ravensight-playtest 0.3.1 → 0.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -45,9 +45,32 @@ Credential Manager, libsecret on Linux). With no keychain available it falls
45
45
  back to a file with 0600 permissions under your home directory, and says so.
46
46
  In CI, set `RAVENSIGHT_TOKEN` instead and skip `login` entirely.
47
47
 
48
+ ## Upgrading to 0.3.2
49
+
50
+ - **A plain `run` plays one persona, not two.** The default persona set is now
51
+ `curious-kid` alone, so a first run costs one persona run. Name more with
52
+ `--personas curious-kid,grumpy-veteran` (or `personas` in
53
+ `.ravensight/config.json`); `ravensight-playtest personas` lists the slugs.
54
+ A script that relied on two personas by default has to name them.
55
+ - **`run` checks the build before it prices anything.** A web build's URL has
56
+ to answer 2xx or 3xx, or the `preflight` URL in `.ravensight/config.json` has
57
+ to answer what you set (see `run`). A failure exits 1 with no job registered.
58
+ `--skip-preflight` skips it, as it already skipped the native probe.
59
+ - **`brief` and `check` quote the run that `run` would start**: its modules and
60
+ its personas, the default one when you name none, each unit with its own
61
+ price. They used to quote a game profile with no personas.
62
+ - **Two new brief fields stay on this machine**: `expected_noise` and
63
+ `test_account` (see the `brief --from` file). `init` adds
64
+ `.ravensight/brief.local.json` to `.gitignore`.
65
+ - **Out of range report values are normalised before upload** instead of
66
+ failing the run (see how the report step reads what happened, under `run`).
67
+ - **A CSS selector is no longer a target.** A target is a ref, coordinates, a
68
+ line copied from the screen, or text on the screen; anything else is refused
69
+ at once instead of searched for.
70
+
48
71
  ## Commands
49
72
 
50
- Twelve commands. Flags below are the ones the code accepts, not the ones the
73
+ Thirteen commands. Flags below are the ones the code accepts, not the ones the
51
74
  prose elsewhere describes; `--help` on any command prints the same list.
52
75
 
53
76
  Every command takes these four:
@@ -89,9 +112,26 @@ to), `project binding` (the game and environment this directory is linked to,
89
112
  and whether that game is in the token's scope), `brief` (present, complete, and
90
113
  which of goals, success criteria, controls and how to start are empty), `build`
91
114
  (the URL, Godot project or native build a run would use), `automation server`
92
- (native drivers only, see below), then the balance against the next job's price,
93
- the content pack verified by sha256, disk, recommended concurrency, and the two
94
- optional tools.
115
+ (native drivers only, see below), then the content pack verified by sha256, the
116
+ balance against the price of the run `run` would start (its modules and
117
+ personas, the default persona when none is named, each unit with its price),
118
+ disk, recommended concurrency, and the two optional tools. A `--personas` slug
119
+ the pack does not have fails a `personas` row that lists the valid slugs, and
120
+ nothing is priced.
121
+
122
+ For a web build linked by URL (`--build-url` is a `run` flag; `check` reads the
123
+ URL from `.ravensight/config.json`), the `browser` row also loads the first
124
+ screen headless and a `first screen` row measures it the way a persona would
125
+ read it. When the whole text is over the 4,000 characters a persona reads per
126
+ step, or the screen has a list of 50 or more look-alike items, the row is a
127
+ warning that suggests `run --long` with the reason ("first screen is 18,400
128
+ characters, about 4.6x the 4,000 a persona reads per step") and both prices,
129
+ read from the server's free estimate. A light screen passes with its size. A
130
+ first screen that does not load within 20 seconds is a warning, never a
131
+ failure. The page is marked as a playtest before the game's scripts run (the
132
+ same `window.__RAVENSIGHT_PLAYTEST__` a run sets, with the token `check`), so a
133
+ build shipping the Ravensight web SDK has its session refused rather than
134
+ recorded as a real player.
95
135
 
96
136
  Every row is `pass`, `warn` or `fail`, and every row that is not `pass` prints
97
137
  its fix. Exit code 3 when any row fails; a warning never fails, so CI does not
@@ -118,7 +158,7 @@ setup with the expected rows at each step.
118
158
  "ok": true,
119
159
  "reserves_units": false,
120
160
  "note": "check reserves no units and starts no job: ...",
121
- "cli": { "installed": "0.3.1", "latest": "0.3.1", "min_supported": "0.1.0" },
161
+ "cli": { "installed": "0.3.2", "latest": "0.3.2", "min_supported": "0.1.0" },
122
162
  "login": { "present": true, "valid": true, "games": [{ "gameId": "g1", "name": "Pocket Knight" }] },
123
163
  "project": { "game": "g1", "environment": null, "in_scope": true },
124
164
  "brief": { "present": true, "complete": true, "missing": [] },
@@ -156,8 +196,8 @@ automatically. `check` shows that same warning as a row when it applies.
156
196
  | Flag | Effect |
157
197
  |---|---|
158
198
  | `--game <gameId>` | Check against this game instead of the linked one. |
159
- | `--modules <list>` | Price this module list, comma separated. |
160
- | `--personas <list>` | Price this persona list. Without it the balance row prices no personas, so the quote is lower than a real job's. |
199
+ | `--modules <list>` | Price this module list, comma separated. Default the modules `run` registers, `persona_playtest,aggregate_report`. |
200
+ | `--personas <list>` | Price this persona list. Without it, a module list with `persona_playtest` prices the repo config's personas or the default one, as `run` would. |
161
201
  | `--godot-path <path>` | Where Godot is, if it is not on the PATH. |
162
202
  | `--web` | Check the browser even when this repo is not linked to a web build. |
163
203
  | `--driver <name>` | Check the setup for this driver instead of the configured one. |
@@ -167,7 +207,8 @@ automatically. `check` shows that same warning as a row when it applies.
167
207
 
168
208
  Links this repo to a game and writes `.ravensight/config.json`. Refuses a game
169
209
  your token cannot see, so the first job does not fail with a 404. Adds
170
- `.ravensight/jobs/` to an existing `.gitignore`.
210
+ `.ravensight/jobs/` and `.ravensight/brief.local.json` to an existing
211
+ `.gitignore`.
171
212
 
172
213
  Without `--game`, one visible game is linked as is. With several, `init` lists
173
214
  them numbered and asks `Which game? [1-N]` when it is running in a terminal; in
@@ -180,11 +221,18 @@ login).
180
221
  | `--game <gameId>` | The game to link. Optional with one visible game, or in a terminal. |
181
222
  | `--force` | Relink a repo that is already linked. |
182
223
 
224
+ ### personas
225
+
226
+ Lists the persona slugs `--personas` accepts, from the content pack this game
227
+ gets: each one's name, action budget (its patience) and play style, with the
228
+ default marked. A free read. Takes `--game <gameId>` and `--json`.
229
+
183
230
  ### brief
184
231
 
185
232
  Writes the expectations brief: your goals, what success looks like, how the game
186
- is controlled, where a persona starts. Then prints what a job would cost,
187
- whether your balance covers it, and, past 8 personas, the same wall-clock
233
+ is controlled, where a persona starts. Then prints what the run `run` would
234
+ start costs ("A run of 1 persona (curious-kid) would cost $1.00: a persona run
235
+ $1.00 each"; the game profile is a separate job), whether your balance covers it, and, past 8 personas, the same wall-clock
188
236
  warning `run` shows. A brief is required before `run`; the server answers 409
189
237
  `brief_required` without one.
190
238
 
@@ -195,7 +243,7 @@ warning `run` shows. A brief is required before `run`; the server answers 409
195
243
  | `--draft` | Save as a draft even when it looks complete. |
196
244
  | `--from <file>` | Read the fields from a JSON file instead of asking. |
197
245
  | `--game <gameId>` | The game, when this repo is not linked. |
198
- | `--modules <list>`, `--personas <list>` | Price this module and persona list in the estimate it prints. |
246
+ | `--modules <list>`, `--personas <list>` | Price this module and persona list in the estimate it prints. Every unit is named with its own price, such as "profile job $4.00, a persona run $1.00 each". |
199
247
 
200
248
  #### The `brief --from` file
201
249
 
@@ -211,13 +259,43 @@ are what makes a brief complete; the rest are optional.
211
259
  | `build_notes.how_to_start` | string | Where a persona begins: the URL, the menu path, the first thing to press. |
212
260
  | `design_intent_notes` | array of strings | Deliberate choices a persona must not report as bugs. |
213
261
  | `known_issues` | array of strings | Issues you already know about, so they are not reported again. |
262
+ | `expected_noise` | array of strings | Console errors and failed requests that are expected, such as `"401 /auth/me before login"`. Never a finding. Kept on this machine. |
263
+ | `test_account` | `{ "email", "password", "how_to_sign_in" }` | A throwaway account the persona signs in with, so a run starts past sign-up. The password must be at least 8 characters. Kept on this machine. |
264
+
265
+ `expected_noise` and `test_account` never go to the server: its brief schema
266
+ refuses keys it does not list, and a password does not belong in a record the
267
+ studio shares. `brief --from` splits them into `.ravensight/brief.local.json`,
268
+ readable by you only, which `init` adds to `.gitignore`; `brief --show` prints
269
+ them with the password masked, and `run` and `resume` read them from there.
270
+
271
+ An `expected_noise` pattern matches an automatic console note when every status
272
+ code (`401`) and every path (`/auth/me`) in it appears in the note; the other
273
+ words describe it for the reader. A pattern with neither is matched as text,
274
+ and `/.../` is a regular expression. One rule is built in: a 401 on a who am I
275
+ check, a request whose last path segment is `/me`, `/whoami`, `/session` or
276
+ `/current_user` (such as `/auth/me`), is what a signed out page does and is not
277
+ a finding. A 401 anywhere else (`/me/inventory`, `/user/42/save`, `/token`,
278
+ `/login`) is left to the report step. A matching note is marked "expected
279
+ noise" in the report prompt, and the runner drops a finding whose evidence is
280
+ only expected noise, listing it in `report.md`. A note the persona wrote itself
281
+ is never noise.
282
+
283
+ **The test account password crosses the model proxy during play only**: it is
284
+ in the persona's prompt on every play turn, because that is how a persona can
285
+ type it. Use an account made for testing, with nothing behind it you would mind
286
+ losing. The report step never sees it: its prompt has no test account, and the
287
+ password is replaced by `[test password]`, wherever it stands as a whole token, in the notes, the action log, the
288
+ transcript, and in `report.md` and `report.json` before anything is uploaded. A
289
+ password field's value is never shown in an observation.
214
290
 
215
291
  ```json
216
292
  {
217
293
  "goals": [{ "goal": "onboarding", "priority": 1 }, { "goal": "find_bugs", "priority": 2 }],
218
294
  "success_criteria": "A new player reaches level 2\nNobody gets stuck in the menu",
219
295
  "build_notes": { "controls": "Arrow keys to move, space to jump", "how_to_start": "Press Start on the title screen" },
220
- "design_intent_notes": ["The fog on level 1 is intentional"]
296
+ "design_intent_notes": ["The fog on level 1 is intentional"],
297
+ "expected_noise": ["401 /auth/me before login"],
298
+ "test_account": { "email": "qa@example.test", "password": "a-throwaway-password", "how_to_sign_in": "Log in from the top right" }
221
299
  }
222
300
  ```
223
301
 
@@ -230,7 +308,36 @@ run options below.
230
308
  ### run
231
309
 
232
310
  Registers a job, plays it, writes the reports, uploads them, finishes the job.
233
- Prints the price and asks before spending.
311
+ Prints the price and asks before spending. Before any charge, even with
312
+ `--yes`, it prints one bold line with the persona count and the total ("This
313
+ run: 1 persona (curious-kid) on playwright_web, $1.00 in total: a persona run
314
+ $1.00 each"), the balance, and what a persona run buys: the action budget (the
315
+ persona's patience in the pack, lowered by `--max-actions` or the server's
316
+ ceiling) and that sign-up alone can take 8 of those actions. A `test_account`
317
+ in the brief lets the persona sign in instead. An unknown `--personas` slug is
318
+ refused with the valid slugs before anything is priced.
319
+
320
+ `--long` makes every persona run a long run: twice the actions, model budget
321
+ and wall time of a standard run, for a game whose screens are heavy or whose
322
+ goal is far in. It costs twice a standard run. The price is the server's
323
+ estimate for the long run, printed and confirmed like any other, never a figure
324
+ built into the CLI. Locally a long run uses the run's own action limit when the
325
+ server's register answer carries one, otherwise twice the standard ceilings
326
+ (the persona's patience and the per run ceiling); `--max-actions` still only
327
+ lowers it. `check` suggests `--long` when the first screen is heavy.
328
+
329
+ Before the estimate, `run` checks the build is up, so a down server is never
330
+ paid for. Set `preflight` in `.ravensight/config.json` to name what to check:
331
+
332
+ ```json
333
+ { "preflight": { "url": "/api/health", "expect_status": 200 } }
334
+ ```
335
+
336
+ `url` may be relative to the build URL; `expect_status` is a number or a list
337
+ of numbers, and without it any 2xx or 3xx passes (redirects are not followed).
338
+ With no `preflight`, a web build's own URL has to answer 2xx or 3xx; other
339
+ drivers are not checked unless a `preflight.url` is set. A failure prints the
340
+ status it saw (or the network error), exits 1, and registers nothing.
234
341
 
235
342
  With a native driver, `run` first runs the `build` and `automation server` rows
236
343
  of `check`. When either fails it exits 1 before the estimate, the confirmation
@@ -252,14 +359,15 @@ splitting the job in two is what gets everything played.
252
359
  | Flag | Effect |
253
360
  |---|---|
254
361
  | `--game <gameId>` | The game, when this repo is not linked. |
255
- | `--personas <list>` | Comma separated persona slugs. Defaults to the pack's set. |
362
+ | `--personas <list>` | Comma separated persona slugs. Defaults to one, `curious-kid`; `ravensight-playtest personas` lists them. |
256
363
  | `--driver <name>` | `playwright_web`, `godot_driver`, `cli_stdio`, or a native driver: `desktop_driver`, `unity_driver`, `unreal_driver`, `android_driver`, `ios_driver`. |
257
364
  | `--build-url <url>` | The web build to play. |
258
365
  | `--godot-project <dir>` | The Godot project directory. |
259
366
  | `--driver-config <file>` | The native driver config, or the JSON-RPC bridge config for `cli_stdio`. |
260
- | `--skip-preflight` | Native drivers only: register without running the free automation probe first. The job is registered and charged even if the automation server turns out to be down. |
367
+ | `--skip-preflight` | Register without the free checks first: the build health check (any driver) and the automation probe (native drivers). The job is registered and charged even if the build or the automation server turns out to be down. |
261
368
  | `--cli <command>` | The command that starts a text build. |
262
369
  | `--max-actions <n>` | Lower the actions one persona may take. It can only lower the persona's own patience (and the server's per run ceiling), never raise it. |
370
+ | `--long` | A long persona run: twice the actions, model budget and wall time of a standard run, at twice the price. The price is read from the estimate and confirmed before any charge. |
263
371
  | `--concurrency <n>` | Personas at once. Default 2; `check` recommends a number for this machine. |
264
372
  | `--upload-video` | Upload `session.webm` as well as the reports. |
265
373
  | `--upload-transcript` | Upload `transcript.jsonl` as well as the reports. |
@@ -304,6 +412,30 @@ message, the run is completed as `failed`, and the job exits 1.
304
412
 
305
413
  How the report step reads what happened:
306
414
 
415
+ - Values the report schema does not allow are normalised before upload, so a
416
+ run is not lost to them when the model has no budget left for a repair: a
417
+ severity that is a category word (`"severity": "ux"`) moves to `category`
418
+ when the category is missing or not allowed and the severity becomes
419
+ `minor`; any other unknown severity becomes `minor`; an unknown category
420
+ takes a close match the schema has (`ui` is `ux`, `crash` is `bug`), else
421
+ `other` when the schema has it, else `ux` (the server's schema has no
422
+ catch-all today); a value that differs only in case is respelled; any other
423
+ enum field takes the schema's default. Each change is a runner note at the
424
+ end of the finding's description and is listed in `report.md` under
425
+ "Normalised by the runner". The server's schema stays strict.
426
+ - After the report is written, each blocker or major (at most 3) is checked
427
+ against its own evidence: the screenshot it cites taken nearest after the
428
+ observation it cites is sent with its claim, and the question "does this
429
+ image contradict the claim: yes, no, unclear". A yes downgrades it one step,
430
+ with a note naming the screenshot, in both report files. The cost is 3
431
+ calls at most, each one screenshot in and 32 tokens out, drawn from the
432
+ run's own model budget (the one the 80 percent warning keeps room in for
433
+ the report), never charged separately; a refusal ends the checks and keeps
434
+ the report as written.
435
+ - Console notes that match the brief's `expected_noise`, or the built in rule
436
+ (a 401 on a who am I check while signed out), are expected noise
437
+ and never a finding; see the `brief --from` file.
438
+
307
439
  - Console errors are recorded as automatic notes. A 4xx on a request (`Failed
308
440
  to load resource`, `HTTP 404`, `status: 401`, `401 Unauthorized` and similar)
309
441
  is a network note: context, not a bug. On the web a `Failed to load
@@ -369,11 +501,26 @@ clicks and types by ref, which names exactly one element even when two share
369
501
  a name; two such elements are both shown, each with where it is ("in header
370
502
  strip", "in main"). A name that matches twice is answered with the refs to
371
503
  choose from, and a target written as locator code (`getByRole(...)`,
372
- `page.locator(...)`, `css=`, `xpath=`) is refused at once. Failures like these
504
+ `page.locator(...)`, `css=`, `xpath=`) is refused at once. A whole line copied
505
+ from the screen (`button "Run the week" [e12] @(1000,303)`, or any part of one)
506
+ is clicked by its ref, else its `@(x,y)` (refused like a stale ref when the page
507
+ has loaded a new document since); a ref or point inside a quoted name is part
508
+ of the name, and a bare `e4` is text when something on screen is named exactly
509
+ that. A target that is
510
+ none of a ref, coordinates or text on the screen is refused at once rather than
511
+ searched for. A name that matches one element on screen is clicked by that
512
+ element's ref, and a text search skips hidden elements, so a mobile only copy
513
+ hidden by CSS is never the one clicked. Failures like these
373
514
  are the driver's, not the game's: they are recorded as `driver_error`, and a
374
515
  blocker or major resting only on them is dropped before upload, as is any
375
516
  finding resting only on the persona's "waiting for X to finish" notes.
376
517
 
518
+ An action counts as changing the screen when the URL changed, the page loaded
519
+ a new document, the visible tree (field values included) changed, or for typing,
520
+ the field's own value changed, with one short second look for a navigation
521
+ still starting. That is the "screen changed" the report step's action log and
522
+ the downgrade rule read.
523
+
377
524
  Every field, checkbox, radio, slider and select the persona sees carries what
378
525
  it holds now, read from the page rather than from the accessibility text:
379
526
  `value="..."` (cut at 80 characters), `checked` or `not checked`, or
@@ -382,7 +529,9 @@ the page could not answer for shows no value at all, so a typed password never
382
529
  reaches the model.
383
530
 
384
531
  On a long web page the persona's text view of the screen is capped at 4,000
385
- characters. Over the cap, buttons, links and fields are kept first, a long run
532
+ characters. A run of 20 or more look-alike items is always shown as its first
533
+ two and "and N more like it", under the cap too; a shorter run under the cap is
534
+ shown as it is. Over the cap, buttons, links and fields are kept first, a long run
386
535
  of look-alike items (a list of 249 cards) is shown as its first two and "and
387
536
  247 more like it", and what changed since the last look comes next, so a form
388
537
  after a long list still reaches the persona.
@@ -394,7 +543,8 @@ reached. Every run gets its own identity to sign up with, an email
394
543
  short run id, so a second run never hits "that email is taken" from the first.
395
544
  `example.test` is a reserved name and never delivers mail. Point `--build-url` at a guest or demo path, or say where to start in
396
545
  the brief's `how_to_start` note (for example "choose Play as guest"), when the
397
- sign-up is not what you want tested.
546
+ sign-up is not what you want tested, or give the brief a `test_account` the
547
+ persona signs in with instead.
398
548
 
399
549
  ### fix
400
550
 
@@ -586,6 +736,7 @@ In your repo:
586
736
  ```
587
737
  .ravensight/
588
738
  config.json the game link and your defaults
739
+ brief.local.json expected_noise and test_account, this machine only (0600)
589
740
  jobs/<jobId>/
590
741
  state.json the resumable journal
591
742
  upload-queue.jsonl what was uploaded, and what is still owed
@@ -602,8 +753,8 @@ and a local `transcript.jsonl` of the apply loop (which tool, on what, how
602
753
  much came back) that is never uploaded and is how an attempt that wrote
603
754
  nothing gets explained afterwards.
604
755
 
605
- Add `.ravensight/jobs/` to your `.gitignore`; `init` does it for you when you
606
- already have one.
756
+ Add `.ravensight/jobs/` and `.ravensight/brief.local.json` to your
757
+ `.gitignore`; `init` does it for you when you already have one.
607
758
 
608
759
  `fix` also leaves a branch on your working copy, checked out, with the change in
609
760
  your working tree and nothing committed. It is the only thing in this CLI that
@@ -633,6 +784,16 @@ pins it equal to the server's file when a checkout is on the same machine.
633
784
  Findings cite a file and a line range. The source text behind that line is not
634
785
  uploaded.
635
786
 
787
+ `.ravensight/brief.local.json` is never uploaded. Its `expected_noise` goes into
788
+ the persona and report prompts. Its `test_account`, password included, goes into
789
+ the persona's prompt during play only, through the model proxy; the report step
790
+ never gets it, and the password is scrubbed to `[test password]` from the
791
+ transcript, the notes and both report files before upload.
792
+
793
+ `run`'s build health check is one GET to your own build (or the `preflight`
794
+ URL you set), made from your machine.
795
+
796
+
636
797
  A `fix` uploads `changeset.json` and `report.md`. The changeset carries the
637
798
  finding id, the branch name, the commit it branched from, the file paths it
638
799
  changed with their added and removed line counts, the verify verdict, and the
@@ -60,7 +60,7 @@ Every device command times out after 30 seconds and responses are capped at
60
60
  ## Step 1: the account side
61
61
 
62
62
  ```bash
63
- npm install -g ravensight-playtest@0.3.1
63
+ npm install -g ravensight-playtest@0.3.2
64
64
  ravensight-playtest login
65
65
  cd path/to/your/game
66
66
  ravensight-playtest init --game <gameId>
@@ -70,7 +70,7 @@ ravensight-playtest check --driver ios_driver
70
70
  Expected rows (your email, game and version differ):
71
71
 
72
72
  ```
73
- pass cli version installed 0.3.1; server latest 0.3.1, min_supported 0.1.0
73
+ pass cli version installed 0.3.2; server latest 0.3.2, min_supported 0.1.0
74
74
  pass login you@example.com, valid, scoped to 1 game: Pocket Knight (g1)
75
75
  pass project binding linked to Pocket Knight (g1), in the token's scope
76
76
  fail brief this game has no expectations brief, and a persona run needs one
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ravensight-playtest",
3
- "version": "0.3.1",
3
+ "version": "0.3.2",
4
4
  "description": "AI personas play your game and file reports to Ravensight.",
5
5
  "license": "MIT",
6
6
  "homepage": "https://ravensight.io/docs/#playtest",
@@ -53,6 +53,6 @@
53
53
  },
54
54
  "optionalDependencies": {
55
55
  "keytar": "^7.9.0",
56
- "playwright": "^1.49.0"
56
+ "playwright": "^1.51.0"
57
57
  }
58
58
  }
package/src/api/client.js CHANGED
@@ -189,9 +189,15 @@ export function createClient(options = {}) {
189
189
  };
190
190
 
191
191
  const jobs = {
192
- estimate: (gameId, { modules = [], personas = [] } = {}) =>
193
- request('POST', `${gameBase(gameId)}/estimate`, { body: { modules, personas } })
194
- .then(answer => answer.body),
192
+ /**
193
+ * Price a job without charging. `runLength: 'long'` asks for the long
194
+ * persona run's price; anything else leaves `run_length` out, which the
195
+ * server reads as a standard run.
196
+ */
197
+ estimate: (gameId, { modules = [], personas = [], runLength = null } = {}) =>
198
+ request('POST', `${gameBase(gameId)}/estimate`, {
199
+ body: { modules, personas, ...(runLength === 'long' ? { run_length: 'long' } : {}) }
200
+ }).then(answer => answer.body),
195
201
 
196
202
  /**
197
203
  * Register a job: this is the request that moves money.
@@ -0,0 +1,126 @@
1
+ import { join } from 'node:path';
2
+ import { readJson, writeJson } from './fsutil.js';
3
+ import { CliError } from './errors.js';
4
+
5
+ /**
6
+ * The brief fields that stay on this machine: `expected_noise` and
7
+ * `test_account`.
8
+ *
9
+ * The server's expectations brief schema refuses any key it does not list
10
+ * (`additionalProperties: false`, drafts included), and a test account's
11
+ * password has no business in a record the whole studio can read. So these
12
+ * two live in `.ravensight/brief.local.json`, which `init` adds to
13
+ * `.gitignore` and which is written readable by its owner only. `brief --from`
14
+ * accepts them in the same file as the server's fields and splits them off;
15
+ * `run` and `resume` hand them to the persona with the rest of the brief.
16
+ *
17
+ * The password is still sent to the model like any other prompt text: that is
18
+ * how a persona can type it. It is for a throwaway account made for testing.
19
+ */
20
+
21
+ /** The keys this file holds. */
22
+ export const LOCAL_BRIEF_FIELDS = Object.freeze(['expected_noise', 'test_account']);
23
+
24
+ /** The shortest test account password accepted. */
25
+ export const MIN_PASSWORD_CHARS = 8;
26
+
27
+ /** What a password is printed as. */
28
+ export const HIDDEN_PASSWORD = '(hidden)';
29
+
30
+ /**
31
+ * @param {{repoRoot?: string}} [options]
32
+ * @returns {string}
33
+ */
34
+ export function localBriefFile(options = {}) {
35
+ return join(options.repoRoot || process.cwd(), '.ravensight', 'brief.local.json');
36
+ }
37
+
38
+ /**
39
+ * Split a brief fields object into what the server stores and what stays here.
40
+ *
41
+ * @param {Object} fields
42
+ * @returns {{server: Object, local: Object}}
43
+ */
44
+ export function splitLocalFields(fields = {}) {
45
+ const server = {};
46
+ const local = {};
47
+ for (const [key, value] of Object.entries(fields || {})) {
48
+ if (LOCAL_BRIEF_FIELDS.includes(key)) local[key] = value;
49
+ else server[key] = value;
50
+ }
51
+ return { server, local };
52
+ }
53
+
54
+ /**
55
+ * What is wrong with a local brief, one line per problem; empty when it is fine.
56
+ *
57
+ * @param {Object} local
58
+ * @returns {Array<string>}
59
+ */
60
+ export function localBriefProblems(local = {}) {
61
+ const problems = [];
62
+ if (local.expected_noise !== undefined) {
63
+ if (!Array.isArray(local.expected_noise) || !local.expected_noise.every(entry => typeof entry === 'string' && entry.trim())) {
64
+ problems.push('expected_noise must be an array of non empty strings, such as ["401 /auth/me before login"]');
65
+ }
66
+ }
67
+ if (local.test_account !== undefined) {
68
+ const account = local.test_account;
69
+ if (!account || typeof account !== 'object' || Array.isArray(account)) {
70
+ problems.push('test_account must be an object: { "email": ..., "password": ..., "how_to_sign_in": ... }');
71
+ } else {
72
+ for (const key of ['email', 'password']) {
73
+ if (typeof account[key] !== 'string' || !account[key]) problems.push(`test_account.${key} must be a non empty string`);
74
+ }
75
+ // The runner scrubs the password out of every report; a short one
76
+ // ("test", "ux") would also match ordinary words and enum values.
77
+ if (typeof account.password === 'string' && account.password && account.password.length < MIN_PASSWORD_CHARS) {
78
+ problems.push(`test_account.password must be at least ${MIN_PASSWORD_CHARS} characters, so scrubbing it from reports cannot rewrite ordinary words`);
79
+ }
80
+ if (account.how_to_sign_in !== undefined && typeof account.how_to_sign_in !== 'string') problems.push('test_account.how_to_sign_in must be a string');
81
+ }
82
+ }
83
+ return problems;
84
+ }
85
+
86
+ /**
87
+ * The local brief, or {} when there is none. Never throws for a missing file;
88
+ * a file that is not an object is an error naming it.
89
+ *
90
+ * @param {{repoRoot?: string}} [options]
91
+ * @returns {Promise<Object>}
92
+ */
93
+ export async function loadLocalBrief(options = {}) {
94
+ const file = localBriefFile(options);
95
+ const value = await readJson(file, null);
96
+ if (value === null) return {};
97
+ if (typeof value !== 'object' || Array.isArray(value)) throw new CliError(`${file} is not a brief fields object.`);
98
+ return Object.fromEntries(Object.entries(value).filter(([key]) => LOCAL_BRIEF_FIELDS.includes(key)));
99
+ }
100
+
101
+ /**
102
+ * Merge these fields into the local brief and write it, owner readable only.
103
+ *
104
+ * @param {Object} local
105
+ * @param {{repoRoot?: string}} [options]
106
+ * @returns {Promise<Object>} what the file now holds
107
+ */
108
+ export async function saveLocalBrief(local, options = {}) {
109
+ const problems = localBriefProblems(local);
110
+ if (problems.length > 0) throw new CliError(`The local brief fields are not valid: ${problems.join('; ')}.`);
111
+ const merged = { ...(await loadLocalBrief(options)), ...local };
112
+ await writeJson(localBriefFile(options), merged, { mode: 0o600 });
113
+ return merged;
114
+ }
115
+
116
+ /**
117
+ * The local brief as it may be printed: the password masked.
118
+ *
119
+ * @param {Object} local
120
+ * @returns {Object}
121
+ */
122
+ export function displayLocalBrief(local = {}) {
123
+ const out = { ...local };
124
+ if (out.test_account && typeof out.test_account === 'object') out.test_account = { ...out.test_account, ...(out.test_account.password ? { password: HIDDEN_PASSWORD } : {}) };
125
+ return out;
126
+ }
package/src/cli.js CHANGED
@@ -52,13 +52,16 @@ async function loadRunner(name, exportName) {
52
52
  */
53
53
  export const RUNNER_HELP = `
54
54
  Run options (parsed by the runner, see src/run/args.js):
55
- --personas <list> comma separated persona slugs, default the pack's set
55
+ --personas <list> comma separated persona slugs, default one (curious-kid); "personas" lists them
56
56
  --driver <name> playwright_web, godot_driver, desktop_driver, unity_driver, unreal_driver, android_driver, ios_driver or cli_stdio
57
57
  --controller <name> standard or jev action selection (default standard)
58
58
  --build-url <url> the web build to play
59
59
  --godot-project <dir> the Godot project directory
60
60
  --driver-config <file> local WebDriver or JSON-RPC bridge configuration
61
61
  --max-actions <n> lower the actions one persona may take; it can only lower the persona's own patience, never raise it
62
+ --long a long persona run: twice the actions, model budget and wall time of a standard
63
+ run, for a game whose screens are heavy or whose goal is far in. It costs twice
64
+ a standard run; the price is read from the estimate and confirmed before any charge
62
65
  --concurrency <n> personas at once, default 2
63
66
  --upload-video upload session.webm as well as the reports
64
67
  --upload-transcript upload transcript.jsonl as well as the reports
@@ -67,8 +70,9 @@ Run options (parsed by the runner, see src/run/args.js):
67
70
  --headed show the browser
68
71
  --headless hide it
69
72
  --verbose print each turn
70
- --skip-preflight native drivers: register without probing the automation server first;
71
- the job is registered and charged even if the server turns out to be down
73
+ --skip-preflight register without the free checks first: the build health check (any driver)
74
+ and the automation server probe (native drivers); the job is registered and
75
+ charged even if the build or the server turns out to be down
72
76
 
73
77
  Accepted and reported as not applied by this release:
74
78
  --quality, --offline-upload, --build-dir
@@ -179,6 +183,13 @@ export function buildProgram() {
179
183
  }));
180
184
  }));
181
185
 
186
+ common(program.command('personas').description('list the persona slugs --personas accepts'))
187
+ .option('--game <gameId>', 'the game, when this repo is not linked')
188
+ .action(capture(slot, async options => {
189
+ const { personas } = await import('./commands/personas.js');
190
+ return personas(flagsFrom(options, { game: options.game }));
191
+ }));
192
+
182
193
  common(program.command('init').description('link this repo to a Ravensight game'))
183
194
  .option('--game <gameId>', 'the game to link')
184
195
  .option('--force', 'relink a repo that is already linked')