@qaping/cli 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/bin/qaping CHANGED
@@ -15,6 +15,7 @@
15
15
  // qaping wait <ping_id> [--timeout <seconds>]
16
16
  // qaping results <ping_id> one passive fetch, no blocking
17
17
  // qaping rounds [add|rm] the parked rounds this repo is owed
18
+ // qaping issues <game> the game's issue board, read-only
18
19
  // qaping whoami | version | help
19
20
  "use strict";
20
21
  const fs = require("fs");
@@ -50,11 +51,12 @@ function resolveKitDir() {
50
51
  // writes it to ~/.claude/rules/qaping.md and ~/.cursor/rules/qaping.mdc.
51
52
  const RULE_BODY = `This machine has qaping: automatic QA for games — real human playtesters play the developer's builds on their own hardware and report back, and the coding agent runs the whole loop.
52
53
  Whenever the user asks to set up QA for a game, run QA on a patch, check whether a change broke anything, or playtest a game or build, load the qaping skill and follow it — it owns authoring/maintaining QA-PLAN.md in the game repo and the per-patch run.
53
- Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead). Hosted builds are temporary and capped per account — if a publish is refused for that cap, \`qaping builds\` lists them and \`qaping builds rm <slug>\` frees a slot; free one and carry on rather than stopping to ask, taking the ones marked as never-finished uploads first and never deleting a build a round you filed is still running against.
54
- The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a round, free) and qaping_wait (continue a pending round).
54
+ Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead). Hosted builds are kept until you delete them (pass \`--expires-in-hours <n>\` to have one reaped instead) and capped per account — if a publish is refused for that cap, \`qaping builds\` lists them and \`qaping builds rm <slug>\` frees a slot; free one and carry on rather than stopping to ask, taking the ones marked as never-finished uploads first and never deleting a build a round you filed is still running against.
55
+ The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a round, free), qaping_wait (continue a pending round) and qaping_issues (the game's issue board — file a bug, claim a fix, close or reopen; free).
56
+ Bugs live on that board, not in chat or in a report nobody re-reads: file EVERY playtest with \`game\` set to the QA plan's game_slug, so the game's open and claimed-fixed issues ride the round as the playtester's checklist. You can only ever CLAIM a fix (qaping_issues action:'fixed', with the exact build URL the next round will play, claimed BEFORE that round is filed) — only a playtester answering "Gone" on a later round marks one verified, and a "Still here" on a claimed fix regresses it. \`qaping issues <game>\` reads the same board from the shell.
55
57
  Decide at filing time how the round ends: a short round waits in place — the moment filing returns pending, spawn a background task that loops qaping_wait and keep working (foreground polling is the fallback); a long or overnight round is PARKED — file it with an explicit long deadline_seconds, record it immediately with \`qaping rounds add <ping_id>\` (it writes qa-open-rounds.json beside QA-PLAN.md), tell the user exactly what is pending, and end the turn; a later session collects it (\`qaping rounds\` lists them with live status, qaping_results fetches one).
56
58
  A recorded parked round is an honest pending end — never report pending as the answer without one, and never end with an unbacked promise to check later.
57
- Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording and no transcript; web rounds return answers only (no recording, no transcript) — never promise evidence a platform does not ship.
59
+ Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording, and a transcript only when the tester recorded with our recorder app (read transcript_status on each result — a Game Bar session has no transcript); web rounds return answers only (no recording, no transcript) — never promise evidence a session does not ship.
58
60
  QA memory: whenever the developer mentions — in ANY conversation, not just QA runs — something that should always be tested, a fragile area, or a QA lesson learned, offer to record it as a check (or a note on an existing check) in QA-PLAN.md so it is tested from then on.
59
61
  `;
60
62
 
@@ -255,13 +257,18 @@ const ROUNDS_FILE = "qa-open-rounds.json";
255
257
  // what `qaping rounds` costs in requests.
256
258
  const MAX_OPEN_ROUNDS = 20;
257
259
  const PING_ID_RE = /^[0-9a-f-]{36}$/i;
260
+ // The issue board's game slug. Mirrors lib/qa-issues.ts's GAME_SLUG_RE and
261
+ // migration 066's column check exactly — a slug this refuses is a slug the
262
+ // service refuses, and finding that out before the network is the point.
263
+ const GAME_SLUG_RE = /^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$/;
258
264
  const DEFAULT_APP_URL = "https://pingfusi.com";
259
265
 
260
266
  const ROUNDS_USAGE = `usage:
261
267
  qaping rounds [--json] [--offline] [--file <path>]
262
268
  qaping rounds add <ping_id> [--platform windows|macos|ios] [--build <url>]
263
269
  [--minutes N] [--deadline-seconds N]
264
- [--checks a,b,c] [--note "<patch>"] [--json]
270
+ [--game <slug>] [--checks a,b,c]
271
+ [--note "<patch>"] [--json]
265
272
  qaping rounds rm <ping_id> [--json]`;
266
273
 
267
274
  function flagValue(argv, name) {
@@ -440,6 +447,9 @@ function renderRounds(entries, opts) {
440
447
  lines.push("");
441
448
  lines.push(` ${record.ping_id}${facts ? ` — ${facts}` : ""}`);
442
449
  if (record.build) lines.push(` build: ${record.build}`);
450
+ // Which board's issues rode this round — the later session that collects
451
+ // it needs the slug to read what moved (`qaping issues <game>`).
452
+ if (record.game) lines.push(` game: ${record.game}`);
443
453
  if (Array.isArray(record.checks) && record.checks.length) lines.push(` checks: ${record.checks.join(", ")}`);
444
454
  if (record.note) lines.push(` patch: ${record.note}`);
445
455
  if (live) lines.push(` ${roundStatusLine(record, live)}`);
@@ -493,13 +503,14 @@ function resolveTokenFromClientConfigs(kitDir) {
493
503
  return null;
494
504
  }
495
505
 
496
- // One passive tools/call against the qaping mount the same transport and the
497
- // same JSON-or-SSE parse the kit's `wait` leg uses, with the mount read from the
498
- // ONE wrapper object every other command is driven by. Passive: it never renews
499
- // the round's lease, which is exactly why a parked round needs its own explicit
500
- // deadline. Failures are TYPED for the caller: `authRejected` and `perRound`
501
- // separate "this login is dead" from "the service refused this one id".
502
- async function callResultsTool(pingId) {
506
+ // One passive tools/call against the qaping mount, for ANY tool it registers
507
+ // the same transport and the same JSON-or-SSE parse the kit's `wait` leg uses,
508
+ // with the mount read from the ONE wrapper object every other command is driven
509
+ // by. Passive: it never renews a round's lease, which is exactly why a parked
510
+ // round needs its own explicit deadline. Failures are TYPED for the caller:
511
+ // `authRejected` and `perRound` separate "this login is dead" from "the service
512
+ // refused this one call".
513
+ async function callQapingTool(name, args) {
503
514
  const w = qapingWrapper();
504
515
  const kitDir = resolveKitDir();
505
516
  const { resolveToken } = require(path.join(kitDir, "packages", "core", "wire.js"));
@@ -518,7 +529,7 @@ async function callResultsTool(pingId) {
518
529
  jsonrpc: "2.0",
519
530
  id: 1,
520
531
  method: "tools/call",
521
- params: { name: "qaping_results", arguments: { ping_id: pingId } },
532
+ params: { name, arguments: args },
522
533
  }),
523
534
  signal: AbortSignal.timeout(20_000),
524
535
  });
@@ -551,10 +562,18 @@ async function callResultsTool(pingId) {
551
562
  status: structured.status || "pending",
552
563
  structured,
553
564
  result,
565
+ // A tool that refuses inside a successful JSON-RPC result (the board at
566
+ // its cap, a seq that is not there) answers with isError + the same text.
567
+ isError: result.isError === true,
554
568
  text: (result.content && result.content[0] && result.content[0].text) || "",
555
569
  };
556
570
  }
557
571
 
572
+ /** The results tool, by name — every caller below reads the same shape. */
573
+ function callResultsTool(pingId) {
574
+ return callQapingTool("qaping_results", { ping_id: pingId });
575
+ }
576
+
558
577
  async function cmdResults(argv) {
559
578
  const json = argv.includes("--json");
560
579
  // Flags in any position: `qaping results --json <id>` is the natural thing to
@@ -584,6 +603,52 @@ async function cmdResults(argv) {
584
603
  process.exit(0);
585
604
  }
586
605
 
606
+ const ISSUES_USAGE = "usage: qaping issues <game> [--json] [--status open|fixed|verified|closed]";
607
+ const ISSUE_STATUSES = ["open", "fixed", "verified", "closed"];
608
+
609
+ /** argv after the command, minus flags and the values they consume — so a
610
+ * slug that happens to equal a flag's value is still the positional. */
611
+ function positionalArgs(argv, valueFlags) {
612
+ const out = [];
613
+ for (let i = 1; i < argv.length; i++) {
614
+ if (argv[i].startsWith("--")) {
615
+ if (valueFlags.includes(argv[i])) i++;
616
+ continue;
617
+ }
618
+ out.push(argv[i]);
619
+ }
620
+ return out;
621
+ }
622
+
623
+ /**
624
+ * The game's issue board, read-only. The agent moves issues through the MCP
625
+ * tool; this is the human's window on the same board — and it prints the
626
+ * SERVICE's own rendering rather than a second one, so the board a developer
627
+ * reads in the shell and the board an agent reads mid-run cannot disagree.
628
+ */
629
+ async function cmdIssues(argv) {
630
+ const json = argv.includes("--json");
631
+ const game = positionalArgs(argv, ["--status"])[0];
632
+ if (!game || !GAME_SLUG_RE.test(game)) {
633
+ console.error(`✗ not a game slug: ${game || "(missing)"} — lowercase letters, digits and hyphens.\n${ISSUES_USAGE}`);
634
+ // 1, like `results`: a typo must never reach the network, and 2 is
635
+ // reserved by the round verbs for "still pending".
636
+ process.exit(1);
637
+ }
638
+ const status = flagValue(argv, "--status");
639
+ if (status !== undefined && !ISSUE_STATUSES.includes(status)) {
640
+ console.error(`✗ --status must be one of ${ISSUE_STATUSES.join(", ")}.\n${ISSUES_USAGE}`);
641
+ process.exit(1);
642
+ }
643
+ const args = { game, action: "list" };
644
+ if (status) args.status_filter = status;
645
+ const r = await callQapingTool("qaping_issues", args);
646
+ console.log(json
647
+ ? JSON.stringify(Object.keys(r.structured).length ? r.structured : r.result, null, 2)
648
+ : (r.text || JSON.stringify(r.result, null, 2)));
649
+ process.exit(r.isError ? 1 : 0);
650
+ }
651
+
587
652
  async function cmdRoundsList(argv) {
588
653
  const json = argv.includes("--json");
589
654
  const state = readRoundsFile(roundsFilePath(argv));
@@ -650,6 +715,13 @@ function cmdRoundsAdd(argv) {
650
715
  console.error(`✗ --platform must be windows, macos or ios — a web round has none, so omit it.\n${ROUNDS_USAGE}`);
651
716
  process.exit(2);
652
717
  }
718
+ // Validated here, not just recorded: a slug the service would refuse is a
719
+ // board the collecting session could never read.
720
+ const game = flagValue(argv, "--game");
721
+ if (game !== undefined && !GAME_SLUG_RE.test(game)) {
722
+ console.error(`✗ --game must be a slug: lowercase letters, digits and hyphens (the QA plan's game_slug).\n${ROUNDS_USAGE}`);
723
+ process.exit(2);
724
+ }
653
725
  const file = roundsFilePath(argv);
654
726
  const minutes = intFlag(argv, "--minutes");
655
727
  const deadline = intFlag(argv, "--deadline-seconds");
@@ -658,6 +730,7 @@ function cmdRoundsAdd(argv) {
658
730
  const build = flagValue(argv, "--build");
659
731
  const patch = {};
660
732
  if (platform) patch.platform = platform;
733
+ if (game) patch.game = game;
661
734
  if (build) patch.build = build;
662
735
  if (minutes != null) patch.est_minutes = minutes;
663
736
  if (deadline != null) patch.deadline_seconds = deadline;
@@ -741,6 +814,7 @@ usage:
741
814
  qaping rounds the parked rounds this repo is owed, with live status
742
815
  qaping rounds add <ping_id> … record a parked round so a later session collects it
743
816
  qaping rounds rm <ping_id> forget one, once it is collected or expired
817
+ qaping issues <game> the game's issue board (read-only; the agent moves it)
744
818
  qaping whoami
745
819
  qaping version
746
820
 
@@ -757,6 +831,7 @@ function route(cmd) {
757
831
  if (cmd === "publish") return "publish";
758
832
  if (cmd === "rounds") return "rounds";
759
833
  if (cmd === "results") return "results";
834
+ if (cmd === "issues") return "issues";
760
835
  return "unknown";
761
836
  }
762
837
 
@@ -783,6 +858,7 @@ function main() {
783
858
  case "publish": return void cmdPublish(argv);
784
859
  case "rounds": return void cmdRounds(argv).catch(die);
785
860
  case "results": return void cmdResults(argv).catch(die);
861
+ case "issues": return void cmdIssues(argv).catch(die);
786
862
  default:
787
863
  console.error(HELP);
788
864
  process.exit(1);
@@ -798,5 +874,6 @@ function main() {
798
874
  if (require.main === module) main();
799
875
  module.exports = {
800
876
  route, resolveKitDir, qapingWrapper, vendorFlags, isSourceCheckout, RULE_BODY, HELP,
801
- ROUNDS_FILE, MAX_OPEN_ROUNDS, parseRounds, upsertRound, removeRound, visibleUntil, renderRounds,
877
+ ROUNDS_FILE, MAX_OPEN_ROUNDS, GAME_SLUG_RE,
878
+ parseRounds, upsertRound, removeRound, visibleUntil, renderRounds,
802
879
  };
@@ -0,0 +1,122 @@
1
+ # qaping — CLI commands
2
+
3
+ Everything the `qaping` binary does. Your agent runs most of these for you;
4
+ `setup`, `builds` and `rounds` are the ones you will type yourself.
5
+
6
+ ```
7
+ qaping setup [claude-code|cursor|codex] [--force]
8
+ qaping remove [--client <c>]
9
+ qaping publish-build <game.zip> --platform windows|macos
10
+ qaping builds
11
+ qaping builds rm <slug>
12
+ qaping publish <built-dir>
13
+ qaping wait <ping_id> [--timeout <seconds>]
14
+ qaping results <ping_id> [--json]
15
+ qaping rounds [--json] [--offline] [--file <path>]
16
+ qaping rounds add <ping_id> [--platform windows|macos|ios] [--build <url>]
17
+ [--minutes N] [--deadline-seconds N]
18
+ [--game <slug>] [--checks a,b,c]
19
+ [--note "<patch>"] [--json]
20
+ qaping rounds rm <ping_id> [--json]
21
+ qaping issues <game> [--json] [--status open|fixed|verified|closed]
22
+ qaping whoami
23
+ qaping version
24
+ ```
25
+
26
+ ## Getting in and out
27
+
28
+ ### `qaping setup [claude-code|cursor|codex] [--force]`
29
+
30
+ Registers the qaping MCP server with your coding agent, installs the `qaping`
31
+ skill and its rule, and signs you in (a browser window opens once). With no
32
+ client name it detects what is installed. `--force` rewrites an existing
33
+ entry.
34
+
35
+ ### `qaping remove [--client <c>]`
36
+
37
+ Removes qaping's MCP entry, skill and rule from the client. It touches only
38
+ qaping's own state — nothing else you have installed, and not your login.
39
+
40
+ ### `qaping whoami`
41
+
42
+ Which account this machine is signed in as. Results are scoped to the account
43
+ that filed the round, so check this before deciding a round is gone.
44
+
45
+ ### `qaping version`
46
+
47
+ Prints the installed version.
48
+
49
+ ## Hosting a build
50
+
51
+ ### `qaping publish-build <game.zip> --platform windows|macos`
52
+
53
+ Uploads a zipped native build and prints the `/b/<slug>` URL a round is filed
54
+ against. You hold a small number of hosted builds at a time; the oldest is
55
+ listed first by `qaping builds`.
56
+
57
+ ### `qaping builds` / `qaping builds rm <slug>`
58
+
59
+ Lists the hosted builds you hold, or deletes one now to free its slot.
60
+
61
+ ### `qaping publish <built-dir>`
62
+
63
+ Hosts a browser-playable game's built output and prints the public URL. Web
64
+ rounds are filed with that URL and no platform.
65
+
66
+ ## Rounds
67
+
68
+ ### `qaping wait <ping_id> [--timeout <seconds>]`
69
+
70
+ Waits on a filed round, renewing the lease that keeps it visible to new
71
+ playtesters, and prints the results when they land. The agent normally runs
72
+ this in a background task.
73
+
74
+ ### `qaping results <ping_id> [--json]`
75
+
76
+ One passive fetch, no waiting. Exit code `0` means results are in, `2` means
77
+ still pending, `1` anything else (expired, not yours, unreachable).
78
+
79
+ ### `qaping rounds`
80
+
81
+ The parked rounds this repo is owed, read from `qa-open-rounds.json` at the
82
+ repo root, each with its live status. `--offline` skips the status lookup;
83
+ `--file` points at another ledger.
84
+
85
+ ### `qaping rounds add <ping_id> …`
86
+
87
+ Records a round the agent chose to park — an overnight run, a slow native
88
+ claim — so a later session collects it instead of filing a duplicate. Add
89
+ `--build`, `--minutes`, `--deadline-seconds`, `--game`, `--checks` and
90
+ `--note` so the record explains itself; `--game` is the QA plan's
91
+ `game_slug`, so the session that collects the round knows which issue board
92
+ it moved.
93
+
94
+ ### `qaping rounds rm <ping_id>`
95
+
96
+ Forgets a parked round once it is collected or expired.
97
+
98
+ ## The issue board
99
+
100
+ ### `qaping issues <game> [--json]`
101
+
102
+ Prints the game's issue board: every bug that outlived a round, with its
103
+ status — open, fixed (you claim it is fixed), verified (a playtester
104
+ confirmed it is gone), closed. `--status` prints one status only; `--json`
105
+ prints the machine-readable rows.
106
+
107
+ Read-only, on purpose. Your agent files, claims, closes and reopens issues
108
+ through the `qaping_issues` MCP tool while it runs the loop; this is the
109
+ window you read it through. Nothing on the board costs credits — only rounds
110
+ do — and nothing turns *verified* except a playtester answering "Gone" on a
111
+ round filed after the fix was claimed.
112
+
113
+ ## Files qaping leaves in your repo
114
+
115
+ | file | what it is | commit it? |
116
+ |---|---|---|
117
+ | `QA-PLAN.md` | the checks this game must pass; the agent authors, you own | yes — see [the format](QA-PLAN-FORMAT.md) |
118
+ | `qa-rounds.jsonl` | append-only history, one line per finished round | yes |
119
+ | `qa-open-rounds.json` | rounds filed and not yet collected | no — gitignore it |
120
+
121
+ The issue board is not a file: it lives on the service, one board per game,
122
+ and `qaping issues <game>` reads it. The plan's `game_slug` is what names it.
@@ -11,12 +11,19 @@ YAML at the very top of the file:
11
11
  ```yaml
12
12
  ---
13
13
  game: Solar Drift
14
+ game_slug: solar-drift # the issue board's id — minted ONCE, never renamed
14
15
  build_command: ./scripts/package.sh --release # the ONE command that produces a shippable build
15
16
  platforms: [windows] # windows | macos | ios | web
16
17
  input: XInput gamepad # optional; default keyboard+mouse
17
18
  ---
18
19
  ```
19
20
 
21
+ `game_slug` is minted once from the game name — lowercase, every run of
22
+ non-alphanumerics collapsed to a single `-`, no leading or trailing `-`, ≤64
23
+ characters — and is then permanent, exactly like a check id: it names the
24
+ game's issue board, and renaming it orphans every issue filed under the old
25
+ one. Every playtest round is filed with it as `game`.
26
+
20
27
  `build_command` is recorded once at setup so every future run can build without
21
28
  asking. `platforms: [web]` means a browser-playable game — its "build" is a
22
29
  static directory, not a zip. `input` declares the hardware the
@@ -88,6 +95,26 @@ setup: "?save=campaign-mid"
88
95
  anything missing, corrupted, or visually wrong.
89
96
  ````
90
97
 
98
+ ## Bugs live on the issue board; checks live here
99
+
100
+ A check is what the game must ALWAYS do — it belongs in this file, forever. A
101
+ bug is what the game does wrong right now — it belongs on the game's issue
102
+ board (`qaping issues <game_slug>`, moved by the agent through the
103
+ `qaping_issues` tool), where it is numbered, carried across rounds, and
104
+ retired when it is gone. Do not write bugs into `QA-PLAN.md`: a check written
105
+ to describe one bug ("the shop does not close when you press B twice") ages
106
+ into a puzzle the moment the bug is fixed.
107
+
108
+ The board is what makes a fix provable. The agent may CLAIM a fix (with the
109
+ exact build URL the next round will play); only a playtester answering
110
+ "Gone" on a round filed after that claim turns the issue *verified*, and a
111
+ "Still here" on a claimed fix regresses it. Because every round is filed with
112
+ `game`, the board's open and claimed-fixed issues ride it as the playtester's
113
+ checklist — no step budget spent, no bug forgotten between patches.
114
+
115
+ A check and the bug that came from it stay linked through the issue's
116
+ `check_id`: the same id this file gives the check.
117
+
91
118
  ## The round ledger — qa-rounds.jsonl
92
119
 
93
120
  Beside the plan, at the game repo root, sits `qa-rounds.jsonl`: one JSON line
@@ -96,6 +123,8 @@ appended per round, committed alongside the plan updates —
96
123
  ```json
97
124
  {"round_id": "…", "report_url": "…", "build": "<sha or hosted url>",
98
125
  "evidence_grade": "…", "checks": [{"id": "…", "outcome": "…"}],
126
+ "issues": {"game": "solar-drift", "verified": [3], "regressed": [5],
127
+ "new": [7], "still_open": [2]},
99
128
  "verdict": "…", "credits": 20, "follow_ups": ["…"]}
100
129
  ```
101
130
 
@@ -106,8 +135,11 @@ page (what the results tool returns), never a PR or issue link. There is no
106
135
  evidence-link field, deliberately: signed recording URLs expire in days.
107
136
  The ledger is the machine truth of what was asked and answered, per check,
108
137
  per round; the plan's `last_verified` and `state` are derived from it and
109
- stay the dev-readable contract. Open `follow_ups` are what the next filing
110
- must carry forward.
138
+ stay the dev-readable contract. `issues` is what this round did to the issue
139
+ board, by number (omitted when the round carried no `game`). `follow_ups` is
140
+ NON-BUG follow-ups only — a check to add, a build affordance to write, a
141
+ question to settle; bugs go on the board, where they are numbered and
142
+ re-checked. Open `follow_ups` are what the next filing must carry forward.
111
143
 
112
144
  ## Rounds still open — qa-open-rounds.json
113
145
 
@@ -118,12 +150,13 @@ PARKED: filed with an explicit long deadline, then recorded in
118
150
  ```json
119
151
  {"rounds": [{"ping_id": "<uuid>", "filed_at": "<ISO>", "platform": "windows",
120
152
  "build": "<the hosted or store URL>", "est_minutes": 10,
121
- "deadline_seconds": 86400, "checks": ["<check ids>"],
122
- "note": "<the patch this round covers>"}]}
153
+ "deadline_seconds": 86400, "game": "solar-drift",
154
+ "checks": ["<check ids>"], "note": "<the patch this round covers>"}]}
123
155
  ```
124
156
 
125
157
  `qaping rounds` lists it with each round's live status, `qaping rounds add`
126
- records one, `qaping rounds rm` drops it once collected. This file is the
158
+ (with `--game` for the board this round carries) records one,
159
+ `qaping rounds rm` drops it once collected. This file is the
127
160
  opposite of the ledger and never merges with it: mutable OPEN state, one entry
128
161
  per round still owed, emptied as rounds land. It is not history — gitignore it.
129
162
 
package/docs/icon.svg ADDED
@@ -0,0 +1 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64"><rect x="4" y="4" width="56" height="56" rx="14" fill="#FF7A45"/><path d="M20 33.5l8 8 16-17" fill="none" stroke="#fff" stroke-width="6.5" stroke-linecap="round" stroke-linejoin="round"/></svg>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qaping/cli",
3
- "version": "0.2.0",
3
+ "version": "0.3.0",
4
4
  "homepage": "https://qaping.dev",
5
5
  "description": "qaping: automatic QA for your game. Your coding agent authors and maintains the QA plan, ships each patch's build to real human playtesters, and reports what broke.",
6
6
  "keywords": [
@@ -25,7 +25,7 @@
25
25
  "node": "^20.17.0 || ^22.13.0 || >=23.5.0"
26
26
  },
27
27
  "dependencies": {
28
- "pingfusi": "^0.16.1"
28
+ "pingfusi": "^0.17.1"
29
29
  },
30
30
  "publishConfig": {
31
31
  "access": "public"
@@ -0,0 +1,19 @@
1
+ {
2
+ "note": "sha256 of every SKILL.md version this package has shipped (LF-normalized). setup refreshes an installed skill whose hash is listed here — it is ours — and preserves anything else as a local edit. APPEND-ONLY; regenerate with: node scripts/gen-skill-hashes.js [<skill-root>...]",
3
+ "skills": {
4
+ "qaping": [
5
+ "184aa97ba0d529eea36be3c6eb4a5f4c1db8e2a65a0230e03814276a18a92354",
6
+ "217ffcc582e068424eefbb69376825ac71433fa92f34e3546bd02d8521ef008c",
7
+ "2fe5cafeef48146d8c9cc263ae1aaef764720027d17c0af3ab879e6853cf4a26",
8
+ "38cdc3289db6a544bc0cc828f4981a3f4ed6b39036ce81eebbaf4cbe9c47a5d0",
9
+ "490ff02895fdc6a2eb3a0b671628a8d4eee3efd906782eb45d99eac40bf59846",
10
+ "5462b858f58d183a020a5b2fa214dd1da6856e71a238316641f13d70eb0f0df4",
11
+ "5d0751c023af0a1cf69f152ba587b10c34087c7d5299f2d4b0e782f1ad0d66b0",
12
+ "6d0a5a1b430a8a0fd1c7c06b726de657e7fe82ac37ce69c9446539065d2ce751",
13
+ "71e64cd2b58996236c7fd3375a324ca2ae7b3e4028f7b358a389c562464c8bf3",
14
+ "8d817af568154734cd893122ce2e40ef2cd20cb725abbf318f134318157e5b6c",
15
+ "966bd642f1ee052691c7974a186e9f8f50644bce0ea010b6f0b444931879dbad",
16
+ "b9d08318e69290759bf31e149d64207b9d186faac9ccd01bc1db3ffcff29506c"
17
+ ]
18
+ }
19
+ }
@@ -11,9 +11,12 @@ build, ship the build to real human playtesters, and report ONE typed outcome
11
11
  per check — never a vibe. The check is the unit of value: its id rides from
12
12
  the plan onto the wire, into the report, and into the round ledger. Over
13
13
  patches, checks migrate from human rounds to repo code tests, so QA gets
14
- cheaper. All service contact goes through three MCP tools
15
- `qaping_playtest`, `qaping_results`, `qaping_wait` plus
16
- `qaping publish-build` for hosting the build.
14
+ cheaper. Checks are what the game must always do; BUGS are what it does wrong
15
+ right now, and they live on the game's issue board, where a fix you claim
16
+ turns verified only when a playtester says it is gone. All service contact
17
+ goes through four MCP tools — `qaping_playtest`, `qaping_results`,
18
+ `qaping_wait`, `qaping_issues` — plus `qaping publish-build` for hosting the
19
+ build.
17
20
 
18
21
  ## Ground rules (read before filing anything)
19
22
 
@@ -55,9 +58,12 @@ cheaper. All service contact goes through three MCP tools —
55
58
  informed.
56
59
  - **Evidence varies by platform — never promise what a platform does not
57
60
  ship.** macOS and iOS rounds return a screen+voice recording plus an inline
58
- [mm:ss] think-aloud transcript. Windows rounds return a recording and NO
59
- transcript (`transcript_status:'unavailable'`) never promise one; plan to
60
- watch the recording for timestamps. WEB rounds return answers only — no
61
+ [mm:ss] think-aloud transcript. Windows rounds return a recording, and a
62
+ transcript only when the tester recorded with our recorder app — read
63
+ `transcript_status` on each result: `'ok'` means an inline transcript,
64
+ `'unavailable'` means a Game Bar session with NO transcript. It is a
65
+ per-session fact, never promise one for a Windows round; plan to watch an
66
+ `'unavailable'` recording for timestamps. WEB rounds return answers only — no
61
67
  recording, no transcript: the tester's option picks and free text are the
62
68
  entire deliverable, which is why every check rides an option step.
63
69
  - **Pay is flat per session, never per bug.** Never offer or imply a bounty.
@@ -101,6 +107,60 @@ once per recorded `ping_id`. The file is this, and nothing else:
101
107
  `qa-open-rounds.json` is mutable OPEN state; `qa-rounds.jsonl` is append-only
102
108
  history written after results land. Never merge the two.
103
109
 
110
+ ## The issue board — bugs live here
111
+
112
+ Checks are the standing contract; bugs are what is wrong today, and they live
113
+ on the game's issue board — one board per `game_slug`, numbered `#1, #2, …`
114
+ for the life of the game, read and moved with `qaping_issues`. The board is
115
+ what carries a bug across rounds: the checklist a round hands the playtester
116
+ is built from it, and it is the only place a fix can ever be confirmed.
117
+
118
+ | status | who moves it there | how |
119
+ |---|---|---|
120
+ | `open` | you (`action:'create'`), or a playtester filing a bug from inside a round | the live state, and where a regression lands |
121
+ | `fixed` | YOU only (`action:'fixed'`, `build` REQUIRED) | a CLAIM, not a verification |
122
+ | `verified` | a PLAYTESTER only — a "Gone" on a round filed AFTER your claim | there is no verify action and there never will be |
123
+ | `closed` | you only (`action:'close'` + reason wontfix / invalid / duplicate / cannot_reproduce) | leaves the checklist; never turns green |
124
+
125
+ A "Still here" on a `fixed` or `verified` issue REGRESSES it back to `open`
126
+ and counts the regression. A "Gone" on an issue nobody claimed fixed changes
127
+ nothing — it may simply not have reproduced this session, and the board
128
+ records that as a not-seen streak for you to act on (close it
129
+ `cannot_reproduce` once you believe it).
130
+
131
+ - **The board is the source of truth for bugs; the dev's GitHub issues are
132
+ the mirror (MUST).** File it on the board first, then `gh issue create` for
133
+ the dev's own triage, then put that URL back on the board with
134
+ `action:'edit'` + `external_ref`. A bug that lives only in GitHub is never
135
+ re-checked by anyone, because only the board rides a round.
136
+ - **Issue titles and bodies speak player language (MUST)** — the same rule
137
+ the steps live under, for the same reason: a stranger reads them
138
+ mid-session on their own machine. "The shop closes when you press B twice",
139
+ never "NRE in ShopController.OnClose". No file paths, no stack traces, no
140
+ ticket ids. Title ≤120 chars, body ≤1000.
141
+ - **No build, no claim (MUST).** `action:'fixed'` takes the exact build URL
142
+ the NEXT round will play, and the claim happens BEFORE that round is filed.
143
+ A fix claimed after the round exists was not in the build that playtester
144
+ ran, so their answer cannot confirm it — the board leaves it amber and says
145
+ why.
146
+ - **`game` on every playtest filing (MUST).** The board's open and
147
+ claimed-fixed issues — up to 12, claimed-fixed first — ride the round as
148
+ the playtester's checklist after free play. It costs no step budget, adds
149
+ ~30 seconds per issue inside est_minutes, and is the ONLY path to
150
+ `verified`. Omit it and the round has no memory.
151
+ - **Never leave a playtester-filed issue untriaged (MUST).** A bug reported
152
+ from inside a round lands `open` with origin `reviewer`, and answering it
153
+ is part of reporting that round — accept it, or close it duplicate /
154
+ invalid / cannot_reproduce.
155
+ - **Where the checklist is answered today, honestly.** The Windows browser
156
+ runner renders it. Rounds answered in the reviewer app (macOS, iOS) and web
157
+ rounds carry the board but do not show it yet, so their issues come back
158
+ unanswered — which is not evidence about the bug. An issue nobody answered
159
+ never moves; never report an unanswered checklist as "no longer
160
+ reproducing".
161
+ - Nothing on the board costs credits — only rounds do. `qaping issues <game>`
162
+ reads the same board from the shell.
163
+
104
164
  ## The review interface (know your tester)
105
165
 
106
166
  <!-- Distilled from QAPING_REVIEWER_INTERFACE.md (monorepo root, INTERNAL,
@@ -151,10 +211,15 @@ that assumes they read your repo.
151
211
  frontmatter so every future run can build without asking again.
152
212
  3. Author `QA-PLAN.md` at the repo root in EXACTLY this format (self-contained
153
213
  here — do not go looking for other docs). Plan frontmatter (YAML at the
154
- very top): `game`, `build_command` (the ONE shippable-build command),
155
- `platforms: [windows|macos|ios|web]` (`web` = a browser-playable game — its
156
- "build" is a static directory, not a zip), optional `input` (hardware;
157
- default keyboard+mouse, or touch for a mobile-web game).
214
+ very top): `game`, `game_slug`, `build_command` (the ONE shippable-build
215
+ command), `platforms: [windows|macos|ios|web]` (`web` = a browser-playable
216
+ game — its "build" is a static directory, not a zip), optional `input`
217
+ (hardware; default keyboard+mouse, or touch for a mobile-web game).
218
+ **Mint `game_slug` ONCE**, from the game name: lowercase, every run of
219
+ non-alphanumerics collapsed to a single `-`, no leading or trailing `-`,
220
+ ≤64 chars ("Solar Drift" → `solar-drift`). It is the issue board's id and
221
+ is NEVER renamed — the same rule check ids live under, for the same reason:
222
+ a renamed slug orphans every issue filed under the old one.
158
223
  Then one `##` section per check — the heading is the check's short name —
159
224
  opening with a fenced yaml block of `id` (stable slug, never renamed),
160
225
  `rung: human | code`, `state` (`unverified | passed | ambiguous | failing`
@@ -217,6 +282,10 @@ checks testimony-only — mark that in the plan and report at that grade.
217
282
 
218
283
  ## RUN — per patch
219
284
 
285
+ 0. **Read the board.** `qaping_issues` with the plan's `game_slug` (action
286
+ `list`): what is open, what you claimed fixed and is still waiting on a
287
+ playtester, what came back REGRESSED. This is the bug half of "what
288
+ happened since last time"; the ledger's last line is the check half.
220
289
  1. **Select.** Read the diff (or PR). Pick the checks it can plausibly affect,
221
290
  plus the always-run criticals (the critical-path block above) and every
222
291
  check the ledger left NOT-ASKED or Blocked last round. Tell the user which
@@ -252,11 +321,19 @@ checks testimony-only — mark that in the plan and report at that grade.
252
321
  players open the link on their own devices. When filing a WEB game: send
253
322
  `url` + `est_minutes` and OMIT `platform` entirely — a web round is claimed
254
323
  by players in their own browser, phones included.
255
- 5. **File** ONE `qaping_playtest` per composed session, always with
324
+ 5. **Claim fixes (MUST, before filing).** For every board issue this patch
325
+ fixes: `qaping_issues` `action:'fixed'` with its `seq`, `build` set to the
326
+ exact URL this round will play, and a `note` written for the playtester
327
+ ("the shop stays open now"). Claim first, file second — a claim recorded
328
+ after the round exists cannot be confirmed by it, and the results say so
329
+ instead of turning it green. No build URL, no claim.
330
+ 6. **File** ONE `qaping_playtest` per composed session, always with
256
331
  `questionnaire:'none'` — on a QA-PLAN round your checks are the questions;
257
332
  the standard instrument is a separate experience round (below), never a
258
- default. A WEB session additionally files `require_evidence:'none'`
259
- (MUST) left at its default, the finish silently demands a pinned
333
+ default and always with `game` set to the plan's `game_slug` (MUST), so
334
+ the board's live issues ride the round as the playtester's checklist.
335
+ A WEB session additionally files `require_evidence:'none'` (MUST) — left
336
+ at its default, the finish silently demands a pinned
260
337
  screenshot-comment no step asked for. est_minutes is an
261
338
  honest sum of the steps' directed play — the service
262
339
  accepts 5–30 minutes (10 is the standard session; outside that
@@ -282,11 +359,14 @@ checks testimony-only — mark that in the plan and report at that grade.
282
359
  player-visible expectation. One whose precondition cannot be forced
283
360
  opens with a verifying option step ("Before this session: had you ever
284
361
  played this game before?" — yes / no / not sure);
285
- - a recheck of a prior finding is a clearly-labeled step at its natural
286
- play-order point (first when its required state demands it, last
287
- otherwise) "VERIFY FIX: last round saw X does it still happen?" —
288
- and prior findings stay OUT of `instructions` entirely, so the rest of
289
- the pass is an independent look;
362
+ - **a recheck of a known bug is NEVER a step.** The board rides the
363
+ checklist: `game` on the filing attaches the open and claimed-fixed
364
+ issues, the playtester answers them (Still here / Gone / Couldn't test)
365
+ after free play, and that costs none of the step budget. Never author a
366
+ step that re-asks an issue already on the board — a step and a checklist
367
+ row about the same bug buy one answer twice — and prior findings stay
368
+ OUT of `instructions` entirely, so the rest of the pass is an
369
+ independent look;
290
370
  - a closing **break-it slice** (~2–3 minutes) authored from the diff:
291
371
  "this patch changed X — try to break it: <concrete abuse ideas from the
292
372
  diff>. Report the weirdest thing you saw AND list what you tried."
@@ -307,18 +387,24 @@ checks testimony-only — mark that in the plan and report at that grade.
307
387
  the standard questionnaire rides) — rows past 20 are answered by the
308
388
  tester and silently dropped at finish, which is why this gate exists. A
309
389
  mismatch (the budget forcing cuts) fails loudly PRE-SPEND: batch another
310
- session or state the deferral out loud — never silently truncate.
311
- 6. **Wait or park decide now** (ground rule 1). WAIT: spawn a background
390
+ session or state the deferral out loud — never silently truncate. Then
391
+ print `issues: <seqs attached>` from the filing's own result (it names the
392
+ checklist it built, and anything the 12-issue cap left off), or
393
+ `issues: none` on an empty board — so what this round RE-CHECKS is on the
394
+ record beside what it covers.
395
+ 7. **Wait or park — decide now** (ground rule 1). WAIT: spawn a background
312
396
  task looping `qaping_wait` and keep working — foreground polling is the
313
397
  fallback. PARK: the filing already carried an explicit `deadline_seconds`
314
398
  sized to the span, so record it — `qaping rounds add <ping_id> --build
315
399
  <url> --minutes N --checks <ids> --note "<patch>"` — tell the user what is
316
400
  pending, and end the turn; a later session collects it. A promise to check
317
401
  later is never a wait, and never a substitute for the record.
318
- 7. **Report** from `qaping_results`:
402
+ 8. **Report** from `qaping_results`:
319
403
  - **Evidence line first (MUST).** Every report opens with its grade:
320
- `Evidence: recording + transcript` (macOS/iOS) / `Evidence: recording
321
- only` (Windows) / `Evidence: testimony only (web round)`.
404
+ `Evidence: recording + transcript` (macOS/iOS, or a Windows result with
405
+ `transcript_status:'ok'`) / `Evidence: recording only` (a Windows result
406
+ with `transcript_status:'unavailable'` — a Game Bar session) /
407
+ `Evidence: testimony only (web round)`.
322
408
  - **Computed headline (MUST).** Derive the one verdict sentence from the
323
409
  per-check outcomes, never from the tester's own verdict: `PASS` only
324
410
  when every selected check returned Pass; `PASS — N unverified (<ids>)`
@@ -326,16 +412,23 @@ checks testimony-only — mark that in the plan and report at that grade.
326
412
  on any Fail. Quote the tester's global verdict beneath it as one input —
327
413
  it is never the headline. (The iterate rule already says unresolved
328
414
  findings mean fix and refile — unresolved never headlines as PASS.)
415
+ - **Issues line (MUST), under the headline.** One line read off the
416
+ results' issue block: `issues: #3 verified, #5 REGRESSED, 2 new` —
417
+ verified first, regressions next, then how many bugs playtesters filed
418
+ from inside the session. `issues: none moved` when the board rode along
419
+ and nothing changed; nothing at all when the round carried no `game`. A
420
+ REGRESSED issue is a failure of a fix you claimed: say so in words, in
421
+ the same breath as the headline.
329
422
  - **One row per SELECTED check (MUST)**, keyed by id, outcome from
330
423
  {Pass, Fail, Blocked, NOT-ASKED}. "Couldn't test" and "Didn't reach this
331
424
  part" are Blocked, with the why. A check that reached the wire as more
332
- than one row (a precondition step, a VERIFY FIX pair) takes its outcome
333
- from the WORST row: any Fail → Fail; else any Blocked or unanswered row
334
- leaves the check unverified and the headline counts it ("PASS — N
335
- unverified") — never plain Pass over a Blocked sub-step. A check whose precondition came back
336
- unverified or disproven is VOID — report it as Blocked ("VOID:
337
- precondition unverified"), never as a pass or fail, and leave
338
- `last_verified` untouched. A selected check that never made the wire or
425
+ than one row (a precondition step plus its observation step) takes its
426
+ outcome from the WORST row: any Fail → Fail; else any Blocked or
427
+ unanswered row leaves the check unverified and the headline counts it
428
+ ("PASS — N unverified") — never plain Pass over a Blocked sub-step. A
429
+ check whose precondition came back unverified or disproven is VOID —
430
+ report it as Blocked ("VOID: precondition unverified"),
431
+ never as a pass or fail, and leave `last_verified` untouched. A selected check that never made the wire or
339
432
  never came back is NOT-ASKED and carries forward into the next filing
340
433
  automatically. Cite [mm:ss] where a transcript or recording exists; on a
341
434
  web round cite the step answers — there are no timestamps to cite.
@@ -344,8 +437,15 @@ checks testimony-only — mark that in the plan and report at that grade.
344
437
  - **Ledger append (MUST).** Append ONE JSON line to `qa-rounds.jsonl` at
345
438
  the game repo root, committed alongside the plan updates:
346
439
  `{"round_id": …, "report_url": …, "build": "<sha or hosted url>",
347
- "evidence_grade": …, "checks": [{"id": …, "outcome": …}], "verdict": …,
348
- "credits": …, "follow_ups": []}`. `report_url` is the round's service
440
+ "evidence_grade": …, "checks": [{"id": …, "outcome": …}],
441
+ "issues": {"game": "<game_slug>", "verified": [3], "regressed": [5],
442
+ "new": [7, 8], "still_open": [2]}, "verdict": …, "credits": …,
443
+ "follow_ups": […]}`. The `issues` object is the board's movement this
444
+ round, by seq (omit it entirely when the round carried no `game`), and
445
+ `follow_ups` is NON-BUG follow-ups only from now on — a check to add, a
446
+ build affordance to write, a question to settle. A bug goes on the
447
+ board, where it is numbered, re-checked and closable; free text nobody
448
+ re-reads is where bugs went to die. `report_url` is the round's service
349
449
  report page (the `report_url` the results tool returns — never a PR or
350
450
  issue link). No evidence-link field, deliberately: signed recording URLs
351
451
  expire in days, so durable evidence pointers live in the report/issues,
@@ -367,14 +467,33 @@ checks testimony-only — mark that in the plan and report at that grade.
367
467
  a fragile area, or a QA lesson learned, offer to record it as a check
368
468
  (or a note on an existing check) in `QA-PLAN.md` right then, so the plan
369
469
  accretes knowledge instead of losing it to chat scrollback.
370
- - Issues: one `gh issue create` per finding drafts for the dev's own
371
- triage, never near-duplicates, never one issue per symptom of the same
372
- bug. Before reproduction the body carries "observed once, unreproduced"
373
- plus a proposed severity from consequence (blocks progress / degrades /
374
- cosmetic). The label CONFIRMED is reserved for a targeted round or
375
- unambiguous in-round evidence; a CONFIRMED issue carries repro steps
376
- rebuilt from the tester's answers and a timestamp link into the
377
- recording where one exists (web rounds have none — cite answers).
470
+ - **Triage every new issue in THIS turn (MUST).** Each `NEW #n` in the
471
+ results is a bug a playtester filed from inside the session, and it is
472
+ answered before the report is finished: accept it (`gh issue create` for
473
+ the dev's triage, then `qaping_issues` `action:'edit'` with
474
+ `external_ref` set to that URL), or close it `duplicate` with
475
+ `duplicate_of_seq`, `invalid`, or `cannot_reproduce`. Never leave one
476
+ sitting; an untriaged board stops being worth reading, and the next
477
+ round re-asks a bug nobody decided about.
478
+ - **A REGRESSED issue reopens its mirror (MUST).** The fix did not hold:
479
+ `gh issue reopen <n>` on the GitHub issue, name it in the PR comment,
480
+ and never claim `fixed` on it again without a new build to claim
481
+ against.
482
+ - **A verified issue closes its mirror.** An issue a playtester turned
483
+ `verified` is settled: close the GitHub issue citing the round. Leave
484
+ the board row green — never `action:'close'` a verified issue to tidy
485
+ up; closed is for what you decided about, not for what was confirmed.
486
+ - Findings YOU file: one board issue per finding (`action:'create'`, in
487
+ player language, severity from consequence — `blocker` cannot play past
488
+ it / `major` / `minor` / `cosmetic`), then one `gh issue create` per
489
+ finding as the dev's own draft — never near-duplicates, never one issue
490
+ per symptom of the same bug — and `action:'edit'` the board row with the
491
+ `external_ref` you just created. Before reproduction the body carries
492
+ "observed once, unreproduced" plus that proposed severity. The label
493
+ CONFIRMED is reserved for a targeted round or unambiguous in-round
494
+ evidence; a CONFIRMED issue carries repro steps rebuilt from the
495
+ tester's answers and a timestamp link into the recording where one
496
+ exists (web rounds have none — cite answers).
378
497
 
379
498
  ## The ambiguity ladder
380
499
 
@@ -384,7 +503,11 @@ precondition forced via `setup` (or verified by the opening option step),
384
503
  plus the disambiguation steps. That round settles it: Pass → `passed` (note
385
504
  the fluke); Fail → `failing`, and the issue upgrades to CONFIRMED. Never
386
505
  resolve an ambiguity by re-buying the whole batch, and never label a
387
- once-seen fail CONFIRMED without that round or hard evidence.
506
+ once-seen fail CONFIRMED without that round or hard evidence. When the
507
+ targeted round settles it as a real bug, the board issue for it carries
508
+ `check_id: <the check's id>` — set at `create`, or with `action:'edit'` on a
509
+ playtester-filed issue — so the check and the bug that made it ambiguous stay
510
+ findable from each other.
388
511
 
389
512
  ## Paired rounds (two-state comparisons)
390
513