@qaping/cli 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/qaping +90 -13
- package/docs/COMMANDS.md +122 -0
- package/docs/QA-PLAN-FORMAT.md +38 -5
- package/docs/icon.svg +1 -0
- package/package.json +2 -2
- package/skill/SHIPPED-HASHES.json +19 -0
- package/skill/qaping/SKILL.md +164 -41
package/bin/qaping
CHANGED
|
@@ -15,6 +15,7 @@
|
|
|
15
15
|
// qaping wait <ping_id> [--timeout <seconds>]
|
|
16
16
|
// qaping results <ping_id> one passive fetch, no blocking
|
|
17
17
|
// qaping rounds [add|rm] the parked rounds this repo is owed
|
|
18
|
+
// qaping issues <game> the game's issue board, read-only
|
|
18
19
|
// qaping whoami | version | help
|
|
19
20
|
"use strict";
|
|
20
21
|
const fs = require("fs");
|
|
@@ -50,11 +51,12 @@ function resolveKitDir() {
|
|
|
50
51
|
// writes it to ~/.claude/rules/qaping.md and ~/.cursor/rules/qaping.mdc.
|
|
51
52
|
const RULE_BODY = `This machine has qaping: automatic QA for games — real human playtesters play the developer's builds on their own hardware and report back, and the coding agent runs the whole loop.
|
|
52
53
|
Whenever the user asks to set up QA for a game, run QA on a patch, check whether a change broke anything, or playtest a game or build, load the qaping skill and follow it — it owns authoring/maintaining QA-PLAN.md in the game repo and the per-patch run.
|
|
53
|
-
Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead). Hosted builds are
|
|
54
|
-
The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a round, free)
|
|
54
|
+
Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead). Hosted builds are kept until you delete them (pass \`--expires-in-hours <n>\` to have one reaped instead) and capped per account — if a publish is refused for that cap, \`qaping builds\` lists them and \`qaping builds rm <slug>\` frees a slot; free one and carry on rather than stopping to ask, taking the ones marked as never-finished uploads first and never deleting a build a round you filed is still running against.
|
|
55
|
+
The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a round, free), qaping_wait (continue a pending round) and qaping_issues (the game's issue board — file a bug, claim a fix, close or reopen; free).
|
|
56
|
+
Bugs live on that board, not in chat or in a report nobody re-reads: file EVERY playtest with \`game\` set to the QA plan's game_slug, so the game's open and claimed-fixed issues ride the round as the playtester's checklist. You can only ever CLAIM a fix (qaping_issues action:'fixed', with the exact build URL the next round will play, claimed BEFORE that round is filed) — only a playtester answering "Gone" on a later round marks one verified, and a "Still here" on a claimed fix regresses it. \`qaping issues <game>\` reads the same board from the shell.
|
|
55
57
|
Decide at filing time how the round ends: a short round waits in place — the moment filing returns pending, spawn a background task that loops qaping_wait and keep working (foreground polling is the fallback); a long or overnight round is PARKED — file it with an explicit long deadline_seconds, record it immediately with \`qaping rounds add <ping_id>\` (it writes qa-open-rounds.json beside QA-PLAN.md), tell the user exactly what is pending, and end the turn; a later session collects it (\`qaping rounds\` lists them with live status, qaping_results fetches one).
|
|
56
58
|
A recorded parked round is an honest pending end — never report pending as the answer without one, and never end with an unbacked promise to check later.
|
|
57
|
-
Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording and no transcript; web rounds return answers only (no recording, no transcript) — never promise evidence a
|
|
59
|
+
Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording, and a transcript only when the tester recorded with our recorder app (read transcript_status on each result — a Game Bar session has no transcript); web rounds return answers only (no recording, no transcript) — never promise evidence a session does not ship.
|
|
58
60
|
QA memory: whenever the developer mentions — in ANY conversation, not just QA runs — something that should always be tested, a fragile area, or a QA lesson learned, offer to record it as a check (or a note on an existing check) in QA-PLAN.md so it is tested from then on.
|
|
59
61
|
`;
|
|
60
62
|
|
|
@@ -255,13 +257,18 @@ const ROUNDS_FILE = "qa-open-rounds.json";
|
|
|
255
257
|
// what `qaping rounds` costs in requests.
|
|
256
258
|
const MAX_OPEN_ROUNDS = 20;
|
|
257
259
|
const PING_ID_RE = /^[0-9a-f-]{36}$/i;
|
|
260
|
+
// The issue board's game slug. Mirrors lib/qa-issues.ts's GAME_SLUG_RE and
|
|
261
|
+
// migration 066's column check exactly — a slug this refuses is a slug the
|
|
262
|
+
// service refuses, and finding that out before the network is the point.
|
|
263
|
+
const GAME_SLUG_RE = /^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$/;
|
|
258
264
|
const DEFAULT_APP_URL = "https://pingfusi.com";
|
|
259
265
|
|
|
260
266
|
const ROUNDS_USAGE = `usage:
|
|
261
267
|
qaping rounds [--json] [--offline] [--file <path>]
|
|
262
268
|
qaping rounds add <ping_id> [--platform windows|macos|ios] [--build <url>]
|
|
263
269
|
[--minutes N] [--deadline-seconds N]
|
|
264
|
-
[--checks a,b,c]
|
|
270
|
+
[--game <slug>] [--checks a,b,c]
|
|
271
|
+
[--note "<patch>"] [--json]
|
|
265
272
|
qaping rounds rm <ping_id> [--json]`;
|
|
266
273
|
|
|
267
274
|
function flagValue(argv, name) {
|
|
@@ -440,6 +447,9 @@ function renderRounds(entries, opts) {
|
|
|
440
447
|
lines.push("");
|
|
441
448
|
lines.push(` ${record.ping_id}${facts ? ` — ${facts}` : ""}`);
|
|
442
449
|
if (record.build) lines.push(` build: ${record.build}`);
|
|
450
|
+
// Which board's issues rode this round — the later session that collects
|
|
451
|
+
// it needs the slug to read what moved (`qaping issues <game>`).
|
|
452
|
+
if (record.game) lines.push(` game: ${record.game}`);
|
|
443
453
|
if (Array.isArray(record.checks) && record.checks.length) lines.push(` checks: ${record.checks.join(", ")}`);
|
|
444
454
|
if (record.note) lines.push(` patch: ${record.note}`);
|
|
445
455
|
if (live) lines.push(` ${roundStatusLine(record, live)}`);
|
|
@@ -493,13 +503,14 @@ function resolveTokenFromClientConfigs(kitDir) {
|
|
|
493
503
|
return null;
|
|
494
504
|
}
|
|
495
505
|
|
|
496
|
-
// One passive tools/call against the qaping mount
|
|
497
|
-
// same JSON-or-SSE parse the kit's `wait` leg uses,
|
|
498
|
-
// ONE wrapper object every other command is driven
|
|
499
|
-
//
|
|
500
|
-
// deadline. Failures are TYPED for the caller:
|
|
501
|
-
// separate "this login is dead" from "the service
|
|
502
|
-
|
|
506
|
+
// One passive tools/call against the qaping mount, for ANY tool it registers —
|
|
507
|
+
// the same transport and the same JSON-or-SSE parse the kit's `wait` leg uses,
|
|
508
|
+
// with the mount read from the ONE wrapper object every other command is driven
|
|
509
|
+
// by. Passive: it never renews a round's lease, which is exactly why a parked
|
|
510
|
+
// round needs its own explicit deadline. Failures are TYPED for the caller:
|
|
511
|
+
// `authRejected` and `perRound` separate "this login is dead" from "the service
|
|
512
|
+
// refused this one call".
|
|
513
|
+
async function callQapingTool(name, args) {
|
|
503
514
|
const w = qapingWrapper();
|
|
504
515
|
const kitDir = resolveKitDir();
|
|
505
516
|
const { resolveToken } = require(path.join(kitDir, "packages", "core", "wire.js"));
|
|
@@ -518,7 +529,7 @@ async function callResultsTool(pingId) {
|
|
|
518
529
|
jsonrpc: "2.0",
|
|
519
530
|
id: 1,
|
|
520
531
|
method: "tools/call",
|
|
521
|
-
params: { name
|
|
532
|
+
params: { name, arguments: args },
|
|
522
533
|
}),
|
|
523
534
|
signal: AbortSignal.timeout(20_000),
|
|
524
535
|
});
|
|
@@ -551,10 +562,18 @@ async function callResultsTool(pingId) {
|
|
|
551
562
|
status: structured.status || "pending",
|
|
552
563
|
structured,
|
|
553
564
|
result,
|
|
565
|
+
// A tool that refuses inside a successful JSON-RPC result (the board at
|
|
566
|
+
// its cap, a seq that is not there) answers with isError + the same text.
|
|
567
|
+
isError: result.isError === true,
|
|
554
568
|
text: (result.content && result.content[0] && result.content[0].text) || "",
|
|
555
569
|
};
|
|
556
570
|
}
|
|
557
571
|
|
|
572
|
+
/** The results tool, by name — every caller below reads the same shape. */
|
|
573
|
+
function callResultsTool(pingId) {
|
|
574
|
+
return callQapingTool("qaping_results", { ping_id: pingId });
|
|
575
|
+
}
|
|
576
|
+
|
|
558
577
|
async function cmdResults(argv) {
|
|
559
578
|
const json = argv.includes("--json");
|
|
560
579
|
// Flags in any position: `qaping results --json <id>` is the natural thing to
|
|
@@ -584,6 +603,52 @@ async function cmdResults(argv) {
|
|
|
584
603
|
process.exit(0);
|
|
585
604
|
}
|
|
586
605
|
|
|
606
|
+
const ISSUES_USAGE = "usage: qaping issues <game> [--json] [--status open|fixed|verified|closed]";
|
|
607
|
+
const ISSUE_STATUSES = ["open", "fixed", "verified", "closed"];
|
|
608
|
+
|
|
609
|
+
/** argv after the command, minus flags and the values they consume — so a
|
|
610
|
+
* slug that happens to equal a flag's value is still the positional. */
|
|
611
|
+
function positionalArgs(argv, valueFlags) {
|
|
612
|
+
const out = [];
|
|
613
|
+
for (let i = 1; i < argv.length; i++) {
|
|
614
|
+
if (argv[i].startsWith("--")) {
|
|
615
|
+
if (valueFlags.includes(argv[i])) i++;
|
|
616
|
+
continue;
|
|
617
|
+
}
|
|
618
|
+
out.push(argv[i]);
|
|
619
|
+
}
|
|
620
|
+
return out;
|
|
621
|
+
}
|
|
622
|
+
|
|
623
|
+
/**
|
|
624
|
+
* The game's issue board, read-only. The agent moves issues through the MCP
|
|
625
|
+
* tool; this is the human's window on the same board — and it prints the
|
|
626
|
+
* SERVICE's own rendering rather than a second one, so the board a developer
|
|
627
|
+
* reads in the shell and the board an agent reads mid-run cannot disagree.
|
|
628
|
+
*/
|
|
629
|
+
async function cmdIssues(argv) {
|
|
630
|
+
const json = argv.includes("--json");
|
|
631
|
+
const game = positionalArgs(argv, ["--status"])[0];
|
|
632
|
+
if (!game || !GAME_SLUG_RE.test(game)) {
|
|
633
|
+
console.error(`✗ not a game slug: ${game || "(missing)"} — lowercase letters, digits and hyphens.\n${ISSUES_USAGE}`);
|
|
634
|
+
// 1, like `results`: a typo must never reach the network, and 2 is
|
|
635
|
+
// reserved by the round verbs for "still pending".
|
|
636
|
+
process.exit(1);
|
|
637
|
+
}
|
|
638
|
+
const status = flagValue(argv, "--status");
|
|
639
|
+
if (status !== undefined && !ISSUE_STATUSES.includes(status)) {
|
|
640
|
+
console.error(`✗ --status must be one of ${ISSUE_STATUSES.join(", ")}.\n${ISSUES_USAGE}`);
|
|
641
|
+
process.exit(1);
|
|
642
|
+
}
|
|
643
|
+
const args = { game, action: "list" };
|
|
644
|
+
if (status) args.status_filter = status;
|
|
645
|
+
const r = await callQapingTool("qaping_issues", args);
|
|
646
|
+
console.log(json
|
|
647
|
+
? JSON.stringify(Object.keys(r.structured).length ? r.structured : r.result, null, 2)
|
|
648
|
+
: (r.text || JSON.stringify(r.result, null, 2)));
|
|
649
|
+
process.exit(r.isError ? 1 : 0);
|
|
650
|
+
}
|
|
651
|
+
|
|
587
652
|
async function cmdRoundsList(argv) {
|
|
588
653
|
const json = argv.includes("--json");
|
|
589
654
|
const state = readRoundsFile(roundsFilePath(argv));
|
|
@@ -650,6 +715,13 @@ function cmdRoundsAdd(argv) {
|
|
|
650
715
|
console.error(`✗ --platform must be windows, macos or ios — a web round has none, so omit it.\n${ROUNDS_USAGE}`);
|
|
651
716
|
process.exit(2);
|
|
652
717
|
}
|
|
718
|
+
// Validated here, not just recorded: a slug the service would refuse is a
|
|
719
|
+
// board the collecting session could never read.
|
|
720
|
+
const game = flagValue(argv, "--game");
|
|
721
|
+
if (game !== undefined && !GAME_SLUG_RE.test(game)) {
|
|
722
|
+
console.error(`✗ --game must be a slug: lowercase letters, digits and hyphens (the QA plan's game_slug).\n${ROUNDS_USAGE}`);
|
|
723
|
+
process.exit(2);
|
|
724
|
+
}
|
|
653
725
|
const file = roundsFilePath(argv);
|
|
654
726
|
const minutes = intFlag(argv, "--minutes");
|
|
655
727
|
const deadline = intFlag(argv, "--deadline-seconds");
|
|
@@ -658,6 +730,7 @@ function cmdRoundsAdd(argv) {
|
|
|
658
730
|
const build = flagValue(argv, "--build");
|
|
659
731
|
const patch = {};
|
|
660
732
|
if (platform) patch.platform = platform;
|
|
733
|
+
if (game) patch.game = game;
|
|
661
734
|
if (build) patch.build = build;
|
|
662
735
|
if (minutes != null) patch.est_minutes = minutes;
|
|
663
736
|
if (deadline != null) patch.deadline_seconds = deadline;
|
|
@@ -741,6 +814,7 @@ usage:
|
|
|
741
814
|
qaping rounds the parked rounds this repo is owed, with live status
|
|
742
815
|
qaping rounds add <ping_id> … record a parked round so a later session collects it
|
|
743
816
|
qaping rounds rm <ping_id> forget one, once it is collected or expired
|
|
817
|
+
qaping issues <game> the game's issue board (read-only; the agent moves it)
|
|
744
818
|
qaping whoami
|
|
745
819
|
qaping version
|
|
746
820
|
|
|
@@ -757,6 +831,7 @@ function route(cmd) {
|
|
|
757
831
|
if (cmd === "publish") return "publish";
|
|
758
832
|
if (cmd === "rounds") return "rounds";
|
|
759
833
|
if (cmd === "results") return "results";
|
|
834
|
+
if (cmd === "issues") return "issues";
|
|
760
835
|
return "unknown";
|
|
761
836
|
}
|
|
762
837
|
|
|
@@ -783,6 +858,7 @@ function main() {
|
|
|
783
858
|
case "publish": return void cmdPublish(argv);
|
|
784
859
|
case "rounds": return void cmdRounds(argv).catch(die);
|
|
785
860
|
case "results": return void cmdResults(argv).catch(die);
|
|
861
|
+
case "issues": return void cmdIssues(argv).catch(die);
|
|
786
862
|
default:
|
|
787
863
|
console.error(HELP);
|
|
788
864
|
process.exit(1);
|
|
@@ -798,5 +874,6 @@ function main() {
|
|
|
798
874
|
if (require.main === module) main();
|
|
799
875
|
module.exports = {
|
|
800
876
|
route, resolveKitDir, qapingWrapper, vendorFlags, isSourceCheckout, RULE_BODY, HELP,
|
|
801
|
-
ROUNDS_FILE, MAX_OPEN_ROUNDS,
|
|
877
|
+
ROUNDS_FILE, MAX_OPEN_ROUNDS, GAME_SLUG_RE,
|
|
878
|
+
parseRounds, upsertRound, removeRound, visibleUntil, renderRounds,
|
|
802
879
|
};
|
package/docs/COMMANDS.md
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
1
|
+
# qaping — CLI commands
|
|
2
|
+
|
|
3
|
+
Everything the `qaping` binary does. Your agent runs most of these for you;
|
|
4
|
+
`setup`, `builds` and `rounds` are the ones you will type yourself.
|
|
5
|
+
|
|
6
|
+
```
|
|
7
|
+
qaping setup [claude-code|cursor|codex] [--force]
|
|
8
|
+
qaping remove [--client <c>]
|
|
9
|
+
qaping publish-build <game.zip> --platform windows|macos
|
|
10
|
+
qaping builds
|
|
11
|
+
qaping builds rm <slug>
|
|
12
|
+
qaping publish <built-dir>
|
|
13
|
+
qaping wait <ping_id> [--timeout <seconds>]
|
|
14
|
+
qaping results <ping_id> [--json]
|
|
15
|
+
qaping rounds [--json] [--offline] [--file <path>]
|
|
16
|
+
qaping rounds add <ping_id> [--platform windows|macos|ios] [--build <url>]
|
|
17
|
+
[--minutes N] [--deadline-seconds N]
|
|
18
|
+
[--game <slug>] [--checks a,b,c]
|
|
19
|
+
[--note "<patch>"] [--json]
|
|
20
|
+
qaping rounds rm <ping_id> [--json]
|
|
21
|
+
qaping issues <game> [--json] [--status open|fixed|verified|closed]
|
|
22
|
+
qaping whoami
|
|
23
|
+
qaping version
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Getting in and out
|
|
27
|
+
|
|
28
|
+
### `qaping setup [claude-code|cursor|codex] [--force]`
|
|
29
|
+
|
|
30
|
+
Registers the qaping MCP server with your coding agent, installs the `qaping`
|
|
31
|
+
skill and its rule, and signs you in (a browser window opens once). With no
|
|
32
|
+
client name it detects what is installed. `--force` rewrites an existing
|
|
33
|
+
entry.
|
|
34
|
+
|
|
35
|
+
### `qaping remove [--client <c>]`
|
|
36
|
+
|
|
37
|
+
Removes qaping's MCP entry, skill and rule from the client. It touches only
|
|
38
|
+
qaping's own state — nothing else you have installed, and not your login.
|
|
39
|
+
|
|
40
|
+
### `qaping whoami`
|
|
41
|
+
|
|
42
|
+
Which account this machine is signed in as. Results are scoped to the account
|
|
43
|
+
that filed the round, so check this before deciding a round is gone.
|
|
44
|
+
|
|
45
|
+
### `qaping version`
|
|
46
|
+
|
|
47
|
+
Prints the installed version.
|
|
48
|
+
|
|
49
|
+
## Hosting a build
|
|
50
|
+
|
|
51
|
+
### `qaping publish-build <game.zip> --platform windows|macos`
|
|
52
|
+
|
|
53
|
+
Uploads a zipped native build and prints the `/b/<slug>` URL a round is filed
|
|
54
|
+
against. You hold a small number of hosted builds at a time; the oldest is
|
|
55
|
+
listed first by `qaping builds`.
|
|
56
|
+
|
|
57
|
+
### `qaping builds` / `qaping builds rm <slug>`
|
|
58
|
+
|
|
59
|
+
Lists the hosted builds you hold, or deletes one now to free its slot.
|
|
60
|
+
|
|
61
|
+
### `qaping publish <built-dir>`
|
|
62
|
+
|
|
63
|
+
Hosts a browser-playable game's built output and prints the public URL. Web
|
|
64
|
+
rounds are filed with that URL and no platform.
|
|
65
|
+
|
|
66
|
+
## Rounds
|
|
67
|
+
|
|
68
|
+
### `qaping wait <ping_id> [--timeout <seconds>]`
|
|
69
|
+
|
|
70
|
+
Waits on a filed round, renewing the lease that keeps it visible to new
|
|
71
|
+
playtesters, and prints the results when they land. The agent normally runs
|
|
72
|
+
this in a background task.
|
|
73
|
+
|
|
74
|
+
### `qaping results <ping_id> [--json]`
|
|
75
|
+
|
|
76
|
+
One passive fetch, no waiting. Exit code `0` means results are in, `2` means
|
|
77
|
+
still pending, `1` anything else (expired, not yours, unreachable).
|
|
78
|
+
|
|
79
|
+
### `qaping rounds`
|
|
80
|
+
|
|
81
|
+
The parked rounds this repo is owed, read from `qa-open-rounds.json` at the
|
|
82
|
+
repo root, each with its live status. `--offline` skips the status lookup;
|
|
83
|
+
`--file` points at another ledger.
|
|
84
|
+
|
|
85
|
+
### `qaping rounds add <ping_id> …`
|
|
86
|
+
|
|
87
|
+
Records a round the agent chose to park — an overnight run, a slow native
|
|
88
|
+
claim — so a later session collects it instead of filing a duplicate. Add
|
|
89
|
+
`--build`, `--minutes`, `--deadline-seconds`, `--game`, `--checks` and
|
|
90
|
+
`--note` so the record explains itself; `--game` is the QA plan's
|
|
91
|
+
`game_slug`, so the session that collects the round knows which issue board
|
|
92
|
+
it moved.
|
|
93
|
+
|
|
94
|
+
### `qaping rounds rm <ping_id>`
|
|
95
|
+
|
|
96
|
+
Forgets a parked round once it is collected or expired.
|
|
97
|
+
|
|
98
|
+
## The issue board
|
|
99
|
+
|
|
100
|
+
### `qaping issues <game> [--json]`
|
|
101
|
+
|
|
102
|
+
Prints the game's issue board: every bug that outlived a round, with its
|
|
103
|
+
status — open, fixed (you claim it is fixed), verified (a playtester
|
|
104
|
+
confirmed it is gone), closed. `--status` prints one status only; `--json`
|
|
105
|
+
prints the machine-readable rows.
|
|
106
|
+
|
|
107
|
+
Read-only, on purpose. Your agent files, claims, closes and reopens issues
|
|
108
|
+
through the `qaping_issues` MCP tool while it runs the loop; this is the
|
|
109
|
+
window you read it through. Nothing on the board costs credits — only rounds
|
|
110
|
+
do — and nothing turns *verified* except a playtester answering "Gone" on a
|
|
111
|
+
round filed after the fix was claimed.
|
|
112
|
+
|
|
113
|
+
## Files qaping leaves in your repo
|
|
114
|
+
|
|
115
|
+
| file | what it is | commit it? |
|
|
116
|
+
|---|---|---|
|
|
117
|
+
| `QA-PLAN.md` | the checks this game must pass; the agent authors, you own | yes — see [the format](QA-PLAN-FORMAT.md) |
|
|
118
|
+
| `qa-rounds.jsonl` | append-only history, one line per finished round | yes |
|
|
119
|
+
| `qa-open-rounds.json` | rounds filed and not yet collected | no — gitignore it |
|
|
120
|
+
|
|
121
|
+
The issue board is not a file: it lives on the service, one board per game,
|
|
122
|
+
and `qaping issues <game>` reads it. The plan's `game_slug` is what names it.
|
package/docs/QA-PLAN-FORMAT.md
CHANGED
|
@@ -11,12 +11,19 @@ YAML at the very top of the file:
|
|
|
11
11
|
```yaml
|
|
12
12
|
---
|
|
13
13
|
game: Solar Drift
|
|
14
|
+
game_slug: solar-drift # the issue board's id — minted ONCE, never renamed
|
|
14
15
|
build_command: ./scripts/package.sh --release # the ONE command that produces a shippable build
|
|
15
16
|
platforms: [windows] # windows | macos | ios | web
|
|
16
17
|
input: XInput gamepad # optional; default keyboard+mouse
|
|
17
18
|
---
|
|
18
19
|
```
|
|
19
20
|
|
|
21
|
+
`game_slug` is minted once from the game name — lowercase, every run of
|
|
22
|
+
non-alphanumerics collapsed to a single `-`, no leading or trailing `-`, ≤64
|
|
23
|
+
characters — and is then permanent, exactly like a check id: it names the
|
|
24
|
+
game's issue board, and renaming it orphans every issue filed under the old
|
|
25
|
+
one. Every playtest round is filed with it as `game`.
|
|
26
|
+
|
|
20
27
|
`build_command` is recorded once at setup so every future run can build without
|
|
21
28
|
asking. `platforms: [web]` means a browser-playable game — its "build" is a
|
|
22
29
|
static directory, not a zip. `input` declares the hardware the
|
|
@@ -88,6 +95,26 @@ setup: "?save=campaign-mid"
|
|
|
88
95
|
anything missing, corrupted, or visually wrong.
|
|
89
96
|
````
|
|
90
97
|
|
|
98
|
+
## Bugs live on the issue board; checks live here
|
|
99
|
+
|
|
100
|
+
A check is what the game must ALWAYS do — it belongs in this file, forever. A
|
|
101
|
+
bug is what the game does wrong right now — it belongs on the game's issue
|
|
102
|
+
board (`qaping issues <game_slug>`, moved by the agent through the
|
|
103
|
+
`qaping_issues` tool), where it is numbered, carried across rounds, and
|
|
104
|
+
retired when it is gone. Do not write bugs into `QA-PLAN.md`: a check written
|
|
105
|
+
to describe one bug ("the shop does not close when you press B twice") ages
|
|
106
|
+
into a puzzle the moment the bug is fixed.
|
|
107
|
+
|
|
108
|
+
The board is what makes a fix provable. The agent may CLAIM a fix (with the
|
|
109
|
+
exact build URL the next round will play); only a playtester answering
|
|
110
|
+
"Gone" on a round filed after that claim turns the issue *verified*, and a
|
|
111
|
+
"Still here" on a claimed fix regresses it. Because every round is filed with
|
|
112
|
+
`game`, the board's open and claimed-fixed issues ride it as the playtester's
|
|
113
|
+
checklist — no step budget spent, no bug forgotten between patches.
|
|
114
|
+
|
|
115
|
+
A check and the bug that came from it stay linked through the issue's
|
|
116
|
+
`check_id`: the same id this file gives the check.
|
|
117
|
+
|
|
91
118
|
## The round ledger — qa-rounds.jsonl
|
|
92
119
|
|
|
93
120
|
Beside the plan, at the game repo root, sits `qa-rounds.jsonl`: one JSON line
|
|
@@ -96,6 +123,8 @@ appended per round, committed alongside the plan updates —
|
|
|
96
123
|
```json
|
|
97
124
|
{"round_id": "…", "report_url": "…", "build": "<sha or hosted url>",
|
|
98
125
|
"evidence_grade": "…", "checks": [{"id": "…", "outcome": "…"}],
|
|
126
|
+
"issues": {"game": "solar-drift", "verified": [3], "regressed": [5],
|
|
127
|
+
"new": [7], "still_open": [2]},
|
|
99
128
|
"verdict": "…", "credits": 20, "follow_ups": ["…"]}
|
|
100
129
|
```
|
|
101
130
|
|
|
@@ -106,8 +135,11 @@ page (what the results tool returns), never a PR or issue link. There is no
|
|
|
106
135
|
evidence-link field, deliberately: signed recording URLs expire in days.
|
|
107
136
|
The ledger is the machine truth of what was asked and answered, per check,
|
|
108
137
|
per round; the plan's `last_verified` and `state` are derived from it and
|
|
109
|
-
stay the dev-readable contract.
|
|
110
|
-
|
|
138
|
+
stay the dev-readable contract. `issues` is what this round did to the issue
|
|
139
|
+
board, by number (omitted when the round carried no `game`). `follow_ups` is
|
|
140
|
+
NON-BUG follow-ups only — a check to add, a build affordance to write, a
|
|
141
|
+
question to settle; bugs go on the board, where they are numbered and
|
|
142
|
+
re-checked. Open `follow_ups` are what the next filing must carry forward.
|
|
111
143
|
|
|
112
144
|
## Rounds still open — qa-open-rounds.json
|
|
113
145
|
|
|
@@ -118,12 +150,13 @@ PARKED: filed with an explicit long deadline, then recorded in
|
|
|
118
150
|
```json
|
|
119
151
|
{"rounds": [{"ping_id": "<uuid>", "filed_at": "<ISO>", "platform": "windows",
|
|
120
152
|
"build": "<the hosted or store URL>", "est_minutes": 10,
|
|
121
|
-
"deadline_seconds": 86400, "
|
|
122
|
-
"note": "<the patch this round covers>"}]}
|
|
153
|
+
"deadline_seconds": 86400, "game": "solar-drift",
|
|
154
|
+
"checks": ["<check ids>"], "note": "<the patch this round covers>"}]}
|
|
123
155
|
```
|
|
124
156
|
|
|
125
157
|
`qaping rounds` lists it with each round's live status, `qaping rounds add`
|
|
126
|
-
|
|
158
|
+
(with `--game` for the board this round carries) records one,
|
|
159
|
+
`qaping rounds rm` drops it once collected. This file is the
|
|
127
160
|
opposite of the ledger and never merges with it: mutable OPEN state, one entry
|
|
128
161
|
per round still owed, emptied as rounds land. It is not history — gitignore it.
|
|
129
162
|
|
package/docs/icon.svg
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" width="64" height="64"><rect x="4" y="4" width="56" height="56" rx="14" fill="#FF7A45"/><path d="M20 33.5l8 8 16-17" fill="none" stroke="#fff" stroke-width="6.5" stroke-linecap="round" stroke-linejoin="round"/></svg>
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qaping/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"homepage": "https://qaping.dev",
|
|
5
5
|
"description": "qaping: automatic QA for your game. Your coding agent authors and maintains the QA plan, ships each patch's build to real human playtesters, and reports what broke.",
|
|
6
6
|
"keywords": [
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
"node": "^20.17.0 || ^22.13.0 || >=23.5.0"
|
|
26
26
|
},
|
|
27
27
|
"dependencies": {
|
|
28
|
-
"pingfusi": "^0.
|
|
28
|
+
"pingfusi": "^0.17.1"
|
|
29
29
|
},
|
|
30
30
|
"publishConfig": {
|
|
31
31
|
"access": "public"
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
{
|
|
2
|
+
"note": "sha256 of every SKILL.md version this package has shipped (LF-normalized). setup refreshes an installed skill whose hash is listed here — it is ours — and preserves anything else as a local edit. APPEND-ONLY; regenerate with: node scripts/gen-skill-hashes.js [<skill-root>...]",
|
|
3
|
+
"skills": {
|
|
4
|
+
"qaping": [
|
|
5
|
+
"184aa97ba0d529eea36be3c6eb4a5f4c1db8e2a65a0230e03814276a18a92354",
|
|
6
|
+
"217ffcc582e068424eefbb69376825ac71433fa92f34e3546bd02d8521ef008c",
|
|
7
|
+
"2fe5cafeef48146d8c9cc263ae1aaef764720027d17c0af3ab879e6853cf4a26",
|
|
8
|
+
"38cdc3289db6a544bc0cc828f4981a3f4ed6b39036ce81eebbaf4cbe9c47a5d0",
|
|
9
|
+
"490ff02895fdc6a2eb3a0b671628a8d4eee3efd906782eb45d99eac40bf59846",
|
|
10
|
+
"5462b858f58d183a020a5b2fa214dd1da6856e71a238316641f13d70eb0f0df4",
|
|
11
|
+
"5d0751c023af0a1cf69f152ba587b10c34087c7d5299f2d4b0e782f1ad0d66b0",
|
|
12
|
+
"6d0a5a1b430a8a0fd1c7c06b726de657e7fe82ac37ce69c9446539065d2ce751",
|
|
13
|
+
"71e64cd2b58996236c7fd3375a324ca2ae7b3e4028f7b358a389c562464c8bf3",
|
|
14
|
+
"8d817af568154734cd893122ce2e40ef2cd20cb725abbf318f134318157e5b6c",
|
|
15
|
+
"966bd642f1ee052691c7974a186e9f8f50644bce0ea010b6f0b444931879dbad",
|
|
16
|
+
"b9d08318e69290759bf31e149d64207b9d186faac9ccd01bc1db3ffcff29506c"
|
|
17
|
+
]
|
|
18
|
+
}
|
|
19
|
+
}
|
package/skill/qaping/SKILL.md
CHANGED
|
@@ -11,9 +11,12 @@ build, ship the build to real human playtesters, and report ONE typed outcome
|
|
|
11
11
|
per check — never a vibe. The check is the unit of value: its id rides from
|
|
12
12
|
the plan onto the wire, into the report, and into the round ledger. Over
|
|
13
13
|
patches, checks migrate from human rounds to repo code tests, so QA gets
|
|
14
|
-
cheaper.
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
cheaper. Checks are what the game must always do; BUGS are what it does wrong
|
|
15
|
+
right now, and they live on the game's issue board, where a fix you claim
|
|
16
|
+
turns verified only when a playtester says it is gone. All service contact
|
|
17
|
+
goes through four MCP tools — `qaping_playtest`, `qaping_results`,
|
|
18
|
+
`qaping_wait`, `qaping_issues` — plus `qaping publish-build` for hosting the
|
|
19
|
+
build.
|
|
17
20
|
|
|
18
21
|
## Ground rules (read before filing anything)
|
|
19
22
|
|
|
@@ -55,9 +58,12 @@ cheaper. All service contact goes through three MCP tools —
|
|
|
55
58
|
informed.
|
|
56
59
|
- **Evidence varies by platform — never promise what a platform does not
|
|
57
60
|
ship.** macOS and iOS rounds return a screen+voice recording plus an inline
|
|
58
|
-
[mm:ss] think-aloud transcript. Windows rounds return a recording and
|
|
59
|
-
transcript
|
|
60
|
-
|
|
61
|
+
[mm:ss] think-aloud transcript. Windows rounds return a recording, and a
|
|
62
|
+
transcript only when the tester recorded with our recorder app — read
|
|
63
|
+
`transcript_status` on each result: `'ok'` means an inline transcript,
|
|
64
|
+
`'unavailable'` means a Game Bar session with NO transcript. It is a
|
|
65
|
+
per-session fact, never promise one for a Windows round; plan to watch an
|
|
66
|
+
`'unavailable'` recording for timestamps. WEB rounds return answers only — no
|
|
61
67
|
recording, no transcript: the tester's option picks and free text are the
|
|
62
68
|
entire deliverable, which is why every check rides an option step.
|
|
63
69
|
- **Pay is flat per session, never per bug.** Never offer or imply a bounty.
|
|
@@ -101,6 +107,60 @@ once per recorded `ping_id`. The file is this, and nothing else:
|
|
|
101
107
|
`qa-open-rounds.json` is mutable OPEN state; `qa-rounds.jsonl` is append-only
|
|
102
108
|
history written after results land. Never merge the two.
|
|
103
109
|
|
|
110
|
+
## The issue board — bugs live here
|
|
111
|
+
|
|
112
|
+
Checks are the standing contract; bugs are what is wrong today, and they live
|
|
113
|
+
on the game's issue board — one board per `game_slug`, numbered `#1, #2, …`
|
|
114
|
+
for the life of the game, read and moved with `qaping_issues`. The board is
|
|
115
|
+
what carries a bug across rounds: the checklist a round hands the playtester
|
|
116
|
+
is built from it, and it is the only place a fix can ever be confirmed.
|
|
117
|
+
|
|
118
|
+
| status | who moves it there | how |
|
|
119
|
+
|---|---|---|
|
|
120
|
+
| `open` | you (`action:'create'`), or a playtester filing a bug from inside a round | the live state, and where a regression lands |
|
|
121
|
+
| `fixed` | YOU only (`action:'fixed'`, `build` REQUIRED) | a CLAIM, not a verification |
|
|
122
|
+
| `verified` | a PLAYTESTER only — a "Gone" on a round filed AFTER your claim | there is no verify action and there never will be |
|
|
123
|
+
| `closed` | you only (`action:'close'` + reason wontfix / invalid / duplicate / cannot_reproduce) | leaves the checklist; never turns green |
|
|
124
|
+
|
|
125
|
+
A "Still here" on a `fixed` or `verified` issue REGRESSES it back to `open`
|
|
126
|
+
and counts the regression. A "Gone" on an issue nobody claimed fixed changes
|
|
127
|
+
nothing — it may simply not have reproduced this session, and the board
|
|
128
|
+
records that as a not-seen streak for you to act on (close it
|
|
129
|
+
`cannot_reproduce` once you believe it).
|
|
130
|
+
|
|
131
|
+
- **The board is the source of truth for bugs; the dev's GitHub issues are
|
|
132
|
+
the mirror (MUST).** File it on the board first, then `gh issue create` for
|
|
133
|
+
the dev's own triage, then put that URL back on the board with
|
|
134
|
+
`action:'edit'` + `external_ref`. A bug that lives only in GitHub is never
|
|
135
|
+
re-checked by anyone, because only the board rides a round.
|
|
136
|
+
- **Issue titles and bodies speak player language (MUST)** — the same rule
|
|
137
|
+
the steps live under, for the same reason: a stranger reads them
|
|
138
|
+
mid-session on their own machine. "The shop closes when you press B twice",
|
|
139
|
+
never "NRE in ShopController.OnClose". No file paths, no stack traces, no
|
|
140
|
+
ticket ids. Title ≤120 chars, body ≤1000.
|
|
141
|
+
- **No build, no claim (MUST).** `action:'fixed'` takes the exact build URL
|
|
142
|
+
the NEXT round will play, and the claim happens BEFORE that round is filed.
|
|
143
|
+
A fix claimed after the round exists was not in the build that playtester
|
|
144
|
+
ran, so their answer cannot confirm it — the board leaves it amber and says
|
|
145
|
+
why.
|
|
146
|
+
- **`game` on every playtest filing (MUST).** The board's open and
|
|
147
|
+
claimed-fixed issues — up to 12, claimed-fixed first — ride the round as
|
|
148
|
+
the playtester's checklist after free play. It costs no step budget, adds
|
|
149
|
+
~30 seconds per issue inside est_minutes, and is the ONLY path to
|
|
150
|
+
`verified`. Omit it and the round has no memory.
|
|
151
|
+
- **Never leave a playtester-filed issue untriaged (MUST).** A bug reported
|
|
152
|
+
from inside a round lands `open` with origin `reviewer`, and answering it
|
|
153
|
+
is part of reporting that round — accept it, or close it duplicate /
|
|
154
|
+
invalid / cannot_reproduce.
|
|
155
|
+
- **Where the checklist is answered today, honestly.** The Windows browser
|
|
156
|
+
runner renders it. Rounds answered in the reviewer app (macOS, iOS) and web
|
|
157
|
+
rounds carry the board but do not show it yet, so their issues come back
|
|
158
|
+
unanswered — which is not evidence about the bug. An issue nobody answered
|
|
159
|
+
never moves; never report an unanswered checklist as "no longer
|
|
160
|
+
reproducing".
|
|
161
|
+
- Nothing on the board costs credits — only rounds do. `qaping issues <game>`
|
|
162
|
+
reads the same board from the shell.
|
|
163
|
+
|
|
104
164
|
## The review interface (know your tester)
|
|
105
165
|
|
|
106
166
|
<!-- Distilled from QAPING_REVIEWER_INTERFACE.md (monorepo root, INTERNAL,
|
|
@@ -151,10 +211,15 @@ that assumes they read your repo.
|
|
|
151
211
|
frontmatter so every future run can build without asking again.
|
|
152
212
|
3. Author `QA-PLAN.md` at the repo root in EXACTLY this format (self-contained
|
|
153
213
|
here — do not go looking for other docs). Plan frontmatter (YAML at the
|
|
154
|
-
very top): `game`, `build_command` (the ONE shippable-build
|
|
155
|
-
`platforms: [windows|macos|ios|web]` (`web` = a browser-playable
|
|
156
|
-
"build" is a static directory, not a zip), optional `input`
|
|
157
|
-
default keyboard+mouse, or touch for a mobile-web game).
|
|
214
|
+
very top): `game`, `game_slug`, `build_command` (the ONE shippable-build
|
|
215
|
+
command), `platforms: [windows|macos|ios|web]` (`web` = a browser-playable
|
|
216
|
+
game — its "build" is a static directory, not a zip), optional `input`
|
|
217
|
+
(hardware; default keyboard+mouse, or touch for a mobile-web game).
|
|
218
|
+
**Mint `game_slug` ONCE**, from the game name: lowercase, every run of
|
|
219
|
+
non-alphanumerics collapsed to a single `-`, no leading or trailing `-`,
|
|
220
|
+
≤64 chars ("Solar Drift" → `solar-drift`). It is the issue board's id and
|
|
221
|
+
is NEVER renamed — the same rule check ids live under, for the same reason:
|
|
222
|
+
a renamed slug orphans every issue filed under the old one.
|
|
158
223
|
Then one `##` section per check — the heading is the check's short name —
|
|
159
224
|
opening with a fenced yaml block of `id` (stable slug, never renamed),
|
|
160
225
|
`rung: human | code`, `state` (`unverified | passed | ambiguous | failing`
|
|
@@ -217,6 +282,10 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
217
282
|
|
|
218
283
|
## RUN — per patch
|
|
219
284
|
|
|
285
|
+
0. **Read the board.** `qaping_issues` with the plan's `game_slug` (action
|
|
286
|
+
`list`): what is open, what you claimed fixed and is still waiting on a
|
|
287
|
+
playtester, what came back REGRESSED. This is the bug half of "what
|
|
288
|
+
happened since last time"; the ledger's last line is the check half.
|
|
220
289
|
1. **Select.** Read the diff (or PR). Pick the checks it can plausibly affect,
|
|
221
290
|
plus the always-run criticals (the critical-path block above) and every
|
|
222
291
|
check the ledger left NOT-ASKED or Blocked last round. Tell the user which
|
|
@@ -252,11 +321,19 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
252
321
|
players open the link on their own devices. When filing a WEB game: send
|
|
253
322
|
`url` + `est_minutes` and OMIT `platform` entirely — a web round is claimed
|
|
254
323
|
by players in their own browser, phones included.
|
|
255
|
-
5. **
|
|
324
|
+
5. **Claim fixes (MUST, before filing).** For every board issue this patch
|
|
325
|
+
fixes: `qaping_issues` `action:'fixed'` with its `seq`, `build` set to the
|
|
326
|
+
exact URL this round will play, and a `note` written for the playtester
|
|
327
|
+
("the shop stays open now"). Claim first, file second — a claim recorded
|
|
328
|
+
after the round exists cannot be confirmed by it, and the results say so
|
|
329
|
+
instead of turning it green. No build URL, no claim.
|
|
330
|
+
6. **File** ONE `qaping_playtest` per composed session, always with
|
|
256
331
|
`questionnaire:'none'` — on a QA-PLAN round your checks are the questions;
|
|
257
332
|
the standard instrument is a separate experience round (below), never a
|
|
258
|
-
default
|
|
259
|
-
|
|
333
|
+
default — and always with `game` set to the plan's `game_slug` (MUST), so
|
|
334
|
+
the board's live issues ride the round as the playtester's checklist.
|
|
335
|
+
A WEB session additionally files `require_evidence:'none'` (MUST) — left
|
|
336
|
+
at its default, the finish silently demands a pinned
|
|
260
337
|
screenshot-comment no step asked for. est_minutes is an
|
|
261
338
|
honest sum of the steps' directed play — the service
|
|
262
339
|
accepts 5–30 minutes (10 is the standard session; outside that
|
|
@@ -282,11 +359,14 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
282
359
|
player-visible expectation. One whose precondition cannot be forced
|
|
283
360
|
opens with a verifying option step ("Before this session: had you ever
|
|
284
361
|
played this game before?" — yes / no / not sure);
|
|
285
|
-
- a recheck of a
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
and
|
|
289
|
-
|
|
362
|
+
- **a recheck of a known bug is NEVER a step.** The board rides the
|
|
363
|
+
checklist: `game` on the filing attaches the open and claimed-fixed
|
|
364
|
+
issues, the playtester answers them (Still here / Gone / Couldn't test)
|
|
365
|
+
after free play, and that costs none of the step budget. Never author a
|
|
366
|
+
step that re-asks an issue already on the board — a step and a checklist
|
|
367
|
+
row about the same bug buy one answer twice — and prior findings stay
|
|
368
|
+
OUT of `instructions` entirely, so the rest of the pass is an
|
|
369
|
+
independent look;
|
|
290
370
|
- a closing **break-it slice** (~2–3 minutes) authored from the diff:
|
|
291
371
|
"this patch changed X — try to break it: <concrete abuse ideas from the
|
|
292
372
|
diff>. Report the weirdest thing you saw AND list what you tried."
|
|
@@ -307,18 +387,24 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
307
387
|
the standard questionnaire rides) — rows past 20 are answered by the
|
|
308
388
|
tester and silently dropped at finish, which is why this gate exists. A
|
|
309
389
|
mismatch (the budget forcing cuts) fails loudly PRE-SPEND: batch another
|
|
310
|
-
session or state the deferral out loud — never silently truncate.
|
|
311
|
-
|
|
390
|
+
session or state the deferral out loud — never silently truncate. Then
|
|
391
|
+
print `issues: <seqs attached>` from the filing's own result (it names the
|
|
392
|
+
checklist it built, and anything the 12-issue cap left off), or
|
|
393
|
+
`issues: none` on an empty board — so what this round RE-CHECKS is on the
|
|
394
|
+
record beside what it covers.
|
|
395
|
+
7. **Wait or park — decide now** (ground rule 1). WAIT: spawn a background
|
|
312
396
|
task looping `qaping_wait` and keep working — foreground polling is the
|
|
313
397
|
fallback. PARK: the filing already carried an explicit `deadline_seconds`
|
|
314
398
|
sized to the span, so record it — `qaping rounds add <ping_id> --build
|
|
315
399
|
<url> --minutes N --checks <ids> --note "<patch>"` — tell the user what is
|
|
316
400
|
pending, and end the turn; a later session collects it. A promise to check
|
|
317
401
|
later is never a wait, and never a substitute for the record.
|
|
318
|
-
|
|
402
|
+
8. **Report** from `qaping_results`:
|
|
319
403
|
- **Evidence line first (MUST).** Every report opens with its grade:
|
|
320
|
-
`Evidence: recording + transcript` (macOS/iOS
|
|
321
|
-
|
|
404
|
+
`Evidence: recording + transcript` (macOS/iOS, or a Windows result with
|
|
405
|
+
`transcript_status:'ok'`) / `Evidence: recording only` (a Windows result
|
|
406
|
+
with `transcript_status:'unavailable'` — a Game Bar session) /
|
|
407
|
+
`Evidence: testimony only (web round)`.
|
|
322
408
|
- **Computed headline (MUST).** Derive the one verdict sentence from the
|
|
323
409
|
per-check outcomes, never from the tester's own verdict: `PASS` only
|
|
324
410
|
when every selected check returned Pass; `PASS — N unverified (<ids>)`
|
|
@@ -326,16 +412,23 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
326
412
|
on any Fail. Quote the tester's global verdict beneath it as one input —
|
|
327
413
|
it is never the headline. (The iterate rule already says unresolved
|
|
328
414
|
findings mean fix and refile — unresolved never headlines as PASS.)
|
|
415
|
+
- **Issues line (MUST), under the headline.** One line read off the
|
|
416
|
+
results' issue block: `issues: #3 verified, #5 REGRESSED, 2 new` —
|
|
417
|
+
verified first, regressions next, then how many bugs playtesters filed
|
|
418
|
+
from inside the session. `issues: none moved` when the board rode along
|
|
419
|
+
and nothing changed; nothing at all when the round carried no `game`. A
|
|
420
|
+
REGRESSED issue is a failure of a fix you claimed: say so in words, in
|
|
421
|
+
the same breath as the headline.
|
|
329
422
|
- **One row per SELECTED check (MUST)**, keyed by id, outcome from
|
|
330
423
|
{Pass, Fail, Blocked, NOT-ASKED}. "Couldn't test" and "Didn't reach this
|
|
331
424
|
part" are Blocked, with the why. A check that reached the wire as more
|
|
332
|
-
than one row (a precondition step
|
|
333
|
-
from the WORST row: any Fail → Fail; else any Blocked or
|
|
334
|
-
leaves the check unverified and the headline counts it
|
|
335
|
-
unverified") — never plain Pass over a Blocked sub-step. A
|
|
336
|
-
unverified or disproven is VOID —
|
|
337
|
-
|
|
338
|
-
`last_verified` untouched. A selected check that never made the wire or
|
|
425
|
+
than one row (a precondition step plus its observation step) takes its
|
|
426
|
+
outcome from the WORST row: any Fail → Fail; else any Blocked or
|
|
427
|
+
unanswered row leaves the check unverified and the headline counts it
|
|
428
|
+
("PASS — N unverified") — never plain Pass over a Blocked sub-step. A
|
|
429
|
+
check whose precondition came back unverified or disproven is VOID —
|
|
430
|
+
report it as Blocked ("VOID: precondition unverified"),
|
|
431
|
+
never as a pass or fail, and leave `last_verified` untouched. A selected check that never made the wire or
|
|
339
432
|
never came back is NOT-ASKED and carries forward into the next filing
|
|
340
433
|
automatically. Cite [mm:ss] where a transcript or recording exists; on a
|
|
341
434
|
web round cite the step answers — there are no timestamps to cite.
|
|
@@ -344,8 +437,15 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
344
437
|
- **Ledger append (MUST).** Append ONE JSON line to `qa-rounds.jsonl` at
|
|
345
438
|
the game repo root, committed alongside the plan updates:
|
|
346
439
|
`{"round_id": …, "report_url": …, "build": "<sha or hosted url>",
|
|
347
|
-
"evidence_grade": …, "checks": [{"id": …, "outcome": …}],
|
|
348
|
-
"
|
|
440
|
+
"evidence_grade": …, "checks": [{"id": …, "outcome": …}],
|
|
441
|
+
"issues": {"game": "<game_slug>", "verified": [3], "regressed": [5],
|
|
442
|
+
"new": [7, 8], "still_open": [2]}, "verdict": …, "credits": …,
|
|
443
|
+
"follow_ups": […]}`. The `issues` object is the board's movement this
|
|
444
|
+
round, by seq (omit it entirely when the round carried no `game`), and
|
|
445
|
+
`follow_ups` is NON-BUG follow-ups only from now on — a check to add, a
|
|
446
|
+
build affordance to write, a question to settle. A bug goes on the
|
|
447
|
+
board, where it is numbered, re-checked and closable; free text nobody
|
|
448
|
+
re-reads is where bugs went to die. `report_url` is the round's service
|
|
349
449
|
report page (the `report_url` the results tool returns — never a PR or
|
|
350
450
|
issue link). No evidence-link field, deliberately: signed recording URLs
|
|
351
451
|
expire in days, so durable evidence pointers live in the report/issues,
|
|
@@ -367,14 +467,33 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
367
467
|
a fragile area, or a QA lesson learned, offer to record it as a check
|
|
368
468
|
(or a note on an existing check) in `QA-PLAN.md` right then, so the plan
|
|
369
469
|
accretes knowledge instead of losing it to chat scrollback.
|
|
370
|
-
-
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
470
|
+
- **Triage every new issue in THIS turn (MUST).** Each `NEW #n` in the
|
|
471
|
+
results is a bug a playtester filed from inside the session, and it is
|
|
472
|
+
answered before the report is finished: accept it (`gh issue create` for
|
|
473
|
+
the dev's triage, then `qaping_issues` `action:'edit'` with
|
|
474
|
+
`external_ref` set to that URL), or close it — `duplicate` with
|
|
475
|
+
`duplicate_of_seq`, `invalid`, or `cannot_reproduce`. Never leave one
|
|
476
|
+
sitting; an untriaged board stops being worth reading, and the next
|
|
477
|
+
round re-asks a bug nobody decided about.
|
|
478
|
+
- **A REGRESSED issue reopens its mirror (MUST).** The fix did not hold:
|
|
479
|
+
`gh issue reopen <n>` on the GitHub issue, name it in the PR comment,
|
|
480
|
+
and never claim `fixed` on it again without a new build to claim
|
|
481
|
+
against.
|
|
482
|
+
- **A verified issue closes its mirror.** An issue a playtester turned
|
|
483
|
+
`verified` is settled: close the GitHub issue citing the round. Leave
|
|
484
|
+
the board row green — never `action:'close'` a verified issue to tidy
|
|
485
|
+
up; closed is for what you decided about, not for what was confirmed.
|
|
486
|
+
- Findings YOU file: one board issue per finding (`action:'create'`, in
|
|
487
|
+
player language, severity from consequence — `blocker` cannot play past
|
|
488
|
+
it / `major` / `minor` / `cosmetic`), then one `gh issue create` per
|
|
489
|
+
finding as the dev's own draft — never near-duplicates, never one issue
|
|
490
|
+
per symptom of the same bug — and `action:'edit'` the board row with the
|
|
491
|
+
`external_ref` you just created. Before reproduction the body carries
|
|
492
|
+
"observed once, unreproduced" plus that proposed severity. The label
|
|
493
|
+
CONFIRMED is reserved for a targeted round or unambiguous in-round
|
|
494
|
+
evidence; a CONFIRMED issue carries repro steps rebuilt from the
|
|
495
|
+
tester's answers and a timestamp link into the recording where one
|
|
496
|
+
exists (web rounds have none — cite answers).
|
|
378
497
|
|
|
379
498
|
## The ambiguity ladder
|
|
380
499
|
|
|
@@ -384,7 +503,11 @@ precondition forced via `setup` (or verified by the opening option step),
|
|
|
384
503
|
plus the disambiguation steps. That round settles it: Pass → `passed` (note
|
|
385
504
|
the fluke); Fail → `failing`, and the issue upgrades to CONFIRMED. Never
|
|
386
505
|
resolve an ambiguity by re-buying the whole batch, and never label a
|
|
387
|
-
once-seen fail CONFIRMED without that round or hard evidence.
|
|
506
|
+
once-seen fail CONFIRMED without that round or hard evidence. When the
|
|
507
|
+
targeted round settles it as a real bug, the board issue for it carries
|
|
508
|
+
`check_id: <the check's id>` — set at `create`, or with `action:'edit'` on a
|
|
509
|
+
playtester-filed issue — so the check and the bug that made it ambiguous stay
|
|
510
|
+
findable from each other.
|
|
388
511
|
|
|
389
512
|
## Paired rounds (two-state comparisons)
|
|
390
513
|
|