@qaping/cli 0.1.5 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/qaping +606 -5
- package/docs/COMMANDS.md +122 -0
- package/docs/QA-PLAN-FORMAT.md +53 -2
- package/docs/icon.svg +1 -0
- package/package.json +2 -2
- package/skill/SHIPPED-HASHES.json +19 -0
- package/skill/qaping/SKILL.md +246 -58
package/skill/qaping/SKILL.md
CHANGED
|
@@ -11,24 +11,46 @@ build, ship the build to real human playtesters, and report ONE typed outcome
|
|
|
11
11
|
per check — never a vibe. The check is the unit of value: its id rides from
|
|
12
12
|
the plan onto the wire, into the report, and into the round ledger. Over
|
|
13
13
|
patches, checks migrate from human rounds to repo code tests, so QA gets
|
|
14
|
-
cheaper.
|
|
15
|
-
|
|
16
|
-
|
|
14
|
+
cheaper. Checks are what the game must always do; BUGS are what it does wrong
|
|
15
|
+
right now, and they live on the game's issue board, where a fix you claim
|
|
16
|
+
turns verified only when a playtester says it is gone. All service contact
|
|
17
|
+
goes through four MCP tools — `qaping_playtest`, `qaping_results`,
|
|
18
|
+
`qaping_wait`, `qaping_issues` — plus `qaping publish-build` for hosting the
|
|
19
|
+
build.
|
|
17
20
|
|
|
18
21
|
## Ground rules (read before filing anything)
|
|
19
22
|
|
|
20
|
-
- **Playtesters are real people, and
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
23
|
+
- **Playtesters are real people, and a round outlives a quick turn.
|
|
24
|
+
Decide at filing time: wait, or park.** A session takes real minutes, often
|
|
25
|
+
hours, to be claimed and played. Which of the two endings this round gets is
|
|
26
|
+
chosen BEFORE you file, not discovered when pending surprises you.
|
|
27
|
+
- **WAIT — short rounds you can sit through.** Each `qaping_wait` renews the
|
|
28
|
+
lease that keeps the round visible to new claimants. The moment a filing
|
|
29
|
+
returns pending, SPAWN A BACKGROUND TASK that loops `qaping_wait` with the
|
|
30
|
+
ping_id until results arrive, and keep working on other things — that is
|
|
31
|
+
the intended shape, the same as every pingfusi round. Only if your harness
|
|
32
|
+
has no background tasks: keep polling in the foreground. A one-shot print
|
|
33
|
+
mode where neither is possible is not a WAIT round at all — it is a PARK
|
|
34
|
+
(below): file with an explicit `deadline_seconds`, record it with `qaping
|
|
35
|
+
rounds add <ping_id>`, say what is pending, and ask to be re-invoked to
|
|
36
|
+
collect it — never promise polling you cannot do (an ended session cannot
|
|
37
|
+
poll). A round already filed on the default lease is recorded anyway,
|
|
38
|
+
flagged as likely to expire unclaimed.
|
|
39
|
+
- **PARK — long or overnight rounds.** For a session nobody will sit
|
|
40
|
+
through — an overnight run, `n_target > 1` on a small pool, a native round
|
|
41
|
+
whose claim is slow, an explicit "check tomorrow", or a one-shot mode —
|
|
42
|
+
file with an explicit `deadline_seconds` sized to the span you are willing
|
|
43
|
+
to wait (the tool schema states the ceiling) plus an `idempotency_key`,
|
|
44
|
+
then IMMEDIATELY record it: `qaping rounds add <ping_id> --build <url>
|
|
45
|
+
--minutes N --checks <ids> --note "<patch>"`, which writes
|
|
46
|
+
`qa-open-rounds.json` at the repo root (gitignore it — open state, not
|
|
47
|
+
history). Then tell the user exactly what is pending and what will collect
|
|
48
|
+
it, and end the turn cleanly.
|
|
49
|
+
- **A recorded parked round is an honest pending end; an unrecorded "I'll
|
|
50
|
+
check later" never is.** Never park on the default lease: with no explicit
|
|
51
|
+
`deadline_seconds` an unwaited round leaves the feed within minutes and
|
|
52
|
+
expires unclaimed (a round died exactly that way, live). On resume,
|
|
53
|
+
collect the recorded round — never file a duplicate.
|
|
32
54
|
- **Costs are duration-billed**: 2 credits per minute of play, per playtester
|
|
33
55
|
(`est_minutes × 2 × players`). State the estimate to the user BEFORE filing,
|
|
34
56
|
and set the wait expectation honestly in the same message: claim time is
|
|
@@ -36,9 +58,12 @@ cheaper. All service contact goes through three MCP tools —
|
|
|
36
58
|
informed.
|
|
37
59
|
- **Evidence varies by platform — never promise what a platform does not
|
|
38
60
|
ship.** macOS and iOS rounds return a screen+voice recording plus an inline
|
|
39
|
-
[mm:ss] think-aloud transcript. Windows rounds return a recording and
|
|
40
|
-
transcript
|
|
41
|
-
|
|
61
|
+
[mm:ss] think-aloud transcript. Windows rounds return a recording, and a
|
|
62
|
+
transcript only when the tester recorded with our recorder app — read
|
|
63
|
+
`transcript_status` on each result: `'ok'` means an inline transcript,
|
|
64
|
+
`'unavailable'` means a Game Bar session with NO transcript. It is a
|
|
65
|
+
per-session fact, never promise one for a Windows round; plan to watch an
|
|
66
|
+
`'unavailable'` recording for timestamps. WEB rounds return answers only — no
|
|
42
67
|
recording, no transcript: the tester's option picks and free text are the
|
|
43
68
|
entire deliverable, which is why every check rides an option step.
|
|
44
69
|
- **Pay is flat per session, never per bug.** Never offer or imply a bounty.
|
|
@@ -51,6 +76,91 @@ cheaper. All service contact goes through three MCP tools —
|
|
|
51
76
|
- **QA-PLAN.md is the dev's file.** Commit it only with their approval; never
|
|
52
77
|
delete or demote a human check without them agreeing.
|
|
53
78
|
|
|
79
|
+
## Open rounds — collect FIRST
|
|
80
|
+
|
|
81
|
+
Before ANY qaping work in a repo — a new run, a plan edit, a plain "did it
|
|
82
|
+
come back yet?" — read `qa-open-rounds.json` at the repo root. `qaping rounds`
|
|
83
|
+
lists it with each round's live status; without the CLI, call `qaping_results`
|
|
84
|
+
once per recorded `ping_id`. The file is this, and nothing else:
|
|
85
|
+
|
|
86
|
+
```json
|
|
87
|
+
{"rounds": [{"ping_id": "<uuid>", "filed_at": "<ISO>", "platform": "windows",
|
|
88
|
+
"build": "<the /b/<slug> or store URL>", "est_minutes": 10,
|
|
89
|
+
"deadline_seconds": 86400, "checks": ["boot-to-menu"],
|
|
90
|
+
"note": "<the patch this round covers>"}]}
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
- **READY** (results are in): collect it BEFORE starting new work — the full
|
|
94
|
+
RUN report ritual (evidence line, computed headline, per-check rows, PR
|
|
95
|
+
comment, issues, plan updates, ledger append), then
|
|
96
|
+
`qaping rounds rm <ping_id>`.
|
|
97
|
+
- **Still pending**: say so plainly, with what it is still waiting on, and
|
|
98
|
+
leave it recorded. Never refile it and never start a second round on the
|
|
99
|
+
same checks while it is open.
|
|
100
|
+
- **Expired or unreadable**: report that honestly — an expired round bought
|
|
101
|
+
nothing — then `qaping rounds rm <ping_id>` and offer a refile (a fresh
|
|
102
|
+
build URL if the old one expired, and a longer `deadline_seconds`).
|
|
103
|
+
- **Not visible from this login** reads identically to gone, and is not the
|
|
104
|
+
same thing: results are asker-scoped, so a round filed from another account
|
|
105
|
+
is alive and invisible. Check `qaping whoami` before deleting that record.
|
|
106
|
+
|
|
107
|
+
`qa-open-rounds.json` is mutable OPEN state; `qa-rounds.jsonl` is append-only
|
|
108
|
+
history written after results land. Never merge the two.
|
|
109
|
+
|
|
110
|
+
## The issue board — bugs live here
|
|
111
|
+
|
|
112
|
+
Checks are the standing contract; bugs are what is wrong today, and they live
|
|
113
|
+
on the game's issue board — one board per `game_slug`, numbered `#1, #2, …`
|
|
114
|
+
for the life of the game, read and moved with `qaping_issues`. The board is
|
|
115
|
+
what carries a bug across rounds: the checklist a round hands the playtester
|
|
116
|
+
is built from it, and it is the only place a fix can ever be confirmed.
|
|
117
|
+
|
|
118
|
+
| status | who moves it there | how |
|
|
119
|
+
|---|---|---|
|
|
120
|
+
| `open` | you (`action:'create'`), or a playtester filing a bug from inside a round | the live state, and where a regression lands |
|
|
121
|
+
| `fixed` | YOU only (`action:'fixed'`, `build` REQUIRED) | a CLAIM, not a verification |
|
|
122
|
+
| `verified` | a PLAYTESTER only — a "Gone" on a round filed AFTER your claim | there is no verify action and there never will be |
|
|
123
|
+
| `closed` | you only (`action:'close'` + reason wontfix / invalid / duplicate / cannot_reproduce) | leaves the checklist; never turns green |
|
|
124
|
+
|
|
125
|
+
A "Still here" on a `fixed` or `verified` issue REGRESSES it back to `open`
|
|
126
|
+
and counts the regression. A "Gone" on an issue nobody claimed fixed changes
|
|
127
|
+
nothing — it may simply not have reproduced this session, and the board
|
|
128
|
+
records that as a not-seen streak for you to act on (close it
|
|
129
|
+
`cannot_reproduce` once you believe it).
|
|
130
|
+
|
|
131
|
+
- **The board is the source of truth for bugs; the dev's GitHub issues are
|
|
132
|
+
the mirror (MUST).** File it on the board first, then `gh issue create` for
|
|
133
|
+
the dev's own triage, then put that URL back on the board with
|
|
134
|
+
`action:'edit'` + `external_ref`. A bug that lives only in GitHub is never
|
|
135
|
+
re-checked by anyone, because only the board rides a round.
|
|
136
|
+
- **Issue titles and bodies speak player language (MUST)** — the same rule
|
|
137
|
+
the steps live under, for the same reason: a stranger reads them
|
|
138
|
+
mid-session on their own machine. "The shop closes when you press B twice",
|
|
139
|
+
never "NRE in ShopController.OnClose". No file paths, no stack traces, no
|
|
140
|
+
ticket ids. Title ≤120 chars, body ≤1000.
|
|
141
|
+
- **No build, no claim (MUST).** `action:'fixed'` takes the exact build URL
|
|
142
|
+
the NEXT round will play, and the claim happens BEFORE that round is filed.
|
|
143
|
+
A fix claimed after the round exists was not in the build that playtester
|
|
144
|
+
ran, so their answer cannot confirm it — the board leaves it amber and says
|
|
145
|
+
why.
|
|
146
|
+
- **`game` on every playtest filing (MUST).** The board's open and
|
|
147
|
+
claimed-fixed issues — up to 12, claimed-fixed first — ride the round as
|
|
148
|
+
the playtester's checklist after free play. It costs no step budget, adds
|
|
149
|
+
~30 seconds per issue inside est_minutes, and is the ONLY path to
|
|
150
|
+
`verified`. Omit it and the round has no memory.
|
|
151
|
+
- **Never leave a playtester-filed issue untriaged (MUST).** A bug reported
|
|
152
|
+
from inside a round lands `open` with origin `reviewer`, and answering it
|
|
153
|
+
is part of reporting that round — accept it, or close it duplicate /
|
|
154
|
+
invalid / cannot_reproduce.
|
|
155
|
+
- **Where the checklist is answered today, honestly.** The Windows browser
|
|
156
|
+
runner renders it. Rounds answered in the reviewer app (macOS, iOS) and web
|
|
157
|
+
rounds carry the board but do not show it yet, so their issues come back
|
|
158
|
+
unanswered — which is not evidence about the bug. An issue nobody answered
|
|
159
|
+
never moves; never report an unanswered checklist as "no longer
|
|
160
|
+
reproducing".
|
|
161
|
+
- Nothing on the board costs credits — only rounds do. `qaping issues <game>`
|
|
162
|
+
reads the same board from the shell.
|
|
163
|
+
|
|
54
164
|
## The review interface (know your tester)
|
|
55
165
|
|
|
56
166
|
<!-- Distilled from QAPING_REVIEWER_INTERFACE.md (monorepo root, INTERNAL,
|
|
@@ -101,10 +211,15 @@ that assumes they read your repo.
|
|
|
101
211
|
frontmatter so every future run can build without asking again.
|
|
102
212
|
3. Author `QA-PLAN.md` at the repo root in EXACTLY this format (self-contained
|
|
103
213
|
here — do not go looking for other docs). Plan frontmatter (YAML at the
|
|
104
|
-
very top): `game`, `build_command` (the ONE shippable-build
|
|
105
|
-
`platforms: [windows|macos|ios|web]` (`web` = a browser-playable
|
|
106
|
-
"build" is a static directory, not a zip), optional `input`
|
|
107
|
-
default keyboard+mouse, or touch for a mobile-web game).
|
|
214
|
+
very top): `game`, `game_slug`, `build_command` (the ONE shippable-build
|
|
215
|
+
command), `platforms: [windows|macos|ios|web]` (`web` = a browser-playable
|
|
216
|
+
game — its "build" is a static directory, not a zip), optional `input`
|
|
217
|
+
(hardware; default keyboard+mouse, or touch for a mobile-web game).
|
|
218
|
+
**Mint `game_slug` ONCE**, from the game name: lowercase, every run of
|
|
219
|
+
non-alphanumerics collapsed to a single `-`, no leading or trailing `-`,
|
|
220
|
+
≤64 chars ("Solar Drift" → `solar-drift`). It is the issue board's id and
|
|
221
|
+
is NEVER renamed — the same rule check ids live under, for the same reason:
|
|
222
|
+
a renamed slug orphans every issue filed under the old one.
|
|
108
223
|
Then one `##` section per check — the heading is the check's short name —
|
|
109
224
|
opening with a fenced yaml block of `id` (stable slug, never renamed),
|
|
110
225
|
`rung: human | code`, `state` (`unverified | passed | ambiguous | failing`
|
|
@@ -167,6 +282,10 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
167
282
|
|
|
168
283
|
## RUN — per patch
|
|
169
284
|
|
|
285
|
+
0. **Read the board.** `qaping_issues` with the plan's `game_slug` (action
|
|
286
|
+
`list`): what is open, what you claimed fixed and is still waiting on a
|
|
287
|
+
playtester, what came back REGRESSED. This is the bug half of "what
|
|
288
|
+
happened since last time"; the ledger's last line is the check half.
|
|
170
289
|
1. **Select.** Read the diff (or PR). Pick the checks it can plausibly affect,
|
|
171
290
|
plus the always-run criticals (the critical-path block above) and every
|
|
172
291
|
check the ledger left NOT-ASKED or Blocked last round. Tell the user which
|
|
@@ -187,19 +306,34 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
187
306
|
static directory (web).
|
|
188
307
|
4. **Ship**, by delivery: NATIVE uploads — `qaping publish-build <game.zip>
|
|
189
308
|
--platform windows|macos` → prints a `/b/<slug>` URL (temporary hosting;
|
|
190
|
-
filing a playtest extends the build through the round;
|
|
191
|
-
|
|
192
|
-
|
|
309
|
+
filing a playtest extends the build through the round; a CHANGED build gets
|
|
310
|
+
a new URL, while re-publishing an unchanged zip returns the same build).
|
|
311
|
+
You hold only a handful of hosted builds at once: if a publish is refused
|
|
312
|
+
for that cap, `qaping builds` lists what you are holding and
|
|
313
|
+
`qaping builds rm <slug>` frees a slot — do that and carry on rather than
|
|
314
|
+
stopping to ask. The listing marks each build: never-finished uploads are
|
|
315
|
+
safe to delete first, and any build an open round is still using is marked
|
|
316
|
+
`IN USE` — never delete one of those, whichever session filed it. If every
|
|
317
|
+
build is in use, say so and wait for a round to finish rather than guessing. Store-delivered games file with their Steam store
|
|
318
|
+
page, TestFlight public link, or App Store page as `url`. WEB games — host the built output
|
|
193
319
|
at a public URL: the dev's own hosting if they have it, else
|
|
194
320
|
`qaping publish <built-dir>` prints a hosted URL. Localhost never works —
|
|
195
321
|
players open the link on their own devices. When filing a WEB game: send
|
|
196
322
|
`url` + `est_minutes` and OMIT `platform` entirely — a web round is claimed
|
|
197
323
|
by players in their own browser, phones included.
|
|
198
|
-
5. **
|
|
324
|
+
5. **Claim fixes (MUST, before filing).** For every board issue this patch
|
|
325
|
+
fixes: `qaping_issues` `action:'fixed'` with its `seq`, `build` set to the
|
|
326
|
+
exact URL this round will play, and a `note` written for the playtester
|
|
327
|
+
("the shop stays open now"). Claim first, file second — a claim recorded
|
|
328
|
+
after the round exists cannot be confirmed by it, and the results say so
|
|
329
|
+
instead of turning it green. No build URL, no claim.
|
|
330
|
+
6. **File** ONE `qaping_playtest` per composed session, always with
|
|
199
331
|
`questionnaire:'none'` — on a QA-PLAN round your checks are the questions;
|
|
200
332
|
the standard instrument is a separate experience round (below), never a
|
|
201
|
-
default
|
|
202
|
-
|
|
333
|
+
default — and always with `game` set to the plan's `game_slug` (MUST), so
|
|
334
|
+
the board's live issues ride the round as the playtester's checklist.
|
|
335
|
+
A WEB session additionally files `require_evidence:'none'` (MUST) — left
|
|
336
|
+
at its default, the finish silently demands a pinned
|
|
203
337
|
screenshot-comment no step asked for. est_minutes is an
|
|
204
338
|
honest sum of the steps' directed play — the service
|
|
205
339
|
accepts 5–30 minutes (10 is the standard session; outside that
|
|
@@ -225,11 +359,14 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
225
359
|
player-visible expectation. One whose precondition cannot be forced
|
|
226
360
|
opens with a verifying option step ("Before this session: had you ever
|
|
227
361
|
played this game before?" — yes / no / not sure);
|
|
228
|
-
- a recheck of a
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
and
|
|
232
|
-
|
|
362
|
+
- **a recheck of a known bug is NEVER a step.** The board rides the
|
|
363
|
+
checklist: `game` on the filing attaches the open and claimed-fixed
|
|
364
|
+
issues, the playtester answers them (Still here / Gone / Couldn't test)
|
|
365
|
+
after free play, and that costs none of the step budget. Never author a
|
|
366
|
+
step that re-asks an issue already on the board — a step and a checklist
|
|
367
|
+
row about the same bug buy one answer twice — and prior findings stay
|
|
368
|
+
OUT of `instructions` entirely, so the rest of the pass is an
|
|
369
|
+
independent look;
|
|
233
370
|
- a closing **break-it slice** (~2–3 minutes) authored from the diff:
|
|
234
371
|
"this patch changed X — try to break it: <concrete abuse ideas from the
|
|
235
372
|
diff>. Report the weirdest thing you saw AND list what you tried."
|
|
@@ -250,14 +387,24 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
250
387
|
the standard questionnaire rides) — rows past 20 are answered by the
|
|
251
388
|
tester and silently dropped at finish, which is why this gate exists. A
|
|
252
389
|
mismatch (the budget forcing cuts) fails loudly PRE-SPEND: batch another
|
|
253
|
-
session or state the deferral out loud — never silently truncate.
|
|
254
|
-
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
390
|
+
session or state the deferral out loud — never silently truncate. Then
|
|
391
|
+
print `issues: <seqs attached>` from the filing's own result (it names the
|
|
392
|
+
checklist it built, and anything the 12-issue cap left off), or
|
|
393
|
+
`issues: none` on an empty board — so what this round RE-CHECKS is on the
|
|
394
|
+
record beside what it covers.
|
|
395
|
+
7. **Wait or park — decide now** (ground rule 1). WAIT: spawn a background
|
|
396
|
+
task looping `qaping_wait` and keep working — foreground polling is the
|
|
397
|
+
fallback. PARK: the filing already carried an explicit `deadline_seconds`
|
|
398
|
+
sized to the span, so record it — `qaping rounds add <ping_id> --build
|
|
399
|
+
<url> --minutes N --checks <ids> --note "<patch>"` — tell the user what is
|
|
400
|
+
pending, and end the turn; a later session collects it. A promise to check
|
|
401
|
+
later is never a wait, and never a substitute for the record.
|
|
402
|
+
8. **Report** from `qaping_results`:
|
|
258
403
|
- **Evidence line first (MUST).** Every report opens with its grade:
|
|
259
|
-
`Evidence: recording + transcript` (macOS/iOS
|
|
260
|
-
|
|
404
|
+
`Evidence: recording + transcript` (macOS/iOS, or a Windows result with
|
|
405
|
+
`transcript_status:'ok'`) / `Evidence: recording only` (a Windows result
|
|
406
|
+
with `transcript_status:'unavailable'` — a Game Bar session) /
|
|
407
|
+
`Evidence: testimony only (web round)`.
|
|
261
408
|
- **Computed headline (MUST).** Derive the one verdict sentence from the
|
|
262
409
|
per-check outcomes, never from the tester's own verdict: `PASS` only
|
|
263
410
|
when every selected check returned Pass; `PASS — N unverified (<ids>)`
|
|
@@ -265,16 +412,23 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
265
412
|
on any Fail. Quote the tester's global verdict beneath it as one input —
|
|
266
413
|
it is never the headline. (The iterate rule already says unresolved
|
|
267
414
|
findings mean fix and refile — unresolved never headlines as PASS.)
|
|
415
|
+
- **Issues line (MUST), under the headline.** One line read off the
|
|
416
|
+
results' issue block: `issues: #3 verified, #5 REGRESSED, 2 new` —
|
|
417
|
+
verified first, regressions next, then how many bugs playtesters filed
|
|
418
|
+
from inside the session. `issues: none moved` when the board rode along
|
|
419
|
+
and nothing changed; nothing at all when the round carried no `game`. A
|
|
420
|
+
REGRESSED issue is a failure of a fix you claimed: say so in words, in
|
|
421
|
+
the same breath as the headline.
|
|
268
422
|
- **One row per SELECTED check (MUST)**, keyed by id, outcome from
|
|
269
423
|
{Pass, Fail, Blocked, NOT-ASKED}. "Couldn't test" and "Didn't reach this
|
|
270
424
|
part" are Blocked, with the why. A check that reached the wire as more
|
|
271
|
-
than one row (a precondition step
|
|
272
|
-
from the WORST row: any Fail → Fail; else any Blocked or
|
|
273
|
-
leaves the check unverified and the headline counts it
|
|
274
|
-
unverified") — never plain Pass over a Blocked sub-step. A
|
|
275
|
-
unverified or disproven is VOID —
|
|
276
|
-
|
|
277
|
-
`last_verified` untouched. A selected check that never made the wire or
|
|
425
|
+
than one row (a precondition step plus its observation step) takes its
|
|
426
|
+
outcome from the WORST row: any Fail → Fail; else any Blocked or
|
|
427
|
+
unanswered row leaves the check unverified and the headline counts it
|
|
428
|
+
("PASS — N unverified") — never plain Pass over a Blocked sub-step. A
|
|
429
|
+
check whose precondition came back unverified or disproven is VOID —
|
|
430
|
+
report it as Blocked ("VOID: precondition unverified"),
|
|
431
|
+
never as a pass or fail, and leave `last_verified` untouched. A selected check that never made the wire or
|
|
278
432
|
never came back is NOT-ASKED and carries forward into the next filing
|
|
279
433
|
automatically. Cite [mm:ss] where a transcript or recording exists; on a
|
|
280
434
|
web round cite the step answers — there are no timestamps to cite.
|
|
@@ -283,8 +437,15 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
283
437
|
- **Ledger append (MUST).** Append ONE JSON line to `qa-rounds.jsonl` at
|
|
284
438
|
the game repo root, committed alongside the plan updates:
|
|
285
439
|
`{"round_id": …, "report_url": …, "build": "<sha or hosted url>",
|
|
286
|
-
"evidence_grade": …, "checks": [{"id": …, "outcome": …}],
|
|
287
|
-
"
|
|
440
|
+
"evidence_grade": …, "checks": [{"id": …, "outcome": …}],
|
|
441
|
+
"issues": {"game": "<game_slug>", "verified": [3], "regressed": [5],
|
|
442
|
+
"new": [7, 8], "still_open": [2]}, "verdict": …, "credits": …,
|
|
443
|
+
"follow_ups": […]}`. The `issues` object is the board's movement this
|
|
444
|
+
round, by seq (omit it entirely when the round carried no `game`), and
|
|
445
|
+
`follow_ups` is NON-BUG follow-ups only from now on — a check to add, a
|
|
446
|
+
build affordance to write, a question to settle. A bug goes on the
|
|
447
|
+
board, where it is numbered, re-checked and closable; free text nobody
|
|
448
|
+
re-reads is where bugs went to die. `report_url` is the round's service
|
|
288
449
|
report page (the `report_url` the results tool returns — never a PR or
|
|
289
450
|
issue link). No evidence-link field, deliberately: signed recording URLs
|
|
290
451
|
expire in days, so durable evidence pointers live in the report/issues,
|
|
@@ -292,6 +453,10 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
292
453
|
`last_verified` stays in QA-PLAN.md as the dev-readable contract derived
|
|
293
454
|
from it, and "carry findings forward" is a read of the last line's
|
|
294
455
|
follow_ups, not prose archaeology.
|
|
456
|
+
- **Close a parked round (MUST).** In the same step as the ledger append,
|
|
457
|
+
drop this round from `qa-open-rounds.json` with
|
|
458
|
+
`qaping rounds rm <ping_id>`, so the open-state file never outlives the
|
|
459
|
+
round it names.
|
|
295
460
|
- Update `QA-PLAN.md` from the outcomes: Pass → `state: passed` and
|
|
296
461
|
`last_verified` this build/date; a Fail with hard evidence → `failing`;
|
|
297
462
|
a Fail that could be dirty state or tester noise → `ambiguous`;
|
|
@@ -302,14 +467,33 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
302
467
|
a fragile area, or a QA lesson learned, offer to record it as a check
|
|
303
468
|
(or a note on an existing check) in `QA-PLAN.md` right then, so the plan
|
|
304
469
|
accretes knowledge instead of losing it to chat scrollback.
|
|
305
|
-
-
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
470
|
+
- **Triage every new issue in THIS turn (MUST).** Each `NEW #n` in the
|
|
471
|
+
results is a bug a playtester filed from inside the session, and it is
|
|
472
|
+
answered before the report is finished: accept it (`gh issue create` for
|
|
473
|
+
the dev's triage, then `qaping_issues` `action:'edit'` with
|
|
474
|
+
`external_ref` set to that URL), or close it — `duplicate` with
|
|
475
|
+
`duplicate_of_seq`, `invalid`, or `cannot_reproduce`. Never leave one
|
|
476
|
+
sitting; an untriaged board stops being worth reading, and the next
|
|
477
|
+
round re-asks a bug nobody decided about.
|
|
478
|
+
- **A REGRESSED issue reopens its mirror (MUST).** The fix did not hold:
|
|
479
|
+
`gh issue reopen <n>` on the GitHub issue, name it in the PR comment,
|
|
480
|
+
and never claim `fixed` on it again without a new build to claim
|
|
481
|
+
against.
|
|
482
|
+
- **A verified issue closes its mirror.** An issue a playtester turned
|
|
483
|
+
`verified` is settled: close the GitHub issue citing the round. Leave
|
|
484
|
+
the board row green — never `action:'close'` a verified issue to tidy
|
|
485
|
+
up; closed is for what you decided about, not for what was confirmed.
|
|
486
|
+
- Findings YOU file: one board issue per finding (`action:'create'`, in
|
|
487
|
+
player language, severity from consequence — `blocker` cannot play past
|
|
488
|
+
it / `major` / `minor` / `cosmetic`), then one `gh issue create` per
|
|
489
|
+
finding as the dev's own draft — never near-duplicates, never one issue
|
|
490
|
+
per symptom of the same bug — and `action:'edit'` the board row with the
|
|
491
|
+
`external_ref` you just created. Before reproduction the body carries
|
|
492
|
+
"observed once, unreproduced" plus that proposed severity. The label
|
|
493
|
+
CONFIRMED is reserved for a targeted round or unambiguous in-round
|
|
494
|
+
evidence; a CONFIRMED issue carries repro steps rebuilt from the
|
|
495
|
+
tester's answers and a timestamp link into the recording where one
|
|
496
|
+
exists (web rounds have none — cite answers).
|
|
313
497
|
|
|
314
498
|
## The ambiguity ladder
|
|
315
499
|
|
|
@@ -319,7 +503,11 @@ precondition forced via `setup` (or verified by the opening option step),
|
|
|
319
503
|
plus the disambiguation steps. That round settles it: Pass → `passed` (note
|
|
320
504
|
the fluke); Fail → `failing`, and the issue upgrades to CONFIRMED. Never
|
|
321
505
|
resolve an ambiguity by re-buying the whole batch, and never label a
|
|
322
|
-
once-seen fail CONFIRMED without that round or hard evidence.
|
|
506
|
+
once-seen fail CONFIRMED without that round or hard evidence. When the
|
|
507
|
+
targeted round settles it as a real bug, the board issue for it carries
|
|
508
|
+
`check_id: <the check's id>` — set at `create`, or with `action:'edit'` on a
|
|
509
|
+
playtester-filed issue — so the check and the bug that made it ambiguous stay
|
|
510
|
+
findable from each other.
|
|
323
511
|
|
|
324
512
|
## Paired rounds (two-state comparisons)
|
|
325
513
|
|