@qaping/cli 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/bin/qaping CHANGED
@@ -44,7 +44,8 @@ const RULE_BODY = `This machine has qaping: automatic QA for games — real huma
44
44
  Whenever the user asks to set up QA for a game, run QA on a patch, check whether a change broke anything, or playtest a game or build, load the qaping skill and follow it — it owns authoring/maintaining QA-PLAN.md in the game repo and the per-patch run.
45
45
  Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead).
46
46
  The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a finished round, free) and qaping_wait (continue a pending round — whenever a filing or wait returns pending, call qaping_wait again immediately; never report pending as the answer).
47
- Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording + questionnaire and no transcript; never promise one.
47
+ Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording and no transcript; web rounds return answers only (no recording, no transcript) — never promise evidence a platform does not ship.
48
+ QA memory: whenever the developer mentions — in ANY conversation, not just QA runs — something that should always be tested, a fragile area, or a QA lesson learned, offer to record it as a check (or a note on an existing check) in QA-PLAN.md so it is tested from then on.
48
49
  `;
49
50
 
50
51
  // The wrapper object harness/setup.js's opts.wrapper API takes; also the one
@@ -183,12 +184,21 @@ function cmdPublishBuild(argv) {
183
184
  .main(argv.slice(1), { brandCommand: "qaping publish-build", nextStepToolName: "qaping_playtest" });
184
185
  }
185
186
 
187
+ function cmdPublish(argv) {
188
+ // WEB games: their "build" is a static directory, hosted (not zipped) — the
189
+ // kit's hosted-draft publisher, brand-seamed like publish-build above. The
190
+ // printed URL files as `url` on qaping_playtest with NO platform field.
191
+ return require(path.join(resolveKitDir(), "harness", "publish.js"))
192
+ .main(argv.slice(1), { brandCommand: "qaping publish" });
193
+ }
194
+
186
195
  const HELP = `qaping — automatic QA for your game: real human playtesters, driven by your coding agent
187
196
 
188
197
  usage:
189
198
  qaping setup [claude-code|cursor|codex] [--force]
190
199
  qaping remove [--client <c>] remove qaping's MCP entries, skill and rule
191
200
  qaping publish-build <game.zip> --platform windows|macos
201
+ qaping publish <built-dir> host a WEB game's built output (prints the URL to file with)
192
202
  qaping wait <ping_id> [--timeout <seconds>]
193
203
  qaping whoami
194
204
  qaping version
@@ -202,6 +212,7 @@ function route(cmd) {
202
212
  if (cmd === "remove" || cmd === "uninstall") return "remove";
203
213
  if (cmd === "wait" || cmd === "whoami") return "vendor";
204
214
  if (cmd === "publish-build") return "publish-build";
215
+ if (cmd === "publish") return "publish";
205
216
  return "unknown";
206
217
  }
207
218
 
@@ -217,6 +228,7 @@ function main() {
217
228
  // vendored default (the stock mount's wait) is not on /api/mcp/qaping.
218
229
  case "vendor": return process.exit(spawnVendor(argv[0] === "wait" ? [...argv, "--wait-tool", "qaping_wait"] : argv));
219
230
  case "publish-build": return void cmdPublishBuild(argv);
231
+ case "publish": return void cmdPublish(argv);
220
232
  default:
221
233
  console.error(HELP);
222
234
  process.exit(1);
@@ -12,15 +12,16 @@ YAML at the very top of the file:
12
12
  ---
13
13
  game: Solar Drift
14
14
  build_command: ./scripts/package.sh --release # the ONE command that produces a shippable build
15
- platforms: [windows] # windows | macos | ios
15
+ platforms: [windows] # windows | macos | ios | web
16
16
  input: XInput gamepad # optional; default keyboard+mouse
17
17
  ---
18
18
  ```
19
19
 
20
20
  `build_command` is recorded once at setup so every future run can build without
21
- asking. `input` declares the hardware the first-15-minutes check (and any check
22
- that needs it) is played on rounds must ask playtesters to confirm what they
23
- actually used.
21
+ asking. `platforms: [web]` means a browser-playable game its "build" is a
22
+ static directory, not a zip. `input` declares the hardware the
23
+ first-15-minutes check (and any check that needs it) is played on — rounds
24
+ must ask playtesters to confirm what they actually used.
24
25
 
25
26
  ## Checks
26
27
 
@@ -30,19 +31,38 @@ opens with a fenced yaml block, then the steps as prose:
30
31
  ```yaml
31
32
  id: save-loads # stable slug — never reused, never renamed
32
33
  rung: human # human | code
34
+ state: unverified # unverified | passed | ambiguous | failing
33
35
  origin: authored # "authored", or the round id that minted/last changed it
34
36
  last_verified: 2026-08-25 # date or build tag of the last passing verification
37
+ setup: "?save=campaign-mid" # optional: the link/flag that forces this check's precondition
35
38
  ```
36
39
 
37
40
  - **Steps are prose for a person**: numbered, imperative, written so a
38
- playtester who has never seen the game can follow them they become the
39
- round's instructions verbatim.
41
+ playtester who has never seen the game can follow them. At filing time each
42
+ check becomes ONE id-tagged step with typed verdict options, so end the
43
+ steps in an observation ask ("describe what you saw"), never a
44
+ confirmation ("confirm it appears").
45
+ - **`state` is the check's verdict history**: `unverified` (never verified on
46
+ the current behavior), `passed` (last verified answer was Pass),
47
+ `ambiguous` (a fail that could be dirty state or tester noise — refiles
48
+ alone as a targeted round), `failing` (a fail backed by hard evidence or a
49
+ targeted round). Blocked/VOID outcomes change nothing here.
50
+ - **`setup` names the test affordance** that forces the check's precondition —
51
+ a URL param (`?fresh=1`, `?state=<checkpoint>`), a flag, a bundled-fixture
52
+ load control. The filed step then starts with "open <exact link>". A check
53
+ whose precondition can be neither forced nor verified is testimony-only —
54
+ say so in its prose.
55
+ - **Fixtures are provisioned by the dev's repo**: a previous-version save
56
+ ships inside the build (or seeds via URL param on web) and is named in the
57
+ check's steps — an anonymous playtester in a fresh browser cannot possess
58
+ it.
40
59
  - **Order the file critical-path first**: boots to menu, previous-version save
41
60
  loads, first 15 minutes on the declared input hardware, core loop, settings
42
61
  persist — then game-specific checks.
43
62
  - A `rung: code` check keeps its prose (it documents intent) and adds one
44
63
  `test:` line naming the repo test that now asserts it, e.g.
45
- `` test: `tests/save_compat.test.ts` ``.
64
+ `` test: `tests/save_compat.test.ts` ``. Mechanically-assertable checks
65
+ enter at `rung: code` on day one — human rounds are for judgment.
46
66
 
47
67
  ## Example section
48
68
 
@@ -52,20 +72,51 @@ last_verified: 2026-08-25 # date or build tag of the last passing verification
52
72
  ```yaml
53
73
  id: save-loads
54
74
  rung: human
75
+ state: unverified
55
76
  origin: authored
56
- last_verified: 2026-08-25
77
+ last_verified: never
78
+ setup: "?save=campaign-mid"
57
79
  ```
58
80
 
59
- 1. From the main menu, choose Continue.
60
- 2. Load the provided save "campaign-mid.sav" (made on the previous release).
61
- 3. Confirm the game resumes in the desert outpost with the inventory intact.
62
- 4. Play for one minute; note anything missing, corrupted, or visually wrong.
81
+ 1. Open the game with the provided link (it loads the save "campaign-mid.sav",
82
+ made on the previous release, from the build's bundled fixtures).
83
+ 2. Play for one minute in the desert outpost.
84
+ 3. Describe what you saw: where you resumed, what the inventory held, and
85
+ anything missing, corrupted, or visually wrong.
63
86
  ````
64
87
 
88
+ ## The round ledger — qa-rounds.jsonl
89
+
90
+ Beside the plan, at the game repo root, sits `qa-rounds.jsonl`: one JSON line
91
+ appended per round, committed alongside the plan updates —
92
+
93
+ ```json
94
+ {"round_id": "…", "report_url": "…", "build": "<sha or hosted url>",
95
+ "evidence_grade": "…", "checks": [{"id": "…", "outcome": "…"}],
96
+ "verdict": "…", "credits": 20, "follow_ups": ["…"]}
97
+ ```
98
+
99
+ Outcomes are `Pass | Fail | Blocked | NOT-ASKED` — a VOID (precondition
100
+ unverified/disproven) is recorded as `Blocked` with a "VOID: …" note in the
101
+ report, never its own enum value. `report_url` is the round's service report
102
+ page (what the results tool returns), never a PR or issue link. There is no
103
+ evidence-link field, deliberately: signed recording URLs expire in days.
104
+ The ledger is the machine truth of what was asked and answered, per check,
105
+ per round; the plan's `last_verified` and `state` are derived from it and
106
+ stay the dev-readable contract. Open `follow_ups` are what the next filing
107
+ must carry forward.
108
+
65
109
  ## Maintenance rules
66
110
 
67
- - Every run updates the run checks' `last_verified`; findings-driven edits
68
- record the round id as `origin`.
69
- - New checks start `rung: human`, `origin: authored`.
111
+ - Every run updates the run checks' `state` and `last_verified` from typed
112
+ outcomes (Pass stamps both; Blocked/VOID/NOT-ASKED touch neither);
113
+ findings-driven edits record the round id as `origin`.
114
+ - New checks start `rung: human` (unless mechanically assertable — then
115
+ `rung: code` with a repo test written at entry), `state: unverified`,
116
+ `origin: authored`.
70
117
  - Promotion (human → code) and any deletion/demotion happen only with the
71
118
  developer's explicit agreement.
119
+ - **QA memory**: the plan accretes from conversation, not just QA runs — when
120
+ the developer mentions something that should always be tested, a fragile
121
+ area, or a lesson learned, the agent offers to record it as a check (or a
122
+ note on an existing check) right then.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@qaping/cli",
3
- "version": "0.1.0",
3
+ "version": "0.1.2",
4
4
  "description": "qaping: automatic QA for your game. Your coding agent authors and maintains the QA plan, ships each patch's build to real human playtesters, and reports what broke.",
5
5
  "keywords": [
6
6
  "qaping",
@@ -1,36 +1,61 @@
1
1
  ---
2
2
  name: qaping
3
- description: Run automatic QA for a game repo through qaping — real human playtesters play the developer's builds on their own hardware, and the agent runs the whole loop. Use when the user says "set up QA for my game", "run QA on this patch", "did this patch break anything", "playtest my game" or "playtest this build", or asks for regression checks, patch verification, or human playtesting of a game. Covers both halves of the product - authoring and maintaining QA-PLAN.md in the game repo, and the per-patch run (read the diff, build, publish-build, file qaping_playtest rounds, report on the PR, file issues, promote settled checks into repo code tests). Do not use for web apps, websites, or UI review of anything that is not a game build — the qaping tools test game builds only.
3
+ description: Run automatic QA for a game repo through qaping — real human playtesters play the developer's builds on their own hardware, and the agent runs the whole loop. Use when the user says "set up QA for my game", "run QA on this patch", "did this patch break anything", "playtest my game" or "playtest this build", or asks for regression checks, patch verification, or human playtesting of a game. Covers both halves of the product - authoring and maintaining QA-PLAN.md in the game repo, and the per-patch run (read the diff, build, publish-build, file qaping_playtest rounds, report on the PR, file issues, promote settled checks into repo code tests). Browser-playable web GAMES count as games (they file as web rounds — url only, no platform). Do not use for web apps, websites, or UI review of anything that is not a game — the qaping tools test games only.
4
4
  ---
5
5
 
6
6
  # qaping — the QA loop for a game repo
7
7
 
8
8
  Two jobs, one file. SETUP authors `QA-PLAN.md` — the committed list of checks
9
- this game must pass. RUN executes it per patch: pick the affected checks, build,
10
- ship the build to real human playtesters, report what they found. Over patches,
11
- checks migrate from human rounds to repo code tests, so QA gets cheaper. All
12
- service contact goes through three MCP tools `qaping_playtest`,
13
- `qaping_results`, `qaping_wait` plus `qaping publish-build` for hosting
14
- the build.
9
+ this game must pass. RUN executes it per patch: pick the affected checks,
10
+ build, ship the build to real human playtesters, and report ONE typed outcome
11
+ per check never a vibe. The check is the unit of value: its id rides from
12
+ the plan onto the wire, into the report, and into the round ledger. Over
13
+ patches, checks migrate from human rounds to repo code tests, so QA gets
14
+ cheaper. All service contact goes through three MCP tools —
15
+ `qaping_playtest`, `qaping_results`, `qaping_wait` — plus
16
+ `qaping publish-build` for hosting the build.
15
17
 
16
18
  ## Ground rules (read before filing anything)
17
19
 
18
20
  - **Playtesters are real people.** A session takes real minutes to be claimed
19
21
  and played. File, then `qaping_wait`; whenever a filing or wait returns
20
- pending, immediately call `qaping_wait` again with the same ping_id. Never
21
- report pending as the final answer, never file a duplicate round.
22
+ pending, immediately call `qaping_wait` again with the same ping_id — keep
23
+ polling, or the round goes invisible to new claimants when the lease lapses.
24
+ Never report pending as the final answer, never file a duplicate round.
22
25
  - **Costs are duration-billed**: 2 credits per minute of play, per playtester
23
- (`est_minutes × 2 × players`). State the estimate to the user BEFORE filing.
24
- - **Windows rounds return a recording + questionnaire and NO transcript**
25
- (`transcript_status:'unavailable'`)never promise one; plan to watch the
26
- recording for timestamps. macOS and iOS rounds include an inline [mm:ss]
27
- think-aloud transcript.
28
- - **Machine replay is not self-serve yet.** Never claim an automated replay ran.
29
- The only automated rung you can run today is a `rung: code` test in the
30
- dev's own repo/CI.
26
+ (`est_minutes × 2 × players`). State the estimate to the user BEFORE filing,
27
+ and set the wait expectation honestly in the same message: claim time is
28
+ unbounded on a small pool "push now or wait for QA" is their call, made
29
+ informed.
30
+ - **Evidence varies by platform — never promise what a platform does not
31
+ ship.** macOS and iOS rounds return a screen+voice recording plus an inline
32
+ [mm:ss] think-aloud transcript. Windows rounds return a recording and NO
33
+ transcript (`transcript_status:'unavailable'`) never promise one; plan to
34
+ watch the recording for timestamps. WEB rounds return answers only — no
35
+ recording, no transcript: the tester's option picks and free text are the
36
+ entire deliverable, which is why every check rides an option step.
37
+ - **Pay is flat per session, never per bug.** Never offer or imply a bounty.
38
+ Never steer a tester's OS-security decision (stopping at a warning is their
39
+ penalty-free call); steering their PLAY — steps, links, forced states — is
40
+ the job.
41
+ - **Machine replay is not self-serve yet.** Never claim an automated replay
42
+ ran. The only automated rung you can run today is a `rung: code` test in
43
+ the dev's own repo/CI.
31
44
  - **QA-PLAN.md is the dev's file.** Commit it only with their approval; never
32
45
  delete or demote a human check without them agreeing.
33
46
 
47
+ ## Know your tester
48
+
49
+ The person who claims your round is on their phone, scrolling a feed. They
50
+ read `instructions` BEFORE claiming — front-load hardware and requirements
51
+ there so the wrong tester never claims. They play with your steps visible on
52
+ the same small screen, with zero context and no dev tools; attention is
53
+ highest in the first minutes; setup friction is unpaid time. So: one action
54
+ per step, exact UI labels (the button says "Continue", not "the resume
55
+ option"), the most important check first, state-sensitive checks forced by a
56
+ link (see "Make the build testable"), and never a step that assumes they read
57
+ your repo.
58
+
34
59
  ## SETUP — author QA-PLAN.md (first run in a repo)
35
60
 
36
61
  1. Read the game repo: engine, how a shippable build is produced, save system,
@@ -38,75 +63,231 @@ the build.
38
63
  2. Ask the dev ONCE for their build command (the one command that produces a
39
64
  shippable build) and which platforms they ship; record both in the plan's
40
65
  frontmatter so every future run can build without asking again.
41
- 3. Author `QA-PLAN.md` at the repo root, format per `docs/QA-PLAN-FORMAT.md`
42
- (shipped with this package read it first). Order checks critical-path
43
- first, then game-specific ones:
66
+ 3. Author `QA-PLAN.md` at the repo root in EXACTLY this format (self-contained
67
+ here do not go looking for other docs). Plan frontmatter (YAML at the
68
+ very top): `game`, `build_command` (the ONE shippable-build command),
69
+ `platforms: [windows|macos|ios|web]` (`web` = a browser-playable game — its
70
+ "build" is a static directory, not a zip), optional `input` (hardware;
71
+ default keyboard+mouse, or touch for a mobile-web game).
72
+ Then one `##` section per check — the heading is the check's short name —
73
+ opening with a fenced yaml block of `id` (stable slug, never renamed),
74
+ `rung: human | code`, `state` (`unverified | passed | ambiguous | failing`
75
+ — new checks start `unverified`), `origin` ("authored" or the round id that
76
+ last changed it), `last_verified` (date or build tag), and optionally
77
+ `setup` (the link or flag that forces this check's precondition — see
78
+ "Make the build testable"), followed by the steps as numbered prose a
79
+ playtester who has never seen the game can follow. Author each check to
80
+ end in an observation ask, with its precondition stated as a verifiable
81
+ step-zero fact (forced by `setup`, else verified by an opening question).
82
+ A `rung: code` check keeps its prose and adds one `test:` line naming the
83
+ repo test that asserts it. Order checks critical-path first, then
84
+ game-specific ones:
44
85
  1. the game boots to the main menu
45
86
  2. a previous-version save loads
46
87
  3. the first 15 minutes play clean on the declared input hardware
47
88
  4. the core loop works (name it concretely — the thing this game is)
48
89
  5. settings persist across a restart
49
- Every check carries `{id, rung, origin, last_verified}`. New checks start
50
- `rung: human`, `origin: authored`.
51
- 4. Tell the dev how runs get triggered, honestly: for now the loop runs from
90
+ 4. **Enter each check at the right rung.** A check that is mechanically
91
+ assertable a save file loads, a config parses, an asset path resolves —
92
+ enters at `rung: code` from day one: write the repo test then and there,
93
+ never "start it human and promote later". Human rounds are reserved for
94
+ what only a human can judge. Where a save must still be human-tested, the
95
+ dev provides the fixture: the previous release's save ships inside the
96
+ build (or seeds via URL param on web), named in the check's steps — a
97
+ stranger in a fresh browser cannot possess your old save.
98
+ 5. **Gyms**: for a mechanic worth isolating, you may build a tiny
99
+ deterministic test scene or page in the dev's repo — one mechanic,
100
+ machine-readable pass/fail. Ordinary dev work; offer it, never force it.
101
+ 6. Tell the dev how runs get triggered, honestly: for now the loop runs from
52
102
  their own machine or agent — after a patch lands, they say "run QA on this
53
- patch" and the RUN half below executes. Do NOT offer to wire a CI workflow
54
- file: a plain CI runner has no qaping MCP connection and no login, so a
55
- committed workflow could not run this loop today. A CI-triggered recipe is
56
- a documented follow-up; until it ships, per-patch QA is an ask-your-agent
57
- step.
58
- 5. Show the dev the plan and commit only on their approval.
103
+ patch". Do NOT offer to wire a CI workflow file: a plain CI runner has no
104
+ qaping MCP connection and no login, so a committed workflow could not run
105
+ this loop today; a CI-triggered recipe is a documented follow-up. Until it
106
+ ships, per-patch QA is an ask-your-agent step.
107
+ 7. Show the dev the plan and commit only on their approval.
108
+
109
+ ## Make the build testable
110
+
111
+ Before publishing a test build, you SHOULD add test affordances to the dev's
112
+ own code — ordinary dev work that turns self-reported state into forced,
113
+ verifiable state:
114
+
115
+ - URL params or flags that force states: `?fresh=1` (a true first visit),
116
+ `?difficulty=medium`, `?state=<checkpoint>` (start at a checkpoint).
117
+ - Bundled save fixtures plus a load control (the previous release's save,
118
+ shipped in the build).
119
+ - Seeded randomness, so a reported oddity reproduces.
120
+ - A corner build stamp (build hash + first-run/save-state flag; a `?qa=1`
121
+ overlay on web) with a step asking the tester to read it back — a read-back
122
+ fact is verified; self-report is not.
123
+
124
+ Record each check's hook in its `setup` field; that check's filed step then
125
+ STARTS with "open <exact link>". The unreviewed-developer-build disclosure
126
+ already covers shipping these; the no-steering rule forbids steering SAFETY
127
+ decisions only. A store-delivered build that cannot surface state keeps its
128
+ checks testimony-only — mark that in the plan and report at that grade.
59
129
 
60
130
  ## RUN — per patch
61
131
 
62
132
  1. **Select.** Read the diff (or PR). Pick the checks it can plausibly affect,
63
- plus the always-run criticals (the critical-path block above). Tell the user
64
- which checks run and why.
65
- 2. **Build** via the recorded build command; zip the result.
66
- 3. **Ship:** `qaping publish-build <game.zip> --platform windows|macos`
67
- prints a `/b/<slug>` URL (temporary hosting; filing a playtest extends the
68
- build through the round; each publish mints a NEW URL). A store-delivered
69
- game files with its Steam store page, TestFlight public link, or App Store
70
- page as `url` instead.
71
- 4. **File** ONE `qaping_playtest` per batch of human checks a single session
72
- can cover, `est_minutes` an honest sum of the steps the service accepts
73
- 5–30 minutes (10 is the standard session; outside that range the filing is
74
- refused, not clamped), and the clock is play time: download/install happens
75
- before it starts. The authored
76
- `instructions` + `steps` must embed:
77
- - the selected checks' steps, in play order (the plan's prose, verbatim
78
- where possible);
79
- - the previous round's findings as context "last round reported X
80
- recheck" so regressions get re-examined on every patch;
81
- - the hardware-verification convention: when the game needs more than
82
- keyboard+mouse, say so up front in `instructions` (playtesters self-select
83
- before claiming) AND add an explicit step asking which input device was
84
- actually used, with options the answer comes back verified in
85
- `steps_result`.
86
- For a regression batch that knows exactly what it is testing, send
87
- `questionnaire:'none'` so your steps are the only questions; leave the
88
- standard questionnaire on for a first baseline round.
89
- 5. **Wait:** `qaping_wait` until results arrive (see ground rules).
90
- 6. **Report** from `qaping_results`:
91
- - PR comment via `gh pr comment`: ONE verdict sentence first, then one line
92
- per check ✅/❌ with a recording timestamp ([mm:ss] from the transcript
93
- where present; from watching the recording on Windows) then credits
94
- spent.
95
- - One `gh issue create` per CONFIRMED bug: title, repro steps rebuilt from
96
- the playtester's answers, a timestamp link into the recording. Issues are
97
- drafts for the dev's own triage — never near-duplicates, never one issue
98
- per symptom of the same bug.
99
- - Update `QA-PLAN.md`: each run check's `last_verified` becomes this
100
- build/date; a check changed because of this round records the round id as
101
- `origin`.
133
+ plus the always-run criticals (the critical-path block above) and every
134
+ check the ledger left NOT-ASKED or Blocked last round. Tell the user which
135
+ checks run and why.
136
+ 2. **Compose sessions.** Partition the selected checks by required state —
137
+ fresh-install / progression / settings-mutating and put state-mutating
138
+ checks LAST within their session. Budget play content: sum the checks'
139
+ directed play time; if it exceeds est_minutes, split into parallel
140
+ 5-minute rounds (credit-neutral: two focused 5-minute rounds cost one
141
+ diluted 10-minute round) or raise est_minutes within the cap, and state
142
+ the trade beside the cost estimate. Depth beyond the budget rotates across
143
+ patches; note in the plan which slice ran last.
144
+ 3. **Build** via the recorded build command, with each selected check's
145
+ `setup` affordance in place; zip the result (native) or keep the built
146
+ static directory (web).
147
+ 4. **Ship**, by delivery: NATIVE uploads `qaping publish-build <game.zip>
148
+ --platform windows|macos` → prints a `/b/<slug>` URL (temporary hosting;
149
+ filing a playtest extends the build through the round; each publish mints a
150
+ NEW URL). Store-delivered games file with their Steam store page, TestFlight
151
+ public link, or App Store page as `url`. WEB games — host the built output
152
+ at a public URL: the dev's own hosting if they have it, else
153
+ `qaping publish <built-dir>` prints a hosted URL. Localhost never works
154
+ players open the link on their own devices. When filing a WEB game: send
155
+ `url` + `est_minutes` and OMIT `platform` entirely — a web round is claimed
156
+ by players in their own browser, phones included.
157
+ 5. **File** ONE `qaping_playtest` per composed session, always with
158
+ `questionnaire:'none'` on a QA-PLAN round your checks are the questions;
159
+ the standard instrument is a separate experience round (below), never a
160
+ default. est_minutes is an honest sum of the steps' directed play — the
161
+ service accepts 5–30 minutes (10 is the standard session; outside that
162
+ range the filing is refused, not clamped), and the clock is play time:
163
+ download/install happens before it starts. The authored steps MUST be:
164
+ - **one step per selected check**, its text opening with the id token —
165
+ `[spin-readable] …` the plan prose rewritten as an observation
166
+ question ("describe what the start menu showed and what you tapped
167
+ first", never "confirm the menu appears"), with options exactly
168
+ ["Pass as described", "Fail — describe exactly what you saw",
169
+ "Couldn't test say why", "Didn't reach this part"] — the pick comes
170
+ back verified in steps_result;
171
+ - a check with a `setup` hook opens "open <exact link>"; one whose
172
+ precondition cannot be forced opens with a verifying option step
173
+ ("Before opening: had you ever played this game before?" — yes / no /
174
+ not sure);
175
+ - a recheck of a prior finding is a clearly-labeled step at its natural
176
+ play-order point (first when its required state demands it, last
177
+ otherwise) — "VERIFY FIX: last round saw X — does it still happen?" —
178
+ and prior findings stay OUT of `instructions` entirely, so the rest of
179
+ the pass is an independent look;
180
+ - a closing **break-it slice** (~2–3 minutes) authored from the diff:
181
+ "this patch changed X — try to break it: <concrete abuse ideas from the
182
+ diff>. Report the weirdest thing you saw AND list what you tried."
183
+ Finding nothing is a fine answer when we know what was attempted; frame
184
+ it in `instructions` as "these checks are the territory — break anything
185
+ near them";
186
+ - hardware verification where it matters: needs beyond keyboard+mouse go
187
+ up front in `instructions` AND an explicit step asks which input device
188
+ was actually used, with options.
189
+ **Filing gate (MUST):** print `this round covers: <ids>` per session and
190
+ assert the ids — unioned across every session this run files — equal the
191
+ Select step's list BEFORE calling qaping_playtest. A mismatch (e.g. the
192
+ 20-step cap forcing cuts) fails loudly PRE-SPEND: batch another session or
193
+ state the deferral out loud — never silently truncate.
194
+ 6. **Wait:** `qaping_wait` until results arrive (see ground rules).
195
+ 7. **Report** from `qaping_results`:
196
+ - **Evidence line first (MUST).** Every report opens with its grade:
197
+ `Evidence: recording + transcript` (macOS/iOS) / `Evidence: recording
198
+ only` (Windows) / `Evidence: testimony only (web round)`.
199
+ - **Computed headline (MUST).** Derive the one verdict sentence from the
200
+ per-check outcomes, never from the tester's own verdict: `PASS` only
201
+ when every selected check returned Pass; `PASS — N unverified (<ids>)`
202
+ when nothing failed but any check is Blocked or NOT-ASKED; `FAIL: <id>`
203
+ on any Fail. Quote the tester's global verdict beneath it as one input —
204
+ it is never the headline. (The iterate rule already says unresolved
205
+ findings mean fix and refile — unresolved never headlines as PASS.)
206
+ - **One row per SELECTED check (MUST)**, keyed by id, outcome from
207
+ {Pass, Fail, Blocked, NOT-ASKED}. "Couldn't test" and "Didn't reach this
208
+ part" are Blocked, with the why. A check whose precondition came back
209
+ unverified or disproven is VOID — report it as Blocked ("VOID:
210
+ precondition unverified"), never as a pass or fail, and leave
211
+ `last_verified` untouched. A selected check that never made the wire or
212
+ never came back is NOT-ASKED and carries forward into the next filing
213
+ automatically. Cite [mm:ss] where a transcript or recording exists; on a
214
+ web round cite the step answers — there are no timestamps to cite.
215
+ - PR comment via `gh pr comment`: evidence line, the computed verdict
216
+ sentence first, then the per-check rows, then credits spent.
217
+ - **Ledger append (MUST).** Append ONE JSON line to `qa-rounds.jsonl` at
218
+ the game repo root, committed alongside the plan updates:
219
+ `{"round_id": …, "report_url": …, "build": "<sha or hosted url>",
220
+ "evidence_grade": …, "checks": [{"id": …, "outcome": …}], "verdict": …,
221
+ "credits": …, "follow_ups": […]}`. `report_url` is the round's service
222
+ report page (the `report_url` the results tool returns — never a PR or
223
+ issue link). No evidence-link field, deliberately: signed recording URLs
224
+ expire in days, so durable evidence pointers live in the report/issues,
225
+ not the ledger. The ledger is machine truth;
226
+ `last_verified` stays in QA-PLAN.md as the dev-readable contract derived
227
+ from it, and "carry findings forward" is a read of the last line's
228
+ follow_ups, not prose archaeology.
229
+ - Update `QA-PLAN.md` from the outcomes: Pass → `state: passed` and
230
+ `last_verified` this build/date; a Fail with hard evidence → `failing`;
231
+ a Fail that could be dirty state or tester noise → `ambiguous`;
232
+ Blocked/NOT-ASKED touch nothing. A check changed because of this round
233
+ records the round id as `origin`.
234
+ - **QA memory (standing rule, not just here)**: whenever the developer
235
+ mentions — in any conversation — something that should always be tested,
236
+ a fragile area, or a QA lesson learned, offer to record it as a check
237
+ (or a note on an existing check) in `QA-PLAN.md` right then, so the plan
238
+ accretes knowledge instead of losing it to chat scrollback.
239
+ - Issues: one `gh issue create` per finding — drafts for the dev's own
240
+ triage, never near-duplicates, never one issue per symptom of the same
241
+ bug. Before reproduction the body carries "observed once, unreproduced"
242
+ plus a proposed severity from consequence (blocks progress / degrades /
243
+ cosmetic). The label CONFIRMED is reserved for a targeted round or
244
+ unambiguous in-round evidence; a CONFIRMED issue carries repro steps
245
+ rebuilt from the tester's answers and a timestamp link into the
246
+ recording where one exists (web rounds have none — cite answers).
247
+
248
+ ## The ambiguity ladder
249
+
250
+ An `ambiguous` or first-time-failing check never re-runs inside the next
251
+ batch — it refiles ALONE as a targeted ~5-minute round: one check, its
252
+ precondition forced via `setup` (or verified by the opening option step),
253
+ plus the disambiguation steps. That round settles it: Pass → `passed` (note
254
+ the fluke); Fail → `failing`, and the issue upgrades to CONFIRMED. Never
255
+ resolve an ambiguity by re-buying the whole batch, and never label a
256
+ once-seen fail CONFIRMED without that round or hard evidence.
257
+
258
+ ## Differential rounds (web feel checks)
259
+
260
+ A feel/physics/balance regression judged absolute buys little — a stranger
261
+ answers "did spin feel right" in a vacuum. On WEB builds, where switching
262
+ costs seconds, such a check MAY file as ONE session carrying BOTH builds:
263
+ publish the previous build alongside the current one (the ledger's `build`
264
+ field makes it recoverable), put the second URL in `instructions`, and author
265
+ the steps as "play 2 minutes on build A, then build B — name anything that
266
+ changed and where". est_minutes honestly covers both halves. Caveat the
267
+ report: a tester told to hunt changes reports noise as change, and order
268
+ effects apply. Store-delivered and uploaded native builds stay single-build —
269
+ no second-URL surface exists for them.
270
+
271
+ ## The experience round (opt-in, never a default)
272
+
273
+ The standard 11-item player-experience questionnaire never rides a QA-PLAN
274
+ round. Offer it separately, at milestones, as an experience round: no checks
275
+ attached, questionnaire left on, n_target up to 3 players, priced
276
+ transparently (est_minutes × 2 × players — state it first). Single-tester
277
+ scores are trend data — meaningful only after 3+ rounds on the same game,
278
+ never compared across testers or builds at n=1. If 8 or more of the 11 scale
279
+ answers are identical, report "low-differentiation response — treat scores as
280
+ unreliable" and never average them into a comparison.
102
281
 
103
282
  ## PROMOTE — make the suite cheaper
104
283
 
105
284
  When a check has passed and is mechanically assertable — a save file loads, a
106
285
  config parses, an asset path resolves — offer to write a normal test in the
107
- repo's own test suite. On the dev's yes: write the test, flip the check's
286
+ repo's own test suite; the ledger's per-check pass history is the evidence
287
+ that justifies the offer. On the dev's yes: write the test, flip the check's
108
288
  `rung` to `code`, keep the prose (it documents intent) and add a `test:` line
109
289
  naming the test file. `rung: code` checks then run in the dev's own CI for
110
- free — that is the point: every check starts human and gets cheaper over time.
111
- Checks about feel, difficulty, or ambiguous visuals stay `rung: human`
112
- permanently. Never delete or demote a human check without the dev agreeing.
290
+ free — that is the point: every check starts as human as it must and gets
291
+ cheaper over time. Checks about feel, difficulty, or ambiguous visuals stay
292
+ `rung: human` permanently. Never delete or demote a human check without the
293
+ dev agreeing.