@qaping/cli 0.1.2 → 0.1.4

package/bin/qaping

@@ -43,7 +43,7 @@ function resolveKitDir() {
 const RULE_BODY = `This machine has qaping: automatic QA for games — real human playtesters play the developer's builds on their own hardware and report back, and the coding agent runs the whole loop.
 Whenever the user asks to set up QA for a game, run QA on a patch, check whether a change broke anything, or playtest a game or build, load the qaping skill and follow it — it owns authoring/maintaining QA-PLAN.md in the game repo and the per-patch run.
 Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead).
-The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a finished round, free) and qaping_wait (continue a pending round — whenever a filing or wait returns pending, call qaping_wait again immediately; never report pending as the answer).
+The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a finished round, free) and qaping_wait (continue a pending round — on pending, spawn a background task that loops qaping_wait and keep working; foreground polling is the fallback, and never report pending as the answer or end with an unbacked promise to check later).
 Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording and no transcript; web rounds return answers only (no recording, no transcript) — never promise evidence a platform does not ship.
 QA memory: whenever the developer mentions — in ANY conversation, not just QA runs — something that should always be tested, a fragile area, or a QA lesson learned, offer to record it as a check (or a note on an existing check) in QA-PLAN.md so it is tested from then on.
 `;

package/docs/QA-PLAN-FORMAT.md

@@ -49,7 +49,10 @@ setup: "?save=campaign-mid"  # optional: the link/flag that forces this check's
   targeted round). Blocked/VOID outcomes change nothing here.
 - **`setup` names the test affordance** that forces the check's precondition —
   a URL param (`?fresh=1`, `?state=<checkpoint>`), a flag, a bundled-fixture
-  load control. The filed step then starts with "open <exact link>". A check
+  load control. At filing time the hook rides the ROUND's `url` — the review
+  interface loads it itself; testers cannot open links or type URLs — and
+  the filed step states only the player-visible expectation, never the
+  mechanism. A check
   whose precondition can be neither forced nor verified is testimony-only —
   say so in its prose.
 - **Fixtures are provisioned by the dev's repo**: a previous-version save
@@ -78,8 +81,8 @@ last_verified: never
 setup: "?save=campaign-mid"
 ```
 
-1. Open the game with the provided link (it loads the save "campaign-mid.sav",
-   made on the previous release, from the build's bundled fixtures).
+1. The game opens already loaded into the save "campaign-mid.sav" (made on
+   the previous release; the round's link forces it — no action needed).
 2. Play for one minute in the desert outpost.
 3. Describe what you saw: where you resumed, what the inventory held, and
    anything missing, corrupted, or visually wrong.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@qaping/cli",
-  "version": "0.1.2",
+  "version": "0.1.4",
   "description": "qaping: automatic QA for your game. Your coding agent authors and maintains the QA plan, ships each patch's build to real human playtesters, and reports what broke.",
   "keywords": [
     "qaping",

package/skill/qaping/SKILL.md

@@ -17,11 +17,18 @@ cheaper. All service contact goes through three MCP tools —
 
 ## Ground rules (read before filing anything)
 
-- **Playtesters are real people.** A session takes real minutes to be claimed
-  and played. File, then `qaping_wait`; whenever a filing or wait returns
-  pending, immediately call `qaping_wait` again with the same ping_id — keep
-  polling, or the round goes invisible to new claimants when the lease lapses.
-  Never report pending as the final answer, never file a duplicate round.
+- **Playtesters are real people, and waiting is background work.** A session
+  takes real minutes to be claimed and played, and the wait chain matters: each
+  `qaping_wait` renews the lease that keeps the round visible to new claimants.
+  The moment a filing returns pending, SPAWN A BACKGROUND TASK that loops
+  `qaping_wait` with the ping_id until results arrive, and keep working on
+  other things — that is the intended shape, the same as every pingfusi round.
+  Only if your harness has no background tasks: keep polling in the foreground.
+  Only in a one-shot print mode where neither is possible: say plainly that the
+  round is pending, give the ping_id, and ask to be re-invoked to continue —
+  never promise polling you cannot do (an ended session cannot poll; a round
+  went invisible and expired unclaimed exactly this way, live). Never report
+  pending as the final answer, never file a duplicate round.
 - **Costs are duration-billed**: 2 credits per minute of play, per playtester
   (`est_minutes × 2 × players`). State the estimate to the user BEFORE filing,
   and set the wait expectation honestly in the same message: claim time is
@@ -44,17 +51,46 @@ cheaper. All service contact goes through three MCP tools —
 - **QA-PLAN.md is the dev's file.** Commit it only with their approval; never
   delete or demote a human check without them agreeing.
 
-## Know your tester
+## The review interface (know your tester)
 
-The person who claims your round is on their phone, scrolling a feed. They
-read `instructions` BEFORE claiming — front-load hardware and requirements
-there so the wrong tester never claims. They play with your steps visible on
-the same small screen, with zero context and no dev tools; attention is
-highest in the first minutes; setup friction is unpaid time. So: one action
-per step, exact UI labels (the button says "Continue", not "the resume
-option"), the most important check first, state-sensitive checks forced by a
-link (see "Make the build testable"), and never a step that assumes they read
-your repo.
+<!-- Distilled from QAPING_REVIEWER_INTERFACE.md (monorepo root, INTERNAL,
+     code-cited). Re-derive there when a reviewer surface changes. -->
+
+Binding, derived from the review interfaces' code. The person who claims a
+web round is on their phone, scrolling a feed. They read `instructions`
+BEFORE claiming — front-load hardware and requirements there so the wrong
+tester never claims. On claim the game auto-loads into an embedded browser
+view with your steps beside it on the same small screen, with zero context;
+attention is highest in the first minutes; setup friction is unpaid time.
+
+The tester CAN: play the already-loaded game (touch, scroll, type); go
+Back; Reload — which KEEPS storage, so a reload is never a fresh state; see
+all steps at once and answer in any order; tap ONE option per option step;
+type a note on any question step; pin a comment to a page element and draw
+on it; pick one verdict; bail out with a reason.
+
+The tester CANNOT:
+- open tabs or windows — window.open and target=_blank links are dead
+  INSIDE the game too, so a game whose Play button opens a new window is
+  broken-for-review: flag it to the dev instead of filing;
+- type or edit a URL: the round's `url` is the only page-load there is, and
+  links in step text are inert on the phone — never put links in steps;
+- clear storage or go incognito: fresh state comes ONLY from a setup URL
+  param the round's `url` already carries;
+- use devtools, attach files, or record — web rounds return answers only;
+- use a second device;
+- see `instructions` or `expected` once inside the review — EVERYTHING the
+  tester needs at answer time lives in the step text (≤300 chars each).
+
+**Steps speak player language (MUST).** Things to tap and see in the game —
+never flags, URL parameters, storage, or code concepts. A test hook is
+exercised by the ROUND's `url` carrying it; the step describes only the
+player-visible expectation ("the How to play option should be offered
+first"), never the mechanism. Never ask to close/reopen anything or "start
+fresh" — that state is a separate round whose `url` forces it. And as
+always: one action per step, exact UI labels (the button says "Continue",
+not "the resume option"), the most important check first, and never a step
+that assumes they read your repo.
 
 ## SETUP — author QA-PLAN.md (first run in a repo)
 
@@ -121,8 +157,10 @@ verifiable state:
   overlay on web) with a step asking the tester to read it back — a read-back
   fact is verified; self-report is not.
 
-Record each check's hook in its `setup` field; that check's filed step then
-STARTS with "open <exact link>". The unreviewed-developer-build disclosure
+Record each check's hook in its `setup` field; at filing time the hook
+rides the ROUND's `url` — the game loads it itself, so the tester never
+sees or types it — and the filed step states only the player-visible
+expectation. The unreviewed-developer-build disclosure
 already covers shipping these; the no-steering rule forbids steering SAFETY
 decisions only. A store-delivered build that cannot surface state keeps its
 checks testimony-only — mark that in the plan and report at that grade.
@@ -135,7 +173,10 @@ checks testimony-only — mark that in the plan and report at that grade.
    checks run and why.
 2. **Compose sessions.** Partition the selected checks by required state —
    fresh-install / progression / settings-mutating — and put state-mutating
-   checks LAST within their session. Budget play content: sum the checks'
+   checks LAST within their session. A session has ONE state: its round
+   `url` (carrying the forcing setup param) is the only state control that
+   exists — testers cannot clear storage or start fresh themselves — so
+   checks needing different forced states never share a session. Budget play content: sum the checks'
    directed play time; if it exceeds est_minutes, split into parallel
    5-minute rounds (credit-neutral: two focused 5-minute rounds cost one
    diluted 10-minute round) or raise est_minutes within the cap, and state
@@ -157,8 +198,11 @@ checks testimony-only — mark that in the plan and report at that grade.
 5. **File** ONE `qaping_playtest` per composed session, always with
    `questionnaire:'none'` — on a QA-PLAN round your checks are the questions;
    the standard instrument is a separate experience round (below), never a
-   default. est_minutes is an honest sum of the steps' directed play — the
-   service accepts 5–30 minutes (10 is the standard session; outside that
+   default. A WEB session additionally files `require_evidence:'none'`
+   (MUST) — left at its default, the finish silently demands a pinned
+   screenshot-comment no step asked for. est_minutes is an
+   honest sum of the steps' directed play — the service
+   accepts 5–30 minutes (10 is the standard session; outside that
    range the filing is refused, not clamped), and the clock is play time:
    download/install happens before it starts. The authored steps MUST be:
    - **one step per selected check**, its text opening with the id token —
@@ -167,11 +211,20 @@ checks testimony-only — mark that in the plan and report at that grade.
      first", never "confirm the menu appears"), with options exactly
      ["Pass — as described", "Fail — describe exactly what you saw",
      "Couldn't test — say why", "Didn't reach this part"] — the pick comes
-     back verified in steps_result;
-   - a check with a `setup` hook opens "open <exact link>"; one whose
-     precondition cannot be forced opens with a verifying option step
-     ("Before opening: had you ever played this game before?" — yes / no /
-     not sure);
+     back verified in steps_result. EXCEPTION: when the check asks the
+     tester to DISCRIMINATE between named states ("which difficulty did the
+     bot play like?", "which screen did you land on?"), the options are the
+     NAMED candidates themselves ("rookie-level" / "harder than that") plus
+     "Couldn't tell — say why" — generic Pass/Fail on a discrimination
+     yields contradictions instead of a signal (measured live);
+   - option labels ≤40 chars, 2–4 options per step; never author a final
+     step whose options duplicate the verdict options — the phone hides it
+     as a duplicate of the verdict control;
+   - a check with a `setup` hook is forced by filing the session's `url`
+     WITH the hook — the game loads it itself; the step states only the
+     player-visible expectation. One whose precondition cannot be forced
+     opens with a verifying option step ("Before this session: had you ever
+     played this game before?" — yes / no / not sure);
    - a recheck of a prior finding is a clearly-labeled step at its natural
      play-order point (first when its required state demands it, last
      otherwise) — "VERIFY FIX: last round saw X — does it still happen?" —
@@ -180,6 +233,9 @@ checks testimony-only — mark that in the plan and report at that grade.
    - a closing **break-it slice** (~2–3 minutes) authored from the diff:
      "this patch changed X — try to break it: <concrete abuse ideas from the
      diff>. Report the weirdest thing you saw AND list what you tried."
+     Abuse ideas target PLAYER-VISIBLE behaviors the diff changed — things a
+     player can tap, see, or provoke — never internal mechanisms (a flag, a
+     cache, a storage key means nothing to a player).
      Finding nothing is a fine answer when we know what was attempted; frame
      it in `instructions` as "these checks are the territory — break anything
      near them";
@@ -188,10 +244,16 @@ checks testimony-only — mark that in the plan and report at that grade.
      was actually used, with options.
    **Filing gate (MUST):** print `this round covers: <ids>` per session and
    assert the ids — unioned across every session this run files — equal the
-   Select step's list BEFORE calling qaping_playtest. A mismatch (e.g. the
-   20-step cap forcing cuts) fails loudly PRE-SPEND: batch another session or
-   state the deferral out loud — never silently truncate.
-6. **Wait:** `qaping_wait` until results arrive (see ground rules).
+   Select step's list BEFORE calling qaping_playtest. The wire keeps only
+   the FIRST 20 steps_result rows and a play directive is prepended, so with
+   `questionnaire:'none'` budget at most 19 authored steps per session (8 if
+   the standard questionnaire rides) — rows past 20 are answered by the
+   tester and silently dropped at finish, which is why this gate exists. A
+   mismatch (the budget forcing cuts) fails loudly PRE-SPEND: batch another
+   session or state the deferral out loud — never silently truncate.
+6. **Wait, in the background:** spawn a background task looping `qaping_wait`
+   and keep working (see ground rules) — foreground polling is the fallback,
+   and a promise to check later is never a wait.
 7. **Report** from `qaping_results`:
    - **Evidence line first (MUST).** Every report opens with its grade:
      `Evidence: recording + transcript` (macOS/iOS) / `Evidence: recording
@@ -205,7 +267,11 @@ checks testimony-only — mark that in the plan and report at that grade.
      findings mean fix and refile — unresolved never headlines as PASS.)
    - **One row per SELECTED check (MUST)**, keyed by id, outcome from
      {Pass, Fail, Blocked, NOT-ASKED}. "Couldn't test" and "Didn't reach this
-     part" are Blocked, with the why. A check whose precondition came back
+     part" are Blocked, with the why. A check that reached the wire as more
+     than one row (a precondition step, a VERIFY FIX pair) takes its outcome
+     from the WORST row: any Fail → Fail; else any Blocked or unanswered row
+     leaves the check unverified and the headline counts it ("PASS — N
+     unverified") — never plain Pass over a Blocked sub-step. A check whose precondition came back
      unverified or disproven is VOID — report it as Blocked ("VOID:
      precondition unverified"), never as a pass or fail, and leave
      `last_verified` untouched. A selected check that never made the wire or
@@ -255,18 +321,19 @@ the fluke); Fail → `failing`, and the issue upgrades to CONFIRMED. Never
 resolve an ambiguity by re-buying the whole batch, and never label a
 once-seen fail CONFIRMED without that round or hard evidence.
 
-## Differential rounds (web feel checks)
+## Paired rounds (two-state comparisons)
 
 A feel/physics/balance regression judged absolute buys little — a stranger
-answers "did spin feel right" in a vacuum. On WEB builds, where switching
-costs seconds, such a check MAY file as ONE session carrying BOTH builds:
-publish the previous build alongside the current one (the ledger's `build`
-field makes it recoverable), put the second URL in `instructions`, and author
-the steps as "play 2 minutes on build A, then build B — name anything that
-changed and where". est_minutes honestly covers both halves. Caveat the
-report: a tester told to hunt changes reports noise as change, and order
-effects apply. Store-delivered and uploaded native builds stay single-build —
-no second-URL surface exists for them.
+answers "did spin feel right" in a vacuum. But a comparison never rides one
+round: never two links in one round — links in step text are inert on the
+phone and `instructions` go unread in-review, so a second URL simply never
+gets opened. When a check needs TWO states compared (with/without a hook,
+old build vs new), file TWO paired rounds with the SAME steps, one per
+state, each state forced by its own round `url` (the ledger's `build` field
+keeps the old build recoverable), and compare the answers yourself in the
+report. Caveat it: different testers played each side, and a tester told to
+hunt changes reports noise as change. Native builds pair the same way, one
+build per round — re-publish the previous zip if its /b/<slug> expired.
 
 ## The experience round (opt-in, never a default)