@qaping/cli 0.1.4 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bin/qaping +563 -6
- package/docs/QA-PLAN-FORMAT.md +18 -0
- package/package.json +3 -2
- package/skill/qaping/SKILL.md +83 -18
package/bin/qaping
CHANGED
|
@@ -11,7 +11,10 @@
|
|
|
11
11
|
// qaping setup [claude-code|cursor|codex] [--force] one-command onboarding
|
|
12
12
|
// qaping remove|uninstall [--client <c>] sweep ONLY qaping's state
|
|
13
13
|
// qaping publish-build <game.zip> --platform windows|macos
|
|
14
|
+
// qaping builds [rm <slug>] what is hosted, and free a slot
|
|
14
15
|
// qaping wait <ping_id> [--timeout <seconds>]
|
|
16
|
+
// qaping results <ping_id> one passive fetch, no blocking
|
|
17
|
+
// qaping rounds [add|rm] the parked rounds this repo is owed
|
|
15
18
|
// qaping whoami | version | help
|
|
16
19
|
"use strict";
|
|
17
20
|
const fs = require("fs");
|
|
@@ -22,6 +25,11 @@ const { spawnSync } = require("child_process");
|
|
|
22
25
|
|
|
23
26
|
const QPKG = path.resolve(__dirname, "..");
|
|
24
27
|
|
|
28
|
+
// The name npm actually serves. The bare `qaping` is refused by the registry's
|
|
29
|
+
// similarity rule (petition open, QAPING_PLAN.md §8), so every global install —
|
|
30
|
+
// the wrapper's own and the one the kit's setup performs — must use this.
|
|
31
|
+
const NPM_PACKAGE = "@qaping/cli";
|
|
32
|
+
|
|
25
33
|
// The kit is a normal npm dependency when qaping is installed, and a workspace
|
|
26
34
|
// sibling in the monorepo checkout (where node_modules may not be linked yet)
|
|
27
35
|
// — resolve the package, never a hardcoded install path.
|
|
@@ -42,8 +50,10 @@ function resolveKitDir() {
|
|
|
42
50
|
// writes it to ~/.claude/rules/qaping.md and ~/.cursor/rules/qaping.mdc.
|
|
43
51
|
const RULE_BODY = `This machine has qaping: automatic QA for games — real human playtesters play the developer's builds on their own hardware and report back, and the coding agent runs the whole loop.
|
|
44
52
|
Whenever the user asks to set up QA for a game, run QA on a patch, check whether a change broke anything, or playtest a game or build, load the qaping skill and follow it — it owns authoring/maintaining QA-PLAN.md in the game repo and the per-patch run.
|
|
45
|
-
Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead).
|
|
46
|
-
The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a
|
|
53
|
+
Builds ship with \`qaping publish-build <game.zip> --platform windows|macos\`; the printed /b/<slug> URL is what a round is filed against (store-delivered games file with their Steam/TestFlight/App Store URL instead). Hosted builds are temporary and capped per account — if a publish is refused for that cap, \`qaping builds\` lists them and \`qaping builds rm <slug>\` frees a slot; free one and carry on rather than stopping to ask, taking the ones marked as never-finished uploads first and never deleting a build a round you filed is still running against.
|
|
54
|
+
The MCP tools are qaping_playtest (file a round with real human playtesters), qaping_results (fetch a round, free) and qaping_wait (continue a pending round).
|
|
55
|
+
Decide at filing time how the round ends: a short round waits in place — the moment filing returns pending, spawn a background task that loops qaping_wait and keep working (foreground polling is the fallback); a long or overnight round is PARKED — file it with an explicit long deadline_seconds, record it immediately with \`qaping rounds add <ping_id>\` (it writes qa-open-rounds.json beside QA-PLAN.md), tell the user exactly what is pending, and end the turn; a later session collects it (\`qaping rounds\` lists them with live status, qaping_results fetches one).
|
|
56
|
+
A recorded parked round is an honest pending end — never report pending as the answer without one, and never end with an unbacked promise to check later.
|
|
47
57
|
Playtests are duration-billed at 2 credits per minute of play per playtester — state the estimated cost before filing. Windows rounds return a recording and no transcript; web rounds return answers only (no recording, no transcript) — never promise evidence a platform does not ship.
|
|
48
58
|
QA memory: whenever the developer mentions — in ANY conversation, not just QA runs — something that should always be tested, a fragile area, or a QA lesson learned, offer to record it as a check (or a note on an existing check) in QA-PLAN.md so it is tested from then on.
|
|
49
59
|
`;
|
|
@@ -55,7 +65,7 @@ function qapingWrapper() {
|
|
|
55
65
|
brand: "qaping",
|
|
56
66
|
// npm registry name ≠ bin name: the similarity rule blocks bare "qaping"
|
|
57
67
|
// (petition open, QAPING_PLAN §8), so the global install pulls @qaping/cli.
|
|
58
|
-
installPackage:
|
|
68
|
+
installPackage: NPM_PACKAGE,
|
|
59
69
|
appUrl: process.env.QAPING_APP_URL || undefined,
|
|
60
70
|
mcpPath: "/api/mcp/qaping",
|
|
61
71
|
serverKey: "qaping",
|
|
@@ -122,6 +132,32 @@ function defaultIO(kitDir) {
|
|
|
122
132
|
|
|
123
133
|
const VALID_CLIENTS = ["claude-desktop", "claude-code", "cursor", "codex"];
|
|
124
134
|
|
|
135
|
+
// The kit's setup step 2 installs the global bin as `npm i -g <brand>` unless
|
|
136
|
+
// it is handed an installPackage — and a PUBLISHED kit older than that option
|
|
137
|
+
// ignores it, so it runs `npm i -g qaping`, a name npm's similarity rule
|
|
138
|
+
// refuses to host (404). Hit live on a fresh machine: setup configured every
|
|
139
|
+
// MCP client and reused the login, then reported "Setup incomplete" over that
|
|
140
|
+
// one 404.
|
|
141
|
+
//
|
|
142
|
+
// So the wrapper installs its OWN bin first, under the name npm serves. Once
|
|
143
|
+
// `qaping` is on PATH the kit's step finds it and logs "already installed
|
|
144
|
+
// globally", whatever kit version is underneath. npx runs from a cache dir
|
|
145
|
+
// that is NOT a persistent install — exactly the case that step exists for —
|
|
146
|
+
// so a cache/tmp path does not count as already installed.
|
|
147
|
+
function ensureGlobalBin(io) {
|
|
148
|
+
if (isSourceCheckout()) return;
|
|
149
|
+
const existing = io.which("qaping");
|
|
150
|
+
const persistent =
|
|
151
|
+
existing && !/[\\/](_npx|_cacache|npm-cache|Temp|tmp)[\\/]/i.test(existing);
|
|
152
|
+
if (persistent) return;
|
|
153
|
+
io.log(`Installing the qaping command (npm i -g ${NPM_PACKAGE})…`);
|
|
154
|
+
const r = io.run("npm", ["i", "-g", NPM_PACKAGE]);
|
|
155
|
+
if (!r || r.error || r.status !== 0) {
|
|
156
|
+
io.log(`⚠ could not install the qaping command — run it yourself: npm i -g ${NPM_PACKAGE}`);
|
|
157
|
+
io.log(" (setup continues; the MCP tools your agent calls do not need it, but `qaping publish-build` does)");
|
|
158
|
+
}
|
|
159
|
+
}
|
|
160
|
+
|
|
125
161
|
async function cmdSetup(argv) {
|
|
126
162
|
const kitDir = resolveKitDir();
|
|
127
163
|
const { setup } = require(path.join(kitDir, "harness", "setup.js"));
|
|
@@ -129,7 +165,9 @@ async function cmdSetup(argv) {
|
|
|
129
165
|
// accepts both the positional form (`setup cursor`) and `setup --client cursor`
|
|
130
166
|
const args = argv.filter((a) => a !== "--force");
|
|
131
167
|
const client = ((args[1] === "--client" ? args[2] : args[1]) || "").toLowerCase();
|
|
132
|
-
const
|
|
168
|
+
const io = defaultIO(kitDir);
|
|
169
|
+
ensureGlobalBin(io);
|
|
170
|
+
const r = await setup(io, {
|
|
133
171
|
home: os.homedir(),
|
|
134
172
|
sourceCheckout: isSourceCheckout(),
|
|
135
173
|
resolveToken,
|
|
@@ -184,6 +222,15 @@ function cmdPublishBuild(argv) {
|
|
|
184
222
|
.main(argv.slice(1), { brandCommand: "qaping publish-build", nextStepToolName: "qaping_playtest" });
|
|
185
223
|
}
|
|
186
224
|
|
|
225
|
+
// The management half of publish-build: an account holds a small number of
|
|
226
|
+
// hosted builds at once, and until this existed a QA loop that hit that cap
|
|
227
|
+
// could not even name the builds holding the slots — it stopped and asked a
|
|
228
|
+
// human. Same brand seam: every remedy this prints says `qaping`.
|
|
229
|
+
function cmdBuilds(argv) {
|
|
230
|
+
return require(path.join(resolveKitDir(), "harness", "builds.js"))
|
|
231
|
+
.main(argv.slice(1), { brandCommand: "qaping builds" });
|
|
232
|
+
}
|
|
233
|
+
|
|
187
234
|
function cmdPublish(argv) {
|
|
188
235
|
// WEB games: their "build" is a static directory, hosted (not zipped) — the
|
|
189
236
|
// kit's hosted-draft publisher, brand-seamed like publish-build above. The
|
|
@@ -192,14 +239,508 @@ function cmdPublish(argv) {
|
|
|
192
239
|
.main(argv.slice(1), { brandCommand: "qaping publish" });
|
|
193
240
|
}
|
|
194
241
|
|
|
242
|
+
// ─── parked rounds: the state a LATER session collects from ─────────────────
|
|
243
|
+
//
|
|
244
|
+
// A playtest legitimately runs for hours, so an agent that files one has two
|
|
245
|
+
// honest endings: wait in place (the background qaping_wait chain), or PARK —
|
|
246
|
+
// file with an explicit long deadline, record the round, end the turn. This
|
|
247
|
+
// file is that record: OPEN state, mutable, one entry per round still owed,
|
|
248
|
+
// sitting beside QA-PLAN.md at the game repo root. It is NOT qa-rounds.jsonl —
|
|
249
|
+
// that ledger is append-only history written AFTER results land, and the two
|
|
250
|
+
// are never merged. Signed URLs never go in here; they expire in days and the
|
|
251
|
+
// record has to outlive the session that wrote it.
|
|
252
|
+
const ROUNDS_FILE = "qa-open-rounds.json";
|
|
253
|
+
// A repo owed more open rounds than this is a runaway loop, not a QA plan. The
|
|
254
|
+
// listing spends one passive fetch per round, so this is also the ceiling on
|
|
255
|
+
// what `qaping rounds` costs in requests.
|
|
256
|
+
const MAX_OPEN_ROUNDS = 20;
|
|
257
|
+
const PING_ID_RE = /^[0-9a-f-]{36}$/i;
|
|
258
|
+
const DEFAULT_APP_URL = "https://pingfusi.com";
|
|
259
|
+
|
|
260
|
+
const ROUNDS_USAGE = `usage:
|
|
261
|
+
qaping rounds [--json] [--offline] [--file <path>]
|
|
262
|
+
qaping rounds add <ping_id> [--platform windows|macos|ios] [--build <url>]
|
|
263
|
+
[--minutes N] [--deadline-seconds N]
|
|
264
|
+
[--checks a,b,c] [--note "<patch>"] [--json]
|
|
265
|
+
qaping rounds rm <ping_id> [--json]`;
|
|
266
|
+
|
|
267
|
+
function flagValue(argv, name) {
|
|
268
|
+
const i = argv.indexOf(name);
|
|
269
|
+
return i >= 0 ? argv[i + 1] : undefined;
|
|
270
|
+
}
|
|
271
|
+
|
|
272
|
+
function intFlag(argv, name) {
|
|
273
|
+
const raw = flagValue(argv, name);
|
|
274
|
+
if (raw === undefined) return null;
|
|
275
|
+
const n = Number.parseInt(raw, 10);
|
|
276
|
+
return Number.isFinite(n) && n > 0 ? n : null;
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
function roundsFilePath(argv) {
|
|
280
|
+
const override = flagValue(argv, "--file");
|
|
281
|
+
return override ? path.resolve(override) : path.join(process.cwd(), ROUNDS_FILE);
|
|
282
|
+
}
|
|
283
|
+
|
|
284
|
+
// Tolerant on read, strict on shape: an entry without a ping_id is not a round
|
|
285
|
+
// anyone can collect, so it is dropped rather than rendered as a mystery.
|
|
286
|
+
//
|
|
287
|
+
// A bare top-level ARRAY is normalized, not refused: the skill and the mount
|
|
288
|
+
// prose both sanction hand-writing this file, and a list of records is the
|
|
289
|
+
// likeliest thing a hand writes. ANY OTHER shape is a loud failure — reading an
|
|
290
|
+
// unrecognized document as "no rounds" makes the next `rounds add` overwrite
|
|
291
|
+
// records nobody ever saw, which is data loss dressed as a fresh start.
|
|
292
|
+
function parseRounds(raw) {
|
|
293
|
+
let data;
|
|
294
|
+
try { data = JSON.parse(raw); }
|
|
295
|
+
catch { throw new Error(`${ROUNDS_FILE} is not valid JSON — fix or delete it`); }
|
|
296
|
+
const rounds = Array.isArray(data) ? data
|
|
297
|
+
: (data && Array.isArray(data.rounds) ? data.rounds : null);
|
|
298
|
+
if (!rounds) {
|
|
299
|
+
throw new Error(`${ROUNDS_FILE} is valid JSON but not a rounds file — expected {"rounds": [ … ]} (a bare array of records is also read). Refusing to overwrite it; fix or delete it.`);
|
|
300
|
+
}
|
|
301
|
+
return { rounds: rounds.filter((r) => r && typeof r.ping_id === "string") };
|
|
302
|
+
}
|
|
303
|
+
|
|
304
|
+
/** null when the file does not exist — "no rounds recorded here" is not an error. */
|
|
305
|
+
function readRoundsFile(file) {
|
|
306
|
+
let raw;
|
|
307
|
+
try { raw = fs.readFileSync(file, "utf8"); }
|
|
308
|
+
catch { return null; }
|
|
309
|
+
return parseRounds(raw);
|
|
310
|
+
}
|
|
311
|
+
|
|
312
|
+
// Write via tmp-then-rename: rename(2) is atomic on POSIX, so a reader (or a
|
|
313
|
+
// crash) sees the whole old file or the whole new one, never a truncated one.
|
|
314
|
+
function writeRoundsFile(file, state) {
|
|
315
|
+
const tmp = `${file}.${process.pid}.tmp`;
|
|
316
|
+
fs.writeFileSync(tmp, JSON.stringify({ rounds: state.rounds }, null, 2) + "\n");
|
|
317
|
+
try { fs.renameSync(tmp, file); }
|
|
318
|
+
catch (e) { try { fs.unlinkSync(tmp); } catch { /* best effort */ } throw e; }
|
|
319
|
+
}
|
|
320
|
+
|
|
321
|
+
// ── the write lock ──────────────────────────────────────────────────────────
|
|
322
|
+
//
|
|
323
|
+
// Every mutation is a read-modify-write, and the two callers that run
|
|
324
|
+
// unattended — an agent recording a fresh round while a background poller
|
|
325
|
+
// removes a collected one — race. Unlocked, the loser's record is simply gone.
|
|
326
|
+
// A lockfile (O_EXCL create, the one atomic primitive every filesystem gives)
|
|
327
|
+
// serializes them; dependency-free, because a QA repo must not grow one.
|
|
328
|
+
const LOCK_RETRIES = 10;
|
|
329
|
+
const LOCK_RETRY_MS = 50;
|
|
330
|
+
const LOCK_STALE_MS = 5000;
|
|
331
|
+
|
|
332
|
+
/** Blocking sleep with no dependency and no busy-spin (allowed on Node's main thread). */
|
|
333
|
+
function sleepSync(ms) {
|
|
334
|
+
Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
|
|
335
|
+
}
|
|
336
|
+
|
|
337
|
+
function acquireRoundsLock(file) {
|
|
338
|
+
const lock = `${file}.lock`;
|
|
339
|
+
for (let attempt = 0; attempt < LOCK_RETRIES; attempt++) {
|
|
340
|
+
try { return fs.openSync(lock, "wx"); }
|
|
341
|
+
catch (e) {
|
|
342
|
+
if (!e || e.code !== "EEXIST") throw e;
|
|
343
|
+
// A killed writer must not wedge the repo forever — an old lock is stolen,
|
|
344
|
+
// loudly, because a stolen lock is also how a real concurrent write is lost.
|
|
345
|
+
let age = 0;
|
|
346
|
+
try { age = Date.now() - fs.statSync(lock).mtimeMs; } catch { age = 0; }
|
|
347
|
+
if (age > LOCK_STALE_MS) {
|
|
348
|
+
console.error(`⚠ stale lock on ${ROUNDS_FILE} (${Math.round(age / 1000)}s old) — taking it over`);
|
|
349
|
+
try { fs.unlinkSync(lock); } catch { /* someone else got there first */ }
|
|
350
|
+
continue;
|
|
351
|
+
}
|
|
352
|
+
sleepSync(LOCK_RETRY_MS);
|
|
353
|
+
}
|
|
354
|
+
}
|
|
355
|
+
throw new Error(`${ROUNDS_FILE} is locked by another qaping process — nothing was written; try again in a moment`);
|
|
356
|
+
}
|
|
357
|
+
|
|
358
|
+
function releaseRoundsLock(file, fd) {
|
|
359
|
+
try { fs.closeSync(fd); } catch { /* best effort */ }
|
|
360
|
+
try { fs.unlinkSync(`${file}.lock`); } catch { /* best effort */ }
|
|
361
|
+
}
|
|
362
|
+
|
|
363
|
+
/**
|
|
364
|
+
* The ONE read-modify-write path. `mutate(state)` returns `{ state, …extras }`;
|
|
365
|
+
* a null/absent `state` means "decided not to write" (and never truncates).
|
|
366
|
+
* Both the read and the write happen inside the lock — the point of the lock.
|
|
367
|
+
*/
|
|
368
|
+
function mutateRoundsFile(file, mutate) {
|
|
369
|
+
const fd = acquireRoundsLock(file);
|
|
370
|
+
try {
|
|
371
|
+
const out = mutate(readRoundsFile(file) || { rounds: [] }) || {};
|
|
372
|
+
if (out.state) writeRoundsFile(file, out.state);
|
|
373
|
+
return out;
|
|
374
|
+
} finally {
|
|
375
|
+
releaseRoundsLock(file, fd);
|
|
376
|
+
}
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
function upsertRound(state, record) {
|
|
380
|
+
const rounds = state.rounds.slice();
|
|
381
|
+
const i = rounds.findIndex((r) => r.ping_id === record.ping_id);
|
|
382
|
+
if (i >= 0) rounds[i] = record; else rounds.push(record);
|
|
383
|
+
return { rounds };
|
|
384
|
+
}
|
|
385
|
+
|
|
386
|
+
function removeRound(state, pingId) {
|
|
387
|
+
const rounds = state.rounds.filter((r) => r.ping_id !== pingId);
|
|
388
|
+
return { state: { rounds }, removed: rounds.length !== state.rounds.length };
|
|
389
|
+
}
|
|
390
|
+
|
|
391
|
+
// How long the round stays offered to NEW players: the explicit deadline the
|
|
392
|
+
// filing set, counted from when it was filed. Absent either fact there is
|
|
393
|
+
// nothing honest to print, so nothing is printed.
|
|
394
|
+
function visibleUntil(record) {
|
|
395
|
+
const filed = Date.parse(record.filed_at || "");
|
|
396
|
+
if (!Number.isFinite(filed) || !record.deadline_seconds) return null;
|
|
397
|
+
return new Date(filed + record.deadline_seconds * 1000).toISOString();
|
|
398
|
+
}
|
|
399
|
+
|
|
400
|
+
function roundStatusLine(record, live) {
|
|
401
|
+
const status = live.status || "pending";
|
|
402
|
+
if (status === "complete") return `READY — collect now: qaping results ${record.ping_id}`;
|
|
403
|
+
if (status === "pending") {
|
|
404
|
+
const seen = `${live.n_received ?? 0}/${live.n_target ?? "?"}`;
|
|
405
|
+
const until = visibleUntil(record);
|
|
406
|
+
return `still pending (${seen})${until ? ` — visible until ${until}` : ""}; keep waiting: qaping wait ${record.ping_id}`;
|
|
407
|
+
}
|
|
408
|
+
// One round's own failure, not the run's: the rest of the listing is real.
|
|
409
|
+
if (status === "unavailable") {
|
|
410
|
+
return `status unavailable (${live.error || "the service refused this id"}) — the other rounds are unaffected; retry: qaping results ${record.ping_id}`;
|
|
411
|
+
}
|
|
412
|
+
// Results are asker-scoped server-side, so a round filed under a DIFFERENT
|
|
413
|
+
// login reads exactly like one that never existed. Never prescribe deleting
|
|
414
|
+
// the only record of a round on that evidence.
|
|
415
|
+
if (status === "not_found") {
|
|
416
|
+
return `not visible from this login (wrong account?) or expired — verify before \`qaping rounds rm ${record.ping_id}\` (\`qaping whoami\` shows which login this is)`;
|
|
417
|
+
}
|
|
418
|
+
return `expired or unreadable (${status}) — report it, then: qaping rounds rm ${record.ping_id}`;
|
|
419
|
+
}
|
|
420
|
+
|
|
421
|
+
/**
|
|
422
|
+
* Pure render of the listing. entries: [{ record, live }], live null when unfetched.
|
|
423
|
+
* opts.total is the TRUE recorded count — entries may be a capped slice of it, and
|
|
424
|
+
* a count line that reports the slice as the total is a lie a later session acts on.
|
|
425
|
+
*/
|
|
426
|
+
function renderRounds(entries, opts) {
|
|
427
|
+
const o = opts || {};
|
|
428
|
+
const total = Number.isFinite(o.total) ? o.total : entries.length;
|
|
429
|
+
const notice = o.notice || (o.offline ? "(live status unavailable — offline)" : null);
|
|
430
|
+
const lines = [total > entries.length
|
|
431
|
+
? `${total} open round(s) recorded in ${ROUNDS_FILE} — showing live status for first ${entries.length} of ${total}:`
|
|
432
|
+
: `${total} open round(s) recorded in ${ROUNDS_FILE}:`];
|
|
433
|
+
if (notice) lines.push(notice);
|
|
434
|
+
for (const { record, live } of entries) {
|
|
435
|
+
const facts = [
|
|
436
|
+
record.platform || null,
|
|
437
|
+
record.est_minutes ? `${record.est_minutes} min` : null,
|
|
438
|
+
record.filed_at ? `filed ${record.filed_at}` : null,
|
|
439
|
+
].filter(Boolean).join(" · ");
|
|
440
|
+
lines.push("");
|
|
441
|
+
lines.push(` ${record.ping_id}${facts ? ` — ${facts}` : ""}`);
|
|
442
|
+
if (record.build) lines.push(` build: ${record.build}`);
|
|
443
|
+
if (Array.isArray(record.checks) && record.checks.length) lines.push(` checks: ${record.checks.join(", ")}`);
|
|
444
|
+
if (record.note) lines.push(` patch: ${record.note}`);
|
|
445
|
+
if (live) lines.push(` ${roundStatusLine(record, live)}`);
|
|
446
|
+
}
|
|
447
|
+
return lines.join("\n");
|
|
448
|
+
}
|
|
449
|
+
|
|
450
|
+
// The app base these passive fetches use. `qaping wait` rides the vendored
|
|
451
|
+
// installer, which honors two kit-side env names besides this brand's own — so
|
|
452
|
+
// resolving a different base here would make two verbs disagree about which
|
|
453
|
+
// service a staging machine talks to. Same order as the vendor's. The second
|
|
454
|
+
// name is spelled from parts on purpose: it is a wire-only env name, and the
|
|
455
|
+
// reviewer-side brand never appears literally on a dev-facing surface (this
|
|
456
|
+
// package's own leak scan is what enforces that).
|
|
457
|
+
const LEGACY_APP_URL_ENVS = [["PING", "HUMANS", "_APP_URL"].join(""), "PINGFUSI_APP_URL"];
|
|
458
|
+
|
|
459
|
+
function resolveAppUrl() {
|
|
460
|
+
const fromEnv = LEGACY_APP_URL_ENVS.map((name) => process.env[name]).find(Boolean);
|
|
461
|
+
return (qapingWrapper().appUrl || fromEnv || DEFAULT_APP_URL).replace(/\/+$/, "");
|
|
462
|
+
}
|
|
463
|
+
|
|
464
|
+
// wire.js's resolveToken reads ~/.claude.json under the WIRE CONTRACT's MCP
|
|
465
|
+
// server keys — a list that predates this brand and does not contain `qaping`.
|
|
466
|
+
// A machine whose only login is the qaping MCP entry (written by the client's
|
|
467
|
+
// own config, with no credentials stash) therefore reads as logged out. This
|
|
468
|
+
// mirrors the vendored installer's resolveLocalToken minimally, bin-local, so
|
|
469
|
+
// the kit stays untouched: the qaping key first, then the contract's own order
|
|
470
|
+
// — which supplies the legacy names without spelling a retired brand here.
|
|
471
|
+
function resolveTokenFromClientConfigs(kitDir) {
|
|
472
|
+
const keys = ["qaping"];
|
|
473
|
+
try {
|
|
474
|
+
const contract = require(path.join(kitDir, "packages", "core", "wire-contract.gen.js"));
|
|
475
|
+
for (const k of contract.MCP_SERVER_KEYS || []) if (!keys.includes(k)) keys.push(k);
|
|
476
|
+
} catch { keys.push("pingfusi"); }
|
|
477
|
+
const home = os.homedir();
|
|
478
|
+
const desktop = process.platform === "darwin"
|
|
479
|
+
? path.join(home, "Library", "Application Support", "Claude", "claude_desktop_config.json")
|
|
480
|
+
: process.platform === "win32"
|
|
481
|
+
? path.join(process.env.APPDATA || path.join(home, "AppData", "Roaming"), "Claude", "claude_desktop_config.json")
|
|
482
|
+
: path.join(home, ".config", "Claude", "claude_desktop_config.json");
|
|
483
|
+
for (const p of [path.join(home, ".claude.json"), desktop, path.join(home, ".cursor", "mcp.json")]) {
|
|
484
|
+
try {
|
|
485
|
+
const servers = (JSON.parse(fs.readFileSync(p, "utf8")) || {}).mcpServers || {};
|
|
486
|
+
for (const key of keys) {
|
|
487
|
+
const headers = (servers[key] || {}).headers || {};
|
|
488
|
+
const m = /Bearer\s+(\S+)/.exec(headers.Authorization || headers.authorization || "");
|
|
489
|
+
if (m) return m[1];
|
|
490
|
+
}
|
|
491
|
+
} catch { /* absent or unreadable config: just not a login source */ }
|
|
492
|
+
}
|
|
493
|
+
return null;
|
|
494
|
+
}
|
|
495
|
+
|
|
496
|
+
// One passive tools/call against the qaping mount — the same transport and the
|
|
497
|
+
// same JSON-or-SSE parse the kit's `wait` leg uses, with the mount read from the
|
|
498
|
+
// ONE wrapper object every other command is driven by. Passive: it never renews
|
|
499
|
+
// the round's lease, which is exactly why a parked round needs its own explicit
|
|
500
|
+
// deadline. Failures are TYPED for the caller: `authRejected` and `perRound`
|
|
501
|
+
// separate "this login is dead" from "the service refused this one id".
|
|
502
|
+
async function callResultsTool(pingId) {
|
|
503
|
+
const w = qapingWrapper();
|
|
504
|
+
const kitDir = resolveKitDir();
|
|
505
|
+
const { resolveToken } = require(path.join(kitDir, "packages", "core", "wire.js"));
|
|
506
|
+
const token = resolveToken() || resolveTokenFromClientConfigs(kitDir);
|
|
507
|
+
if (!token) throw new Error("no qaping login on this machine — run `qaping setup` first");
|
|
508
|
+
let res;
|
|
509
|
+
try {
|
|
510
|
+
res = await fetch(`${resolveAppUrl()}${w.mcpPath}`, {
|
|
511
|
+
method: "POST",
|
|
512
|
+
headers: {
|
|
513
|
+
"content-type": "application/json",
|
|
514
|
+
accept: "application/json, text/event-stream",
|
|
515
|
+
authorization: `Bearer ${token}`,
|
|
516
|
+
},
|
|
517
|
+
body: JSON.stringify({
|
|
518
|
+
jsonrpc: "2.0",
|
|
519
|
+
id: 1,
|
|
520
|
+
method: "tools/call",
|
|
521
|
+
params: { name: "qaping_results", arguments: { ping_id: pingId } },
|
|
522
|
+
}),
|
|
523
|
+
signal: AbortSignal.timeout(20_000),
|
|
524
|
+
});
|
|
525
|
+
} catch (e) {
|
|
526
|
+
throw new Error(`could not reach the qaping service (${(e && e.message) || e})`);
|
|
527
|
+
}
|
|
528
|
+
// A rejected login is a fact about the RUN, not about the round: the body is
|
|
529
|
+
// plain text, so without this it parses as garbage and reads as "offline",
|
|
530
|
+
// sending the developer to debug their network instead of their login.
|
|
531
|
+
if (res.status === 401 || res.status === 403) {
|
|
532
|
+
const e = new Error(`qaping login rejected (HTTP ${res.status}) — run \`qaping setup\` to re-link this machine`);
|
|
533
|
+
e.authRejected = true;
|
|
534
|
+
throw e;
|
|
535
|
+
}
|
|
536
|
+
const raw = await res.text();
|
|
537
|
+
const m = raw.match(/data: (.*)/);
|
|
538
|
+
let payload;
|
|
539
|
+
try { payload = JSON.parse(m ? m[1] : raw); }
|
|
540
|
+
catch { throw new Error(`unreadable response from the qaping service (HTTP ${res.status})`); }
|
|
541
|
+
if (payload.error) {
|
|
542
|
+
// The service answered — it just refused THIS id (a bad ping_id is -32602).
|
|
543
|
+
// Marked so a listing can degrade one row instead of the whole pass.
|
|
544
|
+
const e = new Error(payload.error.message || "MCP error");
|
|
545
|
+
e.perRound = true;
|
|
546
|
+
throw e;
|
|
547
|
+
}
|
|
548
|
+
const result = payload.result || {};
|
|
549
|
+
const structured = result.structuredContent || {};
|
|
550
|
+
return {
|
|
551
|
+
status: structured.status || "pending",
|
|
552
|
+
structured,
|
|
553
|
+
result,
|
|
554
|
+
text: (result.content && result.content[0] && result.content[0].text) || "",
|
|
555
|
+
};
|
|
556
|
+
}
|
|
557
|
+
|
|
558
|
+
async function cmdResults(argv) {
|
|
559
|
+
const json = argv.includes("--json");
|
|
560
|
+
// Flags in any position: `qaping results --json <id>` is the natural thing to
|
|
561
|
+
// type, and reading argv[1] positionally made it a usage error.
|
|
562
|
+
const pingId = argv.slice(1).find((a) => !a.startsWith("--"));
|
|
563
|
+
if (!pingId || !PING_ID_RE.test(pingId)) {
|
|
564
|
+
console.error("usage: qaping results <ping_id> [--json]");
|
|
565
|
+
// 1, never 2: exit 2 is THIS command's "still pending" signal (a background
|
|
566
|
+
// harness reads it and waits), so a typo must not answer "pending". Same
|
|
567
|
+
// code the vendored `wait` exits on its own usage error.
|
|
568
|
+
process.exit(1);
|
|
569
|
+
}
|
|
570
|
+
const r = await callResultsTool(pingId);
|
|
571
|
+
if (r.status === "not_found") {
|
|
572
|
+
// Results are asker-scoped server-side, so this is nearly always a
|
|
573
|
+
// wrong-account token rather than a wrong id — say which.
|
|
574
|
+
console.error(`✗ no round ${pingId} on this account — results are asker-scoped: \`qaping results\` only reads rounds filed with the same qaping login (\`qaping whoami\` shows which one this is).`);
|
|
575
|
+
process.exit(1);
|
|
576
|
+
}
|
|
577
|
+
const payload = Object.keys(r.structured).length ? r.structured : r.result;
|
|
578
|
+
console.log(json ? JSON.stringify(payload, null, 2) : (r.text || JSON.stringify(payload, null, 2)));
|
|
579
|
+
if (r.status === "pending") {
|
|
580
|
+
// stderr, so --json stdout stays a parseable document.
|
|
581
|
+
console.error(`still pending — \`qaping wait ${pingId}\` to block, or leave it recorded and check later.`);
|
|
582
|
+
process.exit(2);
|
|
583
|
+
}
|
|
584
|
+
process.exit(0);
|
|
585
|
+
}
|
|
586
|
+
|
|
587
|
+
async function cmdRoundsList(argv) {
|
|
588
|
+
const json = argv.includes("--json");
|
|
589
|
+
const state = readRoundsFile(roundsFilePath(argv));
|
|
590
|
+
if (!state || state.rounds.length === 0) {
|
|
591
|
+
console.log(json ? JSON.stringify({ rounds: [], total: 0, truncated: false, notice: null }, null, 2)
|
|
592
|
+
: `no open rounds recorded here (${ROUNDS_FILE})`);
|
|
593
|
+
process.exit(0);
|
|
594
|
+
}
|
|
595
|
+
// The cap bounds the FETCHES (one passive call each), not the truth: the
|
|
596
|
+
// count line and the JSON both report every recorded round, and say so when
|
|
597
|
+
// only the first N carry live status.
|
|
598
|
+
const total = state.rounds.length;
|
|
599
|
+
const rounds = state.rounds.slice(0, MAX_OPEN_ROUNDS);
|
|
600
|
+
const offlineFlag = argv.includes("--offline");
|
|
601
|
+
let notice = offlineFlag ? "(live status unavailable — offline)" : null;
|
|
602
|
+
let live = new Map();
|
|
603
|
+
if (!offlineFlag) {
|
|
604
|
+
for (const record of rounds) {
|
|
605
|
+
try {
|
|
606
|
+
const r = await callResultsTool(record.ping_id);
|
|
607
|
+
live.set(record.ping_id, {
|
|
608
|
+
status: r.status,
|
|
609
|
+
n_received: r.structured.n_received ?? null,
|
|
610
|
+
n_target: r.structured.n_target ?? null,
|
|
611
|
+
});
|
|
612
|
+
} catch (e) {
|
|
613
|
+
if (e && e.perRound) {
|
|
614
|
+
// The service answered and refused this id — that is this round's
|
|
615
|
+
// status, not the run's. The rest of the listing still gets fetched.
|
|
616
|
+
live.set(record.ping_id, { status: "unavailable", error: e.message });
|
|
617
|
+
continue;
|
|
618
|
+
}
|
|
619
|
+
// Login and transport fail for the whole pass; name WHICH, because the
|
|
620
|
+
// remedies are nothing alike.
|
|
621
|
+
notice = e && e.authRejected
|
|
622
|
+
? "(live status unavailable — login rejected: run `qaping setup`)"
|
|
623
|
+
: `(live status unavailable — ${(e && e.message) || "offline"})`;
|
|
624
|
+
live = new Map();
|
|
625
|
+
break;
|
|
626
|
+
}
|
|
627
|
+
}
|
|
628
|
+
}
|
|
629
|
+
const entries = rounds.map((record) => ({ record, live: live.get(record.ping_id) || null }));
|
|
630
|
+
console.log(json
|
|
631
|
+
? JSON.stringify({
|
|
632
|
+
rounds: entries.map((e) => ({ ...e.record, live: e.live })),
|
|
633
|
+
total,
|
|
634
|
+
truncated: total > rounds.length,
|
|
635
|
+
notice,
|
|
636
|
+
}, null, 2)
|
|
637
|
+
: renderRounds(entries, { total, notice }));
|
|
638
|
+
process.exit(0);
|
|
639
|
+
}
|
|
640
|
+
|
|
641
|
+
function cmdRoundsAdd(argv) {
|
|
642
|
+
const json = argv.includes("--json");
|
|
643
|
+
const pingId = argv[2];
|
|
644
|
+
if (!pingId || pingId.startsWith("--") || !PING_ID_RE.test(pingId)) {
|
|
645
|
+
console.error(`✗ not a ping id: ${pingId || "(missing)"}\n${ROUNDS_USAGE}`);
|
|
646
|
+
process.exit(2);
|
|
647
|
+
}
|
|
648
|
+
const platform = flagValue(argv, "--platform");
|
|
649
|
+
if (platform && !["windows", "macos", "ios"].includes(platform)) {
|
|
650
|
+
console.error(`✗ --platform must be windows, macos or ios — a web round has none, so omit it.\n${ROUNDS_USAGE}`);
|
|
651
|
+
process.exit(2);
|
|
652
|
+
}
|
|
653
|
+
const file = roundsFilePath(argv);
|
|
654
|
+
const minutes = intFlag(argv, "--minutes");
|
|
655
|
+
const deadline = intFlag(argv, "--deadline-seconds");
|
|
656
|
+
const checks = (flagValue(argv, "--checks") || "").split(",").map((s) => s.trim()).filter(Boolean);
|
|
657
|
+
const note = flagValue(argv, "--note");
|
|
658
|
+
const build = flagValue(argv, "--build");
|
|
659
|
+
const patch = {};
|
|
660
|
+
if (platform) patch.platform = platform;
|
|
661
|
+
if (build) patch.build = build;
|
|
662
|
+
if (minutes != null) patch.est_minutes = minutes;
|
|
663
|
+
if (deadline != null) patch.deadline_seconds = deadline;
|
|
664
|
+
if (checks.length) patch.checks = checks;
|
|
665
|
+
if (note) patch.note = note;
|
|
666
|
+
// Read and write inside ONE lock: a background poller's `rounds rm` landing
|
|
667
|
+
// between this read and this write used to drop whichever record lost.
|
|
668
|
+
const { record, count, updated } = mutateRoundsFile(file, (state) => {
|
|
669
|
+
const existing = state.rounds.find((r) => r.ping_id === pingId);
|
|
670
|
+
// Re-adding the same id CORRECTS a record, never duplicates it — and keeps
|
|
671
|
+
// the original filed_at, which is what the visibility window counts from.
|
|
672
|
+
const rec = {
|
|
673
|
+
...(existing || {}),
|
|
674
|
+
ping_id: pingId,
|
|
675
|
+
filed_at: (existing && existing.filed_at) || new Date().toISOString(),
|
|
676
|
+
...patch,
|
|
677
|
+
};
|
|
678
|
+
const next = upsertRound(state, rec);
|
|
679
|
+
return { state: next, record: rec, count: next.rounds.length, updated: !!existing };
|
|
680
|
+
});
|
|
681
|
+
if (json) {
|
|
682
|
+
console.log(JSON.stringify({ file, recorded: record, rounds: count }, null, 2));
|
|
683
|
+
process.exit(0);
|
|
684
|
+
}
|
|
685
|
+
console.log(`✓ recorded round ${pingId} in ${ROUNDS_FILE}${updated ? " (updated)" : ""}`);
|
|
686
|
+
console.log(` collect it later: \`qaping rounds\` lists what this repo is owed; \`qaping results ${pingId}\` fetches this one.`);
|
|
687
|
+
console.log(` ${ROUNDS_FILE} is open state, not history — add it to .gitignore.`);
|
|
688
|
+
if (count > MAX_OPEN_ROUNDS) {
|
|
689
|
+
console.log(` ⚠ ${count} rounds recorded — \`qaping rounds\` fetches live status for the first ${MAX_OPEN_ROUNDS}. Collect some.`);
|
|
690
|
+
}
|
|
691
|
+
process.exit(0);
|
|
692
|
+
}
|
|
693
|
+
|
|
694
|
+
function cmdRoundsRm(argv) {
|
|
695
|
+
const json = argv.includes("--json");
|
|
696
|
+
const pingId = argv[2];
|
|
697
|
+
if (!pingId || pingId.startsWith("--")) {
|
|
698
|
+
console.error(`✗ qaping rounds rm needs a ping id.\n${ROUNDS_USAGE}`);
|
|
699
|
+
process.exit(2);
|
|
700
|
+
}
|
|
701
|
+
const file = roundsFilePath(argv);
|
|
702
|
+
// Same lock as `add`: this is the verb a background poller runs while the
|
|
703
|
+
// agent that filed the next round is recording it.
|
|
704
|
+
const { removed, count } = mutateRoundsFile(file, (state) => {
|
|
705
|
+
const { state: next, removed: didRemove } = removeRound(state, pingId);
|
|
706
|
+
// No write at all when nothing matched — a no-op must not rewrite the file.
|
|
707
|
+
return { state: didRemove ? next : null, removed: didRemove, count: next.rounds.length };
|
|
708
|
+
});
|
|
709
|
+
if (!removed) {
|
|
710
|
+
console.error(`✗ no round ${pingId} recorded in ${ROUNDS_FILE}`);
|
|
711
|
+
process.exit(1);
|
|
712
|
+
}
|
|
713
|
+
console.log(json ? JSON.stringify({ removed: pingId, rounds: count }, null, 2)
|
|
714
|
+
: `✓ removed ${pingId} from ${ROUNDS_FILE} (${count} open round(s) left)`);
|
|
715
|
+
process.exit(0);
|
|
716
|
+
}
|
|
717
|
+
|
|
718
|
+
async function cmdRounds(argv) {
|
|
719
|
+
const sub = argv[1];
|
|
720
|
+
if (sub === "add") return cmdRoundsAdd(argv);
|
|
721
|
+
if (sub === "rm" || sub === "delete" || sub === "remove") return cmdRoundsRm(argv);
|
|
722
|
+
if (sub === "--help" || sub === "-h") { console.log(ROUNDS_USAGE); process.exit(0); }
|
|
723
|
+
if (sub && !sub.startsWith("--")) {
|
|
724
|
+
console.error(`✗ unknown: qaping rounds ${sub}\n${ROUNDS_USAGE}`);
|
|
725
|
+
process.exit(2);
|
|
726
|
+
}
|
|
727
|
+
return cmdRoundsList(argv);
|
|
728
|
+
}
|
|
729
|
+
|
|
195
730
|
const HELP = `qaping — automatic QA for your game: real human playtesters, driven by your coding agent
|
|
196
731
|
|
|
197
732
|
usage:
|
|
198
733
|
qaping setup [claude-code|cursor|codex] [--force]
|
|
199
734
|
qaping remove [--client <c>] remove qaping's MCP entries, skill and rule
|
|
200
735
|
qaping publish-build <game.zip> --platform windows|macos
|
|
736
|
+
qaping builds the hosted builds you hold (oldest first)
|
|
737
|
+
qaping builds rm <slug> delete one now, freeing its slot
|
|
201
738
|
qaping publish <built-dir> host a WEB game's built output (prints the URL to file with)
|
|
202
739
|
qaping wait <ping_id> [--timeout <seconds>]
|
|
740
|
+
qaping results <ping_id> one passive fetch — exit 0 news, 2 still pending, 1 otherwise
|
|
741
|
+
qaping rounds the parked rounds this repo is owed, with live status
|
|
742
|
+
qaping rounds add <ping_id> … record a parked round so a later session collects it
|
|
743
|
+
qaping rounds rm <ping_id> forget one, once it is collected or expired
|
|
203
744
|
qaping whoami
|
|
204
745
|
qaping version
|
|
205
746
|
|
|
@@ -212,23 +753,36 @@ function route(cmd) {
|
|
|
212
753
|
if (cmd === "remove" || cmd === "uninstall") return "remove";
|
|
213
754
|
if (cmd === "wait" || cmd === "whoami") return "vendor";
|
|
214
755
|
if (cmd === "publish-build") return "publish-build";
|
|
756
|
+
if (cmd === "builds") return "builds";
|
|
215
757
|
if (cmd === "publish") return "publish";
|
|
758
|
+
if (cmd === "rounds") return "rounds";
|
|
759
|
+
if (cmd === "results") return "results";
|
|
216
760
|
return "unknown";
|
|
217
761
|
}
|
|
218
762
|
|
|
763
|
+
// The async commands' rejection handler: a clean one-line message, never a raw
|
|
764
|
+
// stack — same contract as the sync catch in main() below.
|
|
765
|
+
function die(e) {
|
|
766
|
+
console.error(`✗ ${(e && e.message) || e}`);
|
|
767
|
+
process.exit(1);
|
|
768
|
+
}
|
|
769
|
+
|
|
219
770
|
function main() {
|
|
220
771
|
const argv = process.argv.slice(2);
|
|
221
772
|
try {
|
|
222
773
|
switch (route(argv[0])) {
|
|
223
774
|
case "version": return console.log(require("../package.json").version);
|
|
224
775
|
case "help": return console.log(HELP);
|
|
225
|
-
case "setup": return void cmdSetup(argv).catch(
|
|
776
|
+
case "setup": return void cmdSetup(argv).catch(die);
|
|
226
777
|
case "remove": return cmdRemove(argv);
|
|
227
778
|
// `wait` also names the wait tool the qaping mount registers — the
|
|
228
779
|
// vendored default (the stock mount's wait) is not on /api/mcp/qaping.
|
|
229
780
|
case "vendor": return process.exit(spawnVendor(argv[0] === "wait" ? [...argv, "--wait-tool", "qaping_wait"] : argv));
|
|
230
781
|
case "publish-build": return void cmdPublishBuild(argv);
|
|
782
|
+
case "builds": return void cmdBuilds(argv);
|
|
231
783
|
case "publish": return void cmdPublish(argv);
|
|
784
|
+
case "rounds": return void cmdRounds(argv).catch(die);
|
|
785
|
+
case "results": return void cmdResults(argv).catch(die);
|
|
232
786
|
default:
|
|
233
787
|
console.error(HELP);
|
|
234
788
|
process.exit(1);
|
|
@@ -242,4 +796,7 @@ function main() {
|
|
|
242
796
|
}
|
|
243
797
|
|
|
244
798
|
if (require.main === module) main();
|
|
245
|
-
module.exports = {
|
|
799
|
+
module.exports = {
|
|
800
|
+
route, resolveKitDir, qapingWrapper, vendorFlags, isSourceCheckout, RULE_BODY, HELP,
|
|
801
|
+
ROUNDS_FILE, MAX_OPEN_ROUNDS, parseRounds, upsertRound, removeRound, visibleUntil, renderRounds,
|
|
802
|
+
};
|
package/docs/QA-PLAN-FORMAT.md
CHANGED
|
@@ -109,6 +109,24 @@ per round; the plan's `last_verified` and `state` are derived from it and
|
|
|
109
109
|
stay the dev-readable contract. Open `follow_ups` are what the next filing
|
|
110
110
|
must carry forward.
|
|
111
111
|
|
|
112
|
+
## Rounds still open — qa-open-rounds.json
|
|
113
|
+
|
|
114
|
+
A playtest may run for many hours, so a round the agent cannot sit through is
|
|
115
|
+
PARKED: filed with an explicit long deadline, then recorded in
|
|
116
|
+
`qa-open-rounds.json` (same directory) so a LATER session collects it —
|
|
117
|
+
|
|
118
|
+
```json
|
|
119
|
+
{"rounds": [{"ping_id": "<uuid>", "filed_at": "<ISO>", "platform": "windows",
|
|
120
|
+
"build": "<the hosted or store URL>", "est_minutes": 10,
|
|
121
|
+
"deadline_seconds": 86400, "checks": ["<check ids>"],
|
|
122
|
+
"note": "<the patch this round covers>"}]}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
`qaping rounds` lists it with each round's live status, `qaping rounds add`
|
|
126
|
+
records one, `qaping rounds rm` drops it once collected. This file is the
|
|
127
|
+
opposite of the ledger and never merges with it: mutable OPEN state, one entry
|
|
128
|
+
per round still owed, emptied as rounds land. It is not history — gitignore it.
|
|
129
|
+
|
|
112
130
|
## Maintenance rules
|
|
113
131
|
|
|
114
132
|
- Every run updates the run checks' `state` and `last_verified` from typed
|
package/package.json
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qaping/cli",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
|
+
"homepage": "https://qaping.dev",
|
|
4
5
|
"description": "qaping: automatic QA for your game. Your coding agent authors and maintains the QA plan, ships each patch's build to real human playtesters, and reports what broke.",
|
|
5
6
|
"keywords": [
|
|
6
7
|
"qaping",
|
|
@@ -24,7 +25,7 @@
|
|
|
24
25
|
"node": "^20.17.0 || ^22.13.0 || >=23.5.0"
|
|
25
26
|
},
|
|
26
27
|
"dependencies": {
|
|
27
|
-
"pingfusi": "^0.16.
|
|
28
|
+
"pingfusi": "^0.16.1"
|
|
28
29
|
},
|
|
29
30
|
"publishConfig": {
|
|
30
31
|
"access": "public"
|
package/skill/qaping/SKILL.md
CHANGED
|
@@ -17,18 +17,37 @@ cheaper. All service contact goes through three MCP tools —
|
|
|
17
17
|
|
|
18
18
|
## Ground rules (read before filing anything)
|
|
19
19
|
|
|
20
|
-
- **Playtesters are real people, and
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
20
|
+
- **Playtesters are real people, and a round outlives a quick turn.
|
|
21
|
+
Decide at filing time: wait, or park.** A session takes real minutes, often
|
|
22
|
+
hours, to be claimed and played. Which of the two endings this round gets is
|
|
23
|
+
chosen BEFORE you file, not discovered when pending surprises you.
|
|
24
|
+
- **WAIT — short rounds you can sit through.** Each `qaping_wait` renews the
|
|
25
|
+
lease that keeps the round visible to new claimants. The moment a filing
|
|
26
|
+
returns pending, SPAWN A BACKGROUND TASK that loops `qaping_wait` with the
|
|
27
|
+
ping_id until results arrive, and keep working on other things — that is
|
|
28
|
+
the intended shape, the same as every pingfusi round. Only if your harness
|
|
29
|
+
has no background tasks: keep polling in the foreground. A one-shot print
|
|
30
|
+
mode where neither is possible is not a WAIT round at all — it is a PARK
|
|
31
|
+
(below): file with an explicit `deadline_seconds`, record it with `qaping
|
|
32
|
+
rounds add <ping_id>`, say what is pending, and ask to be re-invoked to
|
|
33
|
+
collect it — never promise polling you cannot do (an ended session cannot
|
|
34
|
+
poll). A round already filed on the default lease is recorded anyway,
|
|
35
|
+
flagged as likely to expire unclaimed.
|
|
36
|
+
- **PARK — long or overnight rounds.** For a session nobody will sit
|
|
37
|
+
through — an overnight run, `n_target > 1` on a small pool, a native round
|
|
38
|
+
whose claim is slow, an explicit "check tomorrow", or a one-shot mode —
|
|
39
|
+
file with an explicit `deadline_seconds` sized to the span you are willing
|
|
40
|
+
to wait (the tool schema states the ceiling) plus an `idempotency_key`,
|
|
41
|
+
then IMMEDIATELY record it: `qaping rounds add <ping_id> --build <url>
|
|
42
|
+
--minutes N --checks <ids> --note "<patch>"`, which writes
|
|
43
|
+
`qa-open-rounds.json` at the repo root (gitignore it — open state, not
|
|
44
|
+
history). Then tell the user exactly what is pending and what will collect
|
|
45
|
+
it, and end the turn cleanly.
|
|
46
|
+
- **A recorded parked round is an honest pending end; an unrecorded "I'll
|
|
47
|
+
check later" never is.** Never park on the default lease: with no explicit
|
|
48
|
+
`deadline_seconds` an unwaited round leaves the feed within minutes and
|
|
49
|
+
expires unclaimed (a round died exactly that way, live). On resume,
|
|
50
|
+
collect the recorded round — never file a duplicate.
|
|
32
51
|
- **Costs are duration-billed**: 2 credits per minute of play, per playtester
|
|
33
52
|
(`est_minutes × 2 × players`). State the estimate to the user BEFORE filing,
|
|
34
53
|
and set the wait expectation honestly in the same message: claim time is
|
|
@@ -51,6 +70,37 @@ cheaper. All service contact goes through three MCP tools —
|
|
|
51
70
|
- **QA-PLAN.md is the dev's file.** Commit it only with their approval; never
|
|
52
71
|
delete or demote a human check without them agreeing.
|
|
53
72
|
|
|
73
|
+
## Open rounds — collect FIRST
|
|
74
|
+
|
|
75
|
+
Before ANY qaping work in a repo — a new run, a plan edit, a plain "did it
|
|
76
|
+
come back yet?" — read `qa-open-rounds.json` at the repo root. `qaping rounds`
|
|
77
|
+
lists it with each round's live status; without the CLI, call `qaping_results`
|
|
78
|
+
once per recorded `ping_id`. The file is this, and nothing else:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{"rounds": [{"ping_id": "<uuid>", "filed_at": "<ISO>", "platform": "windows",
|
|
82
|
+
"build": "<the /b/<slug> or store URL>", "est_minutes": 10,
|
|
83
|
+
"deadline_seconds": 86400, "checks": ["boot-to-menu"],
|
|
84
|
+
"note": "<the patch this round covers>"}]}
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
- **READY** (results are in): collect it BEFORE starting new work — the full
|
|
88
|
+
RUN report ritual (evidence line, computed headline, per-check rows, PR
|
|
89
|
+
comment, issues, plan updates, ledger append), then
|
|
90
|
+
`qaping rounds rm <ping_id>`.
|
|
91
|
+
- **Still pending**: say so plainly, with what it is still waiting on, and
|
|
92
|
+
leave it recorded. Never refile it and never start a second round on the
|
|
93
|
+
same checks while it is open.
|
|
94
|
+
- **Expired or unreadable**: report that honestly — an expired round bought
|
|
95
|
+
nothing — then `qaping rounds rm <ping_id>` and offer a refile (a fresh
|
|
96
|
+
build URL if the old one expired, and a longer `deadline_seconds`).
|
|
97
|
+
- **Not visible from this login** reads identically to gone, and is not the
|
|
98
|
+
same thing: results are asker-scoped, so a round filed from another account
|
|
99
|
+
is alive and invisible. Check `qaping whoami` before deleting that record.
|
|
100
|
+
|
|
101
|
+
`qa-open-rounds.json` is mutable OPEN state; `qa-rounds.jsonl` is append-only
|
|
102
|
+
history written after results land. Never merge the two.
|
|
103
|
+
|
|
54
104
|
## The review interface (know your tester)
|
|
55
105
|
|
|
56
106
|
<!-- Distilled from QAPING_REVIEWER_INTERFACE.md (monorepo root, INTERNAL,
|
|
@@ -187,9 +237,16 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
187
237
|
static directory (web).
|
|
188
238
|
4. **Ship**, by delivery: NATIVE uploads — `qaping publish-build <game.zip>
|
|
189
239
|
--platform windows|macos` → prints a `/b/<slug>` URL (temporary hosting;
|
|
190
|
-
filing a playtest extends the build through the round;
|
|
191
|
-
|
|
192
|
-
|
|
240
|
+
filing a playtest extends the build through the round; a CHANGED build gets
|
|
241
|
+
a new URL, while re-publishing an unchanged zip returns the same build).
|
|
242
|
+
You hold only a handful of hosted builds at once: if a publish is refused
|
|
243
|
+
for that cap, `qaping builds` lists what you are holding and
|
|
244
|
+
`qaping builds rm <slug>` frees a slot — do that and carry on rather than
|
|
245
|
+
stopping to ask. The listing marks each build: never-finished uploads are
|
|
246
|
+
safe to delete first, and any build an open round is still using is marked
|
|
247
|
+
`IN USE` — never delete one of those, whichever session filed it. If every
|
|
248
|
+
build is in use, say so and wait for a round to finish rather than guessing. Store-delivered games file with their Steam store
|
|
249
|
+
page, TestFlight public link, or App Store page as `url`. WEB games — host the built output
|
|
193
250
|
at a public URL: the dev's own hosting if they have it, else
|
|
194
251
|
`qaping publish <built-dir>` prints a hosted URL. Localhost never works —
|
|
195
252
|
players open the link on their own devices. When filing a WEB game: send
|
|
@@ -251,9 +308,13 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
251
308
|
tester and silently dropped at finish, which is why this gate exists. A
|
|
252
309
|
mismatch (the budget forcing cuts) fails loudly PRE-SPEND: batch another
|
|
253
310
|
session or state the deferral out loud — never silently truncate.
|
|
254
|
-
6. **Wait
|
|
255
|
-
and keep working
|
|
256
|
-
|
|
311
|
+
6. **Wait or park — decide now** (ground rule 1). WAIT: spawn a background
|
|
312
|
+
task looping `qaping_wait` and keep working — foreground polling is the
|
|
313
|
+
fallback. PARK: the filing already carried an explicit `deadline_seconds`
|
|
314
|
+
sized to the span, so record it — `qaping rounds add <ping_id> --build
|
|
315
|
+
<url> --minutes N --checks <ids> --note "<patch>"` — tell the user what is
|
|
316
|
+
pending, and end the turn; a later session collects it. A promise to check
|
|
317
|
+
later is never a wait, and never a substitute for the record.
|
|
257
318
|
7. **Report** from `qaping_results`:
|
|
258
319
|
- **Evidence line first (MUST).** Every report opens with its grade:
|
|
259
320
|
`Evidence: recording + transcript` (macOS/iOS) / `Evidence: recording
|
|
@@ -292,6 +353,10 @@ checks testimony-only — mark that in the plan and report at that grade.
|
|
|
292
353
|
`last_verified` stays in QA-PLAN.md as the dev-readable contract derived
|
|
293
354
|
from it, and "carry findings forward" is a read of the last line's
|
|
294
355
|
follow_ups, not prose archaeology.
|
|
356
|
+
- **Close a parked round (MUST).** In the same step as the ledger append,
|
|
357
|
+
drop this round from `qa-open-rounds.json` with
|
|
358
|
+
`qaping rounds rm <ping_id>`, so the open-state file never outlives the
|
|
359
|
+
round it names.
|
|
295
360
|
- Update `QA-PLAN.md` from the outcomes: Pass → `state: passed` and
|
|
296
361
|
`last_verified` this build/date; a Fail with hard evidence → `failing`;
|
|
297
362
|
a Fail that could be dirty state or tester noise → `ambiguous`;
|