@piwitests/reporter 0.25.0 → 0.26.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # Piwi Dashboard Reporter
2
2
 
3
- A custom Playwright reporter that sends test results to a [Piwi Dashboard](https://piwitests.github.io) server. It handles uploading test results, HTML reports, trace files, and performance metrics — with optional live streaming of results as tests execute.
3
+ A custom Playwright reporter that sends test results to a [Piwi Dashboard](https://piwitests.dev) server. It handles uploading test results, HTML reports, trace files, and performance metrics — with optional live streaming of results as tests execute.
4
4
 
5
- 📖 **[Full documentation](https://piwitests.github.io/reporter)**
5
+ 📖 **[Full documentation](https://piwitests.dev/reporter)**
6
6
 
7
7
  ## Installation
8
8
 
@@ -18,14 +18,14 @@ From your Playwright project, `npx @piwitests/reporter init` installs the report
18
18
  npx @piwitests/reporter init --server-url http://localhost:3000 --project my-project
19
19
  ```
20
20
 
21
- Every step is idempotent (safe to re-run); a config shape it will not rewrite is reported as `manual` with the exact change to make, never mangled. Add `--dry-run` to preview or `--json` for a machine-readable plan an agent can act on. It also installs the [Piwi agent skills](https://piwitests.github.io/mcp#agent-skills) so your coding agent can investigate failures, heal locators, and stabilize flaky tests. Run `npx @piwitests/reporter init --help` for all options, or wire it up by hand with the steps below.
21
+ Every step is idempotent (safe to re-run); a config shape it will not rewrite is reported as `manual` with the exact change to make, never mangled. Add `--dry-run` to preview or `--json` for a machine-readable plan an agent can act on. It also installs the [Piwi agent skills](https://piwitests.dev/mcp#agent-skills) so your coding agent can investigate failures, heal locators, and stabilize flaky tests. Run `npx @piwitests/reporter init --help` for all options, or wire it up by hand with the steps below.
22
22
 
23
23
  > The package is `@piwitests/reporter`; its command is `piwi`. Invoke it through the package name (`npx @piwitests/reporter <command>`) so npx resolves this package — `npx piwi` would fetch an unrelated `piwi` from npm. Once the reporter is a project dependency, `npx piwi <command>` resolves the local binary and works too.
24
24
 
25
25
  ## Quick start
26
26
 
27
27
  `wrapConfig` is the recommended setup. It injects the reporter **and** a global
28
- setup step (so the run shows up as "initialising" while your `globalSetup` runs),
28
+ setup step (so the run shows up as "initializing" while your `globalSetup` runs),
29
29
  and forwards your options to that setup:
30
30
 
31
31
  ```typescript
@@ -184,7 +184,7 @@ Then import `test` from your fixtures file in every spec — a spec that imports
184
184
 
185
185
  Capture works for the `page` fixture, `browser.newPage()`, `browser.newContext().newPage()`, and popups. Everything is only collected when `collectPerformanceMetrics` is `true` (the default); locator snapshots can be disabled separately with `captureLocators: false`.
186
186
 
187
- Without the fixtures you still get full run history, statuses, errors, traces, reports, streaming, and clustering — the fixtures add the slow-endpoint, Web Vitals, console, ARIA, and locator-healing layers. See the [capture fixtures guide](https://piwitests.github.io/capture-fixtures) for the full feature matrix and composition patterns.
187
+ Without the fixtures you still get full run history, statuses, errors, traces, reports, streaming, and clustering — the fixtures add the slow-endpoint, Web Vitals, console, ARIA, and locator-healing layers. See the [capture fixtures guide](https://piwitests.dev/capture-fixtures) for the full feature matrix and composition patterns.
188
188
 
189
189
  ## AI steps
190
190
 
@@ -204,7 +204,7 @@ export const test = extendPiwiAi(extendPiwiFixtures(base))
204
204
 
205
205
  Author missing entries once in `resolve` mode (`PIWI_AI=resolve`, pointed at a dashboard with an AI provider configured), commit the artifacts, and CI replays them offline. `{param}` placeholders are type-checked and masked out of everything sent to the model. Manage the committed entries with `piwi ai check | resolve | prune`.
206
206
 
207
- See the [AI steps guide](https://piwitests.github.io/ai-steps) for the authoring/replay lifecycle, the safety model (allowlisted, drift-guarded, postcondition-verified), and the full option/env-var reference.
207
+ See the [AI steps guide](https://piwitests.dev/ai-steps) for the authoring/replay lifecycle, the safety model (allowlisted, drift-guarded, postcondition-verified), and the full option/env-var reference.
208
208
 
209
209
  ## Authentication
210
210
 
@@ -298,7 +298,7 @@ addition:
298
298
 
299
299
  ## Requirements
300
300
 
301
- - Node.js 18 or higher (the reporter runs inside your test project — the dashboard *server* itself targets Node 24+, or use its Docker image)
301
+ - Node.js 18 or higher (the reporter runs inside your test project — the dashboard *server* itself targets Node 22+, or use its Docker image)
302
302
  - Playwright Test 1.40 or higher
303
303
  - Running Piwi Dashboard server
304
304
 
package/dist/cli/index.js CHANGED
@@ -414,7 +414,7 @@ function formatGateResult(result) {
414
414
  const { facts } = result;
415
415
  const lines = [
416
416
  result.passed ? `\u2714 Piwi gate passed \u2014 ${facts.projectName} run #${facts.runId}` : `\u2716 Piwi gate failed \u2014 ${facts.projectName} run #${facts.runId}`,
417
- ` ${facts.totalTests} tests, ${facts.failedTests} failed, ${facts.newRegressions} new, ${facts.newFlaky} newly flaky`
417
+ ` ${facts.totalTests} tests, ${facts.failedTests} failed, ${facts.newRegressions} new, ${facts.newFlaky} newly flaky, ${facts.flakyTests} flaky`
418
418
  ];
419
419
  if (facts.quarantinedFailures > 0) {
420
420
  lines.push(
@@ -453,6 +453,8 @@ Policy (at least one is required):
453
453
  --max-new-flaky <n> Fail when more than n tests newly became flaky
454
454
  --max-quarantined <n> Fail when more than n tests are quarantined
455
455
  --fail-on-new-cluster Fail when this run introduced a new failure cluster
456
+ --fail-on-flaky Fail when this run contains any flaky test
457
+ --require-selection <key> Fail when a test the named selection matches did not run or failed
456
458
 
457
459
  Other:
458
460
  --json Print the raw result as JSON instead of a summary
@@ -475,9 +477,9 @@ function readCount(argv, name) {
475
477
  if (!Number.isFinite(n) || n < 0) throw new Error(`${name} expects a non-negative number, got "${raw}"`);
476
478
  return Math.floor(n);
477
479
  }
478
- function readRunIdFromFile(path9) {
480
+ function readRunIdFromFile(path10) {
479
481
  try {
480
- const parsed = JSON.parse(fs3.readFileSync(path9, "utf-8"));
482
+ const parsed = JSON.parse(fs3.readFileSync(path10, "utf-8"));
481
483
  const runId = Number(parsed.runId);
482
484
  return Number.isFinite(runId) && runId > 0 ? runId : null;
483
485
  } catch {
@@ -504,7 +506,9 @@ function parseGateArgs(argv, env) {
504
506
  maxNewRegressions: readCount(argv, "--max-new-regressions"),
505
507
  maxNewFlaky: readCount(argv, "--max-new-flaky"),
506
508
  maxQuarantined: readCount(argv, "--max-quarantined"),
507
- failOnNewCluster: argv.includes("--fail-on-new-cluster")
509
+ failOnNewCluster: argv.includes("--fail-on-new-cluster"),
510
+ failOnFlaky: argv.includes("--fail-on-flaky"),
511
+ requireSelection: readOption2(argv, "--require-selection")
508
512
  };
509
513
  return { serverUrl, apiKey: readOption2(argv, "--api-key") ?? env.PIWI_API_KEY ?? null, runId, policy };
510
514
  }
@@ -770,7 +774,12 @@ function ensureGitignoreEntry(existing, entry = ".env") {
770
774
  var fs5 = __toESM(require("fs"));
771
775
  var path7 = __toESM(require("path"));
772
776
  var SETUP_SKILL = "setup-piwi";
773
- var WORKFLOW_SKILLS = ["investigate-failure", "apply-locator-healing", "stabilize-flaky-tests"];
777
+ var WORKFLOW_SKILLS = [
778
+ "investigate-failure",
779
+ "apply-locator-healing",
780
+ "stabilize-flaky-tests",
781
+ "run-the-right-tests"
782
+ ];
774
783
  var ALL_SKILLS = [SETUP_SKILL, ...WORKFLOW_SKILLS];
775
784
  var DEFAULT_SKILLS_DIR = path7.join(".claude", "skills");
776
785
  function findTemplatesDir(fromDir) {
@@ -1186,8 +1195,415 @@ Piwi setup${opts.dryRun ? " (dry run \u2014 nothing written)" : ""} for "${opts.
1186
1195
  return steps.some((s) => s.status === "error") ? 1 : 0;
1187
1196
  }
1188
1197
 
1189
- // src/cli/index.ts
1198
+ // src/cli/select.ts
1199
+ var fs7 = __toESM(require("fs"));
1200
+ var path9 = __toESM(require("path"));
1201
+ var import_node_child_process3 = require("child_process");
1202
+ var import_node_module = require("module");
1203
+
1204
+ // src/internal/support/selection-client.ts
1205
+ function authHeaders(apiKey) {
1206
+ return apiKey ? { "X-API-Key": apiKey } : {};
1207
+ }
1208
+ async function resolveProjectId(options) {
1209
+ if (!options.project) throw new Error("No project \u2014 pass --project or set PIWI_PROJECT_NAME");
1210
+ if (/^\d+$/.test(options.project)) return Number(options.project);
1211
+ const res = await fetch(`${options.serverUrl}/api/projects/menu`, { headers: authHeaders(options.apiKey) });
1212
+ if (!res.ok) throw new Error(`Could not list projects (dashboard returned ${res.status})`);
1213
+ const body = await res.json();
1214
+ const match = body.items?.find((p) => p.name.toLowerCase() === options.project.toLowerCase());
1215
+ if (!match) throw new Error(`No project named "${options.project}" on this dashboard`);
1216
+ return match.id;
1217
+ }
1218
+ async function fetchResolution(options, projectId) {
1219
+ const params = new URLSearchParams({ format: options.format ?? "args" });
1220
+ if (options.pkgRunner) params.set("pkgRunner", options.pkgRunner);
1221
+ if (options.budgetMs != null) params.set("budgetMs", String(options.budgetMs));
1222
+ if (options.shard) params.set("shard", options.shard);
1223
+ if (options.order) params.set("order", options.order);
1224
+ const url = `${options.serverUrl}/api/projects/${projectId}/selections/${encodeURIComponent(options.key)}/resolve?${params}`;
1225
+ const res = await fetch(url, { headers: authHeaders(options.apiKey) });
1226
+ if (!res.ok) {
1227
+ const body = await res.json().catch(() => ({}));
1228
+ throw new Error(body.message || `Dashboard returned ${res.status} resolving "${options.key}"`);
1229
+ }
1230
+ return await res.json();
1231
+ }
1232
+ async function fetchImpact(options, projectId, changedFiles) {
1233
+ const url = `${options.serverUrl}/api/projects/${projectId}/selections/impact`;
1234
+ const res = await fetch(url, {
1235
+ method: "POST",
1236
+ headers: { "Content-Type": "application/json", ...authHeaders(options.apiKey) },
1237
+ body: JSON.stringify({
1238
+ changedFiles,
1239
+ format: options.format ?? "args",
1240
+ shard: options.shard ?? void 0,
1241
+ order: options.order ?? void 0
1242
+ })
1243
+ });
1244
+ if (!res.ok) {
1245
+ const body = await res.json().catch(() => ({}));
1246
+ throw new Error(body.message || `Dashboard returned ${res.status} resolving impact`);
1247
+ }
1248
+ return await res.json();
1249
+ }
1250
+
1251
+ // src/cli/select.ts
1252
+ var EXIT_OK3 = 0;
1253
+ var EXIT_ERROR3 = 2;
1190
1254
  var USAGE5 = `
1255
+ piwi select / piwi run \u2014 run a saved selection of tests
1256
+
1257
+ Usage:
1258
+ npx @piwitests/reporter select <key> [options] print the Playwright args
1259
+ npx @piwitests/reporter run <key> [options] [-- <playwright args>]
1260
+ npx @piwitests/reporter run impact --base <ref> run the tests your diff impacts
1261
+
1262
+ Connection:
1263
+ --server-url <url> Dashboard URL (env PIWI_DASHBOARD_URL)
1264
+ --api-key <key> API key (env PIWI_API_KEY)
1265
+ --project <name|id> Project (env PIWI_PROJECT_NAME)
1266
+
1267
+ Selection:
1268
+ --format <fmt> args (file:line, default) | grep | files | json
1269
+ --budget <duration> Cap total time, e.g. 5m, 90s, 300000 (ms)
1270
+ --shard <i/n> Keep only shard i of n, balanced by test duration
1271
+ --fail-fast Order the least-reliable tests first (fail-fast)
1272
+ --base <ref> For "impact": the ref to diff the working tree against
1273
+
1274
+ Behavior:
1275
+ --strict Fail (exit 2) instead of falling back when unreachable
1276
+ --pkg-runner <cmd> Package runner for the printed command (default npx)
1277
+ --json Print the full resolution as JSON (select only)
1278
+ -h, --help Show this help
1279
+
1280
+ Exit codes: 0 ok, 1 the test run failed (run only), 2 could not resolve.
1281
+ `.trim();
1282
+ var IMPACT_KEY = "impact";
1283
+ var VALUE_FLAGS = /* @__PURE__ */ new Set([
1284
+ "--server-url",
1285
+ "--api-key",
1286
+ "--project",
1287
+ "--format",
1288
+ "--budget",
1289
+ "--shard",
1290
+ "--base",
1291
+ "--pkg-runner"
1292
+ ]);
1293
+ function readOption5(argv, name) {
1294
+ const withEquals = argv.find((arg) => arg.startsWith(`${name}=`));
1295
+ if (withEquals) return withEquals.slice(name.length + 1);
1296
+ const index = argv.indexOf(name);
1297
+ if (index === -1) return void 0;
1298
+ const value = argv[index + 1];
1299
+ return value && !value.startsWith("-") ? value : void 0;
1300
+ }
1301
+ function findKey(argv) {
1302
+ for (let i = 0; i < argv.length; i++) {
1303
+ const tok = argv[i];
1304
+ if (tok.startsWith("-")) {
1305
+ if (!tok.includes("=") && VALUE_FLAGS.has(tok)) i++;
1306
+ continue;
1307
+ }
1308
+ return tok;
1309
+ }
1310
+ return void 0;
1311
+ }
1312
+ function parseDuration(raw) {
1313
+ const match = raw.trim().match(/^(\d+(?:\.\d+)?)(ms|s|m|h)?$/);
1314
+ if (!match) return null;
1315
+ const value = Number(match[1]);
1316
+ const unit = match[2];
1317
+ const factor = unit === "h" ? 36e5 : unit === "m" ? 6e4 : unit === "s" ? 1e3 : 1;
1318
+ const ms = Math.round(value * factor);
1319
+ return ms > 0 ? ms : null;
1320
+ }
1321
+ function parseSelectArgs(argv, env) {
1322
+ const dashIndex = argv.indexOf("--");
1323
+ const own = dashIndex === -1 ? argv : argv.slice(0, dashIndex);
1324
+ const extra = dashIndex === -1 ? [] : argv.slice(dashIndex + 1);
1325
+ const serverUrl = (readOption5(own, "--server-url") ?? env.PIWI_DASHBOARD_URL ?? "").replace(/\/$/, "");
1326
+ if (!serverUrl) throw new Error("No dashboard URL \u2014 pass --server-url or set PIWI_DASHBOARD_URL");
1327
+ const key = findKey(own);
1328
+ if (!key) throw new Error("No selection key \u2014 usage: piwi select <key>");
1329
+ const budgetRaw = readOption5(own, "--budget");
1330
+ let budgetMs = null;
1331
+ if (budgetRaw !== void 0) {
1332
+ budgetMs = parseDuration(budgetRaw);
1333
+ if (budgetMs === null) throw new Error(`--budget expects a duration like 5m or 90s, got "${budgetRaw}"`);
1334
+ }
1335
+ const shard = readOption5(own, "--shard") ?? null;
1336
+ if (shard !== null && !/^\d+\s*\/\s*\d+$/.test(shard)) {
1337
+ throw new Error(`--shard expects an "i/n" spec like 2/4, got "${shard}"`);
1338
+ }
1339
+ return {
1340
+ serverUrl,
1341
+ apiKey: readOption5(own, "--api-key") ?? env.PIWI_API_KEY ?? null,
1342
+ project: readOption5(own, "--project") ?? env.PIWI_PROJECT_NAME ?? "",
1343
+ key,
1344
+ format: readOption5(own, "--format") ?? "args",
1345
+ budgetMs,
1346
+ shard,
1347
+ order: own.includes("--fail-fast") ? "failureLikelihood" : null,
1348
+ base: readOption5(own, "--base") ?? null,
1349
+ strict: own.includes("--strict"),
1350
+ pkgRunner: readOption5(own, "--pkg-runner") ?? "npx",
1351
+ json: own.includes("--json"),
1352
+ extra
1353
+ };
1354
+ }
1355
+ var CACHE_FILE = path9.join(".piwi", "selection-cache.json");
1356
+ function cacheKey(projectId, args) {
1357
+ return `${projectId}:${args.key}:${args.format}:${args.budgetMs ?? 0}:${args.shard ?? ""}:${args.order ?? ""}`;
1358
+ }
1359
+ function readCache(projectId, args) {
1360
+ try {
1361
+ const store = JSON.parse(fs7.readFileSync(CACHE_FILE, "utf-8"));
1362
+ return store[cacheKey(projectId, args)] ?? null;
1363
+ } catch {
1364
+ return null;
1365
+ }
1366
+ }
1367
+ function writeCache(projectId, args, resolution) {
1368
+ try {
1369
+ let store = {};
1370
+ try {
1371
+ store = JSON.parse(fs7.readFileSync(CACHE_FILE, "utf-8"));
1372
+ } catch {
1373
+ }
1374
+ store[cacheKey(projectId, args)] = resolution;
1375
+ fs7.mkdirSync(path9.dirname(CACHE_FILE), { recursive: true });
1376
+ fs7.writeFileSync(CACHE_FILE, JSON.stringify(store, null, 2));
1377
+ } catch {
1378
+ }
1379
+ }
1380
+ async function resolveWithCache(args, projectId) {
1381
+ try {
1382
+ const resolution = await fetchResolution(args, projectId);
1383
+ writeCache(projectId, args, resolution);
1384
+ return { resolution, fromCache: false };
1385
+ } catch (e) {
1386
+ if (args.strict) throw e;
1387
+ const cached = readCache(projectId, args);
1388
+ if (cached) {
1389
+ console.error(`piwi: dashboard unreachable, using cached resolution \u2014 ${e.message}`);
1390
+ return { resolution: cached, fromCache: true };
1391
+ }
1392
+ return null;
1393
+ }
1394
+ }
1395
+ function printWarnings(resolution) {
1396
+ for (const w of resolution.warnings) console.error(`piwi: warning [${w.code}] ${w.message}`);
1397
+ }
1398
+ function gitChangedFiles(base) {
1399
+ const out = (0, import_node_child_process3.execFileSync)("git", ["diff", "--name-only", base], {
1400
+ encoding: "utf-8",
1401
+ stdio: ["ignore", "pipe", "ignore"]
1402
+ });
1403
+ return [
1404
+ ...new Set(
1405
+ out.split("\n").map((line) => line.trim()).filter(Boolean)
1406
+ )
1407
+ ];
1408
+ }
1409
+ async function loadImpact(args, projectId) {
1410
+ const files = gitChangedFiles(args.base);
1411
+ return fetchImpact(args, projectId, files);
1412
+ }
1413
+ async function runSelectImpact(args) {
1414
+ let impact;
1415
+ try {
1416
+ const projectId = await resolveProjectId(args);
1417
+ impact = await loadImpact(args, projectId);
1418
+ } catch (e) {
1419
+ console.error(`piwi select: ${e.message}`);
1420
+ return EXIT_ERROR3;
1421
+ }
1422
+ printWarnings(impact);
1423
+ console.error(
1424
+ `piwi select: ${impact.impact.changedFiles} changed file(s) \u2192 ${impact.estimate.count} impacted test(s)${impact.impact.widened ? " (widened to full suite)" : ""}`
1425
+ );
1426
+ if (args.json) {
1427
+ console.log(JSON.stringify(impact, null, 2));
1428
+ return EXIT_OK3;
1429
+ }
1430
+ console.log(impact.impact.widened ? "" : impact.materialization.args.join(" "));
1431
+ return EXIT_OK3;
1432
+ }
1433
+ async function runSelect(argv, env = process.env) {
1434
+ if (argv.includes("-h") || argv.includes("--help")) {
1435
+ console.log(USAGE5);
1436
+ return EXIT_OK3;
1437
+ }
1438
+ let args;
1439
+ try {
1440
+ args = parseSelectArgs(argv, env);
1441
+ } catch (e) {
1442
+ console.error(`piwi select: ${e.message}
1443
+ `);
1444
+ console.error(USAGE5);
1445
+ return EXIT_ERROR3;
1446
+ }
1447
+ if (args.key === IMPACT_KEY) {
1448
+ if (!args.base) {
1449
+ console.error("piwi select: impact needs a base ref \u2014 pass --base <ref> (e.g. --base origin/main)");
1450
+ return EXIT_ERROR3;
1451
+ }
1452
+ return runSelectImpact(args);
1453
+ }
1454
+ let resolution;
1455
+ try {
1456
+ const projectId = await resolveProjectId(args);
1457
+ resolution = await fetchResolution(args, projectId);
1458
+ } catch (e) {
1459
+ console.error(`piwi select: ${e.message}`);
1460
+ return EXIT_ERROR3;
1461
+ }
1462
+ printWarnings(resolution);
1463
+ if (resolution.estimate.count === 0) {
1464
+ console.error(`piwi select: "${args.key}" resolved to 0 tests`);
1465
+ return EXIT_ERROR3;
1466
+ }
1467
+ if (args.json) console.log(JSON.stringify(resolution, null, 2));
1468
+ else console.log(resolution.materialization.args.join(" "));
1469
+ return EXIT_OK3;
1470
+ }
1471
+ function resolvePlaywrightCli() {
1472
+ const require2 = (0, import_node_module.createRequire)(path9.join(process.cwd(), "noop.js"));
1473
+ for (const id of ["playwright/cli", "@playwright/test/cli", "playwright/lib/cli/cli"]) {
1474
+ try {
1475
+ return require2.resolve(id);
1476
+ } catch {
1477
+ }
1478
+ }
1479
+ return null;
1480
+ }
1481
+ function spawnPlaywright(pkgRunner, playwrightArgs, env) {
1482
+ const cli = resolvePlaywrightCli();
1483
+ const child = cli ? (0, import_node_child_process3.spawn)(process.execPath, [cli, "test", ...playwrightArgs], { stdio: "inherit", env }) : (0, import_node_child_process3.spawn)(process.platform === "win32" ? `${pkgRunner}.cmd` : pkgRunner, ["playwright", "test", ...playwrightArgs], {
1484
+ stdio: "inherit",
1485
+ env
1486
+ });
1487
+ return new Promise((resolve5) => {
1488
+ child.on("error", (err) => {
1489
+ console.error(`piwi run: could not start Playwright \u2014 ${err.message}`);
1490
+ resolve5(EXIT_ERROR3);
1491
+ });
1492
+ child.on("exit", (code) => resolve5(code ?? EXIT_ERROR3));
1493
+ });
1494
+ }
1495
+ async function runRunImpact(args, env) {
1496
+ let projectId;
1497
+ try {
1498
+ projectId = await resolveProjectId(args);
1499
+ } catch (e) {
1500
+ if (args.strict) {
1501
+ console.error(`piwi run: ${e.message}`);
1502
+ return EXIT_ERROR3;
1503
+ }
1504
+ console.error(`piwi run: ${e.message} \u2014 running the full suite`);
1505
+ return spawnPlaywright(args.pkgRunner, args.extra, env);
1506
+ }
1507
+ let impact;
1508
+ try {
1509
+ impact = await loadImpact(args, projectId);
1510
+ } catch (e) {
1511
+ if (args.strict) {
1512
+ console.error(`piwi run: ${e.message}`);
1513
+ return EXIT_ERROR3;
1514
+ }
1515
+ console.error(`piwi run: ${e.message} \u2014 running the full suite`);
1516
+ return spawnPlaywright(args.pkgRunner, args.extra, env);
1517
+ }
1518
+ printWarnings(impact);
1519
+ if (impact.impact.widened) {
1520
+ console.error(
1521
+ `piwi run: impact widened to the full suite (${impact.impact.unmappedSourceFiles.length} unmapped source file(s))`
1522
+ );
1523
+ return spawnPlaywright(args.pkgRunner, args.extra, env);
1524
+ }
1525
+ if (impact.estimate.count === 0) {
1526
+ console.error(`piwi run: no tests impacted by ${impact.impact.changedFiles} changed file(s) \u2014 nothing to run`);
1527
+ return EXIT_OK3;
1528
+ }
1529
+ const runEnv = {
1530
+ ...env,
1531
+ PIWI_SELECTION: IMPACT_KEY,
1532
+ PIWI_SELECTION_VERSION: "0",
1533
+ PIWI_SELECTION_HASH: impact.resolvedHash,
1534
+ PIWI_SELECTION_COUNT: String(impact.estimate.count)
1535
+ };
1536
+ console.error(`piwi run: impact \u2192 ${impact.estimate.count} test(s)`);
1537
+ return spawnPlaywright(args.pkgRunner, [...impact.materialization.args, ...args.extra], runEnv);
1538
+ }
1539
+ async function runRun(argv, env = process.env) {
1540
+ if (argv.includes("-h") || argv.includes("--help")) {
1541
+ console.log(USAGE5);
1542
+ return EXIT_OK3;
1543
+ }
1544
+ let args;
1545
+ try {
1546
+ args = parseSelectArgs(argv, env);
1547
+ } catch (e) {
1548
+ console.error(`piwi run: ${e.message}
1549
+ `);
1550
+ console.error(USAGE5);
1551
+ return EXIT_ERROR3;
1552
+ }
1553
+ if (args.format === "json") {
1554
+ console.error("piwi run: --format json cannot be run; use args, grep or files");
1555
+ return EXIT_ERROR3;
1556
+ }
1557
+ if (args.key === IMPACT_KEY) {
1558
+ if (!args.base) {
1559
+ console.error("piwi run: impact needs a base ref \u2014 pass --base <ref> (e.g. --base origin/main)");
1560
+ return EXIT_ERROR3;
1561
+ }
1562
+ return runRunImpact(args, env);
1563
+ }
1564
+ let projectId;
1565
+ try {
1566
+ projectId = await resolveProjectId(args);
1567
+ } catch (e) {
1568
+ if (args.strict) {
1569
+ console.error(`piwi run: ${e.message}`);
1570
+ return EXIT_ERROR3;
1571
+ }
1572
+ console.error(`piwi run: ${e.message} \u2014 running the full suite`);
1573
+ return spawnPlaywright(args.pkgRunner, args.extra, env);
1574
+ }
1575
+ let outcome;
1576
+ try {
1577
+ outcome = await resolveWithCache(args, projectId);
1578
+ } catch (e) {
1579
+ console.error(`piwi run: ${e.message}`);
1580
+ return EXIT_ERROR3;
1581
+ }
1582
+ if (!outcome) {
1583
+ console.error("piwi run: dashboard unreachable and no cached resolution \u2014 running the full suite");
1584
+ return spawnPlaywright(args.pkgRunner, args.extra, env);
1585
+ }
1586
+ const { resolution } = outcome;
1587
+ printWarnings(resolution);
1588
+ if (resolution.estimate.count === 0) {
1589
+ console.error(`piwi run: "${args.key}" resolved to 0 tests`);
1590
+ return EXIT_ERROR3;
1591
+ }
1592
+ const runEnv = {
1593
+ ...env,
1594
+ PIWI_SELECTION: resolution.key ?? args.key,
1595
+ PIWI_SELECTION_VERSION: String(resolution.version ?? 0),
1596
+ PIWI_SELECTION_HASH: resolution.resolvedHash,
1597
+ PIWI_SELECTION_COUNT: String(resolution.estimate.count)
1598
+ };
1599
+ console.error(
1600
+ `piwi run: ${args.key} \u2192 ${resolution.estimate.count} tests${resolution.materialization.format !== args.format ? ` (materialized as ${resolution.materialization.format})` : ""}`
1601
+ );
1602
+ return spawnPlaywright(args.pkgRunner, [...resolution.materialization.args, ...args.extra], runEnv);
1603
+ }
1604
+
1605
+ // src/cli/index.ts
1606
+ var USAGE6 = `
1191
1607
  piwi \u2014 companion commands for the Piwi Dashboard reporter
1192
1608
 
1193
1609
  Usage:
@@ -1197,6 +1613,8 @@ Commands:
1197
1613
  init Wire a Playwright project up to a Piwi Dashboard
1198
1614
  skills Install the Piwi agent skills into this project
1199
1615
  gate Fail a CI job on the dashboard's analysis of a run
1616
+ select Print the Playwright args for a saved test selection
1617
+ run Run a saved test selection with playwright test
1200
1618
  ai Manage committed natural-language AI-step artifacts
1201
1619
 
1202
1620
  Run \`npx @piwitests/reporter <command> --help\` for a command's options.
@@ -1212,17 +1630,21 @@ async function main() {
1212
1630
  return runSkills(rest, findTemplatesDir(__dirname));
1213
1631
  case "gate":
1214
1632
  return runGate(rest);
1633
+ case "select":
1634
+ return runSelect(rest);
1635
+ case "run":
1636
+ return runRun(rest);
1215
1637
  case "ai":
1216
1638
  return runAi(rest);
1217
1639
  case void 0:
1218
1640
  case "-h":
1219
1641
  case "--help":
1220
- console.log(USAGE5);
1642
+ console.log(USAGE6);
1221
1643
  return 0;
1222
1644
  default:
1223
1645
  console.error(`piwi: unknown command "${command}"
1224
1646
  `);
1225
- console.error(USAGE5);
1647
+ console.error(USAGE6);
1226
1648
  return 2;
1227
1649
  }
1228
1650
  }
@@ -81,6 +81,7 @@ var DEFAULTS = {
81
81
  streaming: true,
82
82
  streamingBatchSize: 5,
83
83
  streamingBatchDelay: 2e3,
84
+ failOnFlakyTests: false,
84
85
  username: null,
85
86
  password: null,
86
87
  apiKey: null,
@@ -100,6 +101,7 @@ var PIWI_ENV_KEYS = {
100
101
  streamingBatchSize: "PIWI_STREAMING_BATCH_SIZE",
101
102
  streamingBatchDelay: "PIWI_STREAMING_BATCH_DELAY",
102
103
  liveFileUploads: "PIWI_LIVE_FILE_UPLOADS",
104
+ failOnFlakyTests: "PIWI_FAIL_ON_FLAKY_TESTS",
103
105
  uploadTraces: "PIWI_UPLOAD_TRACES",
104
106
  uploadReport: "PIWI_UPLOAD_REPORT",
105
107
  captureLocators: "PIWI_CAPTURE_LOCATORS",
@@ -135,6 +137,7 @@ var ENV_FALLBACK_SPECS = [
135
137
  { option: "streamingBatchSize", env: PIWI_ENV_KEYS.streamingBatchSize, kind: "number" },
136
138
  { option: "streamingBatchDelay", env: PIWI_ENV_KEYS.streamingBatchDelay, kind: "number" },
137
139
  { option: "liveFileUploads", env: PIWI_ENV_KEYS.liveFileUploads, kind: "bool" },
140
+ { option: "failOnFlakyTests", env: PIWI_ENV_KEYS.failOnFlakyTests, kind: "bool" },
138
141
  { option: "uploadTraces", env: PIWI_ENV_KEYS.uploadTraces, kind: "bool" },
139
142
  { option: "uploadReport", env: PIWI_ENV_KEYS.uploadReport, kind: "bool" },
140
143
  { option: "captureLocators", env: PIWI_ENV_KEYS.captureLocators, kind: "bool" },
@@ -506,7 +509,7 @@ function createGlobalSetup(options, userSetup) {
506
509
  projectName: opts.projectName
507
510
  })
508
511
  );
509
- logger.debug(`Global setup: initialising run #${response.runId}`);
512
+ logger.debug(`Global setup: initializing run #${response.runId}`);
510
513
  }
511
514
  } catch (error) {
512
515
  logger.warn(`Could not register global setup: ${errorMessage(error)}`);
package/dist/index.d.ts CHANGED
@@ -88,6 +88,14 @@ interface PiwiDashboardOptions {
88
88
  streamingBatchSize?: number;
89
89
  /** Max delay (ms) before flushing pending events during streaming. Defaults to `2000`. */
90
90
  streamingBatchDelay?: number;
91
+ /**
92
+ * Fail the run when any test was flaky (passed only after a retry). Forwarded
93
+ * to Playwright's native `failOnFlakyTests` config option (Playwright 1.52+)
94
+ * when the reporter is installed via `wrapConfig`, so a flaky-only run exits
95
+ * non-zero without any server round-trip. Defaults to `false`. Can also be
96
+ * set with `PIWI_FAIL_ON_FLAKY_TESTS`.
97
+ */
98
+ failOnFlakyTests?: boolean;
91
99
  /** Username for dashboard login (use `apiKey` instead when possible) */
92
100
  username?: string | null;
93
101
  /** Password for dashboard login (used with `username`) */
@@ -239,6 +247,8 @@ declare class PiwiDashboardReporter {
239
247
  private plannedTests;
240
248
  /** Ids of tests that actually reported via `onTestEnd`, to find the ones that never ran. */
241
249
  private reportedTestIds;
250
+ /** Per-test attempt history, keyed by `test.id`; snapshotted onto every attempt's payload. */
251
+ private attemptsByTest;
242
252
  private instanceId;
243
253
  private runLabel;
244
254
  private shardInfo;
@@ -248,6 +258,8 @@ declare class PiwiDashboardReporter {
248
258
  private viaDesktopApp;
249
259
  private isFullRun;
250
260
  private filterDetails;
261
+ /** Configured `maxFailures` (0 = unlimited) — disambiguates an interrupted run's unrun reason. */
262
+ private maxFailures;
251
263
  private httpClient;
252
264
  private uploader;
253
265
  private fileHandler;
@@ -265,6 +277,12 @@ declare class PiwiDashboardReporter {
265
277
  onTestBegin(test: TestCase, result: TestResult): void;
266
278
  /** Track suite-level setup steps (beforeAll/afterAll) not tied to any test */
267
279
  private setupSteps;
280
+ /**
281
+ * Step categories streamed live while the run executes. `pw:assert` is
282
+ * excluded: it is the polling noise of `expect()`, not a step a human
283
+ * watches; the meaningful readout is the `pw:expect` wrapper around it.
284
+ */
285
+ private static readonly LIVE_STEP_CATEGORIES;
268
286
  /** Playwright reporter hook: called when a step (including hook/fixture) begins */
269
287
  onStepBegin(test: TestCase | undefined, _result: TestResult | undefined, step: any): void;
270
288
  /** Playwright reporter hook: called when a step (including hook/fixture) ends */
@@ -276,13 +294,44 @@ declare class PiwiDashboardReporter {
276
294
  * (no `onTestEnd`) — typically because `maxFailures` cut the run short. These
277
295
  * carry no result, so they're emitted with zero duration and no error. In
278
296
  * streaming mode they're queued as complete events so the pre-finish drain
279
- * sends them alongside the rest.
297
+ * sends them alongside the rest. The `reason` (global timeout / max failures /
298
+ * interrupted) is resolved once from the overall run status by the caller.
280
299
  */
281
300
  private materializeUnrunTests;
282
301
  /** Playwright reporter hook: called when the full test run finishes */
283
302
  onEnd(result: FullResult): Promise<void>;
284
303
  }
285
304
 
305
+ interface ResolveSelectionResult {
306
+ key: string;
307
+ version: number;
308
+ resolvedHash: string;
309
+ resolvedCount: number;
310
+ /** A `--grep` regex that selects the resolved tests, when the grep format was produced. */
311
+ grep?: string;
312
+ /** File (or `file:line`) tokens, when grep fell back to a file materialization. */
313
+ files?: string[];
314
+ }
315
+ interface ResolveSelectionOptions {
316
+ /** Selection key; defaults to the `PIWI_SELECTION` env var. */
317
+ key?: string;
318
+ serverUrl?: string;
319
+ apiKey?: string | null;
320
+ /** Project name or id; defaults to `PIWI_PROJECT_NAME`. */
321
+ project?: string;
322
+ budgetMs?: number | null;
323
+ /** Throw instead of falling back to the full suite when resolution fails. */
324
+ strict?: boolean;
325
+ env?: NodeJS.ProcessEnv;
326
+ }
327
+ /**
328
+ * Resolve the configured selection. Returns `undefined` when no selection is
329
+ * named or (in non-strict mode) when the dashboard cannot be reached, so the
330
+ * config runs the full suite. On success it also writes `PIWI_SELECTION_*` so
331
+ * the reporter stamps the run with the resolved selection.
332
+ */
333
+ declare function resolveSelection(options?: ResolveSelectionOptions): Promise<ResolveSelectionResult | undefined>;
334
+
286
335
  /**
287
336
  * Parameters are first-class: a template like `row for {name}` is the cache key,
288
337
  * so one entry serves every value. Placeholders survive compilation as `{{name}}`
@@ -345,4 +394,4 @@ declare function extendPiwiAi<TestArgs extends FixtureArgs, WorkerArgs extends F
345
394
  * exported from this file, it isn't part of the supported API.
346
395
  */
347
396
 
348
- export { type AiMode, type AiOnMiss, type PiwiAi, type PiwiDashboardOptions, PiwiDashboardReporter, createGlobalSetup, PiwiDashboardReporter as default, extendPiwiAi, piwiAiFixtures, wrapConfig };
397
+ export { type AiMode, type AiOnMiss, type PiwiAi, type PiwiDashboardOptions, PiwiDashboardReporter, type ResolveSelectionOptions, type ResolveSelectionResult, createGlobalSetup, PiwiDashboardReporter as default, extendPiwiAi, piwiAiFixtures, resolveSelection, wrapConfig };
package/dist/index.js CHANGED
@@ -37,6 +37,7 @@ __export(index_exports, {
37
37
  extendPiwiFixtures: () => extendPiwiFixtures,
38
38
  piwiAiFixtures: () => piwiAiFixtures,
39
39
  piwiFixtures: () => piwiFixtures,
40
+ resolveSelection: () => resolveSelection,
40
41
  wrapConfig: () => wrapConfig
41
42
  });
42
43
  module.exports = __toCommonJS(index_exports);
@@ -79,6 +80,7 @@ var DEFAULTS = {
79
80
  streaming: true,
80
81
  streamingBatchSize: 5,
81
82
  streamingBatchDelay: 2e3,
83
+ failOnFlakyTests: false,
82
84
  username: null,
83
85
  password: null,
84
86
  apiKey: null,
@@ -98,6 +100,7 @@ var PIWI_ENV_KEYS = {
98
100
  streamingBatchSize: "PIWI_STREAMING_BATCH_SIZE",
99
101
  streamingBatchDelay: "PIWI_STREAMING_BATCH_DELAY",
100
102
  liveFileUploads: "PIWI_LIVE_FILE_UPLOADS",
103
+ failOnFlakyTests: "PIWI_FAIL_ON_FLAKY_TESTS",
101
104
  uploadTraces: "PIWI_UPLOAD_TRACES",
102
105
  uploadReport: "PIWI_UPLOAD_REPORT",
103
106
  captureLocators: "PIWI_CAPTURE_LOCATORS",
@@ -116,6 +119,12 @@ var PIWI_ENV_KEYS = {
116
119
  aiScreenshotFallback: "PIWI_AI_SCREENSHOT_FALLBACK"
117
120
  };
118
121
  var PIWI_DESKTOP_CONFIG_ENV = "PIWI_DESKTOP_CONFIG";
122
+ var PIWI_SELECTION_ENV = {
123
+ key: "PIWI_SELECTION",
124
+ version: "PIWI_SELECTION_VERSION",
125
+ hash: "PIWI_SELECTION_HASH",
126
+ count: "PIWI_SELECTION_COUNT"
127
+ };
119
128
  function readBool(val) {
120
129
  if (val === void 0) return void 0;
121
130
  return val === "true";
@@ -133,6 +142,7 @@ var ENV_FALLBACK_SPECS = [
133
142
  { option: "streamingBatchSize", env: PIWI_ENV_KEYS.streamingBatchSize, kind: "number" },
134
143
  { option: "streamingBatchDelay", env: PIWI_ENV_KEYS.streamingBatchDelay, kind: "number" },
135
144
  { option: "liveFileUploads", env: PIWI_ENV_KEYS.liveFileUploads, kind: "bool" },
145
+ { option: "failOnFlakyTests", env: PIWI_ENV_KEYS.failOnFlakyTests, kind: "bool" },
136
146
  { option: "uploadTraces", env: PIWI_ENV_KEYS.uploadTraces, kind: "bool" },
137
147
  { option: "uploadReport", env: PIWI_ENV_KEYS.uploadReport, kind: "bool" },
138
148
  { option: "captureLocators", env: PIWI_ENV_KEYS.captureLocators, kind: "bool" },
@@ -430,8 +440,8 @@ function resolveOverallStatus(result, counters) {
430
440
  const STATUS_MAP = {
431
441
  passed: "passed",
432
442
  failed: "failed",
433
- timedout: "failed",
434
- interrupted: "failed"
443
+ timedout: "timedout",
444
+ interrupted: "interrupted"
435
445
  };
436
446
  if (result?.status) return STATUS_MAP[result.status] ?? "failed";
437
447
  if (counters.failedTests === 0 && counters.timedOutTests === 0 && counters.totalTests > 0) return "passed";
@@ -448,6 +458,7 @@ function toWireTestCase(tc) {
448
458
  timeout: rest.timeout ?? null,
449
459
  error: rest.error,
450
460
  retries: rest.retries,
461
+ attempts: rest.attempts || null,
451
462
  workerIndex: rest.workerIndex ?? null,
452
463
  shardIndex: rest.shardIndex ?? null,
453
464
  startedAt: rest.startedAt ?? null,
@@ -470,7 +481,9 @@ function toWireTestCase(tc) {
470
481
  testAnnotations: rest.testAnnotations ?? null,
471
482
  tags: rest.tags ?? null,
472
483
  testMeta: rest.testMeta ?? null,
473
- locatorSnapshots: rest.locatorSnapshots || null
484
+ locatorSnapshots: rest.locatorSnapshots || null,
485
+ didNotRunReason: rest.didNotRunReason ?? null,
486
+ blockedBy: rest.blockedBy ?? null
474
487
  };
475
488
  }
476
489
  function serializeRun(payload, opts) {
@@ -523,8 +536,8 @@ var Uploader = class {
523
536
  auth
524
537
  );
525
538
  this.logger.info(`Successfully uploaded test results`);
526
- if (response.testRunId) {
527
- this.logger.info(`Test Run ID: ${response.testRunId}, Project ID: ${response.projectId}`);
539
+ if (response.runId) {
540
+ this.logger.info(`Test Run ID: ${response.runId}, Project ID: ${response.projectId}`);
528
541
  }
529
542
  return response;
530
543
  }
@@ -538,8 +551,8 @@ var Uploader = class {
538
551
  await this.appendFilesToForm(form, payload.testCases, reportOptions.uploadTraces);
539
552
  const response = await this.httpClient.postFormData("/api/test-runs/upload", form, auth);
540
553
  this.logger.info(`Successfully uploaded test results with files`);
541
- if (response.testRunId) {
542
- this.logger.info(`Test Run ID: ${response.testRunId}, Project ID: ${response.projectId}`);
554
+ if (response.runId) {
555
+ this.logger.info(`Test Run ID: ${response.runId}, Project ID: ${response.projectId}`);
543
556
  }
544
557
  if (response.reports) {
545
558
  for (const r of response.reports) this.logger.info(`${r.label}: ${r.path}`);
@@ -549,7 +562,7 @@ var Uploader = class {
549
562
  /** Upload report files for an already-submitted streaming run */
550
563
  async uploadReportsForStreamingRun(projectName, runId, reportOptions, startTime, auth) {
551
564
  const form = new import_form_data2.default();
552
- form.append("testRunId", String(runId));
565
+ form.append("runId", String(runId));
553
566
  form.append("projectName", projectName);
554
567
  form.append(
555
568
  "testRun",
@@ -1012,6 +1025,39 @@ var FileHandler = class {
1012
1025
 
1013
1026
  // src/internal/collect/metadata-collector.ts
1014
1027
  var import_node_child_process = require("child_process");
1028
+ var DETACHED_HEAD = "HEAD";
1029
+ function resolveScmBranch(env, gitBranch) {
1030
+ const pick = (...names) => {
1031
+ for (const name of names) {
1032
+ const value = env[name]?.trim();
1033
+ if (value) return value;
1034
+ }
1035
+ return void 0;
1036
+ };
1037
+ const override = env.PIWI_BRANCH?.trim();
1038
+ if (override) return override;
1039
+ let provider;
1040
+ if (env.JENKINS_URL) provider = pick("CHANGE_BRANCH", "BRANCH_NAME");
1041
+ else if (env.GITHUB_ACTIONS) provider = pick("GITHUB_HEAD_REF", "GITHUB_REF_NAME");
1042
+ else if (env.GITLAB_CI) provider = pick("CI_MERGE_REQUEST_SOURCE_BRANCH_NAME", "CI_COMMIT_REF_NAME");
1043
+ else if (env.CIRCLECI) provider = pick("CIRCLE_BRANCH");
1044
+ else if (env.TRAVIS) provider = pick("TRAVIS_PULL_REQUEST_BRANCH", "TRAVIS_BRANCH");
1045
+ else if (env.TF_BUILD) provider = pick("SYSTEM_PULLREQUEST_SOURCEBRANCH", "BUILD_SOURCEBRANCHNAME");
1046
+ else if (env.BITBUCKET_BUILD_NUMBER) provider = pick("BITBUCKET_BRANCH");
1047
+ if (provider) return normalizeRef(provider);
1048
+ const trimmed = gitBranch?.trim();
1049
+ if (trimmed && trimmed !== DETACHED_HEAD) return trimmed;
1050
+ return void 0;
1051
+ }
1052
+ function resolveScmPrNumber(env) {
1053
+ const githubRef = env.GITHUB_ACTIONS ? env.GITHUB_REF?.match(/^refs\/pull\/(\d+)\/(?:merge|head)$/) : void 0;
1054
+ const raw = githubRef?.[1] ?? env.CI_MERGE_REQUEST_IID ?? env.BITBUCKET_PR_ID ?? env.SYSTEM_PULLREQUEST_PULLREQUESTNUMBER ?? env.CHANGE_ID;
1055
+ const trimmed = raw?.trim();
1056
+ return trimmed && /^\d+$/.test(trimmed) ? trimmed : void 0;
1057
+ }
1058
+ function normalizeRef(ref) {
1059
+ return ref.replace(/^refs\/heads\//, "");
1060
+ }
1015
1061
  var MetadataCollector = class {
1016
1062
  constructor(logger = new Logger()) {
1017
1063
  this.logger = logger;
@@ -1117,6 +1163,7 @@ var MetadataCollector = class {
1117
1163
  }
1118
1164
  collectScmInfo(_options) {
1119
1165
  const scm = {};
1166
+ let gitBranch;
1120
1167
  try {
1121
1168
  const execOpts = {
1122
1169
  encoding: "utf8",
@@ -1125,7 +1172,7 @@ var MetadataCollector = class {
1125
1172
  stdio: ["ignore", "pipe", "ignore"]
1126
1173
  };
1127
1174
  scm.commit = (0, import_node_child_process.execSync)("git rev-parse HEAD", execOpts).trim();
1128
- scm.branch = (0, import_node_child_process.execSync)("git rev-parse --abbrev-ref HEAD", execOpts).trim();
1175
+ gitBranch = (0, import_node_child_process.execSync)("git rev-parse --abbrev-ref HEAD", execOpts).trim();
1129
1176
  scm.author = (0, import_node_child_process.execSync)('git log -1 --pretty=format:"%an"', execOpts).trim();
1130
1177
  scm.commitMessage = (0, import_node_child_process.execSync)('git log -1 --pretty=format:"%s"', execOpts).trim();
1131
1178
  try {
@@ -1135,6 +1182,10 @@ var MetadataCollector = class {
1135
1182
  } catch (error) {
1136
1183
  this.logger.debug(`Git info not available: ${errorMessage(error)}`);
1137
1184
  }
1185
+ const branch = resolveScmBranch(process.env, gitBranch);
1186
+ if (branch) scm.branch = branch;
1187
+ const prNumber = resolveScmPrNumber(process.env);
1188
+ if (prNumber) scm.prNumber = prNumber;
1138
1189
  return Object.keys(scm).length > 0 ? scm : void 0;
1139
1190
  }
1140
1191
  collectCiInfo() {
@@ -1421,8 +1472,8 @@ var StreamManager = class {
1421
1472
  if (this.pendingBeginEvents.length > 0) {
1422
1473
  this.pendingEvents = [...this.pendingBeginEvents, ...this.pendingEvents];
1423
1474
  this.pendingBeginEvents = [];
1424
- this.flush();
1425
1475
  }
1476
+ if (this.pendingEvents.length > 0) this.flush();
1426
1477
  }
1427
1478
  } catch (error) {
1428
1479
  if (error instanceof HttpError && (error.status === 401 || error.status === 403)) {
@@ -1934,6 +1985,17 @@ function detectCliFileFilters(argv = process.argv) {
1934
1985
  return files;
1935
1986
  }
1936
1987
 
1988
+ // src/internal/support/selection-env.ts
1989
+ function readSelectionStamp(env = process.env) {
1990
+ const key = env[PIWI_SELECTION_ENV.key];
1991
+ if (!key) return null;
1992
+ const version = Number(env[PIWI_SELECTION_ENV.version]);
1993
+ const resolvedCount = Number(env[PIWI_SELECTION_ENV.count]);
1994
+ const resolvedHash = env[PIWI_SELECTION_ENV.hash] ?? "";
1995
+ if (!Number.isFinite(version) || !Number.isFinite(resolvedCount) || !resolvedHash) return null;
1996
+ return { key, version, resolvedHash, resolvedCount };
1997
+ }
1998
+
1937
1999
  // src/public/global-setup.ts
1938
2000
  var path10 = __toESM(require("path"));
1939
2001
  var fs11 = __toESM(require("fs"));
@@ -2024,7 +2086,7 @@ function createGlobalSetup(options, userSetup) {
2024
2086
  projectName: opts.projectName
2025
2087
  })
2026
2088
  );
2027
- logger.debug(`Global setup: initialising run #${response.runId}`);
2089
+ logger.debug(`Global setup: initializing run #${response.runId}`);
2028
2090
  }
2029
2091
  } catch (error) {
2030
2092
  logger.warn(`Could not register global setup: ${errorMessage(error)}`);
@@ -2067,8 +2129,12 @@ function wrapConfig(config, piwiOptions) {
2067
2129
  globalSetupModules.push(...orig);
2068
2130
  }
2069
2131
  globalSetupModules.push(resolveSetupModule());
2132
+ const forwarded = {};
2133
+ const failOnFlaky = piwiOptions?.failOnFlakyTests ?? readBool(process.env[PIWI_ENV_KEYS.failOnFlakyTests]);
2134
+ if (failOnFlaky === true) forwarded.failOnFlakyTests = true;
2070
2135
  return {
2071
2136
  ...config,
2137
+ ...forwarded,
2072
2138
  reporter: injectReporter(config.reporter, piwiOptions),
2073
2139
  globalSetup: globalSetupModules.length === 1 ? globalSetupModules[0] : globalSetupModules
2074
2140
  };
@@ -2093,6 +2159,45 @@ function classifyStatus(rawStatus, annotations) {
2093
2159
  const intentional = annotations.some((a) => a.type === "skip" || a.type === "fixme");
2094
2160
  return intentional ? "skipped" : "didnotrun";
2095
2161
  }
2162
+ function resolveUnrunReason(runStatus, opts) {
2163
+ if (runStatus === "timedout") return "global-timeout";
2164
+ if (opts.maxFailures > 0 && opts.failures >= opts.maxFailures) return "max-failures";
2165
+ return "interrupted";
2166
+ }
2167
+ function fileOf(location) {
2168
+ return location.replace(/:\d+:\d+$/, "");
2169
+ }
2170
+ function suiteKey(suitePath) {
2171
+ return (suitePath ?? []).join("");
2172
+ }
2173
+ function sharedPrefixDepth(a, b) {
2174
+ let depth = 0;
2175
+ while (depth < a.length && depth < b.length && a[depth] === b[depth]) depth++;
2176
+ return depth;
2177
+ }
2178
+ function linkBlockedTests(cases) {
2179
+ const blockers = cases.filter((c) => c.status === "failed" || c.status === "timedOut" || c.status === "timedout");
2180
+ if (blockers.length === 0) return;
2181
+ for (const c of cases) {
2182
+ if (c.didNotRunReason !== "previous-failure") continue;
2183
+ const file = fileOf(c.location);
2184
+ const inFile = blockers.filter((b) => fileOf(b.location) === file);
2185
+ if (inFile.length === 0) continue;
2186
+ const key = suiteKey(c.suitePath);
2187
+ let blocker = inFile.find((b) => suiteKey(b.suitePath) === key);
2188
+ if (!blocker) {
2189
+ let bestDepth = 0;
2190
+ for (const b of inFile) {
2191
+ const depth = sharedPrefixDepth(c.suitePath ?? [], b.suitePath ?? []);
2192
+ if (depth > bestDepth) {
2193
+ bestDepth = depth;
2194
+ blocker = b;
2195
+ }
2196
+ }
2197
+ }
2198
+ if (blocker) c.blockedBy = blocker.location;
2199
+ }
2200
+ }
2096
2201
 
2097
2202
  // ../core/src/test-meta.ts
2098
2203
  var PIWI_ANNOTATION_PREFIX = "piwi:";
@@ -2462,7 +2567,7 @@ var RunSubmitter = class {
2462
2567
  auth
2463
2568
  );
2464
2569
  this.recovery.clear();
2465
- return { done: true, output: this.buildOutput(response?.testRunId, response?.projectId, run, overallStatus) };
2570
+ return { done: true, output: this.buildOutput(response?.runId, response?.projectId, run, overallStatus) };
2466
2571
  } catch (error) {
2467
2572
  if (error instanceof HttpError && error.status === 401 && !auth) {
2468
2573
  this.logAuthRequired(run.options.serverUrl);
@@ -2478,7 +2583,7 @@ var RunSubmitter = class {
2478
2583
  try {
2479
2584
  const response = await this.uploader.uploadJSON(payload, auth);
2480
2585
  this.recovery.clear();
2481
- return { done: true, output: this.buildOutput(response?.testRunId, response?.projectId, run, overallStatus) };
2586
+ return { done: true, output: this.buildOutput(response?.runId, response?.projectId, run, overallStatus) };
2482
2587
  } catch (error) {
2483
2588
  if (error instanceof HttpError && error.status === 401 && !auth) {
2484
2589
  this.logAuthRequired(run.options.serverUrl);
@@ -2505,7 +2610,7 @@ function testLocation(test) {
2505
2610
  const relativeFilePath = path14.relative(process.cwd(), test.location.file).split(path14.sep).join("/");
2506
2611
  return `${relativeFilePath}:${test.location.line}:${test.location.column}`;
2507
2612
  }
2508
- var PiwiDashboardReporter = class {
2613
+ var PiwiDashboardReporter = class _PiwiDashboardReporter {
2509
2614
  constructor(rawOptions = {}) {
2510
2615
  this.testCases = [];
2511
2616
  this.startTime = null;
@@ -2521,11 +2626,15 @@ var PiwiDashboardReporter = class {
2521
2626
  this.plannedTests = [];
2522
2627
  /** Ids of tests that actually reported via `onTestEnd`, to find the ones that never ran. */
2523
2628
  this.reportedTestIds = /* @__PURE__ */ new Set();
2629
+ /** Per-test attempt history, keyed by `test.id`; snapshotted onto every attempt's payload. */
2630
+ this.attemptsByTest = /* @__PURE__ */ new Map();
2524
2631
  this.runLabel = null;
2525
2632
  this.shardInfo = null;
2526
2633
  this.metadata = {};
2527
2634
  this.isFullRun = true;
2528
2635
  this.filterDetails = null;
2636
+ /** Configured `maxFailures` (0 = unlimited) — disambiguates an interrupted run's unrun reason. */
2637
+ this.maxFailures = 0;
2529
2638
  this.streamManager = null;
2530
2639
  /** Track suite-level setup steps (beforeAll/afterAll) not tied to any test */
2531
2640
  this.setupSteps = [];
@@ -2573,6 +2682,7 @@ var PiwiDashboardReporter = class {
2573
2682
  }
2574
2683
  this.startTime = (/* @__PURE__ */ new Date()).toISOString();
2575
2684
  this.playwrightVersion = config.version;
2685
+ this.maxFailures = config.maxFailures ?? 0;
2576
2686
  this.logger.info(
2577
2687
  `Starting test run for project: ${this.options.projectName} (Playwright v${this.playwrightVersion})`
2578
2688
  );
@@ -2582,14 +2692,18 @@ var PiwiDashboardReporter = class {
2582
2692
  const grep = grepRe && grepRe.source !== ".*" ? grepRe.source : void 0;
2583
2693
  const grepInvert = grepInvertRe?.source;
2584
2694
  const fileFilters = detectCliFileFilters();
2585
- if (grep || grepInvert || fileFilters.length > 0) {
2695
+ const selection = readSelectionStamp();
2696
+ if (grep || grepInvert || fileFilters.length > 0 || selection) {
2586
2697
  this.isFullRun = false;
2587
2698
  this.filterDetails = {
2588
2699
  ...grep ? { grep } : {},
2589
2700
  ...grepInvert ? { grepInvert } : {},
2590
- ...fileFilters.length > 0 ? { files: fileFilters } : {}
2701
+ ...fileFilters.length > 0 ? { files: fileFilters } : {},
2702
+ ...selection ? { selection } : {}
2591
2703
  };
2592
- this.logger.info("Partial run detected (filter active)");
2704
+ this.logger.info(
2705
+ selection ? `Selection run detected (${selection.key})` : "Partial run detected (filter active)"
2706
+ );
2593
2707
  }
2594
2708
  this.metadata = this.metadataCollector.collect(config, suite, this.options);
2595
2709
  this.plannedTests = suite.allTests();
@@ -2626,11 +2740,19 @@ var PiwiDashboardReporter = class {
2626
2740
  this.streamManager.queueBeginEvent(toWireTestCase(beginEvent));
2627
2741
  }
2628
2742
  }
2743
+ static {
2744
+ /**
2745
+ * Step categories streamed live while the run executes. `pw:assert` is
2746
+ * excluded: it is the polling noise of `expect()`, not a step a human
2747
+ * watches; the meaningful readout is the `pw:expect` wrapper around it.
2748
+ */
2749
+ this.LIVE_STEP_CATEGORIES = /* @__PURE__ */ new Set(["hook", "fixture", "pw:api", "pw:expect"]);
2750
+ }
2629
2751
  /** Playwright reporter hook: called when a step (including hook/fixture) begins */
2630
2752
  onStepBegin(test, _result, step) {
2631
- if (!this.enabled || !this.streamManager?.enabled) return;
2753
+ if (!this.enabled || !this.streamManager) return;
2632
2754
  const cat = step.category;
2633
- if (cat !== "hook" && cat !== "fixture") return;
2755
+ if (!_PiwiDashboardReporter.LIVE_STEP_CATEGORIES.has(cat)) return;
2634
2756
  const event = {
2635
2757
  type: "step-begin",
2636
2758
  title: step.title,
@@ -2644,10 +2766,9 @@ var PiwiDashboardReporter = class {
2644
2766
  }
2645
2767
  /** Playwright reporter hook: called when a step (including hook/fixture) ends */
2646
2768
  onStepEnd(test, _result, step) {
2769
+ if (!this.enabled || !this.streamManager) return;
2647
2770
  const cat = step.category;
2648
- if (cat === "pw:api") return;
2649
- if (!this.enabled || !this.streamManager?.enabled) return;
2650
- if (cat !== "hook" && cat !== "fixture") return;
2771
+ if (!_PiwiDashboardReporter.LIVE_STEP_CATEGORIES.has(cat)) return;
2651
2772
  const workerIndex = workerIndexOf(_result);
2652
2773
  const startedAt = step.startTime instanceof Date ? step.startTime.getTime() : null;
2653
2774
  const event = {
@@ -2662,7 +2783,7 @@ var PiwiDashboardReporter = class {
2662
2783
  startedAt
2663
2784
  };
2664
2785
  this.streamManager?.queueEvent(event);
2665
- if (!test && startedAt) {
2786
+ if (!test && startedAt && (cat === "hook" || cat === "fixture")) {
2666
2787
  this.setupSteps.push({
2667
2788
  title: step.title,
2668
2789
  category: cat,
@@ -2682,6 +2803,14 @@ var PiwiDashboardReporter = class {
2682
2803
  const annotations = mergeAnnotations(test, result);
2683
2804
  const status = classifyStatus(result.status, annotations);
2684
2805
  const tags = collectTestTags(test);
2806
+ const attempts = this.attemptsByTest.get(test.id) ?? [];
2807
+ attempts.push({
2808
+ retry: result.retry,
2809
+ status,
2810
+ duration: result.duration,
2811
+ startedAt: result.startTime ? result.startTime.getTime() : null
2812
+ });
2813
+ this.attemptsByTest.set(test.id, attempts);
2685
2814
  const testCase = {
2686
2815
  type: "complete",
2687
2816
  title: test.title,
@@ -2693,6 +2822,7 @@ var PiwiDashboardReporter = class {
2693
2822
  timeout: test.timeout ?? null,
2694
2823
  error: buildErrorText(result),
2695
2824
  retries: result.retry,
2825
+ attempts: attempts.map((a) => ({ ...a })),
2696
2826
  workerIndex: workerIndexOf(result),
2697
2827
  shardIndex: this.shardInfo?.current ?? null,
2698
2828
  startedAt: result.startTime ? result.startTime.getTime() : null,
@@ -2702,7 +2832,10 @@ var PiwiDashboardReporter = class {
2702
2832
  suiteConfig,
2703
2833
  testAnnotations: annotations.length ? annotations : null,
2704
2834
  tags: tags.length ? tags : null,
2705
- testMeta: collectTestMetadata(annotations)
2835
+ testMeta: collectTestMetadata(annotations),
2836
+ // An annotation-less skip reclassified to `didnotrun` is a serial-group
2837
+ // cascade: an earlier test failed and Playwright skipped the rest.
2838
+ didNotRunReason: status === "didnotrun" ? "previous-failure" : null
2706
2839
  };
2707
2840
  if (result.status === "failed" || result.status === "timedOut") {
2708
2841
  const failingLine = extractFailingLine(testCase.error, test.location.file, test.location.line);
@@ -2746,6 +2879,7 @@ var PiwiDashboardReporter = class {
2746
2879
  break;
2747
2880
  }
2748
2881
  this.testCases.push(testCase);
2882
+ if (status === "didnotrun") linkBlockedTests(this.testCases);
2749
2883
  if (this.streamManager) {
2750
2884
  this.streamManager.queueEvent(toWireTestCase(testCase));
2751
2885
  if (this.options.liveFileUploads) this.streamManager.scheduleLiveUpload(testCase);
@@ -2756,9 +2890,10 @@ var PiwiDashboardReporter = class {
2756
2890
  * (no `onTestEnd`) — typically because `maxFailures` cut the run short. These
2757
2891
  * carry no result, so they're emitted with zero duration and no error. In
2758
2892
  * streaming mode they're queued as complete events so the pre-finish drain
2759
- * sends them alongside the rest.
2893
+ * sends them alongside the rest. The `reason` (global timeout / max failures /
2894
+ * interrupted) is resolved once from the overall run status by the caller.
2760
2895
  */
2761
- materializeUnrunTests() {
2896
+ materializeUnrunTests(reason) {
2762
2897
  for (const test of this.plannedTests) {
2763
2898
  if (this.reportedTestIds.has(test.id)) continue;
2764
2899
  const { suitePath, suiteConfig } = this.metadataCollector.getSuiteInfo(test);
@@ -2782,7 +2917,8 @@ var PiwiDashboardReporter = class {
2782
2917
  suiteConfig,
2783
2918
  testAnnotations: declaredAnnotations.length ? declaredAnnotations : null,
2784
2919
  tags: tags.length ? tags : null,
2785
- testMeta: collectTestMetadata(declaredAnnotations)
2920
+ testMeta: collectTestMetadata(declaredAnnotations),
2921
+ didNotRunReason: reason
2786
2922
  };
2787
2923
  this.testCases.push(testCase);
2788
2924
  this.totalTests++;
@@ -2795,7 +2931,11 @@ var PiwiDashboardReporter = class {
2795
2931
  /** Playwright reporter hook: called when the full test run finishes */
2796
2932
  async onEnd(result) {
2797
2933
  if (!this.enabled) return;
2798
- this.materializeUnrunTests();
2934
+ const unrunReason = resolveUnrunReason(result?.status, {
2935
+ maxFailures: this.maxFailures,
2936
+ failures: this.failedTests + this.timedOutTests
2937
+ });
2938
+ this.materializeUnrunTests(unrunReason);
2799
2939
  await this.submitter.submit(
2800
2940
  {
2801
2941
  options: this.options,
@@ -2821,6 +2961,79 @@ var PiwiDashboardReporter = class {
2821
2961
  }
2822
2962
  };
2823
2963
 
2964
+ // src/internal/support/selection-client.ts
2965
+ function authHeaders(apiKey) {
2966
+ return apiKey ? { "X-API-Key": apiKey } : {};
2967
+ }
2968
+ async function resolveProjectId(options) {
2969
+ if (!options.project) throw new Error("No project \u2014 pass --project or set PIWI_PROJECT_NAME");
2970
+ if (/^\d+$/.test(options.project)) return Number(options.project);
2971
+ const res = await fetch(`${options.serverUrl}/api/projects/menu`, { headers: authHeaders(options.apiKey) });
2972
+ if (!res.ok) throw new Error(`Could not list projects (dashboard returned ${res.status})`);
2973
+ const body = await res.json();
2974
+ const match = body.items?.find((p) => p.name.toLowerCase() === options.project.toLowerCase());
2975
+ if (!match) throw new Error(`No project named "${options.project}" on this dashboard`);
2976
+ return match.id;
2977
+ }
2978
+ async function fetchResolution(options, projectId) {
2979
+ const params = new URLSearchParams({ format: options.format ?? "args" });
2980
+ if (options.pkgRunner) params.set("pkgRunner", options.pkgRunner);
2981
+ if (options.budgetMs != null) params.set("budgetMs", String(options.budgetMs));
2982
+ if (options.shard) params.set("shard", options.shard);
2983
+ if (options.order) params.set("order", options.order);
2984
+ const url = `${options.serverUrl}/api/projects/${projectId}/selections/${encodeURIComponent(options.key)}/resolve?${params}`;
2985
+ const res = await fetch(url, { headers: authHeaders(options.apiKey) });
2986
+ if (!res.ok) {
2987
+ const body = await res.json().catch(() => ({}));
2988
+ throw new Error(body.message || `Dashboard returned ${res.status} resolving "${options.key}"`);
2989
+ }
2990
+ return await res.json();
2991
+ }
2992
+
2993
+ // src/public/selection.ts
2994
+ async function resolveSelection(options = {}) {
2995
+ const env = options.env ?? process.env;
2996
+ const key = options.key ?? env[PIWI_SELECTION_ENV.key];
2997
+ if (!key) return void 0;
2998
+ const serverUrl = (options.serverUrl ?? env[PIWI_ENV_KEYS.serverUrl] ?? "").replace(/\/$/, "");
2999
+ const apiKey = options.apiKey ?? env[PIWI_ENV_KEYS.apiKey] ?? null;
3000
+ const project = options.project ?? env[PIWI_ENV_KEYS.projectName] ?? "";
3001
+ if (!serverUrl) {
3002
+ if (options.strict) throw new Error("resolveSelection: no dashboard URL (set PIWI_DASHBOARD_URL)");
3003
+ return void 0;
3004
+ }
3005
+ const clientOptions = { serverUrl, apiKey, project, key, format: "grep", budgetMs: options.budgetMs ?? null };
3006
+ try {
3007
+ const projectId = await resolveProjectId(clientOptions);
3008
+ const resolution = await fetchResolution(clientOptions, projectId);
3009
+ if (resolution.estimate.count === 0) {
3010
+ if (options.strict) throw new Error(`resolveSelection: "${key}" resolved to 0 tests`);
3011
+ console.warn(`[Piwi Dashboard] selection "${key}" resolved to 0 tests \u2014 running the full suite`);
3012
+ return void 0;
3013
+ }
3014
+ env[PIWI_SELECTION_ENV.key] = resolution.key ?? key;
3015
+ env[PIWI_SELECTION_ENV.version] = String(resolution.version ?? 0);
3016
+ env[PIWI_SELECTION_ENV.hash] = resolution.resolvedHash;
3017
+ env[PIWI_SELECTION_ENV.count] = String(resolution.estimate.count);
3018
+ const args = resolution.materialization.args;
3019
+ const result = {
3020
+ key: resolution.key ?? key,
3021
+ version: resolution.version ?? 0,
3022
+ resolvedHash: resolution.resolvedHash,
3023
+ resolvedCount: resolution.estimate.count
3024
+ };
3025
+ if (args[0] === "--grep") result.grep = args[1];
3026
+ else result.files = args;
3027
+ return result;
3028
+ } catch (e) {
3029
+ if (options.strict) throw e;
3030
+ console.warn(
3031
+ `[Piwi Dashboard] selection "${key}" could not be resolved \u2014 running the full suite (${e.message})`
3032
+ );
3033
+ return void 0;
3034
+ }
3035
+ }
3036
+
2824
3037
  // src/internal/capture/capture-fixtures.ts
2825
3038
  var import_node_zlib = require("zlib");
2826
3039
 
@@ -6362,5 +6575,6 @@ var index_default = PiwiDashboardReporter;
6362
6575
  extendPiwiFixtures,
6363
6576
  piwiAiFixtures,
6364
6577
  piwiFixtures,
6578
+ resolveSelection,
6365
6579
  wrapConfig
6366
6580
  });
package/package.json CHANGED
@@ -1,9 +1,9 @@
1
1
  {
2
2
  "name": "@piwitests/reporter",
3
- "version": "0.25.0",
3
+ "version": "0.26.1",
4
4
  "description": "Playwright reporter that streams results, traces and HTML reports to a Piwi Dashboard instance",
5
5
  "url": "https://github.com/PiwiTests/platform",
6
- "homepage": "https://piwitests.github.io",
6
+ "homepage": "https://piwitests.dev",
7
7
  "repository": {
8
8
  "type": "git",
9
9
  "url": "https://github.com/PiwiTests/platform"
@@ -5,7 +5,7 @@ description: Investigate a failed test run recorded in Piwi Dashboard and propos
5
5
 
6
6
  # Investigate a Piwi failure
7
7
 
8
- Turn a failed run in [Piwi Dashboard](https://piwitests.github.io) into a grounded diagnosis and a concrete fix. Piwi has already gathered the evidence — the error text, the steps that ran, console output, failing network calls, a suggested locator, and the source diff since the last passing run. Use that instead of guessing.
8
+ Turn a failed run in [Piwi Dashboard](https://piwitests.dev) into a grounded diagnosis and a concrete fix. Piwi has already gathered the evidence — the error text, the steps that ran, console output, failing network calls, a suggested locator, and the source diff since the last passing run. Use that instead of guessing.
9
9
 
10
10
  ## How you reach Piwi
11
11
 
@@ -0,0 +1,38 @@
1
+ ---
2
+ name: run-the-right-tests
3
+ description: Pick and run the right subset of Playwright tests using Piwi's data-driven selections, instead of always running the whole suite. Use when the user asks to "run smoke tests", "run the right tests", "just run what's relevant", "verify this fix", or wants a fast, targeted test loop rather than the full run.
4
+ ---
5
+
6
+ # Run the right tests with Piwi selections
7
+
8
+ Running the whole suite for every change is slow, and hand-maintained grep patterns go stale. A Piwi **selection** is a named, declarative subset of the suite that Piwi resolves from run history — smoke, critical-and-stable, recently-broken, the best five minutes. This skill picks the selection that fits the task and runs exactly those tests.
9
+
10
+ ## How you reach Piwi
11
+
12
+ Prefer the **Piwi MCP server** if it is connected (`list_selections`, `resolve_selection`, `preview_selection`). Otherwise use the reporter CLI (`npx @piwitests/reporter select|run`) with `PIWI_DASHBOARD_URL` / `PIWI_API_KEY` / `PIWI_PROJECT_NAME` set, or the dashboard's **Selections** tab.
13
+
14
+ ## Steps
15
+
16
+ 1. **See what selections exist.** Call `list_selections` for the project. Every project has the built-ins `failed` (tests whose latest run failed) and `quarantine-free`, plus any the team saved (often `smoke`). Pick the one that matches the intent:
17
+ - About to make a broad change, or want a quick confidence check → `smoke` (or `preview_selection` with `{ "include": [{ "tags": ["smoke"] }] }`).
18
+ - Iterating on a fix → `failed`, or a `{ "include": [{ "failedInLastRuns": 5 }] }` preview.
19
+ - Time-boxed → resolve any selection with a `budgetMs` (e.g. 300000 for five minutes).
20
+
21
+ 2. **Resolve it to a command.** Call `resolve_selection` with the key (and optional `budgetMs`). You get back the matching tests, an estimate, and a ready-to-run `playwright test` command. Check the count and warnings first — a selection that matches **zero** tests is a red flag (too narrow, or nothing qualifies), and a `quarantined-included` warning means a flaky test is in the set.
22
+
23
+ 3. **Run it.** The most reliable way is the reporter CLI, which resolves and runs in one step and stamps the run so the dashboard names the subset:
24
+ ```
25
+ npx @piwitests/reporter run <key>
26
+ ```
27
+ Or run the materialized command `resolve_selection` returned directly with `playwright test`. Pass extra Playwright args after `--` (e.g. `-- --workers=4`).
28
+
29
+ 4. **After a fix, verify with the same selection.** Re-run the selection that covered the failure and confirm it now passes. In CI, `npx @piwitests/reporter gate --require-selection <key>` fails the build if any test the selection currently matches did not run or did not pass — catching a smoke job that silently shrank.
30
+
31
+ 5. **Report.** State which selection you ran, how many tests it resolved to, and the outcome. If no saved selection fit, suggest one (a `preview_selection` definition the team could save) rather than falling back to the whole suite silently.
32
+
33
+ ## Guardrails
34
+
35
+ - A selection that resolves to **zero** tests is never "nothing to do" — it means the definition is wrong. Stop and say so.
36
+ - Selections choose what to *run*; they never hide a failure. Quarantine is the tool for a test whose verdict should not block a merge — do not use a selection to route around a red test.
37
+ - Don't invent a selection key. Use `list_selections` to see the real ones, and `preview_selection` for an ad-hoc subset.
38
+ - The full suite stays the baseline. Selections are for the fast loops between full runs, not a permanent replacement.
@@ -5,7 +5,7 @@ description: Wire a Playwright project up to a Piwi Dashboard — install the re
5
5
 
6
6
  # Set up Piwi in a Playwright project
7
7
 
8
- Connect a Playwright test suite to a [Piwi Dashboard](https://piwitests.github.io) so every run is uploaded, kept, and analyzed. The mechanical work is done by a deterministic command; your job is to gather the right inputs, run it, finish anything it flags, and prove a run reaches the dashboard.
8
+ Connect a Playwright test suite to a [Piwi Dashboard](https://piwitests.dev) so every run is uploaded, kept, and analyzed. The mechanical work is done by a deterministic command; your job is to gather the right inputs, run it, finish anything it flags, and prove a run reaches the dashboard.
9
9
 
10
10
  ## Before you start
11
11