@piwitests/reporter 0.60.0 → 0.62.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@piwitests/reporter",
-  "version": "0.60.0",
+  "version": "0.62.0",
   "description": "Playwright reporter that streams results, traces and HTML reports to a Piwi Dashboard instance",
   "url": "https://github.com/PiwiTests/piwi",
   "homepage": "https://piwitests.dev",

package/templates/skills/apply-locator-healing/SKILL.md

@@ -15,7 +15,7 @@ Prefer the **Piwi MCP server** if it is connected (`get_locator_healing`, `expla
 
 1. **Find the failing case.** From a run the user names, or the latest failed run (`list_recent_activity` → `get_run` with a failed filter). Identify the case whose failure is a locator that matched nothing (a timeout on `click` / `fill` / `expect(locator)`).
 
-2. **Get the healed locator.** Call `get_locator_healing` for that case. It returns the **recommended durable locator** plus ranked alternatives, each stamped with the **call site** (file and line) where the original locator was used. Read the call site — that is the exact spot to edit.
+2. **Get the healed locator.** Call `get_locator_healing` for that case. It returns the **recommended durable locator** plus ranked alternatives, each stamped with the **call site** (file and line) where the original locator was used, and a ready-to-apply `edit`. Read the call site — that is the exact spot to edit. An answer with `applicable: false` means the locator resolved and the failure came after it: do not rewrite the selector.
 
 3. **Confirm it's really a locator problem.** The suggestion is only right if the element still exists under a new selector. Skim the error and the ARIA snapshot (`get_test_run_case` / `explain_failure`): if the element is genuinely gone or the page errored before rendering, this is not a healing case — hand it to the `investigate-failure` skill instead.
 
@@ -25,7 +25,7 @@ Prefer the **Piwi MCP server** if it is connected (`get_locator_healing`, `expla
 
 6. **Report.** Show the before/after locator, the file and line, and the verifying result. If several tests shared the selector, list every place you changed.
 
-7. **Report back to Piwi.** Commit with the trailer `Piwi-Cluster: <clusterId>` as the last line of the message (the cluster of the failing case; `get_fix_plan` returns the line in `verify.commitTrailer`), then call `report_fix_attempt` with that cluster, `kind: "locator-edit"`, the commit or the branch, and the `edit` (`filePath`, `line`, `from`: the old locator, `to`: the one you applied). Do it once per call site you changed. Piwi records the attempt verified when the test passes on that commit, and counts the healing suggestion as adopted.
+7. **Report back to Piwi.** Commit with the trailer `Piwi-Cluster: <clusterId>` as the last line of the message (the cluster of the failing case; `get_fix_plan` returns the line in `verify.commitTrailer`), then call `report_fix_attempt` with that cluster, `kind: "locator-edit"`, the commit or the branch, and the `edit` (`filePath`, `line`, `from`: the old locator, `to`: the one you applied). Do it once per call site you changed. Piwi records the attempt verified when the test passes on that commit, and records the healing suggestion applied once a run uses the recommended locator at that call site. `report_fix_attempt` needs the Maintainer role or above on the project.
 
 ## Guardrails
 

package/templates/skills/fix-a-reported-bug/SKILL.md

@@ -13,7 +13,8 @@ fixes it, and proves the fix with that same test.
 
 Prefer the **Piwi MCP server** if it is connected: `get_bug_report` (the steps in words, what was expected, the
 evidence, the reproductions, the tests that visit the page) and `render_steps` with `bugReportId` (the failing test).
-Without it, the CLI does the same with the dashboard's URL and an API key: `npx @piwitests/reporter bug <id>`.
+Without it, the CLI does the same: `npx @piwitests/reporter bug <id>`, with the dashboard's URL and an API key in the
+`PIWI_DASHBOARD_URL` and `PIWI_API_KEY` environment variables, the project's `.env`, or `--server-url` and `--api-key`.
 
 ## Steps
 
@@ -21,10 +22,12 @@ Without it, the CLI does the same with the dashboard's URL and an API key: `npx
    the reporter's note, the failed requests and console errors. The evidence often names the cause (a `POST … 500`).
 
 2. **Write the failing test.** `npx @piwitests/reporter bug <id> --write` writes the spec to the project's bugs folder
-   (`tests/bugs` unless the project names another) and runs it once. It is marked `test.fail()` and annotated
-   `piwi:bug <id>`, so it passes while the bug exists. Exit code 0 means the bug reproduces here. Exit code 1 means it
-   does not: read the output. A step that finds no element means the page or the data differ here (a user, a cart, a
-   flag the report assumed): make the test's setup provide them, in the project's style, rather than deleting steps.
+   (`tests/bugs` unless the project names another) and runs it once. It is marked `test.fail()`, tagged `@bug` and
+   annotated `piwi:bug <id>`, so it passes while the bug exists. Exit code 0 means the bug reproduces here. Exit code 1
+   means it does not: read the output. Exit code 2 means the report could not be read or the file could not be written
+   (`--force` replaces an existing file). A step that finds no element means the page or the data differ here (a
+   user, a cart, a flag the report assumed): make the test's setup provide them, in the project's style, rather than
+   deleting steps.
 
 3. **Look at the tests that already visit the page** (`missedBy` in `get_bug_report`). If one of them reaches the
    element, the missing piece is often one assertion in that test; say so in your report, and keep the bug's own spec
@@ -35,9 +38,9 @@ Without it, the CLI does the same with the dashboard's URL and an API key: `npx
 5. **Prove the fix.** Remove the `test.fail()` line from the bug's spec and run it: it must pass. Run the tests that
    visit the page too (`npx playwright test <their files>`), so the fix breaks nothing around it.
 
-6. **Report.** The cause, the fix, the spec file, and the two runs. Once the spec runs in CI without `test.fail()`, Piwi
-   closes the report on its own; before that, a run where the spec still carries `test.fail()` and passes marks it
-   "looks fixed".
+6. **Report.** The cause, the fix, the spec file, and the two runs. Once the spec passes without `test.fail()` in a
+   run on the project's default branch (not a `piwi bug` run), Piwi closes the report on its own; before that, such a
+   run where the spec still carries `test.fail()` and passes marks it "looks fixed".
 
 ## Guardrails
 

package/templates/skills/investigate-failure/SKILL.md

@@ -15,7 +15,7 @@ Prefer the **Piwi MCP server** if it is connected to this agent (tools named `li
 
 1. **Find the run.** If the user gave a run URL or id, use it. Otherwise call `list_recent_activity` (or `list_runs` for a specific project) and take the most recent failed run. Confirm with the user if several projects are in play.
 
-2. **Get the failures.** Call `get_run` with a failed-status filter to list the failing cases. For a fast single-call evidence bundle on one case, use `explain_failure` (error + steps + console + locator fix + diagnosis context).
+2. **Get the failures.** Call `get_run` (its default `statusFilter` is `failed`) to list the failing cases. For a fast single-call evidence bundle on one case, use `explain_failure` (error + steps + console + locator fix + diagnosis context).
 
 3. **Group by cause.** Call `get_failure_groups` (or `list_clusters` / `get_cluster`) — failures that share a root cause are clustered, so you fix one thing, not five. Work cluster by cluster.
 
@@ -25,9 +25,9 @@ Prefer the **Piwi MCP server** if it is connected to this agent (tools named `li
 
 6. **Propose the fix.** Make the smallest change that addresses the root cause, in the actual source or spec files. Prefer Piwi's own suggestions where they exist — for a broken selector, the ranked replacement from `get_locator_healing` (the `apply-locator-healing` skill does exactly this). Show the diff.
 
-7. **Verify.** Re-run the affected spec(s): `npx playwright test <file>`. Confirm they pass and the new run is green in the dashboard (`get_run_insights` compares against the last green run: regressions cleared, nothing new broken).
+7. **Verify.** Re-run the affected spec(s): `npx playwright test <file>`. Confirm they pass and the new run is green in the dashboard (`get_run_insights` compares the run with its baseline, usually the last passing run: regressions cleared, nothing new broken).
 
-8. **Report back.** Piwi learns from what you tell it, so end every investigation with these calls (they need reporter or admin access; each is logged with your API key):
+8. **Report back.** Piwi learns from what you tell it, so end every investigation with these calls (each is logged with your API key; `record_diagnosis` and `report_fix_attempt` need the Maintainer role or above on the project, rating a diagnosis needs read access only):
    - **Your diagnosis.** If the cluster had no diagnosis, or yours differs, record it with `record_diagnosis` (`clusterId`, or `executionId` for a failure in no cluster or with a cause of its own): the model you run on and the diagnosis in the schema the tool describes (summary, confidenceScore, severity, ranked hypotheses with their evidence, suggestedFix with the unified-diff `patch`). Piwi validates the patch against the source that failed and shows the diagnosis as written by an agent.
    - **Your rating of Piwi's diagnosis.** If you read one with `get_cluster_diagnosis`, rate it with `submit_diagnosis_feedback`: `up` when it pointed at the real cause, `down` with a one-line note on what it got wrong when it did not. A `down` keeps the next diagnosis from repeating it.
    - **Your fix.** Commit the change with the trailer `get_fix_plan` returns in `verify.commitTrailer` (`Piwi-Cluster: <clusterId>`) as the last line of the commit message, then call `report_fix_attempt` with the cluster, the kind (`patch`, `locator-edit` or `fix-plan`), the commit or the branch, the patch you applied and the `diagnosisId` you followed. When the tests pass on that commit, Piwi records the attempt verified and shows it on the cluster's activity.

package/templates/skills/setup-piwi/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: setup-piwi
-description: Wire a Playwright project up to a Piwi Dashboard — install the reporter, wrap the config, add capture fixtures, and confirm a run lands. Use when the user asks to "set up Piwi", "connect Playwright to the dashboard", "start reporting test results", or after they mention a Piwi/dashboard server URL.
+description: Use when the user asks to "set up Piwi", "connect Playwright to the dashboard", "start reporting test results", or mentions a Piwi or dashboard server URL. Wires a Playwright project up to a Piwi Dashboard — install the reporter, wrap the config, add capture fixtures, and confirm a run lands.
 ---
 
 # Set up Piwi in a Playwright project
@@ -9,7 +9,7 @@ Connect a Playwright test suite to a [Piwi Dashboard](https://piwitests.dev) so
 
 ## Before you start
 
-Find out two things (ask the user only if you cannot infer them):
+Find out three things (ask the user only if you cannot infer them):
 
 - **Dashboard URL** — e.g. `http://localhost:3000` for a local server, or a deployed URL. Defaults to `http://localhost:3000`.
 - **Project name** — the label runs are grouped under. Defaults to the package or folder name.
@@ -21,16 +21,19 @@ Confirm this is a Playwright project: there should be a `playwright.config.ts|js
 
 ## Do it
 
-Run the initializer from the project root. It installs `@piwitests/reporter`, wraps `defineConfig(...)` with `wrapConfig(...)`, creates `tests/fixtures.ts`, and records `PIWI_*` settings in `.env.example`:
+Run the initializer from the project root. It installs `@piwitests/reporter`, wraps `defineConfig(...)` with `wrapConfig(...)`, creates `tests/fixtures.ts` (`tests/fixtures.js` for a JavaScript config), records the connection in `.env.example`, and installs the Piwi workflow skills into `.claude/skills/`:
 
 ```bash
 npx @piwitests/reporter init --server-url <dashboard-url> --project <name> --json
 ```
 
 - Preview first with `--dry-run` if you want to see the plan before anything is written.
-- `--json` prints a `steps[]` array. Read it. Each step has a `status`:
+- `--api-key <key>` also writes the key and the URL into `.env` and adds `.env` to `.gitignore`.
+- `--skills none` installs no skills; `--skills-dir <path>` installs them where your agent reads skills from.
+- `--json` prints a `steps[]` array and the `nextSteps` to tell the user. Read it. Each step has a `status`:
   - `created` / `updated` / `already` — done, nothing more to do.
-  - `manual` — the tool would not edit that file safely. The `detail` field is the exact change to make; apply it yourself (see below).
+  - `manual` — the tool would not edit that file safely, or was told not to run the install. The `detail` field is the exact change to make; apply it yourself (see below).
+  - `skipped` — a skill file exists and was changed by hand; `--force` replaces it.
   - `error` — something failed (usually the install); the `detail` says what. Fix it and re-run — `init` is idempotent.
 
 ### Finishing `manual` steps
@@ -50,7 +53,7 @@ change it to import from the fixtures file instead, e.g. `import { test, expect
 
 ## Authentication
 
-If the dashboard has auth enabled, runs need an API key. Have the user create one in the dashboard (**Settings → Account → API keys**; keys start with `pd_`), then put it in `.env` as `PIWI_API_KEY=pd_...` and make sure `.env` is git-ignored. The reporter reads `PIWI_DASHBOARD_URL`, `PIWI_API_KEY` and `PIWI_PROJECT_NAME` from the `.env` next to `playwright.config`, so the config needs no dotenv loading; a variable set in the environment wins over the file. Never hardcode a key into `playwright.config.ts` or commit it. In CI, where `.env` is not checked out, pass it as a secret (`PIWI_API_KEY`).
+If the dashboard has auth enabled, runs need an API key. Have the user create one in the dashboard (**Settings → Account → API keys**; keys start with `pd_`), then put it in `.env` as `PIWI_API_KEY=pd_...` (or re-run `init` with `--api-key`) and make sure `.env` is git-ignored. The reporter reads `PIWI_DASHBOARD_URL`, `PIWI_API_KEY` and `PIWI_PROJECT_NAME` from the `.env` next to `playwright.config`, so the config needs no dotenv loading; a variable set in the environment wins over the file. Never hardcode a key into `playwright.config.ts` or commit it. In CI, where `.env` is not checked out, pass it as a secret (`PIWI_API_KEY`).
 
 ## Verify it worked
 
@@ -58,8 +61,8 @@ Setup is not done until a run reaches the dashboard.
 
 1. Run one spec: `npx playwright test` (or a single file to keep it quick).
 2. Confirm the run landed. Deterministically: set `PIWI_OUTPUT_FILE=piwi-run.json` when running, then read `runUrl`/`status` from that JSON file — its existence with a `runUrl` proves the upload. Otherwise look for the `View run: <url>` line the reporter prints, or open the dashboard and check the project's latest run.
-3. If nothing appears: verify the dashboard is reachable at the URL, that auth (if on) has a valid key, and that traces are enabled (`use: { trace: 'retain-on-failure' }`).
+3. If nothing appears: read the reporter's `[Piwi Dashboard]` lines (set `PIWI_VERBOSE=true` for more), verify the dashboard is reachable at the URL, and that auth (if on) has a valid key. When the reporter could not deliver the run, it saves it and sends it with the next run.
 
 ## Wrap up
 
-Tell the user what changed (config, fixtures, `.env.example`), what they still need to do (API key if auth is on; rewire remaining specs), and the run URL you verified. If they use CI, offer to add the reporter's env there too.
+Tell the user what changed (config, fixtures, `.env.example`, the skills), what they still need to do (API key if auth is on; rewire remaining specs), and the run URL you verified. If they use CI, offer to add the reporter's env there too.

package/templates/skills/stabilize-flaky-tests/SKILL.md

@@ -5,7 +5,7 @@ description: Find the flakiest Playwright tests from Piwi's flaky analysis, make
 
 # Stabilize flaky tests with Piwi
 
-Piwi scores every test's flakiness over its whole run history and ranks it by impact, so you spend effort on the tests that actually cost the team — not whichever one failed most recently. For each test it also ranks **suspects** from that history (a route that is slower when it fails, a test running alongside, load), and its **Flake Lab** replays each suspect as a condition next to a control until the failure reproduces. A reproduced condition proves the cause, and the same condition later proves the fix. This skill follows that loop: rank, reproduce, fix, verify.
+Piwi scores every test's flakiness over its recent runs, with the CI time each one wastes, so you spend effort on the tests that actually cost the team — not whichever one failed most recently. For each test it also ranks **suspects** from that history (a route that is slower when it fails, a test running alongside, load), and its **Flake Lab** replays each suspect as a condition next to a control until the failure reproduces. A reproduced condition proves the cause, and the same condition later proves the fix. This skill follows that loop: rank, reproduce, fix, verify.
 
 ## How you reach Piwi
 
@@ -13,7 +13,7 @@ Prefer the **Piwi MCP server** if it is connected (`list_flaky_tests`, `get_flak
 
 ## Steps
 
-1. **Rank the flaky tests.** Call `list_flaky_tests` for the project and take the top few by impact. Do not try to fix the whole list at once.
+1. **Rank the flaky tests.** Call `list_flaky_tests` for the project (highest flakiness score first, each with its `impact` and `wastedCiMinutes`) and take the few that cost the most. Do not try to fix the whole list at once.
 
 2. **Read the suspects.** For a chosen test, call `get_flake_profile`. Each suspect carries raw counts ("slow in 7 of 8 failures and 3 of 44 passes"), the condition that would test it, and, once the lab has run, its latest result (`lab`). `experiments` lists what the lab already found; if an experiment reproduced the test, skip to step 4. Read one failing execution (`get_test_run_case` / `explain_failure`) for the error at the moment it flaked. A test whose root cause is `order` depends on state an earlier test leaves behind: `compare_coverage` with its `testCaseId` names that test and the code whose path changes, so reproduce by running that test before this one in a single worker, and fix the shared state, not the wait.
 
@@ -23,7 +23,7 @@ Prefer the **Piwi MCP server** if it is connected (`list_flaky_tests`, `get_flak
    npx @piwitests/reporter flake <testCaseId>
    ```
 
-   It runs a control, then one arm per suspect, retries off, and counts only failures with the same error as in CI. Exit code `0` means an arm reproduced the failure ("reproduced by delaying GET /api/cart to 1.8 s: 3/4 against 0/10, p = 0.011"); `1` means nothing did; `2` is an error (read its message: often the fixtures are not in use). Use `--suspect <n>` to test one suspect, `--all` to also try every condition at once. If the output warns that this checkout differs from the commit of the failures, say so in your report.
+   `<testCaseId>` can also be the test's spec file and line (`tests/checkout.spec.ts:42`). It runs a control, then one arm per suspect, retries off, and counts only failures with the same error as in CI. Exit code `0` means an arm reproduced the failure (`Verdict: reproduced by delay GET /api/cart 1.8 s (3/4 against 0/10, p = 0.011)`); `1` means nothing did; `2` is an error (read its message: often the fixtures are not in use). Use `--suspect <n>` to test one suspect, `--all` to also try every condition at once. If the output warns that this checkout differs from the commit of the failures, say so in your report.
 
    If nothing reproduces, do not guess a fix from the suspects alone. Report the arms and their counts, and fall back to reading the failure and the attempts diff.
 

package/templates/skills/write-the-missing-test/SKILL.md

@@ -15,21 +15,24 @@ drafts it from the graph, and opens it in the same change.
 
 Prefer the **Piwi MCP server** if it is connected (`list_scenario_gaps`,
 `draft_scenario`, `get_change_coverage`, `get_coverage`, `triage_gap`). Otherwise use the dashboard's **Gaps**
-view, or the reporter CLI with `PIWI_DASHBOARD_URL` / `PIWI_API_KEY` /
-`PIWI_PROJECT_NAME` set.
+view, and work from what it shows plus the repo.
 
 ## Steps
 
-1. **Find the gaps.** On a change, call `get_change_coverage` for the current
-   branch first — the files whose `reach` is `unreached` (the uncovered
-   changes) and the `lines.notRun` of the files it did reach are what the
-   change left untested. Otherwise call `list_scenario_gaps` for the project. Filter to what you are
-   working on with `class`, `feature`, `minScore` or `pr`. Read the classes:
+1. **Find the gaps.** On a change, call `get_change_coverage` first, with the
+   change's `base` and `head` commits (or a `run` of it): the files whose `reach`
+   is `unreached` (the uncovered changes) and the `lines.notRun` of the files it
+   did reach are what the change left untested. Otherwise call
+   `list_scenario_gaps` for the project. Filter to what you are working on with
+   `class`, `feature`, `minScore` or `pr`. Read the classes:
    - **blind-spot** — no trusted test reaches it (a route, a page, a control, a
      new error path).
    - **false-comfort** — tests reach it and a probe showed they would not notice
      it breaking. The most valuable to fix: the catalog says it is covered.
    - **fragile** — reached by a single, flaky, quarantined or long-skipped test.
+   - **unhandled** / **degraded** — a server probe made a route or a dependency
+     fail, and the application did not handle it, or degraded visibly: an
+     error-state scenario is missing.
 
 2. **Pick one.** Take the highest-scored gap in scope. Every gap is *observed
    reach*, never instrumented coverage, and every evidence line pairs "no test"
@@ -52,12 +55,12 @@ view, or the reporter CLI with `PIWI_DASHBOARD_URL` / `PIWI_API_KEY` /
    the draft named over hand-rolled locators.
 
 5. **Run just this test, then verify.** Run the new spec with a file filter
-   (`npx @piwitests/reporter run` or `playwright test <file>`). Confirm it passes
+   (`npx playwright test <file>`). Confirm it passes
    for the right reason: temporarily break the behavior it asserts and see it
    fail. Open it in the same pull request as the change that motivated it.
 
 6. **Prove it reaches its target lines.** When the project records server
-   coverage (an ASP.NET Core backend in a coverage build), run the new test with
+   coverage (an ASP.NET Core or Nitro backend built for coverage), run the new test with
    it on and reporting to Piwi:
 
    ```bash
@@ -83,7 +86,7 @@ view, or the reporter CLI with `PIWI_DASHBOARD_URL` / `PIWI_API_KEY` /
    the gap may need a fixture.
 
 8. **Report back to Piwi.** Tell Piwi what you decided about the gap with
-   `triage_gap` (reporter or admin access): `verb: "accept"` for the gap you
+   `triage_gap` (Maintainer role or above on the project): `verb: "accept"` for the gap you
    wrote the test for, `covered-by` with the covering test's id when a test you
    found already covers it, or `dismiss` with the reason (`not-worth-testing`,
    `covered-elsewhere`, `wrong`) when the evidence did not hold. Each verdict counts toward the precision of the