@reportforge/playwright-pdf 0.19.0 → 0.20.0

package/CHANGELOG.md

@@ -3,6 +3,37 @@
 All notable changes to this project will be documented in this file.
 Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
+## [0.20.0] - 2026-07-11
+
+### Added
+
+- **On-device training for failure analysis.** Label locally-collected failure samples (`npx @reportforge/playwright-pdf label-feedback`), then `train-model` trains and cross-validates (LOOCV under 50 rows, seeded stratified 5-fold above) a personal classifier layer entirely on your machine. It activates only when it measurably beats the shipped base model on your own labeled set (champion/challenger gate; the layer answers only past a confidence threshold calibrated to >=90% held-out precision, renders `strong` at >=95%). `evaluate-model` prints the base vs. base+local comparison any time without writing anything.
+- **New `failureAnalysis` options**: `localModel` (use the gate-passed on-device model, default `true`), `localModelPath` (point at a repo-committed model file, from `train-model --out --project`, to share a team-trained layer with ephemeral CI machines; the file contains only redacted token statistics), `collectScope` (`'blind-spots' | 'all'`; `'all'` also collects confidently-classified failures so wrong-but-confident predictions can be corrected by labeling).
+- The feedback CSV gained a `label` column (`category,margin,label,text`). Legacy 3-column files parse forever and migrate on first write. Human labels survive re-sighting of the same failure and are exempt from ring-buffer eviction. A `project.json` marker in each feedback dir powers the `--project` filter and the labeler's project display.
+
+### Changed
+
+- `export-feedback` output is normalized to the v2 header and reframed as "review or move your training data" (local training is now the primary consumer; there is still no upload).
+- Classification is now a model chain: deterministic rules -> local layer (when present, gate-passed, and confident) -> base model. Per-test chip strength is resolved at the producing source; local-model margins are never compared against the base thresholds.
+
+### Fixed
+
+- The assertion-timeout rule also recognizes the modern Playwright call-log shape (`expect.toBeVisible with timeout 5000ms`) alongside the classic `Expect "toBeVisible"` form (found via a web error-corpus sweep).
+
+### Security
+
+- The no-network guarantee for failure analysis is mechanically enforced: a CI source scan fails if any analysis/training code names a network primitive (`fetch`, `node:http(s)`, sockets, HTTP clients, WebSocket, or a hardcoded URL). The docs gained a full "Data & network surface" inventory table.
+
+---
+
+## [0.19.1] - 2026-07-11
+
+### Fixed
+
+- **A blank browser window no longer flashes during report generation on Windows.** Chrome's new headless mode (the only mode left in Chrome 132+) briefly uncloaks a window on some Windows/GPU combinations; the PDF renderer now parks that window far off-screen. No effect on PDF output or other platforms.
+
+---
+
 ## [0.19.0] - 2026-07-11
 
 ### Added

package/README.md

@@ -58,7 +58,8 @@ Three templates, one reporter. Every PDF is fully offline: fonts, charts, and sc
 - **Notifications**: post pass/fail summaries to Slack, Teams, Discord, or email after each run; configurable trigger (`always` / `failure` / `success`) per channel; email and Discord support optional PDF attachment via `attachPdf`
 - **Flakiness trend**: top-N flakiest tests table with per-test dot sparkline across stored history runs in the `detailed` template; `flakinessTopN` option (default 5, `0` disables)
 - **Offline failure analysis**: sorts each failure into one of 7 root-cause buckets and renders a root-cause chip under each failed test in the `detailed` PDF (dominant-cause one-liner in `executive`). No network, no AI service: a small embedded classifier that updates over your existing license refresh; pin it any time. Full details at [reportforge.org/docs/advanced/failure-analysis](https://reportforge.org/docs/advanced/failure-analysis).
-- `npx @reportforge/playwright-pdf export-feedback`: exports locally-collected unrecognized failures (tokenized, local-only) for optional manual sharing.
+- **On-device training**: teach the classifier your team's failures without any data leaving your machine. Label locally-collected samples (`npx @reportforge/playwright-pdf label-feedback`), then `train-model` trains and cross-validates a personal layer on-device; it activates only when it measurably beats the base model on your own labeled set (`evaluate-model` shows the comparison any time). Commit the model file to your repo via `failureAnalysis.localModelPath` to share it with CI. Training data, the trained model, and evaluation are all local files; the analysis code cannot even name a network primitive (CI-enforced source scan).
+- `npx @reportforge/playwright-pdf export-feedback`: merges your locally-collected feedback (tokenized + redacted) into one CSV for review or moving between machines. Local file only; there is no upload.
 - **Rich execution capture**: opt-in "Steps to Reproduce" outline (a nested, copy-pasteable Markdown list of your `test.step`s and assertions; add `apiSteps` for the full click/fill/check trail), Node console tail, and trace/video links under each failed test in the defect log. Read straight from the Playwright step tree: no fixtures, no test-code changes. Enable via `capture` (`steps` / `apiSteps` / `console` / `evidence`); off by default.
 - **Configurable sections**: add or remove any report section per template via `sections`.
 - **Executive report brief band**: leads with a plain-language brief (trend + root causes) and one hero pass-rate number instead of a flat KPI grid
@@ -198,7 +199,7 @@ All options are the second element of the reporter tuple.
 | `maxFileSizeMb` | `number` | `8` | Soft cap on the final PDF size in MB. If exceeded, the report is re-rendered once with the `'max'` compression preset. |
 | `shardResults` | `string \| string[]` | n/a | Glob or path array of Playwright JSON shard report files to merge into one PDF. See the Shard Merging docs. |
 | `notify` | `object` | n/a | Notification channels: `slack`, `teams` (each `{ url, enabled, on }`), `discord` (`{ url, enabled, on, attachPdf }`), `email` (`{ to, enabled, on, attachPdf }`). See the Notifications docs. |
-| `failureAnalysis` | `object` | `{ enabled: true }` | Offline failure root-cause analysis (embedded Naive Bayes; no runtime network). Fields: `enabled` (default `true`), `maxClusters` (default `10`), `minStrength` (`weak`\|`moderate`\|`strong`, default `weak`), `maxFailuresToAnalyse` (default `500`), `collectUnclassified` (default `true`), `autoUpdateModel` (default `true`). See the Failure Analysis docs. |
+| `failureAnalysis` | `object` | `{ enabled: true }` | Offline failure root-cause analysis (embedded Naive Bayes; no runtime network) with on-device personalization: label collected failures, then the `train-model` CLI trains + evaluates a local layer entirely on your machine; it activates only when it beats the base model, and nothing leaves the machine. Fields: `enabled` (default `true`), `maxClusters` (default `10`), `minStrength` (`weak`\|`moderate`\|`strong`, default `weak`), `maxFailuresToAnalyse` (default `500`), `collectUnclassified` (default `true`), `collectScope` (`blind-spots`\|`all`, default `blind-spots`), `autoUpdateModel` (default `true`), `localModel` (use the gate-passed on-device model, default `true`), `localModelPath` (share a team-trained model via a repo-committed file). See the Failure Analysis docs. |
 | `capture` | `object` | `{}` (off) | Opt-in rich execution capture for the defect log (reporter-side; no fixtures, no test-code changes). Fields: `steps` (a copy-pasteable "Steps to Reproduce" outline built from the Playwright step tree; each row is a Markdown list line indented under its parent `test.step`, default `false`), `apiSteps` (include every top-level `pw:api` action (click/fill/check/goto) in the outline for a full action trail; without it the outline shows only `test.step` intent + `expect` assertions, default `false`), `console` (Node stdout/stderr tail, default `false`), `evidence` (trace/video file links, default `false`), `maxSteps` (default `50`), `maxConsoleLines` (default `50`). Renders in the Defect Log section of the `detailed` template. |
 | `live` | `object` | `{}` (off) | Opt-in live test-execution streaming. When `enabled`, the reporter prints an unguessable watch link to the CI logs at run start and streams per-test progress to the hosted dashboard while tests run; all shards of one CI run converge on a single live view. Fields: `enabled` (default `false`), `runId` (override the auto-derived run id), `serverUrl` (override the streaming server), `steps` (`none`\|`failed`\|`intent`\|`all`, default `failed`; `intent` shows only your `test.step` names plus any failing step), `console` (stream stdout/stderr tails, default `false`), `flushMs` (batch debounce 500-10000, default `2000`). The PDF at the end of the run is unaffected. Requires an active subscription with the `live` entitlement. See the Live Runs docs. |
 | `historyFile` | `string` | `~/.reportforge/{key}/history.json` | Path to the history JSON file. Relative paths resolve from `cwd`. Enables pass-rate trending charts in the `detailed` template. |