@reportforge/playwright-pdf 0.28.0 → 0.30.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -59,10 +59,11 @@ Three templates, one reporter. Every PDF is fully offline: fonts, charts, and sc
59
59
  - **Flakiness trend**: top-N flakiest tests table with per-test dot sparkline across stored history runs in the `detailed` template; `flakinessTopN` option (default 5, `0` disables)
60
60
  - **Pass-rate trend**: cross-run trend chart from stored history; opt-in `remoteHistory` keeps the trend alive on ephemeral CI runners by storing aggregate numbers only (pass rate, counts, duration, verdict; no test titles or error text, and the branch name never travels in plaintext), with no extra network calls
61
61
  - **Run-over-run diff**: every report shows what changed since the previous run on the same branch: new failures, fixed tests, and still-failing tests, and the same summary rides along in Slack, Teams, Discord, and email notifications
62
+ - **Cross-browser comparison**: multi-project runs get a per-test matrix across projects in the `detailed` template — divergent outcomes first (fails-only-in-webkit at a glance), identical tests collapsed to a count; works for Node.js multi-project configs and pytest runs with multiple `--browser` values
62
63
  - **Offline failure analysis**: sorts each failure into one of 7 root-cause buckets and renders a root-cause chip under each failed test in the `detailed` PDF (dominant-cause one-liner in `executive`). No network, no AI service: a small embedded classifier with automatic updates; pin it any time. Full details at [reportforge.org/docs/advanced/failure-analysis](https://reportforge.org/docs/advanced/failure-analysis).
63
64
  - **On-device training**: teach the classifier your team's failures without any data leaving your machine. Label locally-collected samples (`npx @reportforge/playwright-pdf label-feedback`), then `train-model` trains and cross-validates a personal layer on-device; it activates only when it measurably beats the base model on your own labeled set (`evaluate-model` shows the comparison any time). Commit the model file to your repo via `failureAnalysis.localModelPath` to share it with CI. Training data, the trained model, and evaluation are all local files; the analysis code cannot even name a network primitive (enforced by an automated source audit).
64
65
  - `npx @reportforge/playwright-pdf export-feedback`: merges your locally-collected feedback (tokenized + redacted) into one CSV for review or moving between machines. Local file only; there is no upload.
65
- - **Rich execution capture**: opt-in "Steps to Reproduce" outline (a nested, copy-pasteable Markdown list of your `test.step`s and assertions; add `apiSteps` for the full click/fill/check trail), Node console tail, and trace/video links under each failed test in the defect log. Read straight from the Playwright step tree: no fixtures, no test-code changes. Enable via `capture` (`steps` / `apiSteps` / `console` / `evidence`); off by default.
66
+ - **Rich execution capture**: opt-in "Steps to Reproduce" outline (a nested, copy-pasteable Markdown list of your `test.step`s and assertions; add `apiSteps` for the full click/fill/check trail), Node console tail, and trace/video links under each failed test in the defect log. Read straight from the Playwright step tree: no fixtures, no test-code changes. Enable via `capture` (`steps` / `apiSteps` / `console` / `evidence`); off by default. Set `evidenceUrlTemplate` to turn those trace/video paths into clickable CI artifact links with a QR code beside each one, so even a printed report jumps straight to the trace viewer.
66
67
  - **Configurable sections**: add or remove any report section per template via `sections`.
67
68
  - **Executive report brief band**: leads with a plain-language brief (trend + root causes) and one hero pass-rate number instead of a flat KPI grid
68
69
  - **Data protection**: AES-256 password-protected PDFs (no external tools) and opt-in redaction of credentials/PII in report content and live streams
@@ -179,7 +180,7 @@ template = "detailed"
179
180
  outputFile = "reports/{date}-{branch}-report.pdf"
180
181
  ```
181
182
 
182
- The plugin reuses the Node runtime and Chromium that Playwright for Python already ships, so there is nothing else to install. pytest-xdist and pytest-rerunfailures are supported; failure screenshots from `--screenshot only-on-failure` are embedded. Live runs and step trees are Node.js-only today. Full guide: [reportforge.org/docs/python](https://reportforge.org/docs/python).
183
+ The plugin reuses the Node runtime and Chromium that Playwright for Python already ships, so there is nothing else to install. pytest-xdist and pytest-rerunfailures are supported; failure screenshots from `--screenshot only-on-failure` are embedded, and trace/video artifacts from `--tracing` / `--video` feed the defect log's evidence links (pair with `evidenceUrlTemplate` for clickable CI links). Passing `--browser` more than once splits the report into one project per browser and unlocks the cross-browser comparison section. Live runs and step trees are Node.js-only today. Full guide: [reportforge.org/docs/python](https://reportforge.org/docs/python).
183
184
 
184
185
  ---
185
186
 
@@ -194,7 +195,7 @@ set RF_LICENSE_KEY=RFSU-XXXX-XXXX-XXXX-XXXX
194
195
  dotnet test --logger "reportforge;output=reports/run.pdf;template=detailed"
195
196
  ```
196
197
 
197
- Full options come from a JSON file with the same camelCase names as this reporter: `--logger "reportforge;optionsFile=rf-options.json"`. The logger reuses the Node runtime Microsoft.Playwright places in your test output and a Playwright-managed Chromium, so there is nothing else to install. Retried tests render as flaky with their attempt history; categories become tags. Live runs and step trees are Node.js-only today. Full guide: [reportforge.org/docs/dotnet](https://reportforge.org/docs/dotnet).
198
+ Full options come from a JSON file with the same camelCase names as this reporter: `--logger "reportforge;optionsFile=rf-options.json"`. The logger reuses the Node runtime Microsoft.Playwright places in your test output and a Playwright-managed Chromium, so there is nothing else to install. Retried tests render as flaky with their attempt history; categories become tags. Trace archives and videos attached via `TestContext.AddTestAttachment` feed the defect log's evidence links (pair with `evidenceUrlTemplate` for clickable CI links). A `dotnet test` run drives one browser, so the cross-browser comparison section stays hidden here; live runs and step trees are Node.js-only today. Full guide: [reportforge.org/docs/dotnet](https://reportforge.org/docs/dotnet).
198
199
 
199
200
  ---
200
201
 
@@ -252,10 +253,12 @@ All options are the second element of the reporter tuple.
252
253
  | `showTrend` | `boolean` | `true` | Show the pass-rate sparkline and delta badge in the `detailed` template. Set to `false` to disable history tracking entirely. |
253
254
  | `remoteHistory` | `boolean` | `false` | Opt-in server-side trend store so the pass-rate trend survives ephemeral CI runners (the local history file is wiped with the workspace, leaving the chart stuck at one data point). One small authenticated request per run carrying aggregate numbers only — no test titles, no error text, and the branch name never travels in plaintext. Falls back to the local history file on any failure. See the History docs. |
254
255
  | `flakinessTopN` | `number` | `5` | Maximum flaky tests to show in the flakiness table (`detailed` template). Set to `0` to disable the table entirely. |
256
+ | `quarantineThreshold` | `number` | `40` | Flake-rate percentage (0-100) at which a test gets a QUARANTINE chip in the flakiness table, plus a callout with the candidate count. Applies once a test has 3+ history runs with flakiness data. Set to `0` to disable flagging. |
257
+ | `evidenceUrlTemplate` | `string` | n/a | URL template that turns local trace/video artifact paths (from `capture.evidence`) into clickable CI links with a QR code in the defect log. Must contain `{path}`, replaced with the forward-slashed artifact path, e.g. `https://ci.example.com/job/123/artifacts/{path}`. Unset = plain paths as before. |
255
258
  | `slowTestThreshold` | `number` | `10` | Minimum timed-test count before `SLOW` badges appear in the suite breakdown: below this, every test is trivially the "slowest" so badges stay hidden. On larger runs only genuine outliers (at least 2× the median test duration) are badged, so the badge stays rare. Set to `0` to drop the run-size gate (outliers still required). |
256
259
  | `requirementTagPattern` | `string` | `'^@?[A-Z][A-Z0-9]*-\\d+$'` | Regex deciding which tags count as requirement IDs in the Requirements Traceability section (`detailed` template). Matching tags (ticket shapes like `@ODP-5328`, `REQ-001`) get the traceability matrix; everything else (`@regression`, `@sanity`) collapses into a compact tag summary instead of repeating identical rows. Set to `''` to disable the split and list every tag in the matrix. |
257
260
  | `templatePath` | `string \| string[]` | n/a | Path to a custom Handlebars (`.hbs`) template file. Takes precedence over `template`. Pass an array to generate one PDF per custom template. See the Custom Templates docs. |
258
- | `sections` | `object` | per-template | Override which report sections appear, on top of the chosen template's defaults. Flat keys are the baseline for every chosen template; per-template keys (`minimal`\|`detailed`\|`executive`) override per template. 20 keys total. Block toggles: `coverPage`, `analysisOneliner`, `releaseGate`, `summary`, `charts`, `trend`, `requirementsMatrix`, `ciEnvironment`, `suiteBreakdown`, `failureDeepDive`, `failureAnalysis`, `slowTests`, `defectLog`, `briefBand`, `runDiff`. Display modifiers: `passRate`, `fullEnvironment`, `retries`, `fullFailures`, `stackTraces`. `runDiff` (block toggle, on by default in all three templates) shows new failures, fixed tests, and still-failing tests compared with the previous run on the same branch. See the Report Sections docs. |
261
+ | `sections` | `object` | per-template | Override which report sections appear, on top of the chosen template's defaults. Flat keys are the baseline for every chosen template; per-template keys (`minimal`\|`detailed`\|`executive`) override per template. 21 keys total. Block toggles: `coverPage`, `analysisOneliner`, `releaseGate`, `summary`, `charts`, `trend`, `requirementsMatrix`, `ciEnvironment`, `suiteBreakdown`, `failureDeepDive`, `failureAnalysis`, `slowTests`, `defectLog`, `briefBand`, `runDiff`, `browserMatrix`. Display modifiers: `passRate`, `fullEnvironment`, `retries`, `fullFailures`, `stackTraces`. `runDiff` (block toggle, on by default in all three templates) shows new failures, fixed tests, and still-failing tests compared with the previous run on the same branch. `browserMatrix` (block toggle, on by default in `detailed`) compares per-test outcomes across the run's projects and appears only when the run has 2+ projects. See the Report Sections docs. |
259
262
  <!-- AUTOGEN:options-table END -->
260
263
 
261
264
  Full reference: [reportforge.org/docs/configuration](https://reportforge.org/docs/configuration).
@@ -408,7 +411,9 @@ Full details at [reportforge.org/docs/advanced/security](https://reportforge.org
408
411
 
409
412
  After each run the reporter appends a summary entry to a local JSON file and renders a pass-rate sparkline + verdict row in the `detailed` template.
410
413
 
411
- **Since Last Run**: every report also gets a section (`runDiff`, on by default in all three templates) comparing the current run against the previous run on the same branch: new failures, fixed tests, and tests still failing, with the same summary line carried into Slack, Teams, Discord, and email notifications. The first run after upgrading has no comparable prior run to diff against, so the section starts appearing from the second run onward. It reads only the local history file, so test names never leave your machine; see the ephemeral-CI note below for keeping that file around between runs. Like the flakiness table, it rides history tracking (`showTrend`, on by default), and the notification line follows history availability rather than the `runDiff` section toggle.
414
+ **Flaky intelligence**: the flakiness table (`detailed` template) flags quarantine candidates tests whose flake rate reaches `quarantineThreshold` (default 40%) across 3+ recorded runs get a QUARANTINE chip and a callout with the candidate count, and every row shows its current streak (consecutive latest runs flaky). Set `quarantineThreshold: 0` to turn flagging off.
415
+
416
+ **Since Last Run**: every report also gets a section (`runDiff`, on by default in all three templates) comparing the current run against the previous run on the same branch: new failures, newly flaky tests, fixed tests, and tests still failing, with the same summary line carried into Slack, Teams, Discord, and email notifications. The first run after upgrading has no comparable prior run to diff against, so the section starts appearing from the second run onward. It reads only the local history file, so test names never leave your machine; see the ephemeral-CI note below for keeping that file around between runs. Like the flakiness table, it rides history tracking (`showTrend`, on by default), and the notification line follows history availability rather than the `runDiff` section toggle.
412
417
 
413
418
  **Local dev (zero config)**: history is written automatically to `~/.reportforge/{projectKey}/history.json`. The sparkline appears once two or more runs have been recorded.
414
419
 
@@ -542,6 +547,8 @@ reporter: [['@reportforge/playwright-pdf', {
542
547
  | `slowTests` | `SLOW` badge on the duration-ranked slowest tests, shown inline in the breakdown |
543
548
  | `defectLog` | `DEF-####` numbered failure table (severity, duration) + opt-in repro detail (`capture`) |
544
549
  | `briefBand` | Executive brief band: one plain-language sentence (trend delta + root causes) and a hero pass-rate stat |
550
+ | `runDiff` | Since Last Run: new failures, fixed tests, and still-failing tests vs. the previous run on the same branch |
551
+ | `browserMatrix` | Cross-browser comparison: per-test outcomes side by side across the run's projects — divergent tests first, identical ones collapsed to a count. Appears only when the run has 2+ projects |
545
552
 
546
553
  #### Display modifiers: tune a section that's on
547
554
 
@@ -573,6 +580,8 @@ Omit `sections` entirely and each template renders exactly this (✓ = on):
573
580
  | `slowTests` | – | ✓ | ✓ |
574
581
  | `defectLog` | – | ✓ | – |
575
582
  | `briefBand` | – | – | ✓ |
583
+ | `runDiff` | ✓ | ✓ | ✓ |
584
+ | `browserMatrix` | – | ✓ | – |
576
585
  | `passRate` | ✓ | ✓ | ✓ |
577
586
  | `fullEnvironment` | – | ✓ | – |
578
587
  | `retries` | ✓ | ✓ | – |
@@ -583,7 +592,7 @@ Omit `sections` entirely and each template renders exactly this (✓ = on):
583
592
 
584
593
  - **`trend` needs `charts`.** The trend line, run-history, and flakiness table live inside the charts block, so `trend: true` does nothing with `charts: false` (it's coerced off). It also needs history data; keep the top-level `showTrend` option on.
585
594
  - **Inline failure detail needs `suiteBreakdown`.** `failureDeepDive` (failure detail), `failureAnalysis` (root-cause chip), and `slowTests` (`SLOW` badge) now render *inside* the suite breakdown, so they show nothing when `suiteBreakdown` is off. The chip (`failureAnalysis`) also needs `failureDeepDive`: it lives inside the failure detail, so it's coerced off without it.
586
- - **Data-gated sections**: `failureAnalysis`, `requirementsMatrix`, `slowTests`, `defectLog` render only when there's matching data (classified failures, tagged tests, a meaningful slow set, failures). Toggling them on with no data shows nothing.
595
+ - **Data-gated sections**: `failureAnalysis`, `requirementsMatrix`, `slowTests`, `defectLog` render only when there's matching data (classified failures, tagged tests, a meaningful slow set, failures). Toggling them on with no data shows nothing. `browserMatrix` is data-gated too: it needs 2+ projects in the run (multiple Playwright projects, or `pytest --browser` given twice), so single-browser runs never show it.
587
596
  - **Render-layer only.** `sections` controls what's drawn, never what's collected; the data switches stay separate: `showTrend` (history), `flakinessTopN` (flaky-table size), `failureAnalysis.enabled` (classifier), `includeScreenshots` (images).
588
597
  - **Custom templates.** `sections` applies to the built-in templates; a custom `templatePath` controls its own layout (every section available) and ignores `sections`. You'll get a warning if both are set.
589
598
 
package/dist/cli/index.js CHANGED
@@ -293,20 +293,26 @@ function buildDemoCharts(passed, failed, skipped, flaky, total) {
293
293
  delta: 71 - 76
294
294
  // -5 vs previous run
295
295
  },
296
+ flakinessQuarantineTotal: 1,
297
+ flakinessQuarantineThreshold: 40,
296
298
  flakinessTable: [
297
299
  {
298
300
  title: "should handle session timeout gracefully",
299
301
  flakeRate: 60,
300
302
  flakeCount: 3,
301
303
  totalRuns: 5,
302
- sparkline: [true, false, true, false, true]
304
+ sparkline: [true, false, true, false, true],
305
+ streak: 1,
306
+ quarantine: true
303
307
  },
304
308
  {
305
309
  title: "should apply valid promo code",
306
310
  flakeRate: 20,
307
311
  flakeCount: 1,
308
312
  totalRuns: 5,
309
- sparkline: [false, false, false, true, false]
313
+ sparkline: [false, false, false, true, false],
314
+ streak: 0,
315
+ quarantine: false
310
316
  }
311
317
  ]
312
318
  };
@@ -320,6 +326,9 @@ function buildDemoRunDiff() {
320
326
  stillFailing: [
321
327
  { id: "co-3", title: "should reject expired promo code" }
322
328
  ],
329
+ newlyFlaky: [
330
+ { id: "auth-4", title: "should handle session timeout gracefully" }
331
+ ],
323
332
  fixed: [
324
333
  { id: "co-5", title: "should complete Stripe checkout" }
325
334
  ],
@@ -6381,6 +6390,7 @@ var init_defaults = __esm({
6381
6390
  historySize: 10,
6382
6391
  showTrend: true,
6383
6392
  flakinessTopN: 5,
6393
+ quarantineThreshold: 40,
6384
6394
  slowTestThreshold: 10,
6385
6395
  // Ticket-ID shape: @ODP-5328, REQ-001, JIRA-123. Tags matching this land in
6386
6396
  // the Requirements Traceability matrix; the rest fall to the tag summary.
@@ -6518,6 +6528,7 @@ var init_sections = __esm({
6518
6528
  "defectLog",
6519
6529
  "briefBand",
6520
6530
  "runDiff",
6531
+ "browserMatrix",
6521
6532
  // display modifiers
6522
6533
  "passRate",
6523
6534
  "fullEnvironment",
@@ -6542,6 +6553,7 @@ var init_sections = __esm({
6542
6553
  defectLog: false,
6543
6554
  briefBand: false,
6544
6555
  runDiff: true,
6556
+ browserMatrix: false,
6545
6557
  passRate: true,
6546
6558
  fullEnvironment: false,
6547
6559
  retries: true,
@@ -6564,6 +6576,7 @@ var init_sections = __esm({
6564
6576
  defectLog: true,
6565
6577
  briefBand: false,
6566
6578
  runDiff: true,
6579
+ browserMatrix: true,
6567
6580
  passRate: true,
6568
6581
  fullEnvironment: true,
6569
6582
  retries: true,
@@ -6591,6 +6604,7 @@ var init_sections = __esm({
6591
6604
  defectLog: false,
6592
6605
  briefBand: true,
6593
6606
  runDiff: true,
6607
+ browserMatrix: false,
6594
6608
  passRate: true,
6595
6609
  fullEnvironment: false,
6596
6610
  retries: false,
@@ -12246,6 +12260,10 @@ var init_schema = __esm({
12246
12260
  // local history file is wiped every run. Aggregate numbers only.
12247
12261
  remoteHistory: external_exports.boolean().default(false),
12248
12262
  flakinessTopN: external_exports.number().int().min(0, "flakinessTopN must be 0 or greater").optional().default(DEFAULT_OPTIONS.flakinessTopN),
12263
+ quarantineThreshold: external_exports.number().int().min(0, "quarantineThreshold must be between 0 and 100").max(100, "quarantineThreshold must be between 0 and 100").optional().default(DEFAULT_OPTIONS.quarantineThreshold),
12264
+ evidenceUrlTemplate: external_exports.string().refine((s) => s.includes("{path}"), {
12265
+ message: "evidenceUrlTemplate must contain the {path} token"
12266
+ }).optional(),
12249
12267
  slowTestThreshold: external_exports.number().int().min(0, "slowTestThreshold must be 0 or greater").optional().default(DEFAULT_OPTIONS.slowTestThreshold),
12250
12268
  requirementTagPattern: external_exports.string().refine((v) => {
12251
12269
  try {
package/dist/index.d.mts CHANGED
@@ -1,11 +1,11 @@
1
1
  import { Reporter, FullConfig, Suite, TestCase, TestResult, TestStep, FullResult } from '@playwright/test/reporter';
2
2
 
3
3
  /**
4
- * Canonical section catalog. 15 block toggles (section present/absent) followed
4
+ * Canonical section catalog. 16 block toggles (section present/absent) followed
5
5
  * by 5 display modifiers (tweak a section that is on). Single source of truth —
6
6
  * the Zod schema in src/config/schema.ts builds its flag shape from this array.
7
7
  */
8
- declare const SECTION_KEYS: readonly ["coverPage", "analysisOneliner", "releaseGate", "summary", "charts", "trend", "requirementsMatrix", "ciEnvironment", "suiteBreakdown", "failureDeepDive", "failureAnalysis", "slowTests", "defectLog", "briefBand", "runDiff", "passRate", "fullEnvironment", "retries", "fullFailures", "stackTraces"];
8
+ declare const SECTION_KEYS: readonly ["coverPage", "analysisOneliner", "releaseGate", "summary", "charts", "trend", "requirementsMatrix", "ciEnvironment", "suiteBreakdown", "failureDeepDive", "failureAnalysis", "slowTests", "defectLog", "briefBand", "runDiff", "browserMatrix", "passRate", "fullEnvironment", "retries", "fullFailures", "stackTraces"];
9
9
  type SectionKey = (typeof SECTION_KEYS)[number];
10
10
  /** A partial map of section flags — used both as the flat baseline and per-template. */
11
11
  type SectionFlags = Partial<Record<SectionKey, boolean>>;
@@ -368,6 +368,24 @@ interface ReporterOptions {
368
368
  * @default 5
369
369
  */
370
370
  flakinessTopN?: number;
371
+ /**
372
+ * Flake-rate percentage (0-100) at which a test is flagged as a quarantine
373
+ * candidate in the flakiness table: a QUARANTINE chip on the row plus a
374
+ * callout with the candidate count. A test qualifies once its flake rate
375
+ * over the stored history reaches the threshold across at least 3 runs
376
+ * that carry flakiness data. Set to 0 to disable flagging.
377
+ * @default 40
378
+ */
379
+ quarantineThreshold?: number;
380
+ /**
381
+ * URL template that turns local trace/video artifact paths (surfaced by
382
+ * `capture.evidence`) into clickable CI links with a QR code in the defect
383
+ * log. Must contain the `{path}` token, replaced with the forward-slashed
384
+ * artifact path (each segment percent-encoded), e.g.
385
+ * `https://ci.example.com/job/123/artifacts/{path}`.
386
+ * Unset = evidence renders as plain paths, exactly as before.
387
+ */
388
+ evidenceUrlTemplate?: string;
371
389
  /**
372
390
  * Minimum number of timed tests before `SLOW` badges appear in the suite
373
391
  * breakdown — below this, every test is trivially the "slowest", so badges
@@ -406,18 +424,22 @@ interface ReporterOptions {
406
424
  * every chosen template; per-template keys (`minimal` / `detailed` /
407
425
  * `executive`) override the baseline for that template only.
408
426
  *
409
- * 20 keys total: 15 block toggles, 5 display modifiers.
427
+ * 21 keys total: 16 block toggles, 5 display modifiers.
410
428
  *
411
429
  * Block toggles: `coverPage`, `analysisOneliner`, `releaseGate`, `summary`,
412
430
  * `charts`, `trend`, `requirementsMatrix`, `ciEnvironment`, `suiteBreakdown`,
413
431
  * `failureDeepDive`, `failureAnalysis`, `slowTests`, `defectLog`, `briefBand`,
414
- * `runDiff`.
432
+ * `runDiff`, `browserMatrix`.
415
433
  * Display modifiers: `passRate`, `fullEnvironment`, `retries`, `fullFailures`,
416
434
  * `stackTraces`.
417
435
  *
418
436
  * `runDiff` shows new failures, fixed tests, and still-failing tests against
419
437
  * the previous run on the same branch; on by default in all three templates.
420
438
  *
439
+ * `browserMatrix` compares per-test outcomes across the run's projects
440
+ * (divergent tests first, identical ones collapsed to a count); on by default
441
+ * in `detailed` and self-hides when the run has fewer than two projects.
442
+ *
421
443
  * `trend` renders only when `charts` is also on and history data exists (`showTrend` enabled). Sections gated on data
422
444
  * (`failureAnalysis`, `requirementsMatrix`, `slowTests`, `defectLog`) appear
423
445
  * only when matching data exists — this option never changes data collection.
package/dist/index.d.ts CHANGED
@@ -1,11 +1,11 @@
1
1
  import { Reporter, FullConfig, Suite, TestCase, TestResult, TestStep, FullResult } from '@playwright/test/reporter';
2
2
 
3
3
  /**
4
- * Canonical section catalog. 15 block toggles (section present/absent) followed
4
+ * Canonical section catalog. 16 block toggles (section present/absent) followed
5
5
  * by 5 display modifiers (tweak a section that is on). Single source of truth —
6
6
  * the Zod schema in src/config/schema.ts builds its flag shape from this array.
7
7
  */
8
- declare const SECTION_KEYS: readonly ["coverPage", "analysisOneliner", "releaseGate", "summary", "charts", "trend", "requirementsMatrix", "ciEnvironment", "suiteBreakdown", "failureDeepDive", "failureAnalysis", "slowTests", "defectLog", "briefBand", "runDiff", "passRate", "fullEnvironment", "retries", "fullFailures", "stackTraces"];
8
+ declare const SECTION_KEYS: readonly ["coverPage", "analysisOneliner", "releaseGate", "summary", "charts", "trend", "requirementsMatrix", "ciEnvironment", "suiteBreakdown", "failureDeepDive", "failureAnalysis", "slowTests", "defectLog", "briefBand", "runDiff", "browserMatrix", "passRate", "fullEnvironment", "retries", "fullFailures", "stackTraces"];
9
9
  type SectionKey = (typeof SECTION_KEYS)[number];
10
10
  /** A partial map of section flags — used both as the flat baseline and per-template. */
11
11
  type SectionFlags = Partial<Record<SectionKey, boolean>>;
@@ -368,6 +368,24 @@ interface ReporterOptions {
368
368
  * @default 5
369
369
  */
370
370
  flakinessTopN?: number;
371
+ /**
372
+ * Flake-rate percentage (0-100) at which a test is flagged as a quarantine
373
+ * candidate in the flakiness table: a QUARANTINE chip on the row plus a
374
+ * callout with the candidate count. A test qualifies once its flake rate
375
+ * over the stored history reaches the threshold across at least 3 runs
376
+ * that carry flakiness data. Set to 0 to disable flagging.
377
+ * @default 40
378
+ */
379
+ quarantineThreshold?: number;
380
+ /**
381
+ * URL template that turns local trace/video artifact paths (surfaced by
382
+ * `capture.evidence`) into clickable CI links with a QR code in the defect
383
+ * log. Must contain the `{path}` token, replaced with the forward-slashed
384
+ * artifact path (each segment percent-encoded), e.g.
385
+ * `https://ci.example.com/job/123/artifacts/{path}`.
386
+ * Unset = evidence renders as plain paths, exactly as before.
387
+ */
388
+ evidenceUrlTemplate?: string;
371
389
  /**
372
390
  * Minimum number of timed tests before `SLOW` badges appear in the suite
373
391
  * breakdown — below this, every test is trivially the "slowest", so badges
@@ -406,18 +424,22 @@ interface ReporterOptions {
406
424
  * every chosen template; per-template keys (`minimal` / `detailed` /
407
425
  * `executive`) override the baseline for that template only.
408
426
  *
409
- * 20 keys total: 15 block toggles, 5 display modifiers.
427
+ * 21 keys total: 16 block toggles, 5 display modifiers.
410
428
  *
411
429
  * Block toggles: `coverPage`, `analysisOneliner`, `releaseGate`, `summary`,
412
430
  * `charts`, `trend`, `requirementsMatrix`, `ciEnvironment`, `suiteBreakdown`,
413
431
  * `failureDeepDive`, `failureAnalysis`, `slowTests`, `defectLog`, `briefBand`,
414
- * `runDiff`.
432
+ * `runDiff`, `browserMatrix`.
415
433
  * Display modifiers: `passRate`, `fullEnvironment`, `retries`, `fullFailures`,
416
434
  * `stackTraces`.
417
435
  *
418
436
  * `runDiff` shows new failures, fixed tests, and still-failing tests against
419
437
  * the previous run on the same branch; on by default in all three templates.
420
438
  *
439
+ * `browserMatrix` compares per-test outcomes across the run's projects
440
+ * (divergent tests first, identical ones collapsed to a count); on by default
441
+ * in `detailed` and self-hides when the run has fewer than two projects.
442
+ *
421
443
  * `trend` renders only when `charts` is also on and history data exists (`showTrend` enabled). Sections gated on data
422
444
  * (`failureAnalysis`, `requirementsMatrix`, `slowTests`, `defectLog`) appear
423
445
  * only when matching data exists — this option never changes data collection.