@reportforge/playwright-pdf 0.21.0 → 0.22.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +28 -24
- package/SECURITY.md +2 -2
- package/dist/cli/index.js +65 -53
- package/dist/index.d.mts +19 -8
- package/dist/index.d.ts +19 -8
- package/dist/index.js +51 -39
- package/dist/index.mjs +51 -39
- package/dist/templates/partials/requirements-matrix.hbs +20 -4
- package/dist/templates/styles/base.css +23 -0
- package/package.json +1 -2
- package/CHANGELOG.md +0 -602
package/README.md
CHANGED
|
@@ -57,14 +57,15 @@ Three templates, one reporter. Every PDF is fully offline: fonts, charts, and sc
|
|
|
57
57
|
- **Shard merging**: combine N Playwright JSON shard reports into one PDF via `shardResults`; accepts glob patterns or explicit paths
|
|
58
58
|
- **Notifications**: post pass/fail summaries to Slack, Teams, Discord, or email after each run; configurable trigger (`always` / `failure` / `success`) per channel; email and Discord support optional PDF attachment via `attachPdf`
|
|
59
59
|
- **Flakiness trend**: top-N flakiest tests table with per-test dot sparkline across stored history runs in the `detailed` template; `flakinessTopN` option (default 5, `0` disables)
|
|
60
|
-
- **
|
|
61
|
-
- **
|
|
60
|
+
- **Pass-rate trend**: cross-run trend chart from stored history; opt-in `remoteHistory` keeps the trend alive on ephemeral CI runners by storing aggregate numbers only (pass rate, counts, duration, verdict; no test titles or error text, and the branch name never travels in plaintext), with no extra network calls
|
|
61
|
+
- **Offline failure analysis**: sorts each failure into one of 7 root-cause buckets and renders a root-cause chip under each failed test in the `detailed` PDF (dominant-cause one-liner in `executive`). No network, no AI service: a small embedded classifier with automatic updates; pin it any time. Full details at [reportforge.org/docs/advanced/failure-analysis](https://reportforge.org/docs/advanced/failure-analysis).
|
|
62
|
+
- **On-device training**: teach the classifier your team's failures without any data leaving your machine. Label locally-collected samples (`npx @reportforge/playwright-pdf label-feedback`), then `train-model` trains and cross-validates a personal layer on-device; it activates only when it measurably beats the base model on your own labeled set (`evaluate-model` shows the comparison any time). Commit the model file to your repo via `failureAnalysis.localModelPath` to share it with CI. Training data, the trained model, and evaluation are all local files; the analysis code cannot even name a network primitive (enforced by an automated source audit).
|
|
62
63
|
- `npx @reportforge/playwright-pdf export-feedback`: merges your locally-collected feedback (tokenized + redacted) into one CSV for review or moving between machines. Local file only; there is no upload.
|
|
63
64
|
- **Rich execution capture**: opt-in "Steps to Reproduce" outline (a nested, copy-pasteable Markdown list of your `test.step`s and assertions; add `apiSteps` for the full click/fill/check trail), Node console tail, and trace/video links under each failed test in the defect log. Read straight from the Playwright step tree: no fixtures, no test-code changes. Enable via `capture` (`steps` / `apiSteps` / `console` / `evidence`); off by default.
|
|
64
65
|
- **Configurable sections**: add or remove any report section per template via `sections`.
|
|
65
66
|
- **Executive report brief band**: leads with a plain-language brief (trend + root causes) and one hero pass-rate number instead of a flat KPI grid
|
|
66
|
-
- **Hybrid licensing**: one short key unlocks it; reports keep rendering offline via a cached
|
|
67
|
-
- **Update notice**: when a newer reporter version is published, each run ends with one info line naming it and the update command;
|
|
67
|
+
- **Hybrid licensing**: one short key unlocks it; reports keep rendering offline via a locally cached license between refreshes
|
|
68
|
+
- **Update notice**: when a newer reporter version is published, each run ends with one info line naming it and the update command; no extra network calls
|
|
68
69
|
|
|
69
70
|
---
|
|
70
71
|
|
|
@@ -107,14 +108,14 @@ npx playwright test
|
|
|
107
108
|
|
|
108
109
|
### Requirements
|
|
109
110
|
|
|
110
|
-
- Node.js ≥
|
|
111
|
+
- Node.js ≥ 20
|
|
111
112
|
- `@playwright/test` ≥ 1.40 (peer dependency)
|
|
112
|
-
- `puppeteer-core` ≥ 21 (peer dependency)
|
|
113
|
+
- `puppeteer-core` ≥ 21 < 26 (peer dependency)
|
|
113
114
|
- Chrome or Chromium installed on the machine
|
|
114
115
|
|
|
115
116
|
### Chrome setup
|
|
116
117
|
|
|
117
|
-
The reporter
|
|
118
|
+
The reporter locates Chrome via the `puppeteerExecutablePath` option first, then the `PUPPETEER_EXECUTABLE_PATH` environment variable, then `chrome-finder` (system Chrome), then puppeteer-core's bundled Chromium if present. Full per-OS instructions: [reportforge.org/docs/running-tests#chrome-setup](https://reportforge.org/docs/running-tests#chrome-setup).
|
|
118
119
|
|
|
119
120
|
**Windows**
|
|
120
121
|
|
|
@@ -199,15 +200,16 @@ All options are the second element of the reporter tuple.
|
|
|
199
200
|
| `maxFileSizeMb` | `number` | `8` | Soft cap on the final PDF size in MB. If exceeded, the report is re-rendered once with the `'max'` compression preset. |
|
|
200
201
|
| `shardResults` | `string \| string[]` | n/a | Glob or path array of Playwright JSON shard report files to merge into one PDF. See the Shard Merging docs. |
|
|
201
202
|
| `notify` | `object` | n/a | Notification channels: `slack`, `teams` (each `{ url, enabled, on }`), `discord` (`{ url, enabled, on, attachPdf }`), `email` (`{ to, enabled, on, attachPdf }`). See the Notifications docs. |
|
|
202
|
-
| `failureAnalysis` | `object` | `{ enabled: true }` | Offline failure root-cause analysis (embedded
|
|
203
|
+
| `failureAnalysis` | `object` | `{ enabled: true }` | Offline failure root-cause analysis (embedded classifier; no runtime network) with on-device personalization: label collected failures, then the `train-model` CLI trains + evaluates a local layer entirely on your machine; it activates only when it beats the base model, and nothing leaves the machine. Fields: `enabled` (default `true`), `maxClusters` (default `10`), `minStrength` (`weak`\|`moderate`\|`strong`, default `weak`), `maxFailuresToAnalyse` (default `500`), `collectUnclassified` (default `true`), `collectScope` (`blind-spots`\|`all`, default `blind-spots`), `autoUpdateModel` (default `true`), `localModel` (use the gate-passed on-device model, default `true`), `localModelPath` (share a team-trained model via a repo-committed file). See the Failure Analysis docs. |
|
|
203
204
|
| `capture` | `object` | `{}` (off) | Opt-in rich execution capture for the defect log (reporter-side; no fixtures, no test-code changes). Fields: `steps` (a copy-pasteable "Steps to Reproduce" outline built from the Playwright step tree; each row is a Markdown list line indented under its parent `test.step`, default `false`), `apiSteps` (include every top-level `pw:api` action (click/fill/check/goto) in the outline for a full action trail; without it the outline shows only `test.step` intent + `expect` assertions, default `false`), `console` (Node stdout/stderr tail, default `false`), `evidence` (trace/video file links, default `false`), `maxSteps` (default `50`), `maxConsoleLines` (default `50`). Renders in the Defect Log section of the `detailed` template. |
|
|
204
205
|
| `live` | `object` | `{}` (off) | Opt-in live test-execution streaming. When `enabled`, the reporter prints an unguessable watch link to the CI logs at run start and streams per-test progress to the hosted dashboard while tests run; all shards of one CI run converge on a single live view. Fields: `enabled` (default `false`), `runId` (override the auto-derived run id), `serverUrl` (override the streaming server), `steps` (`none`\|`failed`\|`intent`\|`all`, default `failed`; `intent` shows only your `test.step` names plus any failing step), `console` (stream stdout/stderr tails, default `false`), `flushMs` (batch debounce 500-10000, default `2000`). The PDF at the end of the run is unaffected. Requires an active subscription with the `live` entitlement. See the Live Runs docs. |
|
|
205
206
|
| `historyFile` | `string` | `~/.reportforge/{key}/history.json` | Path to the history JSON file. Relative paths resolve from `cwd`. Enables pass-rate trending charts in the `detailed` template. |
|
|
206
207
|
| `historySize` | `number` | `10` | Maximum number of test runs to keep in the history file (integer ≥ 2). Older runs are pruned on append. |
|
|
207
208
|
| `showTrend` | `boolean` | `true` | Show the pass-rate sparkline and delta badge in the `detailed` template. Set to `false` to disable history tracking entirely. |
|
|
208
|
-
| `remoteHistory` | `boolean` | `false` | Opt-in server-side trend store so the pass-rate trend survives ephemeral CI runners (the local history file is wiped with the workspace, leaving the chart stuck at one data point). One request per run
|
|
209
|
+
| `remoteHistory` | `boolean` | `false` | Opt-in server-side trend store so the pass-rate trend survives ephemeral CI runners (the local history file is wiped with the workspace, leaving the chart stuck at one data point). One small authenticated request per run carrying aggregate numbers only — no test titles, no error text, and the branch name never travels in plaintext. Falls back to the local history file on any failure. See the History docs. |
|
|
209
210
|
| `flakinessTopN` | `number` | `5` | Maximum flaky tests to show in the flakiness table (`detailed` template). Set to `0` to disable the table entirely. |
|
|
210
211
|
| `slowTestThreshold` | `number` | `10` | Minimum timed-test count before `SLOW` badges appear in the suite breakdown: below this, every test is trivially the "slowest" so badges stay hidden. On larger runs only genuine outliers (at least 2× the median test duration) are badged, so the badge stays rare. Set to `0` to drop the run-size gate (outliers still required). |
|
|
212
|
+
| `requirementTagPattern` | `string` | `'^@?[A-Z][A-Z0-9]*-\\d+$'` | Regex deciding which tags count as requirement IDs in the Requirements Traceability section (`detailed` template). Matching tags (ticket shapes like `@ODP-5328`, `REQ-001`) get the traceability matrix; everything else (`@regression`, `@sanity`) collapses into a compact tag summary instead of repeating identical rows. Set to `''` to disable the split and list every tag in the matrix. |
|
|
211
213
|
| `templatePath` | `string \| string[]` | n/a | Path to a custom Handlebars (`.hbs`) template file. Takes precedence over `template`. Pass an array to generate one PDF per custom template. See the Custom Templates docs. |
|
|
212
214
|
| `sections` | `object` | per-template | Override which report sections appear, on top of the chosen template's defaults. Flat keys are the baseline for every chosen template; per-template keys (`minimal`\|`detailed`\|`executive`) override per template. Block toggles: `coverPage`, `analysisOneliner`, `releaseGate`, `summary`, `charts`, `trend`, `requirementsMatrix`, `ciEnvironment`, `suiteBreakdown`, `failureDeepDive`, `failureAnalysis`, `slowTests`, `defectLog`, `briefBand`. Display modifiers: `passRate`, `fullEnvironment`, `retries`, `fullFailures`, `stackTraces`. See the Report Sections docs. |
|
|
213
215
|
<!-- AUTOGEN:options-table END -->
|
|
@@ -237,7 +239,7 @@ reporter: [
|
|
|
237
239
|
|
|
238
240
|
`steps: 'intent'` shows only your `test.step` names (plus any failing step): the cleanest, most readable trail. `'all'` adds every action and assertion beneath its parent step; `'failed'` (default) keeps intent + assertions + failures.
|
|
239
241
|
|
|
240
|
-
Sharded runs need no extra config:
|
|
242
|
+
Sharded runs need no extra config: every shard of one CI run converges on the same live view automatically. Set `RF_LIVE_RUN_ID` to override when your CI is not auto-detected.
|
|
241
243
|
|
|
242
244
|
Live streaming is best-effort and entitlement-gated: if the server is unreachable the run is unaffected, and an active subscription with the `live` feature is required. If you've configured Slack/Teams/Discord notifications, the watch link is also posted to those channels when the run starts.
|
|
243
245
|
|
|
@@ -329,7 +331,7 @@ reporter: [['@reportforge/playwright-pdf', {
|
|
|
329
331
|
}]]
|
|
330
332
|
```
|
|
331
333
|
|
|
332
|
-
One request per run
|
|
334
|
+
One small authenticated request per run, carrying aggregate numbers only (pass rate, counts, duration, verdict; no test titles, no error text, and the branch name never travels in plaintext). The server keeps roughly the last 100 runs per project + branch, expiring after 180 days, and returns them for the chart; any failure falls back to the local file. The flakiness table stays local-only by design (it needs test titles, which never leave your machine). Bitbucket Pipelines users especially: Bitbucket caches are immutable once written, so `remoteHistory` is the practical path there.
|
|
333
335
|
|
|
334
336
|
**CI caching alternative** (keeps even aggregate numbers off the server): set `historyFile` to a project-relative path and cache it between runs:
|
|
335
337
|
|
|
@@ -438,17 +440,18 @@ reporter: [['@reportforge/playwright-pdf', {
|
|
|
438
440
|
|---|---|
|
|
439
441
|
| `coverPage` | Full-page cover: title, verdict badge, KPI stats strip, branch + commit |
|
|
440
442
|
| `analysisOneliner` | One-line failure-analysis summary banner |
|
|
441
|
-
| `releaseGate` | Ship/hold recommendation banner (APPROVED TO SHIP / HOLD) |
|
|
443
|
+
| `releaseGate` | Ship/hold recommendation banner (APPROVED TO SHIP / HOLD, or a neutral NO TESTS RAN when the run executed zero tests) |
|
|
442
444
|
| `summary` | KPI strip (total · passed · failed · timed-out · skipped · flaky), verdict, duration, pass rate, flaky callout |
|
|
443
445
|
| `charts` | Pass-rate doughnut + stacked suite-results bar |
|
|
444
446
|
| `trend` | Pass-rate trend line, run-history table, and flakiness table (rendered inside `charts`) |
|
|
445
|
-
| `requirementsMatrix` |
|
|
447
|
+
| `requirementsMatrix` | Requirement-ID tags as a traceability matrix with pass-rate bars, plus a category tag summary |
|
|
446
448
|
| `ciEnvironment` | Branch, commit, browsers, CI provider, OS, Node, Playwright, projects, workers |
|
|
447
449
|
| `suiteBreakdown` | Per-suite → per-test table (status, duration, tags, retries). Failed tests carry their failure detail, root-cause chip, and a `SLOW` badge inline (the three keys below) |
|
|
448
450
|
| `failureDeepDive` | Inline failure detail under each failed test in the breakdown: error message, stack trace, screenshot, retry history |
|
|
449
451
|
| `failureAnalysis` | Inline per-test root-cause chip under each failed test (category + representative error + match strength, from the offline classifier) |
|
|
450
452
|
| `slowTests` | `SLOW` badge on the duration-ranked slowest tests, shown inline in the breakdown |
|
|
451
453
|
| `defectLog` | `DEF-####` numbered failure table (severity, duration) + opt-in repro detail (`capture`) |
|
|
454
|
+
| `briefBand` | Executive brief band: one plain-language sentence (trend delta + root causes) and a hero pass-rate stat |
|
|
452
455
|
|
|
453
456
|
#### Display modifiers: tune a section that's on
|
|
454
457
|
|
|
@@ -467,7 +470,7 @@ Omit `sections` entirely and each template renders exactly this (✓ = on):
|
|
|
467
470
|
| Section | minimal | detailed | executive |
|
|
468
471
|
|---|:-:|:-:|:-:|
|
|
469
472
|
| `coverPage` | – | – | ✓ |
|
|
470
|
-
| `analysisOneliner` | – | – |
|
|
473
|
+
| `analysisOneliner` | – | – | – |
|
|
471
474
|
| `releaseGate` | ✓ | ✓ | ✓ |
|
|
472
475
|
| `summary` | ✓ | ✓ | ✓ |
|
|
473
476
|
| `charts` | – | ✓ | ✓ |
|
|
@@ -479,6 +482,7 @@ Omit `sections` entirely and each template renders exactly this (✓ = on):
|
|
|
479
482
|
| `failureAnalysis` | – | ✓ | – |
|
|
480
483
|
| `slowTests` | – | ✓ | ✓ |
|
|
481
484
|
| `defectLog` | – | ✓ | – |
|
|
485
|
+
| `briefBand` | – | – | ✓ |
|
|
482
486
|
| `passRate` | ✓ | ✓ | ✓ |
|
|
483
487
|
| `fullEnvironment` | – | ✓ | – |
|
|
484
488
|
| `retries` | ✓ | ✓ | – |
|
|
@@ -621,10 +625,10 @@ stage('Playwright Tests') {
|
|
|
621
625
|
ReportForge ships a **hybrid offline-first** model:
|
|
622
626
|
|
|
623
627
|
1. Paste your `RFSU-…` key into the config once (or set `RF_LICENSE_KEY`).
|
|
624
|
-
2. On first run the reporter activates against the license server
|
|
625
|
-
3. Every subsequent run is **fully offline**: the cached
|
|
626
|
-
4.
|
|
627
|
-
5. Each subscription allows up to **25 active machines
|
|
628
|
+
2. On first run the reporter activates against the license server and caches a signed license at `~/.reportforge/license.json`.
|
|
629
|
+
3. Every subsequent run is **fully offline**: the cached license is validated on-device. No network call.
|
|
630
|
+
4. The reporter refreshes the cache automatically in the background before it expires.
|
|
631
|
+
5. Each subscription allows up to **25 active machines**. Machines that haven't run in 30 days free up automatically.
|
|
628
632
|
|
|
629
633
|
Cancel the subscription and the reporter keeps working until the end of the current billing period, then stops generating PDFs until you restart. **A license issue never aborts your test run**: the reporter logs a warning and skips PDF generation; your Playwright tests still pass.
|
|
630
634
|
|
|
@@ -639,14 +643,14 @@ Full lifecycle + offline failure modes: [reportforge.org/docs/licensing](https:/
|
|
|
639
643
|
|
|
640
644
|
## Requirements Traceability
|
|
641
645
|
|
|
642
|
-
The `detailed` template generates a requirements matrix from test tags. Tag tests to link them to requirements:
|
|
646
|
+
The `detailed` template generates a requirements matrix from test tags. Tag tests with ticket-shaped IDs to link them to requirements:
|
|
643
647
|
|
|
644
648
|
```ts
|
|
645
|
-
test('
|
|
646
|
-
test.describe('@
|
|
649
|
+
test('user can login', { tag: ['@REQ-101', '@smoke'] }, async ({ page }) => { /* ... */ });
|
|
650
|
+
test.describe('payment', { tag: '@PAY-204' }, () => { /* ... */ });
|
|
647
651
|
```
|
|
648
652
|
|
|
649
|
-
|
|
653
|
+
Tags matching `requirementTagPattern` (default: ticket shapes like `@REQ-101`, `@ODP-5328`, `JIRA-42`) each get a matrix row — test count, pass/fail counts, and a pass-rate bar — sorted by ID. Category tags (`@smoke`, `@regression`) collapse into a compact tag summary below the matrix instead of repeating one near-identical row per tag. Set `requirementTagPattern: ''` to disable the split and list every tag in the matrix.
|
|
650
654
|
|
|
651
655
|
---
|
|
652
656
|
|
|
@@ -675,7 +679,7 @@ Common issues:
|
|
|
675
679
|
- **`Chrome/Chromium not found`** → install Google Chrome Stable; don't use the Ubuntu `chromium-browser` snap.
|
|
676
680
|
- **Charts blank** → ensure `npm run bundle-chartjs` ran at build; the reporter waits up to 30s for `window.__chartsReady`.
|
|
677
681
|
- **`pdfPassword` ignored** → requires `qpdf` on `PATH` (`apt install qpdf` / `brew install qpdf` / `choco install qpdf`).
|
|
678
|
-
- **No PDF generated** → verify `RF_LICENSE_KEY` is set and your subscription is active. `RF_DEBUG=1` will show the activation result; missing key, expired subscription, network failure, or no
|
|
682
|
+
- **No PDF generated** → verify `RF_LICENSE_KEY` is set and your subscription is active. `RF_DEBUG=1` will show the activation result; missing key, expired subscription, network failure, or no license cache all skip PDF generation.
|
|
679
683
|
|
|
680
684
|
Full guide: [reportforge.org/docs/troubleshooting](https://reportforge.org/docs/troubleshooting).
|
|
681
685
|
|
|
@@ -693,7 +697,7 @@ Full documentation at [reportforge.org/docs](https://reportforge.org/docs):
|
|
|
693
697
|
- [Templates](https://reportforge.org/docs/templates): minimal, detailed, executive
|
|
694
698
|
- [Live runs](https://reportforge.org/docs/advanced/live): stream per-test progress to a shared watch link
|
|
695
699
|
- [Report Sections](https://reportforge.org/docs/configuration#report-sections): add or remove sections per template
|
|
696
|
-
- [License & offline behaviour](https://reportforge.org/docs/licensing):
|
|
700
|
+
- [License & offline behaviour](https://reportforge.org/docs/licensing): license cache, machine cap, cancellation
|
|
697
701
|
- [Troubleshooting](https://reportforge.org/docs/troubleshooting): common issues and debug flags
|
|
698
702
|
- [Changelog](https://reportforge.org/changelog): version history and release notes
|
|
699
703
|
|
package/SECURITY.md
CHANGED
|
@@ -40,8 +40,8 @@ Out of scope:
|
|
|
40
40
|
|
|
41
41
|
- Denial of service / volumetric attacks.
|
|
42
42
|
- Reports about the 25-machine licensing cap working as documented.
|
|
43
|
-
- Vulnerabilities exclusively in third-party services we integrate
|
|
44
|
-
|
|
43
|
+
- Vulnerabilities exclusively in third-party services we integrate (for
|
|
44
|
+
example our payment provider, Razorpay) — report those upstream.
|
|
45
45
|
|
|
46
46
|
## Supported versions
|
|
47
47
|
|
package/dist/cli/index.js
CHANGED
|
@@ -95,7 +95,7 @@ var init_stats_utils = __esm({
|
|
|
95
95
|
});
|
|
96
96
|
|
|
97
97
|
// src/demo/fixture.ts
|
|
98
|
-
function makeTest(id, title, status, duration, retries = 0) {
|
|
98
|
+
function makeTest(id, title, status, duration, retries = 0, tags = []) {
|
|
99
99
|
return {
|
|
100
100
|
id,
|
|
101
101
|
title,
|
|
@@ -103,7 +103,7 @@ function makeTest(id, title, status, duration, retries = 0) {
|
|
|
103
103
|
status,
|
|
104
104
|
duration,
|
|
105
105
|
retryCount: retries,
|
|
106
|
-
tags
|
|
106
|
+
tags,
|
|
107
107
|
annotations: []
|
|
108
108
|
};
|
|
109
109
|
}
|
|
@@ -381,20 +381,20 @@ var init_fixture = __esm({
|
|
|
381
381
|
machineLimit: 25
|
|
382
382
|
};
|
|
383
383
|
DEMO_AUTH_TESTS = [
|
|
384
|
-
makeTest("auth-1", "should login with valid credentials", "passed", 1240),
|
|
385
|
-
makeTest("auth-2", "should show error for wrong password", "passed", 890),
|
|
386
|
-
makeTest("auth-3", "should require email confirmation", "passed", 1100),
|
|
387
|
-
makeTest("auth-4", "should handle session timeout gracefully", "flaky", 3200, 1),
|
|
388
|
-
makeTest("auth-5", "should block brute-force attempts after 5 tries", "failed", 15e3, 2),
|
|
389
|
-
makeTest("auth-6", "should logout and clear session cookies", "passed", 780)
|
|
384
|
+
makeTest("auth-1", "should login with valid credentials", "passed", 1240, 0, ["@auth", "@smoke", "@SHOP-1201"]),
|
|
385
|
+
makeTest("auth-2", "should show error for wrong password", "passed", 890, 0, ["@auth", "@smoke", "@SHOP-1201"]),
|
|
386
|
+
makeTest("auth-3", "should require email confirmation", "passed", 1100, 0, ["@auth", "@regression", "@SHOP-1188"]),
|
|
387
|
+
makeTest("auth-4", "should handle session timeout gracefully", "flaky", 3200, 1, ["@auth", "@regression"]),
|
|
388
|
+
makeTest("auth-5", "should block brute-force attempts after 5 tries", "failed", 15e3, 2, ["@auth", "@regression", "@SHOP-1349"]),
|
|
389
|
+
makeTest("auth-6", "should logout and clear session cookies", "passed", 780, 0, ["@auth", "@smoke"])
|
|
390
390
|
];
|
|
391
391
|
DEMO_CHECKOUT_TESTS = [
|
|
392
|
-
makeTest("co-1", "should add product to cart", "passed", 1050),
|
|
393
|
-
makeTest("co-2", "should apply valid promo code", "passed", 2100),
|
|
394
|
-
makeTest("co-3", "should reject expired promo code", "failed", 8e3, 2),
|
|
395
|
-
makeTest("co-4", "should calculate shipping by region", "passed", 1800),
|
|
396
|
-
makeTest("co-5", "should complete Stripe checkout", "passed", 4200),
|
|
397
|
-
makeTest("co-6", "should send order confirmation email", "skipped", 0)
|
|
392
|
+
makeTest("co-1", "should add product to cart", "passed", 1050, 0, ["@checkout", "@smoke", "@SHOP-1042"]),
|
|
393
|
+
makeTest("co-2", "should apply valid promo code", "passed", 2100, 0, ["@checkout", "@regression", "@SHOP-1277"]),
|
|
394
|
+
makeTest("co-3", "should reject expired promo code", "failed", 8e3, 2, ["@checkout", "@regression", "@SHOP-1277"]),
|
|
395
|
+
makeTest("co-4", "should calculate shipping by region", "passed", 1800, 0, ["@checkout", "@regression"]),
|
|
396
|
+
makeTest("co-5", "should complete Stripe checkout", "passed", 4200, 0, ["@checkout", "@smoke", "@SHOP-1042"]),
|
|
397
|
+
makeTest("co-6", "should send order confirmation email", "skipped", 0, 0, ["@checkout"])
|
|
398
398
|
];
|
|
399
399
|
DEMO_CATALOG_TESTS = [
|
|
400
400
|
makeTest("cat-1", "should display product grid with filters", "passed", 980),
|
|
@@ -6362,7 +6362,10 @@ var init_defaults = __esm({
|
|
|
6362
6362
|
historySize: 10,
|
|
6363
6363
|
showTrend: true,
|
|
6364
6364
|
flakinessTopN: 5,
|
|
6365
|
-
slowTestThreshold: 10
|
|
6365
|
+
slowTestThreshold: 10,
|
|
6366
|
+
// Ticket-ID shape: @ODP-5328, REQ-001, JIRA-123. Tags matching this land in
|
|
6367
|
+
// the Requirements Traceability matrix; the rest fall to the tag summary.
|
|
6368
|
+
requirementTagPattern: "^@?[A-Z][A-Z0-9]*-\\d+$"
|
|
6366
6369
|
};
|
|
6367
6370
|
}
|
|
6368
6371
|
});
|
|
@@ -6620,7 +6623,7 @@ function deriveOptions(s) {
|
|
|
6620
6623
|
showFullFailures: s.fullFailures
|
|
6621
6624
|
};
|
|
6622
6625
|
}
|
|
6623
|
-
var import_handlebars, import_fs, import_path, A4_HEIGHT_MM, PAGE_MARGIN_Y_MM, PX_PER_MM, SUITE_CHART_SMALL_MAX, SUITE_CHART_ROWS_PER_CANVAS, SUITE_CHART_ROW_PX, SUITE_CHART_PAD_PX, PAGE_CONTENT_HEIGHT_PX, SLOW_MEDIAN_FACTOR, TemplateEngine;
|
|
6626
|
+
var import_handlebars, import_fs, import_path, A4_HEIGHT_MM, PAGE_MARGIN_Y_MM, PX_PER_MM, SUITE_CHART_SMALL_MAX, SUITE_CHART_ROWS_PER_CANVAS, SUITE_CHART_ROW_PX, SUITE_CHART_PAD_PX, PAGE_CONTENT_HEIGHT_PX, NUMERIC_COLLATOR, SLOW_MEDIAN_FACTOR, TemplateEngine;
|
|
6624
6627
|
var init_engine = __esm({
|
|
6625
6628
|
"src/templates/engine.ts"() {
|
|
6626
6629
|
"use strict";
|
|
@@ -6640,6 +6643,7 @@ var init_engine = __esm({
|
|
|
6640
6643
|
SUITE_CHART_ROW_PX = 18;
|
|
6641
6644
|
SUITE_CHART_PAD_PX = 56;
|
|
6642
6645
|
PAGE_CONTENT_HEIGHT_PX = (A4_HEIGHT_MM - 2 * PAGE_MARGIN_Y_MM) * PX_PER_MM;
|
|
6646
|
+
NUMERIC_COLLATOR = new Intl.Collator(void 0, { numeric: true });
|
|
6643
6647
|
SLOW_MEDIAN_FACTOR = 2;
|
|
6644
6648
|
TemplateEngine = class {
|
|
6645
6649
|
constructor() {
|
|
@@ -6666,17 +6670,22 @@ var init_engine = __esm({
|
|
|
6666
6670
|
setChartjsBundle(inline) {
|
|
6667
6671
|
this.chartjsInline = inline;
|
|
6668
6672
|
}
|
|
6669
|
-
render(data, template,
|
|
6673
|
+
render(data, template, opts = {}) {
|
|
6670
6674
|
const layoutPath = import_path.default.join(this.templatesDir, "layouts", `${template}.hbs`);
|
|
6671
6675
|
if (!import_fs.default.existsSync(layoutPath)) {
|
|
6672
6676
|
throw new Error(`[reportforge] Template layout not found: ${layoutPath}`);
|
|
6673
6677
|
}
|
|
6674
6678
|
const layoutSource = import_fs.default.readFileSync(layoutPath, "utf8");
|
|
6675
6679
|
const compiledLayout = this.handlebars.compile(layoutSource);
|
|
6676
|
-
const ctx = this.
|
|
6680
|
+
const ctx = this.assembleContext(
|
|
6681
|
+
data,
|
|
6682
|
+
resolveSections(template, opts.sections),
|
|
6683
|
+
this.readStyle(`${template}.css`),
|
|
6684
|
+
opts
|
|
6685
|
+
);
|
|
6677
6686
|
return compiledLayout(ctx);
|
|
6678
6687
|
}
|
|
6679
|
-
renderFromPath(data, filePath,
|
|
6688
|
+
renderFromPath(data, filePath, opts = {}) {
|
|
6680
6689
|
if (!this.customTemplateCache.has(filePath)) {
|
|
6681
6690
|
if (!import_fs.default.existsSync(filePath)) {
|
|
6682
6691
|
throw new Error(`[reportforge] Custom template not found: ${filePath}`);
|
|
@@ -6685,46 +6694,27 @@ var init_engine = __esm({
|
|
|
6685
6694
|
this.customTemplateCache.set(filePath, this.handlebars.compile(source));
|
|
6686
6695
|
}
|
|
6687
6696
|
const compiledFn = this.customTemplateCache.get(filePath);
|
|
6688
|
-
return compiledFn(this.
|
|
6697
|
+
return compiledFn(this.assembleContext(data, allSectionsOn(), "", opts));
|
|
6689
6698
|
}
|
|
6690
|
-
|
|
6691
|
-
const sections = allSectionsOn();
|
|
6692
|
-
const slow = this.buildSlowTests(data);
|
|
6693
|
-
return {
|
|
6694
|
-
...data,
|
|
6695
|
-
projects: this.enrichProjects(data, slow, sections, slowTestThreshold ?? DEFAULT_OPTIONS.slowTestThreshold),
|
|
6696
|
-
options: deriveOptions(sections),
|
|
6697
|
-
sections,
|
|
6698
|
-
tagGroups: this.buildTagGroups(data),
|
|
6699
|
-
slowTests: slow.ranked,
|
|
6700
|
-
baseCSS: this.fontsBundleCss + "\n" + this.readStyle("base.css"),
|
|
6701
|
-
templateCSS: "",
|
|
6702
|
-
chartjsScript: this.buildChartjsScript(),
|
|
6703
|
-
watermarkPageHeightPx: PAGE_CONTENT_HEIGHT_PX,
|
|
6704
|
-
releaseGateTally: buildFailureTally(data.analysis),
|
|
6705
|
-
suiteChartLarge: data.charts.suiteResults.length > SUITE_CHART_SMALL_MAX,
|
|
6706
|
-
suiteChartChunks: data.charts.suiteResults.length > SUITE_CHART_SMALL_MAX ? buildSuiteChartChunks(data.charts.suiteResults) : []
|
|
6707
|
-
};
|
|
6708
|
-
}
|
|
6709
|
-
buildContext(data, template, userSections, slowTestThreshold) {
|
|
6710
|
-
const baseCSS = this.fontsBundleCss + "\n" + this.readStyle("base.css");
|
|
6711
|
-
const templateCSS = this.readStyle(`${template}.css`);
|
|
6712
|
-
const sections = resolveSections(template, userSections);
|
|
6699
|
+
assembleContext(data, sections, templateCSS, opts) {
|
|
6713
6700
|
const options = deriveOptions(sections);
|
|
6714
6701
|
const slow = this.buildSlowTests(data);
|
|
6702
|
+
const tags = this.buildTagGroups(data, opts.requirementTagPattern);
|
|
6715
6703
|
return {
|
|
6716
6704
|
...data,
|
|
6717
6705
|
projects: this.enrichProjects(
|
|
6718
6706
|
data,
|
|
6719
6707
|
slow,
|
|
6720
6708
|
sections,
|
|
6721
|
-
slowTestThreshold ?? DEFAULT_OPTIONS.slowTestThreshold
|
|
6709
|
+
opts.slowTestThreshold ?? DEFAULT_OPTIONS.slowTestThreshold
|
|
6722
6710
|
),
|
|
6723
6711
|
options,
|
|
6724
6712
|
sections,
|
|
6725
|
-
tagGroups:
|
|
6713
|
+
tagGroups: tags.all,
|
|
6714
|
+
requirementTagGroups: tags.requirements,
|
|
6715
|
+
categoryTagGroups: tags.categories,
|
|
6726
6716
|
slowTests: slow.ranked,
|
|
6727
|
-
baseCSS,
|
|
6717
|
+
baseCSS: this.fontsBundleCss + "\n" + this.readStyle("base.css"),
|
|
6728
6718
|
templateCSS,
|
|
6729
6719
|
chartjsScript: this.buildChartjsScript(),
|
|
6730
6720
|
watermarkPageHeightPx: PAGE_CONTENT_HEIGHT_PX,
|
|
@@ -6736,7 +6726,7 @@ var init_engine = __esm({
|
|
|
6736
6726
|
buildChartjsScript() {
|
|
6737
6727
|
return this.chartjsInline ? `<script>${this.chartjsInline}</script>` : "<!-- Chart.js not bundled; charts disabled --><script>window.__chartsReady=true;</script>";
|
|
6738
6728
|
}
|
|
6739
|
-
buildTagGroups(data) {
|
|
6729
|
+
buildTagGroups(data, requirementTagPattern) {
|
|
6740
6730
|
const tagMap = /* @__PURE__ */ new Map();
|
|
6741
6731
|
for (const project of data.projects) {
|
|
6742
6732
|
for (const suite of this.flattenSuites(project.suites)) {
|
|
@@ -6751,11 +6741,20 @@ var init_engine = __esm({
|
|
|
6751
6741
|
}
|
|
6752
6742
|
}
|
|
6753
6743
|
}
|
|
6754
|
-
|
|
6755
|
-
|
|
6756
|
-
|
|
6757
|
-
|
|
6758
|
-
|
|
6744
|
+
const regex = new RegExp(requirementTagPattern ?? DEFAULT_OPTIONS.requirementTagPattern);
|
|
6745
|
+
const requirements = [];
|
|
6746
|
+
const categories = [];
|
|
6747
|
+
for (const [tag, counts] of tagMap) {
|
|
6748
|
+
const row = {
|
|
6749
|
+
tag,
|
|
6750
|
+
...counts,
|
|
6751
|
+
coveragePct: counts.total > 0 ? Math.round(counts.passed / counts.total * 100) : 0
|
|
6752
|
+
};
|
|
6753
|
+
(regex.test(tag) ? requirements : categories).push(row);
|
|
6754
|
+
}
|
|
6755
|
+
requirements.sort((a, b) => NUMERIC_COLLATOR.compare(a.tag, b.tag));
|
|
6756
|
+
categories.sort((a, b) => b.total - a.total || a.tag.localeCompare(b.tag));
|
|
6757
|
+
return { all: [...requirements, ...categories], requirements, categories };
|
|
6759
6758
|
}
|
|
6760
6759
|
buildSlowTests(data) {
|
|
6761
6760
|
const tests = [];
|
|
@@ -7858,7 +7857,12 @@ var init_PdfGenerator = __esm({
|
|
|
7858
7857
|
}
|
|
7859
7858
|
const templateLabel = source.kind === "builtin" ? source.id : source.label;
|
|
7860
7859
|
logger.info(`Rendering template: ${templateLabel}`);
|
|
7861
|
-
const
|
|
7860
|
+
const renderOpts = {
|
|
7861
|
+
sections: options.sections,
|
|
7862
|
+
slowTestThreshold: options.slowTestThreshold,
|
|
7863
|
+
requirementTagPattern: options.requirementTagPattern
|
|
7864
|
+
};
|
|
7865
|
+
const html = source.kind === "builtin" ? this.templateEngine.render(renderData, source.id, renderOpts) : this.templateEngine.renderFromPath(renderData, source.absolutePath, renderOpts);
|
|
7862
7866
|
const tempHtmlPath = await this.htmlWriter.write(html);
|
|
7863
7867
|
const fileUrl = this.htmlWriter.toFileUrl(tempHtmlPath);
|
|
7864
7868
|
try {
|
|
@@ -12133,6 +12137,14 @@ var init_schema = __esm({
|
|
|
12133
12137
|
remoteHistory: external_exports.boolean().default(false),
|
|
12134
12138
|
flakinessTopN: external_exports.number().int().min(0, "flakinessTopN must be 0 or greater").optional().default(DEFAULT_OPTIONS.flakinessTopN),
|
|
12135
12139
|
slowTestThreshold: external_exports.number().int().min(0, "slowTestThreshold must be 0 or greater").optional().default(DEFAULT_OPTIONS.slowTestThreshold),
|
|
12140
|
+
requirementTagPattern: external_exports.string().refine((v) => {
|
|
12141
|
+
try {
|
|
12142
|
+
new RegExp(v);
|
|
12143
|
+
return true;
|
|
12144
|
+
} catch {
|
|
12145
|
+
return false;
|
|
12146
|
+
}
|
|
12147
|
+
}, "requirementTagPattern must be a valid regular expression").optional().default(DEFAULT_OPTIONS.requirementTagPattern),
|
|
12136
12148
|
failureAnalysis: failureAnalysisConfigSchema,
|
|
12137
12149
|
capture: captureConfigSchema,
|
|
12138
12150
|
live: liveConfigSchema,
|
package/dist/index.d.mts
CHANGED
|
@@ -202,7 +202,7 @@ interface ReporterOptions {
|
|
|
202
202
|
};
|
|
203
203
|
/**
|
|
204
204
|
* Offline failure root-cause analysis. Classifies failures into categories
|
|
205
|
-
* using an embedded
|
|
205
|
+
* using an embedded classifier model and renders a clustered breakdown in
|
|
206
206
|
* the `detailed` PDF (one-liner in `executive`). No network calls at runtime.
|
|
207
207
|
* Optionally personalizes on YOUR machine: label collected failures
|
|
208
208
|
* (`label-feedback`), then `train-model` trains + evaluates a local layer that
|
|
@@ -230,9 +230,9 @@ interface ReporterOptions {
|
|
|
230
230
|
*/
|
|
231
231
|
collectScope?: 'blind-spots' | 'all';
|
|
232
232
|
/**
|
|
233
|
-
* Auto-update the BASE classifier model
|
|
234
|
-
*
|
|
235
|
-
*
|
|
233
|
+
* Auto-update the BASE classifier model (offline-tolerant; the bundled
|
|
234
|
+
* model is always the floor; delivered models are signature-verified).
|
|
235
|
+
* Set false to pin the bundled model for reproducibility.
|
|
236
236
|
* Default: true
|
|
237
237
|
*/
|
|
238
238
|
autoUpdateModel?: boolean;
|
|
@@ -335,10 +335,10 @@ interface ReporterOptions {
|
|
|
335
335
|
/**
|
|
336
336
|
* Store the pass-rate trend on the ReportForge server so it survives
|
|
337
337
|
* ephemeral CI runners (where the local history file is wiped every run).
|
|
338
|
-
* One request per run
|
|
339
|
-
*
|
|
340
|
-
*
|
|
341
|
-
*
|
|
338
|
+
* One small authenticated request per run, carrying AGGREGATE NUMBERS ONLY —
|
|
339
|
+
* pass rate, counts, duration, verdict. No test titles, no error text; even
|
|
340
|
+
* the branch name never travels in plaintext. Falls back to the local
|
|
341
|
+
* history file on any failure.
|
|
342
342
|
* @default false
|
|
343
343
|
*/
|
|
344
344
|
remoteHistory?: boolean;
|
|
@@ -357,6 +357,17 @@ interface ReporterOptions {
|
|
|
357
357
|
* @default 10
|
|
358
358
|
*/
|
|
359
359
|
slowTestThreshold?: number;
|
|
360
|
+
/**
|
|
361
|
+
* Regex (string) deciding which tags count as requirement IDs in the
|
|
362
|
+
* Requirements Traceability section. Matching tags get the traceability
|
|
363
|
+
* matrix; the rest collapse into a compact tag summary, so suite-level
|
|
364
|
+
* category tags (`@regression`, `@sanity`) no longer repeat identical
|
|
365
|
+
* rows. The default matches ticket-ID shapes such as `@ODP-5328`,
|
|
366
|
+
* `REQ-001`, or `JIRA-123` (uppercase key, dash, digits). Set to `''` to
|
|
367
|
+
* disable the split and list every tag in the matrix (legacy behavior).
|
|
368
|
+
* @default '^@?[A-Z][A-Z0-9]*-\\d+$'
|
|
369
|
+
*/
|
|
370
|
+
requirementTagPattern?: string;
|
|
360
371
|
/**
|
|
361
372
|
* Path to a custom Handlebars (.hbs) template file. When set, takes
|
|
362
373
|
* precedence over `template`. Relative paths resolve from `process.cwd()`.
|
package/dist/index.d.ts
CHANGED
|
@@ -202,7 +202,7 @@ interface ReporterOptions {
|
|
|
202
202
|
};
|
|
203
203
|
/**
|
|
204
204
|
* Offline failure root-cause analysis. Classifies failures into categories
|
|
205
|
-
* using an embedded
|
|
205
|
+
* using an embedded classifier model and renders a clustered breakdown in
|
|
206
206
|
* the `detailed` PDF (one-liner in `executive`). No network calls at runtime.
|
|
207
207
|
* Optionally personalizes on YOUR machine: label collected failures
|
|
208
208
|
* (`label-feedback`), then `train-model` trains + evaluates a local layer that
|
|
@@ -230,9 +230,9 @@ interface ReporterOptions {
|
|
|
230
230
|
*/
|
|
231
231
|
collectScope?: 'blind-spots' | 'all';
|
|
232
232
|
/**
|
|
233
|
-
* Auto-update the BASE classifier model
|
|
234
|
-
*
|
|
235
|
-
*
|
|
233
|
+
* Auto-update the BASE classifier model (offline-tolerant; the bundled
|
|
234
|
+
* model is always the floor; delivered models are signature-verified).
|
|
235
|
+
* Set false to pin the bundled model for reproducibility.
|
|
236
236
|
* Default: true
|
|
237
237
|
*/
|
|
238
238
|
autoUpdateModel?: boolean;
|
|
@@ -335,10 +335,10 @@ interface ReporterOptions {
|
|
|
335
335
|
/**
|
|
336
336
|
* Store the pass-rate trend on the ReportForge server so it survives
|
|
337
337
|
* ephemeral CI runners (where the local history file is wiped every run).
|
|
338
|
-
* One request per run
|
|
339
|
-
*
|
|
340
|
-
*
|
|
341
|
-
*
|
|
338
|
+
* One small authenticated request per run, carrying AGGREGATE NUMBERS ONLY —
|
|
339
|
+
* pass rate, counts, duration, verdict. No test titles, no error text; even
|
|
340
|
+
* the branch name never travels in plaintext. Falls back to the local
|
|
341
|
+
* history file on any failure.
|
|
342
342
|
* @default false
|
|
343
343
|
*/
|
|
344
344
|
remoteHistory?: boolean;
|
|
@@ -357,6 +357,17 @@ interface ReporterOptions {
|
|
|
357
357
|
* @default 10
|
|
358
358
|
*/
|
|
359
359
|
slowTestThreshold?: number;
|
|
360
|
+
/**
|
|
361
|
+
* Regex (string) deciding which tags count as requirement IDs in the
|
|
362
|
+
* Requirements Traceability section. Matching tags get the traceability
|
|
363
|
+
* matrix; the rest collapse into a compact tag summary, so suite-level
|
|
364
|
+
* category tags (`@regression`, `@sanity`) no longer repeat identical
|
|
365
|
+
* rows. The default matches ticket-ID shapes such as `@ODP-5328`,
|
|
366
|
+
* `REQ-001`, or `JIRA-123` (uppercase key, dash, digits). Set to `''` to
|
|
367
|
+
* disable the split and list every tag in the matrix (legacy behavior).
|
|
368
|
+
* @default '^@?[A-Z][A-Z0-9]*-\\d+$'
|
|
369
|
+
*/
|
|
370
|
+
requirementTagPattern?: string;
|
|
360
371
|
/**
|
|
361
372
|
* Path to a custom Handlebars (.hbs) template file. When set, takes
|
|
362
373
|
* precedence over `template`. Relative paths resolve from `process.cwd()`.
|