reporting-labs 0.4.3 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +100 -2
- package/bin/cli.js +6 -1
- package/bin/config-template.txt +3 -0
- package/bin/merge.js +173 -0
- package/dist/reporter.d.ts +6 -1
- package/dist/reporter.js +53 -7
- package/dist/types.d.ts +8 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -8,6 +8,10 @@ reportingLabs turns a test run into a single HTML file. No server. No upload. No
|
|
|
8
8
|
|
|
9
9
|
Today it ships with a **Playwright** reporter. WebdriverIO, Cypress and Jest/Vitest are on the roadmap.
|
|
10
10
|
|
|
11
|
+
<p align="center"><img src="https://raw.githubusercontent.com/naveenautomationlabs/reporting-labs/main/docs/quickstart.gif" alt="reportingLabs in three steps: install, tag your tests, open one HTML file" width="900"></p>
|
|
12
|
+
|
|
13
|
+
<p align="center"><em>Install, tag your tests, open one HTML file.</em></p>
|
|
14
|
+
|
|
11
15
|
<p align="center"><img src="https://raw.githubusercontent.com/naveenautomationlabs/reporting-labs/main/docs/overview.png" alt="Overview page of a reportingLabs report" width="900"></p>
|
|
12
16
|
|
|
13
17
|
## Quick start (2 minutes)
|
|
@@ -46,9 +50,10 @@ Your other reporters (list, html, blob...) keep working as before.
|
|
|
46
50
|
|
|
47
51
|
```bash
|
|
48
52
|
npx playwright test
|
|
49
|
-
open reporting-labs/index.html
|
|
50
53
|
```
|
|
51
54
|
|
|
55
|
+
The report opens in your browser when something failed (`open: 'always'` to open every time, `'never'` to stay quiet). It is at `reporting-labs/index.html`.
|
|
56
|
+
|
|
52
57
|
Done. Everything below is optional.
|
|
53
58
|
|
|
54
59
|
> **In a hurry?** Skip the config file and pass options inline: `reporter: [['reporting-labs', { title: 'My app' }]]`.
|
|
@@ -84,6 +89,7 @@ One line per test. With this the report can rank failures by priority, group the
|
|
|
84
89
|
- Known keys: `priority`, `severity`, `owner`, `feature`, `epic`, `story`, `issue`, `component`, `team`. Any other key you pass is shown too.
|
|
85
90
|
- Your Playwright tags like `@sanity` or `@regression` stay as they are and still show on the test.
|
|
86
91
|
- To make story and epic keys clickable, set `links` in the config: `links: { story: 'https://yourteam.atlassian.net/browse/{id}' }`.
|
|
92
|
+
- Forgot one? After every run the console lists the tests that have no `meta()`, with file and line. Turn it off with `warnMissingMeta: false`.
|
|
87
93
|
- `priority`, `severity`, `feature` and `owner` each get a tab in the Breakdown chart and a filter on the Tests page. Want the same for your own key, say `meta({ team: 'web' })`? Add it to `dimensions` in the config: `dimensions: ['priority', 'severity', 'feature', 'owner', 'team']`.
|
|
88
94
|
|
|
89
95
|
### `log()`: a line in the report
|
|
@@ -296,8 +302,12 @@ Every option is optional. `npx reporting-labs init` writes them all, with commen
|
|
|
296
302
|
| `embedAttachments` | `true` | Screenshots inside the HTML (one file) |
|
|
297
303
|
| `embedLimit` | 2 MB | Bigger attachments are copied to `./assets` |
|
|
298
304
|
| `embedVideos` | `false` | Videos inside the HTML too (bigger file, no folder issues) |
|
|
305
|
+
| `emitJson` | `true` | Also write `report.json` alongside `index.html` (used by `merge`) |
|
|
306
|
+
| `jsonFile` | `'report.json'` | File name of the JSON blob |
|
|
299
307
|
| `embedFonts` | `true` | Bundle the fonts (~140 KB) so it looks the same offline |
|
|
300
308
|
| `announce` | `true` | Print the report path after the run |
|
|
309
|
+
| `open` | `'on-failure'` | Open the report in the browser after the run: `'on-failure'`, `'always'` or `'never'`. Never opens in CI |
|
|
310
|
+
| `warnMissingMeta` | `true` | After the run, list the tests that have no `meta()` in the console, so nobody on the team forgets |
|
|
301
311
|
|
|
302
312
|
If your reporter list differs between CI and local, add the same line to both:
|
|
303
313
|
|
|
@@ -322,7 +332,7 @@ Two things to set up:
|
|
|
322
332
|
1. **Publish the report.** Upload the `reporting-labs/` folder as a build artifact (or archive it in Jenkins). Videos and large files sit in `reporting-labs/assets/`, so keep the folder together.
|
|
323
333
|
2. **Keep the history.** `reporting-labs.history.json` powers the trend and the new vs known failures. On GitHub Actions save it with `actions/cache`. On Jenkins the workspace usually keeps it on its own.
|
|
324
334
|
|
|
325
|
-
Ready-to-copy samples: [
|
|
335
|
+
Ready-to-copy samples: [github-actions.yml](https://github.com/naveenautomationlabs/reporting-labs/blob/main/docs/ci/github-actions.yml), [Jenkinsfile](https://github.com/naveenautomationlabs/reporting-labs/blob/main/docs/ci/Jenkinsfile) and [gitlab-ci.yml](https://github.com/naveenautomationlabs/reporting-labs/blob/main/docs/ci/gitlab-ci.yml). Each one includes an optional Slack (and, for Jenkins, email) step you can uncomment.
|
|
326
336
|
|
|
327
337
|
```yaml
|
|
328
338
|
# GitHub Actions, the two steps that matter
|
|
@@ -340,6 +350,94 @@ Ready-to-copy samples: [docs/ci/github-actions.yml](https://github.com/naveenaut
|
|
|
340
350
|
|
|
341
351
|
**Jenkins note.** Jenkins blocks inline JavaScript by default, so a single-file report shows up blank inside the Jenkins HTML Publisher (Playwright's own HTML report has the same issue). Download the archived artifact and open it locally, or ask an admin to relax the policy in the script console: `System.setProperty("hudson.model.DirectoryBrowserSupport.CSP", "")`.
|
|
342
352
|
|
|
353
|
+
### Split your run across shards, then merge into one report
|
|
354
|
+
|
|
355
|
+
Big test suite? Run it faster by splitting it into shards. Each shard is a separate CI job, all running at the same time. When they finish, join them into one report.
|
|
356
|
+
|
|
357
|
+
**How it works:**
|
|
358
|
+
|
|
359
|
+
1. Each shard runs your tests. Each one writes its own `reporting-labs/index.html` and a small `report.json` next to it. Save each shard's folder as a CI artifact.
|
|
360
|
+
2. In one last job, download all the shard folders and run one command:
|
|
361
|
+
|
|
362
|
+
```bash
|
|
363
|
+
npx reporting-labs merge ./all-shards -o merged/
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
You get one report with:
|
|
367
|
+
|
|
368
|
+
- All tests from every shard in one list, sorted by priority
|
|
369
|
+
- Combined pass / fail / flaky numbers on top
|
|
370
|
+
- One Trend chart, one Environment card, one Failure Clusters view
|
|
371
|
+
- Screenshots and videos kept in `merged/assets/shard-1-of-4/`, `merged/assets/shard-2-of-4/`, so nothing overwrites
|
|
372
|
+
|
|
373
|
+
**GitHub Actions example** (replace `npx playwright test` with your own command):
|
|
374
|
+
|
|
375
|
+
```yaml
|
|
376
|
+
jobs:
|
|
377
|
+
test:
|
|
378
|
+
strategy:
|
|
379
|
+
matrix: { shard: [1, 2, 3, 4] }
|
|
380
|
+
runs-on: ubuntu-latest
|
|
381
|
+
steps:
|
|
382
|
+
- uses: actions/checkout@v4
|
|
383
|
+
- uses: actions/setup-node@v4
|
|
384
|
+
with: { node-version: 20, cache: npm }
|
|
385
|
+
- run: npm ci
|
|
386
|
+
- run: npx playwright install --with-deps
|
|
387
|
+
- run: npx playwright test --shard=${{ matrix.shard }}/4
|
|
388
|
+
- uses: actions/upload-artifact@v4
|
|
389
|
+
if: always()
|
|
390
|
+
with:
|
|
391
|
+
name: report-shard-${{ matrix.shard }}
|
|
392
|
+
path: reporting-labs/
|
|
393
|
+
retention-days: 30
|
|
394
|
+
|
|
395
|
+
merge:
|
|
396
|
+
if: always()
|
|
397
|
+
needs: test
|
|
398
|
+
runs-on: ubuntu-latest
|
|
399
|
+
steps:
|
|
400
|
+
- uses: actions/checkout@v4
|
|
401
|
+
- uses: actions/setup-node@v4
|
|
402
|
+
with: { node-version: 20, cache: npm }
|
|
403
|
+
- run: npm ci
|
|
404
|
+
- uses: actions/download-artifact@v4
|
|
405
|
+
with: { path: all-shards, pattern: 'report-shard-*' }
|
|
406
|
+
- run: npx reporting-labs merge all-shards -o merged/
|
|
407
|
+
- uses: actions/upload-artifact@v4
|
|
408
|
+
with: { name: merged-report, path: merged/, retention-days: 30 }
|
|
409
|
+
```
|
|
410
|
+
|
|
411
|
+
**Notes:**
|
|
412
|
+
|
|
413
|
+
- The reporter always writes `report.json` alongside `index.html`, so `merge` just works. Turn it off with `emitJson: false` if you do not want it.
|
|
414
|
+
- Not using shards? Ignore this section. The single-run report keeps working exactly as before.
|
|
415
|
+
|
|
416
|
+
### Slack, email, Teams: use your CI's own integration
|
|
417
|
+
|
|
418
|
+
reportingLabs deliberately does not build its own Slack or email sender. Every CI already has a first-class integration you can lean on, and it stays out of the way of the report itself:
|
|
419
|
+
|
|
420
|
+
- **GitHub Actions:** `slackapi/slack-github-action` posts a message with the run URL and artifact link. `dawidd6/action-send-mail` handles email. Both are one YAML block, both take a repo secret. See the [github-actions.yml sample](https://github.com/naveenautomationlabs/reporting-labs/blob/main/docs/ci/github-actions.yml).
|
|
421
|
+
- **Jenkins:** the Slack Notification plugin (`slackSend`) and the Email Extension plugin (`emailext`) do the same, plus they can attach `reporting-labs/index.html`. See the [Jenkinsfile sample](https://github.com/naveenautomationlabs/reporting-labs/blob/main/docs/ci/Jenkinsfile).
|
|
422
|
+
- **GitLab CI:** the Slack integration in Project Settings posts pipeline results with no code at all. For a rich message, add a `notify` job with `curl` to your webhook.
|
|
423
|
+
- **CircleCI, Bitbucket, Azure Pipelines:** each has a native Slack orb / task; a plain `curl` to the webhook also works from any shell step.
|
|
424
|
+
|
|
425
|
+
Two reasons this stays outside the reporter:
|
|
426
|
+
|
|
427
|
+
1. Your admin most likely already set up notifications for build and deploy. The same channel serves test results with no new moving parts.
|
|
428
|
+
2. Notification delivery (auth, TLS, retries, corporate relays) is a full topic on its own. CI integrations handle it, this reporter stays a single HTML file.
|
|
429
|
+
|
|
430
|
+
If you need a richer machine-readable summary in the same message, add Playwright's own JSON reporter next to reportingLabs and pipe it to your Slack step:
|
|
431
|
+
|
|
432
|
+
```ts
|
|
433
|
+
reporter: [
|
|
434
|
+
['reporting-labs', require('./reporting-labs.config').default],
|
|
435
|
+
['json', { outputFile: 'results.json' }],
|
|
436
|
+
]
|
|
437
|
+
```
|
|
438
|
+
|
|
439
|
+
Your Slack step can then read `results.json` for pass / fail counts and top failures.
|
|
440
|
+
|
|
343
441
|
## Good to know
|
|
344
442
|
|
|
345
443
|
- **One file.** Screenshots and fonts are inside `index.html`, so it works from an email or a CI artifact. Videos and large files go to `./assets` next to it. Keep the folder together when you share it.
|
package/bin/cli.js
CHANGED
|
@@ -5,7 +5,11 @@ const args = process.argv.slice(2);
|
|
|
5
5
|
const cmd = args[0];
|
|
6
6
|
const has = f => args.includes(f);
|
|
7
7
|
|
|
8
|
-
if (cmd === '
|
|
8
|
+
if (cmd === 'merge') {
|
|
9
|
+
const { merge, parseArgs } = require('./merge');
|
|
10
|
+
const { dirs, opts } = parseArgs(args.slice(1));
|
|
11
|
+
merge(dirs, opts);
|
|
12
|
+
} else if (cmd === 'init') {
|
|
9
13
|
const js = has('--js');
|
|
10
14
|
const file = path.resolve(js ? 'reporting-labs.config.js' : 'reporting-labs.config.ts');
|
|
11
15
|
if (fs.existsSync(file) && !has('--force')) {
|
|
@@ -37,6 +41,7 @@ if (cmd === 'init') {
|
|
|
37
41
|
npx reporting-labs init write reporting-labs.config.ts with every option, commented
|
|
38
42
|
npx reporting-labs init --js same, as reporting-labs.config.js
|
|
39
43
|
npx reporting-labs init --force overwrite an existing config file
|
|
44
|
+
npx reporting-labs merge <dirs> combine several shard runs into one report (see 'merge --help')
|
|
40
45
|
|
|
41
46
|
Then in playwright.config.ts:
|
|
42
47
|
import reportingLabs from './reporting-labs.config';
|
package/bin/config-template.txt
CHANGED
|
@@ -31,10 +31,13 @@ const config: ReportingLabsOptions = {
|
|
|
31
31
|
// ── Output ───────────────────────────────────────────────────────────────────
|
|
32
32
|
// outputFolder: 'reporting-labs', // where index.html and copied attachments go
|
|
33
33
|
// outputFile: 'index.html',
|
|
34
|
+
// emitJson: true, // also write report.json (used by `reporting-labs merge` for sharded runs)
|
|
34
35
|
// embedAttachments: true, // inline screenshots as base64: one file, opens anywhere
|
|
35
36
|
// embedLimit: 2 * 1024 * 1024, // attachments bigger than this (bytes) are copied as files
|
|
36
37
|
// embedVideos: false, // true = videos inside the HTML too (bigger file, no folder issues)
|
|
37
38
|
// announce: true, // print the report path after the run
|
|
39
|
+
// open: 'on-failure', // open the report in the browser: 'on-failure' | 'always' | 'never'; never in CI
|
|
40
|
+
// warnMissingMeta: true, // after the run, list tests that have no meta() in the console
|
|
38
41
|
|
|
39
42
|
// ── Test details ─────────────────────────────────────────────────────────────
|
|
40
43
|
// Values come from meta({ priority, severity, owner, feature, epic, story, ... }) in your tests.
|
package/bin/merge.js
ADDED
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
// Merge several shard runs into one report.
|
|
2
|
+
// Usage: node bin/merge.js <shardDir1> <shardDir2> ... [-o <outDir>]
|
|
3
|
+
// (invoked by bin/cli.js as `npx reporting-labs merge ...`)
|
|
4
|
+
|
|
5
|
+
'use strict';
|
|
6
|
+
const fs = require('fs');
|
|
7
|
+
const path = require('path');
|
|
8
|
+
|
|
9
|
+
function usage(exit = 0) {
|
|
10
|
+
console.log(
|
|
11
|
+
`reporting-labs merge
|
|
12
|
+
|
|
13
|
+
Combine several shard runs into one report.
|
|
14
|
+
|
|
15
|
+
Usage:
|
|
16
|
+
npx reporting-labs merge <shard-dir> [<shard-dir> ...] [--out <dir>] [--file <name>]
|
|
17
|
+
|
|
18
|
+
Arguments:
|
|
19
|
+
<shard-dir> A folder that contains report.json (the shard's reporting-labs output).
|
|
20
|
+
You can also pass a parent folder that contains many shards, e.g.
|
|
21
|
+
"npx reporting-labs merge ./all-shards" where each subfolder is a shard.
|
|
22
|
+
|
|
23
|
+
Options:
|
|
24
|
+
--out, -o <dir> Where the merged report is written. Default: reporting-labs-merged
|
|
25
|
+
--file <name> Output HTML file name. Default: index.html
|
|
26
|
+
--title <text> Override the report title. Default: the first shard's title
|
|
27
|
+
--help, -h Show this message
|
|
28
|
+
|
|
29
|
+
Example (GitHub Actions):
|
|
30
|
+
# each shard uploads its own reporting-labs/ folder as an artifact,
|
|
31
|
+
# a final job downloads them all under ./all-shards/, then:
|
|
32
|
+
npx reporting-labs merge ./all-shards -o merged/`);
|
|
33
|
+
process.exit(exit);
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
function parseArgs(argv) {
|
|
37
|
+
const dirs = [];
|
|
38
|
+
const opts = { out: 'reporting-labs-merged', file: 'index.html', title: undefined };
|
|
39
|
+
for (let i = 0; i < argv.length; i++) {
|
|
40
|
+
const a = argv[i];
|
|
41
|
+
if (a === '--help' || a === '-h') usage(0);
|
|
42
|
+
else if (a === '--out' || a === '-o') opts.out = argv[++i];
|
|
43
|
+
else if (a === '--file') opts.file = argv[++i];
|
|
44
|
+
else if (a === '--title') opts.title = argv[++i];
|
|
45
|
+
else if (a.startsWith('-')) { console.error('unknown flag: ' + a); usage(1); }
|
|
46
|
+
else dirs.push(a);
|
|
47
|
+
}
|
|
48
|
+
if (!dirs.length) usage(1);
|
|
49
|
+
return { dirs, opts };
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
/** Expand each arg: if the arg is a shard dir (has report.json) keep it; otherwise treat as a parent and expand its immediate subdirs that have report.json. */
|
|
53
|
+
function resolveShardDirs(inputs) {
|
|
54
|
+
const out = [];
|
|
55
|
+
for (const p of inputs) {
|
|
56
|
+
const abs = path.resolve(p);
|
|
57
|
+
if (!fs.existsSync(abs)) { console.error('missing:', abs); process.exit(1); }
|
|
58
|
+
if (fs.existsSync(path.join(abs, 'report.json'))) out.push(abs);
|
|
59
|
+
else {
|
|
60
|
+
const kids = fs.readdirSync(abs).map(k => path.join(abs, k)).filter(k => fs.statSync(k).isDirectory() && fs.existsSync(path.join(k, 'report.json')));
|
|
61
|
+
if (!kids.length) { console.error(`no report.json under ${abs}, nor in any subfolder`); process.exit(1); }
|
|
62
|
+
out.push(...kids);
|
|
63
|
+
}
|
|
64
|
+
}
|
|
65
|
+
return out;
|
|
66
|
+
}
|
|
67
|
+
|
|
68
|
+
function rewriteAttachmentPath(a, prefix) {
|
|
69
|
+
if (!a) return a;
|
|
70
|
+
// The reporter emits 'src' pointing at 'assets/<name>' for file-backed attachments;
|
|
71
|
+
// namespace it under the shard's folder so multiple shards can share one merged assets/.
|
|
72
|
+
const src = a.src;
|
|
73
|
+
if (typeof src === 'string' && src.startsWith('assets/')) return { ...a, src: 'assets/' + prefix + '/' + src.slice('assets/'.length) };
|
|
74
|
+
return a;
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
function copyDir(from, to) {
|
|
78
|
+
if (!fs.existsSync(from)) return;
|
|
79
|
+
fs.mkdirSync(to, { recursive: true });
|
|
80
|
+
for (const entry of fs.readdirSync(from, { withFileTypes: true })) {
|
|
81
|
+
const src = path.join(from, entry.name), dst = path.join(to, entry.name);
|
|
82
|
+
if (entry.isDirectory()) copyDir(src, dst);
|
|
83
|
+
else fs.copyFileSync(src, dst);
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
function shardLabel(dir, i, data) {
|
|
88
|
+
if (data && data.shard && data.shard.total) return `shard-${data.shard.current}-of-${data.shard.total}`;
|
|
89
|
+
return `shard-${i + 1}`;
|
|
90
|
+
}
|
|
91
|
+
|
|
92
|
+
function merge(inputs, opts) {
|
|
93
|
+
const dirs = resolveShardDirs(inputs);
|
|
94
|
+
console.log(`Merging ${dirs.length} shard${dirs.length === 1 ? '' : 's'} into ${opts.out}`);
|
|
95
|
+
|
|
96
|
+
const shards = dirs.map((d, i) => {
|
|
97
|
+
const data = JSON.parse(fs.readFileSync(path.join(d, 'report.json'), 'utf8'));
|
|
98
|
+
return { dir: d, label: shardLabel(d, i, data), data };
|
|
99
|
+
});
|
|
100
|
+
|
|
101
|
+
const first = shards[0].data;
|
|
102
|
+
// Compose combined data. Start from the first shard's options/env, then merge in test rows.
|
|
103
|
+
const combined = {
|
|
104
|
+
...first,
|
|
105
|
+
title: opts.title || first.title,
|
|
106
|
+
startTime: Math.min(...shards.map(s => s.data.startTime || 0)),
|
|
107
|
+
duration: Math.max(...shards.map(s => s.data.duration || 0)),
|
|
108
|
+
generatedAt: Date.now(),
|
|
109
|
+
stats: { passed: 0, failed: 0, skipped: 0, flaky: 0, timedOut: 0, interrupted: 0, total: 0 },
|
|
110
|
+
tests: [],
|
|
111
|
+
globalErrors: [],
|
|
112
|
+
globalOutput: [],
|
|
113
|
+
projects: [],
|
|
114
|
+
workers: shards.reduce((n, s) => n + (s.data.workers || 0), 0),
|
|
115
|
+
shard: undefined, // no longer sharded
|
|
116
|
+
env: [...(first.env || [])],
|
|
117
|
+
history: first.history || [], // all shards share the same history file
|
|
118
|
+
};
|
|
119
|
+
|
|
120
|
+
const projSet = new Set();
|
|
121
|
+
for (const s of shards) {
|
|
122
|
+
const stats = s.data.stats || {};
|
|
123
|
+
for (const k of Object.keys(combined.stats)) combined.stats[k] += stats[k] || 0;
|
|
124
|
+
(s.data.projects || []).forEach(p => projSet.add(p));
|
|
125
|
+
|
|
126
|
+
for (const t of (s.data.tests || [])) {
|
|
127
|
+
const nt = {
|
|
128
|
+
...t,
|
|
129
|
+
results: (t.results || []).map(r => ({
|
|
130
|
+
...r,
|
|
131
|
+
attachments: (r.attachments || []).map(a => rewriteAttachmentPath(a, s.label)),
|
|
132
|
+
})),
|
|
133
|
+
};
|
|
134
|
+
combined.tests.push(nt);
|
|
135
|
+
}
|
|
136
|
+
for (const e of (s.data.globalErrors || [])) combined.globalErrors.push(e);
|
|
137
|
+
for (const o of (s.data.globalOutput || [])) combined.globalOutput.push(o);
|
|
138
|
+
}
|
|
139
|
+
combined.projects = [...projSet].sort();
|
|
140
|
+
combined.env.push({ k: 'Shards', v: shards.map(s => s.label).join(', ') });
|
|
141
|
+
if (combined.runStatus === undefined || combined.runStatus === 'passed') {
|
|
142
|
+
const anyFail = shards.some(s => s.data.runStatus && s.data.runStatus !== 'passed');
|
|
143
|
+
if (anyFail) combined.runStatus = 'failed';
|
|
144
|
+
}
|
|
145
|
+
|
|
146
|
+
// Prepare output folder
|
|
147
|
+
fs.mkdirSync(opts.out, { recursive: true });
|
|
148
|
+
const assetsRoot = path.join(opts.out, 'assets');
|
|
149
|
+
fs.rmSync(assetsRoot, { recursive: true, force: true });
|
|
150
|
+
|
|
151
|
+
for (const s of shards) {
|
|
152
|
+
const src = path.join(s.dir, 'assets');
|
|
153
|
+
if (fs.existsSync(src)) copyDir(src, path.join(assetsRoot, s.label));
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
// Render combined HTML using the reporter's own template.
|
|
157
|
+
const { renderHtml } = require(path.resolve(__dirname, '..', 'dist', 'template.js'));
|
|
158
|
+
const outFile = path.join(opts.out, opts.file);
|
|
159
|
+
fs.writeFileSync(outFile, renderHtml(combined), 'utf8');
|
|
160
|
+
fs.writeFileSync(path.join(opts.out, 'report.json'), JSON.stringify(combined), 'utf8');
|
|
161
|
+
|
|
162
|
+
const st = combined.stats;
|
|
163
|
+
const rel = path.relative(process.cwd(), outFile);
|
|
164
|
+
console.log(` merged: ${st.total} tests · ${st.passed} passed · ${st.failed + st.timedOut + st.interrupted} failed · ${st.flaky} flaky · ${st.skipped} skipped`);
|
|
165
|
+
console.log(` written to ${rel}`);
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
module.exports = { merge, parseArgs };
|
|
169
|
+
|
|
170
|
+
if (require.main === module) {
|
|
171
|
+
const { dirs, opts } = parseArgs(process.argv.slice(2));
|
|
172
|
+
merge(dirs, opts);
|
|
173
|
+
}
|
package/dist/reporter.d.ts
CHANGED
|
@@ -20,7 +20,12 @@ export default class ReportingLabsReporter implements Reporter {
|
|
|
20
20
|
private pushOutput;
|
|
21
21
|
onBegin(config: FullConfig, suite: Suite): void;
|
|
22
22
|
onEnd(result: FullResult): Promise<void>;
|
|
23
|
-
/**
|
|
23
|
+
/** One short list of tests that carry no meta() at all, so the whole team keeps the report useful. */
|
|
24
|
+
private printMissingMeta;
|
|
25
|
+
/** Open the report in the default browser, like Playwright's HTML reporter. Never in CI. */
|
|
26
|
+
private maybeOpen;
|
|
27
|
+
/** Videos and traces are finalized asynchronously by the runner; wait until every file-backed attachment stops growing.
|
|
28
|
+
* All files are polled in parallel so total wait is bounded by the slowest single file, not the sum. */
|
|
24
29
|
private settleAttachmentFiles;
|
|
25
30
|
/** Runtime facts for the Environment card: Playwright, Node, OS, browsers, CI job, git commit. */
|
|
26
31
|
private collectEnv;
|
package/dist/reporter.js
CHANGED
|
@@ -202,13 +202,56 @@ class ReportingLabsReporter {
|
|
|
202
202
|
};
|
|
203
203
|
const file = path.join(this.outDir, this.options.outputFile ?? 'index.html');
|
|
204
204
|
fs.writeFileSync(file, (0, template_1.renderHtml)(data), 'utf8');
|
|
205
|
+
if (this.options.emitJson !== false) {
|
|
206
|
+
const jsonFile = path.join(this.outDir, this.options.jsonFile ?? 'report.json');
|
|
207
|
+
fs.writeFileSync(jsonFile, JSON.stringify(data), 'utf8');
|
|
208
|
+
}
|
|
205
209
|
if (this.options.announce !== false) {
|
|
206
210
|
const rel = path.relative(process.cwd(), file);
|
|
207
|
-
console.log(`\n reporting-labs: report written to ${rel}
|
|
211
|
+
console.log(`\n reporting-labs: report written to ${rel}`);
|
|
212
|
+
this.printMissingMeta(tests);
|
|
213
|
+
console.log('');
|
|
214
|
+
}
|
|
215
|
+
this.maybeOpen(file, result);
|
|
216
|
+
}
|
|
217
|
+
/** One short list of tests that carry no meta() at all, so the whole team keeps the report useful. */
|
|
218
|
+
printMissingMeta(tests) {
|
|
219
|
+
if (this.options.warnMissingMeta === false)
|
|
220
|
+
return;
|
|
221
|
+
const seen = new Set();
|
|
222
|
+
const missing = tests.filter(t => Object.keys(t.meta).length === 0 && !seen.has(t.file + ':' + t.line) && seen.add(t.file + ':' + t.line));
|
|
223
|
+
if (!missing.length)
|
|
224
|
+
return;
|
|
225
|
+
const total = new Set(tests.map(t => t.file + ':' + t.line)).size;
|
|
226
|
+
console.log(` reporting-labs: ${missing.length} of ${total} tests have no meta()`);
|
|
227
|
+
const show = missing.slice(0, 15);
|
|
228
|
+
const w = Math.max(...show.map(t => (t.file + ':' + t.line).length));
|
|
229
|
+
for (const t of show)
|
|
230
|
+
console.log(` ${(t.file + ':' + t.line).padEnd(w)} ${t.title}`);
|
|
231
|
+
if (missing.length > show.length)
|
|
232
|
+
console.log(` … and ${missing.length - show.length} more`);
|
|
233
|
+
console.log(" Add meta({ priority: 'P1', owner: 'name', feature: 'area' }) at the top of the test. Set warnMissingMeta: false to hide this.");
|
|
234
|
+
}
|
|
235
|
+
/** Open the report in the default browser, like Playwright's HTML reporter. Never in CI. */
|
|
236
|
+
maybeOpen(file, result) {
|
|
237
|
+
const mode = this.options.open ?? 'on-failure';
|
|
238
|
+
if (mode === 'never' || process.env.CI)
|
|
239
|
+
return;
|
|
240
|
+
if (mode === 'on-failure' && result.status === 'passed')
|
|
241
|
+
return;
|
|
242
|
+
try {
|
|
243
|
+
const cmd = process.platform === 'darwin' ? ['open', [file]]
|
|
244
|
+
: process.platform === 'win32' ? ['cmd', ['/c', 'start', '', file]]
|
|
245
|
+
: ['xdg-open', [file]];
|
|
246
|
+
const child = (0, child_process_1.spawn)(cmd[0], [...cmd[1]], { detached: true, stdio: 'ignore' });
|
|
247
|
+
child.on('error', () => { });
|
|
248
|
+
child.unref();
|
|
208
249
|
}
|
|
250
|
+
catch { /* ignore */ }
|
|
209
251
|
}
|
|
210
252
|
// ---- helpers -------------------------------------------------------------
|
|
211
|
-
/** Videos and traces are finalized asynchronously by the runner; wait until every file-backed attachment stops growing.
|
|
253
|
+
/** Videos and traces are finalized asynchronously by the runner; wait until every file-backed attachment stops growing.
|
|
254
|
+
* All files are polled in parallel so total wait is bounded by the slowest single file, not the sum. */
|
|
212
255
|
async settleAttachmentFiles() {
|
|
213
256
|
const paths = new Set();
|
|
214
257
|
for (const test of this.suite.allTests())
|
|
@@ -216,6 +259,8 @@ class ReportingLabsReporter {
|
|
|
216
259
|
for (const a of r.attachments)
|
|
217
260
|
if (a.path)
|
|
218
261
|
paths.add(a.path);
|
|
262
|
+
if (!paths.size)
|
|
263
|
+
return;
|
|
219
264
|
const size = (p) => { try {
|
|
220
265
|
return fs.statSync(p).size;
|
|
221
266
|
}
|
|
@@ -223,18 +268,19 @@ class ReportingLabsReporter {
|
|
|
223
268
|
return -1;
|
|
224
269
|
} };
|
|
225
270
|
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
|
|
226
|
-
|
|
271
|
+
const wait = async (p) => {
|
|
227
272
|
let last = size(p);
|
|
273
|
+
if (last < 0)
|
|
274
|
+
return; // never appeared; nothing to wait for
|
|
228
275
|
for (let i = 0; i < 20; i++) { // up to ~5 s per file
|
|
229
|
-
if (last < 0)
|
|
230
|
-
break; // not on disk (yet); nothing to wait for
|
|
231
276
|
await sleep(250);
|
|
232
277
|
const now = size(p);
|
|
233
278
|
if (now === last && now > 0)
|
|
234
|
-
|
|
279
|
+
return; // stable and non-empty: settled
|
|
235
280
|
last = now;
|
|
236
281
|
}
|
|
237
|
-
}
|
|
282
|
+
};
|
|
283
|
+
await Promise.all([...paths].map(wait));
|
|
238
284
|
}
|
|
239
285
|
/** Runtime facts for the Environment card: Playwright, Node, OS, browsers, CI job, git commit. */
|
|
240
286
|
collectEnv(base) {
|
package/dist/types.d.ts
CHANGED
|
@@ -13,6 +13,10 @@ export interface ReportingLabsOptions {
|
|
|
13
13
|
outputFolder?: string;
|
|
14
14
|
/** Output HTML file name inside outputFolder. Default: "index.html" */
|
|
15
15
|
outputFile?: string;
|
|
16
|
+
/** Also write a machine-readable `report.json` alongside `index.html`. Used by `reporting-labs merge` to combine shard runs. Default: true. */
|
|
17
|
+
emitJson?: boolean;
|
|
18
|
+
/** File name of the JSON blob inside outputFolder. Default: "report.json" */
|
|
19
|
+
jsonFile?: string;
|
|
16
20
|
/** Inline screenshots as base64 (single file, opens anywhere). Default: true */
|
|
17
21
|
embedAttachments?: boolean;
|
|
18
22
|
/** Max size (bytes) of a single attachment to embed. Larger ones are copied as files. Default: 2 MB */
|
|
@@ -58,6 +62,10 @@ export interface ReportingLabsOptions {
|
|
|
58
62
|
embedFonts?: boolean;
|
|
59
63
|
/** Print report path to the console after the run. Default: true */
|
|
60
64
|
announce?: boolean;
|
|
65
|
+
/** After the run, list the tests that have no meta() in the console. Default: true */
|
|
66
|
+
warnMissingMeta?: boolean;
|
|
67
|
+
/** Open the report in the browser after the run: 'on-failure' (default), 'always' or 'never'. Never opens in CI. */
|
|
68
|
+
open?: 'on-failure' | 'always' | 'never';
|
|
61
69
|
/** Company / project block in the header. */
|
|
62
70
|
project?: {
|
|
63
71
|
name?: string;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "reporting-labs",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.6.0",
|
|
4
4
|
"description": "reportingLabs – beautiful, customizable single-file HTML test reports. One line of config. Playwright adapter included, more runners on the way.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"test-report",
|