testdossier 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +45 -14
- package/lib/testdossier.mjs +354 -52
- package/package.json +4 -7
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 TestDossier
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,21 +1,45 @@
|
|
|
1
1
|
# TestDossier CLI
|
|
2
2
|
|
|
3
|
-
Upload a completed test report from a developer laptop, test machine, or CI
|
|
3
|
+
Upload a completed test report from a developer laptop, test machine, or CI
|
|
4
|
+
job. Run these steps in the project containing your automated tests:
|
|
4
5
|
|
|
5
|
-
Create `.env.testdossier
|
|
6
|
+
1. Create `.env.testdossier`:
|
|
6
7
|
|
|
7
8
|
```dotenv
|
|
8
9
|
TD_CI_TOKEN=replace_with_your_td_token
|
|
9
10
|
```
|
|
10
11
|
|
|
11
|
-
Replace the placeholder with the CI-ingestion token shown once by
|
|
12
|
-
|
|
12
|
+
2. Replace the placeholder with the CI-ingestion token shown once by
|
|
13
|
+
TestDossier, add the file to `.gitignore`, and protect it:
|
|
13
14
|
|
|
14
15
|
```bash
|
|
15
16
|
chmod 600 .env.testdossier
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
3. Run your tests so they create a supported report. For example, with
|
|
20
|
+
Playwright's JSON reporter configured to write `playwright-results.json`:
|
|
21
|
+
|
|
22
|
+
```bash
|
|
23
|
+
npx playwright test
|
|
24
|
+
test -f playwright-results.json
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
4. Upload the completed report:
|
|
28
|
+
|
|
29
|
+
```bash
|
|
16
30
|
npx testdossier upload playwright-results.json --env-file .env.testdossier
|
|
17
31
|
```
|
|
18
32
|
|
|
33
|
+
`playwright-results.json` is an example filename, not a file supplied by
|
|
34
|
+
TestDossier. Use the actual report path produced by your runner. Repeat steps 3
|
|
35
|
+
and 4 whenever you want to publish a new run.
|
|
36
|
+
If tests run in multiple shards or batches, merge all shard reports before
|
|
37
|
+
uploading; selecting one shard's file uploads only that shard.
|
|
38
|
+
After merging, the CLI automatically sends JSON reports (Playwright, Cypress,
|
|
39
|
+
Cucumber, generic) larger than 200 tests as sequential pages under one stable
|
|
40
|
+
Test Run. JUnit XML larger than 200 tests is rejected before anything is sent;
|
|
41
|
+
split it per suite or switch the runner to a JSON reporter.
|
|
42
|
+
|
|
19
43
|
The CLI auto-detects Playwright JSON, Cypress JSON, Cucumber JSON, JUnit XML,
|
|
20
44
|
and TestDossier's generic JSON format. It has zero runtime dependencies and
|
|
21
45
|
requires Node.js 18 or newer.
|
|
@@ -36,18 +60,18 @@ rejected except for localhost development.
|
|
|
36
60
|
Before sending anything, inspect what the CLI detected:
|
|
37
61
|
|
|
38
62
|
```bash
|
|
39
|
-
npx testdossier upload playwright-results.json --dry-run
|
|
63
|
+
npx --yes testdossier@0.1.3 upload playwright-results.json --dry-run
|
|
40
64
|
```
|
|
41
65
|
|
|
42
66
|
## Supported reports
|
|
43
67
|
|
|
44
68
|
| Runner or format | Example |
|
|
45
69
|
|---|---|
|
|
46
|
-
| Playwright JSON | `npx testdossier upload playwright-results.json` |
|
|
47
|
-
| Cypress JSON | `npx testdossier upload cypress-results.json` |
|
|
48
|
-
| Cucumber JSON | `npx testdossier upload cucumber-results.json` |
|
|
49
|
-
| JUnit XML | `npx testdossier upload junit-results.xml` |
|
|
50
|
-
| Generic JSON | `npx testdossier upload testdossier-ci.json` |
|
|
70
|
+
| Playwright JSON | `npx --yes testdossier@0.1.3 upload playwright-results.json` |
|
|
71
|
+
| Cypress JSON | `npx --yes testdossier@0.1.3 upload cypress-results.json` |
|
|
72
|
+
| Cucumber JSON | `npx --yes testdossier@0.1.3 upload cucumber-results.json` |
|
|
73
|
+
| JUnit XML | `npx --yes testdossier@0.1.3 upload junit-results.xml` |
|
|
74
|
+
| Generic JSON | `npx --yes testdossier@0.1.3 upload testdossier-ci.json` |
|
|
51
75
|
|
|
52
76
|
Detection uses the report's contents, not its filename. If a custom reporter
|
|
53
77
|
creates an ambiguous shape, pass `--format playwright`, `--format cypress`,
|
|
@@ -103,12 +127,19 @@ npx testdossier upload junit-results.xml \
|
|
|
103
127
|
```
|
|
104
128
|
|
|
105
129
|
`--run-id` is the idempotency key. Repeating the same upload with the same run
|
|
106
|
-
ID returns the existing ingest result rather than creating a duplicate
|
|
107
|
-
|
|
108
|
-
|
|
130
|
+
ID returns the existing ingest result rather than creating a duplicate; the CLI
|
|
131
|
+
prints a note when the server answers with such a replay. Different report
|
|
132
|
+
files uploaded under one run ID accumulate as separate pages of that run (the
|
|
133
|
+
page identity includes the report's file name and a path fingerprint). Without
|
|
134
|
+
a run ID, the CLI creates a new local one. Transient network, rate-limit,
|
|
135
|
+
conflict, and server errors are retried with the same run ID.
|
|
109
136
|
|
|
110
137
|
Run `npx testdossier --help` for every option.
|
|
111
138
|
|
|
112
139
|
If `.env.testdossier` is ever committed or shared, revoke that token in
|
|
113
140
|
TestDossier immediately and create a replacement. Hosted CI should use the
|
|
114
|
-
provider's encrypted secret storage instead of an env file.
|
|
141
|
+
provider's encrypted secret storage instead of an env file. The Access Tokens
|
|
142
|
+
dialog generates provider-specific GitHub Actions, GitLab CI, and Jenkins
|
|
143
|
+
snippets that run after the test report is created. Those recipes pin the CLI,
|
|
144
|
+
link back to the originating CI run, retry transient failures, and keep the
|
|
145
|
+
token out of command history.
|
package/lib/testdossier.mjs
CHANGED
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
import { createHash, randomUUID } from "node:crypto";
|
|
2
2
|
import { readFile, stat } from "node:fs/promises";
|
|
3
|
+
import { basename } from "node:path";
|
|
3
4
|
|
|
4
|
-
export const VERSION = "0.1.
|
|
5
|
+
export const VERSION = "0.1.3";
|
|
5
6
|
const DEFAULT_ORIGIN = "https://testdossier.com";
|
|
6
7
|
const MAX_REPORT_BYTES = 25 * 1024 * 1024;
|
|
8
|
+
const MAX_TESTS_PER_PAGE = 200;
|
|
7
9
|
const RETRYABLE_STATUSES = new Set([408, 409, 425, 429, 500, 502, 503, 504]);
|
|
8
10
|
const FORMAT_LABELS = {
|
|
9
11
|
cypress: "Cypress JSON",
|
|
@@ -40,48 +42,260 @@ function matchingJsonFormats(value) {
|
|
|
40
42
|
return matches;
|
|
41
43
|
}
|
|
42
44
|
|
|
43
|
-
function
|
|
45
|
+
function isObject(value) {
|
|
46
|
+
return !!value && typeof value === "object" && !Array.isArray(value);
|
|
47
|
+
}
|
|
48
|
+
|
|
49
|
+
function validPlaywrightSuites(suites) {
|
|
50
|
+
return Array.isArray(suites) && suites.every((suite) => {
|
|
51
|
+
if (!isObject(suite)) return false;
|
|
52
|
+
if (suite.specs != null && !Array.isArray(suite.specs)) return false;
|
|
53
|
+
if (suite.suites != null && !validPlaywrightSuites(suite.suites)) return false;
|
|
54
|
+
return (suite.specs || []).every((spec) => {
|
|
55
|
+
if (!isObject(spec)) return false;
|
|
56
|
+
if (spec.tests != null && !Array.isArray(spec.tests)) return false;
|
|
57
|
+
return (spec.tests || []).every((test) =>
|
|
58
|
+
isObject(test) &&
|
|
59
|
+
(test.results == null ||
|
|
60
|
+
(Array.isArray(test.results) && test.results.every(isObject))));
|
|
61
|
+
});
|
|
62
|
+
});
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
function validJsonFormat(format, value) {
|
|
66
|
+
if (format === "generic") return isObject(value) && Array.isArray(value.tests);
|
|
67
|
+
if (format === "playwright") return isObject(value) && validPlaywrightSuites(value.suites);
|
|
68
|
+
if (format === "cypress") {
|
|
69
|
+
return isObject(value) && Array.isArray(value.runs) && value.runs.every((run) =>
|
|
70
|
+
isObject(run) &&
|
|
71
|
+
(run.tests == null ||
|
|
72
|
+
(Array.isArray(run.tests) && run.tests.every((test) =>
|
|
73
|
+
isObject(test) && (test.attempts == null || Array.isArray(test.attempts))))) &&
|
|
74
|
+
(run.screenshots == null || Array.isArray(run.screenshots)));
|
|
75
|
+
}
|
|
76
|
+
return Array.isArray(value) && value.every((feature) =>
|
|
77
|
+
isObject(feature) &&
|
|
78
|
+
(feature.elements == null ||
|
|
79
|
+
(Array.isArray(feature.elements) && feature.elements.every((element) =>
|
|
80
|
+
isObject(element) && (element.steps == null || Array.isArray(element.steps))))));
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
function invalidFormatMessage(format) {
|
|
84
|
+
const expected = {
|
|
85
|
+
cypress: "a JSON object with well-formed runs[]",
|
|
86
|
+
playwright: "a JSON object with well-formed suites[]",
|
|
87
|
+
cucumber: "a JSON array of features with well-formed elements[]",
|
|
88
|
+
generic: "a JSON object with tests[]",
|
|
89
|
+
}[format];
|
|
90
|
+
return `${FORMAT_LABELS[format]} must be ${expected}.`;
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
export function analyzeReport(text, requestedFormat = "auto") {
|
|
44
94
|
const trimmed = text.replace(/^\uFEFF/, "").trimStart();
|
|
45
|
-
if (
|
|
46
|
-
|
|
95
|
+
if (!trimmed) throw new CliError("report file is empty.");
|
|
96
|
+
if (requestedFormat === "junit" || (requestedFormat === "auto" && trimmed.startsWith("<"))) {
|
|
97
|
+
if (!/^(?:<\?xml[\s\S]*?\?>\s*)?(?:<!--[\s\S]*?-->\s*)*<(?:[\w.-]+:)?testsuites?\b/i.test(trimmed)) {
|
|
47
98
|
throw new CliError("JUnit reports must be XML containing a <testsuite> or <testsuites> root.");
|
|
48
99
|
}
|
|
49
|
-
return;
|
|
100
|
+
return { format: "junit", document: null };
|
|
50
101
|
}
|
|
51
102
|
|
|
52
103
|
const value = parseJson(trimmed);
|
|
53
104
|
const matches = matchingJsonFormats(value);
|
|
54
|
-
if (
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
}
|
|
61
|
-
|
|
105
|
+
if (requestedFormat === "auto") {
|
|
106
|
+
if (matches.length === 0) {
|
|
107
|
+
throw new CliError(
|
|
108
|
+
"could not detect the report format; expected Cypress runs[], Playwright suites[], " +
|
|
109
|
+
"a Cucumber feature array, generic tests[], or JUnit XML.",
|
|
110
|
+
);
|
|
111
|
+
}
|
|
112
|
+
if (matches.length > 1) {
|
|
113
|
+
throw new CliError(`report shape is ambiguous (${matches.join(", ")}); pass --format explicitly.`);
|
|
114
|
+
}
|
|
115
|
+
const format = matches[0];
|
|
116
|
+
if (!validJsonFormat(format, value)) throw new CliError(invalidFormatMessage(format));
|
|
117
|
+
return { format, document: value };
|
|
62
118
|
}
|
|
119
|
+
if (!matches.includes(requestedFormat) || !validJsonFormat(requestedFormat, value)) {
|
|
120
|
+
throw new CliError(invalidFormatMessage(requestedFormat));
|
|
121
|
+
}
|
|
122
|
+
return { format: requestedFormat, document: value };
|
|
63
123
|
}
|
|
64
124
|
|
|
65
125
|
export function detectReportFormat(text) {
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
126
|
+
return analyzeReport(text).format;
|
|
127
|
+
}
|
|
128
|
+
|
|
129
|
+
function playwrightTestNodes(suites, found = []) {
|
|
130
|
+
for (const suite of Array.isArray(suites) ? suites : []) {
|
|
131
|
+
if (!suite || typeof suite !== "object") continue;
|
|
132
|
+
for (const spec of Array.isArray(suite.specs) ? suite.specs : []) {
|
|
133
|
+
if (!spec || typeof spec !== "object") continue;
|
|
134
|
+
for (const test of Array.isArray(spec.tests) ? spec.tests : []) {
|
|
135
|
+
found.push(test);
|
|
136
|
+
}
|
|
137
|
+
}
|
|
138
|
+
playwrightTestNodes(suite.suites, found);
|
|
71
139
|
}
|
|
140
|
+
return found;
|
|
141
|
+
}
|
|
72
142
|
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
143
|
+
function selectedPlaywrightSuite(suite, selectedTests) {
|
|
144
|
+
if (!suite || typeof suite !== "object") return null;
|
|
145
|
+
const next = { ...suite };
|
|
146
|
+
next.specs = (Array.isArray(suite.specs) ? suite.specs : [])
|
|
147
|
+
.filter((spec) => spec && typeof spec === "object")
|
|
148
|
+
.map((spec) => ({
|
|
149
|
+
...spec,
|
|
150
|
+
tests: (Array.isArray(spec.tests) ? spec.tests : []).filter((test) => selectedTests.has(test)),
|
|
151
|
+
}))
|
|
152
|
+
.filter((spec) => spec.tests.length > 0);
|
|
153
|
+
next.suites = (Array.isArray(suite.suites) ? suite.suites : [])
|
|
154
|
+
.map((child) => selectedPlaywrightSuite(child, selectedTests))
|
|
155
|
+
.filter(Boolean);
|
|
156
|
+
return next.specs.length > 0 || next.suites.length > 0 ? next : null;
|
|
157
|
+
}
|
|
158
|
+
|
|
159
|
+
// Counts <testcase> elements the way the server's parser does: real tags
|
|
160
|
+
// only, so a failure message inside CDATA that happens to contain
|
|
161
|
+
// "<testcase" is not counted.
|
|
162
|
+
function countJunitTests(text) {
|
|
163
|
+
const stripped = text
|
|
164
|
+
.replace(/<!\[CDATA\[[\s\S]*?\]\]>/g, "")
|
|
165
|
+
.replace(/<!--[\s\S]*?-->/g, "");
|
|
166
|
+
return (stripped.match(/<testcase[\s/>]/gi) || []).length;
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
function isCucumberScenario(element) {
|
|
170
|
+
return !!element && typeof element === "object" &&
|
|
171
|
+
String(element.type || "scenario").toLowerCase() === "scenario";
|
|
172
|
+
}
|
|
173
|
+
|
|
174
|
+
function pagedSlices(items, maxTests) {
|
|
175
|
+
const slices = [];
|
|
176
|
+
for (let start = 0; start < items.length; start += maxTests) {
|
|
177
|
+
slices.push(items.slice(start, start + maxTests));
|
|
178
|
+
}
|
|
179
|
+
return slices;
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
function cypressTestFailed(test) {
|
|
183
|
+
const attempts = Array.isArray(test.attempts) ? test.attempts : [];
|
|
184
|
+
const state = attempts[attempts.length - 1]?.state || test.state;
|
|
185
|
+
return String(state || "").toLowerCase() === "failed";
|
|
186
|
+
}
|
|
187
|
+
|
|
188
|
+
function normalizedCypressTitle(value) {
|
|
189
|
+
return String(value || "").toLowerCase().replace(/[^a-z0-9]+/g, " ").trim();
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
function cypressScreenshotMatchesTest(screenshot, test) {
|
|
193
|
+
if (!isObject(screenshot)) return false;
|
|
194
|
+
const title = Array.isArray(test.title) ? test.title : [String(test.title || "")];
|
|
195
|
+
const haystack = normalizedCypressTitle(`${screenshot.name || ""} ${screenshot.path || ""}`);
|
|
196
|
+
const fullTitle = normalizedCypressTitle(title.join(" "));
|
|
197
|
+
const leafTitle = normalizedCypressTitle(title[title.length - 1] || "");
|
|
198
|
+
return !!haystack &&
|
|
199
|
+
((fullTitle.length >= 4 && haystack.includes(fullTitle)) ||
|
|
200
|
+
(leafTitle.length >= 4 && haystack.includes(leafTitle)));
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
function selectedCypressRun(run, selectedTests) {
|
|
204
|
+
const runTests = Array.isArray(run.tests) ? run.tests : [];
|
|
205
|
+
const tests = runTests.filter((test) => selectedTests.has(test));
|
|
206
|
+
if (tests.length === 0) return null;
|
|
207
|
+
|
|
208
|
+
// The server associates run-level screenshots using the complete run's set
|
|
209
|
+
// of failures. Keep that context while paging: otherwise two failures split
|
|
210
|
+
// across pages each look like the run's sole failure and both inherit every
|
|
211
|
+
// unmatched screenshot.
|
|
212
|
+
const allFailures = runTests.filter(cypressTestFailed);
|
|
213
|
+
const pageFailures = tests.filter(cypressTestFailed);
|
|
214
|
+
let screenshots = [];
|
|
215
|
+
if (allFailures.length === 1 && pageFailures.length === 1) {
|
|
216
|
+
screenshots = Array.isArray(run.screenshots) ? run.screenshots : [];
|
|
217
|
+
} else if (allFailures.length > 1 && pageFailures.length > 0) {
|
|
218
|
+
screenshots = (Array.isArray(run.screenshots) ? run.screenshots : [])
|
|
219
|
+
.filter((screenshot) => pageFailures.some((test) => cypressScreenshotMatchesTest(screenshot, test)));
|
|
220
|
+
}
|
|
221
|
+
|
|
222
|
+
return {
|
|
223
|
+
...run,
|
|
224
|
+
tests,
|
|
225
|
+
screenshots,
|
|
226
|
+
video: pageFailures.length > 0 ? run.video : null,
|
|
227
|
+
};
|
|
228
|
+
}
|
|
229
|
+
|
|
230
|
+
export function splitReportPages(format, report, maxTests = MAX_TESTS_PER_PAGE, document = null) {
|
|
231
|
+
if (!Number.isInteger(maxTests) || maxTests < 1 || maxTests > MAX_TESTS_PER_PAGE) {
|
|
232
|
+
throw new CliError(`page size must be an integer from 1 to ${MAX_TESTS_PER_PAGE}.`);
|
|
233
|
+
}
|
|
234
|
+
if (format === "junit") {
|
|
235
|
+
const count = countJunitTests(report.toString("utf8"));
|
|
236
|
+
if (count > maxTests) {
|
|
237
|
+
throw new CliError(
|
|
238
|
+
`JUnit report has ${count} tests; the server accepts at most ${maxTests} per request ` +
|
|
239
|
+
"and JUnit XML cannot be paged automatically. Split it per suite, or switch the runner " +
|
|
240
|
+
"to a JSON reporter (Playwright, Cypress, Cucumber, or generic), which the CLI pages for you.",
|
|
241
|
+
);
|
|
242
|
+
}
|
|
243
|
+
return [{ body: report, tests: count }];
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
const doc = document ?? parseJson(report.toString("utf8").replace(/^\uFEFF/, ""));
|
|
247
|
+
if (format === "generic") {
|
|
248
|
+
const tests = doc.tests;
|
|
249
|
+
if (tests.length <= maxTests) return [{ body: report, tests: tests.length }];
|
|
250
|
+
return pagedSlices(tests, maxTests).map((slice) => ({
|
|
251
|
+
body: Buffer.from(JSON.stringify({ ...doc, tests: slice })),
|
|
252
|
+
tests: slice.length,
|
|
253
|
+
}));
|
|
80
254
|
}
|
|
81
|
-
|
|
82
|
-
|
|
255
|
+
|
|
256
|
+
if (format === "playwright") {
|
|
257
|
+
const tests = playwrightTestNodes(doc.suites);
|
|
258
|
+
if (tests.length <= maxTests) return [{ body: report, tests: tests.length }];
|
|
259
|
+
return pagedSlices(tests, maxTests).map((slice) => {
|
|
260
|
+
const selected = new Set(slice);
|
|
261
|
+
const suites = doc.suites
|
|
262
|
+
.map((suite) => selectedPlaywrightSuite(suite, selected))
|
|
263
|
+
.filter(Boolean);
|
|
264
|
+
return { body: Buffer.from(JSON.stringify({ ...doc, suites })), tests: slice.length };
|
|
265
|
+
});
|
|
266
|
+
}
|
|
267
|
+
|
|
268
|
+
if (format === "cypress") {
|
|
269
|
+
const runs = doc.runs;
|
|
270
|
+
const tests = runs.flatMap((run) => Array.isArray(run.tests) ? run.tests : []);
|
|
271
|
+
if (tests.length <= maxTests) return [{ body: report, tests: tests.length }];
|
|
272
|
+
return pagedSlices(tests, maxTests).map((slice) => {
|
|
273
|
+
const selected = new Set(slice);
|
|
274
|
+
const pageRuns = runs
|
|
275
|
+
.map((run) => selectedCypressRun(run, selected))
|
|
276
|
+
.filter(Boolean);
|
|
277
|
+
return { body: Buffer.from(JSON.stringify({ ...doc, runs: pageRuns })), tests: slice.length };
|
|
278
|
+
});
|
|
83
279
|
}
|
|
84
|
-
|
|
280
|
+
|
|
281
|
+
// cucumber: the test unit is a scenario element. Background (and other
|
|
282
|
+
// non-scenario) elements apply to every scenario in their feature, so
|
|
283
|
+
// they are replicated onto every page that carries that feature.
|
|
284
|
+
const features = (Array.isArray(doc) ? doc : []).filter((f) => f && typeof f === "object");
|
|
285
|
+
const scenarios = features.flatMap((f) =>
|
|
286
|
+
(Array.isArray(f.elements) ? f.elements : []).filter(isCucumberScenario));
|
|
287
|
+
if (scenarios.length <= maxTests) return [{ body: report, tests: scenarios.length }];
|
|
288
|
+
return pagedSlices(scenarios, maxTests).map((slice) => {
|
|
289
|
+
const selected = new Set(slice);
|
|
290
|
+
const pageFeatures = features
|
|
291
|
+
.map((f) => {
|
|
292
|
+
const elements = (Array.isArray(f.elements) ? f.elements : [])
|
|
293
|
+
.filter((el) => !isCucumberScenario(el) || selected.has(el));
|
|
294
|
+
return elements.some((el) => isCucumberScenario(el)) ? { ...f, elements } : null;
|
|
295
|
+
})
|
|
296
|
+
.filter(Boolean);
|
|
297
|
+
return { body: Buffer.from(JSON.stringify(pageFeatures)), tests: slice.length };
|
|
298
|
+
});
|
|
85
299
|
}
|
|
86
300
|
|
|
87
301
|
function isLoopback(hostname) {
|
|
@@ -179,29 +393,58 @@ export function parseUploadArgs(args) {
|
|
|
179
393
|
return options;
|
|
180
394
|
}
|
|
181
395
|
|
|
182
|
-
function
|
|
396
|
+
function normalizedHeader(value, name) {
|
|
183
397
|
if (value == null || value === "") return "";
|
|
184
398
|
const normalized = String(value);
|
|
185
399
|
if (/[\r\n]/.test(normalized)) throw new CliError(`${name} must not contain a newline.`);
|
|
186
|
-
|
|
400
|
+
// fetch rejects header values outside ISO-8859-1; percent-encode anything
|
|
401
|
+
// beyond printable ASCII so a Unicode branch name degrades to a readable
|
|
402
|
+
// token instead of a TypeError that looks like a network failure.
|
|
403
|
+
const ascii = normalized.replace(/[^\x20-\x7e]/gu, (char) => {
|
|
404
|
+
try { return encodeURIComponent(char); } catch { return "_"; }
|
|
405
|
+
});
|
|
406
|
+
return ascii.trim();
|
|
407
|
+
}
|
|
408
|
+
|
|
409
|
+
function boundedHeader(value, name, maxLength) {
|
|
410
|
+
return normalizedHeader(value, name).slice(0, maxLength).trim();
|
|
187
411
|
}
|
|
188
412
|
|
|
189
413
|
function boundedRunId(value) {
|
|
190
|
-
const normalized =
|
|
414
|
+
const normalized = normalizedHeader(value, "run id");
|
|
191
415
|
if (normalized.length <= 120) return normalized;
|
|
192
416
|
const digest = createHash("sha256").update(normalized).digest("hex").slice(0, 20);
|
|
193
417
|
return `${normalized.slice(0, 95)}-${digest}`;
|
|
194
418
|
}
|
|
195
419
|
|
|
420
|
+
function boundedPageBase(value) {
|
|
421
|
+
const normalized = normalizedHeader(value, "report name") || "results";
|
|
422
|
+
if (normalized.length <= 100) return normalized;
|
|
423
|
+
const digest = createHash("sha256").update(normalized).digest("hex").slice(0, 16);
|
|
424
|
+
return `${normalized.slice(0, 83)}-${digest}`;
|
|
425
|
+
}
|
|
426
|
+
|
|
427
|
+
function reportPageBase(reportPath) {
|
|
428
|
+
const reportName = basename(reportPath);
|
|
429
|
+
const pathKey = String(reportPath).replace(/\\/g, "/");
|
|
430
|
+
const pathDigest = createHash("sha256").update(pathKey).digest("hex").slice(0, 12);
|
|
431
|
+
return boundedPageBase(`${reportName}-${pathDigest}`);
|
|
432
|
+
}
|
|
433
|
+
|
|
196
434
|
function defaultRunId() {
|
|
197
435
|
return `local-${Date.now()}-${randomUUID().slice(0, 8)}`;
|
|
198
436
|
}
|
|
199
437
|
|
|
200
438
|
function inferredMetadata(options, env) {
|
|
439
|
+
// GITHUB_JOB is part of the inferred key: two jobs of one workflow run
|
|
440
|
+
// would otherwise share a run id, and the server answers the second
|
|
441
|
+
// upload with an idempotent replay of the first.
|
|
201
442
|
const runId = boundedRunId(
|
|
202
443
|
options.runId ||
|
|
203
444
|
env.TD_RUN_ID ||
|
|
204
|
-
(env.GITHUB_RUN_ID
|
|
445
|
+
(env.GITHUB_RUN_ID
|
|
446
|
+
? [env.GITHUB_RUN_ID, env.GITHUB_JOB, env.GITHUB_RUN_ATTEMPT || 1].filter(Boolean).join("-")
|
|
447
|
+
: "") ||
|
|
205
448
|
(env.CI_PIPELINE_ID ? `${env.CI_PIPELINE_ID}-${env.CI_JOB_ID || 1}` : "") ||
|
|
206
449
|
env.BUILD_TAG ||
|
|
207
450
|
defaultRunId(),
|
|
@@ -350,6 +593,12 @@ Authentication:
|
|
|
350
593
|
Tokens are intentionally not accepted as command-line arguments, keeping
|
|
351
594
|
them out of shell history and process listings.
|
|
352
595
|
|
|
596
|
+
Large reports:
|
|
597
|
+
JSON reports (Playwright, Cypress, Cucumber, generic) above 200 tests are
|
|
598
|
+
automatically sent as sequential pages that remain grouped under one
|
|
599
|
+
TestDossier run. JUnit XML above 200 tests is rejected before upload —
|
|
600
|
+
split it per suite or switch the runner to a JSON reporter.
|
|
601
|
+
|
|
353
602
|
Examples:
|
|
354
603
|
npx testdossier upload playwright-results.json --env-file .env.testdossier
|
|
355
604
|
npx testdossier upload junit-results.xml --env-file .env.testdossier --build abc123
|
|
@@ -369,6 +618,27 @@ function outputSummary(result, stdout, stderr) {
|
|
|
369
618
|
}
|
|
370
619
|
}
|
|
371
620
|
|
|
621
|
+
function aggregatePageResults(results) {
|
|
622
|
+
const aggregate = { counts: {}, warnings: [] };
|
|
623
|
+
const seenWarnings = new Set();
|
|
624
|
+
for (const result of results) {
|
|
625
|
+
if (result && result.counts && typeof result.counts === "object") {
|
|
626
|
+
for (const [name, value] of Object.entries(result.counts)) {
|
|
627
|
+
if (typeof value === "number") {
|
|
628
|
+
aggregate.counts[name] = (aggregate.counts[name] || 0) + value;
|
|
629
|
+
}
|
|
630
|
+
}
|
|
631
|
+
}
|
|
632
|
+
for (const warning of Array.isArray(result && result.warnings) ? result.warnings : []) {
|
|
633
|
+
const key = String(warning);
|
|
634
|
+
if (seenWarnings.has(key)) continue;
|
|
635
|
+
seenWarnings.add(key);
|
|
636
|
+
aggregate.warnings.push(warning);
|
|
637
|
+
}
|
|
638
|
+
}
|
|
639
|
+
return aggregate;
|
|
640
|
+
}
|
|
641
|
+
|
|
372
642
|
export async function runCli(
|
|
373
643
|
args,
|
|
374
644
|
{
|
|
@@ -379,7 +649,7 @@ export async function runCli(
|
|
|
379
649
|
} = {},
|
|
380
650
|
) {
|
|
381
651
|
try {
|
|
382
|
-
if (args
|
|
652
|
+
if (args[0] === "--version") {
|
|
383
653
|
stdout(VERSION);
|
|
384
654
|
return 0;
|
|
385
655
|
}
|
|
@@ -408,12 +678,16 @@ export async function runCli(
|
|
|
408
678
|
}
|
|
409
679
|
const report = await readFile(options.reportPath);
|
|
410
680
|
const reportText = report.toString("utf8");
|
|
411
|
-
const
|
|
412
|
-
|
|
681
|
+
const { format, document } = analyzeReport(reportText, options.format);
|
|
682
|
+
const pages = splitReportPages(format, report, MAX_TESTS_PER_PAGE, document);
|
|
683
|
+
const testCount = pages.reduce((sum, page) => sum + page.tests, 0);
|
|
413
684
|
|
|
414
685
|
const endpoint = normalizeEndpoint(options.url || env.TD_URL || DEFAULT_ORIGIN);
|
|
415
686
|
const destination = new URL(endpoint).origin;
|
|
416
|
-
stdout(
|
|
687
|
+
stdout(
|
|
688
|
+
`Validated ${FORMAT_LABELS[format]} (${report.byteLength} bytes, ${testCount} tests` +
|
|
689
|
+
`${pages.length > 1 ? ` across ${pages.length} pages` : ""}).`,
|
|
690
|
+
);
|
|
417
691
|
if (options.dryRun) {
|
|
418
692
|
stdout(`Dry run complete; no data was sent to ${destination}.`);
|
|
419
693
|
return 0;
|
|
@@ -428,29 +702,57 @@ export async function runCli(
|
|
|
428
702
|
if (typeof fetchImpl !== "function") throw new CliError("Node 18 or newer is required (global fetch is unavailable).");
|
|
429
703
|
|
|
430
704
|
const metadata = inferredMetadata(options, env);
|
|
431
|
-
const
|
|
705
|
+
const baseHeaders = {
|
|
432
706
|
Authorization: `Bearer ${token}`,
|
|
433
707
|
"Content-Type": format === "junit" ? "application/xml" : "application/json",
|
|
434
708
|
"User-Agent": `testdossier-cli/${VERSION}`,
|
|
435
709
|
"X-CI-Run-Id": metadata.runId,
|
|
436
|
-
"X-CI-Page-Id": "results",
|
|
437
710
|
"X-CI-Provider": metadata.provider,
|
|
438
711
|
};
|
|
439
|
-
if (format !== "junit" && format !== "generic")
|
|
440
|
-
if (metadata.build)
|
|
441
|
-
if (metadata.branch)
|
|
442
|
-
if (metadata.ciUrl)
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
712
|
+
if (format !== "junit" && format !== "generic") baseHeaders["X-CI-Format"] = format;
|
|
713
|
+
if (metadata.build) baseHeaders["X-Build"] = metadata.build;
|
|
714
|
+
if (metadata.branch) baseHeaders["X-Branch"] = metadata.branch;
|
|
715
|
+
if (metadata.ciUrl) baseHeaders["X-CI-URL"] = metadata.ciUrl;
|
|
716
|
+
|
|
717
|
+
if (pages.length > 1) {
|
|
718
|
+
stdout(
|
|
719
|
+
`Uploading ${pages.reduce((sum, page) => sum + (page.tests || 0), 0)} tests in ` +
|
|
720
|
+
`${pages.length} pages (run ${metadata.runId}).`,
|
|
721
|
+
);
|
|
722
|
+
}
|
|
723
|
+
// Page ids carry a readable file name plus a path fingerprint so two
|
|
724
|
+
// reports uploaded under one run id become distinct pages even when their
|
|
725
|
+
// base names match (for example chromium/results.json and
|
|
726
|
+
// firefox/results.json).
|
|
727
|
+
const pageBase = reportPageBase(options.reportPath);
|
|
728
|
+
const pageResults = [];
|
|
729
|
+
for (let index = 0; index < pages.length; index += 1) {
|
|
730
|
+
const page = pages[index];
|
|
731
|
+
const result = await postWithRetry({
|
|
732
|
+
endpoint,
|
|
733
|
+
headers: {
|
|
734
|
+
...baseHeaders,
|
|
735
|
+
"X-CI-Page-Id": pages.length === 1 ? pageBase : `${pageBase}-${index + 1}`,
|
|
736
|
+
},
|
|
737
|
+
body: page.body,
|
|
738
|
+
env,
|
|
739
|
+
fetchImpl,
|
|
740
|
+
stderr,
|
|
741
|
+
});
|
|
742
|
+
pageResults.push(result);
|
|
743
|
+
if (pages.length > 1) {
|
|
744
|
+
stdout(`Uploaded page ${index + 1}/${pages.length} (${page.tests} tests).`);
|
|
745
|
+
}
|
|
746
|
+
}
|
|
452
747
|
stdout(`Uploaded ${FORMAT_LABELS[format]} to ${destination} (run ${metadata.runId}).`);
|
|
453
|
-
|
|
748
|
+
const replayed = pageResults.filter((result) => result && result.idempotent_replay === true).length;
|
|
749
|
+
if (replayed > 0) {
|
|
750
|
+
stderr(pageResults.length === 1
|
|
751
|
+
? `Note: the server replayed an earlier upload for run ${metadata.runId}; nothing new was ingested. ` +
|
|
752
|
+
"Pass a unique --run-id if this was not a retry."
|
|
753
|
+
: `Note: ${replayed} of ${pageResults.length} pages were idempotent replays of an earlier upload (run ${metadata.runId}).`);
|
|
754
|
+
}
|
|
755
|
+
outputSummary(aggregatePageResults(pageResults), stdout, stderr);
|
|
454
756
|
return 0;
|
|
455
757
|
} catch (error) {
|
|
456
758
|
const message = error && error.message ? error.message : String(error);
|
package/package.json
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "testdossier",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.3",
|
|
4
4
|
"description": "Upload completed test reports to TestDossier from a local machine or CI.",
|
|
5
|
+
"license": "MIT",
|
|
5
6
|
"type": "module",
|
|
6
7
|
"bin": {
|
|
7
8
|
"testdossier": "bin/testdossier.mjs"
|
|
@@ -10,19 +11,15 @@
|
|
|
10
11
|
"bin",
|
|
11
12
|
"lib",
|
|
12
13
|
"README.md",
|
|
14
|
+
"LICENSE",
|
|
13
15
|
".env.testdossier.example"
|
|
14
16
|
],
|
|
15
17
|
"engines": {
|
|
16
18
|
"node": ">=18"
|
|
17
19
|
},
|
|
18
|
-
"repository": {
|
|
19
|
-
"type": "git",
|
|
20
|
-
"url": "git+https://github.com/devkalu/test-dossier.git",
|
|
21
|
-
"directory": "cli"
|
|
22
|
-
},
|
|
23
20
|
"homepage": "https://testdossier.com",
|
|
24
21
|
"bugs": {
|
|
25
|
-
"
|
|
22
|
+
"email": "support@testdossier.com"
|
|
26
23
|
},
|
|
27
24
|
"keywords": [
|
|
28
25
|
"testing",
|