tolkira 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DESIGN.md +131 -9
- package/GLOSSARY.md +201 -1
- package/README.md +413 -29
- package/SECURITY.md +3 -0
- package/dist/attachment.d.ts +33 -0
- package/dist/attempt-diagnostics.d.ts +2 -0
- package/dist/ci.d.ts +12 -0
- package/dist/cli.js +6881 -2691
- package/dist/cli.js.map +4 -4
- package/dist/compat/cucumber-html-reporter.cjs +12 -0
- package/dist/compat/cucumber-html-reporter.d.cts +10 -0
- package/dist/compat/cucumber-html-reporter.js +263 -26
- package/dist/compat/cucumber-html-reporter.js.map +4 -4
- package/dist/compat/options.d.ts +4 -0
- package/dist/email.d.ts +18 -0
- package/dist/history.d.ts +41 -0
- package/dist/i18n/catalogs.d.ts +7 -0
- package/dist/index.d.ts +15 -7
- package/dist/index.js +5301 -1709
- package/dist/index.js.map +4 -4
- package/dist/input/background.d.ts +4 -0
- package/dist/input/detect.d.ts +30 -0
- package/dist/input/load.d.ts +3 -0
- package/dist/input/playwright.d.ts +14 -0
- package/dist/options.d.ts +18 -0
- package/dist/output.d.ts +9 -1
- package/dist/pdf.d.ts +9 -0
- package/dist/render/ansi.d.ts +2 -0
- package/dist/render/attachments.d.ts +3 -0
- package/dist/render/ci.d.ts +4 -0
- package/dist/render/client-bundle.d.ts +1 -0
- package/dist/render/client-i18n.d.ts +7 -0
- package/dist/render/details.d.ts +5 -0
- package/dist/render/dom-contract.d.ts +42 -0
- package/dist/render/format.d.ts +34 -0
- package/dist/render/html.d.ts +2 -9
- package/dist/render/i18n.d.ts +30 -0
- package/dist/render/markdown.d.ts +2 -0
- package/dist/render/matrix.d.ts +3 -0
- package/dist/render/print-script.d.ts +1 -0
- package/dist/render/report-client.d.ts +1 -0
- package/dist/render/report.d.ts +4 -0
- package/dist/render/search.d.ts +21 -0
- package/dist/render/trends.d.ts +7 -0
- package/dist/render/triage.d.ts +5 -0
- package/dist/render/types.d.ts +26 -0
- package/dist/report/index-run.d.ts +69 -0
- package/dist/report/model.d.ts +96 -6
- package/dist/report/status.d.ts +12 -0
- package/dist/stream.d.ts +15 -0
- package/dist/summary.d.ts +17 -2
- package/dist/text.d.ts +15 -0
- package/dist/warnings.d.ts +6 -0
- package/docs/AGENT-INTERFACE.md +504 -33
- package/docs/CI.md +185 -0
- package/docs/COMPATIBILITY.md +121 -16
- package/docs/CUCUMBER-COMPATIBILITY.md +232 -15
- package/docs/EMAIL.md +273 -0
- package/docs/I18N.md +241 -0
- package/docs/JUNIT-XML.md +46 -4
- package/docs/LIMITATIONS.md +21 -0
- package/docs/PDF.md +255 -0
- package/llms.txt +63 -0
- package/package.json +46 -8
package/docs/EMAIL.md
ADDED
|
@@ -0,0 +1,273 @@
|
|
|
1
|
+
# Email summary
|
|
2
|
+
|
|
3
|
+
`--email <file>` writes one self-contained HTML email summary from the same
|
|
4
|
+
result files and the same locale as the report. It is opt-in: without the flag
|
|
5
|
+
no email file is created and the one-file report default is unchanged.
|
|
6
|
+
|
|
7
|
+
The email is a small at-a-glance body for a status mail, a chat message or a CI
|
|
8
|
+
mail step. The full report stays the detailed artifact and, when one is written,
|
|
9
|
+
is attached or shared separately. Tolkira never sends mail, opens a connection or
|
|
10
|
+
talks to a mail server; it writes one HTML file.
|
|
11
|
+
|
|
12
|
+
```shell
|
|
13
|
+
# Email next to the report; the email names report.html
|
|
14
|
+
tolkira test-results/ --out report.html --email email.html
|
|
15
|
+
|
|
16
|
+
# Email as the only output; it states that no full report is attached
|
|
17
|
+
tolkira test-results/ --email email.html
|
|
18
|
+
|
|
19
|
+
# Email plus machine output; stdout stays JSON only
|
|
20
|
+
tolkira test-results/ --out report.html --email email.html --json > summary.json
|
|
21
|
+
|
|
22
|
+
# PDF first, then the email that names it; no report HTML
|
|
23
|
+
tolkira test-results/ --pdf report.pdf --email email.html
|
|
24
|
+
|
|
25
|
+
# German email without a report
|
|
26
|
+
tolkira test-results/ --email email.html --locale de
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
## CLI and API
|
|
30
|
+
|
|
31
|
+
`--email` is an output like `--out`, not a mode. It composes with `--out`,
|
|
32
|
+
`--json`, `--ci`, `--ci-out` and `--pdf`, alone or together; `--ci-out` still
|
|
33
|
+
requires `--ci markdown`. At least one input and a non-empty email path are
|
|
34
|
+
required. The
|
|
35
|
+
CLI confirms the file with `Email written to <path>` only when stdout is free
|
|
36
|
+
(the email alone or with `--out`); with `--json` or `--ci` on stdout, the machine
|
|
37
|
+
output is unchanged and no extra line is printed. With `--ci-out`, the CI text
|
|
38
|
+
goes to the file, so stdout is free for the normal report and email
|
|
39
|
+
confirmations unless `--json` also uses stdout.
|
|
40
|
+
|
|
41
|
+
`--title` sets the heading and the suggested subject; `--accent` sets the top
|
|
42
|
+
border colour even when no report is written. A title that carries a branch or
|
|
43
|
+
build name therefore carries it into the email.
|
|
44
|
+
|
|
45
|
+
`--locale` applies to the email as well as the report and is accepted with
|
|
46
|
+
`--email` even without `--out`; `--pdf` is the other report-like output that
|
|
47
|
+
accepts `--locale`. With only `--json` or `--ci`, it remains a usage
|
|
48
|
+
error, and the CLI, JSON and CI text stay English.
|
|
49
|
+
|
|
50
|
+
The email path is validated like the other outputs. It must not be empty, must
|
|
51
|
+
not collide with an input, the logo, `--out`, `--ci-out`, `--history` or `--pdf`
|
|
52
|
+
(resolved and case-aware on Windows and macOS), and must not be written inside an
|
|
53
|
+
input directory, even when the target does not exist yet or is reached through a
|
|
54
|
+
junction or symlink. It is written atomically after the report and the PDF. The
|
|
55
|
+
order is
|
|
56
|
+
report, PDF, email, history, `--ci-out`, stdout. On an email write failure the CLI
|
|
57
|
+
exits 1, names the email path, keeps an already written report, skips the history
|
|
58
|
+
and `--ci-out`, prints no JSON/CI stdout and leaves no temporary file behind. The
|
|
59
|
+
`Email written to` confirmation strips control characters from the path.
|
|
60
|
+
|
|
61
|
+
With `--pdf` the email note names the full artifact that exists: the `--out`
|
|
62
|
+
report when one is written, otherwise the PDF. The PDF is always written before
|
|
63
|
+
the email, so a PDF failure skips the email instead of naming a missing file.
|
|
64
|
+
|
|
65
|
+
Library use returns exactly the same HTML for the same summary, options and
|
|
66
|
+
clock:
|
|
67
|
+
|
|
68
|
+
```ts
|
|
69
|
+
import { renderEmail, summarizeReport } from 'tolkira';
|
|
70
|
+
|
|
71
|
+
const summary = await summarizeReport(['test-results/']);
|
|
72
|
+
const html = renderEmail(summary, {
|
|
73
|
+
title: 'Nightly run',
|
|
74
|
+
locale: 'de',
|
|
75
|
+
reportName: 'report.html',
|
|
76
|
+
accent: '#1b5e20',
|
|
77
|
+
now: new Date('2026-11-20T09:30:00.000Z'),
|
|
78
|
+
});
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
| Option | Meaning |
|
|
82
|
+
| --- | --- |
|
|
83
|
+
| `title` | Suggested subject and visible heading text after escaping; default `Tolkira report` |
|
|
84
|
+
| `locale` | Catalog and formatting language; default `en`; unsupported tags throw |
|
|
85
|
+
| `reportName` | Full report name; only the basename reaches the note |
|
|
86
|
+
| `accent` | Hex colour validated with the report rule; invalid values are ignored |
|
|
87
|
+
| `now` | Generation time, labelled `Generated`; default current time |
|
|
88
|
+
|
|
89
|
+
`renderEmail` writes nothing, sends nothing and makes no request. An unrecognised
|
|
90
|
+
summary status falls back to the localized `Unknown` label and colour, an invalid
|
|
91
|
+
`now` omits the generated date instead of throwing, and `reportName` is always
|
|
92
|
+
reduced to its basename. There is no logo option: a report logo can be megabytes
|
|
93
|
+
and mail clients clip or strip it.
|
|
94
|
+
|
|
95
|
+
## What the email contains
|
|
96
|
+
|
|
97
|
+
- The title and overall status as the visible heading and suggested subject, for
|
|
98
|
+
example `Nightly run — Failed`.
|
|
99
|
+
- The generation time (UTC, locale-formatted), labelled `Generated`. It is the
|
|
100
|
+
time the email was rendered, not the run start.
|
|
101
|
+
- The attempted pass rate as a percentage, with a full-sentence note
|
|
102
|
+
(`The pass rate counts only executed (non-skipped) scenarios.`). An
|
|
103
|
+
all-unexecuted run shows
|
|
104
|
+
`No scenarios attempted`, its unexecuted count and `0 scenarios with results`;
|
|
105
|
+
an empty run (no scenario rows) shows `0 / 0 passed`, displays the empty state
|
|
106
|
+
and omits the failed-scenarios section. The attempted-only note is hidden only
|
|
107
|
+
for the empty run.
|
|
108
|
+
- Every non-zero scenario status with its localized label, the unexecuted count,
|
|
109
|
+
and the total labelled as scenarios with results.
|
|
110
|
+
- Feature, step, duration and inline-attachment totals, including the raw
|
|
111
|
+
attachment byte count; the attachment count is labelled `Attachments in the
|
|
112
|
+
run`. English digits are grouped (`8,509 bytes`) and durations are human
|
|
113
|
+
(`12 ms`, `12.3 s`, `1 min 5 s`, `1 h 2 min`); the report helpers and the JSON
|
|
114
|
+
summary keep their existing numbers.
|
|
115
|
+
- The first five failed scenarios in summary order, with feature, scenario name,
|
|
116
|
+
status and duration. The heading, note and failed names wrap anywhere, and
|
|
117
|
+
displayed feature, scenario and report-name strings are capped at 200
|
|
118
|
+
characters with an ellipsis, so one hostile name cannot break the layout or the
|
|
119
|
+
size budget. The report has the steps, errors, tags and attachments, so the
|
|
120
|
+
email omits them. More failures are a count such as `3 more failed scenarios`.
|
|
121
|
+
- A no-failure line keyed off the overall status: a passed run says
|
|
122
|
+
`No failed scenarios.`, while a pending, undefined or unknown run names its
|
|
123
|
+
status and is never called successful. An empty run shows
|
|
124
|
+
`No scenarios in this run.`
|
|
125
|
+
- A report note and the footer.
|
|
126
|
+
|
|
127
|
+
The **top-five rule** keeps the body small and stable: the report carries the
|
|
128
|
+
detail when it is attached, so five rows are enough to decide what to open first.
|
|
129
|
+
The rows are the first five failed scenarios in summary order, not the five
|
|
130
|
+
slowest and not a per-feature table. The cap is fixed in this slice; it is not a
|
|
131
|
+
CLI option.
|
|
132
|
+
|
|
133
|
+
Allure 3's `mail` plugin takes a different trade-off: it links the hosted report,
|
|
134
|
+
lists up to 20 failed or broken scenarios and adds short errors. Tolkira has no
|
|
135
|
+
link (the email is offline and carries no URL), caps the list at five to keep the
|
|
136
|
+
mail small, and keeps error text out to avoid leaking failure messages and
|
|
137
|
+
secret material. A configurable cap and short snippets are B-026 and B-025.
|
|
138
|
+
|
|
139
|
+
With `--out`, the report note names only the basename (`report.html`) and asks
|
|
140
|
+
the recipient to attach or share it separately. No directory, absolute path or
|
|
141
|
+
URL is included. Without `--out`, the note says `No full report is attached.`;
|
|
142
|
+
the usage examples show the report and email pair, which is the recommended
|
|
143
|
+
workflow.
|
|
144
|
+
|
|
145
|
+
## Opening, copying and attaching
|
|
146
|
+
|
|
147
|
+
Open the `.html` file in a browser to preview it. To send a short mail, copy the
|
|
148
|
+
rendered page from the browser into the mail body; the inline styles survive in
|
|
149
|
+
most clients. For a complete artifact, attach `email.html` as the message body
|
|
150
|
+
source or attach `report.html` beside a short text body.
|
|
151
|
+
|
|
152
|
+
The HTML file has no mail headers, so it cannot set a subject by itself. The
|
|
153
|
+
heading is `{title} — {status}`; copy it into the subject line as the suggested
|
|
154
|
+
subject, or let the CI mail step derive one from the summary. A pasted body does
|
|
155
|
+
not carry the `<title>` element, so the visible heading is the reliable subject
|
|
156
|
+
text.
|
|
157
|
+
|
|
158
|
+
## Privacy and test data
|
|
159
|
+
|
|
160
|
+
The email contains feature and scenario names, statuses, counts and durations,
|
|
161
|
+
the title (which may carry a branch or build name) and, with `--out`, the report
|
|
162
|
+
basename. It does not contain step bodies, error messages, tags, attachment
|
|
163
|
+
content or filesystem paths. There is no redaction mode: if names are sensitive,
|
|
164
|
+
do not generate the email, or treat the file like the report. The footer states
|
|
165
|
+
that the file makes no network requests and has no tracking. Tolkira adds no
|
|
166
|
+
tracking pixel, remote image, font or stylesheet.
|
|
167
|
+
|
|
168
|
+
The email is inline-only by design. It counts inline attachments from the
|
|
169
|
+
summary, but does not embed screenshots, videos or logs. Attach the full report
|
|
170
|
+
for the evidence.
|
|
171
|
+
|
|
172
|
+
## Localisation and formatting
|
|
173
|
+
|
|
174
|
+
The email uses the active `--locale` catalog and sets `<html lang>` and `dir`.
|
|
175
|
+
Labels, counts, percentages and dates use the active locale; durations use an
|
|
176
|
+
email-local human form with the locale's decimal separator (`12.3 s` in English,
|
|
177
|
+
`12,3 s` in German). German uses `20.11.2026, 09:30 UTC`, comma decimals and
|
|
178
|
+
German labels, including `Vollständiger Bericht: …` and `Es ist kein vollständiger
|
|
179
|
+
Bericht angehängt.`. A test guards that the German report note stays fully German.
|
|
180
|
+
Test data and the default `Tolkira report` title render verbatim. The JSON
|
|
181
|
+
summary, CI text and CLI messages stay English. See [`I18N.md`](I18N.md) for the
|
|
182
|
+
catalog rules.
|
|
183
|
+
|
|
184
|
+
## Mail-client limits
|
|
185
|
+
|
|
186
|
+
- The template is a 600 px table (`width: 100%`, `max-width: 600px`) with inline
|
|
187
|
+
styles plus a small `<style>` block that applies below 480 px. A 600 px view
|
|
188
|
+
keeps the 2 by 2 metric grid; narrower views stack it. Real clients
|
|
189
|
+
differ in CSS support; the table carries the layout when `<style>` is stripped.
|
|
190
|
+
- Gmail clips large HTML bodies at about 102 KB. A standard summary stays well
|
|
191
|
+
under 100 KB because it lists five failures, not every scenario, and caps long
|
|
192
|
+
names.
|
|
193
|
+
- Images are often blocked; this slice has no image at all, so the email renders
|
|
194
|
+
completely without them.
|
|
195
|
+
- There is no hidden preheader in this slice. A pasted or attached body often
|
|
196
|
+
drops one, so it is deferred to a sending or `.eml` follow-up rather than ruled
|
|
197
|
+
out. The first visible line is the `Tolkira` eyebrow, then the heading and
|
|
198
|
+
status.
|
|
199
|
+
- There is no automatic subject and no `.eml` draft in this slice.
|
|
200
|
+
- Very old clients may not support the media query, forced-colours fallback or
|
|
201
|
+
system font stack; the content remains readable, the heading and note wrap
|
|
202
|
+
anywhere, and names are capped at 200 characters.
|
|
203
|
+
- Browser preview is not a guarantee of a specific mail client's rendering.
|
|
204
|
+
Check the actual client once if the mail is business-critical.
|
|
205
|
+
|
|
206
|
+
## CI mail-step example
|
|
207
|
+
|
|
208
|
+
Generate the body and the report in one step, then send them with a mail action.
|
|
209
|
+
The subject comes from `summary.json`, not from `job.status`: Tolkira exits 0
|
|
210
|
+
when tests fail, so a red run can still be a successful mail step. This GitHub
|
|
211
|
+
Actions sketch uses `dawidd6/action-send-mail`; the guard skips the send when the
|
|
212
|
+
body is missing or when a fork pull request has no mail secrets.
|
|
213
|
+
|
|
214
|
+
```yaml
|
|
215
|
+
- name: Generate report and email body
|
|
216
|
+
if: always()
|
|
217
|
+
run: >
|
|
218
|
+
npx tolkira test-results/ --out report.html --email email.html
|
|
219
|
+
--title "CI run" --json > summary.json
|
|
220
|
+
|
|
221
|
+
- name: Read test status
|
|
222
|
+
id: status
|
|
223
|
+
if: always() && hashFiles('summary.json') != ''
|
|
224
|
+
run: echo "value=$(jq -r .status summary.json)" >> "$GITHUB_OUTPUT"
|
|
225
|
+
|
|
226
|
+
- name: Send status mail
|
|
227
|
+
if: >-
|
|
228
|
+
always() && hashFiles('email.html') != '' &&
|
|
229
|
+
steps.status.outputs.value != '' &&
|
|
230
|
+
(github.event_name != 'pull_request' || !github.event.pull_request.head.repo.fork)
|
|
231
|
+
uses: dawidd6/action-send-mail@v3
|
|
232
|
+
with:
|
|
233
|
+
server_address: smtp.example.com
|
|
234
|
+
server_port: 587
|
|
235
|
+
username: ${{ secrets.MAIL_USERNAME }}
|
|
236
|
+
password: ${{ secrets.MAIL_PASSWORD }}
|
|
237
|
+
subject: "CI run: ${{ steps.status.outputs.value }}"
|
|
238
|
+
html_body: file://email.html
|
|
239
|
+
attachments: report.html,summary.json
|
|
240
|
+
to: qa@example.com
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
The SMTP settings and the send step belong to your CI. Tolkira only writes the
|
|
244
|
+
files. Jenkins `emailext` needs the HTML body as a string, the MIME type and an
|
|
245
|
+
attachment pattern:
|
|
246
|
+
|
|
247
|
+
```groovy
|
|
248
|
+
emailext(
|
|
249
|
+
subject: "CI run: ${status}",
|
|
250
|
+
body: readFile('email.html'),
|
|
251
|
+
mimeType: 'text/html',
|
|
252
|
+
attachmentsPattern: 'report.html',
|
|
253
|
+
to: 'qa@example.com'
|
|
254
|
+
)
|
|
255
|
+
```
|
|
256
|
+
|
|
257
|
+
Read `${status}` from `summary.json` (`jq -r '.status'`) for the same reason.
|
|
258
|
+
GitLab has no job-level report mail action; use its pipeline notifications or a
|
|
259
|
+
generic SMTP step and attach the same files.
|
|
260
|
+
|
|
261
|
+
## Out of scope and follow-ups
|
|
262
|
+
|
|
263
|
+
- **Sending and SMTP:** never part of Tolkira. Use a CI mail step or your mail
|
|
264
|
+
client.
|
|
265
|
+
- **`.eml`:** a real subject and a directly openable draft are a local follow-up
|
|
266
|
+
(B-023); the first slice is an HTML body because that is what CI mail steps
|
|
267
|
+
consume.
|
|
268
|
+
- **Logo or any image:** omitted; a small capped variant is B-024.
|
|
269
|
+
- **`--meta` in the email:** not included; build/branch/PR metadata or short
|
|
270
|
+
error snippets are B-025.
|
|
271
|
+
- **Hidden preheader:** deferred with `.eml` or a sending path, not impossible.
|
|
272
|
+
- **Error snippets or a configurable failed-scenario cap:** B-026; the report has
|
|
273
|
+
the detail.
|
package/docs/I18N.md
ADDED
|
@@ -0,0 +1,241 @@
|
|
|
1
|
+
# Localisation
|
|
2
|
+
|
|
3
|
+
Tolkira renders the report shell, the optional email summary and the opt-in PDF in
|
|
4
|
+
English (`en`)
|
|
5
|
+
or German (`de`). The locale is chosen per report with `--locale <tag>` or the
|
|
6
|
+
render API; one report carries one locale, and there is no in-report switcher.
|
|
7
|
+
The JSON summary, CI text and CLI output stay English. This page is the contract
|
|
8
|
+
for translators, reviewers and agents.
|
|
9
|
+
|
|
10
|
+
## Usage
|
|
11
|
+
|
|
12
|
+
```shell
|
|
13
|
+
tolkira test-results/ --out report.html --locale de
|
|
14
|
+
tolkira test-results/ --out report.html --locale de-AT
|
|
15
|
+
tolkira test-results/ --email email.html --locale de
|
|
16
|
+
tolkira test-results/ --pdf report.pdf --locale de
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
`--locale` requires `--out`, `--pdf` or `--email`. An empty or unknown tag exits
|
|
20
|
+
2 and names the supported locales. Matching is exact first, then by primary
|
|
21
|
+
subtag: `de-AT` uses the German catalog and records `lang="de-AT"`. The default
|
|
22
|
+
is `en`; there is no host, environment or browser detection. The PDF follows the
|
|
23
|
+
report locale; no PDF-specific catalog key exists.
|
|
24
|
+
|
|
25
|
+
```ts
|
|
26
|
+
await generateReport(inputs, { locale: 'de' });
|
|
27
|
+
await renderReport(inputs, { locale: 'de' });
|
|
28
|
+
new CucumberHtmlStream({ locale: 'de' });
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
`RenderOptions.locale` is optional; `de-AT` keeps its regional tag for
|
|
32
|
+
`<html lang>` while using the `de` catalog. Unknown API locales throw.
|
|
33
|
+
`renderEmail(summary, { locale })` uses the same resolution and throws on an
|
|
34
|
+
unsupported tag.
|
|
35
|
+
|
|
36
|
+
## Catalogs
|
|
37
|
+
|
|
38
|
+
| File | Shape |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| `src/i18n/locales/en.json` | `{ "<key>": { "message": "...", "description": "..." } }` |
|
|
41
|
+
| `src/i18n/locales/de.json` | `{ "<key>": "..." }` |
|
|
42
|
+
|
|
43
|
+
- One flat file per locale, dotted keys, sorted with the default
|
|
44
|
+
`Array.prototype.sort()`. The file name is the locale tag; the generator and
|
|
45
|
+
`npm run i18n:check` derive the tag from it.
|
|
46
|
+
- English is the source of truth and carries a non-empty description per key.
|
|
47
|
+
Target catalogs are value-only and must stay in key parity.
|
|
48
|
+
- Messages are full sentences with `{placeholders}`, not concatenated
|
|
49
|
+
fragments. Use `{count, plural, one {...} other {...}}` for counts.
|
|
50
|
+
- `src/i18n/catalogs.ts` is generated by `scripts/build-i18n.mjs` and committed;
|
|
51
|
+
never edit it by hand. `npm run assets` regenerates it. Its public type is a
|
|
52
|
+
`MessageKey` union plus `Record<MessageKey, string>` catalog maps, so the
|
|
53
|
+
emitted `catalogs.d.ts` does not repeat every message as a literal type.
|
|
54
|
+
- `npm run i18n:check` compares the generated module with the JSON sources and
|
|
55
|
+
validates key parity, placeholders, active plural categories per target (each
|
|
56
|
+
target tag gets its own `Intl.PluralRules`), `other`, non-empty values, sorted
|
|
57
|
+
keys and the absence of HTML/`select` syntax. It also rejects ASCII double
|
|
58
|
+
quotes, messages over 2000 characters and `=N` plural branches. It runs before
|
|
59
|
+
`assets` in `npm run check` and `npm test`, so a stale module fails.
|
|
60
|
+
|
|
61
|
+
### Supported syntax
|
|
62
|
+
|
|
63
|
+
| Syntax | Example |
|
|
64
|
+
| --- | --- |
|
|
65
|
+
| Placeholder | `Hallo {name}` |
|
|
66
|
+
| Plural | `{count, plural, one {# Datei} other {# Dateien}}` |
|
|
67
|
+
| `#` in a plural | `#` renders the count; English keeps it raw, other locales use `Intl.NumberFormat` |
|
|
68
|
+
| All plural categories | The server and the client support every category returned by `Intl.PluralRules` (`one`, `few`, `many`, `other`, and so on); an absent category uses `other` |
|
|
69
|
+
| Numeric parameters | `{count}` and `#` are locale-formatted in every visible count; English report output stays raw, the email groups digits |
|
|
70
|
+
|
|
71
|
+
Not supported: `select`, nested plural/select, HTML (`<`/`>`), ASCII double
|
|
72
|
+
quotes (use typographic quotes such as `„…“`), messages over 2000 characters,
|
|
73
|
+
`=N` plural branches, unbalanced braces and empty placeholders. The catalog
|
|
74
|
+
check rejects all of these except a stray `}`, which only the renderer rejects.
|
|
75
|
+
|
|
76
|
+
### Adding a locale
|
|
77
|
+
|
|
78
|
+
1. Copy `en.json` to `<tag>.json`, where the file name is the locale tag, for
|
|
79
|
+
example `fr.json` or `de-AT.json`.
|
|
80
|
+
2. Translate every `message`; keep placeholders and `#` unchanged, use
|
|
81
|
+
typographic quotes, stay under 2000 characters and do not add `=N`
|
|
82
|
+
branches. Sort the keys and remove the descriptions from the new file.
|
|
83
|
+
3. Run `npm run assets`. `scripts/build-i18n.mjs` discovers every
|
|
84
|
+
`src/i18n/locales/*.json`, derives each tag from its file name, emits one
|
|
85
|
+
`Record<MessageKey, string>` catalog per locale and adds it to `catalogs`; no
|
|
86
|
+
manual `SUPPORTED_LOCALES` or `CATALOGS` edit is needed. `npm run i18n:check`
|
|
87
|
+
validates each target with its own `Intl.PluralRules` tag.
|
|
88
|
+
4. Only for an RTL locale, add its tag to `DIRECTIONS` in
|
|
89
|
+
`src/render/i18n.ts`; the default is `ltr`.
|
|
90
|
+
5. Add or update tests and docs, including the CLI usage line that names the
|
|
91
|
+
supported locales and any test that asserts that list. Then run
|
|
92
|
+
`npm run i18n:check`, the focused render tests, `npm run check` and the
|
|
93
|
+
browser gate at 320, 390, 768, 1024 and 1440 px plus 320 px at 200 % text
|
|
94
|
+
zoom.
|
|
95
|
+
|
|
96
|
+
A regional variant is a separate JSON file whose tag matches the region, such as
|
|
97
|
+
`de-AT.json`; without one, `de-AT` falls back to `de`. If a target file is
|
|
98
|
+
missing an English key or contains an extra key, the generator and
|
|
99
|
+
`npm run i18n:check` fail with the file and key instead of dropping it.
|
|
100
|
+
|
|
101
|
+
## Fallback and formatting
|
|
102
|
+
|
|
103
|
+
- On the server, a missing key falls back to English, then to the key itself.
|
|
104
|
+
- A missing plural category uses `other`.
|
|
105
|
+
- All visible report counts are locale-formatted, on the server and in the
|
|
106
|
+
client; English report digits stay raw. Other locales use `Intl.NumberFormat`,
|
|
107
|
+
so German decimals use a comma and percentages include the locale's no-break
|
|
108
|
+
space. The report's English percentages, durations and dates keep the previous
|
|
109
|
+
output; non-English durations keep their unit labels and use a locale decimal
|
|
110
|
+
separator. The email groups English digits (`8,509 bytes`) and uses a short
|
|
111
|
+
human duration form (`12 ms`, `12.3 s`, `1 min 5 s`, `1 h 2 min`), because it
|
|
112
|
+
is read outside the report shell.
|
|
113
|
+
- German dates are `DD.MM.YYYY, HH:MM UTC`; that order is German, not a rule
|
|
114
|
+
for every non-English locale. The date parts come from
|
|
115
|
+
`Intl.DateTimeFormat` with `timeZone: 'UTC'`. `de` is currently the only
|
|
116
|
+
non-English catalog; a new locale needs its own date order and tests.
|
|
117
|
+
- Counts used for sorting, `data-*` attributes, URL state and the summary stay
|
|
118
|
+
numeric.
|
|
119
|
+
- The report always embeds the active `client.*` messages as an inert JSON
|
|
120
|
+
block. The client uses that block as its only source: the old English fallback
|
|
121
|
+
map is gone, and if the block or a key is missing the client shows the key
|
|
122
|
+
itself. The client supports every `Intl.PluralRules` category and formats `#`
|
|
123
|
+
and numeric parameters; the browser bundle contains no non-English catalog.
|
|
124
|
+
|
|
125
|
+
## What is not translated
|
|
126
|
+
|
|
127
|
+
Test data renders verbatim in every locale: feature, scenario, rule and step
|
|
128
|
+
names, Gherkin keywords, tags, doc strings, data tables, error text, attachment
|
|
129
|
+
content and names, metadata values and URIs. So do machine identifiers: status
|
|
130
|
+
ids, `data-*` values, CSS classes, URL parameters, JSON keys and values, CLI
|
|
131
|
+
commands, exit codes and the default title `Tolkira report` (it feeds the
|
|
132
|
+
history id). The CLI CI formatter, CLI usage/errors and the JSON summary stay
|
|
133
|
+
English.
|
|
134
|
+
|
|
135
|
+
Synthetic report labels such as `Feature:`, `Scenario:`, `Attempt:`, `Started:`,
|
|
136
|
+
`Rule:` and `Steps` are chrome and are translated. Clipboard labels are
|
|
137
|
+
translated; the copied names and text are not.
|
|
138
|
+
|
|
139
|
+
## Email labels
|
|
140
|
+
|
|
141
|
+
The email summary is a second localized surface with its own `email.*` keys. It
|
|
142
|
+
reuses `status.*`, `report.scenarioWithResults`, `report.featureCount`,
|
|
143
|
+
`report.stepCount`, `report.attachmentCount`, `format.bytes`,
|
|
144
|
+
`format.noAttempts` and `format.emptyRatio`, sets
|
|
145
|
+
`<html lang>` and `dir`, and formats its counts, percentages and `Generated` date
|
|
146
|
+
with the same translator. Counts use the locale's number format, and durations
|
|
147
|
+
use the email-local human form in every locale. Email test data, the optional
|
|
148
|
+
title and the report basename render verbatim. The email is server-rendered, so it
|
|
149
|
+
does not use the client `client.*` block.
|
|
150
|
+
|
|
151
|
+
| Key | German |
|
|
152
|
+
| --- | --- |
|
|
153
|
+
| `email.title` | `{title} — {status}` |
|
|
154
|
+
| `email.generated` | `Erstellt: {date}` |
|
|
155
|
+
| `email.passRate` | `Erfolgsquote` |
|
|
156
|
+
| `email.passRateNote` | `Die Erfolgsquote zählt nur ausgeführte (nicht übersprungene) Szenarien.` |
|
|
157
|
+
| `email.features` | `Features` |
|
|
158
|
+
| `email.steps` | `Schritte` |
|
|
159
|
+
| `email.duration` | `Dauer` |
|
|
160
|
+
| `email.attachments` | `Anhänge im Lauf` |
|
|
161
|
+
| `email.unexecuted` | `{count} nicht ausgeführt` |
|
|
162
|
+
| `email.failedTitle` | `Fehlgeschlagene Szenarien` |
|
|
163
|
+
| `email.moreFailures` | `{count, plural, one {# weiteres fehlgeschlagenes Szenario} other {# weitere fehlgeschlagene Szenarien}}` |
|
|
164
|
+
| `email.noFailures` | `Keine fehlgeschlagenen Szenarien.` |
|
|
165
|
+
| `email.noFailuresStatus` | `Keine fehlgeschlagenen Szenarien. Laufstatus: {status}.` |
|
|
166
|
+
| `email.empty` | `Keine Szenarien in diesem Lauf.` |
|
|
167
|
+
| `email.reportNote` | `Vollständiger Bericht: {name}. Bitte separat anhängen oder teilen.` |
|
|
168
|
+
| `email.reportMissing` | `Es ist kein vollständiger Bericht angehängt.` |
|
|
169
|
+
| `email.footer` | `Erstellt von Tolkira. Eine Datei. Keine Netzwerkanfragen. Kein Tracking.` |
|
|
170
|
+
|
|
171
|
+
The report note is recipient-facing and must stay fully German. A guard test
|
|
172
|
+
fails if the German output falls back to the English `Full report` label or
|
|
173
|
+
mixes English copy into the note; the standalone note uses
|
|
174
|
+
`email.reportMissing`, not a CLI instruction to run `--out`.
|
|
175
|
+
|
|
176
|
+
The PDF adds no catalog keys. It prints the report HTML with the resolved locale,
|
|
177
|
+
so its labels, counts, percentages, durations and dates come from the report
|
|
178
|
+
catalog and translator. `--pdf` therefore counts as a locale target in the CLI
|
|
179
|
+
guard.
|
|
180
|
+
|
|
181
|
+
## German glossary
|
|
182
|
+
|
|
183
|
+
| English | German |
|
|
184
|
+
| --- | --- |
|
|
185
|
+
| Report | Bericht |
|
|
186
|
+
| Details | Details |
|
|
187
|
+
| Trends | Trends |
|
|
188
|
+
| Scenario | Szenario |
|
|
189
|
+
| Step | Schritt |
|
|
190
|
+
| Attachment | Anhang |
|
|
191
|
+
| Run | Lauf |
|
|
192
|
+
| Passed | Bestanden |
|
|
193
|
+
| Failed | Fehlgeschlagen |
|
|
194
|
+
| Skipped | Übersprungen |
|
|
195
|
+
| Pending | Ausstehend |
|
|
196
|
+
| Flaky | flaky |
|
|
197
|
+
| Undefined | Undefiniert |
|
|
198
|
+
| Ambiguous | Mehrdeutig |
|
|
199
|
+
| Pass rate | Erfolgsquote |
|
|
200
|
+
| Test time | Testzeit |
|
|
201
|
+
| Email summary | E-Mail-Zusammenfassung |
|
|
202
|
+
| Generated | Erstellt |
|
|
203
|
+
| Duration | Dauer |
|
|
204
|
+
| Full report | Vollständiger Bericht |
|
|
205
|
+
| Feature | Feature |
|
|
206
|
+
| Hook | Hook |
|
|
207
|
+
| Rule | Rule |
|
|
208
|
+
| URI | URI |
|
|
209
|
+
|
|
210
|
+
`flaky` is a technical term in both catalogs (`report.flakySummary`,
|
|
211
|
+
`report.flakyBadge`); the German report keeps it untranslated. `@flaky` test
|
|
212
|
+
tags render verbatim, are matched lowercase and exactly, and are inherited, so
|
|
213
|
+
a feature, rule or outline tag can badge and count every affected row.
|
|
214
|
+
|
|
215
|
+
The address form is impersonal; avoid `du` and `Sie`. The full
|
|
216
|
+
`Fehlgeschlagen` chip is used and wraps; there is no short form. `de` is the
|
|
217
|
+
generic catalog and `de-AT` uses it. Final copy also uses `Laufzeit` for the
|
|
218
|
+
runtime value, `Lauf-Hooks` for run-level hooks in the test-time notes,
|
|
219
|
+
`Aufgezeichnet am` for the recorded-at column, `Basisverzeichnisse` for the
|
|
220
|
+
allowed roots and `Fehlerdetails kopieren` for the copy-failure action. The
|
|
221
|
+
attachment notice and the Details pass-rate note are full sentences; the German
|
|
222
|
+
`triage.moreInReport` has `one`/`other` branches and the German `triage.lead`
|
|
223
|
+
agrees its verb with the need count, not the total.
|
|
224
|
+
|
|
225
|
+
## Agent checklist
|
|
226
|
+
|
|
227
|
+
1. Read this guide and the active OpenSpec change.
|
|
228
|
+
2. Find the key with `rg '"' src/i18n/locales/en.json`, or narrow by prefix with
|
|
229
|
+
`rg '"client\.'` / `rg '"report\.'` or another area name.
|
|
230
|
+
3. Edit only that key in `en.json` and every target catalog; keep placeholders
|
|
231
|
+
and `#` identical. Use typographic quotes, keep the message below 2000
|
|
232
|
+
characters and do not add `=N` branches.
|
|
233
|
+
4. Add or update the English description when the meaning changes.
|
|
234
|
+
5. Run `npm run i18n:check` and the focused Vitest suites.
|
|
235
|
+
6. Run `npm run check`; for report changes also run the browser gate.
|
|
236
|
+
7. Check the longest German string at 320 px and 200 % text zoom.
|
|
237
|
+
|
|
238
|
+
Do not edit `src/i18n/catalogs.ts`, translate test data, add detection, or add a
|
|
239
|
+
non-English catalog to the client bundle. The report always embeds only the
|
|
240
|
+
active `client.*` messages as an inert, JSON-escaped data block; the client
|
|
241
|
+
formatter lives in `src/render/client-i18n.ts`.
|
package/docs/JUNIT-XML.md
CHANGED
|
@@ -3,7 +3,9 @@
|
|
|
3
3
|
Tolkira accepts JUnit XML from any framework or CI system that can write it.
|
|
4
4
|
This is a **basic** input mode: suites, test cases, statuses, durations,
|
|
5
5
|
failures and `system-out`/`system-err`. It does not claim Gherkin steps,
|
|
6
|
-
screenshots, videos or traces.
|
|
6
|
+
screenshots, videos or traces. Playwright results have a native JSON adapter;
|
|
7
|
+
see [`PLAYWRIGHT-JSON.md`](PLAYWRIGHT-JSON.md) and prefer that over Playwright's
|
|
8
|
+
JUnit output.
|
|
7
9
|
|
|
8
10
|
## Usage
|
|
9
11
|
|
|
@@ -23,17 +25,35 @@ tolkira reports/junit.xml reports/cucumber.json --out report.html
|
|
|
23
25
|
| JUnit XML | Report |
|
|
24
26
|
| --- | --- |
|
|
25
27
|
| `<testsuite name="...">` | feature; `name` falls back to `classname` or `JUnit suite N` |
|
|
28
|
+
| `<testcase classname="...">` | splits the suite only for a nested-class shape (a `$` name or a dotted outer prefix); classname templates keep the suite feature |
|
|
29
|
+
| `<testsuite description="...">` | feature description, rendered as safe Markdown |
|
|
26
30
|
| `<testcase name="...">` | scenario with one synthetic step |
|
|
31
|
+
| `<testcase description="...">` | scenario description, rendered as safe Markdown |
|
|
27
32
|
| `<failure>` / `<error>` | failed; `message` and text become the step error |
|
|
28
33
|
| `<skipped>` | skipped |
|
|
29
34
|
| no status child | passed |
|
|
30
35
|
| `time="1.234"` | 1234 ms |
|
|
31
|
-
| `system-out` / `system-err` | `text/plain` attachments |
|
|
36
|
+
| `system-out` / `system-err` | `text/plain` attachments (testcase level only) |
|
|
32
37
|
| `file` attribute | not used yet; the input file is the URI |
|
|
33
38
|
|
|
34
39
|
Suites and test cases are recomputed; `<testsuite tests="...">` totals are not
|
|
35
40
|
trusted.
|
|
36
41
|
|
|
42
|
+
## Verified producer shapes
|
|
43
|
+
|
|
44
|
+
`test/junit-producers.test.ts` covers fixtures modelled on common producers.
|
|
45
|
+
They are synthetic but real-shaped, and each records its provenance in a
|
|
46
|
+
comment plus `test/fixtures/junit/PROVENANCE.md`:
|
|
47
|
+
|
|
48
|
+
| Fixture | Producer shape | Covered behaviour |
|
|
49
|
+
| --- | --- | --- |
|
|
50
|
+
| `jest-junit.xml` | jest-junit 16.x | `<testsuites>`, `<testsuite>`, passed/failed/skipped, `<failure>`, suite-level output tolerated and ignored |
|
|
51
|
+
| `pytest.xml` | pytest 8.x `--junitxml` | `<properties>`, passed/failed/skipped, `<failure>`, testcase-level output attached |
|
|
52
|
+
| `gradle.xml` | Gradle 8.x test task | root `<testsuite>`, `<properties>`, passed/failed/error/skipped, `<failure>`, `<error>`, suite-level output tolerated and ignored |
|
|
53
|
+
|
|
54
|
+
These are regression fixtures, not a claim that every dialect is supported.
|
|
55
|
+
Suite-level `system-out`/`system-err` is intentionally not attached.
|
|
56
|
+
|
|
37
57
|
## Example
|
|
38
58
|
|
|
39
59
|
```xml
|
|
@@ -56,12 +76,34 @@ one.
|
|
|
56
76
|
|
|
57
77
|
- No screenshots, videos, traces or arbitrary attachments.
|
|
58
78
|
- No Gherkin steps, doc strings or data tables.
|
|
79
|
+
- `description` is not a standard JUnit XML attribute. Tolkira reads it on
|
|
80
|
+
`<testsuite>` and `<testcase>` as a best-effort feature/scenario description
|
|
81
|
+
when a producer writes it; dialects without it keep plain names. The text is
|
|
82
|
+
rendered as the same safe Markdown as Cucumber descriptions and never enters
|
|
83
|
+
the JSON summary.
|
|
59
84
|
- `system-out`/`system-err` are text only; HTML is escaped and never executed.
|
|
60
85
|
Only testcase-level output is mapped; suite-level output is ignored.
|
|
86
|
+
- `<!DOCTYPE>` and `<!ENTITY>` declarations are rejected as a parser-safety
|
|
87
|
+
measure. The same text inside comments, CDATA sections or processing
|
|
88
|
+
instructions is data and is accepted.
|
|
61
89
|
- A document with no test cases is rejected with a clear error instead of
|
|
62
90
|
rendered as an empty run.
|
|
63
|
-
-
|
|
64
|
-
|
|
91
|
+
- Grouping follows `<testsuite>` except for a nested-class shape: at least two
|
|
92
|
+
distinct non-empty testcase `classname` values and either a classname with `$`
|
|
93
|
+
or one classname as a dotted outer prefix of another. Such a suite becomes one
|
|
94
|
+
feature per classname, using the classname alone as the feature name; testcases
|
|
95
|
+
without a classname stay in a suite-named feature. Other multi-classname
|
|
96
|
+
suites keep the suite feature, so a jest-junit `"{classname} {title}"` template
|
|
97
|
+
does not turn every test into a feature. All testcases, statuses, durations and
|
|
98
|
+
the source file are preserved, so Maven Surefire's `Outer$Nested` class rows
|
|
99
|
+
are no longer folded into the outer feature.
|
|
100
|
+
- Test time is the sum of `<testcase time>` values. Suite wall time (including
|
|
101
|
+
Spring context setup) is not included, so do not compare it directly with a
|
|
102
|
+
Surefire total (B-037).
|
|
103
|
+
- JUnit XML dialects differ; Tolkira handles the common Jenkins/Gradle/Maven/
|
|
104
|
+
pytest/Playwright shape and fails clearly otherwise. Playwright JSON is the
|
|
105
|
+
richer Playwright path; this JUnit mode cannot recover projects, retries,
|
|
106
|
+
nested steps or attachments.
|
|
65
107
|
- The existing 100 MB total input limit applies.
|
|
66
108
|
|
|
67
109
|
## Why a parser dependency
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Limitations
|
|
2
|
+
|
|
3
|
+
Last verified: **2026-11-22**. "Verified" means the repository state on that
|
|
4
|
+
date: the cited fixture or test exists and passes. It does not mean every
|
|
5
|
+
producer was re-run on that date.
|
|
6
|
+
|
|
7
|
+
| Area | Status | Last verified | What it means |
|
|
8
|
+
| --- | --- | --- | --- |
|
|
9
|
+
| Behave 1.3.3 | Partial | 2026-11-19 | Real runs (`behave-real.json`, `behave-real-attachments.json`) cover pass/fail/error statuses, seconds-based durations, a multiline doc string, a data table, a multiline error message and an inline PNG. The recorded rich run contains one `error` status; `hook_error` and `cleanup_error` are covered by synthetic tests. Behave's formatter omits error text for `error` and `hook_error` step results; `cleanup_error` is recorded on the scenario/feature during context cleanup and is accepted defensively in a step result. Dry-run/WIP statuses (`untested`, `untested_pending`, `untested_undefined`, `pending_warn`) and `xfailed`/`xpassed` are unsupported, and scenario-level hook failures without a step result are not reconstructed. |
|
|
10
|
+
| JUnit XML | Basic | 2026-11-13 | Suites, test cases, statuses, durations, failures, skipped cases and `system-out`/`system-err`. No Gherkin steps, screenshots, traces or arbitrary attachments. JUnit XML is not a replacement for a native Playwright report; Playwright projects should use the Playwright JSON adapter. |
|
|
11
|
+
| Legacy JSON outlines | Flat | 2026-11-13 | Examples arrive expanded. Tolkira keeps those rows flat and does not reconstruct the original outline table. Messages outlines are grouped. |
|
|
12
|
+
| Rules and backgrounds in the summary | HTML only | 2026-11-13 | The version-2 JSON summary exposes a flat scenario list and inherited tags. Rules and backgrounds are an HTML presentation feature. |
|
|
13
|
+
| Themes and charts | Not supported | 2026-11-13 | Branding is title, metadata, logo, accent colour and dark mode. The old cucumber-html-reporter themes and charts are not reproduced. |
|
|
14
|
+
| External attachments | Not supported | 2026-11-19 | Output stays one file. Current limits: 100 MB total input, 50 MB per attachment, 100 MB total attachments, with a warning above 10 MB. Decision: [ADR 0006](adr/0006-inline-attachments.md). |
|
|
15
|
+
| History and trends | Opt-in | 2026-11-19 | `--history <file>` keeps the last 30 aggregate runs in a schema-version-1 sidecar and adds a Trends tab from the second run. Entries hold id, recorded time, resolved title, status, counts, duration and attachment totals only: no scenario names, step text, paths, tags or attachment content. There is no scenario-level history, multi-suite grouping or dashboard, and the CLI is the only writer, so concurrent generation on one sidecar can lose an update. A report embeds the prior-run aggregates it was generated with. |
|
|
16
|
+
| Localisation | en, de | 2026-11-20 | The report shell and the opt-in email summary take their labels and accessibility text from the active catalog; all visible counts are locale-formatted (English report digits stay raw; the email groups them), including the client. Test data, Gherkin keywords, attachment content and names, metadata and URIs render verbatim in every locale; the JSON summary and CI text stay English. `de-AT` uses `de` but keeps its `lang` tag. Only left-to-right locales ship: there is no RTL layout, pseudo-locale, localized default title or in-report language switcher. |
|
|
17
|
+
| Native Playwright adapter | Basic | 2026-11-22 | The Playwright JSON reporter is read natively: projects, file/describe rules, retries, nested steps, expected/unexpected outcomes, tags, annotations, inline and in-tree path attachments, `stdout`/`stderr` and run-level errors. The final attempt is the visible one. No trace viewer or HTML-report replay is claimed; trace zip attachments are skipped. Path attachments outside the input file's directory and the process working directory are skipped; ones that exist as regular files are counted and surface as a run-level HTML notice, a per-scenario notice with that scenario's final-attempt count, and, when HTML or a PDF is written, a CLI warning, while missing files, symlinks, URL/UNC paths, trace zips and earlier attempts are not counted. |
|
|
18
|
+
| Flaky visibility | Display only | 2026-11-22 | A scenario whose pickle tag set (own or inherited) includes the lowercase, exact `@flaky` tag shows a localized badge, and when at least one exists the run metadata gets a separate count such as `1 tagged @flaky`, distinct from the status counts. Feature, rule and outline-level tags badge every affected row, including every outline example, so the count can exceed the visible tag labels; a failed or unexecuted tagged scenario also counts. The badge and count are report-only (HTML/PDF), not a computed flake rate or detection, and do not appear in the JSON summary, CI or email; the scenario keeps its existing status everywhere. There is no flaky status, first-attempt error/screenshot or retry history, no extra summary field, and no separate filter; the existing tag filter still selects `@flaky`. |
|
|
19
|
+
| Browser and accessibility coverage | Ongoing | 2026-11-13 | Chromium, Firefox and WebKit at 320, 390, 768, 1024 and 1440 px, plus 320 px at 200 % text zoom. This is not a formal WCAG audit. |
|
|
20
|
+
| Print and PDF | Browser print plus opt-in PDF | 2026-11-20 | With JavaScript, printing includes Report, Details and Trends when present and opens every detail; the screen state returns where the browser fires `afterprint`, and a missed event cannot stack panels because the reveal is print-scoped. Without JavaScript only the active Report prints and closed details stay closed. An active search or filter carries into print and narrows the Report view. Interactive chrome and video/audio attachments do not print; very wide matrices may need landscape; the browser's headers/footers, URL and page numbers stay user-controlled (the footer can reveal a local path). `--pdf <file>` prints the same complete archive through the optional Playwright Chromium engine: A4 portrait via `format: 'A4'`, 12 mm print CSS margins, `printBackground`, no header/footer/URL/page number, Chromium-only, no CLI filter state (a fresh page), no PDF-only logo/Trends, no Letter/custom paper. HTTP(S) images in caller HTML are aborted; inline data URIs render. It needs `playwright-core` (about 13 MB) plus the Chromium headless shell (285.8 MB unpacked) and helpers (about 4 MB; ffmpeg/winldd; roughly 303 MB unpacked total; the full Chromium build is not required), installed in the project where Tolkira runs; a missing engine exits 1 with the install command. `page.pdf` has no timeout option and there is no 30 s browser print default: `setContent` gets the load timeout and an explicit deadline bounds the font/image wait and the print (default five minutes; `--pdf-timeout`/`PdfOptions.timeoutMs` override it). The default Chromium launch keeps `chromiumSandbox: false`; the CSP and escaped attachments are the boundary, so run untrusted attachments in a container or VM (B-029). There is no separate PDF size cap. A print button, repeated print title, custom page-number control, tagged PDF and per-page header remain absent. Volume measured on the maintainer's machine (Chromium, 2026-11-20), not CI: 2,000 scenarios take about 5 s and produce about 670 A4 pages, 4,000 about 21 s, and 10,000 about 2.7 min and about 3,337 pages; CI can be slower, so raise `--pdf-timeout`. There is no print cap beyond the normal 10 MB size warning: clear filters for the whole run, and use the failures-only filter for large suites. Sparse attachment-only content can leave nearly empty pages, a known cosmetic limit. The PDF is not byte-identical across engine, font or OS versions. |
|
|
21
|
+
| Email summary | Opt-in | 2026-11-20 | `--email <file>` writes one self-contained HTML mail body from the summary; without the flag no email file exists. It has no SMTP, sending, scheduling, recipients, `.eml` draft, logo, image, `--meta` content or redaction mode, and no hidden preheader in this slice (deferred). It lists the first five failed scenarios in summary order (fixed cap), not the five slowest, counts inline attachments only, and names the report by basename or says that no full report is attached. The heading, note and failed names wrap anywhere and displayed names are capped at 200 characters; English digits are grouped (`8,509 bytes`) and durations are human (`12 ms`, `12.3 s`, `1 min 5 s`, `1 h 2 min`). A standard summary stays under 100 KB. Browser preview at 600 and 320 px is tested; Gmail/Outlook/Apple Mail rendering, clipping and image blocking are not covered, and the HTML body cannot set a mail subject. |
|