tolkira 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/DESIGN.md +131 -9
  2. package/GLOSSARY.md +201 -1
  3. package/README.md +413 -29
  4. package/SECURITY.md +3 -0
  5. package/dist/attachment.d.ts +33 -0
  6. package/dist/attempt-diagnostics.d.ts +2 -0
  7. package/dist/ci.d.ts +12 -0
  8. package/dist/cli.js +6881 -2691
  9. package/dist/cli.js.map +4 -4
  10. package/dist/compat/cucumber-html-reporter.cjs +12 -0
  11. package/dist/compat/cucumber-html-reporter.d.cts +10 -0
  12. package/dist/compat/cucumber-html-reporter.js +263 -26
  13. package/dist/compat/cucumber-html-reporter.js.map +4 -4
  14. package/dist/compat/options.d.ts +4 -0
  15. package/dist/email.d.ts +18 -0
  16. package/dist/history.d.ts +41 -0
  17. package/dist/i18n/catalogs.d.ts +7 -0
  18. package/dist/index.d.ts +15 -7
  19. package/dist/index.js +5301 -1709
  20. package/dist/index.js.map +4 -4
  21. package/dist/input/background.d.ts +4 -0
  22. package/dist/input/detect.d.ts +30 -0
  23. package/dist/input/load.d.ts +3 -0
  24. package/dist/input/playwright.d.ts +14 -0
  25. package/dist/options.d.ts +18 -0
  26. package/dist/output.d.ts +9 -1
  27. package/dist/pdf.d.ts +9 -0
  28. package/dist/render/ansi.d.ts +2 -0
  29. package/dist/render/attachments.d.ts +3 -0
  30. package/dist/render/ci.d.ts +4 -0
  31. package/dist/render/client-bundle.d.ts +1 -0
  32. package/dist/render/client-i18n.d.ts +7 -0
  33. package/dist/render/details.d.ts +5 -0
  34. package/dist/render/dom-contract.d.ts +42 -0
  35. package/dist/render/format.d.ts +34 -0
  36. package/dist/render/html.d.ts +2 -9
  37. package/dist/render/i18n.d.ts +30 -0
  38. package/dist/render/markdown.d.ts +2 -0
  39. package/dist/render/matrix.d.ts +3 -0
  40. package/dist/render/print-script.d.ts +1 -0
  41. package/dist/render/report-client.d.ts +1 -0
  42. package/dist/render/report.d.ts +4 -0
  43. package/dist/render/search.d.ts +21 -0
  44. package/dist/render/trends.d.ts +7 -0
  45. package/dist/render/triage.d.ts +5 -0
  46. package/dist/render/types.d.ts +26 -0
  47. package/dist/report/index-run.d.ts +69 -0
  48. package/dist/report/model.d.ts +96 -6
  49. package/dist/report/status.d.ts +12 -0
  50. package/dist/stream.d.ts +15 -0
  51. package/dist/summary.d.ts +17 -2
  52. package/dist/text.d.ts +15 -0
  53. package/dist/warnings.d.ts +6 -0
  54. package/docs/AGENT-INTERFACE.md +504 -33
  55. package/docs/CI.md +185 -0
  56. package/docs/COMPATIBILITY.md +121 -16
  57. package/docs/CUCUMBER-COMPATIBILITY.md +232 -15
  58. package/docs/EMAIL.md +273 -0
  59. package/docs/I18N.md +241 -0
  60. package/docs/JUNIT-XML.md +46 -4
  61. package/docs/LIMITATIONS.md +21 -0
  62. package/docs/PDF.md +255 -0
  63. package/llms.txt +63 -0
  64. package/package.json +46 -8
package/docs/CI.md ADDED
@@ -0,0 +1,185 @@
1
+ # CI output
2
+
3
+ Tolkira can format the same stable summary it uses for `--json` as CI text.
4
+ The formatter is offline and deterministic: it reads the in-memory summary,
5
+ makes no HTTP, Git or telemetry request, and emits the same bytes for the same
6
+ input and options.
7
+
8
+ ```shell
9
+ # GitHub Actions annotations on stdout
10
+ tolkira test-results/ --ci github
11
+
12
+ # Markdown on stdout
13
+ tolkira test-results/ --ci markdown
14
+
15
+ # Markdown in a standalone file, with or without HTML
16
+ tolkira test-results/ --ci markdown --ci-out tolkira-summary.md
17
+ tolkira test-results/ --out report.html --ci markdown --ci-out tolkira-summary.md
18
+
19
+ # Report, PDF and CI file in one run; confirmations go to stdout
20
+ tolkira test-results/ --out report.html --pdf report.pdf \
21
+ --ci markdown --ci-out tolkira-summary.md
22
+ ```
23
+
24
+ | Option | Meaning |
25
+ | --- | --- |
26
+ | `--ci github` | Print GitHub Actions workflow commands to stdout. |
27
+ | `--ci markdown` | Print a deterministic Markdown summary to stdout or `--ci-out`. |
28
+ | `--ci-out <file>` | Write the Markdown summary to a standalone file instead of stdout. Requires `--ci markdown`. |
29
+ | `--title <text>` | Report title; also the Markdown heading, escaped so it stays one inert heading. |
30
+ | `--logo <file>` | Embed a logo in the HTML report. Requires `--out`. |
31
+
32
+ `--ci` does not require `--out`. Without `--ci-out`, the CI text is the only
33
+ thing on stdout, so a redirect or a step summary receives clean output.
34
+ `--json` and `--ci` share stdout only when `--ci-out` keeps the CI text out of
35
+ stdout.
36
+
37
+ ## GitHub Actions
38
+
39
+ GitHub only reads workflow commands from stdout. Run `--ci github` as a normal
40
+ step and do not redirect its stdout:
41
+
42
+ ```yaml
43
+ - name: Cucumber annotations
44
+ if: always()
45
+ run: npx tolkira test-results/ --ci github
46
+
47
+ - name: Cucumber job summary
48
+ if: always()
49
+ run: npx tolkira test-results/ --ci markdown >> "$GITHUB_STEP_SUMMARY"
50
+ ```
51
+
52
+ The first step adds an error annotation for every failed, undefined and
53
+ ambiguous scenario and a notice for executed skipped, pending and unknown
54
+ scenarios. A final `::notice title=Tolkira::` line reports the exact scenario
55
+ counts and the unexecuted count. An empty run produces one `No scenarios`
56
+ notice; when the run status is failed, the run error is emitted before it.
57
+ Unexecuted scenarios are counted, never annotated individually. When the run
58
+ status is failed but no scenario produced an error, for example because a
59
+ global hook failed, Tolkira emits one
60
+ `::error title=Tolkira::Run failed: ...` annotation and marks the final notice
61
+ as `run failed`.
62
+
63
+ The second step appends Markdown to the job summary. Use `>>`, not `>`, so the
64
+ content from earlier steps survives. Do not point `--ci-out` at
65
+ `$GITHUB_STEP_SUMMARY`: `--ci-out` replaces the whole file atomically, while
66
+ `>>` appends. `--ci-out` is for a dedicated Markdown file, a GitLab artifact or
67
+ a later upload step.
68
+
69
+ GitHub mode caps at 50 error annotations and 20 notice annotations. When a cap
70
+ is hit, one `::notice title=Tolkira::<N> more annotations omitted` line states
71
+ the remainder, and the final count notice keeps the exact totals. GitHub
72
+ Enterprise Server applies its own limits to notice annotations; individual
73
+ annotations can be dropped there, but the final count notice still carries the
74
+ exact numbers.
75
+
76
+ The summary URI is report-relative, not a CI workspace path, so annotations
77
+ carry no `file=` property. The feature, scenario name and first failure message
78
+ provide the identity. Scenarios that share a feature and name are therefore
79
+ indistinguishable in annotations; the final count notice still includes them.
80
+
81
+ ## GitLab
82
+
83
+ GitLab does not understand GitHub workflow commands. Keep the Markdown summary
84
+ as an artifact:
85
+
86
+ ```yaml
87
+ cucumber:
88
+ script:
89
+ - npx tolkira test-results/ --out report.html --ci markdown --ci-out tolkira-summary.md
90
+ artifacts:
91
+ when: always
92
+ paths:
93
+ - report.html
94
+ - tolkira-summary.md
95
+ ```
96
+
97
+ A GitLab job that only runs Tolkira stays green even when tests failed, because
98
+ the report command exits 0 by design. Do not read the job status as the test
99
+ outcome: read the counts from the artifact, or let the test runner fail the job
100
+ separately.
101
+
102
+ Or without HTML, keep only the summary as the artifact:
103
+
104
+ ```yaml
105
+ cucumber-summary:
106
+ script:
107
+ - npx tolkira test-results/ --ci markdown --ci-out tolkira-summary.md
108
+ artifacts:
109
+ when: always
110
+ paths:
111
+ - tolkira-summary.md
112
+ ```
113
+
114
+ Markdown mode writes a heading, a complete counts table including the overall
115
+ status, the failed scenarios and the skipped scenario names. Dynamic names,
116
+ URIs and messages are rendered as inert code spans or fenced blocks, never as
117
+ links, images or raw HTML; line breaks in a name, URI or feature collapse to a
118
+ space so they cannot start a new Markdown block. The failed list is capped at
119
+ 50 scenarios, each failure message at 500 raw characters before formatting, and
120
+ the skipped name list at 20 names. The counts stay complete and every omitted
121
+ count is stated.
122
+
123
+ ## stdout contract
124
+
125
+ `--json` and CI text both target stdout, so they combine only when CI goes to a
126
+ file:
127
+
128
+ | `--ci` | `--json` | `--ci-out` | `--email` | stdout |
129
+ | --- | --- | --- | --- | --- |
130
+ | yes | no | no | any | CI text only |
131
+ | yes | no | yes | no | `Report written to ...`, or nothing without `--out` |
132
+ | yes | no | yes | yes | `Report written to ...` and `Email written to ...`, or the email line alone without `--out` |
133
+ | yes | yes | yes | any | JSON summary only |
134
+ | yes | yes | no | any | usage error |
135
+
136
+ The human confirmation lines (`Report written to ...`,
137
+ `PDF written to ...`, `Email written to ...`) are suppressed whenever CI text or
138
+ the JSON summary owns stdout, so annotations, Markdown and JSON are never mixed
139
+ with other output. `--email` and `--pdf` write their files whenever the command
140
+ is accepted and change neither the CI text nor the JSON bytes. Their own
141
+ confirmations appear only when stdout is free; `--json --ci-out` suppresses the
142
+ PDF line too. The PDF is written after the report and before the email, history
143
+ and CI file, so a PDF failure keeps the report and skips the later outputs;
144
+ `PDF written to` appears between the report and email lines. `--pdf-timeout`
145
+ overrides the PDF render deadline without changing any CI output.
146
+
147
+ Report warnings, such as skipped path attachments, go to stderr when HTML or a
148
+ PDF is written. They never enter stdout, the JSON summary or the CI text.
149
+
150
+ `--ci-out` must not overwrite an input file, the logo, the `--out` file, the
151
+ `--pdf` file or the
152
+ `--email` file. The check resolves each path through its nearest existing
153
+ ancestor, so a symlinked or junctioned directory cannot hide a target that does
154
+ not exist yet, and it treats paths as the same file regardless of case on Windows
155
+ and macOS. It runs before report generation, so a rejected combination writes
156
+ neither HTML nor the CI file. The CI file is written atomically as a standalone
157
+ file: the contents are replaced, not appended.
158
+
159
+ ## Exit codes
160
+
161
+ CI output does not change the exit codes:
162
+
163
+ | Code | Meaning |
164
+ | --- | --- |
165
+ | 0 | Report or CI text generated; tests may still have failed |
166
+ | 1 | An input could not be parsed or rendered, or a file could not be written |
167
+ | 2 | Invalid usage, including an invalid `--ci` combination |
168
+
169
+ A failed test run still exits 0 so the report and annotations can be published.
170
+ Use the annotations, the Markdown counts or the JSON summary for the test
171
+ outcome. A status mail step must derive its subject from the summary status, not
172
+ from the job status, for the same reason. A `--ci-out` write failure exits 1 after
173
+ the HTML report has been written.
174
+
175
+ ## Messages are not redacted
176
+
177
+ Failure messages, scenario names and URIs are copied from the input verbatim.
178
+ Tolkira does not redact secrets. Do not rely on CI masking for annotations:
179
+ keep credentials out of test data. GitLab does not mask them, a local Markdown
180
+ file never does, and GitHub masking applies to registered secrets in the job
181
+ log, not as a guarantee for every annotation. If a message may contain secret
182
+ material, write the CI output to a restricted artifact instead of the job log.
183
+
184
+ The formatter never fetches a remote asset and never calls a CI provider API.
185
+ The text is derived from the in-memory summary only.
@@ -2,11 +2,16 @@
2
2
 
3
3
  **Status: experimental.** The compatibility subpath accepts the legacy options
4
4
  object and renders with Tolkira. It is a migration aid, not a promise of visual
5
- parity. The shim is ESM-only and asynchronous: use Node 22 or later,
6
- `import { generate }` and `await generate(...)` before reading the report.
5
+ parity. The shim is asynchronous: use Node 22 or later, then either
6
+ `import { generate }` or `require(...)`, and await `generate(...)` before
7
+ reading the report.
7
8
 
8
9
  ## Before and after
9
10
 
11
+ ```shell
12
+ npm i -D tolkira
13
+ ```
14
+
10
15
  ```js
11
16
  // before
12
17
  const reporter = require('cucumber-html-reporter');
@@ -33,8 +38,32 @@ await generate({
33
38
  });
34
39
  ```
35
40
 
36
- Only the import changes, plus `await` and Node 22 or later. `theme` is ignored
37
- with a warning; everything else maps to Tolkira. The callback form still works:
41
+ The option names are unchanged. ESM projects import the shim; CommonJS
42
+ projects require it. The wrapper is async, so await the promise (or use the
43
+ callback) before reading the report. The core package and CLI remain ESM-only.
44
+
45
+ A CommonJS project can require the shim directly:
46
+
47
+ ```js
48
+ // report.cjs
49
+ const { generate } = require('tolkira/compat/cucumber-html-reporter');
50
+
51
+ async function main() {
52
+ await generate({
53
+ jsonFile: 'reports/cucumber.json',
54
+ output: 'reports/report.html',
55
+ });
56
+ }
57
+
58
+ main().catch((error) => {
59
+ console.error(error);
60
+ process.exitCode = 1;
61
+ });
62
+ ```
63
+
64
+ The wrapper exposes only the named `generate` function; there is no default
65
+ export. `theme` is ignored with a warning; everything else maps to Tolkira. The
66
+ callback form still works:
38
67
 
39
68
  ```js
40
69
  generate(options, (error) => {
@@ -42,6 +71,11 @@ generate(options, (error) => {
42
71
  });
43
72
  ```
44
73
 
74
+ Callback caveat: callback-only code can ignore the returned promise. Tolkira
75
+ marks that ignored promise handled so it does not crash the process; awaiting
76
+ the same call still throws. The callback receives an `Error` object, not the
77
+ legacy string. If the callback itself throws, handle that error yourself.
78
+
45
79
  ## Mapping
46
80
 
47
81
  | Old option | Tolkira behaviour |
@@ -61,6 +95,33 @@ generate(options, (error) => {
61
95
  `jsonFile` wins when both `jsonFile` and `jsonDir` are set. If both are missing,
62
96
  generation fails with a usage error.
63
97
 
98
+ With `jsonDir` and no `jsonFile`, the shim also writes the old reporter's merged
99
+ legacy array to `<output>.json` after the HTML. Features merge by `uri`, or
100
+ under `Feature: URI Not Present` when the `uri` is absent; duplicate features
101
+ concatenate their `elements`, every element gets `retried: 0`, and the file uses
102
+ two-space indentation. The write is atomic and creates missing parent
103
+ directories. The `<output>.json` path is reserved for the sidecar: if an earlier
104
+ run left it inside `jsonDir`, it is excluded from the report inputs and replaced
105
+ when every feature URI in it also appears in another input, so a second group
106
+ run does not duplicate features. On the strict path a file at that path is
107
+ refused before the report is written, and stays unchanged, when it is not a
108
+ valid legacy feature array or has a feature URI that no other input represents;
109
+ the error names the sidecar and the input. With `ignoreBadJsonFile: true`, an
110
+ invalid file at that reserved path is skipped and the sidecar replaces it
111
+ instead. Every input is parsed once before the report when the sidecar is
112
+ written: it must be a legacy feature array (an array of objects), and one leading
113
+ UTF-8 BOM is accepted. Non-array or non-object content, including Playwright
114
+ JSON, fails before the HTML with an error naming the file. The old
115
+ reporter's timestamp sort and filename-derived retry counter are not reproduced:
116
+ features keep their input order and `retried` stays `0` for every element, which
117
+ Seed-Test's group analysis does not read.
118
+ `jsonFile` never writes the sidecar, even when `jsonDir` is also set.
119
+
120
+ The shim consumes legacy Cucumber JSON, which has no Messages `meta.ci`
121
+ envelope. CI job and git details therefore do not appear automatically; pass
122
+ them as `metadata` entries (for example `CI`, `Build`, `Commit` and `Branch`).
123
+ The core Messages path reads `meta.ci` and renders it in HTML only.
124
+
64
125
  ## What is not reproduced
65
126
 
66
127
  - **Themes and charts.** The old Bootstrap, hierarchy and Foundation themes are
@@ -70,28 +131,51 @@ generation fails with a usage error.
70
131
  Very large runs can hit the 10 MB size warning; split assets are a later item.
71
132
  - **The old rendered HTML.** The shim consumes the input JSON and options, not
72
133
  the generated HTML. If you only have the old HTML, export the JSON again.
73
- - **The `output + '.json'` side effect.** Tolkira does not write a merged JSON
74
- copy next to the report.
75
- - **Custom CSS/JS, i18n, email reports, CDN mode and browser launch.** These
76
- conflict with the single-file, no-network baseline or are out of scope.
134
+ - **Custom CSS/JS, MCHR's UI language set, CDN mode and browser launch.** MCHR
135
+ 4.6 chooses one UI language at build time (`language` or
136
+ `--language`/`--lang`) from en, es, fr, pt and ar, including RTL Arabic;
137
+ German is not in that set, and MCHR has no in-report switcher. Tolkira
138
+ supports en and de per report and maps the MCHR option to `--locale`. Custom
139
+ CSS/JS, CDN mode and browser launch conflict with the single-file, no-network
140
+ baseline or are out of scope. Tolkira's core `--email <file>` is the nearest
141
+ equivalent to MCHR's separate email report, not a shim option; see
142
+ [`EMAIL.md`](EMAIL.md) and the MCHR section below.
77
143
 
78
144
  ## Seed-Test
79
145
 
80
146
  Seed-Test 2.0's dev/AI integration branch is ESM. Use
81
147
  `import { generate } from 'tolkira/compat/cucumber-html-reporter'` and await
82
- the call before reading the HTML. Seed-Test 1.x is CommonJS and needs an ESM
83
- wrapper or an upgrade; the shim does not provide a CommonJS entry point.
84
- Seed-Test's group mode also expects a sidecar at `output + '.json'`; Tolkira
85
- does not write that file.
148
+ the call before reading the HTML. The old default import becomes the named
149
+ `generate`, and the `reporter.Options` type becomes
150
+ `CucumberHtmlReporterOptions`. Seed-Test 1.x is CommonJS and can require the
151
+ same subpath, then await `generate` or use its callback; its current
152
+ `reporter.generate(options)` call without `await` must change too. The core
153
+ package stays ESM-only.
154
+ Seed-Test's group mode also reads the merged array at `output + '.json'`; a
155
+ `jsonDir` run writes it before `generate` resolves, so await the call and read
156
+ the sidecar afterwards.
86
157
 
87
158
  ## Limits
88
159
 
89
- - The shim accepts legacy Cucumber JSON. It does not turn Messages into legacy
90
- JSON or vice versa.
160
+ - The shim accepts legacy Cucumber JSON and passes `jsonFile` through the core
161
+ loader; its documented contract remains legacy JSON. It does not turn
162
+ Messages into legacy JSON or vice versa. Legacy JSON arrives with expanded
163
+ outline examples, so the shim renders them as flat scenarios. The core CLI
164
+ and API additionally accept Playwright JSON natively.
91
165
  - The usual Tolkira limits apply: 100 MB total input, 50 MB per attachment,
92
166
  100 MB total attachments.
93
167
  - The shim is experimental. The option set is frozen; new features are added to
94
168
  the core CLI/API, not to the shim.
169
+ - Printing is the browser's print/Save as PDF, not a shim option: with
170
+ JavaScript the complete report prints by default (an active filter still
171
+ narrows the Report view), without it the active Report only. Video/audio do not
172
+ print, very wide matrices may need landscape, and browser headers/footers and
173
+ page numbers stay user-controlled. The core CLI/API also adds the opt-in
174
+ `--pdf <file>`/`renderPdf` Playwright Chromium export (A4 portrait,
175
+ `--pdf-timeout`; no Letter/custom paper or landscape); it is not exposed as a
176
+ cucumber-html-reporter option. The shim itself has no email option either; the
177
+ core CLI/API adds the opt-in `--email` summary. See [`PDF.md`](PDF.md) and
178
+ [`EMAIL.md`](EMAIL.md).
95
179
 
96
180
  ## `multiple-cucumber-html-reporter`
97
181
 
@@ -104,5 +188,26 @@ tolkira path/to/jsonDir --out reports/index.html --title "My report"
104
188
  ```
105
189
 
106
190
  Map `jsonDir` to the input directory, `reportPath`/`reportName` to `--out`,
107
- `metadata` to `--meta key=value` and `brandLogo` to `--logo`. Dashboard,
108
- history, trends, i18n and the old folder report are not reproduced.
191
+ `metadata` to `--meta key=value` and `brandLogo` to `--logo`. MCHR 4.6 selects
192
+ its UI language at build time with `language` or `--language`/`--lang` from en,
193
+ es, fr, pt and ar (including RTL Arabic; no German); map it to Tolkira's
194
+ `--locale en|de`. Neither has an in-report switcher; Tolkira's one file carries
195
+ one locale, while MCHR chooses its language at build time and writes its report
196
+ folder. MCHR localizes its default report titles; Tolkira keeps `Tolkira report`
197
+ unless `--title` overrides it. An unknown MCHR language warns and falls back to
198
+ English on the API/config path, while its `--language`/`--lang` CLI throws.
199
+ Tolkira rejects it too: the CLI exits 2 and the API throws.
200
+ MCHR's `--email`/`emailReport` is a boolean and writes
201
+ `reportPath/email-report.html`; Tolkira's `--email` takes a required file path, so
202
+ copying MCHR's bare flag exits 2. MCHR renders a 960 px Tailwind/CDN page with
203
+ per-feature statistic cards, metadata/logo and the five **slowest** scenarios.
204
+ Tolkira writes a 600 px inline, offline mail body with aggregate counts and the
205
+ first five **failed** scenarios in summary order, with no per-feature cards,
206
+ metadata or logo. MCHR and Allure 3's `mail` plugin both only generate the file
207
+ and leave delivery to a mail step: no sending is shared, not a Tolkira
208
+ difference. Attach the report separately.
209
+ The dashboard, scenario-level history and the old folder report are not
210
+ reproduced; history is available only as the opt-in aggregate `--history`
211
+ sidecar described in [`AGENT-INTERFACE.md`](AGENT-INTERFACE.md#run-history). The
212
+ core CLI/API also accepts `--locale en|de` for the report shell and email; the
213
+ legacy `cucumber-html-reporter` shim option set stays frozen.
@@ -1,18 +1,24 @@
1
1
  # Cucumber input compatibility
2
2
 
3
- 2026-11-04. "Tested" means a real producer output is checked in under
4
- `test/fixtures/legacy/` and parsed and rendered by the fixture tests (`test/legacy-version.test.ts`,
5
- `test/legacy-jvm.test.ts`, `test/legacy-ruby.test.ts` and
6
- `test/messages-producers.test.ts`). Each fixture is a two-scenario feature: one
7
- passes, one fails with `Error: boom`.
3
+ 2026-11-19. "Tested" means a real producer output is checked in under
4
+ `test/fixtures/legacy/` or `test/fixtures/behave/` and parsed and rendered by the fixture tests
5
+ (`test/legacy.test.ts`, `test/legacy-version.test.ts`, `test/legacy-jvm.test.ts`,
6
+ `test/legacy-ruby.test.ts`, `test/behave-producers.test.ts` and
7
+ `test/messages-producers.test.ts`). The cucumber-js, cucumber-jvm and
8
+ cucumber-ruby fixtures are two-scenario features: one passes and one fails with
9
+ a producer-specific error (the cucumber-js fixtures use `Error: boom`). The
10
+ Behave producer fixture is a three-scenario real run described below.
11
+
12
+ The dated cross-format limitations table is
13
+ [`docs/LIMITATIONS.md`](LIMITATIONS.md) (last verified 2026-11-19).
8
14
 
9
15
  ## Legacy JSON
10
16
 
11
17
  | Producer | Tested | Notes |
12
18
  | --- | --- | --- |
13
19
  | cucumber-js | 2.3.1, 3.2.1, 4.2.1, 5.1.0, 6.0.7, 7.3.2, 13.3.0 | The shape is stable enough. 2.x has no `type` on elements and uses absolute URIs; duration units vary below 6.0.5. |
14
- | Behave | 1.3.3 real run (`behave-real.json`) | Duration is seconds and is normalised to milliseconds. A real run with attachments and doc strings is still outstanding; the synthetic `behave.json` covers those shapes. |
15
- | cucumber-jvm | 7.22.1 real run | The legacy `json` formatter still ships in 7.x/8.x. The fixture covers `classpath:` URIs, nanosecond durations and a Java assertion. |
20
+ | Behave | 1.3.3 real runs (`test/fixtures/legacy/behave-real.json`, `test/fixtures/behave/behave-real-attachments.json`) | Duration is seconds and is normalised to milliseconds. The rich real run covers a multiline doc string, a data table, an inline PNG, a multiline error message and the `error` status. `test/fixtures/legacy/behave.json` is hand-shaped regression coverage, not proof of producer detection. |
21
+ | cucumber-jvm | 8.1.0 host run (`test/fixtures/legacy/cucumber-jvm-hook-output.json`) | The calculator example emits `before`/`after` hook `output`; 12 real single-line hook results render as `text/plain` hook attachments with the host text. The `\n` array join is unit-tested with a synthetic two-line output; the host capture itself is single-line. The 7.22.1 fixture still covers `classpath:` URIs, nanosecond durations and a Java assertion. |
16
22
  | cucumber-ruby | 12.0.1 real run | The legacy `json` formatter still ships; the fixture covers relative paths, nanosecond durations and a Ruby `RuntimeError`. |
17
23
 
18
24
  ## Messages
@@ -24,6 +30,206 @@ passes, one fails with `Error: boom`.
24
30
  | cucumber-ruby | 12.0.1 real run | `message` formatter; `exception.message` carries the readable error while `message` is the stack, so the report merges them. |
25
31
  | Behave | not applicable | No Messages formatter. |
26
32
 
33
+ ## JUnit XML
34
+
35
+ JUnit is basic mode: suites, test cases, statuses, durations, failures, errors,
36
+ skipped cases and testcase-level `system-out`/`system-err`. Fixtures modelled
37
+ on real output shapes, with provenance in `test/fixtures/junit/`, cover:
38
+
39
+ | Producer | Tested shape | Notes |
40
+ | --- | --- | --- |
41
+ | jest-junit | 16.x | Synthetic but real-shaped; `<testsuites>` root; suite-level `<system-out>` is tolerated, not mapped. |
42
+ | pytest | 8.x `--junitxml` | Synthetic but real-shaped; `<properties>` plus testcase-level `system-out`/`system-err` are mapped. |
43
+ | Gradle | 8.x test task | Synthetic but real-shaped; root `<testsuite>`, `<properties>`, `<failure>`/`<error>`/`<skipped>`; suite-level output is tolerated, not mapped. |
44
+
45
+ See [`JUNIT-XML.md`](JUNIT-XML.md) for the mapping and limits.
46
+
47
+ ## Scenario outlines
48
+
49
+ Messages input carries the Gherkin document, so Tolkira groups executed
50
+ examples under one outline section and numbers them `#1.1`, `#1.2`, `#2.1`.
51
+ Examples blocks and rows whose test case never started are omitted, matching the
52
+ official formatter. A row whose selected test case started with zero raw
53
+ results stays visible as an unexecuted row with its original numbering and no
54
+ outcome. Outline and examples tags, descriptions and keywords come from the
55
+ AST; the substituted values remain visible in the steps. Outlines are ordered
56
+ failed-first like the rest of the report; examples inside an outline keep their
57
+ Gherkin order and numbering.
58
+
59
+ Legacy JSON has no Gherkin document and arrives with examples already expanded.
60
+ Tolkira keeps those rows flat and does not invent an outline. A Messages
61
+ outline without examples also falls back to a flat row; this is a deliberate
62
+ difference from the official formatter, which renders the outline with no
63
+ examples.
64
+
65
+ ## Rules, backgrounds and tags
66
+
67
+ Messages input carries rules and backgrounds in the Gherkin document. Tolkira
68
+ renders a rule as a section with its keyword, name, description and tags.
69
+ Scenarios and outlines stay inside their rule. Feature-level items and rule
70
+ sections are ordered by severity and then by first input position; scenarios do
71
+ not move across rule boundaries. Features without rules keep their previous
72
+ layout.
73
+
74
+ Feature tags render once in the feature header. Rule tags render once on the
75
+ rule section. Scenario tags render on their scenario row or, for an outline, on
76
+ the outline heading; examples tags render once on the examples heading. Example
77
+ rows show no tags. Feature, rule, scenario and examples tags remain in the
78
+ pickle tag set for the tag filter, search and the JSON summary, but they are
79
+ not repeated on every row. A rule section is hidden when a search, status
80
+ or tag filter leaves no visible scenario below it.
81
+
82
+ Feature and rule backgrounds render their keyword, name and description once
83
+ when the name or description is non-empty. Their steps stay in the scenario
84
+ step lists, as Cucumber compiles them into each pickle; Tolkira does not render
85
+ a second background step list. The `backgrounds` Compatibility Kit sample has an
86
+ anonymous background and therefore shows no background heading.
87
+
88
+ Legacy JSON has no reliable rule or background structure. Its normalisation
89
+ synthesizes a Background node for step resolution, marks it synthetic and the
90
+ report model skips it. Legacy feature and scenario tags remain best effort.
91
+ JUnit has neither rules nor backgrounds. The JSON summary stays version 2 and
92
+ continues to expose only the flat scenario list and inherited `scenarios[].tags`;
93
+ rule and background structure is HTML-only.
94
+
95
+ ## Hooks
96
+
97
+ Messages per-scenario hooks render separately from steps. The subtitle counts
98
+ both; named hooks show `Before: name`; passed hooks collapse behind one
99
+ `N hooks` disclosure, while failed and skipped hooks stay inline with their
100
+ error and attachments. Skipped hooks are kept because they explain the skip;
101
+ the official formatter filters some of them. BeforeAll/AfterAll render in a
102
+ Hooks section below the feature list; a named run hook shows its bare name.
103
+ The section is omitted when empty, while the official formatter shows an empty
104
+ section.
105
+
106
+ Legacy JSON has per-scenario before/after hooks but no test-run hooks. Hook
107
+ `output` from a non-null string or string array is kept as a `text/plain`
108
+ attachment named `Before hook output` or `After hook output`, with array lines
109
+ joined by `\n`; null, empty strings/arrays and non-hook output add none. It
110
+ counts as a normal scenario attachment and renders with the hook. This is an
111
+ improvement over cucumber-html-reporter 7.2.0, which ignores hook output; it is
112
+ not a parity claim. The real 8.1.0 host fixture uses single-line outputs; the
113
+ `\n` array join is unit-tested with a synthetic two-line output. JUnit has no
114
+ hooks. The JSON summary stays version 2: hooks stay in `scenarios[].steps` with
115
+ `keyword: "Hook"`, plus an optional `name`. Summary step counts include hooks;
116
+ the HTML counts steps and hooks separately.
117
+
118
+ ## CI metadata and attachments
119
+
120
+ Messages input can carry `meta.ci`. Tolkira reads the CI name and URL, the git
121
+ remote, revision, branch and tag, and shows them in the report header and
122
+ Details. Missing fields are omitted; only http(s) values become links, and a
123
+ missing name falls back to the URL as link text. A safe http(s), scp-like or
124
+ `ssh://` GitHub remote plus a revision links the 7-character commit to the full
125
+ https revision and keeps the full value in the link title; other remotes and
126
+ schemes show the revision as plain text. In a mixed run the first non-empty CI block
127
+ wins, and meta synthesized by the legacy adapter cannot erase it. CI metadata is
128
+ Messages-only; legacy Cucumber JSON has no CI envelope, so pass CI values
129
+ through the `metadata` option instead. The JSON summary never includes CI
130
+ fields and stays schema version 2.
131
+
132
+ Attachment names use the Cucumber `fileName` or the legacy JSON `name` when
133
+ present. Unnamed attachments
134
+ fall back to a deterministic `attachment.<ext>` name in Messages and legacy
135
+ JSON; known parameterised media types strip parameters and keep their
136
+ extension (`text/uri-list` becomes `.txt` and `text/x.cucumber.log+plain`
137
+ becomes `.log`), while unknown or forged types normalise to
138
+ `application/octet-stream` and `attachment.bin`. Names are text, never
139
+ filesystem paths. `application/json` attachments are pretty-printed without
140
+ re-serialising values, so big numbers and duplicate keys survive; malformed
141
+ JSON and bodies above 262,144 UTF-8 bytes render as escaped raw text.
142
+ `text/uri-list` entries become links only for http(s); other schemes and
143
+ protocol-relative values stay escaped text. `text/x.cucumber.log+plain` logs
144
+ render safe theme-aware SGR, 256-colour and truecolour spans plus a copy
145
+ action; non-SGR control sequences are stripped, and logs above 262,144 UTF-8
146
+ bytes render as escaped text without colour parsing. `text/html` attachments
147
+ are escaped text, and SVG attachments are only used as an `<img>` data URI;
148
+ neither is injected as markup. Attachment counts and the offline single-file
149
+ contract are unchanged.
150
+
151
+ ## Step diagnostics
152
+
153
+ Messages input carries `suggestion` and `stepDefinition` envelopes. For an
154
+ undefined step Tolkira shows "No step definition found." and renders every
155
+ suggestion snippet as escaped code with its language label. For an ambiguous
156
+ step it shows "Multiple matching step definitions found:" and each matching
157
+ pattern with its source reference in the relative `uri:line` form.
158
+ Multiple snippets and multiple definitions are supported. A passed step can
159
+ carry definition ids but never shows a diagnostic.
160
+
161
+ The lead is status-based: it appears even when the producer sent no suggestion
162
+ or definition envelope. Snippets and patterns are Messages-only. Legacy JSON has
163
+ no such envelopes, so an undefined or ambiguous legacy step shows the lead and
164
+ nothing else; Tolkira never invents a definition. JUnit has no Gherkin step
165
+ statuses and renders no diagnostics.
166
+
167
+ The `undefined`, `undefined-multiple`, `ambiguous`, `all-statuses` and `empty`
168
+ Compatibility Kit samples are checked in as fixtures. `ambiguous` already has
169
+ two matching definitions; there is no separate `ambiguous-multiple` sample.
170
+
171
+ ## Markdown descriptions
172
+
173
+ Feature, rule, background, scenario, outline and examples descriptions render a
174
+ safe GFM subset: headings, emphasis, inline and fenced code, http(s)/mailto
175
+ links, ordered and unordered lists, tables and blockquotes. Raw HTML is escaped
176
+ and never executed. Links carry `rel="noopener noreferrer nofollow"`; relative,
177
+ `javascript:`, `data:` and `file:` targets stay plain text. Images, including
178
+ reference-style and linked images, render as alt text only, so the report makes
179
+ no network request and no image becomes an anchor. Description headings never
180
+ emit `h1`, `h2`, `h5` or `h6`; tables sit in the focusable scroll region.
181
+
182
+ The same renderer covers Messages, legacy JSON and JUnit. JUnit has no standard
183
+ description field; Tolkira reads the non-standard `description` attributes on
184
+ `<testsuite>` and `<testcase>` when present and otherwise keeps plain names.
185
+ Descriptions render only in the HTML report and do not change the JSON summary.
186
+ The Markdown fixtures are local hand-written inputs, not producer captures:
187
+ `markdown-descriptions.ndjson` is authoritative, `markdown-descriptions.feature`
188
+ is an abridged illustration of the Gherkin source,
189
+ `legacy/markdown-description.json` covers the legacy adapter and
190
+ `junit/markdown-description.xml` covers the JUnit description attributes.
191
+
192
+ ## Search and URL state
193
+
194
+ The search box accepts the official Cucumber tag-expression grammar when the
195
+ whole query is tag-shaped: `@a and not @b`, `not @a`, parentheses and escaped
196
+ tag characters. Text mixed with tags and invalid expressions fall back to the
197
+ existing substring search, with no error. Status filters and the tag dropdown
198
+ combine with a valid expression using AND, and the final visible set still opens
199
+ at most 20 newly expanded rows. Clearing the tag dropdown always applies
200
+ immediately, even while a tag expression is active.
201
+
202
+ The report stores `query`, every repeated lowercase `hide` value and `tag` in
203
+ the URL, restores them on load and synchronises the search input, status
204
+ buttons and tag select. `?hide=` also accepts the `unexecuted` pseudo-status,
205
+ restored only with the preset chip state; a pruned value is removed from the
206
+ URL. Updates use `replaceState` and preserve the hash and unrelated parameters.
207
+ Empty, unknown and uppercase parameters are ignored. When `replaceState` is
208
+ unavailable or throws (for example on `file://`), filtering keeps working. URL
209
+ state and tag expressions are progressive enhancements; the static report stays
210
+ readable without JavaScript.
211
+
212
+ ## Auto-expansion
213
+
214
+ The report opens every feature with an executed scenario whose status is not
215
+ `passed`, including undefined, ambiguous, pending, skipped and unknown. Passed
216
+ scenarios stay collapsed unless they carry an image. A feature whose scenarios
217
+ are all unexecuted opens; a mixed feature opens only for an executed non-passed
218
+ or image scenario, not merely because it contains an unexecuted row. The
219
+ unexecuted rows themselves stay collapsed and show no outcome. This applies to
220
+ Messages, legacy JSON and JUnit input because it is a status rule, not an
221
+ input-format rule. Text search and a valid tag expression still open at most 20
222
+ rows, counting only rows they newly open.
223
+
224
+ The JSON summary stays version 2. Undefined summary steps may carry
225
+ `suggestions`; ambiguous summary steps may carry `definitions`. The fields are
226
+ optional and additive, so `schemaVersion` does not change. Scenario counts
227
+ cover executed scenarios only; an unexecuted scenario keeps `status:
228
+ "unknown"` with `executed: false` and `counts.scenarios.total` can be smaller
229
+ than `scenarios.length`. A legacy scenario with no steps and no hooks is
230
+ unexecuted, so its scenario total drops by one compared with version 1 while
231
+ legacy steps without a result remain executed.
232
+
27
233
  ## Breaking changes that matter
28
234
 
29
235
  - **3.2.1 (2017):** JSON mime-type revert; duration handling was still in flux.
@@ -36,9 +242,13 @@ passes, one fails with `Error: boom`.
36
242
  - **8.0–13.x:** Messages additions are additive for the fields Tolkira reads
37
243
  (`workerId`, `Hook.type`, `testRunStartedId`, test-run hooks, external
38
244
  attachments).
39
- - **Behave 1.3.3:** JSON duration is in seconds, and features/elements carry
40
- `status` and `location` strings. The adapter detects that dialect and scales
41
- durations; Behave has no Messages formatter.
245
+ - **Behave 1.3.3:** JSON duration is in seconds; multiline `text` and
246
+ `error_message` are arrays of lines; step results use `error` for
247
+ non-assertion exceptions and `hook_error` for failing step hooks.
248
+ `cleanup_error` is recorded on the scenario/feature during context cleanup,
249
+ not on a step; the adapter accepts it defensively if a producer writes it as
250
+ a step result. The adapter detects that dialect, scales durations, joins line
251
+ arrays and maps those statuses to failed. Behave has no Messages formatter.
42
252
  - **cucumber-jvm 7.22.1:** legacy JSON uses `classpath:` URIs, nanosecond
43
253
  durations and Java stack traces. The shape matches the Cucumber legacy
44
254
  dialect and needs no adapter change.
@@ -63,10 +273,15 @@ passes, one fails with `Error: boom`.
63
273
  ## Reproduce
64
274
 
65
275
  ```shell
66
- npx vitest run test/legacy-version.test.ts
276
+ npx vitest run test/legacy-version.test.ts test/legacy.test.ts test/behave-producers.test.ts
67
277
  ```
68
278
 
69
- The fixtures are generated from a minimal feature with:
279
+ The Behave rich run is recorded in `test/fixtures/behave/`; its
280
+ `PROVENANCE.md` has the container command that regenerates the JSON without
281
+ touching the host Python installation.
282
+
283
+ The cucumber-js, cucumber-jvm and cucumber-ruby fixtures are generated from a
284
+ minimal feature with:
70
285
 
71
286
  ```gherkin
72
287
  Feature: Old cucumber compatibility
@@ -78,9 +293,11 @@ Feature: Old cucumber compatibility
78
293
  Given a failing step
79
294
  ```
80
295
 
81
- The step definitions are one empty function and one that throws `Error: boom`.
82
- The versions were installed in isolated temp projects; the checked-in JSON is
83
- the producer output, not a hand-written approximation.
296
+ The step definitions are one empty function and one that fails with a
297
+ producer-specific error: `Error: boom` for cucumber-js,
298
+ `java.lang.AssertionError: boom` for cucumber-jvm and `boom (RuntimeError)` for
299
+ cucumber-ruby. The versions were installed in isolated temp projects; the
300
+ checked-in JSON is the producer output, not a hand-written approximation.
84
301
 
85
302
  ### Fixture toolchains
86
303