@surea11y/core 1.7.0 → 1.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +103 -1
- package/README.md +157 -54
- package/docs/ACT_RULE_MAPPING.md +8 -7
- package/docs/API_STABILITY.md +18 -5
- package/docs/BINDING_AUTHORS_GUIDE.md +2 -2
- package/docs/CI_INTEGRATIONS.md +43 -0
- package/docs/DESIGN_CHALLENGES.md +97 -3
- package/docs/EARL.md +2 -2
- package/docs/ENGINE_OPTIONS.md +81 -3
- package/docs/I18N.md +62 -20
- package/docs/JUNIT.md +73 -0
- package/docs/LIMITATIONS.md +1 -0
- package/docs/OUTPUT_SCHEMA.md +19 -6
- package/docs/REPORT.md +7 -2
- package/docs/RULE_AUTHORING.md +73 -6
- package/docs/RULE_CATALOG.md +139 -116
- package/docs/RULE_EXAMPLES.md +2189 -0
- package/docs/RULE_HELPERS.md +62 -5
- package/docs/RULE_TAXONOMY.md +2 -2
- package/docs/SARIF.md +2 -1
- package/docs/WCAG_CONFORMANCE.md +56 -3
- package/package.json +34 -11
- package/profiles/index.js +14 -0
- package/src/checks/automatic/area-alt-present.js +87 -31
- package/src/checks/automatic/aria-braille-equivalent.js +25 -7
- package/src/checks/automatic/aria-hidden-focus.js +74 -18
- package/src/checks/automatic/aria-prohibited-attr.js +17 -4
- package/src/checks/automatic/aria-required-attr.js +29 -0
- package/src/checks/automatic/aria-role-name-present.js +19 -2
- package/src/checks/automatic/aria-valid-attr-value.js +28 -16
- package/src/checks/automatic/autocomplete-valid.js +26 -11
- package/src/checks/automatic/avoid-inline-spacing.js +105 -40
- package/src/checks/automatic/button-name-present.js +2 -1
- package/src/checks/automatic/canvas-text-alternative-present.js +105 -14
- package/src/checks/automatic/combobox-name-present.js +34 -51
- package/src/checks/automatic/contrast-computable.js +35 -4
- package/src/checks/automatic/contrast-enhanced.js +4 -4
- package/src/checks/automatic/contrast-minimum.js +45 -11
- package/src/checks/automatic/css-orientation-lock.js +152 -30
- package/src/checks/automatic/definition-list-children-valid.js +67 -23
- package/src/checks/automatic/deprecated-elements-not-used.js +43 -38
- package/src/checks/automatic/dialog-name-present.js +28 -9
- package/src/checks/automatic/duplicate-id.js +6 -2
- package/src/checks/automatic/identical-iframes-same-purpose.js +4 -4
- package/src/checks/automatic/iframe-focusable-content.js +7 -4
- package/src/checks/automatic/iframe-title-unique.js +36 -81
- package/src/checks/automatic/input-image-alt-present.js +32 -20
- package/src/checks/automatic/label-in-name.js +40 -13
- package/src/checks/automatic/language-page-present.js +12 -6
- package/src/checks/automatic/link-in-text-block.js +272 -60
- package/src/checks/automatic/link-name-present.js +13 -5
- package/src/checks/automatic/list-children-valid.js +18 -1
- package/src/checks/automatic/listbox-name-present.js +19 -49
- package/src/checks/automatic/listitem-parent-valid.js +4 -3
- package/src/checks/automatic/meta-refresh-no-exceptions.js +3 -3
- package/src/checks/automatic/page-title-present.js +16 -4
- package/src/checks/automatic/progressbar-name-present.js +11 -1
- package/src/checks/automatic/role-img-text-alternative-present.js +9 -5
- package/src/checks/automatic/searchbox-name-present.js +32 -49
- package/src/checks/automatic/server-side-image-map-absent.js +48 -28
- package/src/checks/automatic/slider-name-present.js +38 -52
- package/src/checks/automatic/spinbutton-name-present.js +32 -49
- package/src/checks/automatic/target-size-minimum.js +0 -11
- package/src/checks/automatic/td-has-header.js +41 -5
- package/src/checks/automatic/text-spacing-content-loss.js +548 -0
- package/src/checks/automatic/textbox-name-present.js +32 -49
- package/src/checks/automatic/valid-lang.js +15 -10
- package/src/checks/manual/area-alt-quality-manual.js +113 -31
- package/src/checks/manual/canvas-text-alternative-quality-manual.js +0 -2
- package/src/checks/manual/css-focus-indicator-suppressed-manual.js +23 -10
- package/src/checks/manual/css-hidden-focus.js +215 -7
- package/src/checks/manual/form-control-label-quality-manual.js +109 -5
- package/src/checks/manual/heading-order-manual.js +9 -1
- package/src/checks/manual/heading-quality-manual.js +143 -9
- package/src/checks/manual/img-alt-decorative-manual.js +6 -3
- package/src/checks/manual/input-image-alt-quality-manual.js +81 -13
- package/src/checks/manual/link-name-quality-manual.js +130 -4
- package/src/checks/manual/media-transcript-present-manual.js +65 -8
- package/src/checks/manual/mouse-only-event-handlers-manual.js +66 -5
- package/src/checks/manual/no-autoplay-audio-manual.js +93 -6
- package/src/checks/manual/p-as-heading-manual.js +89 -44
- package/src/checks/manual/page-title-patterns-manual.js +77 -8
- package/src/checks/manual/skip-link-manual.js +42 -14
- package/src/checks/manual/table-fake-caption-manual.js +32 -1
- package/src/checks/manual/video-caption-manual.js +47 -24
- package/src/checks/manual-review.js +0 -4
- package/src/core.js +14285 -2219
- package/src/coverage/en301549-map.js +187 -0
- package/src/coverage/standards.js +279 -0
- package/src/coverage/wcag-facets.js +1119 -0
- package/src/coverage/wcag-version-map.js +101 -0
- package/src/en301549.js +33 -0
- package/src/junit.js +321 -0
- package/src/profile-kit.js +163 -0
- package/src/report.js +343 -74
- package/src/sarif.js +34 -3
- package/src/wcag.js +105 -0
- package/surea11y.browser.js +5 -4
- package/surea11y.i18n.de.js +1 -1
- package/surea11y.i18n.es.js +1 -1
- package/surea11y.i18n.fr.js +1 -1
- package/surea11y.i18n.ja.js +3 -0
- package/src/checks/manual/area-alt-decorative-manual.js +0 -255
package/docs/I18N.md
CHANGED
|
@@ -4,16 +4,23 @@ Every rule's title/description, and every occurrence's summary/hint, is localize
|
|
|
4
4
|
|
|
5
5
|
## Current locale coverage
|
|
6
6
|
|
|
7
|
-
| Locale |
|
|
7
|
+
| Locale | Files | Keys | Values identical to English |
|
|
8
8
|
|---|---|---|---|
|
|
9
|
-
| `en` (English) | `src/i18n/en.json` |
|
|
10
|
-
| `fr` (French) | `src/i18n/fr.json` |
|
|
11
|
-
| `de` (German) | `src/i18n/de.json` |
|
|
12
|
-
| `es` (Spanish) | `src/i18n/es.json` |
|
|
9
|
+
| `en` (English) | `src/i18n/en.json` | 841 | — (the canonical/fallback set) |
|
|
10
|
+
| `fr` (French) | `src/i18n/fr.json` | 841 | 3 — *Orientation*, *Interruptions*, *Occurrences*, which are the same words in French |
|
|
11
|
+
| `de` (German) | `src/i18n/de.json` | 841 | 0 |
|
|
12
|
+
| `es` (Spanish) | `src/i18n/es.json` | 841 | 2 — *Selector*, the same word in Spanish |
|
|
13
|
+
| `ja` (Japanese) | `src/i18n/ja.json` | 841 | 0 |
|
|
14
|
+
|
|
15
|
+
The last column is what `npm run i18n:report` measures: values that match the English text character for character. That catches a key nobody has translated yet, but it also counts a word that is simply the same in both languages, and it cannot see an English word left inside an otherwise translated sentence. Those are found by reading the text; none remain in the locales above.
|
|
13
16
|
|
|
14
17
|
Locale files are plain JSON: a flat map of key to translated string, in the same key order as `en.json`. Nothing else lives in them, so contributing a language means editing text and never touching code.
|
|
15
18
|
|
|
16
|
-
|
|
19
|
+
A locale's text may be split between several files. `src/i18n/<locale>.json` holds the engine's own messages; a profile's `profiles/<key>/i18n/<locale>.json` holds those of its rules (see [`profiles/README.md`](../profiles/README.md)). Each file is synced against the `en.json` next to it, and the build merges them into one dictionary per locale. A key lives in exactly one of them: the build fails if two files define it.
|
|
20
|
+
|
|
21
|
+
A profile chooses its languages: the locale files in its `i18n/` folder are the list. It always has `en.json`; a locale core has and the profile does not shows that profile's messages in English, as the engine does for any key a locale lacks. Because the profile chose it, `engine.locale.reason` stays `ok`: the build records the keys each locale leaves out that way, and only a key missing otherwise makes it `partial-dictionary`. Core's own messages are never left out by choice, so a locale only a profile has is reported as `partial-dictionary`.
|
|
22
|
+
|
|
23
|
+
Every locale file carries every key the `en.json` next to it has. Keeping it that way is the job of `npm run i18n:sync`: run it after any change to `en.json` and it rewrites every non-English locale file to match — adding keys that are new, dropping keys `en.json` no longer has, and leaving existing translations alone. A key it adds is seeded with the English text, which counts as untranslated until someone replaces it.
|
|
17
24
|
|
|
18
25
|
Forget to run it and the build fails: `tests/i18n-sync.test.js` and `tests/i18n/i18n-locale-completeness.test.js` reject a missing key, an orphaned key, or any file that `i18n:sync` would rewrite. `npm run i18n:check` reports the same thing without touching the files, and `npm run i18n:report` prints per-locale coverage.
|
|
19
26
|
|
|
@@ -55,7 +62,7 @@ Graceful fallback has one drawback: ask for a language the build doesn't carry a
|
|
|
55
62
|
"engine": {
|
|
56
63
|
"tag": "a11ycore",
|
|
57
64
|
"schemaVersion": "1.0.0",
|
|
58
|
-
"locale": { "requested": "
|
|
65
|
+
"locale": { "requested": "ko", "resolved": "en", "reason": "unknown-locale" }
|
|
59
66
|
}
|
|
60
67
|
```
|
|
61
68
|
|
|
@@ -66,8 +73,8 @@ Graceful fallback has one drawback: ask for a language the build doesn't carry a
|
|
|
66
73
|
| `ok` | You got exactly what you asked for, and that dictionary carries every key. |
|
|
67
74
|
| `primary-subtag` | Your code carried a subtag with no dictionary of its own, so its base language was used — you asked for `de-DE` and got `de`. Normal and expected; nothing to fix. A difference in case alone is not this: `DE` reports `ok`. |
|
|
68
75
|
| `dictionary-not-loaded` | The project ships that language, but this copy of the engine doesn't carry it and none was supplied. In practice: the standalone browser bundle without its locale side file. |
|
|
69
|
-
| `unknown-locale` | The project has no such translation at all, so English was used. `
|
|
70
|
-
| `partial-dictionary` | The dictionary was used but is missing some keys, so those individual strings fell back to English. |
|
|
76
|
+
| `unknown-locale` | The project has no such translation at all, so English was used. `ko` and `pt-BR` both land here today. |
|
|
77
|
+
| `partial-dictionary` | The dictionary was used but is missing some keys, so those individual strings fell back to English. A profile's messages in a language the profile does not offer also show in English, but do not count here. |
|
|
71
78
|
|
|
72
79
|
Treat the list as open — a later release can add a value, so match on the ones you care about and let the rest fall through a default.
|
|
73
80
|
|
|
@@ -83,11 +90,14 @@ Which languages are available depends on how you load the engine.
|
|
|
83
90
|
| A binding (Playwright, Cypress, …) | Every locale, built in. Nothing to configure. |
|
|
84
91
|
| `surea11y.browser.js` in a `<script>` tag | English. Load `surea11y.i18n.<locale>.js` after it for anything else. |
|
|
85
92
|
|
|
86
|
-
The bundle is split because it travels over the network to every page that uses it, and no page needs
|
|
93
|
+
The bundle is split because it travels over the network to every page that uses it, and no page needs every language. Keeping English inline and the rest optional took about 280 KB off the download and stops it growing as languages are added. Nothing else changes: the Node package and the bindings are unaffected.
|
|
87
94
|
|
|
88
95
|
```html
|
|
89
96
|
<script src="surea11y.browser.js"></script>
|
|
90
|
-
<script src="surea11y.i18n.
|
|
97
|
+
<script src="surea11y.i18n.ja.js"></script>
|
|
98
|
+
<script>
|
|
99
|
+
const result = a11ycore.runa11yCoreInPage(location.href, null, { locale: 'ja' }, null);
|
|
100
|
+
</script>
|
|
91
101
|
```
|
|
92
102
|
|
|
93
103
|
Ask for a language whose file you didn't load and you get English, with `engine.locale.reason` set to `dictionary-not-loaded` — different from `unknown-locale`, which means the project has no such translation at all.
|
|
@@ -105,11 +115,35 @@ runDomRulesInPage(url, null, {
|
|
|
105
115
|
|
|
106
116
|
Keys you don't supply fall back normally, so a partial override is fine.
|
|
107
117
|
|
|
118
|
+
## Notes on individual locales
|
|
119
|
+
|
|
120
|
+
**Japanese (`ja`).** Terms that WCAG 2.2 defines follow the Japanese translation published by WAIC (Web Accessibility Infrastructure Committee): 達成基準 for success criterion, アクセシブルな名前 for accessible name, テキストによる代替 for text alternative, 支援技術 for assistive technology, and the WAIC names for the criteria themselves, such as ラベルを含む名前 (name), コントラスト (最低限) and ターゲットのサイズ (最低限). Messages use the です/ます style, and hints ask for a fix with 〜してください. A few conventions worth keeping when you edit it:
|
|
121
|
+
|
|
122
|
+
- Where an English title says *should*, the Japanese one ends in 〜が望ましい (or says 推奨), and *(manual review)* becomes (手動確認). A title that says *must* is phrased as a requirement. Keep that split: it is how a reader tells advisory findings from confirmed failures.
|
|
123
|
+
- A `cantTell` summary says what a person has to decide, usually ending in 人による確認が必要です or in a sentence stating what could not be determined. It never reads as a failure.
|
|
124
|
+
- A `pass` message describes the check that ran (計算できたすべてのテキストが…基準値を満たしています), not the page. Nothing in the dictionary claims conformance.
|
|
125
|
+
- Quoted page content uses 「…」, so `{{name}}` becomes 「{{name}}」. Code keeps its ASCII quotes: `alt=""`, `role="img"`.
|
|
126
|
+
- Rule descriptions quote the phrases a rule matches in both languages (「こちら」、"click here"), since the rules recognize both.
|
|
127
|
+
|
|
128
|
+
Some parameter values are CSS or attribute vocabulary and are interpolated as-is in every locale: `{{fontWeightLabel}}` (`bold`, `700`), `{{labelSource}}` and `{{nameMechanism}}` (`aria-label`, `title`).
|
|
129
|
+
|
|
130
|
+
## Language-specific matching
|
|
131
|
+
|
|
132
|
+
Five rules judge text by known phrases, so they carry word lists as well as messages: `link-name-quality` ("click here", 「こちら」), `heading-quality` ("Untitled", 「見出し」), `form-control-label-quality` ("Label", 「入力欄」), `page-title-patterns` ("Home", 「トップページ」) and `media-alternative-transcript-evidence` ("transcript", 「文字起こし」). Each list covers `en`, `de`, `es`, `fr` and `ja`.
|
|
133
|
+
|
|
134
|
+
English is always checked. The list for the element's own language, taken from the nearest `lang` attribute (the `<html>` element's for a page title), is added on top. Checking every list everywhere would flag words that are generic in one language and a real name in another, such as "Suite" or "Plus" on an English page. Transcript words are the exception: they are distinctive in every language and a page can link a transcript in another language, so all of them are always checked. Text is NFKC-normalized first, so full-width forms such as 「見出し2」 match.
|
|
135
|
+
|
|
136
|
+
`page-title-patterns` also counts each Chinese, Japanese or Korean character as two when deciding whether a title is too short, since 「お問い合わせ」 is a complete title in six characters.
|
|
137
|
+
|
|
138
|
+
Adding a language means adding its words to these lists as well as translating its dictionary.
|
|
139
|
+
|
|
108
140
|
## Where keys are used
|
|
109
141
|
|
|
110
142
|
Two independent key namespaces, both resolved the same way:
|
|
111
143
|
|
|
112
144
|
- **Rule-level**: `meta.i18n.titleKey` / `meta.i18n.descriptionKey` — resolve a rule's `title`/`description` on every `checksResults[]` entry.
|
|
145
|
+
- **HTML report**: the `report_*` keys label the page `@surea11y/core/report` renders — headings, table columns, outcome and severity names, the headline, the pager. The report reads them in the locale the scan resolved to, so translating a dictionary translates the report too.
|
|
146
|
+
- **Composite-level**: `meta.titleKey` / `meta.descriptionKey` in `src/catalogs/composites.wcag.js` — resolve each `rulesResults[]` rollup's `title`/`description`. The catalog's own `title`/`description` must match the English value, which `tests/i18n/i18n-composite-titles.test.js` checks. Titles use each language's published name for the success criterion.
|
|
113
147
|
- **Occurrence-level**: `i18n.summaryKey` / `i18n.hintKey`, with `i18n.params` for `{{placeholder}}` interpolation — resolve an occurrence's `summary`/`hint`. See [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md#an-occurrence-occurrencesi).
|
|
114
148
|
|
|
115
149
|
Both are included in the result alongside the already-resolved text, so you can re-render in a different locale from a saved result without re-scanning — see the `i18n` field's presence in `OUTPUT_SCHEMA.md`.
|
|
@@ -132,10 +166,18 @@ npm install
|
|
|
132
166
|
npm run i18n:new pt-BR
|
|
133
167
|
```
|
|
134
168
|
|
|
135
|
-
That writes `src/i18n/pt-BR.json
|
|
169
|
+
That writes `src/i18n/pt-BR.json`, containing every key `en.json` has, in the same order, seeded with the English text as a placeholder. **Use the shortest code that identifies the language** — `pt`, `nl`, `pl`. A file named `pt.json` serves everyone who asks for `pt`, `pt-BR` or `pt-PT`, because a code with a subtag falls back to its base language. Name it `pt-BR.json` and only people who ask for exactly that get it; `pt` speakers elsewhere fall through to English.
|
|
136
170
|
|
|
137
171
|
Add a regional file only when the wording genuinely has to differ, and add it alongside the base language rather than instead of it.
|
|
138
172
|
|
|
173
|
+
A profile's messages are a file of their own, added with `--profile`:
|
|
174
|
+
|
|
175
|
+
```sh
|
|
176
|
+
npm run i18n:new -- pt-BR --profile <key>
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
writes `profiles/<key>/i18n/pt-BR.json` the same way. Without it, that profile's rules show in English in that locale.
|
|
180
|
+
|
|
139
181
|
The command refuses to overwrite a file that already exists. To pick up work on an existing locale, edit it directly.
|
|
140
182
|
|
|
141
183
|
### 3. Translate the values
|
|
@@ -161,7 +203,7 @@ Four things to leave alone:
|
|
|
161
203
|
|
|
162
204
|
Write for someone fixing the page, not for a specialist: say what is wrong and what to do about it. Where your language has established accessibility vocabulary (a national WCAG translation, a government standard), follow it rather than inventing terms.
|
|
163
205
|
|
|
164
|
-
If a string is genuinely identical in your language, leave it.
|
|
206
|
+
If a string is genuinely identical in your language, leave it. The report counts it as identical to English, which is correct and needs no action.
|
|
165
207
|
|
|
166
208
|
### 4. Check your progress
|
|
167
209
|
|
|
@@ -169,7 +211,7 @@ If a string is genuinely identical in your language, leave it. It is counted as
|
|
|
169
211
|
npm run i18n:report
|
|
170
212
|
```
|
|
171
213
|
|
|
172
|
-
Prints, per locale, how many values differ from English, plus any missing or orphaned keys.
|
|
214
|
+
Prints, per locale, how many values differ from English, plus any missing or orphaned keys. A value that still matches the English text is counted as not yet translated. While you work, that is your progress signal; once you are done, whatever it still counts should be words your language shares with English.
|
|
173
215
|
|
|
174
216
|
You do not need every string on day one. Per-string fallback means an unfinished locale renders in your language where you have translated it and in English everywhere else, which is exactly how `fr`, `de` and `es` started. Ship what you have.
|
|
175
217
|
|
|
@@ -182,25 +224,25 @@ npm test
|
|
|
182
224
|
|
|
183
225
|
`npm run build` also emits `surea11y.i18n.<locale>.js` for the standalone browser bundle — generated, so there is nothing for you to write.
|
|
184
226
|
|
|
185
|
-
Then open a pull request touching `src/i18n/<locale>.json
|
|
227
|
+
Then open a pull request touching `src/i18n/<locale>.json` (and a profile's `profiles/<key>/i18n/<locale>.json` if you translated its messages too), plus one row in the coverage table at the top of this file. Tell us which language and, if you use one, which national terminology standard you followed — that helps whoever reviews it later.
|
|
186
228
|
|
|
187
229
|
Once a locale is in the repository it is maintained with the rest of the engine: when a new rule adds strings, `npm run i18n:sync` seeds them in your file in English and `npm run i18n:report` shows them as outstanding.
|
|
188
230
|
|
|
189
231
|
## Maintaining the locales
|
|
190
232
|
|
|
191
|
-
When you add, rename or remove a key in `src/i18n/en.json`:
|
|
233
|
+
When you add, rename or remove a key in `src/i18n/en.json` or a profile's `i18n/en.json`:
|
|
192
234
|
|
|
193
235
|
```sh
|
|
194
236
|
npm run i18n:sync
|
|
195
237
|
```
|
|
196
238
|
|
|
197
|
-
Every other locale is rewritten to match — new keys seeded in English, removed keys dropped, existing translations untouched. The command is idempotent and prints what it changed per
|
|
239
|
+
Every other locale file in the same folder is rewritten to match its `en.json` — new keys seeded in English, removed keys dropped, existing translations untouched. A locale a profile has no file for is left out: it never creates one. The command is idempotent and prints what it changed per file.
|
|
198
240
|
|
|
199
241
|
| Command | Does |
|
|
200
242
|
|---|---|
|
|
201
|
-
| `npm run i18n:new <locale>` | Create
|
|
202
|
-
| `npm run i18n:sync` | Bring every locale file back in line with `en.json`. Add `-- <locale>` to restrict it to one. |
|
|
243
|
+
| `npm run i18n:new <locale>` | Create core's file for the locale from `src/i18n/en.json`; with `-- <locale> --profile <key>`, a profile's from its own. Refuses to overwrite. |
|
|
244
|
+
| `npm run i18n:sync` | Bring every locale file back in line with its `en.json`. Add `-- <locale>` to restrict it to one. |
|
|
203
245
|
| `npm run i18n:check` | Same comparison, writes nothing, exits non-zero on drift. |
|
|
204
|
-
| `npm run i18n:report` |
|
|
246
|
+
| `npm run i18n:report` | Translation coverage per folder and locale: core's, then each profile's for the locales it has. |
|
|
205
247
|
|
|
206
248
|
`npm test` fails if a locale file has drifted, so an added key cannot reach `main` without every locale carrying it.
|
package/docs/JUNIT.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# JUnit report
|
|
2
|
+
|
|
3
|
+
`@surea11y/core/junit` renders a scan result as JUnit XML, the test report format CI dashboards read natively: GitLab's merge request test widget, Azure DevOps' Tests tab, Jenkins and CircleCI.
|
|
4
|
+
|
|
5
|
+
```js
|
|
6
|
+
const { renderJunitReport } = require('@surea11y/core/junit');
|
|
7
|
+
const { runDomRulesInPage } = require('@surea11y/core');
|
|
8
|
+
|
|
9
|
+
const result = runDomRulesInPage(url, null, { profile: 'wcag22-aa' }, null);
|
|
10
|
+
require('fs').writeFileSync('surea11y.junit.xml', renderJunitReport(result));
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`renderJunitReport(result, options)` is a pure function: it returns the XML as a string and never touches the filesystem. It works just as well on a result saved as JSON earlier, such as the output of `surea11y scan <target> --json` from [`@surea11y/cli`](https://github.com/SureA11y/cli#readme) — see [`CI_INTEGRATIONS.md`](./CI_INTEGRATIONS.md#junit-test-reports).
|
|
14
|
+
|
|
15
|
+
## Shape
|
|
16
|
+
|
|
17
|
+
One `<testsuite>` per WCAG Success Criterion, one `<testcase>` per rule mapped to it:
|
|
18
|
+
|
|
19
|
+
```xml
|
|
20
|
+
<testsuites name="surea11y" tests="14" failures="2" errors="0" skipped="4" time="0">
|
|
21
|
+
<testsuite name="WCAG 1.1.1 Non-text content: text alternatives" tests="1" failures="1" errors="0" skipped="0" time="0">
|
|
22
|
+
<properties>
|
|
23
|
+
<property name="wcagCriterion" value="1.1.1"/>
|
|
24
|
+
<property name="wcagLevel" value="A"/>
|
|
25
|
+
<property name="en301549" value="9.1.1.1"/>
|
|
26
|
+
<property name="criterionOutcome" value="fail"/>
|
|
27
|
+
<property name="engine" value="a11ycore"/>
|
|
28
|
+
<property name="schemaVersion" value="1.0.0"/>
|
|
29
|
+
<property name="wcagVersion" value="2.2"/>
|
|
30
|
+
<property name="profile" value="wcag22-aa"/>
|
|
31
|
+
<property name="locale" value="en"/>
|
|
32
|
+
<property name="url" value="https://example.test/"/>
|
|
33
|
+
</properties>
|
|
34
|
+
<testcase classname="wcag-1.1.1" name="img-alt-present" time="0">
|
|
35
|
+
<failure type="fail" message="1 failing occurrence: Missing alt attribute on <img>.">- Missing alt attribute on <img>. Add an alt attribute (use alt="" only for decorative images).
|
|
36
|
+
selector: html > body > main > img
|
|
37
|
+
html: <img src="a.png"></failure>
|
|
38
|
+
</testcase>
|
|
39
|
+
</testsuite>
|
|
40
|
+
</testsuites>
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
- **Suites follow the criterion**, because that is what people track, and **testcases follow the rule** rather than the occurrence, so a defect repeated forty times on a page is one failing test whose body lists all forty, and test counts stay stable between runs.
|
|
44
|
+
- Suites come from each rule's own WCAG mappings, not from the composites, so every rule that ran is reported even when composites were excluded. The composite, when it ran, supplies the suite's title and the `criterionOutcome` property. A rule mapped to two criteria appears in both suites. A rule mapped to no criterion goes into a final `Other checks` suite with `classname="other"`.
|
|
45
|
+
- The run's `engine`, `schemaVersion`, `wcagVersion`, `profile`, `optInRules` (comma-separated tags, when `engineOptions.optInRules` added rules), `locale` and `url` are repeated as properties on every suite, each only when the result has it.
|
|
46
|
+
- Suites are ordered by criterion, numerically (1.4.3 before 1.4.10), and testcases by rule id.
|
|
47
|
+
- `en301549` properties name the EN 301 549 clause that restates the criterion, where there is one and the scan asked for EN 301 549 clauses (see [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md#en-301-549)).
|
|
48
|
+
|
|
49
|
+
## Outcomes
|
|
50
|
+
|
|
51
|
+
| Rule outcome | JUnit | Why |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| `fail` | `<failure type="fail">`, one line per failing occurrence (message, hint, selector, markup) | The deterministic, gating case. |
|
|
54
|
+
| `cantTell` | `<skipped>` saying how many occurrences need manual review | JUnit has no "could not tell". Skipped surfaces it without turning a build red, the same line SARIF draws with `warning`. |
|
|
55
|
+
| `pass` | a bare `<testcase>` | |
|
|
56
|
+
| `notApplicable` | left out | A page has hundreds; none says anything. `includeNotApplicable: true` adds them as `<skipped message="Not applicable">`. |
|
|
57
|
+
|
|
58
|
+
A `fail` rule that also has `cantTell` occurrences reports the failures in `<failure>` and the undecided ones in `<system-out>`, where dashboards show test output. A skipped `cantTell` rule does the same.
|
|
59
|
+
|
|
60
|
+
## Options
|
|
61
|
+
|
|
62
|
+
| Option | Default | Effect |
|
|
63
|
+
|---|---|---|
|
|
64
|
+
| `cantTellAs` | `'skipped'` | `'failure'` reports `cantTell` rules as `<failure type="cantTell">`, for a pipeline that must not pass while anything is undecided. |
|
|
65
|
+
| `includeNotApplicable` | `false` | Include `notApplicable` rules as skipped tests. |
|
|
66
|
+
| `baselineEntries` | none | Entries from [`BASELINE.md`](./BASELINE.md)'s `buildBaselineEntries()`. Fail occurrences recorded there are dropped, exactly as in SARIF. A rule whose every failure is already known is `<skipped message="N known failures recorded in the baseline">`, not passing: it did not pass. |
|
|
67
|
+
| `name` | `'surea11y'` | The `name` of the root `<testsuites>`, for telling several pages' reports apart in one dashboard. |
|
|
68
|
+
|
|
69
|
+
## Determinism
|
|
70
|
+
|
|
71
|
+
The engine has no clock, and this report does not invent one: every `time` is `"0"`, and a `timestamp` attribute appears on each suite only when the result carries one (`engineOptions.timestamp`). The same scan always renders byte-identical XML, so a report can be committed or diffed.
|
|
72
|
+
|
|
73
|
+
Text is escaped for XML, and characters XML 1.0 forbids even when escaped (most control characters, lone surrogates) are dropped, since markup captured from a page can contain them.
|
package/docs/LIMITATIONS.md
CHANGED
|
@@ -17,6 +17,7 @@ surea11y is a **static DOM scan**: it reads the DOM tree and computed styles at
|
|
|
17
17
|
- **`<dialog>` and other elements hidden by the default UA stylesheet** (no `open` attribute, `display: none` by spec), along with any other subtree hidden via `display:none`, `visibility:hidden`, `[hidden]`, or closed `<details>`, are **excluded from rule evaluation by default** — matching the visibility-aware behavior of other established engines. This is a deliberate default (`engineOptions.includeHiddenElements: false`), not an oversight: hidden content isn't reachable by assistive technology or keyboard until it's shown, so flagging a markup defect inside it by default would often be noise. Set `engineOptions.includeHiddenElements: true` to evaluate hidden/collapsed subtrees anyway — e.g. to catch a markup defect (like a broken ARIA ID reference) before a dialog ever opens. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#engineoptions--the-rest) for the option and exactly which hiding mechanisms it covers.
|
|
18
18
|
- **jsdom's computed `text-shadow` is unreliable on a second read of the same element.** Confirmed in jsdom 29.1.1: reading a computed `text-shadow` value a second time on the same element — through any accessor, from any freshly-requested `CSSStyleDeclaration` for that element, regardless of caching — silently returns a different, wrong "no shadow" value instead of the real declared one. The first read is always correct. This engine works around it internally by reading each element's `text-shadow` exactly once per run and caching the result (see `__textShadowInfoEl` in `src/core/contrast-helpers.js`), so a single scan is unaffected. It only surfaces if you read `getComputedStyle(el).textShadow` yourself, more than once, against the same jsdom-parsed element — a real browser has no such bug.
|
|
19
19
|
- **Under jsdom, scan time grows with the square of DOM depth, not with element count.** jsdom resolves inherited CSS by walking an element's ancestor chain on every `getComputedStyle` call, so one call per element costs O(elements x depth). Measured on jsdom 29.1.1 with no engine code involved: 4000 elements in one chain take 8.3s of `getComputedStyle` alone, against 0.25s for the same 4000 as siblings. The engine's own ancestor walks are capped and stay linear, so this is jsdom's cost rather than the rules'. It affects `runDomRulesInPage` and anything built on it, including `@surea11y/test-matchers`; a real browser computes inherited style natively and does not have this shape. Component frameworks routinely nest 100-300 deep, which is comfortably fast — it becomes noticeable past roughly 1000.
|
|
20
|
+
- **In a real browser, some rules change the page for the time of their check, and the page's own scripts can react.** To measure what the browser draws, `css-hidden-focus` focuses elements as the keyboard would, and switches transitions off on the element it measures; `text-spacing-content-loss` adds a style sheet with the WCAG 1.4.12 spacing. All of this is put back before the rule returns: focus, scroll position, style attributes and the added style sheet. What cannot be put back is what the page's scripts do in response, since focusing an element fires its `focus` handlers: a carousel may move to the focused slide or enable a button. The findings are not affected, but if you scan a page you go on using, reload it afterwards.
|
|
20
21
|
- **Static markup vs. live/post-hydration DOM state.** The rule logic itself is DOM-source-agnostic — it evaluates whatever DOM it's handed, whether that's jsdom-parsed static HTML (Pattern 1) or an already-loaded, already-hydrated real browser tab (Pattern 2, see [`INTEGRATION.md`](./INTEGRATION.md)). But the CLI (`npx @surea11y/cli scan <url>`) specifically fetches static HTML only, with no JS execution — see [the CLI docs](https://github.com/SureA11y/cli/blob/main/docs/CLI.md). For a JS-framework-hydrated widget whose server-rendered markup intentionally ships one state before client JS syncs it (e.g. `<input type="checkbox" aria-checked="true">` shipped before client JS sets the native `checked` property to match on hydration — an extremely common, entirely legitimate pattern), a CLI scan only sees the pre-hydration markup. A scan running inside an actual loaded browser tab sees the post-hydration state instead, so the two can disagree on exactly this class of element for reasons that have nothing to do with rule correctness. That's why `aria-checked-state-mismatch` is capped at `manual`/`cantTell` rather than a hard `fail`. If you need live-DOM accuracy for hydration-sensitive checks, run the library directly against an already-loaded page via Pattern 2, not the static-fetch CLI.
|
|
21
22
|
|
|
22
23
|
## Not attempted: judgment calls that aren't automatable safely
|
package/docs/OUTPUT_SCHEMA.md
CHANGED
|
@@ -19,7 +19,9 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
19
19
|
tag: string,
|
|
20
20
|
schemaVersion: string,
|
|
21
21
|
locale: { requested: string, resolved: string, reason: string },
|
|
22
|
-
wcagVersion: "2.0" | "2.1" | "2.2"
|
|
22
|
+
wcagVersion: "2.0" | "2.1" | "2.2",
|
|
23
|
+
profile?: string, // "wcag22-aa", "en301549-v4.1.1", "en301549-v3.2.1", "section508", or a registered standard's own
|
|
24
|
+
mappings?: string[] // e.g. ["en301549"] or ["en301549:V3.2.1"]
|
|
23
25
|
},
|
|
24
26
|
url: string | null,
|
|
25
27
|
title: string | null,
|
|
@@ -37,15 +39,19 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
37
39
|
| `engine.tag` | The engine's own identity tag, currently `"a11ycore"`. Every rule (built-in or custom) carries it in `meta.tags` — rule `ruleId`s themselves are bare (no prefix). |
|
|
38
40
|
| `engine.schemaVersion` | The result-schema version (`"1.0.0"`). Bump-worthy if this document's shape ever changes incompatibly — pin to it if you're parsing output programmatically. See [`API_STABILITY.md`](./API_STABILITY.md) for the full stable/unstable field list and version-bump policy. |
|
|
39
41
|
| `engine.locale` | Which dictionary the run actually used. `requested` is your `engineOptions.locale` after trimming (`"en"` if you passed nothing or a non-string); `resolved` is the locale whose dictionary was used; `reason` explains the pairing. Because locale fallback is graceful and per-string, asking for a language the build does not carry produces English text rather than an error — this field is how you find that out without reading the strings. Reported once per result: a run uses one dictionary throughout. |
|
|
40
|
-
| `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key. `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
|
|
42
|
+
| `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key (a profile's messages in a language that profile does not offer show in English, by its choice, and do not count; see [`I18N.md`](./I18N.md)). `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
|
|
41
43
|
| `engine.wcagVersion` | Which version of WCAG this run was conformance-tested against: your `engineOptions.wcagVersion`, or what your version-origin tags implied, or the default `"2.2"`. It affects one thing today — a rule mapped only to SC 4.1.1 Parsing cannot `fail` under a 2.2 target (see `checksResults[i].wcagVersionScope` below). Reported once per result: a run has one target throughout. |
|
|
44
|
+
| `engine.profile` | Present only when an `engineOptions.profile` selected this run's rules, and names the profile, lowercased. Absent when none was asked for, and also when one was asked for but did not apply (unknown name, or an include in `runOnly` or `engineOptions` chose the rules instead), so its presence is how you confirm a run really targeted that profile. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
|
|
45
|
+
| `engine.profileExcludes` | Present only when the applied profile leaves rules out (`exclude` in its standard's registry entry): `{ rules, criteria }`, the rules it names and the WCAG criteria it waives. Their rules and WCAG rollups did not run. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
|
|
46
|
+
| `engine.optInRules` | Present only when `engineOptions.optInRules` ran at least one opt-in rule that the rest of the selection would not have run. Lists their tags. Its presence means the result includes rules for requirements beyond the targeted standard, such as a national standard's, so a failure may not be a WCAG failure. Absent under a WCAG profile, where unlocking selects nothing, and when a standard's own profile ran its rules. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#running-every-rule-optinrules). |
|
|
47
|
+
| `engine.mappings` | Present only when the run names a standard besides WCAG in `meta.normativeMappings`, through `engineOptions.mappings` or a standard's profile. Lists them canonically, in table order: `"en301549"` for every version, `"en301549:V3.2.1"` for one. Absent means every result names WCAG only. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#other-standards-mappings). |
|
|
42
48
|
| `url` | The `pageUrl` argument you passed in, or `document.location.href` if you passed `null`/omitted it, or `null` if neither is available. |
|
|
43
49
|
| `title` | `document.title` at scan time, or `null`. |
|
|
44
50
|
| `timestamp` | **Not auto-generated.** Only set if you pass `engineOptions.timestamp` as a non-empty string — the engine has no built-in clock (deterministic-by-design). If you want a scan timestamp in the result, supply it yourself. |
|
|
45
51
|
| `perfStats` | `null` unless `engineOptions.perfStats: true`. Internal timing/counters — shape not covered by this document, treat as debug-only. |
|
|
46
52
|
| `contextSelector` | The (trimmed) `contextSelector` argument you passed — a string, an array of strings (multi-region scanning, see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)), or `null` if none/empty. |
|
|
47
53
|
| `checksResults` | One entry per **atomic rule** that ran (every rule not filtered out by `runOnly` — see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). **Every loaded rule produces an entry, even ones that outcome `notApplicable`** — this is not a "violations only" list. |
|
|
48
|
-
| `rulesResults` | One entry per **composite (
|
|
54
|
+
| `rulesResults` | One entry per **composite (rollup) rule** that ran — see [Composite result](#a-composite-result-rulesresultsi) and [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md). Normally one per WCAG Success Criterion; a run under the profile of a standard with rollups of its own also gets those (`meta.standard` set). Empty array if no composite matched the current `runOnly`/tag filter. |
|
|
49
55
|
| `overriddenBuiltinIds` | Rule ids where an `engineOptions.customRules` entry shared its `id` with a built-in rule, so the custom implementation replaced the built-in one for this scan (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Always an array; empty when no collision occurred. Also logged via `console.warn` at scan time, since a same-named custom rule is as likely to be an accidental collision as a deliberate override. |
|
|
50
56
|
|
|
51
57
|
## Cross-frame result (`runa11yCoreAcrossFrames`)
|
|
@@ -87,7 +93,7 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
87
93
|
normative: boolean,
|
|
88
94
|
atomic: boolean,
|
|
89
95
|
category: "perceivable" | "operable" | "understandable" | "robust" | null,
|
|
90
|
-
normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel
|
|
96
|
+
normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel?: string, wcagSc?: string[] }>,
|
|
91
97
|
standard: string | null,
|
|
92
98
|
applicability: string,
|
|
93
99
|
expectation: string,
|
|
@@ -97,6 +103,8 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
97
103
|
},
|
|
98
104
|
engineOptions: object, // the resolved engineOptions this rule actually ran under
|
|
99
105
|
schemaVersion: string,
|
|
106
|
+
rollupIds: string[], // the rulesResults entries that group this rule in this run; [] if none
|
|
107
|
+
data?: object, // page-level data a rule reports whatever its outcome; see below
|
|
100
108
|
wcagVersionScope?: { // present only when the target WCAG version changed this outcome
|
|
101
109
|
target: "2.0" | "2.1" | "2.2",
|
|
102
110
|
removedSc: string[],
|
|
@@ -110,8 +118,10 @@ Notes:
|
|
|
110
118
|
|
|
111
119
|
- **`outcome` vs `outcomeNormalized`**: identical except `notApplicable` becomes `"inapplicable"` in `outcomeNormalized`. Both are provided so you can match either your own vocabulary or the engine's internal one.
|
|
112
120
|
- **`type: "manual"` rules can never report `outcome: "fail"`.** If a manual rule's own logic would have said `fail`, the engine coerces it to `cantTell` and appends an explanatory note to `error` — this is enforced centrally (`policy.coerceManualFailToCantTell`, on by default under the `a11y` policy contract; see [`POLICY.md`](./POLICY.md)), not something each rule has to remember. `fail` is reserved for deterministic, `type: "automatic"` findings only.
|
|
113
|
-
- **`
|
|
121
|
+
- **`rollupIds`** lists the composites in `rulesResults` that group this rule in this run. An empty list means the rule's findings appear in no rollup, so a consumer that reads only `rulesResults` never sees them; `heading-order`, for instance, belongs to no WCAG rollup.
|
|
122
|
+
- **`meta.normativeMappings`** is how a check result ties back to a WCAG Success Criterion — `[]` for rules with no formal WCAG mapping (this engine calls them advisory `type: "manual"` rules). See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md) for how these roll up. The list is not WCAG-only. When the scan asks for another standard (`engineOptions.mappings`, or a standard's profile), each WCAG criterion is followed by that standard's corresponding requirement, such as the EN 301 549 clause that restates it (`{ standard: "EN 301 549", version: "V3.2.1" | "V4.1.1", requirement: "9.1.1.1", title, wcagSc: ["1.1.1"] }`, one entry per version that includes the criterion, `wcagSc` naming the criteria it corresponds to), or the requirements of a standard mapped rule by rule that the rule checks (same shape, `wcagSc` empty for a requirement WCAG does not make, and any fields of that standard's own); and a rule may also cite WCAG's Understanding documents (`type: "Understanding"`). Filter on `standard` (and on the absence of `type`) before reading `requirement` as a Success Criterion. A composite's WCAG entry is always first. See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md#en-301-549).
|
|
114
123
|
- **`wcagVersionScope`**: only present when the run's target WCAG version turned this rule's `fail` into a `cantTell` — today that means a rule mapped to SC 4.1.1 Parsing (`duplicate-id`) under the default 2.2 target, since 2.2 removed that criterion. `removedSc` lists the criteria that stopped existing, `target` is the version that removed them, and `coercedFrom` is the outcome the rule itself reported. The occurrences are the rule's own, unchanged — nothing was dropped, only the conformance verdict was. Absent on every other result, and **never** reported through `error`: nothing went wrong. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#filtering-by-wcag-version-21-vs-22).
|
|
124
|
+
- **`data`**: present only on a rule that reports something about the whole page, whatever its outcome. No core rule does today; a profile's rule may, such as one returning the page's own entry for a probe that compares pages (see `probes` in [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Like `data.details` on an occurrence, it is not a stable contract.
|
|
115
125
|
- **`error`**: only present if the rule implementation threw an uncaught exception, or if the manual-fail coercion above fired. A thrown rule always surfaces as `outcome: "cantTell"` with `occurrences: []` and `error` set to the exception message — the engine never lets one broken rule crash the whole scan.
|
|
116
126
|
- **`engineOptions`** on each result is the *resolved* options object (after locale/contrast defaults were applied), not literally what you passed in — useful for confirming what a given rule actually saw, especially the resolved `locale` and `contrast.mode`/`contrast.rootCanvasFallback`.
|
|
117
127
|
|
|
@@ -175,7 +185,7 @@ Every automatic rule that can report `cantTell` carries this, and a test holds t
|
|
|
175
185
|
|
|
176
186
|
## A composite result (`rulesResults[i]`)
|
|
177
187
|
|
|
178
|
-
Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Shape is the same envelope as a check result, with composite-specific `data.details`:
|
|
188
|
+
Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Under the profile of a standard with rollups of its own, there are also those, with the standard's wording as `title` and its name as `meta.standard`; they may be the only rollup some of its findings have, such as `heading-order`'s. Shape is the same envelope as a check result, with composite-specific `data.details`:
|
|
179
189
|
|
|
180
190
|
```ts
|
|
181
191
|
{
|
|
@@ -186,6 +196,9 @@ Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wc
|
|
|
186
196
|
data: {
|
|
187
197
|
details: {
|
|
188
198
|
reasonCode: string, // e.g. "composite.rollup.fail.anyFail"
|
|
199
|
+
standard?: string, // a standard's own rollup only, with version and criterion
|
|
200
|
+
version?: string,
|
|
201
|
+
criterion?: string,
|
|
189
202
|
checksIds: string[], // every atomic ruleId this composite rolls up
|
|
190
203
|
contributors: Array<{ testId: string, outcome: string, severity: string | null }>,
|
|
191
204
|
metrics: { failCount, cantTellCount, notApplicableCount, passCount, missingCount }
|
package/docs/REPORT.md
CHANGED
|
@@ -12,10 +12,15 @@ Open `report.html` directly from disk. Works alongside any other output mode —
|
|
|
12
12
|
|
|
13
13
|
- **Hero**: one plain-language headline ("N of M applicable checks passed") plus a stacked bar and legend (icon + label + count — status is never color-only) broken down by outcome (`fail`/`cantTell`/`pass`/`notApplicable`).
|
|
14
14
|
- **Worth reviewing**: one card per rule with `fail`/`cantTell` occurrences (not one per occurrence — a rule with many identical occurrences is one thing worth attention, not many), each showing severity, WCAG SC chip(s), a representative occurrence, and the total occurrence count. Capped at the 24 highest-priority rules with an overflow note past that.
|
|
15
|
-
- **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion, its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
|
|
15
|
+
- **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion (with the EN 301 549 clause that restates it, where there is one and the scan asked for EN 301 549 clauses), its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
|
|
16
|
+
- **A standard's own rollup**: only when the scan ran the rules of a standard that has rollups of its own (its profile, or `engineOptions.optInRules`), one row per rollup, with the standard's wording, the requirements of the rules that decided the outcome, the breakdown and the contributing rules.
|
|
16
17
|
- **Full technical data** (collapsed by default): a scorecard (tiles per outcome) and a searchable, filterable (by outcome), paginated table of every individual occurrence across the whole scan.
|
|
17
18
|
|
|
18
|
-
The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
|
|
19
|
+
The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, the WCAG version the scan targeted the `engineOptions.profile` it used (when there was one), the opt-in rule tags `engineOptions.optInRules` added (when it added any), and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
|
|
20
|
+
|
|
21
|
+
The whole page is written in the locale the scan resolved to: headings, table columns, outcome and severity names, the headline, the pager, and the date format, all from the `report_*` keys in the same dictionaries as the findings. `<html lang>` names that locale. The outcome codes in the occurrence filters (`fail`, `cantTell`, …) stay as they are, because they are the values a reader searches for in the JSON result.
|
|
22
|
+
|
|
23
|
+
One case keeps English labels: a scan in a language the engine does not ship, run with a caller-supplied `engineOptions.messages` dictionary. That dictionary is not part of the result, so the report cannot read labels from it; its findings are then marked with their own language (`lang="nl"`, for instance), so a screen reader still reads them with the right voice.
|
|
19
24
|
|
|
20
25
|
## Library usage
|
|
21
26
|
|
package/docs/RULE_AUTHORING.md
CHANGED
|
@@ -114,7 +114,7 @@ const meta = {
|
|
|
114
114
|
title: 'Non-text Content',
|
|
115
115
|
conformanceLevel: 'A'
|
|
116
116
|
}
|
|
117
|
-
],
|
|
117
|
+
], // WCAG only: EN 301 549 clauses are derived at build time (WCAG_CONFORMANCE.md#en-301-549)
|
|
118
118
|
|
|
119
119
|
defaultSeverity: 'minor' | 'moderate' | 'serious' | 'critical',
|
|
120
120
|
category: 'perceivable' | 'operable' | 'understandable' | 'robust',
|
|
@@ -141,8 +141,9 @@ The build/runtime resolves i18n by:
|
|
|
141
141
|
2) falling back to `en` if missing,
|
|
142
142
|
3) falling back to the literal `title`/`description` strings if still missing.
|
|
143
143
|
|
|
144
|
-
Add the key and its English text to `src/i18n/en.json
|
|
145
|
-
`npm run i18n:sync` so every other
|
|
144
|
+
Add the key and its English text to `src/i18n/en.json` (a profile's rule:
|
|
145
|
+
`profiles/<name>/i18n/en.json`), then run `npm run i18n:sync` so every other
|
|
146
|
+
locale picks it up. `npm test` fails if you
|
|
146
147
|
forget. See [`I18N.md`](./I18N.md).
|
|
147
148
|
|
|
148
149
|
#### `meta.tags`
|
|
@@ -150,6 +151,35 @@ Tags are used for grouping/filtering. Typical tag families in this ruleset inclu
|
|
|
150
151
|
- WCAG tagging: `wcag2a`, `wcag111`
|
|
151
152
|
- domain: `nontext`, `images`, plus element-specific tags
|
|
152
153
|
- nature: `atomic`, plus `automatic` or `manual`
|
|
154
|
+
- another standard's own requirement: that standard's rule tag (see below)
|
|
155
|
+
|
|
156
|
+
#### Rules for another standard's own requirements
|
|
157
|
+
A rule that checks something WCAG does not require, but another standard does (a doctype or presentational attributes, say), declares no WCAG mapping (`wcagSc: []`, `normativeMappings: []`) and carries that standard's rule tag. The tag makes it **opt-in**: it runs only under the standard's profile, a selection that includes the tag, or its own id, never in a default or WCAG run ([`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#opt-in-rules)). That is what lets it report `fail`: its failures are failures of that standard, and only a scan targeting it sees them. Its module goes in that standard's profile rather than in `src/checks/`: `profiles/<key>/rules/automatic/` or `profiles/<key>/rules/manual/`. The build compiles it into the engine like any other rule. Its test and scenario page go in the profile too, in `profiles/<key>/tests/rules/` and `profiles/<key>/tests/fixtures/`. Map it to the standard's requirements the usual way (a row in the profile's rule map). Rule tags come from each standard's `ruleTag` in the registry, `src/coverage/standards.js` (a profile's from its `index.js`). The sample profile core's tests run against, `tests/fixtures/profiles/sample/`, has two such rules.
|
|
158
|
+
|
|
159
|
+
#### Rule variants
|
|
160
|
+
When another standard's requirement is a core rule with different thresholds (contrast at 7:1, say, or bold text large from 18.5px rather than WCAG's 14pt), write it as a **variant**, not a copy. The core rule declares the thresholds it reads from `ctx.config` as `settings`, with WCAG's values as defaults:
|
|
161
|
+
|
|
162
|
+
```js
|
|
163
|
+
// src/checks/automatic/contrast-minimum.js
|
|
164
|
+
const settings = { boldLargeMinPx: null, largeTextRatio: 3, normalTextRatio: 4.5 };
|
|
165
|
+
module.exports = { id, meta, runInPage, settings };
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
The variant is data, in the standard's profile:
|
|
169
|
+
|
|
170
|
+
```js
|
|
171
|
+
// profiles/<key>/rules/automatic/sample-contrast-enhanced.js
|
|
172
|
+
module.exports = {
|
|
173
|
+
id: 'sample-contrast-enhanced',
|
|
174
|
+
from: 'contrast-minimum',
|
|
175
|
+
config: { normalTextRatio: 7, largeTextRatio: 4.5 },
|
|
176
|
+
meta: { /* its own title, description, i18n, tags... as any rule's */ }
|
|
177
|
+
};
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
The build runs the base rule's `runInPage` and `applicability` under the variant's id and meta, with its `config` in `ctx.config`. A rule's settings are never the caller's: the runner drops a caller's value for one (`engineOptions.rules[ruleId]`), so the base rule always runs at its defaults and the variant at its `config`, while the caller's other config, such as `excludeSelectors`, still applies. A message key of the base's that starts with the base's prefix (its `meta.i18n.titleKey` without `_title`, `contrastMinimum`) is read from the variant's prefix instead (`sampleContrastEnhanced`), so the variant's dictionary has the same keys under its own prefix; the rule validator checks they exist. A fix to the base reaches every variant. The build refuses a variant whose base does not exist, is itself a variant, or declares no `settings`, and a setting the base does not declare or of another type. A base rule that caches verdicts depending on its settings keys those caches by them, as `contrast-minimum` does.
|
|
181
|
+
|
|
182
|
+
Add a setting to a core rule when a standard needs it, with a default that keeps the rule's behaviour; the settings a rule declares are for its variants, not for callers, and stay outside semver until a profile can live outside this repository (see [`API_STABILITY.md`](./API_STABILITY.md#explicitly-unstable-not-covered-by-semver)).
|
|
153
183
|
|
|
154
184
|
#### `meta.coverage.facetsBySc`
|
|
155
185
|
This is the repo’s explicit **coverage model** for an SC.
|
|
@@ -317,7 +347,23 @@ The rule must return:
|
|
|
317
347
|
|
|
318
348
|
Examples:
|
|
319
349
|
|
|
320
|
-
### 8.2
|
|
350
|
+
### 8.2 What `ctx` carries
|
|
351
|
+
|
|
352
|
+
`runInPage(ctx)` and `applicability(ctx)` receive the same object, built-in and custom rules alike:
|
|
353
|
+
|
|
354
|
+
| Field | What it is |
|
|
355
|
+
|---|---|
|
|
356
|
+
| `document`, `window` | The page being scanned. |
|
|
357
|
+
| `root` | The roots the scan covers: the document, or what `contextSelector` resolved to. |
|
|
358
|
+
| `contextSelector` | The selector that scoped the run, if any. |
|
|
359
|
+
| `rule` | The rule's resolved definition: `ruleId`, `defaultSeverity`, `defaultConfidence`, `type`, `meta`... |
|
|
360
|
+
| `config` | `engineOptions.rules[ruleId]`, this rule's settings, if the caller gave any (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). |
|
|
361
|
+
| `standard` | The standard and version the run targets, `{ key, name, version }` (`{ key: 'en301549', name: 'EN 301 549', version: 'V4.1.1' }`), when a standard's profile selected the run; `null` otherwise (no profile, a WCAG profile, or rules chosen by tag or id). A rule whose behaviour differs between versions of its standard reads it here, and does what holds for every version when it is `null`. |
|
|
362
|
+
| `helpers` | The helpers documented in [`RULE_HELPERS.md`](./RULE_HELPERS.md). |
|
|
363
|
+
| `engineOptions` | The scan's options as resolved. |
|
|
364
|
+
| `inputs.probes` | Evidence the host application supplied (`engineOptions.probes`). |
|
|
365
|
+
|
|
366
|
+
### 8.3 Outcome conventions used by these rules
|
|
321
367
|
|
|
322
368
|
Automatic:
|
|
323
369
|
- `notApplicable` if no applicable targets
|
|
@@ -385,7 +431,8 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
|
|
|
385
431
|
### 11.1 The fixture file
|
|
386
432
|
|
|
387
433
|
- Path: `tests/fixtures/<rule-slug>-all-scenarios.html`, where `<rule-slug>` is the rule
|
|
388
|
-
id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`).
|
|
434
|
+
id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`). A
|
|
435
|
+
profile's rule keeps it in the profile: `profiles/<name>/tests/fixtures/`.
|
|
389
436
|
- Structure: a real HTML page (`<!doctype html>`, `<title>`, minimal inline `<style>`)
|
|
390
437
|
containing numbered scenario blocks, each:
|
|
391
438
|
```html
|
|
@@ -405,6 +452,24 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
|
|
|
405
452
|
(eligible, FAIL)", "C. Ineligible (excluded from accessibility tree, skipped)").
|
|
406
453
|
- Cover every branch the rule's own logic distinguishes: pass, fail (each distinct
|
|
407
454
|
`reasonCode`), notApplicable/skipped, and — for manual rules — cantTell.
|
|
455
|
+
- A whole-document rule (`page-title-present`, `meta-refresh-timing-absent`, `region`)
|
|
456
|
+
can only demonstrate one outcome per page. Its fixture declares a single bare
|
|
457
|
+
`.case-title` with no `.case` wrapper, and the page itself is the case; the marker is
|
|
458
|
+
compared against the rule-level outcome, so `PASS` and `NEUTRAL` are distinguished
|
|
459
|
+
there. Cover the remaining branches with inline tests rather than near-identical
|
|
460
|
+
fixture files.
|
|
461
|
+
- One fixture shared by several rules that expect different things of the same case
|
|
462
|
+
(`tests/fixtures/contrast-all-scenarios.html` serves `contrast-minimum`,
|
|
463
|
+
`contrast-enhanced` and `contrast-computable`) carries a per-rule marker as a
|
|
464
|
+
`data-outcome-<rule-id>` attribute on the `.case`, which overrides the shared
|
|
465
|
+
`.case-title` for that rule. Use an attribute rather than more text when the rules
|
|
466
|
+
under test evaluate text: a `.case-title` added to a contrast case is one more text
|
|
467
|
+
node to check. A marker word the parser does not recognise (`MIXED`, `UNSTATED`)
|
|
468
|
+
asserts nothing, for a case whose outcome the fixture does not state.
|
|
469
|
+
- `npm run fixtures:markers:check` replays every fixture and fails when a marker no
|
|
470
|
+
longer matches what the rule reports; `scripts/data/fixture-markers.json` records the
|
|
471
|
+
cases that already disagree, so that set can only shrink. A profile's rules have
|
|
472
|
+
their record in the profile's own `scripts/data/fixture-markers.json`.
|
|
408
473
|
|
|
409
474
|
### 11.2 Known, acceptable exceptions to "one fixture, many cases"
|
|
410
475
|
|
|
@@ -480,7 +545,9 @@ npm run fixtures:index
|
|
|
480
545
|
This writes `tests/fixtures/INDEX.md` (human-readable), `tests/fixtures/index.json`
|
|
481
546
|
(machine-readable — every rule, its fixture path, and parsed pass/fail/cantTell case
|
|
482
547
|
counts, for external tooling to enumerate and load fixtures directly) and
|
|
483
|
-
`tests/fixtures/index.html` (the same listing as a browsable page).
|
|
548
|
+
`tests/fixtures/index.html` (the same listing as a browsable page). A profile's rules
|
|
549
|
+
get the same three files in the profile's `tests/fixtures/`, with paths relative to the
|
|
550
|
+
profile's folder. Commit all three
|
|
484
551
|
alongside the fixture and test changes. A rule shipped without its fixture is treated
|
|
485
552
|
the same as a rule shipped without tests — not done. `npm run fixtures:check` reports
|
|
486
553
|
a stale index without rewriting it, and CI fails on one.
|