@surea11y/core 1.7.0 → 1.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +94 -1
- package/README.md +157 -54
- package/docs/ACT_RULE_MAPPING.md +2 -2
- package/docs/API_STABILITY.md +18 -5
- package/docs/BINDING_AUTHORS_GUIDE.md +2 -2
- package/docs/CI_INTEGRATIONS.md +43 -0
- package/docs/DESIGN_CHALLENGES.md +97 -3
- package/docs/EARL.md +1 -1
- package/docs/ENGINE_OPTIONS.md +81 -3
- package/docs/I18N.md +62 -20
- package/docs/JUNIT.md +73 -0
- package/docs/LIMITATIONS.md +1 -0
- package/docs/OUTPUT_SCHEMA.md +19 -6
- package/docs/REPORT.md +7 -2
- package/docs/RULE_AUTHORING.md +73 -6
- package/docs/RULE_CATALOG.md +139 -116
- package/docs/RULE_EXAMPLES.md +2189 -0
- package/docs/RULE_HELPERS.md +62 -5
- package/docs/RULE_TAXONOMY.md +2 -2
- package/docs/SARIF.md +2 -1
- package/docs/WCAG_CONFORMANCE.md +56 -3
- package/package.json +34 -11
- package/profiles/index.js +14 -0
- package/src/checks/automatic/area-alt-present.js +87 -31
- package/src/checks/automatic/aria-braille-equivalent.js +25 -7
- package/src/checks/automatic/aria-hidden-focus.js +74 -18
- package/src/checks/automatic/aria-prohibited-attr.js +17 -4
- package/src/checks/automatic/aria-required-attr.js +29 -0
- package/src/checks/automatic/aria-role-name-present.js +19 -2
- package/src/checks/automatic/aria-valid-attr-value.js +28 -16
- package/src/checks/automatic/autocomplete-valid.js +152 -26
- package/src/checks/automatic/avoid-inline-spacing.js +105 -40
- package/src/checks/automatic/button-name-present.js +2 -1
- package/src/checks/automatic/canvas-text-alternative-present.js +105 -14
- package/src/checks/automatic/combobox-name-present.js +34 -51
- package/src/checks/automatic/contrast-computable.js +35 -4
- package/src/checks/automatic/contrast-enhanced.js +4 -4
- package/src/checks/automatic/contrast-minimum.js +45 -11
- package/src/checks/automatic/css-orientation-lock.js +152 -30
- package/src/checks/automatic/definition-list-children-valid.js +67 -23
- package/src/checks/automatic/deprecated-elements-not-used.js +43 -38
- package/src/checks/automatic/dialog-name-present.js +28 -9
- package/src/checks/automatic/duplicate-id.js +6 -2
- package/src/checks/automatic/identical-iframes-same-purpose.js +4 -4
- package/src/checks/automatic/iframe-focusable-content.js +7 -4
- package/src/checks/automatic/iframe-title-unique.js +36 -81
- package/src/checks/automatic/input-image-alt-present.js +32 -20
- package/src/checks/automatic/label-in-name.js +40 -13
- package/src/checks/automatic/language-page-present.js +12 -6
- package/src/checks/automatic/link-in-text-block.js +272 -60
- package/src/checks/automatic/link-name-present.js +13 -5
- package/src/checks/automatic/list-children-valid.js +18 -1
- package/src/checks/automatic/listbox-name-present.js +19 -49
- package/src/checks/automatic/listitem-parent-valid.js +4 -3
- package/src/checks/automatic/meta-refresh-no-exceptions.js +3 -3
- package/src/checks/automatic/page-title-present.js +16 -4
- package/src/checks/automatic/progressbar-name-present.js +11 -1
- package/src/checks/automatic/role-img-text-alternative-present.js +9 -5
- package/src/checks/automatic/searchbox-name-present.js +32 -49
- package/src/checks/automatic/server-side-image-map-absent.js +48 -28
- package/src/checks/automatic/slider-name-present.js +38 -52
- package/src/checks/automatic/spinbutton-name-present.js +32 -49
- package/src/checks/automatic/target-size-minimum.js +0 -11
- package/src/checks/automatic/td-has-header.js +41 -5
- package/src/checks/automatic/text-spacing-content-loss.js +548 -0
- package/src/checks/automatic/textbox-name-present.js +32 -49
- package/src/checks/automatic/valid-lang.js +15 -10
- package/src/checks/manual/area-alt-quality-manual.js +113 -31
- package/src/checks/manual/canvas-text-alternative-quality-manual.js +0 -2
- package/src/checks/manual/css-focus-indicator-suppressed-manual.js +23 -10
- package/src/checks/manual/css-hidden-focus.js +215 -7
- package/src/checks/manual/form-control-label-quality-manual.js +109 -5
- package/src/checks/manual/heading-order-manual.js +9 -1
- package/src/checks/manual/heading-quality-manual.js +143 -9
- package/src/checks/manual/img-alt-decorative-manual.js +6 -3
- package/src/checks/manual/input-image-alt-quality-manual.js +81 -13
- package/src/checks/manual/link-name-quality-manual.js +130 -4
- package/src/checks/manual/media-transcript-present-manual.js +65 -8
- package/src/checks/manual/mouse-only-event-handlers-manual.js +66 -5
- package/src/checks/manual/no-autoplay-audio-manual.js +93 -6
- package/src/checks/manual/p-as-heading-manual.js +89 -44
- package/src/checks/manual/page-title-patterns-manual.js +77 -8
- package/src/checks/manual/skip-link-manual.js +42 -14
- package/src/checks/manual/table-fake-caption-manual.js +32 -1
- package/src/checks/manual/video-caption-manual.js +47 -24
- package/src/checks/manual-review.js +0 -4
- package/src/core.js +14419 -2231
- package/src/coverage/en301549-map.js +187 -0
- package/src/coverage/standards.js +279 -0
- package/src/coverage/wcag-facets.js +1119 -0
- package/src/coverage/wcag-version-map.js +101 -0
- package/src/en301549.js +33 -0
- package/src/junit.js +321 -0
- package/src/profile-kit.js +163 -0
- package/src/report.js +343 -74
- package/src/sarif.js +34 -3
- package/src/wcag.js +105 -0
- package/surea11y.browser.js +5 -4
- package/surea11y.i18n.de.js +1 -1
- package/surea11y.i18n.es.js +1 -1
- package/surea11y.i18n.fr.js +1 -1
- package/surea11y.i18n.ja.js +3 -0
- package/src/checks/manual/area-alt-decorative-manual.js +0 -255
package/docs/JUNIT.md
ADDED
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
# JUnit report
|
|
2
|
+
|
|
3
|
+
`@surea11y/core/junit` renders a scan result as JUnit XML, the test report format CI dashboards read natively: GitLab's merge request test widget, Azure DevOps' Tests tab, Jenkins and CircleCI.
|
|
4
|
+
|
|
5
|
+
```js
|
|
6
|
+
const { renderJunitReport } = require('@surea11y/core/junit');
|
|
7
|
+
const { runDomRulesInPage } = require('@surea11y/core');
|
|
8
|
+
|
|
9
|
+
const result = runDomRulesInPage(url, null, { profile: 'wcag22-aa' }, null);
|
|
10
|
+
require('fs').writeFileSync('surea11y.junit.xml', renderJunitReport(result));
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`renderJunitReport(result, options)` is a pure function: it returns the XML as a string and never touches the filesystem. It works just as well on a result saved as JSON earlier, such as the output of `surea11y scan <target> --json` from [`@surea11y/cli`](https://github.com/SureA11y/cli#readme) — see [`CI_INTEGRATIONS.md`](./CI_INTEGRATIONS.md#junit-test-reports).
|
|
14
|
+
|
|
15
|
+
## Shape
|
|
16
|
+
|
|
17
|
+
One `<testsuite>` per WCAG Success Criterion, one `<testcase>` per rule mapped to it:
|
|
18
|
+
|
|
19
|
+
```xml
|
|
20
|
+
<testsuites name="surea11y" tests="14" failures="2" errors="0" skipped="4" time="0">
|
|
21
|
+
<testsuite name="WCAG 1.1.1 Non-text content: text alternatives" tests="1" failures="1" errors="0" skipped="0" time="0">
|
|
22
|
+
<properties>
|
|
23
|
+
<property name="wcagCriterion" value="1.1.1"/>
|
|
24
|
+
<property name="wcagLevel" value="A"/>
|
|
25
|
+
<property name="en301549" value="9.1.1.1"/>
|
|
26
|
+
<property name="criterionOutcome" value="fail"/>
|
|
27
|
+
<property name="engine" value="a11ycore"/>
|
|
28
|
+
<property name="schemaVersion" value="1.0.0"/>
|
|
29
|
+
<property name="wcagVersion" value="2.2"/>
|
|
30
|
+
<property name="profile" value="wcag22-aa"/>
|
|
31
|
+
<property name="locale" value="en"/>
|
|
32
|
+
<property name="url" value="https://example.test/"/>
|
|
33
|
+
</properties>
|
|
34
|
+
<testcase classname="wcag-1.1.1" name="img-alt-present" time="0">
|
|
35
|
+
<failure type="fail" message="1 failing occurrence: Missing alt attribute on <img>.">- Missing alt attribute on <img>. Add an alt attribute (use alt="" only for decorative images).
|
|
36
|
+
selector: html > body > main > img
|
|
37
|
+
html: <img src="a.png"></failure>
|
|
38
|
+
</testcase>
|
|
39
|
+
</testsuite>
|
|
40
|
+
</testsuites>
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
- **Suites follow the criterion**, because that is what people track, and **testcases follow the rule** rather than the occurrence, so a defect repeated forty times on a page is one failing test whose body lists all forty, and test counts stay stable between runs.
|
|
44
|
+
- Suites come from each rule's own WCAG mappings, not from the composites, so every rule that ran is reported even when composites were excluded. The composite, when it ran, supplies the suite's title and the `criterionOutcome` property. A rule mapped to two criteria appears in both suites. A rule mapped to no criterion goes into a final `Other checks` suite with `classname="other"`.
|
|
45
|
+
- The run's `engine`, `schemaVersion`, `wcagVersion`, `profile`, `optInRules` (comma-separated tags, when `engineOptions.optInRules` added rules), `locale` and `url` are repeated as properties on every suite, each only when the result has it.
|
|
46
|
+
- Suites are ordered by criterion, numerically (1.4.3 before 1.4.10), and testcases by rule id.
|
|
47
|
+
- `en301549` properties name the EN 301 549 clause that restates the criterion, where there is one and the scan asked for EN 301 549 clauses (see [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md#en-301-549)).
|
|
48
|
+
|
|
49
|
+
## Outcomes
|
|
50
|
+
|
|
51
|
+
| Rule outcome | JUnit | Why |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| `fail` | `<failure type="fail">`, one line per failing occurrence (message, hint, selector, markup) | The deterministic, gating case. |
|
|
54
|
+
| `cantTell` | `<skipped>` saying how many occurrences need manual review | JUnit has no "could not tell". Skipped surfaces it without turning a build red, the same line SARIF draws with `warning`. |
|
|
55
|
+
| `pass` | a bare `<testcase>` | |
|
|
56
|
+
| `notApplicable` | left out | A page has hundreds; none says anything. `includeNotApplicable: true` adds them as `<skipped message="Not applicable">`. |
|
|
57
|
+
|
|
58
|
+
A `fail` rule that also has `cantTell` occurrences reports the failures in `<failure>` and the undecided ones in `<system-out>`, where dashboards show test output. A skipped `cantTell` rule does the same.
|
|
59
|
+
|
|
60
|
+
## Options
|
|
61
|
+
|
|
62
|
+
| Option | Default | Effect |
|
|
63
|
+
|---|---|---|
|
|
64
|
+
| `cantTellAs` | `'skipped'` | `'failure'` reports `cantTell` rules as `<failure type="cantTell">`, for a pipeline that must not pass while anything is undecided. |
|
|
65
|
+
| `includeNotApplicable` | `false` | Include `notApplicable` rules as skipped tests. |
|
|
66
|
+
| `baselineEntries` | none | Entries from [`BASELINE.md`](./BASELINE.md)'s `buildBaselineEntries()`. Fail occurrences recorded there are dropped, exactly as in SARIF. A rule whose every failure is already known is `<skipped message="N known failures recorded in the baseline">`, not passing: it did not pass. |
|
|
67
|
+
| `name` | `'surea11y'` | The `name` of the root `<testsuites>`, for telling several pages' reports apart in one dashboard. |
|
|
68
|
+
|
|
69
|
+
## Determinism
|
|
70
|
+
|
|
71
|
+
The engine has no clock, and this report does not invent one: every `time` is `"0"`, and a `timestamp` attribute appears on each suite only when the result carries one (`engineOptions.timestamp`). The same scan always renders byte-identical XML, so a report can be committed or diffed.
|
|
72
|
+
|
|
73
|
+
Text is escaped for XML, and characters XML 1.0 forbids even when escaped (most control characters, lone surrogates) are dropped, since markup captured from a page can contain them.
|
package/docs/LIMITATIONS.md
CHANGED
|
@@ -17,6 +17,7 @@ surea11y is a **static DOM scan**: it reads the DOM tree and computed styles at
|
|
|
17
17
|
- **`<dialog>` and other elements hidden by the default UA stylesheet** (no `open` attribute, `display: none` by spec), along with any other subtree hidden via `display:none`, `visibility:hidden`, `[hidden]`, or closed `<details>`, are **excluded from rule evaluation by default** — matching the visibility-aware behavior of other established engines. This is a deliberate default (`engineOptions.includeHiddenElements: false`), not an oversight: hidden content isn't reachable by assistive technology or keyboard until it's shown, so flagging a markup defect inside it by default would often be noise. Set `engineOptions.includeHiddenElements: true` to evaluate hidden/collapsed subtrees anyway — e.g. to catch a markup defect (like a broken ARIA ID reference) before a dialog ever opens. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#engineoptions--the-rest) for the option and exactly which hiding mechanisms it covers.
|
|
18
18
|
- **jsdom's computed `text-shadow` is unreliable on a second read of the same element.** Confirmed in jsdom 29.1.1: reading a computed `text-shadow` value a second time on the same element — through any accessor, from any freshly-requested `CSSStyleDeclaration` for that element, regardless of caching — silently returns a different, wrong "no shadow" value instead of the real declared one. The first read is always correct. This engine works around it internally by reading each element's `text-shadow` exactly once per run and caching the result (see `__textShadowInfoEl` in `src/core/contrast-helpers.js`), so a single scan is unaffected. It only surfaces if you read `getComputedStyle(el).textShadow` yourself, more than once, against the same jsdom-parsed element — a real browser has no such bug.
|
|
19
19
|
- **Under jsdom, scan time grows with the square of DOM depth, not with element count.** jsdom resolves inherited CSS by walking an element's ancestor chain on every `getComputedStyle` call, so one call per element costs O(elements x depth). Measured on jsdom 29.1.1 with no engine code involved: 4000 elements in one chain take 8.3s of `getComputedStyle` alone, against 0.25s for the same 4000 as siblings. The engine's own ancestor walks are capped and stay linear, so this is jsdom's cost rather than the rules'. It affects `runDomRulesInPage` and anything built on it, including `@surea11y/test-matchers`; a real browser computes inherited style natively and does not have this shape. Component frameworks routinely nest 100-300 deep, which is comfortably fast — it becomes noticeable past roughly 1000.
|
|
20
|
+
- **In a real browser, some rules change the page for the time of their check, and the page's own scripts can react.** To measure what the browser draws, `css-hidden-focus` focuses elements as the keyboard would, and switches transitions off on the element it measures; `text-spacing-content-loss` adds a style sheet with the WCAG 1.4.12 spacing. All of this is put back before the rule returns: focus, scroll position, style attributes and the added style sheet. What cannot be put back is what the page's scripts do in response, since focusing an element fires its `focus` handlers: a carousel may move to the focused slide or enable a button. The findings are not affected, but if you scan a page you go on using, reload it afterwards.
|
|
20
21
|
- **Static markup vs. live/post-hydration DOM state.** The rule logic itself is DOM-source-agnostic — it evaluates whatever DOM it's handed, whether that's jsdom-parsed static HTML (Pattern 1) or an already-loaded, already-hydrated real browser tab (Pattern 2, see [`INTEGRATION.md`](./INTEGRATION.md)). But the CLI (`npx @surea11y/cli scan <url>`) specifically fetches static HTML only, with no JS execution — see [the CLI docs](https://github.com/SureA11y/cli/blob/main/docs/CLI.md). For a JS-framework-hydrated widget whose server-rendered markup intentionally ships one state before client JS syncs it (e.g. `<input type="checkbox" aria-checked="true">` shipped before client JS sets the native `checked` property to match on hydration — an extremely common, entirely legitimate pattern), a CLI scan only sees the pre-hydration markup. A scan running inside an actual loaded browser tab sees the post-hydration state instead, so the two can disagree on exactly this class of element for reasons that have nothing to do with rule correctness. That's why `aria-checked-state-mismatch` is capped at `manual`/`cantTell` rather than a hard `fail`. If you need live-DOM accuracy for hydration-sensitive checks, run the library directly against an already-loaded page via Pattern 2, not the static-fetch CLI.
|
|
21
22
|
|
|
22
23
|
## Not attempted: judgment calls that aren't automatable safely
|
package/docs/OUTPUT_SCHEMA.md
CHANGED
|
@@ -19,7 +19,9 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
19
19
|
tag: string,
|
|
20
20
|
schemaVersion: string,
|
|
21
21
|
locale: { requested: string, resolved: string, reason: string },
|
|
22
|
-
wcagVersion: "2.0" | "2.1" | "2.2"
|
|
22
|
+
wcagVersion: "2.0" | "2.1" | "2.2",
|
|
23
|
+
profile?: string, // "wcag22-aa", "en301549-v4.1.1", "en301549-v3.2.1", "section508", or a registered standard's own
|
|
24
|
+
mappings?: string[] // e.g. ["en301549"] or ["en301549:V3.2.1"]
|
|
23
25
|
},
|
|
24
26
|
url: string | null,
|
|
25
27
|
title: string | null,
|
|
@@ -37,15 +39,19 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
37
39
|
| `engine.tag` | The engine's own identity tag, currently `"a11ycore"`. Every rule (built-in or custom) carries it in `meta.tags` — rule `ruleId`s themselves are bare (no prefix). |
|
|
38
40
|
| `engine.schemaVersion` | The result-schema version (`"1.0.0"`). Bump-worthy if this document's shape ever changes incompatibly — pin to it if you're parsing output programmatically. See [`API_STABILITY.md`](./API_STABILITY.md) for the full stable/unstable field list and version-bump policy. |
|
|
39
41
|
| `engine.locale` | Which dictionary the run actually used. `requested` is your `engineOptions.locale` after trimming (`"en"` if you passed nothing or a non-string); `resolved` is the locale whose dictionary was used; `reason` explains the pairing. Because locale fallback is graceful and per-string, asking for a language the build does not carry produces English text rather than an error — this field is how you find that out without reading the strings. Reported once per result: a run uses one dictionary throughout. |
|
|
40
|
-
| `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key. `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
|
|
42
|
+
| `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key (a profile's messages in a language that profile does not offer show in English, by its choice, and do not count; see [`I18N.md`](./I18N.md)). `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
|
|
41
43
|
| `engine.wcagVersion` | Which version of WCAG this run was conformance-tested against: your `engineOptions.wcagVersion`, or what your version-origin tags implied, or the default `"2.2"`. It affects one thing today — a rule mapped only to SC 4.1.1 Parsing cannot `fail` under a 2.2 target (see `checksResults[i].wcagVersionScope` below). Reported once per result: a run has one target throughout. |
|
|
44
|
+
| `engine.profile` | Present only when an `engineOptions.profile` selected this run's rules, and names the profile, lowercased. Absent when none was asked for, and also when one was asked for but did not apply (unknown name, or an include in `runOnly` or `engineOptions` chose the rules instead), so its presence is how you confirm a run really targeted that profile. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
|
|
45
|
+
| `engine.profileExcludes` | Present only when the applied profile leaves rules out (`exclude` in its standard's registry entry): `{ rules, criteria }`, the rules it names and the WCAG criteria it waives. Their rules and WCAG rollups did not run. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
|
|
46
|
+
| `engine.optInRules` | Present only when `engineOptions.optInRules` ran at least one opt-in rule that the rest of the selection would not have run. Lists their tags. Its presence means the result includes rules for requirements beyond the targeted standard, such as a national standard's, so a failure may not be a WCAG failure. Absent under a WCAG profile, where unlocking selects nothing, and when a standard's own profile ran its rules. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#running-every-rule-optinrules). |
|
|
47
|
+
| `engine.mappings` | Present only when the run names a standard besides WCAG in `meta.normativeMappings`, through `engineOptions.mappings` or a standard's profile. Lists them canonically, in table order: `"en301549"` for every version, `"en301549:V3.2.1"` for one. Absent means every result names WCAG only. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#other-standards-mappings). |
|
|
42
48
|
| `url` | The `pageUrl` argument you passed in, or `document.location.href` if you passed `null`/omitted it, or `null` if neither is available. |
|
|
43
49
|
| `title` | `document.title` at scan time, or `null`. |
|
|
44
50
|
| `timestamp` | **Not auto-generated.** Only set if you pass `engineOptions.timestamp` as a non-empty string — the engine has no built-in clock (deterministic-by-design). If you want a scan timestamp in the result, supply it yourself. |
|
|
45
51
|
| `perfStats` | `null` unless `engineOptions.perfStats: true`. Internal timing/counters — shape not covered by this document, treat as debug-only. |
|
|
46
52
|
| `contextSelector` | The (trimmed) `contextSelector` argument you passed — a string, an array of strings (multi-region scanning, see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)), or `null` if none/empty. |
|
|
47
53
|
| `checksResults` | One entry per **atomic rule** that ran (every rule not filtered out by `runOnly` — see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). **Every loaded rule produces an entry, even ones that outcome `notApplicable`** — this is not a "violations only" list. |
|
|
48
|
-
| `rulesResults` | One entry per **composite (
|
|
54
|
+
| `rulesResults` | One entry per **composite (rollup) rule** that ran — see [Composite result](#a-composite-result-rulesresultsi) and [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md). Normally one per WCAG Success Criterion; a run under the profile of a standard with rollups of its own also gets those (`meta.standard` set). Empty array if no composite matched the current `runOnly`/tag filter. |
|
|
49
55
|
| `overriddenBuiltinIds` | Rule ids where an `engineOptions.customRules` entry shared its `id` with a built-in rule, so the custom implementation replaced the built-in one for this scan (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Always an array; empty when no collision occurred. Also logged via `console.warn` at scan time, since a same-named custom rule is as likely to be an accidental collision as a deliberate override. |
|
|
50
56
|
|
|
51
57
|
## Cross-frame result (`runa11yCoreAcrossFrames`)
|
|
@@ -87,7 +93,7 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
87
93
|
normative: boolean,
|
|
88
94
|
atomic: boolean,
|
|
89
95
|
category: "perceivable" | "operable" | "understandable" | "robust" | null,
|
|
90
|
-
normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel
|
|
96
|
+
normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel?: string, wcagSc?: string[] }>,
|
|
91
97
|
standard: string | null,
|
|
92
98
|
applicability: string,
|
|
93
99
|
expectation: string,
|
|
@@ -97,6 +103,8 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
97
103
|
},
|
|
98
104
|
engineOptions: object, // the resolved engineOptions this rule actually ran under
|
|
99
105
|
schemaVersion: string,
|
|
106
|
+
rollupIds: string[], // the rulesResults entries that group this rule in this run; [] if none
|
|
107
|
+
data?: object, // page-level data a rule reports whatever its outcome; see below
|
|
100
108
|
wcagVersionScope?: { // present only when the target WCAG version changed this outcome
|
|
101
109
|
target: "2.0" | "2.1" | "2.2",
|
|
102
110
|
removedSc: string[],
|
|
@@ -110,8 +118,10 @@ Notes:
|
|
|
110
118
|
|
|
111
119
|
- **`outcome` vs `outcomeNormalized`**: identical except `notApplicable` becomes `"inapplicable"` in `outcomeNormalized`. Both are provided so you can match either your own vocabulary or the engine's internal one.
|
|
112
120
|
- **`type: "manual"` rules can never report `outcome: "fail"`.** If a manual rule's own logic would have said `fail`, the engine coerces it to `cantTell` and appends an explanatory note to `error` — this is enforced centrally (`policy.coerceManualFailToCantTell`, on by default under the `a11y` policy contract; see [`POLICY.md`](./POLICY.md)), not something each rule has to remember. `fail` is reserved for deterministic, `type: "automatic"` findings only.
|
|
113
|
-
- **`
|
|
121
|
+
- **`rollupIds`** lists the composites in `rulesResults` that group this rule in this run. An empty list means the rule's findings appear in no rollup, so a consumer that reads only `rulesResults` never sees them; `heading-order`, for instance, belongs to no WCAG rollup.
|
|
122
|
+
- **`meta.normativeMappings`** is how a check result ties back to a WCAG Success Criterion — `[]` for rules with no formal WCAG mapping (this engine calls them advisory `type: "manual"` rules). See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md) for how these roll up. The list is not WCAG-only. When the scan asks for another standard (`engineOptions.mappings`, or a standard's profile), each WCAG criterion is followed by that standard's corresponding requirement, such as the EN 301 549 clause that restates it (`{ standard: "EN 301 549", version: "V3.2.1" | "V4.1.1", requirement: "9.1.1.1", title, wcagSc: ["1.1.1"] }`, one entry per version that includes the criterion, `wcagSc` naming the criteria it corresponds to), or the requirements of a standard mapped rule by rule that the rule checks (same shape, `wcagSc` empty for a requirement WCAG does not make, and any fields of that standard's own); and a rule may also cite WCAG's Understanding documents (`type: "Understanding"`). Filter on `standard` (and on the absence of `type`) before reading `requirement` as a Success Criterion. A composite's WCAG entry is always first. See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md#en-301-549).
|
|
114
123
|
- **`wcagVersionScope`**: only present when the run's target WCAG version turned this rule's `fail` into a `cantTell` — today that means a rule mapped to SC 4.1.1 Parsing (`duplicate-id`) under the default 2.2 target, since 2.2 removed that criterion. `removedSc` lists the criteria that stopped existing, `target` is the version that removed them, and `coercedFrom` is the outcome the rule itself reported. The occurrences are the rule's own, unchanged — nothing was dropped, only the conformance verdict was. Absent on every other result, and **never** reported through `error`: nothing went wrong. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#filtering-by-wcag-version-21-vs-22).
|
|
124
|
+
- **`data`**: present only on a rule that reports something about the whole page, whatever its outcome. No core rule does today; a profile's rule may, such as one returning the page's own entry for a probe that compares pages (see `probes` in [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Like `data.details` on an occurrence, it is not a stable contract.
|
|
115
125
|
- **`error`**: only present if the rule implementation threw an uncaught exception, or if the manual-fail coercion above fired. A thrown rule always surfaces as `outcome: "cantTell"` with `occurrences: []` and `error` set to the exception message — the engine never lets one broken rule crash the whole scan.
|
|
116
126
|
- **`engineOptions`** on each result is the *resolved* options object (after locale/contrast defaults were applied), not literally what you passed in — useful for confirming what a given rule actually saw, especially the resolved `locale` and `contrast.mode`/`contrast.rootCanvasFallback`.
|
|
117
127
|
|
|
@@ -175,7 +185,7 @@ Every automatic rule that can report `cantTell` carries this, and a test holds t
|
|
|
175
185
|
|
|
176
186
|
## A composite result (`rulesResults[i]`)
|
|
177
187
|
|
|
178
|
-
Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Shape is the same envelope as a check result, with composite-specific `data.details`:
|
|
188
|
+
Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Under the profile of a standard with rollups of its own, there are also those, with the standard's wording as `title` and its name as `meta.standard`; they may be the only rollup some of its findings have, such as `heading-order`'s. Shape is the same envelope as a check result, with composite-specific `data.details`:
|
|
179
189
|
|
|
180
190
|
```ts
|
|
181
191
|
{
|
|
@@ -186,6 +196,9 @@ Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wc
|
|
|
186
196
|
data: {
|
|
187
197
|
details: {
|
|
188
198
|
reasonCode: string, // e.g. "composite.rollup.fail.anyFail"
|
|
199
|
+
standard?: string, // a standard's own rollup only, with version and criterion
|
|
200
|
+
version?: string,
|
|
201
|
+
criterion?: string,
|
|
189
202
|
checksIds: string[], // every atomic ruleId this composite rolls up
|
|
190
203
|
contributors: Array<{ testId: string, outcome: string, severity: string | null }>,
|
|
191
204
|
metrics: { failCount, cantTellCount, notApplicableCount, passCount, missingCount }
|
package/docs/REPORT.md
CHANGED
|
@@ -12,10 +12,15 @@ Open `report.html` directly from disk. Works alongside any other output mode —
|
|
|
12
12
|
|
|
13
13
|
- **Hero**: one plain-language headline ("N of M applicable checks passed") plus a stacked bar and legend (icon + label + count — status is never color-only) broken down by outcome (`fail`/`cantTell`/`pass`/`notApplicable`).
|
|
14
14
|
- **Worth reviewing**: one card per rule with `fail`/`cantTell` occurrences (not one per occurrence — a rule with many identical occurrences is one thing worth attention, not many), each showing severity, WCAG SC chip(s), a representative occurrence, and the total occurrence count. Capped at the 24 highest-priority rules with an overflow note past that.
|
|
15
|
-
- **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion, its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
|
|
15
|
+
- **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion (with the EN 301 549 clause that restates it, where there is one and the scan asked for EN 301 549 clauses), its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
|
|
16
|
+
- **A standard's own rollup**: only when the scan ran the rules of a standard that has rollups of its own (its profile, or `engineOptions.optInRules`), one row per rollup, with the standard's wording, the requirements of the rules that decided the outcome, the breakdown and the contributing rules.
|
|
16
17
|
- **Full technical data** (collapsed by default): a scorecard (tiles per outcome) and a searchable, filterable (by outcome), paginated table of every individual occurrence across the whole scan.
|
|
17
18
|
|
|
18
|
-
The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
|
|
19
|
+
The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, the WCAG version the scan targeted the `engineOptions.profile` it used (when there was one), the opt-in rule tags `engineOptions.optInRules` added (when it added any), and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
|
|
20
|
+
|
|
21
|
+
The whole page is written in the locale the scan resolved to: headings, table columns, outcome and severity names, the headline, the pager, and the date format, all from the `report_*` keys in the same dictionaries as the findings. `<html lang>` names that locale. The outcome codes in the occurrence filters (`fail`, `cantTell`, …) stay as they are, because they are the values a reader searches for in the JSON result.
|
|
22
|
+
|
|
23
|
+
One case keeps English labels: a scan in a language the engine does not ship, run with a caller-supplied `engineOptions.messages` dictionary. That dictionary is not part of the result, so the report cannot read labels from it; its findings are then marked with their own language (`lang="nl"`, for instance), so a screen reader still reads them with the right voice.
|
|
19
24
|
|
|
20
25
|
## Library usage
|
|
21
26
|
|
package/docs/RULE_AUTHORING.md
CHANGED
|
@@ -114,7 +114,7 @@ const meta = {
|
|
|
114
114
|
title: 'Non-text Content',
|
|
115
115
|
conformanceLevel: 'A'
|
|
116
116
|
}
|
|
117
|
-
],
|
|
117
|
+
], // WCAG only: EN 301 549 clauses are derived at build time (WCAG_CONFORMANCE.md#en-301-549)
|
|
118
118
|
|
|
119
119
|
defaultSeverity: 'minor' | 'moderate' | 'serious' | 'critical',
|
|
120
120
|
category: 'perceivable' | 'operable' | 'understandable' | 'robust',
|
|
@@ -141,8 +141,9 @@ The build/runtime resolves i18n by:
|
|
|
141
141
|
2) falling back to `en` if missing,
|
|
142
142
|
3) falling back to the literal `title`/`description` strings if still missing.
|
|
143
143
|
|
|
144
|
-
Add the key and its English text to `src/i18n/en.json
|
|
145
|
-
`npm run i18n:sync` so every other
|
|
144
|
+
Add the key and its English text to `src/i18n/en.json` (a profile's rule:
|
|
145
|
+
`profiles/<name>/i18n/en.json`), then run `npm run i18n:sync` so every other
|
|
146
|
+
locale picks it up. `npm test` fails if you
|
|
146
147
|
forget. See [`I18N.md`](./I18N.md).
|
|
147
148
|
|
|
148
149
|
#### `meta.tags`
|
|
@@ -150,6 +151,35 @@ Tags are used for grouping/filtering. Typical tag families in this ruleset inclu
|
|
|
150
151
|
- WCAG tagging: `wcag2a`, `wcag111`
|
|
151
152
|
- domain: `nontext`, `images`, plus element-specific tags
|
|
152
153
|
- nature: `atomic`, plus `automatic` or `manual`
|
|
154
|
+
- another standard's own requirement: that standard's rule tag (see below)
|
|
155
|
+
|
|
156
|
+
#### Rules for another standard's own requirements
|
|
157
|
+
A rule that checks something WCAG does not require, but another standard does (a doctype or presentational attributes, say), declares no WCAG mapping (`wcagSc: []`, `normativeMappings: []`) and carries that standard's rule tag. The tag makes it **opt-in**: it runs only under the standard's profile, a selection that includes the tag, or its own id, never in a default or WCAG run ([`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#opt-in-rules)). That is what lets it report `fail`: its failures are failures of that standard, and only a scan targeting it sees them. Its module goes in that standard's profile rather than in `src/checks/`: `profiles/<key>/rules/automatic/` or `profiles/<key>/rules/manual/`. The build compiles it into the engine like any other rule. Its test and scenario page go in the profile too, in `profiles/<key>/tests/rules/` and `profiles/<key>/tests/fixtures/`. Map it to the standard's requirements the usual way (a row in the profile's rule map). Rule tags come from each standard's `ruleTag` in the registry, `src/coverage/standards.js` (a profile's from its `index.js`). The sample profile core's tests run against, `tests/fixtures/profiles/sample/`, has two such rules.
|
|
158
|
+
|
|
159
|
+
#### Rule variants
|
|
160
|
+
When another standard's requirement is a core rule with different thresholds (contrast at 7:1, say, or bold text large from 18.5px rather than WCAG's 14pt), write it as a **variant**, not a copy. The core rule declares the thresholds it reads from `ctx.config` as `settings`, with WCAG's values as defaults:
|
|
161
|
+
|
|
162
|
+
```js
|
|
163
|
+
// src/checks/automatic/contrast-minimum.js
|
|
164
|
+
const settings = { boldLargeMinPx: null, largeTextRatio: 3, normalTextRatio: 4.5 };
|
|
165
|
+
module.exports = { id, meta, runInPage, settings };
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
The variant is data, in the standard's profile:
|
|
169
|
+
|
|
170
|
+
```js
|
|
171
|
+
// profiles/<key>/rules/automatic/sample-contrast-enhanced.js
|
|
172
|
+
module.exports = {
|
|
173
|
+
id: 'sample-contrast-enhanced',
|
|
174
|
+
from: 'contrast-minimum',
|
|
175
|
+
config: { normalTextRatio: 7, largeTextRatio: 4.5 },
|
|
176
|
+
meta: { /* its own title, description, i18n, tags... as any rule's */ }
|
|
177
|
+
};
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
The build runs the base rule's `runInPage` and `applicability` under the variant's id and meta, with its `config` in `ctx.config`. A rule's settings are never the caller's: the runner drops a caller's value for one (`engineOptions.rules[ruleId]`), so the base rule always runs at its defaults and the variant at its `config`, while the caller's other config, such as `excludeSelectors`, still applies. A message key of the base's that starts with the base's prefix (its `meta.i18n.titleKey` without `_title`, `contrastMinimum`) is read from the variant's prefix instead (`sampleContrastEnhanced`), so the variant's dictionary has the same keys under its own prefix; the rule validator checks they exist. A fix to the base reaches every variant. The build refuses a variant whose base does not exist, is itself a variant, or declares no `settings`, and a setting the base does not declare or of another type. A base rule that caches verdicts depending on its settings keys those caches by them, as `contrast-minimum` does.
|
|
181
|
+
|
|
182
|
+
Add a setting to a core rule when a standard needs it, with a default that keeps the rule's behaviour; the settings a rule declares are for its variants, not for callers, and stay outside semver until a profile can live outside this repository (see [`API_STABILITY.md`](./API_STABILITY.md#explicitly-unstable-not-covered-by-semver)).
|
|
153
183
|
|
|
154
184
|
#### `meta.coverage.facetsBySc`
|
|
155
185
|
This is the repo’s explicit **coverage model** for an SC.
|
|
@@ -317,7 +347,23 @@ The rule must return:
|
|
|
317
347
|
|
|
318
348
|
Examples:
|
|
319
349
|
|
|
320
|
-
### 8.2
|
|
350
|
+
### 8.2 What `ctx` carries
|
|
351
|
+
|
|
352
|
+
`runInPage(ctx)` and `applicability(ctx)` receive the same object, built-in and custom rules alike:
|
|
353
|
+
|
|
354
|
+
| Field | What it is |
|
|
355
|
+
|---|---|
|
|
356
|
+
| `document`, `window` | The page being scanned. |
|
|
357
|
+
| `root` | The roots the scan covers: the document, or what `contextSelector` resolved to. |
|
|
358
|
+
| `contextSelector` | The selector that scoped the run, if any. |
|
|
359
|
+
| `rule` | The rule's resolved definition: `ruleId`, `defaultSeverity`, `defaultConfidence`, `type`, `meta`... |
|
|
360
|
+
| `config` | `engineOptions.rules[ruleId]`, this rule's settings, if the caller gave any (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). |
|
|
361
|
+
| `standard` | The standard and version the run targets, `{ key, name, version }` (`{ key: 'en301549', name: 'EN 301 549', version: 'V4.1.1' }`), when a standard's profile selected the run; `null` otherwise (no profile, a WCAG profile, or rules chosen by tag or id). A rule whose behaviour differs between versions of its standard reads it here, and does what holds for every version when it is `null`. |
|
|
362
|
+
| `helpers` | The helpers documented in [`RULE_HELPERS.md`](./RULE_HELPERS.md). |
|
|
363
|
+
| `engineOptions` | The scan's options as resolved. |
|
|
364
|
+
| `inputs.probes` | Evidence the host application supplied (`engineOptions.probes`). |
|
|
365
|
+
|
|
366
|
+
### 8.3 Outcome conventions used by these rules
|
|
321
367
|
|
|
322
368
|
Automatic:
|
|
323
369
|
- `notApplicable` if no applicable targets
|
|
@@ -385,7 +431,8 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
|
|
|
385
431
|
### 11.1 The fixture file
|
|
386
432
|
|
|
387
433
|
- Path: `tests/fixtures/<rule-slug>-all-scenarios.html`, where `<rule-slug>` is the rule
|
|
388
|
-
id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`).
|
|
434
|
+
id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`). A
|
|
435
|
+
profile's rule keeps it in the profile: `profiles/<name>/tests/fixtures/`.
|
|
389
436
|
- Structure: a real HTML page (`<!doctype html>`, `<title>`, minimal inline `<style>`)
|
|
390
437
|
containing numbered scenario blocks, each:
|
|
391
438
|
```html
|
|
@@ -405,6 +452,24 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
|
|
|
405
452
|
(eligible, FAIL)", "C. Ineligible (excluded from accessibility tree, skipped)").
|
|
406
453
|
- Cover every branch the rule's own logic distinguishes: pass, fail (each distinct
|
|
407
454
|
`reasonCode`), notApplicable/skipped, and — for manual rules — cantTell.
|
|
455
|
+
- A whole-document rule (`page-title-present`, `meta-refresh-timing-absent`, `region`)
|
|
456
|
+
can only demonstrate one outcome per page. Its fixture declares a single bare
|
|
457
|
+
`.case-title` with no `.case` wrapper, and the page itself is the case; the marker is
|
|
458
|
+
compared against the rule-level outcome, so `PASS` and `NEUTRAL` are distinguished
|
|
459
|
+
there. Cover the remaining branches with inline tests rather than near-identical
|
|
460
|
+
fixture files.
|
|
461
|
+
- One fixture shared by several rules that expect different things of the same case
|
|
462
|
+
(`tests/fixtures/contrast-all-scenarios.html` serves `contrast-minimum`,
|
|
463
|
+
`contrast-enhanced` and `contrast-computable`) carries a per-rule marker as a
|
|
464
|
+
`data-outcome-<rule-id>` attribute on the `.case`, which overrides the shared
|
|
465
|
+
`.case-title` for that rule. Use an attribute rather than more text when the rules
|
|
466
|
+
under test evaluate text: a `.case-title` added to a contrast case is one more text
|
|
467
|
+
node to check. A marker word the parser does not recognise (`MIXED`, `UNSTATED`)
|
|
468
|
+
asserts nothing, for a case whose outcome the fixture does not state.
|
|
469
|
+
- `npm run fixtures:markers:check` replays every fixture and fails when a marker no
|
|
470
|
+
longer matches what the rule reports; `scripts/data/fixture-markers.json` records the
|
|
471
|
+
cases that already disagree, so that set can only shrink. A profile's rules have
|
|
472
|
+
their record in the profile's own `scripts/data/fixture-markers.json`.
|
|
408
473
|
|
|
409
474
|
### 11.2 Known, acceptable exceptions to "one fixture, many cases"
|
|
410
475
|
|
|
@@ -480,7 +545,9 @@ npm run fixtures:index
|
|
|
480
545
|
This writes `tests/fixtures/INDEX.md` (human-readable), `tests/fixtures/index.json`
|
|
481
546
|
(machine-readable — every rule, its fixture path, and parsed pass/fail/cantTell case
|
|
482
547
|
counts, for external tooling to enumerate and load fixtures directly) and
|
|
483
|
-
`tests/fixtures/index.html` (the same listing as a browsable page).
|
|
548
|
+
`tests/fixtures/index.html` (the same listing as a browsable page). A profile's rules
|
|
549
|
+
get the same three files in the profile's `tests/fixtures/`, with paths relative to the
|
|
550
|
+
profile's folder. Commit all three
|
|
484
551
|
alongside the fixture and test changes. A rule shipped without its fixture is treated
|
|
485
552
|
the same as a rule shipped without tests — not done. `npm run fixtures:check` reports
|
|
486
553
|
a stale index without rewriting it, and CI fails on one.
|