@surea11y/core 1.6.0 → 1.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (120) hide show
  1. package/CHANGELOG.md +140 -0
  2. package/README.md +179 -90
  3. package/docs/ACT_RULE_MAPPING.md +10 -8
  4. package/docs/API_STABILITY.md +67 -6
  5. package/docs/BINDING_AUTHORS_GUIDE.md +104 -2
  6. package/docs/CI_INTEGRATIONS.md +43 -0
  7. package/docs/DESIGN_CHALLENGES.md +162 -2
  8. package/docs/EARL.md +100 -0
  9. package/docs/ENGINE_OPTIONS.md +109 -5
  10. package/docs/I18N.md +62 -20
  11. package/docs/INTEGRATION.md +4 -2
  12. package/docs/JUNIT.md +73 -0
  13. package/docs/LIMITATIONS.md +4 -1
  14. package/docs/OUTPUT_SCHEMA.md +62 -11
  15. package/docs/POLICY.md +1 -1
  16. package/docs/REPORT.md +7 -2
  17. package/docs/RULE_AUTHORING.md +83 -17
  18. package/docs/RULE_CATALOG.md +212 -139
  19. package/docs/RULE_EXAMPLES.md +2189 -0
  20. package/docs/RULE_HELPERS.md +390 -0
  21. package/docs/RULE_TAXONOMY.md +27 -6
  22. package/docs/SARIF.md +23 -3
  23. package/docs/WCAG_CONFORMANCE.md +64 -3
  24. package/package.json +41 -12
  25. package/profiles/index.js +14 -0
  26. package/src/checks/automatic/area-alt-present.js +87 -31
  27. package/src/checks/automatic/aria-allowed-attr.js +6 -0
  28. package/src/checks/automatic/aria-allowed-role.js +32 -23
  29. package/src/checks/automatic/aria-braille-equivalent.js +43 -17
  30. package/src/checks/automatic/aria-conditional-attr.js +17 -10
  31. package/src/checks/automatic/aria-deprecated-role.js +12 -0
  32. package/src/checks/automatic/aria-hidden-body.js +1 -1
  33. package/src/checks/automatic/aria-hidden-focus.js +74 -18
  34. package/src/checks/automatic/aria-prohibited-attr.js +22 -4
  35. package/src/checks/automatic/aria-prohibited-children.js +6 -6
  36. package/src/checks/automatic/aria-required-attr.js +88 -12
  37. package/src/checks/automatic/aria-required-children.js +33 -16
  38. package/src/checks/automatic/aria-required-parent.js +32 -6
  39. package/src/checks/automatic/aria-role-name-present.js +20 -3
  40. package/src/checks/automatic/aria-roles-valid.js +52 -21
  41. package/src/checks/automatic/aria-valid-attr-value.js +89 -24
  42. package/src/checks/automatic/aria-valid-attr.js +14 -9
  43. package/src/checks/automatic/autocomplete-valid.js +152 -26
  44. package/src/checks/automatic/avoid-inline-spacing.js +207 -15
  45. package/src/checks/automatic/button-name-present.js +2 -1
  46. package/src/checks/automatic/canvas-text-alternative-present.js +105 -14
  47. package/src/checks/automatic/combobox-name-present.js +34 -51
  48. package/src/checks/automatic/contrast-computable.js +45 -4
  49. package/src/checks/automatic/contrast-enhanced.js +16 -4
  50. package/src/checks/automatic/contrast-minimum.js +57 -11
  51. package/src/checks/automatic/css-orientation-lock.js +171 -12
  52. package/src/checks/automatic/definition-list-children-valid.js +67 -23
  53. package/src/checks/automatic/deprecated-elements-not-used.js +43 -38
  54. package/src/checks/automatic/dialog-name-present.js +28 -9
  55. package/src/checks/automatic/duplicate-id-aria.js +5 -0
  56. package/src/checks/automatic/duplicate-id.js +19 -10
  57. package/src/checks/automatic/form-control-single-label.js +9 -0
  58. package/src/checks/automatic/identical-iframes-same-purpose.js +229 -0
  59. package/src/checks/automatic/iframe-focusable-content.js +12 -4
  60. package/src/checks/automatic/iframe-title-unique.js +36 -81
  61. package/src/checks/automatic/input-image-alt-present.js +32 -20
  62. package/src/checks/automatic/label-in-name.js +78 -69
  63. package/src/checks/automatic/language-page-present.js +12 -6
  64. package/src/checks/automatic/link-in-text-block.js +512 -44
  65. package/src/checks/automatic/link-name-present.js +13 -5
  66. package/src/checks/automatic/list-children-valid.js +18 -1
  67. package/src/checks/automatic/listbox-name-present.js +19 -49
  68. package/src/checks/automatic/listitem-parent-valid.js +4 -3
  69. package/src/checks/automatic/meta-refresh-no-exceptions.js +3 -3
  70. package/src/checks/automatic/page-title-present.js +16 -4
  71. package/src/checks/automatic/progressbar-name-present.js +11 -1
  72. package/src/checks/automatic/{role-img-alt-present.js → role-img-text-alternative-present.js} +9 -5
  73. package/src/checks/automatic/searchbox-name-present.js +32 -49
  74. package/src/checks/automatic/server-side-image-map-absent.js +48 -28
  75. package/src/checks/automatic/slider-name-present.js +38 -52
  76. package/src/checks/automatic/spinbutton-name-present.js +32 -49
  77. package/src/checks/automatic/target-size-minimum.js +84 -16
  78. package/src/checks/automatic/td-has-header.js +60 -23
  79. package/src/checks/automatic/text-spacing-content-loss.js +548 -0
  80. package/src/checks/automatic/textbox-name-present.js +32 -49
  81. package/src/checks/automatic/valid-lang.js +15 -10
  82. package/src/checks/manual/area-alt-quality-manual.js +113 -31
  83. package/src/checks/manual/canvas-text-alternative-quality-manual.js +0 -2
  84. package/src/checks/manual/css-focus-indicator-suppressed-manual.js +23 -10
  85. package/src/checks/manual/css-hidden-focus.js +215 -7
  86. package/src/checks/manual/form-control-label-quality-manual.js +243 -29
  87. package/src/checks/manual/heading-order-manual.js +9 -1
  88. package/src/checks/manual/heading-quality-manual.js +143 -9
  89. package/src/checks/manual/img-alt-decorative-manual.js +6 -3
  90. package/src/checks/manual/input-image-alt-quality-manual.js +81 -13
  91. package/src/checks/manual/landmark-complementary-is-top-level-manual.js +231 -0
  92. package/src/checks/manual/link-name-quality-manual.js +130 -4
  93. package/src/checks/manual/media-transcript-present-manual.js +65 -8
  94. package/src/checks/manual/mouse-only-event-handlers-manual.js +66 -5
  95. package/src/checks/manual/no-autoplay-audio-manual.js +93 -6
  96. package/src/checks/manual/p-as-heading-manual.js +89 -44
  97. package/src/checks/manual/page-title-patterns-manual.js +77 -8
  98. package/src/checks/manual/password-paste-enabled-manual.js +255 -0
  99. package/src/checks/manual/skip-link-manual.js +42 -14
  100. package/src/checks/manual/table-fake-caption-manual.js +32 -1
  101. package/src/checks/manual/video-caption-manual.js +47 -24
  102. package/src/checks/manual-review.js +0 -4
  103. package/src/core.js +18061 -46194
  104. package/src/coverage/en301549-map.js +187 -0
  105. package/src/coverage/standards.js +279 -0
  106. package/src/coverage/wcag-facets.js +1119 -0
  107. package/src/coverage/wcag-version-map.js +101 -0
  108. package/src/earl.js +144 -0
  109. package/src/en301549.js +33 -0
  110. package/src/junit.js +321 -0
  111. package/src/profile-kit.js +163 -0
  112. package/src/report.js +343 -74
  113. package/src/sarif.js +56 -5
  114. package/src/wcag.js +105 -0
  115. package/surea11y.browser.js +11 -41039
  116. package/surea11y.i18n.de.js +2 -21
  117. package/surea11y.i18n.es.js +2 -21
  118. package/surea11y.i18n.fr.js +2 -21
  119. package/surea11y.i18n.ja.js +3 -0
  120. package/src/checks/manual/area-alt-decorative-manual.js +0 -255
@@ -18,7 +18,10 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
18
18
  engine: {
19
19
  tag: string,
20
20
  schemaVersion: string,
21
- locale: { requested: string, resolved: string, reason: string }
21
+ locale: { requested: string, resolved: string, reason: string },
22
+ wcagVersion: "2.0" | "2.1" | "2.2",
23
+ profile?: string, // "wcag22-aa", "en301549-v4.1.1", "en301549-v3.2.1", "section508", or a registered standard's own
24
+ mappings?: string[] // e.g. ["en301549"] or ["en301549:V3.2.1"]
22
25
  },
23
26
  url: string | null,
24
27
  title: string | null,
@@ -36,14 +39,19 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
36
39
  | `engine.tag` | The engine's own identity tag, currently `"a11ycore"`. Every rule (built-in or custom) carries it in `meta.tags` — rule `ruleId`s themselves are bare (no prefix). |
37
40
  | `engine.schemaVersion` | The result-schema version (`"1.0.0"`). Bump-worthy if this document's shape ever changes incompatibly — pin to it if you're parsing output programmatically. See [`API_STABILITY.md`](./API_STABILITY.md) for the full stable/unstable field list and version-bump policy. |
38
41
  | `engine.locale` | Which dictionary the run actually used. `requested` is your `engineOptions.locale` after trimming (`"en"` if you passed nothing or a non-string); `resolved` is the locale whose dictionary was used; `reason` explains the pairing. Because locale fallback is graceful and per-string, asking for a language the build does not carry produces English text rather than an error — this field is how you find that out without reading the strings. Reported once per result: a run uses one dictionary throughout. |
39
- | `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key. `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
42
+ | `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key (a profile's messages in a language that profile does not offer show in English, by its choice, and do not count; see [`I18N.md`](./I18N.md)). `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
43
+ | `engine.wcagVersion` | Which version of WCAG this run was conformance-tested against: your `engineOptions.wcagVersion`, or what your version-origin tags implied, or the default `"2.2"`. It affects one thing today — a rule mapped only to SC 4.1.1 Parsing cannot `fail` under a 2.2 target (see `checksResults[i].wcagVersionScope` below). Reported once per result: a run has one target throughout. |
44
+ | `engine.profile` | Present only when an `engineOptions.profile` selected this run's rules, and names the profile, lowercased. Absent when none was asked for, and also when one was asked for but did not apply (unknown name, or an include in `runOnly` or `engineOptions` chose the rules instead), so its presence is how you confirm a run really targeted that profile. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
45
+ | `engine.profileExcludes` | Present only when the applied profile leaves rules out (`exclude` in its standard's registry entry): `{ rules, criteria }`, the rules it names and the WCAG criteria it waives. Their rules and WCAG rollups did not run. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
46
+ | `engine.optInRules` | Present only when `engineOptions.optInRules` ran at least one opt-in rule that the rest of the selection would not have run. Lists their tags. Its presence means the result includes rules for requirements beyond the targeted standard, such as a national standard's, so a failure may not be a WCAG failure. Absent under a WCAG profile, where unlocking selects nothing, and when a standard's own profile ran its rules. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#running-every-rule-optinrules). |
47
+ | `engine.mappings` | Present only when the run names a standard besides WCAG in `meta.normativeMappings`, through `engineOptions.mappings` or a standard's profile. Lists them canonically, in table order: `"en301549"` for every version, `"en301549:V3.2.1"` for one. Absent means every result names WCAG only. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#other-standards-mappings). |
40
48
  | `url` | The `pageUrl` argument you passed in, or `document.location.href` if you passed `null`/omitted it, or `null` if neither is available. |
41
49
  | `title` | `document.title` at scan time, or `null`. |
42
50
  | `timestamp` | **Not auto-generated.** Only set if you pass `engineOptions.timestamp` as a non-empty string — the engine has no built-in clock (deterministic-by-design). If you want a scan timestamp in the result, supply it yourself. |
43
51
  | `perfStats` | `null` unless `engineOptions.perfStats: true`. Internal timing/counters — shape not covered by this document, treat as debug-only. |
44
52
  | `contextSelector` | The (trimmed) `contextSelector` argument you passed — a string, an array of strings (multi-region scanning, see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)), or `null` if none/empty. |
45
53
  | `checksResults` | One entry per **atomic rule** that ran (every rule not filtered out by `runOnly` — see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). **Every loaded rule produces an entry, even ones that outcome `notApplicable`** — this is not a "violations only" list. |
46
- | `rulesResults` | One entry per **composite (WCAG-SC rollup) rule** that ran — see [Composite result](#a-composite-result-rulesresultsi) and [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md). Empty array if no composite matched the current `runOnly`/tag filter. |
54
+ | `rulesResults` | One entry per **composite (rollup) rule** that ran — see [Composite result](#a-composite-result-rulesresultsi) and [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md). Normally one per WCAG Success Criterion; a run under the profile of a standard with rollups of its own also gets those (`meta.standard` set). Empty array if no composite matched the current `runOnly`/tag filter. |
47
55
  | `overriddenBuiltinIds` | Rule ids where an `engineOptions.customRules` entry shared its `id` with a built-in rule, so the custom implementation replaced the built-in one for this scan (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Always an array; empty when no collision occurred. Also logged via `console.warn` at scan time, since a same-named custom rule is as likely to be an accidental collision as a deliberate override. |
48
56
 
49
57
  ## Cross-frame result (`runa11yCoreAcrossFrames`)
@@ -85,7 +93,7 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
85
93
  normative: boolean,
86
94
  atomic: boolean,
87
95
  category: "perceivable" | "operable" | "understandable" | "robust" | null,
88
- normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel: string }>,
96
+ normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel?: string, wcagSc?: string[] }>,
89
97
  standard: string | null,
90
98
  applicability: string,
91
99
  expectation: string,
@@ -95,6 +103,13 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
95
103
  },
96
104
  engineOptions: object, // the resolved engineOptions this rule actually ran under
97
105
  schemaVersion: string,
106
+ rollupIds: string[], // the rulesResults entries that group this rule in this run; [] if none
107
+ data?: object, // page-level data a rule reports whatever its outcome; see below
108
+ wcagVersionScope?: { // present only when the target WCAG version changed this outcome
109
+ target: "2.0" | "2.1" | "2.2",
110
+ removedSc: string[],
111
+ coercedFrom: "fail"
112
+ },
98
113
  error?: string // present only if the rule threw — see below
99
114
  }
100
115
  ```
@@ -102,14 +117,19 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
102
117
  Notes:
103
118
 
104
119
  - **`outcome` vs `outcomeNormalized`**: identical except `notApplicable` becomes `"inapplicable"` in `outcomeNormalized`. Both are provided so you can match either your own vocabulary or the engine's internal one.
105
- - **`type: "manual"` rules can never report `outcome: "fail"`.** If a manual rule's own logic would have said `fail`, the engine coerces it to `cantTell` and appends an explanatory note to `error` — this is enforced centrally (`policy.coerceManualFailToCantTell`, on by default under the `a11y` policy contract; see [`POLICY.md`](./POLICY.md)), not something each rule has to remember. `fail` is reserved for deterministic, high-confidence, `type: "automatic"` findings only.
106
- - **`meta.normativeMappings`** is how a check result ties back to a WCAG Success Criterion — `[]` for rules with no formal WCAG mapping (this engine calls them advisory `type: "manual"` rules). See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md) for how these roll up.
120
+ - **`type: "manual"` rules can never report `outcome: "fail"`.** If a manual rule's own logic would have said `fail`, the engine coerces it to `cantTell` and appends an explanatory note to `error` — this is enforced centrally (`policy.coerceManualFailToCantTell`, on by default under the `a11y` policy contract; see [`POLICY.md`](./POLICY.md)), not something each rule has to remember. `fail` is reserved for deterministic, `type: "automatic"` findings only.
121
+ - **`rollupIds`** lists the composites in `rulesResults` that group this rule in this run. An empty list means the rule's findings appear in no rollup, so a consumer that reads only `rulesResults` never sees them; `heading-order`, for instance, belongs to no WCAG rollup.
122
+ - **`meta.normativeMappings`** is how a check result ties back to a WCAG Success Criterion — `[]` for rules with no formal WCAG mapping (this engine calls them advisory `type: "manual"` rules). See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md) for how these roll up. The list is not WCAG-only. When the scan asks for another standard (`engineOptions.mappings`, or a standard's profile), each WCAG criterion is followed by that standard's corresponding requirement, such as the EN 301 549 clause that restates it (`{ standard: "EN 301 549", version: "V3.2.1" | "V4.1.1", requirement: "9.1.1.1", title, wcagSc: ["1.1.1"] }`, one entry per version that includes the criterion, `wcagSc` naming the criteria it corresponds to), or the requirements of a standard mapped rule by rule that the rule checks (same shape, `wcagSc` empty for a requirement WCAG does not make, and any fields of that standard's own); and a rule may also cite WCAG's Understanding documents (`type: "Understanding"`). Filter on `standard` (and on the absence of `type`) before reading `requirement` as a Success Criterion. A composite's WCAG entry is always first. See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md#en-301-549).
123
+ - **`wcagVersionScope`**: only present when the run's target WCAG version turned this rule's `fail` into a `cantTell` — today that means a rule mapped to SC 4.1.1 Parsing (`duplicate-id`) under the default 2.2 target, since 2.2 removed that criterion. `removedSc` lists the criteria that stopped existing, `target` is the version that removed them, and `coercedFrom` is the outcome the rule itself reported. The occurrences are the rule's own, unchanged — nothing was dropped, only the conformance verdict was. Absent on every other result, and **never** reported through `error`: nothing went wrong. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#filtering-by-wcag-version-21-vs-22).
124
+ - **`data`**: present only on a rule that reports something about the whole page, whatever its outcome. No core rule does today; a profile's rule may, such as one returning the page's own entry for a probe that compares pages (see `probes` in [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Like `data.details` on an occurrence, it is not a stable contract.
107
125
  - **`error`**: only present if the rule implementation threw an uncaught exception, or if the manual-fail coercion above fired. A thrown rule always surfaces as `outcome: "cantTell"` with `occurrences: []` and `error` set to the exception message — the engine never lets one broken rule crash the whole scan.
108
126
  - **`engineOptions`** on each result is the *resolved* options object (after locale/contrast defaults were applied), not literally what you passed in — useful for confirming what a given rule actually saw, especially the resolved `locale` and `contrast.mode`/`contrast.rootCanvasFallback`.
109
127
 
110
128
  ## An occurrence (`occurrences[i]`)
111
129
 
112
- Only present when `outcome` is `fail` or `cantTell` (a `pass`/`notApplicable` result has `occurrences: []` — this engine does not enumerate the elements it passed, only the ones it flagged).
130
+ Normally present only when `outcome` is `fail` or `cantTell`: a `pass` result has `occurrences: []`, since this engine does not enumerate the elements it passed, only the ones it flagged.
131
+
132
+ `notApplicable` is the one exception. A rule that had nothing to judge may attach a single occurrence saying why, and the contrast rules do exactly that when no text had a computable background — the difference between "checked, nothing to flag" and "could not check" is one this engine reports rather than hides. Such an occurrence describes the scan, not an element, so its `selector` is empty. Do not read `occurrences.length` as a violation count without checking `outcome` first.
113
133
 
114
134
  ```ts
115
135
  {
@@ -119,6 +139,13 @@ Only present when `outcome` is `fail` or `cantTell` (a `pass`/`notApplicable` re
119
139
  summary: string,
120
140
  hint: string,
121
141
  i18n: { summaryKey: string, hintKey: string, params: object } | null,
142
+ occurrenceOutcome?: "fail" | "cantTell", // present when the rule graded its findings into tiers
143
+ uncertainty?: { // present only on a cantTell-tier occurrence
144
+ code: "not-computable" | "runtime-dependent" | "spec-only"
145
+ | "equivalence-unknown" | "judgement-required" | "out-of-scope",
146
+ needed?: string, // what would settle the question
147
+ evidence?: object // what the rule did establish, rule-specific
148
+ },
122
149
  data: {
123
150
  visibilityFilter?: { eligible: boolean, targetSet: string, accEligible: boolean | null, reasons: string[] },
124
151
  details?: object // rule-specific, non-normative — see below
@@ -128,18 +155,37 @@ Only present when `outcome` is `fail` or `cantTell` (a `pass`/`notApplicable` re
128
155
 
129
156
  | Field | Meaning |
130
157
  |---|---|
131
- | `selector` | A best-effort CSS selector built to resolve back to the flagged element (see `helpers.buildSelector` in `RULE_AUTHORING.md`). Not guaranteed unique in adversarial DOM shapes, but the engine actively verifies it resolves to the reported element before using it. |
158
+ | `selector` | A best-effort CSS selector built to resolve back to the flagged element (see `helpers.buildSelector` in `RULE_AUTHORING.md`). Not guaranteed unique in adversarial DOM shapes, but the engine actively verifies it resolves to the reported element before using it. The exception is a rule whose finding *is* an absent element: `page-title-present` reports `head > title` with an `html` of `<title>(missing)</title>`, neither of which is on the page. Both are constants, so the fingerprint they feed stays stable, but do not treat `selector` as resolvable or `html` as real markup without checking the rule reported something that exists. |
132
159
  | `html` | An outer-HTML snippet of the flagged element — use this as your primary "which element" signal when `includeShadowDom: true` (selectors don't pierce shadow boundaries). |
133
160
  | `structuralPath` | The flagged element's sibling-index path from `documentElement` down to it (e.g. `[1, 0, 2]`) — `[]` if the element *is* `documentElement`, `null` if it couldn't be determined. A more robust element-identity mechanism than `selector` alone: it survives DOM changes a selector string wouldn't (an id/class rename, for instance), at the cost of not being usable as an actual CSS selector. Computed from the element reference when the rule kept one, otherwise by re-resolving `selector` against the document (same caveat as `selector` itself: a non-unique selector could resolve to a different element than intended). |
134
161
  | `summary` | Human-readable, already localized ("This button has no accessible name."). |
135
162
  | `hint` | Human-readable remediation guidance, already localized. |
136
163
  | `i18n` | The raw translation keys behind `summary`/`hint`, if you want to re-render them in a different locale yourself without re-running the scan. `null` if the occurrence didn't use key-based i18n. |
137
164
  | `data.visibilityFilter` | Present on most occurrences: why the engine considered this element eligible (or not) under whichever eligibility model the rule used. `eligible` is that result; `targetSet` says which model produced it (`'dom'`: raw DOM/CSS visibility — most rules; `'acc'`: accessibility-tree eligibility). `accEligible` mirrors `eligible` only when `targetSet` is `'acc'`, otherwise `null` — don't read it as a second, independent signal. `reasons` is a list of machine-readable exclusion codes when `eligible: false`. |
138
- | `data.details` | Rule-specific structured data (e.g. `reasonCode`, computed metrics, resolved references) — **non-normative**: useful for building richer UI or debugging, but never changes what `outcome`/`severity` mean. Shape varies per rule; treat as best-effort extra context, not a stable contract. |
165
+ | `data.details` | Rule-specific structured data (computed metrics, resolved references) — **non-normative**: useful for building richer UI or debugging, but never changes what `outcome`/`severity` mean. Shape varies per rule; treat as best-effort extra context, not a stable contract. The one exception is `data.details.reasonCode`, which **is** stable: it identifies *which* of a rule's findings this is, and together with `ruleId` and `html` forms the fingerprint baselines and SARIF are keyed on. A rule may gain a new reason code in a minor release; a shipped one does not change. See [`API_STABILITY.md`](./API_STABILITY.md#finding-identity). |
166
+ | `occurrenceOutcome` | Which tier this occurrence belongs to, on a rule that graded its findings into a confident `fail` tier and a needs-review `cantTell` tier. A rule reporting one tier only omits it, in which case the result's own `outcome` is the occurrence's tier. This is why a `fail` result can carry `cantTell`-tier occurrences: the aggregate outcome stays singular so CI can still gate on it, without discarding the findings that only warranted review. |
167
+ | `uncertainty` | Why this finding could not be decided — see [Uncertainty codes](#uncertainty-codes) below. |
168
+
169
+ ### Uncertainty codes
170
+
171
+ A `cantTell` says the engine did not decide. `uncertainty` says **why**, from a closed vocabulary, so a consumer can branch on the reason rather than parse a summary string. It is present only on a `cantTell`-tier occurrence: a `fail`-tier one would be claiming the rule both decided and did not, so the engine drops it.
172
+
173
+ | `code` | Meaning | Typical shape |
174
+ |---|---|---|
175
+ | `not-computable` | The evidence the rule needed could not be read in this environment. | A cross-origin stylesheet, a background colour that resolves to no value, an `src` that will not resolve. |
176
+ | `runtime-dependent` | The markup cannot settle it because script decides at runtime. | An `aria-controls` naming an element the widget builds when it opens. |
177
+ | `spec-only` | A real specification violation, but the exposed name, role and value survive it, so no Success Criterion is established as failed. | An ARIA attribute whose absence the specification supplies a default for. |
178
+ | `equivalence-unknown` | Two things may or may not serve the same purpose, and neither the markup nor the content settles it. | Two frames sharing an accessible name but embedding different resources. |
179
+ | `judgement-required` | The question is inherently a human call. | Whether an undersized target is essential; every `type: "manual"` rule. |
180
+ | `out-of-scope` | The finding is real but falls outside the standard this run targets. | A rule mapped only to a criterion the target WCAG version removed. |
181
+
182
+ `needed` states, in one sentence, what would settle the question — the thing a reviewer has to go and check. `evidence` carries what the rule *did* establish, so the reviewer starts from the engine's work rather than repeating it; its shape is rule-specific and, like `data.details`, not a stable contract. The `code` is: new codes may be added in a minor release, but an existing one does not change meaning, so branch on the codes you know and treat an unrecognised one as "needs review" rather than an error.
183
+
184
+ Every automatic rule that can report `cantTell` carries this, and a test holds that line so a new one cannot arrive without it. The `out-of-scope` code is attached by the engine rather than by a rule, on the same occurrences that produce a result-level `wcagVersionScope`. Manual rules do not carry it: `judgement-required` is what `type: "manual"` already means, so repeating it per occurrence would say nothing the result does not.
139
185
 
140
186
  ## A composite result (`rulesResults[i]`)
141
187
 
142
- Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Shape is the same envelope as a check result, with composite-specific `data.details`:
188
+ Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Under the profile of a standard with rollups of its own, there are also those, with the standard's wording as `title` and its name as `meta.standard`; they may be the only rollup some of its findings have, such as `heading-order`'s. Shape is the same envelope as a check result, with composite-specific `data.details`:
143
189
 
144
190
  ```ts
145
191
  {
@@ -150,6 +196,9 @@ Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wc
150
196
  data: {
151
197
  details: {
152
198
  reasonCode: string, // e.g. "composite.rollup.fail.anyFail"
199
+ standard?: string, // a standard's own rollup only, with version and criterion
200
+ version?: string,
201
+ criterion?: string,
153
202
  checksIds: string[], // every atomic ruleId this composite rolls up
154
203
  contributors: Array<{ testId: string, outcome: string, severity: string | null }>,
155
204
  metrics: { failCount, cantTellCount, notApplicableCount, passCount, missingCount }
@@ -164,7 +213,7 @@ Rollup precedence (deterministic, in this order): **any contributor `fail` → c
164
213
 
165
214
  | Outcome | Meaning | Can appear on `type: "manual"`? |
166
215
  |---|---|---|
167
- | `fail` | Deterministic, high-confidence, normative violation — no heuristics, no guessing. | No (coerced to `cantTell`) |
216
+ | `fail` | Deterministic, normative violation — the decision procedure guesses at nothing. | No (coerced to `cantTell`) |
168
217
  | `pass` | The rule's applicable target(s) exist and none were flagged. | Yes |
169
218
  | `cantTell` | Requires human judgment — either genuinely ambiguous, or a `manual` rule's advisory finding. | Yes |
170
219
  | `notApplicable` | The rule found no elements it applies to on this page/scope. | Yes |
@@ -176,6 +225,8 @@ Rollup precedence (deterministic, in this order): **any contributor `fail` → c
176
225
  - `severity`: `minor` < `moderate` < `serious` < `critical` — the rule author's assessment of user impact, independent of `confidence`.
177
226
  - `confidence`: `low` < `medium` < `high` — how certain the engine is that a `fail`/`cantTell` verdict is correct. Both are informational metadata for prioritization; neither changes `outcome`'s meaning.
178
227
 
228
+ A `fail` is not always `confidence: "high"`, and that is not a contradiction. The outcome describes the decision procedure — it resolved the question without guessing — while `confidence` describes the model that decision was made against. A handful of automatic rules decide deterministically against something that is itself an approximation (the curated WAI-ARIA role tables, the native-role mappings, an accessibility tree inferred from static markup) and report `medium`: `aria-required-children`, `aria-prohibited-children`, `aria-required-parent`, `aria-allowed-attr`, `form-control-programmatic-label-present`, `svg-image-text-alternative-present`, `video-poster-text-alternative-present` and `target-size-minimum`. `confidence` is on every result, so a consumer that wants only the most certain failures can gate on it directly; `policy.allowedConfidence` will not do it for you, since a disallowed value is replaced with the rule's own `defaultConfidence` rather than changing the outcome (see [`POLICY.md`](./POLICY.md)).
229
+
179
230
  ## Worked example
180
231
 
181
232
  Scanning `<img src="logo.png">` (no `alt`) and `<button></button>` (no accessible name), scoped to just those two rules via `runOnly: { includeRuleIds: [...] }` (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md) — this is **not** a bare array):
package/docs/POLICY.md CHANGED
@@ -68,4 +68,4 @@ Neither of these ever throws — policy resolution is designed to always produce
68
68
 
69
69
  ## Why this exists as a separate layer
70
70
 
71
- Keeping outcome-integrity rules (like "manual rules can't fail") in a policy layer — rather than hard-coded into every rule, or worse, left to each rule author's discretion — means the guarantee holds even if a rule's own logic has a bug, and means different consumers can have different appetites for risk (a CI gate vs. an internal audit dashboard) without forking the rule set itself. This protects the engine's core guarantee: `fail` must always mean "deterministic, high-confidence, normative violation," full stop.
71
+ Keeping outcome-integrity rules (like "manual rules can't fail") in a policy layer — rather than hard-coded into every rule, or worse, left to each rule author's discretion — means the guarantee holds even if a rule's own logic has a bug, and means different consumers can have different appetites for risk (a CI gate vs. an internal audit dashboard) without forking the rule set itself. This protects the engine's core guarantee: `fail` must always mean "deterministic, normative violation," full stop — see [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md#severity-and-confidence-values) for why that is not the same as "high-confidence."
package/docs/REPORT.md CHANGED
@@ -12,10 +12,15 @@ Open `report.html` directly from disk. Works alongside any other output mode —
12
12
 
13
13
  - **Hero**: one plain-language headline ("N of M applicable checks passed") plus a stacked bar and legend (icon + label + count — status is never color-only) broken down by outcome (`fail`/`cantTell`/`pass`/`notApplicable`).
14
14
  - **Worth reviewing**: one card per rule with `fail`/`cantTell` occurrences (not one per occurrence — a rule with many identical occurrences is one thing worth attention, not many), each showing severity, WCAG SC chip(s), a representative occurrence, and the total occurrence count. Capped at the 24 highest-priority rules with an overflow note past that.
15
- - **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion, its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
15
+ - **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion (with the EN 301 549 clause that restates it, where there is one and the scan asked for EN 301 549 clauses), its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
16
+ - **A standard's own rollup**: only when the scan ran the rules of a standard that has rollups of its own (its profile, or `engineOptions.optInRules`), one row per rollup, with the standard's wording, the requirements of the rules that decided the outcome, the breakdown and the contributing rules.
16
17
  - **Full technical data** (collapsed by default): a scorecard (tiles per outcome) and a searchable, filterable (by outcome), paginated table of every individual occurrence across the whole scan.
17
18
 
18
- The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
19
+ The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, the WCAG version the scan targeted the `engineOptions.profile` it used (when there was one), the opt-in rule tags `engineOptions.optInRules` added (when it added any), and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
20
+
21
+ The whole page is written in the locale the scan resolved to: headings, table columns, outcome and severity names, the headline, the pager, and the date format, all from the `report_*` keys in the same dictionaries as the findings. `<html lang>` names that locale. The outcome codes in the occurrence filters (`fail`, `cantTell`, …) stay as they are, because they are the values a reader searches for in the JSON result.
22
+
23
+ One case keeps English labels: a scan in a language the engine does not ship, run with a caller-supplied `engineOptions.messages` dictionary. That dictionary is not part of the result, so the report cannot read labels from it; its findings are then marked with their own language (`lang="nl"`, for instance), so a screen reader still reads them with the right voice.
19
24
 
20
25
  ## Library usage
21
26
 
@@ -114,7 +114,7 @@ const meta = {
114
114
  title: 'Non-text Content',
115
115
  conformanceLevel: 'A'
116
116
  }
117
- ],
117
+ ], // WCAG only: EN 301 549 clauses are derived at build time (WCAG_CONFORMANCE.md#en-301-549)
118
118
 
119
119
  defaultSeverity: 'minor' | 'moderate' | 'serious' | 'critical',
120
120
  category: 'perceivable' | 'operable' | 'understandable' | 'robust',
@@ -141,8 +141,9 @@ The build/runtime resolves i18n by:
141
141
  2) falling back to `en` if missing,
142
142
  3) falling back to the literal `title`/`description` strings if still missing.
143
143
 
144
- Add the key and its English text to `src/i18n/en.json`, then run
145
- `npm run i18n:sync` so every other locale picks it up. `npm test` fails if you
144
+ Add the key and its English text to `src/i18n/en.json` (a profile's rule:
145
+ `profiles/<name>/i18n/en.json`), then run `npm run i18n:sync` so every other
146
+ locale picks it up. `npm test` fails if you
146
147
  forget. See [`I18N.md`](./I18N.md).
147
148
 
148
149
  #### `meta.tags`
@@ -150,6 +151,35 @@ Tags are used for grouping/filtering. Typical tag families in this ruleset inclu
150
151
  - WCAG tagging: `wcag2a`, `wcag111`
151
152
  - domain: `nontext`, `images`, plus element-specific tags
152
153
  - nature: `atomic`, plus `automatic` or `manual`
154
+ - another standard's own requirement: that standard's rule tag (see below)
155
+
156
+ #### Rules for another standard's own requirements
157
+ A rule that checks something WCAG does not require, but another standard does (a doctype or presentational attributes, say), declares no WCAG mapping (`wcagSc: []`, `normativeMappings: []`) and carries that standard's rule tag. The tag makes it **opt-in**: it runs only under the standard's profile, a selection that includes the tag, or its own id, never in a default or WCAG run ([`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#opt-in-rules)). That is what lets it report `fail`: its failures are failures of that standard, and only a scan targeting it sees them. Its module goes in that standard's profile rather than in `src/checks/`: `profiles/<key>/rules/automatic/` or `profiles/<key>/rules/manual/`. The build compiles it into the engine like any other rule. Its test and scenario page go in the profile too, in `profiles/<key>/tests/rules/` and `profiles/<key>/tests/fixtures/`. Map it to the standard's requirements the usual way (a row in the profile's rule map). Rule tags come from each standard's `ruleTag` in the registry, `src/coverage/standards.js` (a profile's from its `index.js`). The sample profile core's tests run against, `tests/fixtures/profiles/sample/`, has two such rules.
158
+
159
+ #### Rule variants
160
+ When another standard's requirement is a core rule with different thresholds (contrast at 7:1, say, or bold text large from 18.5px rather than WCAG's 14pt), write it as a **variant**, not a copy. The core rule declares the thresholds it reads from `ctx.config` as `settings`, with WCAG's values as defaults:
161
+
162
+ ```js
163
+ // src/checks/automatic/contrast-minimum.js
164
+ const settings = { boldLargeMinPx: null, largeTextRatio: 3, normalTextRatio: 4.5 };
165
+ module.exports = { id, meta, runInPage, settings };
166
+ ```
167
+
168
+ The variant is data, in the standard's profile:
169
+
170
+ ```js
171
+ // profiles/<key>/rules/automatic/sample-contrast-enhanced.js
172
+ module.exports = {
173
+ id: 'sample-contrast-enhanced',
174
+ from: 'contrast-minimum',
175
+ config: { normalTextRatio: 7, largeTextRatio: 4.5 },
176
+ meta: { /* its own title, description, i18n, tags... as any rule's */ }
177
+ };
178
+ ```
179
+
180
+ The build runs the base rule's `runInPage` and `applicability` under the variant's id and meta, with its `config` in `ctx.config`. A rule's settings are never the caller's: the runner drops a caller's value for one (`engineOptions.rules[ruleId]`), so the base rule always runs at its defaults and the variant at its `config`, while the caller's other config, such as `excludeSelectors`, still applies. A message key of the base's that starts with the base's prefix (its `meta.i18n.titleKey` without `_title`, `contrastMinimum`) is read from the variant's prefix instead (`sampleContrastEnhanced`), so the variant's dictionary has the same keys under its own prefix; the rule validator checks they exist. A fix to the base reaches every variant. The build refuses a variant whose base does not exist, is itself a variant, or declares no `settings`, and a setting the base does not declare or of another type. A base rule that caches verdicts depending on its settings keys those caches by them, as `contrast-minimum` does.
181
+
182
+ Add a setting to a core rule when a standard needs it, with a default that keeps the rule's behaviour; the settings a rule declares are for its variants, not for callers, and stay outside semver until a profile can live outside this repository (see [`API_STABILITY.md`](./API_STABILITY.md#explicitly-unstable-not-covered-by-semver)).
153
183
 
154
184
  #### `meta.coverage.facetsBySc`
155
185
  This is the repo’s explicit **coverage model** for an SC.
@@ -254,17 +284,15 @@ that one is on you.
254
284
 
255
285
  ## 6) Helpers contract used by rules (ctx.helpers)
256
286
 
257
- Rules use helpers returned by `createDomHelpers()`.
287
+ Rules use helpers returned by `createDomHelpers()`. The most load-bearing ones —
288
+ `queryAllSmart` (query with shadow/hidden/exclude handling built in),
289
+ `getAccessibleNameInfo`/`getAccessibleDescriptionInfo`/`getTextAlternativeInfo` (naming),
290
+ `isAccTreeEligible`/`getEligibilityInfo` (visibility), `getRoleInfo`/`getFocusableInfo`
291
+ (role/focus) — cover most rules.
258
292
 
259
- Helpers observed in this repo include:
260
- - `queryAll`, `queryAllDeep`, `queryAllSmart`
261
- - `getOuterHtmlSnippet`
262
- - `buildSimpleSelector`, `buildSelector`
263
- - `isAccTreeEligible`, `getEligibilityInfo`
264
- - `resolveIdRefs`, `getTextFromIdRefs`
265
- - `getAccessibleNameInfo`, `getAccessibleDescriptionInfo`
266
- - `getTextAlternativeInfo`
267
- - `getRoleInfo`, `getFocusableInfo`
293
+ **See [`RULE_HELPERS.md`](./RULE_HELPERS.md) for the full reference** (~35 helpers plus
294
+ the `contrast.*`/`aria.*` namespaces), with what each one does and when to reach for it
295
+ instead of reimplementing the logic in a new rule.
268
296
 
269
297
  ### 6.1 Shadow DOM scanning
270
298
 
@@ -319,7 +347,23 @@ The rule must return:
319
347
 
320
348
  Examples:
321
349
 
322
- ### 8.2 Outcome conventions used by these rules
350
+ ### 8.2 What `ctx` carries
351
+
352
+ `runInPage(ctx)` and `applicability(ctx)` receive the same object, built-in and custom rules alike:
353
+
354
+ | Field | What it is |
355
+ |---|---|
356
+ | `document`, `window` | The page being scanned. |
357
+ | `root` | The roots the scan covers: the document, or what `contextSelector` resolved to. |
358
+ | `contextSelector` | The selector that scoped the run, if any. |
359
+ | `rule` | The rule's resolved definition: `ruleId`, `defaultSeverity`, `defaultConfidence`, `type`, `meta`... |
360
+ | `config` | `engineOptions.rules[ruleId]`, this rule's settings, if the caller gave any (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). |
361
+ | `standard` | The standard and version the run targets, `{ key, name, version }` (`{ key: 'en301549', name: 'EN 301 549', version: 'V4.1.1' }`), when a standard's profile selected the run; `null` otherwise (no profile, a WCAG profile, or rules chosen by tag or id). A rule whose behaviour differs between versions of its standard reads it here, and does what holds for every version when it is `null`. |
362
+ | `helpers` | The helpers documented in [`RULE_HELPERS.md`](./RULE_HELPERS.md). |
363
+ | `engineOptions` | The scan's options as resolved. |
364
+ | `inputs.probes` | Evidence the host application supplied (`engineOptions.probes`). |
365
+
366
+ ### 8.3 Outcome conventions used by these rules
323
367
 
324
368
  Automatic:
325
369
  - `notApplicable` if no applicable targets
@@ -387,7 +431,8 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
387
431
  ### 11.1 The fixture file
388
432
 
389
433
  - Path: `tests/fixtures/<rule-slug>-all-scenarios.html`, where `<rule-slug>` is the rule
390
- id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`).
434
+ id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`). A
435
+ profile's rule keeps it in the profile: `profiles/<name>/tests/fixtures/`.
391
436
  - Structure: a real HTML page (`<!doctype html>`, `<title>`, minimal inline `<style>`)
392
437
  containing numbered scenario blocks, each:
393
438
  ```html
@@ -407,6 +452,24 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
407
452
  (eligible, FAIL)", "C. Ineligible (excluded from accessibility tree, skipped)").
408
453
  - Cover every branch the rule's own logic distinguishes: pass, fail (each distinct
409
454
  `reasonCode`), notApplicable/skipped, and — for manual rules — cantTell.
455
+ - A whole-document rule (`page-title-present`, `meta-refresh-timing-absent`, `region`)
456
+ can only demonstrate one outcome per page. Its fixture declares a single bare
457
+ `.case-title` with no `.case` wrapper, and the page itself is the case; the marker is
458
+ compared against the rule-level outcome, so `PASS` and `NEUTRAL` are distinguished
459
+ there. Cover the remaining branches with inline tests rather than near-identical
460
+ fixture files.
461
+ - One fixture shared by several rules that expect different things of the same case
462
+ (`tests/fixtures/contrast-all-scenarios.html` serves `contrast-minimum`,
463
+ `contrast-enhanced` and `contrast-computable`) carries a per-rule marker as a
464
+ `data-outcome-<rule-id>` attribute on the `.case`, which overrides the shared
465
+ `.case-title` for that rule. Use an attribute rather than more text when the rules
466
+ under test evaluate text: a `.case-title` added to a contrast case is one more text
467
+ node to check. A marker word the parser does not recognise (`MIXED`, `UNSTATED`)
468
+ asserts nothing, for a case whose outcome the fixture does not state.
469
+ - `npm run fixtures:markers:check` replays every fixture and fails when a marker no
470
+ longer matches what the rule reports; `scripts/data/fixture-markers.json` records the
471
+ cases that already disagree, so that set can only shrink. A profile's rules have
472
+ their record in the profile's own `scripts/data/fixture-markers.json`.
410
473
 
411
474
  ### 11.2 Known, acceptable exceptions to "one fixture, many cases"
412
475
 
@@ -482,6 +545,9 @@ npm run fixtures:index
482
545
  This writes `tests/fixtures/INDEX.md` (human-readable), `tests/fixtures/index.json`
483
546
  (machine-readable — every rule, its fixture path, and parsed pass/fail/cantTell case
484
547
  counts, for external tooling to enumerate and load fixtures directly) and
485
- `tests/fixtures/index.html` (the same listing as a browsable page). Commit all three
548
+ `tests/fixtures/index.html` (the same listing as a browsable page). A profile's rules
549
+ get the same three files in the profile's `tests/fixtures/`, with paths relative to the
550
+ profile's folder. Commit all three
486
551
  alongside the fixture and test changes. A rule shipped without its fixture is treated
487
- the same as a rule shipped without tests — not done.
552
+ the same as a rule shipped without tests — not done. `npm run fixtures:check` reports
553
+ a stale index without rewriting it, and CI fails on one.