@surea11y/core 1.6.0 → 1.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +140 -0
- package/README.md +179 -90
- package/docs/ACT_RULE_MAPPING.md +10 -8
- package/docs/API_STABILITY.md +67 -6
- package/docs/BINDING_AUTHORS_GUIDE.md +104 -2
- package/docs/CI_INTEGRATIONS.md +43 -0
- package/docs/DESIGN_CHALLENGES.md +162 -2
- package/docs/EARL.md +100 -0
- package/docs/ENGINE_OPTIONS.md +109 -5
- package/docs/I18N.md +62 -20
- package/docs/INTEGRATION.md +4 -2
- package/docs/JUNIT.md +73 -0
- package/docs/LIMITATIONS.md +4 -1
- package/docs/OUTPUT_SCHEMA.md +62 -11
- package/docs/POLICY.md +1 -1
- package/docs/REPORT.md +7 -2
- package/docs/RULE_AUTHORING.md +83 -17
- package/docs/RULE_CATALOG.md +212 -139
- package/docs/RULE_EXAMPLES.md +2189 -0
- package/docs/RULE_HELPERS.md +390 -0
- package/docs/RULE_TAXONOMY.md +27 -6
- package/docs/SARIF.md +23 -3
- package/docs/WCAG_CONFORMANCE.md +64 -3
- package/package.json +41 -12
- package/profiles/index.js +14 -0
- package/src/checks/automatic/area-alt-present.js +87 -31
- package/src/checks/automatic/aria-allowed-attr.js +6 -0
- package/src/checks/automatic/aria-allowed-role.js +32 -23
- package/src/checks/automatic/aria-braille-equivalent.js +43 -17
- package/src/checks/automatic/aria-conditional-attr.js +17 -10
- package/src/checks/automatic/aria-deprecated-role.js +12 -0
- package/src/checks/automatic/aria-hidden-body.js +1 -1
- package/src/checks/automatic/aria-hidden-focus.js +74 -18
- package/src/checks/automatic/aria-prohibited-attr.js +22 -4
- package/src/checks/automatic/aria-prohibited-children.js +6 -6
- package/src/checks/automatic/aria-required-attr.js +88 -12
- package/src/checks/automatic/aria-required-children.js +33 -16
- package/src/checks/automatic/aria-required-parent.js +32 -6
- package/src/checks/automatic/aria-role-name-present.js +20 -3
- package/src/checks/automatic/aria-roles-valid.js +52 -21
- package/src/checks/automatic/aria-valid-attr-value.js +89 -24
- package/src/checks/automatic/aria-valid-attr.js +14 -9
- package/src/checks/automatic/autocomplete-valid.js +152 -26
- package/src/checks/automatic/avoid-inline-spacing.js +207 -15
- package/src/checks/automatic/button-name-present.js +2 -1
- package/src/checks/automatic/canvas-text-alternative-present.js +105 -14
- package/src/checks/automatic/combobox-name-present.js +34 -51
- package/src/checks/automatic/contrast-computable.js +45 -4
- package/src/checks/automatic/contrast-enhanced.js +16 -4
- package/src/checks/automatic/contrast-minimum.js +57 -11
- package/src/checks/automatic/css-orientation-lock.js +171 -12
- package/src/checks/automatic/definition-list-children-valid.js +67 -23
- package/src/checks/automatic/deprecated-elements-not-used.js +43 -38
- package/src/checks/automatic/dialog-name-present.js +28 -9
- package/src/checks/automatic/duplicate-id-aria.js +5 -0
- package/src/checks/automatic/duplicate-id.js +19 -10
- package/src/checks/automatic/form-control-single-label.js +9 -0
- package/src/checks/automatic/identical-iframes-same-purpose.js +229 -0
- package/src/checks/automatic/iframe-focusable-content.js +12 -4
- package/src/checks/automatic/iframe-title-unique.js +36 -81
- package/src/checks/automatic/input-image-alt-present.js +32 -20
- package/src/checks/automatic/label-in-name.js +78 -69
- package/src/checks/automatic/language-page-present.js +12 -6
- package/src/checks/automatic/link-in-text-block.js +512 -44
- package/src/checks/automatic/link-name-present.js +13 -5
- package/src/checks/automatic/list-children-valid.js +18 -1
- package/src/checks/automatic/listbox-name-present.js +19 -49
- package/src/checks/automatic/listitem-parent-valid.js +4 -3
- package/src/checks/automatic/meta-refresh-no-exceptions.js +3 -3
- package/src/checks/automatic/page-title-present.js +16 -4
- package/src/checks/automatic/progressbar-name-present.js +11 -1
- package/src/checks/automatic/{role-img-alt-present.js → role-img-text-alternative-present.js} +9 -5
- package/src/checks/automatic/searchbox-name-present.js +32 -49
- package/src/checks/automatic/server-side-image-map-absent.js +48 -28
- package/src/checks/automatic/slider-name-present.js +38 -52
- package/src/checks/automatic/spinbutton-name-present.js +32 -49
- package/src/checks/automatic/target-size-minimum.js +84 -16
- package/src/checks/automatic/td-has-header.js +60 -23
- package/src/checks/automatic/text-spacing-content-loss.js +548 -0
- package/src/checks/automatic/textbox-name-present.js +32 -49
- package/src/checks/automatic/valid-lang.js +15 -10
- package/src/checks/manual/area-alt-quality-manual.js +113 -31
- package/src/checks/manual/canvas-text-alternative-quality-manual.js +0 -2
- package/src/checks/manual/css-focus-indicator-suppressed-manual.js +23 -10
- package/src/checks/manual/css-hidden-focus.js +215 -7
- package/src/checks/manual/form-control-label-quality-manual.js +243 -29
- package/src/checks/manual/heading-order-manual.js +9 -1
- package/src/checks/manual/heading-quality-manual.js +143 -9
- package/src/checks/manual/img-alt-decorative-manual.js +6 -3
- package/src/checks/manual/input-image-alt-quality-manual.js +81 -13
- package/src/checks/manual/landmark-complementary-is-top-level-manual.js +231 -0
- package/src/checks/manual/link-name-quality-manual.js +130 -4
- package/src/checks/manual/media-transcript-present-manual.js +65 -8
- package/src/checks/manual/mouse-only-event-handlers-manual.js +66 -5
- package/src/checks/manual/no-autoplay-audio-manual.js +93 -6
- package/src/checks/manual/p-as-heading-manual.js +89 -44
- package/src/checks/manual/page-title-patterns-manual.js +77 -8
- package/src/checks/manual/password-paste-enabled-manual.js +255 -0
- package/src/checks/manual/skip-link-manual.js +42 -14
- package/src/checks/manual/table-fake-caption-manual.js +32 -1
- package/src/checks/manual/video-caption-manual.js +47 -24
- package/src/checks/manual-review.js +0 -4
- package/src/core.js +18061 -46194
- package/src/coverage/en301549-map.js +187 -0
- package/src/coverage/standards.js +279 -0
- package/src/coverage/wcag-facets.js +1119 -0
- package/src/coverage/wcag-version-map.js +101 -0
- package/src/earl.js +144 -0
- package/src/en301549.js +33 -0
- package/src/junit.js +321 -0
- package/src/profile-kit.js +163 -0
- package/src/report.js +343 -74
- package/src/sarif.js +56 -5
- package/src/wcag.js +105 -0
- package/surea11y.browser.js +11 -41039
- package/surea11y.i18n.de.js +2 -21
- package/surea11y.i18n.es.js +2 -21
- package/surea11y.i18n.fr.js +2 -21
- package/surea11y.i18n.ja.js +3 -0
- package/src/checks/manual/area-alt-decorative-manual.js +0 -255
package/docs/OUTPUT_SCHEMA.md
CHANGED
|
@@ -18,7 +18,10 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
18
18
|
engine: {
|
|
19
19
|
tag: string,
|
|
20
20
|
schemaVersion: string,
|
|
21
|
-
locale: { requested: string, resolved: string, reason: string }
|
|
21
|
+
locale: { requested: string, resolved: string, reason: string },
|
|
22
|
+
wcagVersion: "2.0" | "2.1" | "2.2",
|
|
23
|
+
profile?: string, // "wcag22-aa", "en301549-v4.1.1", "en301549-v3.2.1", "section508", or a registered standard's own
|
|
24
|
+
mappings?: string[] // e.g. ["en301549"] or ["en301549:V3.2.1"]
|
|
22
25
|
},
|
|
23
26
|
url: string | null,
|
|
24
27
|
title: string | null,
|
|
@@ -36,14 +39,19 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
36
39
|
| `engine.tag` | The engine's own identity tag, currently `"a11ycore"`. Every rule (built-in or custom) carries it in `meta.tags` — rule `ruleId`s themselves are bare (no prefix). |
|
|
37
40
|
| `engine.schemaVersion` | The result-schema version (`"1.0.0"`). Bump-worthy if this document's shape ever changes incompatibly — pin to it if you're parsing output programmatically. See [`API_STABILITY.md`](./API_STABILITY.md) for the full stable/unstable field list and version-bump policy. |
|
|
38
41
|
| `engine.locale` | Which dictionary the run actually used. `requested` is your `engineOptions.locale` after trimming (`"en"` if you passed nothing or a non-string); `resolved` is the locale whose dictionary was used; `reason` explains the pairing. Because locale fallback is graceful and per-string, asking for a language the build does not carry produces English text rather than an error — this field is how you find that out without reading the strings. Reported once per result: a run uses one dictionary throughout. |
|
|
39
|
-
| `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key. `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
|
|
42
|
+
| `engine.locale.reason` | `"ok"` — you got the dictionary you asked for, and it carries every key (a profile's messages in a language that profile does not offer show in English, by its choice, and do not count; see [`I18N.md`](./I18N.md)). `"primary-subtag"` — your code had a subtag with no dictionary of its own, so its base language was used: `"de-DE"` resolves to `"de"`. `"dictionary-not-loaded"` — the project ships that language, but this build does not carry it and none was supplied (the standalone browser bundle, without its locale side file). `"unknown-locale"` — the project has no such translation at all. `"partial-dictionary"` — the dictionary was used but is missing keys English has, so those strings fell back to English. Treat the set as open; later releases can add to it. |
|
|
43
|
+
| `engine.wcagVersion` | Which version of WCAG this run was conformance-tested against: your `engineOptions.wcagVersion`, or what your version-origin tags implied, or the default `"2.2"`. It affects one thing today — a rule mapped only to SC 4.1.1 Parsing cannot `fail` under a 2.2 target (see `checksResults[i].wcagVersionScope` below). Reported once per result: a run has one target throughout. |
|
|
44
|
+
| `engine.profile` | Present only when an `engineOptions.profile` selected this run's rules, and names the profile, lowercased. Absent when none was asked for, and also when one was asked for but did not apply (unknown name, or an include in `runOnly` or `engineOptions` chose the rules instead), so its presence is how you confirm a run really targeted that profile. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
|
|
45
|
+
| `engine.profileExcludes` | Present only when the applied profile leaves rules out (`exclude` in its standard's registry entry): `{ rules, criteria }`, the rules it names and the WCAG criteria it waives. Their rules and WCAG rollups did not run. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#conformance-profiles). |
|
|
46
|
+
| `engine.optInRules` | Present only when `engineOptions.optInRules` ran at least one opt-in rule that the rest of the selection would not have run. Lists their tags. Its presence means the result includes rules for requirements beyond the targeted standard, such as a national standard's, so a failure may not be a WCAG failure. Absent under a WCAG profile, where unlocking selects nothing, and when a standard's own profile ran its rules. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#running-every-rule-optinrules). |
|
|
47
|
+
| `engine.mappings` | Present only when the run names a standard besides WCAG in `meta.normativeMappings`, through `engineOptions.mappings` or a standard's profile. Lists them canonically, in table order: `"en301549"` for every version, `"en301549:V3.2.1"` for one. Absent means every result names WCAG only. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#other-standards-mappings). |
|
|
40
48
|
| `url` | The `pageUrl` argument you passed in, or `document.location.href` if you passed `null`/omitted it, or `null` if neither is available. |
|
|
41
49
|
| `title` | `document.title` at scan time, or `null`. |
|
|
42
50
|
| `timestamp` | **Not auto-generated.** Only set if you pass `engineOptions.timestamp` as a non-empty string — the engine has no built-in clock (deterministic-by-design). If you want a scan timestamp in the result, supply it yourself. |
|
|
43
51
|
| `perfStats` | `null` unless `engineOptions.perfStats: true`. Internal timing/counters — shape not covered by this document, treat as debug-only. |
|
|
44
52
|
| `contextSelector` | The (trimmed) `contextSelector` argument you passed — a string, an array of strings (multi-region scanning, see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)), or `null` if none/empty. |
|
|
45
53
|
| `checksResults` | One entry per **atomic rule** that ran (every rule not filtered out by `runOnly` — see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). **Every loaded rule produces an entry, even ones that outcome `notApplicable`** — this is not a "violations only" list. |
|
|
46
|
-
| `rulesResults` | One entry per **composite (
|
|
54
|
+
| `rulesResults` | One entry per **composite (rollup) rule** that ran — see [Composite result](#a-composite-result-rulesresultsi) and [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md). Normally one per WCAG Success Criterion; a run under the profile of a standard with rollups of its own also gets those (`meta.standard` set). Empty array if no composite matched the current `runOnly`/tag filter. |
|
|
47
55
|
| `overriddenBuiltinIds` | Rule ids where an `engineOptions.customRules` entry shared its `id` with a built-in rule, so the custom implementation replaced the built-in one for this scan (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Always an array; empty when no collision occurred. Also logged via `console.warn` at scan time, since a same-named custom rule is as likely to be an accidental collision as a deliberate override. |
|
|
48
56
|
|
|
49
57
|
## Cross-frame result (`runa11yCoreAcrossFrames`)
|
|
@@ -85,7 +93,7 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
85
93
|
normative: boolean,
|
|
86
94
|
atomic: boolean,
|
|
87
95
|
category: "perceivable" | "operable" | "understandable" | "robust" | null,
|
|
88
|
-
normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel
|
|
96
|
+
normativeMappings: Array<{ standard: string, version: string, requirement: string, title: string, conformanceLevel?: string, wcagSc?: string[] }>,
|
|
89
97
|
standard: string | null,
|
|
90
98
|
applicability: string,
|
|
91
99
|
expectation: string,
|
|
@@ -95,6 +103,13 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
95
103
|
},
|
|
96
104
|
engineOptions: object, // the resolved engineOptions this rule actually ran under
|
|
97
105
|
schemaVersion: string,
|
|
106
|
+
rollupIds: string[], // the rulesResults entries that group this rule in this run; [] if none
|
|
107
|
+
data?: object, // page-level data a rule reports whatever its outcome; see below
|
|
108
|
+
wcagVersionScope?: { // present only when the target WCAG version changed this outcome
|
|
109
|
+
target: "2.0" | "2.1" | "2.2",
|
|
110
|
+
removedSc: string[],
|
|
111
|
+
coercedFrom: "fail"
|
|
112
|
+
},
|
|
98
113
|
error?: string // present only if the rule threw — see below
|
|
99
114
|
}
|
|
100
115
|
```
|
|
@@ -102,14 +117,19 @@ This is the exact shape of the object returned by `runDomRulesInPage(...)` / `ru
|
|
|
102
117
|
Notes:
|
|
103
118
|
|
|
104
119
|
- **`outcome` vs `outcomeNormalized`**: identical except `notApplicable` becomes `"inapplicable"` in `outcomeNormalized`. Both are provided so you can match either your own vocabulary or the engine's internal one.
|
|
105
|
-
- **`type: "manual"` rules can never report `outcome: "fail"`.** If a manual rule's own logic would have said `fail`, the engine coerces it to `cantTell` and appends an explanatory note to `error` — this is enforced centrally (`policy.coerceManualFailToCantTell`, on by default under the `a11y` policy contract; see [`POLICY.md`](./POLICY.md)), not something each rule has to remember. `fail` is reserved for deterministic,
|
|
106
|
-
- **`
|
|
120
|
+
- **`type: "manual"` rules can never report `outcome: "fail"`.** If a manual rule's own logic would have said `fail`, the engine coerces it to `cantTell` and appends an explanatory note to `error` — this is enforced centrally (`policy.coerceManualFailToCantTell`, on by default under the `a11y` policy contract; see [`POLICY.md`](./POLICY.md)), not something each rule has to remember. `fail` is reserved for deterministic, `type: "automatic"` findings only.
|
|
121
|
+
- **`rollupIds`** lists the composites in `rulesResults` that group this rule in this run. An empty list means the rule's findings appear in no rollup, so a consumer that reads only `rulesResults` never sees them; `heading-order`, for instance, belongs to no WCAG rollup.
|
|
122
|
+
- **`meta.normativeMappings`** is how a check result ties back to a WCAG Success Criterion — `[]` for rules with no formal WCAG mapping (this engine calls them advisory `type: "manual"` rules). See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md) for how these roll up. The list is not WCAG-only. When the scan asks for another standard (`engineOptions.mappings`, or a standard's profile), each WCAG criterion is followed by that standard's corresponding requirement, such as the EN 301 549 clause that restates it (`{ standard: "EN 301 549", version: "V3.2.1" | "V4.1.1", requirement: "9.1.1.1", title, wcagSc: ["1.1.1"] }`, one entry per version that includes the criterion, `wcagSc` naming the criteria it corresponds to), or the requirements of a standard mapped rule by rule that the rule checks (same shape, `wcagSc` empty for a requirement WCAG does not make, and any fields of that standard's own); and a rule may also cite WCAG's Understanding documents (`type: "Understanding"`). Filter on `standard` (and on the absence of `type`) before reading `requirement` as a Success Criterion. A composite's WCAG entry is always first. See [`WCAG_CONFORMANCE.md`](./WCAG_CONFORMANCE.md#en-301-549).
|
|
123
|
+
- **`wcagVersionScope`**: only present when the run's target WCAG version turned this rule's `fail` into a `cantTell` — today that means a rule mapped to SC 4.1.1 Parsing (`duplicate-id`) under the default 2.2 target, since 2.2 removed that criterion. `removedSc` lists the criteria that stopped existing, `target` is the version that removed them, and `coercedFrom` is the outcome the rule itself reported. The occurrences are the rule's own, unchanged — nothing was dropped, only the conformance verdict was. Absent on every other result, and **never** reported through `error`: nothing went wrong. See [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#filtering-by-wcag-version-21-vs-22).
|
|
124
|
+
- **`data`**: present only on a rule that reports something about the whole page, whatever its outcome. No core rule does today; a profile's rule may, such as one returning the page's own entry for a probe that compares pages (see `probes` in [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). Like `data.details` on an occurrence, it is not a stable contract.
|
|
107
125
|
- **`error`**: only present if the rule implementation threw an uncaught exception, or if the manual-fail coercion above fired. A thrown rule always surfaces as `outcome: "cantTell"` with `occurrences: []` and `error` set to the exception message — the engine never lets one broken rule crash the whole scan.
|
|
108
126
|
- **`engineOptions`** on each result is the *resolved* options object (after locale/contrast defaults were applied), not literally what you passed in — useful for confirming what a given rule actually saw, especially the resolved `locale` and `contrast.mode`/`contrast.rootCanvasFallback`.
|
|
109
127
|
|
|
110
128
|
## An occurrence (`occurrences[i]`)
|
|
111
129
|
|
|
112
|
-
|
|
130
|
+
Normally present only when `outcome` is `fail` or `cantTell`: a `pass` result has `occurrences: []`, since this engine does not enumerate the elements it passed, only the ones it flagged.
|
|
131
|
+
|
|
132
|
+
`notApplicable` is the one exception. A rule that had nothing to judge may attach a single occurrence saying why, and the contrast rules do exactly that when no text had a computable background — the difference between "checked, nothing to flag" and "could not check" is one this engine reports rather than hides. Such an occurrence describes the scan, not an element, so its `selector` is empty. Do not read `occurrences.length` as a violation count without checking `outcome` first.
|
|
113
133
|
|
|
114
134
|
```ts
|
|
115
135
|
{
|
|
@@ -119,6 +139,13 @@ Only present when `outcome` is `fail` or `cantTell` (a `pass`/`notApplicable` re
|
|
|
119
139
|
summary: string,
|
|
120
140
|
hint: string,
|
|
121
141
|
i18n: { summaryKey: string, hintKey: string, params: object } | null,
|
|
142
|
+
occurrenceOutcome?: "fail" | "cantTell", // present when the rule graded its findings into tiers
|
|
143
|
+
uncertainty?: { // present only on a cantTell-tier occurrence
|
|
144
|
+
code: "not-computable" | "runtime-dependent" | "spec-only"
|
|
145
|
+
| "equivalence-unknown" | "judgement-required" | "out-of-scope",
|
|
146
|
+
needed?: string, // what would settle the question
|
|
147
|
+
evidence?: object // what the rule did establish, rule-specific
|
|
148
|
+
},
|
|
122
149
|
data: {
|
|
123
150
|
visibilityFilter?: { eligible: boolean, targetSet: string, accEligible: boolean | null, reasons: string[] },
|
|
124
151
|
details?: object // rule-specific, non-normative — see below
|
|
@@ -128,18 +155,37 @@ Only present when `outcome` is `fail` or `cantTell` (a `pass`/`notApplicable` re
|
|
|
128
155
|
|
|
129
156
|
| Field | Meaning |
|
|
130
157
|
|---|---|
|
|
131
|
-
| `selector` | A best-effort CSS selector built to resolve back to the flagged element (see `helpers.buildSelector` in `RULE_AUTHORING.md`). Not guaranteed unique in adversarial DOM shapes, but the engine actively verifies it resolves to the reported element before using it. |
|
|
158
|
+
| `selector` | A best-effort CSS selector built to resolve back to the flagged element (see `helpers.buildSelector` in `RULE_AUTHORING.md`). Not guaranteed unique in adversarial DOM shapes, but the engine actively verifies it resolves to the reported element before using it. The exception is a rule whose finding *is* an absent element: `page-title-present` reports `head > title` with an `html` of `<title>(missing)</title>`, neither of which is on the page. Both are constants, so the fingerprint they feed stays stable, but do not treat `selector` as resolvable or `html` as real markup without checking the rule reported something that exists. |
|
|
132
159
|
| `html` | An outer-HTML snippet of the flagged element — use this as your primary "which element" signal when `includeShadowDom: true` (selectors don't pierce shadow boundaries). |
|
|
133
160
|
| `structuralPath` | The flagged element's sibling-index path from `documentElement` down to it (e.g. `[1, 0, 2]`) — `[]` if the element *is* `documentElement`, `null` if it couldn't be determined. A more robust element-identity mechanism than `selector` alone: it survives DOM changes a selector string wouldn't (an id/class rename, for instance), at the cost of not being usable as an actual CSS selector. Computed from the element reference when the rule kept one, otherwise by re-resolving `selector` against the document (same caveat as `selector` itself: a non-unique selector could resolve to a different element than intended). |
|
|
134
161
|
| `summary` | Human-readable, already localized ("This button has no accessible name."). |
|
|
135
162
|
| `hint` | Human-readable remediation guidance, already localized. |
|
|
136
163
|
| `i18n` | The raw translation keys behind `summary`/`hint`, if you want to re-render them in a different locale yourself without re-running the scan. `null` if the occurrence didn't use key-based i18n. |
|
|
137
164
|
| `data.visibilityFilter` | Present on most occurrences: why the engine considered this element eligible (or not) under whichever eligibility model the rule used. `eligible` is that result; `targetSet` says which model produced it (`'dom'`: raw DOM/CSS visibility — most rules; `'acc'`: accessibility-tree eligibility). `accEligible` mirrors `eligible` only when `targetSet` is `'acc'`, otherwise `null` — don't read it as a second, independent signal. `reasons` is a list of machine-readable exclusion codes when `eligible: false`. |
|
|
138
|
-
| `data.details` | Rule-specific structured data (
|
|
165
|
+
| `data.details` | Rule-specific structured data (computed metrics, resolved references) — **non-normative**: useful for building richer UI or debugging, but never changes what `outcome`/`severity` mean. Shape varies per rule; treat as best-effort extra context, not a stable contract. The one exception is `data.details.reasonCode`, which **is** stable: it identifies *which* of a rule's findings this is, and together with `ruleId` and `html` forms the fingerprint baselines and SARIF are keyed on. A rule may gain a new reason code in a minor release; a shipped one does not change. See [`API_STABILITY.md`](./API_STABILITY.md#finding-identity). |
|
|
166
|
+
| `occurrenceOutcome` | Which tier this occurrence belongs to, on a rule that graded its findings into a confident `fail` tier and a needs-review `cantTell` tier. A rule reporting one tier only omits it, in which case the result's own `outcome` is the occurrence's tier. This is why a `fail` result can carry `cantTell`-tier occurrences: the aggregate outcome stays singular so CI can still gate on it, without discarding the findings that only warranted review. |
|
|
167
|
+
| `uncertainty` | Why this finding could not be decided — see [Uncertainty codes](#uncertainty-codes) below. |
|
|
168
|
+
|
|
169
|
+
### Uncertainty codes
|
|
170
|
+
|
|
171
|
+
A `cantTell` says the engine did not decide. `uncertainty` says **why**, from a closed vocabulary, so a consumer can branch on the reason rather than parse a summary string. It is present only on a `cantTell`-tier occurrence: a `fail`-tier one would be claiming the rule both decided and did not, so the engine drops it.
|
|
172
|
+
|
|
173
|
+
| `code` | Meaning | Typical shape |
|
|
174
|
+
|---|---|---|
|
|
175
|
+
| `not-computable` | The evidence the rule needed could not be read in this environment. | A cross-origin stylesheet, a background colour that resolves to no value, an `src` that will not resolve. |
|
|
176
|
+
| `runtime-dependent` | The markup cannot settle it because script decides at runtime. | An `aria-controls` naming an element the widget builds when it opens. |
|
|
177
|
+
| `spec-only` | A real specification violation, but the exposed name, role and value survive it, so no Success Criterion is established as failed. | An ARIA attribute whose absence the specification supplies a default for. |
|
|
178
|
+
| `equivalence-unknown` | Two things may or may not serve the same purpose, and neither the markup nor the content settles it. | Two frames sharing an accessible name but embedding different resources. |
|
|
179
|
+
| `judgement-required` | The question is inherently a human call. | Whether an undersized target is essential; every `type: "manual"` rule. |
|
|
180
|
+
| `out-of-scope` | The finding is real but falls outside the standard this run targets. | A rule mapped only to a criterion the target WCAG version removed. |
|
|
181
|
+
|
|
182
|
+
`needed` states, in one sentence, what would settle the question — the thing a reviewer has to go and check. `evidence` carries what the rule *did* establish, so the reviewer starts from the engine's work rather than repeating it; its shape is rule-specific and, like `data.details`, not a stable contract. The `code` is: new codes may be added in a minor release, but an existing one does not change meaning, so branch on the codes you know and treat an unrecognised one as "needs review" rather than an error.
|
|
183
|
+
|
|
184
|
+
Every automatic rule that can report `cantTell` carries this, and a test holds that line so a new one cannot arrive without it. The `out-of-scope` code is attached by the engine rather than by a rule, on the same occurrences that produce a result-level `wcagVersionScope`. Manual rules do not carry it: `judgement-required` is what `type: "manual"` already means, so repeating it per occurrence would say nothing the result does not.
|
|
139
185
|
|
|
140
186
|
## A composite result (`rulesResults[i]`)
|
|
141
187
|
|
|
142
|
-
Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Shape is the same envelope as a check result, with composite-specific `data.details`:
|
|
188
|
+
Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wcag-1.1.1-non-text-content` rolls up 22 atomic rules). Under the profile of a standard with rollups of its own, there are also those, with the standard's wording as `title` and its name as `meta.standard`; they may be the only rollup some of its findings have, such as `heading-order`'s. Shape is the same envelope as a check result, with composite-specific `data.details`:
|
|
143
189
|
|
|
144
190
|
```ts
|
|
145
191
|
{
|
|
@@ -150,6 +196,9 @@ Composites roll multiple atomic rules up to one WCAG Success Criterion (e.g. `wc
|
|
|
150
196
|
data: {
|
|
151
197
|
details: {
|
|
152
198
|
reasonCode: string, // e.g. "composite.rollup.fail.anyFail"
|
|
199
|
+
standard?: string, // a standard's own rollup only, with version and criterion
|
|
200
|
+
version?: string,
|
|
201
|
+
criterion?: string,
|
|
153
202
|
checksIds: string[], // every atomic ruleId this composite rolls up
|
|
154
203
|
contributors: Array<{ testId: string, outcome: string, severity: string | null }>,
|
|
155
204
|
metrics: { failCount, cantTellCount, notApplicableCount, passCount, missingCount }
|
|
@@ -164,7 +213,7 @@ Rollup precedence (deterministic, in this order): **any contributor `fail` → c
|
|
|
164
213
|
|
|
165
214
|
| Outcome | Meaning | Can appear on `type: "manual"`? |
|
|
166
215
|
|---|---|---|
|
|
167
|
-
| `fail` | Deterministic,
|
|
216
|
+
| `fail` | Deterministic, normative violation — the decision procedure guesses at nothing. | No (coerced to `cantTell`) |
|
|
168
217
|
| `pass` | The rule's applicable target(s) exist and none were flagged. | Yes |
|
|
169
218
|
| `cantTell` | Requires human judgment — either genuinely ambiguous, or a `manual` rule's advisory finding. | Yes |
|
|
170
219
|
| `notApplicable` | The rule found no elements it applies to on this page/scope. | Yes |
|
|
@@ -176,6 +225,8 @@ Rollup precedence (deterministic, in this order): **any contributor `fail` → c
|
|
|
176
225
|
- `severity`: `minor` < `moderate` < `serious` < `critical` — the rule author's assessment of user impact, independent of `confidence`.
|
|
177
226
|
- `confidence`: `low` < `medium` < `high` — how certain the engine is that a `fail`/`cantTell` verdict is correct. Both are informational metadata for prioritization; neither changes `outcome`'s meaning.
|
|
178
227
|
|
|
228
|
+
A `fail` is not always `confidence: "high"`, and that is not a contradiction. The outcome describes the decision procedure — it resolved the question without guessing — while `confidence` describes the model that decision was made against. A handful of automatic rules decide deterministically against something that is itself an approximation (the curated WAI-ARIA role tables, the native-role mappings, an accessibility tree inferred from static markup) and report `medium`: `aria-required-children`, `aria-prohibited-children`, `aria-required-parent`, `aria-allowed-attr`, `form-control-programmatic-label-present`, `svg-image-text-alternative-present`, `video-poster-text-alternative-present` and `target-size-minimum`. `confidence` is on every result, so a consumer that wants only the most certain failures can gate on it directly; `policy.allowedConfidence` will not do it for you, since a disallowed value is replaced with the rule's own `defaultConfidence` rather than changing the outcome (see [`POLICY.md`](./POLICY.md)).
|
|
229
|
+
|
|
179
230
|
## Worked example
|
|
180
231
|
|
|
181
232
|
Scanning `<img src="logo.png">` (no `alt`) and `<button></button>` (no accessible name), scoped to just those two rules via `runOnly: { includeRuleIds: [...] }` (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md) — this is **not** a bare array):
|
package/docs/POLICY.md
CHANGED
|
@@ -68,4 +68,4 @@ Neither of these ever throws — policy resolution is designed to always produce
|
|
|
68
68
|
|
|
69
69
|
## Why this exists as a separate layer
|
|
70
70
|
|
|
71
|
-
Keeping outcome-integrity rules (like "manual rules can't fail") in a policy layer — rather than hard-coded into every rule, or worse, left to each rule author's discretion — means the guarantee holds even if a rule's own logic has a bug, and means different consumers can have different appetites for risk (a CI gate vs. an internal audit dashboard) without forking the rule set itself. This protects the engine's core guarantee: `fail` must always mean "deterministic,
|
|
71
|
+
Keeping outcome-integrity rules (like "manual rules can't fail") in a policy layer — rather than hard-coded into every rule, or worse, left to each rule author's discretion — means the guarantee holds even if a rule's own logic has a bug, and means different consumers can have different appetites for risk (a CI gate vs. an internal audit dashboard) without forking the rule set itself. This protects the engine's core guarantee: `fail` must always mean "deterministic, normative violation," full stop — see [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md#severity-and-confidence-values) for why that is not the same as "high-confidence."
|
package/docs/REPORT.md
CHANGED
|
@@ -12,10 +12,15 @@ Open `report.html` directly from disk. Works alongside any other output mode —
|
|
|
12
12
|
|
|
13
13
|
- **Hero**: one plain-language headline ("N of M applicable checks passed") plus a stacked bar and legend (icon + label + count — status is never color-only) broken down by outcome (`fail`/`cantTell`/`pass`/`notApplicable`).
|
|
14
14
|
- **Worth reviewing**: one card per rule with `fail`/`cantTell` occurrences (not one per occurrence — a rule with many identical occurrences is one thing worth attention, not many), each showing severity, WCAG SC chip(s), a representative occurrence, and the total occurrence count. Capped at the 24 highest-priority rules with an overflow note past that.
|
|
15
|
-
- **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion, its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
|
|
15
|
+
- **WCAG rollup**: grouped by conformance level (A / AA / AAA), sourced directly from the engine's own `rulesResults[]` composite rollups (`docs/WCAG_CONFORMANCE.md`) — one row per Success Criterion (with the EN 301 549 clause that restates it, where there is one and the scan asked for EN 301 549 clauses), its outcome, a pass/fail/needs-review/n/a breakdown, and which atomic rules contributed. This is real engine data, not an invented grouping — the same rollup you'd get from the raw JSON's `rulesResults`.
|
|
16
|
+
- **A standard's own rollup**: only when the scan ran the rules of a standard that has rollups of its own (its profile, or `engineOptions.optInRules`), one row per rollup, with the standard's wording, the requirements of the rules that decided the outcome, the breakdown and the contributing rules.
|
|
16
17
|
- **Full technical data** (collapsed by default): a scorecard (tiles per outcome) and a searchable, filterable (by outcome), paginated table of every individual occurrence across the whole scan.
|
|
17
18
|
|
|
18
|
-
The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
|
|
19
|
+
The meta bar under the header carries the rule count, occurrence count, engine tag, schema version, the WCAG version the scan targeted the `engineOptions.profile` it used (when there was one), the opt-in rule tags `engineOptions.optInRules` added (when it added any), and the locale the scan resolved to. Locale fallback is per-string and invisible in the text itself, so a report requested in a language the engine does not carry reads as an ordinary English one — the chip names the requested locale alongside the resolved one when the two differ. See [`I18N.md`](./I18N.md).
|
|
20
|
+
|
|
21
|
+
The whole page is written in the locale the scan resolved to: headings, table columns, outcome and severity names, the headline, the pager, and the date format, all from the `report_*` keys in the same dictionaries as the findings. `<html lang>` names that locale. The outcome codes in the occurrence filters (`fail`, `cantTell`, …) stay as they are, because they are the values a reader searches for in the JSON result.
|
|
22
|
+
|
|
23
|
+
One case keeps English labels: a scan in a language the engine does not ship, run with a caller-supplied `engineOptions.messages` dictionary. That dictionary is not part of the result, so the report cannot read labels from it; its findings are then marked with their own language (`lang="nl"`, for instance), so a screen reader still reads them with the right voice.
|
|
19
24
|
|
|
20
25
|
## Library usage
|
|
21
26
|
|
package/docs/RULE_AUTHORING.md
CHANGED
|
@@ -114,7 +114,7 @@ const meta = {
|
|
|
114
114
|
title: 'Non-text Content',
|
|
115
115
|
conformanceLevel: 'A'
|
|
116
116
|
}
|
|
117
|
-
],
|
|
117
|
+
], // WCAG only: EN 301 549 clauses are derived at build time (WCAG_CONFORMANCE.md#en-301-549)
|
|
118
118
|
|
|
119
119
|
defaultSeverity: 'minor' | 'moderate' | 'serious' | 'critical',
|
|
120
120
|
category: 'perceivable' | 'operable' | 'understandable' | 'robust',
|
|
@@ -141,8 +141,9 @@ The build/runtime resolves i18n by:
|
|
|
141
141
|
2) falling back to `en` if missing,
|
|
142
142
|
3) falling back to the literal `title`/`description` strings if still missing.
|
|
143
143
|
|
|
144
|
-
Add the key and its English text to `src/i18n/en.json
|
|
145
|
-
`npm run i18n:sync` so every other
|
|
144
|
+
Add the key and its English text to `src/i18n/en.json` (a profile's rule:
|
|
145
|
+
`profiles/<name>/i18n/en.json`), then run `npm run i18n:sync` so every other
|
|
146
|
+
locale picks it up. `npm test` fails if you
|
|
146
147
|
forget. See [`I18N.md`](./I18N.md).
|
|
147
148
|
|
|
148
149
|
#### `meta.tags`
|
|
@@ -150,6 +151,35 @@ Tags are used for grouping/filtering. Typical tag families in this ruleset inclu
|
|
|
150
151
|
- WCAG tagging: `wcag2a`, `wcag111`
|
|
151
152
|
- domain: `nontext`, `images`, plus element-specific tags
|
|
152
153
|
- nature: `atomic`, plus `automatic` or `manual`
|
|
154
|
+
- another standard's own requirement: that standard's rule tag (see below)
|
|
155
|
+
|
|
156
|
+
#### Rules for another standard's own requirements
|
|
157
|
+
A rule that checks something WCAG does not require, but another standard does (a doctype or presentational attributes, say), declares no WCAG mapping (`wcagSc: []`, `normativeMappings: []`) and carries that standard's rule tag. The tag makes it **opt-in**: it runs only under the standard's profile, a selection that includes the tag, or its own id, never in a default or WCAG run ([`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md#opt-in-rules)). That is what lets it report `fail`: its failures are failures of that standard, and only a scan targeting it sees them. Its module goes in that standard's profile rather than in `src/checks/`: `profiles/<key>/rules/automatic/` or `profiles/<key>/rules/manual/`. The build compiles it into the engine like any other rule. Its test and scenario page go in the profile too, in `profiles/<key>/tests/rules/` and `profiles/<key>/tests/fixtures/`. Map it to the standard's requirements the usual way (a row in the profile's rule map). Rule tags come from each standard's `ruleTag` in the registry, `src/coverage/standards.js` (a profile's from its `index.js`). The sample profile core's tests run against, `tests/fixtures/profiles/sample/`, has two such rules.
|
|
158
|
+
|
|
159
|
+
#### Rule variants
|
|
160
|
+
When another standard's requirement is a core rule with different thresholds (contrast at 7:1, say, or bold text large from 18.5px rather than WCAG's 14pt), write it as a **variant**, not a copy. The core rule declares the thresholds it reads from `ctx.config` as `settings`, with WCAG's values as defaults:
|
|
161
|
+
|
|
162
|
+
```js
|
|
163
|
+
// src/checks/automatic/contrast-minimum.js
|
|
164
|
+
const settings = { boldLargeMinPx: null, largeTextRatio: 3, normalTextRatio: 4.5 };
|
|
165
|
+
module.exports = { id, meta, runInPage, settings };
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
The variant is data, in the standard's profile:
|
|
169
|
+
|
|
170
|
+
```js
|
|
171
|
+
// profiles/<key>/rules/automatic/sample-contrast-enhanced.js
|
|
172
|
+
module.exports = {
|
|
173
|
+
id: 'sample-contrast-enhanced',
|
|
174
|
+
from: 'contrast-minimum',
|
|
175
|
+
config: { normalTextRatio: 7, largeTextRatio: 4.5 },
|
|
176
|
+
meta: { /* its own title, description, i18n, tags... as any rule's */ }
|
|
177
|
+
};
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
The build runs the base rule's `runInPage` and `applicability` under the variant's id and meta, with its `config` in `ctx.config`. A rule's settings are never the caller's: the runner drops a caller's value for one (`engineOptions.rules[ruleId]`), so the base rule always runs at its defaults and the variant at its `config`, while the caller's other config, such as `excludeSelectors`, still applies. A message key of the base's that starts with the base's prefix (its `meta.i18n.titleKey` without `_title`, `contrastMinimum`) is read from the variant's prefix instead (`sampleContrastEnhanced`), so the variant's dictionary has the same keys under its own prefix; the rule validator checks they exist. A fix to the base reaches every variant. The build refuses a variant whose base does not exist, is itself a variant, or declares no `settings`, and a setting the base does not declare or of another type. A base rule that caches verdicts depending on its settings keys those caches by them, as `contrast-minimum` does.
|
|
181
|
+
|
|
182
|
+
Add a setting to a core rule when a standard needs it, with a default that keeps the rule's behaviour; the settings a rule declares are for its variants, not for callers, and stay outside semver until a profile can live outside this repository (see [`API_STABILITY.md`](./API_STABILITY.md#explicitly-unstable-not-covered-by-semver)).
|
|
153
183
|
|
|
154
184
|
#### `meta.coverage.facetsBySc`
|
|
155
185
|
This is the repo’s explicit **coverage model** for an SC.
|
|
@@ -254,17 +284,15 @@ that one is on you.
|
|
|
254
284
|
|
|
255
285
|
## 6) Helpers contract used by rules (ctx.helpers)
|
|
256
286
|
|
|
257
|
-
Rules use helpers returned by `createDomHelpers()`.
|
|
287
|
+
Rules use helpers returned by `createDomHelpers()`. The most load-bearing ones —
|
|
288
|
+
`queryAllSmart` (query with shadow/hidden/exclude handling built in),
|
|
289
|
+
`getAccessibleNameInfo`/`getAccessibleDescriptionInfo`/`getTextAlternativeInfo` (naming),
|
|
290
|
+
`isAccTreeEligible`/`getEligibilityInfo` (visibility), `getRoleInfo`/`getFocusableInfo`
|
|
291
|
+
(role/focus) — cover most rules.
|
|
258
292
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
- `buildSimpleSelector`, `buildSelector`
|
|
263
|
-
- `isAccTreeEligible`, `getEligibilityInfo`
|
|
264
|
-
- `resolveIdRefs`, `getTextFromIdRefs`
|
|
265
|
-
- `getAccessibleNameInfo`, `getAccessibleDescriptionInfo`
|
|
266
|
-
- `getTextAlternativeInfo`
|
|
267
|
-
- `getRoleInfo`, `getFocusableInfo`
|
|
293
|
+
**See [`RULE_HELPERS.md`](./RULE_HELPERS.md) for the full reference** (~35 helpers plus
|
|
294
|
+
the `contrast.*`/`aria.*` namespaces), with what each one does and when to reach for it
|
|
295
|
+
instead of reimplementing the logic in a new rule.
|
|
268
296
|
|
|
269
297
|
### 6.1 Shadow DOM scanning
|
|
270
298
|
|
|
@@ -319,7 +347,23 @@ The rule must return:
|
|
|
319
347
|
|
|
320
348
|
Examples:
|
|
321
349
|
|
|
322
|
-
### 8.2
|
|
350
|
+
### 8.2 What `ctx` carries
|
|
351
|
+
|
|
352
|
+
`runInPage(ctx)` and `applicability(ctx)` receive the same object, built-in and custom rules alike:
|
|
353
|
+
|
|
354
|
+
| Field | What it is |
|
|
355
|
+
|---|---|
|
|
356
|
+
| `document`, `window` | The page being scanned. |
|
|
357
|
+
| `root` | The roots the scan covers: the document, or what `contextSelector` resolved to. |
|
|
358
|
+
| `contextSelector` | The selector that scoped the run, if any. |
|
|
359
|
+
| `rule` | The rule's resolved definition: `ruleId`, `defaultSeverity`, `defaultConfidence`, `type`, `meta`... |
|
|
360
|
+
| `config` | `engineOptions.rules[ruleId]`, this rule's settings, if the caller gave any (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)). |
|
|
361
|
+
| `standard` | The standard and version the run targets, `{ key, name, version }` (`{ key: 'en301549', name: 'EN 301 549', version: 'V4.1.1' }`), when a standard's profile selected the run; `null` otherwise (no profile, a WCAG profile, or rules chosen by tag or id). A rule whose behaviour differs between versions of its standard reads it here, and does what holds for every version when it is `null`. |
|
|
362
|
+
| `helpers` | The helpers documented in [`RULE_HELPERS.md`](./RULE_HELPERS.md). |
|
|
363
|
+
| `engineOptions` | The scan's options as resolved. |
|
|
364
|
+
| `inputs.probes` | Evidence the host application supplied (`engineOptions.probes`). |
|
|
365
|
+
|
|
366
|
+
### 8.3 Outcome conventions used by these rules
|
|
323
367
|
|
|
324
368
|
Automatic:
|
|
325
369
|
- `notApplicable` if no applicable targets
|
|
@@ -387,7 +431,8 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
|
|
|
387
431
|
### 11.1 The fixture file
|
|
388
432
|
|
|
389
433
|
- Path: `tests/fixtures/<rule-slug>-all-scenarios.html`, where `<rule-slug>` is the rule
|
|
390
|
-
id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`).
|
|
434
|
+
id itself (e.g. `tab-name-present` → `tab-name-present-all-scenarios.html`). A
|
|
435
|
+
profile's rule keeps it in the profile: `profiles/<name>/tests/fixtures/`.
|
|
391
436
|
- Structure: a real HTML page (`<!doctype html>`, `<title>`, minimal inline `<style>`)
|
|
392
437
|
containing numbered scenario blocks, each:
|
|
393
438
|
```html
|
|
@@ -407,6 +452,24 @@ exercised as real pages), not just embedded as strings inside `.test.js` files.
|
|
|
407
452
|
(eligible, FAIL)", "C. Ineligible (excluded from accessibility tree, skipped)").
|
|
408
453
|
- Cover every branch the rule's own logic distinguishes: pass, fail (each distinct
|
|
409
454
|
`reasonCode`), notApplicable/skipped, and — for manual rules — cantTell.
|
|
455
|
+
- A whole-document rule (`page-title-present`, `meta-refresh-timing-absent`, `region`)
|
|
456
|
+
can only demonstrate one outcome per page. Its fixture declares a single bare
|
|
457
|
+
`.case-title` with no `.case` wrapper, and the page itself is the case; the marker is
|
|
458
|
+
compared against the rule-level outcome, so `PASS` and `NEUTRAL` are distinguished
|
|
459
|
+
there. Cover the remaining branches with inline tests rather than near-identical
|
|
460
|
+
fixture files.
|
|
461
|
+
- One fixture shared by several rules that expect different things of the same case
|
|
462
|
+
(`tests/fixtures/contrast-all-scenarios.html` serves `contrast-minimum`,
|
|
463
|
+
`contrast-enhanced` and `contrast-computable`) carries a per-rule marker as a
|
|
464
|
+
`data-outcome-<rule-id>` attribute on the `.case`, which overrides the shared
|
|
465
|
+
`.case-title` for that rule. Use an attribute rather than more text when the rules
|
|
466
|
+
under test evaluate text: a `.case-title` added to a contrast case is one more text
|
|
467
|
+
node to check. A marker word the parser does not recognise (`MIXED`, `UNSTATED`)
|
|
468
|
+
asserts nothing, for a case whose outcome the fixture does not state.
|
|
469
|
+
- `npm run fixtures:markers:check` replays every fixture and fails when a marker no
|
|
470
|
+
longer matches what the rule reports; `scripts/data/fixture-markers.json` records the
|
|
471
|
+
cases that already disagree, so that set can only shrink. A profile's rules have
|
|
472
|
+
their record in the profile's own `scripts/data/fixture-markers.json`.
|
|
410
473
|
|
|
411
474
|
### 11.2 Known, acceptable exceptions to "one fixture, many cases"
|
|
412
475
|
|
|
@@ -482,6 +545,9 @@ npm run fixtures:index
|
|
|
482
545
|
This writes `tests/fixtures/INDEX.md` (human-readable), `tests/fixtures/index.json`
|
|
483
546
|
(machine-readable — every rule, its fixture path, and parsed pass/fail/cantTell case
|
|
484
547
|
counts, for external tooling to enumerate and load fixtures directly) and
|
|
485
|
-
`tests/fixtures/index.html` (the same listing as a browsable page).
|
|
548
|
+
`tests/fixtures/index.html` (the same listing as a browsable page). A profile's rules
|
|
549
|
+
get the same three files in the profile's `tests/fixtures/`, with paths relative to the
|
|
550
|
+
profile's folder. Commit all three
|
|
486
551
|
alongside the fixture and test changes. A rule shipped without its fixture is treated
|
|
487
|
-
the same as a rule shipped without tests — not done.
|
|
552
|
+
the same as a rule shipped without tests — not done. `npm run fixtures:check` reports
|
|
553
|
+
a stale index without rewriting it, and CI fails on one.
|