@surea11y/core 1.1.2 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +48 -4
- package/LICENSE +373 -21
- package/README.md +70 -1
- package/bin/core.js +240 -11
- package/docs/API_STABILITY.md +61 -0
- package/docs/BASELINE.md +66 -0
- package/docs/BINDING_AUTHORS_GUIDE.md +9 -9
- package/docs/CI_INTEGRATIONS.md +103 -0
- package/docs/CLI.md +80 -1
- package/docs/ENGINE_OPTIONS.md +4 -0
- package/docs/INTEGRATION.md +19 -1
- package/docs/OUTPUT_SCHEMA.md +2 -2
- package/docs/REPORT.md +33 -0
- package/docs/RULE_AUTHORING.md +31 -0
- package/docs/RULE_CATALOG.md +1 -1
- package/docs/SARIF.md +59 -0
- package/package.json +15 -4
- package/src/baseline.js +0 -0
- package/src/catalogs/composites.wcag.js +414 -450
- package/src/checks/automatic/area-alt-present.js +59 -25
- package/src/checks/automatic/aria-allowed-attr.js +193 -33
- package/src/checks/automatic/aria-allowed-role.js +21 -7
- package/src/checks/automatic/aria-braille-equivalent.js +32 -10
- package/src/checks/automatic/aria-conditional-attr.js +24 -7
- package/src/checks/automatic/aria-deprecated-role.js +22 -8
- package/src/checks/automatic/aria-hidden-body.js +50 -19
- package/src/checks/automatic/aria-hidden-focus.js +408 -53
- package/src/checks/automatic/aria-prohibited-attr.js +296 -22
- package/src/checks/automatic/aria-prohibited-children.js +125 -29
- package/src/checks/automatic/aria-required-attr.js +23 -8
- package/src/checks/automatic/aria-required-children.js +37 -14
- package/src/checks/automatic/aria-required-parent.js +48 -14
- package/src/checks/automatic/aria-role-name-present.js +47 -21
- package/src/checks/automatic/aria-roles-valid.js +22 -12
- package/src/checks/automatic/aria-valid-attr-value.js +28 -7
- package/src/checks/automatic/aria-valid-attr.js +17 -5
- package/src/checks/automatic/autocomplete-valid.js +74 -16
- package/src/checks/automatic/avoid-inline-spacing.js +20 -6
- package/src/checks/automatic/binary-control-name-present.js +60 -50
- package/src/checks/automatic/button-name-present.js +48 -18
- package/src/checks/automatic/bypass-blocks-present.js +50 -25
- package/src/checks/automatic/canvas-text-alternative-present.js +57 -26
- package/src/checks/automatic/combobox-name-present.js +38 -45
- package/src/checks/automatic/contrast-computable.js +361 -341
- package/src/checks/automatic/contrast-enhanced.js +487 -466
- package/src/checks/automatic/contrast-minimum.js +486 -465
- package/src/checks/automatic/css-orientation-lock.js +39 -9
- package/src/checks/automatic/definition-list-children-valid.js +40 -19
- package/src/checks/automatic/deprecated-elements-not-used.js +21 -7
- package/src/checks/automatic/dialog-name-present.js +37 -75
- package/src/checks/automatic/dlitem-parent-valid.js +23 -8
- package/src/checks/automatic/duplicate-id-aria.js +24 -6
- package/src/checks/automatic/embed-text-alternative-present.js +86 -35
- package/src/checks/automatic/form-control-programmatic-label-present.js +79 -196
- package/src/checks/automatic/form-control-single-label.js +47 -10
- package/src/checks/automatic/html-xml-lang-mismatch.js +43 -19
- package/src/checks/automatic/iframe-focusable-content.js +26 -11
- package/src/checks/automatic/iframe-name-present.js +31 -9
- package/src/checks/automatic/iframe-title-unique.js +29 -8
- package/src/checks/automatic/img-alt-present.js +47 -43
- package/src/checks/automatic/input-image-alt-present.js +143 -112
- package/src/checks/automatic/label-in-name.js +50 -22
- package/src/checks/automatic/language-page-present.js +117 -109
- package/src/checks/automatic/link-in-text-block.js +59 -19
- package/src/checks/automatic/link-name-present.js +45 -14
- package/src/checks/automatic/list-children-valid.js +26 -9
- package/src/checks/automatic/listbox-name-present.js +39 -19
- package/src/checks/automatic/listitem-parent-valid.js +18 -6
- package/src/checks/automatic/menuitem-name-present.js +39 -61
- package/src/checks/automatic/meta-refresh-no-exceptions.js +37 -8
- package/src/checks/automatic/meta-refresh-timing-absent.js +28 -6
- package/src/checks/automatic/meta-viewport-zoom-enabled.js +32 -7
- package/src/checks/automatic/meter-name-present.js +36 -33
- package/src/checks/automatic/nested-interactive-controls-absent.js +31 -10
- package/src/checks/automatic/object-text-alternative-present.js +91 -39
- package/src/checks/automatic/option-name-present.js +38 -21
- package/src/checks/automatic/page-title-present.js +26 -7
- package/src/checks/automatic/progressbar-name-present.js +41 -34
- package/src/checks/automatic/role-img-alt-present.js +209 -157
- package/src/checks/automatic/searchbox-name-present.js +39 -19
- package/src/checks/automatic/server-side-image-map-absent.js +23 -8
- package/src/checks/automatic/slider-name-present.js +40 -47
- package/src/checks/automatic/spinbutton-name-present.js +39 -19
- package/src/checks/automatic/summary-name-present.js +37 -17
- package/src/checks/automatic/svg-image-text-alternative-present.js +114 -47
- package/src/checks/automatic/svg-text-alternative-present.js +246 -226
- package/src/checks/automatic/tab-name-present.js +37 -60
- package/src/checks/automatic/table-headers-attr-valid.js +24 -8
- package/src/checks/automatic/table-th-has-data-cells.js +22 -8
- package/src/checks/automatic/target-size-minimum.js +118 -48
- package/src/checks/automatic/td-has-header.js +29 -11
- package/src/checks/automatic/textbox-name-present.js +39 -19
- package/src/checks/automatic/tooltip-name-present.js +37 -18
- package/src/checks/automatic/treeitem-name-present.js +38 -21
- package/src/checks/automatic/valid-lang.js +20 -6
- package/src/checks/automatic/video-poster-text-alternative-present.js +79 -36
- package/src/checks/manual/accesskeys-manual.js +14 -5
- package/src/checks/manual/area-alt-decorative-manual.js +192 -193
- package/src/checks/manual/area-alt-quality-manual.js +182 -141
- package/src/checks/manual/aria-checked-state-mismatch-manual.js +34 -11
- package/src/checks/manual/aria-text-manual.js +14 -6
- package/src/checks/manual/canvas-text-alternative-quality-manual.js +149 -114
- package/src/checks/manual/css-hidden-focus.js +196 -165
- package/src/checks/manual/embed-text-alternative-quality-manual.js +171 -160
- package/src/checks/manual/empty-heading-manual.js +24 -12
- package/src/checks/manual/empty-table-header-manual.js +21 -14
- package/src/checks/manual/focus-order-semantics-manual.js +45 -10
- package/src/checks/manual/form-control-programmatic-label-quality-manual.js +207 -246
- package/src/checks/manual/heading-order-manual.js +22 -12
- package/src/checks/manual/identical-links-same-purpose-manual.js +34 -12
- package/src/checks/manual/image-redundant-alt-manual.js +17 -7
- package/src/checks/manual/img-alt-decorative-manual.js +131 -96
- package/src/checks/manual/img-alt-quality-manual.js +176 -127
- package/src/checks/manual/input-image-alt-decorative-manual.js +125 -92
- package/src/checks/manual/input-image-alt-quality-manual.js +125 -92
- package/src/checks/manual/label-title-only-manual.js +14 -5
- package/src/checks/manual/landmark-banner-is-top-level-manual.js +95 -29
- package/src/checks/manual/landmark-contentinfo-is-top-level-manual.js +82 -29
- package/src/checks/manual/landmark-main-is-top-level-manual.js +64 -23
- package/src/checks/manual/landmark-no-duplicate-banner-manual.js +54 -23
- package/src/checks/manual/landmark-no-duplicate-contentinfo-manual.js +54 -23
- package/src/checks/manual/landmark-no-duplicate-main-manual.js +17 -8
- package/src/checks/manual/landmark-one-main-manual.js +35 -21
- package/src/checks/manual/landmark-unique-manual.js +73 -45
- package/src/checks/manual/link-name-quality-manual.js +43 -12
- package/src/checks/manual/media-transcript-present-manual.js +35 -22
- package/src/checks/manual/meta-viewport-large-manual.js +25 -6
- package/src/checks/manual/mouse-only-event-handlers-manual.js +38 -11
- package/src/checks/manual/no-autoplay-audio-manual.js +20 -6
- package/src/checks/manual/object-text-alternative-quality-manual.js +175 -154
- package/src/checks/manual/p-as-heading-manual.js +22 -7
- package/src/checks/manual/page-has-heading-one-manual.js +40 -23
- package/src/checks/manual/page-title-patterns-manual.js +87 -51
- package/src/checks/manual/presentation-role-conflict-manual.js +59 -19
- package/src/checks/manual/region-manual.js +275 -62
- package/src/checks/manual/scope-attr-valid-manual.js +11 -9
- package/src/checks/manual/scrollable-region-focusable-manual.js +37 -11
- package/src/checks/manual/skip-link-manual.js +35 -12
- package/src/checks/manual/svg-text-alternative-quality-manual.js +206 -165
- package/src/checks/manual/tabindex-manual.js +11 -9
- package/src/checks/manual/table-duplicate-name-manual.js +17 -7
- package/src/checks/manual/table-fake-caption-manual.js +24 -7
- package/src/checks/manual/video-caption-manual.js +15 -4
- package/src/checks/manual-review.js +56 -12
- package/src/core/aria-helpers.js +1128 -823
- package/src/core/contrast-helpers.js +1217 -1062
- package/src/core/dom-helpers.js +4176 -3886
- package/src/core/dom-runner.js +720 -596
- package/src/core/frame-messaging.js +189 -138
- package/src/core/frame-scan.js +94 -82
- package/src/core/rollup-composites.js +94 -102
- package/src/core/rule-meta.js +71 -35
- package/src/core.js +37939 -29121
- package/src/i18n/en.js +1194 -887
- package/src/i18n/fr.js +1136 -793
- package/src/policy/contracts.js +13 -13
- package/src/policy/resolvePolicy.js +48 -44
- package/src/report.js +502 -0
- package/src/sarif.js +175 -0
- package/surea11y.browser.js +36042 -0
package/README.md
CHANGED
|
@@ -198,6 +198,66 @@ For complete integration examples, see `docs/INTEGRATION.md`.
|
|
|
198
198
|
|
|
199
199
|
---
|
|
200
200
|
|
|
201
|
+
#### Standalone browser bundle
|
|
202
|
+
|
|
203
|
+
For a page that isn't driven by an automation framework at all — a manual
|
|
204
|
+
QA pass, a bookmarklet, a browser extension — the package also ships a
|
|
205
|
+
self-contained bundle that needs no `require`, no bundler, and no build
|
|
206
|
+
step:
|
|
207
|
+
|
|
208
|
+
```html
|
|
209
|
+
<script src="node_modules/@surea11y/core/surea11y.browser.js"></script>
|
|
210
|
+
<script>
|
|
211
|
+
const result = a11ycore.runa11yCoreInPage(
|
|
212
|
+
location.href, // pageUrl
|
|
213
|
+
null, // contextSelector
|
|
214
|
+
{}, // engineOptions
|
|
215
|
+
null // runOnly
|
|
216
|
+
);
|
|
217
|
+
|
|
218
|
+
console.log(result.checksResults.filter(r => r.outcome === "fail"));
|
|
219
|
+
</script>
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
Loading `surea11y.browser.js` defines a single global, `a11ycore`, exposing
|
|
223
|
+
the same `runa11yCoreInPage` function described above — calling it runs a
|
|
224
|
+
real scan against the page it's loaded into and returns the same result
|
|
225
|
+
shape documented in [Understanding the Results](#understanding-the-results).
|
|
226
|
+
|
|
227
|
+
`contextSelector`, `engineOptions`, and `runOnly` are the same three
|
|
228
|
+
arguments described throughout this README and `docs/ENGINE_OPTIONS.md` —
|
|
229
|
+
nothing about calling the engine changes just because it's loaded this
|
|
230
|
+
way. A scan scoped to one region, filtered to a specific WCAG level, with
|
|
231
|
+
a couple of known-noisy selectors excluded, looks like this:
|
|
232
|
+
|
|
233
|
+
```html
|
|
234
|
+
<script src="node_modules/@surea11y/core/surea11y.browser.js"></script>
|
|
235
|
+
<script>
|
|
236
|
+
const result = a11ycore.runa11yCoreInPage(
|
|
237
|
+
location.href,
|
|
238
|
+
"#main", // contextSelector: scan only this region
|
|
239
|
+
{
|
|
240
|
+
excludeSelectors: ["#cookie-banner", ".intercom-launcher"],
|
|
241
|
+
contrast: { mode: "auditorAssist" } // trade some false-positive protection for more findings
|
|
242
|
+
},
|
|
243
|
+
{ tags: ["wcag2a", "wcag2aa"] } // runOnly: WCAG 2.0 A/AA rules only
|
|
244
|
+
);
|
|
245
|
+
|
|
246
|
+
console.log(result.checksResults.filter(r => r.outcome === "fail"));
|
|
247
|
+
</script>
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
This bundle is generated from the same rule sources as the rest of the
|
|
251
|
+
engine (`npm run build` regenerates it alongside `src/core.js`), so its
|
|
252
|
+
behavior never drifts from the library's. It intentionally exposes only
|
|
253
|
+
`runa11yCoreInPage` — cross-frame scanning
|
|
254
|
+
(`runa11yCoreAcrossFrames`/`a11yCoreEnableFrameResponder`) requires the
|
|
255
|
+
embedded frame to also load the engine and opt in, which doesn't fit a
|
|
256
|
+
single dropped-in `<script>` tag; reach for the npm package directly if
|
|
257
|
+
you need that.
|
|
258
|
+
|
|
259
|
+
---
|
|
260
|
+
|
|
201
261
|
## Understanding the Results
|
|
202
262
|
|
|
203
263
|
Every scan returns a structured JSON document designed for both
|
|
@@ -270,7 +330,12 @@ and progressively explore more advanced features.
|
|
|
270
330
|
| Document | Description |
|
|
271
331
|
|---|---|
|
|
272
332
|
| `docs/OUTPUT_SCHEMA.md` | Complete description of every field returned by the engine. |
|
|
333
|
+
| `docs/API_STABILITY.md` | Semver guarantees on the result shape, and the rule-ID deprecation policy. |
|
|
273
334
|
| `docs/CLI.md` | CLI commands, options, exit codes and examples. |
|
|
335
|
+
| `docs/BASELINE.md` | CI baseline/allowlist: gate builds only on new violations. |
|
|
336
|
+
| `docs/REPORT.md` | Self-contained HTML report: browsable summary, WCAG rollup, filterable occurrence table. |
|
|
337
|
+
| `docs/SARIF.md` | SARIF 2.1.0 report for GitHub Code Scanning and other SARIF dashboards. |
|
|
338
|
+
| `docs/CI_INTEGRATIONS.md` | GitHub Actions and Bitbucket Pipelines templates wrapping the CLI. |
|
|
274
339
|
| `docs/ENGINE_OPTIONS.md` | Configuration, filtering, policies and localization. |
|
|
275
340
|
| `docs/INTEGRATION.md` | Using surea11y with jsdom, Playwright, Puppeteer, Selenium, Cypress and other drivers. |
|
|
276
341
|
| `docs/BINDING_AUTHORS_GUIDE.md` | Building new framework integrations on top of the engine. |
|
|
@@ -338,6 +403,8 @@ The repository is organised so that the accessibility engine, rule
|
|
|
338
403
|
implementations and supporting infrastructure remain clearly separated.
|
|
339
404
|
|
|
340
405
|
```text
|
|
406
|
+
surea11y.browser.js # Generated standalone browser bundle
|
|
407
|
+
|
|
341
408
|
bin/
|
|
342
409
|
core.js # CLI entry point
|
|
343
410
|
|
|
@@ -420,10 +487,12 @@ supported versions and the preferred disclosure process.
|
|
|
420
487
|
|
|
421
488
|
## License
|
|
422
489
|
|
|
423
|
-
This project is released under the
|
|
490
|
+
This project is released under the Mozilla Public License 2.0 (MPL-2.0).
|
|
424
491
|
|
|
425
492
|
See the accompanying `LICENSE` file for the complete license text.
|
|
426
493
|
|
|
494
|
+
MPL-2.0 is file-level copyleft: it applies to `@surea11y/core`'s own source files, not to code that merely depends on it. A project that installs `@surea11y/core` as a normal package dependency and imports its public API — without copying or modifying this repository's source files — is unaffected by MPL-2.0 and may keep its own license (including a permissive one like MIT).
|
|
495
|
+
|
|
427
496
|
---
|
|
428
497
|
|
|
429
498
|
## Final Notes
|
package/bin/core.js
CHANGED
|
@@ -21,6 +21,9 @@ const fs = require('fs');
|
|
|
21
21
|
const path = require('path');
|
|
22
22
|
|
|
23
23
|
const pkg = require('../package.json');
|
|
24
|
+
const { buildBaselineEntries, matchBaseline } = require('../src/baseline.js');
|
|
25
|
+
const { renderHtmlReport } = require('../src/report.js');
|
|
26
|
+
const { renderSarifReport } = require('../src/sarif.js');
|
|
24
27
|
|
|
25
28
|
// Piping output to `head`/`less`/etc. closes stdout early — without this,
|
|
26
29
|
// the next write throws an unhandled EPIPE and crashes with a raw stack
|
|
@@ -43,18 +46,31 @@ Options:
|
|
|
43
46
|
--exclude-rules <ids> Comma-separated rule IDs to exclude
|
|
44
47
|
--tags <tags> Comma-separated tags to run (e.g. wcag2a,wcag2aa)
|
|
45
48
|
--context <selector> CSS selector to scope the scan to a subtree
|
|
49
|
+
--custom-rules <path> Load runtime custom rules from a JS file (repeatable)
|
|
50
|
+
--write-baseline <path> Write every current "fail" occurrence to <path>; never fails the build
|
|
51
|
+
--baseline <path> Gate only on occurrences not already recorded in <path>
|
|
52
|
+
--html <path> Write a self-contained, browsable HTML report to <path>
|
|
53
|
+
--sarif <path> Write a SARIF 2.1.0 report to <path> (for GitHub Code Scanning etc.)
|
|
46
54
|
-h, --help Show this help
|
|
47
55
|
-v, --version Show the installed version
|
|
48
56
|
|
|
49
57
|
Exit codes:
|
|
50
|
-
0 scan completed, no "fail" outcomes
|
|
51
|
-
1 scan completed, at least one "fail" outcome
|
|
58
|
+
0 scan completed, no "fail" outcomes (or no *new* ones, with --baseline)
|
|
59
|
+
1 scan completed, at least one "fail" outcome (or *new* one, with --baseline)
|
|
52
60
|
2 usage error or the scan itself could not run (bad path/URL, network failure, etc.)
|
|
53
61
|
|
|
54
62
|
Examples:
|
|
55
63
|
surea11y scan ./index.html
|
|
56
64
|
surea11y scan https://example.com/ --tags wcag2a,wcag2aa
|
|
57
65
|
surea11y scan ./index.html --json > result.json
|
|
66
|
+
surea11y scan ./index.html --write-baseline baseline.json
|
|
67
|
+
surea11y scan ./index.html --baseline baseline.json
|
|
68
|
+
surea11y scan ./index.html --html report.html
|
|
69
|
+
surea11y scan ./index.html --baseline baseline.json --sarif results.sarif
|
|
70
|
+
surea11y scan ./index.html --custom-rules ./a11y-rules.js
|
|
71
|
+
|
|
72
|
+
See docs/BASELINE.md for the baseline/allowlist mechanism, docs/REPORT.md for the HTML report,
|
|
73
|
+
docs/SARIF.md for the SARIF report, docs/CLI.md#custom-rules for custom rules.
|
|
58
74
|
`);
|
|
59
75
|
}
|
|
60
76
|
|
|
@@ -81,6 +97,21 @@ function parseArgs(argv) {
|
|
|
81
97
|
case '--context':
|
|
82
98
|
out.context = argv[++i];
|
|
83
99
|
break;
|
|
100
|
+
case '--custom-rules':
|
|
101
|
+
(out.customRules = out.customRules || []).push(argv[++i]);
|
|
102
|
+
break;
|
|
103
|
+
case '--baseline':
|
|
104
|
+
out.baseline = argv[++i];
|
|
105
|
+
break;
|
|
106
|
+
case '--write-baseline':
|
|
107
|
+
out.writeBaseline = argv[++i];
|
|
108
|
+
break;
|
|
109
|
+
case '--html':
|
|
110
|
+
out.html = argv[++i];
|
|
111
|
+
break;
|
|
112
|
+
case '--sarif':
|
|
113
|
+
out.sarif = argv[++i];
|
|
114
|
+
break;
|
|
84
115
|
case '-h':
|
|
85
116
|
case '--help':
|
|
86
117
|
out.help = true;
|
|
@@ -102,7 +133,12 @@ function isUrl(s) {
|
|
|
102
133
|
|
|
103
134
|
function formatError(err) {
|
|
104
135
|
const base = err && err.message ? err.message : String(err);
|
|
105
|
-
const cause =
|
|
136
|
+
const cause =
|
|
137
|
+
err && err.cause && err.cause.message
|
|
138
|
+
? err.cause.message
|
|
139
|
+
: err && err.cause
|
|
140
|
+
? String(err.cause)
|
|
141
|
+
: '';
|
|
106
142
|
return cause ? `${base}: ${cause}` : base;
|
|
107
143
|
}
|
|
108
144
|
|
|
@@ -122,7 +158,7 @@ async function loadHtml(target) {
|
|
|
122
158
|
return { html: fs.readFileSync(resolved, 'utf8'), url: `file://${resolved}` };
|
|
123
159
|
}
|
|
124
160
|
|
|
125
|
-
function buildEngineOptions(args) {
|
|
161
|
+
function buildEngineOptions(args, customRules) {
|
|
126
162
|
const engineOptions = {};
|
|
127
163
|
if (args.locale) engineOptions.locale = args.locale;
|
|
128
164
|
if (args.rules || args.excludeRules) {
|
|
@@ -131,23 +167,66 @@ function buildEngineOptions(args) {
|
|
|
131
167
|
if (args.excludeRules) engineOptions.rules.exclude = args.excludeRules;
|
|
132
168
|
}
|
|
133
169
|
if (args.tags) engineOptions.tags = { include: args.tags };
|
|
170
|
+
if (customRules && customRules.length) engineOptions.customRules = customRules;
|
|
134
171
|
return engineOptions;
|
|
135
172
|
}
|
|
136
173
|
|
|
137
|
-
|
|
174
|
+
// Loads one --custom-rules file. The CLI runs custom rules in the same
|
|
175
|
+
// process/JS realm as the scan itself (unlike a binding whose engineOptions
|
|
176
|
+
// crosses page.evaluate()'s serialization boundary), so a descriptor's
|
|
177
|
+
// runInPage/applicability may be a real function -- see the customRules
|
|
178
|
+
// section of docs/ENGINE_OPTIONS.md for the full descriptor contract this
|
|
179
|
+
// mirrors. Validated up front (not left to the engine's own silent-skip of
|
|
180
|
+
// invalid entries) so a typo in a hand-authored file fails loudly as a usage
|
|
181
|
+
// error instead of quietly producing a scan with one fewer rule than expected.
|
|
182
|
+
function loadCustomRulesFile(customRulesPath) {
|
|
183
|
+
const resolved = path.resolve(process.cwd(), customRulesPath);
|
|
184
|
+
|
|
185
|
+
let loaded;
|
|
186
|
+
try {
|
|
187
|
+
loaded = require(resolved);
|
|
188
|
+
} catch (err) {
|
|
189
|
+
throw new Error(`Could not load custom rules file "${customRulesPath}": ${formatError(err)}`, {
|
|
190
|
+
cause: err
|
|
191
|
+
});
|
|
192
|
+
}
|
|
193
|
+
|
|
194
|
+
const descriptors = Array.isArray(loaded) ? loaded : [loaded];
|
|
195
|
+
|
|
196
|
+
for (const d of descriptors) {
|
|
197
|
+
const hasId = d && typeof d === 'object' && typeof d.id === 'string' && d.id.trim();
|
|
198
|
+
const hasRunInPage =
|
|
199
|
+
d &&
|
|
200
|
+
(typeof d.runInPage === 'function' ||
|
|
201
|
+
(typeof d.runInPage === 'string' && d.runInPage.trim()));
|
|
202
|
+
if (!hasId || !hasRunInPage) {
|
|
203
|
+
throw new Error(
|
|
204
|
+
`Custom rules file "${customRulesPath}" must export a rule descriptor ({ id, runInPage, ... }) or an array of them. See docs/CLI.md#custom-rules.`
|
|
205
|
+
);
|
|
206
|
+
}
|
|
207
|
+
}
|
|
208
|
+
|
|
209
|
+
return descriptors;
|
|
210
|
+
}
|
|
211
|
+
|
|
212
|
+
function printSummary(result, baselineMatch) {
|
|
138
213
|
const byOutcome = { pass: 0, fail: 0, cantTell: 0, notApplicable: 0 };
|
|
139
214
|
for (const r of result.checksResults) {
|
|
140
215
|
if (Object.prototype.hasOwnProperty.call(byOutcome, r.outcome)) byOutcome[r.outcome] += 1;
|
|
141
216
|
}
|
|
142
217
|
|
|
143
218
|
process.stdout.write(`\nsurea11y scan: ${result.url || '(no url)'}\n`);
|
|
144
|
-
process.stdout.write(
|
|
219
|
+
process.stdout.write(
|
|
220
|
+
` pass: ${byOutcome.pass} fail: ${byOutcome.fail} cantTell: ${byOutcome.cantTell} notApplicable: ${byOutcome.notApplicable}\n\n`
|
|
221
|
+
);
|
|
145
222
|
|
|
146
223
|
const fails = result.checksResults.filter((r) => r.outcome === 'fail');
|
|
147
224
|
if (fails.length) {
|
|
148
225
|
process.stdout.write(`FAIL (${fails.length} rule(s)):\n`);
|
|
149
226
|
for (const r of fails) {
|
|
150
|
-
process.stdout.write(
|
|
227
|
+
process.stdout.write(
|
|
228
|
+
`\n ${r.ruleId} (${r.severity}, ${r.occurrences.length} occurrence(s))\n`
|
|
229
|
+
);
|
|
151
230
|
for (const occ of r.occurrences.slice(0, 5)) {
|
|
152
231
|
process.stdout.write(` - ${occ.selector || '(no selector)'}\n ${occ.summary}\n`);
|
|
153
232
|
if (occ.hint) process.stdout.write(` hint: ${occ.hint}\n`);
|
|
@@ -161,8 +240,57 @@ function printSummary(result) {
|
|
|
161
240
|
|
|
162
241
|
const cantTells = result.checksResults.filter((r) => r.outcome === 'cantTell');
|
|
163
242
|
if (cantTells.length) {
|
|
164
|
-
process.stdout.write(
|
|
243
|
+
process.stdout.write(
|
|
244
|
+
`cantTell — needs human review (${cantTells.length} rule(s)): ${cantTells.map((r) => r.ruleId).join(', ')}\n\n`
|
|
245
|
+
);
|
|
246
|
+
}
|
|
247
|
+
|
|
248
|
+
if (baselineMatch) {
|
|
249
|
+
process.stdout.write(
|
|
250
|
+
`baseline: ${baselineMatch.knownCount} known, ${baselineMatch.newCount} new, ${baselineMatch.staleCount} stale (no longer detected)\n`
|
|
251
|
+
);
|
|
252
|
+
if (baselineMatch.newCount) {
|
|
253
|
+
process.stdout.write(`\nNEW (not in baseline, ${baselineMatch.newCount} occurrence(s)):\n`);
|
|
254
|
+
for (const occ of baselineMatch.newOccurrences.slice(0, 5)) {
|
|
255
|
+
process.stdout.write(
|
|
256
|
+
` - ${occ.ruleId}: ${occ.selector || '(no selector)'}\n ${occ.summary}\n`
|
|
257
|
+
);
|
|
258
|
+
}
|
|
259
|
+
if (baselineMatch.newOccurrences.length > 5) {
|
|
260
|
+
process.stdout.write(` ... and ${baselineMatch.newOccurrences.length - 5} more\n`);
|
|
261
|
+
}
|
|
262
|
+
}
|
|
263
|
+
process.stdout.write('\n');
|
|
264
|
+
}
|
|
265
|
+
}
|
|
266
|
+
|
|
267
|
+
function loadBaselineFile(baselinePath) {
|
|
268
|
+
let raw;
|
|
269
|
+
try {
|
|
270
|
+
raw = fs.readFileSync(baselinePath, 'utf8');
|
|
271
|
+
} catch (err) {
|
|
272
|
+
throw new Error(
|
|
273
|
+
`Could not read baseline file "${baselinePath}": ${formatError(err)}. Run with --write-baseline ${baselinePath} first to create one.`,
|
|
274
|
+
{ cause: err }
|
|
275
|
+
);
|
|
276
|
+
}
|
|
277
|
+
|
|
278
|
+
let parsed;
|
|
279
|
+
try {
|
|
280
|
+
parsed = JSON.parse(raw);
|
|
281
|
+
} catch (err) {
|
|
282
|
+
throw new Error(`Baseline file "${baselinePath}" is not valid JSON: ${formatError(err)}`, {
|
|
283
|
+
cause: err
|
|
284
|
+
});
|
|
285
|
+
}
|
|
286
|
+
|
|
287
|
+
if (!parsed || parsed.version !== 1 || !Array.isArray(parsed.entries)) {
|
|
288
|
+
throw new Error(
|
|
289
|
+
`Baseline file "${baselinePath}" is not a supported baseline (expected { version: 1, entries: [...] }). Regenerate it with --write-baseline.`
|
|
290
|
+
);
|
|
165
291
|
}
|
|
292
|
+
|
|
293
|
+
return parsed;
|
|
166
294
|
}
|
|
167
295
|
|
|
168
296
|
async function runScan(args) {
|
|
@@ -173,6 +301,38 @@ async function runScan(args) {
|
|
|
173
301
|
return;
|
|
174
302
|
}
|
|
175
303
|
|
|
304
|
+
if (args.baseline && args.writeBaseline) {
|
|
305
|
+
process.stderr.write(
|
|
306
|
+
'Error: --baseline and --write-baseline cannot be used together in the same run. See --help.\n'
|
|
307
|
+
);
|
|
308
|
+
process.exitCode = 2;
|
|
309
|
+
return;
|
|
310
|
+
}
|
|
311
|
+
|
|
312
|
+
let baselineFile = null;
|
|
313
|
+
if (args.baseline) {
|
|
314
|
+
try {
|
|
315
|
+
baselineFile = loadBaselineFile(args.baseline);
|
|
316
|
+
} catch (err) {
|
|
317
|
+
process.stderr.write(`Error: ${formatError(err)}\n`);
|
|
318
|
+
process.exitCode = 2;
|
|
319
|
+
return;
|
|
320
|
+
}
|
|
321
|
+
}
|
|
322
|
+
|
|
323
|
+
let customRules = [];
|
|
324
|
+
if (args.customRules && args.customRules.length) {
|
|
325
|
+
try {
|
|
326
|
+
for (const customRulesPath of args.customRules) {
|
|
327
|
+
customRules = customRules.concat(loadCustomRulesFile(customRulesPath));
|
|
328
|
+
}
|
|
329
|
+
} catch (err) {
|
|
330
|
+
process.stderr.write(`Error: ${formatError(err)}\n`);
|
|
331
|
+
process.exitCode = 2;
|
|
332
|
+
return;
|
|
333
|
+
}
|
|
334
|
+
}
|
|
335
|
+
|
|
176
336
|
let html, url;
|
|
177
337
|
try {
|
|
178
338
|
({ html, url } = await loadHtml(target));
|
|
@@ -186,7 +346,9 @@ async function runScan(args) {
|
|
|
186
346
|
try {
|
|
187
347
|
({ JSDOM } = require('jsdom'));
|
|
188
348
|
} catch {
|
|
189
|
-
process.stderr.write(
|
|
349
|
+
process.stderr.write(
|
|
350
|
+
'Error: the surea11y CLI requires jsdom. Run `npm install jsdom` (it should already be a dependency of this package — this likely means a broken install).\n'
|
|
351
|
+
);
|
|
190
352
|
process.exitCode = 2;
|
|
191
353
|
return;
|
|
192
354
|
}
|
|
@@ -199,11 +361,76 @@ async function runScan(args) {
|
|
|
199
361
|
|
|
200
362
|
let result;
|
|
201
363
|
try {
|
|
202
|
-
result = runDomRulesInPage(
|
|
364
|
+
result = runDomRulesInPage(
|
|
365
|
+
url,
|
|
366
|
+
args.context || null,
|
|
367
|
+
buildEngineOptions(args, customRules),
|
|
368
|
+
null
|
|
369
|
+
);
|
|
203
370
|
} finally {
|
|
204
371
|
dom.window.close();
|
|
205
372
|
}
|
|
206
373
|
|
|
374
|
+
if (args.html) {
|
|
375
|
+
fs.writeFileSync(
|
|
376
|
+
args.html,
|
|
377
|
+
renderHtmlReport(result, { title: `surea11y scan report — ${target}` })
|
|
378
|
+
);
|
|
379
|
+
process.stderr.write(`Wrote HTML report to: ${args.html}\n`);
|
|
380
|
+
}
|
|
381
|
+
|
|
382
|
+
if (args.sarif) {
|
|
383
|
+
fs.writeFileSync(
|
|
384
|
+
args.sarif,
|
|
385
|
+
renderSarifReport(result, {
|
|
386
|
+
toolVersion: pkg.version,
|
|
387
|
+
informationUri: (pkg.homepage || '').replace(/#.*$/, ''),
|
|
388
|
+
baselineEntries: baselineFile ? baselineFile.entries : undefined
|
|
389
|
+
})
|
|
390
|
+
);
|
|
391
|
+
process.stderr.write(`Wrote SARIF report to: ${args.sarif}\n`);
|
|
392
|
+
}
|
|
393
|
+
|
|
394
|
+
if (args.writeBaseline) {
|
|
395
|
+
const entries = buildBaselineEntries(result);
|
|
396
|
+
const payload = { version: 1, generatedAt: new Date().toISOString(), entries };
|
|
397
|
+
fs.writeFileSync(args.writeBaseline, JSON.stringify(payload, null, 2) + '\n');
|
|
398
|
+
|
|
399
|
+
if (args.json) {
|
|
400
|
+
process.stdout.write(
|
|
401
|
+
JSON.stringify(
|
|
402
|
+
{
|
|
403
|
+
...result,
|
|
404
|
+
baseline: { mode: 'write', path: args.writeBaseline, entries: entries.length }
|
|
405
|
+
},
|
|
406
|
+
null,
|
|
407
|
+
2
|
|
408
|
+
) + '\n'
|
|
409
|
+
);
|
|
410
|
+
} else {
|
|
411
|
+
printSummary(result);
|
|
412
|
+
}
|
|
413
|
+
process.stderr.write(
|
|
414
|
+
`Wrote ${entries.length} occurrence(s) to baseline: ${args.writeBaseline}\n`
|
|
415
|
+
);
|
|
416
|
+
process.exitCode = 0;
|
|
417
|
+
return;
|
|
418
|
+
}
|
|
419
|
+
|
|
420
|
+
if (baselineFile) {
|
|
421
|
+
const match = matchBaseline(result, baselineFile.entries);
|
|
422
|
+
|
|
423
|
+
if (args.json) {
|
|
424
|
+
process.stdout.write(
|
|
425
|
+
JSON.stringify({ ...result, baseline: { mode: 'check', ...match } }, null, 2) + '\n'
|
|
426
|
+
);
|
|
427
|
+
} else {
|
|
428
|
+
printSummary(result, match);
|
|
429
|
+
}
|
|
430
|
+
process.exitCode = match.newCount > 0 ? 1 : 0;
|
|
431
|
+
return;
|
|
432
|
+
}
|
|
433
|
+
|
|
207
434
|
if (args.json) {
|
|
208
435
|
process.stdout.write(JSON.stringify(result, null, 2) + '\n');
|
|
209
436
|
} else {
|
|
@@ -229,7 +456,9 @@ async function main() {
|
|
|
229
456
|
|
|
230
457
|
const [command, ...rest] = args._;
|
|
231
458
|
if (command !== 'scan') {
|
|
232
|
-
process.stderr.write(
|
|
459
|
+
process.stderr.write(
|
|
460
|
+
`Error: unknown command "${command}". Only "scan" is supported. See --help.\n`
|
|
461
|
+
);
|
|
233
462
|
process.exitCode = 2;
|
|
234
463
|
return;
|
|
235
464
|
}
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
# API stability & versioning
|
|
2
|
+
|
|
3
|
+
`@surea11y/core` has real downstream consumers today: 5 first-party framework bindings (Playwright, Puppeteer, Selenium, WebdriverIO, Cypress) published to npm, plus a Jest/Vitest matcher (`@surea11y/test-matchers`, `toHaveNoA11yViolations()`) — all pinned to a `^1.1.0`-style semver range. Until now, "what counts as a breaking change" was implicit — discoverable only by reading source, not written down anywhere. This document makes that contract explicit.
|
|
4
|
+
|
|
5
|
+
## Stable fields (covered by semver)
|
|
6
|
+
|
|
7
|
+
Removing, renaming, or changing the type/meaning of any of these is a **major** version bump:
|
|
8
|
+
|
|
9
|
+
- Top-level result: `engine.tag`, `engine.schemaVersion`, `url`, `checksResults` (an array), `rulesResults` (an array).
|
|
10
|
+
- Each `checksResults[i]` / `rulesResults[i]` entry: `ruleId`, `outcome`, `outcomeNormalized`, `severity`, `confidence`, `type`, `title`, `description`, `meta` (including `meta.normativeMappings`, `meta.deprecated`/`.deprecation` — see below), `engineOptions`, `schemaVersion`.
|
|
11
|
+
- Each occurrence (`occurrences[i]`): `selector`, `html`, `summary`, `hint`, `i18n`, `structuralPath`.
|
|
12
|
+
- The rule **catalog** (`getChecksCatalog()`/`getRulesCatalog()`, a separate surface from a scan result — see `RULE_AUTHORING.md`): `ruleId`, `title`, `description`, `tags`, `wcagSc`, `normativeMappings`, `defaultSeverity`, `defaultConfidence`, `type`, `deprecated`/`.deprecation`. Note `tags` lives here, not on a per-scan `checksResults[i].meta` — the two surfaces intentionally carry different subsets of a rule's metadata.
|
|
13
|
+
|
|
14
|
+
This list is deliberately not a new, invented guarantee — it codifies what the 6 real consumers above (and `docs/OUTPUT_SCHEMA.md`'s own worked examples) already depend on today, either directly or as documented shape.
|
|
15
|
+
|
|
16
|
+
## Explicitly unstable (not covered by semver)
|
|
17
|
+
|
|
18
|
+
- `perfStats` and `ruleTimings` — internal timing/debug counters, only present when `engineOptions.perfStats`/`.profileRules` is set. Shape not covered by this document.
|
|
19
|
+
- `occurrences[i].data.details` — rule-specific, non-normative extra context. Shape varies per rule and may change in a patch release; treat as best-effort, not a stable contract (this was already noted in `docs/OUTPUT_SCHEMA.md` before this document existed).
|
|
20
|
+
- `ruleInterfaceVersion` / `ruleVersion` on a rule's meta — currently unused scaffolding (every rule defaults to the same two static strings; nothing meaningfully sets or consumes them today). Not part of this contract until they're actually wired up to mean something.
|
|
21
|
+
|
|
22
|
+
## What triggers which version bump
|
|
23
|
+
|
|
24
|
+
- **Patch**: a correctness fix that changes *which* outcome a rule produces for the same input, without changing the shape or mechanism. Example: the fragment-scan applicability fix (`engineOptions.fragment`, see `ENGINE_OPTIONS.md`) changed several rules from incorrectly `fail`ing on a scoped subtree to correctly `notApplicable` — that's a patch, not a major bump, because no stable field's *shape* changed, only a bug got fixed. Don't over-index on "any output change = major" — bug fixes are expected to change output.
|
|
25
|
+
- **Minor**: adding a new stable field, adding a new rule to the catalog, or marking an existing rule `deprecated` (see below).
|
|
26
|
+
- **Major**: removing or renaming a stable field, changing a stable field's type or meaning, or removing a rule ID once its deprecation notice period has passed. Paired with an `engine.schemaVersion` bump specifically when the *shape* changes (as opposed to package-level major bumps for other reasons, e.g. dropping support for an old Node version).
|
|
27
|
+
|
|
28
|
+
`engine.schemaVersion` has been `"1.0.0"` since the engine's first release and has never needed a bump — nothing has changed a stable field's shape yet. Adding the `deprecated`/`deprecation` meta fields described below is purely additive (new optional fields, ignored safely by anything not looking for them), so it does **not** warrant a schema bump either — this is the policy's first real application.
|
|
29
|
+
|
|
30
|
+
## Rule-ID deprecation policy
|
|
31
|
+
|
|
32
|
+
A rule can be marked deprecated in its own `meta`:
|
|
33
|
+
|
|
34
|
+
```js
|
|
35
|
+
const meta = {
|
|
36
|
+
// ...
|
|
37
|
+
deprecated: true,
|
|
38
|
+
deprecation: {
|
|
39
|
+
replacedBy: 'new-rule-id', // or null if there's no direct replacement
|
|
40
|
+
reason: 'Why this rule is being retired.',
|
|
41
|
+
sinceVersion: '1.2.0' // the package version this was first marked deprecated in
|
|
42
|
+
}
|
|
43
|
+
};
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
`meta.deprecated: true` requires both `deprecation.reason` and `deprecation.sinceVersion` — `normalizeRuleMeta` (`src/core/rule-meta.js`) throws a clear build-time error otherwise, the same way it already validates `meta.i18n.titleKey`.
|
|
47
|
+
|
|
48
|
+
**A deprecated rule keeps running and producing results completely normally** — `pass`/`fail`/`cantTell`/`notApplicable` exactly as before. Deprecation is a catalog-level signal (visible via `getChecksCatalog()`, and in `docs/RULE_CATALOG.md`) for integrators to plan a migration on their own schedule — deliberately **not** an automatic exclusion (there is no `engineOptions.excludeDeprecated` flag). Silently dropping a rule's results the moment it's deprecated would be exactly the kind of surprise this document exists to prevent.
|
|
49
|
+
|
|
50
|
+
The process:
|
|
51
|
+
1. Mark the rule `deprecated: true` with `deprecation.reason`/`.replacedBy`/`.sinceVersion` set. Document it under `CHANGELOG.md`'s `### Deprecated` section (a standard Keep-a-Changelog category that's been in this project's changelog template since the beginning but never actually used until now).
|
|
52
|
+
2. Leave it running normally for at least one full minor version cycle after the deprecation, so integrators pinned to `^x.y.0` have a real chance to see it before it's gone.
|
|
53
|
+
3. Remove the rule file entirely in a future **major** version, documented under `### Removed`.
|
|
54
|
+
|
|
55
|
+
No rule has been deprecated yet as of this document's introduction — this is the mechanism, ready for the first real case.
|
|
56
|
+
|
|
57
|
+
## See also
|
|
58
|
+
|
|
59
|
+
- [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md) — the full result shape this document's stability rules apply to.
|
|
60
|
+
- [`RULE_AUTHORING.md`](./RULE_AUTHORING.md) §4.1 — the full rule `meta` contract, including `deprecated`/`deprecation`.
|
|
61
|
+
- [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md) — `engineOptions.fragment`, referenced above as a worked example of a patch-level behavior fix.
|
package/docs/BASELINE.md
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
# Baseline / allowlist
|
|
2
|
+
|
|
3
|
+
A strict CI gate ("fail the build on any `fail` outcome") is an adoption blocker for a team scanning an existing, imperfect site for the first time — they can't ship anything until every pre-existing violation is fixed. A baseline lets a team say "these N are already known — don't gate on them, but fail the moment a genuinely new one appears."
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
# Once: record every current fail occurrence, does not fail the build.
|
|
7
|
+
surea11y scan ./dist/index.html --write-baseline baseline.json
|
|
8
|
+
|
|
9
|
+
# Commit baseline.json to git.
|
|
10
|
+
|
|
11
|
+
# From then on, in CI: only a NEW (not-yet-baselined) fail occurrence gates the build.
|
|
12
|
+
surea11y scan ./dist/index.html --baseline baseline.json
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
The baseline file is meant to be committed to git, so accepting more accessibility debt becomes an explicit, reviewable decision — a new violation shows up as a new row in the file's diff in a PR, not a silent change in a count.
|
|
16
|
+
|
|
17
|
+
## When this is useful
|
|
18
|
+
|
|
19
|
+
- **Adopting scanning on an existing site.** The motivating case above: a first scan on a mature site can turn up hundreds of pre-existing violations. Baselining them once unblocks gating immediately instead of requiring "fix everything first."
|
|
20
|
+
- **Incremental remediation with a visible burn-down.** Fix violations in batches, then periodically regenerate the baseline with `--write-baseline`. Each regeneration's git diff *shrinks* as entries disappear — the file itself becomes a progress tracker, and a PR that fixes a batch of issues shows exactly what got cleaned up.
|
|
21
|
+
- **Third-party/vendor content you don't control.** A page embeds a widget (chat bubble, ad unit, payment iframe) with known, unfixable-by-you violations. Baselining just those specific occurrences is more precise than excluding the whole subtree via `excludeSelectors` (see [`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)) — new issues introduced anywhere else near it, including inside your own code, still gate normally.
|
|
22
|
+
- **Regression safety net during a large refactor.** Mid-migration, some rules may legitimately fail temporarily in ways already tracked and being fixed across many PRs. Baselining the known-in-progress set still catches *unrelated* new regressions from other PRs during the same window, instead of turning the gate off entirely or blocking every PR on it.
|
|
23
|
+
- **Staged rollout of a new or stricter rule.** Turning on a rule (or a whole WCAG level) the codebase isn't clean against yet: baseline the existing gaps, enable gating immediately, and prevent any *new* violations of that rule while the backlog is cleaned up separately — rather than waiting to enable the rule until the codebase is already compliant.
|
|
24
|
+
|
|
25
|
+
## How matching works
|
|
26
|
+
|
|
27
|
+
Nothing in an occurrence's shape is a perfect, page-position-independent identity (see [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md)): `selector` and `structuralPath` are both derived from the element's position in the DOM, so either can shift when *unrelated* markup changes elsewhere on the page, even though the flagged element itself never changed.
|
|
28
|
+
|
|
29
|
+
Instead, a baseline entry's identity is:
|
|
30
|
+
|
|
31
|
+
```
|
|
32
|
+
ruleId + reasonCode + html
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
where `reasonCode` is the rule-specific code from `occurrence.data.details.reasonCode` (defaulting to `"DEFAULT"` when a rule doesn't set one), and `html` is the occurrence's outer-HTML snippet. This is content-based rather than position-based: it survives the flagged element moving around the page (a reorder, an unrelated sibling added/removed) as long as the flagged element's own markup doesn't change. `selector` is still recorded in the baseline file, but purely for human context when reading a diff — it is never used for matching.
|
|
36
|
+
|
|
37
|
+
**Known limitation**: an element whose *own* markup includes dynamic content (a timestamp, a live counter, a randomly-generated id) will never match itself across two scans, since its `html` snippet differs every time. If your pages hit this case, the baseline mechanism won't help for those specific rules/elements — see "Alternative" below.
|
|
38
|
+
|
|
39
|
+
Matching counts occurrences, not just presence: if a page has 3 elements that produce byte-identical `ruleId`+`reasonCode`+`html` (e.g. the same broken component repeated 3 times) and the baseline recorded only 1 of them, a fresh scan reports 1 known and 2 new — not all 3 as known.
|
|
40
|
+
|
|
41
|
+
Baseline entries that don't match anything in a fresh scan are reported as **stale** (the violation was presumably fixed) — this is informational only and never gates the build; regenerate the baseline with `--write-baseline` periodically to clean these up.
|
|
42
|
+
|
|
43
|
+
## File format
|
|
44
|
+
|
|
45
|
+
```json
|
|
46
|
+
{
|
|
47
|
+
"version": 1,
|
|
48
|
+
"generatedAt": "2026-07-30T12:00:00.000Z",
|
|
49
|
+
"entries": [
|
|
50
|
+
{ "ruleId": "img-alt-present", "reasonCode": "DEFAULT", "selector": "html > body > img", "html": "<img src=\"logo.png\">" }
|
|
51
|
+
]
|
|
52
|
+
}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
`--baseline <path>` rejects a file that isn't `{ version: 1, entries: [...] }` with a clear exit-2 error rather than guessing at an older/different format.
|
|
56
|
+
|
|
57
|
+
## Combining with `--json`
|
|
58
|
+
|
|
59
|
+
When `--baseline` or `--write-baseline` is used together with `--json`, the printed object gains a `baseline` key alongside the normal engine result (this is CLI-output-only — it is not part of the engine's own result contract described in `OUTPUT_SCHEMA.md`):
|
|
60
|
+
|
|
61
|
+
- `--write-baseline`: `{ mode: "write", path, entries: <number written> }`
|
|
62
|
+
- `--baseline`: `{ mode: "check", totalFail, knownCount, newCount, staleCount, newOccurrences: [...] }`
|
|
63
|
+
|
|
64
|
+
## Alternative: diff two scans yourself
|
|
65
|
+
|
|
66
|
+
If your pages don't fit this model (heavy dynamic content inside the flagged elements themselves), the always-available fallback is to diff two full `--json` outputs yourself — see [`INTEGRATION.md`](./INTEGRATION.md#ci-gating-a-build-on-the-result). That gives you full control over the identity/matching logic at the cost of writing it yourself.
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Binding authors' guide
|
|
2
2
|
|
|
3
|
-
Written for whoever builds the *next* framework binding on top of
|
|
3
|
+
Written for whoever builds the *next* framework binding on top of `@surea11y/core` (Puppeteer, Cypress, Selenium, WebdriverIO, whatever comes after) — not for someone consuming a binding, and not for someone calling the engine directly. If that's you, see [`INTEGRATION.md`](./INTEGRATION.md) instead.
|
|
4
4
|
|
|
5
|
-
The first real binding,
|
|
5
|
+
The first real binding, `@surea11y/playwright` (a sibling project, not part of this repo), has already worked through most of the design questions a new binding hits. This doc exists so the next one doesn't have to re-derive them — it's a checklist and a map of "what the engine already gives you for free" vs. "what every binding has to build itself," backed by what that binding actually did, not theory.
|
|
6
6
|
|
|
7
7
|
## Core-engine vs. binding-layer: know which side you're on
|
|
8
8
|
|
|
@@ -10,19 +10,19 @@ Some engine-parity features (relative to other established engines) are **engine
|
|
|
10
10
|
|
|
11
11
|
**Already engine-level, works the moment you call `runa11yCoreInPage`/`runDomRulesInPage` — no binding code needed:**
|
|
12
12
|
- All rule execution, WCAG SC mapping, composite rollups.
|
|
13
|
-
- `runOnly`/`engineOptions.rules`/`.tags`/`.tests` rule selection — including [WCAG-version filtering](./ENGINE_OPTIONS.md#filtering-by-wcag-version-21-vs-22) (`wcag21a`/`wcag22aa`-style tags): if your binding has any kind of `.withTags()`/`.options()` passthrough that forwards `runOnly`/`engineOptions` generically, WCAG-version filtering already works through it with zero extra code — just document the tag vocabulary for your users, the way
|
|
13
|
+
- `runOnly`/`engineOptions.rules`/`.tags`/`.tests` rule selection — including [WCAG-version filtering](./ENGINE_OPTIONS.md#filtering-by-wcag-version-21-vs-22) (`wcag21a`/`wcag22aa`-style tags): if your binding has any kind of `.withTags()`/`.options()` passthrough that forwards `runOnly`/`engineOptions` generically, WCAG-version filtering already works through it with zero extra code — just document the tag vocabulary for your users, the way `@surea11y/playwright`'s README does.
|
|
14
14
|
- `structuralPath` on every `fail`/`cantTell` occurrence — if your binding passes occurrences through unreshaped (don't strip fields you don't recognize), this reaches your consumers automatically.
|
|
15
15
|
- `engineOptions.customRules` — runtime rule registration. Works through a generic `engineOptions` passthrough too, **but** see the caveat below: if your binding crosses a serialization boundary (see next section), your consumers must pass `runInPage`/`applicability` as `fn.toString()` source, not a live function. Worth a dedicated `.withCustomRules([...])` convenience method for ergonomics, but not required for the feature to work.
|
|
16
|
-
- Cross-frame scanning **if** your driver reaches every frame itself already (Puppeteer/Cypress/Selenium all can, via CDP or equivalent) — you don't need `runa11yCoreAcrossFrames`/`a11yCoreEnableFrameResponder` at all. Just call the engine once per frame your driver already gives you and merge the results yourself (see
|
|
16
|
+
- Cross-frame scanning **if** your driver reaches every frame itself already (Puppeteer/Cypress/Selenium all can, via CDP or equivalent) — you don't need `runa11yCoreAcrossFrames`/`a11yCoreEnableFrameResponder` at all. Just call the engine once per frame your driver already gives you and merge the results yourself (see `@surea11y/playwright`'s `.frames(true)`, which does exactly this — no engine change was needed for it). The `postMessage`-based cross-frame functions exist specifically for the *no-automation-driver* case (a plain injected script) and are the wrong tool for a driver-based binding.
|
|
17
17
|
|
|
18
18
|
**Binding-layer — your binding has to build these itself, the engine won't:**
|
|
19
|
-
- **Element references.** The engine returns `selector`/`structuralPath` strings, never a live handle — it has no concept of your driver's element-reference type. Resolve `occurrences[i].selector` back to a real handle yourself (Playwright's approach: `page.evaluateHandle` instead of `page.evaluate`, then `elementHandle.$(selector)` per occurrence — see `.elementRef(true)` in
|
|
20
|
-
- **Result verbosity/reporter filtering.** The engine deliberately always returns every rule's outcome, including `pass`/`notApplicable` — "not a violations-only list" is a stated engine design choice (see [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md)), not an oversight to work around. If your consumers want a trimmed view for CI-scale output, that's a post-filter your binding adds (
|
|
19
|
+
- **Element references.** The engine returns `selector`/`structuralPath` strings, never a live handle — it has no concept of your driver's element-reference type. Resolve `occurrences[i].selector` back to a real handle yourself (Playwright's approach: `page.evaluateHandle` instead of `page.evaluate`, then `elementHandle.$(selector)` per occurrence — see `.elementRef(true)` in `@surea11y/playwright`).
|
|
20
|
+
- **Result verbosity/reporter filtering.** The engine deliberately always returns every rule's outcome, including `pass`/`notApplicable` — "not a violations-only list" is a stated engine design choice (see [`OUTPUT_SCHEMA.md`](./OUTPUT_SCHEMA.md)), not an oversight to work around. If your consumers want a trimmed view for CI-scale output, that's a post-filter your binding adds (`@surea11y/playwright`'s `.reportOnly(['fail','cantTell'])` is a simple array-filter over the full result — no engine change).
|
|
21
21
|
- **Formatted failure output for your framework's own assertion/reporting style** (e.g. Playwright/Jest-style multi-line failure messages). The engine's raw result is framework-agnostic on purpose; shaping it into "what shows up in a failed test's stack trace" is squarely binding territory.
|
|
22
22
|
|
|
23
23
|
## The serialization-boundary caveat
|
|
24
24
|
|
|
25
|
-
If your binding drives a *separate JS realm* (a browser page/tab is a different realm than your Node test process — this is Playwright/Puppeteer/Selenium's situation, not Cypress's, since Cypress test code already runs in-browser), anything you hand to `page.evaluate()`-equivalent gets structurally cloned/JSON-serialized. **Live functions do not survive that boundary.** This bit
|
|
25
|
+
If your binding drives a *separate JS realm* (a browser page/tab is a different realm than your Node test process — this is Playwright/Puppeteer/Selenium's situation, not Cypress's, since Cypress test code already runs in-browser), anything you hand to `page.evaluate()`-equivalent gets structurally cloned/JSON-serialized. **Live functions do not survive that boundary.** This bit `@surea11y/playwright` in two places:
|
|
26
26
|
1. `runa11yCoreInPage` itself is designed around this — it's fully self-contained (`.toString()`-serializable, no closure over outer scope) specifically so it can be reconstructed from source inside the page realm.
|
|
27
27
|
2. `engineOptions.customRules[].runInPage`/`.applicability` accept a function-source string for exactly this reason — a binding crossing this boundary must tell its consumers to pass `fn.toString()`, not `fn`. Document this prominently; it's an easy trap (a function looks like it should just work as an argument until it silently fails to serialize).
|
|
28
28
|
|
|
@@ -30,7 +30,7 @@ If your binding runs in the *same* realm as the page (a browser extension conten
|
|
|
30
30
|
|
|
31
31
|
## Things to check before shipping a new binding
|
|
32
32
|
|
|
33
|
-
A short list, derived from what the audit pass on
|
|
33
|
+
A short list, derived from what the audit pass on `@surea11y/playwright` actually found missing on a first pass (per its own `ROADMAP.md`) — worth checking explicitly rather than assuming your binding's generic passthrough covers them:
|
|
34
34
|
- [ ] Combinations of your own filtering methods behave sanely together (e.g. include+exclude on the same ID, tag-include + tag-exclude on the same tag) — these interact through the engine's `includeMode`/exclude-always-wins semantics ([`ENGINE_OPTIONS.md`](./ENGINE_OPTIONS.md)), test them explicitly rather than assuming.
|
|
35
35
|
- [ ] `structuralPath` and `customRules` still work correctly when combined with whatever binding-layer features you build (verbosity filtering, element refs, per-frame scanning) — a filter applied after the fact should never silently drop fields a consumer expects on a surviving occurrence.
|
|
36
36
|
- [ ] If you support cross-frame scanning via your own driver, confirm each frame's result gets the same normalization (selector/structuralPath/severity) as a single-document scan — don't let a "per-frame" code path silently skip the shared result-shaping logic.
|
|
@@ -38,4 +38,4 @@ A short list, derived from what the audit pass on `surea11y-playwright` actually
|
|
|
38
38
|
|
|
39
39
|
## A known engine-side tradeoff worth knowing about
|
|
40
40
|
|
|
41
|
-
`src/core.js` is not small (~3.1MB as of 2026-07-22) because the bundler-free, no-driver-context functions (`runa11yCoreInPage`, `runa11yCoreAcrossFrames`, `a11yCoreEnableFrameResponder`) each carry their own complete self-contained copy of the rule catalog. If your binding only ever uses `require('@surea11y/core')` in Node and injects `runa11yCoreInPage.toString()` into the page (the same pattern
|
|
41
|
+
`src/core.js` is not small (~3.1MB as of 2026-07-22) because the bundler-free, no-driver-context functions (`runa11yCoreInPage`, `runa11yCoreAcrossFrames`, `a11yCoreEnableFrameResponder`) each carry their own complete self-contained copy of the rule catalog. If your binding only ever uses `require('@surea11y/core')` in Node and injects `runa11yCoreInPage.toString()` into the page (the same pattern `@surea11y/playwright` uses), your actual browser-injected payload is unaffected by this — only your Node-side `require()` footprint grows. Worth knowing if your binding's own package size matters to your consumers.
|