ultra11y 2.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +223 -0
- package/docker/Dockerfile +7 -0
- package/docker/package.json +9 -0
- package/docker/runner.mjs +23 -0
- package/package.json +73 -0
- package/scripts/bench.mjs +151 -0
- package/scripts/build-act-corpus.mjs +143 -0
- package/scripts/build-adjudication.mjs +503 -0
- package/scripts/build-pack-rgaa.mjs +397 -0
- package/scripts/build-standards.mjs +548 -0
- package/scripts/copy-bundle.mjs +39 -0
- package/scripts/import-pack.mjs +107 -0
- package/scripts/sync-engine.mjs +84 -0
- package/scripts/sync-version.mjs +43 -0
- package/scripts/ultra11y.mjs +52739 -0
- package/scripts/vendor/act-testcases.json +12297 -0
- package/scripts/vendor/rgaa/SOURCE.json +8 -0
- package/scripts/vendor/rgaa/criteres.json +4144 -0
- package/scripts/vendor/rgaa/glossaire.json +480 -0
- package/scripts/vendor/wcag-2.2-fr.json +82 -0
- package/scripts/vendor/wcag-2.2-sc-universe.json +950 -0
- package/scripts/vendor/wcag-2.2-sc.json +574 -0
- package/scripts/verify-rgaa-source.mjs +126 -0
- package/scripts/verify-skill-bundle.mjs +91 -0
- package/skills/review-a11y/SKILL.md +123 -0
- package/skills/review-a11y/scripts/ultra11y.mjs +52739 -0
- package/skills/ultra11y/SKILL.md +292 -0
- package/skills/ultra11y/references/act.md +163 -0
- package/skills/ultra11y/references/adjudication.md +599 -0
- package/skills/ultra11y/references/audit.md +100 -0
- package/skills/ultra11y/references/authoring.md +37 -0
- package/skills/ultra11y/references/automation.md +113 -0
- package/skills/ultra11y/references/correction.md +41 -0
- package/skills/ultra11y/references/criteria.md +129 -0
- package/skills/ultra11y/references/cross-file.md +82 -0
- package/skills/ultra11y/references/dynamic.md +152 -0
- package/skills/ultra11y/references/false-positives.md +81 -0
- package/skills/ultra11y/references/fix.md +46 -0
- package/skills/ultra11y/references/focus-and-logic.md +64 -0
- package/skills/ultra11y/references/forbidden-patterns.md +61 -0
- package/skills/ultra11y/references/forms-and-errors.md +147 -0
- package/skills/ultra11y/references/guidance.md +55 -0
- package/skills/ultra11y/references/judgment.md +90 -0
- package/skills/ultra11y/references/media-and-motion.md +116 -0
- package/skills/ultra11y/references/methodology.md +72 -0
- package/skills/ultra11y/references/naming.md +117 -0
- package/skills/ultra11y/references/orchestration.md +44 -0
- package/skills/ultra11y/references/packs.md +193 -0
- package/skills/ultra11y/references/prd.md +104 -0
- package/skills/ultra11y/references/rendered.md +110 -0
- package/skills/ultra11y/references/scale.md +49 -0
- package/skills/ultra11y/references/standards.md +97 -0
- package/skills/ultra11y/references/structure.md +138 -0
- package/skills/ultra11y/references/verify.md +88 -0
- package/skills/ultra11y/references/widgets.md +106 -0
- package/skills/ultra11y/scripts/ultra11y.mjs +52739 -0
- package/src/act.ts +441 -0
- package/src/adjudicate.ts +630 -0
- package/src/aria.ts +230 -0
- package/src/audit.ts +396 -0
- package/src/auditor.ts +399 -0
- package/src/autofill.ts +102 -0
- package/src/axe-map.ts +294 -0
- package/src/baseline.ts +75 -0
- package/src/capture.ts +182 -0
- package/src/check.ts +243 -0
- package/src/cli.ts +1927 -0
- package/src/color.ts +102 -0
- package/src/config.ts +182 -0
- package/src/crawl.ts +75 -0
- package/src/criteria.ts +183 -0
- package/src/data/adjudication.json +770 -0
- package/src/data/guidance/rgaa.json +2395 -0
- package/src/data/standards/rgaa.glossary.json +478 -0
- package/src/data/standards/rgaa.json +3454 -0
- package/src/data/wcag-universe.json +530 -0
- package/src/data/wcag.json +1178 -0
- package/src/discover.ts +203 -0
- package/src/fix/codemods.ts +76 -0
- package/src/fix/diff.ts +64 -0
- package/src/fix/edits.ts +126 -0
- package/src/fix.ts +215 -0
- package/src/gh.ts +214 -0
- package/src/glob.ts +206 -0
- package/src/graph/build.ts +114 -0
- package/src/graph/engine-resolve.ts +0 -0
- package/src/graph/graph.ts +126 -0
- package/src/graph/imports.ts +359 -0
- package/src/graph/resolve.ts +73 -0
- package/src/graph/tsconfig.ts +113 -0
- package/src/grounding.ts +119 -0
- package/src/guidance/index.ts +45 -0
- package/src/guidance/types.ts +36 -0
- package/src/init.ts +113 -0
- package/src/mcp/handlers.ts +388 -0
- package/src/mcp/http.ts +260 -0
- package/src/mcp/prompts.ts +183 -0
- package/src/mcp/protocol.ts +187 -0
- package/src/mcp/resources.ts +136 -0
- package/src/mcp/server.ts +228 -0
- package/src/mcp/stdio.ts +119 -0
- package/src/mcp/tools.ts +347 -0
- package/src/messages.ts +1011 -0
- package/src/name.ts +174 -0
- package/src/orchestrate-templates.ts +231 -0
- package/src/orchestrate.ts +186 -0
- package/src/output.ts +108 -0
- package/src/pack.ts +187 -0
- package/src/parse/html.ts +258 -0
- package/src/parse/jsx-ast.ts +67 -0
- package/src/parse/jsx-bridge.ts +247 -0
- package/src/parse/jsx.ts +19 -0
- package/src/parse/source.ts +75 -0
- package/src/prd.ts +406 -0
- package/src/project-lock.ts +42 -0
- package/src/render.ts +307 -0
- package/src/report.ts +399 -0
- package/src/rules/colors.ts +125 -0
- package/src/rules/cross-registry.ts +264 -0
- package/src/rules/cross-rule.ts +83 -0
- package/src/rules/forms.ts +590 -0
- package/src/rules/frames.ts +33 -0
- package/src/rules/headings.ts +227 -0
- package/src/rules/images.ts +200 -0
- package/src/rules/links.ts +193 -0
- package/src/rules/mandatory.ts +173 -0
- package/src/rules/multimedia.ts +77 -0
- package/src/rules/navigation.ts +190 -0
- package/src/rules/presentation.ts +75 -0
- package/src/rules/registry.ts +57 -0
- package/src/rules/rule.ts +122 -0
- package/src/rules/scripts-aria.ts +636 -0
- package/src/rules/tables.ts +264 -0
- package/src/rules/timing.ts +55 -0
- package/src/sample.ts +176 -0
- package/src/scan-local.ts +866 -0
- package/src/scan.ts +538 -0
- package/src/standards/derive.ts +195 -0
- package/src/standards/index.ts +31 -0
- package/src/standards/pack-rules.ts +111 -0
- package/src/standards/pack.ts +81 -0
- package/src/standards/registry.ts +100 -0
- package/src/standards/types.ts +202 -0
- package/src/standards/validate.ts +544 -0
- package/src/standards/vocabulary.ts +75 -0
- package/src/types.ts +355 -0
- package/src/util.ts +32 -0
- package/src/vendor/codeindex-engine.d.mts +1009 -0
- package/src/vendor/codeindex-engine.mjs +13701 -0
- package/src/vendor/engine.meta.json +9 -0
- package/src/verify.ts +275 -0
- package/src/wcag.ts +128 -0
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
# Auditor backlog (`prd`) + GitHub issues
|
|
2
|
+
|
|
3
|
+
`prd` turns an `AuditResult` into an **auditor-legible conformance backlog**, grouped by WCAG
|
|
4
|
+
success criterion (or, with `--standard <pack>`, by a country standard's criteria). It is the
|
|
5
|
+
"action" counterpart of `report` (which is the compliance document) — and they are not just
|
|
6
|
+
similar, they render the literal **same auditor block** (`src/auditor.ts` `renderAuditorUnit`,
|
|
7
|
+
fed by the same `prdUnits`): `report`'s "Non-conformities by priority" section (§2) IS one of
|
|
8
|
+
these blocks per NC criterion, grouped by severity exactly like the `prd` backlog and the
|
|
9
|
+
`--gh-issues`/`--gh-single` issue bodies. Change the block once, all four deliverables change
|
|
10
|
+
together — there is no separate "report wording" to keep in sync.
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
node scripts/ultra11y.mjs audit "src/**/*.tsx" --graph --json > audit.json
|
|
14
|
+
node scripts/ultra11y.mjs prd --in audit.json # auditor backlog (default)
|
|
15
|
+
node scripts/ultra11y.mjs prd --in audit.json --split criterion # one file per criterion
|
|
16
|
+
node scripts/ultra11y.mjs prd --in audit.json --gh-issues # + one GitHub issue per criterion
|
|
17
|
+
node scripts/ultra11y.mjs prd --in audit.json --gh-single # + ONE consolidated GitHub issue (whole audit)
|
|
18
|
+
node scripts/ultra11y.mjs prd --in audit.json --standard rgaa --lang fr # rendered with the RGAA (fr) vocabulary
|
|
19
|
+
node scripts/ultra11y.mjs prd --in audit.json --format doc # product-requirements doc (epics/stories/AC)
|
|
20
|
+
node scripts/ultra11y.mjs prd --in audit.json --format remediation # legacy dev fix-backlog
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Output
|
|
24
|
+
|
|
25
|
+
- **Default (`--format audit`) — the auditor conformance block**: one document
|
|
26
|
+
`audits/prd-YYYY-MM-DD.md`, sectioned by priority (🔴 blocking → 🟠 major → 🟡 minor). Each
|
|
27
|
+
criterion becomes a full **ticket** rendered **with the active standard's vocabulary** (see
|
|
28
|
+
`references/standards.md` → *Auditor vocabulary*):
|
|
29
|
+
1. **Auditor block** — **theme** (RGAA *Thématique* / WCAG core *Principle · Guideline*),
|
|
30
|
+
**criterion** + its official wording, **test(s)** (RGAA test numbers `11.6.1` / WCAG
|
|
31
|
+
techniques), **WCAG** mapping + level, **Priorité**, the **finding** (non-conformity,
|
|
32
|
+
labelled with the standard's *non-conformant* verdict), the **expected** conformant state,
|
|
33
|
+
a **verification** method, and the occurrence **checklist** (`file:line`) with the
|
|
34
|
+
cross-file **definition site** (`related`) when present.
|
|
35
|
+
2. **Partie technique / Technical details** — **impacted files** (source paths), **impacted
|
|
36
|
+
pages / URLs** (served locations, each with its sample **page name + auth flag** when
|
|
37
|
+
`scan --sample` provenance is present), the **expected change** + a before/after
|
|
38
|
+
**guidance example** (`references/guidance.md`), **Critères d'acceptation / Acceptance
|
|
39
|
+
criteria** as a Given/When/Then checkbox list, and **Complexité / Complexity** (a
|
|
40
|
+
deterministic t-shirt size + story points).
|
|
41
|
+
3. **Contexte de reproduction / Reproduction context** — emitted only when ≥1 occurrence
|
|
42
|
+
cites a served URL static grounding could not resolve, or a sample page behind
|
|
43
|
+
authentication: the URL, whether auth is required, and the required-state / reproduction
|
|
44
|
+
steps. This is what lets a developer reproduce a scan/`--sample` finding.
|
|
45
|
+
|
|
46
|
+
Sections 2–3 are opt-out with **`--no-technical`** (a pure-auditor consumption drops the
|
|
47
|
+
ticket scaffolding, keeping only the auditor block). Everything is localized by `--lang
|
|
48
|
+
fr|en`. The **GitHub issues** below use this same block by default.
|
|
49
|
+
- **Advisory findings render as recommendations, never NCs.** A non-normative recommendation
|
|
50
|
+
(an advisory pack rule, or an agent `recommendations[]` verdict) is rendered under
|
|
51
|
+
« Recommandations (non normatives) » / "Recommendations (non-normative)" with a 💡 marker and
|
|
52
|
+
the « Recommandation (non normative) » tag — a distinct, non-parseable list so `verify` never
|
|
53
|
+
captures it as an NC claim. An advisory finding riding along in an otherwise-NC criterion is
|
|
54
|
+
split out under « Recommandations associées ». It never enters the occurrence checklist,
|
|
55
|
+
never flips a criterion to NC, and never counts toward `conformancePct`.
|
|
56
|
+
- **`--format remediation` (legacy dev backlog)**: the previous developer-oriented block —
|
|
57
|
+
fix(es), an **effort estimate** (S/M/L), a **before/after example** from the implementation
|
|
58
|
+
guidance (`references/guidance.md`), and the occurrence checklist.
|
|
59
|
+
- **`--format doc` (product-requirements doc)**: `audits/prd-doc-YYYY-MM-DD.md` — epics
|
|
60
|
+
grouped by theme, one **user story** per criterion, **Given/When/Then** acceptance criteria
|
|
61
|
+
templated from the real SC title/techniques (anchored to W3C text), and the task list.
|
|
62
|
+
- **`--split criterion`**: a `prd-<criterion>-YYYY-MM-DD.md` file per criterion (handy for batching).
|
|
63
|
+
- The markdown is **always** written, even with `--gh-issues`.
|
|
64
|
+
- **`--json`**: emits a machine-readable object instead of the file paths —
|
|
65
|
+
`{paths, units, gh?}` where `units` is the structured per-criterion backlog an agent can consume.
|
|
66
|
+
|
|
67
|
+
## Language of the prose (French deliverables)
|
|
68
|
+
|
|
69
|
+
When you author or extend PRD/issue prose in French (extra context, acceptance criteria,
|
|
70
|
+
issue comments), keep technical tokens in English exactly as the engine's fr catalog does:
|
|
71
|
+
attribute/element/role names and their values are code, not prose — `aria-live` stays
|
|
72
|
+
`aria-live` (never « région live »), same for `tabindex`, `alt`, `role="alert"`. Normative
|
|
73
|
+
standard vocabulary (e.g. RGAA « lien d'évitement ») keeps its official French wording.
|
|
74
|
+
|
|
75
|
+
## GitHub issues (`--gh-issues` / `--gh-single`, opt-in)
|
|
76
|
+
|
|
77
|
+
- Uses the **`gh` CLI** (which handles its own auth) — **no** npm dependency, no key in
|
|
78
|
+
ultra11y.
|
|
79
|
+
- **`--gh-issues` → one issue per criterion** (regardless of `--split`), stable title
|
|
80
|
+
`"[a11y] WCAG <sc> — <title>"` (or `"[a11y] <PACK> <id> — …"` under `--standard`), labels
|
|
81
|
+
`accessibility`, `wcag` (or the pack key), severity. The body is the **auditor conformance
|
|
82
|
+
block** (theme, criterion + wording, test(s), WCAG + level, finding, expected, verification,
|
|
83
|
+
`file:line` occurrences + definition site) — same as the default backlog, in the active
|
|
84
|
+
standard's vocabulary. `--format remediation` files the legacy dev body instead.
|
|
85
|
+
- **`--gh-single` → ONE consolidated issue** for the whole audit, stable title
|
|
86
|
+
`"[a11y] WCAG — Accessibility audit"` (or `"[a11y] <PACK> — Accessibility audit"`). The body
|
|
87
|
+
is the full backlog **sectioned by severity** (🔴 blocking → 🟠 major → 🟡 minor), each
|
|
88
|
+
criterion carrying the auditor block. Labelled by the **most severe** criterion.
|
|
89
|
+
- `--gh-single` **wins** if both flags are passed.
|
|
90
|
+
- **De-dupe by title**: an existing issue (open or closed) is skipped, so re-running never
|
|
91
|
+
creates duplicates. The consolidated title carries **no count or date**, so it stays stable
|
|
92
|
+
across re-runs.
|
|
93
|
+
- **Caveat — `--lang` changes the title**: the issue title embeds the localized criterion/pack
|
|
94
|
+
title (e.g. `[a11y] WCAG 1.4.3 — Contrast (Minimum)` vs `[a11y] WCAG 1.4.3 — Contraste (minimum)`).
|
|
95
|
+
Since de-dupe matches on the **exact title string**, re-running `prd --gh-issues` with a
|
|
96
|
+
**different `--lang`** than a previous run does **not** match the earlier issue and creates a
|
|
97
|
+
new one instead of updating it. Keep `--lang` consistent across re-runs against the same
|
|
98
|
+
repo, or accept the duplicate and close the stale one manually.
|
|
99
|
+
- **Graceful degradation**: if `gh` is absent / unauthenticated, the command says so and exits
|
|
100
|
+
`0` — the markdown was still produced.
|
|
101
|
+
|
|
102
|
+
> `prd` reads the `AuditResult` produced by `audit` (ideally `--graph` for cross-file
|
|
103
|
+
> coverage); it reuses the criterion titles, severities, messages, remediations and
|
|
104
|
+
> `file:line` already computed.
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# Audit the RENDERED HTML (component libraries: DSFR, MUI…)
|
|
2
|
+
|
|
3
|
+
Auditing a component library's **JSX sources** gives **false negatives**:
|
|
4
|
+
`<Button iconId="fr-icon-add-line" />` (DSFR) actually renders
|
|
5
|
+
`<button class="fr-btn fr-icon-add-line">` where the icon is a CSS pseudo-element
|
|
6
|
+
(decorative) and the **accessible name comes only from `title`**. That HTML lives in
|
|
7
|
+
`node_modules` at runtime — no source analysis sees it. So for those components, **audit
|
|
8
|
+
what the JS produces**, not the source.
|
|
9
|
+
|
|
10
|
+
## Honest by default
|
|
11
|
+
|
|
12
|
+
When `audit` sees a JSX/TSX file rendering components imported from a **library** (a package
|
|
13
|
+
specifier, e.g. `@codegouvfr/react-dsfr`), it does not invent a verdict: it adds a **scope
|
|
14
|
+
risk** to the report ("preliminary source verdict — audit the build or `scan`") and names the
|
|
15
|
+
library(ies). A source verdict is therefore never silently "complete".
|
|
16
|
+
|
|
17
|
+
**`.vue`/`.svelte`/`.astro` single-file components are the same case.** They are parsed as
|
|
18
|
+
SOURCE templates (the HTML path with component case preserved), so `<slot>`/`{@render}`,
|
|
19
|
+
`v-for`/`{#each}`, dynamic bindings (`:aria-label`) and child components are invisible to
|
|
20
|
+
static analysis. Every finding raised on them is flagged **`preliminary: true`** and the run
|
|
21
|
+
adds a **`scope.sourceTemplate`** caveat ("audited SOURCE template only — audit the rendered
|
|
22
|
+
output"). Treat those verdicts as provisional: audit the rendered HTML (a static Storybook
|
|
23
|
+
build is ideal for SFCs) or `scan` a live page before concluding, and refute any source
|
|
24
|
+
finding the rendered DOM disproves (`verify --apply`). See `references/false-positives.md`.
|
|
25
|
+
|
|
26
|
+
## Zero-touch test-render captures (recommended)
|
|
27
|
+
|
|
28
|
+
Rather than hand-write a render list, let your existing tests do the rendering. Install the
|
|
29
|
+
capture harvester once and every component your tests already render is serialized to a
|
|
30
|
+
provenance-tagged `.html` the static engine audits at full fidelity — the real markup a
|
|
31
|
+
library/SFC emits, not its source call:
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
node scripts/ultra11y.mjs render --setup # writes .ultra11y/capture-setup.mjs + prints the runner wiring
|
|
35
|
+
# wire it into your runner (Vitest: setupFiles + globals:true; Jest: setupFilesAfterEnv), then:
|
|
36
|
+
npm test # every rendered component → .ultra11y/captures/*.html
|
|
37
|
+
node scripts/ultra11y.mjs audit # auto-ingests .ultra11y/captures; findings attribute to the source component
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
- **Attribution.** Each capture carries `<!-- ultra11y:capture source="src/Button.tsx" component="Button" … -->`,
|
|
41
|
+
so a finding on the rendered DOM is reported against the SOURCE component, not the anonymous
|
|
42
|
+
capture file. Storybook/manual dumps with no marker still audit as plain HTML.
|
|
43
|
+
- **Coverage + gate.** `render --coverage` lists which components have a capture vs which are
|
|
44
|
+
still opaque-source-only blind spots; `audit --require-captures` fails when an opaque/control
|
|
45
|
+
component lacks one — so "all components" actually get rendered-DOM coverage.
|
|
46
|
+
`.vue`/`.svelte`/`.astro` self-defined components now count in this coverage universe too
|
|
47
|
+
(previously invisible to the graph), so a previously-green `--require-captures` gate can newly
|
|
48
|
+
report blind spots on an SFC-heavy repo — that is intended (a named blind spot beats
|
|
49
|
+
invisibility); capture those components or scope the gate.
|
|
50
|
+
- **Commit `.ultra11y/captures`.** In diff-scoped mode (`--changed`/`--staged`/`--since`) the
|
|
51
|
+
audit automatically pulls in the captures whose provenance points at a diffed source file —
|
|
52
|
+
the pre-commit gate sees the real rendered markup for a touched component even though the
|
|
53
|
+
(unchanged) capture itself isn't part of the diff. After a component change, re-run your
|
|
54
|
+
tests so its capture refreshes, and stage the refreshed capture with the source change.
|
|
55
|
+
`fix` never rewrites captures (generated output). Add `.gitattributes`:
|
|
56
|
+
`.ultra11y/captures/*.html text eol=lf` for stable cross-platform diffs.
|
|
57
|
+
- **Storybook**: with **portable stories** (composeStories) rendered in your test suite, the
|
|
58
|
+
harvester captures them for free. For per-story HTML produced another way (e.g.
|
|
59
|
+
`@storybook/test-runner`), `render --storybook [<storybook-static>] [--captures <html-dir>]`
|
|
60
|
+
attributes each file to its source component via the Storybook index, into `.ultra11y/captures`.
|
|
61
|
+
- ultra11y renders nothing itself — your test toolchain (jsdom/happy-dom) does. Disable with `ULTRA11Y_CAPTURES=off`.
|
|
62
|
+
- The harvester pipeline is covered by a real end-to-end test in ultra11y's own suite (a
|
|
63
|
+
spawned vitest+jsdom project renders, the capture lands with correct provenance, and the
|
|
64
|
+
engine ingests it) — the escaping scheme it embeds is locked byte-identical to the parser's.
|
|
65
|
+
|
|
66
|
+
## Get rendered HTML
|
|
67
|
+
|
|
68
|
+
`node scripts/ultra11y.mjs render [<dir>]` detects the framework and prints the
|
|
69
|
+
"build → audit" recipe for the project (and flags the libraries it detects). Other routes,
|
|
70
|
+
simplest to most faithful:
|
|
71
|
+
|
|
72
|
+
1. **Build output** (recommended): build the site/pages, then audit the emitted HTML with the
|
|
73
|
+
static engine — it is real HTML, audited at full fidelity:
|
|
74
|
+
```
|
|
75
|
+
npx astro build # or next build (output:'export'), vite build…
|
|
76
|
+
node scripts/ultra11y.mjs audit "dist/**/*.html"
|
|
77
|
+
```
|
|
78
|
+
**Storybook is the one exception**: a bare `storybook build` does **NOT** emit one HTML
|
|
79
|
+
file per story — `storybook-static/` is a single-page-app shell (`iframe.html`/`index.html`)
|
|
80
|
+
that renders stories client-side; auditing `storybook-static/**/*.html` sees only that shell,
|
|
81
|
+
not your components. Get REAL per-story HTML first, either via `@storybook/test-runner`
|
|
82
|
+
(renders every story in a real browser — dump/audit its output) or **portable stories**
|
|
83
|
+
(`composeStories`) rendered in your OWN test suite, where the zero-touch capture harvester
|
|
84
|
+
(`render --setup`) captures them for free. Only then does `render --storybook
|
|
85
|
+
[<storybook-static>] [--captures <html-dir>]` do anything useful: it ATTRIBUTES that
|
|
86
|
+
already-produced per-story HTML to its source component via the Storybook index — it does
|
|
87
|
+
not render anything itself. Point it at nothing but the shell and it now fails honestly
|
|
88
|
+
(exit 1, not a silent no-op) when candidate HTML existed but none could be attributed.
|
|
89
|
+
2. **SSR snapshot**: `render --scaffold` writes `ultra11y-render.tsx`, a `react-dom/server`
|
|
90
|
+
harness (`renderToStaticMarkup`) that imports **your** components and writes HTML into
|
|
91
|
+
`audits/rendered/`. **The written file is INERT until you fill in `COMPONENTS`** — running
|
|
92
|
+
it as-is writes nothing (it self-reports `COMPONENTS is empty` and exits), and the CLI's own
|
|
93
|
+
write-time message says so. Fill in the list, run it with your toolchain (`npx tsx
|
|
94
|
+
ultra11y-render.tsx`), then audit `audits/rendered`. ultra11y does not embed React — your
|
|
95
|
+
project renders.
|
|
96
|
+
3. **Headless browser**: `scan <url>` (Docker tier, axe-core) against the running app — the
|
|
97
|
+
most faithful, and required for the **rendering** criteria (computed contrast 1.4.3, reflow
|
|
98
|
+
1.4.10). `--merge` folds the result into a static AuditResult.
|
|
99
|
+
|
|
100
|
+
## Choosing
|
|
101
|
+
|
|
102
|
+
- Components WITH a test suite → zero-touch captures (`render --setup`), then `audit --require-captures`.
|
|
103
|
+
- Components / design system (no tests) → portable stories + `@storybook/test-runner` (real
|
|
104
|
+
per-story HTML, then `render --storybook` to attribute it) **or** an SSR snapshot
|
|
105
|
+
(`render --scaffold`), then `audit`. A bare `storybook build` alone is NOT enough (see above).
|
|
106
|
+
- Full pages / SSG → build output (`dist`/`out`), then `audit`.
|
|
107
|
+
- Rendering criteria (contrast, focus, zoom) → `scan` (browser).
|
|
108
|
+
- In all cases, do not conclude "conforming" on a library component from the source: that is
|
|
109
|
+
what the scope risk reminds you — resolve it on the produced HTML (captures / `scan`), then let
|
|
110
|
+
the AI agent adjudicate the criterion (`verify --manual`, gated).
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Audit at scale — focus smartly
|
|
2
|
+
|
|
3
|
+
You never audit "all" of a huge repo. The engine reads only the **markup** (HTML/JSX/Vue/
|
|
4
|
+
Svelte/Astro), streams it (bounded memory, one file at a time, the `Doc` is dropped after use),
|
|
5
|
+
and you **focus** it. Recommended loop:
|
|
6
|
+
|
|
7
|
+
## The loop
|
|
8
|
+
|
|
9
|
+
1. **Map without loading everything.** The engine ignores `node_modules`, `.git`, `dist`,
|
|
10
|
+
`build`, `.next`, `out`, `coverage`, `audits` by default, and skips **test/spec/story
|
|
11
|
+
markup** (`*.test.*`, `*.spec.*`, `*.stories.*`, `__tests__/`, `__mocks__/`,
|
|
12
|
+
`.storybook/`) — bad-by-design fixtures, never shipped UI (the drop count is logged).
|
|
13
|
+
Pass `--no-default-excludes` to include them, or name a file / `--include` it directly to
|
|
14
|
+
re-admit. Narrow further:
|
|
15
|
+
```
|
|
16
|
+
node scripts/ultra11y.mjs audit "apps/web/**/*.{html,tsx}" --json > audit.json
|
|
17
|
+
```
|
|
18
|
+
2. **Pick the slice that matters.** Three levers, most targeted to broadest:
|
|
19
|
+
- **The diff** (hooks/CI, PR review): `--changed` (vs `HEAD`) or `--since <ref>` — audits
|
|
20
|
+
only the changed markup files (via `git diff`, no tree walk).
|
|
21
|
+
- **Shared templates/components first**: the engine **prioritizes** layouts, templates,
|
|
22
|
+
entry pages, then `components/`, `shared/`, `ui/`, `design-system/`, then leaves. A partial
|
|
23
|
+
run therefore covers the highest-impact markup first.
|
|
24
|
+
- **An explicit cap**: `--max-files <n>` bounds the number of files audited (highest-priority
|
|
25
|
+
first); truncation is **always logged** (never a silent drop) and recorded in the report.
|
|
26
|
+
3. **De-duplicate the repetitive.** An identical component repeated N times is audited **once**
|
|
27
|
+
(`--dedup exact` by default; `normalized` ignores inter-tag whitespace; `off` disables). The
|
|
28
|
+
report cites the canonical file.
|
|
29
|
+
4. **Audit → fix → re-audit → widen.** On the chosen slice: read the `AuditResult`, complete the
|
|
30
|
+
judgment, apply the fixes (`references/fix.md`), re-audit to prove no regression, then widen
|
|
31
|
+
the scope.
|
|
32
|
+
|
|
33
|
+
## Why it is safe at scale
|
|
34
|
+
|
|
35
|
+
- **Bounded memory**: file-by-file stream; only the finding count (not source size) stays in
|
|
36
|
+
memory.
|
|
37
|
+
- **Deterministic**: stable order (priority then path) → reproducible audits, stable canonical
|
|
38
|
+
file choice, `check:build` holds.
|
|
39
|
+
- **Incremental**: `--changed`/`--since` make the audit proportional to the diff, not the repo —
|
|
40
|
+
what makes hooks/CI viable (see `references/automation.md`). Honest caveat: the diff modes
|
|
41
|
+
pay a **fixed cost of a few `git` invocations** before auditing anything, so on a small repo
|
|
42
|
+
a full scan can actually be the faster of the two. The win starts around a few hundred
|
|
43
|
+
markup files and grows from there.
|
|
44
|
+
- **One pass over the tree**: `--graph` discovers markup and the `.ts/.js` modules it resolves
|
|
45
|
+
through, but walks the tree once and parses each markup file once — the graph pass hands its
|
|
46
|
+
parsed documents to the audit pass instead of every file being read and parsed twice.
|
|
47
|
+
|
|
48
|
+
> In `--changed` mode, de-duplication is disabled: a changed file is always audited, never
|
|
49
|
+
> merged with a file that was not read.
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# Standards packs — WCAG core + pluggable country standards
|
|
2
|
+
|
|
3
|
+
ultra11y's engine is keyed on **WCAG 2.2 Level AA** — the worldwide standard. Country
|
|
4
|
+
accessibility standards ship as **standards packs**: small in-repo JSON files that map a
|
|
5
|
+
national standard's criteria onto WCAG success criteria. The WCAG verdict is canonical and
|
|
6
|
+
gated; a pack report is a *derived view* projected from the same audit.
|
|
7
|
+
|
|
8
|
+
## Model
|
|
9
|
+
|
|
10
|
+
- **Core = WCAG 2.2 AA.** 55 success criteria (Level A + AA), the engine's canonical key.
|
|
11
|
+
SC ids/titles/levels are derived from the W3C source (https://github.com/w3c/wcag).
|
|
12
|
+
**French SC/guideline/principle titles are the official W3C AUTHORIZED translation**
|
|
13
|
+
(https://www.w3.org/Translations/WCAG22-fr/, vendored at `scripts/vendor/wcag-2.2-fr.json`)
|
|
14
|
+
baked into the dataset — `--lang fr` output cites the normative French wording, never a
|
|
15
|
+
machine translation (the build fails if a core-AA title is missing).
|
|
16
|
+
- **Packs = country standards.** Each pack (`src/data/standards/<key>.json`) declares its
|
|
17
|
+
own localized criteria and, per criterion, the WCAG SC ids it maps to. The engine audits
|
|
18
|
+
WCAG; `derivePackResults` projects that onto the pack's criteria with the same
|
|
19
|
+
NC-dominates rule. A pack carries **no** engine rules or automatability — only the mapping.
|
|
20
|
+
- **The SC universe backs validation and scoping.** Alongside the AA core the engine ships
|
|
21
|
+
the full WCAG 2.x SC universe (every real SC at every level, plus the removed ones, from
|
|
22
|
+
the same vendored W3C source). A pack criterion may legitimately map to a real
|
|
23
|
+
out-of-core SC (an AAA criterion, or the removed `4.1.1`): validation accepts it with a
|
|
24
|
+
warning, and a criterion whose mapping is ENTIRELY out-of-core derives as **out of engine
|
|
25
|
+
scope** — status `manual` with a dedicated justification (never a silent NA). Shipped
|
|
26
|
+
example: **RGAA 8.1** (doctype → the removed 4.1.1). A well-formed id that never existed
|
|
27
|
+
(`9.9.9`) is rejected outright. See `references/packs.md`.
|
|
28
|
+
- **Pack locales ≠ the CLI's `Lang`.** A pack's `LocaleString`s may carry any BCP-47-ish
|
|
29
|
+
tag (`pt-BR`, `nl-BE`…) — vocabulary and locales are decoupled from the CLI's own
|
|
30
|
+
`--lang fr|en` UI frame (whose `auto` default resolves conversation → repo `<html lang>`
|
|
31
|
+
→ the pack's `defaultLocale` → English).
|
|
32
|
+
|
|
33
|
+
## Using a pack
|
|
34
|
+
|
|
35
|
+
`--standard <pack>` re-keys the output of `report`, `prd`, `criteria`, `check` and `verify`.
|
|
36
|
+
The default is the WCAG core.
|
|
37
|
+
|
|
38
|
+
```
|
|
39
|
+
node scripts/ultra11y.mjs report --in audits/audit-latest.json --standard rgaa # → audits/rgaa-YYYY-MM-DD.md
|
|
40
|
+
node scripts/ultra11y.mjs criteria --standard rgaa --theme 1 # a pack theme
|
|
41
|
+
node scripts/ultra11y.mjs criteria --standard rgaa 8.3 # one pack criterion (shows its WCAG SCs)
|
|
42
|
+
node scripts/ultra11y.mjs check --report audits/rgaa-YYYY-MM-DD.md --standard rgaa
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
An unknown `--standard` value errors out (never a silent fallback to WCAG).
|
|
46
|
+
|
|
47
|
+
## Auditor vocabulary
|
|
48
|
+
|
|
49
|
+
The `prd` **auditor block** (the default output) and the **GitHub issues** it files are
|
|
50
|
+
rendered with the **active standard's own vocabulary** — the nouns an auditor of that standard
|
|
51
|
+
reads. A pack declares them under an optional `vocabulary` object (`LocaleString`s):
|
|
52
|
+
|
|
53
|
+
| term | RGAA (fr) | WCAG core (en) | fallback |
|
|
54
|
+
|---|---|---|---|
|
|
55
|
+
| `theme` | Thématique | Principle · Guideline | Theme |
|
|
56
|
+
| `criterion` | Critère | Success criterion | Criterion |
|
|
57
|
+
| `test` | Test | Technique | Test |
|
|
58
|
+
| `conformant` | Conforme (C) | Pass | Conformant |
|
|
59
|
+
| `nonConformant` | Non conforme (NC) | Fail | Non-conformant |
|
|
60
|
+
| `notApplicable` | Non applicable (NA) | Not applicable | Not applicable |
|
|
61
|
+
| `auditorHeading` | Critère d'accessibilité | Accessibility criterion | Accessibility criterion |
|
|
62
|
+
|
|
63
|
+
Every field is optional: what a pack omits falls back to a generic default, the WCAG core keeps
|
|
64
|
+
its own built-in set (it is not a pack), and `--lang fr|en` picks the locale. So a new country
|
|
65
|
+
pack ships its terminology and the auditor output speaks its language with **no engine change** —
|
|
66
|
+
the modular seam that makes `prd` work for each country, not just France. Resolution lives in
|
|
67
|
+
`src/standards/vocabulary.ts`; the RGAA terms are set in `scripts/build-pack-rgaa.mjs`.
|
|
68
|
+
|
|
69
|
+
## Shipped packs
|
|
70
|
+
|
|
71
|
+
- **RGAA 4.1.2** (`--standard rgaa`) — France, © DINUM, Licence Ouverte / Etalab 2.0.
|
|
72
|
+
13 themes, 106 criteria, French. The flagship example pack.
|
|
73
|
+
|
|
74
|
+
## International equivalence (version-accurate)
|
|
75
|
+
|
|
76
|
+
Several standards incorporate WCAG by reference, but **not all at the same version** — state
|
|
77
|
+
the right one rather than blanket-claiming 2.2:
|
|
78
|
+
|
|
79
|
+
- **Section 508** (US, revised 2018) incorporates **WCAG 2.0 Level AA** for web content.
|
|
80
|
+
- **EN 301 549** (EU): v3.2.1 references **WCAG 2.1**; v4 (2024+) references **WCAG 2.2**.
|
|
81
|
+
- **AODA** (Ontario, Canada) references **WCAG 2.0 AA**.
|
|
82
|
+
|
|
83
|
+
A WCAG 2.2 AA audit therefore covers the web requirements of these standards at their
|
|
84
|
+
respective WCAG versions; a country pack adds that standard's own numbering and wording.
|
|
85
|
+
|
|
86
|
+
## Adding your country
|
|
87
|
+
|
|
88
|
+
Two ways, depending on whether you want it shipped or just plugged locally:
|
|
89
|
+
|
|
90
|
+
- **Built-in (shipped):** drop a pack JSON under `src/data/standards/`, register it in
|
|
91
|
+
`src/standards/registry.ts`, add a test, and open a PR. The full authoring contract
|
|
92
|
+
(pack schema, locale rules, licence requirements, the WCAG-SC mapping, tests) is in
|
|
93
|
+
**`../../CONTRIBUTING.md`**.
|
|
94
|
+
- **Runtime (no rebuild):** author the pack JSON, gate it with `pack check`, and load it
|
|
95
|
+
with `--pack ./pack.json` (or a `.ultra11yrc.json`). An external standard or rule source
|
|
96
|
+
can be **AI-ingested** into a pack + implementation guidance and validated by the same
|
|
97
|
+
gate — see **`packs.md`** and **`guidance.md`**.
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
# Structure — headings, landmarks, lists, tables
|
|
2
|
+
|
|
3
|
+
1.3.1 is the criterion that fails most often in real audits, because it covers everything a
|
|
4
|
+
sighted user reads from layout alone. The test is always the same: **is the relationship I
|
|
5
|
+
can see also programmatically determinable?**
|
|
6
|
+
|
|
7
|
+
## Headings
|
|
8
|
+
|
|
9
|
+
Headings are the primary navigation aid for screen-reader users — most jump between them
|
|
10
|
+
rather than reading linearly. Judge the OUTLINE, not the styling.
|
|
11
|
+
|
|
12
|
+
```html
|
|
13
|
+
<h1>Quarterly report</h1>
|
|
14
|
+
<h2>Revenue</h2>
|
|
15
|
+
<h3>Europe</h3>
|
|
16
|
+
<h3>Asia</h3>
|
|
17
|
+
<h2>Costs</h2>
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
- Never skip a level going down (`h2` → `h4`). Going back up is fine (`h3` → `h2`).
|
|
21
|
+
- Style with CSS, not by picking a level. A "small heading" is still an `h2` if it opens a
|
|
22
|
+
second-level section.
|
|
23
|
+
- A visually styled `<div class="title">` is not a heading. Either mark it up, or accept it
|
|
24
|
+
is not part of the outline.
|
|
25
|
+
- One `<h1>` per page is a convention, not a rule — the engine reports it as a
|
|
26
|
+
recommendation, never a non-conformity.
|
|
27
|
+
|
|
28
|
+
Engine rules: `heading-order-skip`, `empty-heading`; `h1-missing`/`h1-multiple` (advisory).
|
|
29
|
+
Yours: whether each heading actually **describes** its section (2.4.6).
|
|
30
|
+
|
|
31
|
+
## Landmarks
|
|
32
|
+
|
|
33
|
+
```html
|
|
34
|
+
<header> <!-- banner -->
|
|
35
|
+
<nav aria-label="Main">…</nav>
|
|
36
|
+
</header>
|
|
37
|
+
<main id="content"> <!-- exactly one per page -->
|
|
38
|
+
…
|
|
39
|
+
</main>
|
|
40
|
+
<aside aria-label="Related articles">…</aside> <!-- complementary -->
|
|
41
|
+
<footer>…</footer> <!-- contentinfo -->
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
- Every piece of content should sit inside a landmark; text stranded between them is
|
|
45
|
+
unreachable by landmark navigation.
|
|
46
|
+
- Name repeated landmarks: two `<nav>`s need `aria-label` to be told apart. One `<nav>`
|
|
47
|
+
needs no label — "Navigation navigation" is what a redundant one produces.
|
|
48
|
+
- `<main>` is the target of the skip link, and needs `tabindex="-1"` to accept focus.
|
|
49
|
+
|
|
50
|
+
Engine rules: `missing-main-landmark`, `multiple-main-landmark`, `nav-landmark-missing`,
|
|
51
|
+
`nav-landmark-unnamed`, `skip-link-target-missing`.
|
|
52
|
+
|
|
53
|
+
## Lists
|
|
54
|
+
|
|
55
|
+
```html
|
|
56
|
+
<ul> <!-- unordered: order carries no meaning -->
|
|
57
|
+
<li>Coffee</li>
|
|
58
|
+
</ul>
|
|
59
|
+
<ol> <!-- ordered: steps, rankings -->
|
|
60
|
+
<li>Preheat the oven</li>
|
|
61
|
+
</ol>
|
|
62
|
+
<dl> <!-- name/value pairs: glossary, spec sheet, recap screens -->
|
|
63
|
+
<dt>Weight</dt><dd>1.2 kg</dd>
|
|
64
|
+
<div><dt>Colour</dt><dd>Black</dd></div> <!-- a grouping <div> is allowed -->
|
|
65
|
+
</dl>
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Screen readers announce "list, 5 items" — that count is the information. A run of `<div>`s
|
|
69
|
+
separated by `<br>` announces nothing. Only `<li>` (and `<script>`/`<template>`) may be a
|
|
70
|
+
direct child of `<ul>`/`<ol>`; only `<dt>`/`<dd>` (optionally wrapped in a `<div>`) of `<dl>`.
|
|
71
|
+
|
|
72
|
+
**The recap-screen pattern** is the one most often got wrong: a "Your details" summary built
|
|
73
|
+
from `<div class="label">` / `<div class="value">` pairs conveys the relationship visually
|
|
74
|
+
only. Use `<dl>`, or associate the pair programmatically. Engine rules: `list-structure`,
|
|
75
|
+
`dl-structure`.
|
|
76
|
+
|
|
77
|
+
## Tables
|
|
78
|
+
|
|
79
|
+
A table is for **data**. Using one for layout is not a failure in itself, but it must then
|
|
80
|
+
be marked `role="presentation"` and carry no data markup.
|
|
81
|
+
|
|
82
|
+
```html
|
|
83
|
+
<table>
|
|
84
|
+
<caption>Rainfall by city, 2025</caption> <!-- the table's accessible name -->
|
|
85
|
+
<thead>
|
|
86
|
+
<tr><th scope="col">City</th><th scope="col">January</th></tr>
|
|
87
|
+
</thead>
|
|
88
|
+
<tbody>
|
|
89
|
+
<tr><th scope="row">Paris</th><td>52 mm</td></tr>
|
|
90
|
+
</tbody>
|
|
91
|
+
</table>
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
- `scope` takes exactly `col`, `row`, `colgroup`, `rowgroup`. `scope="column"` is invalid and
|
|
95
|
+
silently does nothing (engine rule: `scope-value-invalid` is a declared gap — check it by
|
|
96
|
+
eye until it lands).
|
|
97
|
+
- `<caption>` is the name; keep it SHORT. A long explanation belongs in a paragraph
|
|
98
|
+
associated via `aria-describedby`, not in the caption (RGAA 5.5).
|
|
99
|
+
|
|
100
|
+
**Complex tables** — irregular headers, spans, two header rows — need explicit wiring, since
|
|
101
|
+
`scope` can no longer express the relationship:
|
|
102
|
+
|
|
103
|
+
```html
|
|
104
|
+
<table>
|
|
105
|
+
<caption>Opening hours</caption>
|
|
106
|
+
<tr><td></td><th id="morning">Morning</th><th id="afternoon">Afternoon</th></tr>
|
|
107
|
+
<tr><th id="mon">Monday</th>
|
|
108
|
+
<td headers="mon morning">9–12</td>
|
|
109
|
+
<td headers="mon afternoon">14–18</td></tr>
|
|
110
|
+
</table>
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
Every `headers` id must name a header cell **of the same table** (engine rule:
|
|
114
|
+
`headers-attr-dangling`), and a header no cell references heads nothing (`th-no-data-cells`).
|
|
115
|
+
Mixing `scope` and `headers` in one table is where associations quietly get lost — pick one.
|
|
116
|
+
|
|
117
|
+
`colspan`/`rowspan` are fine in a simple table; once they make the header structure
|
|
118
|
+
irregular, switch to `headers`/`id`. Engine rules: `data-table-no-headers`,
|
|
119
|
+
`table-caption-missing`, `layout-table-data-markup`, `sortable-header-no-aria-sort`.
|
|
120
|
+
|
|
121
|
+
## 1.3.2 Meaningful Sequence
|
|
122
|
+
|
|
123
|
+
The DOM order is the reading order. CSS that reorders content visually
|
|
124
|
+
(`order`, `flex-direction: row-reverse`, `position: absolute`, `grid-area`) makes the two
|
|
125
|
+
diverge, and the divergence becomes a failure when it changes the meaning.
|
|
126
|
+
|
|
127
|
+
Test: read the DOM top to bottom. Does the sequence still make sense? A sidebar declared
|
|
128
|
+
between two paragraphs of an article will be read in the middle of the article.
|
|
129
|
+
|
|
130
|
+
## What the engine settles, and what stays yours
|
|
131
|
+
|
|
132
|
+
| Decided by the engine | Yours to adjudicate |
|
|
133
|
+
|---|---|
|
|
134
|
+
| Heading level skips, empty headings, malformed lists and description lists | Whether a heading **describes** its section (2.4.6) |
|
|
135
|
+
| Missing/duplicate `<main>`, unnamed repeated `<nav>`, broken skip-link target | Whether every region of content sits in a landmark |
|
|
136
|
+
| Data table with no headers, no caption, dangling `headers`, unreferenced `<th>` | Whether a table is data or layout in the first place |
|
|
137
|
+
| — | Whether a visual relationship (recap screens, columns) exists programmatically (1.3.1) |
|
|
138
|
+
| — | Whether the DOM order preserves meaning (1.3.2) |
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
# `check` and `verify` gates
|
|
2
|
+
|
|
3
|
+
Two assurance levels before shipping a report, plus the agent-adjudication worklist.
|
|
4
|
+
|
|
5
|
+
## `check` — structural integrity
|
|
6
|
+
|
|
7
|
+
```
|
|
8
|
+
node scripts/ultra11y.mjs check --report audits/wcag-YYYY-MM-DD.md
|
|
9
|
+
```
|
|
10
|
+
Fails (non-zero) if: one of the 5 sections is missing, a cited criterion id does not exist in
|
|
11
|
+
the active standard, an `NA` criterion has no justification, or the pass rate is absent. This
|
|
12
|
+
is the baseline anti-hallucination guard. `--quiet` emits only the exit code; `--json` the
|
|
13
|
+
structured list of problems. For a country pack report, add `--standard <pack>` so the right
|
|
14
|
+
id grammar/registry is validated.
|
|
15
|
+
|
|
16
|
+
### Semantic gate — `check --report … --semantic`
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
node scripts/ultra11y.mjs check --report audits/wcag-YYYY-MM-DD.md --semantic [--verdicts VERIFY.todo.json]
|
|
20
|
+
```
|
|
21
|
+
Fails **closed** if there is no adjudicated verdicts artifact (never green-but-inactive),
|
|
22
|
+
re-runs the **uncapped** coverage check, and re-grounds every passing verdict against the cited
|
|
23
|
+
source. `--verdicts` points at the artifact (default `VERIFY.todo.json` next to the report).
|
|
24
|
+
|
|
25
|
+
### Applicability gate — `check --report … --standard rgaa --in audit.json`
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
node scripts/ultra11y.mjs check --report audits/rgaa-YYYY-MM-DD.md --standard rgaa --in audit.json
|
|
29
|
+
```
|
|
30
|
+
Re-derives the pack view from the audit and fails on any **over- or under-projected** NC — the
|
|
31
|
+
RGAA applicability gate, so a pack report can neither drop nor invent a criterion vs the WCAG core.
|
|
32
|
+
|
|
33
|
+
## `verify` — adversarial verification of non-conformities
|
|
34
|
+
|
|
35
|
+
```
|
|
36
|
+
node scripts/ultra11y.mjs verify --report audits/wcag-YYYY-MM-DD.md --semantic
|
|
37
|
+
```
|
|
38
|
+
Generates a **worklist** `VERIFY.md` + `VERIFY.todo.json`: one entry per non-conformity
|
|
39
|
+
(criterion ↔ `file:line` ↔ cited claim), capped by `--max-verify` (default 40). For each
|
|
40
|
+
entry, open the cited code and assign a verdict in `VERIFY.todo.json`:
|
|
41
|
+
|
|
42
|
+
- `supported` — the non-conformity is real;
|
|
43
|
+
- `partial` — real but the criterion/wording is imprecise;
|
|
44
|
+
- `refuted` — false (the cited element is actually conforming);
|
|
45
|
+
- `unsupported` — the cited element is not enough to decide.
|
|
46
|
+
|
|
47
|
+
In `--semantic` mode, explicitly confirm the cited snippet **supports** the non-conformity.
|
|
48
|
+
The worklist-generation step also accepts `--json` (emits `{mdPath, todoPath, count, items}`
|
|
49
|
+
so an agent can fill verdicts in-memory). **Prioritise `preliminary` findings** (SFC/library
|
|
50
|
+
source, see `references/false-positives.md`): refute the ones the rendered DOM disproves.
|
|
51
|
+
Then apply the gate:
|
|
52
|
+
```
|
|
53
|
+
node scripts/ultra11y.mjs verify --apply VERIFY.todo.json
|
|
54
|
+
```
|
|
55
|
+
The gate fails (non-zero) if any entry is `refuted`, `unsupported`, or unadjudicated. Goal: no
|
|
56
|
+
fabricated non-conformity survives into the final report.
|
|
57
|
+
|
|
58
|
+
## `verify --manual` — adjudicate the residual (judgment / needs-rendering) criteria
|
|
59
|
+
|
|
60
|
+
The **AI agent** decides the `manual` criteria itself, gated. Run from the audit's cwd
|
|
61
|
+
(harvesting re-reads the audited source files):
|
|
62
|
+
```
|
|
63
|
+
node scripts/ultra11y.mjs verify --report audits/wcag-YYYY-MM-DD.md --in audit.json --manual --out .
|
|
64
|
+
```
|
|
65
|
+
This emits an **ADJUDICATION worklist** — `ADJUDICATE.todo.json` + `ADJUDICATE.md`, one item per
|
|
66
|
+
residual criterion, pre-loaded with the engine's harvested evidence (every image's `alt`, every
|
|
67
|
+
link's text + context, the literal colour pairs, control labels, the heading outline, ARIA state,
|
|
68
|
+
`tabindex`, lang-of-parts). Fill each item's `verdict` (provenance `decidedBy: "agent"` recorded):
|
|
69
|
+
|
|
70
|
+
- `C` / `NA` — with a `justification` from the evidence;
|
|
71
|
+
- `NC` — with ≥1 **groundable** finding (`file`/`line`/`message`/`snippet`);
|
|
72
|
+
- `manual` — with a `reason`: `"needs-rendered-dom"` (decide via `scan`) or `"undecidable"`.
|
|
73
|
+
|
|
74
|
+
Fold the verdicts back FAIL-CLOSED:
|
|
75
|
+
```
|
|
76
|
+
node scripts/ultra11y.mjs verify --apply ADJUDICATE.todo.json --in audit.json --out .
|
|
77
|
+
```
|
|
78
|
+
It rejects a null verdict, a `C`/`NA` without a `justification`, an `NC` without a groundable
|
|
79
|
+
finding, a `manual` without a `reason`, or any uncovered residual criterion. Agent `NC`s become
|
|
80
|
+
real `agent:<sc>` findings that re-render in the report's §2 and re-enter the verify worklist;
|
|
81
|
+
`report`/`prd` re-render with the adjudicated statuses; §5 shrinks to only still-`manual` items.
|
|
82
|
+
|
|
83
|
+
## Residual risk
|
|
84
|
+
|
|
85
|
+
After adjudication, only the criteria the agent marked `manual` (`needs-rendered-dom` /
|
|
86
|
+
`undecidable`) stay in the report's "to assess manually" section — never mark them conforming
|
|
87
|
+
without a recorded, gated justification (agent adjudication via `verify --manual`, or `scan`
|
|
88
|
+
evidence for the rendering criteria).
|