ultra11y 2.32.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/.codex-plugin/plugin.json +1 -1
- package/.opencode/plugins/ultra11y.js +1 -1
- package/README.md +14 -2
- package/package.json +1 -1
- package/scripts/ultra11y.mjs +1133 -222
- package/skills/review-a11y/SKILL.md +1 -1
- package/skills/review-a11y/scripts/ultra11y.mjs +1133 -222
- package/skills/ultra11y/SKILL.md +17 -11
- package/skills/ultra11y/references/audit.md +1 -1
- package/skills/ultra11y/references/ci.md +63 -0
- package/skills/ultra11y/references/judgment.md +8 -0
- package/skills/ultra11y/references/methodology.md +1 -1
- package/skills/ultra11y/references/orchestrators.md +26 -11
- package/skills/ultra11y/references/prd.md +9 -36
- package/skills/ultra11y/references/tickets.md +156 -0
- package/skills/ultra11y/scripts/ultra11y.mjs +1133 -222
- package/src/cli.ts +386 -63
- package/src/config.ts +10 -0
- package/src/gh-cli.ts +39 -0
- package/src/mcp/handlers.ts +28 -0
- package/src/mcp/tools.ts +27 -0
- package/src/pages.ts +13 -0
- package/src/pr-comment.ts +78 -0
- package/src/tickets/grain.ts +218 -0
- package/src/tickets/http.ts +107 -0
- package/src/tickets/providers/github.ts +155 -0
- package/src/tickets/providers/gitlab.ts +158 -0
- package/src/tickets/providers/jira.ts +152 -0
- package/src/tickets/providers/remote.ts +40 -0
- package/src/tickets/push.ts +94 -0
- package/src/tickets/registry.ts +45 -0
- package/src/tickets/render.ts +157 -0
- package/src/tickets/types.ts +154 -0
- package/src/types.ts +1 -1
- package/src/gh.ts +0 -333
package/skills/ultra11y/SKILL.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ultra11y
|
|
3
3
|
description: "Use to AUDIT existing HTML/CSS/JSX against WCAG 2.2 AA accessibility and produce a dated auditor-conformance report, OR to AUTHOR/REVIEW accessible markup (native-HTML-first, ARIA last). An install-free engine (`node scripts/ultra11y.mjs`, no keys) runs 78 static checks tied to WCAG criteria — alt/lang/title, unlabeled fields, empty links/buttons, tables, headings, ARIA vocabulary, label-in-name, autocomplete — measured against the official W3C ACT corpus. The engine decides 3 of the 55 criteria; the AI agent adjudicates the 38 judgment ones from harvested evidence + a per-criterion decision protocol (`verify --manual`, gated, fannable via `orchestrate`), and the 14 rendering ones go to `scan` — never silently conforming. Library/SFC code is audited as RENDERED captures (`render --setup`); country standards are pluggable packs (`--standard rgaa`, `scan --sample`). check/verify reject invented non-conformities. Triggers: 'audit WCAG/a11y', 'make accessible', 'fix a11y', 'audit RGAA'."
|
|
4
|
-
when_to_use: "Invoke when the user asks for an accessibility AUDIT or a formal deliverable: audit a repo, site or page against WCAG 2.2 AA or a country standard (RGAA, Section 508, EN 301 549); produce a dated conformance report, a criterion grid, a PRD
|
|
4
|
+
when_to_use: "Invoke when the user asks for an accessibility AUDIT or a formal deliverable: audit a repo, site or page against WCAG 2.2 AA or a country standard (RGAA, Section 508, EN 301 549); produce a dated conformance report, a criterion grid, a PRD backlog, or tickets (GitHub/GitLab/Jira); author accessible markup from scratch; wire the repo gate (init --hook/--ci) or a rendered-DOM capture pipeline. For a review of the code UNDER CHANGE (diff, branch, PR), use the `review-a11y` skill instead."
|
|
5
5
|
license: MIT
|
|
6
6
|
metadata:
|
|
7
|
-
version:
|
|
7
|
+
version: 3.0.0
|
|
8
8
|
---
|
|
9
9
|
|
|
10
10
|
# ultra11y — audit WCAG 2.2 AA and write accessible markup
|
|
@@ -58,7 +58,7 @@ adjudicates. So the engine's clean run is a starting point, never a verdict —
|
|
|
58
58
|
> `<html lang>` → the active standard's default locale → English) — a scripted/CI
|
|
59
59
|
> fallback, not a substitute for passing `--lang` yourself.
|
|
60
60
|
> 7. **Technical tokens stay in English, even in French prose.** In any French deliverable
|
|
61
|
-
> you write (report commentary, PRD,
|
|
61
|
+
> you write (report commentary, PRD, tracker tickets, judgment verdicts), attribute/
|
|
62
62
|
> element/role names and their values are code, not prose — never translate them:
|
|
63
63
|
> `aria-live` stays `aria-live` (never « région live »), same for `tabindex`, `alt`,
|
|
64
64
|
> `role="alert"`, landmark role names. The engine's own fr catalog follows this; match
|
|
@@ -90,7 +90,7 @@ Domain knowledge first, then the tooling. Read the one that matches the question
|
|
|
90
90
|
| `references/false-positives.md` | Where a finding may still be wrong, and how to refute it |
|
|
91
91
|
| `references/rendered.md` · `references/automation.md` | Auditing produced HTML, captures, hooks and CI |
|
|
92
92
|
| `references/harnesses.md` | Where this runs: Claude Code, Codex CLI, OpenCode, and the AGENTS.md fallback |
|
|
93
|
-
| `references/orchestrators.md` | Driving the engine from a workflow engine / non-GitHub CI: pinning, the six-step pipeline, exit codes,
|
|
93
|
+
| `references/orchestrators.md` | Driving the engine from a workflow engine / non-GitHub CI: pinning, the six-step pipeline, exit codes, the ticket set as JSON for any tracker |
|
|
94
94
|
| `references/ci.md` | `--format sarif\|github`: inline PR annotations, code scanning, job summary |
|
|
95
95
|
| `references/pages.md` | Page snapshots + the per-page criterion grid (`pages`), and its honesty rules |
|
|
96
96
|
| `references/e2e.md` | `render --e2e`: auditing a page during your Playwright/Cypress run |
|
|
@@ -99,7 +99,8 @@ Domain knowledge first, then the tooling. Read the one that matches the question
|
|
|
99
99
|
| `references/dynamic.md` | The `scan` tier: runtimes, probes, authenticated pages |
|
|
100
100
|
| `references/scale.md` | Focusing an audit on a large repository |
|
|
101
101
|
| `references/fix.md` · `references/correction.md` | Applying fixes, by priority, without regressions |
|
|
102
|
-
| `references/prd.md` | The auditor block as a backlog
|
|
102
|
+
| `references/prd.md` | The auditor block as a backlog (markdown) |
|
|
103
|
+
| `references/tickets.md` | Filing that backlog as tickets: GitHub, GitLab, Jira, and at which granularity |
|
|
103
104
|
| `references/standards.md` · `references/packs.md` · `references/guidance.md` | Country standards, authoring a pack, implementation guidance |
|
|
104
105
|
| `references/methodology.md` | Statuses, pass rate, severities, report format |
|
|
105
106
|
| `references/cross-file.md` | `--graph`: imports and cross-file rules |
|
|
@@ -109,7 +110,7 @@ Domain knowledge first, then the tooling. Read the one that matches the question
|
|
|
109
110
|
|
|
110
111
|
- **"Audit / compliance report"** → `node scripts/ultra11y.mjs audit … --json`, then
|
|
111
112
|
`report` (synthesis table + one **auditor conformance block** per NC criterion — same
|
|
112
|
-
block `prd
|
|
113
|
+
block `prd` and `tickets` use), then `check`; read **`references/audit.md`**.
|
|
113
114
|
- **"Code rendered by a library (DSFR, MUI…) or a `.vue`/`.svelte`/`.astro` SFC / avoid
|
|
114
115
|
false negatives"** → audit the **produced HTML**, not the source template. Easiest:
|
|
115
116
|
install the zero-touch **capture** harvester (`render --setup`) so your tests serialize
|
|
@@ -128,13 +129,17 @@ Domain knowledge first, then the tooling. Read the one that matches the question
|
|
|
128
129
|
`audit --graph` resolves imports and applies cross-file rules (an icon-only component
|
|
129
130
|
used without a name, an anchor target in another file…), no browser; read
|
|
130
131
|
**`references/cross-file.md`**.
|
|
131
|
-
- **"Generate the fix markdown / PRDs
|
|
132
|
+
- **"Generate the fix markdown / PRDs"** → `prd` (the SAME auditor
|
|
132
133
|
conformance block `report`'s NC section renders — theme/criterion/test/WCAG+level/
|
|
133
134
|
finding/expected/verification in the active standard's vocabulary — as a backlog);
|
|
134
135
|
`--split criterion`, `--format doc` for a product-requirements doc, `--format remediation`
|
|
135
|
-
for the legacy dev backlog
|
|
136
|
-
body) or `--gh-single` for a single consolidated issue via the `gh` CLI); read
|
|
136
|
+
for the legacy dev backlog). It writes markdown only; read
|
|
137
137
|
**`references/prd.md`**.
|
|
138
|
+
- **"Open tickets / issues for this (GitHub · GitLab · Jira)"** → `tickets`, a separate
|
|
139
|
+
command that reads the audit and files it, writing no markdown. `--grain` picks what one
|
|
140
|
+
ticket is: per criterion (default), per page, per page+criterion, per file, or one
|
|
141
|
+
consolidated. De-dupe is by exact title, so re-running never duplicates. Always
|
|
142
|
+
`--dry-run` first; read **`references/tickets.md`**.
|
|
138
143
|
- **"Plug or author a standards pack (RGAA & beyond), AI-ingest external rules"** →
|
|
139
144
|
`--pack`/`.ultra11yrc.json` to load at runtime, `pack check` to gate it (the
|
|
140
145
|
anti-hallucination guardrail), `pack scaffold` to start one; concrete before/after
|
|
@@ -273,7 +278,8 @@ node scripts/ultra11y.mjs verify --report audits/wcag-YYYY-MM-DD.md --in audits/
|
|
|
273
278
|
node scripts/ultra11y.mjs verify --apply audits/ADJUDICATE.todo.json --in audits/audit-latest.json --out audits
|
|
274
279
|
node scripts/ultra11y.mjs orchestrate --run audits # fan the judgment phases out (--eco for the sequential path)
|
|
275
280
|
node scripts/ultra11y.mjs report --in audits/audit-latest.json --out audits # → audits/wcag-YYYY-MM-DD.md
|
|
276
|
-
node scripts/ultra11y.mjs prd
|
|
281
|
+
node scripts/ultra11y.mjs prd --in audits/audit-latest.json # the same auditor block as a backlog
|
|
282
|
+
node scripts/ultra11y.mjs tickets --in audits/audit-latest.json --dry-run # file it: GitHub/GitLab/Jira
|
|
277
283
|
node scripts/ultra11y.mjs check --report audits/wcag-YYYY-MM-DD.md # integrity gate
|
|
278
284
|
node scripts/ultra11y.mjs criteria 1.4.3 # one success criterion (--list for all)
|
|
279
285
|
node scripts/ultra11y.mjs fix "src/**/*.html" --write --iterate # apply the safe codemods to a fixpoint
|
|
@@ -312,7 +318,7 @@ drive the judgment and content stages:
|
|
|
312
318
|
4. **Re-audit** (on the render where relevant) and repeat.
|
|
313
319
|
5. **Deliver the auditor block.** `report` (compliance doc: synthesis + one auditor
|
|
314
320
|
conformance block — theme/criterion/test/WCAG+level/finding/expected/verification —
|
|
315
|
-
per NC criterion) and `prd` (the same blocks as an actionable backlog
|
|
321
|
+
per NC criterion) and `prd` (the same blocks as an actionable backlog; `tickets`
|
|
316
322
|
filing one GitHub issue per criterion with that identical block) are two views of the
|
|
317
323
|
ONE building block, in the language of this conversation (pass `--lang` explicitly —
|
|
318
324
|
Core rule 5).
|
|
@@ -42,7 +42,7 @@ non-conformity.
|
|
|
42
42
|
→ `audits/wcag-YYYY-MM-DD.md` (5 sections, see `references/methodology.md`); section 2
|
|
43
43
|
renders one **auditor conformance block** per NC criterion (theme, criterion + official
|
|
44
44
|
wording, test(s), WCAG mapping + level, finding, expected state, verification,
|
|
45
|
-
`file:line` occurrences), grouped by severity — the SAME block `prd` and
|
|
45
|
+
`file:line` occurrences), grouped by severity — the SAME block `prd` and `tickets`
|
|
46
46
|
emit (see `references/prd.md`), so report and backlog never drift. For a country
|
|
47
47
|
standard, add `--standard rgaa` (see `references/standards.md`). Pass `--lang` to match
|
|
48
48
|
your conversation; without it, `auto` resolves the audit's `scope.langs` → the
|
|
@@ -53,6 +53,69 @@ run and an RGAA run coexist rather than overwriting each other. The Markdown rep
|
|
|
53
53
|
through `check` before being uploaded, so CI cannot publish a report citing an invented
|
|
54
54
|
criterion.
|
|
55
55
|
|
|
56
|
+
## Adjudicating the judgment criteria in CI (`adjudicate`)
|
|
57
|
+
|
|
58
|
+
Of the 55 WCAG 2.2 AA criteria the engine decides a handful; **38 are judgment calls**, and
|
|
59
|
+
under RGAA **81 of 106** can only ever derive `manual`. Inside a coding agent the agent rules
|
|
60
|
+
on them. In CI nobody does, so they stay « à évaluer » and the published conformance rate is
|
|
61
|
+
partial by construction. `adjudicate` closes that — opt-in, in two modes.
|
|
62
|
+
|
|
63
|
+
```yaml
|
|
64
|
+
env:
|
|
65
|
+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} # the job's env, never an input
|
|
66
|
+
steps:
|
|
67
|
+
- uses: maxgfr/ultra11y@v2
|
|
68
|
+
with:
|
|
69
|
+
standard: rgaa
|
|
70
|
+
adjudicate: api # or `agent`
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
| input | default | |
|
|
74
|
+
|---|---|---|
|
|
75
|
+
| `adjudicate` | `none` | `api` · `agent` · `none` |
|
|
76
|
+
| `adjudicate-model` | *(empty)* | model id for `api`; else `$ULTRA11Y_LLM_MODEL`, else the engine default |
|
|
77
|
+
| `gate-adjudicated` | `false` | let a model-ruled NC fail the job |
|
|
78
|
+
|
|
79
|
+
**`api`** sends the worklist to the Messages API in batches of 8 and folds the verdicts back.
|
|
80
|
+
**`agent`** emits the worklist plus `orchestrate --eco`'s runbook and hands them to a
|
|
81
|
+
`claude-code-action` run, then folds the same way. Both end in the **same fail-closed gate** an
|
|
82
|
+
agent's verdicts pass through, so neither can assert a conformance the engine refuses.
|
|
83
|
+
|
|
84
|
+
The difference between them is **evidence, not trust**. `judge` rules from the harvested
|
|
85
|
+
evidence alone — capped at 30 items per criterion, snippets truncated — while the agent can
|
|
86
|
+
open the cited files and read around them, which is what *link purpose **in context*** actually
|
|
87
|
+
asks for. Expect the API mode to answer `manual` more often; that is it being honest, not
|
|
88
|
+
broken. The metric that separates them is the count of criteria left to assess.
|
|
89
|
+
|
|
90
|
+
**Both modes read `ANTHROPIC_API_KEY` from the job environment, never from an input** — a
|
|
91
|
+
composite action's steps inherit it, and a secret that never travels through an input is one
|
|
92
|
+
fewer secret to leak into a log. When the key is absent the tier **skips itself**: the job stays
|
|
93
|
+
green, the report is still written, and the criteria stay « à évaluer ». That is not an edge
|
|
94
|
+
case — it is precisely what a **fork's pull request** looks like, where secrets are never
|
|
95
|
+
exposed. An adjudication that fails mid-flight (a rate-limited batch refuses the whole fold) is
|
|
96
|
+
absorbed the same way, because losing the verdicts must never cost you the audit.
|
|
97
|
+
|
|
98
|
+
**Cost is per run and does not amortise.** Roughly $0.20 for a WCAG run and $0.50 for RGAA at
|
|
99
|
+
Sonnet pricing — the worklist is re-sent whole each time and no batch is large enough to earn
|
|
100
|
+
prompt caching. On a busy repository that is real money per push; this belongs on the default
|
|
101
|
+
branch or a schedule far more often than on every PR.
|
|
102
|
+
|
|
103
|
+
**`gate-adjudicated` trades reproducibility for reach.** By default the gate re-audits the
|
|
104
|
+
**source**, so the red/green is a pure function of the commit whatever a model said about it.
|
|
105
|
+
Turn it on and the gate evaluates the adjudicated audit instead (`audit --in`), letting a
|
|
106
|
+
model-ruled non-conformity fail the job — and accepting that two runs on the same commit may
|
|
107
|
+
now disagree. Default off, deliberately.
|
|
108
|
+
|
|
109
|
+
```sh
|
|
110
|
+
# The same re-gate, outside the action:
|
|
111
|
+
node scripts/ultra11y.mjs audit --in audits/audit-latest.json --fail-on blocking
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
`--in` re-gates an audit that already exists rather than computing one, which is the only way
|
|
115
|
+
to gate on verdicts a second detection pass would not see. It refuses every flag that would
|
|
116
|
+
change *what* is audited (`--since`, `--baseline`, paths…): a gate that silently dropped your
|
|
117
|
+
scoping would be a gate nobody could trust.
|
|
118
|
+
|
|
56
119
|
# The `--format` renderings
|
|
57
120
|
|
|
58
121
|
A gate tells a developer *that* the build failed; it does not put the non-conformity **on the
|
|
@@ -151,6 +151,14 @@ A rejected adjudication leaves `audit-latest.json` untouched.
|
|
|
151
151
|
**Strictly opt-in.** With no key the command explains itself and exits 2; nothing else in the
|
|
152
152
|
engine changes.
|
|
153
153
|
|
|
154
|
+
**In the GitHub Action.** `adjudicate: api` runs exactly the command above; `adjudicate: agent`
|
|
155
|
+
hands the same worklist — plus `orchestrate --eco`'s runbook and the adjudicator contract — to
|
|
156
|
+
a `claude-code-action` run, then folds it with `verify --apply`. The agent mode exists because
|
|
157
|
+
`judge` rules from the harvested evidence alone (30 items per criterion, snippets truncated)
|
|
158
|
+
while an agent can open the cited files, which is what *link purpose in context* really needs.
|
|
159
|
+
Both skip themselves when `ANTHROPIC_API_KEY` is absent from the job environment, and both
|
|
160
|
+
absorb their own failure rather than taking the audit down with them. See `references/ci.md`.
|
|
161
|
+
|
|
154
162
|
`--max` bounds the spend and says out loud which criteria it did not submit. It is **refused
|
|
155
163
|
together with `--apply`**, before any call is made: the gate rejects an incomplete
|
|
156
164
|
adjudication by design, so the pair could only ever bill you for a guaranteed failure. Drop
|
|
@@ -54,7 +54,7 @@ content, so the verdict is provisional — confirm against the rendered DOM or r
|
|
|
54
54
|
(C/NC/NA/to assess), (2) non-conformities by priority — one **auditor conformance block**
|
|
55
55
|
per NC criterion (theme, criterion + official wording, test(s), WCAG mapping + level,
|
|
56
56
|
finding, expected state, verification, `file:line` occurrences), the SAME block `prd` and
|
|
57
|
-
|
|
57
|
+
`tickets` emit (see `references/tickets.md`), (3) conforming criteria, (4) justified
|
|
58
58
|
not-applicable criteria, (5) criteria to assess manually.
|
|
59
59
|
|
|
60
60
|
## Worldwide: WCAG core, country standards as packs
|
|
@@ -64,6 +64,13 @@ verdict with no justification, an NC with no resolvable `normativeRef`, a criter
|
|
|
64
64
|
from the adjudication — each is refused by name, and the run stops. That is what keeps the
|
|
65
65
|
final report defensible.
|
|
66
66
|
|
|
67
|
+
**The shipped GitHub Action now runs this pipeline for you.** `adjudicate: agent` performs
|
|
68
|
+
steps 2–4 — `verify --manual`, then `orchestrate --run audits --phase adjudicate --eco`, then a
|
|
69
|
+
`claude-code-action` run pointed at the emitted `RUNBOOK.md`, then `verify --apply`. Reach for
|
|
70
|
+
the raw pipeline above when your orchestrator is *not* GitHub Actions, or when you want to fan
|
|
71
|
+
step 3 out across subagents rather than walk the eco path. Either way steps 1, 2, 4, 5 and 6
|
|
72
|
+
stay deterministic, and only step 3 involves a model.
|
|
73
|
+
|
|
67
74
|
## The adjudication contract
|
|
68
75
|
|
|
69
76
|
`ADJUDICATE.todo.json` declares what may be written into it, so the filler never has to read
|
|
@@ -102,34 +109,40 @@ the criterion in `residualRisks`, where the report declares it as a residual ris
|
|
|
102
109
|
map an undecided criterion to `C`** — that is the one way to turn this pipeline back into
|
|
103
110
|
the thing it replaces.
|
|
104
111
|
|
|
105
|
-
## Filing findings on a tracker
|
|
112
|
+
## Filing findings on a tracker
|
|
106
113
|
|
|
107
|
-
|
|
114
|
+
`tickets` files into **GitHub, GitLab or Jira** directly (see `references/tickets.md`). For
|
|
115
|
+
any other board — a native kanban, an internal tool — take the same items as JSON and file
|
|
116
|
+
them yourself:
|
|
108
117
|
|
|
109
118
|
```sh
|
|
110
|
-
$ENGINE
|
|
111
|
-
# → $RUN/issues-<date>.json
|
|
119
|
+
$ENGINE tickets --in "$RUN/audit-latest.json" --out "$RUN" --lang en
|
|
120
|
+
# → $RUN/issues-<date>.json (writing the set files nothing; add --dry-run to be explicit)
|
|
112
121
|
```
|
|
113
122
|
|
|
123
|
+
`--grain` decides what one item is: `criterion` (default), `page`, `page-criterion`,
|
|
124
|
+
`single` or `file`. The envelope and the item shape are the same at every grain.
|
|
125
|
+
|
|
114
126
|
```jsonc
|
|
115
127
|
{
|
|
116
128
|
"tool": "ultra11y", "kind": "issues", "schemaVersion": 1,
|
|
117
|
-
"standard": "wcag", "date": "2026-08-12", "count": 4,
|
|
129
|
+
"standard": "wcag", "grain": "criterion", "date": "2026-08-12", "count": 4,
|
|
118
130
|
"issues": [{
|
|
119
|
-
"criteriaId": "1.1.1",
|
|
120
131
|
"title": "[a11y] WCAG 1.1.1 — Non-text Content", // the de-dupe grain: stable across runs
|
|
121
132
|
"body": "…", // the auditor block, rendered
|
|
122
133
|
"labels": ["accessibility", "wcag", "bloquant"],
|
|
123
134
|
"severity": "bloquant",
|
|
124
135
|
"advisory": false, // true ⇒ a good practice, NOT a non-conformity
|
|
136
|
+
"scope": { "grain": "criterion", "criteriaId": "1.1.1" },
|
|
125
137
|
"occurrences": [{ "file": "src/Page.tsx", "line": 4, "selector": "img", "message": "…" }]
|
|
126
138
|
}]
|
|
127
139
|
}
|
|
128
140
|
```
|
|
129
141
|
|
|
130
|
-
|
|
131
|
-
with forty `occurrences`, not forty tickets. De-dupe on `title
|
|
132
|
-
|
|
142
|
+
At the default grain, one issue per criterion, not per occurrence — a missing `alt` on forty
|
|
143
|
+
images is one item with forty `occurrences`, not forty tickets. **De-dupe is on `title`**, so
|
|
144
|
+
re-running files nothing twice; keep `--lang` stable across runs, since a criterion's wording
|
|
145
|
+
is part of its title. `advisory: true` marks a non-normative recommendation: it must not be triaged as a
|
|
133
146
|
non-conformity, and it never enters the conformance rate.
|
|
134
147
|
|
|
135
148
|
## Where output lands
|
|
@@ -140,7 +153,8 @@ repo it does not own — write the run's intermediates out-of-tree and let only
|
|
|
140
153
|
dated report land in the project, if at all.
|
|
141
154
|
|
|
142
155
|
`.ultra11y/pages/` is written only by the page-snapshot commands (`scan`, `snapshot`,
|
|
143
|
-
`dev`); `--root` relocates it. `audit`, `report`, `prd`, `verify` and `check`
|
|
156
|
+
`dev`); `--root` relocates it. `audit`, `report`, `prd`, `tickets`, `verify` and `check`
|
|
157
|
+
never touch it.
|
|
144
158
|
|
|
145
159
|
## Exit codes
|
|
146
160
|
|
|
@@ -149,7 +163,8 @@ dated report land in the project, if at all.
|
|
|
149
163
|
| `audit` | reported (no `--fail-on`), or clean | findings ≥ `--fail-on` | usage / bad input |
|
|
150
164
|
| `verify --apply` | folded | the adjudication was refused (see `issues[]`) | usage / bad input |
|
|
151
165
|
| `check` | report is sound | integrity failure | report not found / bad input |
|
|
152
|
-
| `prd` | written |
|
|
166
|
+
| `prd` | written | — (it writes markdown and files nothing) | usage / bad input |
|
|
167
|
+
| `tickets` | filed, skipped, or nothing to file | a creation failed, the provider is unusable, or no page in scope | usage / bad input / past `--max-tickets` |
|
|
153
168
|
|
|
154
169
|
Read exit codes from the process, **not** from a pipeline — `cmd --json | head` reports
|
|
155
170
|
`head`'s status, which is how a failing gate reads as a pass.
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Auditor backlog (`prd`)
|
|
1
|
+
# Auditor backlog (`prd`)
|
|
2
2
|
|
|
3
3
|
`prd` turns an `AuditResult` into an **auditor-legible conformance backlog**, grouped by WCAG
|
|
4
4
|
success criterion (or, with `--standard <pack>`, by a country standard's criteria). It is the
|
|
@@ -6,15 +6,15 @@ success criterion (or, with `--standard <pack>`, by a country standard's criteri
|
|
|
6
6
|
similar, they render the literal **same auditor block** (`src/auditor.ts` `renderAuditorUnit`,
|
|
7
7
|
fed by the same `prdUnits`): `report`'s "Non-conformities by priority" section (§2) IS one of
|
|
8
8
|
these blocks per NC criterion, grouped by severity exactly like the `prd` backlog and the
|
|
9
|
-
|
|
10
|
-
together — there is no separate "report wording" to keep in sync.
|
|
9
|
+
ticket bodies `tickets` files (`references/tickets.md`). Change the block once, all four
|
|
10
|
+
deliverables change together — there is no separate "report wording" to keep in sync.
|
|
11
|
+
|
|
12
|
+
`prd` writes MARKDOWN and files nothing. Filing tickets is `tickets`.
|
|
11
13
|
|
|
12
14
|
```
|
|
13
15
|
node scripts/ultra11y.mjs audit "src/**/*.tsx" --graph --json > audit.json
|
|
14
16
|
node scripts/ultra11y.mjs prd --in audit.json # auditor backlog (default)
|
|
15
17
|
node scripts/ultra11y.mjs prd --in audit.json --split criterion # one file per criterion
|
|
16
|
-
node scripts/ultra11y.mjs prd --in audit.json --gh-issues # + one GitHub issue per criterion
|
|
17
|
-
node scripts/ultra11y.mjs prd --in audit.json --gh-single # + ONE consolidated GitHub issue (whole audit)
|
|
18
18
|
node scripts/ultra11y.mjs prd --in audit.json --standard rgaa --lang fr # rendered with the RGAA (fr) vocabulary
|
|
19
19
|
node scripts/ultra11y.mjs prd --in audit.json --format doc # product-requirements doc (epics/stories/AC)
|
|
20
20
|
node scripts/ultra11y.mjs prd --in audit.json --format remediation # legacy dev fix-backlog
|
|
@@ -60,9 +60,8 @@ node scripts/ultra11y.mjs prd --in audit.json --format remediation # legacy dev
|
|
|
60
60
|
grouped by theme, one **user story** per criterion, **Given/When/Then** acceptance criteria
|
|
61
61
|
templated from the real SC title/techniques (anchored to W3C text), and the task list.
|
|
62
62
|
- **`--split criterion`**: a `prd-<criterion>-YYYY-MM-DD.md` file per criterion (handy for batching).
|
|
63
|
-
- The markdown is **always** written, even with `--gh-issues`.
|
|
64
63
|
- **`--json`**: emits a machine-readable object instead of the file paths —
|
|
65
|
-
`{paths, units
|
|
64
|
+
`{paths, units}` where `units` is the structured per-criterion backlog an agent can consume.
|
|
66
65
|
|
|
67
66
|
## Language of the prose (French deliverables)
|
|
68
67
|
|
|
@@ -72,33 +71,7 @@ attribute/element/role names and their values are code, not prose — `aria-live
|
|
|
72
71
|
`aria-live` (never « région live »), same for `tabindex`, `alt`, `role="alert"`. Normative
|
|
73
72
|
standard vocabulary (e.g. RGAA « lien d'évitement ») keeps its official French wording.
|
|
74
73
|
|
|
75
|
-
##
|
|
76
|
-
|
|
77
|
-
- Uses the **`gh` CLI** (which handles its own auth) — **no** npm dependency, no key in
|
|
78
|
-
ultra11y.
|
|
79
|
-
- **`--gh-issues` → one issue per criterion** (regardless of `--split`), stable title
|
|
80
|
-
`"[a11y] WCAG <sc> — <title>"` (or `"[a11y] <PACK> <id> — …"` under `--standard`), labels
|
|
81
|
-
`accessibility`, `wcag` (or the pack key), severity. The body is the **auditor conformance
|
|
82
|
-
block** (theme, criterion + wording, test(s), WCAG + level, finding, expected, verification,
|
|
83
|
-
`file:line` occurrences + definition site) — same as the default backlog, in the active
|
|
84
|
-
standard's vocabulary. `--format remediation` files the legacy dev body instead.
|
|
85
|
-
- **`--gh-single` → ONE consolidated issue** for the whole audit, stable title
|
|
86
|
-
`"[a11y] WCAG — Accessibility audit"` (or `"[a11y] <PACK> — Accessibility audit"`). The body
|
|
87
|
-
is the full backlog **sectioned by severity** (🔴 blocking → 🟠 major → 🟡 minor), each
|
|
88
|
-
criterion carrying the auditor block. Labelled by the **most severe** criterion.
|
|
89
|
-
- `--gh-single` **wins** if both flags are passed.
|
|
90
|
-
- **De-dupe by title**: an existing issue (open or closed) is skipped, so re-running never
|
|
91
|
-
creates duplicates. The consolidated title carries **no count or date**, so it stays stable
|
|
92
|
-
across re-runs.
|
|
93
|
-
- **Caveat — `--lang` changes the title**: the issue title embeds the localized criterion/pack
|
|
94
|
-
title (e.g. `[a11y] WCAG 1.4.3 — Contrast (Minimum)` vs `[a11y] WCAG 1.4.3 — Contraste (minimum)`).
|
|
95
|
-
Since de-dupe matches on the **exact title string**, re-running `prd --gh-issues` with a
|
|
96
|
-
**different `--lang`** than a previous run does **not** match the earlier issue and creates a
|
|
97
|
-
new one instead of updating it. Keep `--lang` consistent across re-runs against the same
|
|
98
|
-
repo, or accept the duplicate and close the stale one manually.
|
|
99
|
-
- **Graceful degradation**: if `gh` is absent / unauthenticated, the command says so and exits
|
|
100
|
-
`0` — the markdown was still produced.
|
|
74
|
+
## Filing these as tickets
|
|
101
75
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
> `file:line` already computed.
|
|
76
|
+
See **`references/tickets.md`**: `tickets` files the same auditor blocks into GitHub, GitLab
|
|
77
|
+
or Jira, at a granularity you choose (per criterion, per page, per file, or one consolidated).
|
|
@@ -0,0 +1,156 @@
|
|
|
1
|
+
# Tracker tickets (`tickets`) — GitHub · GitLab · Jira
|
|
2
|
+
|
|
3
|
+
`tickets` reads an `AuditResult` and **files it into a tracker**. It writes no Markdown.
|
|
4
|
+
`prd` and `report` write Markdown and file nothing. That split is deliberate: you could
|
|
5
|
+
previously not push without also producing a document, nor produce a document without
|
|
6
|
+
risking a push.
|
|
7
|
+
|
|
8
|
+
```
|
|
9
|
+
node scripts/ultra11y.mjs audit "src/**/*.tsx" --graph --out audit.json
|
|
10
|
+
|
|
11
|
+
node scripts/ultra11y.mjs tickets --in audit.json --dry-run # ALWAYS start here
|
|
12
|
+
node scripts/ultra11y.mjs tickets --in audit.json # one ticket per criterion
|
|
13
|
+
node scripts/ultra11y.mjs tickets --in audit.json --grain page # one ticket per page
|
|
14
|
+
node scripts/ultra11y.mjs tickets --in audit.json --grain page-criterion # one per page × criterion
|
|
15
|
+
node scripts/ultra11y.mjs tickets --in audit.json --grain single # one consolidated ticket
|
|
16
|
+
node scripts/ultra11y.mjs tickets --in audit.json --grain file # one ticket per source file
|
|
17
|
+
node scripts/ultra11y.mjs tickets --in audit.json --provider gitlab --standard rgaa --lang fr
|
|
18
|
+
node scripts/ultra11y.mjs tickets --in audit.json --provider jira --max-tickets 20
|
|
19
|
+
node scripts/ultra11y.mjs tickets --in audit.json --json # machine-readable plan + result
|
|
20
|
+
node scripts/ultra11y.mjs tickets --in audit.json --out ./run # the ticket SET as JSON, for any other board
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## Filing on a tracker ultra11y does not speak
|
|
24
|
+
|
|
25
|
+
`--out <dir>` writes the tracker-agnostic set to `<dir>/issues-<date>.json` — a stable path a
|
|
26
|
+
workflow engine can read without parsing a payload that also carries prose. It works at every
|
|
27
|
+
grain, carries `occurrences[]` so a board with inline anchors needs nothing else, and writing
|
|
28
|
+
it **files nothing**. `schemaVersion` is what lets a consumer pin. See
|
|
29
|
+
`references/orchestrators.md`.
|
|
30
|
+
|
|
31
|
+
The ticket **body** is the same auditor block `report` §2 and the `prd` backlog render
|
|
32
|
+
(`src/auditor.ts`). There is one audit truth; the tracker sees exactly what the document says.
|
|
33
|
+
|
|
34
|
+
## Granularity (`--grain`)
|
|
35
|
+
|
|
36
|
+
What counts as ONE ticket. Default `criterion`.
|
|
37
|
+
|
|
38
|
+
| Grain | One ticket is | Title |
|
|
39
|
+
|---|---|---|
|
|
40
|
+
| `criterion` | a criterion, across the whole scope | `[a11y] WCAG 1.1.1 — Non-text Content` |
|
|
41
|
+
| `single` | the entire audit | `[a11y] WCAG — Accessibility audit` |
|
|
42
|
+
| `page` | a page and every non-conformity on it | `[a11y] WCAG [page:accueil] — Accessibility audit` |
|
|
43
|
+
| `page-criterion` | a criterion **on** a page | `[a11y] WCAG [page:accueil] 1.1.1 — Non-text Content` |
|
|
44
|
+
| `file` | a source file | `[a11y] WCAG [file:src/Button.tsx] — Accessibility audit` |
|
|
45
|
+
|
|
46
|
+
`WCAG` becomes the pack name under `--standard` (`[a11y] RGAA 1.1 — …`). A non-normative
|
|
47
|
+
recommendation gets the ` (recommendation)` suffix and the `recommendation` label, so it is
|
|
48
|
+
never triaged as a non-conformity.
|
|
49
|
+
|
|
50
|
+
Pick by who does the work: `criterion` suits one person fixing one rule everywhere,
|
|
51
|
+
`page`/`page-criterion` suit a team that owns pages, `file` suits a component owner,
|
|
52
|
+
`single` suits "one tracking ticket, we'll work from the report".
|
|
53
|
+
|
|
54
|
+
### The page grains keep the page honesty rules
|
|
55
|
+
|
|
56
|
+
- A page whose real rendered DOM was audited (`basis: snapshot`) can be conforming. A page
|
|
57
|
+
assembled only by **source attribution** (`basis: attributed`) cannot: its ticket carries
|
|
58
|
+
the warning that absence of a finding is not conformity. See `references/pages.md`.
|
|
59
|
+
- Findings **no page can claim** (shared components, files outside any route) get their OWN
|
|
60
|
+
ticket, `[a11y] WCAG [unattributed] — Accessibility audit`. They are never dropped and
|
|
61
|
+
never spread across pages — spreading would invent non-conformities.
|
|
62
|
+
- With **no page in scope**, `--grain page` exits `1` and tells you to capture snapshots
|
|
63
|
+
(`render --e2e`) or scan a sample (`scan --sample`). It never files zero tickets in silence.
|
|
64
|
+
|
|
65
|
+
A `file` ticket carries no conformance rate: a file is not a page.
|
|
66
|
+
|
|
67
|
+
## De-duplication — the title IS the key
|
|
68
|
+
|
|
69
|
+
There is no hidden marker, no state file, no id map. The **exact title** is the identity,
|
|
70
|
+
compared client-side. That is the only key that survives all three trackers without
|
|
71
|
+
persisting anything, and it means re-running never creates duplicates.
|
|
72
|
+
|
|
73
|
+
Two consequences worth knowing before the first run:
|
|
74
|
+
|
|
75
|
+
- **Upgrading from `--gh-issues` costs nothing.** The `criterion` and `single` titles are
|
|
76
|
+
byte-identical to the pre-v3 ones, so tickets already filed are recognised and skipped.
|
|
77
|
+
- **`--lang` forks the title** for grains carrying a criterion wording (`[a11y] WCAG 1.4.3 —
|
|
78
|
+
Contrast (Minimum)` vs `— Contraste (minimum)`). Keep `--lang` consistent across runs
|
|
79
|
+
against the same tracker. The FRAME is never localized: "Accessibility audit" stays English
|
|
80
|
+
at `--lang fr`, precisely so it cannot fork.
|
|
81
|
+
- Renaming a snapshot directory changes the page id, hence the title. Same caveat.
|
|
82
|
+
|
|
83
|
+
If the tracker cannot be listed (permissions, rate limit), the run degrades to **create**
|
|
84
|
+
rather than dropping the backlog, and says the skip count is unverified.
|
|
85
|
+
|
|
86
|
+
## Providers and transports
|
|
87
|
+
|
|
88
|
+
`--provider auto` (default) reads `ULTRA11Y_TICKET_PROVIDER`, then `.ultra11yrc.json`, then
|
|
89
|
+
the `origin` git remote. **Jira is never auto-detected** — it owns no git remote — so name it.
|
|
90
|
+
|
|
91
|
+
`--transport auto` (default) prefers the CLI, which owns its own auth so ultra11y never holds
|
|
92
|
+
a token; it falls back to REST when only a token is available, and says what BOTH were
|
|
93
|
+
missing when neither works.
|
|
94
|
+
|
|
95
|
+
**REST is not redundant with the CLI.** `gh` ships on GitHub-*hosted* runners only: container
|
|
96
|
+
jobs, self-hosted runners and every non-GitHub CI have a token and no binary. GHES works
|
|
97
|
+
through `GITHUB_API_URL`, which Actions sets for you.
|
|
98
|
+
|
|
99
|
+
| | GitHub | GitLab | Jira |
|
|
100
|
+
|---|---|---|---|
|
|
101
|
+
| CLI | `gh` | `glab` | — (REST only) |
|
|
102
|
+
| token | `GH_TOKEN` → `GITHUB_TOKEN` | `GITLAB_TOKEN` | `JIRA_EMAIL`+`JIRA_API_TOKEN` (Cloud) or `JIRA_TOKEN` (Server/DC) |
|
|
103
|
+
| target | `GITHUB_REPOSITORY` → `ULTRA11Y_GITHUB_REPO` → git remote | `CI_PROJECT_ID` → `ULTRA11Y_GITLAB_PROJECT` → git remote | `ULTRA11Y_JIRA_PROJECT` (the project KEY) |
|
|
104
|
+
| API base | `GITHUB_API_URL` | `CI_API_V4_URL` | `ULTRA11Y_JIRA_URL` |
|
|
105
|
+
|
|
106
|
+
Also for Jira: `ULTRA11Y_JIRA_ISSUE_TYPE` (default `Task`) and `ULTRA11Y_JIRA_API` (`2`|`3`).
|
|
107
|
+
|
|
108
|
+
**Credentials live in the environment only.** A `token` / `apiToken` / `password` / `secret`
|
|
109
|
+
key anywhere in the config's `tickets` block is a hard error (exit 2) — `.ultra11yrc.json` is
|
|
110
|
+
committed, so a token in it is a leaked token.
|
|
111
|
+
|
|
112
|
+
### Per-tracker specifics
|
|
113
|
+
|
|
114
|
+
- **GitHub** — the issues endpoint also returns pull requests; they are filtered out, so a PR
|
|
115
|
+
title can never suppress a real ticket. A `422` on labels retries unlabelled rather than
|
|
116
|
+
losing the ticket. Bodies are clamped to 65 536 characters with a pointer to the PRD.
|
|
117
|
+
- **GitLab** — `CI_JOB_TOKEN` **cannot create issues**; when that is the token in play, the
|
|
118
|
+
failure says so instead of returning a bare 403. Missing labels are auto-created by GitLab.
|
|
119
|
+
- **Jira** — API v3 requires ADF, not Markdown, so the body is rendered as one ADF paragraph
|
|
120
|
+
per line: the text is faithful, the formatting is not. Set `ULTRA11Y_JIRA_API=2` for the v2
|
|
121
|
+
endpoint (plain string) on Server/DC. Severity maps to priority (Highest/High/Low); a
|
|
122
|
+
project that has not configured `priority` or `labels` gets a retry without them. Every
|
|
123
|
+
ticket carries a constant `ultra11y` label so the de-dupe JQL can bound its search — the
|
|
124
|
+
JQL only narrows candidates, the exact title match still decides.
|
|
125
|
+
|
|
126
|
+
## Guards
|
|
127
|
+
|
|
128
|
+
- **`--max-tickets`** (default 200) — `page-criterion` on a large RGAA audit runs to the
|
|
129
|
+
hundreds, and a creation is hard to undo. Past the limit the command **refuses** and names
|
|
130
|
+
the escape hatch. It never truncates silently.
|
|
131
|
+
- **`--dry-run`** — resolves the transport, lists what exists, prints create-vs-skip, creates
|
|
132
|
+
nothing, exits 0. Run it first, every time.
|
|
133
|
+
- **Exit codes** — `0` filed/skipped/nothing to file · `1` at least one creation failed, the
|
|
134
|
+
provider is unusable, or `--grain page*` with no page in scope · `2` usage, config, or the
|
|
135
|
+
`--max-tickets` guard.
|
|
136
|
+
|
|
137
|
+
An unusable provider exits **1**, not 0. A push command that files nothing and reports green
|
|
138
|
+
is a silent failure.
|
|
139
|
+
|
|
140
|
+
## Configuration
|
|
141
|
+
|
|
142
|
+
```jsonc
|
|
143
|
+
// .ultra11yrc.json — flags always win
|
|
144
|
+
{ "tickets": { "provider": "gitlab", "grain": "page", "maxTickets": 100 } }
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
## Migrating from `prd --gh-issues`
|
|
148
|
+
|
|
149
|
+
| Before | Now |
|
|
150
|
+
|---|---|
|
|
151
|
+
| `prd --in a.json --gh-issues` | `tickets --in a.json --provider github --grain criterion` |
|
|
152
|
+
| `prd --in a.json --gh-single` | `tickets --in a.json --provider github --grain single` |
|
|
153
|
+
| `prd --in a.json --out d --issues-json` | `tickets --in a.json --out d --grain criterion` |
|
|
154
|
+
|
|
155
|
+
All three old flags now exit `2` and print the replacement. `prd` still writes the same Markdown;
|
|
156
|
+
its `--json` payload no longer carries a `gh` key.
|