ultra11y 2.31.2 → 2.32.0

package/skills/ultra11y/SKILL.md

@@ -4,7 +4,7 @@ description: "Use to AUDIT existing HTML/CSS/JSX against WCAG 2.2 AA accessibili
 when_to_use: "Invoke when the user asks for an accessibility AUDIT or a formal deliverable: audit a repo, site or page against WCAG 2.2 AA or a country standard (RGAA, Section 508, EN 301 549); produce a dated conformance report, a criterion grid, a PRD remediation backlog or GitHub issues; author accessible markup from scratch; wire the repo gate (init --hook/--ci) or a rendered-DOM capture pipeline. For a review of the code UNDER CHANGE (diff, branch, PR), use the `review-a11y` skill instead."
 license: MIT
 metadata:
-  version: 2.31.2
+  version: 2.32.0
 ---
 
 # ultra11y — audit WCAG 2.2 AA and write accessible markup
@@ -38,21 +38,26 @@ adjudicates. So the engine's clean run is a starting point, never a verdict —
 > **Core rules:**
 > 1. **Never invent a non-conformity**: every `NC` cites a real, resolvable element (`check` verifies it).
 > 2. **Native HTML first, ARIA last**; never duplicate implicit semantics.
-> 3. **Residual is explicit, never silently conforming**: the AI agent *adjudicates* every
+> 3. **Never invent a conformity either**: a `C` is cited exactly like an `NC` — it names in
+>    `citations[]` the harvested evidence it cleared, each anchor resolvable and drawn from
+>    the evidence that criterion was actually shown. A criterion presented with *no* evidence
+>    cannot be `C` at all; it stays `manual`. And an adjudicated `C` is never merged with an
+>    engine-decided one: reports list it separately and keep it out of the automatic pass rate.
+> 4. **Residual is explicit, never silently conforming**: the AI agent *adjudicates* every
 >    *judgment* criterion itself (`verify --manual`, gated), and the *rendering* criteria go to
 >    `scan`; any criterion still unproven stays "to assess manually" — no status without a
 >    recorded, justified verdict.
-> 4. **The FINAL rendered semantic HTML must be correct.** The engine sees only source; a
+> 5. **The FINAL rendered semantic HTML must be correct.** The engine sees only source; a
 >    component library (DSFR/MUI…) or `.vue`/`.svelte`/`.astro` SFC hides the real markup, so
 >    a green source audit is not proof. Verify the produced semantic HTML — install the
 >    zero-touch **capture** harvester (`render --setup`) so every component your tests render
 >    is serialized to `.ultra11y/captures` and audited, with `audit --require-captures` gating
 >    the blind spots. See `references/automation.md` / `rendered.md`.
-> 5. **Language**: ALWAYS pass `--lang` matching the language of your conversation with
+> 6. **Language**: ALWAYS pass `--lang` matching the language of your conversation with
 >    the user; ask the user when ambiguous. Without the flag the CLI auto-detects (repo
 >    `<html lang>` → the active standard's default locale → English) — a scripted/CI
 >    fallback, not a substitute for passing `--lang` yourself.
-> 6. **Technical tokens stay in English, even in French prose.** In any French deliverable
+> 7. **Technical tokens stay in English, even in French prose.** In any French deliverable
 >    you write (report commentary, PRD, GitHub issues, judgment verdicts), attribute/
 >    element/role names and their values are code, not prose — never translate them:
 >    `aria-live` stays `aria-live` (never « région live »), same for `tabindex`, `alt`,
@@ -85,6 +90,7 @@ Domain knowledge first, then the tooling. Read the one that matches the question
 | `references/false-positives.md` | Where a finding may still be wrong, and how to refute it |
 | `references/rendered.md` · `references/automation.md` | Auditing produced HTML, captures, hooks and CI |
 | `references/harnesses.md` | Where this runs: Claude Code, Codex CLI, OpenCode, and the AGENTS.md fallback |
+| `references/orchestrators.md` | Driving the engine from a workflow engine / non-GitHub CI: pinning, the six-step pipeline, exit codes, `--issues-json` for any tracker |
 | `references/ci.md` | `--format sarif\|github`: inline PR annotations, code scanning, job summary |
 | `references/pages.md` | Page snapshots + the per-page criterion grid (`pages`), and its honesty rules |
 | `references/e2e.md` | `render --e2e`: auditing a page during your Playwright/Cypress run |

package/skills/ultra11y/references/orchestrators.md

@@ -0,0 +1,176 @@
+# Drive the engine from an orchestrator
+
+The skill and the GitHub Action assume a person or a coding harness is in the loop. An
+**orchestrator** — an agent workflow engine, a CI job that is not GitHub, a cron runner —
+is different: nothing reads a banner, nothing retries on a hunch, and the only things that
+travel between steps are exit codes and JSON. This page is the contract for that caller.
+
+The shape that matters: **the engine finds, the agent adjudicates, the gates refuse.** Keep
+the detection deterministic and there is no model between "a defect exists" and "a defect is
+counted" — which is the whole reason to wire the engine in rather than ask a model to audit.
+
+## Resolving the engine
+
+Pin the version. `npx` resolves `latest` otherwise, so what runs can change with no commit
+anywhere.
+
+```sh
+npm_config_update_notifier=false npx -y ultra11y@2.31.2 --version   # → 2.31.2, exit 0
+```
+
+- **`npm_config_update_notifier=false` is not optional.** Without it npm prints its
+  "New major version available" notice into the output a step is about to parse.
+- **`ULTRA11Y_BIN`** points at an engine you already have (a vendored `ultra11y.mjs`, a
+  container layer). The engine is one self-contained file with no runtime dependencies, so
+  vendoring is a supported path — that is exactly what the skills and the Action do.
+- **Node ≥ 22.18.** No other system dependency, no API key. (`judge` takes a key; nothing
+  else does. `scan` needs a browser tier — see `references/dynamic.md`.)
+- **Verify what you resolved, and fail if it is not what you asked for.** A step that lets a
+  missing engine fall through reports zero findings, which reads exactly like a clean audit.
+  This is the one failure mode worth being loud about.
+
+## The pipeline
+
+Six steps. Four are deterministic; only step 3 involves a model.
+
+```sh
+ENGINE="npx -y ultra11y@2.31.2"        # or "node $ULTRA11Y_BIN"
+RUN=/tmp/a11y-run                          # out-of-tree: see "Where output lands"
+
+# 1. FIND — the static engine. No model. This is the finding set.
+$ENGINE audit "<globs>" --graph --out "$RUN" --json --lang en > /dev/null
+#    …or, on a pull request, exactly what the branch introduced:
+$ENGINE audit --since "$BASE_REF" --graph --out "$RUN" --json --lang en > /dev/null
+
+# 2. WORKLIST — a draft report, then the criteria a static pass cannot decide.
+$ENGINE report --in "$RUN/audit-latest.json" --out "$RUN" --lang en > /dev/null
+$ENGINE verify --report "$RUN"/wcag-*.md --in "$RUN/audit-latest.json" --manual --out "$RUN" --json
+
+# 3. ADJUDICATE — hand $RUN/ADJUDICATE.todo.json to the agent; it fills every `verdict`.
+#    The file carries its own `contract` (see below) — the agent needs nothing else.
+
+# 4. FOLD — fail-closed. Exit 1 with per-criterion reasons if a verdict is unsupported.
+$ENGINE verify --apply "$RUN/adjudication.json" --in "$RUN/audit-latest.json" --out "$RUN" --json
+
+# 5. REPORT — deterministic rendering of the adjudicated audit.
+$ENGINE report --in "$RUN/audit-latest.json" --out audits --standard rgaa --lang fr
+
+# 6. GATE — integrity. Exit ≠ 0 on an invented criterion or unjustified NA.
+$ENGINE check --report audits/rgaa-*.md --in "$RUN/audit-latest.json" --json
+```
+
+Step 4 is the one to understand: it is a **deterministic gate on the model's output**. A
+verdict with no justification, an NC with no resolvable `normativeRef`, a criterion missing
+from the adjudication — each is refused by name, and the run stops. That is what keeps the
+final report defensible.
+
+## The adjudication contract
+
+`ADJUDICATE.todo.json` declares what may be written into it, so the filler never has to read
+ultra11y's source:
+
+```jsonc
+{
+  "contract": {
+    "verdicts": ["C", "NC", "NA", "manual"],
+    "manualReasons": ["needs-rendered-dom", "undecidable"],
+    "requires": {
+      "C":  "a non-empty justification AND citations[] naming the harvested evidence it cleared …",
+      "NA": "a non-empty justification",
+      "NC": "at least one groundable finding, each citing a normativeRef that resolves against the active standard",
+      "manual": "a reason ∈ {needs-rendered-dom, undecidable}"
+    }
+  },
+  "items": [ /* one per criterion the engine could not decide; fill `verdict` */ ]
+}
+```
+
+Note what `C` costs: a clearing verdict is gated exactly like an accusing one. It must
+name in `citations[]` the evidence it cleared, each anchor resolvable and drawn from that
+criterion's own harvested evidence — and a criterion presented with **no** evidence cannot
+be `C` at all. This is what stops "C everywhere with a plausible sentence" from passing.
+
+The header is **advisory**. `verify --apply` validates against its own constants, so editing
+the contract in the file widens nothing.
+
+Verdicts are matched **case-insensitively** (`na` is `NA`, `MANUAL` is `manual`) — a spelling
+accident is not a disagreement about accessibility. A verdict outside the vocabulary is still
+refused, and the message names the four that are not.
+
+`manual` is the honest verdict for anything that genuinely needs a rendered page. It keeps
+the criterion in `residualRisks`, where the report declares it as a residual risk. **Never
+map an undecided criterion to `C`** — that is the one way to turn this pipeline back into
+the thing it replaces.
+
+## Filing findings on a tracker that is not GitHub
+
+`--gh-issues` shells out to the `gh` CLI. For any other board, take the same items as JSON:
+
+```sh
+$ENGINE prd --in "$RUN/audit-latest.json" --out "$RUN" --issues-json --lang en
+# → $RUN/issues-<date>.json
+```
+
+```jsonc
+{
+  "tool": "ultra11y", "kind": "issues", "schemaVersion": 1,
+  "standard": "wcag", "date": "2026-08-12", "count": 4,
+  "issues": [{
+    "criteriaId": "1.1.1",
+    "title": "[a11y] WCAG 1.1.1 — Non-text Content",   // the de-dupe grain: stable across runs
+    "body": "…",                                       // the auditor block, rendered
+    "labels": ["accessibility", "wcag", "bloquant"],
+    "severity": "bloquant",
+    "advisory": false,                                 // true ⇒ a good practice, NOT a non-conformity
+    "occurrences": [{ "file": "src/Page.tsx", "line": 4, "selector": "img", "message": "…" }]
+  }]
+}
+```
+
+One issue per criterion, not per occurrence — a missing `alt` on forty images is one item
+with forty `occurrences`, not forty tickets. De-dupe on `title` and re-running files nothing
+twice. `advisory: true` marks a non-normative recommendation: it must not be triaged as a
+non-conformity, and it never enters the conformance rate.
+
+## Where output lands
+
+`--out <dir>` fully contains what a command writes: pointed outside the checkout, an audit
+leaves the target repository byte-identical. That matters when the orchestrator audits a
+repo it does not own — write the run's intermediates out-of-tree and let only the final
+dated report land in the project, if at all.
+
+`.ultra11y/pages/` is written only by the page-snapshot commands (`scan`, `snapshot`,
+`dev`); `--root` relocates it. `audit`, `report`, `prd`, `verify` and `check` never touch it.
+
+## Exit codes
+
+| | 0 | 1 | 2 |
+|---|---|---|---|
+| `audit` | reported (no `--fail-on`), or clean | findings ≥ `--fail-on` | usage / bad input |
+| `verify --apply` | folded | the adjudication was refused (see `issues[]`) | usage / bad input |
+| `check` | report is sound | integrity failure | report not found / bad input |
+| `prd` | written | `--gh-*` had failures | usage / bad input |
+
+Read exit codes from the process, **not** from a pipeline — `cmd --json | head` reports
+`head`'s status, which is how a failing gate reads as a pass.
+
+## What the JSON tells you
+
+- `audit --json` → the `AuditResult` (also at `<out>/audit-latest.json`): `findings[]` with
+  stable `findingId`, `criteriaId`, `severity`, `file`, `line`; `residualRisks[]`;
+  `conformancePct` — the **automatic static pass rate**, not a conformance rate.
+- `verify --apply --json` → `{ ok, auditPath, applied, stillManual, conformancePct, findings, grounding }`
+  on success; `{ ok: false, issues[], audit }` when refused.
+- `check --json` → `{ ok, issues[] }`.
+
+## Honesty rules an orchestrator has to carry
+
+The engine is explicit about what it did not decide; a wrapper that flattens that away
+produces exactly the false clean bill of health this design exists to prevent.
+
+1. **A missing engine is a failure, never an empty finding set.**
+2. **Report the residual criteria.** The rendering criteria (contrast, focus, zoom, reflow)
+   are undecided until `scan` runs. Say so; do not let a step summary imply full coverage.
+3. **`conformancePct` is the static pass rate.** Label it as such.
+4. **Never let a model write the counts.** Take them from the JSON — the numbers exist
+   precisely so nothing has to be re-derived by a model that may drop a field.

package/skills/ultra11y/references/pages.md

@@ -205,9 +205,13 @@ Two things it deliberately does NOT do:
   page sheet, the compliance report, the PRD and the GitHub issue are the same block — and
   the occurrence lines stay parseable by `verify`.
 
-The screenshot is *referenced*, never copied: `.ultra11y/pages/<id>/screen.png` is already on
-disk, so the path is resolved relative to wherever the file lands. A page with no screenshot
-says so, rather than showing nothing.
+The screenshot is **copied next to the report** when you pass `--out`: each page's
+`.ultra11y/pages/<id>/screen.png` lands in `<out>/assets/<id>.png` and is linked relatively
+from there. That is what makes the output directory self-contained — CI uploads `audits/`
+alone as the artefact, and a `../../.ultra11y/…` reference would be a broken image in every
+sheet the reviewer opens. Without `--out` (streaming to stdout) the path stays relative,
+since there is no directory to keep intact. A page with no screenshot says so, rather than
+showing nothing.
 
 Sheets are named `page-<id>.md`: a page whose id is `index` — the ordinary id for an
 `index.html` target — would otherwise be written over the index itself.