ultra11y 5.3.3 → 5.4.0

package/skills/ultra11y/SKILL.md

@@ -4,7 +4,7 @@ description: "Use to AUDIT existing HTML/CSS/JSX against WCAG 2.2 AA accessibili
 when_to_use: "Invoke when the user asks for an accessibility AUDIT or a formal deliverable: audit a repo, site or page against WCAG 2.2 AA or a country standard (RGAA, Section 508, EN 301 549); produce a dated conformance report, a criterion grid, a PRD backlog, or tickets (GitHub/GitLab/Jira); author accessible markup from scratch; wire the repo gate (init --hook/--ci) or a rendered-DOM capture pipeline. For a review of the code UNDER CHANGE (diff, branch, PR), use the `review-a11y` skill instead."
 license: MIT
 metadata:
-  version: 5.3.3
+  version: 5.4.0
 ---
 
 # ultra11y — audit WCAG 2.2 AA and write accessible markup
@@ -54,7 +54,7 @@ adjudicates. So the engine's clean run is a starting point, never a verdict —
 >    decides what its terms mean. An auditor block that cites `8.3.1` after reading it is
 >    grounded; one that cites it from memory is a guess wearing a reference. Before auditing
 >    against a country standard, run the plan (`ultra11y_method`): it tells you how many of its
->    criteria no tool will decide for you — for RGAA, **58 of 106**. See `references/mcp.md`.
+>    criteria no tool will decide for you — for RGAA, **59 of 106**. See `references/mcp.md`.
 > 6. **The FINAL rendered semantic HTML must be correct.** The engine sees only source; a
 >    component library (DSFR/MUI…) or `.vue`/`.svelte`/`.astro` SFC hides the real markup, so
 >    a green source audit is not proof. Verify the produced semantic HTML — install the

package/skills/ultra11y/references/ci.md

@@ -62,12 +62,54 @@ run and an RGAA run coexist rather than overwriting each other. The Markdown rep
 through `check` before being uploaded, so CI cannot publish a report citing an invented
 criterion.
 
-## Adjudicating the judgment criteria in CI (`adjudicate`)
+## The verdict ledger (`ledger`) — a complete grid in CI, with no model
 
 Of the 55 WCAG 2.2 AA criteria the engine decides a handful; **38 are judgment calls**, and
-under RGAA **81 of 106** can only ever derive `manual`. Inside a coding agent the agent rules
-on them. In CI nobody does, so they stay « à évaluer » and the published conformance rate is
-partial by construction. `adjudicate` closes that — opt-in, in two modes.
+under RGAA **59 of 106** can only ever derive `manual`. Inside a coding agent the agent rules on
+them. In CI nobody does — so before the ledger, a CI grid was partial by construction and the
+only way to close it was to pay for an agent pass on every run.
+
+A **ledger** is a file of verdicts committed to the audited repository
+(`.ultra11y/verdicts/<standard>.json`), each carrying its justification, its citations, and a
+FINGERPRINT of the evidence it was ruled against:
+
+```yaml
+- uses: maxgfr/ultra11y@v5
+  with:
+    standard: rgaa
+    ledger: 'true'          # or an explicit path
+```
+
+Replaying it is **a re-verification, not a cache hit**. Every stored verdict is rebuilt into an
+ordinary adjudication over the evidence re-derived from the audit in front of it, and folded
+through the same `applyAdjudication` — same coverage checks in both directions, same citation
+matching against the criterion's own anchors, same content-level re-grounding. A verdict nobody
+can prove is refused in CI exactly as it would be in a session.
+
+Three properties are worth knowing:
+
+- **A stale verdict is dropped, never carried.** When the code under a criterion changes, the
+  evidence changes with it, the fingerprint stops matching, and the criterion returns to « à
+  évaluer » saying so — with the date it was recorded and both evidence counts. A verdict that
+  silently outlives the code it described is the one thing a conformance deliverable cannot
+  afford. The log names every stale and every uncovered criterion, so a refresh pass knows
+  exactly what to re-adjudicate.
+- **A reformatting is not a change.** The fingerprint ignores line numbers and the replay
+  re-anchors stored citations onto today's lines, so adding a comment at the top of a file does
+  not invalidate every verdict in it.
+- **A missing or refused ledger never fails the job.** The criteria stay « à évaluer », which is
+  where they would have been without one.
+
+To FILL it, run an adjudication with `ledger` set: the accepted verdicts are recorded on the way
+out. Only accepted ones — a refused verdict in the ledger would be laundered back in on the next
+replay. A defensible split is a weekly (or `workflow_dispatch`) job that adjudicates and commits
+the refreshed ledger, and a pull-request job that only replays it.
+
+## Adjudicating the judgment criteria in CI (`adjudicate`)
+
+`adjudicate` is what FILLS the ledger — opt-in, in two modes. It runs **after** the replay, so a
+paid pass only ever covers what the ledger did not: on a repository whose ledger is current, it
+has nothing left to rule on and costs nothing.
 
 ```yaml
 env:

package/skills/ultra11y/references/judgment.md

@@ -71,6 +71,31 @@ test).
    the **produced** HTML (see `render` / audit the build), otherwise the verdict is a false
    negative.
 
+## When the adjudicator has no shell (CI)
+
+`verify --manual` writes the worklist **twice**, plus one brief per criterion:
+
+| file | for |
+|---|---|
+| `ADJUDICATE.todo.json` · `ADJUDICATE.md` | a session **with a shell**: evidence inline, edit in place, fold yourself |
+| `ADJUDICATE.verdicts.json` | the **only file to write** when you have no shell — verdicts, no evidence |
+| `adjudicate/<criteriaId>.md` | one small brief per criterion: its evidence, its protocol, its citable tests |
+
+Why the split exists, measured rather than assumed: under RGAA the worklist is 96 criteria
+carrying 1590 harvested anchors — **536 KB** of JSON and **466 KB** of Markdown. An agent given
+`Read/Grep/Glob/Edit/Write` cannot work with that: reading either document swamps its context,
+and filling 96 verdicts inside half a megabyte is 96 exact-match edits. A real CI run spent 75
+of 424 turns, hit 17 permission denials trying to run the commands the runbook prescribes, and
+returned the file untouched — so the fail-closed fold discarded all 96 verdicts and every
+criterion stayed « to assess », in a job that reported success. The verdicts file is **37 KB**
+for the same worklist, and each brief is a few KB.
+
+`verify --apply` accepts either file. Given the verdicts-only one it **re-derives the evidence
+from the audit** (the worklist is a pure function of the audit) and then runs the identical
+fold: same coverage checks in both directions, same citation matching against *that criterion's*
+own anchors, same refusals. The smaller surface buys nothing from the gate — a `C` that cites
+an anchor it was never shown is refused exactly as before.
+
 ## Adversarial verification of the non-conformities — `verify --report`
 
 Unchanged: `node scripts/ultra11y.mjs verify --report audits/wcag-YYYY-MM-DD.md` writes `VERIFY.md`
@@ -92,7 +117,7 @@ Then `node scripts/ultra11y.mjs verify --apply VERIFY.todo.json` is green again
 ## Under a country standard, the worklist speaks that standard
 
 `verify --manual --standard rgaa` keys the worklist by **RGAA criteria**, not WCAG success
-criteria — which is the granularity that matters, since 99 of RGAA's 106 criteria can only
+criteria — which is the granularity that matters, since 59 of RGAA's 106 criteria can only
 ever derive `manual`. Each item carries, inline:
 
 - the criterion's **numbered tests, in full** (`11.2.1` … `11.2.6`) — what actually has to be
@@ -138,15 +163,27 @@ gate refuses:
 
 | It returns | The gate does |
 |---|---|
-| `C`/`NA` with no justification | rejects the whole adjudication |
-| `NC` citing nothing, or a `file:line` that does not resolve against real source | rejects |
-| `NC` citing a `normativeRef` belonging to another criterion | rejects |
-| `manual` with no reason | rejects |
+| `C`/`NA` with no justification | refuses THAT criterion |
+| `NC` citing nothing, or a `file:line` that does not resolve against real source | refuses it |
+| `NC` citing a `normativeRef` belonging to another criterion | refuses it |
+| `manual` with no reason | refuses it |
 | a verdict for a criterion nobody asked about | dropped before it reaches the gate |
-| fewer verdicts than criteria (a truncated `--max` run, a failed batch) | rejects on coverage |
+| fewer verdicts than criteria (a bounded `--max` run, a failed batch) | the missing ones stay « à évaluer »; the rest lands |
 | all `manual` with reasons | **accepts** — that is a correct answer, not a failure |
 
-A rejected adjudication leaves `audit-latest.json` untouched.
+**The fold is fail-closed per VERDICT, not per FILE.** No refused verdict is ever applied, and a
+refusal costs its own criterion and nothing else: that criterion stays « à évaluer » carrying the
+refusal as its reason, and every verdict that proved itself lands. This is not a loosening of the
+gate — every check above is unchanged — it is a change of blast radius, and it was measured:
+a CI run filled 95 of 96 verdicts correctly, one came back `null`, and the old file-level fold
+discarded all 96, so a $16 adjudication published « à évaluer » across the whole grid in a job
+that reported success.
+
+`--strict` restores the all-or-nothing fold for a caller who genuinely wants it (a deliverable
+signed off in one pass); there, a rejected adjudication leaves `audit-latest.json` untouched.
+
+Whatever lands can be RECORDED with `--ledger <path>` and replayed on a later run with no model
+in the loop — see `references/ci.md`.
 
 **Strictly opt-in.** With no key the command explains itself and exits 2; nothing else in the
 engine changes.
@@ -159,8 +196,9 @@ while an agent can open the cited files, which is what *link purpose in context*
 Both skip themselves when `ANTHROPIC_API_KEY` is absent from the job environment, and both
 absorb their own failure rather than taking the audit down with them. See `references/ci.md`.
 
-`--max` bounds the spend and says out loud which criteria it did not submit. It is **refused
-together with `--apply`**, before any call is made: the gate rejects an incomplete
-adjudication by design, so the pair could only ever bill you for a guaranteed failure. Drop
-`--max` to adjudicate everything, or drop `--apply` to produce the worklist now and fold it
-in later.
+`--max` bounds the spend and says out loud which criteria it did not submit. It now composes
+with `--apply`: the fold is per-verdict, so a bounded run lands what it covered and the rest stay
+« à évaluer » — which is exactly what bounding spend is for. (It used to be refused outright, and
+rightly so while the fold was all-or-nothing: the pair could only ever bill a full run to
+guarantee a coverage failure. That refusal survives under `--apply --strict`, where it is still
+true.)

package/skills/ultra11y/references/pages.md

@@ -292,10 +292,19 @@ criterion means one definite failure fired *somewhere* — not that the engine c
 criterion. On a page where the failure did not fire, the engine knows "no definite failure
 here", which is not "conforming": alt relevance, link purpose and reading order are still
 nobody's verdict. So only the `static` criteria (the ones with an applicability predicate and
-rules, `src/audit.ts` `APPLICABLE`) earn `C` by silence — under RGAA that is 4.10 and
-8.3/8.4/8.5/8.6. Everything else stays « à évaluer » until `scan` or an adjudication decides
-it. Before this rule a page with no images scored 100% on « chaque image a-t-elle une
-alternative pertinente ? », a rate computed over criteria nobody had assessed.
+rules, `src/audit.ts` `APPLICABLE`) earn `C` by silence — under RGAA that is **8.3 and 8.5, and
+those two only**. 4.10, 8.4 and 8.6 look like they should qualify and do not: each is flagged
+`judgment: true`, because the criterion asks more than its mapped success criterion does (8.6
+asks whether the page title is *pertinent*; 2.4.2 only that a title exists), so `judgmentGuard`
+turns their `C` into `manual` on purpose. Everything else stays « à évaluer » until `scan`, the
+snapshot tier or an adjudication decides it. Before this rule a page with no images scored 100%
+on « chaque image a-t-elle une alternative pertinente ? », a rate computed over criteria nobody
+had assessed.
+
+What a criterion CAN be decided as, without any of that, is a separate question: since the
+engine learned to prove applicability (`src/audit.ts` `SUBJECT_MATTER`), a criterion whose
+subject matter is absent from the whole scope closes as a justified `NA` — so a repository with
+no media reports RGAA theme 4 as not applicable rather than as « à évaluer ».
 
 A non-normative recommendation never flips a page criterion to `NC`, exactly as in core.