a11y-loop 0.2.3 → 0.2.5

package/.claude-plugin/marketplace.json

@@ -6,13 +6,13 @@
   },
   "metadata": {
     "description": "Makes AI coding agents decide accessibility while the work is still being planned and write accessible UI by default, then verify it in a real browser across the states they built — and say exactly what could not be checked. Ships the a11y-loop Agent Skill, a /a11y-plan command, and a plan-mode gate that declines a UI plan with no accessibility content in it.",
-    "version": "0.2.3"
+    "version": "0.2.5"
   },
   "plugins": [
     {
       "name": "a11y-loop",
       "source": "./",
-      "version": "0.2.3",
+      "version": "0.2.5",
       "description": "Accessibility in the plan, not after the pull request. Adds §0 plan rules (conformance target, per-component criteria, the product decisions that foreclose accessibility, color tokens before components), standing WCAG 2.2 AA generation rules, and a PreToolUse hook on ExitPlanMode that declines a UI-touching plan with no accessibility content — once, handing back the section to fill in. Verification is the a11y-loop CLI: axe-core in Chromium across default, dark, forced-colors, reduced-motion and 320px passes.",
       "author": {
         "name": "Chan Meng",

package/.claude-plugin/plugin.json

@@ -2,7 +2,7 @@
   "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "a11y-loop",
   "displayName": "a11y-loop",
-  "version": "0.2.3",
+  "version": "0.2.5",
   "description": "Hard-gates plan mode: a plan that changes UI has to say what it decided about accessibility before it can be approved. Ships the a11y-loop skill and a /a11y-plan command alongside it.",
   "author": {
     "name": "Chan Meng",

package/README.md

@@ -80,6 +80,13 @@ a11y-loop is two things working together:
 2. **A Node CLI** (`a11y-loop audit` / `contrast --fix` / `diff`) that verifies the result in a
    real browser, feeds failures back to the agent to fix, and re-audits until the loop converges.
 
+**About the name.** `a11y` is the standard numeronym for *accessibility*: the first letter, the last
+letter, and the 11 letters between them replaced by their count — the same construction as `i18n`
+(internationalization) and `k8s` (Kubernetes). Spelled out it reads as "ally", which is the pun the
+accessibility community has always liked, but `11` is the conventional form. `loop` is the other half
+of the method: the CLI is not a one-shot audit but a write → audit → fix → re-audit cycle that runs
+until the report converges.
+
 The skill follows the open [Agent Skills](https://agentskills.io/specification) standard, so the
 same `SKILL.md` is portable across 40+ clients — Claude Code, Cursor, GitHub Copilot, Codex,
 Gemini CLI, and more — not just one vendor's agent. The CLI runs the same checks standalone in CI
@@ -152,7 +159,9 @@ graph LR
 `2` **Checks axe-core can't run** — tab order, focus visibility (including focus-ring contrast),
 dialog focus trap / Escape / focus-return, target size (24×24 CSS px, SC 2.5.8), reduced-motion
 effectiveness, ambiguous link text, div-as-button, and positive `tabindex`. These sit alongside
-axe-core, not instead of it.
+axe-core, not instead of it. The dialog and tab-order checks understand **portalled** dialogs —
+a `<div role="dialog">` in a portal, the way Base UI, Radix, Headless UI and anything on floating-ui
+ships one — not only a native `<dialog>` opened with `showModal()`.
 
 `3` **State coverage, not just page-load** — `--interact` drives the states an agent just built
 (a modal opened, a menu expanded, a form in its error state) through the same five passes, because
@@ -263,9 +272,9 @@ illustration of the effect's shape, not a controlled study or a precise effect s
 - **SARIF conversion:** [axe-sarif-converter](https://github.com/microsoft/axe-sarif-converter)
 - **Agent integration:** the open [Agent Skills](https://agentskills.io/specification) standard —
   `SKILL.md` + `references/`, no client-proprietary format
-- **Tests:** the built-in `node --test` runner, 337+ tests across unit and integration suites,
-  including an 18-fixture seeded-violation matrix, a demo end-to-end run, and a forced-colors
-  gradient regression test
+- **Tests:** the built-in `node --test` runner, 378 tests across unit and integration suites,
+  including a 19-fixture matrix of seeded violations and correct-behaviour regression guards, a
+  demo end-to-end run, and a forced-colors gradient regression test
 
 ## 🏗️ Architecture
 
@@ -550,11 +559,17 @@ npm run test:integration  # integration tests only (drives real Chromium)
 The optional plugin layer is plain Node with no dependencies. Exercise the plan gate directly:
 
 ```bash
-echo '{"session_id":"s1","tool_name":"ExitPlanMode","tool_input":{"plan":"Build a React modal with Tailwind CSS."}}' \
-  | node hooks/plan-gate.mjs
-claude plugin validate . --strict
+echo '{"session_id":"s1","tool_name":"ExitPlanMode","tool_input":{"plan":"Add a settings page in React: a modal dialog, a members table with row actions, tabs, and a dark mode toggle styled with Tailwind CSS."}}' \
+  | node hooks/plan-gate.mjs        # expect a "deny" verdict
+claude plugin validate . --strict   # shape only — see AGENTS.md on why this is not proof
 ```
 
+Silence and exit 0 is the gate's normal answer, so an example that prints nothing proves nothing:
+the plan has to clear 40 characters and be unambiguously about UI, or you are testing the pass path
+by accident. Re-running the *same* plan returns `defer` rather than `deny` — the loop guard spends
+each plan's hash once, deliberately — so prefix with `CLAUDE_PLUGIN_DATA="$(mktemp -d)"` for a clean
+verdict. See [`AGENTS.md`](AGENTS.md) for the rest of the plugin-layer gotchas.
+
 See [`AGENTS.md`](AGENTS.md) for AI-agent-oriented project conventions, the fixture-manifest
 testing pattern, the loop discipline expected when touching `demo/` or other UI, and the rule that
 keeps `skill/` free of any Claude-Code-specific mechanism.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "a11y-loop",
-  "version": "0.2.3",
+  "version": "0.2.5",
   "description": "Makes AI coding agents decide accessibility while the work is still being planned and write accessible UI by default, then proves what it can prove with a real browser audit across the states they built — and tells you exactly what it could not check.",
   "type": "module",
   "bin": {

package/skill/a11y-loop/SKILL.md

@@ -28,7 +28,7 @@ compatibility: >-
   The generation and honesty rules apply with or without the CLI; every step
   that says "audit" requires it.
 metadata:
-  "a11y-loop/version": "0.2.3"
+  "a11y-loop/version": "0.2.5"
 allowed-tools: 'Bash(a11y-loop *) Bash(npx a11y-loop *) Bash(node ${CLAUDE_SKILL_DIR}/../../src/cli.js *)'
 ---
 

package/src/lib/browser-utils.js

@@ -133,6 +133,10 @@ export const HELPERS_SOURCE = `
     // this, every check that surveys "visible" elements would flood a report
     // with the rest of the page once a modal is open — not a real defect,
     // just this helper not knowing the platform already handled it.
+    //
+    // A PORTALLED dialog needs no equivalent clause here: it makes the rest of
+    // the page unreachable by marking it aria-hidden, which the check above
+    // already catches. modalDialogRoot() is what reads that shape.
     var modal = document.querySelector(':modal');
     if (modal && !modal.contains(el)) return false;
     return true;
@@ -190,15 +194,62 @@ export const HELPERS_SOURCE = `
     return 'rgb(255, 255, 255)';
   }
 
+  /**
+   * Has the page been taken out of reach around this element? That is how a
+   * portalled dialog declares modality when it does not set aria-modal: the
+   * library marks the dialog's body-level SIBLINGS aria-hidden (or inert) and
+   * leaves its own portal alone. Base UI sets no aria-modal attribute at all
+   * and relies on this entirely, which is the currently recommended pattern.
+   */
+  function outsideIsHidden(el) {
+    var top = el;
+    while (top.parentElement && top.parentElement !== document.body) top = top.parentElement;
+    return Array.prototype.some.call(document.body.children, function (sibling) {
+      if (sibling === top || sibling.contains(el)) return false;
+      if (!isRendered(sibling)) return false;
+      return sibling.getAttribute('aria-hidden') === 'true' || sibling.hasAttribute('inert');
+    });
+  }
+
+  /**
+   * The open modal dialog, or null. The :modal pseudo-class is the cheap
+   * answer and the right one for a native <dialog> opened with showModal() —
+   * but it is ONLY that. Every React popup library (Base UI, Radix, Headless
+   * UI, floating-ui) ships a portalled <div role="dialog"> instead, which
+   * :modal never matches, so anything relying on it alone silently treats an
+   * open sheet as though no dialog were there at all.
+   *
+   * The portalled shape is recognised by what it actually does: a visible
+   * dialog-role element that is itself exposed, and that has either declared
+   * aria-modal="true" or hidden the rest of the page around itself. The LAST
+   * such element in document order wins, because a portal appends to <body> —
+   * so the most recently opened dialog is the topmost one.
+   */
+  function modalDialogRoot() {
+    var native = document.querySelector(':modal');
+    if (native) return native;
+
+    var open = Array.prototype.filter.call(
+      document.querySelectorAll('[role="dialog"], [role="alertdialog"], [aria-modal="true"]'),
+      function (el) {
+        if (el.getAttribute('aria-modal') === 'false') return false;
+        if (!isVisible(el)) return false;
+        return el.getAttribute('aria-modal') === 'true' || outsideIsHidden(el);
+      },
+    );
+    return open.length ? open[open.length - 1] : null;
+  }
+
   /**
    * Where a forward Tab walk should be scoped: inside the open modal dialog
-   * if there is one (everything outside it is inert and cannot be reached),
-   * otherwise the whole document. tabbable() itself has no idea a native
-   * dialog element made the rest of the page inert, so callers must pass
-   * this as the container rather than always walking document.body.
+   * if there is one (everything outside it is unreachable while it is open),
+   * otherwise the whole document. tabbable() has no idea a dialog is open —
+   * it reads display and visibility, not aria-hidden, and knows nothing about
+   * :modal — so callers must pass this as the container rather than always
+   * walking document.body.
    */
   function tabbableRoot() {
-    return document.querySelector(':modal') || document.body;
+    return modalDialogRoot() || document.body;
   }
 
   /** The CSS path of whatever was last clicked, or null if nothing was. */
@@ -212,6 +263,7 @@ export const HELPERS_SOURCE = `
     accessibleName: accessibleName,
     isRendered: isRendered,
     isVisible: isVisible,
+    modalDialogRoot: modalDialogRoot,
     tabbableRoot: tabbableRoot,
     lastClickSelector: lastClickSelector,
     interactiveElements: interactiveElements,

package/src/lib/checks/dialog.js

@@ -16,6 +16,28 @@ import { makeFinding, SEVERITY } from '../finding.js';
 /** Tab presses used to probe a focus trap. */
 export const TRAP_PROBE_STEPS = 12;
 
+/**
+ * How many consecutive presses may land outside the dialog on something no user
+ * can act on before it stops counting as a wrap and starts counting as an escape.
+ *
+ * Not a fudge factor — it is the length of a real wrap, measured. A portalled
+ * dialog wraps through a CHAIN: floating-ui (Base UI, Radix) renders a focus
+ * guard beside the floating element and another pair at the edges of <body>, and
+ * hands focus back an animation frame later, so a forward Tab off the end of the
+ * sheet goes guard → body → back inside. Observed on a live Base UI sheet, three
+ * runs: two of them spent two presses out (guard, then body) and one spent a
+ * single press. Allowing exactly one press, as this check used to, therefore
+ * reported a perfectly trapped dialog as broken about two runs in three — which
+ * is what made the row look flaky rather than wrong.
+ *
+ * It stays small on purpose, and it only applies AFTER focus has been inside the
+ * dialog at least once: presses spent outside before that are the probe finding
+ * its way in from wherever the earlier surveys left focus, not a wrap. Landing on
+ * a genuine, exposed element outside the dialog is an escape on the first press
+ * either way, whatever this bound says.
+ */
+export const MAX_WRAP_TICKS = 3;
+
 /**
  * @param {object} observation
  * @param {string} observation.selector
@@ -196,6 +218,11 @@ export async function captureDialogInitialState(page, opts = {}) {
         selector: helpers.cssPath(dialog),
         html: helpers.shortHtml(dialog),
         trigger,
+        // Was this dialog MODAL at the moment it opened? Recorded now because
+        // it is the only way to tell, later, between a dialog that never
+        // trapped anything and one whose modal treatment was torn down while
+        // the probe was running. See surveyDialog.
+        modal: helpers.modalDialogRoot() === dialog,
         focusMovedIntoDialog: Boolean(active && dialog.contains(active)),
         initialFocus: active ? helpers.cssPath(active) : null,
       };
@@ -204,6 +231,26 @@ export async function captureDialogInitialState(page, opts = {}) {
   );
 }
 
+/**
+ * Let the page finish reacting to a key before reading where focus went.
+ *
+ * A focus trap built on requestAnimationFrame — floating-ui's enqueueFocus, and
+ * so Base UI's and Radix's — hands focus back one frame after a guard receives
+ * it, and cancels a pending hand-back when another arrives. Synthetic Tab
+ * presses can be dispatched faster than a frame, which cancels the redirect over
+ * and over and leaves focus parked outside the dialog: the probe outruns the
+ * page and then reports the page for not keeping up. No keyboard user can press
+ * Tab twice inside one frame, so waiting two frames does not weaken the check —
+ * it stops it measuring itself.
+ *
+ * @param {import('playwright').Page} page
+ */
+async function settleFrames(page) {
+  await page.evaluate(
+    () => new Promise((resolve) => requestAnimationFrame(() => requestAnimationFrame(resolve))),
+  );
+}
+
 /**
  * Probe focus trap, Escape, and focus return, given a dialog snapshot already
  * captured by `captureDialogInitialState`. This drives the keyboard and
@@ -217,22 +264,66 @@ export async function captureDialogInitialState(page, opts = {}) {
 export async function surveyDialog(page, found) {
   if (!found) return null;
 
+  // The probe asks exactly one question: with focus INSIDE this dialog, can Tab
+  // take it out? So it has to start inside. By the time it runs, three other
+  // surveys have each walked the page with real Tab presses and left focus
+  // wherever they happened to finish — and on a portalled dialog, focus sitting
+  // outside is itself enough for the library's focus-out handling to begin
+  // dismantling the modal treatment it put up. The probe would then walk a page
+  // that is no longer behind a modal and report a working trap as broken.
+  //
+  // Measured on a live Base UI sheet, ten runs: without this, four began outside
+  // the dialog and every one of those four reported the trap broken. With it,
+  // ten of ten began inside and none did.
+  const enteredDialog = await page.evaluate(
+    ({ selector }) => {
+      const helpers = window.__a11yLoop;
+      const dialog = Array.from(document.querySelectorAll(selector)).find((el) =>
+        helpers.isVisible(el),
+      );
+      if (!dialog) return false;
+      if (dialog.contains(document.activeElement)) return true;
+      const stops = window.tabbable ? window.tabbable.tabbable(dialog) : [];
+      if (!stops.length) return false;
+      stops[0].focus();
+      return dialog.contains(document.activeElement);
+    },
+    { selector: DIALOG_SELECTOR },
+  );
+
   // Focus trap: Tab repeatedly and watch for focus leaving the dialog.
   //
-  // Real Chromium, for a genuinely native <dialog> opened with showModal(),
-  // transiently moves document.activeElement to document.body for exactly
-  // one Tab press when wrapping past the dialog's last (or before its first)
-  // focusable descendant, then redirects back inside on the very next press —
-  // body itself stays inert throughout, so nothing is actually reachable
-  // there. Treating that single step as an escape would flag a correctly
-  // trapped native dialog as broken. It is only a real failure if landing
-  // outside the dialog is on a genuine element, or if it does not recover on
-  // the immediately following press.
-  let focusTrapped = true;
+  // Every modal dialog spends one tick outside itself when it wraps, and the
+  // shape of that tick depends on who implements the trap:
+  //
+  //  - Real Chromium, for a genuinely native <dialog> opened with showModal(),
+  //    transiently moves document.activeElement to document.body for exactly
+  //    one Tab press when wrapping past the dialog's last (or before its first)
+  //    focusable descendant, then redirects back inside on the very next press.
+  //    Body itself stays inert throughout, so nothing is reachable there.
+  //  - A portalled <div role="dialog"> — Base UI, Radix, Headless UI, anything
+  //    on floating-ui — wraps through a focus guard instead: a focusable span
+  //    outside the dialog, hidden from assistive technology, whose whole job is
+  //    to catch the wrap and hand focus back. Nothing is reachable there either.
+  //
+  // Treating either as an escape flags a correctly trapped dialog as broken,
+  // and that is precisely what happened to portalled dialogs while this gate
+  // asked `:modal` — which matches only the native case. It is a real failure
+  // if focus lands outside the dialog on a genuine, exposed element, or if it
+  // does not come back within MAX_WRAP_TICKS presses: a portalled dialog wraps
+  // through a chain of those ticks, not a single one.
+
+  // A dialog focus cannot be placed inside is a dialog whose trap this check
+  // cannot judge, so it says so with null rather than guessing — dialogFindings
+  // reports only on an explicit false.
+  let focusTrapped = enteredDialog ? true : null;
   let escapedTo = null;
-  let sawUnconfirmedTransient = false;
-  for (let i = 0; i < TRAP_PROBE_STEPS; i++) {
+  let insideAtLeastOnce = false;
+  let consecutiveWrapTicks = 0;
+  let tornDownMidProbe = false;
+  for (let i = 0; enteredDialog && i < TRAP_PROBE_STEPS; i++) {
     await page.keyboard.press('Tab');
+    await settleFrames(page);
     const check = await page.evaluate(
       ({ selector }) => {
         const helpers = window.__a11yLoop;
@@ -240,13 +331,31 @@ export async function surveyDialog(page, found) {
           helpers.isVisible(el),
         );
         const active = document.activeElement;
-        if (!dialog || !active) return { inside: false, transient: false, label: null };
-        if (dialog.contains(active)) return { inside: true, transient: false, label: null };
+        if (!dialog || !active) {
+          return { inside: false, transient: false, label: null, stillModalRoot: false };
+        }
+        if (dialog.contains(active)) {
+          return { inside: true, transient: false, label: null, stillModalRoot: true };
+        }
         const isBodyOrRoot = active === document.body || active === document.documentElement;
-        const stillModal = typeof dialog.matches === 'function' && dialog.matches(':modal');
+        // A focus guard is an EMPTY element that is not exposed to assistive
+        // technology — every library builds one the same way, as a bare
+        // aria-hidden span with nothing in it. Emptiness is what separates it
+        // from the page behind the dialog, which is also aria-hidden but is
+        // full of real content: focus landing THERE is a trap that failed, and
+        // must still be reported.
+        const isFocusGuard =
+          !isBodyOrRoot &&
+          !helpers.isVisible(active) &&
+          active.children.length === 0 &&
+          !(active.textContent || '').trim();
+        const stillModal =
+          (typeof dialog.matches === 'function' && dialog.matches(':modal')) ||
+          helpers.modalDialogRoot() === dialog;
         return {
           inside: false,
-          transient: isBodyOrRoot && stillModal,
+          transient: (isBodyOrRoot || isFocusGuard) && stillModal,
+          stillModalRoot: helpers.modalDialogRoot() === dialog,
           label: isBodyOrRoot ? 'the browser UI / document root' : helpers.cssPath(active),
         };
       },
@@ -254,18 +363,40 @@ export async function surveyDialog(page, found) {
     );
 
     if (check.inside) {
-      sawUnconfirmedTransient = false;
+      insideAtLeastOnce = true;
+      consecutiveWrapTicks = 0;
       continue;
     }
-    if (check.transient && !sawUnconfirmedTransient) {
-      sawUnconfirmedTransient = true;
+    if (check.transient && (!insideAtLeastOnce || consecutiveWrapTicks < MAX_WRAP_TICKS)) {
+      if (insideAtLeastOnce) consecutiveWrapTicks += 1;
       continue;
     }
+    // The dialog was modal when it opened and is not any more: the page took
+    // the modal treatment down — the backdrop, the aria-hidden on everything
+    // behind it — while this probe was running, usually because the sheet is
+    // closing. Focus is free to walk the page because the page let it, not
+    // because a trap failed. There is nothing to judge here, so judge nothing:
+    // reporting a trap failure off a dialog that stopped being modal mid-probe
+    // is how a correctly trapped sheet ends up accused about one run in seven.
+    if (found.modal && !check.stillModalRoot) {
+      tornDownMidProbe = true;
+      break;
+    }
     focusTrapped = false;
     escapedTo = check.label;
     break;
   }
 
+  if (tornDownMidProbe) {
+    return {
+      ...found,
+      focusTrapped: null,
+      escapedTo: null,
+      escapeClosed: null,
+      focusReturnedToTrigger: null,
+    };
+  }
+
   await page.keyboard.press('Escape');
   await page.waitForTimeout(150);
 

package/src/lib/checks/keyboard.js

@@ -155,10 +155,12 @@ export function keyboardFindings(survey, ctx) {
  * matters whenever a state setup function has clicked something, which is most of
  * the time. A focused sentinel at the start of the body fixes the origin.
  *
- * "Top of the document" is not always `document.body`: if a native `<dialog>`
- * is open, everything outside it is inert, so a sentinel inserted into `body`
- * could never be focused at all. The sentinel goes into whichever element
- * `tabbableRoot()` says is actually reachable right now.
+ * "Top of the document" is not always `document.body`: if a modal dialog is
+ * open, everything outside it is out of reach, so a sentinel inserted into
+ * `body` is either unfocusable (a native `<dialog>`) or sits in the very region
+ * the dialog's focus trap exists to keep focus out of (a portalled one — as far
+ * from the sheet as the document allows). The sentinel goes into whichever
+ * element `tabbableRoot()` says is actually reachable right now.
  *
  * @param {import('playwright').Page} page
  */