a11y-loop 0.2.5 → 0.2.6

package/.claude-plugin/marketplace.json

@@ -6,13 +6,13 @@
   },
   "metadata": {
     "description": "Makes AI coding agents decide accessibility while the work is still being planned and write accessible UI by default, then verify it in a real browser across the states they built — and say exactly what could not be checked. Ships the a11y-loop Agent Skill, a /a11y-plan command, and a plan-mode gate that declines a UI plan with no accessibility content in it.",
-    "version": "0.2.5"
+    "version": "0.2.6"
   },
   "plugins": [
     {
       "name": "a11y-loop",
       "source": "./",
-      "version": "0.2.5",
+      "version": "0.2.6",
       "description": "Accessibility in the plan, not after the pull request. Adds §0 plan rules (conformance target, per-component criteria, the product decisions that foreclose accessibility, color tokens before components), standing WCAG 2.2 AA generation rules, and a PreToolUse hook on ExitPlanMode that declines a UI-touching plan with no accessibility content — once, handing back the section to fill in. Verification is the a11y-loop CLI: axe-core in Chromium across default, dark, forced-colors, reduced-motion and 320px passes.",
       "author": {
         "name": "Chan Meng",

package/.claude-plugin/plugin.json

@@ -2,7 +2,7 @@
   "$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
   "name": "a11y-loop",
   "displayName": "a11y-loop",
-  "version": "0.2.5",
+  "version": "0.2.6",
   "description": "Hard-gates plan mode: a plan that changes UI has to say what it decided about accessibility before it can be approved. Ships the a11y-loop skill and a /a11y-plan command alongside it.",
   "author": {
     "name": "Chan Meng",

package/README.md

@@ -49,6 +49,7 @@ check.
 - [🎥 Promo Video](#-promo-video)
 - [✨ Key Features](#-key-features)
 - [📊 Honest Coverage](#-honest-coverage)
+- [🔬 Field Notes](#-field-notes)
 - [📈 Benchmark](#-benchmark)
 - [🛠️ Tech Stack](#-tech-stack)
 - [🏗️ Architecture](#-architecture)
@@ -234,6 +235,10 @@ of the other ~87%.
 - axe-core's own `incomplete` results are surfaced as `needsReview`, not suppressed. See
   `evals/benchmark-results.md` for a live example of two such findings being investigated and
   resolved rather than dismissed.
+- **The engine is wrong sometimes, and the honest thing is to say where.** A handful of rows have
+  a known non-defect explanation — contrast in the forced-colors pass being the loudest. They are
+  written up in [Field Notes](#-field-notes), with what to check before dismissing one. Verifying a
+  row is not the same as suppressing it, and a11y-loop suppresses nothing.
 - **SARIF caveat, stated honestly:** GitHub Code Scanning only displays SARIF results that carry a
   file-path location. a11y-loop's findings are located by rendered URL + CSS selector, which
   Code Scanning drops on ingestion — a naive upload produces an empty Code Scanning view. SARIF
@@ -244,6 +249,96 @@ of the other ~87%.
   Section 508 to the 2.0/2.1 AA subset, EU EN 301 549 to 2.1 AA (moving to 2.2 AA around October
   2026), NZ and UK to the full 2.2 AA set.
 
+## 🔬 Field Notes
+
+Things a report says that are not what they look like. These came out of running a11y-loop against
+a production app across a full audit-fix cycle; each one cost an hour to work out from the report
+alone, so it is written down here. Every number below is reproducible from a checkout at axe-core
+4.12.1.
+
+### `color-contrast` in the forced-colors pass is usually not a defect
+
+axe-core reads an element's foreground from `-webkit-text-fill-color`, falling back to `color` only
+when that is unset. Chromium's forced-colors emulation forces `color` and the background, and
+leaves `-webkit-text-fill-color` at the author's value — on every element, including ones that
+never declared it. So in the forced-colors pass axe compares the author's foreground against the
+forced background.
+
+Dark-on-light text survives that comparison. Light-on-dark text collapses:
+
+| Element | Default pass | forced-colors pass |
+|---|---|---|
+| `#6d6975` text on `#ffffff` | 4.9:1, silent | silent |
+| `#e8e6ef` text on a `#141218` card | 14:1, silent | `color-contrast`, 1.23:1 — *"foreground color: #e8e6ef, background color: #ffffff"* |
+
+The tell is inside the finding: the reported foreground is the author's color while the background
+is the forced `#ffffff` or `#000000`. On a page with any dark surface this arrives as a cluster
+covering every line of text on it. On one production page it accounted for the large majority of a
+153-finding run.
+
+**Do not apply the suggestion.** It is computed from the same mismatched pair: for the row above it
+proposes darkening `#e8e6ef` to `#77757d`, which then fails in the default pass. **Do not suppress
+the rule either.** Confirm the pair in the default and dark passes, check forced colors by eye or
+with the OS setting, and say in the hand-off that forced colors was reviewed rather than measured.
+
+### `div-button` on `<body>` or a portal root is an outside-press handler
+
+Popup libraries dismiss on a press outside the popup by attaching a click handler to the document
+body or to the portal container. `div-button` looks for exactly that shape — a click handler on an
+element with no role, no `tabindex` and no keyboard path — so while a popup is open it reports
+`html > body` and the portal root.
+
+The keyboard equivalent of "press outside to dismiss" is Escape, which those libraries wire
+separately. So the question the row should send you to is `dialog-escape-does-not-close`: if
+Escape dismisses the popup, the `div-button` row on the body is a false positive of this check.
+It is a real failure only when that handler is the only way to activate something.
+
+### Contrast sampled mid-animation measures a frame, not the design
+
+An element fading in is composited at partial opacity, and axe flattens that against what is behind
+it. Measured on a production page: `#6d6975` text at opacity 0.917 was read as `#797580`. The
+difference is small, but it is enough to move a value that sits near 4.5:1 across the line, and
+enough for two runs of the same page to disagree.
+
+If a contrast finding names a color you cannot find anywhere in the source, this is usually why.
+`openPage()` waits 120 ms after load for webfonts and short entrance animations; a longer reveal
+outlives it. Re-run, have the `--interact` state wait for the animation to settle before it
+returns, or read the number off the reduced-motion pass — which is the stable one, provided the
+app honors the preference rather than ignoring it.
+
+### `keyboard-unreachable` can mean the Tab budget ran out
+
+The keyboard survey presses Tab up to `MAX_TAB_STEPS` (60) times, then reports everything
+`tabbable()` expected but the walk never reached. Past 60 tab stops in one state, the tail of the
+document is reported unreachable because the budget ended, not because focus could not get there.
+The rule's own message names the budget.
+
+A cluster of unreachable findings that is contiguous and sits at the end of document order is the
+signature. Confirm by hand with the five-minute keyboard test in
+[`references/manual-testing.md`](skill/a11y-loop/references/manual-testing.md), and audit narrower
+states so each walk fits.
+
+### Invoking the CLI through a Windows junction silently does nothing
+
+`src/cli.js` runs `main()` only when `import.meta.url === pathToFileURL(process.argv[1]).href`.
+Node resolves `import.meta.url` through a junction to the real path while `process.argv[1]` keeps
+the literal path you typed, so the guard is false, nothing runs, and the process exits **0 with no
+output** — which reads like a pass:
+
+```bash
+node D:\link-to-repo\src\cli.js --version   # prints nothing, exit 0
+node D:\github_repository\a11y-loop\src\cli.js --version   # 0.2.6
+```
+
+This bites git worktrees and any checkout reached through a junction or symlink. Use the real path,
+the installed `a11y-loop` binary, or import the entry point and call it directly — `main` is
+exported for this:
+
+```js
+import { main } from './src/cli.js';
+process.exitCode = await main(['audit', 'http://localhost:3000', '--json']);
+```
+
 ## 📈 Benchmark
 
 A small, illustrative comparison in [`evals/benchmark-results.md`](evals/benchmark-results.md):
@@ -293,7 +388,7 @@ graph TD
     C --> F["diff<br/>FIXED / NEW / REMAINING"]
     D --> G["Playwright + Chromium"]
     G --> H["axe-core"]
-    G --> I["a11y-loop's own checks<br/>focus, dialog trap, target size,<br/>reduced motion, link text, div-button"]
+    G --> I["a11y-loop's own checks<br/>focus, dialog trap (native and portalled),<br/>target size, reduced motion,<br/>link text, div-button"]
     D -->|--interact| J["Drive built states:<br/>modal open, menu expanded, form error"]
     H --> K["JSON report<br/>WCAG SC + ACT IDs + provenance"]
     I --> K

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "a11y-loop",
-  "version": "0.2.5",
+  "version": "0.2.6",
   "description": "Makes AI coding agents decide accessibility while the work is still being planned and write accessible UI by default, then proves what it can prove with a real browser audit across the states they built — and tells you exactly what it could not check.",
   "type": "module",
   "bin": {

package/skill/a11y-loop/SKILL.md

@@ -28,7 +28,7 @@ compatibility: >-
   The generation and honesty rules apply with or without the CLI; every step
   that says "audit" requires it.
 metadata:
-  "a11y-loop/version": "0.2.5"
+  "a11y-loop/version": "0.2.6"
 allowed-tools: 'Bash(a11y-loop *) Bash(npx a11y-loop *) Bash(node ${CLAUDE_SKILL_DIR}/../../src/cli.js *)'
 ---
 
@@ -274,6 +274,13 @@ ignored reduced motion, SPA focus loss, unannounced updates, positive
    Opened a modal? Added an error state? Added a route, a tab panel, an
    expanded menu, an async list? Each one is a state. Audit it.
 
+   While a modal dialog is open the keyboard survey is scoped to that dialog —
+   a native `<dialog>` opened with `showModal()`, or a portalled
+   `<div role="dialog">` that has hidden the page around it, which is what
+   every popup library ships. That is deliberate: inside an open modal, the
+   page behind it is not supposed to be reachable. Give the page behind it its
+   own state if you need it audited.
+
    ```
    A11Y audit http://localhost:5173 --interact .a11y/states.mjs --json --out .a11y/run-1.json
    ```
@@ -282,6 +289,17 @@ ignored reduced motion, SPA focus loss, unannounced updates, positive
    needsReview item is a place the engine knows it could not decide — usually
    contrast over a gradient, image, or translucent layer. Resolve it by
    reasoning about the source, not by ignoring it.
+
+   A few rows have a known non-defect explanation: a `color-contrast` cluster
+   in the forced-colors pass, a `div-button` on `<body>` while a popup is open,
+   a contrast value sampled while something was animating, a run of
+   `keyboard-unreachable` after a long Tab walk.
+   [references/manual-testing.md](references/manual-testing.md) lists each with
+   its tell. Check a row against it before changing code — fixing a false
+   positive breaks working UI, and the suggested color on a forced-colors
+   contrast row fails in the default pass. Verifying a row is not suppressing
+   it: never disable a rule, and say in your report which rows you verified and
+   how.
 4. **Fix in the source files**, never in the report or by suppressing a rule.
    For contrast findings, use the report's `suggestions[]` or
    `A11Y contrast <fg> <bg> --fix`, which returns both a lighter and a darker

package/skill/a11y-loop/references/manual-testing.md

@@ -112,6 +112,44 @@ content is any good.
   notwithstanding"), which means it stays silent wherever it is unsure. Silence
   is not a pass.
 
+### Rows with a known non-defect explanation
+
+Four of these have been seen on a production app. Check a row against them
+before changing code for it — fixing a false positive breaks working UI — and
+then say in the report which rows you verified and how. Verifying a row is not
+the same as suppressing it, and nothing here is a reason to disable a rule.
+
+- **`color-contrast` in the forced-colors pass is usually not a defect.** axe
+  reads an element's foreground from `-webkit-text-fill-color` before `color`.
+  Chromium's forced-colors emulation forces `color` and the background but
+  leaves `-webkit-text-fill-color` at the author's value, on every element,
+  so the pass compares the author's foreground against the forced background.
+  Dark-on-light text survives that; light-on-dark text reads near 1:1 and every
+  line on a dark surface is reported. The tell is a finding whose foreground is
+  a color you can find in your source and whose background is `#ffffff` or
+  `#000000`. Confirm the pair in the default and dark passes, check forced
+  colors by eye or with the OS setting, and report it as reviewed rather than
+  measured. **Never apply the suggested color** — it is derived from the same
+  mismatched pair and fails in the default pass.
+- **A click handler on `<body>` or on a portal root is how popup libraries
+  dismiss on an outside press**, and a keyboard-path check sees a clickable
+  element with no role, no `tabindex` and no keyboard handling. The keyboard
+  equivalent of "press outside" is Escape, wired separately. Check that Escape
+  closes the popup; if it does, the row on the body is explained.
+- **Contrast measured while something is animating is a measurement of one
+  frame.** An element fading in is composited at partial opacity and flattened
+  against what is behind it: at opacity 0.917, `#6d6975` was read as `#797580`.
+  That is enough to move a value near 4.5:1 across the line and enough for two
+  runs of the same page to disagree. If a finding names a color that is nowhere
+  in your source, this is usually why. Let entrance animations finish before
+  the state returns, or read the number from the reduced-motion pass if the
+  page honors the preference.
+- **A Tab walk has a budget** — 60 presses. Anything the walk did not reach
+  inside it is reported unreachable because the budget ended, not because focus
+  could not get there. A contiguous cluster at the end of document order is the
+  signature; confirm it by hand with the keyboard test below, and audit
+  narrower states so each walk fits.
+
 ---
 
 ## A five-minute keyboard test anyone can run
@@ -168,6 +206,39 @@ from the specification, which is why the APG says outright that testing with
 real assistive technology is essential and that its own examples target spec
 compliance rather than AT bug workarounds.
 
+### Automating a screen-reader pass
+
+A screen-reader pass can be scripted — [Guidepup](https://www.guidepup.dev/)
+drives NVDA on Windows and VoiceOver on macOS — and it is worth doing, because
+it turns "focus moves somewhere and something is announced" into a transcript
+you can quote and re-run. Four conditions, each of which decides whether the
+output is evidence or decoration:
+
+- **Drive a portable copy of the screen reader, not the installed one**, so a
+  run cannot inherit a person's own settings. Pin its language explicitly
+  (NVDA: `general.language`, e.g. `en`) — it otherwise follows the OS locale,
+  and a transcript in an unexpected language is unreadable as evidence.
+- **Set the synthesizer to `silence`.** The spoken text is what you are
+  capturing; the audio is not, and rendering it makes runs slow and
+  machine-dependent.
+- **A driver's relay log is focus speech, not everything spoken.** Guidepup's
+  `spokenPhraseLog()` reports what it was handed as focus moved. Live-region
+  announcements — `role="status"`, `role="alert"`, `aria-live` — do not
+  reliably appear there, so their absence proves nothing. A claim that a live
+  region spoke has to come from the screen reader's own debug log (NVDA:
+  `nvda.log` at DEBUG level), where the announcement is recorded by the screen
+  reader itself. Anything less is an assumption about the one mechanism most
+  likely to silently not fire.
+- **Copy the log out before stopping the screen reader.** `nvda.stop()` removes
+  the temporary config directory the log lives in; read it afterwards and you
+  will find nothing.
+
+What this buys you is a regression you can detect and a transcript you can
+quote. What it does not buy you is a judgment about whether the interface is
+usable — that is still answered only by people who use a screen reader every
+day. Report an automated pass as what it is: one screen reader, one version,
+one set of states.
+
 ---
 
 ## Involving disabled users