automatica11y 0.3.1 → 0.3.3

package/AGENTS.md

@@ -29,4 +29,4 @@ Both print files that live in `skills/automatica11y-runner/` in the repository a
 - Heavy dependencies (Playwright, esbuild, the rule engines) load only when a command needs them. A test checks that `--version`, `doctor`, `guide`, and `--plan` never import them.
 - Keep findings from axe-core and IBM Equal Access separate. Never add their counts together or convert one engine's scale into the other's.
 - A gap, an error, a failed target, or a result that isn't testable is a finding. It never counts as a pass. A report says "no automated violations found" only where an engine found none, and never says "accessible."
-- There are two skills. `skills/automatica11y/` is a tiny bootstrap that people copy. It sends an agent to `guide`. `skills/automatica11y-runner/` holds the full steps and ships with the tool. The runner names the version series it works with. The series is the major and minor version while the major version is 0 (`0.2`), and the major version alone from 1.0 on. Update the series named in `skills/automatica11y-runner/SKILL.md` (written like `0.2.x`) whenever the series of the version in `package.json` changes, for example from `0.2.x` to `0.3.0`. A change from `0.2.5` to `0.2.6` needs no update. A test fails if they disagree.
+- There are two skills. `skills/automatica11y/` is a tiny bootstrap that people copy. It sends an agent to `guide skill`. `skills/automatica11y-runner/` holds the full steps and ships with the tool. The runner names the version series it works with. The series is the major and minor version while the major version is 0 (`0.2`), and the major version alone from 1.0 on. Update the series named in `skills/automatica11y-runner/SKILL.md` (written like `0.2.x`) whenever the series of the version in `package.json` changes, for example from `0.2.x` to `0.3.0`. A change from `0.2.5` to `0.2.6` needs no update. A test fails if they disagree.

package/README.md

@@ -44,10 +44,11 @@ A local `.html` file is served over `http://localhost`, never `file://`. A stati
 
 ## What it checks.
 
-Three tiers run by default. Use `--tiers` to pick fewer.
+Four tiers run by default. Use `--tiers` to pick fewer.
 
 - **Rules.** axe-core and IBM Equal Access run side by side. They overlap, and each catches things the other misses. Their findings are reported separately and never added together. axe-core reports an impact (`minor` to `critical`). IBM reports its Toolkit level, a staged adoption scale where level 1 is essential, high-impact requirements. The two scales aren't comparable.
 - **Interactions.** Keyboard and focus checks for nine archetypes: button, link, dialog, menu, tabs, combobox, form-field, accordion, and tooltip. Each check runs on a fresh page.
+- **Computed checks.** automatica11y's own measurements from resolved styles in the browser, for the trigger of each archetype fixture: text contrast in rest, hover, keyboard focus, and pressed states (1.4.3), the contrast of the control's edge, fill, or icon (1.4.11), and the contrast and thickness of the focus indicator (1.4.11, and 2.4.13 at level AAA). A page that uses a gradient, an image, or transparency behind the control can't be reduced to one color, so that check reports `undetermined`, which is a gap and never a pass. These results are reported on their own and never added to the rule engines' counts.
 - **Virtual screen reader.** The announcements a simulated screen reader makes, recorded as data. The output is simulated. It isn't a real screen reader, and real ones announce things differently.
 
 Every result says what it ran, or why it didn't. A gap, a failure, or a result that can't be tested is a finding. It never counts as a pass.
@@ -59,7 +60,7 @@ Every result says what it ran, or why it didn't. A gap, a failure, or a result t
 | `--wcag 2.0\|2.1\|2.2` | `2.2` | The WCAG version. |
 | `--level A\|AA\|AAA` | `AA` | The conformance level. IBM Equal Access has no AAA rules, so it runs its AA rules and says so. |
 | `--engine axe,ibm` | both | Which rule engines run. |
-| `--tiers rules,interactions,vsr` | all | Which tiers run. |
+| `--tiers rules,interactions,computed,vsr` | all | Which tiers run. |
 | `--archetypes a,b` | all | Limit npm and Storybook targets to these archetypes. |
 | `--lib-a11y on,off` | both | For libraries with opt-in accessibility features. See [the fixture guide](skills/automatica11y-runner/references/fixtures.md). |
 | `--mapping <file>` | none | A mapping file for npm targets. |
@@ -111,7 +112,7 @@ npx automatica11y@latest guide fixtures  # how to write the fixtures an npm pack
 
 The guidance ships with the tool, so it always matches the version you run. Tell your agent to run `npx automatica11y@latest guide` and follow it, then ask for things like "How accessible is Radix Dialog?" or "Compare the accessibility of React Aria and Headless UI." The agent needs to run shell commands and read and write files. Nothing here is tied to one agent.
 
-**Skills.** If your agent loads skills from a folder, copy [`skills/automatica11y`](skills/automatica11y) into it. That's one small file, `SKILL.md`. It advertises the tool to the agent, and sends it to `guide`. It names no version, so it doesn't go stale. The full steps are the [`automatica11y-runner`](skills/automatica11y-runner) skill, which ships in the package and is what `guide skill` prints. Copy it too if you want the steps available without the network.
+**Skills.** If your agent loads skills from a folder, copy [`skills/automatica11y`](skills/automatica11y) into it. That's one small file, `SKILL.md`. It advertises the tool to the agent, and sends it to `guide skill`. It names no version, so it doesn't go stale. The full steps are the [`automatica11y-runner`](skills/automatica11y-runner) skill, which ships in the package and is what `guide skill` prints. Copy it too if you want the steps available without the network.
 
 **AGENTS.md.** [`AGENTS.md`](AGENTS.md) is for agents that read it but don't load skills. It points to the same steps, and tells contributors how to run and change the code.
 
@@ -134,4 +135,12 @@ The run exits 1 when the check trips. Needs-review items never trip it.
 
 ## License.
 
-MIT. See [LICENSE](LICENSE).
+MIT. See [LICENSE](LICENSE). The WCAG data below has its own terms.
+
+## Attribution.
+
+automatica11y reads WCAG criterion numbers, names, levels, and versions from the W3C's published JSON, [wcag.json](https://www.w3.org/WAI/WCAG22/wcag.json). The package ships that file in `src/data/` without changes.
+
+Source: [Web Content Accessibility Guidelines (WCAG) 2.2](https://www.w3.org/TR/WCAG22/), W3C. The JSON is used under the [terms in the W3C WCAG repository](https://github.com/w3c/wcag/blob/main/11ty/json/README.md): the source is credited with a link, and the content isn't changed. See also the [W3C Document License](https://www.w3.org/copyright/document-license/) and [W3C Intellectual Rights](https://www.w3.org/copyright/intellectual-rights/). The links that reports build to each criterion are added by automatica11y and aren't part of the W3C data.
+
+Every report repeats this credit in its closing section. To refresh the data, run `npm run update-wcag`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "automatica11y",
-  "version": "0.3.1",
+  "version": "0.3.3",
   "description": "Test and compare the accessibility of web pages, Storybook builds, and npm component libraries.",
   "license": "MIT",
   "type": "module",
@@ -19,6 +19,7 @@
   "scripts": {
     "test": "node --test test/*.test.js",
     "lint": "tsc -p jsconfig.json",
+    "update-wcag": "node scripts/update-wcag.js",
     "prepublishOnly": "npm run lint"
   },
   "dependencies": {

package/skills/automatica11y/SKILL.md

@@ -13,11 +13,13 @@ automatica11y tests and compares web accessibility. This skill only gets you sta
 3. Run this, read all of what it prints, and follow it:
 
    ```bash
-   npx --yes automatica11y@latest guide
+   npx --yes automatica11y@latest guide skill
    ```
 
-   If the command exits with an error or prints no usable output, tell the user it failed, include the error text, and stop. Don't continue with partial instructions.
+   If the error mentions `ETARGET` (npm says a version doesn't exist, usually because its local list is out of date), run the command once more with `--prefer-online`: `npx --yes --prefer-online automatica11y@latest guide skill`.
 
-   After reading the guide and the steps it sends you to, pass only settings they list as supported. If the user named a setting they don't list as supported, tell the user which setting is unsupported, list the supported values from the guide, and ask which to use. Don't ask again for anything they've already said.
+   If the command exits with an error or prints no usable output, including after that retry, tell the user it failed, include the error text, and stop. Don't continue with partial instructions.
+
+   After reading the steps, pass only settings they list as supported. If the user named a setting they don't list as supported, tell the user which setting is unsupported, list the supported values from the steps, and ask which to use. Don't ask again for anything they've already said.
 
 4. If you can't run shell commands, or `npx` can't reach the npm registry, tell the user and stop. Don't guess at results.

package/skills/automatica11y-runner/SKILL.md

@@ -14,6 +14,8 @@ The tool is **not** an attestation or certification tool. Automated checks cover
 
 ## 1. Check the version
 
+Do sections 1 and 2 before you run an audit. A question to the user about a missing target or an unsupported setting (section 3) can come before or after them.
+
 This skill works with automatica11y **0.3.x**. Run:
 
 ```bash
@@ -39,7 +41,7 @@ npx --yes automatica11y@latest audit <target> [options]
 npx --yes automatica11y@latest compare <target> <target> [<target>...] [options]
 ```
 
-Use `audit` for one target and `compare` for two or more. A target is `[label=]<spec>`. The label is optional and names the target in the report.
+Use `audit` for one target and `compare` for two or more, even when they're different kinds, such as a live page and an npm package. The report then opens with a warning that the evidence isn't equivalent. A target is `[label=]<spec>`. The label is optional and names the target in the report.
 
 | The user means | The spec is |
 |---|---|
@@ -57,7 +59,7 @@ Options you can set, and nothing else:
 | `--wcag 2.0\|2.1\|2.2` | `2.2` | The user names a WCAG version. |
 | `--level A\|AA\|AAA` | `AA` | The user names a level. IBM Equal Access has no AAA rules, so it runs its AA rules and says so. |
 | `--engine axe,ibm` | both | The user wants one rule engine. |
-| `--tiers rules,interactions,vsr` | all three | The user wants fewer checks. |
+| `--tiers rules,interactions,computed,vsr` | all four | The user wants fewer checks. |
 | `--archetypes a,b` | all | The user cares about some components. Choose from button, link, dialog, menu, tabs, combobox, form-field, accordion, tooltip, chart. |
 | `--lib-a11y on,off` | both | Only for libraries with opt-in accessibility features. |
 | `--mapping <file>` | none | You wrote or edited a mapping file. |
@@ -72,7 +74,7 @@ Use the target and the settings the user already gave, and ask only for what's m
 
 Pass only the options in the table above, with the values it lists. If the user names a setting the tool doesn't have, or a value outside those lists (for example, WCAG 3.0), tell them which setting is unsupported, list the supported values, and ask which to use. Don't substitute a default or invent a value.
 
-Show the user the exact command before you run it. If a target is ambiguous, ask one question, then go on.
+Show the user the exact command before you run it. If you have questions (a target that could mean two things, a missing target, an unsupported setting), ask them together in one message, then go on.
 
 Don't set the fail flags unless the user asks for gating. They change the exit code. They don't change the results.
 
@@ -82,9 +84,9 @@ A package's components can't be guessed from its name. The first run installs th
 
 1. Run the audit once. Read `<out>/mapping.json` and the report's **Archetypes** table.
 2. Treat the mapping as a guess. Check each `export` or `tag` against what the user asked about.
-3. For each archetype marked `needs-fixture` that matters to the request, write `fixtures/<target id>/<archetype>.jsx` (`.js` for web components) in the working directory. Follow the contract and the examples below. `references/fixtures.md` has more: the mapping file, states, and library accessibility options. It should sit next to this file. If you can't open it, run `npx --yes automatica11y@latest guide fixtures` to print it. The contract here is enough for the first fixture.
+3. Each archetype in `mapping.json` has a status. `needs-fixture` means the tool found the component but can't build it from a template. `no-match` means no export or custom element looks like that archetype. For each archetype marked `needs-fixture` that matters to the request, write `fixtures/<target id>/<archetype>.jsx` (`.js` for web components) in the working directory. Follow the contract and the examples below. `references/fixtures.md` has more: the mapping file, states, and library accessibility options. It should sit next to this file. If you can't open it, run `npx --yes automatica11y@latest guide fixtures` to print it. The contract here is enough for the first fixture.
 4. Run the same command again. The tool finds fixtures in that folder without `--mapping`.
-5. Don't invent fixtures for archetypes the user didn't ask about. A gap is an honest result.
+5. Don't invent fixtures for archetypes the user didn't ask about. For a `no-match` archetype, write a fixture only if the library's documentation names a component, or a documented way, to make it. Otherwise leave it as a gap. A gap is an honest result.
 
 **The fixture contract, in brief.**
 
@@ -138,7 +140,7 @@ Run the command. Note the exit code:
 
 | Code | Meaning |
 |---|---|
-| 0 | The run completed. Findings don't change this unless a fail flag was set. |
+| 0 | The run completed. Findings don't change this unless a fail flag was set. A target that failed while others ran still exits 0, and the failure is in the results. |
 | 1 | The run completed and a fail flag tripped. |
 | 2 | The command was wrong. Read the message, fix it, and run again. |
 | 3 | An environment problem. Relay the fix. |
@@ -154,14 +156,16 @@ Write the narrative from `results.json`. Never write from memory, and never repe
 2. For a comparison, say that every target used the same archetypes, WCAG version, level, and rules.
 3. Open with the coverage matrix (target by archetype by tier). Then give the findings.
 4. Keep violations, needs-review items, and passes in separate lists. Never merge them.
-5. Break findings out by archetype and by impact.
+5. Break findings out by archetype. Within an archetype, order axe-core findings by impact and IBM findings by Toolkit level.
 6. Use "no automated violations found" only for an engine that reported none for that target. For an engine that reported violations, list them as the tool reports them. Never say "accessible," "compliant," or "passes WCAG."
 7. Don't print a single score. If someone insists, pair any number with the coverage matrix and the automated-coverage caveat.
 8. Label virtual screen reader output **simulated**. Label library accessibility options **on** or **off** on every result that has one.
 9. Treat a gap, a not-testable result, an error, or a failed target as a finding. It never counts as a pass.
 10. If the comparison mixes component targets and page targets, open with a warning that the evidence isn't equivalent.
 11. Label every rule finding with its engine. Report axe-core and IBM Equal Access separately. Never add their counts together. Impact is axe-core's own label. IBM Toolkit level is IBM's staged adoption scale (1 is essential, high-impact requirements). Don't convert one into the other.
-12. End with a plain method note. Say what automated tools can't catch: whether alt text is meaningful, whether link and heading text make sense in context, cognitive load, real focus and reading order in use, and how real screen readers behave. Those need a person.
+12. Report computed checks (contrast measured from resolved styles) in their own section. Give the measured ratio next to the ratio the criterion needs, and name the state or the method. Never add them to axe-core or IBM counts. Treat `undetermined` as a gap, never as a pass. When a control has visible text and a pale edge, the result is `not-applicable`, because the text identifies the control. Say that, and leave it for a person to confirm.
+13. When you name a WCAG criterion, use the number and name exactly as `report.md` prints them, and keep its "WCAG data" credit to the W3C. Don't write criterion names or levels from memory.
+14. End with a plain method note. Say what automated tools can't catch: whether alt text is meaningful, whether link and heading text make sense in context, cognitive load, real focus and reading order in use, and how real screen readers behave. Those need a person.
 
 Use the structure of `report.md`. Quote selectors and rule IDs exactly as `results.json` has them.
 
@@ -173,8 +177,9 @@ Say these things plainly. Don't soften them, and don't fill in a result.
 - **Not applicable.** The package has no rendering surface, such as a utility library. There's nothing to test.
 - **Not testable.** The content is a canvas with no alternative, or sits in a closed shadow root. The rule engines can't see it, so the result is untested, not clean. The virtual screen reader also can't read open shadow roots.
 - **Gap.** The archetype has no usable fixture or no matching export. Say what the archetype needs.
-- **Error.** An interaction check couldn't finish. It's untested, not failed.
-- **Failed target.** The target can't be reached, isn't a web page, or couldn't be built. The tool records it as failed with a reason. Tell the user which target failed, using the reason from `results.json`. Don't retry with guesses. If every target failed (exit code 4), stop. If others ran, report them, and list the failed target as a gap in coverage.
+- **Error.** An interaction or computed check couldn't finish. It's untested, not failed.
+- **Undetermined.** A computed check found a gradient, an image, or transparency behind the control, so it can't reduce the page to one color. It's untested, not clean.
+- **Failed target.** The target can't be reached, isn't a web page, or couldn't be built. The tool records it as failed with a reason. Tell the user which target failed, using the reason from `results.json`. Don't retry with guesses. If an npm target failed with a network or install error (for example `ETARGET`), you may run the same command once more. If it fails again, report it. If every target failed (exit code 4), stop. If others ran, report them, and list the failed target as a gap in coverage.
 
 ## 8. Stay out of setup
 

package/src/commands/common.js

@@ -237,7 +237,7 @@ export async function runSavedPlan(argv, io) {
 const FLAGS = `Options:
   --wcag <2.0|2.1|2.2>        WCAG version. Default 2.2.
   --level <A|AA|AAA>          Conformance level. Default AA.
-  --tiers <list>              rules, interactions, vsr. Default all three.
+  --tiers <list>              rules, interactions, computed, vsr. Default all four.
   --engine <list>             axe, ibm. Default both.
   --archetypes <list>         Limit npm and Storybook targets to these archetypes.
   --lib-a11y <list>           on, off. Default both.

package/src/data/README.md

@@ -0,0 +1,19 @@
+# WCAG data.
+
+`wcag-2.2.json` is the W3C's published JSON serialization of WCAG 2.2, copied without changes from <https://www.w3.org/WAI/WCAG22/wcag.json>. `wcag-2.2.source.json` records where and when it was downloaded, and the SHA-256 of the file. A test fails if the file and that hash disagree.
+
+automatica11y reads criterion numbers, names, levels, and versions from this file. It doesn't edit or extend the data. The links it builds from each criterion's `id` are added by automatica11y and aren't part of the W3C data.
+
+## Attribution.
+
+Source: [Web Content Accessibility Guidelines (WCAG) 2.2](https://www.w3.org/TR/WCAG22/), W3C. The JSON is published under the [terms in the W3C WCAG repository](https://github.com/w3c/wcag/blob/main/11ty/json/README.md): attribute the original source with a link, and don't change the content. See also the [W3C Document License](https://www.w3.org/copyright/document-license/) and [W3C Intellectual Rights](https://www.w3.org/copyright/intellectual-rights/).
+
+Copyright © World Wide Web Consortium. W3C® liability, trademark and permissive document license rules apply.
+
+## Updating.
+
+```bash
+npm run update-wcag
+```
+
+The W3C publishes new versions monthly. The script downloads the file, checks that it has principles and terms, and writes it unchanged.