@josueavalosjim/taste-check 0.4.0 → 0.5.0

package/README.md

@@ -1,22 +1,28 @@
 # taste-check
 
-Two deterministic checks for a design system you already have. It computes WCAG
-contrast ratios from your own custom properties, and flags class names and
-literal values that are not on your own approved list.
+Design review in CI, with a line down the middle of it.
 
-It has no opinion about which colours you use or which classes are allowed.
-You supply both.
+Some of design review is measurable. Contrast ratios, values that should have
+been tokens, class names nobody approved. taste-check measures those and fails
+your build on them, either from your token file or from a page it renders in a
+real browser.
 
-```bash
-npx @josueavalosjim/taste-check --config tastecheck.config.json
-```
+The rest is judgment, and it cannot be measured. For that it sends your
+screenshots and your checklist to a model, and reports what comes back as an
+opinion rather than a result.
 
-Installed as a dependency, the command is just `taste-check`:
+Keeping those two apart is the whole design. A tool that blurs them either
+blocks your pipeline on a coin flip or quietly downgrades a real failure to a
+suggestion.
 
 ```bash
 npm i -D @josueavalosjim/taste-check
 ```
 
+```bash
+npx @josueavalosjim/taste-check
+```
+
 ```
 contrast ok, 10 pairs across 2 themes
     ok  17.76:1  needs 4.5  light  --text-strong on --surface  body text
@@ -28,20 +34,37 @@ treatments FAILED
   FAIL  src/Promo.jsx:8 inline value "#ff0055" on <a> is a one-off. Use a token, or add it to approvedValues.
 ```
 
+```bash
+npx @josueavalosjim/taste-check judge
+```
+
+```
+judge ok, 5 lines against 2 screenshots, 1 to read
+  NOTE  fail   Nothing important is cut off at the edge: the third card runs
+                past the right edge of the frame at roughly x=798 of 800.
+```
+
 Zero runtime dependencies. Node 22 or newer.
 
 ## Why this exists
 
-Most quality tooling checks compliance and has no point of view. axe tells you
-an element fails 4.5:1. It cannot tell you that your borders answer to 3:1
-while your captions answer to 4.5:1, because that distinction is yours, not the
-spec's.
+Linters have a gate and no taste. They enforce what someone could write down
+as a rule, which is why they can tell you an element fails 4.5:1 but not that
+your borders answer to 3:1 while your captions answer to 4.5:1. That
+distinction is yours, not the spec's.
 
-Design systems drift in a specific way: a value gets hardcoded because the
-token did not quite fit, a class gets invented because nobody knew the approved
-one existed. Written rules do not stop it. A rule in a stylesheet comment is
-enforced by people re-reading stylesheets, and nobody re-reads a stylesheet
-while writing markup. So the rules get a check that runs instead.
+The newer AI design reviewers have taste and no gate. They will tell you what
+is wrong with a screen, in prose, in a chat window, and some of them arrive
+with a few hundred opinions already loaded about what good looks like.
+
+Both halves are useful and they need to stay separable. Run the same model
+twice over the same unchanged screen and you can get two different answers,
+which is fine for advice and disqualifying for a build gate. So the measured
+half here gates, the judged half does not, and the tool will not let you
+confuse one for the other by accident.
+
+It also ships no design rules of its own. Not a palette, not a class list, not
+a contrast floor, not a checklist. You supply all of it.
 
 ## What counts as a failure
 
@@ -58,6 +81,12 @@ Each of these exits 1 rather than passing quietly:
 A check that cannot fail is worse than no check, because it goes green and gets
 quoted as evidence.
 
+The judge is held to the same rule from the other side. Its verdicts never
+affect the exit code by default, but a judge that could not run does: no
+screenshots, a command that died, a reply that was not JSON or that skipped a
+checklist line. "Did not run" and "found nothing" must not print the same
+thing.
+
 ## The contrast check
 
 Give it your custom properties, describe your themes, and list what must clear
@@ -152,6 +181,60 @@ direction to be wrong in.
 Template literal holes are read into rather than blanked, so a class written
 inside `` `card ${on ? 'card--on' : ''}` `` is seen.
 
+## Measuring the rendered page
+
+The contrast check above reads your token file, which tells you what a colour
+is declared to be. `taste-check runtime` opens the page and reads what is
+actually painted.
+
+```json
+{
+  "runtime": {
+    "url": "http://localhost:3000",
+    "states": [
+      { "name": "light" },
+      { "name": "dark", "before": "localStorage.setItem('theme', 'dark')" }
+    ],
+    "targets": [
+      { "selector": ".caption", "prop": "color", "min": 4.5 },
+      { "selector": ".panel", "prop": "borderTopColor", "min": 3.0 },
+      { "selector": ".row.is-selected", "prop": "backgroundColor", "min": 1.25, "againstParent": true }
+    ]
+  }
+}
+```
+
+```bash
+taste-check runtime
+```
+
+It launches a headless Chromium, or connects to one you already have if you
+give it an `endpoint`, and it never closes a browser it did not start. There is
+no dependency for this. The Chrome DevTools Protocol is JSON over the WebSocket
+Node already ships, and the part of it needed here is two methods.
+
+Two things this does that reading tokens cannot.
+
+**It composites the whole background stack.** White text on a 75% black scrim
+over a near-white page measures 10.57:1. Stop at the first opaque ancestor, the
+way the check this was ported from did, and you measure the white against the
+page instead and get 1.04:1. That is a failure the design has not earned, and
+with the colours the other way round the same shortcut hands you a pass it has
+not earned either.
+
+**It can measure a state.** `before` runs on every navigation ahead of the
+page's own scripts, which is where a theme belongs because the page reads it at
+boot. `after` runs once there is a document, for opening a panel or focusing a
+field. Each state's setup is removed before the next one, so they cannot leak.
+
+Use `againstParent` when the thing being measured is a fill rather than a
+foreground. A selected row with its own background measured against itself
+scores 1.00, which is a pass that means nothing.
+
+A selector that matches nothing is a failure. So is a border colour on an edge
+with no width: you asked for the contrast of something that is not being drawn,
+and the question is wrong rather than the answer being zero.
+
 ## The judge
 
 Everything above measures. This asks the question a measurement cannot: not
@@ -238,9 +321,15 @@ from anywhere.
 | `judge.shotCommand` | Optional command run first to produce those screenshots. |
 | `judge.command` | The model command. Prompt on stdin, image paths as arguments. |
 | `judge.failOn` | `"never"` (default) or `"fail"`. Whether a verdict blocks. |
+| `runtime.url` | The page to measure. A `file://` URL works. |
+| `runtime.endpoint` | An existing CDP endpoint. Given one, taste-check connects rather than launching, and never closes a browser it did not start. |
+| `runtime.browserPath` | Path to a Chromium. Falls back to `CHROME_PATH`, then the usual locations. |
+| `runtime.states[]` | `before` runs ahead of the page's scripts, `after` once loaded, `waitFor` is a selector. |
+| `runtime.targets[]` | `selector`, `prop`, `min`, and `againstParent` when measuring a fill. |
 
 ```
-taste-check [options]         Run the deterministic checks
+taste-check [options]         Run the deterministic checks over your files
+taste-check runtime [options] Measure contrast on a rendered page
 taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
 
   -c, --config <path>   Config file (default: tastecheck.config.json)
@@ -252,6 +341,33 @@ taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
 Exit code is 1 if any check fails, 0 if every check ran and passed. The judge
 plays by the rules in its own section above.
 
+## Where it sits next to other tools
+
+This is a small tool with a narrow claim, and several of these are better than
+it at the thing they do. Reach for them.
+
+**axe, pa11y.** They run against a real rendered page and catch far more than
+contrast. taste-check checks pairs you declare in a config, before a page
+exists and without a browser. Use both. If you only run one accessibility
+tool, run axe.
+
+**stylelint, eslint.** General code quality, with an enormous rule ecosystem.
+Nothing here replaces them.
+
+**@lapidist/design-lint.** More thorough than taste-check on the token and
+component side: it parses properly rather than scanning, knows about
+frameworks, autofixes, and manages deprecations. If enforcing tokens in code is
+the whole of your problem, it is the better fit.
+
+**Checklist Design and similar agent skills.** They arrive with a hundred or
+more published checklists and review conversationally. If you want good
+opinions supplied, take theirs. taste-check supplies none on purpose and runs
+in CI with an exit code instead.
+
+What is left, and the reason this exists: nothing above draws a line between
+the part that can gate a build and the part that cannot. The linters have no
+judgment, the judges have no gate.
+
 ## What this does not do
 
 Read this before trusting a green run.
@@ -287,13 +403,16 @@ invisible to it.
 
 Not built. Written down so the shape is clear.
 
-**A runtime mode**, closing the gap named above by measuring `getComputedStyle`
-in a real browser, as an optional peer dependency so the core stays free of one.
-
 **YAML configs**, once there is a reason to take on a parser.
 
+**SARIF output**, so findings land in a code scanning tab rather than only in
+a log.
+
+**A way to run the judge from an agent skill**, not only from a shell.
+
 **`lab()` and `lch()`**, which need the D50 white point and a chromatic
-adaptation step that `oklch()` does not. Worth doing the same way: derive it,
+adaptation step that `oklch()` does not. Completeness rather than reach, so
+it sits behind the others. Worth doing the same way when it happens: derive it,
 then check every case against a browser rather than trusting the matrices.
 
 ## Development

package/bin/taste-check.mjs

@@ -8,12 +8,13 @@
  * not a quiet skip.
  */
 import { load } from '../src/config.mjs';
-import { judge, run } from '../src/index.mjs';
+import { judge, run, runtime } from '../src/index.mjs';
 import { failed, toJson, toText } from '../src/report.mjs';
 
 const USAGE = `taste-check
 
-  taste-check [options]         Run the deterministic checks
+  taste-check [options]         Run the deterministic checks over your files
+  taste-check runtime [options] Measure contrast on a rendered page
   taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
 
 Options:
@@ -25,6 +26,11 @@ Options:
 
 Exit code is 1 if any check fails, 0 if every check ran and passed.
 
+runtime is a separate command because it needs a browser and a server that
+is already up. It measures what is actually painted, compositing every
+background layer behind an element rather than stopping at the first opaque
+one, and it can put the page into a state first.
+
 The judge is a separate command because it runs a model, and a model's
 verdict is not reproducible. Its verdicts print as notes and do not affect
 the exit code unless judge.failOn is set to "fail". Whether the judge ran
@@ -35,8 +41,8 @@ function parseArgs(argv) {
   const options = { config: 'tastecheck.config.json', only: null, json: false, command: 'check' };
   // One positional, and only in first position, so a stray argument is an
   // error rather than something silently ignored.
-  if (argv[0] === 'judge') {
-    options.command = 'judge';
+  if (argv[0] === 'judge' || argv[0] === 'runtime') {
+    options.command = argv[0];
     argv = argv.slice(1);
   }
   for (let i = 0; i < argv.length; i += 1) {
@@ -95,18 +101,18 @@ if (!loaded.ok) {
   process.exit(1);
 }
 
-if (options.command === 'judge' && options.only) {
-  die('--only applies to the deterministic checks, not to judge');
+if (options.command !== 'check' && options.only) {
+  die(`--only applies to the deterministic checks, not to ${options.command}`);
 }
 
-if (options.command === 'judge' && !loaded.config.judge) {
-  die(`${options.config} defines no "judge" block.`);
+if (options.command !== 'check' && !loaded.config[options.command]) {
+  die(`${options.config} defines no "${options.command}" block.`);
 }
 
-const results =
-  options.command === 'judge'
-    ? judge(loaded.config, loaded.dir)
-    : run(loaded.config, loaded.dir, { only: options.only });
+let results;
+if (options.command === 'judge') results = judge(loaded.config, loaded.dir);
+else if (options.command === 'runtime') results = await runtime(loaded.config, loaded.dir);
+else results = run(loaded.config, loaded.dir, { only: options.only });
 
 if (!results.length) {
   die(`nothing to run. ${options.config} defines no ${options.only ?? 'contrast or treatments'} check.`);

package/package.json

@@ -1,11 +1,12 @@
 {
   "name": "@josueavalosjim/taste-check",
-  "version": "0.4.0",
-  "description": "Deterministic design-system checks: WCAG contrast over your own tokens, and a one-off value linter over your own approved list. No palette, no class list, no opinions shipped.",
+  "version": "0.5.0",
+  "description": "Design review in CI with a line down the middle: measured checks that gate the build (WCAG contrast from your tokens or from a real rendered page, one-off values in your markup) and a fresh-eyes model judge whose verdicts stay advisory. Zero dependencies. Ships no design rules of its own.",
   "keywords": [
     "accessibility",
     "cli",
     "contrast",
+    "design-review",
     "design-system",
     "design-tokens",
     "lint",

package/schema/config.schema.json

@@ -17,6 +17,11 @@
         "treatments"
       ]
     },
+    {
+      "required": [
+        "runtime"
+      ]
+    },
     {
       "required": [
         "judge"
@@ -224,6 +229,100 @@
           "description": "Whether a \"fail\" verdict affects the exit code. Defaults to never: a model's verdict is not reproducible, so it does not gate a build unless you decide it should. Independent of this, a judge that could not run always exits 1."
         }
       }
+    },
+    "runtime": {
+      "type": "object",
+      "description": "Contrast measured on a rendered page rather than derived from a token file. Run with `taste-check runtime`. Needs a Chromium and a server that is already up.",
+      "additionalProperties": false,
+      "required": [
+        "url",
+        "targets"
+      ],
+      "properties": {
+        "url": {
+          "type": "string",
+          "description": "The page to measure. A file:// URL works."
+        },
+        "endpoint": {
+          "type": "string",
+          "description": "An existing CDP endpoint, for example http://127.0.0.1:9222. Given this, taste-check connects instead of launching, and never closes a browser it did not start."
+        },
+        "browserPath": {
+          "type": "string",
+          "description": "Path to a Chromium. Falls back to CHROME_PATH, then the usual locations for the platform."
+        },
+        "timeout": {
+          "type": "number",
+          "exclusiveMinimum": 0,
+          "description": "Milliseconds to wait for the browser and for each page operation. Default 15000."
+        },
+        "states": {
+          "type": "array",
+          "minItems": 1,
+          "description": "States to measure the page in. Omit for a single default state.",
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": [
+              "name"
+            ],
+            "properties": {
+              "name": {
+                "type": "string",
+                "minLength": 1
+              },
+              "before": {
+                "type": "string",
+                "description": "JavaScript run on every navigation ahead of the page's own scripts. Where a theme goes, because the page reads it at boot."
+              },
+              "after": {
+                "type": "string",
+                "description": "JavaScript run once the page has loaded. For opening a panel or focusing a field."
+              },
+              "waitFor": {
+                "type": "string",
+                "description": "A selector to wait for before measuring."
+              }
+            }
+          }
+        },
+        "targets": {
+          "type": "array",
+          "minItems": 1,
+          "description": "What to measure. A selector matching nothing is a failure, as is a border colour on an edge with no width.",
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": [
+              "selector",
+              "prop",
+              "min"
+            ],
+            "properties": {
+              "selector": {
+                "type": "string",
+                "minLength": 1
+              },
+              "prop": {
+                "type": "string",
+                "minLength": 1,
+                "description": "A computed style property carrying the colour, for example color or borderTopColor."
+              },
+              "min": {
+                "type": "number",
+                "exclusiveMinimum": 0
+              },
+              "label": {
+                "type": "string"
+              },
+              "againstParent": {
+                "type": "boolean",
+                "description": "Measure against what is behind the element rather than its own background. Use when the thing being measured is a fill: measuring a fill against itself scores 1.00 and means nothing."
+              }
+            }
+          }
+        }
+      }
     }
   }
 }

package/src/cdp.mjs

@@ -0,0 +1,252 @@
+/**
+ * A very small Chrome DevTools Protocol client, over the WebSocket Node ships
+ * with. No dependencies.
+ *
+ * The alternative was a peer dependency on a browser automation library, which
+ * would have been less code here and several hundred megabytes there. CDP is
+ * JSON over a socket; the part of it this needs is Page.navigate and
+ * Runtime.evaluate, and that part is small enough to own.
+ *
+ * This is not a browser automation library and should not grow into one. If a
+ * feature here starts wanting selectors, waiting strategies or a frame tree,
+ * that is the point to take the dependency instead.
+ */
+import { spawn } from 'node:child_process';
+import { existsSync, mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+/** Where a Chromium lives on each platform, in the order worth trying. */
+const BROWSERS = {
+  darwin: [
+    '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
+    '/Applications/Chromium.app/Contents/MacOS/Chromium',
+    '/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge',
+  ],
+  linux: [
+    '/usr/bin/google-chrome',
+    '/usr/bin/google-chrome-stable',
+    '/usr/bin/chromium',
+    '/usr/bin/chromium-browser',
+    '/snap/bin/chromium',
+  ],
+  win32: [
+    'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
+    'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
+  ],
+};
+
+export function findBrowser(configured) {
+  const candidates = [configured, process.env.CHROME_PATH, ...(BROWSERS[process.platform] ?? [])];
+  for (const path of candidates) {
+    if (path && existsSync(path)) return path;
+  }
+  return null;
+}
+
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+/** Poll the endpoint until Chrome answers, or give up with a useful message. */
+async function waitForEndpoint(base, timeout) {
+  const deadline = Date.now() + timeout;
+  let last;
+  while (Date.now() < deadline) {
+    try {
+      const res = await fetch(`${base}/json/version`);
+      if (res.ok) return await res.json();
+    } catch (error) {
+      last = error;
+    }
+    await sleep(100);
+  }
+  throw new Error(`no CDP endpoint at ${base} after ${timeout}ms${last ? `: ${last.message}` : ''}`);
+}
+
+/**
+ * A page to evaluate against, plus a close() that tears down whatever this
+ * function created and nothing it did not. Connecting to a browser somebody
+ * else started must not kill it.
+ */
+export async function connect({ endpoint, browserPath, timeout = 15000 }) {
+  let child = null;
+  let profile = null;
+  let base = endpoint;
+
+  if (!base) {
+    const binary = findBrowser(browserPath);
+    if (!binary) {
+      throw new Error(
+        'no Chrome or Chromium found. Set runtime.browserPath, or CHROME_PATH, or ' +
+          'start a browser with --remote-debugging-port and set runtime.endpoint.',
+      );
+    }
+    // Port 0 lets the OS choose, so two runs never collide. Chrome writes the
+    // port it actually took into the profile directory.
+    profile = mkdtempSync(join(tmpdir(), 'taste-check-cdp-'));
+    child = spawn(
+      binary,
+      [
+        '--headless=new',
+        '--remote-debugging-port=0',
+        `--user-data-dir=${profile}`,
+        '--no-first-run',
+        '--no-default-browser-check',
+        '--disable-extensions',
+        '--disable-background-networking',
+        '--hide-scrollbars',
+        'about:blank',
+      ],
+      { stdio: ['ignore', 'ignore', 'pipe'] },
+    );
+    const port = await new Promise((resolve, reject) => {
+      let buffer = '';
+      const onData = (chunk) => {
+        buffer += chunk;
+        const match = buffer.match(/ws:\/\/127\.0\.0\.1:(\d+)\//);
+        if (match) settle(resolve, Number(match[1]));
+      };
+      const onExit = (code) => settle(reject, new Error(`the browser exited with code ${code}`));
+      const timer = setTimeout(
+        () => settle(reject, new Error('the browser did not report a debugging port')),
+        timeout,
+      );
+      const settle = (fn, value) => {
+        clearTimeout(timer);
+        child.stderr.off('data', onData);
+        child.off('exit', onExit);
+        fn(value);
+      };
+      child.stderr.on('data', onData);
+      child.on('exit', onExit);
+    });
+    // Nothing reads the browser's stderr after this, and a piped stream with
+    // a live child on the other end holds the event loop open. Without both
+    // of these the process sits idle until the browser happens to exit, which
+    // reads as a mysteriously slow run rather than as a leak.
+    child.stderr.destroy();
+    child.unref();
+    base = `http://127.0.0.1:${port}`;
+  }
+
+  await waitForEndpoint(base, timeout);
+  const list = await (await fetch(`${base}/json/list`)).json();
+  const target =
+    list.find((t) => t.type === 'page') ??
+    (await (await fetch(`${base}/json/new?about:blank`, { method: 'PUT' })).json());
+
+  const ws = new WebSocket(target.webSocketDebuggerUrl);
+  await new Promise((resolve, reject) => {
+    ws.onopen = resolve;
+    ws.onerror = () => reject(new Error(`could not open a socket to ${target.webSocketDebuggerUrl}`));
+  });
+
+  let id = 0;
+  const pending = new Map();
+  ws.onmessage = (event) => {
+    const message = JSON.parse(event.data);
+    if (!message.id || !pending.has(message.id)) return;
+    const { resolve, reject } = pending.get(message.id);
+    pending.delete(message.id);
+    if (message.error) reject(new Error(message.error.message));
+    else resolve(message.result);
+  };
+
+  // Every timer here is cleared on the happy path. An uncancelled one keeps
+  // the event loop alive until it fires, which turns a fast run into a wait
+  // for the full timeout and looks exactly like a slow browser.
+  const send = (method, params = {}) =>
+    new Promise((resolve, reject) => {
+      const n = ++id;
+      const timer = setTimeout(() => {
+        if (pending.delete(n)) reject(new Error(`${method} timed out after ${timeout}ms`));
+      }, timeout);
+      pending.set(n, {
+        resolve: (value) => {
+          clearTimeout(timer);
+          resolve(value);
+        },
+        reject: (error) => {
+          clearTimeout(timer);
+          reject(error);
+        },
+      });
+      ws.send(JSON.stringify({ id: n, method, params }));
+    });
+
+  await send('Page.enable');
+  await send('Runtime.enable');
+
+  return {
+    send,
+    /** Navigate and wait for the load event rather than for a fixed delay. */
+    async goto(url) {
+      let onMessage;
+      let timer;
+      const loaded = new Promise((resolve) => {
+        onMessage = (event) => {
+          if (JSON.parse(event.data).method === 'Page.loadEventFired') resolve();
+        };
+        ws.addEventListener('message', onMessage);
+        timer = setTimeout(resolve, timeout);
+      });
+      try {
+        await send('Page.navigate', { url });
+        await loaded;
+      } finally {
+        ws.removeEventListener('message', onMessage);
+        clearTimeout(timer);
+      }
+    },
+    /**
+     * Register a script to run on every navigation, before the page's own
+     * scripts. This is where a theme goes: setting localStorage after load and
+     * reloading would wipe anything the reload undoes, and setting it without
+     * a reload is too late for a page that reads it on boot.
+     */
+    async onNewDocument(source) {
+      const { identifier } = await send('Page.addScriptToEvaluateOnNewDocument', { source });
+      return identifier;
+    },
+    /** Remove one, so a state cannot leak into the next one. */
+    async removeNewDocumentScript(identifier) {
+      await send('Page.removeScriptToEvaluateOnNewDocument', { identifier });
+    },
+    /** Evaluate in the page and return the value, awaiting a promise result. */
+    async evaluate(expression) {
+      const { result, exceptionDetails } = await send('Runtime.evaluate', {
+        expression,
+        returnByValue: true,
+        awaitPromise: true,
+      });
+      if (exceptionDetails) {
+        throw new Error(exceptionDetails.exception?.description ?? exceptionDetails.text);
+      }
+      return result.value;
+    },
+    /**
+     * Best effort, and deliberately incapable of throwing. Teardown failing
+     * must never take down a run whose measurements already succeeded, and a
+     * temp directory the browser is still writing into is not worth a crash.
+     */
+    close() {
+      try {
+        ws.close();
+      } catch {
+        /* already gone */
+      }
+      if (!child) return;
+      try {
+        child.kill();
+      } catch {
+        /* already gone */
+      }
+      if (profile) {
+        try {
+          rmSync(profile, { recursive: true, force: true, maxRetries: 10, retryDelay: 50 });
+        } catch {
+          /* the OS will collect it */
+        }
+      }
+    },
+  };
+}

package/src/color.mjs

@@ -262,6 +262,22 @@ export function parseColor(input) {
   return err(`"${text}" is not a colour this tool can parse`);
 }
 
+/**
+ * Flatten a stack of backgrounds, nearest the element first, into one opaque
+ * colour.
+ *
+ * The check this ports from walked up to the first ancestor with alpha over
+ * 0.99 and measured against that, which quietly discards every translucent
+ * layer in between. A caption on a dark scrim over a light page is measured
+ * against the light page, and reports a pass it has not earned. Compositing
+ * the whole stack is what is actually painted.
+ */
+export function flatten(layers) {
+  let ground = layers[layers.length - 1].slice(0, 3);
+  for (let i = layers.length - 2; i >= 0; i -= 1) ground = composite(layers[i], ground);
+  return ground;
+}
+
 /** Composite a translucent foreground over an opaque ground. */
 export function composite(fg, bg) {
   return [0, 1, 2].map((i) => fg[3] * fg[i] + (1 - fg[3]) * bg[i]);

package/src/config.mjs

@@ -151,14 +151,88 @@ function validateJudge(judge, errors) {
   }
 }
 
+function validateRuntime(runtime, errors) {
+  rejectUnknown(
+    runtime,
+    ['url', 'endpoint', 'browserPath', 'timeout', 'states', 'targets'],
+    'runtime',
+    errors,
+  );
+  if (typeof runtime.url !== 'string' || !runtime.url) {
+    errors.push('runtime.url must be the page to measure');
+  }
+  for (const key of ['endpoint', 'browserPath']) {
+    if (runtime[key] !== undefined && (typeof runtime[key] !== 'string' || !runtime[key])) {
+      errors.push(`runtime.${key} must be a non-empty string`);
+    }
+  }
+  if (runtime.timeout !== undefined && (typeof runtime.timeout !== 'number' || runtime.timeout <= 0)) {
+    errors.push('runtime.timeout must be a positive number of milliseconds');
+  }
+  if (runtime.states !== undefined) {
+    if (!Array.isArray(runtime.states) || !runtime.states.length) {
+      errors.push('runtime.states must be a non-empty array. Omit it for a single default state.');
+    } else {
+      const names = new Set();
+      runtime.states.forEach((state, i) => {
+        const where = `runtime.states[${i}]`;
+        if (!isPlainObject(state)) {
+          errors.push(`${where} must be an object`);
+          return;
+        }
+        rejectUnknown(state, ['name', 'before', 'after', 'waitFor'], where, errors);
+        if (typeof state.name !== 'string' || !state.name) errors.push(`${where}.name must be a string`);
+        else if (names.has(state.name)) errors.push(`${where}.name "${state.name}" is used twice`);
+        else names.add(state.name);
+        for (const key of ['before', 'after', 'waitFor']) {
+          if (state[key] !== undefined && typeof state[key] !== 'string') {
+            errors.push(`${where}.${key} must be a string`);
+          }
+        }
+      });
+    }
+  }
+  if (!Array.isArray(runtime.targets) || !runtime.targets.length) {
+    errors.push('runtime.targets must be a non-empty array. A check with no targets cannot fail.');
+    return;
+  }
+  runtime.targets.forEach((target, i) => {
+    const where = `runtime.targets[${i}]`;
+    if (!isPlainObject(target)) {
+      errors.push(`${where} must be an object`);
+      return;
+    }
+    rejectUnknown(target, ['selector', 'prop', 'min', 'label', 'againstParent'], where, errors);
+    for (const key of ['selector', 'prop']) {
+      if (typeof target[key] !== 'string' || !target[key]) {
+        errors.push(`${where}.${key} must be a non-empty string`);
+      }
+    }
+    if (typeof target.min !== 'number' || !Number.isFinite(target.min) || target.min <= 0) {
+      errors.push(`${where}.min must be a positive number`);
+    }
+    if (target.label !== undefined && typeof target.label !== 'string') {
+      errors.push(`${where}.label must be a string`);
+    }
+    if (target.againstParent !== undefined && typeof target.againstParent !== 'boolean') {
+      errors.push(`${where}.againstParent must be a boolean`);
+    }
+  });
+}
+
 /** Validate a parsed config, returning a list of human-readable problems. */
 export function validate(config) {
   const errors = [];
   if (!isPlainObject(config)) return ['the config must be a JSON object'];
-  rejectUnknown(config, ['$schema', 'contrast', 'treatments', 'judge'], 'the config', errors);
+  rejectUnknown(config, ['$schema', 'contrast', 'treatments', 'judge', 'runtime'], 'the config', errors);
 
-  if (config.contrast === undefined && config.treatments === undefined && config.judge === undefined) {
-    errors.push('the config must define at least one of "contrast", "treatments" or "judge"');
+  if (
+    config.contrast === undefined &&
+    config.treatments === undefined &&
+    config.judge === undefined &&
+    config.runtime === undefined
+  ) {
+    errors.push('the config must define at least one of "contrast", "treatments", "runtime" or "judge"');
   }
   if (config.contrast !== undefined) {
     if (isPlainObject(config.contrast)) validateContrast(config.contrast, errors);
@@ -172,6 +246,10 @@ export function validate(config) {
     if (isPlainObject(config.judge)) validateJudge(config.judge, errors);
     else errors.push('judge must be an object');
   }
+  if (config.runtime !== undefined) {
+    if (isPlainObject(config.runtime)) validateRuntime(config.runtime, errors);
+    else errors.push('runtime must be an object');
+  }
   return errors;
 }
 

package/src/index.mjs

@@ -5,6 +5,8 @@
 export { runContrast } from './contrast.mjs';
 export { runTreatments } from './treatments.mjs';
 export { runJudge, buildPrompt, checklistLines, extractJson } from './judge.mjs';
+export { runRuntime } from './runtime.mjs';
+export { connect, findBrowser } from './cdp.mjs';
 export { load, validate } from './config.mjs';
 export { toText, toJson, failed } from './report.mjs';
 export { parseColor, contrastRatio, composite, luminance } from './color.mjs';
@@ -14,6 +16,7 @@ export { openTags, classesOf } from './treatments.mjs';
 import { runContrast } from './contrast.mjs';
 import { runTreatments } from './treatments.mjs';
 import { runJudge } from './judge.mjs';
+import { runRuntime } from './runtime.mjs';
 
 /**
  * Run the deterministic checks a config asks for. `only` narrows to one by
@@ -38,3 +41,12 @@ export function run(config, cwd, { only = null } = {}) {
 export function judge(config, cwd) {
   return [runJudge(config.judge, cwd)];
 }
+
+/**
+ * Measure a rendered page. Separate from `run` because it needs a browser and
+ * a server that is already up, which is a heavier precondition than reading
+ * files off disk, and not one to impose on the check people put in a hook.
+ */
+export async function runtime(config, cwd) {
+  return [await runRuntime(config.runtime, cwd)];
+}

package/src/report.mjs

@@ -52,7 +52,7 @@ function judgeLines(result) {
 }
 
 export const linesFor = (result) => {
-  if (result.name === 'contrast') return contrastLines(result);
+  if (result.name === 'contrast' || result.name === 'runtime') return contrastLines(result);
   if (result.name === 'judge') return judgeLines(result);
   return treatmentLines(result);
 };
@@ -79,7 +79,9 @@ export function toText(results) {
       out.push(`${result.name} ok, ${result.summary}`);
       // A clean contrast run still shows its margins. Nothing else in the
       // report tells you which pair is one nudge away from failing.
-      if (result.name === 'contrast') for (const l of lines) out.push(`  ${MARK.ok}${l.text}`);
+      if (result.name === 'contrast' || result.name === 'runtime') {
+        for (const l of lines) out.push(`  ${MARK.ok}${l.text}`);
+      }
       out.push('');
       continue;
     }

package/src/runtime.mjs

@@ -0,0 +1,161 @@
+/**
+ * Contrast measured off a rendered page.
+ *
+ * This is the check the contrast module is a static approximation of. A token
+ * file says what a colour is declared to be. A rendered page says what is
+ * actually painted, which is a different question the moment anything is
+ * translucent, inherited, set by a component, or overridden by a state.
+ *
+ * Two things it does that reading tokens cannot:
+ *
+ *   1. It composites the whole background stack, nearest layer first, rather
+ *      than measuring against the first opaque ancestor. White text on a dark
+ *      scrim over a light page reads 1.04:1 if you skip the scrim and 10.57:1
+ *      if you do not. One of those is a false failure, and the same shortcut
+ *      produces a false pass with the colours the other way round.
+ *
+ *   2. It can measure a state, because it can put the page into one first.
+ *      A theme, an opened panel, a focused input.
+ *
+ * A selector that matches nothing is a failure, not a skipped target. So is a
+ * border with no width: you asked for the contrast of an edge that is not
+ * being drawn, and the honest answer is that the question is wrong.
+ */
+import { contrastRatio, flatten, isOpaque, parseColor } from './color.mjs';
+import { connect } from './cdp.mjs';
+
+/**
+ * Runs in the page. Returns the foreground and the stack of backgrounds behind
+ * it, and leaves every judgement to the caller so that the reasoning lives in
+ * one place rather than half here and half in a string.
+ */
+const MEASURE = `(targets) => targets.map(({ selector, prop, againstParent }) => {
+  const el = document.querySelector(selector);
+  if (!el) return { error: 'matched no element' };
+  if (!el.getClientRects().length) return { error: 'matched an element that is not rendered' };
+  const cs = getComputedStyle(el);
+  const fg = cs[prop];
+  if (!fg) return { error: 'has no ' + prop };
+  if (prop.startsWith('border')) {
+    const side = prop.replace(/^border|Color$/g, '');
+    const width = parseFloat(cs['border' + (side || 'Top') + 'Width']);
+    if (!width) return { error: 'has a ' + prop + ' but no width, so no edge is drawn' };
+  }
+  const layers = [];
+  let node = againstParent ? el.parentElement : el;
+  if (!node) return { error: 'has no parent to measure against' };
+  while (node) {
+    const bg = getComputedStyle(node).backgroundColor;
+    const m = bg && bg.match(/rgba?\\(([^)]+)\\)/);
+    if (m) {
+      const p = m[1].split(/[,\\/]/).map((v) => parseFloat(v.trim()));
+      const alpha = p.length > 3 ? p[3] : 1;
+      if (alpha > 0) {
+        layers.push(bg);
+        if (alpha > 0.999) break;
+      }
+    }
+    node = node.parentElement;
+  }
+  const root = getComputedStyle(document.documentElement).backgroundColor;
+  return { fg, layers, root, text: (el.textContent || '').trim().slice(0, 40) };
+})`;
+
+export async function runRuntime(config, cwd, { connect: open = connect } = {}) {
+  const samples = [];
+  const problems = [];
+  const { url, endpoint, browserPath, timeout, states = [{ name: 'default' }], targets } = config;
+
+  let page;
+  try {
+    page = await open({ endpoint, browserPath, timeout });
+  } catch (error) {
+    problems.push(error.message);
+    return { name: 'runtime', samples, problems, summary: '' };
+  }
+
+  try {
+    for (const state of states) {
+      // `before` runs on every navigation ahead of the page's own scripts, so
+      // a theme read at boot sees it. Running it after load and reloading
+      // would undo any DOM change; running it after load without reloading is
+      // too late for anything the page reads once. It is removed again after
+      // the state is measured, or the next state inherits it.
+      let initScript = null;
+      if (state.before) initScript = await page.onNewDocument(state.before);
+
+      await page.goto(url);
+      // `after` is the other half: opening a panel, focusing a field, anything
+      // that only exists once there is a document to act on.
+      if (state.after) await page.evaluate(state.after);
+      if (state.waitFor) {
+        await page.evaluate(
+          `new Promise((resolve, reject) => {
+            const done = () => document.querySelector(${JSON.stringify(state.waitFor)});
+            if (done()) return resolve(true);
+            const t = setInterval(() => { if (done()) { clearInterval(t); resolve(true); } }, 50);
+            setTimeout(() => { clearInterval(t); reject(new Error('waitFor ${state.waitFor} never appeared')); }, 10000);
+          })`,
+        );
+      }
+
+      const measured = await page.evaluate(`(${MEASURE})(${JSON.stringify(targets)})`);
+
+      targets.forEach((target, i) => {
+        const found = measured[i];
+        const where = `${target.selector} { ${target.prop} }`;
+        if (found.error) {
+          problems.push(`state "${state.name}": ${where} ${found.error}`);
+          return;
+        }
+        const fg = parseColor(found.fg);
+        if (!fg.ok) {
+          problems.push(`state "${state.name}": ${where} ${fg.reason}`);
+          return;
+        }
+        const layers = [];
+        for (const raw of found.layers) {
+          const parsed = parseColor(raw);
+          if (!parsed.ok) {
+            problems.push(`state "${state.name}": ${where} background ${parsed.reason}`);
+            return;
+          }
+          layers.push(parsed.rgba);
+        }
+        // Nothing opaque anywhere up the tree means the canvas is showing
+        // through, and the canvas is whatever the root paints.
+        if (!layers.length || !isOpaque(layers[layers.length - 1])) {
+          const root = parseColor(found.root);
+          layers.push(root.ok && isOpaque(root.rgba) ? root.rgba : [255, 255, 255, 1]);
+        }
+
+        const ground = flatten(layers);
+        const ratio = contrastRatio(fg.rgba, ground);
+        samples.push({
+          theme: state.name,
+          fg: target.selector,
+          bg: `${target.prop}, ${layers.length} ${layers.length === 1 ? 'layer' : 'layers'}`,
+          note: target.label ?? found.text ?? '',
+          ratio,
+          min: target.min,
+          pass: ratio >= target.min,
+        });
+      });
+
+      if (initScript) await page.removeNewDocumentScript(initScript);
+    }
+  } catch (error) {
+    problems.push(`the page could not be measured: ${error.message}`);
+  } finally {
+    page.close();
+  }
+
+  return {
+    name: 'runtime',
+    samples,
+    problems,
+    summary: `${samples.length} ${samples.length === 1 ? 'target' : 'targets'} across ${
+      states.length
+    } ${states.length === 1 ? 'state' : 'states'}`,
+  };
+}