@josueavalosjim/taste-check 0.4.1 → 0.5.1

package/README.md

@@ -4,7 +4,8 @@ Design review in CI, with a line down the middle of it.
 
 Some of design review is measurable. Contrast ratios, values that should have
 been tokens, class names nobody approved. taste-check measures those and fails
-your build on them.
+your build on them, either from your token file or from a page it renders in a
+real browser.
 
 The rest is judgment, and it cannot be measured. For that it sends your
 screenshots and your checklist to a model, and reports what comes back as an
@@ -180,6 +181,60 @@ direction to be wrong in.
 Template literal holes are read into rather than blanked, so a class written
 inside `` `card ${on ? 'card--on' : ''}` `` is seen.
 
+## Measuring the rendered page
+
+The contrast check above reads your token file, which tells you what a colour
+is declared to be. `taste-check runtime` opens the page and reads what is
+actually painted.
+
+```json
+{
+  "runtime": {
+    "url": "http://localhost:3000",
+    "states": [
+      { "name": "light" },
+      { "name": "dark", "before": "localStorage.setItem('theme', 'dark')" }
+    ],
+    "targets": [
+      { "selector": ".caption", "prop": "color", "min": 4.5 },
+      { "selector": ".panel", "prop": "borderTopColor", "min": 3.0 },
+      { "selector": ".row.is-selected", "prop": "backgroundColor", "min": 1.25, "againstParent": true }
+    ]
+  }
+}
+```
+
+```bash
+taste-check runtime
+```
+
+It launches a headless Chromium, or connects to one you already have if you
+give it an `endpoint`, and it never closes a browser it did not start. There is
+no dependency for this. The Chrome DevTools Protocol is JSON over the WebSocket
+Node already ships, and the part of it needed here is two methods.
+
+Two things this does that reading tokens cannot.
+
+**It composites the whole background stack.** White text on a 75% black scrim
+over a near-white page measures 10.57:1. Stop at the first opaque ancestor, the
+way the check this was ported from did, and you measure the white against the
+page instead and get 1.04:1. That is a failure the design has not earned, and
+with the colours the other way round the same shortcut hands you a pass it has
+not earned either.
+
+**It can measure a state.** `before` runs on every navigation ahead of the
+page's own scripts, which is where a theme belongs because the page reads it at
+boot. `after` runs once there is a document, for opening a panel or focusing a
+field. Each state's setup is removed before the next one, so they cannot leak.
+
+Use `againstParent` when the thing being measured is a fill rather than a
+foreground. A selected row with its own background measured against itself
+scores 1.00, which is a pass that means nothing.
+
+A selector that matches nothing is a failure. So is a border colour on an edge
+with no width: you asked for the contrast of something that is not being drawn,
+and the question is wrong rather than the answer being zero.
+
 ## The judge
 
 Everything above measures. This asks the question a measurement cannot: not
@@ -266,9 +321,15 @@ from anywhere.
 | `judge.shotCommand` | Optional command run first to produce those screenshots. |
 | `judge.command` | The model command. Prompt on stdin, image paths as arguments. |
 | `judge.failOn` | `"never"` (default) or `"fail"`. Whether a verdict blocks. |
+| `runtime.url` | The page to measure. A `file://` URL works. |
+| `runtime.endpoint` | An existing CDP endpoint. Given one, taste-check connects rather than launching, and never closes a browser it did not start. |
+| `runtime.browserPath` | Path to a Chromium. Falls back to `CHROME_PATH`, then the usual locations. |
+| `runtime.states[]` | `before` runs ahead of the page's scripts, `after` once loaded, `waitFor` is a selector. |
+| `runtime.targets[]` | `selector`, `prop`, `min`, and `againstParent` when measuring a fill. |
 
 ```
-taste-check [options]         Run the deterministic checks
+taste-check [options]         Run the deterministic checks over your files
+taste-check runtime [options] Measure contrast on a rendered page
 taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
 
   -c, --config <path>   Config file (default: tastecheck.config.json)
@@ -342,12 +403,6 @@ invisible to it.
 
 Not built. Written down so the shape is clear.
 
-**A runtime mode**, closing the gap named above by measuring `getComputedStyle`
-in a real browser, as an optional peer dependency so the core stays free of
-one. This is the one that matters most. An advisory judge is only worth
-listening to if the measured half beside it is genuinely measured, and reading
-a token file is the weaker version of that.
-
 **YAML configs**, once there is a reason to take on a parser.
 
 **SARIF output**, so findings land in a code scanning tab rather than only in

package/bin/taste-check.mjs

@@ -8,12 +8,13 @@
  * not a quiet skip.
  */
 import { load } from '../src/config.mjs';
-import { judge, run } from '../src/index.mjs';
+import { judge, run, runtime } from '../src/index.mjs';
 import { failed, toJson, toText } from '../src/report.mjs';
 
 const USAGE = `taste-check
 
-  taste-check [options]         Run the deterministic checks
+  taste-check [options]         Run the deterministic checks over your files
+  taste-check runtime [options] Measure contrast on a rendered page
   taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
 
 Options:
@@ -25,6 +26,11 @@ Options:
 
 Exit code is 1 if any check fails, 0 if every check ran and passed.
 
+runtime is a separate command because it needs a browser and a server that
+is already up. It measures what is actually painted, compositing every
+background layer behind an element rather than stopping at the first opaque
+one, and it can put the page into a state first.
+
 The judge is a separate command because it runs a model, and a model's
 verdict is not reproducible. Its verdicts print as notes and do not affect
 the exit code unless judge.failOn is set to "fail". Whether the judge ran
@@ -35,8 +41,8 @@ function parseArgs(argv) {
   const options = { config: 'tastecheck.config.json', only: null, json: false, command: 'check' };
   // One positional, and only in first position, so a stray argument is an
   // error rather than something silently ignored.
-  if (argv[0] === 'judge') {
-    options.command = 'judge';
+  if (argv[0] === 'judge' || argv[0] === 'runtime') {
+    options.command = argv[0];
     argv = argv.slice(1);
   }
   for (let i = 0; i < argv.length; i += 1) {
@@ -95,18 +101,18 @@ if (!loaded.ok) {
   process.exit(1);
 }
 
-if (options.command === 'judge' && options.only) {
-  die('--only applies to the deterministic checks, not to judge');
+if (options.command !== 'check' && options.only) {
+  die(`--only applies to the deterministic checks, not to ${options.command}`);
 }
 
-if (options.command === 'judge' && !loaded.config.judge) {
-  die(`${options.config} defines no "judge" block.`);
+if (options.command !== 'check' && !loaded.config[options.command]) {
+  die(`${options.config} defines no "${options.command}" block.`);
 }
 
-const results =
-  options.command === 'judge'
-    ? judge(loaded.config, loaded.dir)
-    : run(loaded.config, loaded.dir, { only: options.only });
+let results;
+if (options.command === 'judge') results = judge(loaded.config, loaded.dir);
+else if (options.command === 'runtime') results = await runtime(loaded.config, loaded.dir);
+else results = run(loaded.config, loaded.dir, { only: options.only });
 
 if (!results.length) {
   die(`nothing to run. ${options.config} defines no ${options.only ?? 'contrast or treatments'} check.`);

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@josueavalosjim/taste-check",
-  "version": "0.4.1",
-  "description": "Design review in CI with a line down the middle: measured checks that gate the build (WCAG contrast over your own tokens, one-off values in your own markup) and a fresh-eyes model judge whose verdicts stay advisory. Ships no design rules of its own.",
+  "version": "0.5.1",
+  "description": "Design review in CI with a line down the middle: measured checks that gate the build (WCAG contrast from your tokens or from a real rendered page, one-off values in your markup) and a fresh-eyes model judge whose verdicts stay advisory. Zero dependencies. Ships no design rules of its own.",
   "keywords": [
     "accessibility",
     "cli",

package/schema/config.schema.json

@@ -17,6 +17,11 @@
         "treatments"
       ]
     },
+    {
+      "required": [
+        "runtime"
+      ]
+    },
     {
       "required": [
         "judge"
@@ -224,6 +229,100 @@
           "description": "Whether a \"fail\" verdict affects the exit code. Defaults to never: a model's verdict is not reproducible, so it does not gate a build unless you decide it should. Independent of this, a judge that could not run always exits 1."
         }
       }
+    },
+    "runtime": {
+      "type": "object",
+      "description": "Contrast measured on a rendered page rather than derived from a token file. Run with `taste-check runtime`. Needs a Chromium and a server that is already up.",
+      "additionalProperties": false,
+      "required": [
+        "url",
+        "targets"
+      ],
+      "properties": {
+        "url": {
+          "type": "string",
+          "description": "The page to measure. A file:// URL works."
+        },
+        "endpoint": {
+          "type": "string",
+          "description": "An existing CDP endpoint, for example http://127.0.0.1:9222. Given this, taste-check connects instead of launching, and never closes a browser it did not start."
+        },
+        "browserPath": {
+          "type": "string",
+          "description": "Path to a Chromium. Falls back to CHROME_PATH, then the usual locations for the platform."
+        },
+        "timeout": {
+          "type": "number",
+          "exclusiveMinimum": 0,
+          "description": "Milliseconds to wait for the browser and for each page operation. Default 15000."
+        },
+        "states": {
+          "type": "array",
+          "minItems": 1,
+          "description": "States to measure the page in. Omit for a single default state.",
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": [
+              "name"
+            ],
+            "properties": {
+              "name": {
+                "type": "string",
+                "minLength": 1
+              },
+              "before": {
+                "type": "string",
+                "description": "JavaScript run on every navigation ahead of the page's own scripts. Where a theme goes, because the page reads it at boot."
+              },
+              "after": {
+                "type": "string",
+                "description": "JavaScript run once the page has loaded. For opening a panel or focusing a field."
+              },
+              "waitFor": {
+                "type": "string",
+                "description": "A selector to wait for before measuring."
+              }
+            }
+          }
+        },
+        "targets": {
+          "type": "array",
+          "minItems": 1,
+          "description": "What to measure. A selector matching nothing is a failure, as is a border colour on an edge with no width.",
+          "items": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": [
+              "selector",
+              "prop",
+              "min"
+            ],
+            "properties": {
+              "selector": {
+                "type": "string",
+                "minLength": 1
+              },
+              "prop": {
+                "type": "string",
+                "minLength": 1,
+                "description": "A computed style property carrying the colour, for example color or borderTopColor."
+              },
+              "min": {
+                "type": "number",
+                "exclusiveMinimum": 0
+              },
+              "label": {
+                "type": "string"
+              },
+              "againstParent": {
+                "type": "boolean",
+                "description": "Measure against what is behind the element rather than its own background. Use when the thing being measured is a fill: measuring a fill against itself scores 1.00 and means nothing."
+              }
+            }
+          }
+        }
+      }
     }
   }
 }

package/src/cdp.mjs

@@ -0,0 +1,272 @@
+/**
+ * A very small Chrome DevTools Protocol client, over the WebSocket Node ships
+ * with. No dependencies.
+ *
+ * The alternative was a peer dependency on a browser automation library, which
+ * would have been less code here and several hundred megabytes there. CDP is
+ * JSON over a socket; the part of it this needs is Page.navigate and
+ * Runtime.evaluate, and that part is small enough to own.
+ *
+ * This is not a browser automation library and should not grow into one. If a
+ * feature here starts wanting selectors, waiting strategies or a frame tree,
+ * that is the point to take the dependency instead.
+ */
+import { spawn } from 'node:child_process';
+import { existsSync, mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+/** Where a Chromium lives on each platform, in the order worth trying. */
+const BROWSERS = {
+  darwin: [
+    '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
+    '/Applications/Chromium.app/Contents/MacOS/Chromium',
+    '/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge',
+  ],
+  linux: [
+    '/usr/bin/google-chrome',
+    '/usr/bin/google-chrome-stable',
+    '/usr/bin/chromium',
+    '/usr/bin/chromium-browser',
+    '/snap/bin/chromium',
+  ],
+  win32: [
+    'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
+    'C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe',
+  ],
+};
+
+export function findBrowser(configured) {
+  const candidates = [configured, process.env.CHROME_PATH, ...(BROWSERS[process.platform] ?? [])];
+  for (const path of candidates) {
+    if (path && existsSync(path)) return path;
+  }
+  return null;
+}
+
+const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+/** Poll the endpoint until Chrome answers, or give up with a useful message. */
+async function waitForEndpoint(base, timeout) {
+  const deadline = Date.now() + timeout;
+  let last;
+  while (Date.now() < deadline) {
+    try {
+      const res = await fetch(`${base}/json/version`);
+      if (res.ok) return await res.json();
+    } catch (error) {
+      last = error;
+    }
+    await sleep(100);
+  }
+  throw new Error(`no CDP endpoint at ${base} after ${timeout}ms${last ? `: ${last.message}` : ''}`);
+}
+
+/**
+ * A page to evaluate against, plus a close() that tears down whatever this
+ * function created and nothing it did not. Connecting to a browser somebody
+ * else started must not kill it.
+ */
+export async function connect({ endpoint, browserPath, timeout = 15000 }) {
+  let child = null;
+  let profile = null;
+  let base = endpoint;
+
+  if (!base) {
+    const binary = findBrowser(browserPath);
+    if (!binary) {
+      throw new Error(
+        'no Chrome or Chromium found. Set runtime.browserPath, or CHROME_PATH, or ' +
+          'start a browser with --remote-debugging-port and set runtime.endpoint.',
+      );
+    }
+    // Port 0 lets the OS choose, so two runs never collide. Chrome writes the
+    // port it actually took into the profile directory.
+    profile = mkdtempSync(join(tmpdir(), 'taste-check-cdp-'));
+    child = spawn(
+      binary,
+      [
+        '--headless=new',
+        '--remote-debugging-port=0',
+        `--user-data-dir=${profile}`,
+        '--no-first-run',
+        '--no-default-browser-check',
+        '--disable-extensions',
+        '--disable-background-networking',
+        '--hide-scrollbars',
+        'about:blank',
+      ],
+      { stdio: ['ignore', 'ignore', 'pipe'] },
+    );
+    const port = await new Promise((resolve, reject) => {
+      let buffer = '';
+      const onData = (chunk) => {
+        buffer += chunk;
+        const match = buffer.match(/ws:\/\/127\.0\.0\.1:(\d+)\//);
+        if (match) settle(resolve, Number(match[1]));
+      };
+      const onExit = (code) => settle(reject, new Error(`the browser exited with code ${code}`));
+      const timer = setTimeout(
+        () => settle(reject, new Error('the browser did not report a debugging port')),
+        timeout,
+      );
+      const settle = (fn, value) => {
+        clearTimeout(timer);
+        child.stderr.off('data', onData);
+        child.off('exit', onExit);
+        fn(value);
+      };
+      child.stderr.on('data', onData);
+      child.on('exit', onExit);
+    });
+    // Nothing reads the browser's stderr after this, and a piped stream with
+    // a live child on the other end holds the event loop open. Without both
+    // of these the process sits idle until the browser happens to exit, which
+    // reads as a mysteriously slow run rather than as a leak.
+    child.stderr.destroy();
+    child.unref();
+    base = `http://127.0.0.1:${port}`;
+  }
+
+  await waitForEndpoint(base, timeout);
+  const list = await (await fetch(`${base}/json/list`)).json();
+  const target =
+    list.find((t) => t.type === 'page') ??
+    (await (await fetch(`${base}/json/new?about:blank`, { method: 'PUT' })).json());
+
+  const ws = new WebSocket(target.webSocketDebuggerUrl);
+  await new Promise((resolve, reject) => {
+    ws.onopen = resolve;
+    ws.onerror = () => reject(new Error(`could not open a socket to ${target.webSocketDebuggerUrl}`));
+  });
+
+  let id = 0;
+  const pending = new Map();
+  ws.onmessage = (event) => {
+    const message = JSON.parse(event.data);
+    if (!message.id || !pending.has(message.id)) return;
+    const { resolve, reject } = pending.get(message.id);
+    pending.delete(message.id);
+    if (message.error) reject(new Error(message.error.message));
+    else resolve(message.result);
+  };
+
+  // Every timer here is cleared on the happy path. An uncancelled one keeps
+  // the event loop alive until it fires, which turns a fast run into a wait
+  // for the full timeout and looks exactly like a slow browser.
+  const send = (method, params = {}) =>
+    new Promise((resolve, reject) => {
+      const n = ++id;
+      const timer = setTimeout(() => {
+        if (pending.delete(n)) reject(new Error(`${method} timed out after ${timeout}ms`));
+      }, timeout);
+      pending.set(n, {
+        resolve: (value) => {
+          clearTimeout(timer);
+          resolve(value);
+        },
+        reject: (error) => {
+          clearTimeout(timer);
+          reject(error);
+        },
+      });
+      ws.send(JSON.stringify({ id: n, method, params }));
+    });
+
+  await send('Page.enable');
+  await send('Runtime.enable');
+
+  return {
+    send,
+    /** Navigate and wait for the load event rather than for a fixed delay. */
+    async goto(url) {
+      let onMessage;
+      let timer;
+      const loaded = new Promise((resolve) => {
+        onMessage = (event) => {
+          if (JSON.parse(event.data).method === 'Page.loadEventFired') resolve();
+        };
+        ws.addEventListener('message', onMessage);
+        timer = setTimeout(resolve, timeout);
+      });
+      try {
+        await send('Page.navigate', { url });
+        await loaded;
+      } finally {
+        ws.removeEventListener('message', onMessage);
+        clearTimeout(timer);
+      }
+    },
+    /**
+     * Register a script to run on every navigation, before the page's own
+     * scripts. This is where a theme goes: setting localStorage after load and
+     * reloading would wipe anything the reload undoes, and setting it without
+     * a reload is too late for a page that reads it on boot.
+     */
+    async onNewDocument(source) {
+      const { identifier } = await send('Page.addScriptToEvaluateOnNewDocument', { source });
+      return identifier;
+    },
+    /** Remove one, so a state cannot leak into the next one. */
+    async removeNewDocumentScript(identifier) {
+      await send('Page.removeScriptToEvaluateOnNewDocument', { identifier });
+    },
+    /** Evaluate in the page and return the value, awaiting a promise result. */
+    async evaluate(expression) {
+      const { result, exceptionDetails } = await send('Runtime.evaluate', {
+        expression,
+        returnByValue: true,
+        awaitPromise: true,
+      });
+      if (exceptionDetails) {
+        throw new Error(exceptionDetails.exception?.description ?? exceptionDetails.text);
+      }
+      return result.value;
+    },
+    /**
+     * Waits for the browser to actually exit before removing its profile.
+     *
+     * Killing and deleting in the same breath loses a race with Chrome, which
+     * is still flushing its profile when the signal arrives. Best effort was
+     * not good enough: it left a directory behind on nearly every run, and a
+     * tool that litters the temp directory every time CI runs is not one to
+     * inflict on anybody.
+     *
+     * Still incapable of throwing. Teardown must never take down a run whose
+     * measurements already succeeded.
+     */
+    async close() {
+      try {
+        ws.close();
+      } catch {
+        /* already gone */
+      }
+      if (!child) return;
+      try {
+        if (child.exitCode === null && child.signalCode === null) {
+          const exited = new Promise((resolve) => child.once('exit', resolve));
+          child.kill();
+          // SIGTERM is usually enough. If it is not, stop being polite.
+          const forced = new Promise((resolve) => setTimeout(resolve, 2000)).then(() => {
+            try {
+              child.kill('SIGKILL');
+            } catch {
+              /* already gone */
+            }
+            return new Promise((resolve) => setTimeout(resolve, 300));
+          });
+          await Promise.race([exited, forced]);
+        }
+      } catch {
+        /* already gone */
+      }
+      if (profile) {
+        try {
+          rmSync(profile, { recursive: true, force: true, maxRetries: 10, retryDelay: 50 });
+        } catch {
+          /* the OS will collect it */
+        }
+      }
+    },
+  };
+}

package/src/color.mjs

@@ -262,6 +262,22 @@ export function parseColor(input) {
   return err(`"${text}" is not a colour this tool can parse`);
 }
 
+/**
+ * Flatten a stack of backgrounds, nearest the element first, into one opaque
+ * colour.
+ *
+ * The check this ports from walked up to the first ancestor with alpha over
+ * 0.99 and measured against that, which quietly discards every translucent
+ * layer in between. A caption on a dark scrim over a light page is measured
+ * against the light page, and reports a pass it has not earned. Compositing
+ * the whole stack is what is actually painted.
+ */
+export function flatten(layers) {
+  let ground = layers[layers.length - 1].slice(0, 3);
+  for (let i = layers.length - 2; i >= 0; i -= 1) ground = composite(layers[i], ground);
+  return ground;
+}
+
 /** Composite a translucent foreground over an opaque ground. */
 export function composite(fg, bg) {
   return [0, 1, 2].map((i) => fg[3] * fg[i] + (1 - fg[3]) * bg[i]);

package/src/config.mjs

@@ -151,14 +151,88 @@ function validateJudge(judge, errors) {
   }
 }
 
+function validateRuntime(runtime, errors) {
+  rejectUnknown(
+    runtime,
+    ['url', 'endpoint', 'browserPath', 'timeout', 'states', 'targets'],
+    'runtime',
+    errors,
+  );
+  if (typeof runtime.url !== 'string' || !runtime.url) {
+    errors.push('runtime.url must be the page to measure');
+  }
+  for (const key of ['endpoint', 'browserPath']) {
+    if (runtime[key] !== undefined && (typeof runtime[key] !== 'string' || !runtime[key])) {
+      errors.push(`runtime.${key} must be a non-empty string`);
+    }
+  }
+  if (runtime.timeout !== undefined && (typeof runtime.timeout !== 'number' || runtime.timeout <= 0)) {
+    errors.push('runtime.timeout must be a positive number of milliseconds');
+  }
+  if (runtime.states !== undefined) {
+    if (!Array.isArray(runtime.states) || !runtime.states.length) {
+      errors.push('runtime.states must be a non-empty array. Omit it for a single default state.');
+    } else {
+      const names = new Set();
+      runtime.states.forEach((state, i) => {
+        const where = `runtime.states[${i}]`;
+        if (!isPlainObject(state)) {
+          errors.push(`${where} must be an object`);
+          return;
+        }
+        rejectUnknown(state, ['name', 'before', 'after', 'waitFor'], where, errors);
+        if (typeof state.name !== 'string' || !state.name) errors.push(`${where}.name must be a string`);
+        else if (names.has(state.name)) errors.push(`${where}.name "${state.name}" is used twice`);
+        else names.add(state.name);
+        for (const key of ['before', 'after', 'waitFor']) {
+          if (state[key] !== undefined && typeof state[key] !== 'string') {
+            errors.push(`${where}.${key} must be a string`);
+          }
+        }
+      });
+    }
+  }
+  if (!Array.isArray(runtime.targets) || !runtime.targets.length) {
+    errors.push('runtime.targets must be a non-empty array. A check with no targets cannot fail.');
+    return;
+  }
+  runtime.targets.forEach((target, i) => {
+    const where = `runtime.targets[${i}]`;
+    if (!isPlainObject(target)) {
+      errors.push(`${where} must be an object`);
+      return;
+    }
+    rejectUnknown(target, ['selector', 'prop', 'min', 'label', 'againstParent'], where, errors);
+    for (const key of ['selector', 'prop']) {
+      if (typeof target[key] !== 'string' || !target[key]) {
+        errors.push(`${where}.${key} must be a non-empty string`);
+      }
+    }
+    if (typeof target.min !== 'number' || !Number.isFinite(target.min) || target.min <= 0) {
+      errors.push(`${where}.min must be a positive number`);
+    }
+    if (target.label !== undefined && typeof target.label !== 'string') {
+      errors.push(`${where}.label must be a string`);
+    }
+    if (target.againstParent !== undefined && typeof target.againstParent !== 'boolean') {
+      errors.push(`${where}.againstParent must be a boolean`);
+    }
+  });
+}
+
 /** Validate a parsed config, returning a list of human-readable problems. */
 export function validate(config) {
   const errors = [];
   if (!isPlainObject(config)) return ['the config must be a JSON object'];
-  rejectUnknown(config, ['$schema', 'contrast', 'treatments', 'judge'], 'the config', errors);
+  rejectUnknown(config, ['$schema', 'contrast', 'treatments', 'judge', 'runtime'], 'the config', errors);
 
-  if (config.contrast === undefined && config.treatments === undefined && config.judge === undefined) {
-    errors.push('the config must define at least one of "contrast", "treatments" or "judge"');
+  if (
+    config.contrast === undefined &&
+    config.treatments === undefined &&
+    config.judge === undefined &&
+    config.runtime === undefined
+  ) {
+    errors.push('the config must define at least one of "contrast", "treatments", "runtime" or "judge"');
   }
   if (config.contrast !== undefined) {
     if (isPlainObject(config.contrast)) validateContrast(config.contrast, errors);
@@ -172,6 +246,10 @@ export function validate(config) {
     if (isPlainObject(config.judge)) validateJudge(config.judge, errors);
     else errors.push('judge must be an object');
   }
+  if (config.runtime !== undefined) {
+    if (isPlainObject(config.runtime)) validateRuntime(config.runtime, errors);
+    else errors.push('runtime must be an object');
+  }
   return errors;
 }
 

package/src/index.mjs

@@ -5,6 +5,8 @@
 export { runContrast } from './contrast.mjs';
 export { runTreatments } from './treatments.mjs';
 export { runJudge, buildPrompt, checklistLines, extractJson } from './judge.mjs';
+export { runRuntime } from './runtime.mjs';
+export { connect, findBrowser } from './cdp.mjs';
 export { load, validate } from './config.mjs';
 export { toText, toJson, failed } from './report.mjs';
 export { parseColor, contrastRatio, composite, luminance } from './color.mjs';
@@ -14,6 +16,7 @@ export { openTags, classesOf } from './treatments.mjs';
 import { runContrast } from './contrast.mjs';
 import { runTreatments } from './treatments.mjs';
 import { runJudge } from './judge.mjs';
+import { runRuntime } from './runtime.mjs';
 
 /**
  * Run the deterministic checks a config asks for. `only` narrows to one by
@@ -38,3 +41,12 @@ export function run(config, cwd, { only = null } = {}) {
 export function judge(config, cwd) {
   return [runJudge(config.judge, cwd)];
 }
+
+/**
+ * Measure a rendered page. Separate from `run` because it needs a browser and
+ * a server that is already up, which is a heavier precondition than reading
+ * files off disk, and not one to impose on the check people put in a hook.
+ */
+export async function runtime(config, cwd) {
+  return [await runRuntime(config.runtime, cwd)];
+}

package/src/report.mjs

@@ -52,7 +52,7 @@ function judgeLines(result) {
 }
 
 export const linesFor = (result) => {
-  if (result.name === 'contrast') return contrastLines(result);
+  if (result.name === 'contrast' || result.name === 'runtime') return contrastLines(result);
   if (result.name === 'judge') return judgeLines(result);
   return treatmentLines(result);
 };
@@ -79,7 +79,9 @@ export function toText(results) {
       out.push(`${result.name} ok, ${result.summary}`);
       // A clean contrast run still shows its margins. Nothing else in the
       // report tells you which pair is one nudge away from failing.
-      if (result.name === 'contrast') for (const l of lines) out.push(`  ${MARK.ok}${l.text}`);
+      if (result.name === 'contrast' || result.name === 'runtime') {
+        for (const l of lines) out.push(`  ${MARK.ok}${l.text}`);
+      }
       out.push('');
       continue;
     }

package/src/runtime.mjs

@@ -0,0 +1,161 @@
+/**
+ * Contrast measured off a rendered page.
+ *
+ * This is the check the contrast module is a static approximation of. A token
+ * file says what a colour is declared to be. A rendered page says what is
+ * actually painted, which is a different question the moment anything is
+ * translucent, inherited, set by a component, or overridden by a state.
+ *
+ * Two things it does that reading tokens cannot:
+ *
+ *   1. It composites the whole background stack, nearest layer first, rather
+ *      than measuring against the first opaque ancestor. White text on a dark
+ *      scrim over a light page reads 1.04:1 if you skip the scrim and 10.57:1
+ *      if you do not. One of those is a false failure, and the same shortcut
+ *      produces a false pass with the colours the other way round.
+ *
+ *   2. It can measure a state, because it can put the page into one first.
+ *      A theme, an opened panel, a focused input.
+ *
+ * A selector that matches nothing is a failure, not a skipped target. So is a
+ * border with no width: you asked for the contrast of an edge that is not
+ * being drawn, and the honest answer is that the question is wrong.
+ */
+import { contrastRatio, flatten, isOpaque, parseColor } from './color.mjs';
+import { connect } from './cdp.mjs';
+
+/**
+ * Runs in the page. Returns the foreground and the stack of backgrounds behind
+ * it, and leaves every judgement to the caller so that the reasoning lives in
+ * one place rather than half here and half in a string.
+ */
+const MEASURE = `(targets) => targets.map(({ selector, prop, againstParent }) => {
+  const el = document.querySelector(selector);
+  if (!el) return { error: 'matched no element' };
+  if (!el.getClientRects().length) return { error: 'matched an element that is not rendered' };
+  const cs = getComputedStyle(el);
+  const fg = cs[prop];
+  if (!fg) return { error: 'has no ' + prop };
+  if (prop.startsWith('border')) {
+    const side = prop.replace(/^border|Color$/g, '');
+    const width = parseFloat(cs['border' + (side || 'Top') + 'Width']);
+    if (!width) return { error: 'has a ' + prop + ' but no width, so no edge is drawn' };
+  }
+  const layers = [];
+  let node = againstParent ? el.parentElement : el;
+  if (!node) return { error: 'has no parent to measure against' };
+  while (node) {
+    const bg = getComputedStyle(node).backgroundColor;
+    const m = bg && bg.match(/rgba?\\(([^)]+)\\)/);
+    if (m) {
+      const p = m[1].split(/[,\\/]/).map((v) => parseFloat(v.trim()));
+      const alpha = p.length > 3 ? p[3] : 1;
+      if (alpha > 0) {
+        layers.push(bg);
+        if (alpha > 0.999) break;
+      }
+    }
+    node = node.parentElement;
+  }
+  const root = getComputedStyle(document.documentElement).backgroundColor;
+  return { fg, layers, root, text: (el.textContent || '').trim().slice(0, 40) };
+})`;
+
+export async function runRuntime(config, cwd, { connect: open = connect } = {}) {
+  const samples = [];
+  const problems = [];
+  const { url, endpoint, browserPath, timeout, states = [{ name: 'default' }], targets } = config;
+
+  let page;
+  try {
+    page = await open({ endpoint, browserPath, timeout });
+  } catch (error) {
+    problems.push(error.message);
+    return { name: 'runtime', samples, problems, summary: '' };
+  }
+
+  try {
+    for (const state of states) {
+      // `before` runs on every navigation ahead of the page's own scripts, so
+      // a theme read at boot sees it. Running it after load and reloading
+      // would undo any DOM change; running it after load without reloading is
+      // too late for anything the page reads once. It is removed again after
+      // the state is measured, or the next state inherits it.
+      let initScript = null;
+      if (state.before) initScript = await page.onNewDocument(state.before);
+
+      await page.goto(url);
+      // `after` is the other half: opening a panel, focusing a field, anything
+      // that only exists once there is a document to act on.
+      if (state.after) await page.evaluate(state.after);
+      if (state.waitFor) {
+        await page.evaluate(
+          `new Promise((resolve, reject) => {
+            const done = () => document.querySelector(${JSON.stringify(state.waitFor)});
+            if (done()) return resolve(true);
+            const t = setInterval(() => { if (done()) { clearInterval(t); resolve(true); } }, 50);
+            setTimeout(() => { clearInterval(t); reject(new Error('waitFor ${state.waitFor} never appeared')); }, 10000);
+          })`,
+        );
+      }
+
+      const measured = await page.evaluate(`(${MEASURE})(${JSON.stringify(targets)})`);
+
+      targets.forEach((target, i) => {
+        const found = measured[i];
+        const where = `${target.selector} { ${target.prop} }`;
+        if (found.error) {
+          problems.push(`state "${state.name}": ${where} ${found.error}`);
+          return;
+        }
+        const fg = parseColor(found.fg);
+        if (!fg.ok) {
+          problems.push(`state "${state.name}": ${where} ${fg.reason}`);
+          return;
+        }
+        const layers = [];
+        for (const raw of found.layers) {
+          const parsed = parseColor(raw);
+          if (!parsed.ok) {
+            problems.push(`state "${state.name}": ${where} background ${parsed.reason}`);
+            return;
+          }
+          layers.push(parsed.rgba);
+        }
+        // Nothing opaque anywhere up the tree means the canvas is showing
+        // through, and the canvas is whatever the root paints.
+        if (!layers.length || !isOpaque(layers[layers.length - 1])) {
+          const root = parseColor(found.root);
+          layers.push(root.ok && isOpaque(root.rgba) ? root.rgba : [255, 255, 255, 1]);
+        }
+
+        const ground = flatten(layers);
+        const ratio = contrastRatio(fg.rgba, ground);
+        samples.push({
+          theme: state.name,
+          fg: target.selector,
+          bg: `${target.prop}, ${layers.length} ${layers.length === 1 ? 'layer' : 'layers'}`,
+          note: target.label ?? found.text ?? '',
+          ratio,
+          min: target.min,
+          pass: ratio >= target.min,
+        });
+      });
+
+      if (initScript) await page.removeNewDocumentScript(initScript);
+    }
+  } catch (error) {
+    problems.push(`the page could not be measured: ${error.message}`);
+  } finally {
+    await page.close();
+  }
+
+  return {
+    name: 'runtime',
+    samples,
+    problems,
+    summary: `${samples.length} ${samples.length === 1 ? 'target' : 'targets'} across ${
+      states.length
+    } ${states.length === 1 ? 'state' : 'states'}`,
+  };
+}