@josueavalosjim/taste-check 0.10.0 → 0.12.0

package/README.md

@@ -3,9 +3,9 @@
 Design review in CI, with a line down the middle of it.
 
 Some of design review is measurable. Contrast ratios, values that should have
-been tokens, class names nobody approved. taste-check measures those and fails
-your build on them, either from your token file or from a page it renders in a
-real browser.
+been tokens, tokens that were never real, class names nobody approved.
+taste-check measures those and fails your build on them, either from your token
+file or from a page it renders in a real browser.
 
 The rest is judgment, and it cannot be measured. For that it sends your
 screenshots and your checklist to a model, and reports what comes back as an
@@ -32,6 +32,9 @@ contrast ok, 10 pairs across 2 themes
 treatments FAILED
   FAIL  src/Promo.jsx:7 class "promo-huge" on <div> is not approved
   FAIL  src/Promo.jsx:8 inline value "#ff0055" on <a> is a one-off. Use a token, or add it to approvedValues.
+
+tokens FAILED
+  FAIL  src/Promo.jsx:8 "--color-primary-500" is referenced here but declared in no token file. Check the name, or add it to allow.
 ```
 
 ```bash
@@ -72,7 +75,9 @@ in your repo: what runs is the copy you edited, never its own.
 Each of these exits 1 rather than passing quietly:
 
 - a pair naming a token that does not exist
+- markup referencing a token that is declared nowhere
 - a file pattern matching no files
+- a token file that declares no custom properties at all
 - a theme whose scopes resolve no tokens
 - a single scope inside a theme that selects nothing, even when the theme has
   tokens from its other scopes
@@ -182,6 +187,55 @@ direction to be wrong in.
 Template literal holes are read into rather than blanked, so a class written
 inside `` `card ${on ? 'card--on' : ''}` `` is seen.
 
+## The tokens check
+
+Give it the stylesheet that declares your tokens and the markup that uses them.
+It reports every `var(--x)` in your markup where `--x` is declared nowhere.
+
+```json
+{
+  "tokens": {
+    "declaredIn": ["src/styles/tokens.css"],
+    "files": ["src/**/*.jsx"],
+    "allow": ["--scroll-progress"],
+    "allowPrefixes": ["--radix-"]
+  }
+}
+```
+
+The failure it is built for is a token name that was never real:
+`var(--color-primary-500)` in a system that says `--brand-action-bg`. A name
+like that is not a typo. It is what gets written when the code is produced
+without the token file open, which is most of what a model does, and it looks
+correct in review because it looks like a token.
+
+**It scans markup, not stylesheets, and that is deliberate.** See
+[where it sits](#where-it-sits-next-to-other-tools). Stylelint already owns the
+CSS side and does it better.
+
+For markup the scan is the whole file rather than the tags, because `var(--`
+means one thing everywhere it appears. So a reference in an object declared
+above the `return`, in a styled-components template, or in a constant in
+another module is found, and nesting comes free: both names in
+`var(--a, var(--b))` are checked, not only the outer one.
+
+`allow` and `allowPrefixes` are the escape hatch, and you will need them. A
+custom property set from JavaScript with `setProperty`, or one owned by a
+library's stylesheet, is legitimately absent from your token file.
+
+Two things it does not treat as failures. A bare `--x` written where you meant
+`var(--x)` is stylelint's `custom-property-no-missing-var-function`, not this.
+And a run that finds no references at all is reported in the summary rather
+than failed, because a codebase that keeps its styling in stylesheets has none
+and is not broken:
+
+```
+tokens ok, 1 token file, 84 declared, 41 files scanned, 0 references
+```
+
+If that zero is a surprise, your `files` pattern is pointed at the wrong tree.
+A pattern matching no files at all is a failure, as everywhere else here.
+
 ## Measuring the rendered page
 
 The contrast check above reads your token file, which tells you what a colour
@@ -416,6 +470,10 @@ it is the way it is.
 | `treatments.approvedClasses` | Every class allowed to appear. |
 | `treatments.allowPrefixes` | Prefixes that are always allowed, as a deliberate escape hatch. |
 | `treatments.approvedValues` | Literal values allowed inside inline styles. |
+| `tokens.declaredIn` | Stylesheets declaring the tokens. Every custom property in them counts, in any scope. |
+| `tokens.files` | Markup to scan for `var()` references. |
+| `tokens.allow` | Names that need no declaration, for ones set from JavaScript. Must start with `--`. |
+| `tokens.allowPrefixes` | Name prefixes that are always allowed, for a family a library owns. |
 | `judge.checklist` | Your checklist file. List items are judged, prose is not. |
 | `judge.shots` | Screenshots to hand the judge. Matching nothing is a failure. |
 | `judge.shotCommand` | Optional command run first to produce those screenshots. |
@@ -433,7 +491,7 @@ taste-check runtime [options] Measure contrast on a rendered page
 taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
 
   -c, --config <path>   Config file (default: tastecheck.config.json)
-      --only <name>     Run one check: contrast or treatments
+      --only <name>     Run one check: contrast, treatments or tokens
       --format <kind>   text (default), json, or sarif
       --json            Alias for --format json
       --version         Print the version
@@ -493,6 +551,16 @@ tool, run axe.
 **stylelint, eslint.** General code quality, with an enormous rule ecosystem.
 Nothing here replaces them.
 
+**stylelint for tokens in CSS specifically.** Its core
+`no-unknown-custom-properties` reports a `var()` naming a custom property
+nothing declares, and its `referenceFiles` option covers the case that matters,
+where the tokens live in one file and the uses live in another. That is the
+same check taste-check runs, done first-party, so run stylelint on your
+stylesheets. taste-check only scans markup, where stylelint does not go: its
+CSS-in-JS syntax is deprecated, and a `style={{ color: open ? 'var(--a)' :
+'var(--b)' }}` prop is a JavaScript object literal that no CSS parser reads at
+all.
+
 **@lapidist/design-lint.** More thorough than taste-check on the token and
 component side: it parses properly rather than scanning, knows about
 frameworks, autofixes, and manages deprecations. If enforcing tokens in code is
@@ -536,6 +604,11 @@ within one channel unit.
 them. If your tokens rely on `.a.b` beating `.b`, list the scopes in the order
 you want.
 
+**It does not lint your stylesheets.** The tokens check reads your CSS to learn
+what is declared and then never looks at it again. A `var()` in a `.css` file
+naming nothing is not reported here. That is stylelint's
+`no-unknown-custom-properties`, named above.
+
 **Component indirection is invisible.** Classes applied by a helper
 function, a `clsx` call importing names from elsewhere, or CSS-in-JS are
 invisible to it.
@@ -564,10 +637,15 @@ publisher over OIDC, so no npm token is stored in the repository and every
 release carries a provenance attestation. For a minor or major bump, run
 `npm version minor` or `npm version major` and `git push --follow-tags`.
 
-34 tests. Most of them plant a violation into a fixture that was passing a
+183 tests. Most of them plant a violation into a fixture that was passing a
 moment earlier and demand it gets caught: a token darkened below its floor, an
 unapproved class added to a clean file, a class buried in a template literal
-hole, a file pattern pointed at nothing.
+hole, a fabricated token hidden in a ternary, a file pattern pointed at
+nothing.
+
+The count is not the point and is easy to inflate. What the suite is actually
+worth is measured by breaking the source on purpose and checking the right
+test goes red, which is where the 0.12.0 fixes came from.
 
 ## License
 

package/bin/taste-check.mjs

@@ -28,7 +28,7 @@ const USAGE = `taste-check
 
 Options:
   -c, --config <path>   Config file (default: tastecheck.config.json)
-      --only <name>     Run one check: contrast or treatments
+      --only <name>     Run one check: contrast, treatments or tokens
       --new [path]      checklist only: scaffold a starter, refusing to
                         overwrite one that exists
       --lint            checklist only: report lines a judge cannot answer
@@ -112,8 +112,8 @@ function parseArgs(argv) {
     else if (arg === '-c' || arg === '--config') options.config = next();
     else if (arg === '--only') {
       options.only = next();
-      if (options.only !== 'contrast' && options.only !== 'treatments') {
-        throw new Error(`--only takes "contrast" or "treatments", not "${options.only}"`);
+      if (!['contrast', 'treatments', 'tokens'].includes(options.only)) {
+        throw new Error(`--only takes "contrast", "treatments" or "tokens", not "${options.only}"`);
       }
     } else if (arg === '--json') options.format = 'json';
     else if (arg === '--lint') options.lint = true;
@@ -268,7 +268,7 @@ else if (options.command === 'runtime') results = await runtime(loaded.config, l
 else results = run(loaded.config, loaded.dir, { only: options.only });
 
 if (!results.length) {
-  die(`nothing to run. ${options.config} defines no ${options.only ?? 'contrast or treatments'} check.`);
+  die(`nothing to run. ${options.config} defines no ${options.only ?? 'contrast, treatments or tokens'} check.`);
 }
 
 if (options.format === 'sarif') {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@josueavalosjim/taste-check",
-  "version": "0.10.0",
+  "version": "0.12.0",
   "description": "Design review in CI with a line down the middle: measured checks that gate the build (WCAG contrast from your tokens or from a real rendered page, one-off values in your markup) and a fresh-eyes model judge whose verdicts stay advisory. Zero dependencies. Ships no design rules of its own.",
   "keywords": [
     "accessibility",

package/schema/config.schema.json

@@ -17,6 +17,11 @@
         "treatments"
       ]
     },
+    {
+      "required": [
+        "tokens"
+      ]
+    },
     {
       "required": [
         "runtime"
@@ -260,6 +265,62 @@
         }
       }
     },
+    "tokens": {
+      "type": "object",
+      "description": "Custom properties referenced by var() in markup, against the tokens the system declares.",
+      "additionalProperties": false,
+      "required": [
+        "declaredIn",
+        "files"
+      ],
+      "properties": {
+        "declaredIn": {
+          "type": "array",
+          "minItems": 1,
+          "description": "Stylesheets that declare the tokens. Every custom property declared anywhere in them counts, in any scope or at-rule.",
+          "items": {
+            "type": "string"
+          }
+        },
+        "files": {
+          "type": "array",
+          "minItems": 1,
+          "description": "Markup to scan for var() references. A pattern matching nothing is a failure, never a clean run.",
+          "items": {
+            "type": "string"
+          }
+        },
+        "allow": {
+          "type": "array",
+          "description": "Custom property names that are fine to reference without being declared, for ones set from JavaScript or by a third-party stylesheet.",
+          "items": {
+            "type": "string",
+            "pattern": "^--"
+          }
+        },
+        "allowPrefixes": {
+          "type": "array",
+          "description": "Name prefixes that are always allowed, for a whole family a library owns.",
+          "items": {
+            "type": "string"
+          }
+        },
+        "$comment": {
+          "oneOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "array",
+              "items": {
+                "type": "string"
+              }
+            }
+          ],
+          "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
+        }
+      }
+    },
     "judge": {
       "type": "object",
       "description": "A fresh-eyes review of your screenshots against your checklist, run by a model command you supply. Run with `taste-check judge`. Verdicts are advisory unless failOn says otherwise.",

package/src/config.mjs

@@ -151,6 +151,19 @@ function validateTreatments(treatments, errors) {
   }
 }
 
+function validateTokens(tokens, errors) {
+  rejectUnknown(tokens, ['declaredIn', 'files', 'allow', 'allowPrefixes'], 'tokens', errors);
+  stringArray(tokens.declaredIn, 'tokens.declaredIn', errors);
+  stringArray(tokens.files, 'tokens.files', errors);
+  stringArray(tokens.allow, 'tokens.allow', errors, { required: false });
+  stringArray(tokens.allowPrefixes, 'tokens.allowPrefixes', errors, { required: false });
+  for (const name of tokens.allow ?? []) {
+    if (typeof name === 'string' && !name.startsWith('--')) {
+      errors.push(`tokens.allow has "${name}", which is not a custom property name. Names start with "--".`);
+    }
+  }
+}
+
 function validateJudge(judge, errors) {
   rejectUnknown(judge, ['checklist', 'shots', 'shotCommand', 'command', 'failOn'], 'judge', errors);
   if (typeof judge.checklist !== 'string' || !judge.checklist) {
@@ -246,15 +259,18 @@ function validateRuntime(runtime, errors) {
 export function validate(config) {
   const errors = [];
   if (!isPlainObject(config)) return ['the config must be a JSON object'];
-  rejectUnknown(config, ['$schema', 'contrast', 'treatments', 'judge', 'runtime'], 'the config', errors);
+  rejectUnknown(config, ['$schema', 'contrast', 'treatments', 'tokens', 'judge', 'runtime'], 'the config', errors);
 
   if (
     config.contrast === undefined &&
     config.treatments === undefined &&
+    config.tokens === undefined &&
     config.judge === undefined &&
     config.runtime === undefined
   ) {
-    errors.push('the config must define at least one of "contrast", "treatments", "runtime" or "judge"');
+    errors.push(
+      'the config must define at least one of "contrast", "treatments", "tokens", "runtime" or "judge"',
+    );
   }
   if (config.contrast !== undefined) {
     if (isPlainObject(config.contrast)) validateContrast(config.contrast, errors);
@@ -264,6 +280,10 @@ export function validate(config) {
     if (isPlainObject(config.treatments)) validateTreatments(config.treatments, errors);
     else errors.push('treatments must be an object');
   }
+  if (config.tokens !== undefined) {
+    if (isPlainObject(config.tokens)) validateTokens(config.tokens, errors);
+    else errors.push('tokens must be an object');
+  }
   if (config.judge !== undefined) {
     if (isPlainObject(config.judge)) validateJudge(config.judge, errors);
     else errors.push('judge must be an object');

package/src/contrast.mjs

@@ -19,7 +19,7 @@
 import { readFileSync } from 'node:fs';
 import { contrastRatio, isOpaque, parseColor } from './color.mjs';
 import { lineAt, parseDeclarations, resolveScopes, resolveValue, unmatchedScopes } from './css.mjs';
-import { expand, label } from './files.mjs';
+import { expandEach, label } from './files.mjs';
 
 /** A token name, or a literal colour, resolved to rgba for one theme. */
 function side(spec, table, theme) {
@@ -40,9 +40,14 @@ export function runContrast(config, cwd) {
   const problems = [];
   const { tokens, themes, pairs } = config;
 
-  const files = expand(tokens, cwd);
+  const { files, empty: deadPatterns } = expandEach(tokens, cwd);
+  // Per pattern. A dead one beside a live one leaves the union non-empty, so
+  // a renamed token file stops being read without anything saying so.
+  for (const p of deadPatterns) problems.push(`no token files matched "${p}"`);
   if (!files.length) {
-    problems.push(`no token files matched ${tokens.map((t) => `"${t}"`).join(', ')}`);
+    if (!deadPatterns.length) {
+      problems.push(`no token files matched ${tokens.map((t) => `"${t}"`).join(', ')}`);
+    }
     return { name: 'contrast', samples, problems, summary: '' };
   }
 

package/src/css.mjs

@@ -39,11 +39,6 @@
  */
 const CONDITIONAL = /^@(media|supports|container|scope|document)\b/;
 
-/** Blank out comments, keeping every offset and newline so lines stay true. */
-function blankComments(css) {
-  return css.replace(/\/\*[\s\S]*?\*\//g, (m) => m.replace(/[^\n]/g, ' '));
-}
-
 const normalize = (selector) => selector.replace(/\s+/g, ' ').replace(/'/g, '"').trim();
 
 /**
@@ -51,16 +46,25 @@ const normalize = (selector) => selector.replace(/\s+/g, ' ').replace(/'/g, '"')
  * under and the at-rules it is nested inside.
  */
 export function parseDeclarations(css) {
-  const source = blankComments(css);
+  const source = css;
   const decls = [];
   const stack = [];
   let buffer = '';
   let bufferStart = 0;
+  /* Offset of the first non-whitespace character in the buffer, recorded as
+     it is read rather than derived afterwards by trimming. Deriving it worked
+     only while comments were blanked to spaces up front: once they are stepped
+     over instead, the source still holds the comment text, so trimming stops
+     at the "/" and every declaration under a comment reported the comment's
+     line as its own. */
+  let ink = -1;
   let quote = null;
 
-  const flush = (end) => {
+  const flush = () => {
     const text = buffer.trim();
     buffer = '';
+    const at = ink;
+    ink = -1;
     if (!text) return;
     const colon = text.indexOf(':');
     if (colon === -1) return;
@@ -77,13 +81,14 @@ export function parseDeclarations(css) {
       value,
       selector,
       atRules: stack.filter((s) => s.startsWith('@')),
-      index: bufferStart + (source.slice(bufferStart, end).length - source.slice(bufferStart, end).trimStart().length),
+      index: at === -1 ? bufferStart : at,
     });
   };
 
   for (let i = 0; i < source.length; i += 1) {
     const c = source[i];
     if (quote) {
+      if (ink === -1) ink = i;
       buffer += c;
       if (c === '\\') {
         buffer += source[i + 1] ?? '';
@@ -91,29 +96,44 @@ export function parseDeclarations(css) {
       } else if (c === quote) quote = null;
       continue;
     }
+    /* Comments are skipped here rather than blanked before the walk.
+       Blanking first cannot see quotes, so a "/*" inside one string value and
+       a "*\/" inside another erased every declaration between them. A theme
+       override lost that way does not fail: resolveScopes falls back to the
+       base value and the dark theme gets measured against light numbers, which
+       is the exact miss this file exists to catch. Offsets stay true because
+       nothing is rewritten, only stepped over. */
+    if (c === '/' && source[i + 1] === '*') {
+      const end = source.indexOf('*\/', i + 2);
+      i = end === -1 ? source.length : end + 1;
+      continue;
+    }
     if (c === '"' || c === "'") {
       quote = c;
+      if (ink === -1) ink = i;
       buffer += c;
       continue;
     }
     if (c === '{') {
       stack.push(buffer.trim().replace(/\s+/g, ' '));
       buffer = '';
+      ink = -1;
       bufferStart = i + 1;
       continue;
     }
     if (c === '}') {
-      flush(i);
+      flush();
       stack.pop();
       bufferStart = i + 1;
       continue;
     }
     if (c === ';') {
-      flush(i);
+      flush();
       bufferStart = i + 1;
       continue;
     }
     if (!buffer) bufferStart = i;
+    if (ink === -1 && c.trim()) ink = i;
     buffer += c;
   }
   return decls;

package/src/files.mjs

@@ -56,25 +56,44 @@ function walk(dir, out) {
 }
 
 /**
- * Absolute paths matching the patterns, relative to `cwd`, sorted and
- * de-duplicated so a run is reproducible and a report is diffable.
+ * Absolute paths matching the patterns, plus the patterns that matched
+ * nothing.
+ *
+ * The misses are returned rather than inferred from an empty result, because
+ * a list is checked as a union. Two patterns, one naming src and one naming
+ * components, with components renamed: the src half still returns every file
+ * it matched, so the union is not empty and that whole second tree silently
+ * stops being checked. This file's own header promises a typo is reported
+ * where it happened, and only a per-pattern answer keeps that promise.
  */
-export function expand(patterns, cwd) {
+export function expandEach(patterns, cwd) {
   const found = new Set();
+  const empty = [];
   for (const pattern of patterns) {
+    const before = found.size;
     const absolute = isAbsolute(pattern) ? pattern : resolve(cwd, pattern);
     if (!/[*?]/.test(pattern)) {
       if (existsSync(absolute) && statSync(absolute).isFile()) found.add(absolute);
+      else empty.push(pattern);
       continue;
     }
     const base = resolve(cwd, baseOf(pattern));
-    if (!existsSync(base) || !statSync(base).isDirectory()) continue;
+    if (!existsSync(base) || !statSync(base).isDirectory()) {
+      empty.push(pattern);
+      continue;
+    }
     const test = toRegExp(isAbsolute(pattern) ? pattern : resolve(cwd, pattern).split(sep).join('/'));
     for (const file of walk(base, [])) {
       if (test.test(file.split(sep).join('/'))) found.add(file);
     }
+    if (found.size === before) empty.push(pattern);
   }
-  return [...found].sort();
+  return { files: [...found].sort(), empty };
+}
+
+/** Just the paths, for callers with nothing useful to say about a miss. */
+export function expand(patterns, cwd) {
+  return expandEach(patterns, cwd).files;
 }
 
 /** A path as you would paste it into an editor. */

package/src/index.mjs

@@ -4,6 +4,7 @@
  */
 export { runContrast } from './contrast.mjs';
 export { runTreatments } from './treatments.mjs';
+export { runTokens, varRefs } from './tokens.mjs';
 export { runJudge, prepareJudge, gradeVerdict, buildPrompt, checklistLines, checklistEntries, extractJson } from './judge.mjs';
 export { runRuntime } from './runtime.mjs';
 export { connect, findBrowser } from './cdp.mjs';
@@ -15,6 +16,7 @@ export { openTags, classesOf } from './treatments.mjs';
 
 import { runContrast } from './contrast.mjs';
 import { runTreatments } from './treatments.mjs';
+import { runTokens } from './tokens.mjs';
 import { runJudge } from './judge.mjs';
 import { runRuntime } from './runtime.mjs';
 
@@ -34,6 +36,9 @@ export function run(config, cwd, { only = null } = {}) {
   if (config.treatments && (!only || only === 'treatments')) {
     results.push(runTreatments(config.treatments, cwd));
   }
+  if (config.tokens && (!only || only === 'tokens')) {
+    results.push(runTokens(config.tokens, cwd));
+  }
   return results;
 }
 

package/src/judge.mjs

@@ -41,7 +41,7 @@
 import { execFileSync } from 'node:child_process';
 import { readFileSync } from 'node:fs';
 import { resolve } from 'node:path';
-import { expand, label } from './files.mjs';
+import { expandEach, label } from './files.mjs';
 
 /**
  * The instructions wrapped around the user's checklist.
@@ -182,7 +182,21 @@ export function prepareJudge(config, cwd) {
     if (!made.ok) return { ok: false, result: empty([made.reason], failOn) };
   }
 
-  const images = expand(shots, cwd);
+  const { files: images, empty: deadShots } = expandEach(shots, cwd);
+  /* A dead shot pattern is the same failure as no shots at all, one screen at
+     a time: the judge answers about the frames it was given and says nothing
+     about the one nobody handed it. */
+  if (deadShots.length) {
+    return {
+      ok: false,
+      result: empty(
+        deadShots.map(
+          (p) => `no screenshots matched "${p}". A screen the judge never saw cannot pass.`,
+        ),
+        failOn,
+      ),
+    };
+  }
   if (!images.length) {
     return {
       ok: false,

package/src/report.mjs

@@ -54,6 +54,10 @@ function judgeLines(result) {
 export const linesFor = (result) => {
   if (result.name === 'contrast' || result.name === 'runtime') return contrastLines(result);
   if (result.name === 'judge') return judgeLines(result);
+  // tokens and treatments both report {file, line, message} failures, so they
+  // render the same way. Named rather than left to the fallthrough: a check
+  // that renders by accident is a check that stops rendering by accident.
+  if (result.name === 'tokens') return treatmentLines(result);
   return treatmentLines(result);
 };
 

package/src/runtime.mjs

@@ -38,11 +38,19 @@ const MEASURE = `(targets) => targets.map(({ selector, prop, againstParent }) =>
   if (!el) return { error: 'matched no element' };
   if (!el.getClientRects().length) return { error: 'matched an element that is not rendered' };
   const cs = getComputedStyle(el);
-  const fg = cs[prop];
+  // Read either spelling. A config saying border-top-color is the natural
+  // thing to write and the validator accepts it, so it must not fall through
+  // to an undefined lookup.
+  const fg = prop.includes('-') ? cs.getPropertyValue(prop) : cs[prop];
   if (!fg) return { error: 'has no ' + prop };
-  if (prop.startsWith('border')) {
-    const side = prop.replace(/^border|Color$/g, '');
-    const width = parseFloat(cs['border' + (side || 'Top') + 'Width']);
+  if (/^border/i.test(prop)) {
+    /* Both spellings again. Stripping only the camelCase affixes turned
+       border-top-color into "-top-color" and then read borderTopWidth as
+       border-top-colorWidth, which is undefined, so every dash-case edge
+       reported "no width, so no edge is drawn" over a real border. */
+    const side = prop.replace(/^border-?/i, '').replace(/-?color$/i, '');
+    const dashed = side ? side.replace(/([A-Z])/g, '-$1').replace(/^-/, '').toLowerCase() : 'top';
+    const width = parseFloat(cs.getPropertyValue('border-' + dashed + '-width'));
     if (!width) return { error: 'has a ' + prop + ' but no width, so no edge is drawn' };
   }
   const layers = [];

package/src/sarif.mjs

@@ -32,6 +32,8 @@ const RULES = [
   ['treatments/unapproved-class', 'error', 'A class name that is not on the approved list.'],
   ['treatments/one-off-value', 'error', 'A literal colour or length hardcoded into an inline style.'],
   ['treatments/unscannable', 'error', 'No markup matched, so nothing was checked.'],
+  ['tokens/undefined', 'error', 'Markup references a custom property that no token file declares.'],
+  ['tokens/unscannable', 'error', 'The tokens check could not run: no token file, no markup, or a token file that declares nothing.'],
   ['runtime/below-floor', 'error', 'A target on the rendered page is under the ratio it was given.'],
   ['runtime/unmeasurable', 'error', 'A target could not be measured: no element, nothing rendered, an edge with no width, or no browser.'],
   ['judge/verdict', 'note', 'A checklist line the judge did not pass. Advisory: a model verdict is not reproducible.'],
@@ -118,7 +120,7 @@ function resultsFor(check, ctx) {
     return out;
   }
 
-  if (check.name === 'treatments') {
+  if (check.name === 'treatments' || check.name === 'tokens') {
     for (const failure of check.failures ?? []) {
       out.push({
         ruleId: failure.rule,
@@ -130,7 +132,7 @@ function resultsFor(check, ctx) {
     }
     for (const problem of check.problems ?? []) {
       out.push({
-        ruleId: 'treatments/unscannable',
+        ruleId: `${check.name}/unscannable`,
         key: problem,
         level: 'error',
         message: { text: problem },

package/src/tokens.mjs

@@ -0,0 +1,135 @@
+/**
+ * The undefined-token check: every `var(--x)` in markup, against the tokens
+ * the system actually declares.
+ *
+ * ── Why markup and not stylesheets ──────────────────────────────────────
+ *
+ * Stylelint's core `no-unknown-custom-properties` already does this for CSS,
+ * and its `referenceFiles` option covers the case that matters, where tokens
+ * are declared in one file and consumed in another. Rebuilding that would be
+ * a worse copy of a first-party rule.
+ *
+ * What nothing covers is markup. Stylelint's CSS-in-JS syntax is deprecated,
+ * and `style={{ color: open ? 'var(--a)' : 'var(--b)' }}` is not CSS-in-JS
+ * anyway: it is a JavaScript object literal that no CSS parser reads. That is
+ * the gap, and it is the one a model writing plausible off-system code falls
+ * into, reaching for `--color-primary-500` when the system says `--brand-ink`.
+ *
+ * ── Why a plain scan and not the tag walker ─────────────────────────────
+ *
+ * `treatments.mjs` walks tags because a class name is only a class name in an
+ * attribute, and a bare word elsewhere is noise. `var(--` is not like that.
+ * The string is unambiguous wherever it appears, so scanning the whole file
+ * costs nothing in false positives and picks up references the tag walker
+ * cannot see: a styled-components template, an object of styles declared above
+ * the return, a constant in a helper module.
+ *
+ * Nesting comes free. `var(--a, var(--b))` matches twice, so both names are
+ * checked rather than only the outer one.
+ */
+import { readFileSync } from 'node:fs';
+import { lineAt, parseDeclarations } from './css.mjs';
+import { expandEach, label } from './files.mjs';
+
+/**
+ * The name is `[\w-]+` for the same reason `resolveValue` in css.mjs uses it:
+ * escapes and non-ASCII in a custom property name are legal and vanishingly
+ * rare, and inventing a fuller grammar here would only disagree with the
+ * resolver next door.
+ */
+const VAR_REF = /var\(\s*(--[\w-]+)\s*(,?)/g;
+
+const count = (n, word) => `${n} ${word}${n === 1 ? '' : 's'}`;
+
+/**
+ * Every custom property referenced through `var()`, with the offset of the
+ * name and whether the reference carries a fallback.
+ *
+ * A bare `--x` written outside a `var()` is deliberately not a reference. That
+ * is a different mistake, and stylelint's `custom-property-no-missing-var-function`
+ * is the rule for it.
+ */
+export function varRefs(source) {
+  const found = [];
+  for (const m of source.matchAll(VAR_REF)) {
+    found.push({ name: m[1], index: m.index + m[0].indexOf(m[1]), fallback: m[2] === ',' });
+  }
+  return found;
+}
+
+/** Every custom property name declared anywhere in the token files. */
+function declaredNames(files) {
+  const names = new Set();
+  for (const file of files) {
+    for (const decl of parseDeclarations(readFileSync(file, 'utf8'))) names.add(decl.prop);
+  }
+  return names;
+}
+
+export function runTokens(config, cwd) {
+  const failures = [];
+  const problems = [];
+  const { declaredIn, files: patterns, allow = [], allowPrefixes = [] } = config;
+
+  const quoted = (list) => list.map((p) => `"${p}"`).join(', ');
+  const declared_ = expandEach(declaredIn, cwd);
+  const scanned_ = expandEach(patterns, cwd);
+  const tokenFiles = declared_.files;
+  const files = scanned_.files;
+
+  /* Per pattern, not per list. A list is a union, so one dead pattern beside
+     a live one leaves the union non-empty and that tree stops being checked
+     in silence. */
+  for (const p of declared_.empty) problems.push(`no token files matched "${p}"`);
+  for (const p of scanned_.empty) problems.push(`no markup files matched "${p}"`);
+  if (!tokenFiles.length && !declared_.empty.length) problems.push(`no token files matched ${quoted(declaredIn)}`);
+  const declared = declaredNames(tokenFiles);
+  if (tokenFiles.length && !declared.size) {
+    problems.push(
+      `${count(tokenFiles.length, 'token file')} matched ${quoted(declaredIn)}, ` +
+        `but none of them declares a custom property. Every reference would read as undefined.`,
+    );
+  }
+  if (!files.length && !scanned_.empty.length) problems.push(`no markup files matched ${quoted(patterns)}`);
+
+  // Without a trustworthy declared set there is nothing to compare against,
+  // and reporting every reference in the codebase as undefined would bury the
+  // one line that says why.
+  if (problems.length) return { name: 'tokens', failures, problems, summary: '' };
+
+  const allowed = new Set(allow);
+  let references = 0;
+
+  for (const file of files) {
+    const source = readFileSync(file, 'utf8');
+    for (const { name, index, fallback } of varRefs(source)) {
+      references += 1;
+      if (declared.has(name)) continue;
+      if (allowed.has(name)) continue;
+      if (allowPrefixes.some((p) => name.startsWith(p))) continue;
+      failures.push({
+        rule: 'tokens/undefined',
+        file: label(file, cwd),
+        line: lineAt(source, index),
+        subject: name,
+        message:
+          `"${name}" is referenced here but declared in no token file.` +
+          (fallback ? ' It has a fallback, so it degrades rather than breaks.' : '') +
+          ' Check the name, or add it to allow.',
+      });
+    }
+  }
+
+  // `references` is in the summary rather than a failure on purpose. A zero
+  // can mean a glob pointing at the wrong tree, but it can equally mean a
+  // codebase that keeps its styling in stylesheets, which is not a fault. The
+  // wrong-glob case that can be told apart is the one above: no files matched.
+  return {
+    name: 'tokens',
+    failures,
+    problems,
+    summary:
+      `${count(tokenFiles.length, 'token file')}, ${declared.size} declared, ` +
+      `${count(files.length, 'file')} scanned, ${count(references, 'reference')}`,
+  };
+}

package/src/treatments.mjs

@@ -20,7 +20,8 @@
  * positive is a conversation while a false negative is the bug shipping.
  */
 import { readFileSync } from 'node:fs';
-import { expand, label } from './files.mjs';
+import { lineAt } from './css.mjs';
+import { expandEach, label } from './files.mjs';
 
 /** Literal value shapes worth flagging when they are not on the list. */
 const VALUE_SHAPES = [
@@ -60,10 +61,18 @@ export function* openTags(source, names) {
   }
 }
 
-/** The value of `attr` when it is a plain string literal, else null. */
+/**
+ * The value of `attr` when it is a plain string literal, else null.
+ *
+ * Both quote styles. Matching only double quotes meant class='a b' and
+ * style='...' hit neither this nor the expression path, so the element was
+ * read as carrying no classes and no values: a file full of unapproved
+ * classes reported clean and exited 0. Single quotes are the house style in
+ * a lot of HTML, and `files` accepts .html, so this was not a corner.
+ */
 export function literalAttr(attrs, attr) {
-  const m = attrs.match(new RegExp(`(?:^|\\s)${attr}\\s*=\\s*"([^"]*)"`));
-  return m ? { text: m[1], at: m.index + m[0].indexOf('"') + 1 } : null;
+  const m = attrs.match(new RegExp(`(?:^|\\s)${attr}\\s*=\\s*(["'])([\\s\\S]*?)\\1`));
+  return m ? { text: m[2], at: m.index + m[0].indexOf(m[1]) + 1 } : null;
 }
 
 /** The braced expression for `attr`, brace-balanced, or null. */
@@ -178,8 +187,6 @@ function inlineValues(attrs) {
   return found;
 }
 
-const lineOf = (source, index) => source.slice(0, index).split('\n').length;
-
 export function runTreatments(config, cwd) {
   const failures = [];
   const problems = [];
@@ -191,9 +198,11 @@ export function runTreatments(config, cwd) {
     approvedValues = [],
   } = config;
 
-  const files = expand(patterns, cwd);
+  const { files, empty } = expandEach(patterns, cwd);
+  // Per pattern: a dead one beside a live one leaves the union non-empty.
+  for (const p of empty) problems.push(`no markup files matched "${p}"`);
   if (!files.length) {
-    problems.push(`no markup files matched ${patterns.map((p) => `"${p}"`).join(', ')}`);
+    if (!empty.length) problems.push(`no markup files matched ${patterns.map((p) => `"${p}"`).join(', ')}`);
     return { name: 'treatments', failures, problems, summary: '' };
   }
 
@@ -202,7 +211,7 @@ export function runTreatments(config, cwd) {
 
   for (const file of files) {
     const source = readFileSync(file, 'utf8');
-    const where = (at) => ({ file: label(file, cwd), line: lineOf(source, at) });
+    const where = (at) => ({ file: label(file, cwd), line: lineAt(source, at) });
 
     for (const tag of openTags(source, elements)) {
       for (const { name, at } of classesOf(tag.attrs)) {