@josueavalosjim/taste-check 0.8.0 → 0.10.0

package/README.md

@@ -63,8 +63,9 @@ which is fine for advice and disqualifying for a build gate. So the measured
 half here gates, the judged half does not, and the tool will not let you
 confuse one for the other by accident.
 
-It also ships no design rules of its own. Not a palette, not a class list, not
-a contrast floor, not a checklist. You supply all of it.
+It also enforces no rule it wrote. Not a palette, not a class list, not a
+contrast floor. It will scaffold you a starter checklist, and that file lands
+in your repo: what runs is the copy you edited, never its own.
 
 ## What counts as a failure
 
@@ -284,11 +285,13 @@ told what changed, answer every line, prefer unsure to a guess, do not be
 agreeable, and name the specific thing you are looking at. That part is method
 and it is the same for everyone.
 
-The checklist is yours. taste-check ships none, and there is a test asserting
-the framing mentions no design vocabulary at all, because a rule that arrives
-inside a tool is somebody else's taste with the tool's authority behind it.
-Checklist lines are list items in your file; a heading or a paragraph is prose
-and is not judged.
+The checklist is yours, and there is a test asserting the framing mentions no
+design vocabulary at all, because a rule that arrives at the moment of judgment
+carries the tool's authority rather than its author's. Checklist lines are list
+items in your file; a heading or a paragraph is prose and is not judged.
+
+Starting from nothing is its own problem, so there is a scaffold and a linter
+for the lines you write. Both are in the next section.
 
 ### Letting an agent carry the call
 
@@ -336,12 +339,71 @@ exits 1 whatever `failOn` says:
 Without that split, "the judge did not run" and "the judge found nothing" print
 the same thing.
 
+## Starting a checklist
+
+An empty file is the hardest part of the judge, so:
+
+```bash
+taste-check checklist --new
+```
+
+It writes a starter into your repo. Six lines, and every one is about whether
+the screen is broken rather than whether it is good, because that is the only
+kind of line that is not somebody's taste. It refuses to overwrite a file that
+exists, since the moment it lands it is yours.
+
+That is a scaffold and not a default, and the difference is the whole reason it
+is allowed to exist here. A default judges every user who never opened the file
+against opinions they did not choose. A scaffold is a file you edited and
+committed, and taste-check never reads its own copy. eslint is the same shape:
+it ships rules and no default configuration, and `--init` writes you one.
+
+```bash
+taste-check checklist --lint
+```
+
+This checks your lines and never their content. It has no opinion about what
+you ask for, only about whether asking it produces a verdict:
+
+```
+checklist, 3 to look at
+  unfalsifiable   design-checklist.md:12  "Does it feel premium and modern"
+                  "premium", "modern", "feel" describes a feeling rather than
+                  something visible. A judge answers unsure, every run.
+  not-in-a-still  design-checklist.md:13  "Is the hover animation smooth"
+                  "animation", "hover" cannot be seen in a screenshot.
+  measurable      design-checklist.md:14  "Body copy hits at least 4.5:1"
+                  this asks for a number. Put it in a contrast pair instead.
+```
+
+It also flags a line asking two things at once, since one verdict cannot answer
+both. "The primary action reads as the primary action" passes: it is an opinion,
+a strong one, and none of the linter's business, because it can be settled by
+looking.
+
 ## Config
 
 Point your editor at `schema/config.schema.json` for completion and inline
 docs. Paths inside a config resolve against the config file, so it can be run
 from anywhere.
 
+Unknown keys are rejected, because a misspelled key that silently does nothing
+is a config that looks like it is working. `$comment` is the exception: it is
+allowed wherever an object is, ignored, and takes a string or an array of them.
+A config you are expected to live with for years should let you write down why
+it is the way it is.
+
+```json
+{
+  "$comment": "Guards the ramp itself. The browser suite only covers tokens some selector reaches.",
+  "contrast": {
+    "pairs": [
+      { "$comment": "Documented at 6.13:1 in tokens.css.", "fg": "--ink-muted", "bg": "--bg", "min": 4.5 }
+    ]
+  }
+}
+```
+
 | Key | Meaning |
 | --- | --- |
 | `contrast.tokens` | CSS files holding the custom properties. Later files override earlier ones. |
@@ -438,8 +500,9 @@ the whole of your problem, it is the better fit.
 
 **Checklist Design and similar agent skills.** They arrive with a hundred or
 more published checklists and review conversationally. If you want good
-opinions supplied, take theirs. taste-check supplies none on purpose and runs
-in CI with an exit code instead.
+opinions supplied, take theirs, and they are good. taste-check scaffolds six
+lines about whether a screen is broken and expects the rest to be yours, which
+is more work and a different bargain.
 
 What is left, and the reason this exists: nothing above draws a line between
 the part that can gate a build and the part that cannot. The linters have no

package/bin/taste-check.mjs

@@ -7,10 +7,13 @@
  * exist, or scopes a pair to a theme that is not defined is a failure here,
  * not a quiet skip.
  */
-import { readFileSync } from 'node:fs';
+import { existsSync, readFileSync, writeFileSync } from 'node:fs';
+import { resolve } from 'node:path';
 import { fileURLToPath } from 'node:url';
 
 import { load } from '../src/config.mjs';
+import { STARTER, lintChecklist } from '../src/checklist.mjs';
+import { checklistEntries } from '../src/judge.mjs';
 import { gradeVerdict, judge, prepareJudge, run, runtime } from '../src/index.mjs';
 import { failed, toJson, toText } from '../src/report.mjs';
 import { toSarif } from '../src/sarif.mjs';
@@ -20,10 +23,15 @@ const USAGE = `taste-check
   taste-check [options]         Run the deterministic checks over your files
   taste-check runtime [options] Measure contrast on a rendered page
   taste-check judge [options]   Ask a fresh-eyes judge about your screenshots
+  taste-check checklist --new   Write a starter checklist you then own
+  taste-check checklist --lint  Check your checklist lines can be answered
 
 Options:
   -c, --config <path>   Config file (default: tastecheck.config.json)
       --only <name>     Run one check: contrast or treatments
+      --new [path]      checklist only: scaffold a starter, refusing to
+                        overwrite one that exists
+      --lint            checklist only: report lines a judge cannot answer
       --emit            judge only: print the prompt and stop, for an agent
                         to carry to a model itself
       --verdict <path>  judge only: grade a reply from a file, or - for stdin
@@ -48,6 +56,16 @@ is already up. It measures what is actually painted, compositing every
 background layer behind an element rather than stopping at the first opaque
 one, and it can put the page into a state first.
 
+checklist --new scaffolds; it does not default. The file it writes is yours,
+and taste-check never reads its own copy of it. The six lines in it are about
+whether a screen is broken rather than whether it is good, because that is the
+only kind of line that is not somebody's taste.
+
+checklist --lint is about the form of your lines and never their content. It
+reports a line a judge will answer "unsure" to every run, a line asking two
+things at once, a line about something a still image cannot show, and a line
+asking for a number that belongs in a contrast pair instead.
+
 An agent can carry the model call instead of a shell command. --emit prints
 the prompt and the images; the agent asks a fresh context and pipes the JSON
 back to --verdict -, which checks it against the checklist the same way. Run
@@ -68,10 +86,12 @@ function parseArgs(argv) {
     emit: false,
     verdict: null,
     skill: false,
+    lint: false,
+    scaffold: null,
   };
   // One positional, and only in first position, so a stray argument is an
   // error rather than something silently ignored.
-  if (argv[0] === 'judge' || argv[0] === 'runtime') {
+  if (argv[0] === 'judge' || argv[0] === 'runtime' || argv[0] === 'checklist') {
     options.command = argv[0];
     argv = argv.slice(1);
   }
@@ -96,7 +116,12 @@ function parseArgs(argv) {
         throw new Error(`--only takes "contrast" or "treatments", not "${options.only}"`);
       }
     } else if (arg === '--json') options.format = 'json';
-    else if (arg === '--emit') options.emit = true;
+    else if (arg === '--lint') options.lint = true;
+    else if (arg === '--new') {
+      // The path is optional, so peek rather than demanding a value.
+      const value = argv[i + 1];
+      options.scaffold = value !== undefined && !value.startsWith('-') ? (i += 1, value) : true;
+    } else if (arg === '--emit') options.emit = true;
     else if (arg === '--skill') options.skill = true;
     else if (arg === '--verdict') options.verdict = next();
     else if (arg === '--format') {
@@ -138,6 +163,26 @@ if (options.skill) {
   process.exit(0);
 }
 
+// checklist --new is the one command that has to work before a config exists,
+// because not having anything yet is the situation it is for.
+if (options.command === 'checklist' && options.scaffold) {
+  const named = typeof options.scaffold === 'string' ? options.scaffold : null;
+  const fromConfig = (() => {
+    const loadedForPath = load(options.config);
+    return loadedForPath.ok ? loadedForPath.config.judge?.checklist : null;
+  })();
+  const target = resolve(named ?? fromConfig ?? 'design-checklist.md');
+  if (existsSync(target)) {
+    die(`${target} already exists. It is yours now, so this will not overwrite it.`);
+  }
+  writeFileSync(target, STARTER);
+  const lines = checklistEntries(STARTER).length;
+  console.log(`wrote ${target}`);
+  console.log(`${lines} lines, all of them about whether a screen is broken. The ones worth`);
+  console.log('having are the ones you add. Run taste-check checklist --lint after editing.');
+  process.exit(0);
+}
+
 for (const flag of ['emit', 'verdict']) {
   if (options[flag] && options.command !== 'judge') {
     die(`--${flag} applies to \`taste-check judge\`, not to ${options.command}`);
@@ -155,11 +200,42 @@ if (!loaded.ok) {
 if (options.command !== 'check' && options.only) {
   die(`--only applies to the deterministic checks, not to ${options.command}`);
 }
+if (options.lint && options.command !== 'checklist') {
+  die('--lint applies to `taste-check checklist`, not to ' + options.command);
+}
 
-if (options.command !== 'check' && !loaded.config[options.command]) {
+// checklist is the exception: its file lives under judge.checklist rather than
+// in a block of its own, and it says so itself below.
+if (options.command !== 'check' && options.command !== 'checklist' && !loaded.config[options.command]) {
   die(`${options.config} defines no "${options.command}" block.`);
 }
 
+if (options.command === 'checklist') {
+  if (!options.lint) die('taste-check checklist takes --new or --lint');
+  const path = loaded.config.judge?.checklist;
+  if (!path) die(`${options.config} defines no judge.checklist to lint.`);
+  let entries;
+  try {
+    entries = checklistEntries(readFileSync(resolve(loaded.dir, path), 'utf8'));
+  } catch {
+    die(`cannot read the checklist at ${path}`);
+  }
+  if (!entries.length) {
+    die(`${path} has no checklist lines in it. Lines are list items ("- ..." or "1. ...").`);
+  }
+  const findings = lintChecklist(entries);
+  if (!findings.length) {
+    console.log(`checklist ok, ${entries.length} ${entries.length === 1 ? 'line' : 'lines'} a judge can answer`);
+    process.exit(0);
+  }
+  console.log(`checklist, ${findings.length} to look at`);
+  for (const f of findings) {
+    console.log(`  ${f.kind.padEnd(16)}${path}:${f.line}  ${JSON.stringify(f.text)}`);
+    console.log(`  ${' '.repeat(16)}${f.detail}`);
+  }
+  process.exit(1);
+}
+
 // --emit stops before any model is involved, so it has no findings to report
 // and no exit code to earn. It still refuses to hand back a prompt when the
 // screenshots or the checklist are missing.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@josueavalosjim/taste-check",
-  "version": "0.8.0",
+  "version": "0.10.0",
   "description": "Design review in CI with a line down the middle: measured checks that gate the build (WCAG contrast from your tokens or from a real rendered page, one-off values in your markup) and a fresh-eyes model judge whose verdicts stay advisory. Zero dependencies. Ships no design rules of its own.",
   "keywords": [
     "accessibility",

package/schema/config.schema.json

@@ -89,11 +89,39 @@
                         "atRule": {
                           "type": "string",
                           "description": "Opt into declarations nested in a matching at-rule prelude, for example \"prefers-color-scheme: dark\". Conditional at-rules (@media, @supports, @container, @scope) are ignored without this. Grouping at-rules like @layer are always transparent, so you only name one to narrow a scope to it."
+                        },
+                        "$comment": {
+                          "oneOf": [
+                            {
+                              "type": "string"
+                            },
+                            {
+                              "type": "array",
+                              "items": {
+                                "type": "string"
+                              }
+                            }
+                          ],
+                          "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
                         }
                       }
                     }
                   ]
                 }
+              },
+              "$comment": {
+                "oneOf": [
+                  {
+                    "type": "string"
+                  },
+                  {
+                    "type": "array",
+                    "items": {
+                      "type": "string"
+                    }
+                  }
+                ],
+                "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
               }
             }
           }
@@ -137,9 +165,37 @@
                 "items": {
                   "type": "string"
                 }
+              },
+              "$comment": {
+                "oneOf": [
+                  {
+                    "type": "string"
+                  },
+                  {
+                    "type": "array",
+                    "items": {
+                      "type": "string"
+                    }
+                  }
+                ],
+                "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
               }
             }
           }
+        },
+        "$comment": {
+          "oneOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "array",
+              "items": {
+                "type": "string"
+              }
+            }
+          ],
+          "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
         }
       }
     },
@@ -187,6 +243,20 @@
           "items": {
             "type": "string"
           }
+        },
+        "$comment": {
+          "oneOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "array",
+              "items": {
+                "type": "string"
+              }
+            }
+          ],
+          "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
         }
       }
     },
@@ -227,6 +297,20 @@
           ],
           "default": "never",
           "description": "Whether a \"fail\" verdict affects the exit code. Defaults to never: a model's verdict is not reproducible, so it does not gate a build unless you decide it should. Independent of this, a judge that could not run always exits 1."
+        },
+        "$comment": {
+          "oneOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "array",
+              "items": {
+                "type": "string"
+              }
+            }
+          ],
+          "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
         }
       }
     },
@@ -282,6 +366,20 @@
               "waitFor": {
                 "type": "string",
                 "description": "A selector to wait for before measuring."
+              },
+              "$comment": {
+                "oneOf": [
+                  {
+                    "type": "string"
+                  },
+                  {
+                    "type": "array",
+                    "items": {
+                      "type": "string"
+                    }
+                  }
+                ],
+                "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
               }
             }
           }
@@ -318,11 +416,53 @@
               "againstParent": {
                 "type": "boolean",
                 "description": "Measure against what is behind the element rather than its own background. Use when the thing being measured is a fill: measuring a fill against itself scores 1.00 and means nothing."
+              },
+              "$comment": {
+                "oneOf": [
+                  {
+                    "type": "string"
+                  },
+                  {
+                    "type": "array",
+                    "items": {
+                      "type": "string"
+                    }
+                  }
+                ],
+                "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
               }
             }
           }
+        },
+        "$comment": {
+          "oneOf": [
+            {
+              "type": "string"
+            },
+            {
+              "type": "array",
+              "items": {
+                "type": "string"
+              }
+            }
+          ],
+          "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
         }
       }
+    },
+    "$comment": {
+      "oneOf": [
+        {
+          "type": "string"
+        },
+        {
+          "type": "array",
+          "items": {
+            "type": "string"
+          }
+        }
+      ],
+      "description": "Ignored. Somewhere to write down why this config, or this pair, is the way it is."
     }
   }
 }

package/skills/taste-check-judge/SKILL.md

@@ -78,5 +78,11 @@ old screenshots is not a verdict on the new ones.
 ## What this skill does not contain
 
 Any design rules. The checklist is the user's file, named in their config, and
-if it is empty then this reports nothing and that is correct. A checklist that
-arrived with a tool is somebody else's taste wearing the tool's authority.
+if it is empty then this reports nothing and that is correct.
+
+`taste-check checklist --new` will scaffold a starter into their repo, six
+lines about whether a screen is broken rather than whether it is good. Once it
+is written it is theirs, and what gets judged is whatever they edited it into.
+Do not add lines to it yourself, and do not judge against a rule that is not in
+it. A rule that arrives at the moment of judgment, from the tool or from you,
+is somebody else's taste wearing borrowed authority.

package/src/checklist.mjs

@@ -0,0 +1,147 @@
+/**
+ * Starting a checklist, and checking that the lines in one can be answered.
+ *
+ * Neither of these says what to value. The starter contains only lines about
+ * whether a screen is broken, which is not taste, and the linter is about the
+ * form of a line rather than its content.
+ *
+ * ── Why a starter at all, given this ships no design rules ──────────────
+ *
+ * There is a real difference between a tool that defaults to a checklist and a
+ * tool that writes one into your repo once. A default arrives with the tool's
+ * authority at the moment of judgment: every user who never opened the file is
+ * being judged against opinions they did not choose. A scaffold is a file you
+ * own, edited and committed by you, and what runs is yours. The tool never
+ * reads its own copy.
+ *
+ * eslint is the precedent. It ships rules and no default configuration, and
+ * `--init` writes you one. Nobody thinks eslint has opinions about their code
+ * until they enable some.
+ */
+
+/**
+ * The starter.
+ *
+ * Six lines, and every one of them is about whether the screen is broken
+ * rather than whether it is good. That is deliberate and the header says so:
+ * these are the ones that are not anybody's taste, which means the lines worth
+ * having are the ones you add.
+ */
+export const STARTER = `# Design checklist
+
+Lines here are judged one at a time against a screenshot, and each gets one
+verdict: pass, fail, or unsure.
+
+Everything below is about whether the screen is broken. None of it is taste,
+which is the only reason it could ship inside a tool. The lines worth having
+are the ones you add, and they should be the specific things your work gets
+wrong, in your words.
+
+Two things make a line answerable. It has to be settleable by looking, so
+"nothing is cut off" works where "looks expensive" comes back unsure every
+run. And it has to ask one thing, because a line joining two claims cannot be
+answered by a single verdict.
+
+Anything you can measure belongs in a contrast pair or a runtime target
+instead, where the answer is a number rather than an opinion.
+
+Note that only list items are judged. Everything above this is prose and is
+ignored, so there is room here to write down why these lines and not others.
+
+Run \`taste-check checklist --lint\` after editing.
+
+- Nothing important is cut off at an edge of the frame
+- Every piece of text is legible against what is directly behind it
+- Nothing overlaps something it was not meant to overlap
+- Nothing is obviously misaligned with the things beside it
+- Nothing is still showing a loading, empty, or placeholder state
+- Nothing appears twice that was meant to appear once
+`;
+
+/** Words that describe a feeling rather than something visible in an image. */
+const UNFALSIFIABLE = [
+  'premium', 'modern', 'clean', 'professional', 'polished', 'elegant',
+  'beautiful', 'delightful', 'intuitive', 'slick', 'crisp', 'tasteful',
+  'cohesive', 'harmonious', 'balanced', 'pleasing', 'appealing', 'nice',
+  'good', 'bad', 'ugly', 'feel', 'feels', 'vibe', 'aesthetic',
+];
+
+/** Things a still image cannot show. */
+const NOT_IN_A_STILL = [
+  'animation', 'animate', 'transition', 'hover', 'scroll', 'scrolling',
+  'load time', 'performance', 'sound', 'audio', 'click', 'tap', 'drag',
+  'keyboard', 'focus ring', 'video plays',
+];
+
+const MEASURABLE = /\b\d+(\.\d+)?\s*(:\s*1|px|rem|em|%|pt)\b|\bcontrast ratio\b|\b\d+:\d+\b/i;
+const COMPOUND = /\b(and|or)\b/i;
+
+/**
+ * Check a single line for whether a judge could answer it.
+ *
+ * Never about content. A line can ask for anything at all; these findings are
+ * about whether asking it produces a verdict or a shrug.
+ */
+export function lintLine(text) {
+  const findings = [];
+  const lower = text.toLowerCase();
+  const words = text.trim().split(/\s+/);
+
+  const vague = UNFALSIFIABLE.filter((w) => new RegExp(`\\b${w}\\b`, 'i').test(lower));
+  if (vague.length) {
+    findings.push({
+      kind: 'unfalsifiable',
+      detail:
+        `"${vague.join('", "')}" describes a feeling rather than something visible. ` +
+        `A judge answers unsure, every run.`,
+    });
+  }
+
+  const unseen = NOT_IN_A_STILL.filter((w) => lower.includes(w));
+  if (unseen.length) {
+    findings.push({
+      kind: 'not-in-a-still',
+      detail: `"${unseen.join('", "')}" cannot be seen in a screenshot. The judge only gets images.`,
+    });
+  }
+
+  if (MEASURABLE.test(text)) {
+    findings.push({
+      kind: 'measurable',
+      detail:
+        'this asks for a number. Put it in a contrast pair or a runtime target, ' +
+        'where the answer is measured rather than judged.',
+    });
+  }
+
+  // Only compound when both halves look like claims, so "black and white" or
+  // "cut off at the top or bottom" are left alone.
+  if (COMPOUND.test(text)) {
+    // Three words a side, so "black and white" and "the top or bottom" are
+    // left alone while two real claims are caught.
+    const halves = text.split(/\b(?:and|or)\b/i).map((h) => h.trim().split(/\s+/).filter(Boolean).length);
+    if (halves.length > 1 && halves.every((n) => n >= 3)) {
+      findings.push({
+        kind: 'compound',
+        detail: 'two claims in one line. One verdict cannot answer both, so split it.',
+      });
+    }
+  }
+
+  if (words.length < 4) {
+    findings.push({ kind: 'too-short', detail: 'too short to say what is being asked.' });
+  }
+
+  return findings;
+}
+
+/** Lint every line of a checklist, keeping the line numbers. */
+export function lintChecklist(entries) {
+  const findings = [];
+  for (const entry of entries) {
+    for (const finding of lintLine(entry.text)) {
+      findings.push({ ...finding, line: entry.line, text: entry.text });
+    }
+  }
+  return findings;
+}

package/src/config.mjs

@@ -27,8 +27,28 @@ function stringArray(value, where, errors, { required = true } = {}) {
   return value;
 }
 
+/**
+ * Allowed anywhere an object is, and ignored.
+ *
+ * The validator rejects keys it does not know, which is right: a misspelled
+ * key that silently does nothing is a config that looks like it is working.
+ * The cost is that a config had nowhere to say why it exists. Per-pair labels
+ * carry the reasoning for a pair, and nothing carried the reasoning for the
+ * file. A format people are expected to live with for years should let them
+ * write that down.
+ */
+const COMMENT = '$comment';
+
 function rejectUnknown(object, allowed, where, errors) {
   for (const key of Object.keys(object)) {
+    if (key === COMMENT) {
+      const value = object[key];
+      const ok =
+        typeof value === 'string' ||
+        (Array.isArray(value) && value.every((v) => typeof v === 'string'));
+      if (!ok) errors.push(`${where}.${COMMENT} must be a string, or an array of them`);
+      continue;
+    }
     if (!allowed.includes(key)) {
       errors.push(`${where} has an unknown key "${key}". Allowed: ${allowed.join(', ')}.`);
     }

package/src/judge.mjs

@@ -18,9 +18,12 @@
  * every line, prefer unsure to a guess, quote what you actually see. That part
  * is method and it is portable.
  *
- * The CHECKLIST is yours. taste-check contains no design rules and never will,
- * because a shipped checklist is just somebody else's taste wearing the
- * authority of a tool.
+ * The CHECKLIST is yours. taste-check enforces no rule it wrote: what runs is
+ * the file in your repo. `checklist --new` will scaffold you a starting point,
+ * and the distinction is not a dodge, it is the difference between a default
+ * and a scaffold. A default judges every user who never opened the file
+ * against opinions they did not choose. A scaffold is a file you edited and
+ * committed, and this never reads its own copy of it.
  *
  * ── Where the determinism line falls ────────────────────────────────────
  *