maddox-engine 0.2.0 → 0.3.0

package/README.md

@@ -1,16 +1,16 @@
 # maddox
 
-Scans source code for design-token, motion, and component-state drift against your own design system.
+Scans source code — or a live deployed page — for design-token, motion, and component-state drift against your own design system.
 
 This is the open-source scanning engine behind [Maddox Engine](https://www.maddoxengine.com) — the same code the hosted dashboard and GitHub Action run, extracted so you can run it locally or in CI with no account required.
 
 ## What it checks
 
-- **Colors** — every hex literal in your source, matched against your `@theme` tokens by exact value, then by RGB distance for near-misses.
-- **Motion** — durations and easings used in code, checked against your real motion tokens (duration against duration, ease against ease — never cross-compared).
-- **Component states** — an explicit contract you write yourself (a JSON file naming which states each kind of component must cover: `disabled`, `loading`, `error`, and so on). Confirms the state is referenced in the file; it doesn't verify it renders correctly — that's a static source check, not a visual one.
+- **Colors** — every hex literal in your source (or a live page's rendered HTML/CSS), matched against your `@theme` tokens by exact value, then by RGB distance for near-misses.
+- **Motion** — durations and easings used in code, checked against your real motion tokens (duration against duration, ease against ease — never cross-compared). Source-scan only; see `--url` below.
+- **Component states** — an explicit contract you write yourself (a JSON file naming which states each kind of component must cover: `disabled`, `loading`, `error`, and so on). Confirms the state is referenced in the file; it doesn't verify it renders correctly — that's a static source check, not a visual one. Source-scan only.
 
-This is a static source-code scanner, not a pixel/DOM visual-regression tool. It checks the values developers actually wrote against the tokens that are supposed to govern them, so it catches drift that renders identically to a real token (and so produces zero visual diff) but was never written as one.
+This is a source-code and rendered-output scanner, not a pixel/DOM visual-regression tool. It checks the values a page actually ships against the tokens that are supposed to govern them, so it catches drift that renders identically to a real token (and so produces zero visual diff) but was never written as one.
 
 ## Usage
 
@@ -20,10 +20,11 @@ npx maddox-engine <target-source-dir> <path-to-globals.css-with-@theme-block> [o
 
 Options:
 
-- `--project <name>` — project name (defaults to the target directory's basename)
+- `--project <name>` — project name (defaults to the target directory's basename, or the URL's hostname in `--url` mode)
 - `--motion <path>` — path to a JSON file of motion tokens (e.g. a vendored copy of your motion-tokens export)
-- `--states <path>` — path to a state-contract JSON file (see below)
+- `--states <path>` — path to a state-contract JSON file (see below). No effect in `--url` mode.
 - `--tokens-studio <path>` — path to a Tokens Studio / W3C Design Tokens JSON export (see below); merged with, and taking precedence over, `@theme` on any path both define
+- `--url <page-url>` — scan a deployed page instead of local source (see below); pass a placeholder like `-` for `<target-source-dir>` when using this alone
 - `--format text|json|markdown` — output format (default: `text`)
 - `--fail-below <0-100>` — exit non-zero if the drift health score falls below this threshold; omit to never fail
 
@@ -50,6 +51,21 @@ Ground truth is explicit — nothing is inferred about which states a component
 
 If your tokens live in Figma via the Tokens Studio plugin rather than (or alongside) a Tailwind `@theme` block, export them to JSON and point `--tokens-studio` at the file. A single-set export (the whole file is one token tree) and a multi-set export (top-level keys are set names, e.g. `global`, `dark`) are both supported — for a multi-set export, every set is merged, later sets overriding earlier ones by path. `{alias}` references are resolved automatically. Only token types that resolve to a single comparable value (`color`, `spacing`, `sizing`, `fontSizes`, `borderRadius`, `dimension`) are used — composite types like `typography` or `boxShadow` describe a bundle of properties, not one value to diff against, and are skipped rather than misclassified.
 
+### Scanning a live page
+
+```bash
+npx maddox-engine - ./src/app/globals.css --url https://example.com --format text
+```
+
+`--url` fetches the page's rendered HTML plus every same-origin `<link rel="stylesheet">` it links to, and runs the same color/font-size extraction against what's actually shipped — not what's in the repo at scan time. This is the difference between a source-linter and a production check: a stale CDN cache, a build step that silently drops a token, or a config typo can all make the deployed page diverge from what the source says, and only a live-URL scan catches that.
+
+Two things don't carry over to `--url` mode:
+
+- **Motion** — durations/eases only ever appear in rendered HTML/CSS as static literals if a component hardcodes them there, which real motion libraries don't do (they animate via the JS runtime). Motion findings are dropped from every `--url` scan rather than surfaced as noise from unrelated CSS.
+- **Component states** — state-completeness is a "is this identifier referenced in this file's source" check; a fetched, already-rendered page has no source to check. `--states` is accepted but has no effect in `--url` mode.
+
+Third-party stylesheets (a different origin than the page itself, e.g. a font CDN) are never fetched — only same-origin CSS, so findings only ever point at code the project actually owns.
+
 ### Suggested fixes
 
 Every `near-miss` finding with a resolved nearest token gets a `suggestion` — the concrete replacement text (`var(--token-name)` for a CSS value, or `motionTokens.path.to.value` for a motion token). It's advisory text for a human to apply, not an automatic edit: nothing in this package rewrites your source files. An `unrecognized` value with no close match, and a `missing`-state finding, never get one — there's no safe mechanical fix for either.
@@ -74,11 +90,12 @@ This repo is also a GitHub Action — `omrdev1/maddox-cli` — that runs a scan,
 
 Inputs:
 
-- `target-dir` *(required)* — directory to scan
+- `target-dir` *(required unless `url` is set)* — directory to scan
 - `theme-css` *(required)* — path to the CSS file containing the `@theme` block
 - `motion-tokens` — path to a motion-tokens JSON file
-- `states` — path to a state-contract JSON file
+- `states` — path to a state-contract JSON file. No effect when `url` is set.
 - `tokens-studio` — path to a Tokens Studio / W3C Design Tokens JSON export
+- `url` — a deployed page URL to scan instead of local source
 - `github-token` *(required)* — for posting the PR comment, usually `${{ secrets.GITHUB_TOKEN }}`
 - `fail-below` — fail the build below this health score; omit for comment-only
 - `api-key` / `api-url` — optional, upload results to a [Maddox Engine](https://www.maddoxengine.com) dashboard account for scan history and drift trends across projects (a separate hosted product, not required to use the Action itself)

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "maddox-engine",
-  "version": "0.2.0",
-  "description": "Scans source code for design-token, motion, and component-state drift against your own design system — the same engine behind Maddox Engine's CI checks.",
+  "version": "0.3.0",
+  "description": "Scans source code or a live deployed page for design-token, motion, and component-state drift against your own design system — the same engine behind Maddox Engine's CI checks.",
   "license": "MIT",
   "type": "module",
   "bin": {

package/src/cli.ts

@@ -1,7 +1,7 @@
 #!/usr/bin/env -S npx tsx
 
 import { readFileSync } from "node:fs";
-import { audit, healthScore, loadGroundTruth, loadStateContract } from "./core.js";
+import { audit, auditUrl, healthScore, loadGroundTruth, loadStateContract } from "./core.js";
 import { summarize, toJson, toMarkdown, toText, type ScanSummary } from "./format.js";
 
 async function uploadResult(apiUrl: string, apiKey: string, projectName: string, summary: ScanSummary) {
@@ -36,18 +36,23 @@ function flagValue(name: string): string | undefined {
 async function main() {
   const targetDir = process.argv[2];
   const themeCssPath = process.argv[3];
+  const url = flagValue("--url");
 
-  if (!targetDir || !themeCssPath) {
+  if (!themeCssPath || (!targetDir && !url)) {
     console.error(
       "Usage: pnpm scan <target-source-dir> <path-to-globals.css-with-@theme-block> " +
         "[--project <name>] [--motion <path-to-motion-tokens.json>] " +
         "[--states <path-to-state-contract.json>] [--format text|json|markdown] " +
-        "[--fail-below <0-100>] [--tokens-studio <path-to-tokens-studio-export.json>]"
+        "[--fail-below <0-100>] [--tokens-studio <path-to-tokens-studio-export.json>] " +
+        "[--url <live-page-url>]\n" +
+        "  --url scans a deployed page's rendered HTML/CSS instead of local source " +
+        "(pass a placeholder for <target-source-dir>, e.g. '-', when using --url alone). " +
+        "--states has no effect in --url mode: state-completeness is a source-code check."
     );
     process.exit(1);
   }
 
-  const projectName = flagValue("--project") ?? targetDir.split("/").pop() ?? "unnamed-project";
+  const projectName = flagValue("--project") ?? (url ? new URL(url).hostname : targetDir.split("/").pop()) ?? "unnamed-project";
   const format = flagValue("--format") ?? "text";
 
   // Motion tokens are an optional external JSON file (a plain object like
@@ -64,7 +69,8 @@ async function main() {
 
   const tokensStudioPath = flagValue("--tokens-studio");
   const groundTruth = loadGroundTruth(themeCssPath, motionTokens, tokensStudioPath);
-  const result = await audit(targetDir, groundTruth, stateContract);
+
+  const result = url ? await auditUrl(url, groundTruth) : await audit(targetDir, groundTruth, stateContract);
   const summary = summarize(result.findings, result.filesScanned);
 
   if (format === "json") {

package/src/core.ts

@@ -2,12 +2,14 @@ export * from "./types.js";
 export * from "./groundTruth.js";
 export * from "./tokensStudio.js";
 export * from "./scan.js";
+export * from "./crawl.js";
 export * from "./diff.js";
 export * from "./stateCheck.js";
 export * from "./healthScore.js";
 export * from "./suggestFix.js";
 
 import { scanSource, scanFiles } from "./scan.js";
+import { crawlUrl } from "./crawl.js";
 import { diffUsages } from "./diff.js";
 import { checkStates } from "./stateCheck.js";
 import { attachSuggestions } from "./suggestFix.js";
@@ -32,3 +34,18 @@ export async function audit(
 
   return { findings, filesScanned: scannedFiles.size };
 }
+
+/**
+ * The production-aware counterpart to audit(): fetches a live page (plus
+ * its same-origin stylesheets) instead of walking local source files, and
+ * diffs what's actually shipped against the same ground truth. No state
+ * contract here — component-state completeness is a source-code question
+ * (is the state referenced anywhere in this file's source), which has no
+ * equivalent for a fetched, already-rendered page.
+ */
+export async function auditUrl(pageUrl: string, groundTruth: GroundTruth): Promise<ScanResult> {
+  const { usages, sourcesScanned } = await crawlUrl(pageUrl);
+  const findings = diffUsages(usages, groundTruth);
+  attachSuggestions(findings);
+  return { findings, filesScanned: sourcesScanned.length };
+}

package/src/crawl.ts

@@ -0,0 +1,92 @@
+import { extractFromContent, type RawUsage } from "./scan.js";
+
+// Only ever follow relative/absolute-path stylesheet links resolved
+// against the page's own origin — never a third-party CDN's CSS, since
+// that's not the project's own design system and would only add noise
+// (and, worse, false "unrecognized" findings for code the project never
+// wrote).
+function isSameOrigin(href: string, pageUrl: URL): boolean {
+  try {
+    const resolved = new URL(href, pageUrl);
+    return resolved.origin === pageUrl.origin;
+  } catch {
+    return false;
+  }
+}
+
+// Matches <link ... rel="stylesheet" ... href="..."> and the reverse
+// attribute order, case-insensitively, without a full HTML parser —
+// good enough for the handful of <link> tags a real page head has.
+const STYLESHEET_LINK_RE =
+  /<link\b[^>]*\brel=["']stylesheet["'][^>]*\bhref=["']([^"']+)["'][^>]*>|<link\b[^>]*\bhref=["']([^"']+)["'][^>]*\brel=["']stylesheet["'][^>]*>/gi;
+
+function findStylesheetHrefs(html: string): string[] {
+  const hrefs: string[] = [];
+  STYLESHEET_LINK_RE.lastIndex = 0;
+  let m: RegExpExecArray | null;
+  while ((m = STYLESHEET_LINK_RE.exec(html)) !== null) {
+    hrefs.push(m[1] ?? m[2]);
+  }
+  return hrefs;
+}
+
+export interface CrawlResult {
+  usages: RawUsage[];
+  // The page URL plus every stylesheet URL actually fetched — reported
+  // back so a caller (or the CLI's own output) can show what was scanned,
+  // the live-URL equivalent of scanSource's file count.
+  sourcesScanned: string[];
+}
+
+/**
+ * Fetches a live page's rendered HTML plus every same-origin linked
+ * stylesheet, and runs the same color/font-size/motion literal
+ * extraction scanSource runs over local source files. This is what makes
+ * Maddox "production-aware" rather than local-source-only: it checks
+ * what a real deployed page actually ships, not just what's in the repo
+ * at scan time (a config typo, a stale CDN cache, or a build step that
+ * silently drops a token could all cause the two to diverge).
+ *
+ * Framer Motion / GSAP usage is deliberately dropped from the result.
+ * Real motion libraries set durations/eases via the JS animation runtime,
+ * not static markup, so they never legitimately appear in rendered
+ * HTML/CSS — but framework-compiled CSS reliably DOES contain unrelated
+ * matches for the same shared regex (confirmed against a real deploy:
+ * Tailwind's own `--default-transition-duration:.15s` custom property
+ * matches the duration pattern, `transition-timing-function` matches the
+ * ease pattern). Rather than surface those as noisy false "unrecognized"
+ * findings, motion usages are filtered out of every crawl result before
+ * it's returned — motion checking stays a source-scan-only capability.
+ */
+export async function crawlUrl(pageUrl: string): Promise<CrawlResult> {
+  const url = new URL(pageUrl);
+  const usages: RawUsage[] = [];
+  const sourcesScanned: string[] = [];
+
+  const pageResponse = await fetch(url, { redirect: "follow" });
+  if (!pageResponse.ok) {
+    throw new Error(`Failed to fetch ${url}: ${pageResponse.status} ${pageResponse.statusText}`);
+  }
+  const html = await pageResponse.text();
+  usages.push(...extractFromContent(url.toString(), html));
+  sourcesScanned.push(url.toString());
+
+  const stylesheetHrefs = findStylesheetHrefs(html).filter((href) => isSameOrigin(href, url));
+
+  for (const href of stylesheetHrefs) {
+    const cssUrl = new URL(href, url).toString();
+    try {
+      const cssResponse = await fetch(cssUrl);
+      if (!cssResponse.ok) continue;
+      const css = await cssResponse.text();
+      usages.push(...extractFromContent(cssUrl, css));
+      sourcesScanned.push(cssUrl);
+    } catch {
+      // A single unreachable stylesheet shouldn't fail the whole crawl —
+      // same tolerance readScannedFiles already has for an unreadable
+      // local file.
+    }
+  }
+
+  return { usages: usages.filter((u) => u.kind !== "motion"), sourcesScanned };
+}