styleproof 1.8.1 → 1.9.1

package/CHANGELOG.md

@@ -7,6 +7,20 @@ this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
 
 ## [Unreleased]
 
+## [1.9.1]
+
+### Fixed
+
+- **Framework / non-visual DOM noise no longer registers as a change.** Capture now skips a built-in default set of selectors — `<meta>`, `<title>`, `<link>`, `<script>`, `<style>`, `<base>`, `<noscript>`, `<template>`, and `next-route-announcer` — merged into (not replaced by) the caller's `ignore`. These are elements frameworks stream into the body and reorder (Next.js app-router injects metadata then hoists it) or inject as live regions (Next's a11y route announcer), with no visual box to diff; their churn was surfacing as phantom DOM-added/removed findings on PRs that changed no CSS. A real stylesheet change still shows up in the affected elements' computed styles, not in the `<style>` tag. _Note: this changes the captured element set slightly — re-baseline once after upgrading._
+
+## [1.9.0]
+
+### Added
+
+- **Deterministic captures with no per-repo fixtures.** `defineStyleMapCapture` now records each surface's data responses on the baseline run and replays them on the comparison run, so a diff reflects code, not live-data drift — a backend blip or a flipped status chip no longer reads as a style change on a PR that touched no CSS. Set `STYLEPROOF_REPLAY_FROM=<base dir>` on the head capture; only data URLs (`**/api/**`, configurable via `replayUrl` / `STYLEPROOF_REPLAY_URL`) are intercepted, so the app's own JS/CSS still load live and the captured run runs its own code. If the backend is down during a run, both sides replay the same recording — no phantom diff.
+- **Frozen clock during capture** (`freezeClock`, on by default; `clockTime` to set the instant). Pins `Date.now()` / `new Date()` so time-derived styling (relative-age classes, "stale > 1h" flags) can't drift between runs; timers keep running so settling/polling still work.
+- **Capture self-check** (`STYLEPROOF_SELFCHECK=1` / `selfCheck`). Captures each surface twice and fails with a clear _"non-deterministic capture"_ error (naming the drifting element) if the two differ — so a replay gap or unseeded randomness is caught at setup time instead of surfacing as a phantom change on an unrelated PR.
+
 ## [1.8.1]
 
 ### Fixed

package/README.md

@@ -103,11 +103,12 @@ jobs:
       - run: npx wait-on http://localhost:3000
       - run: STYLEMAP_DIR=base npx playwright test e2e/styleproof.spec.ts
 
-      # capture the PR head
+      # capture the PR head — replay the base's recorded data so the diff is
+      # code, not live-data drift (see "Deterministic by default" below)
       - run: git checkout ${{ github.event.pull_request.head.sha }}
       - run: npm ci && npm run build && (npm run serve &)
       - run: npx wait-on http://localhost:3000
-      - run: STYLEMAP_DIR=head npx playwright test e2e/styleproof.spec.ts
+      - run: STYLEMAP_DIR=head STYLEPROOF_REPLAY_FROM=__stylemaps__/base npx playwright test e2e/styleproof.spec.ts
 
       # report + gate
       - uses: BenSheridanEdwards/styleproof@v1
@@ -121,7 +122,14 @@ jobs:
 
 **4. Require the `StyleProof` status** in branch protection. Now an unsigned visual change can't merge.
 
-> Capture both sides in the **same environment** (same machine, same env vars): if env vars change _what renders_, base and head will diff on DOM no PR touched.
+**Deterministic by default — no fixtures required.** A style diff only means something if both sides saw the same inputs; otherwise live-data drift (a backend blip, a `5m ago` timestamp, a status chip that flips) reads as a style change on a PR that touched no CSS. StyleProof handles this for you:
+
+- **Record / replay.** The base capture records each surface's data responses (anything matching `**/api/**`) to a HAR; the head capture replays them, so the head renders _its_ code against the _base's_ data — the app's own JS/CSS still load live. Backend down during a run? Both sides replay the same recording, so there's no phantom diff. Point the head capture at the base's recording with `STYLEPROOF_REPLAY_FROM=<base dir>` (see the CI step above); tune the data boundary with `STYLEPROOF_REPLAY_URL` / `replayUrl` if your API isn't under `/api`.
+- **Frozen clock.** `Date.now()` / `new Date()` are pinned to a fixed instant, so time-derived styling (`stale > 1h → red`) can't drift. Timers keep running, so settling still works.
+- **Self-check** (`STYLEPROOF_SELFCHECK=1`). Captures each surface twice and fails if they differ — a replay gap or unseeded randomness surfaces as a clear _"non-deterministic capture"_ error, never as a phantom change on an unrelated PR.
+- **Framework noise is skipped by default.** Non-visual and framework-injected elements never count as a change — `<meta>`/`<title>`/`<script>`/`<style>`/… (which Next.js streams into the body then hoists) and live regions like Next's `next-route-announcer`. A real stylesheet change still shows up in the affected elements' computed styles, not in the `<style>` tag. Add your own selectors with `ignore` — they extend this default, they don't replace it.
+
+> Replay covers data the page _fetches_. If your app **server-renders** differently per environment (SSR feature flags, locale), still capture both sides with the same server env so the rendered HTML matches.
 
 **Live pages just work.** Before each capture, StyleProof settles the page — it waits until the computed-style map stops changing, so async content (a fetch, an SSE/WebSocket stream backfilling a grid) is captured loaded, not mid-load. Anything still moving on its own after that is detected as a live region and excluded from the diff, so a stream or ticker never reads as a change — no manual `ignore` needed. Disable or tune with `captureStyleMap(page, { stabilize: false })` / `{ stabilize: { quietFor, timeout } }`.
 

package/dist/capture.d.ts

@@ -65,9 +65,11 @@ export type StyleMap = {
 export type CaptureOptions = {
     /**
      * Selectors for nondeterministic regions (live data, embeds, ads). The
-     * matching elements and their descendants are skipped entirely. Usually
-     * unnecessary now that `stabilize` auto-detects live regions; use it to skip
-     * a region you know is volatile without paying the settle wait for it.
+     * matching elements and their descendants are skipped entirely. Added to a
+     * built-in default that skips framework/non-visual noise (`<meta>`/`<title>`/
+     * `<script>`/`<style>`/… and `next-route-announcer`), so you rarely need it —
+     * `stabilize` also auto-detects live regions. Use it to skip a region you know
+     * is volatile without paying the settle wait for it.
      */
     ignore?: string[];
     /**

package/dist/capture.js

@@ -5,6 +5,26 @@ const INTERACTIVE = 'a, button, input, textarea, select, summary, [role="button"
 // Freeze motion so every captured value is a settled end state, not a frame
 // of an animation or a mid-flight transition after a forced :hover.
 const FREEZE_CSS = '*,*::before,*::after{animation:none!important;transition:none!important}';
+// Always skipped, merged into the caller's `ignore`. Two kinds of noise that
+// are never a visual change worth gating on but churn the DOM between runs:
+//   - non-rendered elements frameworks stream into <body> then hoist (Next.js
+//     app-router injects <meta>/<title>/<link>); they have no box to style, and
+//     their presence/order is nondeterministic. A real stylesheet change still
+//     shows up in the affected elements' computed styles, not in the <style> tag.
+//   - framework-injected live regions / overlays that mount and reorder on their
+//     own (Next.js's a11y route announcer — already a known source of CDP skew).
+const FRAMEWORK_IGNORE = [
+    'meta',
+    'title',
+    'link',
+    'script',
+    'style',
+    'base',
+    'noscript',
+    'template',
+    'next-route-announcer',
+    '[id="__next-route-announcer__"]',
+];
 /** True if `path` is one of `roots` or a structural descendant of one. Shared by
  *  the capture (excluding live regions) and the diff (skipping them). */
 export function isUnder(path, roots) {
@@ -328,6 +348,51 @@ function capturePageTokens() {
     probe.remove();
     return tokens;
 }
+/** Settle the page and return the paths of live regions to exclude. */
+async function detectVolatile(page, ignore, stabilize) {
+    if (stabilize === false)
+        return [];
+    const opt = typeof stabilize === 'object' ? stabilize : {};
+    const volatile = await stabilizePage(page, ignore, opt.interval || 150, opt.quietFor || 600, opt.timeout || 5000);
+    if (volatile.length) {
+        // eslint-disable-next-line no-console
+        console.warn(`styleproof: ${volatile.length} live region(s) kept changing on their own and were excluded from ` +
+            'this capture (nondeterministic — a stream, ticker, or late-loading content). The diff skips them so ' +
+            'they never read as a change. If a real change is being hidden, settle the page in go() or raise stabilize.timeout.');
+    }
+    return volatile;
+}
+/** Drop live regions (and their subtrees) from the base capture, in Node so the
+ *  serialized capturePage stays a pure snapshot. */
+function dropVolatile(elements, volatile) {
+    if (!volatile.length)
+        return;
+    for (const p of Object.keys(elements))
+        if (isUnder(p, volatile))
+            delete elements[p];
+}
+/** Warn once when shadow roots / iframes were skipped (their styles aren't captured or diffed). */
+function warnUntraversed(shadowHosts, sameOriginFrames) {
+    if (!shadowHosts && !sameOriginFrames)
+        return;
+    // eslint-disable-next-line no-console
+    console.warn(`styleproof: ${shadowHosts} shadow host(s) and ${sameOriginFrames} same-origin iframe(s) were ` +
+        'NOT traversed — styles inside shadow roots and frames are not captured or diffed. A refactor inside ' +
+        'one would be reported as identical. See README "Limitations".');
+}
+/** Fold the pre-freeze motion longhands back onto the settled base capture. */
+function mergeMotion(elements, motion) {
+    for (const [p, entry] of Object.entries(elements)) {
+        const m = motion[p];
+        if (!m)
+            continue;
+        Object.assign(entry.style, m.style);
+        for (const [ps, props] of Object.entries(m.pseudo ?? {})) {
+            if (entry.pseudo?.[ps])
+                Object.assign(entry.pseudo[ps], props);
+        }
+    }
+}
 /**
  * Capture the page's complete style map. Drive the page to the state you want
  * first (navigate, open menus); by default the capture then auto-settles the
@@ -335,7 +400,9 @@ function capturePageTokens() {
  * paints after `go()` resolves is captured loaded, not mid-load.
  */
 export async function captureStyleMap(page, options = {}) {
-    const ignore = options.ignore ?? [];
+    // Framework/non-visual noise is always skipped, so it can't read as a DOM
+    // change; the caller's `ignore` adds to it (not replaces it).
+    const ignore = [...FRAMEWORK_IGNORE, ...(options.ignore ?? [])];
     const captureStates = options.captureStates ?? true;
     const maxInteractive = options.maxInteractive ?? 800;
     const stabilize = options.stabilize ?? true;
@@ -346,43 +413,11 @@ export async function captureStyleMap(page, options = {}) {
     // the same loaded state, and collect any region still changing on its own
     // (a live stream/ticker) to exclude — animations are frozen above, so only
     // real content/layout churn lands here.
-    let volatile = [];
-    if (stabilize !== false) {
-        const opt = typeof stabilize === 'object' ? stabilize : {};
-        const interval = opt.interval || 150;
-        const quietFor = opt.quietFor || 600;
-        const timeout = opt.timeout || 5000;
-        volatile = await stabilizePage(page, ignore, interval, quietFor, timeout);
-        if (volatile.length) {
-            // eslint-disable-next-line no-console
-            console.warn(`styleproof: ${volatile.length} live region(s) kept changing on their own and were excluded from ` +
-                'this capture (nondeterministic — a stream, ticker, or late-loading content). The diff skips them so ' +
-                'they never read as a change. If a real change is being hidden, settle the page in go() or raise stabilize.timeout.');
-        }
-    }
+    const volatile = await detectVolatile(page, ignore, stabilize);
     const base = await page.evaluate(capturePage, { ignore, motionOnly: false });
-    // Drop live regions (and their subtrees) detected by the settle pass — done
-    // here, in Node, so the serialized capturePage stays a pure snapshot.
-    if (volatile.length)
-        for (const p of Object.keys(base.elements))
-            if (isUnder(p, volatile))
-                delete base.elements[p];
-    if (base.shadowHosts || base.sameOriginFrames) {
-        // eslint-disable-next-line no-console
-        console.warn(`styleproof: ${base.shadowHosts} shadow host(s) and ${base.sameOriginFrames} same-origin iframe(s) were ` +
-            'NOT traversed — styles inside shadow roots and frames are not captured or diffed. A refactor inside ' +
-            'one would be reported as identical. See README "Limitations".');
-    }
-    for (const [p, entry] of Object.entries(base.elements)) {
-        const m = motion.elements[p];
-        if (!m)
-            continue;
-        Object.assign(entry.style, m.style);
-        for (const [ps, props] of Object.entries(m.pseudo ?? {})) {
-            if (entry.pseudo?.[ps])
-                Object.assign(entry.pseudo[ps], props);
-        }
-    }
+    dropVolatile(base.elements, volatile);
+    warnUntraversed(base.shadowHosts, base.sameOriginFrames);
+    mergeMotion(base.elements, motion.elements);
     let states = {};
     let statesSkipped = false;
     if (captureStates) {

package/dist/runner.d.ts

@@ -32,14 +32,50 @@ export type DefineOptions = {
      * without screenshots still diff, but produce text-only reports.
      */
     screenshots?: boolean;
+    /**
+     * Replay a baseline run's recorded responses so a before/after diff reflects
+     * code, not live-data drift. When set (or via STYLEPROOF_REPLAY_FROM), each
+     * surface replays `<replayFrom>/<key>@<width>.har` for requests matching
+     * `replayUrl`; otherwise the run RECORDS that HAR into its own dir for the
+     * comparison run to replay. Only data URLs are intercepted, so the app's own
+     * JS/CSS still load live — the captured run renders ITS code against the
+     * baseline's data. This is what makes captures deterministic with no per-repo
+     * fixtures: record once on the base, replay on the head.
+     */
+    replayFrom?: string;
+    /**
+     * URL glob for the data boundary to record/replay (default `**\/api/**`, or
+     * STYLEPROOF_REPLAY_URL). Requests outside it (JS/CSS/fonts/images) always
+     * load live so the captured code actually runs.
+     */
+    replayUrl?: string;
+    /**
+     * Freeze `Date.now()`/`new Date()` to a fixed instant so time-derived styling
+     * (relative-age classes, "stale > 1h" flags) can't drift between runs. Timers
+     * keep running, so settling/polling still works. Default true.
+     */
+    freezeClock?: boolean;
+    /** Fixed instant for the frozen clock (default `2025-01-01T00:00:00Z`). */
+    clockTime?: string | number | Date;
+    /**
+     * Capture each surface twice and fail if the computed styles differ — proves
+     * the capture is deterministic (catches a replay gap falling through to the
+     * live backend, or unseeded client randomness) instead of letting it surface
+     * as a phantom change on an unrelated diff. Default from STYLEPROOF_SELFCHECK=1.
+     */
+    selfCheck?: boolean;
 };
 /**
  * Generate one Playwright test per surface × width that captures the style
- * map to `<baseDir>/<dir>/<key>@<width>.json.gz`.
+ * map to `<baseDir>/<dir>/<key>@<width>.json.gz`. Captures are made
+ * deterministic with no per-repo fixtures: the baseline run records each
+ * surface's data responses to a HAR, and the comparison run replays them (set
+ * STYLEPROOF_REPLAY_FROM=<baseline dir> on the comparison capture), while the
+ * clock is frozen so time-derived styling is stable.
  *
  * ```ts
  * // styleproof.spec.ts
  * defineStyleMapCapture({ surfaces: SURFACES, dir: process.env.STYLEMAP_DIR });
  * ```
  */
-export declare function defineStyleMapCapture({ surfaces, dir, baseDir, screenshots, }: DefineOptions): void;
+export declare function defineStyleMapCapture({ surfaces, dir, baseDir, screenshots, replayFrom, replayUrl, freezeClock, clockTime, selfCheck, }: DefineOptions): void;

package/dist/runner.js

@@ -1,34 +1,102 @@
 import { test } from '@playwright/test';
+import fs from 'node:fs';
 import path from 'node:path';
 import { captureStyleMap, saveStyleMap } from './capture.js';
+import { diffStyleMaps } from './diff.js';
+/** One-line description of the first drift finding, for the self-check error. */
+function driftDesc(f) {
+    if (f.kind === 'dom')
+        return `${f.path} ${f.change}`;
+    const p = f.props[0];
+    return p ? `${f.path} ${p.prop}: ${p.before} → ${p.after}` : f.path;
+}
+/**
+ * Pin the surface's inputs: replay the baseline's recorded data (or record ours),
+ * scoped to the data URLs so the app's own JS/CSS still load live, then freeze the
+ * clock so time-derived styling is stable.
+ */
+async function pinInputs(page, harName, s) {
+    if (s.replayFrom) {
+        const har = path.join(s.replayFrom, harName);
+        if (fs.existsSync(har)) {
+            // notFound:'abort' — a request the baseline never recorded fails
+            // deterministically rather than silently hitting the live backend.
+            await page.routeFromHAR(har, { url: s.replayUrl, update: false, notFound: 'abort' });
+        }
+        else {
+            // eslint-disable-next-line no-console
+            console.warn(`styleproof: no replay HAR at ${har} — capturing live (NON-deterministic)`);
+        }
+    }
+    else {
+        await page.routeFromHAR(path.join(s.baseDir, s.dir, harName), {
+            url: s.replayUrl,
+            update: true,
+            updateContent: 'embed', // single portable file, no sidecar resources
+        });
+    }
+    if (s.freezeClock)
+        await page.clock.setFixedTime(new Date(s.clockTime));
+}
+/** Capture the surface again and throw if the computed styles drifted from `first`. */
+async function assertDeterministic(page, surface, first) {
+    await surface.go(page);
+    const again = await captureStyleMap(page, { ignore: surface.ignore ?? [] });
+    const drift = diffStyleMaps(first, again);
+    if (drift.length) {
+        throw new Error(`styleproof self-check failed: ${surface.key} is non-deterministic — ` +
+            `${drift.length} computed-style difference(s) between two captures of the same commit. ` +
+            `Likely a replay gap (a request not in the baseline HAR) or unseeded randomness. ` +
+            `First: ${driftDesc(drift[0])}`);
+    }
+}
+/** Drive one surface at one width to a settled state and save its style map (+ screenshot). */
+async function captureSurface(page, surface, width, s) {
+    test.setTimeout(180_000);
+    await pinInputs(page, `${surface.key}@${width}.har`, s);
+    const height = typeof surface.height === 'function' ? surface.height(width) : (surface.height ?? 800);
+    await page.setViewportSize({ width, height });
+    await surface.go(page);
+    const map = await captureStyleMap(page, { ignore: surface.ignore ?? [] });
+    if (s.selfCheck)
+        await assertDeterministic(page, surface, map);
+    const stem = path.join(s.baseDir, s.dir, `${surface.key}@${width}`);
+    saveStyleMap(`${stem}.json.gz`, map);
+    if (s.screenshots) {
+        // captureStyleMap froze animations/transitions, so this is the same settled
+        // state the map describes.
+        await page.screenshot({ path: `${stem}.png`, fullPage: true, animations: 'disabled' });
+    }
+}
 /**
  * Generate one Playwright test per surface × width that captures the style
- * map to `<baseDir>/<dir>/<key>@<width>.json.gz`.
+ * map to `<baseDir>/<dir>/<key>@<width>.json.gz`. Captures are made
+ * deterministic with no per-repo fixtures: the baseline run records each
+ * surface's data responses to a HAR, and the comparison run replays them (set
+ * STYLEPROOF_REPLAY_FROM=<baseline dir> on the comparison capture), while the
+ * clock is frozen so time-derived styling is stable.
  *
  * ```ts
  * // styleproof.spec.ts
  * defineStyleMapCapture({ surfaces: SURFACES, dir: process.env.STYLEMAP_DIR });
  * ```
  */
-export function defineStyleMapCapture({ surfaces, dir, baseDir = '__stylemaps__', screenshots = true, }) {
+export function defineStyleMapCapture({ surfaces, dir, baseDir = '__stylemaps__', screenshots = true, replayFrom = process.env.STYLEPROOF_REPLAY_FROM, replayUrl = process.env.STYLEPROOF_REPLAY_URL ?? '**/api/**', freezeClock = true, clockTime = '2025-01-01T00:00:00Z', selfCheck = process.env.STYLEPROOF_SELFCHECK === '1', }) {
     test.skip(!dir, 'set STYLEMAP_DIR=<label> to capture computed-style maps');
+    const settings = {
+        dir: dir,
+        baseDir,
+        screenshots,
+        replayFrom,
+        replayUrl,
+        freezeClock,
+        clockTime,
+        selfCheck,
+    };
     test.describe('styleproof capture', () => {
         for (const surface of surfaces) {
             for (const width of surface.widths) {
-                test(`${surface.key} @ ${width}`, async ({ page }) => {
-                    test.setTimeout(180_000);
-                    const height = typeof surface.height === 'function' ? surface.height(width) : (surface.height ?? 800);
-                    await page.setViewportSize({ width, height });
-                    await surface.go(page);
-                    const map = await captureStyleMap(page, { ignore: surface.ignore ?? [] });
-                    const stem = path.join(baseDir, dir, `${surface.key}@${width}`);
-                    saveStyleMap(`${stem}.json.gz`, map);
-                    if (screenshots) {
-                        // captureStyleMap froze animations/transitions, so this is the
-                        // same settled state the map describes.
-                        await page.screenshot({ path: `${stem}.png`, fullPage: true, animations: 'disabled' });
-                    }
-                });
+                test(`${surface.key} @ ${width}`, ({ page }) => captureSurface(page, surface, width, settings));
             }
         }
     });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "styleproof",
-  "version": "1.8.1",
+  "version": "1.9.1",
   "description": "Catch every CSS change before it ships — review PRs and certify refactors by the browser's computed styles, not pixels. Works with any styling system.",
   "keywords": [
     "playwright",