@trueclone/cli 0.2.0 → 0.3.0

package/README.md

@@ -61,17 +61,17 @@ The hand-built portfolio isn't perfect yet, and its report says so: computed sty
 
 ```
 URL ─► capture ─► understand ─► design system ─► build ─► score ─► report
-        3 widths    motion         DESIGN.md        your      7 checks   REPORT.md
+        3 widths    motion         DESIGN.md        your      8 checks   REPORT.md
         HAR, HTML   inventory      globals.css      agent     in parallel
         assets      + REA          tokens.css                  ▲     │
                                                                └─────┘ until every target passes
 ```
 
 1. **Capture** (`trueclone capture`): full-page screenshots at 390, 768 and 1440px, sections, rendered HTML, assets, every network request.
-2. **Understand** (`trueclone motion`, REA): reads the live animation runtime (GSAP timelines, ScrollTriggers, Lenis and Swiper settings, CSS animations, cursor-driven tweens) and, through REA, bundles, source maps, event listeners and API endpoints.
+2. **Understand** (`trueclone motion`, `trueclone understand`): reads the live animation runtime (GSAP timelines, ScrollTriggers, Lenis and Swiper settings, CSS animations, cursor-driven tweens) and, with REA, which scripts actually run while a visitor scrolls, which handler each control has and the code around it, the endpoints in the code, source maps when published: `code.md`, so the agent copies rules instead of guessing them from effects.
 3. **Design system** (`trueclone design`): one canonical `design-tokens.json`, rendered to DESIGN.md, globals.css and the clone's tokens.
-4. **Build**: `trueclone reveals` records the original's scroll reveals frame by frame and its scroll-driven class changes; `trueclone widgets` clicks every tab and menu and records each state; `trueclone seed` turns the script-free mirror into the clone (template mode: layout-measured lorem text and same-size media placeholders) and replays what was recorded. Layout is at 99%+ before anyone writes CSS; your agent writes the remaining behaviour, guided by the skill. `trueclone rules` shows the original's CSS for any element (like DevTools).
-5. **Score** (`trueclone score`): layout pixel diff per section, scroll states replayed, motion curves sampled, interaction scenarios with change assertions, every hover, computed styles at matching points. Reference measurements are cached, so only the clone is re-measured.
+4. **Build**: `trueclone reveals` records, at every width, the original's scroll reveals frame by frame, its scroll-linked curves (parallax) and its scroll-driven class changes; `trueclone widgets` clicks every tab and menu and records each state; `trueclone seed` turns the script-free mirror into the clone (template mode: layout-measured lorem text and same-size media placeholders), cleans what the original's scripts left in the DOM (pin-spacers, canvas buffer sizes) and replays what was recorded. Layout is at 99%+ before anyone writes CSS; your agent writes the remaining behaviour, guided by the skill. `trueclone rules` shows the original's CSS for any element (like DevTools).
+5. **Score** (`trueclone score`): layout pixel diff per section, scroll states replayed, motion curves sampled, interaction scenarios with change assertions, every hover, computed styles at matching points, and the load sequence (how settled each side is at 0.25 to 4 s after navigation, cold cache, same emulated network). Reference measurements are cached, so only the clone is re-measured.
 6. **Report**: `REPORT.md` with the verdict, what to fix first, and what was not copied.
 
 ## Why not just REA?
@@ -86,7 +86,7 @@ trueclone **bundles** [REA](https://github.com/morluto/rea) (Reverse Engineer An
 | Tells you what's wrong | No | "Footer column 2 is at x 189, the original's at 732", plus the original's CSS rule |
 | Outputs | Whatever the agent writes | HTML clone, Next.js app, DESIGN.md, globals.css, motion recipe, REPORT.md |
 
-REA answers *how does this work?* trueclone answers *is my copy identical, and where isn't it?* For the hard parts (a bundled physics rule, a shader, the API a component calls) the skill sends your agent to REA.
+REA answers *how does this work?* trueclone answers *is my copy identical, and where isn't it?* trueclone proves the visual and behavioural match on its own; `trueclone understand` adds REA's code-level evidence (which scripts ran, the handler behind each control with its source, endpoints, original sources when published) so the agent copies rules instead of inferring them. For the hard parts (a bundled physics rule, a shader, a module's data flow) REA's own CLI is one command away.
 
 ## Modes and guardrails
 
@@ -107,13 +107,16 @@ npx trueclone doctor                    # check Node, Chrome and REA
 npx trueclone <url> [name]              # capture + design-system draft
 npx trueclone capture <url> [name] [--mode exact --permission "<owner>"]
 npx trueclone motion <name>             # motion inventory
+npx trueclone understand <name>         # code evidence with REA: which scripts ran, each control's handler with its source -> code.md
 npx trueclone design <name>             # DESIGN.md + globals.css
 npx trueclone reveals <name>            # record scroll reveals and scroll class changes
 npx trueclone widgets <name>            # record every tab and menu state
 npx trueclone seed <name>               # clone/ from the mirror (template: placeholder text and media)
 npx trueclone rules <name> "<selector>" # the original's CSS rules for an element
 npx trueclone textshape <name> --write  # template mode: placeholder text with the original's line breaks
-npx trueclone score <name> [--quick]    # every check, then REPORT.md
+npx trueclone load <name>               # load sequence on both sides: settledness 0.25-4 s after navigation, cold cache, same network
+npx trueclone score <name> [--quick]    # every check, then REPORT.md (--only detail,load --viewport tablet after a fix; unchanged checks are kept)
+npx trueclone explain <name> <y>        # one place on the page: what every check saw there, plus a reference|clone|diff crop
 npx trueclone next <name> --verify      # Next.js + Tailwind + shadcn app, scored
 npx trueclone rea <args>                # bundled REA
 npx trueclone help                      # every step

package/bin/trueclone.mjs

@@ -23,6 +23,7 @@ const STEPS = {
   seed: ['seed.mjs', 'exact mode: start clone/ from the script-free mirror (markup, text, styles, media)'],
   design: ['design.mjs', 'design-tokens.json -> DESIGN.md (linted), shadcn globals.css, clone/tokens.css'],
   motion: ['motion.mjs', 'motion inventory from the live runtime (GSAP/ScrollTrigger, Lenis, Swiper, CSS animations)'],
+  understand: ['understand.mjs', 'code evidence with REA: which scripts ran, each element\'s handler with its source, endpoints, source maps -> reference/code.md'],
   reveals: ['reveals.mjs', 'record one-shot scroll reveals frame by frame, so seed can replay them (any library)'],
   widgets: ['widgets.mjs', 'record every tab and menu state by clicking it, so seed can replay them (template mode)'],
   states: ['states.mjs', 'live scroll states; --clone scores the rebuild step by step'],
@@ -30,7 +31,11 @@ const STEPS = {
   interact: ['interact.mjs', 'interaction tests: click/hover/drag/key scenarios on both sides (interactions.json)'],
   detail: ['detail.mjs', 'style diff at matching points: font, size, weight, line height, colour, background, radius'],
   hovers: ['hovers.mjs', 'hover every interactive element on both sides and compare what changes'],
-  rules: ['rules.mjs', 'every CSS rule matching an element on the reference (+ ::before/::after, @media)'],
+  load: ['load.mjs', 'load sequence: how settled each side is 0.25/0.5/1/2/4 s after navigation, cold cache, same network'],
+  drift: ['drift.mjs', 'has the live page changed since the capture? (page height and sections per width vs meta.json)'],
+  owned: ['owned.mjs', 'which sections of the clone script.js rebuilds at boot (load with scripts blocked vs running)'],
+  rules: ['rules.mjs', 'every CSS rule matching an element on the reference (+ ::before/::after, @media); --clone for the clone'],
+  explain: ['explain.mjs', 'one place on the page: what layout, states, detail and hovers saw there, plus a reference|clone|diff crop'],
   textshape: ['textshape.mjs', 'template mode: placeholder words with the reference word widths, so lines break the same'],
   diff: ['diff.mjs', 'full-page pixel diff per viewport and section (--structure masks text and media)'],
   spec: ['spec.mjs', 'measured boxes and styles of the live page (--clone for the rebuild)'],

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@trueclone/cli",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "A working twin of any website from its URL: HTML clone, Next.js + Tailwind + shadcn app, DESIGN.md and globals.css, proven by layout, motion, interaction and style scores. Bundles REA.",
   "type": "module",
   "bin": {

package/scripts/capture.mjs

@@ -92,6 +92,7 @@ const browser = await openBrowser(flags.cdp);
 const manifest = [];
 const summary = [];
 const rootFontSize = {};
+const inlineStyles = {};
 try {
   for (const vp of VIEWPORTS) {
     const desktop = vp.name === 'desktop';
@@ -150,6 +151,26 @@ try {
     await fs.writeFile(path.join(ref, `${vp.name}.png`), shot);
     const sections = await sectionsOf(page);
     await fs.writeFile(path.join(ref, `${vp.name}.sections.json`), JSON.stringify(sections, null, 2));
+    // inline styles at this width, by element path: scripts size and place things per width (a grid's computed
+    // height, a card's offset), and the rendered HTML only keeps the desktop values. The seed turns what differs
+    // between widths into media-query rules.
+    inlineStyles[vp.name] = await page.evaluate(() => {
+      const SKIP = new Set(['SCRIPT', 'NOSCRIPT', 'TEMPLATE', 'STYLE', 'LINK', 'META']);
+      const pathOf = (el) => {
+        const parts = [];
+        for (let n = el; n && n !== document.body; n = n.parentElement) {
+          const sibs = [...n.parentElement.children].filter((c) => !SKIP.has(c.tagName));
+          parts.unshift(`${n.tagName.toLowerCase()}:${sibs.indexOf(n) + 1}`);
+        }
+        return parts.join('>');
+      };
+      const out = {};
+      for (const el of document.body.querySelectorAll('[style]')) {
+        if (SKIP.has(el.tagName) || (el.closest('svg') && el.tagName !== 'svg')) continue;
+        out[pathOf(el)] = el.getAttribute('style');
+      }
+      return out;
+    });
     const height = await page.evaluate(() => document.documentElement.scrollHeight);
     rootFontSize[vp.name] = await page.evaluate(() => getComputedStyle(document.documentElement).fontSize);
     if (desktop) {
@@ -162,6 +183,23 @@ try {
           el.setAttribute('data-tc-rich', String(i));
           restore[i++] = best.html;
         }
+        // open shadow roots (web components, a Webflow code island) are not part of outerHTML: each becomes a
+        // declarative <template shadowrootmode="open"> inside its host, which the browser turns back into a shadow
+        // root while parsing, no script needed. Deepest first, so a host inside a shadow root is carried too.
+        // Constructed stylesheets (adoptedStyleSheets) are written out as a <style> at the top of the template.
+        const hosts = [];
+        (function find(scope) { for (const el of scope.querySelectorAll('*')) if (el.shadowRoot) { find(el.shadowRoot); hosts.push(el); } })(document);
+        for (const host of hosts) {
+          const t = document.createElement('template');
+          t.setAttribute('shadowrootmode', 'open');
+          t.setAttribute('shadowrootserializable', '');
+          const adopted = (host.shadowRoot.adoptedStyleSheets || []).map((s) => { try { return [...s.cssRules].map((r) => r.cssText).join('\n'); } catch { return ''; } }).filter(Boolean);
+          if (adopted.length) { const st = document.createElement('style'); st.textContent = adopted.join('\n'); t.content.appendChild(st); }
+          for (const n of host.shadowRoot.childNodes) t.content.appendChild(n.cloneNode(true));
+          host.prepend(t);
+          host.setAttribute('data-tc-shadow', '');
+        }
+        if (hosts.length) console.log(`tc-shadow: serialised ${hosts.length} open shadow root(s) as declarative shadow DOM`);
         const doc = document.documentElement.cloneNode(true);
         for (const el of document.querySelectorAll('[data-tc-rich]')) el.removeAttribute('data-tc-rich');
         for (const el of doc.querySelectorAll('[data-tc-rich]')) {
@@ -188,6 +226,7 @@ try {
 }
 
 await fs.writeFile(path.join(ref, 'assets', 'manifest.json'), JSON.stringify(manifest, null, 2));
+await fs.writeFile(path.join(ref, 'inline-styles.json'), JSON.stringify(inlineStyles));
 await fs.writeFile(path.join(ref, 'meta.json'), JSON.stringify({ url, name, mode, permission, hide, capturedAt: new Date().toISOString(), rootFontSize, viewports: summary }, null, 2));
 await buildMirror(manifest, url, mode === 'exact' ? `Exact mode, authorised by: ${permission}.` : "Template mode: the site's own content, kept as a reference only.");
 console.table(summary);
@@ -202,6 +241,24 @@ async function buildMirror(manifest, url, note) {
   const mirror = path.join(ref, 'mirror');
   await fs.rm(mirror, { recursive: true, force: true });
   await fs.mkdir(path.join(mirror, 'css'), { recursive: true });
+  // Rendered twice: the first pass shows which media the capture never saw (lazy images below the fold, hover and
+  // state images, sources behind a media query, fonts behind @import); fetchMissing() saves them and the second pass
+  // points at the local copies. A page is only as complete as its slowest-loading image.
+  let out = await renderMirror(manifest, url, note, mirror);
+  const missing = findRemoteMedia(out);
+  if (missing.length) {
+    const added = await fetchMissing(missing, manifest, url);
+    if (added) out = await renderMirror(manifest, url, note, mirror);
+    await fs.writeFile(path.join(ref, 'assets', 'manifest.json'), JSON.stringify(manifest, null, 2));
+    console.log(`mirror: fetched ${added}/${missing.length} media the capture had not seen (lazy, hover or conditional sources)`);
+  }
+  for (const [file, css] of out.css) await fs.writeFile(path.join(mirror, 'css', file), css);
+  await fs.writeFile(path.join(mirror, 'index.html'), out.html);
+  const left = findRemoteMedia(out);
+  console.log(`mirror: reference/mirror/index.html (${manifest.length} local assets${left.length ? `, ${left.length} remote media left: ${left.slice(0, 3).join(', ')}` : ''})`);
+}
+
+async function renderMirror(manifest, url, note, mirror) {
   const byUrl = new Map(manifest.map((m) => [m.url.split('#')[0], m]));
   const abs = (u, base) => { try { return new URL(u.trim().replace(/^['"]|['"]$/g, '').replaceAll('&amp;', '&'), base).href.split('#')[0]; } catch { return null; } };
   const cssName = (m) => path.basename(m.file);
@@ -218,9 +275,10 @@ async function buildMirror(manifest, url, note) {
     .replace(/@import\s+(['"])([^'"]+)\1/g, (all, q, u) => `@import "${local(abs(u, base), fromDir) || u}"`);
 
   // stylesheets: rewritten copies under mirror/css
+  const css = new Map();
   for (const m of manifest.filter((x) => x.type === 'stylesheet')) {
-    const css = await fs.readFile(path.join(ref, m.file), 'utf8');
-    await fs.writeFile(path.join(mirror, 'css', cssName(m)), rewriteCssUrls(css, m.url, path.join(mirror, 'css')));
+    const text = await fs.readFile(path.join(ref, m.file), 'utf8').catch(() => null);
+    if (text !== null) css.set(cssName(m), rewriteCssUrls(text, m.url, path.join(mirror, 'css')));
   }
 
   let html = await fs.readFile(path.join(ref, 'rendered.html'), 'utf8');
@@ -235,14 +293,74 @@ async function buildMirror(manifest, url, note) {
       const l = local(abs(u, url), mirror);
       return l ? ` ${attr}=${q}${l}${q}` : all;
     })
+    // a subresource-integrity hash belongs to the bytes at the original URL; after the rewrite it only makes the
+    // browser refuse the local copy (and the whole stylesheet with it)
+    .replace(/<link\b[^>]*>/gi, (tag) => tag.replace(/\s(?:integrity|crossorigin)(?:=("[^"]*"|'[^']*'|[^\s>]+))?/gi, ''))
     .replace(/\s(srcset|data-srcset)=("|')([^"']+)\2/gi, (all, attr, q, list) => ` ${attr}=${q}${list.split(',').map((part) => {
       const [u, ...rest] = part.trim().split(/\s+/);
       return [local(abs(u, url), mirror) || u, ...rest].join(' ');
     }).join(', ')}${q}`)
     .replace(/<style\b([^>]*)>([\s\S]*?)<\/style>/gi, (all, attrs, css) => `<style${attrs}>${rewriteCssUrls(css, url, mirror)}</style>`)
-    .replace(/\sstyle=("|')([^"']*url\([^"']*)\1/gi, (all, q, css) => ` style=${q}${rewriteCssUrls(css, url, mirror)}${q}`)
+    // quotes inside a style attribute are HTML-escaped (url(&quot;…&quot;)): unescape before rewriting, re-escape after
+    .replace(/\sstyle=(?:"([^"]*)"|'([^']*)')/gi, (all, d, s) => {
+      const value = d ?? s;
+      if (!/url\(/i.test(value)) return all;
+      const plain = value.replaceAll('&quot;', '"').replaceAll('&#39;', "'");
+      return ` style="${rewriteCssUrls(plain, url, mirror).replaceAll('"', '&quot;')}"`;
+    })
     .replace(/<head([^>]*)>/i, `<head$1>\n<!-- trueclone local mirror of ${url}. ${note.replace(/--/g, '-')} Local reference only: do not deploy. -->\n<meta name="robots" content="noindex,nofollow">`);
-  await fs.writeFile(path.join(mirror, 'index.html'), html);
-  const missing = [...html.matchAll(/\s(?:src|href)=["'](https?:[^"']+)["']/gi)].map((m) => m[1]).filter((u) => !/^https?:\/\/[^/]+\/?$/.test(u));
-  console.log(`mirror: reference/mirror/index.html (${manifest.length} local assets, ${missing.length} remote URLs left, e.g. links)`);
+  return { html, css };
+}
+
+// Media the page still points at on the network after the rewrite: image/video/font/stylesheet URLs in src, srcset,
+// poster, data-src, <link> hrefs and CSS url(). Links to other pages (<a href>) are not media.
+function findRemoteMedia({ html, css }) {
+  const urls = new Set();
+  const media = (u) => u && /^https?:\/\//.test(u) && !/^https?:\/\/[^/]+\/?$/.test(u) && !/\.(js|mjs|json|html?)(\?|$)/i.test(u);
+  const add = (u) => { if (media(u)) urls.add(u.replaceAll('&amp;', '&')); };
+  for (const m of html.matchAll(/\s(?:src|poster|data-src)=["']([^"']+)["']/gi)) add(m[1]);
+  for (const m of html.matchAll(/\s(?:srcset|data-srcset)=["']([^"']+)["']/gi)) for (const part of m[1].split(',')) add(part.trim().split(/\s+/)[0]);
+  for (const m of html.matchAll(/<link\b[^>]*>/gi)) {
+    const rel = /\srel=["']?([^"'\s>]+)/i.exec(m[0])?.[1] || '';
+    const href = /\shref=["']([^"']+)["']/i.exec(m[0])?.[1];
+    if (/stylesheet|icon|apple-touch-icon|preload|prefetch/i.test(rel) && !/\sas=["']?(script|document|fetch)/i.test(m[0])) add(href);
+  }
+  const cssUrls = (text) => { for (const m of text.matchAll(/url\(\s*(?:&quot;|["'])?(https?:\/\/[^)"'&\s]+)/gi)) add(m[1]); };
+  cssUrls(html);
+  for (const text of css.values()) cssUrls(text);
+  return [...urls].slice(0, 300);
+}
+
+async function fetchMissing(urls, manifest, pageUrl) {
+  const typeOf = (u, contentType = '') => {
+    if (/text\/css/.test(contentType) || /\.css(\?|$)/i.test(u)) return 'stylesheet';
+    if (/^font\/|application\/(x-)?font|application\/vnd\.ms-fontobject/.test(contentType) || /\.(woff2?|ttf|otf|eot)(\?|$)/i.test(u)) return 'font';
+    if (/^image\//.test(contentType) || /\.(png|jpe?g|gif|webp|avif|svg|ico|bmp)(\?|$)/i.test(u)) return 'image';
+    return 'media';
+  };
+  let added = 0;
+  const queue = [...urls];
+  const worker = async () => {
+    for (let u = queue.shift(); u; u = queue.shift()) {
+      try {
+        const res = await fetch(u, { headers: { 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36', referer: pageUrl, accept: '*/*' }, signal: AbortSignal.timeout(20000), redirect: 'follow' });
+        if (!res.ok) continue;
+        const body = Buffer.from(await res.arrayBuffer());
+        if (!body.length || body.length > MAX_ASSET_BYTES) continue;
+        const contentType = res.headers.get('content-type') || '';
+        if (/text\/html/.test(contentType)) continue; // a page, not media (a soft 404)
+        const type = typeOf(u, contentType);
+        const parsed = new URL(u);
+        const base = (path.basename(parsed.pathname) || 'index').replace(/[^a-z0-9._-]/gi, '_').slice(-80);
+        const hash = crypto.createHash('sha1').update(u).digest('hex').slice(0, 8);
+        const file = path.join('assets', type, `${hash}-${base}`);
+        await fs.mkdir(path.join(ref, 'assets', type), { recursive: true });
+        await fs.writeFile(path.join(ref, file), body);
+        manifest.push({ url: u, type, file: file.replaceAll('\\', '/'), contentType, bytes: body.length, fetched: 'mirror' });
+        added++;
+      } catch { /* offline, blocked, timed out: stays remote and is listed */ }
+    }
+  };
+  await Promise.all(Array.from({ length: 6 }, worker));
+  return added;
 }

package/scripts/detail.mjs

@@ -155,7 +155,7 @@ async function sample(browser, url, vp, probes) {
       if (y >= max) break;
     }
     await context.close();
-    return { list, results };
+    return { list, results, pageHeight: max + vp.height };
   }
   const list = probes;
   const results = new Array(list.length);
@@ -177,14 +177,27 @@ const report = { name, at: new Date().toISOString(), viewports: [] };
 try {
   for (const vpName of wanted) {
     const vp = VIEWPORTS.find((v) => v.name === vpName);
-    // reference probes are cached until the next capture (--refresh-ref to re-measure)
-    const ref = await cached(root, { tool: 'detail', vp: vpName, max: MAX, v: 8 }, () => sample(browser, meta.url, vp, null), Boolean(flags['refresh-ref']));
+    // reference probes are cached until the next capture (--refresh-ref to re-measure). The reference is read twice:
+    // a probe that does not agree with itself (random motion, a hover that lands by timing, a rotating word) can
+    // never be matched by any clone and is set aside as unstable instead of being handed over as a gap.
+    const ref = await cached(root, { tool: 'detail', vp: vpName, max: MAX, v: 9 }, async () => {
+      const first = await sample(browser, meta.url, vp, null);
+      // a live site changes: a reference read against today's page scores a clone of yesterday's capture against the
+      // wrong thing. The page height is a cheap tell; the fix is a new capture, not a new clone.
+      const captured = meta.viewports?.find((v) => v.viewport === vpName)?.pageHeight;
+      if (captured && first.pageHeight && Math.abs(first.pageHeight - captured) > Math.max(40, captured * 0.01)) console.warn(`WARNING reference drift (${vpName}): the live page is ${first.pageHeight}px tall, the capture was ${captured}px (${meta.capturedAt}); re-capture before trusting this score`);
+      const again = await sample(browser, meta.url, vp, first.list.map(({ x, xs, y, kind, w, h, scroll }) => ({ x, xs, y, kind, w, h, scroll })));
+      const unstable = first.list.map((p, i) => JSON.stringify(first.results[i]?.style) !== JSON.stringify(again.results[i]?.style) || first.results[i]?.sel !== again.results[i]?.sel);
+      return { ...first, unstable };
+    }, Boolean(flags['refresh-ref']));
     const cl = await sample(browser, cloneUrl, vp, ref.list.map(({ x, xs, y, kind, w, h, scroll }) => ({ x, xs, y, kind, w, h, scroll })));
     const issues = [];
+    const unstableList = [];
     let clean = 0, missing = 0;
     ref.list.forEach((p, i) => {
       const a = ref.results[i], b = cl.results[i];
       if (!a?.style) return;
+      if (ref.unstable?.[i]) { unstableList.push({ y: p.y, kind: p.kind, ref: a.sel }); return; }
       if (!b?.style) { missing++; issues.push({ y: p.y, kind: p.kind, ref: a.sel, clone: b?.top || '(nothing)', props: [`no ${p.kind} element at this point on the clone`] }); return; }
       const A = a.style, B = b.style, props = [];
       if (p.kind === 'text') {
@@ -203,10 +216,10 @@ try {
       if (props.length) issues.push({ y: p.y, kind: p.kind, ref: a.sel, clone: b.sel, props });
       else clean++;
     });
-    const total = ref.list.filter((p, i) => ref.results[i]?.style).length;
+    const total = ref.list.filter((p, i) => ref.results[i]?.style && !ref.unstable?.[i]).length;
     const byProp = {};
     issues.forEach((it) => it.props.forEach((pr) => { const k = pr.split(' ')[0] === 'no' ? 'missing element' : pr.split(' ')[0]; byProp[k] = (byProp[k] || 0) + 1; }));
-    report.viewports.push({ viewport: vpName, probes: total, clean, missing, score: +(100 * clean / Math.max(1, total)).toFixed(1), byProp, issues });
+    report.viewports.push({ viewport: vpName, probes: total, clean, missing, unstable: unstableList.length, score: +(100 * clean / Math.max(1, total)).toFixed(1), byProp, issues, unstableProbes: unstableList });
   }
 } finally {
   if (!flags.cdp) await browser.close();
@@ -216,7 +229,7 @@ const out = path.join(root, 'compare', 'detail.json');
 await fs.mkdir(path.dirname(out), { recursive: true });
 await fs.writeFile(out, JSON.stringify(report, null, 2));
 for (const v of report.viewports) {
-  console.log(`\n${v.viewport}: detail match ${v.score}% (${v.clean}/${v.probes} probes clean, ${v.missing} with no counterpart)  by property: ${JSON.stringify(v.byProp)}`);
+  console.log(`\n${v.viewport}: detail match ${v.score}% (${v.clean}/${v.probes} probes clean, ${v.missing} with no counterpart${v.unstable ? `; ${v.unstable} unstable on the reference itself, not scored: ${[...new Set(v.unstableProbes.map((u) => u.ref))].slice(0, 4).join(', ')}` : ''})  by property: ${JSON.stringify(v.byProp)}`);
   // group identical issues so one root cause prints once with a count
   const groups = new Map();
   for (const it of v.issues) { const k = `${it.ref} -> ${it.clone}: ${it.props.join('; ')}`; const g = groups.get(k) || { ...it, n: 0 }; g.n++; groups.set(k, g); }

package/scripts/drift.mjs

@@ -0,0 +1,48 @@
+// Has the live page changed since the capture? Layout and scroll states score against the captured screenshots, but
+// detail, hovers, motion curves, interactions and the load sequence measure the live page: when the site changed in
+// between, those checks compare the clone with something it was never a copy of. One page load per width: the page
+// height and the number of top-level sections against meta.json. Written to compare/drift.json; REPORT.md marks the
+// live-measured checks "stale capture" and the advice is to re-capture, not to touch the clone.
+// usage: node scripts/drift.mjs <name> [--viewport desktop|all]
+import fs from 'node:fs/promises';
+import path from 'node:path';
+import { VIEWPORTS, parseArgs, openBrowser, newPage, settle, sectionsOf } from './lib.mjs';
+
+const { pos, flags } = parseArgs(process.argv.slice(2));
+const [name] = pos;
+if (!name) {
+  console.error('usage: trueclone drift <name> [--viewport desktop|all]');
+  process.exit(1);
+}
+const root = path.resolve('sites', name);
+const meta = JSON.parse(await fs.readFile(path.join(root, 'reference', 'meta.json'), 'utf8'));
+const vps = !flags.viewport || flags.viewport === 'all' ? VIEWPORTS : [VIEWPORTS.find((v) => v.name === flags.viewport)].filter(Boolean);
+
+const browser = await openBrowser(flags.cdp);
+const viewports = [];
+try {
+  for (const vp of vps) {
+    const captured = meta.viewports?.find((v) => v.viewport === vp.name);
+    if (!captured) continue;
+    const { context, page } = await newPage(browser, vp);
+    try {
+      await page.goto(meta.url, { waitUntil: 'load', timeout: 60000 });
+      await settle(page, meta.hide || []);
+      const livePageHeight = await page.evaluate(() => document.documentElement.scrollHeight);
+      const liveSections = (await sectionsOf(page)).length;
+      // 1% (at least 40px) absorbs font hinting and a rotating line of text; a new section or a block that moved is above that
+      const stale = Math.abs(livePageHeight - captured.pageHeight) > Math.max(40, captured.pageHeight * 0.01) || liveSections !== captured.sections;
+      viewports.push({ viewport: vp.name, livePageHeight, capturedPageHeight: captured.pageHeight, liveSections, capturedSections: captured.sections, stale });
+      console.log(`${vp.name}: live ${livePageHeight}px / ${liveSections} sections, captured ${captured.pageHeight}px / ${captured.sections} sections${stale ? '  <- changed since the capture' : '  (same)'}`);
+    } finally {
+      await context.close();
+    }
+  }
+} finally {
+  if (!flags.cdp) await browser.close();
+}
+const stale = viewports.some((v) => v.stale);
+await fs.mkdir(path.join(root, 'compare'), { recursive: true });
+await fs.writeFile(path.join(root, 'compare', 'drift.json'), JSON.stringify({ at: new Date().toISOString(), capturedAt: meta.capturedAt, stale, viewports }, null, 2));
+if (stale) console.warn(`WARNING reference drift: the live page has changed since the capture of ${meta.capturedAt}; detail, hovers, curves, interactions and load now measure a page the clone was never a copy of. Re-capture (trueclone capture, then states, reveals, seed --force) before trusting them.`);
+else console.log(`drift: none since ${meta.capturedAt}`);

package/scripts/explain.mjs

@@ -0,0 +1,87 @@
+// One look at one place on the page: what every check saw there, so a gap is read from the tool's own data instead of
+// re-measured by hand. Given a page y (or a section label) at one width, it prints the layout section's score, the
+// scroll states near it, the detail probes that differ, the hover rows that differ, and writes a side-by-side crop
+// (reference | clone | diff, masked) around that y.
+// usage: node scripts/explain.mjs <name> <y | "section label"> [--viewport desktop] [--range 800]
+import fs from 'node:fs/promises';
+import path from 'node:path';
+import { PNG, parseArgs, crop } from './lib.mjs';
+
+const { pos, flags } = parseArgs(process.argv.slice(2));
+const [name, where] = pos;
+if (!name || where === undefined) {
+  console.error('usage: trueclone explain <name> <y | "section label"> [--viewport desktop] [--range 800]');
+  process.exit(1);
+}
+const vp = flags.viewport || 'desktop';
+const range = Number(flags.range || 800);
+const root = path.resolve('sites', name);
+const read = async (rel) => fs.readFile(path.join(root, rel), 'utf8').then(JSON.parse).catch(() => null);
+
+const layout = await read('compare/report.mask.json');
+const lv = layout?.viewports.find((v) => v.viewport === vp);
+if (!lv) {
+  console.error(`no layout score for ${vp} yet: run trueclone score ${name} first`);
+  process.exit(1);
+}
+const section = /^\d+$/.test(where)
+  ? lv.sections.find((s) => Number(where) >= s.y && Number(where) < s.y + s.height) || null
+  : lv.sections.find((s) => String(s.label).toLowerCase().includes(where.toLowerCase())) || null;
+const y = /^\d+$/.test(where) ? Number(where) : section?.y;
+if (y === undefined) {
+  console.error(`no section labelled like "${where}" at ${vp}; sections: ${lv.sections.map((s) => `${s.y} ${String(s.label).slice(0, 24)}`).join(' | ')}`);
+  process.exit(1);
+}
+const lo = Math.max(0, y - range / 2), hi = y + range / 2;
+const within = (v) => v >= lo && v <= hi;
+
+console.log(`${name} @ ${vp} y ${y} (±${range / 2}px)`);
+console.log(`\nLayout: page ${lv.heightRef}px vs clone ${lv.heightClone}px (${lv.match}% overall)`);
+if (section) console.log(`  section "${String(section.label).slice(0, 60)}" y ${section.y} h ${section.height}: ${section.match}%`);
+for (const s of lv.sections.filter((s) => s !== section && (within(s.y) || within(s.y + s.height)))) console.log(`  nearby "${String(s.label).slice(0, 40)}" y ${s.y}: ${s.match}%`);
+
+const states = await read('compare/states/report.mask.json');
+const sv = states?.viewports?.find((v) => v.viewport === vp);
+const steps = (sv?.steps || sv?.states || []).filter((st) => within(st.scrollY ?? st.y ?? st.scroll ?? -1));
+if (steps.length) console.log(`\nScroll states scrolled to here: ${steps.map((st) => `${st.scrollY ?? st.y ?? st.scroll}px ${st.match ?? st.score}%`).join(', ')}`);
+
+const detail = await read('compare/detail.json');
+const dv = detail?.viewports?.find((v) => v.viewport === vp);
+if (dv) {
+  const here = dv.issues.filter((it) => within(it.y));
+  const unstable = (dv.unstableProbes || []).filter((it) => within(it.y));
+  console.log(`\nDetail: ${here.length} differing probe(s) here${unstable.length ? `, ${unstable.length} unstable on the reference (not scored)` : ''}`);
+  const groups = new Map();
+  for (const it of here) { const k = `${it.ref} -> ${it.clone}: ${it.props.join('; ')}`; const g = groups.get(k) || { ...it, n: 0 }; g.n++; groups.set(k, g); }
+  [...groups.values()].sort((a, b) => b.n - a.n).slice(0, 12).forEach((g) => console.log(`  x${String(g.n).padEnd(3)} y${String(g.y).padEnd(6)} ${g.ref} -> ${g.clone}: ${g.props.join('; ')}`));
+}
+
+const hovers = await read(`compare/hovers.${vp}.json`);
+if (hovers) {
+  const rows = hovers.rows.filter((r) => within(r.y) && !['static', 'ok', 'unstable'].includes(r.status));
+  if (rows.length) { console.log(`\nHovers differing here:`); rows.slice(0, 10).forEach((r) => console.log(`  y${r.y} ${r.ref}: reference changes ${r.refChanges}, clone ${r.cloneChanges} (${r.status})`)); }
+}
+
+// the crop: reference | clone | diff, masked, half size
+try {
+  const imgs = await Promise.all([`reference/${vp}.mask.png`, `compare/${vp}.mask.clone.png`, `compare/${vp}.mask.diff.png`].map(async (f) => PNG.sync.read(await fs.readFile(path.join(root, f)))));
+  const top = Math.max(0, Math.round(lo)), h = Math.min(Math.round(hi) - top, Math.min(...imgs.map((i) => i.height)) - top);
+  const parts = imgs.map((img) => crop(img, top, h));
+  const scale = 0.5, w = Math.round(parts[0].width * scale), hh = Math.round(h * scale);
+  const out = new PNG({ width: w * 3 + 20, height: hh });
+  out.data.fill(255);
+  parts.forEach((p, k) => {
+    for (let yy = 0; yy < hh; yy++) for (let x = 0; x < w; x++) {
+      const sx = Math.min(p.width - 1, Math.round(x / scale)), sy = Math.min(p.height - 1, Math.round(yy / scale));
+      const si = (sy * p.width + sx) * 4, di = (yy * out.width + x + k * (w + 10)) * 4;
+      out.data[di] = p.data[si]; out.data[di + 1] = p.data[si + 1]; out.data[di + 2] = p.data[si + 2]; out.data[di + 3] = 255;
+    }
+  });
+  const file = path.join(root, 'compare', 'explain', `${vp}-${y}.png`);
+  await fs.mkdir(path.dirname(file), { recursive: true });
+  await fs.writeFile(file, PNG.sync.write(out));
+  console.log(`\nCrop (reference | clone | diff, y ${top}-${top + h}): ${path.relative(process.cwd(), file)}`);
+} catch (e) {
+  console.log(`\n(no crop: ${e.message.split('\n')[0]})`);
+}
+console.log(`\nNext: npx trueclone rules ${name} "<selector>" --viewport ${vp} (the reference's CSS) and --clone (the clone's), or npx trueclone score ${name} --only layout,detail --viewport ${vp} after a fix.`);

package/scripts/hovers.mjs

@@ -86,15 +86,23 @@ async function hoverAll(browser, url, list) {
     const after = await page.evaluate(signature, [t.x, vy]);
     out.push({ sel: before?.sel || null, kinds: before && after && before.sel === after.sel ? changedKinds(before, after) : [] });
   }
+  const pageHeight = await page.evaluate(() => document.documentElement.scrollHeight).catch(() => null);
   await context.close();
-  return { list, out };
+  return { list, out, pageHeight };
 }
 
 const browser = await openBrowser(flags.cdp);
 let ref, cl;
 try {
-  // reference hovers are cached until the next capture (--refresh-ref to re-measure)
-  ref = await cached(root, { tool: 'hovers', vp: vp.name, max: MAX, v: 2 }, () => hoverAll(browser, meta.url, null), Boolean(flags['refresh-ref']));
+  // reference hovers are cached until the next capture (--refresh-ref to re-measure); read twice, so a hover effect
+  // that comes and goes by timing on the reference itself is set aside as unstable rather than demanded of the clone
+  ref = await cached(root, { tool: 'hovers', vp: vp.name, max: MAX, v: 3 }, async () => {
+    const first = await hoverAll(browser, meta.url, null);
+    const captured = meta.viewports?.find((v) => v.viewport === vp.name)?.pageHeight;
+    if (captured && first.pageHeight && Math.abs(first.pageHeight - captured) > Math.max(40, captured * 0.01)) console.warn(`WARNING reference drift (${vp.name}): the live page is ${first.pageHeight}px tall, the capture was ${captured}px (${meta.capturedAt}); re-capture before trusting this score`);
+    const again = await hoverAll(browser, meta.url, first.list);
+    return { ...first, unstable: first.out.map((a, i) => a.sel !== again.out[i]?.sel || a.kinds.join() !== (again.out[i]?.kinds || []).join()) };
+  }, Boolean(flags['refresh-ref']));
   cl = await hoverAll(browser, cloneUrl, ref.list);
 } finally {
   if (!flags.cdp) await browser.close();
@@ -102,15 +110,16 @@ try {
 }
 const rows = ref.list.map((t, i) => {
   const a = ref.out[i], b = cl.out[i];
-  const status = !a.kinds.length ? (b.kinds.length ? 'extra' : 'static') : !b.sel ? 'no element' : !b.kinds.length ? 'MISSING' : a.kinds.some((k) => b.kinds.includes(k)) ? 'ok' : 'different';
+  const status = ref.unstable?.[i] ? 'unstable' : !a.kinds.length ? (b.kinds.length ? 'extra' : 'static') : !b.sel ? 'no element' : !b.kinds.length ? 'MISSING' : a.kinds.some((k) => b.kinds.includes(k)) ? 'ok' : 'different';
   return { y: t.y, ref: a.sel || t.sel, refChanges: a.kinds.join(',') || '-', clone: b.sel || '-', cloneChanges: b.kinds.join(',') || '-', status };
 });
-const withHover = rows.filter((r) => r.refChanges !== '-');
+const withHover = rows.filter((r) => r.refChanges !== '-' && r.status !== 'unstable');
+const unstable = rows.filter((r) => r.status === 'unstable').length;
 const ok = withHover.filter((r) => r.status === 'ok').length;
-const report = { name, viewport: vp.name, at: new Date().toISOString(), score: +(100 * ok / Math.max(1, withHover.length)).toFixed(1), rows };
+const report = { name, viewport: vp.name, at: new Date().toISOString(), score: +(100 * ok / Math.max(1, withHover.length)).toFixed(1), unstable, rows };
 const out = path.join(root, 'compare', `hovers.${vp.name}.json`);
 await fs.mkdir(path.dirname(out), { recursive: true });
 await fs.writeFile(out, JSON.stringify(report, null, 2));
-console.log(`hover match (${vp.name}): ${report.score}% of ${withHover.length} reference hover effects reproduced (${rows.length} interactive elements probed)`);
-console.table(rows.filter((r) => r.status !== 'static' && r.status !== 'ok').slice(0, 25));
+console.log(`hover match (${vp.name}): ${report.score}% of ${withHover.length} reference hover effects reproduced (${rows.length} interactive elements probed${unstable ? `, ${unstable} unstable on the reference itself, not scored` : ''})`);
+console.table(rows.filter((r) => r.status !== 'static' && r.status !== 'ok' && r.status !== 'unstable').slice(0, 25));
 console.log(`-> ${path.relative(process.cwd(), out)}`);

package/scripts/lib.mjs

@@ -126,19 +126,22 @@ img,video{object-position:-99999px -99999px!important;background:#888!important}
 svg{background:#888!important} svg *{visibility:hidden!important} iframe{visibility:hidden!important}
 canvas{background:#888!important;filter:contrast(0) brightness(1.066)!important}
 [data-lab-cliptext]{background:none!important;-webkit-text-fill-color:transparent!important}
+[data-lab-bgi]{background-image:none!important;background-color:#888!important}
 [data-lab-bgb]::before,[data-lab-bga]::after{background-image:none!important;background-color:#888!important}`; // canvas = media (WebGL heroes, 3D models): flat grey box like images
 
-// What CSS alone can't select: gradient text (background-clip: text paints the glyphs with its background, so a
-// transparent colour still shows the letters) and photo backgrounds on ::before/::after. Marked with attributes
-// the mask CSS targets; `off` removes them.
+// What CSS alone can't select: photo backgrounds (a script that rewrites an element's inline background every frame
+// would undo an inline override, so the mask is a stylesheet rule on a marker attribute, which inline styles lose
+// to), gradient text (background-clip: text paints the glyphs with its background, so a transparent colour still
+// shows the letters) and photo backgrounds on ::before/::after. `off` removes the markers.
 function maskExtrasInPage(enable) {
-  const marks = ['data-lab-cliptext', 'data-lab-bgb', 'data-lab-bga'];
+  const marks = ['data-lab-bgi', 'data-lab-cliptext', 'data-lab-bgb', 'data-lab-bga'];
   if (!enable) {
     for (const m of marks) for (const el of document.querySelectorAll(`[${m}]`)) el.removeAttribute(m);
     return;
   }
   for (const el of document.querySelectorAll('*')) {
     const s = getComputedStyle(el);
+    if (s.backgroundImage.includes('url(')) el.setAttribute('data-lab-bgi', '');
     if (s.backgroundClip === 'text' || s.webkitBackgroundClip === 'text') el.setAttribute('data-lab-cliptext', '');
     if (getComputedStyle(el, '::before').backgroundImage.includes('url(')) el.setAttribute('data-lab-bgb', '');
     if (getComputedStyle(el, '::after').backgroundImage.includes('url(')) el.setAttribute('data-lab-bga', '');
@@ -148,14 +151,6 @@ function maskExtrasInPage(enable) {
 export async function mask(page) {
   const tag = await page.addStyleTag({ content: MASK_CSS });
   await tag.evaluate((el) => el.setAttribute('data-lab-freeze', ''));
-  await page.evaluate(() => {
-    for (const el of document.querySelectorAll('*')) {
-      if (getComputedStyle(el).backgroundImage.includes('url(')) {
-        el.style.setProperty('background-image', 'none', 'important');
-        el.style.setProperty('background-color', '#888', 'important');
-      }
-    }
-  });
   await page.evaluate(maskExtrasInPage, true);
   await page.waitForTimeout(200);
 }
@@ -173,7 +168,7 @@ export async function freezeLoops(page) {
   }).catch(() => {});
 }
 
-// Reversible version of mask() for live pages: on() masks, off() restores every inline style it touched.
+// Reversible version of mask() for live pages: on() masks, off() removes the stylesheet and every marker.
 export async function maskToggle(page, on) {
   await page.evaluate(([css, enable]) => {
     if (enable) {
@@ -182,19 +177,8 @@ export async function maskToggle(page, on) {
       tag.setAttribute('data-lab-freeze', '');
       tag.textContent = css;
       document.head.appendChild(tag);
-      for (const el of document.querySelectorAll('*')) {
-        if (getComputedStyle(el).backgroundImage.includes('url(')) {
-          el.setAttribute('data-lab-bg', el.style.cssText);
-          el.style.setProperty('background-image', 'none', 'important');
-          el.style.setProperty('background-color', '#888', 'important');
-        }
-      }
     } else {
       document.getElementById('lab-mask')?.remove();
-      for (const el of document.querySelectorAll('[data-lab-bg]')) {
-        el.style.cssText = el.getAttribute('data-lab-bg');
-        el.removeAttribute('data-lab-bg');
-      }
     }
   }, [MASK_CSS, on]);
   await page.evaluate(maskExtrasInPage, on);