@trueclone/cli 0.1.0 → 0.3.0

package/README.md

@@ -23,46 +23,55 @@ npx @trueclone/cli init            # once per project: installs trueclone, the s
 
 ## Proof
 
-Two benchmarks with very different stacks, scored by the tool itself:
+Three benchmarks with very different stacks, scored by the tool itself:
 
 **[rea.tools](https://rea.tools/)**: static docs with a canvas game and interactive demos, exact mode (copied with the owner's permission).
 
 | Check | Desktop | Tablet | Mobile |
 |---|---|---|---|
 | Layout, real pixels | 100% | 100% | 100% |
-| Scroll states | 100% | 100% | 99.99% |
+| Scroll states | 100% | 100% | 100% |
 | Computed styles (≈280 points per width) | 100% | 100% | 100% |
 | Hover effects | 100% | | |
 | Interactions (copy, tabs, calculator, dinosaur game, language menu) | 5/5 | | |
 
-**A Webflow motorsport portfolio** built with GSAP, ScrollTrigger, SplitText, Lenis, Swiper and three.js (97 scroll triggers), template mode (own placeholder content):
+**A Next.js + React + Framer Motion developer landing page** (12,000px, 11 scroll reveals with spring easing, a scroll-state header, marquees, view-mounted components), template mode, **seeded automatically**: no hand-written layout code, no original text or media in the clone.
+
+| Check | Desktop | Tablet | Mobile |
+|---|---|---|---|
+| Layout, structure | 99.99% | 99.93% | 99.94% |
+| Scroll states | 99.96% | 99.92% | 99.94% |
+| Computed styles | 99.5% | 98.7% | 99.2% |
+| Hover effects | 100% | | |
+
+**A Webflow motorsport portfolio** built with GSAP, ScrollTrigger, SplitText, Lenis, Swiper and three.js (97 scroll triggers), template mode, built by hand before seeding existed:
 
 | Check | Desktop | Tablet | Mobile |
 |---|---|---|---|
 | Layout, structure | 99.6% | 99.7% | 96.8% |
-| Scroll states | 97.1% | 96.7% | 96.5% |
-| Motion curves | 96.1% | 98.2% | 100% |
-| Computed styles | 94% | 88.9% | 81.8% |
+| Scroll states | 98.0% | 96.6% | 96.5% |
+| Motion curves | 96.9% | 98.2% | 99.6% |
+| Computed styles | 94% | 88.9% | 85.4% |
 | Hover effects | 100% | | |
 | Interactions (drag sliders, drag gallery, cursor follow, menu, skip) | 5/5 | | |
 
-Not perfect yet, and the report says so: the computed-style check is below its 95% target (placeholder text in highlighted phrases, a scroll-linked colour fade, mobile spacing), and so is mobile layout. The generated Next.js app scores within half a point of the HTML clone on every check. A full scoring run takes 1 to 9 minutes.
+The hand-built portfolio isn't perfect yet, and its report says so: computed styles and mobile layout are below target (placeholder text in highlighted phrases, a scroll-linked colour fade, mobile spacing). The generated Next.js app scores within half a point of the HTML clone on every check. A full scoring run takes 1.5 to 9 minutes.
 
 ## How it works
 
 ```
 URL ─► capture ─► understand ─► design system ─► build ─► score ─► report
-        3 widths    motion         DESIGN.md        your      7 checks   REPORT.md
+        3 widths    motion         DESIGN.md        your      8 checks   REPORT.md
         HAR, HTML   inventory      globals.css      agent     in parallel
         assets      + REA          tokens.css                  ▲     │
                                                                └─────┘ until every target passes
 ```
 
 1. **Capture** (`trueclone capture`): full-page screenshots at 390, 768 and 1440px, sections, rendered HTML, assets, every network request.
-2. **Understand** (`trueclone motion`, REA): reads the live animation runtime (GSAP timelines, ScrollTriggers, Lenis and Swiper settings, CSS animations, cursor-driven tweens) and, through REA, bundles, source maps, event listeners and API endpoints.
+2. **Understand** (`trueclone motion`, `trueclone understand`): reads the live animation runtime (GSAP timelines, ScrollTriggers, Lenis and Swiper settings, CSS animations, cursor-driven tweens) and, with REA, which scripts actually run while a visitor scrolls, which handler each control has and the code around it, the endpoints in the code, source maps when published: `code.md`, so the agent copies rules instead of guessing them from effects.
 3. **Design system** (`trueclone design`): one canonical `design-tokens.json`, rendered to DESIGN.md, globals.css and the clone's tokens.
-4. **Build**: your agent writes the clone, guided by the skill. `trueclone rules` shows the original's CSS for any element (like DevTools), `trueclone textshape` gives placeholder text the original's line breaks, `trueclone seed` starts an exact clone from a script-free mirror.
-5. **Score** (`trueclone score`): layout pixel diff per section, scroll states replayed, motion curves sampled, interaction scenarios with change assertions, every hover, computed styles at matching points. Reference measurements are cached, so only the clone is re-measured.
+4. **Build**: `trueclone reveals` records, at every width, the original's scroll reveals frame by frame, its scroll-linked curves (parallax) and its scroll-driven class changes; `trueclone widgets` clicks every tab and menu and records each state; `trueclone seed` turns the script-free mirror into the clone (template mode: layout-measured lorem text and same-size media placeholders), cleans what the original's scripts left in the DOM (pin-spacers, canvas buffer sizes) and replays what was recorded. Layout is at 99%+ before anyone writes CSS; your agent writes the remaining behaviour, guided by the skill. `trueclone rules` shows the original's CSS for any element (like DevTools).
+5. **Score** (`trueclone score`): layout pixel diff per section, scroll states replayed, motion curves sampled, interaction scenarios with change assertions, every hover, computed styles at matching points, and the load sequence (how settled each side is at 0.25 to 4 s after navigation, cold cache, same emulated network). Reference measurements are cached, so only the clone is re-measured.
 6. **Report**: `REPORT.md` with the verdict, what to fix first, and what was not copied.
 
 ## Why not just REA?
@@ -77,7 +86,7 @@ trueclone **bundles** [REA](https://github.com/morluto/rea) (Reverse Engineer An
 | Tells you what's wrong | No | "Footer column 2 is at x 189, the original's at 732", plus the original's CSS rule |
 | Outputs | Whatever the agent writes | HTML clone, Next.js app, DESIGN.md, globals.css, motion recipe, REPORT.md |
 
-REA answers *how does this work?* trueclone answers *is my copy identical, and where isn't it?* For the hard parts (a bundled physics rule, a shader, the API a component calls) the skill sends your agent to REA.
+REA answers *how does this work?* trueclone answers *is my copy identical, and where isn't it?* trueclone proves the visual and behavioural match on its own; `trueclone understand` adds REA's code-level evidence (which scripts ran, the handler behind each control with its source, endpoints, original sources when published) so the agent copies rules instead of inferring them. For the hard parts (a bundled physics rule, a shader, a module's data flow) REA's own CLI is one command away.
 
 ## Modes and guardrails
 
@@ -98,11 +107,16 @@ npx trueclone doctor                    # check Node, Chrome and REA
 npx trueclone <url> [name]              # capture + design-system draft
 npx trueclone capture <url> [name] [--mode exact --permission "<owner>"]
 npx trueclone motion <name>             # motion inventory
+npx trueclone understand <name>         # code evidence with REA: which scripts ran, each control's handler with its source -> code.md
 npx trueclone design <name>             # DESIGN.md + globals.css
-npx trueclone seed <name>               # exact mode: start clone/ from the mirror
+npx trueclone reveals <name>            # record scroll reveals and scroll class changes
+npx trueclone widgets <name>            # record every tab and menu state
+npx trueclone seed <name>               # clone/ from the mirror (template: placeholder text and media)
 npx trueclone rules <name> "<selector>" # the original's CSS rules for an element
 npx trueclone textshape <name> --write  # template mode: placeholder text with the original's line breaks
-npx trueclone score <name> [--quick]    # every check, then REPORT.md
+npx trueclone load <name>               # load sequence on both sides: settledness 0.25-4 s after navigation, cold cache, same network
+npx trueclone score <name> [--quick]    # every check, then REPORT.md (--only detail,load --viewport tablet after a fix; unchanged checks are kept)
+npx trueclone explain <name> <y>        # one place on the page: what every check saw there, plus a reference|clone|diff crop
 npx trueclone next <name> --verify      # Next.js + Tailwind + shadcn app, scored
 npx trueclone rea <args>                # bundled REA
 npx trueclone help                      # every step

package/bin/trueclone.mjs

@@ -23,12 +23,19 @@ const STEPS = {
   seed: ['seed.mjs', 'exact mode: start clone/ from the script-free mirror (markup, text, styles, media)'],
   design: ['design.mjs', 'design-tokens.json -> DESIGN.md (linted), shadcn globals.css, clone/tokens.css'],
   motion: ['motion.mjs', 'motion inventory from the live runtime (GSAP/ScrollTrigger, Lenis, Swiper, CSS animations)'],
+  understand: ['understand.mjs', 'code evidence with REA: which scripts ran, each element\'s handler with its source, endpoints, source maps -> reference/code.md'],
+  reveals: ['reveals.mjs', 'record one-shot scroll reveals frame by frame, so seed can replay them (any library)'],
+  widgets: ['widgets.mjs', 'record every tab and menu state by clicking it, so seed can replay them (template mode)'],
   states: ['states.mjs', 'live scroll states; --clone scores the rebuild step by step'],
   curves: ['curves.mjs', 'motion score: sampled property curves, reference vs clone (motion-map.json)'],
   interact: ['interact.mjs', 'interaction tests: click/hover/drag/key scenarios on both sides (interactions.json)'],
   detail: ['detail.mjs', 'style diff at matching points: font, size, weight, line height, colour, background, radius'],
   hovers: ['hovers.mjs', 'hover every interactive element on both sides and compare what changes'],
-  rules: ['rules.mjs', 'every CSS rule matching an element on the reference (+ ::before/::after, @media)'],
+  load: ['load.mjs', 'load sequence: how settled each side is 0.25/0.5/1/2/4 s after navigation, cold cache, same network'],
+  drift: ['drift.mjs', 'has the live page changed since the capture? (page height and sections per width vs meta.json)'],
+  owned: ['owned.mjs', 'which sections of the clone script.js rebuilds at boot (load with scripts blocked vs running)'],
+  rules: ['rules.mjs', 'every CSS rule matching an element on the reference (+ ::before/::after, @media); --clone for the clone'],
+  explain: ['explain.mjs', 'one place on the page: what layout, states, detail and hovers saw there, plus a reference|clone|diff crop'],
   textshape: ['textshape.mjs', 'template mode: placeholder words with the reference word widths, so lines break the same'],
   diff: ['diff.mjs', 'full-page pixel diff per viewport and section (--structure masks text and media)'],
   spec: ['spec.mjs', 'measured boxes and styles of the live page (--clone for the rebuild)'],
@@ -38,6 +45,11 @@ const STEPS = {
   report: ['report.mjs', 'REPORT.md from the latest scores: every number, every gap, what cannot be copied'],
 };
 
+// npm/npx are .cmd files on Windows and need a shell; Node warns (DEP0190) when a shell gets separate arguments, so
+// build one quoted command line there instead
+const spawnCmd = (cmd, args, opts = {}) => (process.platform === 'win32'
+  ? spawn([cmd, ...args].map((a) => (/[\s"]/.test(a) ? `"${a.replace(/"/g, '\\"')}"` : a)).join(' '), { ...opts, shell: true })
+  : spawn(cmd, args, opts));
 const run = (file, args) => new Promise((resolve) => {
   const child = spawn(process.execPath, [file, ...args], { stdio: 'inherit' });
   child.on('close', (code) => resolve(code ?? 1));
@@ -88,7 +100,7 @@ async function init() {
   if (!process.argv.includes('--no-install')) {
     if (!(await exists(path.join(cwd, 'package.json')))) await fs.writeFile(path.join(cwd, 'package.json'), `${JSON.stringify({ name: path.basename(cwd).toLowerCase().replace(/[^a-z0-9-]+/g, '-'), private: true }, null, 2)}\n`);
     const code = await new Promise((resolve) => {
-      const child = spawn(process.platform === 'win32' ? 'npm.cmd' : 'npm', ['install', '--save-dev', '--save-exact', `${name}@${version}`], { stdio: 'inherit', shell: process.platform === 'win32' });
+      const child = spawnCmd('npm', ['install', '--save-dev', '--save-exact', `${name}@${version}`], { stdio: 'inherit' });
       child.on('close', (c) => resolve(c ?? 1));
     });
     installed = code === 0 ? `${name}@${version} (dev dependency: npx trueclone <step> now runs it)` : `failed: run npm install -D ${name}@${version} yourself`;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@trueclone/cli",
-  "version": "0.1.0",
+  "version": "0.3.0",
   "description": "A working twin of any website from its URL: HTML clone, Next.js + Tailwind + shadcn app, DESIGN.md and globals.css, proven by layout, motion, interaction and style scores. Bundles REA.",
   "type": "module",
   "bin": {

package/scripts/capture.mjs

@@ -1,6 +1,7 @@
 // Capture a live page as the clone's reference: full-page screenshots, sections, design tokens,
 // rendered HTML, CSS/image/font files and a HAR.
-// usage: node scripts/capture.mjs <url> [name] [--hide ".cookie-banner,#popup"] [--cdp [endpoint]]
+// usage: node scripts/capture.mjs <url> [name] [--hide ".cookie-banner,#popup"] [--cdp [endpoint]] [--mode exact --permission "<owner>"]
+//        node scripts/capture.mjs <url> <name> --mirror-only   (rebuild reference/mirror from the saved capture)
 import fs from 'node:fs/promises';
 import path from 'node:path';
 import crypto from 'node:crypto';
@@ -32,6 +33,14 @@ const SAVED_TYPES = new Set(['stylesheet', 'image', 'font', 'media']);
 const EXTRA_MEDIA = /\.(glb|gltf|bin|hdr|exr|ktx2|basis|mp4|webm|mov|m3u8|ts|m4s|lottie|riv|json)(\?|$)/i;
 const keepResponse = (type, url) => SAVED_TYPES.has(type) || (mode === 'exact' && ['fetch', 'xhr', 'other'].includes(type) && EXTRA_MEDIA.test(url));
 
+// --mirror-only: rebuild reference/mirror from an existing capture (no new screenshots, caches stay valid)
+if (flags['mirror-only']) {
+  const meta = JSON.parse(await fs.readFile(path.join(ref, 'meta.json'), 'utf8'));
+  const saved = JSON.parse(await fs.readFile(path.join(ref, 'assets', 'manifest.json'), 'utf8'));
+  await buildMirror(saved, meta.url, meta.mode === 'exact' ? `Exact mode, authorised by: ${meta.permission}.` : "Template mode: the site's own content, kept as a reference only.");
+  process.exit(0);
+}
+
 // Only clear what capture owns; states/, motion.*, *.spec.json come from other scripts and stay.
 await fs.rm(path.join(ref, 'assets'), { recursive: true, force: true });
 await fs.rm(path.join(ref, 'mirror'), { recursive: true, force: true });
@@ -83,6 +92,7 @@ const browser = await openBrowser(flags.cdp);
 const manifest = [];
 const summary = [];
 const rootFontSize = {};
+const inlineStyles = {};
 try {
   for (const vp of VIEWPORTS) {
     const desktop = vp.name === 'desktop';
@@ -106,16 +116,101 @@ try {
       });
     }
     await page.goto(url, { waitUntil: 'load', timeout: 60000 });
+    // Components that mount only while in view (a lazy wordmark, an embed) are often unmounted again by the time
+    // the DOM is saved. Watch from load: an element that was empty, got content while scrolled into view and is empty
+    // again at snapshot time is saved with the fullest content it had. Things that start full and empty out (a
+    // loader) are left alone.
+    if (desktop) await page.evaluate(() => {
+      const count = (el) => el.getElementsByTagName('*').length;
+      const initial = new WeakMap();
+      for (const el of document.body.getElementsByTagName('*')) initial.set(el, count(el));
+      window.__tcRich = new Map();
+      new MutationObserver((records) => {
+        for (const r of records) {
+          for (let el = r.target, up = 0; el instanceof Element && el !== document.body && up < 8; el = el.parentElement, up++) {
+            const start = initial.has(el) ? initial.get(el) : 0;
+            const n = count(el);
+            if (start > 0 || n === 0) continue;
+            const best = window.__tcRich.get(el);
+            if (!best || n > best.n) window.__tcRich.set(el, { n, html: el.innerHTML });
+          }
+        }
+      }).observe(document.body, { subtree: true, childList: true });
+    });
     await settle(page, hide);
-    await page.screenshot({ path: path.join(ref, `${vp.name}.png`), fullPage: true });
+    // A full-page screenshot brings the whole page into view at once, which mounts view-mounted components and starts
+    // their reveals: shoot until two shots agree, so the reference shows a settled page, not a fade caught at 0%.
+    let shot = await page.screenshot({ fullPage: true });
+    for (let tries = 0; tries < 3; tries++) {
+      await page.waitForTimeout(2000);
+      const again = await page.screenshot({ fullPage: true });
+      const same = again.equals(shot);
+      shot = again;
+      if (same) break;
+    }
+    await fs.writeFile(path.join(ref, `${vp.name}.png`), shot);
     const sections = await sectionsOf(page);
     await fs.writeFile(path.join(ref, `${vp.name}.sections.json`), JSON.stringify(sections, null, 2));
+    // inline styles at this width, by element path: scripts size and place things per width (a grid's computed
+    // height, a card's offset), and the rendered HTML only keeps the desktop values. The seed turns what differs
+    // between widths into media-query rules.
+    inlineStyles[vp.name] = await page.evaluate(() => {
+      const SKIP = new Set(['SCRIPT', 'NOSCRIPT', 'TEMPLATE', 'STYLE', 'LINK', 'META']);
+      const pathOf = (el) => {
+        const parts = [];
+        for (let n = el; n && n !== document.body; n = n.parentElement) {
+          const sibs = [...n.parentElement.children].filter((c) => !SKIP.has(c.tagName));
+          parts.unshift(`${n.tagName.toLowerCase()}:${sibs.indexOf(n) + 1}`);
+        }
+        return parts.join('>');
+      };
+      const out = {};
+      for (const el of document.body.querySelectorAll('[style]')) {
+        if (SKIP.has(el.tagName) || (el.closest('svg') && el.tagName !== 'svg')) continue;
+        out[pathOf(el)] = el.getAttribute('style');
+      }
+      return out;
+    });
     const height = await page.evaluate(() => document.documentElement.scrollHeight);
     rootFontSize[vp.name] = await page.evaluate(() => getComputedStyle(document.documentElement).fontSize);
     if (desktop) {
       const html = await page.evaluate(() => {
+        // restore view-mounted content (see above): only elements that are empty again now
+        const restore = [];
+        let i = 0;
+        for (const [el, best] of window.__tcRich || []) {
+          if (!el.isConnected || el.getElementsByTagName('*').length > 0) continue;
+          el.setAttribute('data-tc-rich', String(i));
+          restore[i++] = best.html;
+        }
+        // open shadow roots (web components, a Webflow code island) are not part of outerHTML: each becomes a
+        // declarative <template shadowrootmode="open"> inside its host, which the browser turns back into a shadow
+        // root while parsing, no script needed. Deepest first, so a host inside a shadow root is carried too.
+        // Constructed stylesheets (adoptedStyleSheets) are written out as a <style> at the top of the template.
+        const hosts = [];
+        (function find(scope) { for (const el of scope.querySelectorAll('*')) if (el.shadowRoot) { find(el.shadowRoot); hosts.push(el); } })(document);
+        for (const host of hosts) {
+          const t = document.createElement('template');
+          t.setAttribute('shadowrootmode', 'open');
+          t.setAttribute('shadowrootserializable', '');
+          const adopted = (host.shadowRoot.adoptedStyleSheets || []).map((s) => { try { return [...s.cssRules].map((r) => r.cssText).join('\n'); } catch { return ''; } }).filter(Boolean);
+          if (adopted.length) { const st = document.createElement('style'); st.textContent = adopted.join('\n'); t.content.appendChild(st); }
+          for (const n of host.shadowRoot.childNodes) t.content.appendChild(n.cloneNode(true));
+          host.prepend(t);
+          host.setAttribute('data-tc-shadow', '');
+        }
+        if (hosts.length) console.log(`tc-shadow: serialised ${hosts.length} open shadow root(s) as declarative shadow DOM`);
         const doc = document.documentElement.cloneNode(true);
+        for (const el of document.querySelectorAll('[data-tc-rich]')) el.removeAttribute('data-tc-rich');
+        for (const el of doc.querySelectorAll('[data-tc-rich]')) {
+          el.innerHTML = restore[Number(el.getAttribute('data-tc-rich'))];
+          el.removeAttribute('data-tc-rich');
+          // marked for the layout diff: a full-page still can't show view-mounted content on either side, so its box
+          // is excluded there (the scroll-state check, with a real viewport, scores it)
+          el.setAttribute('data-tc-viewmount', '');
+        }
         doc.querySelector('style[data-lab-freeze]')?.remove();
+        console.log(`tc-rich: restored ${restore.length} view-mounted element(s)`);
         return '<!DOCTYPE html>\n' + doc.outerHTML;
       });
       await fs.writeFile(path.join(ref, 'rendered.html'), html);
@@ -131,19 +226,39 @@ try {
 }
 
 await fs.writeFile(path.join(ref, 'assets', 'manifest.json'), JSON.stringify(manifest, null, 2));
+await fs.writeFile(path.join(ref, 'inline-styles.json'), JSON.stringify(inlineStyles));
 await fs.writeFile(path.join(ref, 'meta.json'), JSON.stringify({ url, name, mode, permission, hide, capturedAt: new Date().toISOString(), rootFontSize, viewports: summary }, null, 2));
-if (mode === 'exact') await buildMirror();
+await buildMirror(manifest, url, mode === 'exact' ? `Exact mode, authorised by: ${permission}.` : "Template mode: the site's own content, kept as a reference only.");
 console.table(summary);
-console.log(`${manifest.length} assets saved. Reference: ${path.relative(process.cwd(), ref)}${mode === 'exact' ? ` (exact mode; mirror in reference/mirror, authorised by: ${permission})` : ''}`);
+console.log(`${manifest.length} assets saved. Reference: ${path.relative(process.cwd(), ref)}, mirror in reference/mirror${mode === 'exact' ? ` (exact mode, authorised by: ${permission})` : ''}`);
 
-// ---------- Exact mode: static local mirror of the rendered page ----------
+// ---------- Static local mirror of the rendered page ----------
 // Same DOM as rendered.html, asset URLs rewritten to the saved copies, every script and inline handler
 // removed (we never reuse the site's code), forms disabled, noindex. Static by design: motion is rebuilt
-// from the states/motion inventory, not replayed.
-async function buildMirror() {
+// from the states/motion inventory, not replayed. Exact mode seeds the clone from it as is; template mode
+// seeds it with every text and medium swapped for placeholders (seed.mjs).
+async function buildMirror(manifest, url, note) {
   const mirror = path.join(ref, 'mirror');
   await fs.rm(mirror, { recursive: true, force: true });
   await fs.mkdir(path.join(mirror, 'css'), { recursive: true });
+  // Rendered twice: the first pass shows which media the capture never saw (lazy images below the fold, hover and
+  // state images, sources behind a media query, fonts behind @import); fetchMissing() saves them and the second pass
+  // points at the local copies. A page is only as complete as its slowest-loading image.
+  let out = await renderMirror(manifest, url, note, mirror);
+  const missing = findRemoteMedia(out);
+  if (missing.length) {
+    const added = await fetchMissing(missing, manifest, url);
+    if (added) out = await renderMirror(manifest, url, note, mirror);
+    await fs.writeFile(path.join(ref, 'assets', 'manifest.json'), JSON.stringify(manifest, null, 2));
+    console.log(`mirror: fetched ${added}/${missing.length} media the capture had not seen (lazy, hover or conditional sources)`);
+  }
+  for (const [file, css] of out.css) await fs.writeFile(path.join(mirror, 'css', file), css);
+  await fs.writeFile(path.join(mirror, 'index.html'), out.html);
+  const left = findRemoteMedia(out);
+  console.log(`mirror: reference/mirror/index.html (${manifest.length} local assets${left.length ? `, ${left.length} remote media left: ${left.slice(0, 3).join(', ')}` : ''})`);
+}
+
+async function renderMirror(manifest, url, note, mirror) {
   const byUrl = new Map(manifest.map((m) => [m.url.split('#')[0], m]));
   const abs = (u, base) => { try { return new URL(u.trim().replace(/^['"]|['"]$/g, '').replaceAll('&amp;', '&'), base).href.split('#')[0]; } catch { return null; } };
   const cssName = (m) => path.basename(m.file);
@@ -155,13 +270,15 @@ async function buildMirror() {
     return path.relative(fromDir, path.join(ref, m.file)).replaceAll('\\', '/');
   };
   const rewriteCssUrls = (css, base, fromDir) => css
-    .replace(/url\(\s*(['"]?)([^'")]+)\1\s*\)/g, (all, q, u) => (u.startsWith('data:') ? all : `url("${local(abs(u, base), fromDir) || abs(u, base) || u}")`))
+    // url(#id) points inside the document (SVG gradients, filters, masks): never resolve it against the stylesheet
+    .replace(/url\(\s*(['"]?)([^'")]+)\1\s*\)/g, (all, q, u) => (u.startsWith('data:') || u.trim().startsWith('#') ? all : `url("${local(abs(u, base), fromDir) || abs(u, base) || u}")`))
     .replace(/@import\s+(['"])([^'"]+)\1/g, (all, q, u) => `@import "${local(abs(u, base), fromDir) || u}"`);
 
   // stylesheets: rewritten copies under mirror/css
+  const css = new Map();
   for (const m of manifest.filter((x) => x.type === 'stylesheet')) {
-    const css = await fs.readFile(path.join(ref, m.file), 'utf8');
-    await fs.writeFile(path.join(mirror, 'css', cssName(m)), rewriteCssUrls(css, m.url, path.join(mirror, 'css')));
+    const text = await fs.readFile(path.join(ref, m.file), 'utf8').catch(() => null);
+    if (text !== null) css.set(cssName(m), rewriteCssUrls(text, m.url, path.join(mirror, 'css')));
   }
 
   let html = await fs.readFile(path.join(ref, 'rendered.html'), 'utf8');
@@ -176,14 +293,74 @@ async function buildMirror() {
       const l = local(abs(u, url), mirror);
       return l ? ` ${attr}=${q}${l}${q}` : all;
     })
+    // a subresource-integrity hash belongs to the bytes at the original URL; after the rewrite it only makes the
+    // browser refuse the local copy (and the whole stylesheet with it)
+    .replace(/<link\b[^>]*>/gi, (tag) => tag.replace(/\s(?:integrity|crossorigin)(?:=("[^"]*"|'[^']*'|[^\s>]+))?/gi, ''))
     .replace(/\s(srcset|data-srcset)=("|')([^"']+)\2/gi, (all, attr, q, list) => ` ${attr}=${q}${list.split(',').map((part) => {
       const [u, ...rest] = part.trim().split(/\s+/);
       return [local(abs(u, url), mirror) || u, ...rest].join(' ');
     }).join(', ')}${q}`)
     .replace(/<style\b([^>]*)>([\s\S]*?)<\/style>/gi, (all, attrs, css) => `<style${attrs}>${rewriteCssUrls(css, url, mirror)}</style>`)
-    .replace(/\sstyle=("|')([^"']*url\([^"']*)\1/gi, (all, q, css) => ` style=${q}${rewriteCssUrls(css, url, mirror)}${q}`)
-    .replace(/<head([^>]*)>/i, `<head$1>\n<!-- rea-lab local mirror of ${url}. Authorised by: ${permission.replace(/--/g, '-')}. Local reference only: do not deploy. -->\n<meta name="robots" content="noindex,nofollow">`);
-  await fs.writeFile(path.join(mirror, 'index.html'), html);
-  const missing = [...html.matchAll(/\s(?:src|href)=["'](https?:[^"']+)["']/gi)].map((m) => m[1]).filter((u) => !/^https?:\/\/[^/]+\/?$/.test(u));
-  console.log(`mirror: reference/mirror/index.html (${manifest.length} local assets, ${missing.length} remote URLs left, e.g. links)`);
+    // quotes inside a style attribute are HTML-escaped (url(&quot;…&quot;)): unescape before rewriting, re-escape after
+    .replace(/\sstyle=(?:"([^"]*)"|'([^']*)')/gi, (all, d, s) => {
+      const value = d ?? s;
+      if (!/url\(/i.test(value)) return all;
+      const plain = value.replaceAll('&quot;', '"').replaceAll('&#39;', "'");
+      return ` style="${rewriteCssUrls(plain, url, mirror).replaceAll('"', '&quot;')}"`;
+    })
+    .replace(/<head([^>]*)>/i, `<head$1>\n<!-- trueclone local mirror of ${url}. ${note.replace(/--/g, '-')} Local reference only: do not deploy. -->\n<meta name="robots" content="noindex,nofollow">`);
+  return { html, css };
+}
+
+// Media the page still points at on the network after the rewrite: image/video/font/stylesheet URLs in src, srcset,
+// poster, data-src, <link> hrefs and CSS url(). Links to other pages (<a href>) are not media.
+function findRemoteMedia({ html, css }) {
+  const urls = new Set();
+  const media = (u) => u && /^https?:\/\//.test(u) && !/^https?:\/\/[^/]+\/?$/.test(u) && !/\.(js|mjs|json|html?)(\?|$)/i.test(u);
+  const add = (u) => { if (media(u)) urls.add(u.replaceAll('&amp;', '&')); };
+  for (const m of html.matchAll(/\s(?:src|poster|data-src)=["']([^"']+)["']/gi)) add(m[1]);
+  for (const m of html.matchAll(/\s(?:srcset|data-srcset)=["']([^"']+)["']/gi)) for (const part of m[1].split(',')) add(part.trim().split(/\s+/)[0]);
+  for (const m of html.matchAll(/<link\b[^>]*>/gi)) {
+    const rel = /\srel=["']?([^"'\s>]+)/i.exec(m[0])?.[1] || '';
+    const href = /\shref=["']([^"']+)["']/i.exec(m[0])?.[1];
+    if (/stylesheet|icon|apple-touch-icon|preload|prefetch/i.test(rel) && !/\sas=["']?(script|document|fetch)/i.test(m[0])) add(href);
+  }
+  const cssUrls = (text) => { for (const m of text.matchAll(/url\(\s*(?:&quot;|["'])?(https?:\/\/[^)"'&\s]+)/gi)) add(m[1]); };
+  cssUrls(html);
+  for (const text of css.values()) cssUrls(text);
+  return [...urls].slice(0, 300);
+}
+
+async function fetchMissing(urls, manifest, pageUrl) {
+  const typeOf = (u, contentType = '') => {
+    if (/text\/css/.test(contentType) || /\.css(\?|$)/i.test(u)) return 'stylesheet';
+    if (/^font\/|application\/(x-)?font|application\/vnd\.ms-fontobject/.test(contentType) || /\.(woff2?|ttf|otf|eot)(\?|$)/i.test(u)) return 'font';
+    if (/^image\//.test(contentType) || /\.(png|jpe?g|gif|webp|avif|svg|ico|bmp)(\?|$)/i.test(u)) return 'image';
+    return 'media';
+  };
+  let added = 0;
+  const queue = [...urls];
+  const worker = async () => {
+    for (let u = queue.shift(); u; u = queue.shift()) {
+      try {
+        const res = await fetch(u, { headers: { 'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36', referer: pageUrl, accept: '*/*' }, signal: AbortSignal.timeout(20000), redirect: 'follow' });
+        if (!res.ok) continue;
+        const body = Buffer.from(await res.arrayBuffer());
+        if (!body.length || body.length > MAX_ASSET_BYTES) continue;
+        const contentType = res.headers.get('content-type') || '';
+        if (/text\/html/.test(contentType)) continue; // a page, not media (a soft 404)
+        const type = typeOf(u, contentType);
+        const parsed = new URL(u);
+        const base = (path.basename(parsed.pathname) || 'index').replace(/[^a-z0-9._-]/gi, '_').slice(-80);
+        const hash = crypto.createHash('sha1').update(u).digest('hex').slice(0, 8);
+        const file = path.join('assets', type, `${hash}-${base}`);
+        await fs.mkdir(path.join(ref, 'assets', type), { recursive: true });
+        await fs.writeFile(path.join(ref, file), body);
+        manifest.push({ url: u, type, file: file.replaceAll('\\', '/'), contentType, bytes: body.length, fetched: 'mirror' });
+        added++;
+      } catch { /* offline, blocked, timed out: stays remote and is listed */ }
+    }
+  };
+  await Promise.all(Array.from({ length: 6 }, worker));
+  return added;
 }

package/scripts/detail.mjs

@@ -155,7 +155,7 @@ async function sample(browser, url, vp, probes) {
       if (y >= max) break;
     }
     await context.close();
-    return { list, results };
+    return { list, results, pageHeight: max + vp.height };
   }
   const list = probes;
   const results = new Array(list.length);
@@ -177,14 +177,27 @@ const report = { name, at: new Date().toISOString(), viewports: [] };
 try {
   for (const vpName of wanted) {
     const vp = VIEWPORTS.find((v) => v.name === vpName);
-    // reference probes are cached until the next capture (--refresh-ref to re-measure)
-    const ref = await cached(root, { tool: 'detail', vp: vpName, max: MAX, v: 8 }, () => sample(browser, meta.url, vp, null), Boolean(flags['refresh-ref']));
+    // reference probes are cached until the next capture (--refresh-ref to re-measure). The reference is read twice:
+    // a probe that does not agree with itself (random motion, a hover that lands by timing, a rotating word) can
+    // never be matched by any clone and is set aside as unstable instead of being handed over as a gap.
+    const ref = await cached(root, { tool: 'detail', vp: vpName, max: MAX, v: 9 }, async () => {
+      const first = await sample(browser, meta.url, vp, null);
+      // a live site changes: a reference read against today's page scores a clone of yesterday's capture against the
+      // wrong thing. The page height is a cheap tell; the fix is a new capture, not a new clone.
+      const captured = meta.viewports?.find((v) => v.viewport === vpName)?.pageHeight;
+      if (captured && first.pageHeight && Math.abs(first.pageHeight - captured) > Math.max(40, captured * 0.01)) console.warn(`WARNING reference drift (${vpName}): the live page is ${first.pageHeight}px tall, the capture was ${captured}px (${meta.capturedAt}); re-capture before trusting this score`);
+      const again = await sample(browser, meta.url, vp, first.list.map(({ x, xs, y, kind, w, h, scroll }) => ({ x, xs, y, kind, w, h, scroll })));
+      const unstable = first.list.map((p, i) => JSON.stringify(first.results[i]?.style) !== JSON.stringify(again.results[i]?.style) || first.results[i]?.sel !== again.results[i]?.sel);
+      return { ...first, unstable };
+    }, Boolean(flags['refresh-ref']));
     const cl = await sample(browser, cloneUrl, vp, ref.list.map(({ x, xs, y, kind, w, h, scroll }) => ({ x, xs, y, kind, w, h, scroll })));
     const issues = [];
+    const unstableList = [];
     let clean = 0, missing = 0;
     ref.list.forEach((p, i) => {
       const a = ref.results[i], b = cl.results[i];
       if (!a?.style) return;
+      if (ref.unstable?.[i]) { unstableList.push({ y: p.y, kind: p.kind, ref: a.sel }); return; }
       if (!b?.style) { missing++; issues.push({ y: p.y, kind: p.kind, ref: a.sel, clone: b?.top || '(nothing)', props: [`no ${p.kind} element at this point on the clone`] }); return; }
       const A = a.style, B = b.style, props = [];
       if (p.kind === 'text') {
@@ -203,10 +216,10 @@ try {
       if (props.length) issues.push({ y: p.y, kind: p.kind, ref: a.sel, clone: b.sel, props });
       else clean++;
     });
-    const total = ref.list.filter((p, i) => ref.results[i]?.style).length;
+    const total = ref.list.filter((p, i) => ref.results[i]?.style && !ref.unstable?.[i]).length;
     const byProp = {};
     issues.forEach((it) => it.props.forEach((pr) => { const k = pr.split(' ')[0] === 'no' ? 'missing element' : pr.split(' ')[0]; byProp[k] = (byProp[k] || 0) + 1; }));
-    report.viewports.push({ viewport: vpName, probes: total, clean, missing, score: +(100 * clean / Math.max(1, total)).toFixed(1), byProp, issues });
+    report.viewports.push({ viewport: vpName, probes: total, clean, missing, unstable: unstableList.length, score: +(100 * clean / Math.max(1, total)).toFixed(1), byProp, issues, unstableProbes: unstableList });
   }
 } finally {
   if (!flags.cdp) await browser.close();
@@ -216,7 +229,7 @@ const out = path.join(root, 'compare', 'detail.json');
 await fs.mkdir(path.dirname(out), { recursive: true });
 await fs.writeFile(out, JSON.stringify(report, null, 2));
 for (const v of report.viewports) {
-  console.log(`\n${v.viewport}: detail match ${v.score}% (${v.clean}/${v.probes} probes clean, ${v.missing} with no counterpart)  by property: ${JSON.stringify(v.byProp)}`);
+  console.log(`\n${v.viewport}: detail match ${v.score}% (${v.clean}/${v.probes} probes clean, ${v.missing} with no counterpart${v.unstable ? `; ${v.unstable} unstable on the reference itself, not scored: ${[...new Set(v.unstableProbes.map((u) => u.ref))].slice(0, 4).join(', ')}` : ''})  by property: ${JSON.stringify(v.byProp)}`);
   // group identical issues so one root cause prints once with a count
   const groups = new Map();
   for (const it of v.issues) { const k = `${it.ref} -> ${it.clone}: ${it.props.join('; ')}`; const g = groups.get(k) || { ...it, n: 0 }; g.n++; groups.set(k, g); }

package/scripts/diff.mjs

@@ -32,12 +32,27 @@ try {
     if (structure) await mask(page);
     const clonePath = path.join(out, `${vp.name}${suffix}.clone.png`);
     await page.screenshot({ path: clonePath, fullPage: true });
+    // View-mounted content (marked by capture): a full-page still can't show it on either side, so its boxes are
+    // painted the same flat colour in both images; the scroll-state check scores it with a real viewport.
+    const excluded = await page.evaluate(() => [...document.querySelectorAll('[data-tc-viewmount]')]
+      .map((e) => { const r = e.getBoundingClientRect(); return [r.left, r.top + scrollY, r.width, r.height].map(Math.round); })
+      .filter((r) => r[2] > 0 && r[3] > 0));
     await context.close();
 
     const a0 = PNG.sync.read(await fs.readFile(path.join(ref, `${vp.name}${suffix}.png`)));
     const b0 = PNG.sync.read(await fs.readFile(clonePath));
     const w = Math.max(a0.width, b0.width), h = Math.max(a0.height, b0.height);
     const a = pad(a0, w, h), b = pad(b0, w, h);
+    for (const [x, y, rw, rh] of excluded) {
+      for (const img of [a, b]) {
+        for (let yy = Math.max(0, y); yy < Math.min(h, y + rh); yy++) {
+          for (let xx = Math.max(0, x); xx < Math.min(w, x + rw); xx++) {
+            const i = (yy * w + xx) * 4;
+            img.data[i] = 64; img.data[i + 1] = 64; img.data[i + 2] = 64; img.data[i + 3] = 255;
+          }
+        }
+      }
+    }
     const diff = new PNG({ width: w, height: h });
     const bad = match(a, b, diff);
     await fs.writeFile(path.join(out, `${vp.name}${suffix}.diff.png`), PNG.sync.write(diff));
@@ -57,6 +72,7 @@ try {
       match: +(100 - (bad / (w * h)) * 100).toFixed(2),
       heightRef: a0.height,
       heightClone: b0.height,
+      excludedViewMounted: excluded.length,
       worstSections: [...perSection].sort((x, y) => x.match - y.match).slice(0, 5),
       sections: perSection,
     });

package/scripts/drift.mjs

@@ -0,0 +1,48 @@
+// Has the live page changed since the capture? Layout and scroll states score against the captured screenshots, but
+// detail, hovers, motion curves, interactions and the load sequence measure the live page: when the site changed in
+// between, those checks compare the clone with something it was never a copy of. One page load per width: the page
+// height and the number of top-level sections against meta.json. Written to compare/drift.json; REPORT.md marks the
+// live-measured checks "stale capture" and the advice is to re-capture, not to touch the clone.
+// usage: node scripts/drift.mjs <name> [--viewport desktop|all]
+import fs from 'node:fs/promises';
+import path from 'node:path';
+import { VIEWPORTS, parseArgs, openBrowser, newPage, settle, sectionsOf } from './lib.mjs';
+
+const { pos, flags } = parseArgs(process.argv.slice(2));
+const [name] = pos;
+if (!name) {
+  console.error('usage: trueclone drift <name> [--viewport desktop|all]');
+  process.exit(1);
+}
+const root = path.resolve('sites', name);
+const meta = JSON.parse(await fs.readFile(path.join(root, 'reference', 'meta.json'), 'utf8'));
+const vps = !flags.viewport || flags.viewport === 'all' ? VIEWPORTS : [VIEWPORTS.find((v) => v.name === flags.viewport)].filter(Boolean);
+
+const browser = await openBrowser(flags.cdp);
+const viewports = [];
+try {
+  for (const vp of vps) {
+    const captured = meta.viewports?.find((v) => v.viewport === vp.name);
+    if (!captured) continue;
+    const { context, page } = await newPage(browser, vp);
+    try {
+      await page.goto(meta.url, { waitUntil: 'load', timeout: 60000 });
+      await settle(page, meta.hide || []);
+      const livePageHeight = await page.evaluate(() => document.documentElement.scrollHeight);
+      const liveSections = (await sectionsOf(page)).length;
+      // 1% (at least 40px) absorbs font hinting and a rotating line of text; a new section or a block that moved is above that
+      const stale = Math.abs(livePageHeight - captured.pageHeight) > Math.max(40, captured.pageHeight * 0.01) || liveSections !== captured.sections;
+      viewports.push({ viewport: vp.name, livePageHeight, capturedPageHeight: captured.pageHeight, liveSections, capturedSections: captured.sections, stale });
+      console.log(`${vp.name}: live ${livePageHeight}px / ${liveSections} sections, captured ${captured.pageHeight}px / ${captured.sections} sections${stale ? '  <- changed since the capture' : '  (same)'}`);
+    } finally {
+      await context.close();
+    }
+  }
+} finally {
+  if (!flags.cdp) await browser.close();
+}
+const stale = viewports.some((v) => v.stale);
+await fs.mkdir(path.join(root, 'compare'), { recursive: true });
+await fs.writeFile(path.join(root, 'compare', 'drift.json'), JSON.stringify({ at: new Date().toISOString(), capturedAt: meta.capturedAt, stale, viewports }, null, 2));
+if (stale) console.warn(`WARNING reference drift: the live page has changed since the capture of ${meta.capturedAt}; detail, hovers, curves, interactions and load now measure a page the clone was never a copy of. Re-capture (trueclone capture, then states, reveals, seed --force) before trusting them.`);
+else console.log(`drift: none since ${meta.capturedAt}`);