@trueclone/cli 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,11 +1,11 @@
1
1
  # trueclone
2
2
 
3
- **A working twin of any website, from one URL, with scores that prove it.**
3
+ **The exact same website, from one URL, at every screen size, with the checks to prove it.**
4
4
 
5
- Give your coding agent a URL. trueclone captures the site, extracts its design system, guides the rebuild and measures the result against the original until it matches: layout, scroll states, motion curves, interactions, hover effects and computed styles, at phone, tablet and desktop widths. You get the clone as HTML and as a Next.js app, plus the design system as `DESIGN.md` and a shadcn `globals.css`.
5
+ Give your coding agent a URL. trueclone captures the site, extracts its design system, seeds the clone with the original's structure, hands the agent the original's own scripts to port every behaviour from, and measures the result against the original until it matches: layout, scroll states, canvases, motion, interactions, hovers, styles and the load sequence, at the sizes it was built from and at sizes it never saw, plus a frame-by-frame visual review. You get the clone as HTML and as a Next.js app, plus the design system as `DESIGN.md` and a shadcn `globals.css`.
6
6
 
7
7
  ```bash
8
- npx @trueclone/cli init # once per project: installs trueclone, the skill for your agent, REA as an MCP server
8
+ npx @trueclone/cli init # once per project: installs trueclone and the skill for your agent
9
9
  # then tell Claude Code, Codex or Cursor:
10
10
  # "clone https://example.com with trueclone"
11
11
  ```
@@ -23,7 +23,7 @@ npx @trueclone/cli init # once per project: installs trueclone, the s
23
23
 
24
24
  ## Proof
25
25
 
26
- Three benchmarks with very different stacks, scored by the tool itself:
26
+ Three benchmarks with very different stacks, scored by the tool itself. These numbers were measured at the three build sizes only, before trueclone 0.4 added held-out sizes and the visual review; 0.4's benchmark runs replace them.
27
27
 
28
28
  **[rea.tools](https://rea.tools/)**: static docs with a canvas game and interactive demos, exact mode (copied with the owner's permission).
29
29
 
@@ -60,33 +60,25 @@ The hand-built portfolio isn't perfect yet, and its report says so: computed sty
60
60
  ## How it works
61
61
 
62
62
  ```
63
- URL ─► capture ─► understand ─► design system ─► build ─► score ─► report
64
- 3 widths motion DESIGN.md your 8 checks REPORT.md
65
- HAR, HTML inventory globals.css agent in parallel
66
- assets + REA tokens.css ▲ │
67
- └─────┘ until every target passes
63
+ URL ─► capture ─► understand ─► design system ─► seed ─► build ─► score + review ─► report
64
+ 5 sizes the site's DESIGN.md structure your 10 checks at REPORT.md
65
+ scripts own scripts, globals.css + viewport agent build and held-out
66
+ HAR motion runtime tokens.css functions sizes, by eye too
67
+ ▲ │
68
+ └────────────┘ until every check passes
68
69
  ```
69
70
 
70
- 1. **Capture** (`trueclone capture`): full-page screenshots at 390, 768 and 1440px, sections, rendered HTML, assets, every network request.
71
- 2. **Understand** (`trueclone motion`, `trueclone understand`): reads the live animation runtime (GSAP timelines, ScrollTriggers, Lenis and Swiper settings, CSS animations, cursor-driven tweens) and, with REA, which scripts actually run while a visitor scrolls, which handler each control has and the code around it, the endpoints in the code, source maps when published: `code.md`, so the agent copies rules instead of guessing them from effects.
71
+ 1. **Capture** (`trueclone capture`): full-page screenshots at five sizes (390, 768 and 1440 wide to build from; 1280 and 1920 held out, never used to build), sections, rendered HTML, assets, every network request, every script the page ran (with what ran and where each event listener lives), and the inline values scripts set, read at extra sizes inside each of the site's breakpoint ranges.
72
+ 2. **Understand**: `reference/scripts/INDEX.md` (the site's own code, ranked by what ran), the live animation runtime (`trueclone motion`: GSAP timelines, ScrollTriggers, Lenis, Swiper, CSS animations) and the scroll states. The agent ports each behaviour's rule from the site's code instead of guessing it from effects.
72
73
  3. **Design system** (`trueclone design`): one canonical `design-tokens.json`, rendered to DESIGN.md, globals.css and the clone's tokens.
73
- 4. **Build**: `trueclone reveals` records, at every width, the original's scroll reveals frame by frame, its scroll-linked curves (parallax) and its scroll-driven class changes; `trueclone widgets` clicks every tab and menu and records each state; `trueclone seed` turns the script-free mirror into the clone (template mode: layout-measured lorem text and same-size media placeholders), cleans what the original's scripts left in the DOM (pin-spacers, canvas buffer sizes) and replays what was recorded. Layout is at 99%+ before anyone writes CSS; your agent writes the remaining behaviour, guided by the skill. `trueclone rules` shows the original's CSS for any element (like DevTools).
74
- 5. **Score** (`trueclone score`): layout pixel diff per section, scroll states replayed, motion curves sampled, interaction scenarios with change assertions, every hover, computed styles at matching points, and the load sequence (how settled each side is at 0.25 to 4 s after navigation, cold cache, same emulated network). Reference measurements are cached, so only the clone is re-measured.
75
- 6. **Report**: `REPORT.md` with the verdict, what to fix first, and what was not copied.
74
+ 4. **Seed** (`trueclone seed`): the script-free mirror becomes the clone (template mode: layout-measured lorem text and same-size media placeholders). A value a script computed from the screen size is carried as that function (`calc()`), never as one size's number; a value no such function explains is left to the agent and listed. Pins pin, one-shot reveals and scroll class changes replay, shadow roots render, the first paint is held until the clone's scripts have run.
75
+ 5. **Build**: the agent writes `clone/script.js`, scene by scene, from the site's own rules; `clone/patches/` for anything on top of the seed; `trueclone rules` shows the original's CSS for any element.
76
+ 6. **Score and review** (`trueclone score`, `trueclone review`): layout, scroll states, canvases, motion curves, details, hovers (outcome, not just "something changed"), interactions (same end state), load sequence, at the build sizes and the held-out sizes; then the reference and the clone side by side, frame by frame, with a verdict for each.
77
+ 7. **Report**: `REPORT.md` with every check at every size, the review, what to fix first, and what was not copied.
76
78
 
77
- ## Why not just REA?
79
+ ## REA
78
80
 
79
- trueclone **bundles** [REA](https://github.com/morluto/rea) (Reverse Engineer Anything) and uses it, but works without it too.
80
-
81
- | | REA on its own | trueclone |
82
- |---|---|---|
83
- | What it is | A microscope: typed tools that let your agent look inside apps, binaries and pages | A factory with quality control: capture, design system, build guide, scores, report |
84
- | Capture | Your Chrome over CDP, one page state | 3 widths, every scroll state, motion runtime, assets, network |
85
- | Knows when the clone is right | No: the agent judges by eye | Yes: 7 measured checks with targets |
86
- | Tells you what's wrong | No | "Footer column 2 is at x 189, the original's at 732", plus the original's CSS rule |
87
- | Outputs | Whatever the agent writes | HTML clone, Next.js app, DESIGN.md, globals.css, motion recipe, REPORT.md |
88
-
89
- REA answers *how does this work?* trueclone answers *is my copy identical, and where isn't it?* trueclone proves the visual and behavioural match on its own; `trueclone understand` adds REA's code-level evidence (which scripts ran, the handler behind each control with its source, endpoints, original sources when published) so the agent copies rules instead of inferring them. For the hard parts (a bundled physics rule, a shader, a module's data flow) REA's own CLI is one command away.
81
+ trueclone bundles [REA](https://github.com/morluto/rea) (Reverse Engineer Anything) for the rare part no script reading explains (a bundled physics rule, a shader, WASM): `npx trueclone rea --help`. It is not part of the standard flow: the site's scripts, saved and ranked by the capture, carry what the agent needs.
90
82
 
91
83
  ## Modes and guardrails
92
84
 
@@ -103,22 +95,22 @@ REA answers *how does this work?* trueclone answers *is my copy identical, and w
103
95
 
104
96
  ```bash
105
97
  npx @trueclone/cli init # set up a project for your agent (then `npx trueclone ...`)
106
- npx trueclone doctor # check Node, Chrome and REA
98
+ npx trueclone doctor # check Node and Chrome
107
99
  npx trueclone <url> [name] # capture + design-system draft
108
100
  npx trueclone capture <url> [name] [--mode exact --permission "<owner>"]
109
101
  npx trueclone motion <name> # motion inventory
110
- npx trueclone understand <name> # code evidence with REA: which scripts ran, each control's handler with its source -> code.md
111
102
  npx trueclone design <name> # DESIGN.md + globals.css
112
103
  npx trueclone reveals <name> # record scroll reveals and scroll class changes
113
104
  npx trueclone widgets <name> # record every tab and menu state
114
- npx trueclone seed <name> # clone/ from the mirror (template: placeholder text and media)
105
+ npx trueclone seed <name> # clone/ from the mirror (template: placeholder text and media); --patches re-applies clone/patches/ in a second
115
106
  npx trueclone rules <name> "<selector>" # the original's CSS rules for an element
116
107
  npx trueclone textshape <name> --write # template mode: placeholder text with the original's line breaks
117
108
  npx trueclone load <name> # load sequence on both sides: settledness 0.25-4 s after navigation, cold cache, same network
118
109
  npx trueclone score <name> [--quick] # every check, then REPORT.md (--only detail,load --viewport tablet after a fix; unchanged checks are kept)
119
110
  npx trueclone explain <name> <y> # one place on the page: what every check saw there, plus a reference|clone|diff crop
120
- npx trueclone next <name> --verify # Next.js + Tailwind + shadcn app, scored
121
- npx trueclone rea <args> # bundled REA
111
+ npx trueclone review <name> --size wide # reference and clone side by side at a held-out size: load, every section, hover
112
+ npx trueclone next <name> --verify # Next.js + Tailwind + shadcn app, scored into REPORT.next.md
113
+ npx trueclone rea <args> # bundled REA, for the rare deep case
122
114
  npx trueclone help # every step
123
115
  ```
124
116
 
@@ -137,7 +129,7 @@ Node 22.19+, 24.11+ or 26+ (the same as REA), and Google Chrome for captures (it
137
129
 
138
130
  - **Whole sites:** crawl every route, shared header and footer as components.
139
131
  - **Data layer:** captured API responses turned into mock routes and fixtures, so the clone works offline.
140
- - **Benchmarks:** React with Framer Motion, a Next.js site with source maps, a data-driven app; each scored with and without REA.
132
+ - **Benchmarks by kind of site:** static, Tailwind/Next.js, Webflow, Webflow + GSAP, Framer, WordPress builders, WebGL showcases, docs, e-commerce, SPAs, scroll-jacking, embeds: each scored at build and held-out sizes, published here.
141
133
  - **A free web page:** paste a URL, get its DESIGN.md and globals.css.
142
134
 
143
135
  ## Credits
package/bin/trueclone.mjs CHANGED
@@ -19,27 +19,28 @@ const require = createRequire(import.meta.url);
19
19
  const { name, version } = JSON.parse(await fs.readFile(path.join(PKG, 'package.json'), 'utf8'));
20
20
 
21
21
  const STEPS = {
22
- capture: ['capture.mjs', 'capture a live page: screenshots at 390/768/1440, sections, tokens, rendered HTML, assets'],
23
- seed: ['seed.mjs', 'exact mode: start clone/ from the script-free mirror (markup, text, styles, media)'],
22
+ capture: ['capture.mjs', 'capture a live page at 5 sizes (3 to build from, 2 held out), its scripts, assets, rendered HTML, inline values at extra sizes'],
23
+ seed: ['seed.mjs', 'clone/ from the script-free mirror: structure, viewport functions, pins, reveals (--patches: re-apply clone/patches/ only)'],
24
24
  design: ['design.mjs', 'design-tokens.json -> DESIGN.md (linted), shadcn globals.css, clone/tokens.css'],
25
25
  motion: ['motion.mjs', 'motion inventory from the live runtime (GSAP/ScrollTrigger, Lenis, Swiper, CSS animations)'],
26
- understand: ['understand.mjs', 'code evidence with REA: which scripts ran, each element\'s handler with its source, endpoints, source maps -> reference/code.md'],
26
+ understand: ['understand.mjs', 'optional, rarely needed: REA code evidence -> reference/code.md (capture already saves every script, ranked, in reference/scripts/)'],
27
27
  reveals: ['reveals.mjs', 'record one-shot scroll reveals frame by frame, so seed can replay them (any library)'],
28
28
  widgets: ['widgets.mjs', 'record every tab and menu state by clicking it, so seed can replay them (template mode)'],
29
- states: ['states.mjs', 'live scroll states; --clone scores the rebuild step by step'],
29
+ states: ['states.mjs', 'live scroll states at every scored size; --clone scores the rebuild step by step (canvases too)'],
30
30
  curves: ['curves.mjs', 'motion score: sampled property curves, reference vs clone (motion-map.json)'],
31
31
  interact: ['interact.mjs', 'interaction tests: click/hover/drag/key scenarios on both sides (interactions.json)'],
32
32
  detail: ['detail.mjs', 'style diff at matching points: font, size, weight, line height, colour, background, radius'],
33
- hovers: ['hovers.mjs', 'hover every interactive element on both sides and compare what changes'],
33
+ hovers: ['hovers.mjs', 'hover every control and every element with a hover listener on both sides; the outcome must match'],
34
34
  load: ['load.mjs', 'load sequence: how settled each side is 0.25/0.5/1/2/4 s after navigation, cold cache, same network'],
35
35
  drift: ['drift.mjs', 'has the live page changed since the capture? (page height and sections per width vs meta.json)'],
36
36
  owned: ['owned.mjs', 'which sections of the clone script.js rebuilds at boot (load with scripts blocked vs running)'],
37
37
  rules: ['rules.mjs', 'every CSS rule matching an element on the reference (+ ::before/::after, @media); --clone for the clone'],
38
+ review: ['review.mjs', 'reference and clone side by side at a held-out size: load, every section, hover; verdicts go in notes.md'],
38
39
  explain: ['explain.mjs', 'one place on the page: what layout, states, detail and hovers saw there, plus a reference|clone|diff crop'],
39
40
  textshape: ['textshape.mjs', 'template mode: placeholder words with the reference word widths, so lines break the same'],
40
41
  diff: ['diff.mjs', 'full-page pixel diff per viewport and section (--structure masks text and media)'],
41
42
  spec: ['spec.mjs', 'measured boxes and styles of the live page (--clone for the rebuild)'],
42
- tiles: ['tiles.mjs', 'cut a tall screenshot into readable tiles'],
43
+ tiles: ['tiles.mjs', 'the full-page shots cut into one tile per screen: tiles <name> [--size wide] [--clone], or tiles <file.png>'],
43
44
  next: ['next.mjs', 'Next.js + Tailwind v4 + shadcn app from the verified clone; --verify scores it'],
44
45
  score: ['score.mjs', 'every score in parallel groups, time limit per step, writes REPORT.md'],
45
46
  report: ['report.mjs', 'REPORT.md from the latest scores: every number, every gap, what cannot be copied'],
@@ -56,14 +57,14 @@ const run = (file, args) => new Promise((resolve) => {
56
57
  });
57
58
 
58
59
  function help() {
59
- console.log(`trueclone ${version}: a working twin of a website, proven by scores
60
+ console.log(`trueclone ${version}: the exact same website, at every screen size
60
61
 
61
62
  usage
62
- npx @trueclone/cli init set up this folder for your coding agent (installs trueclone)
63
+ npx @trueclone/cli init set up this folder for your coding agent (installs trueclone; --with-rea adds REA as MCP)
63
64
  trueclone <url> [name] capture + design-system draft (template mode)
64
65
  trueclone <step> <name> [options] run one step
65
- trueclone rea <args> bundled REA (Reverse Engineer Anything)
66
- trueclone doctor check Node, browsers and REA
66
+ trueclone rea <args> bundled REA (Reverse Engineer Anything), for the rare deep case
67
+ trueclone doctor check Node and browsers
67
68
 
68
69
  steps
69
70
  ${Object.entries(STEPS).map(([k, [, d]]) => ` ${k.padEnd(10)} ${d}`).join('\n')}
@@ -85,11 +86,15 @@ async function init() {
85
86
  let agents = (await exists(agentsFile)) ? await fs.readFile(agentsFile, 'utf8') : '';
86
87
  agents = agents.includes('<!-- trueclone:start -->') ? agents.replace(/<!-- trueclone:start -->[\s\S]*?<!-- trueclone:end -->\n?/, block) : `${agents}${agents && !agents.endsWith('\n\n') ? '\n' : ''}${block}`;
87
88
  await fs.writeFile(agentsFile, agents);
88
- // 3. REA as an MCP server for agents that speak MCP (Claude Code reads .mcp.json)
89
- const mcpFile = path.join(cwd, '.mcp.json');
90
- const mcp = (await exists(mcpFile)) ? JSON.parse(await fs.readFile(mcpFile, 'utf8')) : {};
91
- mcp.mcpServers = { ...(mcp.mcpServers || {}), rea: { command: 'npx', args: ['-y', `${name}@${version}`, 'rea', 'mcp'] } };
92
- await fs.writeFile(mcpFile, `${JSON.stringify(mcp, null, 2)}\n`);
89
+ // 3. REA as an MCP server only on request (--with-rea): the standard flow reads the site's own scripts from the
90
+ // capture; REA is for the rare deep case and stays reachable as `npx trueclone rea` either way
91
+ const withRea = process.argv.includes('--with-rea');
92
+ if (withRea) {
93
+ const mcpFile = path.join(cwd, '.mcp.json');
94
+ const mcp = (await exists(mcpFile)) ? JSON.parse(await fs.readFile(mcpFile, 'utf8')) : {};
95
+ mcp.mcpServers = { ...(mcp.mcpServers || {}), rea: { command: 'npx', args: ['-y', `${name}@${version}`, 'rea', 'mcp'] } };
96
+ await fs.writeFile(mcpFile, `${JSON.stringify(mcp, null, 2)}\n`);
97
+ }
93
98
  // 4. captures, clones and browser profiles stay local
94
99
  const ignoreFile = path.join(cwd, '.gitignore');
95
100
  let ignore = (await exists(ignoreFile)) ? await fs.readFile(ignoreFile, 'utf8') : '';
@@ -108,8 +113,7 @@ async function init() {
108
113
  console.log(`trueclone set up in ${cwd}
109
114
  .claude/skills/trueclone/SKILL.md the workflow your agent follows
110
115
  AGENTS.md points Codex, Cursor and others at the skill
111
- .mcp.json REA as an MCP server
112
- .gitignore sites/ stays local
116
+ ${withRea ? ' .mcp.json REA as an MCP server\n' : ''} .gitignore sites/ stays local
113
117
  package.json ${installed}
114
118
  next: npx trueclone doctor, then ask your agent "clone <url> with trueclone"`);
115
119
  }
@@ -125,8 +129,8 @@ async function doctor() {
125
129
  try { const b = await chromium.launch(opts); await b.close(); console.log(`ok ${label}`); }
126
130
  catch { console.log(`warn ${label} not found${opts.channel ? ': install Google Chrome' : ': npx playwright install chromium'}`); }
127
131
  }
128
- try { require.resolve('rea-agents/package.json'); console.log('ok REA bundled (trueclone rea doctor checks its decompilers)'); }
129
- catch { console.log('warn REA not installed'); ok = false; }
132
+ try { require.resolve('rea-agents/package.json'); console.log('ok REA bundled, for the rare deep case (trueclone rea doctor checks its decompilers)'); }
133
+ catch { console.log('warn REA not installed (optional)'); }
130
134
  process.exitCode = ok ? 0 : 1;
131
135
  }
132
136
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@trueclone/cli",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "A working twin of any website from its URL: HTML clone, Next.js + Tailwind + shadcn app, DESIGN.md and globals.css, proven by layout, motion, interaction and style scores. Bundles REA.",
5
5
  "type": "module",
6
6
  "bin": {
@@ -22,6 +22,7 @@
22
22
  },
23
23
  "dependencies": {
24
24
  "@google/design.md": "^0.4.0",
25
+ "js-beautify": "2.0.3",
25
26
  "parse5": "^8.0.1",
26
27
  "pixelmatch": "^7.2.0",
27
28
  "playwright": "^1.64.0",
@@ -5,7 +5,8 @@
5
5
  import fs from 'node:fs/promises';
6
6
  import path from 'node:path';
7
7
  import crypto from 'node:crypto';
8
- import { VIEWPORTS, parseArgs, siteName, openBrowser, newPage, settle, sectionsOf, mask } from './lib.mjs';
8
+ import { SCORED, VIEWPORTS, HELD_OUT, parseArgs, siteName, openBrowser, newPage, settle, sectionsOf, mask } from './lib.mjs';
9
+ import { startScriptEvidence } from './scripts-evidence.mjs';
9
10
 
10
11
  const { pos, flags } = parseArgs(process.argv.slice(2));
11
12
  const [url, nameArg] = pos;
@@ -93,10 +94,13 @@ const manifest = [];
93
94
  const summary = [];
94
95
  const rootFontSize = {};
95
96
  const inlineStyles = {};
97
+ const scriptQueries = new Set(); // media queries the page's scripts asked (matchMedia), at any size
96
98
  try {
97
- for (const vp of VIEWPORTS) {
99
+ // the seed sizes and the held-out sizes: the held-out ones are shot and measured for scoring, nothing else
100
+ for (const vp of SCORED) {
98
101
  const desktop = vp.name === 'desktop';
99
102
  const { context, page } = await newPage(browser, vp, desktop ? { recordHar: { path: path.join(ref, 'desktop.har'), content: 'omit' } } : {});
103
+ await page.addInitScript(recordMediaQueries);
100
104
  if (desktop) {
101
105
  page.on('response', async (res) => {
102
106
  const rawType = res.request().resourceType();
@@ -115,6 +119,8 @@ try {
115
119
  } catch { /* redirects, data: urls, aborted */ }
116
120
  });
117
121
  }
122
+ // desktop: the page's own scripts and what ran, saved as evidence (reference/scripts/)
123
+ const finishEvidence = desktop ? await startScriptEvidence(page, url).catch(() => null) : null;
118
124
  await page.goto(url, { waitUntil: 'load', timeout: 60000 });
119
125
  // Components that mount only while in view (a lazy wordmark, an embed) are often unmounted again by the time
120
126
  // the DOM is saved. Watch from load: an element that was empty, got content while scrolled into view and is empty
@@ -138,6 +144,16 @@ try {
138
144
  }).observe(document.body, { subtree: true, childList: true });
139
145
  });
140
146
  await settle(page, hide);
147
+ // inline styles at this size, by element path: scripts size and place things from the viewport (a grid's computed
148
+ // cells, a pin's distance), and the rendered HTML only keeps one size's values. With the extra sizes read below,
149
+ // the seed tells a function of the viewport from a constant. Read here, settled and before any screenshot, in the
150
+ // same state as the extra sizes (a full-page shot mounts content and fires reveals).
151
+ if (!vp.heldOut) inlineStyles[`${vp.width}x${vp.height}`] = await page.evaluate(readInlineStyles);
152
+ for (const q of await page.evaluate(() => window.__tcMediaQueries || []).catch(() => [])) scriptQueries.add(q);
153
+ if (finishEvidence) {
154
+ const ev = await finishEvidence(path.join(ref, 'scripts')).catch((e) => { console.warn(`WARNING scripts not saved: ${e.message}`); return null; });
155
+ if (ev) console.log(`scripts: ${ev.scripts} saved to reference/scripts/ (${ev.ran} ran during load and scroll), ${ev.listeners} event-listener handlers located (INDEX.md)`);
156
+ }
141
157
  // A full-page screenshot brings the whole page into view at once, which mounts view-mounted components and starts
142
158
  // their reveals: shoot until two shots agree, so the reference shows a settled page, not a fade caught at 0%.
143
159
  let shot = await page.screenshot({ fullPage: true });
@@ -151,26 +167,6 @@ try {
151
167
  await fs.writeFile(path.join(ref, `${vp.name}.png`), shot);
152
168
  const sections = await sectionsOf(page);
153
169
  await fs.writeFile(path.join(ref, `${vp.name}.sections.json`), JSON.stringify(sections, null, 2));
154
- // inline styles at this width, by element path: scripts size and place things per width (a grid's computed
155
- // height, a card's offset), and the rendered HTML only keeps the desktop values. The seed turns what differs
156
- // between widths into media-query rules.
157
- inlineStyles[vp.name] = await page.evaluate(() => {
158
- const SKIP = new Set(['SCRIPT', 'NOSCRIPT', 'TEMPLATE', 'STYLE', 'LINK', 'META']);
159
- const pathOf = (el) => {
160
- const parts = [];
161
- for (let n = el; n && n !== document.body; n = n.parentElement) {
162
- const sibs = [...n.parentElement.children].filter((c) => !SKIP.has(c.tagName));
163
- parts.unshift(`${n.tagName.toLowerCase()}:${sibs.indexOf(n) + 1}`);
164
- }
165
- return parts.join('>');
166
- };
167
- const out = {};
168
- for (const el of document.body.querySelectorAll('[style]')) {
169
- if (SKIP.has(el.tagName) || (el.closest('svg') && el.tagName !== 'svg')) continue;
170
- out[pathOf(el)] = el.getAttribute('style');
171
- }
172
- return out;
173
- });
174
170
  const height = await page.evaluate(() => document.documentElement.scrollHeight);
175
171
  rootFontSize[vp.name] = await page.evaluate(() => getComputedStyle(document.documentElement).fontSize);
176
172
  if (desktop) {
@@ -218,7 +214,7 @@ try {
218
214
  }
219
215
  await mask(page);
220
216
  await page.screenshot({ path: path.join(ref, `${vp.name}.mask.png`), fullPage: true });
221
- summary.push({ viewport: vp.name, width: vp.width, pageHeight: height, sections: sections.length });
217
+ summary.push({ viewport: vp.name, width: vp.width, height: vp.height, ...(vp.heldOut ? { heldOut: true } : {}), pageHeight: height, sections: sections.length });
222
218
  await context.close(); // flushes the HAR
223
219
  }
224
220
  } finally {
@@ -226,7 +222,33 @@ try {
226
222
  }
227
223
 
228
224
  await fs.writeFile(path.join(ref, 'assets', 'manifest.json'), JSON.stringify(manifest, null, 2));
229
- await fs.writeFile(path.join(ref, 'inline-styles.json'), JSON.stringify(inlineStyles));
225
+
226
+ // The site's own breakpoints (its @media width queries) and extra sizes inside every range they make: inline styles
227
+ // read there let the seed fit each value a script set as a function of the viewport (a·vw + b·vh + c) per range, or
228
+ // see that it is none and leave it to the agent. Never the held-out sizes: those stay unseen by anything that builds.
229
+ const breakpoints = await siteBreakpoints(manifest, [...scriptQueries]);
230
+ const fitSizes = sizesForFit(breakpoints);
231
+ if (fitSizes.length) {
232
+ const queue = [...fitSizes];
233
+ const browser2 = await openBrowser(flags.cdp);
234
+ try {
235
+ await Promise.all(Array.from({ length: 3 }, async () => {
236
+ for (let s = queue.shift(); s; s = queue.shift()) {
237
+ const { context, page } = await newPage(browser2, { name: `fit${s.width}`, width: s.width, height: s.height });
238
+ try {
239
+ await page.goto(url, { waitUntil: 'load', timeout: 60000 });
240
+ await settle(page, hide);
241
+ inlineStyles[`${s.width}x${s.height}`] = await page.evaluate(readInlineStyles);
242
+ } catch (e) { console.warn(`WARNING inline styles at ${s.width}x${s.height} not read: ${e.message.split('\n')[0]}`); }
243
+ finally { await context.close(); }
244
+ }
245
+ }));
246
+ } finally {
247
+ if (!flags.cdp) await browser2.close();
248
+ }
249
+ }
250
+ await fs.writeFile(path.join(ref, 'inline-styles.json'), JSON.stringify({ breakpoints, scriptQueries: [...scriptQueries], sizes: Object.keys(inlineStyles), styles: inlineStyles }));
251
+ console.log(`inline styles read at ${Object.keys(inlineStyles).length} sizes (site breakpoints: ${breakpoints.join(', ') || 'none found'}; ${scriptQueries.size} width quer${scriptQueries.size === 1 ? 'y' : 'ies'} asked by its scripts)`);
230
252
  await fs.writeFile(path.join(ref, 'meta.json'), JSON.stringify({ url, name, mode, permission, hide, capturedAt: new Date().toISOString(), rootFontSize, viewports: summary }, null, 2));
231
253
  await buildMirror(manifest, url, mode === 'exact' ? `Exact mode, authorised by: ${permission}.` : "Template mode: the site's own content, kept as a reference only.");
232
254
  console.table(summary);
@@ -364,3 +386,93 @@ async function fetchMissing(urls, manifest, pageUrl) {
364
386
  await Promise.all(Array.from({ length: 6 }, worker));
365
387
  return added;
366
388
  }
389
+
390
+ // Inline styles and classes of every styled element, by element path, with the size actually laid out (clientWidth
391
+ // excludes a classic scrollbar; scripts measure that, not the window).
392
+ function readInlineStyles() {
393
+ const SKIP = new Set(['SCRIPT', 'NOSCRIPT', 'TEMPLATE', 'STYLE', 'LINK', 'META']);
394
+ const pathOf = (el) => {
395
+ const parts = [];
396
+ for (let n = el; n && n !== document.body; n = n.parentElement) {
397
+ const sibs = [...n.parentElement.children].filter((c) => !SKIP.has(c.tagName));
398
+ parts.unshift(`${n.tagName.toLowerCase()}:${sibs.indexOf(n) + 1}`);
399
+ }
400
+ return parts.join('>');
401
+ };
402
+ const styles = {};
403
+ for (const el of document.body.querySelectorAll('[style]')) {
404
+ if (SKIP.has(el.tagName) || (el.closest('svg') && el.tagName !== 'svg')) continue;
405
+ styles[pathOf(el)] = [el.getAttribute('style'), String(el.getAttribute('class') || '')];
406
+ }
407
+ return { w: document.documentElement.clientWidth, h: innerHeight, styles };
408
+ }
409
+
410
+ // Every media query the page's scripts ask (GSAP matchMedia, Webflow, a component's own breakpoint): what a script
411
+ // does can switch at a width no stylesheet mentions.
412
+ function recordMediaQueries() {
413
+ const seen = (window.__tcMediaQueries = []);
414
+ const original = window.matchMedia;
415
+ if (!original) return;
416
+ window.matchMedia = function matchMedia(q) {
417
+ if (typeof q === 'string' && /width/.test(q) && !seen.includes(q)) seen.push(q);
418
+ return original.call(this, q);
419
+ };
420
+ }
421
+
422
+ // The width breakpoints the site's stylesheets use at least twice (min-width: 992px is a boundary at 991), and every
423
+ // width its scripts ask about (always kept: a script's breakpoint is behaviour).
424
+ async function siteBreakpoints(manifest, scriptQueries = []) {
425
+ const count = new Map();
426
+ for (const q of scriptQueries) {
427
+ for (const m of q.matchAll(/(min|max)-width\s*:\s*([\d.]+)(px|em|rem)/g)) {
428
+ const px = Number(m[2]) * (m[3] === 'px' ? 1 : 16);
429
+ const edge = Math.round(m[1] === 'min' ? px - 1 : px);
430
+ if (edge >= 300 && edge <= 2600) count.set(edge, (count.get(edge) || 0) + 100);
431
+ }
432
+ }
433
+ const texts = [];
434
+ for (const m of manifest.filter((x) => x.type === 'stylesheet')) texts.push(await fs.readFile(path.join(ref, m.file), 'utf8').catch(() => ''));
435
+ const rendered = await fs.readFile(path.join(ref, 'rendered.html'), 'utf8').catch(() => '');
436
+ for (const m of rendered.matchAll(/<style\b[^>]*>([\s\S]*?)<\/style>/gi)) texts.push(m[1]);
437
+ for (const t of texts) {
438
+ for (const q of t.matchAll(/@media[^{]*/g)) {
439
+ for (const m of q[0].matchAll(/(min|max)-width\s*:\s*([\d.]+)(px|em|rem)/g)) {
440
+ const px = Number(m[2]) * (m[3] === 'px' ? 1 : 16);
441
+ const edge = Math.round(m[1] === 'min' ? px - 1 : px);
442
+ if (edge >= 300 && edge <= 2600) count.set(edge, (count.get(edge) || 0) + 1);
443
+ }
444
+ }
445
+ }
446
+ // breakpoints within 2px of each other are one (991 from max-width, 991 from min-width: 992)
447
+ const edges = [...count].filter(([, n]) => n >= 2).sort((a, b) => a[0] - b[0]);
448
+ const merged = [];
449
+ for (const [e, n] of edges) { const last = merged[merged.length - 1]; if (last && e - last.e <= 2) last.n += n; else merged.push({ e, n }); }
450
+ return merged.sort((a, b) => b.n - a.n).slice(0, 9).map((x) => x.e).sort((a, b) => a - b);
451
+ }
452
+
453
+ // Extra sizes for fitting: at least three points in every breakpoint range (counting the seed sizes in it), at
454
+ // different heights, so a value that depends on the width, the height or both can be told apart from a constant.
455
+ function sizesForFit(breakpoints) {
456
+ const edges = [320, ...breakpoints, 2400];
457
+ const taken = new Set([...VIEWPORTS, ...HELD_OUT].map((v) => `${v.width}x${v.height}`));
458
+ const out = [];
459
+ const heights = [760, 1000, 880];
460
+ for (let i = 0; i < edges.length - 1; i++) {
461
+ const lo = i === 0 ? edges[0] : edges[i] + 1, hi = edges[i + 1];
462
+ if (hi - lo < 8) { // a narrow band (760 and 767 both breakpoints) still has its own rules: sample both edges
463
+ for (const [w, h] of [[lo, heights[i % 3]], [hi, heights[(i + 1) % 3]]]) if (!taken.has(`${w}x${h}`)) { taken.add(`${w}x${h}`); out.push({ width: w, height: h }); }
464
+ continue;
465
+ }
466
+ const seeds = VIEWPORTS.filter((v) => v.width >= lo && v.width <= hi).length;
467
+ const want = Math.max(2, 3 - seeds);
468
+ for (let k = 0; k < want; k++) {
469
+ const w = Math.round(lo + (hi - lo) * [0.25, 0.75, 0.5][k]);
470
+ const h = heights[(i + k) % heights.length];
471
+ if (!taken.has(`${w}x${h}`)) { taken.add(`${w}x${h}`); out.push({ width: w, height: h }); }
472
+ }
473
+ }
474
+ // Two very wide, short screens: most sizes above are taller than 16:9, so a "cover" rule (max(100vw, 177.8vh))
475
+ // would have one point at most on its width side. Two make it checkable (see fitCover in viewport-fit.mjs).
476
+ for (const [w, h] of [[2400, 760], [2200, 900]]) if (!taken.has(`${w}x${h}`)) { taken.add(`${w}x${h}`); out.push({ width: w, height: h }); }
477
+ return out;
478
+ }
@@ -7,7 +7,7 @@
7
7
  // { "name": "quote reveal", "ref": ".is-quote .reveal-line", "clone": ".is-quote .split-line",
8
8
  // "props": ["ty"], "time": { "before": 0, "at": 700, "ms": 1400, "every": 100 } }
9
9
  //
10
- // selector@n picks the n-th match (default 0). Props: opacity, tx, ty, sx, sy, rot, top, left, width, height, saturate.
10
+ // selector@n picks the n-th match (default 0); path:<recorded path> picks an element the reveals recorder saw. Props: opacity, tx, ty, sx, sy, rot, top, left, width, height, saturate.
11
11
  // Viewport-independent ranges: "scroll": { "of": { "ref": "...", "clone": "..." }, "start": "top bottom", "end": "bottom top", "steps": 10 }
12
12
  // resolve each side's own scroll positions, so one map scores desktop, tablet and mobile (--viewport tablet).
13
13
  // usage: node scripts/curves.mjs <name> [--viewport desktop] [--only "<name substring>"]
@@ -29,7 +29,7 @@ const vp = VIEWPORTS.find((v) => v.name === (flags.viewport || 'desktop'));
29
29
 
30
30
  // ---------- Draft a map from the motion inventory ----------
31
31
  if (flags.draft) {
32
- const motion = JSON.parse(await fs.readFile(path.join(ref, `motion.${vp.name}.json`), 'utf8'));
32
+ const motion = JSON.parse(await fs.readFile(path.join(ref, `motion.${vp.name}.json`), 'utf8').catch(() => '{"triggers":[],"onScroll":[]}'));
33
33
  const propsFor = (vars) => {
34
34
  const out = new Set();
35
35
  for (const k of Object.keys(vars || {})) {
@@ -69,9 +69,27 @@ if (flags.draft) {
69
69
  const y = tw.firedAt[0];
70
70
  entries.push({ name: `reveal ${sel}`, ref: sel, clone: 'TODO', props: propsFor({ ...leaf.vars, ...(leaf.vars.startAt || {}) }), time: { before: Math.max(0, y - vp.height), at: y, ms: Math.ceil((tw.duration + 0.4) * 10) * 100, every: 100 } });
71
71
  }
72
+ // Scroll-linked curves the recorder sampled (any library, or none the inventory can read): one entry per element,
73
+ // over the scroll range it was seen moving at this size
74
+ const recorded = JSON.parse(await fs.readFile(path.join(ref, `reveals.${vp.name}.json`), 'utf8').catch(() => '{}')).scrubs || [];
75
+ let fromScrubs = 0;
76
+ for (const s of recorded) {
77
+ const samples = (s.samples || []).filter((x) => x && Number.isFinite(x.y));
78
+ if (samples.length < 2 || seen.has(`path:${s.path}`)) continue;
79
+ const moved = (k) => Math.max(...samples.map((x) => x[k] ?? 0)) - Math.min(...samples.map((x) => x[k] ?? 0));
80
+ const props = [];
81
+ if (moved('o') > 0.02) props.push('opacity');
82
+ if (moved('tx') > 1) props.push('tx');
83
+ if (moved('ty') > 1) props.push('ty');
84
+ if (moved('s') > 0.01) props.push('sx', 'sy');
85
+ if (!props.length) continue;
86
+ seen.add(`path:${s.path}`);
87
+ fromScrubs++;
88
+ entries.push({ name: `scrub ${s.cls || s.path}`, ref: `path:${s.path}`, clone: 'TODO', props, viewports: [vp.name], scroll: { from: samples[0].y, to: samples.at(-1).y, steps: Math.min(12, Math.max(4, samples.length - 1)) } });
89
+ }
72
90
  const out = path.join(root, 'motion-map.draft.json');
73
91
  await fs.writeFile(out, JSON.stringify(entries, null, 2));
74
- console.log(`${entries.length} entries -> ${path.relative(process.cwd(), out)}. Fill in the clone selectors, then save as motion-map.json.`);
92
+ console.log(`${entries.length} entries (${fromScrubs} from recorded scroll curves) -> ${path.relative(process.cwd(), out)}. Fill in the clone selectors, then save as motion-map.json.`);
75
93
  process.exit(0);
76
94
  }
77
95
 
@@ -86,9 +104,28 @@ let map = JSON.parse(mapText);
86
104
  if (typeof flags.only === 'string') map = map.filter((e) => e.name.includes(flags.only));
87
105
  map = map.filter((e) => e.clone && e.clone !== 'TODO' && (!e.viewports || e.viewports.includes(vp.name)));
88
106
 
107
+ // `path:section:3>div:2` is a recorded element path (from trueclone reveals: tag:position among the children that
108
+ // are not script/style/meta, 1-based, from <body>); anything else is a CSS selector with an optional @n.
109
+ // Installed on every page as window.__tcFind.
110
+ function installFind() {
111
+ window.__tcFind = (selector) => {
112
+ if (selector.startsWith('path:')) {
113
+ const SKIP = new Set(['SCRIPT', 'NOSCRIPT', 'TEMPLATE', 'STYLE', 'LINK', 'META']);
114
+ let n = document.body;
115
+ for (const part of selector.slice(5).split('>')) {
116
+ const [tag, i] = part.split(':');
117
+ n = n && [...n.children].filter((c) => !SKIP.has(c.tagName))[Number(i) - 1];
118
+ if (!n || n.tagName.toLowerCase() !== tag) return null;
119
+ }
120
+ return n;
121
+ }
122
+ const [sel, idx] = selector.split('@');
123
+ return document.querySelectorAll(sel)[Number(idx || 0)] || null;
124
+ };
125
+ }
126
+
89
127
  function sampleInPage([selector, props]) {
90
- const [sel, idx] = selector.split('@');
91
- const el = document.querySelectorAll(sel)[Number(idx || 0)];
128
+ const el = window.__tcFind(selector);
92
129
  if (!el) return null;
93
130
  const s = getComputedStyle(el);
94
131
  const m = new DOMMatrixReadOnly(s.transform === 'none' ? undefined : s.transform);
@@ -132,6 +169,7 @@ async function run(url, side, entries, onEntry) {
132
169
  };
133
170
  try {
134
171
  const { context, page } = await newPage(browser, vp, { reducedMotion: 'no-preference' });
172
+ await page.addInitScript(installFind);
135
173
  await page.goto(url, { waitUntil: 'load', timeout: 60000 });
136
174
  await page.mouse.move(5, vp.height - 5);
137
175
  await waitStill(page, { timeout: 15000, interval: 200 });
@@ -139,8 +177,7 @@ async function run(url, side, entries, onEntry) {
139
177
  // Scroll entries: resolve every range first (at the top, before anything is pinned), then visit all sample
140
178
  // positions of all curves in one downward pass, so the page glides short distances and never rewinds.
141
179
  const resolve = (of, spec) => page.evaluate(([sel, s]) => {
142
- const [selector, idx] = sel.split('@');
143
- const el = document.querySelectorAll(selector)[Number(idx || 0)];
180
+ const el = window.__tcFind(sel);
144
181
  if (!el) return 0;
145
182
  const r = el.getBoundingClientRect();
146
183
  const top = r.top + scrollY;
package/scripts/diff.mjs CHANGED
@@ -3,7 +3,7 @@
3
3
  // Without --url it serves sites/<name>/clone/ itself on a free local port.
4
4
  import fs from 'node:fs/promises';
5
5
  import path from 'node:path';
6
- import { VIEWPORTS, parseArgs, openBrowser, newPage, settle, mask, serve, pad, crop, match, PNG } from './lib.mjs';
6
+ import { SCORED, parseArgs, openBrowser, newPage, settle, mask, serve, pad, crop, match, PNG } from './lib.mjs';
7
7
 
8
8
  const { pos, flags } = parseArgs(process.argv.slice(2));
9
9
  const [name] = pos;
@@ -25,7 +25,9 @@ const target = typeof flags.url === 'string' ? flags.url : server.url;
25
25
  const browser = await openBrowser(flags.cdp);
26
26
  const report = { name, mode: structure ? 'structure' : 'visual', source: meta.url, clone: target, at: new Date().toISOString(), viewports: [] };
27
27
  try {
28
- for (const vp of VIEWPORTS) {
28
+ for (const vp of SCORED) {
29
+ // a capture made before held-out sizes existed has no reference for them: skipped, not failed
30
+ if (!(await fs.stat(path.join(ref, `${vp.name}${suffix}.png`)).then(() => true, () => false))) { console.log(`${vp.name}: no reference capture at ${vp.width}x${vp.height} (re-capture to score it)`); continue; }
29
31
  const { context, page } = await newPage(browser, vp);
30
32
  await page.goto(target, { waitUntil: 'load', timeout: 60000 });
31
33
  await settle(page);
package/scripts/drift.mjs CHANGED
@@ -6,7 +6,7 @@
6
6
  // usage: node scripts/drift.mjs <name> [--viewport desktop|all]
7
7
  import fs from 'node:fs/promises';
8
8
  import path from 'node:path';
9
- import { VIEWPORTS, parseArgs, openBrowser, newPage, settle, sectionsOf } from './lib.mjs';
9
+ import { SCORED as VIEWPORTS, parseArgs, openBrowser, newPage, settle, sectionsOf } from './lib.mjs';
10
10
 
11
11
  const { pos, flags } = parseArgs(process.argv.slice(2));
12
12
  const [name] = pos;
@@ -24,19 +24,26 @@ try {
24
24
  for (const vp of vps) {
25
25
  const captured = meta.viewports?.find((v) => v.viewport === vp.name);
26
26
  if (!captured) continue;
27
- const { context, page } = await newPage(browser, vp);
28
- try {
29
- await page.goto(meta.url, { waitUntil: 'load', timeout: 60000 });
30
- await settle(page, meta.hide || []);
31
- const livePageHeight = await page.evaluate(() => document.documentElement.scrollHeight);
32
- const liveSections = (await sectionsOf(page)).length;
33
- // 1% (at least 40px) absorbs font hinting and a rotating line of text; a new section or a block that moved is above that
34
- const stale = Math.abs(livePageHeight - captured.pageHeight) > Math.max(40, captured.pageHeight * 0.01) || liveSections !== captured.sections;
35
- viewports.push({ viewport: vp.name, livePageHeight, capturedPageHeight: captured.pageHeight, liveSections, capturedSections: captured.sections, stale });
36
- console.log(`${vp.name}: live ${livePageHeight}px / ${liveSections} sections, captured ${captured.pageHeight}px / ${captured.sections} sections${stale ? ' <- changed since the capture' : ' (same)'}`);
37
- } finally {
38
- await context.close();
39
- }
27
+ const sample = async () => {
28
+ const { context, page } = await newPage(browser, vp);
29
+ try {
30
+ await page.goto(meta.url, { waitUntil: 'load', timeout: 60000 });
31
+ await settle(page, meta.hide || []);
32
+ return { h: await page.evaluate(() => document.documentElement.scrollHeight), n: (await sectionsOf(page)).length };
33
+ } finally {
34
+ await context.close();
35
+ }
36
+ };
37
+ // 1% (at least 40px) absorbs font hinting and a rotating line of text; a new section or a block that moved is above that
38
+ const differs = (s) => Math.abs(s.h - captured.pageHeight) > Math.max(40, captured.pageHeight * 0.01) || s.n !== captured.sections;
39
+ // a page that renders differently from one load to the next (a random block, a late embed) is not a changed site:
40
+ // a difference counts only when a second load shows it too
41
+ let s = await sample();
42
+ let confirmed = false;
43
+ if (differs(s)) { const again = await sample(); confirmed = differs(again); if (!confirmed) s = again; }
44
+ const stale = confirmed;
45
+ viewports.push({ viewport: vp.name, livePageHeight: s.h, capturedPageHeight: captured.pageHeight, liveSections: s.n, capturedSections: captured.sections, stale });
46
+ console.log(`${vp.name}: live ${s.h}px / ${s.n} sections, captured ${captured.pageHeight}px / ${captured.sections} sections${stale ? ' <- changed since the capture (seen on two loads)' : ' (same)'}`);
40
47
  }
41
48
  } finally {
42
49
  if (!flags.cdp) await browser.close();