@trueclone/cli 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,384 @@
1
+ // Template mode: give the clone's placeholder text the reference's shape. For every text block on the clone, the
2
+ // reference block at the same spot on screen is found, and each reference word is replaced by a lorem word of
3
+ // (nearly) the same rendered width in the reference's own font, keeping capitals, punctuation, number lengths, years. Lines then
4
+ // break where the reference's lines break, so layout and detail scores measure styling instead of copy length.
5
+ // Only placeholder words are written; no reference copy ends up in the clone.
6
+ //
7
+ // Blocks are matched at scroll windows (half a viewport apart, so pinned chapters are seen one by one). A clone
8
+ // block is rewritten only when its text is found verbatim in clone/index.html (or as a string literal in
9
+ // clone/script.js, for lists the script renders from data). Inline parts (a coloured span inside a heading) are
10
+ // shaped part by part. Text that already has the reference's shape (same word count, width within 4%) is kept, and a
11
+ // pair is only trusted when both blocks line up (left within 12px; width within 10% for blocks that fill their parent). Run desktop first, then
12
+ // the other viewports for blocks that only exist there; text that is already placeholder words is never reshaped.
13
+ //
14
+ // usage: node scripts/textshape.mjs <name> [--viewport desktop] [--write] [--debug]
15
+ // node scripts/textshape.mjs <name> --list "<reference row selector>" "<clone row selector>" [--write]
16
+ // (repeated lists such as calendar rows or cards: row n takes row n's shape, block by block; this also
17
+ // works where the list still drifts, and for a placeholder repeated in a data array)
18
+ import fs from 'node:fs/promises';
19
+ import path from 'node:path';
20
+ import { VIEWPORTS, parseArgs, openBrowser, newPage, settle, serve, waitStill } from './lib.mjs';
21
+
22
+ const { pos, flags } = parseArgs(process.argv.slice(2));
23
+ const [name] = pos;
24
+ if (!name) {
25
+ console.error('usage: npm run textshape -- <name> [--viewport desktop] [--write]');
26
+ process.exit(1);
27
+ }
28
+ const root = path.resolve('sites', name);
29
+ const meta = JSON.parse(await fs.readFile(path.join(root, 'reference', 'meta.json'), 'utf8'));
30
+ if (meta.mode === 'exact') {
31
+ console.error('exact mode clones carry the real text already; textshape is for template clones');
32
+ process.exit(1);
33
+ }
34
+ const vp = VIEWPORTS.find((v) => v.name === (flags.viewport || 'desktop'));
35
+ const listAt = process.argv.indexOf('--list');
36
+ const listSelectors = listAt > 0 ? process.argv.slice(listAt + 1, listAt + 3) : null;
37
+ if (listAt > 0 && (listSelectors.length < 2 || listSelectors.some((x) => x.startsWith('--')))) { console.error('--list needs two selectors: the reference row and the clone row'); process.exit(1); }
38
+ const htmlFile = path.join(root, 'clone', 'index.html');
39
+ const jsFile = path.join(root, 'clone', 'script.js');
40
+
41
+ const LOREM = `lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore et dolore
42
+ magna aliqua enim ad minim veniam quis nostrud exercitation ullamco laboris nisi aliquip ex ea commodo consequat duis aute
43
+ irure in reprehenderit voluptate velit esse cillum fugiat nulla pariatur excepteur sint occaecat cupidatat non proident sunt
44
+ culpa qui officia deserunt mollit anim id est laborum curabitur pretium tincidunt lacus nunc vitae faucibus porta felis
45
+ mauris viverra diam vel augue blandit nibh sapien rhoncus massa feugiat tellus integer vulputate morbi risus quam varius
46
+ orci nam libero justo laoreet sem lectus mattis praesent semper ante gravida eget fermentum turpis sodales neque arcu
47
+ egestas phasellus volutpat donec pharetra aenean placerat lobortis primis cras tortor iaculis lacinia suspendisse potenti
48
+ vivamus ultricies posuere cubilia curae maecenas ligula proin accumsan quisque ornare urna pellentesque habitant tristique
49
+ senectus netus malesuada fames ac a at nec mi ne te se eu per`.split(/\s+/).filter(Boolean);
50
+ const LOREM_SET = new Set(LOREM);
51
+
52
+ // Same rule on both pages: a text node belongs to its nearest block-level ancestor, skipping split-text wrappers.
53
+ const BLOCK_RULE = `
54
+ const SPLIT = /(^|[-_\\s])(line|word|char)s?([-_\\s]|$)|split|reveal-/i;
55
+ const own = (el) => [...el.childNodes].some((n) => n.nodeType === 3 && n.textContent.trim());
56
+ const norm = (t) => t.replace(/\\s+/g, ' ').trim();
57
+ const blockOf = (el) => {
58
+ let b = el;
59
+ while (b.parentElement && b.parentElement !== document.body) {
60
+ const d = getComputedStyle(b).display;
61
+ // one-or-two-character inline-blocks are split text too (odometer digits, per-letter hover rolls)
62
+ const splitChar = d === 'inline-block' && norm(b.textContent).length <= 2;
63
+ if (!['inline', 'contents'].includes(d) && !splitChar && !SPLIT.test(String(b.getAttribute('class') || ''))) break;
64
+ b = b.parentElement;
65
+ }
66
+ return b;
67
+ };`;
68
+
69
+ // Clone: text blocks actually on screen, with a stable id and which duplicate (same text) they are.
70
+ const blocksInView = `(() => {${BLOCK_RULE}
71
+ const visible = (el) => {
72
+ // closed <details> content still reports a box but is never painted
73
+ if (el.checkVisibility && !el.checkVisibility({ contentVisibilityAuto: true })) return false;
74
+ for (let p = el; p && p !== document.body; p = p.parentElement) {
75
+ const s = getComputedStyle(p);
76
+ if (s.visibility === 'hidden' || +s.opacity === 0 || s.display === 'none') return false;
77
+ }
78
+ return true;
79
+ };
80
+ // hit tests skip pointer-events:none layers (text overlays often are), so switch it on while testing
81
+ const hit = document.createElement('style');
82
+ hit.textContent = '*{pointer-events:auto!important}';
83
+ document.head.append(hit);
84
+ const found = new Map();
85
+ for (const el of document.body.querySelectorAll('*')) {
86
+ if (!own(el) || ['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEXT', 'TSPAN'].includes(el.tagName.toUpperCase())) continue;
87
+ const b = blockOf(el);
88
+ if (found.has(b) || !visible(b)) continue;
89
+ const range = document.createRange();
90
+ range.selectNodeContents(b);
91
+ const first = [...range.getClientRects()].find((r) => r.width > 2 && r.height > 2);
92
+ if (!first || first.top < 0 || first.bottom > innerHeight || first.left < 0 || first.right > innerWidth) continue;
93
+ const pt = { x: first.left + Math.min(6, first.width / 2), y: first.top + first.height / 2 };
94
+ // clipped text (a closed menu behind a clip-path) drops out of hit testing; decorative layers above the text
95
+ // (guide lines, gradients) are fine, so the block only has to be somewhere in the stack at its point
96
+ if (!document.elementsFromPoint(pt.x, pt.y).some((e) => e === b || b.contains(e))) continue;
97
+ found.set(b, { ...pt, top: first.top });
98
+ }
99
+ window.__labIds = window.__labIds || 0;
100
+ hit.remove();
101
+ return [...found].map(([b, p]) => {
102
+ if (!b.dataset.labId) b.dataset.labId = String(++window.__labIds);
103
+ const text = norm(b.textContent);
104
+ const same = [...document.body.querySelectorAll('*')].filter((e) => norm(e.textContent) === text && ![...e.children].some((c) => norm(c.textContent) === text));
105
+ const dup = same.findIndex((e) => e === b || b.contains(e) || e.contains(b));
106
+ const box = b.getBoundingClientRect();
107
+ const fills = box.width >= 0.9 * b.parentElement.getBoundingClientRect().width;
108
+ const cs = getComputedStyle(b);
109
+ return { id: b.dataset.labId, x: p.x, y: p.y, top: p.top, boxTop: box.top, boxBottom: box.bottom, left: box.left, width: box.width, fills, family: cs.fontFamily.split(',')[0].replace(/["']/g, '').trim().toLowerCase(), size: parseFloat(cs.fontSize), text, dup, dups: same.length };
110
+ });
111
+ })()`;
112
+
113
+ // A block's text split into parts by inline styling (a coloured span is its own part), with each part's font.
114
+ const DESCRIBE = `${BLOCK_RULE}
115
+ const describe = (b) => {
116
+ const parts = [];
117
+ const walk = (node) => {
118
+ for (const n of node.childNodes) {
119
+ if (n.nodeType === 3) {
120
+ if (!n.textContent.trim() && !parts.length) continue;
121
+ let el = n.parentElement;
122
+ while (el !== b && SPLIT.test(String(el.getAttribute('class') || ''))) el = el.parentElement;
123
+ // keyed by the inline element's class, not its identity: split-text libraries clone a coloured span on
124
+ // every line it crosses, and those copies are one part
125
+ const key = el === b ? '' : String(el.getAttribute('class') || el.tagName);
126
+ const last = parts[parts.length - 1];
127
+ if (last && last.key === key && !last.closed) last.text += n.textContent;
128
+ else {
129
+ const s = getComputedStyle(n.parentElement);
130
+ parts.push({ key, text: n.textContent, font: s.fontStyle + ' ' + s.fontWeight + ' ' + s.fontSize + ' ' + s.fontFamily, upper: s.textTransform === 'uppercase', ls: parseFloat(s.letterSpacing) || 0 });
131
+ }
132
+ } else if (n.nodeType === 1) {
133
+ if (n.tagName === 'BR') { const last = parts[parts.length - 1]; if (last) last.closed = true; continue; }
134
+ if (getComputedStyle(n).display === 'none') continue;
135
+ walk(n);
136
+ }
137
+ }
138
+ };
139
+ walk(b);
140
+ return {
141
+ boxTop: b.getBoundingClientRect().top,
142
+ boxBottom: b.getBoundingClientRect().bottom,
143
+ top: (() => { const rg = document.createRange(); rg.selectNodeContents(b); const f = [...rg.getClientRects()].find((x) => x.width > 2 && x.height > 2); return f ? f.top : b.getBoundingClientRect().top; })(),
144
+ left: b.getBoundingClientRect().left,
145
+ width: b.getBoundingClientRect().width,
146
+ fills: b.getBoundingClientRect().width >= 0.9 * b.parentElement.getBoundingClientRect().width,
147
+ family: getComputedStyle(b).fontFamily.split(',')[0].replace(/["']/g, '').trim().toLowerCase(),
148
+ size: parseFloat(getComputedStyle(b).fontSize),
149
+ tag: b.tagName.toLowerCase(),
150
+ cls: String(b.getAttribute('class') || '').split(/\\s+/).slice(0, 2).join('.'),
151
+ text: norm(b.textContent),
152
+ ...(() => { const t = norm(b.textContent); const same = [...document.body.querySelectorAll('*')].filter((e) => norm(e.textContent) === t && ![...e.children].some((c) => norm(c.textContent) === t)); return { dup: same.findIndex((e) => e === b || b.contains(e) || e.contains(b)), dups: same.length }; })(),
153
+ parts: parts.map((p) => ({ text: norm(p.text), font: p.font, upper: p.upper, ls: p.ls })).filter((p) => p.text),
154
+ };
155
+ };`;
156
+
157
+ // Reference: the block under a point.
158
+ const refBlockAt = (x, y) => `(() => {${DESCRIBE}
159
+ const hit = document.createElement('style');
160
+ hit.textContent = '*{pointer-events:auto!important}';
161
+ document.head.append(hit);
162
+ const leaf = document.elementsFromPoint(${x}, ${y}).find(own);
163
+ hit.remove();
164
+ return leaf ? describe(blockOf(leaf)) : null;
165
+ })()`;
166
+
167
+ // List mode: every row of a repeated list (rows, cards), and the text blocks inside each row in reading order.
168
+ const listRows = (sel) => `(() => {${DESCRIBE}
169
+ return [...document.querySelectorAll(${JSON.stringify(sel)})].map((row) => {
170
+ const blocks = [];
171
+ for (const el of row.querySelectorAll('*')) {
172
+ if (!own(el) || ['SCRIPT', 'STYLE', 'TEXT', 'TSPAN'].includes(el.tagName.toUpperCase())) continue;
173
+ const b = blockOf(el);
174
+ // only text that renders: hidden CMS fields (display: none) are not part of the layout
175
+ if (!blocks.includes(b) && row.contains(b) && b.getClientRects().length) blocks.push(b);
176
+ }
177
+ return blocks.map(describe);
178
+ });
179
+ })()`;
180
+
181
+ // Does the clone text already have the reference's shape (same word count, width within 4%)?
182
+ function sameShapeInPage([refParts, cloneParts]) {
183
+ const ctx = document.createElement('canvas').getContext('2d');
184
+ return refParts.every((p, k) => {
185
+ ctx.font = p.font;
186
+ const words = (t) => t.split(/\s+/).filter(Boolean);
187
+ const w = (t) => ctx.measureText(p.upper ? t.toUpperCase() : t).width;
188
+ const c = cloneParts[k] || '';
189
+ return words(p.text).length === words(c).length && Math.abs(w(p.text) - w(c)) <= 0.04 * w(p.text);
190
+ });
191
+ }
192
+
193
+ // Width-matched placeholder words, measured in the reference font.
194
+ function shapeInPage([parts, lorem]) {
195
+ const ctx = document.createElement('canvas').getContext('2d');
196
+ let seed = 7;
197
+ return parts.map((p) => {
198
+ ctx.font = p.font;
199
+ const width = (w) => ctx.measureText(p.upper ? w.toUpperCase() : w).width + p.ls * w.length;
200
+ const pool = lorem.map((w) => [w, width(w)]);
201
+ // match the running width, not just each word: every pick corrects the drift left by the previous ones, so
202
+ // the line-break points stay where the reference's are even over long paragraphs
203
+ let drift = 0;
204
+ return p.text.split(/(\s+)/).map((tok) => {
205
+ if (!tok.trim()) return ' ';
206
+ const m = tok.match(/^([^\p{L}\p{N}]*)([\p{L}\p{N}][\p{L}\p{N}'’-]*?)?([^\p{L}\p{N}]*)$/u);
207
+ if (!m || !m[2]) return tok; // punctuation only
208
+ const [, pre, word, post] = m;
209
+ // numbers keep their length (and years stay), but not their digits: a car number or a podium count is the
210
+ // original's data, not layout
211
+ if (/^(19|20)\d\d$/.test(word)) return tok;
212
+ if (/^[\d.,:/+-]+$/.test(word)) return pre + word.replace(/\d/g, (d) => String((Number(d) + 3) % 10)) + post;
213
+ // candidates get the word's capitalisation before they are measured (capitals are wider)
214
+ const caps = word.length > 1 && word === word.toUpperCase() && /\p{L}/u.test(word);
215
+ const cap = !caps && /\p{Lu}/u.test(word[0]);
216
+ const cased = (w) => (caps ? w.toUpperCase() : cap ? w[0].toUpperCase() + w.slice(1) : w);
217
+ const target = width(word) + drift;
218
+ const ranked = pool.map(([w]) => [cased(w), Math.abs(width(cased(w)) - target)]).sort((a, b) => a[1] - b[1]);
219
+ const close = ranked.filter(([, d]) => d <= ranked[0][1] + 0.5).map(([w]) => w);
220
+ seed = (seed * 31 + word.length) % 9973;
221
+ const out = close[seed % close.length];
222
+ drift = target - width(out);
223
+ return pre + out + post;
224
+ }).join('').replace(/\s+/g, ' ').trim();
225
+ });
226
+ }
227
+
228
+ // Clone source: the block's own text parts in clone/index.html (parsed, no scripts).
229
+ function sourcePartsInPage([blockText, dup, dups, html]) {
230
+ const doc = new DOMParser().parseFromString(html, 'text/html');
231
+ const norm = (s) => s.replace(/\s+/g, ' ').trim();
232
+ const candidates = [...doc.body.querySelectorAll('*')].filter((e) => norm(e.textContent) === blockText && ![...e.children].some((c) => norm(c.textContent) === blockText));
233
+ if (!candidates.length) return { error: 'not found verbatim in the source' };
234
+ if (candidates.length !== dups || dup < 0) return { error: `${candidates.length} identical blocks in the source, ${dups} on the page` };
235
+ const el = candidates[dup];
236
+ const allText = [];
237
+ const tw = doc.createTreeWalker(doc.body, NodeFilter.SHOW_TEXT);
238
+ for (let n = tw.nextNode(); n; n = tw.nextNode()) allText.push(n);
239
+ const parts = [];
240
+ const walk = (node) => {
241
+ for (const n of node.childNodes) {
242
+ if (n.nodeType === 3) {
243
+ if (!n.textContent.trim()) continue;
244
+ const k = allText.indexOf(n);
245
+ const occurrence = allText.filter((t, j) => j < k && norm(t.textContent) === norm(n.textContent)).length;
246
+ const last = parts[parts.length - 1];
247
+ if (last && last.parent === n.parentElement && !last.closed) last.nodes.push({ text: n.textContent, occurrence });
248
+ else parts.push({ parent: n.parentElement, nodes: [{ text: n.textContent, occurrence }] });
249
+ } else if (n.nodeType === 1) {
250
+ if (n.tagName === 'BR') { const last = parts[parts.length - 1]; if (last) last.closed = true; continue; }
251
+ walk(n);
252
+ }
253
+ }
254
+ };
255
+ walk(el);
256
+ return { parts: parts.map((p) => p.nodes) };
257
+ }
258
+
259
+ const LITERAL = /(['"`])((?:\\.|(?!\1).)*)\1/g; // string literals in script.js
260
+ const server = await serve(path.join(root, 'clone'));
261
+ const browser = await openBrowser(flags.cdp);
262
+ const html = await fs.readFile(htmlFile, 'utf8');
263
+ const js = await fs.readFile(jsFile, 'utf8').catch(() => '');
264
+ const plan = [];
265
+ const seen = new Set();
266
+ try {
267
+ const open = async (url) => {
268
+ const { page } = await newPage(browser, vp, { reducedMotion: 'no-preference' });
269
+ await page.goto(url, { waitUntil: 'load', timeout: 60000 });
270
+ await page.mouse.move(5, vp.height - 5);
271
+ await waitStill(page, { timeout: 15000, interval: 200 });
272
+ await settle(page);
273
+ return page;
274
+ };
275
+ const [ref, clone] = await Promise.all([open(meta.url), open(server.url)]);
276
+ // One pair: reference block r, clone block c ({ text, dup, dups }); jsOccurrence set in list mode.
277
+ const consider = async (r, c, where, jsOccurrence) => {
278
+ if (!r || !r.text || r.text === c.text) return; // nothing, or already the same words (generic labels)
279
+ const words = c.text.toLowerCase().match(/\p{L}+/gu) || [];
280
+ if (words.length && words.every((w) => LOREM_SET.has(w))) { // shaped on an earlier pass
281
+ if (flags.debug) console.log(` already placeholder words: "${c.text.slice(0, 40)}"`);
282
+ return;
283
+ }
284
+ const label = `${r.tag}.${r.cls}`;
285
+ let src = await clone.evaluate(sourcePartsInPage, [c.text, c.dup, c.dups, html]);
286
+ if (src.error && r.parts.length === 1) {
287
+ const lits = [...js.matchAll(LITERAL)].filter((m) => m[2] === c.text);
288
+ // a literal repeated across data rows maps by row order in list mode; otherwise only a unique one is safe
289
+ if (jsOccurrence !== undefined && jsOccurrence < lits.length) src = { js: true, parts: [[{ text: c.text, occurrence: jsOccurrence }]] };
290
+ else if (lits.length === 1 && c.dups === 1) src = { js: true, parts: [[{ text: c.text, occurrence: 0 }]] };
291
+ else if (lits.length) src = { error: `string literal repeated ${lits.length}x in script.js (use --list)` };
292
+ }
293
+ if (src.error) { plan.push({ y: where, clone: c.text, ref: label, skip: src.error }); return; }
294
+ if (src.parts.length !== r.parts.length) { plan.push({ y: where, clone: c.text, ref: label, skip: `${src.parts.length} inline parts on the clone vs ${r.parts.length} on the reference` }); return; }
295
+ const cloneParts = src.parts.map((nodes) => nodes.map((n) => n.text).join(' ').replace(/\s+/g, ' ').trim());
296
+ if (await ref.evaluate(sameShapeInPage, [r.parts, cloneParts])) {
297
+ if (flags.debug) console.log(` same shape already: "${c.text.slice(0, 40)}"`);
298
+ return;
299
+ }
300
+ const shaped = await ref.evaluate(shapeInPage, [r.parts, LOREM]);
301
+ plan.push({ y: where, clone: c.text, ref: label, js: src.js, parts: src.parts.map((nodes, k) => ({ nodes, to: shaped[k] })) });
302
+ };
303
+
304
+ if (listSelectors) {
305
+ // List mode: row n of the clone list takes the shape of row n of the reference list, block by block.
306
+ const [refSel, cloneSel] = listSelectors;
307
+ const [refRows, cloneRows] = await Promise.all([ref.evaluate(listRows(refSel)), clone.evaluate(listRows(cloneSel))]);
308
+ if (refRows.length !== cloneRows.length) console.warn(`WARNING ${refRows.length} reference rows vs ${cloneRows.length} clone rows: pairing the first ${Math.min(refRows.length, cloneRows.length)}`);
309
+ const seenText = new Map(); // text -> how many earlier rows used it (literal occurrence order)
310
+ for (let i = 0; i < Math.min(refRows.length, cloneRows.length); i++) {
311
+ if (refRows[i].length !== cloneRows[i].length) { plan.push({ y: `row ${i + 1}`, clone: cloneRows[i].map((x) => x.text).join(' | '), ref: refSel, skip: `${cloneRows[i].length} text blocks vs ${refRows[i].length}` }); continue; }
312
+ for (let j = 0; j < refRows[i].length; j++) {
313
+ const c = cloneRows[i][j], r = refRows[i][j];
314
+ const occurrence = seenText.get(c.text) || 0;
315
+ seenText.set(c.text, occurrence + 1);
316
+ if (r.family !== c.family) { plan.push({ y: `row ${i + 1}`, clone: c.text, ref: `${r.tag}.${r.cls}`, skip: `font ${c.family} vs ${r.family}` }); continue; }
317
+ await consider(r, c, `row ${i + 1}`, occurrence);
318
+ }
319
+ }
320
+ } else {
321
+ const max = await clone.evaluate(() => document.documentElement.scrollHeight - innerHeight);
322
+ for (let y = 0; ; y += Math.round(vp.height / 2)) {
323
+ const at = Math.min(y, max);
324
+ await Promise.all([ref, clone].map(async (p) => { await p.evaluate((yy) => window.scrollTo(0, yy), at); await waitStill(p, { timeout: 3000 }); }));
325
+ for (const c of await clone.evaluate(blocksInView)) {
326
+ if (seen.has(c.id)) continue;
327
+ const r = await ref.evaluate(refBlockAt(c.x, c.y));
328
+ if (!r || !r.text) continue;
329
+ seen.add(c.id);
330
+ // Position pairing only holds where the layout already lines up: the blocks' left edges must agree (and
331
+ // widths, for blocks that fill their parent), and the text must be the same kind (family, size). Elsewhere
332
+ // (a section that still drifts at this width) the block under the point is a different block: use --list.
333
+ const why = Math.abs(r.left - c.left) > 12 || (r.fills && c.fills && Math.abs(r.width - c.width) > Math.max(12, 0.1 * r.width))
334
+ ? `boxes differ (left ${Math.round(r.left)} vs ${Math.round(c.left)}, width ${Math.round(r.width)} vs ${Math.round(c.width)})`
335
+ // vertically the blocks must line up too (drift): by their first lines, centres or bottoms, so top-, centre-
336
+ // and bottom-aligned layouts all pass while text of different length is still being shaped
337
+ : ![[r.top, c.top], [(r.boxTop + r.boxBottom) / 2, (c.boxTop + c.boxBottom) / 2], [r.boxBottom, c.boxBottom]].some(([p, q]) => Math.abs(p - q) <= Math.max(4, 0.4 * c.size))
338
+ ? `first lines at different heights (${Math.round(r.top)} vs ${Math.round(c.top)})`
339
+ : r.family !== c.family || Math.abs(r.size - c.size) > 0.15 * r.size ? `different text (${r.family} ${r.size} vs ${c.family} ${c.size})` : null;
340
+ if (why) {
341
+ if (flags.debug) console.log(` not paired at y${at}: "${c.text.slice(0, 40)}" with ${r.tag}.${r.cls} "${r.text.slice(0, 30)}": ${why}`);
342
+ continue;
343
+ }
344
+ await consider(r, c, at);
345
+ }
346
+ if (at >= max) break;
347
+ }
348
+ }
349
+ } finally {
350
+ if (!flags.cdp) await browser.close();
351
+ server.close();
352
+ }
353
+
354
+ // Apply: each source part's text nodes become the shaped text (the first node gets it all, the rest are emptied).
355
+ const esc = (s) => s.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
356
+ const reEsc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
357
+ const pattern = (text) => new RegExp(text.trim().split(/\s+/).map((w) => reEsc(esc(w)).replace(/'/g, "(?:'|&#39;|&apos;)").replace(/’/g, '(?:’|&rsquo;)')).join('\\s+'), 'g');
358
+ // Edits to the same text are applied from the last occurrence back to the first: replacing an earlier copy
359
+ // first would shift the occurrence numbers of the later ones.
360
+ const edits = [];
361
+ for (const item of plan.filter((p) => p.parts)) {
362
+ if (item.js) { const { text, occurrence } = item.parts[0].nodes[0]; edits.push({ js: true, text, occurrence, to: item.parts[0].to }); continue; }
363
+ for (const part of item.parts) part.nodes.forEach((node, k) => edits.push({ text: node.text, occurrence: node.occurrence, to: k === 0 ? part.to : '' }));
364
+ }
365
+ edits.sort((x, y) => y.occurrence - x.occurrence);
366
+ let out = html;
367
+ let outJs = js;
368
+ for (const e of edits) {
369
+ let n = -1;
370
+ if (e.js) outJs = outJs.replace(LITERAL, (m, q, body) => (body === e.text && ++n === e.occurrence ? q + e.to.split(q).join('\\' + q) + q : m));
371
+ else out = out.replace(pattern(e.text), (m) => (++n === e.occurrence ? esc(e.to) : m));
372
+ }
373
+ const changed = plan.filter((p) => p.parts).reduce((n, p) => n + p.parts.length, 0);
374
+ for (const p of plan) {
375
+ console.log(`y${String(p.y).padEnd(6)} ${p.ref.padEnd(36)} ${p.skip ? `SKIP (${p.skip})` : p.js ? '(script.js)' : ''}`);
376
+ console.log(` clone: ${p.clone.slice(0, 110)}`);
377
+ if (p.parts) p.parts.forEach((x) => console.log(` -> ${x.to.slice(0, 110)}`));
378
+ }
379
+ console.log(`\n${plan.filter((p) => p.parts).length} blocks reshaped, ${plan.filter((p) => p.skip).length} skipped`);
380
+ if (flags.write) {
381
+ await fs.writeFile(htmlFile, out);
382
+ if (outJs !== js) await fs.writeFile(jsFile, outJs);
383
+ console.log(`wrote ${path.relative(process.cwd(), htmlFile)}${outJs !== js ? ' and script.js' : ''} (${changed} text parts)`);
384
+ } else console.log('dry run: add --write to apply');
@@ -0,0 +1,25 @@
1
+ // Cut a tall full-page screenshot into viewport-sized tiles you can actually read.
2
+ // usage: node scripts/tiles.mjs <png> [tileHeight=1200] -> <png-dir>/tiles/<name>-01.png ...
3
+ import fs from 'node:fs/promises';
4
+ import path from 'node:path';
5
+ import { PNG } from 'pngjs';
6
+
7
+ const [file, h = '1200'] = process.argv.slice(2);
8
+ if (!file) {
9
+ console.error('usage: npm run tiles -- <png> [tileHeight]');
10
+ process.exit(1);
11
+ }
12
+ const tileH = Number(h);
13
+ const src = PNG.sync.read(await fs.readFile(file));
14
+ const dir = path.join(path.dirname(file), 'tiles');
15
+ await fs.mkdir(dir, { recursive: true });
16
+ const base = path.basename(file, '.png');
17
+ let n = 0;
18
+ for (let y = 0; y < src.height; y += tileH) {
19
+ const th = Math.min(tileH, src.height - y);
20
+ const out = new PNG({ width: src.width, height: th });
21
+ PNG.bitblt(src, out, 0, y, src.width, th, 0, 0);
22
+ const name = `${base}-${String(++n).padStart(2, '0')}.png`;
23
+ await fs.writeFile(path.join(dir, name), PNG.sync.write(out));
24
+ }
25
+ console.log(`${n} tiles -> ${dir}`);
package/skill/SKILL.md ADDED
@@ -0,0 +1,138 @@
1
+ ---
2
+ name: trueclone
3
+ description: Build a working twin of a website from its URL - HTML/CSS/JS clone, Next.js + Tailwind + shadcn app, DESIGN.md and shadcn globals.css - and prove the match with layout, scroll-state, motion, interaction, hover and style scores at mobile, tablet and desktop widths. Use when the user gives a URL and asks to clone, replicate, rebuild or migrate a site or page, or to extract its design system.
4
+ ---
5
+
6
+ # Build a true clone of a website
7
+
8
+ Everything for a site lives in `sites/<name>/` (gitignored, stays local):
9
+ - `clone/`: plain HTML/CSS/JS, the source of truth.
10
+ - `next/`: the same site as a Next.js + Tailwind v4 + shadcn app, generated from `clone/`.
11
+ - `design-tokens.json`: the **one canonical source** of the design system, rendered to `DESIGN.md` (Google design.md format, linted), `globals.css` (shadcn theme) and `clone/tokens.css`.
12
+ - `REPORT.md`: every score against its target, what to fix first, what was not copied. This is the deliverable.
13
+ - `notes.md`: your notes: stack, motion inventory, decisions, known gaps.
14
+
15
+ Run every step as `npx trueclone <step> <name>` from the project folder (`npx trueclone help` lists them).
16
+
17
+ ## 0. Pick the mode
18
+
19
+ | Mode | Use it when | Content in the clone |
20
+ |---|---|---|
21
+ | `template` (default) | Anyone else's site: learning, layout study, a starting template | Placeholder brand, text, images and logos; scored with `--structure` |
22
+ | `exact` | The user owns the site or is authorised to copy it (a client migration, their own Webflow/Framer/WordPress site) | The site's own text and media |
23
+
24
+ Exact mode refuses to run without `--permission "<who authorised it>"`, and the value is recorded in `meta.json` and in `REPORT.md`. Never use exact mode on a site the user doesn't own or have permission for. If permission is unclear, ask.
25
+
26
+ ## 1. Capture
27
+
28
+ ```powershell
29
+ npx trueclone capture <url> [name] [--hide "<cookie banner / popup selectors>"] [--mode exact --permission "<owner>"]
30
+ npx trueclone states <name> # live scroll states + motion inventory (animations running)
31
+ npx trueclone motion <name> # animation runtime: ScrollTriggers, timelines, reveals, cursor tweens, loops, Lenis/Swiper, CSS
32
+ npx trueclone spec <name> # measured boxes and styles per viewport
33
+ ```
34
+
35
+ `npx trueclone <url> [name]` is a shortcut for capture + design-system draft.
36
+
37
+ `reference/` then holds:
38
+ - full-page PNGs and their `*.mask.png` structure versions, `sections.json` and `rendered.html`;
39
+ - `tokens.json` (colours, type, radii, CSS variables), `assets/` with `manifest.json`, and `desktop.har` (every request, including the data/API calls);
40
+ - **`states/<vp>/NN.png`**: what a visitor sees at every half-viewport scroll step, after motion settles;
41
+ - **`states/<vp>.summary.json`** (the motion inventory): `reveals`, `animated`, `parallax`, `pinned` and `fixed` elements;
42
+ - **`motion.<vp>.md`**: the exact animation recipe when the site uses GSAP/ScrollTrigger, Lenis, Swiper or CSS animations: trigger, start/end, scrub, pin, properties and ranges, duration, easing, stagger. Rebuild these with the same public libraries and these numbers, in your own code;
43
+ - in exact mode, `mirror/index.html`: a script-free local snapshot with assets rewritten to local copies.
44
+
45
+ Read the motion inventory before building. Each entry is a behaviour the clone must have.
46
+
47
+ If the page is blank or blocked, capture through your own Chrome over CDP (`chrome --remote-debugging-port=9222 --user-data-dir=.chrome-profile`, then `--cdp`). Never try to get past bot checks, logins or paywalls.
48
+
49
+ ## 2. Understand it with REA
50
+
51
+ trueclone bundles REA (Reverse Engineer Anything). `npx @trueclone/cli init` registers it as an MCP server; without MCP, run `npx trueclone rea <command>`. With the page open in your CDP Chrome:
52
+ - `inspect_web_page`: DOM, scripts, resources.
53
+ - `analyze_web_bundle` with source maps: framework, vendors (GSAP, ScrollTrigger, Lenis, Swiper, Webflow IX2, three.js), routes and API endpoints.
54
+ - `inspect_web_event_listeners`: hover, drag, wheel and click logic.
55
+
56
+ Use REA when the capture can't tell you how something works: a behaviour's exact rule, bundled shader or physics code, the API a component calls. Write `notes.md`: stack, breakpoints, and the motion inventory turned into a list of behaviours (trigger, property, range).
57
+
58
+ ## 3. Design system
59
+
60
+ ```powershell
61
+ npx trueclone design <name> # drafts design-tokens.json if missing, renders DESIGN.md, globals.css, clone/tokens.css + lints
62
+ npx trueclone design <name> --redraft # fresh draft in design-tokens.draft.json (never overwrites your edits)
63
+ ```
64
+
65
+ Review the draft: role names, the `notes` prose (overview, do's and don'ts), anything marked `origin: generated`. Re-run after every edit. `DESIGN.md` must lint with **0 errors**.
66
+
67
+ ## 4. Build (`clone/`)
68
+
69
+ **Exact mode starts from the mirror:** `npx trueclone seed <name>` copies the script-free mirror into `clone/`, with the site's markup, text, stylesheets and media at local paths. Then write `clone/script.js`: every interaction in your own code, using the parameters you read from the reference (timings, physics constants, offsets, breakpoints). Never copy the site's scripts. JS-set initial states, such as a `<details>` closed on phones, are part of the layout, so check every viewport. Data the page fetches live (a star count, a feed) stays as captured; no requests to third parties.
70
+
71
+ **Template mode builds from scratch:**
72
+
73
+ - `styles.css` starts with `@import url("tokens.css");` and uses its variables. Never hard-code a token value.
74
+ - Use the site's breakpoints and root font-size behaviour (in `design-tokens.json`, under `layout`).
75
+ - Build top to bottom, one section at a time, matching `sections.json` heights. Take structure from `rendered.html`. Write your own placeholder words, then run `npx trueclone textshape <name> --write` (desktop, then `--viewport tablet` and `mobile`): it swaps each placeholder word for a lorem word of the reference word's rendered width, so lines break where the reference's break. Use `--list "<ref row>" "<clone row>"` for repeated lists (calendar rows, cards), and give every data row its own placeholder text.
76
+
77
+ **Both modes:**
78
+
79
+ - **Copy the reference's CSS mechanism, not its look.** Before writing CSS for an element, run `npx trueclone rules <name> "<selector>"` and port the matching rules (with their `@media`). Look-alikes drift in flex/grid: a "capsize" `::before/::after` text trim is not `text-box-trim`, `height: 14.5rem` is not `flex: 1`, and `width: 100%; max-width: 40rem` with default flex shrink is not `flex: 0 10 auto` or `width: min-content`.
80
+ - **Positional selectors:** `:nth-of-type` counts siblings of the same tag only. A logo `<a>` before the column `<div>`s shifts every column by one, so use `:nth-child` or a class.
81
+ - **Stacked media:** keep every layer (colour, grayscale or depth-map images, overlays). Never flatten them into one image.
82
+ - **Canvas/WebGL:** a site can draw its hero or a game on a `<canvas>`. Rebuild the drawing in your own code with the reference's parameters (exact mode) or a placeholder scene. Never drop it silently.
83
+ - **Motion:** implement every inventory entry: reveals, parallax speeds, pinned sections, fixed elements.
84
+ - **Interactions:** sliders, tabs, hover states, cursor followers, menus, counters, demos. Scope query selectors to their own component: a hidden duplicate (a mobile-only list) must never feed a desktop counter.
85
+ - **Sliders in flex/grid:** give the slider and every flex/grid ancestor `min-width: 0`, or Swiper and similar grow to fit their slides and run away. `states --clone` warns when the page height drifts more than 20%.
86
+ - `<meta name="robots" content="noindex">`, forms submit nowhere, no tracking scripts.
87
+
88
+ ## 5. Verify
89
+
90
+ ```powershell
91
+ npx trueclone score <name> --quick # layout, desktop states, interactions: the loop while building (about a minute)
92
+ npx trueclone score <name> # everything, in parallel groups, then REPORT.md
93
+ ```
94
+
95
+ The steps, if you need one alone:
96
+
97
+ ```powershell
98
+ npx trueclone diff <name> [--structure] # layout: full page, frozen (structure = text and media masked)
99
+ npx trueclone states <name> --clone [--structure] # scroll states: the reference's scroll positions replayed on the clone
100
+ npx trueclone curves <name> --draft # motion-map.draft.json from the inventory: fill in clone selectors, save as motion-map.json
101
+ npx trueclone curves <name> [--viewport tablet] # motion: property curves (scroll- and time-driven), reference vs clone
102
+ npx trueclone interact <name> --draft # interactions.draft.json: fill in TODOs, save as interactions.json
103
+ npx trueclone interact <name> # interactions: click/hover/drag/key scenarios + expectChange (geometry, text, state attributes, canvas pixels)
104
+ npx trueclone detail <name> --viewport all # styles at matching points: font, size, weight, line height, colour, boxes
105
+ npx trueclone hovers <name> # hover effects present on the reference but missing on the clone
106
+ ```
107
+
108
+ Reference measurements are cached until the next capture, so only the clone is re-measured. Sites without scroll/time motion or without interactions simply skip those steps.
109
+
110
+ Fix in this order: page height drift, the worst layout section, the worst state step, the lowest motion curve, any interaction marked FAIL, then detail and hovers. `REPORT.md` lists them under "Fix first". For a drift, compare the element's ancestor boxes on both sides until one differs, then `rules` that ancestor: the cause is usually one declaration. Fix the cause; never tune numbers to the score.
111
+
112
+ **Done when:** layout ≥ 98% at every viewport, states within 5 points of layout, motion ≥ 95%, detail ≥ 95%, hovers ≥ 90%, and every interaction passes. Otherwise stop after 8 passes and report what's left and why.
113
+
114
+ How detail measures, so you can trust it:
115
+ - motion on (many originals ignore `prefers-reduced-motion`, and a clone that honours it lays out differently);
116
+ - both sides measured the same way: a hit test at the same point and scroll position, so sticky and pinned content is compared in place and an identical page scores 100%;
117
+ - through `pointer-events: none` overlays, and never on content the browser doesn't paint (closed `<details>`);
118
+ - colours written by scroll-scrubbed JS show up as in-between values: that is a motion difference, so check the motion inventory.
119
+
120
+ The live site can change after capture (a redeploy changes its CSS bundle name). When live-site tools (detail, rules) and screenshot scores (layout, states) disagree about a section, re-capture (`capture`, then `states <name>`); that also resets the cached reference measurements.
121
+
122
+ ## 6. Next.js (after the HTML clone passes)
123
+
124
+ ```powershell
125
+ npx trueclone next <name> --verify # generates sites/<name>/next, installs, builds, serves, and runs every score on it
126
+ ```
127
+
128
+ The HTML clone stays the source of truth: change `clone/`, then regenerate. The app has one component per section, the motion script as a client component (GSAP, Lenis, Swiper and three from npm at the clone's versions), `app/globals.css` = shadcn theme + tokens + the clone's stylesheet + preflight parity, and `components.json` so `npx shadcn add <component>` works. It is done when its scores match the HTML clone's within a point.
129
+
130
+ ## 7. Report
131
+
132
+ `npx trueclone score <name>` writes `REPORT.md` (or run `npx trueclone report <name>`): mode and permission, every score against its target, "Fix first", the motion found on the reference, and what was not copied (server behaviour, tracking, and each data/API call the reference makes). Hand it over with the clone.
133
+
134
+ ## Honest limits
135
+
136
+ - Server-side behaviour is never visible from outside: databases, accounts, payments, search. The clone reproduces the client; data calls are listed in the report.
137
+ - A live site is never pixel-identical twice: rotating content, A/B tests and font hinting vary.
138
+ - Template clones of other people's sites stay local. Exact clones need recorded permission and also stay local unless the owner deploys them.