devlens-mcp 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/settings.json +12 -0
- package/.claude/settings.local.json +17 -0
- package/INSTALLATION_GUIDE.md +354 -0
- package/QUICK_START.md +153 -0
- package/README.md +354 -0
- package/bin/cli.ts +22 -0
- package/bin/register.ts +96 -0
- package/dist/bin/cli.d.ts +3 -0
- package/dist/bin/cli.d.ts.map +1 -0
- package/dist/bin/cli.js +20 -0
- package/dist/bin/cli.js.map +1 -0
- package/dist/bin/register.d.ts +10 -0
- package/dist/bin/register.d.ts.map +1 -0
- package/dist/bin/register.js +92 -0
- package/dist/bin/register.js.map +1 -0
- package/dist/src/config/devlens-config.d.ts +92 -0
- package/dist/src/config/devlens-config.d.ts.map +1 -0
- package/dist/src/config/devlens-config.js +70 -0
- package/dist/src/config/devlens-config.js.map +1 -0
- package/dist/src/index.d.ts +35 -0
- package/dist/src/index.d.ts.map +1 -0
- package/dist/src/index.js +8 -0
- package/dist/src/index.js.map +1 -0
- package/dist/src/metro/cdp-client.d.ts +48 -0
- package/dist/src/metro/cdp-client.d.ts.map +1 -0
- package/dist/src/metro/cdp-client.js +127 -0
- package/dist/src/metro/cdp-client.js.map +1 -0
- package/dist/src/metro/log-collector.d.ts +30 -0
- package/dist/src/metro/log-collector.d.ts.map +1 -0
- package/dist/src/metro/log-collector.js +114 -0
- package/dist/src/metro/log-collector.js.map +1 -0
- package/dist/src/metro/metro-bridge.d.ts +56 -0
- package/dist/src/metro/metro-bridge.d.ts.map +1 -0
- package/dist/src/metro/metro-bridge.js +255 -0
- package/dist/src/metro/metro-bridge.js.map +1 -0
- package/dist/src/metro/network-inspector.d.ts +34 -0
- package/dist/src/metro/network-inspector.d.ts.map +1 -0
- package/dist/src/metro/network-inspector.js +100 -0
- package/dist/src/metro/network-inspector.js.map +1 -0
- package/dist/src/platform/android/adb.d.ts +50 -0
- package/dist/src/platform/android/adb.d.ts.map +1 -0
- package/dist/src/platform/android/adb.js +137 -0
- package/dist/src/platform/android/adb.js.map +1 -0
- package/dist/src/platform/android/android-device.d.ts +21 -0
- package/dist/src/platform/android/android-device.d.ts.map +1 -0
- package/dist/src/platform/android/android-device.js +94 -0
- package/dist/src/platform/android/android-device.js.map +1 -0
- package/dist/src/platform/android/ui-automator.d.ts +17 -0
- package/dist/src/platform/android/ui-automator.d.ts.map +1 -0
- package/dist/src/platform/android/ui-automator.js +126 -0
- package/dist/src/platform/android/ui-automator.js.map +1 -0
- package/dist/src/platform/device-manager.d.ts +28 -0
- package/dist/src/platform/device-manager.d.ts.map +1 -0
- package/dist/src/platform/device-manager.js +185 -0
- package/dist/src/platform/device-manager.js.map +1 -0
- package/dist/src/platform/device.d.ts +86 -0
- package/dist/src/platform/device.d.ts.map +1 -0
- package/dist/src/platform/device.js +7 -0
- package/dist/src/platform/device.js.map +1 -0
- package/dist/src/platform/ios/accessibility.d.ts +17 -0
- package/dist/src/platform/ios/accessibility.d.ts.map +1 -0
- package/dist/src/platform/ios/accessibility.js +159 -0
- package/dist/src/platform/ios/accessibility.js.map +1 -0
- package/dist/src/platform/ios/ios-device.d.ts +22 -0
- package/dist/src/platform/ios/ios-device.d.ts.map +1 -0
- package/dist/src/platform/ios/ios-device.js +97 -0
- package/dist/src/platform/ios/ios-device.js.map +1 -0
- package/dist/src/platform/ios/simctl.d.ts +54 -0
- package/dist/src/platform/ios/simctl.d.ts.map +1 -0
- package/dist/src/platform/ios/simctl.js +192 -0
- package/dist/src/platform/ios/simctl.js.map +1 -0
- package/dist/src/server.d.ts +3 -0
- package/dist/src/server.d.ts.map +1 -0
- package/dist/src/server.js +176 -0
- package/dist/src/server.js.map +1 -0
- package/dist/src/snapshot/formatter.d.ts +18 -0
- package/dist/src/snapshot/formatter.d.ts.map +1 -0
- package/dist/src/snapshot/formatter.js +86 -0
- package/dist/src/snapshot/formatter.js.map +1 -0
- package/dist/src/snapshot/ref-registry.d.ts +67 -0
- package/dist/src/snapshot/ref-registry.d.ts.map +1 -0
- package/dist/src/snapshot/ref-registry.js +169 -0
- package/dist/src/snapshot/ref-registry.js.map +1 -0
- package/dist/src/snapshot/snapshot-differ.d.ts +57 -0
- package/dist/src/snapshot/snapshot-differ.d.ts.map +1 -0
- package/dist/src/snapshot/snapshot-differ.js +153 -0
- package/dist/src/snapshot/snapshot-differ.js.map +1 -0
- package/dist/src/tools/app-tools.d.ts +71 -0
- package/dist/src/tools/app-tools.d.ts.map +1 -0
- package/dist/src/tools/app-tools.js +97 -0
- package/dist/src/tools/app-tools.js.map +1 -0
- package/dist/src/tools/device-tools.d.ts +53 -0
- package/dist/src/tools/device-tools.d.ts.map +1 -0
- package/dist/src/tools/device-tools.js +86 -0
- package/dist/src/tools/device-tools.js.map +1 -0
- package/dist/src/tools/ds-tools.d.ts +65 -0
- package/dist/src/tools/ds-tools.d.ts.map +1 -0
- package/dist/src/tools/ds-tools.js +314 -0
- package/dist/src/tools/ds-tools.js.map +1 -0
- package/dist/src/tools/interaction-tools.d.ts +248 -0
- package/dist/src/tools/interaction-tools.d.ts.map +1 -0
- package/dist/src/tools/interaction-tools.js +391 -0
- package/dist/src/tools/interaction-tools.js.map +1 -0
- package/dist/src/tools/metro-tools.d.ts +115 -0
- package/dist/src/tools/metro-tools.d.ts.map +1 -0
- package/dist/src/tools/metro-tools.js +270 -0
- package/dist/src/tools/metro-tools.js.map +1 -0
- package/dist/src/tools/navigation-tools.d.ts +36 -0
- package/dist/src/tools/navigation-tools.d.ts.map +1 -0
- package/dist/src/tools/navigation-tools.js +60 -0
- package/dist/src/tools/navigation-tools.js.map +1 -0
- package/dist/src/tools/screenshot-tools.d.ts +298 -0
- package/dist/src/tools/screenshot-tools.d.ts.map +1 -0
- package/dist/src/tools/screenshot-tools.js +565 -0
- package/dist/src/tools/screenshot-tools.js.map +1 -0
- package/dist/src/tools/snapshot-tools.d.ts +161 -0
- package/dist/src/tools/snapshot-tools.d.ts.map +1 -0
- package/dist/src/tools/snapshot-tools.js +479 -0
- package/dist/src/tools/snapshot-tools.js.map +1 -0
- package/dist/src/utils/image-preprocess.d.ts +49 -0
- package/dist/src/utils/image-preprocess.d.ts.map +1 -0
- package/dist/src/utils/image-preprocess.js +322 -0
- package/dist/src/utils/image-preprocess.js.map +1 -0
- package/dist/src/utils/retry.d.ts +21 -0
- package/dist/src/utils/retry.d.ts.map +1 -0
- package/dist/src/utils/retry.js +33 -0
- package/dist/src/utils/retry.js.map +1 -0
- package/dist/src/visual/comparator.d.ts +51 -0
- package/dist/src/visual/comparator.d.ts.map +1 -0
- package/dist/src/visual/comparator.js +119 -0
- package/dist/src/visual/comparator.js.map +1 -0
- package/dist/src/visual/layout-analyzer.d.ts +64 -0
- package/dist/src/visual/layout-analyzer.d.ts.map +1 -0
- package/dist/src/visual/layout-analyzer.js +198 -0
- package/dist/src/visual/layout-analyzer.js.map +1 -0
- package/dist/src/visual/screenshot.d.ts +17 -0
- package/dist/src/visual/screenshot.d.ts.map +1 -0
- package/dist/src/visual/screenshot.js +39 -0
- package/dist/src/visual/screenshot.js.map +1 -0
- package/docs/figma-workflow.md +289 -0
- package/docs/setup-guide.md +360 -0
- package/docs/tool-reference.md +622 -0
- package/package.json +57 -0
- package/src/config/devlens-config.ts +76 -0
- package/src/index.ts +5 -0
- package/src/metro/cdp-client.ts +160 -0
- package/src/metro/log-collector.ts +137 -0
- package/src/metro/metro-bridge.ts +307 -0
- package/src/metro/network-inspector.ts +134 -0
- package/src/platform/android/adb.ts +200 -0
- package/src/platform/android/android-device.ts +116 -0
- package/src/platform/android/ui-automator.ts +141 -0
- package/src/platform/device-manager.ts +229 -0
- package/src/platform/device.ts +110 -0
- package/src/platform/ios/accessibility.ts +189 -0
- package/src/platform/ios/ios-device.ts +116 -0
- package/src/platform/ios/simctl.ts +244 -0
- package/src/server.ts +228 -0
- package/src/snapshot/formatter.ts +102 -0
- package/src/snapshot/ref-registry.ts +230 -0
- package/src/snapshot/snapshot-differ.ts +220 -0
- package/src/tools/app-tools.ts +111 -0
- package/src/tools/device-tools.ts +96 -0
- package/src/tools/ds-tools.ts +395 -0
- package/src/tools/interaction-tools.ts +467 -0
- package/src/tools/metro-tools.ts +320 -0
- package/src/tools/navigation-tools.ts +71 -0
- package/src/tools/screenshot-tools.ts +698 -0
- package/src/tools/snapshot-tools.ts +585 -0
- package/src/utils/image-preprocess.ts +430 -0
- package/src/utils/retry.ts +51 -0
- package/src/visual/comparator.ts +191 -0
- package/src/visual/layout-analyzer.ts +283 -0
- package/src/visual/screenshot.ts +49 -0
- package/tsconfig.json +20 -0
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
import type { RefEntry } from "../snapshot/ref-registry.js";
|
|
2
|
+
/**
|
|
3
|
+
* Layout-aware comparison engine.
|
|
4
|
+
*
|
|
5
|
+
* Uses the accessibility tree (via RefRegistry) to identify semantic UI regions,
|
|
6
|
+
* scales those regions from device coordinate space into Figma image space,
|
|
7
|
+
* and compares each region independently via pixelmatch.
|
|
8
|
+
*
|
|
9
|
+
* This gives per-element similarity scores ("Header: 92%, ChatInput: 44%")
|
|
10
|
+
* instead of a single misleading global pixel-diff percentage.
|
|
11
|
+
*/
|
|
12
|
+
export interface RegionComparisonResult {
|
|
13
|
+
ref: string;
|
|
14
|
+
elementType: string;
|
|
15
|
+
text?: string;
|
|
16
|
+
label?: string;
|
|
17
|
+
/** Element bounds in device pixel space */
|
|
18
|
+
deviceBounds: {
|
|
19
|
+
left: number;
|
|
20
|
+
top: number;
|
|
21
|
+
width: number;
|
|
22
|
+
height: number;
|
|
23
|
+
};
|
|
24
|
+
/** Corresponding bounds in Figma image space (after scaling) */
|
|
25
|
+
figmaBounds: {
|
|
26
|
+
left: number;
|
|
27
|
+
top: number;
|
|
28
|
+
width: number;
|
|
29
|
+
height: number;
|
|
30
|
+
};
|
|
31
|
+
similarity: number;
|
|
32
|
+
diffPixels: number;
|
|
33
|
+
totalPixels: number;
|
|
34
|
+
verdict: "excellent" | "good" | "partial" | "poor";
|
|
35
|
+
}
|
|
36
|
+
export interface LayoutReport {
|
|
37
|
+
regions: RegionComparisonResult[];
|
|
38
|
+
/** Weighted average similarity across all analyzed regions */
|
|
39
|
+
overallSimilarity: number;
|
|
40
|
+
/** Number of refs skipped (too small, out of bounds, crop failed) */
|
|
41
|
+
skippedRefs: number;
|
|
42
|
+
deviceDimensions: {
|
|
43
|
+
width: number;
|
|
44
|
+
height: number;
|
|
45
|
+
};
|
|
46
|
+
figmaDimensions: {
|
|
47
|
+
width: number;
|
|
48
|
+
height: number;
|
|
49
|
+
};
|
|
50
|
+
}
|
|
51
|
+
/**
|
|
52
|
+
* Core function: for each ref with sufficient area, scale its device bounds
|
|
53
|
+
* into Figma image space, crop both images, and compare independently.
|
|
54
|
+
*
|
|
55
|
+
* @param deviceBuffer Full-screen device screenshot PNG
|
|
56
|
+
* @param figmaBuffer Preprocessed Figma image PNG (artifacts already removed)
|
|
57
|
+
* @param refs All RefEntry objects from RefRegistry.getAllRefs()
|
|
58
|
+
* @param deviceW Device screen width in pixels
|
|
59
|
+
* @param deviceH Device screen height in pixels
|
|
60
|
+
*/
|
|
61
|
+
export declare function buildLayoutReport(deviceBuffer: Buffer, figmaBuffer: Buffer, refs: RefEntry[], deviceW: number, deviceH: number): Promise<LayoutReport>;
|
|
62
|
+
/** Format layout report as human-readable text for MCP tool output */
|
|
63
|
+
export declare function formatLayoutReport(report: LayoutReport): string;
|
|
64
|
+
//# sourceMappingURL=layout-analyzer.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"layout-analyzer.d.ts","sourceRoot":"","sources":["../../../src/visual/layout-analyzer.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,6BAA6B,CAAC;AAE5D;;;;;;;;;GASG;AAEH,MAAM,WAAW,sBAAsB;IACrC,GAAG,EAAE,MAAM,CAAC;IACZ,WAAW,EAAE,MAAM,CAAC;IACpB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,2CAA2C;IAC3C,YAAY,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,GAAG,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;IAC3E,gEAAgE;IAChE,WAAW,EAAE;QAAE,IAAI,EAAE,MAAM,CAAC;QAAC,GAAG,EAAE,MAAM,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;IAC1E,UAAU,EAAE,MAAM,CAAC;IACnB,UAAU,EAAE,MAAM,CAAC;IACnB,WAAW,EAAE,MAAM,CAAC;IACpB,OAAO,EAAE,WAAW,GAAG,MAAM,GAAG,SAAS,GAAG,MAAM,CAAC;CACpD;AAED,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,sBAAsB,EAAE,CAAC;IAClC,8DAA8D;IAC9D,iBAAiB,EAAE,MAAM,CAAC;IAC1B,qEAAqE;IACrE,WAAW,EAAE,MAAM,CAAC;IACpB,gBAAgB,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;IACpD,eAAe,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAC;CACpD;AA6FD;;;;;;;;;GASG;AACH,wBAAsB,iBAAiB,CACrC,YAAY,EAAE,MAAM,EACpB,WAAW,EAAE,MAAM,EACnB,IAAI,EAAE,QAAQ,EAAE,EAChB,OAAO,EAAE,MAAM,EACf,OAAO,EAAE,MAAM,GACd,OAAO,CAAC,YAAY,CAAC,CAyFvB;AAED,sEAAsE;AACtE,wBAAgB,kBAAkB,CAAC,MAAM,EAAE,YAAY,GAAG,MAAM,CA4C/D"}
|
|
@@ -0,0 +1,198 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __importDefault = (this && this.__importDefault) || function (mod) {
|
|
3
|
+
return (mod && mod.__esModule) ? mod : { "default": mod };
|
|
4
|
+
};
|
|
5
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
6
|
+
exports.buildLayoutReport = buildLayoutReport;
|
|
7
|
+
exports.formatLayoutReport = formatLayoutReport;
|
|
8
|
+
const pixelmatch_1 = __importDefault(require("pixelmatch"));
|
|
9
|
+
/** Minimum element area (px²) for inclusion in layout analysis */
|
|
10
|
+
const MIN_REGION_AREA = 3000;
|
|
11
|
+
/** Color difference threshold for pixelmatch (matches global comparator default) */
|
|
12
|
+
const REGION_THRESHOLD = 0.1;
|
|
13
|
+
function verdictFromSimilarity(s) {
|
|
14
|
+
if (s >= 0.92)
|
|
15
|
+
return "excellent";
|
|
16
|
+
if (s >= 0.80)
|
|
17
|
+
return "good";
|
|
18
|
+
if (s >= 0.60)
|
|
19
|
+
return "partial";
|
|
20
|
+
return "poor";
|
|
21
|
+
}
|
|
22
|
+
/** Scale device-space element bounds into Figma image pixel space */
|
|
23
|
+
function scaleToFigmaSpace(bounds, deviceW, deviceH, figmaW, figmaH) {
|
|
24
|
+
const scaleX = figmaW / deviceW;
|
|
25
|
+
const scaleY = figmaH / deviceH;
|
|
26
|
+
return {
|
|
27
|
+
left: Math.round(bounds.x * scaleX),
|
|
28
|
+
top: Math.round(bounds.y * scaleY),
|
|
29
|
+
width: Math.max(1, Math.round(bounds.width * scaleX)),
|
|
30
|
+
height: Math.max(1, Math.round(bounds.height * scaleY)),
|
|
31
|
+
};
|
|
32
|
+
}
|
|
33
|
+
/** Clamp bounds to image dimensions. Returns null if there's no overlap. */
|
|
34
|
+
function clampBounds(b, maxW, maxH) {
|
|
35
|
+
const left = Math.max(0, b.left);
|
|
36
|
+
const top = Math.max(0, b.top);
|
|
37
|
+
const right = Math.min(maxW, b.left + b.width);
|
|
38
|
+
const bottom = Math.min(maxH, b.top + b.height);
|
|
39
|
+
if (right <= left || bottom <= top)
|
|
40
|
+
return null;
|
|
41
|
+
return { left, top, width: right - left, height: bottom - top };
|
|
42
|
+
}
|
|
43
|
+
/**
|
|
44
|
+
* Crop a PNG buffer to a region and return raw RGBA bytes + dimensions.
|
|
45
|
+
* Uses sharp's raw() output so pixelmatch can consume it directly
|
|
46
|
+
* without a PNG encode/decode round-trip.
|
|
47
|
+
*/
|
|
48
|
+
async function cropToRaw(imageBuffer, bounds) {
|
|
49
|
+
try {
|
|
50
|
+
const sharp = (await import("sharp")).default;
|
|
51
|
+
const result = await sharp(imageBuffer)
|
|
52
|
+
.extract({ left: bounds.left, top: bounds.top, width: bounds.width, height: bounds.height })
|
|
53
|
+
.ensureAlpha()
|
|
54
|
+
.raw()
|
|
55
|
+
.toBuffer({ resolveWithObject: true });
|
|
56
|
+
return { data: result.data, width: result.info.width, height: result.info.height };
|
|
57
|
+
}
|
|
58
|
+
catch {
|
|
59
|
+
return null;
|
|
60
|
+
}
|
|
61
|
+
}
|
|
62
|
+
/**
|
|
63
|
+
* Resize raw RGBA bytes to target dimensions.
|
|
64
|
+
* Used to match device crop and Figma crop pixel dimensions before pixelmatch.
|
|
65
|
+
*/
|
|
66
|
+
async function resizeRaw(raw, targetW, targetH) {
|
|
67
|
+
if (raw.width === targetW && raw.height === targetH)
|
|
68
|
+
return raw;
|
|
69
|
+
try {
|
|
70
|
+
const sharp = (await import("sharp")).default;
|
|
71
|
+
const result = await sharp(raw.data, {
|
|
72
|
+
raw: { width: raw.width, height: raw.height, channels: 4 },
|
|
73
|
+
})
|
|
74
|
+
.resize(targetW, targetH, { fit: "fill" })
|
|
75
|
+
.raw()
|
|
76
|
+
.toBuffer({ resolveWithObject: true });
|
|
77
|
+
return { data: result.data, width: result.info.width, height: result.info.height };
|
|
78
|
+
}
|
|
79
|
+
catch {
|
|
80
|
+
return null;
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
/**
|
|
84
|
+
* Core function: for each ref with sufficient area, scale its device bounds
|
|
85
|
+
* into Figma image space, crop both images, and compare independently.
|
|
86
|
+
*
|
|
87
|
+
* @param deviceBuffer Full-screen device screenshot PNG
|
|
88
|
+
* @param figmaBuffer Preprocessed Figma image PNG (artifacts already removed)
|
|
89
|
+
* @param refs All RefEntry objects from RefRegistry.getAllRefs()
|
|
90
|
+
* @param deviceW Device screen width in pixels
|
|
91
|
+
* @param deviceH Device screen height in pixels
|
|
92
|
+
*/
|
|
93
|
+
async function buildLayoutReport(deviceBuffer, figmaBuffer, refs, deviceW, deviceH) {
|
|
94
|
+
const sharp = (await import("sharp")).default;
|
|
95
|
+
const figmaMeta = await sharp(figmaBuffer).metadata();
|
|
96
|
+
const figmaW = figmaMeta.width;
|
|
97
|
+
const figmaH = figmaMeta.height;
|
|
98
|
+
const regions = [];
|
|
99
|
+
let skippedRefs = 0;
|
|
100
|
+
for (const entry of refs) {
|
|
101
|
+
const { bounds } = entry;
|
|
102
|
+
const area = bounds.width * bounds.height;
|
|
103
|
+
if (area < MIN_REGION_AREA) {
|
|
104
|
+
skippedRefs++;
|
|
105
|
+
continue;
|
|
106
|
+
}
|
|
107
|
+
const deviceBounds = { left: bounds.x, top: bounds.y, width: bounds.width, height: bounds.height };
|
|
108
|
+
const figmaBounds = scaleToFigmaSpace(bounds, deviceW, deviceH, figmaW, figmaH);
|
|
109
|
+
const clampedDevice = clampBounds(deviceBounds, deviceW, deviceH);
|
|
110
|
+
const clampedFigma = clampBounds(figmaBounds, figmaW, figmaH);
|
|
111
|
+
if (!clampedDevice || !clampedFigma) {
|
|
112
|
+
skippedRefs++;
|
|
113
|
+
continue;
|
|
114
|
+
}
|
|
115
|
+
// Crop both in parallel
|
|
116
|
+
const [deviceCrop, figmaCrop] = await Promise.all([
|
|
117
|
+
cropToRaw(deviceBuffer, clampedDevice),
|
|
118
|
+
cropToRaw(figmaBuffer, clampedFigma),
|
|
119
|
+
]);
|
|
120
|
+
if (!deviceCrop || !figmaCrop) {
|
|
121
|
+
skippedRefs++;
|
|
122
|
+
continue;
|
|
123
|
+
}
|
|
124
|
+
// Resize Figma crop to exactly match device crop dimensions
|
|
125
|
+
const figmaResized = await resizeRaw(figmaCrop, deviceCrop.width, deviceCrop.height);
|
|
126
|
+
if (!figmaResized) {
|
|
127
|
+
skippedRefs++;
|
|
128
|
+
continue;
|
|
129
|
+
}
|
|
130
|
+
const totalPixels = deviceCrop.width * deviceCrop.height;
|
|
131
|
+
const diff = Buffer.alloc(totalPixels * 4);
|
|
132
|
+
const diffPixels = (0, pixelmatch_1.default)(deviceCrop.data, figmaResized.data, diff, deviceCrop.width, deviceCrop.height, { threshold: REGION_THRESHOLD, includeAA: false });
|
|
133
|
+
const similarity = 1 - diffPixels / totalPixels;
|
|
134
|
+
regions.push({
|
|
135
|
+
ref: entry.ref,
|
|
136
|
+
elementType: entry.node.type,
|
|
137
|
+
text: entry.node.text,
|
|
138
|
+
label: entry.node.label,
|
|
139
|
+
deviceBounds,
|
|
140
|
+
figmaBounds,
|
|
141
|
+
similarity,
|
|
142
|
+
diffPixels,
|
|
143
|
+
totalPixels,
|
|
144
|
+
verdict: verdictFromSimilarity(similarity),
|
|
145
|
+
});
|
|
146
|
+
}
|
|
147
|
+
// Weighted average by region area
|
|
148
|
+
const totalWeight = regions.reduce((s, r) => s + r.totalPixels, 0);
|
|
149
|
+
const overallSimilarity = totalWeight > 0
|
|
150
|
+
? regions.reduce((s, r) => s + r.similarity * r.totalPixels, 0) / totalWeight
|
|
151
|
+
: 0;
|
|
152
|
+
return {
|
|
153
|
+
regions,
|
|
154
|
+
overallSimilarity,
|
|
155
|
+
skippedRefs,
|
|
156
|
+
deviceDimensions: { width: deviceW, height: deviceH },
|
|
157
|
+
figmaDimensions: { width: figmaW, height: figmaH },
|
|
158
|
+
};
|
|
159
|
+
}
|
|
160
|
+
/** Format layout report as human-readable text for MCP tool output */
|
|
161
|
+
function formatLayoutReport(report) {
|
|
162
|
+
const scaleX = (report.deviceDimensions.width / report.figmaDimensions.width).toFixed(2);
|
|
163
|
+
const scaleY = (report.deviceDimensions.height / report.figmaDimensions.height).toFixed(2);
|
|
164
|
+
const lines = [
|
|
165
|
+
"--- Layout Region Analysis ---",
|
|
166
|
+
`Device: ${report.deviceDimensions.width}x${report.deviceDimensions.height}px`,
|
|
167
|
+
`Figma: ${report.figmaDimensions.width}x${report.figmaDimensions.height}px`,
|
|
168
|
+
`Scale: ${scaleX}x (horizontal) ${scaleY}x (vertical)`,
|
|
169
|
+
`Regions: ${report.regions.length} analyzed, ${report.skippedRefs} too small to compare`,
|
|
170
|
+
"",
|
|
171
|
+
];
|
|
172
|
+
if (report.regions.length === 0) {
|
|
173
|
+
lines.push("No regions to analyze. Run devlens_snapshot first to populate element refs.");
|
|
174
|
+
return lines.join("\n");
|
|
175
|
+
}
|
|
176
|
+
// Sort ascending by similarity — problem areas at the bottom
|
|
177
|
+
const sorted = [...report.regions].sort((a, b) => a.similarity - b.similarity);
|
|
178
|
+
for (const r of sorted) {
|
|
179
|
+
const icon = r.verdict === "excellent" ? "✓"
|
|
180
|
+
: r.verdict === "good" ? "~"
|
|
181
|
+
: r.verdict === "partial" ? "!"
|
|
182
|
+
: "X";
|
|
183
|
+
const pct = (r.similarity * 100).toFixed(1);
|
|
184
|
+
const label = r.text
|
|
185
|
+
? `"${r.text.slice(0, 35)}"`
|
|
186
|
+
: r.label
|
|
187
|
+
? `label="${r.label.slice(0, 35)}"`
|
|
188
|
+
: r.elementType;
|
|
189
|
+
const suffix = (r.verdict === "poor" || r.verdict === "partial")
|
|
190
|
+
? ` <- investigate (${r.diffPixels.toLocaleString()} diff pixels)`
|
|
191
|
+
: "";
|
|
192
|
+
lines.push(` [${icon}] [${r.ref}] ${label}: ${pct}%${suffix}`);
|
|
193
|
+
}
|
|
194
|
+
lines.push("");
|
|
195
|
+
lines.push(`Overall (area-weighted): ${(report.overallSimilarity * 100).toFixed(1)}%`);
|
|
196
|
+
return lines.join("\n");
|
|
197
|
+
}
|
|
198
|
+
//# sourceMappingURL=layout-analyzer.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"layout-analyzer.js","sourceRoot":"","sources":["../../../src/visual/layout-analyzer.ts"],"names":[],"mappings":";;;;;AA4IA,8CA+FC;AAGD,gDA4CC;AA1RD,4DAAoC;AAuCpC,kEAAkE;AAClE,MAAM,eAAe,GAAG,IAAI,CAAC;AAE7B,oFAAoF;AACpF,MAAM,gBAAgB,GAAG,GAAG,CAAC;AAE7B,SAAS,qBAAqB,CAAC,CAAS;IACtC,IAAI,CAAC,IAAI,IAAI;QAAE,OAAO,WAAW,CAAC;IAClC,IAAI,CAAC,IAAI,IAAI;QAAE,OAAO,MAAM,CAAC;IAC7B,IAAI,CAAC,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAChC,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,qEAAqE;AACrE,SAAS,iBAAiB,CACxB,MAA+D,EAC/D,OAAe,EACf,OAAe,EACf,MAAc,EACd,MAAc;IAEd,MAAM,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC;IAChC,MAAM,MAAM,GAAG,MAAM,GAAG,OAAO,CAAC;IAChC,OAAO;QACL,IAAI,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,GAAG,MAAM,CAAC;QACnC,GAAG,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,GAAG,MAAM,CAAC;QAClC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,KAAK,GAAG,MAAM,CAAC,CAAC;QACrD,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC,CAAC;KACxD,CAAC;AACJ,CAAC;AAED,4EAA4E;AAC5E,SAAS,WAAW,CAClB,CAA+D,EAC/D,IAAY,EACZ,IAAY;IAEZ,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;IACjC,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC;IAC/B,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC;IAC/C,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,CAAC,CAAC,GAAG,GAAG,CAAC,CAAC,MAAM,CAAC,CAAC;IAChD,IAAI,KAAK,IAAI,IAAI,IAAI,MAAM,IAAI,GAAG;QAAE,OAAO,IAAI,CAAC;IAChD,OAAO,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,KAAK,GAAG,IAAI,EAAE,MAAM,EAAE,MAAM,GAAG,GAAG,EAAE,CAAC;AAClE,CAAC;AAED;;;;GAIG;AACH,KAAK,UAAU,SAAS,CACtB,WAAmB,EACnB,MAAoE;IAEpE,IAAI,CAAC;QACH,MAAM,KAAK,GAAG,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC;QAC9C,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,WAAW,CAAC;aACpC,OAAO,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC;aAC3F,WAAW,EAAE;aACb,GAAG,EAAE;aACL,QAAQ,CAAC,EAAE,iBAAiB,EAAE,IAAI,EAAE,CAAC,CAAC;QACzC,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,KAAK,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;IACrF,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC;AAED;;;GAGG;AACH,KAAK,UAAU,SAAS,CACtB,GAAoD,EACpD,OAAe,EACf,OAAe;IAEf,IAAI,GAAG,CAAC,KAAK,KAAK,OAAO,IAAI,GAAG,CAAC,MAAM,KAAK,OAAO;QAAE,OAAO,GAAG,CAAC;IAChE,IAAI,CAAC;QACH,MAAM,KAAK,GAAG,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC;QAC9C,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE;YACnC,GAAG,EAAE,EAAE,KAAK,EAAE,GAAG,CAAC,KAAK,EAAE,MAAM,EAAE,GAAG,CAAC,MAAM,EAAE,QAAQ,EAAE,CAAC,EAAE;SAC3D,CAAC;aACC,MAAM,CAAC,OAAO,EAAE,OAAO,EAAE,EAAE,GAAG,EAAE,MAAM,EAAE,CAAC;aACzC,GAAG,EAAE;aACL,QAAQ,CAAC,EAAE,iBAAiB,EAAE,IAAI,EAAE,CAAC,CAAC;QACzC,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,KAAK,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;IACrF,CAAC;IAAC,MAAM,CAAC;QACP,OAAO,IAAI,CAAC;IACd,CAAC;AACH,CAAC;AAED;;;;;;;;;GASG;AACI,KAAK,UAAU,iBAAiB,CACrC,YAAoB,EACpB,WAAmB,EACnB,IAAgB,EAChB,OAAe,EACf,OAAe;IAEf,MAAM,KAAK,GAAG,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC;IAE9C,MAAM,SAAS,GAAG,MAAM,KAAK,CAAC,WAAW,CAAC,CAAC,QAAQ,EAAE,CAAC;IACtD,MAAM,MAAM,GAAG,SAAS,CAAC,KAAM,CAAC;IAChC,MAAM,MAAM,GAAG,SAAS,CAAC,MAAO,CAAC;IAEjC,MAAM,OAAO,GAA6B,EAAE,CAAC;IAC7C,IAAI,WAAW,GAAG,CAAC,CAAC;IAEpB,KAAK,MAAM,KAAK,IAAI,IAAI,EAAE,CAAC;QACzB,MAAM,EAAE,MAAM,EAAE,GAAG,KAAK,CAAC;QACzB,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,GAAG,MAAM,CAAC,MAAM,CAAC;QAE1C,IAAI,IAAI,GAAG,eAAe,EAAE,CAAC;YAC3B,WAAW,EAAE,CAAC;YACd,SAAS;QACX,CAAC;QAED,MAAM,YAAY,GAAG,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,CAAC,EAAE,KAAK,EAAE,MAAM,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC;QACnG,MAAM,WAAW,GAAG,iBAAiB,CAAC,MAAM,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;QAEhF,MAAM,aAAa,GAAG,WAAW,CAAC,YAAY,EAAE,OAAO,EAAE,OAAO,CAAC,CAAC;QAClE,MAAM,YAAY,GAAG,WAAW,CAAC,WAAW,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;QAE9D,IAAI,CAAC,aAAa,IAAI,CAAC,YAAY,EAAE,CAAC;YACpC,WAAW,EAAE,CAAC;YACd,SAAS;QACX,CAAC;QAED,wBAAwB;QACxB,MAAM,CAAC,UAAU,EAAE,SAAS,CAAC,GAAG,MAAM,OAAO,CAAC,GAAG,CAAC;YAChD,SAAS,CAAC,YAAY,EAAE,aAAa,CAAC;YACtC,SAAS,CAAC,WAAW,EAAE,YAAY,CAAC;SACrC,CAAC,CAAC;QAEH,IAAI,CAAC,UAAU,IAAI,CAAC,SAAS,EAAE,CAAC;YAC9B,WAAW,EAAE,CAAC;YACd,SAAS;QACX,CAAC;QAED,4DAA4D;QAC5D,MAAM,YAAY,GAAG,MAAM,SAAS,CAAC,SAAS,EAAE,UAAU,CAAC,KAAK,EAAE,UAAU,CAAC,MAAM,CAAC,CAAC;QACrF,IAAI,CAAC,YAAY,EAAE,CAAC;YAClB,WAAW,EAAE,CAAC;YACd,SAAS;QACX,CAAC;QAED,MAAM,WAAW,GAAG,UAAU,CAAC,KAAK,GAAG,UAAU,CAAC,MAAM,CAAC;QACzD,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,WAAW,GAAG,CAAC,CAAC,CAAC;QAE3C,MAAM,UAAU,GAAG,IAAA,oBAAU,EAC3B,UAAU,CAAC,IAAI,EACf,YAAY,CAAC,IAAI,EACjB,IAAI,EACJ,UAAU,CAAC,KAAK,EAChB,UAAU,CAAC,MAAM,EACjB,EAAE,SAAS,EAAE,gBAAgB,EAAE,SAAS,EAAE,KAAK,EAAE,CAClD,CAAC;QAEF,MAAM,UAAU,GAAG,CAAC,GAAG,UAAU,GAAG,WAAW,CAAC;QAEhD,OAAO,CAAC,IAAI,CAAC;YACX,GAAG,EAAE,KAAK,CAAC,GAAG;YACd,WAAW,EAAE,KAAK,CAAC,IAAI,CAAC,IAAI;YAC5B,IAAI,EAAE,KAAK,CAAC,IAAI,CAAC,IAAI;YACrB,KAAK,EAAE,KAAK,CAAC,IAAI,CAAC,KAAK;YACvB,YAAY;YACZ,WAAW;YACX,UAAU;YACV,UAAU;YACV,WAAW;YACX,OAAO,EAAE,qBAAqB,CAAC,UAAU,CAAC;SAC3C,CAAC,CAAC;IACL,CAAC;IAED,kCAAkC;IAClC,MAAM,WAAW,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;IACnE,MAAM,iBAAiB,GAAG,WAAW,GAAG,CAAC;QACvC,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,UAAU,GAAG,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,GAAG,WAAW;QAC7E,CAAC,CAAC,CAAC,CAAC;IAEN,OAAO;QACL,OAAO;QACP,iBAAiB;QACjB,WAAW;QACX,gBAAgB,EAAE,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE;QACrD,eAAe,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE;KACnD,CAAC;AACJ,CAAC;AAED,sEAAsE;AACtE,SAAgB,kBAAkB,CAAC,MAAoB;IACrD,MAAM,MAAM,GAAG,CAAC,MAAM,CAAC,gBAAgB,CAAC,KAAK,GAAG,MAAM,CAAC,eAAe,CAAC,KAAK,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;IACzF,MAAM,MAAM,GAAG,CAAC,MAAM,CAAC,gBAAgB,CAAC,MAAM,GAAG,MAAM,CAAC,eAAe,CAAC,MAAM,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;IAE3F,MAAM,KAAK,GAAa;QACtB,gCAAgC;QAChC,WAAW,MAAM,CAAC,gBAAgB,CAAC,KAAK,IAAI,MAAM,CAAC,gBAAgB,CAAC,MAAM,IAAI;QAC9E,WAAW,MAAM,CAAC,eAAe,CAAC,KAAK,IAAI,MAAM,CAAC,eAAe,CAAC,MAAM,IAAI;QAC5E,WAAW,MAAM,kBAAkB,MAAM,cAAc;QACvD,YAAY,MAAM,CAAC,OAAO,CAAC,MAAM,cAAc,MAAM,CAAC,WAAW,uBAAuB;QACxF,EAAE;KACH,CAAC;IAEF,IAAI,MAAM,CAAC,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAChC,KAAK,CAAC,IAAI,CAAC,6EAA6E,CAAC,CAAC;QAC1F,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAC1B,CAAC;IAED,6DAA6D;IAC7D,MAAM,MAAM,GAAG,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,UAAU,GAAG,CAAC,CAAC,UAAU,CAAC,CAAC;IAE/E,KAAK,MAAM,CAAC,IAAI,MAAM,EAAE,CAAC;QACvB,MAAM,IAAI,GAAG,CAAC,CAAC,OAAO,KAAK,WAAW,CAAC,CAAC,CAAC,GAAG;YACjC,CAAC,CAAC,CAAC,CAAC,OAAO,KAAK,MAAM,CAAM,CAAC,CAAC,GAAG;gBACjC,CAAC,CAAC,CAAC,CAAC,OAAO,KAAK,SAAS,CAAG,CAAC,CAAC,GAAG;oBACjC,CAAC,CAA6B,GAAG,CAAC;QAC7C,MAAM,GAAG,GAAG,CAAC,CAAC,CAAC,UAAU,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC;QAC5C,MAAM,KAAK,GAAG,CAAC,CAAC,IAAI;YAClB,CAAC,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG;YAC5B,CAAC,CAAC,CAAC,CAAC,KAAK;gBACT,CAAC,CAAC,UAAU,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG;gBACnC,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC;QAElB,MAAM,MAAM,GAAG,CAAC,CAAC,CAAC,OAAO,KAAK,MAAM,IAAI,CAAC,CAAC,OAAO,KAAK,SAAS,CAAC;YAC9D,CAAC,CAAC,qBAAqB,CAAC,CAAC,UAAU,CAAC,cAAc,EAAE,eAAe;YACnE,CAAC,CAAC,EAAE,CAAC;QAEP,KAAK,CAAC,IAAI,CAAC,MAAM,IAAI,MAAM,CAAC,CAAC,GAAG,KAAK,KAAK,KAAK,GAAG,IAAI,MAAM,EAAE,CAAC,CAAC;IAClE,CAAC;IAED,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,4BAA4B,CAAC,MAAM,CAAC,iBAAiB,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC;IAEvF,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC"}
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
import type { Device } from "../platform/device.js";
|
|
2
|
+
import type { Bounds } from "../platform/device.js";
|
|
3
|
+
/**
|
|
4
|
+
* Screenshot utilities — capture, encode, crop, and save screenshots.
|
|
5
|
+
*/
|
|
6
|
+
/** Take a screenshot and return it as base64-encoded PNG */
|
|
7
|
+
export declare function captureScreenshot(device: Device): Promise<{
|
|
8
|
+
buffer: Buffer;
|
|
9
|
+
base64: string;
|
|
10
|
+
}>;
|
|
11
|
+
/** Save a screenshot buffer to disk */
|
|
12
|
+
export declare function saveScreenshot(buffer: Buffer, filePath: string): Promise<void>;
|
|
13
|
+
/** Crop a screenshot to a specific element's bounds */
|
|
14
|
+
export declare function cropToElement(screenshotBuffer: Buffer, bounds: Bounds): Promise<Buffer>;
|
|
15
|
+
/** Generate a default screenshot filename with timestamp */
|
|
16
|
+
export declare function defaultScreenshotPath(): string;
|
|
17
|
+
//# sourceMappingURL=screenshot.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"screenshot.d.ts","sourceRoot":"","sources":["../../../src/visual/screenshot.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,uBAAuB,CAAC;AACpD,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,uBAAuB,CAAC;AAEpD;;GAEG;AAEH,4DAA4D;AAC5D,wBAAsB,iBAAiB,CAAC,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC;IAC/D,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;CAChB,CAAC,CAID;AAED,uCAAuC;AACvC,wBAAsB,cAAc,CAClC,MAAM,EAAE,MAAM,EACd,QAAQ,EAAE,MAAM,GACf,OAAO,CAAC,IAAI,CAAC,CAEf;AAED,uDAAuD;AACvD,wBAAsB,aAAa,CACjC,gBAAgB,EAAE,MAAM,EACxB,MAAM,EAAE,MAAM,GACb,OAAO,CAAC,MAAM,CAAC,CAYjB;AAED,4DAA4D;AAC5D,wBAAgB,qBAAqB,IAAI,MAAM,CAG9C"}
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.captureScreenshot = captureScreenshot;
|
|
4
|
+
exports.saveScreenshot = saveScreenshot;
|
|
5
|
+
exports.cropToElement = cropToElement;
|
|
6
|
+
exports.defaultScreenshotPath = defaultScreenshotPath;
|
|
7
|
+
const promises_1 = require("fs/promises");
|
|
8
|
+
/**
|
|
9
|
+
* Screenshot utilities — capture, encode, crop, and save screenshots.
|
|
10
|
+
*/
|
|
11
|
+
/** Take a screenshot and return it as base64-encoded PNG */
|
|
12
|
+
async function captureScreenshot(device) {
|
|
13
|
+
const buffer = await device.takeScreenshot();
|
|
14
|
+
const base64 = buffer.toString("base64");
|
|
15
|
+
return { buffer, base64 };
|
|
16
|
+
}
|
|
17
|
+
/** Save a screenshot buffer to disk */
|
|
18
|
+
async function saveScreenshot(buffer, filePath) {
|
|
19
|
+
await (0, promises_1.writeFile)(filePath, buffer);
|
|
20
|
+
}
|
|
21
|
+
/** Crop a screenshot to a specific element's bounds */
|
|
22
|
+
async function cropToElement(screenshotBuffer, bounds) {
|
|
23
|
+
const sharp = (await import("sharp")).default;
|
|
24
|
+
return sharp(screenshotBuffer)
|
|
25
|
+
.extract({
|
|
26
|
+
left: Math.max(0, Math.round(bounds.x)),
|
|
27
|
+
top: Math.max(0, Math.round(bounds.y)),
|
|
28
|
+
width: Math.max(1, Math.round(bounds.width)),
|
|
29
|
+
height: Math.max(1, Math.round(bounds.height)),
|
|
30
|
+
})
|
|
31
|
+
.png()
|
|
32
|
+
.toBuffer();
|
|
33
|
+
}
|
|
34
|
+
/** Generate a default screenshot filename with timestamp */
|
|
35
|
+
function defaultScreenshotPath() {
|
|
36
|
+
const timestamp = new Date().toISOString().replace(/[:.]/g, "-");
|
|
37
|
+
return `/tmp/devlens-screenshot-${timestamp}.png`;
|
|
38
|
+
}
|
|
39
|
+
//# sourceMappingURL=screenshot.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"screenshot.js","sourceRoot":"","sources":["../../../src/visual/screenshot.ts"],"names":[],"mappings":";;AASA,8CAOC;AAGD,wCAKC;AAGD,sCAeC;AAGD,sDAGC;AAhDD,0CAAwC;AAIxC;;GAEG;AAEH,4DAA4D;AACrD,KAAK,UAAU,iBAAiB,CAAC,MAAc;IAIpD,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,cAAc,EAAE,CAAC;IAC7C,MAAM,MAAM,GAAG,MAAM,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC;IACzC,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC;AAC5B,CAAC;AAED,uCAAuC;AAChC,KAAK,UAAU,cAAc,CAClC,MAAc,EACd,QAAgB;IAEhB,MAAM,IAAA,oBAAS,EAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;AACpC,CAAC;AAED,uDAAuD;AAChD,KAAK,UAAU,aAAa,CACjC,gBAAwB,EACxB,MAAc;IAEd,MAAM,KAAK,GAAG,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,CAAC;IAE9C,OAAO,KAAK,CAAC,gBAAgB,CAAC;SAC3B,OAAO,CAAC;QACP,IAAI,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;QACvC,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;QACtC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QAC5C,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;KAC/C,CAAC;SACD,GAAG,EAAE;SACL,QAAQ,EAAE,CAAC;AAChB,CAAC;AAED,4DAA4D;AAC5D,SAAgB,qBAAqB;IACnC,MAAM,SAAS,GAAG,IAAI,IAAI,EAAE,CAAC,WAAW,EAAE,CAAC,OAAO,CAAC,OAAO,EAAE,GAAG,CAAC,CAAC;IACjE,OAAO,2BAA2B,SAAS,MAAM,CAAC;AACpD,CAAC"}
|
|
@@ -0,0 +1,289 @@
|
|
|
1
|
+
# Figma to Code Workflow with DevLens
|
|
2
|
+
|
|
3
|
+
This guide explains how to use DevLens together with Figma MCP to create a fully automated **design → code → verify → fix** loop for React Native development.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## Overview
|
|
8
|
+
|
|
9
|
+
```
|
|
10
|
+
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
|
|
11
|
+
│ Figma │────►│ AI │────►│ Code │────►│ Device │
|
|
12
|
+
│ Design │ │ Agent │ │ Editor │ │ Screen │
|
|
13
|
+
└──────────┘ └──────┬───┘ └──────────┘ └────┬─────┘
|
|
14
|
+
│ │
|
|
15
|
+
│◄──── devlens_compare ──────────┘
|
|
16
|
+
│ (similarity < 95%?)
|
|
17
|
+
│ │
|
|
18
|
+
▼ ▼
|
|
19
|
+
Fix Code ◄──── Diff Image
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## Prerequisites
|
|
25
|
+
|
|
26
|
+
1. **Figma MCP** configured in your AI assistant (for reading designs)
|
|
27
|
+
2. **DevLens MCP** configured (for device interaction) — see [Setup Guide](./setup-guide.md)
|
|
28
|
+
3. **`FIGMA_TOKEN`** environment variable set (for `devlens_compare_with_figma`)
|
|
29
|
+
4. **Metro bundler** running (`npx react-native start`)
|
|
30
|
+
5. **Emulator/Simulator** running with your app loaded
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## The Workflow — Step by Step
|
|
35
|
+
|
|
36
|
+
### Step 1: Check Metro Health
|
|
37
|
+
|
|
38
|
+
Before anything else, verify Metro is running and healthy:
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
devlens_metro_status
|
|
42
|
+
→ Running: true, Port: 8081, Packager: running
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
If Metro is down, you'll get actionable instructions:
|
|
46
|
+
```
|
|
47
|
+
Metro is NOT running. Start it with:
|
|
48
|
+
cd your-project && npx react-native start
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
### Step 2: Get the Figma Design
|
|
52
|
+
|
|
53
|
+
The AI reads the Figma design using Figma MCP. Two approaches for comparison:
|
|
54
|
+
|
|
55
|
+
**Option A — Direct URL (Recommended):**
|
|
56
|
+
Pass the Figma URL to `devlens_compare_with_figma` — it fetches the design via Figma REST API and compares automatically. One tool call, no piping.
|
|
57
|
+
|
|
58
|
+
**Option B — File reference:**
|
|
59
|
+
Export the Figma frame as PNG, save to your project (e.g., `./design-refs/home-screen.png`), and pass the path to `devlens_compare_screenshot`.
|
|
60
|
+
|
|
61
|
+
### Step 3: Prompt the AI
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
Build the HomeScreen component based on this Figma design:
|
|
65
|
+
[paste Figma URL]
|
|
66
|
+
|
|
67
|
+
Use DevLens to verify the implementation matches the design.
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
### Step 4: The Automated Verification Loop
|
|
71
|
+
|
|
72
|
+
The AI executes this loop:
|
|
73
|
+
|
|
74
|
+
```
|
|
75
|
+
1. devlens_metro_status ← Verify Metro is healthy
|
|
76
|
+
2. devlens_dismiss_overlays ← Remove debug banners
|
|
77
|
+
3. Generate/update React Native code
|
|
78
|
+
4. devlens_hot_reload ← Trigger Metro fast refresh
|
|
79
|
+
5. devlens_wait_for_screen ← Wait for screen to stabilize
|
|
80
|
+
6. devlens_snapshot(validate: true) ← Catch invisible/zero-size elements
|
|
81
|
+
7. devlens_compare_with_figma ← Compare with Figma design (one call)
|
|
82
|
+
(figmaUrl: "https://figma.com/design/...")
|
|
83
|
+
8. Analyze diff image ← Identify mismatches
|
|
84
|
+
9. Fix code & repeat from step 4 ← Until similarity > 95%
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Key improvements in v0.3.0:
|
|
88
|
+
- **Step 2** removes React Native debug banners that ruin comparison scores
|
|
89
|
+
- **Step 7** uses `devlens_compare_with_figma` — a single tool call that fetches the Figma design via REST API and compares. No base64 piping, no broken image pipeline.
|
|
90
|
+
- **`cropRef`/`region`** on compare tools lets you ignore status bars and focus on specific areas
|
|
91
|
+
|
|
92
|
+
### Step 5: AI Analyzes the Diff
|
|
93
|
+
|
|
94
|
+
After comparison, the AI gets:
|
|
95
|
+
- **Similarity score** (e.g., 72%)
|
|
96
|
+
- **Diff image** with red highlights showing exactly where differences are
|
|
97
|
+
- **Summary** categorizing the match quality
|
|
98
|
+
|
|
99
|
+
The AI reads the diff image to identify specific issues (wrong font size, missing padding, incorrect color) and fixes them.
|
|
100
|
+
|
|
101
|
+
---
|
|
102
|
+
|
|
103
|
+
## Example Conversation
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
User: Build the login screen from Figma.
|
|
107
|
+
Figma URL: https://figma.com/design/abc123/MyApp?node-id=10-200
|
|
108
|
+
|
|
109
|
+
AI: I'll start by checking Metro health and dismissing debug overlays.
|
|
110
|
+
[calls devlens_metro_status → Metro healthy on port 8081]
|
|
111
|
+
[calls devlens_dismiss_overlays → LogBox suppressed]
|
|
112
|
+
[calls get_design_context → reads colors, spacing, typography]
|
|
113
|
+
|
|
114
|
+
I'll create LoginScreen.tsx now.
|
|
115
|
+
[writes LoginScreen.tsx]
|
|
116
|
+
[calls devlens_hot_reload]
|
|
117
|
+
[calls devlens_wait_for_screen(stableMs: 1500)]
|
|
118
|
+
[calls devlens_snapshot(validate: true)]
|
|
119
|
+
|
|
120
|
+
Validation report: all elements visible, no issues.
|
|
121
|
+
|
|
122
|
+
[calls devlens_compare_with_figma(
|
|
123
|
+
figmaUrl: "https://figma.com/design/abc123/MyApp?node-id=10-200")]
|
|
124
|
+
|
|
125
|
+
Similarity: 72%. The diff shows:
|
|
126
|
+
- Title text is 18px but should be 24px
|
|
127
|
+
- Primary button color is #333 but should be #2563EB
|
|
128
|
+
- Bottom padding missing on form container
|
|
129
|
+
|
|
130
|
+
Fixing these issues...
|
|
131
|
+
[updates LoginScreen.tsx]
|
|
132
|
+
[calls devlens_hot_reload]
|
|
133
|
+
[calls devlens_wait_for_screen(stableMs: 1000)]
|
|
134
|
+
[calls devlens_compare_with_figma(
|
|
135
|
+
figmaUrl: "https://figma.com/design/abc123/MyApp?node-id=10-200")]
|
|
136
|
+
|
|
137
|
+
Similarity: 91%. Remaining: input border radius 8px → should be 12px.
|
|
138
|
+
|
|
139
|
+
[fixes border radius]
|
|
140
|
+
[calls devlens_hot_reload]
|
|
141
|
+
[calls devlens_wait_for_screen]
|
|
142
|
+
[calls devlens_compare_with_figma(
|
|
143
|
+
figmaUrl: "https://figma.com/design/abc123/MyApp?node-id=10-200")]
|
|
144
|
+
|
|
145
|
+
Similarity: 97%. The implementation matches the Figma design.
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
---
|
|
149
|
+
|
|
150
|
+
## Pro Tips
|
|
151
|
+
|
|
152
|
+
### 1. Always Start with Metro Status + Dismiss Overlays
|
|
153
|
+
Call `devlens_metro_status` at the beginning of any session, then `devlens_dismiss_overlays` to remove React Native debug banners. This ensures clean screenshots for comparison. If Metro dies mid-session, call status again to diagnose.
|
|
154
|
+
|
|
155
|
+
### 2. Use Validation Mode to Catch Invisible Elements
|
|
156
|
+
When you notice a component is missing from the screenshot but you know the code renders it:
|
|
157
|
+
```
|
|
158
|
+
devlens_snapshot(validate: true)
|
|
159
|
+
```
|
|
160
|
+
This will flag zero-size elements, invisible nodes, and empty containers. Common causes:
|
|
161
|
+
- Image with missing/broken source → renders as 0x0
|
|
162
|
+
- `display: 'none'` or `opacity: 0` styles
|
|
163
|
+
- Parent container collapsing due to missing flex
|
|
164
|
+
|
|
165
|
+
### 3. Use Wait-for-Screen Instead of Sleep
|
|
166
|
+
Instead of:
|
|
167
|
+
```
|
|
168
|
+
devlens_hot_reload → sleep 3 → devlens_screenshot
|
|
169
|
+
```
|
|
170
|
+
Use:
|
|
171
|
+
```
|
|
172
|
+
devlens_hot_reload → devlens_wait_for_screen(stableMs: 1500)
|
|
173
|
+
```
|
|
174
|
+
This waits only as long as needed (faster) and guarantees the screen has stopped changing (more reliable).
|
|
175
|
+
|
|
176
|
+
### 4. Match Screenshot Dimensions
|
|
177
|
+
Export your Figma design at the same dimensions as your emulator/simulator screen. DevLens will resize for comparison, but matching dimensions gives the most accurate results.
|
|
178
|
+
|
|
179
|
+
If dimensions don't match, choose a resize strategy:
|
|
180
|
+
- **`fit`** (default) — resize both to smaller dimensions
|
|
181
|
+
- **`scale`** — resize smaller image up to larger
|
|
182
|
+
- **`crop`** — compare only the overlapping region
|
|
183
|
+
|
|
184
|
+
### 5. Use Element Screenshots for Components
|
|
185
|
+
For comparing individual components (not full screens):
|
|
186
|
+
```
|
|
187
|
+
devlens_snapshot() → get refs
|
|
188
|
+
devlens_element_screenshot(ref: "e5") → screenshot just that component
|
|
189
|
+
devlens_compare_screenshot( → compare against Figma component
|
|
190
|
+
referenceImageBase64: <figma_component_base64>
|
|
191
|
+
)
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
### 6. Adjust Comparison Threshold
|
|
195
|
+
The default threshold (0.1) works well for most cases:
|
|
196
|
+
- **Strict matching** (exact colors): use `threshold: 0.05`
|
|
197
|
+
- **Lenient matching** (layout only): use `threshold: 0.2`
|
|
198
|
+
|
|
199
|
+
### 7. Use Incremental Snapshots in Loops
|
|
200
|
+
After the first full snapshot, use `mode: "incremental"` to save tokens:
|
|
201
|
+
```
|
|
202
|
+
devlens_snapshot(mode: "full") → first time
|
|
203
|
+
devlens_snapshot(mode: "incremental") → subsequent calls (only changes)
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
### 8. Check Metro Logs for Runtime Errors
|
|
207
|
+
If the screen looks wrong, there might be runtime errors:
|
|
208
|
+
```
|
|
209
|
+
devlens_metro_logs(level: "error")
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
### 9. Use Capture Flow for Multi-Screen Verification
|
|
213
|
+
Instead of calling tap → screenshot → back for each screen:
|
|
214
|
+
```
|
|
215
|
+
devlens_capture_flow(steps: [
|
|
216
|
+
{ action: "tap", ref: "e5", element: "Tab 1" },
|
|
217
|
+
{ action: "screenshot", label: "tab1-screen" },
|
|
218
|
+
{ action: "tap", ref: "e6", element: "Tab 2" },
|
|
219
|
+
{ action: "screenshot", label: "tab2-screen" },
|
|
220
|
+
{ action: "tap", ref: "e7", element: "Tab 3" },
|
|
221
|
+
{ action: "screenshot", label: "tab3-screen" }
|
|
222
|
+
])
|
|
223
|
+
```
|
|
224
|
+
This returns all screenshots in a single call — saving ~12 tool calls for 4 screens.
|
|
225
|
+
|
|
226
|
+
### 10. Use Layout Detection for Debugging
|
|
227
|
+
When children are laid out incorrectly (horizontal vs vertical):
|
|
228
|
+
```
|
|
229
|
+
devlens_element_info(ref: "e3")
|
|
230
|
+
→ Inferred Layout: row (children arranged left-to-right)
|
|
231
|
+
```
|
|
232
|
+
If it says `row` but your design expects `column`, you know the `flexDirection` is wrong.
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
## Combining with Other MCP Servers
|
|
237
|
+
|
|
238
|
+
DevLens works great alongside:
|
|
239
|
+
|
|
240
|
+
| MCP Server | Use |
|
|
241
|
+
|-----------|-----|
|
|
242
|
+
| **Figma MCP** | Read design specs, get screenshots as base64 for comparison |
|
|
243
|
+
| **Playwright MCP** | Test web views or web versions of your app |
|
|
244
|
+
| **GitHub MCP** | Create PRs with before/after screenshots |
|
|
245
|
+
|
|
246
|
+
---
|
|
247
|
+
|
|
248
|
+
## Troubleshooting
|
|
249
|
+
|
|
250
|
+
### Similarity is always low (< 50%)
|
|
251
|
+
- **Most common cause**: Figma presentation artifacts (rounded corners, grey backgrounds)
|
|
252
|
+
- **Solution**: Use `preprocessReference: true` (enabled by default for `devlens_compare_with_figma`)
|
|
253
|
+
- Check that the reference image dimensions match the device screen
|
|
254
|
+
- Ensure the correct screen is displayed on the device
|
|
255
|
+
- Verify the app has fully loaded — use `devlens_wait_for_screen` before comparing
|
|
256
|
+
- Try `resizeStrategy: "scale"` if the reference image is smaller than the device screen
|
|
257
|
+
|
|
258
|
+
### Preprocessing removes too much or too little
|
|
259
|
+
The preprocessing uses smart content detection, but you can control it:
|
|
260
|
+
|
|
261
|
+
**If too aggressive** (removes actual content):
|
|
262
|
+
```
|
|
263
|
+
preprocessReference: false // Disable preprocessing
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
**For debugging** (see what was removed):
|
|
267
|
+
- Check the console output — it logs detected artifacts
|
|
268
|
+
- Preprocessed dimensions are shown in comparison results
|
|
269
|
+
|
|
270
|
+
**Known limitations**:
|
|
271
|
+
- Works best when Figma export dimensions match device screen
|
|
272
|
+
- Very complex backgrounds may confuse content detection
|
|
273
|
+
- If similarity is still low after preprocessing, check for dimension mismatches
|
|
274
|
+
|
|
275
|
+
### "Could not connect to Metro"
|
|
276
|
+
- Run `devlens_metro_status` for a detailed health report
|
|
277
|
+
- Metro must be running: `npx react-native start`
|
|
278
|
+
- Check `METRO_PORT` matches your Metro config (default: 8081)
|
|
279
|
+
|
|
280
|
+
### Hot reload doesn't seem to work
|
|
281
|
+
- Verify Metro is watching your files (check Metro terminal output)
|
|
282
|
+
- Use `devlens_wait_for_screen` after hot reload to confirm the screen updated
|
|
283
|
+
- Try a full reload: `devlens_terminate_app` → `devlens_launch_app`
|
|
284
|
+
- Check for syntax errors: `devlens_metro_logs(level: "error")`
|
|
285
|
+
|
|
286
|
+
### Elements appear in code but not on screen
|
|
287
|
+
- Run `devlens_snapshot(validate: true)` to check for zero-size or invisible elements
|
|
288
|
+
- Use `devlens_find_element(text: "missing text")` — it will warn if the element exists but is invisible
|
|
289
|
+
- Common causes: missing image source, parent overflow hidden, style width/height 0
|