webcake-landing-mcp 1.0.79 → 1.0.81
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -3
- package/dist/changelog.json +14 -14
- package/dist/domains/landing/guide.js +3 -1
- package/dist/domains/landing/instructions.js +2 -1
- package/dist/http.js +39 -0
- package/dist/legal.js +6 -0
- package/dist/mcp/response.js +12 -0
- package/dist/persistence/screenshot-client.js +162 -0
- package/dist/persistence/screenshot-playwright.js +182 -0
- package/dist/smoke.js +37 -0
- package/dist/tools/media.js +52 -1
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -172,7 +172,7 @@ server, the `login` browser flow (+ backend contract), and how to grab a JWT by
|
|
|
172
172
|
|-------|---------------|
|
|
173
173
|
| **[Connect your IDE / claude.ai](docs/connect-mcp.md)** | Step-by-step connection for every client (npx & hosted URL), troubleshooting table. |
|
|
174
174
|
| **[Configuration](docs/configuration.md)** | Env vars, `--env` presets, browser `login`, per-request headers, getting a JWT. |
|
|
175
|
-
| **[Tools reference](docs/tools.md)** | All
|
|
175
|
+
| **[Tools reference](docs/tools.md)** | All 22 tools in detail + the step-by-step workflow + model notes. |
|
|
176
176
|
| **[Usage examples](docs/usage-examples.md)** | Three end-to-end walkthroughs: build from a brief, surgical edit, inspect a type. |
|
|
177
177
|
| **[Manual / advanced install](docs/manual-install.md)** | Shell installers, cloned builds, hand-written per-IDE config. |
|
|
178
178
|
| **[Page-element schema](docs/page-element-schema.md)** | The full element-model reference (+ [every special/event](docs/element-specials-reference.md)). |
|
|
@@ -181,13 +181,13 @@ server, the `login` browser flow (+ backend contract), and how to grab a JWT by
|
|
|
181
181
|
|
|
182
182
|
## 🧰 The tools at a glance
|
|
183
183
|
|
|
184
|
-
|
|
184
|
+
22 tools in five groups — full descriptions in **[docs/tools.md](docs/tools.md)**:
|
|
185
185
|
|
|
186
186
|
| Group | Tools | Needs |
|
|
187
187
|
|-------|-------|-------|
|
|
188
188
|
| **Reference** | `get_generation_guide` · `list_elements` · `get_element` · `get_page_schema` | nothing |
|
|
189
189
|
| **Generation** | `new_element` · `new_page_skeleton` · `validate_page` | nothing |
|
|
190
|
-
| **Media** | `search_images` (real Pexels stock photos) · `get_icon_svg` (Material Symbols / Font Awesome icon names → inline SVG via Iconify) · `upload_images` (re-host external images, data: URIs, or local file paths from the user's machine) | nothing |
|
|
190
|
+
| **Media** | `search_images` (real Pexels stock photos) · `get_icon_svg` (Material Symbols / Font Awesome icon names → inline SVG via Iconify) · `upload_images` (re-host external images, data: URIs, or local file paths from the user's machine) · `render_preview` (screenshot a page/URL so the model can see + compare it) | nothing |
|
|
191
191
|
| **Ingest** | `ingest_html` · `ingest_url` (recreate an existing page) | nothing |
|
|
192
192
|
| **Persistence** | `list_organizations` · `create_page` · `list_pages` · `find_pages` · `get_page` · `update_page` · `add_section` · `patch_page` · `publish_page` | `WEBCAKE_API_BASE` + `WEBCAKE_JWT` |
|
|
193
193
|
|
package/dist/changelog.json
CHANGED
|
@@ -1,4 +1,18 @@
|
|
|
1
1
|
[
|
|
2
|
+
{
|
|
3
|
+
"v": "1.0.81",
|
|
4
|
+
"d": "16/06/2026",
|
|
5
|
+
"type": "Changed",
|
|
6
|
+
"en": "The Playwright screenshot engine that backs render_preview's GET /api/render/screenshot route now defaults to JPEG output instead of PNG, reducing…",
|
|
7
|
+
"vi": "Engine chụp màn hình Playwright phục vụ route GET /api/render/screenshot của render_preview nay mặc định xuất ảnh JPEG thay vì PNG, giúp giảm kích…"
|
|
8
|
+
},
|
|
9
|
+
{
|
|
10
|
+
"v": "1.0.80",
|
|
11
|
+
"d": "16/06/2026",
|
|
12
|
+
"type": "Added",
|
|
13
|
+
"en": "New render_preview tool screenshots a page's /preview/<id> or any public URL and returns a PNG the model can see, enabling a…",
|
|
14
|
+
"vi": "Tool mới render_preview chụp màn hình /preview/<id> của một trang hoặc bất kỳ URL công khai nào và trả về ảnh PNG để model có thể NHÌN THẤY kết quả,…"
|
|
15
|
+
},
|
|
2
16
|
{
|
|
3
17
|
"v": "1.0.79",
|
|
4
18
|
"d": "15/06/2026",
|
|
@@ -26,19 +40,5 @@
|
|
|
26
40
|
"type": "Added",
|
|
27
41
|
"en": "validate_page now warns when specials.custom_css sets layout or structural CSS properties (position, top, left, right, bottom, inset, width, height,…",
|
|
28
42
|
"vi": "validate_page nay cảnh báo khi specials.custom_css đặt các thuộc tính CSS layout hoặc cấu trúc (position, top, left, right, bottom, inset, width,…"
|
|
29
|
-
},
|
|
30
|
-
{
|
|
31
|
-
"v": "1.0.75",
|
|
32
|
-
"d": "15/06/2026",
|
|
33
|
-
"type": "Added",
|
|
34
|
-
"en": "New get_icon_svg tool resolves Material Symbols (ms:<name>) and Font Awesome (fa:<name>) icon-font references to real inline SVG markup via the…",
|
|
35
|
-
"vi": "Tool mới get_icon_svg resolve tên icon-font Material Symbols (ms:<name>) và Font Awesome (fa:<name>) thành SVG inline thực sự qua Iconify API công…"
|
|
36
|
-
},
|
|
37
|
-
{
|
|
38
|
-
"v": "1.0.74",
|
|
39
|
-
"d": "13/06/2026",
|
|
40
|
-
"type": "Added",
|
|
41
|
-
"en": "ingest_html and ingest_url now extract Tailwind gradient utilities (bg-gradient-to-*/from-*/via-*/to-*) from the page's class attributes, resolve…",
|
|
42
|
-
"vi": "ingest_html và ingest_url nay trích xuất các utility gradient Tailwind (bg-gradient-to-*/from-*/via-*/to-*) từ các thuộc tính class của trang,…"
|
|
43
43
|
}
|
|
44
44
|
]
|
|
@@ -152,6 +152,7 @@ WORKFLOW (recommended)
|
|
|
152
152
|
4. Assemble { page, popup, settings, options, cartConfigs }.
|
|
153
153
|
5. Call validate_page and fix every error AND every warning — warnings are visible defects (text spilling onto the element below, off-canvas boxes, empty bands at a section's bottom, missing field_name, dead event targets), not advisory polish. Re-validate until the warning list is empty; only a demonstrably false positive may remain (tell the user which and why).
|
|
154
154
|
6. To save: call list_organizations. If the account has EXACTLY ONE organization, create_page will auto-select it — no need to ask. If there are MULTIPLE organizations, show them to the user and ask which to use (highlight is_default as the suggested default); pass the chosen organization_id to create_page. If the user explicitly wants to save without any organization, pass organization_id:"personal". Then create_page (dry_run first, then dry_run:false). Note: create_page itself enforces this — it refuses to guess between multiple orgs and returns the org list asking you to pick.
|
|
155
|
+
7. VISUAL CHECK — do this whenever you built from a reference (clone/adapt) or fidelity matters: after create_page auto-publishes (or after publish_page), actually LOOK at the rendered page and compare it to the reference, then fix what's off. SCREENSHOT IT WITH YOUR OWN CAPABILITY FIRST: if you have a browser/screenshot MCP (e.g. chrome-devtools) or any tool that can capture a URL, screenshot the returned preview_url yourself — it's fresh and unlimited. ONLY if you have no such ability, call render_preview(page_id) (returns a PNG; on a quota/rate-limit it returns ok:false → SKIP the visual check that round, don't fail the build). Put your shot next to the reference and compare section order, colors, spacing, image placement, sizing and text; for EACH mismatch, patch_page the offending element by id, re-publish (publish_page), and screenshot again — loop 2–3 rounds until it matches. The no-domain preview only renders for ~10 min after the last publish, so screenshot promptly and re-publish before re-checking a stale page.
|
|
155
156
|
|
|
156
157
|
EDITING an existing page
|
|
157
158
|
- find_pages / list_pages → let the user pick (or take a page_id from a URL).
|
|
@@ -165,4 +166,5 @@ REFERENCE INPUT (HTML page to clone or adapt)
|
|
|
165
166
|
- ingest_html(html, detail:'full') / ingest_url(url, detail:'full') returns a richer AST for clone-quality rebuilds: CSS custom-property palette (design tokens by name), background_images from stylesheets (hero/CTA bg images invisible to a plain img scan), per-section blocks (repeating card/tile/step structures with title/body/image/cta), li lists, gradients, and images as { src, alt } objects. Use detail:'compact' (default) for a quick layout-only reference.
|
|
166
167
|
- SECTION HEIGHTS: every AST section carries size_hint = { height, basis, css? } — the section's desktop height on the 960px canvas (basis:'css' = an explicit height/min-height found in the source, css holds the raw value e.g. "100vh"; basis:'estimate' = content-volume math). Set each rebuilt section's desktop height FROM its size_hint (±15% to fit your actual element placement) instead of the 800px default, so the page's vertical rhythm tracks the source: a 72px header stays a slim bar, a 1200px pricing band stays tall. Mobile is NOT hinted — redo the height per the mobile text math (stacked content is taller).
|
|
167
168
|
- Map AST section roles to Webcake elements: hero → section (background image/overlay) + text-block H1 + text-block subheading + button; features → group per card (icon rectangle + text-block title + text-block body); stats bar → group with text-block per stat; pricing → group with text-block list + button; footer → section (dark bg) + text-block + links. When the ingested page contains a composite widget (phone/device mockup, chat thread, mini dashboard, browser frame) → rebuild it as ONE html-box, NOT as element soup. The full AST hands you the SOURCE for this: section.widgets = [{ hint, html, css? }] — the widget's cleaned outerHTML plus the stylesheet rules that style it. Build the html-box FROM that html verbatim: inline each css rule into the matching elements' style="" attributes (an html-box has no <style> scope), wrap in a root div with width:100%;height:100%;box-sizing:border-box;overflow:hidden, and size the html-box to the widget's box — do NOT re-imagine the widget's markup from the summary fields.
|
|
168
|
-
- Reference images are the user's assets — for BOTH intents (adapt AND clone), carry every real image URL found in the AST (images, background_images, og_image, canvas src/background) straight into the matching slot; the save auto-hosts it to the Webcake CDN (no upload_images needed, no hotlinking), so never replace it with a search_images stock photo or a placeholder. intent='adapt' rewrites TEXT for the user's brand, not the imagery; search_images only fills slots that have no source image
|
|
169
|
+
- Reference images are the user's assets — for BOTH intents (adapt AND clone), carry every real image URL found in the AST (images, background_images, og_image, canvas src/background) straight into the matching slot; the save auto-hosts it to the Webcake CDN (no upload_images needed, no hotlinking), so never replace it with a search_images stock photo or a placeholder. intent='adapt' rewrites TEXT for the user's brand, not the imagery; search_images only fills slots that have no source image.
|
|
170
|
+
- VERIFY THE CLONE VISUALLY (don't trust the build blind): after saving, do the VISUAL CHECK from BUILDING step 7 — screenshot the rendered preview with your own browser/screenshot capability (or render_preview as fallback) and compare it side-by-side with the reference (screenshot the reference URL too if you only have HTML), then patch_page the elements that don't match and re-check. Eyeballing a reference and rebuilding once typically lands ~60% similar; the look-and-patch loop is what closes the gap.`;
|
|
@@ -41,5 +41,6 @@ MODEL (essentials):
|
|
|
41
41
|
|
|
42
42
|
- BEYOND ELEMENT CAPABILITY: when a reference effect can't be expressed with an element's built-in specials (hover scale/lift/zoom & transitions, gradients, glassmorphism/backdrop-blur, custom shadows, gradient/clipped text, keyframe animations outside the 9 entrance types) DON'T drop it — use the escape hatches so the page is as complete as possible: element specials.custom_css (extra DECLARATIONS in #w-<id>{…}; declarations-only) + specials.custom_class (both need specials.customAdvance:true) for per-element styling, and page settings.extra_css (a full raw stylesheet in <head> — where :hover/@keyframes/media-queries live, target #w-<element id>) + settings.extra_script (raw JS) + settings.bhet/bbet (raw HTML blocks at end of <head>/<body> — for webfont <link>s, <meta>/verification, analytics/pixels, chat widgets, third-party embeds). get_generation_guide has the recipes. SAFETY (custom is raw + global → sloppy custom BREAKS the whole UI): SCOPE every extra_css rule to "#w-<id>" or a specials.custom_class — NEVER a bare tag / "*" / a Webcake-internal class (.section-container/.rectangle-css/.text-block-css/.group-*…); keep custom_css to VISUAL props only (no position/top/left/width/height/display/float); close every bhet/bbet tag; don't restructure "#w-…" elements in extra_script. validate_page flags unscoped selectors, layout props in custom_css, unbalanced braces/tags, missing customAdvance, and CSS/JS in the wrong field — fix every such warning.
|
|
43
43
|
- PREVIEW vs PUBLISH: for review share the EDITOR url (the builder renders the raw source). The editor_url SIGNS THE BROWSER IN automatically (it routes through the builder's /transport with the account token), so it works even when the user isn't logged in — but for the same reason it must go to the PAGE OWNER ONLY, never into anything public. create_page AUTO-PUBLISHES on success (builds the rendered app + publish_html), so a fresh page's preview_url renders right away — but the preview host (preview.localhost:5800 local / staging.webcake.me staging / www.webcake.me prod) only serves it for ~10 MINUTES after the last publish, then shows "Preview page is expired". The EDIT routes (update_page/add_section/patch_page) store source only — after finishing a round of edits, run publish_page({ page_id, dry_run:false }) to rebuild the rendered app (else the preview shows the STALE pre-edit build). ONLY a custom_domain (publish_page({ page_id, custom_domain, dry_run:false })) gives a permanent public URL; without one the page has just the ephemeral preview link — say so and suggest attaching a domain the user already points at Webcake.
|
|
44
|
+
- VISUAL CHECK after saving (do it for every clone/reference build, and whenever the look matters): don't trust the build blind — LOOK at the rendered page and fix what's off. SCREENSHOT IT WITH YOUR OWN CAPABILITY FIRST — if you have a browser/screenshot MCP (e.g. chrome-devtools) or any tool that captures a URL, screenshot the returned preview_url yourself (fresh + unlimited); ONLY if you have none, call render_preview(page_id), which returns a PNG (on a quota/429 it returns ok:false → skip the check that round, never fail the build). Compare your shot to the reference (screenshot the reference URL too when you only have HTML) — section order, colors, spacing, image placement, sizing, text — then patch_page each mismatch by id, re-publish (publish_page), and screenshot again; loop 2–3 rounds. Rebuilding once from a glance lands ~60% similar — the look-and-patch loop is what closes the gap. The no-domain preview only renders ~10 min after the last publish, so shoot promptly and re-publish before re-checking.
|
|
44
45
|
|
|
45
|
-
Start by calling get_generation_guide. Tools: get_generation_guide, list_elements, get_element, new_element, new_page_skeleton, get_page_schema, validate_page, search_images, get_icon_svg, upload_images, ingest_html, ingest_url, list_organizations, create_page, list_pages, find_pages, get_page, update_page, add_section, patch_page, publish_page.`;
|
|
46
|
+
Start by calling get_generation_guide. Tools: get_generation_guide, list_elements, get_element, new_element, new_page_skeleton, get_page_schema, validate_page, search_images, get_icon_svg, upload_images, render_preview, ingest_html, ingest_url, list_organizations, create_page, list_pages, find_pages, get_page, update_page, add_section, patch_page, publish_page.`;
|
package/dist/http.js
CHANGED
|
@@ -21,11 +21,13 @@ import { ICON_SVG, ICON_MIME } from "./branding.js";
|
|
|
21
21
|
import { guideHtml, ogImageSvg, normalizeLang } from "./web-guide.js";
|
|
22
22
|
import { privacyHtml, termsHtml } from "./legal.js";
|
|
23
23
|
import { searchPexels, resolvePexelsKey } from "./persistence/pexels-client.js";
|
|
24
|
+
import { captureWithPlaywright, isAllowedScreenshotUrl } from "./persistence/screenshot-playwright.js";
|
|
24
25
|
import { resolveEnv, ENVIRONMENTS, stripTrailingSlash } from "./persistence/config.js";
|
|
25
26
|
import { buildConnectUrl } from "./auth/login.js";
|
|
26
27
|
import { registerClient, startAuthorize, completeAuthorize, exchangeToken, resolveAccessToken, revokeToken, authServerMetadata, protectedResourceMetadata, } from "./auth/oauth-server.js";
|
|
27
28
|
const MCP_PATH = "/mcp";
|
|
28
29
|
const IMAGES_PATH = "/api/images/search";
|
|
30
|
+
const RENDER_SCREENSHOT_PATH = "/api/render/screenshot";
|
|
29
31
|
// OAuth 2.1 endpoints (the embedded thin Authorization Server — see auth/oauth-server.ts).
|
|
30
32
|
const WELL_KNOWN_PR = "/.well-known/oauth-protected-resource";
|
|
31
33
|
const WELL_KNOWN_AS = "/.well-known/oauth-authorization-server";
|
|
@@ -322,6 +324,40 @@ async function handleImageSearch(req, res) {
|
|
|
322
324
|
const result = await searchPexels(key, params);
|
|
323
325
|
return sendImgJson(result.ok ? 200 : result.status || 502, result);
|
|
324
326
|
}
|
|
327
|
+
/**
|
|
328
|
+
* Self-hosted screenshot route: GET /api/render/screenshot?url=…&full_page=…&width=…
|
|
329
|
+
* Renders the target URL with Playwright (this VPS's own headless Chromium) and
|
|
330
|
+
* returns the PNG bytes — the UNLIMITED engine `render_preview` falls over to when
|
|
331
|
+
* Microlink's free quota is hit (point RENDER_SCREENSHOT_BASE at this host). Returns
|
|
332
|
+
* 503 when Playwright isn't installed here. Blocks private/loopback targets (SSRF).
|
|
333
|
+
*/
|
|
334
|
+
async function handleRenderScreenshot(req, res) {
|
|
335
|
+
const cors = { "access-control-allow-origin": "*", "access-control-allow-headers": "*" };
|
|
336
|
+
if (req.method === "OPTIONS") {
|
|
337
|
+
res.writeHead(204, cors);
|
|
338
|
+
return res.end();
|
|
339
|
+
}
|
|
340
|
+
const sendErr = (status, body) => {
|
|
341
|
+
res.writeHead(status, { "content-type": "application/json", ...cors });
|
|
342
|
+
res.end(JSON.stringify(body));
|
|
343
|
+
};
|
|
344
|
+
const sp = new URL(req.url ?? "/", "http://x").searchParams;
|
|
345
|
+
const target = sp.get("url")?.trim();
|
|
346
|
+
if (!target)
|
|
347
|
+
return sendErr(400, { ok: false, error: "Pass ?url=<public http(s) URL>." });
|
|
348
|
+
const allow = isAllowedScreenshotUrl(target);
|
|
349
|
+
if (!allow.ok)
|
|
350
|
+
return sendErr(400, { ok: false, error: allow.error });
|
|
351
|
+
const fullPage = sp.get("full_page") !== "false";
|
|
352
|
+
const width = sp.get("width") ? Number(sp.get("width")) : undefined;
|
|
353
|
+
const r = await captureWithPlaywright(target, { fullPage, width });
|
|
354
|
+
if (!r.ok) {
|
|
355
|
+
// 503 when the engine is absent (caller should fall back / skip), 502 otherwise.
|
|
356
|
+
return sendErr(r.reason === "not_installed" ? 503 : 502, { ok: false, error: r.error });
|
|
357
|
+
}
|
|
358
|
+
res.writeHead(200, { "content-type": r.mimeType, "cache-control": "no-store", ...cors });
|
|
359
|
+
return res.end(r.data);
|
|
360
|
+
}
|
|
325
361
|
export async function startHttpServer(port) {
|
|
326
362
|
// mcp-session-id -> live transport (each bound to its own McpServer instance).
|
|
327
363
|
const transports = new Map();
|
|
@@ -382,6 +418,9 @@ export async function startHttpServer(port) {
|
|
|
382
418
|
// Shared image proxy (for `npx` clients without their own Pexels key).
|
|
383
419
|
if (path === IMAGES_PATH)
|
|
384
420
|
return handleImageSearch(req, res);
|
|
421
|
+
// Self-hosted screenshot engine (Playwright) — the unlimited fallback for render_preview.
|
|
422
|
+
if (path === RENDER_SCREENSHOT_PATH)
|
|
423
|
+
return handleRenderScreenshot(req, res);
|
|
385
424
|
// OAuth 2.1 endpoints (always served; see handleOAuth). Returns true if handled.
|
|
386
425
|
if (await handleOAuth(req, res, path))
|
|
387
426
|
return;
|
package/dist/legal.js
CHANGED
|
@@ -48,6 +48,10 @@ what categories of data the connector handles, why, who receives it, and how lon
|
|
|
48
48
|
<li><strong>Images.</strong> External image URLs in a page are re-hosted to the Webcake CDN on save. Optional
|
|
49
49
|
stock-photo search uses the Pexels API and icon lookup uses the Iconify API. <em>Purpose:</em> to supply the
|
|
50
50
|
visuals for your page.</li>
|
|
51
|
+
<li><strong>Page preview screenshots.</strong> When you ask the assistant to visually check a page, the page's
|
|
52
|
+
<em>public</em> preview URL is sent to a screenshot service (by default Microlink; optionally a self-hosted
|
|
53
|
+
renderer) which returns a picture of the page. <em>Purpose:</em> so the assistant can see the rendered result and
|
|
54
|
+
compare it to your reference. Only the public URL is sent — never your access token or page-source JSON.</li>
|
|
51
55
|
</ul>
|
|
52
56
|
|
|
53
57
|
<h2>What we store and for how long</h2>
|
|
@@ -70,6 +74,8 @@ what categories of data the connector handles, why, who receives it, and how lon
|
|
|
70
74
|
and serves your pages; governed by Webcake's own terms.</li>
|
|
71
75
|
<li><strong>Pexels</strong> (pexels.com) — stock-photo search, only when you request images.</li>
|
|
72
76
|
<li><strong>Iconify</strong> (iconify.design) — resolves icon names to SVG, only when a page uses icons.</li>
|
|
77
|
+
<li><strong>Microlink</strong> (api.microlink.io) — renders a page's public preview URL to a screenshot, only when
|
|
78
|
+
you request a visual check. A self-hosted renderer can be configured instead; only the public URL is sent.</li>
|
|
73
79
|
</ul>
|
|
74
80
|
|
|
75
81
|
<h2>Data we do NOT collect</h2>
|
package/dist/mcp/response.js
CHANGED
|
@@ -6,6 +6,18 @@ export function text(value) {
|
|
|
6
6
|
const body = typeof value === "string" ? value : JSON.stringify(value, null, 2);
|
|
7
7
|
return { content: [{ type: "text", text: body }] };
|
|
8
8
|
}
|
|
9
|
+
/**
|
|
10
|
+
* Wrap a base64 image as an MCP image content block, optionally followed by a
|
|
11
|
+
* text block (notes/metadata the model should read). Used by render_preview so a
|
|
12
|
+
* multimodal model can SEE the rendered page and compare it to the reference.
|
|
13
|
+
*/
|
|
14
|
+
export function image(dataBase64, mimeType = "image/png", note) {
|
|
15
|
+
const content = [{ type: "image", data: dataBase64, mimeType }];
|
|
16
|
+
if (note !== undefined) {
|
|
17
|
+
content.push({ type: "text", text: typeof note === "string" ? note : JSON.stringify(note, null, 2) });
|
|
18
|
+
}
|
|
19
|
+
return { content };
|
|
20
|
+
}
|
|
9
21
|
/**
|
|
10
22
|
* Directive shipped alongside every non-empty validation-warnings list.
|
|
11
23
|
* Warnings are design defects the customer WILL see (text overlapping the
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Thin screenshot client for the clone-fidelity loop: render a public URL (a
|
|
3
|
+
* page's /preview/<id>, or a reference page being cloned) to a PNG the model can
|
|
4
|
+
* SEE and compare against the reference.
|
|
5
|
+
*
|
|
6
|
+
* CAPABILITY LADDER (decided by the caller, NOT this file):
|
|
7
|
+
* 1. If the AGENT can screenshot itself (a shell + headless browser, e.g. Claude
|
|
8
|
+
* Code local), it should do that — fresh every time, no quota. The MCP
|
|
9
|
+
* `render_preview` tool is the FALLBACK for agents WITHOUT that ability
|
|
10
|
+
* (e.g. the claude.ai remote connector).
|
|
11
|
+
* 2. This client's default engine is Microlink ZERO-CONFIG (no key) — but its
|
|
12
|
+
* free tier is rate-limited PER IP (~50/day). That's fine for a local stdio
|
|
13
|
+
* user (their own IP) but a SHARED quota behind the serve-host proxy, so for
|
|
14
|
+
* remote/multi-user the host should point RENDER_SCREENSHOT_BASE at a KEYED
|
|
15
|
+
* engine (a proxy route that holds a ScreenshotOne/ApiFlash/Microlink-Pro key).
|
|
16
|
+
* 3. On quota/429 or any failure the caller SKIPS gracefully (never throws).
|
|
17
|
+
*
|
|
18
|
+
* Mirrors the Pexels client's direct-vs-proxy shape (see pexels-client.ts). The
|
|
19
|
+
* secret (if any) is resolved like the JWT — env or per-request header, never
|
|
20
|
+
* hard-coded (the repo is public). Requires global fetch (Node 18+).
|
|
21
|
+
*/
|
|
22
|
+
/** Microlink's free screenshot endpoint. `embed=screenshot.url` returns the PNG bytes directly. */
|
|
23
|
+
const MICROLINK_ENDPOINT = "https://api.microlink.io/";
|
|
24
|
+
/** Path of the keyed proxy's screenshot route (served by a host that holds a paid key). */
|
|
25
|
+
export const SCREENSHOT_PROXY_PATH = "/api/render/screenshot";
|
|
26
|
+
/** Fetch timeout — screenshots can be slow (full-page + webfont settle). */
|
|
27
|
+
const SCREENSHOT_TIMEOUT_MS = (() => {
|
|
28
|
+
const v = parseInt(process.env.WEBCAKE_HTTP_TIMEOUT_MS ?? "", 10);
|
|
29
|
+
return Number.isFinite(v) && v > 0 ? v : 60_000;
|
|
30
|
+
})();
|
|
31
|
+
/** Resolve the keyed-proxy base (host route that holds a paid screenshot key), if configured. */
|
|
32
|
+
export function resolveScreenshotProxyBase(override) {
|
|
33
|
+
const base = override ?? process.env.RENDER_SCREENSHOT_BASE;
|
|
34
|
+
return base && base.trim() !== "" ? base.replace(/\/+$/, "") : undefined;
|
|
35
|
+
}
|
|
36
|
+
/** Pull the per-request proxy base header (remote mode), if present. */
|
|
37
|
+
export function screenshotProxyBaseFromHeaders(headers) {
|
|
38
|
+
const v = headers?.["x-render-screenshot-base"];
|
|
39
|
+
const raw = Array.isArray(v) ? v[0] : v;
|
|
40
|
+
return raw && raw.trim() !== "" ? raw.trim() : undefined;
|
|
41
|
+
}
|
|
42
|
+
/** Which engine to try FIRST: "microlink" (free, default) or "proxy" (the keyed/self-hosted route). */
|
|
43
|
+
export function resolveScreenshotPrimary(override) {
|
|
44
|
+
const v = (override ?? process.env.RENDER_SCREENSHOT_PRIMARY ?? "").toLowerCase();
|
|
45
|
+
return v === "proxy" ? "proxy" : "microlink";
|
|
46
|
+
}
|
|
47
|
+
/** Optional Microlink Pro key (raises the free per-IP quota) — env or x-microlink-key header. */
|
|
48
|
+
export function resolveMicrolinkKey(override) {
|
|
49
|
+
const key = override ?? process.env.MICROLINK_API_KEY;
|
|
50
|
+
return key && key.trim() !== "" ? key.trim() : undefined;
|
|
51
|
+
}
|
|
52
|
+
export function microlinkKeyFromHeaders(headers) {
|
|
53
|
+
const v = headers?.["x-microlink-key"];
|
|
54
|
+
const raw = Array.isArray(v) ? v[0] : v;
|
|
55
|
+
return raw && raw.trim() !== "" ? raw.trim() : undefined;
|
|
56
|
+
}
|
|
57
|
+
/**
|
|
58
|
+
* Build the Microlink request URL. We force a cache bypass (`force=true` AND a
|
|
59
|
+
* nonce on the target URL) because the preview URL is stable across patch rounds
|
|
60
|
+
* — without it Microlink serves a STALE shot of the pre-patch page. Pure so the
|
|
61
|
+
* smoke test can assert the query shape without a network call.
|
|
62
|
+
*/
|
|
63
|
+
export function buildMicrolinkUrl(targetUrl, opts = {}, nonce = 0) {
|
|
64
|
+
// Nonce-bust the TARGET url itself (defends against any URL-keyed cache even if
|
|
65
|
+
// `force` is restricted on the free tier).
|
|
66
|
+
const bustTarget = targetUrl + (targetUrl.includes("?") ? "&" : "?") + "_=" + nonce;
|
|
67
|
+
const q = new URLSearchParams();
|
|
68
|
+
q.set("url", bustTarget);
|
|
69
|
+
q.set("screenshot", "true");
|
|
70
|
+
q.set("fullPage", String(opts.fullPage !== false));
|
|
71
|
+
q.set("meta", "false");
|
|
72
|
+
q.set("force", "true");
|
|
73
|
+
q.set("embed", "screenshot.url"); // respond with the PNG bytes, not JSON
|
|
74
|
+
if (opts.width && Number.isFinite(opts.width))
|
|
75
|
+
q.set("viewport.width", String(Math.round(opts.width)));
|
|
76
|
+
return `${MICROLINK_ENDPOINT}?${q.toString()}`;
|
|
77
|
+
}
|
|
78
|
+
/** Read a fetch Response into a ScreenshotResult, classifying quota/errors. */
|
|
79
|
+
async function readImageResponse(res, via) {
|
|
80
|
+
const ct = (res.headers.get("content-type") ?? "").toLowerCase();
|
|
81
|
+
if (res.ok && ct.startsWith("image/")) {
|
|
82
|
+
const buf = Buffer.from(await res.arrayBuffer());
|
|
83
|
+
return { ok: true, status: res.status, dataBase64: buf.toString("base64"), mimeType: ct.split(";")[0].trim(), bytes: buf.length, via };
|
|
84
|
+
}
|
|
85
|
+
// Non-image → an error (JSON or text). 429 == rate limit / quota.
|
|
86
|
+
const body = (await res.text()).slice(0, 300);
|
|
87
|
+
const quota = res.status === 429;
|
|
88
|
+
return {
|
|
89
|
+
ok: false,
|
|
90
|
+
status: res.status,
|
|
91
|
+
via,
|
|
92
|
+
quota_exhausted: quota,
|
|
93
|
+
error: quota
|
|
94
|
+
? `screenshot quota/rate-limit hit (HTTP 429) via ${via} — skip the visual check this round or configure a keyed engine`
|
|
95
|
+
: `${via} returned HTTP ${res.status}${body ? `: ${body}` : ""}`,
|
|
96
|
+
};
|
|
97
|
+
}
|
|
98
|
+
/** Capture via Microlink directly (zero-config; per-IP free quota). */
|
|
99
|
+
export async function captureViaMicrolink(targetUrl, opts, key, nonce = 0) {
|
|
100
|
+
const url = buildMicrolinkUrl(targetUrl, opts, nonce);
|
|
101
|
+
try {
|
|
102
|
+
const res = await fetch(url, {
|
|
103
|
+
method: "GET",
|
|
104
|
+
headers: key ? { "x-api-key": key } : undefined,
|
|
105
|
+
signal: AbortSignal.timeout(SCREENSHOT_TIMEOUT_MS),
|
|
106
|
+
});
|
|
107
|
+
return await readImageResponse(res, "microlink");
|
|
108
|
+
}
|
|
109
|
+
catch (e) {
|
|
110
|
+
if (e?.name === "TimeoutError" || e?.name === "AbortError") {
|
|
111
|
+
return { ok: false, status: 0, via: "microlink", error: `screenshot timed out after ${SCREENSHOT_TIMEOUT_MS}ms` };
|
|
112
|
+
}
|
|
113
|
+
return { ok: false, status: 0, via: "microlink", error: `network error calling Microlink: ${e?.message ?? e}` };
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
/** Capture via a keyed proxy route the host operates: GET <base>/api/render/screenshot?url=…&full_page=…&width=… */
|
|
117
|
+
export async function captureViaProxy(base, targetUrl, opts, nonce = 0) {
|
|
118
|
+
const q = new URLSearchParams();
|
|
119
|
+
q.set("url", targetUrl + (targetUrl.includes("?") ? "&" : "?") + "_=" + nonce);
|
|
120
|
+
q.set("full_page", String(opts.fullPage !== false));
|
|
121
|
+
if (opts.width && Number.isFinite(opts.width))
|
|
122
|
+
q.set("width", String(Math.round(opts.width)));
|
|
123
|
+
const url = `${base}${SCREENSHOT_PROXY_PATH}?${q.toString()}`;
|
|
124
|
+
try {
|
|
125
|
+
const res = await fetch(url, { method: "GET", signal: AbortSignal.timeout(SCREENSHOT_TIMEOUT_MS) });
|
|
126
|
+
return await readImageResponse(res, "proxy");
|
|
127
|
+
}
|
|
128
|
+
catch (e) {
|
|
129
|
+
if (e?.name === "TimeoutError" || e?.name === "AbortError") {
|
|
130
|
+
return { ok: false, status: 0, via: "proxy", error: `screenshot timed out after ${SCREENSHOT_TIMEOUT_MS}ms calling proxy ${base}` };
|
|
131
|
+
}
|
|
132
|
+
return { ok: false, status: 0, via: "proxy", error: `network error calling screenshot proxy ${base}: ${e?.message ?? e}` };
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
/**
|
|
136
|
+
* Capture a screenshot with AUTOMATIC FALLOVER between the two engines:
|
|
137
|
+
* - Microlink direct (free, zero-config) and
|
|
138
|
+
* - the proxy route (a keyed API or the VPS's self-hosted Playwright engine).
|
|
139
|
+
* Tries `primary` first; if it fails — especially Microlink hitting its per-IP
|
|
140
|
+
* quota (HTTP 429) — it retries the other engine. So a free quota is used up
|
|
141
|
+
* first, then traffic auto-switches to the unlimited self-hosted route. With no
|
|
142
|
+
* proxyBase configured it's Microlink only. Never throws; on total failure
|
|
143
|
+
* returns the most informative error (quota_exhausted set when that was the cause).
|
|
144
|
+
*/
|
|
145
|
+
export async function captureScreenshot(targetUrl, opts = {}, resolved = {}, nonce = 0) {
|
|
146
|
+
const primary = resolved.primary ?? resolveScreenshotPrimary();
|
|
147
|
+
const micro = () => captureViaMicrolink(targetUrl, opts, resolved.microlinkKey, nonce);
|
|
148
|
+
const proxy = () => (resolved.proxyBase ? captureViaProxy(resolved.proxyBase, targetUrl, opts, nonce) : null);
|
|
149
|
+
// No fallback engine available → single attempt.
|
|
150
|
+
if (!resolved.proxyBase)
|
|
151
|
+
return micro();
|
|
152
|
+
const first = primary === "proxy" ? proxy() : micro();
|
|
153
|
+
const r1 = await first;
|
|
154
|
+
if (r1.ok)
|
|
155
|
+
return r1;
|
|
156
|
+
// Primary failed (quota or error) → fall over to the other engine.
|
|
157
|
+
const r2 = await (primary === "proxy" ? micro() : proxy());
|
|
158
|
+
if (r2.ok)
|
|
159
|
+
return r2;
|
|
160
|
+
// Both failed → prefer the non-quota error (more actionable); else the quota one.
|
|
161
|
+
return r1.quota_exhausted && !r2.quota_exhausted ? r2 : r1;
|
|
162
|
+
}
|
|
@@ -0,0 +1,182 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Self-hosted screenshot engine (Playwright) for the serve-host's
|
|
3
|
+
* GET /api/render/screenshot route — the UNLIMITED fallback the `render_preview`
|
|
4
|
+
* tool falls over to when Microlink's free per-IP quota is exhausted.
|
|
5
|
+
*
|
|
6
|
+
* Playwright is NOT a package dependency (keeps `npx webcake-landing-mcp` light —
|
|
7
|
+
* no Chromium download for stdio users). It is loaded LAZILY at runtime; absent →
|
|
8
|
+
* captureWithPlaywright returns ok:false and the route replies 503. Install it
|
|
9
|
+
* ONLY on the VPS that runs `serve`:
|
|
10
|
+
*
|
|
11
|
+
* npm i playwright && npx playwright install --with-deps chromium
|
|
12
|
+
* # or, to reuse an already-installed Chrome/Chromium without the download:
|
|
13
|
+
* npm i playwright-core and set CHROME_BIN=/path/to/chrome
|
|
14
|
+
*
|
|
15
|
+
* Launch uses CHROME_BIN / PLAYWRIGHT_CHROMIUM_PATH as executablePath when set
|
|
16
|
+
* (so playwright-core can drive a system browser); otherwise Playwright's own
|
|
17
|
+
* bundled Chromium. One browser is launched and reused; a fresh context per shot.
|
|
18
|
+
*/
|
|
19
|
+
let pwModule = null;
|
|
20
|
+
let triedLoad = false;
|
|
21
|
+
let browserPromise = null;
|
|
22
|
+
/** Lazy-load `playwright` (then `playwright-core`); cache the result (incl. the not-installed case). */
|
|
23
|
+
async function loadPlaywright() {
|
|
24
|
+
if (triedLoad)
|
|
25
|
+
return pwModule;
|
|
26
|
+
triedLoad = true;
|
|
27
|
+
// Variable specifier so tsc treats import() as `any` and does NOT require the
|
|
28
|
+
// module to be installed at build time.
|
|
29
|
+
for (const spec of ["playwright", "playwright-core"]) {
|
|
30
|
+
try {
|
|
31
|
+
pwModule = await import(spec);
|
|
32
|
+
break;
|
|
33
|
+
}
|
|
34
|
+
catch {
|
|
35
|
+
/* not installed — try next */
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
return pwModule;
|
|
39
|
+
}
|
|
40
|
+
/** True when a Playwright package is importable on this host. */
|
|
41
|
+
export async function playwrightInstalled() {
|
|
42
|
+
return !!(await loadPlaywright());
|
|
43
|
+
}
|
|
44
|
+
/** Launch (once) and reuse a headless Chromium. Returns null when unavailable. */
|
|
45
|
+
async function getBrowser() {
|
|
46
|
+
const pw = await loadPlaywright();
|
|
47
|
+
if (!pw)
|
|
48
|
+
return null;
|
|
49
|
+
if (!browserPromise) {
|
|
50
|
+
const launchOpts = { headless: true, args: ["--no-sandbox", "--disable-dev-shm-usage"] };
|
|
51
|
+
const exe = process.env.PLAYWRIGHT_CHROMIUM_PATH || process.env.CHROME_BIN;
|
|
52
|
+
if (exe)
|
|
53
|
+
launchOpts.executablePath = exe;
|
|
54
|
+
browserPromise = pw.chromium.launch(launchOpts).catch((e) => {
|
|
55
|
+
browserPromise = null; // allow a later retry
|
|
56
|
+
throw e;
|
|
57
|
+
});
|
|
58
|
+
}
|
|
59
|
+
try {
|
|
60
|
+
return await browserPromise;
|
|
61
|
+
}
|
|
62
|
+
catch {
|
|
63
|
+
return null;
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
/**
|
|
67
|
+
* Output tuning (env-controlled, so the VPS operator picks the size/quality
|
|
68
|
+
* tradeoff). Default JPEG — a full-page landing screenshot is ~5–10× smaller as
|
|
69
|
+
* JPEG than PNG with no loss that matters for a layout/colour comparison, which
|
|
70
|
+
* shrinks the base64 the model receives. `scale` (deviceScaleFactor) renders at a
|
|
71
|
+
* lower pixel density to cut dimensions too; 1 = crisp, 0.5 = quarter the pixels.
|
|
72
|
+
*/
|
|
73
|
+
function outputOpts() {
|
|
74
|
+
const fmt = (process.env.RENDER_SCREENSHOT_FORMAT ?? "jpeg").toLowerCase();
|
|
75
|
+
const type = fmt === "png" ? "png" : "jpeg";
|
|
76
|
+
const q = parseInt(process.env.RENDER_SCREENSHOT_QUALITY ?? "", 10);
|
|
77
|
+
const quality = type === "jpeg" ? (Number.isFinite(q) && q >= 1 && q <= 100 ? q : 72) : undefined;
|
|
78
|
+
const s = parseFloat(process.env.RENDER_SCREENSHOT_SCALE ?? "");
|
|
79
|
+
const scale = Number.isFinite(s) && s > 0 && s <= 2 ? s : 1;
|
|
80
|
+
return { type, quality, scale };
|
|
81
|
+
}
|
|
82
|
+
/** Screenshot a URL with Playwright. Never throws. */
|
|
83
|
+
export async function captureWithPlaywright(url, opts = {}) {
|
|
84
|
+
const pw = await loadPlaywright();
|
|
85
|
+
if (!pw) {
|
|
86
|
+
return {
|
|
87
|
+
ok: false,
|
|
88
|
+
reason: "not_installed",
|
|
89
|
+
error: "playwright is not installed on this host (run: npm i playwright && npx playwright install chromium)",
|
|
90
|
+
};
|
|
91
|
+
}
|
|
92
|
+
// One shot, with a single self-heal retry: if the cached browser died (crash,
|
|
93
|
+
// or a system Chrome that closed), drop it and relaunch once.
|
|
94
|
+
for (let attempt = 0; attempt < 2; attempt++) {
|
|
95
|
+
let browser = await getBrowser();
|
|
96
|
+
if (!browser)
|
|
97
|
+
return { ok: false, error: "failed to launch headless chromium (check CHROME_BIN / playwright install)" };
|
|
98
|
+
// If the cached handle is already disconnected, force a relaunch.
|
|
99
|
+
if (typeof browser.isConnected === "function" && !browser.isConnected()) {
|
|
100
|
+
browserPromise = null;
|
|
101
|
+
browser = await getBrowser();
|
|
102
|
+
if (!browser)
|
|
103
|
+
return { ok: false, error: "failed to relaunch headless chromium" };
|
|
104
|
+
}
|
|
105
|
+
let ctx;
|
|
106
|
+
try {
|
|
107
|
+
const out = outputOpts();
|
|
108
|
+
ctx = await browser.newContext({
|
|
109
|
+
viewport: { width: opts.width && Number.isFinite(opts.width) ? Math.round(opts.width) : 1280, height: 900 },
|
|
110
|
+
deviceScaleFactor: out.scale,
|
|
111
|
+
});
|
|
112
|
+
const page = await ctx.newPage();
|
|
113
|
+
await page.goto(url, { waitUntil: "load", timeout: 30_000 });
|
|
114
|
+
await page.waitForTimeout(1200); // let webfonts/lazy images settle
|
|
115
|
+
const shot = await page.screenshot({
|
|
116
|
+
fullPage: opts.fullPage !== false,
|
|
117
|
+
type: out.type,
|
|
118
|
+
...(out.type === "jpeg" ? { quality: out.quality } : {}),
|
|
119
|
+
});
|
|
120
|
+
return { ok: true, data: Buffer.from(shot), mimeType: out.type === "png" ? "image/png" : "image/jpeg" };
|
|
121
|
+
}
|
|
122
|
+
catch (e) {
|
|
123
|
+
const msg = String(e?.message ?? e);
|
|
124
|
+
// Browser-died errors → reset and retry once; other errors → fail now.
|
|
125
|
+
const dead = /closed|disconnected|crash|Target page, context or browser/i.test(msg);
|
|
126
|
+
if (dead && attempt === 0) {
|
|
127
|
+
browserPromise = null;
|
|
128
|
+
continue;
|
|
129
|
+
}
|
|
130
|
+
return { ok: false, error: `playwright capture failed: ${msg}` };
|
|
131
|
+
}
|
|
132
|
+
finally {
|
|
133
|
+
if (ctx)
|
|
134
|
+
await ctx.close().catch(() => { });
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
return { ok: false, error: "playwright capture failed after retry" };
|
|
138
|
+
}
|
|
139
|
+
// ---------------------------------------------------------------------------
|
|
140
|
+
// SSRF guard — this route fetches an arbitrary `url`, so block private/loopback
|
|
141
|
+
// targets (someone could otherwise screenshot internal services). Pure + exported
|
|
142
|
+
// for smoke coverage. Opt out with RENDER_ALLOW_PRIVATE=1 (e.g. local dev).
|
|
143
|
+
// ---------------------------------------------------------------------------
|
|
144
|
+
/** True when `hostname` resolves to a loopback / link-local / RFC-1918 private host. */
|
|
145
|
+
export function isPrivateHost(hostname) {
|
|
146
|
+
const h = hostname.toLowerCase().replace(/^\[|\]$/g, ""); // strip IPv6 brackets
|
|
147
|
+
if (h === "localhost" || h.endsWith(".localhost") || h === "::1" || h === "0.0.0.0")
|
|
148
|
+
return true;
|
|
149
|
+
if (/^127\./.test(h))
|
|
150
|
+
return true; // loopback
|
|
151
|
+
if (/^10\./.test(h))
|
|
152
|
+
return true; // private A
|
|
153
|
+
if (/^192\.168\./.test(h))
|
|
154
|
+
return true; // private C
|
|
155
|
+
if (/^172\.(1[6-9]|2\d|3[01])\./.test(h))
|
|
156
|
+
return true; // private B
|
|
157
|
+
if (/^169\.254\./.test(h))
|
|
158
|
+
return true; // link-local
|
|
159
|
+
if (/^0\./.test(h))
|
|
160
|
+
return true; // "this" network
|
|
161
|
+
if (h.startsWith("fc") || h.startsWith("fd"))
|
|
162
|
+
return true; // IPv6 ULA
|
|
163
|
+
if (h.startsWith("fe80"))
|
|
164
|
+
return true; // IPv6 link-local
|
|
165
|
+
return false;
|
|
166
|
+
}
|
|
167
|
+
/** Validate a screenshot target URL: http(s) only, no private host (unless RENDER_ALLOW_PRIVATE). */
|
|
168
|
+
export function isAllowedScreenshotUrl(raw) {
|
|
169
|
+
let u;
|
|
170
|
+
try {
|
|
171
|
+
u = new URL(raw);
|
|
172
|
+
}
|
|
173
|
+
catch {
|
|
174
|
+
return { ok: false, error: "invalid url" };
|
|
175
|
+
}
|
|
176
|
+
if (u.protocol !== "http:" && u.protocol !== "https:")
|
|
177
|
+
return { ok: false, error: "only http(s) urls are allowed" };
|
|
178
|
+
if (!process.env.RENDER_ALLOW_PRIVATE && isPrivateHost(u.hostname)) {
|
|
179
|
+
return { ok: false, error: "private/loopback hosts are blocked (set RENDER_ALLOW_PRIVATE=1 to allow)" };
|
|
180
|
+
}
|
|
181
|
+
return { ok: true };
|
|
182
|
+
}
|
package/dist/smoke.js
CHANGED
|
@@ -15,6 +15,8 @@ import { normalizePhoto, resolvePexelsKey, pexelsKeyFromHeaders, resolvePexelsPr
|
|
|
15
15
|
import { putDraft, getDraft, updateDraft, deleteDraft } from "./persistence/draft-cache.js";
|
|
16
16
|
import { buildConnectUrl, parseCallback } from "./auth/login.js";
|
|
17
17
|
import { isLocalPath, resolveLocalPath, sniffMime, localContentType } from "./tools/media.js";
|
|
18
|
+
import { buildMicrolinkUrl } from "./persistence/screenshot-client.js";
|
|
19
|
+
import { isPrivateHost, isAllowedScreenshotUrl } from "./persistence/screenshot-playwright.js";
|
|
18
20
|
import { iconifyCandidates } from "./persistence/icon-client.js";
|
|
19
21
|
import { collectExternalImageUrls, rewriteImageUrls, isRehostableImageUrl } from "./persistence/rehost.js";
|
|
20
22
|
let failures = 0;
|
|
@@ -1799,5 +1801,40 @@ console.log("== upload_images: localContentType (ext + magic, pure offline) ==")
|
|
|
1799
1801
|
// both unknown → undefined
|
|
1800
1802
|
check("localContentType: unknown magic + unknown ext → undefined", localContentType("xyz", unknownBuf) === undefined);
|
|
1801
1803
|
}
|
|
1804
|
+
// == render_preview: buildMicrolinkUrl (pure offline) ==
|
|
1805
|
+
{
|
|
1806
|
+
console.log("\n== render_preview: buildMicrolinkUrl (pure offline) ==");
|
|
1807
|
+
const u = buildMicrolinkUrl("https://www.webcake.me/preview/abc", { fullPage: true, width: 1280 }, 999);
|
|
1808
|
+
const sp = new URL(u).searchParams;
|
|
1809
|
+
check("buildMicrolinkUrl: hits api.microlink.io", u.startsWith("https://api.microlink.io/?"));
|
|
1810
|
+
check("buildMicrolinkUrl: screenshot=true", sp.get("screenshot") === "true");
|
|
1811
|
+
check("buildMicrolinkUrl: fullPage=true by default", sp.get("fullPage") === "true");
|
|
1812
|
+
check("buildMicrolinkUrl: force=true (cache bypass)", sp.get("force") === "true");
|
|
1813
|
+
check("buildMicrolinkUrl: embed returns PNG bytes", sp.get("embed") === "screenshot.url");
|
|
1814
|
+
check("buildMicrolinkUrl: nonce busts the target url", (sp.get("url") ?? "").includes("_=999"));
|
|
1815
|
+
check("buildMicrolinkUrl: viewport.width passed through", sp.get("viewport.width") === "1280");
|
|
1816
|
+
const noFull = new URL(buildMicrolinkUrl("https://x.test", { fullPage: false }, 1)).searchParams;
|
|
1817
|
+
check("buildMicrolinkUrl: fullPage=false honored", noFull.get("fullPage") === "false");
|
|
1818
|
+
check("buildMicrolinkUrl: width omitted → no viewport.width", noFull.get("viewport.width") === null);
|
|
1819
|
+
}
|
|
1820
|
+
// == /api/render/screenshot: SSRF url guard (pure offline) ==
|
|
1821
|
+
{
|
|
1822
|
+
console.log("\n== render screenshot route: SSRF url guard (pure offline) ==");
|
|
1823
|
+
// private / loopback / link-local hosts are blocked
|
|
1824
|
+
check("isPrivateHost: localhost", isPrivateHost("localhost"));
|
|
1825
|
+
check("isPrivateHost: 127.0.0.1", isPrivateHost("127.0.0.1"));
|
|
1826
|
+
check("isPrivateHost: 10.x", isPrivateHost("10.1.2.3"));
|
|
1827
|
+
check("isPrivateHost: 192.168.x", isPrivateHost("192.168.0.5"));
|
|
1828
|
+
check("isPrivateHost: 172.16-31.x", isPrivateHost("172.20.0.1") && isPrivateHost("172.16.0.1") && isPrivateHost("172.31.255.255"));
|
|
1829
|
+
check("isPrivateHost: 172.15/172.32 are PUBLIC", !isPrivateHost("172.15.0.1") && !isPrivateHost("172.32.0.1"));
|
|
1830
|
+
check("isPrivateHost: 169.254 link-local", isPrivateHost("169.254.1.1"));
|
|
1831
|
+
check("isPrivateHost: IPv6 ::1 / ULA", isPrivateHost("::1") && isPrivateHost("fd00::1"));
|
|
1832
|
+
check("isPrivateHost: public host", !isPrivateHost("www.webcake.me") && !isPrivateHost("8.8.8.8"));
|
|
1833
|
+
// url validation (RENDER_ALLOW_PRIVATE not set in smoke)
|
|
1834
|
+
check("isAllowedScreenshotUrl: public https ok", isAllowedScreenshotUrl("https://www.webcake.me/preview/abc").ok);
|
|
1835
|
+
check("isAllowedScreenshotUrl: ftp rejected", !isAllowedScreenshotUrl("ftp://x.test/a").ok);
|
|
1836
|
+
check("isAllowedScreenshotUrl: localhost rejected", !isAllowedScreenshotUrl("http://localhost:5800/preview/1").ok);
|
|
1837
|
+
check("isAllowedScreenshotUrl: garbage rejected", !isAllowedScreenshotUrl("not a url").ok);
|
|
1838
|
+
}
|
|
1802
1839
|
console.log(`\n${failures === 0 ? "ALL GOOD" : failures + " FAILURE(S)"}`);
|
|
1803
1840
|
process.exit(failures === 0 ? 0 : 1);
|
package/dist/tools/media.js
CHANGED
|
@@ -26,8 +26,9 @@ import { z } from "zod";
|
|
|
26
26
|
import { promises as fs } from "node:fs";
|
|
27
27
|
import { homedir } from "node:os";
|
|
28
28
|
import { fileURLToPath } from "node:url";
|
|
29
|
-
import { text } from "../mcp/response.js";
|
|
29
|
+
import { text, image } from "../mcp/response.js";
|
|
30
30
|
import { searchPexels, searchImagesViaProxy, resolvePexelsKey, resolvePexelsProxyBase, pexelsKeyFromHeaders, } from "../persistence/pexels-client.js";
|
|
31
|
+
import { captureScreenshot, resolveScreenshotProxyBase, screenshotProxyBaseFromHeaders, resolveMicrolinkKey, microlinkKeyFromHeaders, } from "../persistence/screenshot-client.js";
|
|
31
32
|
import { uploadImageMultipart } from "../persistence/webcake-client.js";
|
|
32
33
|
import { resolveIconSvg } from "../persistence/icon-client.js";
|
|
33
34
|
import { configFromHeaders, ENVIRONMENTS, stripTrailingSlash } from "../persistence/config.js";
|
|
@@ -47,6 +48,21 @@ function resolveApiBase(headers) {
|
|
|
47
48
|
// Default to prod.
|
|
48
49
|
return ENVIRONMENTS.prod.apiBase;
|
|
49
50
|
}
|
|
51
|
+
/** Resolve the public preview base (for building /preview/<id> URLs) from headers → env → WEBCAKE_ENV preset → prod default. No JWT needed. */
|
|
52
|
+
function resolvePreviewBase(headers) {
|
|
53
|
+
const overrides = configFromHeaders(headers);
|
|
54
|
+
if (overrides.previewBase)
|
|
55
|
+
return stripTrailingSlash(overrides.previewBase);
|
|
56
|
+
if (process.env.WEBCAKE_PREVIEW_BASE)
|
|
57
|
+
return stripTrailingSlash(process.env.WEBCAKE_PREVIEW_BASE);
|
|
58
|
+
const envName = overrides.env ?? process.env.WEBCAKE_ENV;
|
|
59
|
+
if (envName && envName in ENVIRONMENTS) {
|
|
60
|
+
const p = ENVIRONMENTS[envName].previewBase;
|
|
61
|
+
if (p)
|
|
62
|
+
return stripTrailingSlash(p);
|
|
63
|
+
}
|
|
64
|
+
return ENVIRONMENTS.prod.previewBase;
|
|
65
|
+
}
|
|
50
66
|
const UPLOAD_FETCH_TIMEOUT_MS = 60_000; // 60 s — large bodies can be slow to transfer
|
|
51
67
|
const UPLOAD_MAX_BYTES = 200_000_000; // 200 MB — mirrors the backend multipart Plug.Parsers limit
|
|
52
68
|
/** Map a content_type to its canonical file extension. */
|
|
@@ -252,6 +268,41 @@ export function registerMediaTools(server, allowLocalFiles = true) {
|
|
|
252
268
|
usage: "For each icons[<ref>].svg, make a rectangle in that card's icon slot: copy the svg into BOTH responsive.desktop.config.svgMask AND responsive.mobile.config.svgMask, set styles.background to the icon color, and keep the box SQUARE (width === height) — without styles.background the masked icon is invisible, and a non-square box stretches it. For any ok:false ref, fall back to an emoji inline or skip — never leave a feature card iconless.",
|
|
253
269
|
});
|
|
254
270
|
});
|
|
271
|
+
// 13c) Render a page/URL to a screenshot the model can SEE --------------------
|
|
272
|
+
server.tool("render_preview", "Renders a PUBLIC URL to a PNG and returns it as an image so the model can SEE the result and compare it visually to the reference — the fidelity-check step of the clone loop (build → see → patch_page → re-check). Pass `page_id` to shoot a created page's preview (/preview/<id>) or `url` for any public page (e.g. the reference you're cloning). full_page defaults to true (whole scrollable page). AGENT-FIRST: if YOU already have a screenshot/browser capability (a shell + headless browser, or a screenshot tool), screenshot the preview URL YOURSELF instead — it's fresh and unlimited; use this tool only when you cannot. ENGINE: zero-config via Microlink's free tier (rate-limited ~50/day PER IP, so heavy looping can hit HTTP 429 — then this returns ok:false and you should SKIP the visual check that round, not fail); a host can set RENDER_SCREENSHOT_BASE (or the x-render-screenshot-base header) to a keyed proxy, or MICROLINK_API_KEY / x-microlink-key for a higher quota. NOTE: a no-domain preview only renders for ~10 minutes after the last publish — call this promptly after create_page/publish_page, and re-publish before re-checking a stale page.", {
|
|
273
|
+
page_id: z.string().optional().describe("A created page's id — screenshots its /preview/<id> URL (built from the preview base). Provide page_id OR url."),
|
|
274
|
+
url: z.string().optional().describe("Any public http(s) URL to screenshot (e.g. the reference page being cloned). Wins over page_id."),
|
|
275
|
+
full_page: z.boolean().optional().describe("Capture the whole scrollable page (default true) vs just the viewport."),
|
|
276
|
+
width: z.number().int().min(320).max(2560).optional().describe("Viewport width in px (default 1280; use ~960/1200 to match the page canvas, ~420 for mobile)."),
|
|
277
|
+
}, { title: "Render Preview Screenshot", readOnlyHint: true, openWorldHint: true }, async ({ page_id, url, full_page, width }, extra) => {
|
|
278
|
+
const headers = extra?.requestInfo?.headers;
|
|
279
|
+
let target = url?.trim();
|
|
280
|
+
if (!target && page_id) {
|
|
281
|
+
target = `${resolvePreviewBase(headers)}/preview/${encodeURIComponent(page_id)}`;
|
|
282
|
+
}
|
|
283
|
+
if (!target) {
|
|
284
|
+
return text({ ok: false, error: "Pass `page_id` (to shoot its /preview/<id>) or `url`." });
|
|
285
|
+
}
|
|
286
|
+
const resolved = {
|
|
287
|
+
proxyBase: resolveScreenshotProxyBase(screenshotProxyBaseFromHeaders(headers)),
|
|
288
|
+
microlinkKey: resolveMicrolinkKey(microlinkKeyFromHeaders(headers)),
|
|
289
|
+
};
|
|
290
|
+
const r = await captureScreenshot(target, { fullPage: full_page !== false, width }, resolved, Date.now());
|
|
291
|
+
if (!r.ok) {
|
|
292
|
+
return text({
|
|
293
|
+
ok: false,
|
|
294
|
+
url: target,
|
|
295
|
+
status: r.status,
|
|
296
|
+
via: r.via,
|
|
297
|
+
quota_exhausted: r.quota_exhausted ?? false,
|
|
298
|
+
error: r.error,
|
|
299
|
+
hint: r.quota_exhausted
|
|
300
|
+
? "Screenshot quota/rate-limit reached — SKIP the visual check this round (do not block the build), or set a keyed engine (RENDER_SCREENSHOT_BASE / MICROLINK_API_KEY) and retry."
|
|
301
|
+
: "Couldn't capture the screenshot. If you have your own browser/screenshot ability, shoot the URL yourself; otherwise skip the visual check. Note the /preview/<id> link expires ~10 min after the last publish — re-publish if stale.",
|
|
302
|
+
});
|
|
303
|
+
}
|
|
304
|
+
return image(r.dataBase64, r.mimeType ?? "image/png", `Rendered ${target} (${r.bytes} bytes, via ${r.via}). Compare this to the reference: check section order, colors, spacing, image placement, and text. For each mismatch, patch_page the offending element by id, re-publish, then render_preview again until it matches.`);
|
|
305
|
+
});
|
|
255
306
|
// 14) Upload images to Webcake -----------------------------------------------
|
|
256
307
|
server.tool("upload_images", "Converts external image URLs (typically collected from ingest_html/ingest_url results), data: URIs, or LOCAL FILE PATHS from the user's computer into Webcake-hosted URLs (statics.pancake.vn) by reading/downloading each image and re-uploading it to the Webcake backend via multipart upload (200 MB backend limit). Use this whenever the page is built from a reference HTML/URL (BOTH intents — adapt AND clone), the user supplies their own image URLs, OR the user provides local image files from their machine — pass the path directly in `urls`; NEVER upload a user's local file to a third-party host (catbox, imgur, transfer.sh…) to obtain a URL first. The returned URLs go directly into specials.src — same as search_images results. Processes up to 20 entries per call in parallel, with a 200 MB per-image cap. No Webcake credentials required (the upload endpoint is public). UPLOADS BY DEFAULT (dry_run defaults to FALSE — unlike the page-persistence tools, this touches no account data, so the default is the real upload): the call downloads/reads each entry, uploads it, and returns the images map (original URL → hosted URL); WAIT for that map before assembling the page and never fall back to a placeholder for a slot whose upload succeeded. Pass dry_run:true only to preview what would be processed without any network/filesystem activity. Use search_images instead when you need stock photos. Local file paths are only permitted when the MCP server runs locally (stdio mode); on the remote HTTP transport they are rejected per-entry.", {
|
|
257
308
|
urls: z
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "webcake-landing-mcp",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.81",
|
|
4
4
|
"description": "MCP server exposing Webcake landing-page element schemas + AI usage hints, and persisting LLM-generated page sources to a Webcake backend.",
|
|
5
5
|
"mcpName": "io.github.vuluu2k/webcake-landing-mcp",
|
|
6
6
|
"type": "module",
|
|
@@ -31,6 +31,7 @@
|
|
|
31
31
|
"dev:guide": "node scripts/preview-guide.mjs",
|
|
32
32
|
"smoke": "node dist/smoke.js",
|
|
33
33
|
"inspect": "npm run build && npx -y @modelcontextprotocol/inspector node dist/index.js",
|
|
34
|
+
"pack:mcpb": "node scripts/pack-mcpb.mjs",
|
|
34
35
|
"prepare": "npm run build",
|
|
35
36
|
"prepublishOnly": "npm run build && npm run smoke",
|
|
36
37
|
"release": "node scripts/release.js",
|