@ricsam/r5d-browser 0.0.113 → 0.0.115
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -0
- package/dist/cjs/browser-input.cjs +82 -0
- package/dist/cjs/browser-runtime.cjs +55 -5
- package/dist/cjs/main.cjs +3 -1
- package/dist/cjs/package.json +1 -1
- package/dist/cjs/password-safe-snapshot.cjs +150 -0
- package/dist/mjs/browser-input.mjs +56 -0
- package/dist/mjs/browser-runtime.mjs +55 -5
- package/dist/mjs/main.mjs +3 -1
- package/dist/mjs/package.json +1 -1
- package/dist/mjs/password-safe-snapshot.mjs +125 -0
- package/dist/types/browser-input.d.ts +91 -0
- package/dist/types/browser-runtime.d.ts +3 -1
- package/dist/types/password-safe-snapshot.d.ts +15 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -22,10 +22,16 @@ r5d-browser start --chrome-path "/Applications/Google Chrome.app/Contents/MacOS/
|
|
|
22
22
|
|
|
23
23
|
The profile is stored under `~/.r5d/browser/chrome-profile`. The previous Chrome for Testing profile at `~/.r5d/browser/profile` is preserved but is not migrated automatically, so sign in again the first time you use the new profile. Browser binaries previously installed under `~/.r5d/browser/browsers` are also left untouched, but new versions of `r5d-browser` do not use them. Completed downloads are retained under `~/.r5d/browser/downloads`; agents can list them and copy a selected file into the active session's artifacts. That artifact is then synchronized to every connected worker. Browser screenshots are session artifacts in r5d.dev and are not retained by the browser process.
|
|
24
24
|
|
|
25
|
+
Password input values are replaced with `[redacted]` in `browser_read_page` snapshots, including values used in accessibility labels and iframe content. Reading a snapshot preserves the actual form values.
|
|
26
|
+
|
|
25
27
|
Agents read pages through `browser_read_page`, an accessibility outline with stable element refs (`[ref=e12]`) on interactable elements and `[box=x,y,w,h]` viewport coordinates on everything visible. Refs feed the one-call interaction tools: `browser_mouse_click` clicks a ref (scrolled into view, with actionability checks) or an x/y point, `browser_form_input` sets values on text inputs, selects, checkboxes, and contenteditable elements with trusted events that controlled (React-style) components honor, and `browser_scroll` scrolls an element into view or dispatches trusted wheel input. Same-origin iframe content appears inline with frame-prefixed refs such as `f1e3`; cross-origin frame content may be omitted. Refs come from the most recent snapshot of a tab, so agents re-read the page after navigations or large re-renders. `browser_wait_for` waits for a selector, visible text, a URL glob, or a load state instead of sleeping, and `browser_keyboard_type` has a `keys` mode that presses each character as a real key event (up to 500 characters per call) for type-ahead widgets — characters without key codes, such as emoji, still fall back to text insertion. Screenshots capture the viewport by default; `fullPage: true` captures the scrollable page up to 4000 CSS px of height.
|
|
26
28
|
|
|
27
29
|
Agents running JavaScript in a page can save captured text straight into the active session's artifacts by calling `r5dCreateArtifact(filename, content)`, which takes two strings and must be called before the agent's code returns. Each captured file is written to the session artifacts in r5d.dev, synchronized to every connected worker, and reported back under `$R5D_ROOT/artifacts/$R5D_SESSION_ID`; a single call may create at most 32 files totalling 4 MiB. The function is passed into the agent's code rather than assigned to `window`, so pages can neither call it nor use it to detect automation.
|
|
28
30
|
|
|
31
|
+
For responsive testing that needs an existing login (for example, an app using Google login), agents can call `browser_set_viewport` with `{ tabId, width: 390, height: 844 }`. Both dimensions must be integers from 1 to 4096 CSS pixels. The override is scoped to that tab, even in a shared window: it does not resize the OS window or affect other tabs. It preserves the profile, user agent, touch support, and device pixel ratio; this is layout sizing, not mobile-device emulation. It adds no Chrome launch flags or webdriver spoofing; `navigator.webdriver` remains natively `false`.
|
|
32
|
+
|
|
33
|
+
The viewport survives navigation, reloads, and r5d.dev control-connection reconnects until the agent calls `browser_set_viewport` with `{ tabId, reset: true }`, closes the tab, or exits the browser. Reset restores sizing from the current OS window; omit width/height when resetting. The tool returns the actual width and height. Re-read the page or capture a screenshot after resizing because old coordinates are stale, and reset when testing is finished. To apply a size before the first page load, open `about:blank`, set its viewport, then navigate. An outdated browser client returns an update-and-restart instruction for this tool.
|
|
34
|
+
|
|
29
35
|
As an escape hatch, agents can create loopback-only TCP port forwards from this Mac to the worker bound to a chat when a local flow specifically needs this browser's authenticated state or human interaction, such as testing Google OAuth. Ordinary local application testing should use Playwright directly on the worker without a forward. For example, a forward can map Chrome's `localhost:4323` to `127.0.0.1:3232` on that worker. The relay preserves raw TCP traffic, including HTTP, HTTPS, WebSockets, HMR, and SSE; it never exposes a public listener or permits an arbitrary target host.
|
|
30
36
|
|
|
31
37
|
Port forwards remain active across agent turns and r5d.dev control-connection reconnects. They are owned by the running `r5d-browser` process, so restarting that process ends them. Agents can list and stop forwards, and the user can inspect or stop them from Settings.
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __defProp = Object.defineProperty;
|
|
3
|
+
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
|
|
4
|
+
var __getOwnPropNames = Object.getOwnPropertyNames;
|
|
5
|
+
var __hasOwnProp = Object.prototype.hasOwnProperty;
|
|
6
|
+
var __export = (target, all) => {
|
|
7
|
+
for (var name in all)
|
|
8
|
+
__defProp(target, name, { get: all[name], enumerable: true });
|
|
9
|
+
};
|
|
10
|
+
var __copyProps = (to, from, except, desc) => {
|
|
11
|
+
if (from && typeof from === "object" || typeof from === "function") {
|
|
12
|
+
for (let key of __getOwnPropNames(from))
|
|
13
|
+
if (!__hasOwnProp.call(to, key) && key !== except)
|
|
14
|
+
__defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable });
|
|
15
|
+
}
|
|
16
|
+
return to;
|
|
17
|
+
};
|
|
18
|
+
var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
|
|
19
|
+
var browser_input_exports = {};
|
|
20
|
+
__export(browser_input_exports, {
|
|
21
|
+
MAX_VIEWPORT_DIMENSION: () => MAX_VIEWPORT_DIMENSION,
|
|
22
|
+
normalizeBrowserInput: () => normalizeBrowserInput,
|
|
23
|
+
parseViewportInput: () => parseViewportInput
|
|
24
|
+
});
|
|
25
|
+
module.exports = __toCommonJS(browser_input_exports);
|
|
26
|
+
const browserInputFields = {
|
|
27
|
+
list_tabs: { required: [], optional: [] },
|
|
28
|
+
list_downloads: { required: [], optional: [] },
|
|
29
|
+
get_download: { required: ["downloadId"], optional: ["offset", "maxBytes"] },
|
|
30
|
+
open_tab: { required: [], optional: ["url", "disposition"] },
|
|
31
|
+
close_tab: { required: ["tabId"], optional: [] },
|
|
32
|
+
navigate: { required: ["tabId", "url"], optional: [] },
|
|
33
|
+
set_viewport: { required: ["tabId"], optional: ["width", "height", "reset"] },
|
|
34
|
+
read_page: { required: ["tabId"], optional: ["depth", "maxChars"] },
|
|
35
|
+
screenshot: { required: ["tabId"], optional: ["fullPage"] },
|
|
36
|
+
move_mouse: { required: ["tabId", "x", "y"], optional: [] },
|
|
37
|
+
mouse_click: { required: ["tabId"], optional: ["ref", "x", "y", "button", "clickCount"] },
|
|
38
|
+
scroll: { required: ["tabId"], optional: ["ref", "direction", "amount", "x", "y"] },
|
|
39
|
+
wait_for: { required: ["tabId"], optional: ["selector", "text", "url", "loadState", "state", "timeoutMs"] },
|
|
40
|
+
form_input: { required: ["tabId", "ref"], optional: ["value", "values", "checked"] },
|
|
41
|
+
keyboard: { required: ["tabId", "keys"], optional: ["modifiers"] },
|
|
42
|
+
keyboard_type: { required: ["tabId", "text"], optional: ["mode", "delayMs"] },
|
|
43
|
+
run_js: { required: ["tabId", "code"], optional: [] },
|
|
44
|
+
port_forward: { required: ["workerPort"], optional: ["browserPort"] },
|
|
45
|
+
list_port_forwards: { required: [], optional: [] },
|
|
46
|
+
stop_port_forward: { required: ["forwardId"], optional: [] }
|
|
47
|
+
};
|
|
48
|
+
const MAX_VIEWPORT_DIMENSION = 4096;
|
|
49
|
+
function parseViewportInput(input) {
|
|
50
|
+
if (input.reset !== void 0 && typeof input.reset !== "boolean") throw new Error("set_viewport reset must be a boolean.");
|
|
51
|
+
if (input.reset === true) {
|
|
52
|
+
if (input.width !== void 0 || input.height !== void 0)
|
|
53
|
+
throw new Error("set_viewport accepts width/height or reset: true, not both.");
|
|
54
|
+
return null;
|
|
55
|
+
}
|
|
56
|
+
const { width, height } = input;
|
|
57
|
+
for (const [name, value] of Object.entries({ width, height })) {
|
|
58
|
+
if (typeof value !== "number" || !Number.isInteger(value) || value < 1 || value > MAX_VIEWPORT_DIMENSION) {
|
|
59
|
+
throw new Error(
|
|
60
|
+
`set_viewport requires ${name} to be an integer from 1 to ${MAX_VIEWPORT_DIMENSION} CSS pixels, or reset: true without dimensions.`
|
|
61
|
+
);
|
|
62
|
+
}
|
|
63
|
+
}
|
|
64
|
+
return { width, height };
|
|
65
|
+
}
|
|
66
|
+
function normalizeBrowserInput(action, input) {
|
|
67
|
+
const fields = browserInputFields[action];
|
|
68
|
+
for (const key of fields.required) {
|
|
69
|
+
if (input[key] === null || input[key] === void 0) throw new Error(`${action} requires ${key}.`);
|
|
70
|
+
}
|
|
71
|
+
const normalized = { ...input };
|
|
72
|
+
for (const key of fields.optional) {
|
|
73
|
+
if (normalized[key] === null) delete normalized[key];
|
|
74
|
+
}
|
|
75
|
+
return normalized;
|
|
76
|
+
}
|
|
77
|
+
// Annotate the CommonJS export names for ESM import in node:
|
|
78
|
+
0 && (module.exports = {
|
|
79
|
+
MAX_VIEWPORT_DIMENSION,
|
|
80
|
+
normalizeBrowserInput,
|
|
81
|
+
parseViewportInput
|
|
82
|
+
});
|
|
@@ -33,6 +33,8 @@ __export(browser_runtime_exports, {
|
|
|
33
33
|
module.exports = __toCommonJS(browser_runtime_exports);
|
|
34
34
|
var import_node_fs = __toESM(require("node:fs"), 1);
|
|
35
35
|
var import_node_path = __toESM(require("node:path"), 1);
|
|
36
|
+
var import_browser_input = require("./browser-input.cjs");
|
|
37
|
+
var import_password_safe_snapshot = require("./password-safe-snapshot.cjs");
|
|
36
38
|
const MAX_DOWNLOAD_CHUNK_BYTES = 1024 * 1024;
|
|
37
39
|
const MAX_RUN_JS_RESULT_BYTES = 1024 * 1024;
|
|
38
40
|
const MAX_RUN_JS_ARTIFACT_BYTES = 4 * 1024 * 1024;
|
|
@@ -82,6 +84,7 @@ class BrowserRuntime {
|
|
|
82
84
|
ids = /* @__PURE__ */ new WeakMap();
|
|
83
85
|
pages = /* @__PURE__ */ new Map();
|
|
84
86
|
cursor = /* @__PURE__ */ new Map();
|
|
87
|
+
viewportSessions = /* @__PURE__ */ new Map();
|
|
85
88
|
downloadSaveQueue = Promise.resolve();
|
|
86
89
|
track(page) {
|
|
87
90
|
const existing = this.ids.get(page);
|
|
@@ -98,6 +101,7 @@ class BrowserRuntime {
|
|
|
98
101
|
page.once("close", () => {
|
|
99
102
|
this.pages.delete(id);
|
|
100
103
|
this.cursor.delete(id);
|
|
104
|
+
this.viewportSessions.delete(id);
|
|
101
105
|
});
|
|
102
106
|
return id;
|
|
103
107
|
}
|
|
@@ -233,7 +237,36 @@ class BrowserRuntime {
|
|
|
233
237
|
async drainDownloads() {
|
|
234
238
|
await this.downloadSaveQueue;
|
|
235
239
|
}
|
|
240
|
+
async setViewport(page, tabId, viewport) {
|
|
241
|
+
const existing = this.viewportSessions.get(tabId);
|
|
242
|
+
if (viewport) {
|
|
243
|
+
const cdp = existing ?? await this.context.newCDPSession(page);
|
|
244
|
+
try {
|
|
245
|
+
await cdp.send("Emulation.setDeviceMetricsOverride", { ...viewport, deviceScaleFactor: 0, mobile: false });
|
|
246
|
+
this.viewportSessions.set(tabId, cdp);
|
|
247
|
+
} catch (error) {
|
|
248
|
+
if (!existing) await cdp.detach().catch(() => void 0);
|
|
249
|
+
throw error;
|
|
250
|
+
}
|
|
251
|
+
} else if (existing) {
|
|
252
|
+
await existing.send("Emulation.clearDeviceMetricsOverride");
|
|
253
|
+
this.viewportSessions.delete(tabId);
|
|
254
|
+
await existing.detach().catch(() => void 0);
|
|
255
|
+
}
|
|
256
|
+
this.cursor.delete(tabId);
|
|
257
|
+
if (viewport) {
|
|
258
|
+
const ready = await page.waitForFunction(
|
|
259
|
+
({ width, height }) => window.innerWidth === width && window.innerHeight === height,
|
|
260
|
+
viewport,
|
|
261
|
+
{ timeout: 5e3 }
|
|
262
|
+
);
|
|
263
|
+
await ready.dispose();
|
|
264
|
+
}
|
|
265
|
+
const actual = await page.evaluate(() => ({ width: window.innerWidth, height: window.innerHeight }));
|
|
266
|
+
return { tabId, ...actual, reset: viewport === null };
|
|
267
|
+
}
|
|
236
268
|
async execute(action, input) {
|
|
269
|
+
input = (0, import_browser_input.normalizeBrowserInput)(action, input);
|
|
237
270
|
if (action === "list_tabs") return { tabs: await this.listTabs() };
|
|
238
271
|
if (action === "list_downloads") return { downloads: await this.listDownloads() };
|
|
239
272
|
if (action === "get_download") return await this.getDownloadChunk(input);
|
|
@@ -272,18 +305,20 @@ class BrowserRuntime {
|
|
|
272
305
|
case "navigate":
|
|
273
306
|
await this.gotoSettled(page, String(input.url));
|
|
274
307
|
return { tab: await this.tabInfo(page) };
|
|
308
|
+
case "set_viewport":
|
|
309
|
+
return await this.setViewport(page, tabId, (0, import_browser_input.parseViewportInput)(input));
|
|
275
310
|
case "read_page": {
|
|
276
311
|
const maxChars = clampInt(input.maxChars, 1e3, 2e5, 5e4);
|
|
277
312
|
const depth = input.depth === void 0 ? void 0 : clampInt(input.depth, 1, 50, 50);
|
|
278
313
|
const options = { mode: "ai", boxes: true, timeout: READ_PAGE_TIMEOUT_MS, ...depth === void 0 ? {} : { depth } };
|
|
279
314
|
let snapshot;
|
|
280
315
|
try {
|
|
281
|
-
snapshot = await
|
|
316
|
+
snapshot = await (0, import_password_safe_snapshot.passwordSafeAriaSnapshot)(page, options);
|
|
282
317
|
} catch (error) {
|
|
283
318
|
const message = error instanceof Error ? error.message : String(error);
|
|
284
319
|
if (isDestroyedContext(message)) {
|
|
285
320
|
await page.waitForLoadState("domcontentloaded", { timeout: 5e3 }).catch(() => void 0);
|
|
286
|
-
snapshot = await
|
|
321
|
+
snapshot = await (0, import_password_safe_snapshot.passwordSafeAriaSnapshot)(page, options);
|
|
287
322
|
} else if (/Timeout/i.test(message)) {
|
|
288
323
|
throw new Error(
|
|
289
324
|
`read_page timed out after ${READ_PAGE_TIMEOUT_MS}ms building the outline. The page is very large or busy; retry with a depth limit (e.g. depth: 10).`
|
|
@@ -311,13 +346,28 @@ class BrowserRuntime {
|
|
|
311
346
|
case "screenshot": {
|
|
312
347
|
if (input.fullPage === true) {
|
|
313
348
|
const metrics = await page.evaluate(() => ({
|
|
314
|
-
|
|
315
|
-
|
|
349
|
+
// Match Playwright's full-page bounds, which exclude scrollbar gutters.
|
|
350
|
+
docWidth: Math.max(
|
|
351
|
+
document.documentElement.scrollWidth,
|
|
352
|
+
document.documentElement.offsetWidth,
|
|
353
|
+
document.documentElement.clientWidth,
|
|
354
|
+
document.body?.scrollWidth ?? 0,
|
|
355
|
+
document.body?.offsetWidth ?? 0,
|
|
356
|
+
document.body?.clientWidth ?? 0
|
|
357
|
+
),
|
|
358
|
+
docHeight: Math.max(
|
|
359
|
+
document.documentElement.scrollHeight,
|
|
360
|
+
document.documentElement.offsetHeight,
|
|
361
|
+
document.documentElement.clientHeight,
|
|
362
|
+
document.body?.scrollHeight ?? 0,
|
|
363
|
+
document.body?.offsetHeight ?? 0,
|
|
364
|
+
document.body?.clientHeight ?? 0
|
|
365
|
+
)
|
|
316
366
|
}));
|
|
317
367
|
const width = Math.min(metrics.docWidth, MAX_FULL_PAGE_WIDTH);
|
|
318
368
|
const height = Math.min(metrics.docHeight, MAX_FULL_PAGE_HEIGHT);
|
|
319
369
|
const truncated = metrics.docHeight > MAX_FULL_PAGE_HEIGHT || metrics.docWidth > MAX_FULL_PAGE_WIDTH;
|
|
320
|
-
const bytes2 = truncated ? await page.screenshot({ type: "png", scale: "css", timeout: 15e3, clip: { x: 0, y: 0, width, height } }) : await page.screenshot({ type: "png", scale: "css", timeout: 15e3, fullPage: true });
|
|
370
|
+
const bytes2 = truncated ? await page.screenshot({ type: "png", scale: "css", timeout: 15e3, fullPage: true, clip: { x: 0, y: 0, width, height } }) : await page.screenshot({ type: "png", scale: "css", timeout: 15e3, fullPage: true });
|
|
321
371
|
return { base64: bytes2.toString("base64"), width, height, fullPage: true, pageHeight: metrics.docHeight, truncated };
|
|
322
372
|
}
|
|
323
373
|
const bytes = await page.screenshot({ type: "png", scale: "css" });
|
package/dist/cjs/main.cjs
CHANGED
|
@@ -154,11 +154,13 @@ async function connectLoop(params) {
|
|
|
154
154
|
portForwarding: true,
|
|
155
155
|
runJsArtifacts: true,
|
|
156
156
|
readPage: true,
|
|
157
|
+
passwordRedaction: true,
|
|
157
158
|
refActions: true,
|
|
158
159
|
waitFor: true,
|
|
159
160
|
formInput: true,
|
|
160
161
|
typedKeys: true,
|
|
161
|
-
screenshotFullPage: true
|
|
162
|
+
screenshotFullPage: true,
|
|
163
|
+
viewportSizing: true
|
|
162
164
|
}
|
|
163
165
|
},
|
|
164
166
|
tabs: await params.runtime.listTabs(),
|
package/dist/cjs/package.json
CHANGED
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __defProp = Object.defineProperty;
|
|
3
|
+
var __getOwnPropDesc = Object.getOwnPropertyDescriptor;
|
|
4
|
+
var __getOwnPropNames = Object.getOwnPropertyNames;
|
|
5
|
+
var __hasOwnProp = Object.prototype.hasOwnProperty;
|
|
6
|
+
var __export = (target, all) => {
|
|
7
|
+
for (var name in all)
|
|
8
|
+
__defProp(target, name, { get: all[name], enumerable: true });
|
|
9
|
+
};
|
|
10
|
+
var __copyProps = (to, from, except, desc) => {
|
|
11
|
+
if (from && typeof from === "object" || typeof from === "function") {
|
|
12
|
+
for (let key of __getOwnPropNames(from))
|
|
13
|
+
if (!__hasOwnProp.call(to, key) && key !== except)
|
|
14
|
+
__defProp(to, key, { get: () => from[key], enumerable: !(desc = __getOwnPropDesc(from, key)) || desc.enumerable });
|
|
15
|
+
}
|
|
16
|
+
return to;
|
|
17
|
+
};
|
|
18
|
+
var __toCommonJS = (mod) => __copyProps(__defProp({}, "__esModule", { value: true }), mod);
|
|
19
|
+
var password_safe_snapshot_exports = {};
|
|
20
|
+
__export(password_safe_snapshot_exports, {
|
|
21
|
+
passwordSafeAriaSnapshot: () => passwordSafeAriaSnapshot
|
|
22
|
+
});
|
|
23
|
+
module.exports = __toCommonJS(password_safe_snapshot_exports);
|
|
24
|
+
var import_node_module = require("node:module");
|
|
25
|
+
const import_meta = {};
|
|
26
|
+
const requirePlaywright = (0, import_node_module.createRequire)(typeof __filename === "string" ? __filename : import_meta.url);
|
|
27
|
+
const SUPPORTED_PLAYWRIGHT_VERSION = "1.62.1";
|
|
28
|
+
const DEFAULT_TIMEOUT_MS = 1e4;
|
|
29
|
+
const UNAVAILABLE = "Password-safe browser snapshots are unavailable with this Playwright runtime. Update r5d-browser before retrying.";
|
|
30
|
+
function isServerFrame(value) {
|
|
31
|
+
return !!value && typeof value.selectors?.callOnSelector === "function";
|
|
32
|
+
}
|
|
33
|
+
function captureInUtilityWorld({ elements, injected }, options) {
|
|
34
|
+
const root = elements[0];
|
|
35
|
+
const valueDescriptor = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value");
|
|
36
|
+
const typeDescriptor = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "type");
|
|
37
|
+
const attributeDescriptor = Object.getOwnPropertyDescriptor(Element.prototype, "getAttribute");
|
|
38
|
+
if (!root || typeof injected.ariaSnapshotWithRefs !== "function" || !valueDescriptor?.configurable || typeof valueDescriptor.get !== "function" || typeof typeDescriptor?.get !== "function" || !attributeDescriptor?.configurable || typeof attributeDescriptor.value !== "function") {
|
|
39
|
+
throw new Error("Password-safe snapshot utility APIs are unavailable.");
|
|
40
|
+
}
|
|
41
|
+
const readValue = valueDescriptor.get;
|
|
42
|
+
const readType = typeDescriptor.get;
|
|
43
|
+
const readAttribute = attributeDescriptor.value;
|
|
44
|
+
const isPassword = (element) => element instanceof HTMLInputElement && readType.call(element) === "password";
|
|
45
|
+
try {
|
|
46
|
+
Object.defineProperty(HTMLInputElement.prototype, "value", {
|
|
47
|
+
...valueDescriptor,
|
|
48
|
+
get() {
|
|
49
|
+
const value = readValue.call(this);
|
|
50
|
+
return value && isPassword(this) ? "[redacted]" : value;
|
|
51
|
+
}
|
|
52
|
+
});
|
|
53
|
+
Object.defineProperty(Element.prototype, "getAttribute", {
|
|
54
|
+
...attributeDescriptor,
|
|
55
|
+
value(name) {
|
|
56
|
+
const value = readAttribute.call(this, name);
|
|
57
|
+
return value && String(name).toLowerCase() === "value" && isPassword(this) ? "[redacted]" : value;
|
|
58
|
+
}
|
|
59
|
+
});
|
|
60
|
+
const snapshot = injected.ariaSnapshotWithRefs(root, options);
|
|
61
|
+
if (snapshot && typeof snapshot.then === "function") {
|
|
62
|
+
throw new Error("Password-safe snapshot generation must be synchronous.");
|
|
63
|
+
}
|
|
64
|
+
return snapshot;
|
|
65
|
+
} finally {
|
|
66
|
+
Object.defineProperty(Element.prototype, "getAttribute", attributeDescriptor);
|
|
67
|
+
Object.defineProperty(HTMLInputElement.prototype, "value", valueDescriptor);
|
|
68
|
+
}
|
|
69
|
+
}
|
|
70
|
+
function validateSnapshot(value) {
|
|
71
|
+
const snapshot = value;
|
|
72
|
+
if (!snapshot || typeof snapshot.text !== "string" || !Array.isArray(snapshot.iframeRefs) || !snapshot.iframeDepths || typeof snapshot.iframeDepths !== "object" || snapshot.iframeRefs.some((ref) => typeof ref !== "string" || !/^(f\d+)?e\d+$/.test(ref)) || Object.entries(snapshot.iframeDepths).some(([ref, depth]) => !/^(f\d+)?e\d+$/.test(ref) || !Number.isInteger(depth) || depth < 0)) {
|
|
73
|
+
throw new Error(UNAVAILABLE);
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
async function passwordSafeAriaSnapshot(page, options = {}) {
|
|
77
|
+
const timeout = options.timeout ?? DEFAULT_TIMEOUT_MS;
|
|
78
|
+
if (!Number.isFinite(timeout) || timeout <= 0) throw new Error("Password-safe snapshots require a finite positive timeout.");
|
|
79
|
+
let frame;
|
|
80
|
+
try {
|
|
81
|
+
if (requirePlaywright("playwright-core/package.json").version !== SUPPORTED_PLAYWRIGHT_VERSION) throw new Error(UNAVAILABLE);
|
|
82
|
+
const connection = page._connection;
|
|
83
|
+
if (typeof connection?.toImpl !== "function") throw new Error(UNAVAILABLE);
|
|
84
|
+
frame = connection.toImpl(page.mainFrame());
|
|
85
|
+
} catch {
|
|
86
|
+
throw new Error(UNAVAILABLE);
|
|
87
|
+
}
|
|
88
|
+
if (!isServerFrame(frame)) throw new Error(UNAVAILABLE);
|
|
89
|
+
const deadline = Date.now() + timeout;
|
|
90
|
+
const renderOptions = {
|
|
91
|
+
mode: "ai",
|
|
92
|
+
...options.depth === void 0 ? {} : { depth: options.depth },
|
|
93
|
+
boxes: options.boxes
|
|
94
|
+
};
|
|
95
|
+
const timeoutError = () => new Error(`Timeout ${timeout}ms exceeded building a password-safe browser snapshot.`);
|
|
96
|
+
const bounded = async (operation) => {
|
|
97
|
+
let timer;
|
|
98
|
+
try {
|
|
99
|
+
return await Promise.race([
|
|
100
|
+
operation,
|
|
101
|
+
new Promise((_, reject) => {
|
|
102
|
+
timer = setTimeout(() => reject(timeoutError()), Math.max(0, deadline - Date.now()));
|
|
103
|
+
})
|
|
104
|
+
]);
|
|
105
|
+
} finally {
|
|
106
|
+
clearTimeout(timer);
|
|
107
|
+
}
|
|
108
|
+
};
|
|
109
|
+
const captureFrame = async (currentFrame, selector, frameOptions, isMainFrame = false) => {
|
|
110
|
+
if (!isServerFrame(currentFrame)) throw new Error(UNAVAILABLE);
|
|
111
|
+
if (Date.now() >= deadline) throw timeoutError();
|
|
112
|
+
const captured = await bounded(currentFrame.selectors.callOnSelector(selector, { strict: false }, captureInUtilityWorld, frameOptions));
|
|
113
|
+
if (!captured) {
|
|
114
|
+
if (!isMainFrame) return [];
|
|
115
|
+
await bounded(new Promise((resolve) => setTimeout(resolve, 50)));
|
|
116
|
+
return captureFrame(currentFrame, selector, frameOptions, true);
|
|
117
|
+
}
|
|
118
|
+
if (captured.info?.world !== "utility" || !isServerFrame(captured.frame)) throw new Error(UNAVAILABLE);
|
|
119
|
+
validateSnapshot(captured.result);
|
|
120
|
+
const snapshot = captured.result;
|
|
121
|
+
const renderedRefs = snapshot.iframeRefs.filter((ref) => Object.hasOwn(snapshot.iframeDepths, ref));
|
|
122
|
+
const children = /* @__PURE__ */ new Map();
|
|
123
|
+
await Promise.all(
|
|
124
|
+
renderedRefs.map(async (ref) => {
|
|
125
|
+
const childDepth = frameOptions.depth === void 0 ? void 0 : frameOptions.depth - snapshot.iframeDepths[ref] - 1;
|
|
126
|
+
if (childDepth !== void 0 && childDepth <= 0) return;
|
|
127
|
+
children.set(
|
|
128
|
+
ref,
|
|
129
|
+
await captureFrame(captured.frame, `aria-ref=${ref} >> internal:control=enter-frame >> body,frameset`, {
|
|
130
|
+
...frameOptions,
|
|
131
|
+
...childDepth === void 0 ? {} : { depth: childDepth }
|
|
132
|
+
})
|
|
133
|
+
);
|
|
134
|
+
})
|
|
135
|
+
);
|
|
136
|
+
const lines = [];
|
|
137
|
+
for (const line of snapshot.text.split("\n")) {
|
|
138
|
+
const match = line.match(/^(\s*)- iframe (?:\[active\] )?\[ref=([^\]]*)\]/);
|
|
139
|
+
const child = match ? children.get(match[2]) : void 0;
|
|
140
|
+
lines.push(child?.length ? `${line}:` : line);
|
|
141
|
+
if (match && child) lines.push(...child.map((childLine) => `${match[1]} ${childLine}`));
|
|
142
|
+
}
|
|
143
|
+
return lines;
|
|
144
|
+
};
|
|
145
|
+
return (await captureFrame(frame, "body,frameset", renderOptions, true)).join("\n");
|
|
146
|
+
}
|
|
147
|
+
// Annotate the CommonJS export names for ESM import in node:
|
|
148
|
+
0 && (module.exports = {
|
|
149
|
+
passwordSafeAriaSnapshot
|
|
150
|
+
});
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
const browserInputFields = {
|
|
2
|
+
list_tabs: { required: [], optional: [] },
|
|
3
|
+
list_downloads: { required: [], optional: [] },
|
|
4
|
+
get_download: { required: ["downloadId"], optional: ["offset", "maxBytes"] },
|
|
5
|
+
open_tab: { required: [], optional: ["url", "disposition"] },
|
|
6
|
+
close_tab: { required: ["tabId"], optional: [] },
|
|
7
|
+
navigate: { required: ["tabId", "url"], optional: [] },
|
|
8
|
+
set_viewport: { required: ["tabId"], optional: ["width", "height", "reset"] },
|
|
9
|
+
read_page: { required: ["tabId"], optional: ["depth", "maxChars"] },
|
|
10
|
+
screenshot: { required: ["tabId"], optional: ["fullPage"] },
|
|
11
|
+
move_mouse: { required: ["tabId", "x", "y"], optional: [] },
|
|
12
|
+
mouse_click: { required: ["tabId"], optional: ["ref", "x", "y", "button", "clickCount"] },
|
|
13
|
+
scroll: { required: ["tabId"], optional: ["ref", "direction", "amount", "x", "y"] },
|
|
14
|
+
wait_for: { required: ["tabId"], optional: ["selector", "text", "url", "loadState", "state", "timeoutMs"] },
|
|
15
|
+
form_input: { required: ["tabId", "ref"], optional: ["value", "values", "checked"] },
|
|
16
|
+
keyboard: { required: ["tabId", "keys"], optional: ["modifiers"] },
|
|
17
|
+
keyboard_type: { required: ["tabId", "text"], optional: ["mode", "delayMs"] },
|
|
18
|
+
run_js: { required: ["tabId", "code"], optional: [] },
|
|
19
|
+
port_forward: { required: ["workerPort"], optional: ["browserPort"] },
|
|
20
|
+
list_port_forwards: { required: [], optional: [] },
|
|
21
|
+
stop_port_forward: { required: ["forwardId"], optional: [] }
|
|
22
|
+
};
|
|
23
|
+
const MAX_VIEWPORT_DIMENSION = 4096;
|
|
24
|
+
function parseViewportInput(input) {
|
|
25
|
+
if (input.reset !== void 0 && typeof input.reset !== "boolean") throw new Error("set_viewport reset must be a boolean.");
|
|
26
|
+
if (input.reset === true) {
|
|
27
|
+
if (input.width !== void 0 || input.height !== void 0)
|
|
28
|
+
throw new Error("set_viewport accepts width/height or reset: true, not both.");
|
|
29
|
+
return null;
|
|
30
|
+
}
|
|
31
|
+
const { width, height } = input;
|
|
32
|
+
for (const [name, value] of Object.entries({ width, height })) {
|
|
33
|
+
if (typeof value !== "number" || !Number.isInteger(value) || value < 1 || value > MAX_VIEWPORT_DIMENSION) {
|
|
34
|
+
throw new Error(
|
|
35
|
+
`set_viewport requires ${name} to be an integer from 1 to ${MAX_VIEWPORT_DIMENSION} CSS pixels, or reset: true without dimensions.`
|
|
36
|
+
);
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
return { width, height };
|
|
40
|
+
}
|
|
41
|
+
function normalizeBrowserInput(action, input) {
|
|
42
|
+
const fields = browserInputFields[action];
|
|
43
|
+
for (const key of fields.required) {
|
|
44
|
+
if (input[key] === null || input[key] === void 0) throw new Error(`${action} requires ${key}.`);
|
|
45
|
+
}
|
|
46
|
+
const normalized = { ...input };
|
|
47
|
+
for (const key of fields.optional) {
|
|
48
|
+
if (normalized[key] === null) delete normalized[key];
|
|
49
|
+
}
|
|
50
|
+
return normalized;
|
|
51
|
+
}
|
|
52
|
+
export {
|
|
53
|
+
MAX_VIEWPORT_DIMENSION,
|
|
54
|
+
normalizeBrowserInput,
|
|
55
|
+
parseViewportInput
|
|
56
|
+
};
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
import fs from "node:fs";
|
|
2
2
|
import path from "node:path";
|
|
3
|
+
import { normalizeBrowserInput, parseViewportInput } from "./browser-input.mjs";
|
|
4
|
+
import { passwordSafeAriaSnapshot } from "./password-safe-snapshot.mjs";
|
|
3
5
|
const MAX_DOWNLOAD_CHUNK_BYTES = 1024 * 1024;
|
|
4
6
|
const MAX_RUN_JS_RESULT_BYTES = 1024 * 1024;
|
|
5
7
|
const MAX_RUN_JS_ARTIFACT_BYTES = 4 * 1024 * 1024;
|
|
@@ -49,6 +51,7 @@ class BrowserRuntime {
|
|
|
49
51
|
ids = /* @__PURE__ */ new WeakMap();
|
|
50
52
|
pages = /* @__PURE__ */ new Map();
|
|
51
53
|
cursor = /* @__PURE__ */ new Map();
|
|
54
|
+
viewportSessions = /* @__PURE__ */ new Map();
|
|
52
55
|
downloadSaveQueue = Promise.resolve();
|
|
53
56
|
track(page) {
|
|
54
57
|
const existing = this.ids.get(page);
|
|
@@ -65,6 +68,7 @@ class BrowserRuntime {
|
|
|
65
68
|
page.once("close", () => {
|
|
66
69
|
this.pages.delete(id);
|
|
67
70
|
this.cursor.delete(id);
|
|
71
|
+
this.viewportSessions.delete(id);
|
|
68
72
|
});
|
|
69
73
|
return id;
|
|
70
74
|
}
|
|
@@ -200,7 +204,36 @@ class BrowserRuntime {
|
|
|
200
204
|
async drainDownloads() {
|
|
201
205
|
await this.downloadSaveQueue;
|
|
202
206
|
}
|
|
207
|
+
async setViewport(page, tabId, viewport) {
|
|
208
|
+
const existing = this.viewportSessions.get(tabId);
|
|
209
|
+
if (viewport) {
|
|
210
|
+
const cdp = existing ?? await this.context.newCDPSession(page);
|
|
211
|
+
try {
|
|
212
|
+
await cdp.send("Emulation.setDeviceMetricsOverride", { ...viewport, deviceScaleFactor: 0, mobile: false });
|
|
213
|
+
this.viewportSessions.set(tabId, cdp);
|
|
214
|
+
} catch (error) {
|
|
215
|
+
if (!existing) await cdp.detach().catch(() => void 0);
|
|
216
|
+
throw error;
|
|
217
|
+
}
|
|
218
|
+
} else if (existing) {
|
|
219
|
+
await existing.send("Emulation.clearDeviceMetricsOverride");
|
|
220
|
+
this.viewportSessions.delete(tabId);
|
|
221
|
+
await existing.detach().catch(() => void 0);
|
|
222
|
+
}
|
|
223
|
+
this.cursor.delete(tabId);
|
|
224
|
+
if (viewport) {
|
|
225
|
+
const ready = await page.waitForFunction(
|
|
226
|
+
({ width, height }) => window.innerWidth === width && window.innerHeight === height,
|
|
227
|
+
viewport,
|
|
228
|
+
{ timeout: 5e3 }
|
|
229
|
+
);
|
|
230
|
+
await ready.dispose();
|
|
231
|
+
}
|
|
232
|
+
const actual = await page.evaluate(() => ({ width: window.innerWidth, height: window.innerHeight }));
|
|
233
|
+
return { tabId, ...actual, reset: viewport === null };
|
|
234
|
+
}
|
|
203
235
|
async execute(action, input) {
|
|
236
|
+
input = normalizeBrowserInput(action, input);
|
|
204
237
|
if (action === "list_tabs") return { tabs: await this.listTabs() };
|
|
205
238
|
if (action === "list_downloads") return { downloads: await this.listDownloads() };
|
|
206
239
|
if (action === "get_download") return await this.getDownloadChunk(input);
|
|
@@ -239,18 +272,20 @@ class BrowserRuntime {
|
|
|
239
272
|
case "navigate":
|
|
240
273
|
await this.gotoSettled(page, String(input.url));
|
|
241
274
|
return { tab: await this.tabInfo(page) };
|
|
275
|
+
case "set_viewport":
|
|
276
|
+
return await this.setViewport(page, tabId, parseViewportInput(input));
|
|
242
277
|
case "read_page": {
|
|
243
278
|
const maxChars = clampInt(input.maxChars, 1e3, 2e5, 5e4);
|
|
244
279
|
const depth = input.depth === void 0 ? void 0 : clampInt(input.depth, 1, 50, 50);
|
|
245
280
|
const options = { mode: "ai", boxes: true, timeout: READ_PAGE_TIMEOUT_MS, ...depth === void 0 ? {} : { depth } };
|
|
246
281
|
let snapshot;
|
|
247
282
|
try {
|
|
248
|
-
snapshot = await page
|
|
283
|
+
snapshot = await passwordSafeAriaSnapshot(page, options);
|
|
249
284
|
} catch (error) {
|
|
250
285
|
const message = error instanceof Error ? error.message : String(error);
|
|
251
286
|
if (isDestroyedContext(message)) {
|
|
252
287
|
await page.waitForLoadState("domcontentloaded", { timeout: 5e3 }).catch(() => void 0);
|
|
253
|
-
snapshot = await page
|
|
288
|
+
snapshot = await passwordSafeAriaSnapshot(page, options);
|
|
254
289
|
} else if (/Timeout/i.test(message)) {
|
|
255
290
|
throw new Error(
|
|
256
291
|
`read_page timed out after ${READ_PAGE_TIMEOUT_MS}ms building the outline. The page is very large or busy; retry with a depth limit (e.g. depth: 10).`
|
|
@@ -278,13 +313,28 @@ class BrowserRuntime {
|
|
|
278
313
|
case "screenshot": {
|
|
279
314
|
if (input.fullPage === true) {
|
|
280
315
|
const metrics = await page.evaluate(() => ({
|
|
281
|
-
|
|
282
|
-
|
|
316
|
+
// Match Playwright's full-page bounds, which exclude scrollbar gutters.
|
|
317
|
+
docWidth: Math.max(
|
|
318
|
+
document.documentElement.scrollWidth,
|
|
319
|
+
document.documentElement.offsetWidth,
|
|
320
|
+
document.documentElement.clientWidth,
|
|
321
|
+
document.body?.scrollWidth ?? 0,
|
|
322
|
+
document.body?.offsetWidth ?? 0,
|
|
323
|
+
document.body?.clientWidth ?? 0
|
|
324
|
+
),
|
|
325
|
+
docHeight: Math.max(
|
|
326
|
+
document.documentElement.scrollHeight,
|
|
327
|
+
document.documentElement.offsetHeight,
|
|
328
|
+
document.documentElement.clientHeight,
|
|
329
|
+
document.body?.scrollHeight ?? 0,
|
|
330
|
+
document.body?.offsetHeight ?? 0,
|
|
331
|
+
document.body?.clientHeight ?? 0
|
|
332
|
+
)
|
|
283
333
|
}));
|
|
284
334
|
const width = Math.min(metrics.docWidth, MAX_FULL_PAGE_WIDTH);
|
|
285
335
|
const height = Math.min(metrics.docHeight, MAX_FULL_PAGE_HEIGHT);
|
|
286
336
|
const truncated = metrics.docHeight > MAX_FULL_PAGE_HEIGHT || metrics.docWidth > MAX_FULL_PAGE_WIDTH;
|
|
287
|
-
const bytes2 = truncated ? await page.screenshot({ type: "png", scale: "css", timeout: 15e3, clip: { x: 0, y: 0, width, height } }) : await page.screenshot({ type: "png", scale: "css", timeout: 15e3, fullPage: true });
|
|
337
|
+
const bytes2 = truncated ? await page.screenshot({ type: "png", scale: "css", timeout: 15e3, fullPage: true, clip: { x: 0, y: 0, width, height } }) : await page.screenshot({ type: "png", scale: "css", timeout: 15e3, fullPage: true });
|
|
288
338
|
return { base64: bytes2.toString("base64"), width, height, fullPage: true, pageHeight: metrics.docHeight, truncated };
|
|
289
339
|
}
|
|
290
340
|
const bytes = await page.screenshot({ type: "png", scale: "css" });
|
package/dist/mjs/main.mjs
CHANGED
|
@@ -131,11 +131,13 @@ async function connectLoop(params) {
|
|
|
131
131
|
portForwarding: true,
|
|
132
132
|
runJsArtifacts: true,
|
|
133
133
|
readPage: true,
|
|
134
|
+
passwordRedaction: true,
|
|
134
135
|
refActions: true,
|
|
135
136
|
waitFor: true,
|
|
136
137
|
formInput: true,
|
|
137
138
|
typedKeys: true,
|
|
138
|
-
screenshotFullPage: true
|
|
139
|
+
screenshotFullPage: true,
|
|
140
|
+
viewportSizing: true
|
|
139
141
|
}
|
|
140
142
|
},
|
|
141
143
|
tabs: await params.runtime.listTabs(),
|
package/dist/mjs/package.json
CHANGED
|
@@ -0,0 +1,125 @@
|
|
|
1
|
+
import { createRequire } from "node:module";
|
|
2
|
+
const requirePlaywright = createRequire(typeof __filename === "string" ? __filename : import.meta.url);
|
|
3
|
+
const SUPPORTED_PLAYWRIGHT_VERSION = "1.62.1";
|
|
4
|
+
const DEFAULT_TIMEOUT_MS = 1e4;
|
|
5
|
+
const UNAVAILABLE = "Password-safe browser snapshots are unavailable with this Playwright runtime. Update r5d-browser before retrying.";
|
|
6
|
+
function isServerFrame(value) {
|
|
7
|
+
return !!value && typeof value.selectors?.callOnSelector === "function";
|
|
8
|
+
}
|
|
9
|
+
function captureInUtilityWorld({ elements, injected }, options) {
|
|
10
|
+
const root = elements[0];
|
|
11
|
+
const valueDescriptor = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "value");
|
|
12
|
+
const typeDescriptor = Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, "type");
|
|
13
|
+
const attributeDescriptor = Object.getOwnPropertyDescriptor(Element.prototype, "getAttribute");
|
|
14
|
+
if (!root || typeof injected.ariaSnapshotWithRefs !== "function" || !valueDescriptor?.configurable || typeof valueDescriptor.get !== "function" || typeof typeDescriptor?.get !== "function" || !attributeDescriptor?.configurable || typeof attributeDescriptor.value !== "function") {
|
|
15
|
+
throw new Error("Password-safe snapshot utility APIs are unavailable.");
|
|
16
|
+
}
|
|
17
|
+
const readValue = valueDescriptor.get;
|
|
18
|
+
const readType = typeDescriptor.get;
|
|
19
|
+
const readAttribute = attributeDescriptor.value;
|
|
20
|
+
const isPassword = (element) => element instanceof HTMLInputElement && readType.call(element) === "password";
|
|
21
|
+
try {
|
|
22
|
+
Object.defineProperty(HTMLInputElement.prototype, "value", {
|
|
23
|
+
...valueDescriptor,
|
|
24
|
+
get() {
|
|
25
|
+
const value = readValue.call(this);
|
|
26
|
+
return value && isPassword(this) ? "[redacted]" : value;
|
|
27
|
+
}
|
|
28
|
+
});
|
|
29
|
+
Object.defineProperty(Element.prototype, "getAttribute", {
|
|
30
|
+
...attributeDescriptor,
|
|
31
|
+
value(name) {
|
|
32
|
+
const value = readAttribute.call(this, name);
|
|
33
|
+
return value && String(name).toLowerCase() === "value" && isPassword(this) ? "[redacted]" : value;
|
|
34
|
+
}
|
|
35
|
+
});
|
|
36
|
+
const snapshot = injected.ariaSnapshotWithRefs(root, options);
|
|
37
|
+
if (snapshot && typeof snapshot.then === "function") {
|
|
38
|
+
throw new Error("Password-safe snapshot generation must be synchronous.");
|
|
39
|
+
}
|
|
40
|
+
return snapshot;
|
|
41
|
+
} finally {
|
|
42
|
+
Object.defineProperty(Element.prototype, "getAttribute", attributeDescriptor);
|
|
43
|
+
Object.defineProperty(HTMLInputElement.prototype, "value", valueDescriptor);
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
function validateSnapshot(value) {
|
|
47
|
+
const snapshot = value;
|
|
48
|
+
if (!snapshot || typeof snapshot.text !== "string" || !Array.isArray(snapshot.iframeRefs) || !snapshot.iframeDepths || typeof snapshot.iframeDepths !== "object" || snapshot.iframeRefs.some((ref) => typeof ref !== "string" || !/^(f\d+)?e\d+$/.test(ref)) || Object.entries(snapshot.iframeDepths).some(([ref, depth]) => !/^(f\d+)?e\d+$/.test(ref) || !Number.isInteger(depth) || depth < 0)) {
|
|
49
|
+
throw new Error(UNAVAILABLE);
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
async function passwordSafeAriaSnapshot(page, options = {}) {
|
|
53
|
+
const timeout = options.timeout ?? DEFAULT_TIMEOUT_MS;
|
|
54
|
+
if (!Number.isFinite(timeout) || timeout <= 0) throw new Error("Password-safe snapshots require a finite positive timeout.");
|
|
55
|
+
let frame;
|
|
56
|
+
try {
|
|
57
|
+
if (requirePlaywright("playwright-core/package.json").version !== SUPPORTED_PLAYWRIGHT_VERSION) throw new Error(UNAVAILABLE);
|
|
58
|
+
const connection = page._connection;
|
|
59
|
+
if (typeof connection?.toImpl !== "function") throw new Error(UNAVAILABLE);
|
|
60
|
+
frame = connection.toImpl(page.mainFrame());
|
|
61
|
+
} catch {
|
|
62
|
+
throw new Error(UNAVAILABLE);
|
|
63
|
+
}
|
|
64
|
+
if (!isServerFrame(frame)) throw new Error(UNAVAILABLE);
|
|
65
|
+
const deadline = Date.now() + timeout;
|
|
66
|
+
const renderOptions = {
|
|
67
|
+
mode: "ai",
|
|
68
|
+
...options.depth === void 0 ? {} : { depth: options.depth },
|
|
69
|
+
boxes: options.boxes
|
|
70
|
+
};
|
|
71
|
+
const timeoutError = () => new Error(`Timeout ${timeout}ms exceeded building a password-safe browser snapshot.`);
|
|
72
|
+
const bounded = async (operation) => {
|
|
73
|
+
let timer;
|
|
74
|
+
try {
|
|
75
|
+
return await Promise.race([
|
|
76
|
+
operation,
|
|
77
|
+
new Promise((_, reject) => {
|
|
78
|
+
timer = setTimeout(() => reject(timeoutError()), Math.max(0, deadline - Date.now()));
|
|
79
|
+
})
|
|
80
|
+
]);
|
|
81
|
+
} finally {
|
|
82
|
+
clearTimeout(timer);
|
|
83
|
+
}
|
|
84
|
+
};
|
|
85
|
+
const captureFrame = async (currentFrame, selector, frameOptions, isMainFrame = false) => {
|
|
86
|
+
if (!isServerFrame(currentFrame)) throw new Error(UNAVAILABLE);
|
|
87
|
+
if (Date.now() >= deadline) throw timeoutError();
|
|
88
|
+
const captured = await bounded(currentFrame.selectors.callOnSelector(selector, { strict: false }, captureInUtilityWorld, frameOptions));
|
|
89
|
+
if (!captured) {
|
|
90
|
+
if (!isMainFrame) return [];
|
|
91
|
+
await bounded(new Promise((resolve) => setTimeout(resolve, 50)));
|
|
92
|
+
return captureFrame(currentFrame, selector, frameOptions, true);
|
|
93
|
+
}
|
|
94
|
+
if (captured.info?.world !== "utility" || !isServerFrame(captured.frame)) throw new Error(UNAVAILABLE);
|
|
95
|
+
validateSnapshot(captured.result);
|
|
96
|
+
const snapshot = captured.result;
|
|
97
|
+
const renderedRefs = snapshot.iframeRefs.filter((ref) => Object.hasOwn(snapshot.iframeDepths, ref));
|
|
98
|
+
const children = /* @__PURE__ */ new Map();
|
|
99
|
+
await Promise.all(
|
|
100
|
+
renderedRefs.map(async (ref) => {
|
|
101
|
+
const childDepth = frameOptions.depth === void 0 ? void 0 : frameOptions.depth - snapshot.iframeDepths[ref] - 1;
|
|
102
|
+
if (childDepth !== void 0 && childDepth <= 0) return;
|
|
103
|
+
children.set(
|
|
104
|
+
ref,
|
|
105
|
+
await captureFrame(captured.frame, `aria-ref=${ref} >> internal:control=enter-frame >> body,frameset`, {
|
|
106
|
+
...frameOptions,
|
|
107
|
+
...childDepth === void 0 ? {} : { depth: childDepth }
|
|
108
|
+
})
|
|
109
|
+
);
|
|
110
|
+
})
|
|
111
|
+
);
|
|
112
|
+
const lines = [];
|
|
113
|
+
for (const line of snapshot.text.split("\n")) {
|
|
114
|
+
const match = line.match(/^(\s*)- iframe (?:\[active\] )?\[ref=([^\]]*)\]/);
|
|
115
|
+
const child = match ? children.get(match[2]) : void 0;
|
|
116
|
+
lines.push(child?.length ? `${line}:` : line);
|
|
117
|
+
if (match && child) lines.push(...child.map((childLine) => `${match[1]} ${childLine}`));
|
|
118
|
+
}
|
|
119
|
+
return lines;
|
|
120
|
+
};
|
|
121
|
+
return (await captureFrame(frame, "body,frameset", renderOptions, true)).join("\n");
|
|
122
|
+
}
|
|
123
|
+
export {
|
|
124
|
+
passwordSafeAriaSnapshot
|
|
125
|
+
};
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
declare const browserInputFields: {
|
|
2
|
+
list_tabs: {
|
|
3
|
+
required: never[];
|
|
4
|
+
optional: never[];
|
|
5
|
+
};
|
|
6
|
+
list_downloads: {
|
|
7
|
+
required: never[];
|
|
8
|
+
optional: never[];
|
|
9
|
+
};
|
|
10
|
+
get_download: {
|
|
11
|
+
required: string[];
|
|
12
|
+
optional: string[];
|
|
13
|
+
};
|
|
14
|
+
open_tab: {
|
|
15
|
+
required: never[];
|
|
16
|
+
optional: string[];
|
|
17
|
+
};
|
|
18
|
+
close_tab: {
|
|
19
|
+
required: string[];
|
|
20
|
+
optional: never[];
|
|
21
|
+
};
|
|
22
|
+
navigate: {
|
|
23
|
+
required: string[];
|
|
24
|
+
optional: never[];
|
|
25
|
+
};
|
|
26
|
+
set_viewport: {
|
|
27
|
+
required: string[];
|
|
28
|
+
optional: string[];
|
|
29
|
+
};
|
|
30
|
+
read_page: {
|
|
31
|
+
required: string[];
|
|
32
|
+
optional: string[];
|
|
33
|
+
};
|
|
34
|
+
screenshot: {
|
|
35
|
+
required: string[];
|
|
36
|
+
optional: string[];
|
|
37
|
+
};
|
|
38
|
+
move_mouse: {
|
|
39
|
+
required: string[];
|
|
40
|
+
optional: never[];
|
|
41
|
+
};
|
|
42
|
+
mouse_click: {
|
|
43
|
+
required: string[];
|
|
44
|
+
optional: string[];
|
|
45
|
+
};
|
|
46
|
+
scroll: {
|
|
47
|
+
required: string[];
|
|
48
|
+
optional: string[];
|
|
49
|
+
};
|
|
50
|
+
wait_for: {
|
|
51
|
+
required: string[];
|
|
52
|
+
optional: string[];
|
|
53
|
+
};
|
|
54
|
+
form_input: {
|
|
55
|
+
required: string[];
|
|
56
|
+
optional: string[];
|
|
57
|
+
};
|
|
58
|
+
keyboard: {
|
|
59
|
+
required: string[];
|
|
60
|
+
optional: string[];
|
|
61
|
+
};
|
|
62
|
+
keyboard_type: {
|
|
63
|
+
required: string[];
|
|
64
|
+
optional: string[];
|
|
65
|
+
};
|
|
66
|
+
run_js: {
|
|
67
|
+
required: string[];
|
|
68
|
+
optional: never[];
|
|
69
|
+
};
|
|
70
|
+
port_forward: {
|
|
71
|
+
required: string[];
|
|
72
|
+
optional: string[];
|
|
73
|
+
};
|
|
74
|
+
list_port_forwards: {
|
|
75
|
+
required: never[];
|
|
76
|
+
optional: never[];
|
|
77
|
+
};
|
|
78
|
+
stop_port_forward: {
|
|
79
|
+
required: string[];
|
|
80
|
+
optional: never[];
|
|
81
|
+
};
|
|
82
|
+
};
|
|
83
|
+
export type BrowserInputAction = keyof typeof browserInputFields;
|
|
84
|
+
export declare const MAX_VIEWPORT_DIMENSION = 4096;
|
|
85
|
+
export type BrowserViewport = {
|
|
86
|
+
width: number;
|
|
87
|
+
height: number;
|
|
88
|
+
};
|
|
89
|
+
export declare function parseViewportInput(input: Readonly<Record<string, unknown>>): BrowserViewport | null;
|
|
90
|
+
export declare function normalizeBrowserInput(action: BrowserInputAction, input: Readonly<Record<string, unknown>>): Record<string, unknown>;
|
|
91
|
+
export {};
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import type { BrowserContext } from "playwright-core";
|
|
2
|
-
export type BrowserAction = "list_tabs" | "list_downloads" | "get_download" | "open_tab" | "close_tab" | "navigate" | "read_page" | "screenshot" | "move_mouse" | "mouse_click" | "scroll" | "wait_for" | "form_input" | "keyboard" | "keyboard_type" | "run_js";
|
|
2
|
+
export type BrowserAction = "list_tabs" | "list_downloads" | "get_download" | "open_tab" | "close_tab" | "navigate" | "set_viewport" | "read_page" | "screenshot" | "move_mouse" | "mouse_click" | "scroll" | "wait_for" | "form_input" | "keyboard" | "keyboard_type" | "run_js";
|
|
3
3
|
export type TabInfo = {
|
|
4
4
|
tabId: string;
|
|
5
5
|
windowId: string;
|
|
@@ -18,6 +18,7 @@ export declare class BrowserRuntime {
|
|
|
18
18
|
private readonly ids;
|
|
19
19
|
private readonly pages;
|
|
20
20
|
private readonly cursor;
|
|
21
|
+
private readonly viewportSessions;
|
|
21
22
|
private downloadSaveQueue;
|
|
22
23
|
constructor(context: BrowserContext, downloadsPath: string);
|
|
23
24
|
private track;
|
|
@@ -34,5 +35,6 @@ export declare class BrowserRuntime {
|
|
|
34
35
|
private tabInfo;
|
|
35
36
|
listTabs(): Promise<TabInfo[]>;
|
|
36
37
|
drainDownloads(): Promise<void>;
|
|
38
|
+
private setViewport;
|
|
37
39
|
execute(action: BrowserAction, input: Record<string, unknown>): Promise<unknown>;
|
|
38
40
|
}
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
import type { Page } from "playwright-core";
|
|
2
|
+
export type PasswordSafeSnapshotOptions = {
|
|
3
|
+
mode?: "ai";
|
|
4
|
+
depth?: number;
|
|
5
|
+
boxes?: boolean;
|
|
6
|
+
timeout?: number;
|
|
7
|
+
};
|
|
8
|
+
/**
|
|
9
|
+
* Playwright 1.62.1 has no public snapshot redaction hook. This deliberately narrow
|
|
10
|
+
* adapter uses its in-process server Frame and the same utility-world selector,
|
|
11
|
+
* ref generation, and iframe stitching as ariaSnapshotForFrame in coreBundle.js.
|
|
12
|
+
* The exact version and private API shapes are checked; failure never falls back
|
|
13
|
+
* to page.ariaSnapshot(), which exposes password values in this pinned version.
|
|
14
|
+
*/
|
|
15
|
+
export declare function passwordSafeAriaSnapshot(page: Page, options?: PasswordSafeSnapshotOptions): Promise<string>;
|