pi-chrome 0.15.53 → 0.15.54
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +11 -0
- package/README.md +1 -1
- package/docs/FAQ.md +1 -1
- package/extensions/chrome-profile-bridge/browser-extension/manifest.json +1 -1
- package/extensions/chrome-profile-bridge/browser-extension/service_worker.js +246 -28
- package/extensions/chrome-profile-bridge/index.ts +73 -10
- package/package.json +2 -2
- package/test-suite/unit/background-policy.test.mjs +2 -1
- package/test-suite/unit/hidden-input.test.mjs +279 -0
- package/test-suite/unit/input-reliability.test.mjs +39 -6
- package/test-suite/unit/type-evidence.test.mjs +23 -2
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,17 @@
|
|
|
2
2
|
|
|
3
3
|
All notable user-facing changes to `pi-chrome`.
|
|
4
4
|
|
|
5
|
+
## 0.15.54 — 2026-09-26
|
|
6
|
+
|
|
7
|
+
- **Hidden tabs report instead of silently dropping input.** Chrome ignores trusted input to hidden pages. A page is hidden when it is an inactive tab, its window is minimized, its window is fully covered by another window, or (on macOS) its window became a hidden window tab behind a full-screen window. In a hidden page, mouse presses and keys were dropped while the tool reported success, and mouse moves could hang until a timeout (the source of `Detached while handling command` in background runs). Input tools now check page visibility first. `chrome_type`, `chrome_key`, `chrome_hover`, `chrome_scroll`, `chrome_tap` and `chrome_drag` fail fast with the reason and the options (`background:false`, or `/chrome background off`). `chrome_click` and `chrome_fill` go straight to their existing DOM fallback and report the reason, or reject with `domFallback:false`. Hard background mode is unchanged: nothing is activated or focused.
|
|
8
|
+
- **No hidden automation windows in full screen.** While your Chrome window is full screen, a new automation window would become a hidden macOS window tab or land on another Space. The automation target is now an inactive tab in your window instead.
|
|
9
|
+
- **Targeted `chrome_type` appends.** With a uid/selector, `chrome_type` clicked a random point inside the field, which left the caret mid-text and spliced input into existing content. After the focus click it now moves the caret to the end with Chrome's `moveToEndOfDocument` editing command. Untargeted typing still uses the current caret; `replace:true` is unchanged.
|
|
10
|
+
- **Upload buttons, labels, wrappers, and iframes.** `chrome_upload_file` accepts a label for a file input, or a wrapper/dropzone containing exactly one, and searches same-origin iframes for selectors. For upload buttons that open the native picker (no persistent `<input type=file>`), it intercepts the file chooser (`Page.setInterceptFileChooserDialog`) so no OS dialog appears. The trigger click uses trusted Chrome input on visible pages. On hidden pages it uses `element.click()` with a CDP user gesture, unless `domFallback:false`. It rejects multiple paths for single-file inputs. Adapted from kkunkunya's fork.
|
|
11
|
+
- **No duplicate `change` on upload.** Chrome's `DOM.setFileInputFiles` already fires `input`/`change`, and pi-chrome dispatched a second pair, which could make apps upload twice. It now dispatches only when Chrome did not, and reports `events: native|dispatched`.
|
|
12
|
+
- **Bounded screenshot folder.** Default-path `chrome_screenshot` captures prune `.pi/chrome-screenshots` at capture time. Captures older than 7 days are removed, but the newest 20 are always kept. Only files matching the tool's own naming (including full-page tiles and manifests) are eligible, and explicit `path:` captures are never pruned. `retentionDays` overrides the window; `0` disables pruning. Adapted from ardhiqii's fork.
|
|
13
|
+
- **`input.debug` without a target.** It no longer fails with `Value must be at least 0`; it describes this session's automation tab if one exists.
|
|
14
|
+
- **Validation.** New `hidden-input` unit suite. Live-checked on Chrome/macOS: hidden-tab errors and DOM fallbacks, caret-end typing in inputs and contenteditables, all upload shapes in visible and hidden tabs, single `change` events, and pruning through a real Pi session.
|
|
15
|
+
|
|
5
16
|
## 0.15.53 — 2026-09-26
|
|
6
17
|
|
|
7
18
|
Changes adapted from community forks (ardhiqii, kkunkunya, nihar-oracle, steimerbyte).
|
package/README.md
CHANGED
|
@@ -85,7 +85,7 @@ Run `/chrome revoke` when finished. Use `/chrome authorize` again whenever you w
|
|
|
85
85
|
|
|
86
86
|
This is browser automation, not full OS control. Native Chrome/OS dialogs, password-manager prompts, passkeys/security keys/biometrics, CAPTCHA challenges, cross-origin iframe DOM access, rich multitouch/stylus gestures, and arbitrary desktop apps are outside its reliable tool surface. Some workflows need human assistance.
|
|
87
87
|
|
|
88
|
-
If page inspection or evaluation is blocked, use screenshots and coordinate input where possible. Background pages can throttle rendering or reject focus-gated actions. See the [FAQ](./docs/FAQ.md) for details.
|
|
88
|
+
If page inspection or evaluation is blocked, use screenshots and coordinate input where possible. Background pages can throttle rendering or reject focus-gated actions. Chrome ignores real input to hidden pages (inactive tabs, covered or minimized windows, windows behind a macOS full-screen window); input tools report this instead of pretending to succeed. See the [FAQ](./docs/FAQ.md) for details.
|
|
89
89
|
|
|
90
90
|
## Commands
|
|
91
91
|
|
package/docs/FAQ.md
CHANGED
|
@@ -75,7 +75,7 @@ If the page did not change, take a fresh snapshot or screenshot and check for ov
|
|
|
75
75
|
|
|
76
76
|
## How do I attach a file to a React file input?
|
|
77
77
|
|
|
78
|
-
`chrome_upload_file` — uses Chrome DevTools file-input control
|
|
78
|
+
`chrome_upload_file` — uses Chrome DevTools file-input control; the page gets one `input` + `change`. Target the file input, a label or wrapper containing one, or an upload button: for buttons, the native file chooser is intercepted, so no OS dialog appears. Selectors also search same-origin iframes. Works with React/Vue/Angular controlled inputs.
|
|
79
79
|
|
|
80
80
|
## Can it record videos?
|
|
81
81
|
|
|
@@ -174,6 +174,17 @@ function isPiChromeOwnedTarget(tabId, sessionKey) {
|
|
|
174
174
|
return false;
|
|
175
175
|
}
|
|
176
176
|
|
|
177
|
+
// A new window created while the user's Chrome window is full screen does not become a visible
|
|
178
|
+
// standalone window: macOS either merges it into the full-screen window as a hidden native window
|
|
179
|
+
// tab ("Prefer tabs: in full screen") or places it on another Space. Its page is hidden, so trusted
|
|
180
|
+
// input cannot reach it. Use a grouped background tab in the user's window instead, which is at least
|
|
181
|
+
// visible in the tab strip and does not add a macOS tab bar to the user's full-screen window.
|
|
182
|
+
async function userWindowIsFullscreen() {
|
|
183
|
+
if (typeof chrome.windows?.getLastFocused !== "function") return false;
|
|
184
|
+
const win = await chrome.windows.getLastFocused({ windowTypes: ["normal"] }).catch(() => null);
|
|
185
|
+
return win?.state === "fullscreen";
|
|
186
|
+
}
|
|
187
|
+
|
|
177
188
|
// Create a fresh automation target for `sessionKey`. If this session already has a tab group,
|
|
178
189
|
// create the tab inside that group's window so one Pi session keeps one Chrome tab group (Chrome
|
|
179
190
|
// groups cannot span windows). If no group exists yet, prefer an isolated window; fall back to a
|
|
@@ -187,7 +198,7 @@ async function createAutomationTarget(sessionKey, groupTitle) {
|
|
|
187
198
|
await persistAutomationTargets();
|
|
188
199
|
return tab;
|
|
189
200
|
}
|
|
190
|
-
if (chrome.windows && typeof chrome.windows.create === "function") {
|
|
201
|
+
if (chrome.windows && typeof chrome.windows.create === "function" && !(await userWindowIsFullscreen())) {
|
|
191
202
|
try {
|
|
192
203
|
const win = await chrome.windows.create({ url: "about:blank", focused: false });
|
|
193
204
|
const created = win && Array.isArray(win.tabs) ? win.tabs[0] : undefined;
|
|
@@ -391,7 +402,8 @@ async function attachDebugger(tabId) {
|
|
|
391
402
|
}
|
|
392
403
|
|
|
393
404
|
async function describeInputTarget(tabId) {
|
|
394
|
-
const
|
|
405
|
+
const id = Number(tabId);
|
|
406
|
+
const tab = Number.isInteger(id) && id >= 0 ? await chrome.tabs.get(id).catch(() => null) : null;
|
|
395
407
|
const active = (await chrome.tabs.query({ active: true, lastFocusedWindow: true }).catch(() => []))[0] || null;
|
|
396
408
|
let targets = [];
|
|
397
409
|
try { targets = await new Promise((resolve) => chrome.debugger.getTargets((t) => resolve(t || []))); } catch {}
|
|
@@ -408,7 +420,13 @@ function targetMetaSuffix(meta) {
|
|
|
408
420
|
}
|
|
409
421
|
|
|
410
422
|
async function inputDebug(params) {
|
|
411
|
-
|
|
423
|
+
// No explicit target: describe this session's automation tab if it exists, without creating one.
|
|
424
|
+
let tabId = params?.targetId !== undefined ? Number(params.targetId) : null;
|
|
425
|
+
if (tabId === null) {
|
|
426
|
+
const owned = await resolveOwnedAutomationTarget(sessionKeyOf(params || {})).catch(() => null);
|
|
427
|
+
tabId = owned?.id ?? null;
|
|
428
|
+
}
|
|
429
|
+
const requested = await describeInputTarget(tabId);
|
|
412
430
|
return {
|
|
413
431
|
extensionVersion: chrome.runtime.getManifest().version,
|
|
414
432
|
extensionId: chrome.runtime.id,
|
|
@@ -417,6 +435,57 @@ async function inputDebug(params) {
|
|
|
417
435
|
};
|
|
418
436
|
}
|
|
419
437
|
|
|
438
|
+
// Chrome drops trusted CDP input to hidden pages: mouse presses and keys are silently ignored and
|
|
439
|
+
// mouse moves can hang waiting for a frame. A page is hidden when its tab is inactive, its window is
|
|
440
|
+
// minimized, or (macOS) its window became a native window-tab behind a full-screen window. Check
|
|
441
|
+
// before sending input so the caller gets an actionable error instead of a false success.
|
|
442
|
+
async function pageVisibilityState(tabId) {
|
|
443
|
+
try {
|
|
444
|
+
const evaluated = await cdp(tabId, "Runtime.evaluate", { expression: "document.visibilityState", returnByValue: true });
|
|
445
|
+
const value = evaluated?.result?.value;
|
|
446
|
+
return typeof value === "string" ? value : null;
|
|
447
|
+
} catch {
|
|
448
|
+
return null; // Unknown: do not block input on a failed probe.
|
|
449
|
+
}
|
|
450
|
+
}
|
|
451
|
+
|
|
452
|
+
function hiddenTabReason(tab, win) {
|
|
453
|
+
if (win?.state === "minimized") return "its window is minimized";
|
|
454
|
+
if (tab && tab.active === false) return "it is an inactive tab in its window";
|
|
455
|
+
if (win?.state === "fullscreen") return "its window is behind a full-screen window (on macOS, new windows can become hidden window tabs)";
|
|
456
|
+
return "its window is covered by another window or is on another Space (Chrome treats fully covered windows as hidden)";
|
|
457
|
+
}
|
|
458
|
+
|
|
459
|
+
async function assertTabVisibleForInput(tab, params, label) {
|
|
460
|
+
let state = await pageVisibilityState(tab.id);
|
|
461
|
+
if (state !== "hidden") return;
|
|
462
|
+
if (foregroundRequested(params)) {
|
|
463
|
+
// bringToFront just ran; give Chrome a moment to report the page visible.
|
|
464
|
+
for (let i = 0; i < 6 && state === "hidden"; i++) {
|
|
465
|
+
await sleep(100);
|
|
466
|
+
state = await pageVisibilityState(tab.id);
|
|
467
|
+
}
|
|
468
|
+
if (state !== "hidden") return;
|
|
469
|
+
}
|
|
470
|
+
const fresh = await chrome.tabs.get(tab.id).catch(() => tab);
|
|
471
|
+
const win = typeof chrome.windows?.get === "function" ? await chrome.windows.get(fresh?.windowId ?? tab.windowId).catch(() => null) : null;
|
|
472
|
+
const error = new Error(
|
|
473
|
+
`${label}: tab ${tab.id} is hidden because ${hiddenTabReason(fresh, win)}. Chrome ignores trusted input to hidden pages, so the events would be dropped. ` +
|
|
474
|
+
`Pass background:false for this call (or run /chrome background off) to bring the tab forward. chrome_click and chrome_fill can instead use their DOM fallback (untrusted page events).`,
|
|
475
|
+
);
|
|
476
|
+
error.hiddenTab = true;
|
|
477
|
+
error.windowState = win?.state;
|
|
478
|
+
throw error;
|
|
479
|
+
}
|
|
480
|
+
|
|
481
|
+
// Put the caret at the end of the focused field with Chrome's editing command (platform-neutral,
|
|
482
|
+
// works for inputs, textareas, and contenteditables). Used after chrome_type clicks a field to focus
|
|
483
|
+
// it, so the random click point never splices text into the middle of existing content.
|
|
484
|
+
async function moveCaretToEnd(tabId) {
|
|
485
|
+
await cdp(tabId, "Input.dispatchKeyEvent", { type: "rawKeyDown", key: "End", code: "End", windowsVirtualKeyCode: 35, commands: ["moveToEndOfDocument"] });
|
|
486
|
+
await cdp(tabId, "Input.dispatchKeyEvent", { type: "keyUp", key: "End", code: "End", windowsVirtualKeyCode: 35 });
|
|
487
|
+
}
|
|
488
|
+
|
|
420
489
|
async function detachDebugger(tabId) {
|
|
421
490
|
const entry = attachedTabs.get(tabId);
|
|
422
491
|
if (!entry) return;
|
|
@@ -825,6 +894,7 @@ async function chromeInputClick(params) {
|
|
|
825
894
|
await bringToFront(tab, params);
|
|
826
895
|
try {
|
|
827
896
|
await attachDebugger(tab.id);
|
|
897
|
+
await assertTabVisibleForInput(tab, params, "chrome.click");
|
|
828
898
|
const resolved = await resolveTargetInTab(tab.id, params);
|
|
829
899
|
const point = resolved.rect ? pickInsideRect(resolved.rect) : { x: resolved.x, y: resolved.y };
|
|
830
900
|
await cdpMoveTo(tab.id, point.x, point.y);
|
|
@@ -861,6 +931,7 @@ async function chromeInputHover(params) {
|
|
|
861
931
|
const tab = await getTabByParams(params);
|
|
862
932
|
await bringToFront(tab, params);
|
|
863
933
|
await attachDebugger(tab.id);
|
|
934
|
+
await assertTabVisibleForInput(tab, params, "chrome.hover");
|
|
864
935
|
const resolved = await resolveTargetInTab(tab.id, params);
|
|
865
936
|
const point = resolved.rect ? pickInsideRect(resolved.rect) : { x: resolved.x, y: resolved.y };
|
|
866
937
|
await cdpMoveTo(tab.id, point.x, point.y);
|
|
@@ -872,6 +943,7 @@ async function chromeInputKey(params) {
|
|
|
872
943
|
const tab = await getTabByParams(params);
|
|
873
944
|
await bringToFront(tab, params);
|
|
874
945
|
await attachDebugger(tab.id);
|
|
946
|
+
await assertTabVisibleForInput(tab, params, "chrome.key");
|
|
875
947
|
const key = String(params.key || "");
|
|
876
948
|
if (!key) throw new Error("chrome.key: missing key");
|
|
877
949
|
const mods = params.modifiers || {};
|
|
@@ -954,6 +1026,7 @@ async function chromeInputType(params) {
|
|
|
954
1026
|
const tab = await getTabByParams(params);
|
|
955
1027
|
await bringToFront(tab, params);
|
|
956
1028
|
await attachDebugger(tab.id);
|
|
1029
|
+
await assertTabVisibleForInput(tab, params, "chrome.type");
|
|
957
1030
|
if (params.selector || params.uid) {
|
|
958
1031
|
// Focus target by clicking it first.
|
|
959
1032
|
const resolved = await resolveTargetInTab(tab.id, params);
|
|
@@ -963,6 +1036,9 @@ async function chromeInputType(params) {
|
|
|
963
1036
|
await sleep(rng(45, 110));
|
|
964
1037
|
await cdp(tab.id, "Input.dispatchMouseEvent", { type: "mouseReleased", x: point.x, y: point.y, button: "left", buttons: 0, clickCount: 1, pointerType: "mouse" });
|
|
965
1038
|
await sleep(rng(50, 120));
|
|
1039
|
+
// The click lands at a random point inside the field, which leaves the caret mid-text. Move it to
|
|
1040
|
+
// the end so a targeted chrome_type appends; replace=true selects everything below anyway.
|
|
1041
|
+
if (params.replace !== true) await moveCaretToEnd(tab.id);
|
|
966
1042
|
}
|
|
967
1043
|
const text = String(params.text || "");
|
|
968
1044
|
// chrome_type inserts at the caret. Read the field before and after so a splice into existing
|
|
@@ -1144,6 +1220,7 @@ async function chromeInputFill(params) {
|
|
|
1144
1220
|
await bringToFront(tab, params);
|
|
1145
1221
|
try {
|
|
1146
1222
|
await attachDebugger(tab.id);
|
|
1223
|
+
await assertTabVisibleForInput(tab, params, "chrome.fill");
|
|
1147
1224
|
if (!(params.selector || params.uid)) throw new Error("chrome.fill: selector or uid required");
|
|
1148
1225
|
const resolved = await resolveTargetInTab(tab.id, params);
|
|
1149
1226
|
const point = resolved.rect ? pickInsideRect(resolved.rect) : { x: resolved.x, y: resolved.y };
|
|
@@ -1174,6 +1251,7 @@ async function chromeInputScroll(params) {
|
|
|
1174
1251
|
const tab = await getTabByParams(params);
|
|
1175
1252
|
await bringToFront(tab, params);
|
|
1176
1253
|
await attachDebugger(tab.id);
|
|
1254
|
+
await assertTabVisibleForInput(tab, params, "chrome.scroll");
|
|
1177
1255
|
const resolved = (params.selector || params.uid) ? await resolveTargetInTab(tab.id, params) : { x: 100, y: 100, rect: null };
|
|
1178
1256
|
const x = resolved.rect ? resolved.rect.left + Math.min(resolved.rect.width, 800) / 2 : resolved.x;
|
|
1179
1257
|
const y = resolved.rect ? resolved.rect.top + Math.min(resolved.rect.height, 600) / 2 : resolved.y;
|
|
@@ -1224,6 +1302,7 @@ async function chromeInputTap(params) {
|
|
|
1224
1302
|
const tab = await getTabByParams(params);
|
|
1225
1303
|
await bringToFront(tab, params);
|
|
1226
1304
|
await attachDebugger(tab.id);
|
|
1305
|
+
await assertTabVisibleForInput(tab, params, "chrome.tap");
|
|
1227
1306
|
const resolved = (params.selector || params.uid || (typeof params.x === "number" && typeof params.y === "number"))
|
|
1228
1307
|
? await resolveTargetInTab(tab.id, params)
|
|
1229
1308
|
: null;
|
|
@@ -1240,6 +1319,7 @@ async function chromeInputDrag(params) {
|
|
|
1240
1319
|
const tab = await getTabByParams(params);
|
|
1241
1320
|
await bringToFront(tab, params);
|
|
1242
1321
|
await attachDebugger(tab.id);
|
|
1322
|
+
await assertTabVisibleForInput(tab, params, "chrome.drag");
|
|
1243
1323
|
const from = await resolveTargetInTab(tab.id, { selector: params.fromSelector ?? null, uid: params.fromUid ?? null, x: params.fromX ?? null, y: params.fromY ?? null });
|
|
1244
1324
|
const to = await resolveTargetInTab(tab.id, { selector: params.toSelector ?? null, uid: params.toUid ?? null, x: params.toX ?? null, y: params.toY ?? null });
|
|
1245
1325
|
const fp = from.rect ? pickInsideRect(from.rect) : { x: from.x, y: from.y };
|
|
@@ -1261,6 +1341,100 @@ async function chromeInputDrag(params) {
|
|
|
1261
1341
|
return { input: "chrome", from: fp, to: tp, steps };
|
|
1262
1342
|
}
|
|
1263
1343
|
|
|
1344
|
+
// Pending native file-chooser intercepts, keyed by tabId. chrome_upload_file uses these when the
|
|
1345
|
+
// target is an upload trigger (button/dropzone) rather than a persistent <input type=file>:
|
|
1346
|
+
// Page.setInterceptFileChooserDialog -> click the trigger -> Page.fileChooserOpened ->
|
|
1347
|
+
// DOM.setFileInputFiles(backendNodeId). No OS dialog is shown.
|
|
1348
|
+
const pendingFileChoosers = new Map();
|
|
1349
|
+
const FILE_CHOOSER_TIMEOUT_MS = 8_000;
|
|
1350
|
+
|
|
1351
|
+
function fileChooserTabIdForSource(source) {
|
|
1352
|
+
if (typeof source?.tabId === "number") return source.tabId;
|
|
1353
|
+
if (source?.targetId) {
|
|
1354
|
+
for (const [tabId, entry] of attachedTabs) if (entry?.debuggee?.targetId === source.targetId) return tabId;
|
|
1355
|
+
}
|
|
1356
|
+
return undefined;
|
|
1357
|
+
}
|
|
1358
|
+
|
|
1359
|
+
function handleDebuggerEvent(source, method, params) {
|
|
1360
|
+
if (method !== "Page.fileChooserOpened") return;
|
|
1361
|
+
const tabId = fileChooserTabIdForSource(source);
|
|
1362
|
+
const entry = tabId === undefined ? undefined : pendingFileChoosers.get(tabId);
|
|
1363
|
+
if (!entry) return;
|
|
1364
|
+
pendingFileChoosers.delete(tabId);
|
|
1365
|
+
entry.resolve({ backendNodeId: params?.backendNodeId, mode: params?.mode });
|
|
1366
|
+
}
|
|
1367
|
+
|
|
1368
|
+
if (chrome.debugger && chrome.debugger.onEvent) {
|
|
1369
|
+
chrome.debugger.onEvent.addListener(handleDebuggerEvent);
|
|
1370
|
+
}
|
|
1371
|
+
|
|
1372
|
+
function waitForFileChooser(tabId, timeoutMs) {
|
|
1373
|
+
let entry;
|
|
1374
|
+
const promise = new Promise((resolve, reject) => {
|
|
1375
|
+
entry = { resolve, reject };
|
|
1376
|
+
entry.timer = setTimeout(() => {
|
|
1377
|
+
if (pendingFileChoosers.get(tabId) === entry) pendingFileChoosers.delete(tabId);
|
|
1378
|
+
reject(new Error("chrome.upload: no file chooser opened after clicking the target. Pass the <input type=file> itself, or an element that opens the file picker when clicked."));
|
|
1379
|
+
}, timeoutMs);
|
|
1380
|
+
});
|
|
1381
|
+
pendingFileChoosers.set(tabId, entry);
|
|
1382
|
+
return { promise: promise.finally(() => clearTimeout(entry.timer)), cancel: () => { if (pendingFileChoosers.get(tabId) === entry) pendingFileChoosers.delete(tabId); clearTimeout(entry.timer); } };
|
|
1383
|
+
}
|
|
1384
|
+
|
|
1385
|
+
// Resolve the upload target in the page. Selectors also search same-origin iframes (editors often
|
|
1386
|
+
// host their file input in one). A label for a file input, or a wrapper/dropzone containing exactly
|
|
1387
|
+
// one file input, resolves to that input so no click is needed.
|
|
1388
|
+
function uploadTargetExpression(selector, uid) {
|
|
1389
|
+
return `(() => {
|
|
1390
|
+
const selector = ${JSON.stringify(selector ?? null)};
|
|
1391
|
+
const uid = ${JSON.stringify(uid ?? null)};
|
|
1392
|
+
const state = window.__PI_CHROME_STATE__;
|
|
1393
|
+
const findIn = (doc, depth) => {
|
|
1394
|
+
let found = null;
|
|
1395
|
+
try { found = doc.querySelector(selector); } catch (error) { throw new Error("Invalid selector: " + selector); }
|
|
1396
|
+
if (found || depth >= 3) return found;
|
|
1397
|
+
for (const frame of doc.querySelectorAll("iframe,frame")) {
|
|
1398
|
+
let inner = null;
|
|
1399
|
+
try { inner = frame.contentDocument; } catch {}
|
|
1400
|
+
if (!inner) continue;
|
|
1401
|
+
found = findIn(inner, depth + 1);
|
|
1402
|
+
if (found) return found;
|
|
1403
|
+
}
|
|
1404
|
+
return null;
|
|
1405
|
+
};
|
|
1406
|
+
let el = uid ? state?.elements?.[uid] : (selector ? findIn(document, 0) : null);
|
|
1407
|
+
if (uid && (!el || !el.isConnected)) throw new Error("snapshot uid " + uid + " is stale; refresh chrome_snapshot");
|
|
1408
|
+
if (!el) throw new Error("chrome.upload: target not found: " + (uid || selector));
|
|
1409
|
+
const isFile = (node) => node && node.tagName === "INPUT" && String(node.type).toLowerCase() === "file";
|
|
1410
|
+
if (!isFile(el)) {
|
|
1411
|
+
const control = el.tagName === "LABEL" && isFile(el.control) ? el.control : null;
|
|
1412
|
+
const nested = control ? null : Array.from(el.querySelectorAll ? el.querySelectorAll("input[type=file]") : []);
|
|
1413
|
+
if (control) el = control;
|
|
1414
|
+
else if (nested && nested.length === 1) el = nested[0];
|
|
1415
|
+
}
|
|
1416
|
+
return el;
|
|
1417
|
+
})()`;
|
|
1418
|
+
}
|
|
1419
|
+
|
|
1420
|
+
const UPLOAD_TARGET_INFO_FN = `function() {
|
|
1421
|
+
const isFile = this.tagName === "INPUT" && String(this.type).toLowerCase() === "file";
|
|
1422
|
+
if (!isFile) this.scrollIntoView({ block: "center", inline: "center", behavior: "instant" });
|
|
1423
|
+
const r = this.getBoundingClientRect();
|
|
1424
|
+
let x = r.left + r.width / 2, y = r.top + r.height / 2;
|
|
1425
|
+
let win = this.ownerDocument.defaultView;
|
|
1426
|
+
// Convert iframe-relative coordinates to top-level viewport coordinates (same-origin frames only).
|
|
1427
|
+
while (win && win.frameElement) {
|
|
1428
|
+
const frame = win.frameElement;
|
|
1429
|
+
const fr = frame.getBoundingClientRect();
|
|
1430
|
+
const cs = frame.ownerDocument.defaultView.getComputedStyle(frame);
|
|
1431
|
+
x += fr.left + (parseFloat(cs.borderLeftWidth) || 0) + (parseFloat(cs.paddingLeft) || 0);
|
|
1432
|
+
y += fr.top + (parseFloat(cs.borderTopWidth) || 0) + (parseFloat(cs.paddingTop) || 0);
|
|
1433
|
+
win = win.parent === win ? null : win.parent;
|
|
1434
|
+
}
|
|
1435
|
+
return { tag: this.tagName, isFile, multiple: isFile ? this.multiple === true : undefined, inFrame: this.ownerDocument !== document, x, y, width: r.width, height: r.height };
|
|
1436
|
+
}`;
|
|
1437
|
+
|
|
1264
1438
|
async function chromeInputUpload(params) {
|
|
1265
1439
|
const tab = await getTabByParams(params);
|
|
1266
1440
|
await bringToFront(tab, params);
|
|
@@ -1268,36 +1442,80 @@ async function chromeInputUpload(params) {
|
|
|
1268
1442
|
if (!(params.selector || params.uid)) throw new Error("chrome.upload: selector or uid required");
|
|
1269
1443
|
const paths = Array.isArray(params.paths) ? params.paths.map(String) : [];
|
|
1270
1444
|
if (!paths.length) throw new Error("chrome.upload: no file paths provided");
|
|
1271
|
-
const
|
|
1272
|
-
|
|
1273
|
-
const uid = ${JSON.stringify(params.uid ?? null)};
|
|
1274
|
-
const state = window.__PI_CHROME_STATE__;
|
|
1275
|
-
const el = uid ? state?.elements?.[uid] : (selector ? document.querySelector(selector) : null);
|
|
1276
|
-
if (uid && (!el || !el.isConnected)) throw new Error("snapshot uid " + uid + " is stale; refresh chrome_snapshot");
|
|
1277
|
-
if (!el || el.tagName !== "INPUT" || el.type !== "file") throw new Error("Target must be <input type=file>");
|
|
1278
|
-
el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" });
|
|
1279
|
-
return el;
|
|
1280
|
-
})()`;
|
|
1281
|
-
const evaluated = await cdp(tab.id, "Runtime.evaluate", { expression, objectGroup: "pi-chrome-upload", includeCommandLineAPI: false, returnByValue: false });
|
|
1282
|
-
if (evaluated.exceptionDetails) throw new Error(cdpExceptionText(evaluated.exceptionDetails) || "Could not resolve file input");
|
|
1445
|
+
const evaluated = await cdp(tab.id, "Runtime.evaluate", { expression: uploadTargetExpression(params.selector, params.uid), objectGroup: "pi-chrome-upload", includeCommandLineAPI: false, returnByValue: false });
|
|
1446
|
+
if (evaluated.exceptionDetails) throw new Error(cdpExceptionText(evaluated.exceptionDetails) || "Could not resolve upload target");
|
|
1283
1447
|
const objectId = evaluated.result?.objectId;
|
|
1284
|
-
if (!objectId) throw new Error("Could not resolve
|
|
1448
|
+
if (!objectId) throw new Error("Could not resolve upload target object");
|
|
1285
1449
|
try {
|
|
1286
|
-
await cdp(tab.id, "
|
|
1287
|
-
|
|
1288
|
-
|
|
1289
|
-
|
|
1290
|
-
|
|
1291
|
-
|
|
1292
|
-
|
|
1293
|
-
objectId
|
|
1294
|
-
|
|
1295
|
-
|
|
1296
|
-
|
|
1450
|
+
const infoResult = await cdp(tab.id, "Runtime.callFunctionOn", { objectId, functionDeclaration: UPLOAD_TARGET_INFO_FN, returnByValue: true });
|
|
1451
|
+
const info = infoResult?.result?.value || {};
|
|
1452
|
+
if (info.isFile) {
|
|
1453
|
+
if (paths.length > 1 && info.multiple === false) throw new Error(`chrome.upload: the file input accepts one file but ${paths.length} paths were given`);
|
|
1454
|
+
await cdp(tab.id, "DOM.enable", {}).catch(() => undefined);
|
|
1455
|
+
// Some DOM agents return nodeId:0 (or reject conversion) for a valid remote
|
|
1456
|
+
// element. CDP accepts that same objectId directly, before any file mutation.
|
|
1457
|
+
const requested = await cdp(tab.id, "DOM.requestNode", { objectId }).catch(() => null);
|
|
1458
|
+
const target = requested?.nodeId ? { nodeId: requested.nodeId } : { objectId };
|
|
1459
|
+
// Chrome's setFileInputFiles normally fires input/change itself. Watch for that and only
|
|
1460
|
+
// dispatch our own events when it did not, so apps never see a duplicate change (double upload).
|
|
1461
|
+
await cdp(tab.id, "Runtime.callFunctionOn", {
|
|
1462
|
+
objectId,
|
|
1463
|
+
functionDeclaration: `function() { const el = this; el.__piChromeChangeSeen = false; el.addEventListener("change", () => { el.__piChromeChangeSeen = true; }, { once: true, capture: true }); }`,
|
|
1464
|
+
}).catch(() => undefined);
|
|
1465
|
+
await cdp(tab.id, "DOM.setFileInputFiles", { ...target, files: paths });
|
|
1466
|
+
const notified = await cdp(tab.id, "Runtime.callFunctionOn", {
|
|
1467
|
+
objectId,
|
|
1468
|
+
functionDeclaration: `function() { const seen = this.__piChromeChangeSeen === true; delete this.__piChromeChangeSeen; if (seen) return "native"; this.dispatchEvent(new Event("input", { bubbles: true })); this.dispatchEvent(new Event("change", { bubbles: true })); return "dispatched"; }`,
|
|
1469
|
+
returnByValue: true,
|
|
1470
|
+
}).catch(() => null);
|
|
1471
|
+
const events = notified?.result?.value;
|
|
1472
|
+
return { input: "chrome", mode: "file-input", events: typeof events === "string" ? events : undefined, inFrame: info.inFrame || undefined, uploaded: paths.map((path) => ({ path })) };
|
|
1473
|
+
}
|
|
1474
|
+
return await uploadViaFileChooser(tab, params, paths, objectId, info);
|
|
1297
1475
|
} finally {
|
|
1298
1476
|
await cdp(tab.id, "Runtime.releaseObject", { objectId }).catch(() => undefined);
|
|
1299
1477
|
}
|
|
1300
|
-
|
|
1478
|
+
}
|
|
1479
|
+
|
|
1480
|
+
// The target opens the native picker when clicked. Intercept the chooser so no OS dialog appears,
|
|
1481
|
+
// then attach the files to the input Chrome reports. The click is trusted Chrome input when the page
|
|
1482
|
+
// is visible; a hidden page drops trusted input, so there we call element.click() with a CDP user
|
|
1483
|
+
// gesture instead (an untrusted event, like chrome_click's DOM fallback) unless domFallback:false.
|
|
1484
|
+
async function uploadViaFileChooser(tab, params, paths, objectId, info) {
|
|
1485
|
+
const hidden = (await pageVisibilityState(tab.id)) === "hidden";
|
|
1486
|
+
if (hidden && params.domFallback === false) await assertTabVisibleForInput(tab, params, "chrome.upload");
|
|
1487
|
+
await cdp(tab.id, "Page.enable", {});
|
|
1488
|
+
await cdp(tab.id, "Page.setInterceptFileChooserDialog", { enabled: true });
|
|
1489
|
+
const chooser = waitForFileChooser(tab.id, FILE_CHOOSER_TIMEOUT_MS);
|
|
1490
|
+
let trigger;
|
|
1491
|
+
try {
|
|
1492
|
+
if (!hidden && typeof info.x === "number" && typeof info.y === "number" && info.width > 0 && info.height > 0) {
|
|
1493
|
+
trigger = "chrome";
|
|
1494
|
+
await cdpMoveTo(tab.id, info.x, info.y);
|
|
1495
|
+
await cdp(tab.id, "Input.dispatchMouseEvent", { type: "mousePressed", x: info.x, y: info.y, button: "left", buttons: 1, clickCount: 1, pointerType: "mouse" });
|
|
1496
|
+
await sleep(rng(40, 110));
|
|
1497
|
+
await cdp(tab.id, "Input.dispatchMouseEvent", { type: "mouseReleased", x: info.x, y: info.y, button: "left", buttons: 0, clickCount: 1, pointerType: "mouse" });
|
|
1498
|
+
} else {
|
|
1499
|
+
trigger = "activation-click";
|
|
1500
|
+
await cdp(tab.id, "Runtime.callFunctionOn", { objectId, functionDeclaration: "function() { this.click(); }", userGesture: true, awaitPromise: false });
|
|
1501
|
+
}
|
|
1502
|
+
const opened = await chooser.promise;
|
|
1503
|
+
if (typeof opened.backendNodeId !== "number") throw new Error("chrome.upload: file chooser opened without an input node");
|
|
1504
|
+
if (paths.length > 1 && opened.mode === "selectSingle") throw new Error(`chrome.upload: the file chooser accepts one file but ${paths.length} paths were given`);
|
|
1505
|
+
await cdp(tab.id, "DOM.enable", {}).catch(() => undefined);
|
|
1506
|
+
await cdp(tab.id, "DOM.setFileInputFiles", { backendNodeId: opened.backendNodeId, files: paths });
|
|
1507
|
+
return {
|
|
1508
|
+
input: "chrome",
|
|
1509
|
+
mode: "file-chooser",
|
|
1510
|
+
trigger,
|
|
1511
|
+
reason: trigger === "activation-click" ? "target tab is hidden; Chrome drops trusted input to hidden pages" : undefined,
|
|
1512
|
+
inFrame: info.inFrame || undefined,
|
|
1513
|
+
uploaded: paths.map((path) => ({ path })),
|
|
1514
|
+
};
|
|
1515
|
+
} finally {
|
|
1516
|
+
chooser.cancel();
|
|
1517
|
+
await cdp(tab.id, "Page.setInterceptFileChooserDialog", { enabled: false }).catch(() => undefined);
|
|
1518
|
+
}
|
|
1301
1519
|
}
|
|
1302
1520
|
// ===============================================================
|
|
1303
1521
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
2
2
|
import { Type } from "typebox";
|
|
3
3
|
import { existsSync, readFileSync, statSync } from "node:fs";
|
|
4
|
-
import { mkdir, writeFile } from "node:fs/promises";
|
|
4
|
+
import { mkdir, readdir, unlink, writeFile } from "node:fs/promises";
|
|
5
5
|
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
|
|
6
6
|
import { dirname, join, resolve } from "node:path";
|
|
7
7
|
|
|
@@ -236,6 +236,58 @@ function formatCdpResult(method: string, value: unknown): ToolTextResult {
|
|
|
236
236
|
return { content: [{ type: "text", text: truncateText(text) }], details: { value } };
|
|
237
237
|
}
|
|
238
238
|
|
|
239
|
+
// chrome_screenshot writes a timestamped file per capture (plus -tileN files and a .json manifest for
|
|
240
|
+
// full-page captures) into .pi/chrome-screenshots. Prune that folder at capture time so it stays
|
|
241
|
+
// bounded. Only files matching the tool's own naming are eligible; the newest `keep` captures are always
|
|
242
|
+
// kept, and nothing inside the retention window is removed. Best-effort: errors never fail a capture.
|
|
243
|
+
const SCREENSHOT_NAME_RE = /^(\d{4}-\d{2}-\d{2}T\d{2}-\d{2}-\d{2}-\d{3}Z)(?:-tile\d+)?\.(?:png|jpeg)(?:\.json)?$/;
|
|
244
|
+
const SCREENSHOT_KEEP_NEWEST = 20;
|
|
245
|
+
const SCREENSHOT_RETENTION_DAYS = 7;
|
|
246
|
+
|
|
247
|
+
function screenshotCaptureTime(stamp: string): number {
|
|
248
|
+
const iso = stamp.replace(/^(\d{4}-\d{2}-\d{2}T\d{2})-(\d{2})-(\d{2})-(\d{3})Z$/, "$1:$2:$3.$4Z");
|
|
249
|
+
return Date.parse(iso);
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
function screenshotFilesToPrune(names: string[], now: number, retentionDays = SCREENSHOT_RETENTION_DAYS, keep = SCREENSHOT_KEEP_NEWEST): string[] {
|
|
253
|
+
if (!(retentionDays > 0)) return [];
|
|
254
|
+
const captures = new Map<string, string[]>();
|
|
255
|
+
for (const name of names) {
|
|
256
|
+
const match = SCREENSHOT_NAME_RE.exec(name);
|
|
257
|
+
if (!match) continue;
|
|
258
|
+
const files = captures.get(match[1]) ?? [];
|
|
259
|
+
files.push(name);
|
|
260
|
+
captures.set(match[1], files);
|
|
261
|
+
}
|
|
262
|
+
const cutoff = now - retentionDays * 24 * 60 * 60 * 1000;
|
|
263
|
+
const newestFirst = [...captures.keys()].sort().reverse();
|
|
264
|
+
const prune: string[] = [];
|
|
265
|
+
for (const stamp of newestFirst.slice(keep)) {
|
|
266
|
+
const takenAt = screenshotCaptureTime(stamp);
|
|
267
|
+
if (Number.isFinite(takenAt) && takenAt < cutoff) prune.push(...(captures.get(stamp) ?? []));
|
|
268
|
+
}
|
|
269
|
+
return prune;
|
|
270
|
+
}
|
|
271
|
+
|
|
272
|
+
async function pruneScreenshotDir(dir: string, retentionDays?: number): Promise<number> {
|
|
273
|
+
try {
|
|
274
|
+
const names = await readdir(dir);
|
|
275
|
+
let removed = 0;
|
|
276
|
+
for (const name of screenshotFilesToPrune(names, Date.now(), retentionDays)) {
|
|
277
|
+
try {
|
|
278
|
+
await unlink(join(dir, name));
|
|
279
|
+
removed++;
|
|
280
|
+
} catch {
|
|
281
|
+
// Locked or already gone: keep going.
|
|
282
|
+
}
|
|
283
|
+
}
|
|
284
|
+
return removed;
|
|
285
|
+
} catch {
|
|
286
|
+
return 0;
|
|
287
|
+
}
|
|
288
|
+
}
|
|
289
|
+
// End screenshot pruning.
|
|
290
|
+
|
|
239
291
|
function formatChromeInspect(inspect: any): string {
|
|
240
292
|
if (!inspect || typeof inspect !== "object") return safeJson(inspect);
|
|
241
293
|
const t = inspect.target || {};
|
|
@@ -1066,7 +1118,8 @@ Capability model (important):
|
|
|
1066
1118
|
- Input bypasses page CSP because it is injected at browser input layer, not page JavaScript. Chrome may show the “Pi Chrome Connector started debugging this browser” banner while attached.
|
|
1067
1119
|
- \`chrome_evaluate\` and \`chrome_snapshot\` run in MAIN world via **CDP \`Runtime.evaluate\`**, which is not subject to the page's Content-Security-Policy. They work even on strict-CSP pages (e.g. github.com, many bank/SaaS apps) that block \`'unsafe-eval'\`. \`chrome_navigate initScript\` likewise injects at document_start via CDP and bypasses CSP. \`chrome_screenshot\`, \`chrome_tab\`, and Chrome input also work under any CSP.
|
|
1068
1120
|
- Input tools return structured details and support \`includeSnapshot=true\` on click/type/fill/key. Use the fresh snapshot to verify state instead of repeating blindly. If the action started a navigation, the snapshot waits (up to 5s) for the new page.
|
|
1069
|
-
- \`chrome_type\`
|
|
1121
|
+
- \`chrome_type\` never replaces: with a uid/selector it appends at the end of the field; without one it types at the current caret. Check its before/after report. Use \`chrome_fill\` to replace a field.
|
|
1122
|
+
- Chrome ignores real input to hidden pages (inactive tabs, minimized windows, windows behind a macOS full-screen window). Input tools report this instead of silently doing nothing; retry with background:false only when bringing the tab forward is acceptable.
|
|
1070
1123
|
- \`chrome_cdp\` runs any raw CDP method on a tab (emulation, cookies, PDF, accessibility tree, etc.) when no dedicated chrome_* tool fits; \`chrome_cdp_targets\` diagnoses debugger/overlay conflicts.
|
|
1071
1124
|
|
|
1072
1125
|
Usage rules:
|
|
@@ -1630,7 +1683,7 @@ Usage rules:
|
|
|
1630
1683
|
selector: Type.Optional(Type.String({ description: "CSS selector to click. Prefer uid from chrome_snapshot when available." })),
|
|
1631
1684
|
x: Type.Optional(Type.Number({ description: "Viewport x coordinate if uid/selector is omitted." })),
|
|
1632
1685
|
y: Type.Optional(Type.Number({ description: "Viewport y coordinate if uid/selector is omitted." })),
|
|
1633
|
-
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked
|
|
1686
|
+
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM-dispatched click if Chrome's CDP input path is blocked (extension overlay, debugger failure, or a hidden tab that ignores trusted input)." })),
|
|
1634
1687
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, include a fresh chrome_snapshot result after the click." })),
|
|
1635
1688
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
|
1636
1689
|
targetId: Type.Optional(Type.String()),
|
|
@@ -1654,8 +1707,8 @@ Usage rules:
|
|
|
1654
1707
|
name: "chrome_type",
|
|
1655
1708
|
label: "Chrome Type",
|
|
1656
1709
|
description:
|
|
1657
|
-
"
|
|
1658
|
-
promptSnippet: "Type text at the
|
|
1710
|
+
"Type using Chrome's real input. With a snapshot uid or CSS selector, chrome_type clicks the field and moves the caret to the end, so text is appended to existing content; without one it types at the current caret of the focused element. It does NOT replace existing text. Use chrome_fill (or replace=true) to replace a field's value. Contenteditables use one native text insertion; other fields use key events. Set perCharacter=true for editors needing individual keydown events. The result reports the field value before/after and insertedAt (empty|caret-end|caret-middle|replaced-selection|replaced-all), with a warning when text was spliced into existing content. Pass includeSnapshot=true to verify after typing; if the action starts a navigation, the snapshot waits (up to 5s) for it to load.",
|
|
1711
|
+
promptSnippet: "Type text in Chrome; a uid/selector target appends at the end (does not replace existing text; use chrome_fill or replace=true).",
|
|
1659
1712
|
parameters: Type.Object({
|
|
1660
1713
|
text: Type.String(),
|
|
1661
1714
|
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
|
|
@@ -1695,7 +1748,7 @@ Usage rules:
|
|
|
1695
1748
|
selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })),
|
|
1696
1749
|
perCharacter: Type.Optional(Type.Boolean({ default: false, description: "Send individual key events even in contenteditables. Default: one native text insertion for contenteditables; key events for other fields." })),
|
|
1697
1750
|
submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })),
|
|
1698
|
-
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked
|
|
1751
|
+
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked (extension overlay, debugger failure, or a hidden tab that ignores trusted input)." })),
|
|
1699
1752
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, include a fresh chrome_snapshot result after filling." })),
|
|
1700
1753
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
|
1701
1754
|
targetId: Type.Optional(Type.String()),
|
|
@@ -1845,6 +1898,7 @@ Usage rules:
|
|
|
1845
1898
|
format: Type.Optional(StringEnum(imageFormatValues)),
|
|
1846
1899
|
quality: Type.Optional(Type.Number({ minimum: 0, maximum: 100, description: "JPEG quality 0-100." })),
|
|
1847
1900
|
fullPage: Type.Optional(Type.Boolean({ description: "Capture full-page tiles plus a JSON manifest. Temporarily scrolls the target page; does not activate background tabs." })),
|
|
1901
|
+
retentionDays: Type.Optional(Type.Number({ minimum: 0, description: "Default-folder captures older than this many days are pruned (the newest 20 are always kept). Default 7; 0 disables pruning. Explicit paths are never pruned." })),
|
|
1848
1902
|
targetId: Type.Optional(Type.String()),
|
|
1849
1903
|
urlIncludes: Type.Optional(Type.String()),
|
|
1850
1904
|
titleIncludes: Type.Optional(Type.String()),
|
|
@@ -1855,8 +1909,10 @@ Usage rules:
|
|
|
1855
1909
|
async execute(_id, params, signal, _onUpdate, ctx: ExtensionContext): Promise<ToolTextResult> {
|
|
1856
1910
|
const format = params.format ?? "png";
|
|
1857
1911
|
const cwd = workspaceCwd(ctx);
|
|
1858
|
-
const
|
|
1912
|
+
const screenshotDir = join(cwd, ".pi", "chrome-screenshots");
|
|
1913
|
+
const defaultPath = join(screenshotDir, `${new Date().toISOString().replace(/[:.]/g, "-")}.${format}`);
|
|
1859
1914
|
const outputPath = params.path ? resolve(cwd, params.path) : defaultPath;
|
|
1915
|
+
const prune = () => (params.path ? Promise.resolve(0) : pruneScreenshotDir(screenshotDir, params.retentionDays));
|
|
1860
1916
|
const result = (await authorizedBridgeSend("page.screenshot", params, params.fullPage ? 120_000 : DEFAULT_TIMEOUT_MS, signal)) as {
|
|
1861
1917
|
dataUrl?: string;
|
|
1862
1918
|
method?: string;
|
|
@@ -1880,6 +1936,7 @@ Usage rules:
|
|
|
1880
1936
|
manifest.push({ path: tilePath, y: tile.y });
|
|
1881
1937
|
}
|
|
1882
1938
|
await writeFile(outputPath + ".json", JSON.stringify({ width, height, viewportHeight, dpr, tiles: manifest }, null, 2));
|
|
1939
|
+
await prune();
|
|
1883
1940
|
return {
|
|
1884
1941
|
content: [{ type: "text", text: `Saved ${result.tiles.length} full-page tile(s) for ${width}×${height}px page. Manifest: ${outputPath}.json` }],
|
|
1885
1942
|
details: { manifest: outputPath + ".json", tiles: manifest, dimensions: result.dimensions, tab: result.tab, method: result.method } as unknown as Record<string, unknown>,
|
|
@@ -1888,6 +1945,7 @@ Usage rules:
|
|
|
1888
1945
|
if (!result.dataUrl) throw new Error("Screenshot returned no dataUrl");
|
|
1889
1946
|
const base64 = result.dataUrl.replace(/^data:image\/(?:png|jpeg);base64,/, "");
|
|
1890
1947
|
await writeFile(outputPath, Buffer.from(base64, "base64"));
|
|
1948
|
+
await prune();
|
|
1891
1949
|
return { content: [{ type: "text", text: `Saved Chrome screenshot to ${outputPath}` }], details: { path: outputPath, format, tab: result.tab, method: result.method } };
|
|
1892
1950
|
},
|
|
1893
1951
|
});
|
|
@@ -1987,12 +2045,13 @@ Usage rules:
|
|
|
1987
2045
|
pi.registerTool({
|
|
1988
2046
|
name: "chrome_upload_file",
|
|
1989
2047
|
label: "Chrome Upload File",
|
|
1990
|
-
description: "Attach local files
|
|
2048
|
+
description: "Attach local files using Chrome DevTools file-input control without opening a native file picker. Target the <input type=file>, a label/wrapper containing one, or an upload button that opens the picker (the chooser is intercepted). Selectors also search same-origin iframes. Works with React/Vue/Angular controlled inputs.",
|
|
1991
2049
|
promptSnippet: "Attach local files to a Chrome <input type=file> without opening the native file picker.",
|
|
1992
2050
|
parameters: Type.Object({
|
|
1993
2051
|
uid: Type.Optional(Type.String()),
|
|
1994
2052
|
selector: Type.Optional(Type.String()),
|
|
1995
2053
|
paths: Type.Array(Type.String(), { description: "Local absolute file paths to upload." }),
|
|
2054
|
+
domFallback: Type.Optional(Type.Boolean({ description: "Upload buttons need a click to open the (intercepted) file chooser. If true (default) and the tab is hidden, use a page-level element.click() with a user gesture instead of trusted input. Set false to require trusted input." })),
|
|
1996
2055
|
targetId: Type.Optional(Type.String()),
|
|
1997
2056
|
urlIncludes: Type.Optional(Type.String()),
|
|
1998
2057
|
titleIncludes: Type.Optional(Type.String()),
|
|
@@ -2001,8 +2060,12 @@ Usage rules:
|
|
|
2001
2060
|
async execute(_id, params, signal, _onUpdate, ctx): Promise<ToolTextResult> {
|
|
2002
2061
|
const cwd = workspaceCwd(ctx);
|
|
2003
2062
|
const paths = params.paths.map((p) => resolve(cwd, p));
|
|
2004
|
-
const result = await authorizedBridgeSend("page.upload", { ...params, paths }, DEFAULT_TIMEOUT_MS, signal);
|
|
2005
|
-
|
|
2063
|
+
const result = (await authorizedBridgeSend("page.upload", { ...params, paths }, DEFAULT_TIMEOUT_MS, signal)) as { mode?: string; trigger?: string; inFrame?: boolean } | undefined;
|
|
2064
|
+
const via = result?.mode === "file-chooser"
|
|
2065
|
+
? ` via intercepted file chooser (${result.trigger === "activation-click" ? "page-level click because the tab is hidden" : "trusted click"})`
|
|
2066
|
+
: "";
|
|
2067
|
+
const frame = result?.inFrame ? " inside an iframe" : "";
|
|
2068
|
+
return { content: [{ type: "text", text: `Uploaded ${paths.length} file(s) to ${params.uid ?? params.selector}${frame}${via}` }], details: { result: result as Json } };
|
|
2006
2069
|
},
|
|
2007
2070
|
});
|
|
2008
2071
|
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-chrome",
|
|
3
|
-
"version": "0.15.
|
|
3
|
+
"version": "0.15.54",
|
|
4
4
|
"scripts": {
|
|
5
|
-
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && node test-suite/unit/session-cleanup.test.mjs && node test-suite/unit/background-policy.test.mjs && node test-suite/unit/input-reliability.test.mjs && node test-suite/unit/chrome-command.test.mjs && node test-suite/unit/bridge-resilience.test.mjs && node test-suite/unit/cdp-passthrough.test.mjs && node test-suite/unit/type-evidence.test.mjs",
|
|
5
|
+
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && node test-suite/unit/session-cleanup.test.mjs && node test-suite/unit/background-policy.test.mjs && node test-suite/unit/input-reliability.test.mjs && node test-suite/unit/chrome-command.test.mjs && node test-suite/unit/bridge-resilience.test.mjs && node test-suite/unit/cdp-passthrough.test.mjs && node test-suite/unit/type-evidence.test.mjs && node test-suite/unit/hidden-input.test.mjs",
|
|
6
6
|
"version": "node scripts/sync-manifest-version.js",
|
|
7
7
|
"prepublishOnly": "node scripts/sync-manifest-version.js"
|
|
8
8
|
},
|
|
@@ -50,7 +50,7 @@ function piHarness({ session = "alpha", send } = {}) {
|
|
|
50
50
|
formatChromeInspect: JSON.stringify, summarizeActionResult: () => "", formatIncludedSnapshotText: (_r, text) => text,
|
|
51
51
|
describeTypeEvidence: () => [], formatCdpResult: (_method, value) => ({ content: [{ type: "text", text: JSON.stringify(value) }], details: { value } }),
|
|
52
52
|
workspaceCwd: () => ctx.cwd, ...path,
|
|
53
|
-
mkdir: async () => {}, writeFile: async (...args) => writes.push(args),
|
|
53
|
+
mkdir: async () => {}, writeFile: async (...args) => writes.push(args), pruneScreenshotDir: async () => 0,
|
|
54
54
|
};
|
|
55
55
|
const registrations = indexSource.slice(indexSource.indexOf("function registerChromeTools(pi:"), indexSource.lastIndexOf("\n}"));
|
|
56
56
|
vm.runInNewContext(stripTypeScriptTypes([
|
|
@@ -292,6 +292,7 @@ function workerHarness({ withWindows = true } = {}) {
|
|
|
292
292
|
w, chrome, calls, tabs, focusedWindow: () => focusedWindow,
|
|
293
293
|
cdpResult: (method) => method === "Page.captureScreenshot" ? { data: "c2NyZWVuc2hvdA==" }
|
|
294
294
|
: method === "DOM.requestNode" ? { nodeId: 1 }
|
|
295
|
+
: method === "Runtime.callFunctionOn" ? { result: { type: "object", value: { tag: "INPUT", isFile: true } } }
|
|
295
296
|
: { result: { type: "boolean", value: true, objectId: "file-input" } },
|
|
296
297
|
};
|
|
297
298
|
return h;
|
|
@@ -0,0 +1,279 @@
|
|
|
1
|
+
// Hidden-tab input guard, full-screen automation targets, input.debug without a target, upload
|
|
2
|
+
// targets (label/wrapper/iframe/file-chooser), and screenshot pruning. Chrome and pages are mocked;
|
|
3
|
+
// the hidden-page behavior itself (Chrome drops trusted input) was measured live on Chrome/macOS.
|
|
4
|
+
import assert from "node:assert/strict";
|
|
5
|
+
import fs from "node:fs";
|
|
6
|
+
import os from "node:os";
|
|
7
|
+
import path from "node:path";
|
|
8
|
+
import vm from "node:vm";
|
|
9
|
+
import { stripTypeScriptTypes } from "node:module";
|
|
10
|
+
import { test } from "node:test";
|
|
11
|
+
|
|
12
|
+
const workerSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/browser-extension/service_worker.js", import.meta.url), "utf8");
|
|
13
|
+
const indexSource = fs.readFileSync(new URL("../../extensions/chrome-profile-bridge/index.ts", import.meta.url), "utf8");
|
|
14
|
+
const clone = (value) => (value === undefined ? undefined : JSON.parse(JSON.stringify(value)));
|
|
15
|
+
|
|
16
|
+
function worker({ visibility = ["visible"], tab = { id: 2, windowId: 1, active: false, url: "https://fixture.test/" }, win = { id: 1, state: "normal" } } = {}) {
|
|
17
|
+
const calls = [];
|
|
18
|
+
const listener = { addListener() {}, removeListener() {} };
|
|
19
|
+
const chrome = {
|
|
20
|
+
runtime: { id: "test", getManifest: () => ({ version: "0.0.0" }), onInstalled: listener, onStartup: listener },
|
|
21
|
+
alarms: { create() {}, onAlarm: listener }, action: { onClicked: listener }, webNavigation: { onCommitted: listener },
|
|
22
|
+
debugger: { onDetach: listener, onEvent: listener, getTargets: (cb) => cb([]) },
|
|
23
|
+
storage: { session: { get: async () => ({}), set: async () => {} } },
|
|
24
|
+
scripting: { executeScript: async (options) => { calls.push({ method: "scripting.executeScript", options }); return [{ result: { tag: "BUTTON", url: tab.url } }]; } },
|
|
25
|
+
tabs: {
|
|
26
|
+
get: async (id) => (id === tab.id ? clone(tab) : Promise.reject(new Error(`No tab with id: ${id}`))),
|
|
27
|
+
query: async () => [clone(tab)],
|
|
28
|
+
create: async (props) => { calls.push({ method: "tabs.create", props }); return { id: 77, windowId: 1, url: props.url }; },
|
|
29
|
+
},
|
|
30
|
+
windows: {
|
|
31
|
+
get: async () => clone(win),
|
|
32
|
+
getLastFocused: async () => clone(win),
|
|
33
|
+
create: async (props) => { calls.push({ method: "windows.create", props }); return { id: 9, tabs: [{ id: 90, windowId: 9 }] }; },
|
|
34
|
+
},
|
|
35
|
+
};
|
|
36
|
+
const w = { chrome, console, setTimeout, clearTimeout, setInterval: () => 0, navigator: { userAgent: "unit-test" }, fetch: async () => { throw new Error("offline"); } };
|
|
37
|
+
w.self = w;
|
|
38
|
+
vm.runInNewContext(workerSource, w);
|
|
39
|
+
w.sleep = async () => {};
|
|
40
|
+
w.getTabByParams = async () => clone(tab);
|
|
41
|
+
w.attachDebugger = async () => {};
|
|
42
|
+
w.cdpMoveTo = async () => {};
|
|
43
|
+
w.resolveTargetInTab = async () => ({ found: true, x: 40, y: 50, rect: { left: 10, top: 10, width: 100, height: 20 }, tag: "INPUT" });
|
|
44
|
+
let probes = 0;
|
|
45
|
+
w.cdp = async (_tabId, method, params = {}) => {
|
|
46
|
+
calls.push({ method, params: clone(params) });
|
|
47
|
+
if (method === "Runtime.evaluate" && params.expression === "document.visibilityState") {
|
|
48
|
+
const state = visibility[Math.min(probes++, visibility.length - 1)];
|
|
49
|
+
if (state instanceof Error) throw state;
|
|
50
|
+
return { result: { type: "string", value: state } };
|
|
51
|
+
}
|
|
52
|
+
return {};
|
|
53
|
+
};
|
|
54
|
+
return { w, chrome, calls, inputs: () => calls.filter((c) => c.method.startsWith("Input.")) };
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
// ---- hidden-tab guard ----
|
|
58
|
+
test("typing into a hidden tab fails fast with the reason instead of silently dropping keys", async () => {
|
|
59
|
+
const h = worker({ visibility: ["hidden"] });
|
|
60
|
+
await assert.rejects(h.w.dispatch("page.type", { selector: "#q", text: "hi", background: true }), (error) => {
|
|
61
|
+
assert.match(error.message, /chrome\.type: tab 2 is hidden because it is an inactive tab in its window/);
|
|
62
|
+
assert.match(error.message, /background:false/);
|
|
63
|
+
return true;
|
|
64
|
+
});
|
|
65
|
+
assert.equal(h.inputs().length, 0, "no trusted input sent to a hidden page");
|
|
66
|
+
});
|
|
67
|
+
|
|
68
|
+
for (const [action, params] of [["key", { key: "Enter" }], ["hover", { x: 1, y: 1 }], ["scroll", { deltaY: 10 }], ["tap", { x: 1, y: 1 }], ["drag", { fromX: 1, fromY: 1, toX: 2, toY: 2 }]]) {
|
|
69
|
+
test(`${action} refuses hidden tabs before sending input`, async () => {
|
|
70
|
+
const h = worker({ visibility: ["hidden"] });
|
|
71
|
+
await assert.rejects(h.w.dispatch(`page.${action}`, { ...params, background: true }), new RegExp(`chrome\\.${action}: tab 2 is hidden`));
|
|
72
|
+
assert.equal(h.inputs().length, 0);
|
|
73
|
+
});
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
test("click and fill use their DOM fallback immediately for hidden tabs, or reject with domFallback:false", async () => {
|
|
77
|
+
const click = worker({ visibility: ["hidden"] });
|
|
78
|
+
const clicked = await click.w.dispatch("page.click", { selector: "button", background: true });
|
|
79
|
+
assert.equal(clicked.input, "dom-fallback");
|
|
80
|
+
assert.match(clicked.reason, /hidden/);
|
|
81
|
+
assert.equal(click.inputs().length, 0);
|
|
82
|
+
await assert.rejects(worker({ visibility: ["hidden"] }).w.dispatch("page.click", { selector: "button", background: true, domFallback: false }), /is hidden/);
|
|
83
|
+
|
|
84
|
+
const fill = worker({ visibility: ["hidden"] });
|
|
85
|
+
fill.w.domFillFallback = async (_tabId, _params, cause) => ({ input: "dom-fallback", reason: String(cause.message) });
|
|
86
|
+
const filled = await fill.w.dispatch("page.fill", { selector: "#q", text: "x", background: true });
|
|
87
|
+
assert.equal(filled.input, "dom-fallback");
|
|
88
|
+
assert.match(filled.reason, /hidden/);
|
|
89
|
+
assert.equal(fill.inputs().length, 0);
|
|
90
|
+
});
|
|
91
|
+
|
|
92
|
+
test("hidden reasons distinguish minimized and macOS full-screen windows", async () => {
|
|
93
|
+
const minimized = worker({ visibility: ["hidden"], win: { id: 1, state: "minimized" } });
|
|
94
|
+
await assert.rejects(minimized.w.dispatch("page.key", { key: "a", background: true }), /its window is minimized/);
|
|
95
|
+
const fullscreen = worker({ visibility: ["hidden"], tab: { id: 2, windowId: 1, active: true, url: "https://fixture.test/" }, win: { id: 1, state: "fullscreen" } });
|
|
96
|
+
await assert.rejects(fullscreen.w.dispatch("page.key", { key: "a", background: true }), (error) => {
|
|
97
|
+
assert.match(error.message, /behind a full-screen window/);
|
|
98
|
+
assert.equal(error.windowState, "fullscreen");
|
|
99
|
+
return true;
|
|
100
|
+
});
|
|
101
|
+
});
|
|
102
|
+
|
|
103
|
+
test("visible pages, failed probes, and foreground tabs that become visible proceed with trusted input", async () => {
|
|
104
|
+
const visible = worker();
|
|
105
|
+
assert.equal((await visible.w.dispatch("page.key", { key: "a", background: true })).input, "chrome");
|
|
106
|
+
const unknown = worker({ visibility: [new Error("probe failed")] });
|
|
107
|
+
assert.equal((await unknown.w.dispatch("page.key", { key: "a", background: true })).input, "chrome");
|
|
108
|
+
const foreground = worker({ visibility: ["hidden", "hidden", "visible"] });
|
|
109
|
+
foreground.w.bringToFront = async (tab) => tab;
|
|
110
|
+
assert.equal((await foreground.w.dispatch("page.key", { key: "a", foreground: true })).input, "chrome");
|
|
111
|
+
});
|
|
112
|
+
|
|
113
|
+
// ---- automation target placement ----
|
|
114
|
+
test("automation targets use a background tab, not a new window, while the user's window is full screen", async () => {
|
|
115
|
+
const full = worker({ win: { id: 1, state: "fullscreen" } });
|
|
116
|
+
await full.w.createAutomationTarget("s1", undefined);
|
|
117
|
+
assert.equal(full.calls.some((c) => c.method === "windows.create"), false);
|
|
118
|
+
assert.deepEqual(clone(full.calls.find((c) => c.method === "tabs.create").props), { url: "about:blank", active: false });
|
|
119
|
+
|
|
120
|
+
const normal = worker({ win: { id: 1, state: "maximized" } });
|
|
121
|
+
await normal.w.createAutomationTarget("s2", undefined);
|
|
122
|
+
assert.deepEqual(clone(normal.calls.find((c) => c.method === "windows.create").props), { url: "about:blank", focused: false });
|
|
123
|
+
});
|
|
124
|
+
|
|
125
|
+
// ---- input.debug ----
|
|
126
|
+
test("input.debug without a target does not call tabs.get(-1) and reports the session's own tab", async () => {
|
|
127
|
+
const h = worker();
|
|
128
|
+
h.chrome.tabs.get = async (id) => {
|
|
129
|
+
if (!Number.isInteger(id) || id < 0) throw new Error("Value must be at least 0.");
|
|
130
|
+
return id === 2 ? { id: 2, windowId: 1, url: "https://fixture.test/" } : Promise.reject(new Error("missing"));
|
|
131
|
+
};
|
|
132
|
+
const none = await h.w.dispatch("input.debug", {});
|
|
133
|
+
assert.equal(none.resolvedTab, null);
|
|
134
|
+
h.w.resolveOwnedAutomationTarget = async () => ({ id: 2 });
|
|
135
|
+
const owned = await h.w.dispatch("input.debug", { sessionKey: "s" });
|
|
136
|
+
assert.equal(owned.resolvedTab.id, 2);
|
|
137
|
+
});
|
|
138
|
+
|
|
139
|
+
// ---- upload targets ----
|
|
140
|
+
function uploadWorker({ visibility = "visible", chooserMode = "selectMultiple", opensChooser = true } = {}) {
|
|
141
|
+
const h = worker({ visibility: [visibility] });
|
|
142
|
+
const fileInput = { tagName: "INPUT", type: "file", multiple: true, isConnected: true };
|
|
143
|
+
const frameInput = { tagName: "INPUT", type: "file", multiple: false, isConnected: true };
|
|
144
|
+
const button = { tagName: "BUTTON", isConnected: true, querySelectorAll: () => [] };
|
|
145
|
+
const label = { tagName: "LABEL", control: fileInput, isConnected: true, querySelectorAll: () => [] };
|
|
146
|
+
const dropzone = { tagName: "DIV", isConnected: true, querySelectorAll: () => [fileInput] };
|
|
147
|
+
const frameDoc = { querySelector: (s) => (s === "#frame-file" ? frameInput : null), querySelectorAll: () => [] };
|
|
148
|
+
const doc = {
|
|
149
|
+
querySelector: (s) => ({ "#file": fileInput, "#button": button, "#label": label, "#drop": dropzone }[s] ?? null),
|
|
150
|
+
querySelectorAll: (s) => (s === "iframe,frame" ? [{ contentDocument: frameDoc }] : []),
|
|
151
|
+
};
|
|
152
|
+
const page = vm.createContext({ document: doc, __PI_CHROME_STATE__: { elements: {} } });
|
|
153
|
+
page.window = page;
|
|
154
|
+
const objects = new Map();
|
|
155
|
+
let nextObject = 1;
|
|
156
|
+
const base = h.w.cdp;
|
|
157
|
+
h.w.cdp = async (tabId, method, params = {}) => {
|
|
158
|
+
if (method === "Runtime.evaluate" && params.objectGroup === "pi-chrome-upload") {
|
|
159
|
+
h.calls.push({ method, params: {} });
|
|
160
|
+
try {
|
|
161
|
+
const el = vm.runInContext(params.expression, page);
|
|
162
|
+
const objectId = `obj-${nextObject++}`;
|
|
163
|
+
objects.set(objectId, el);
|
|
164
|
+
return { result: { objectId } };
|
|
165
|
+
} catch (error) {
|
|
166
|
+
return { exceptionDetails: { text: error.message } };
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
const result = await base(tabId, method, params);
|
|
170
|
+
if (method === "Runtime.callFunctionOn" && /getBoundingClientRect/.test(params.functionDeclaration)) {
|
|
171
|
+
const el = objects.get(params.objectId);
|
|
172
|
+
const isFile = el.tagName === "INPUT" && el.type === "file";
|
|
173
|
+
return { result: { value: { tag: el.tagName, isFile, multiple: isFile ? el.multiple : undefined, inFrame: el === frameInput, x: 50, y: 60, width: 80, height: 20 } } };
|
|
174
|
+
}
|
|
175
|
+
const clicked = (method === "Input.dispatchMouseEvent" && params.type === "mouseReleased") || (method === "Runtime.callFunctionOn" && params.userGesture === true);
|
|
176
|
+
if (clicked && opensChooser) setTimeout(() => h.w.handleDebuggerEvent({ tabId: 2 }, "Page.fileChooserOpened", { backendNodeId: 42, mode: chooserMode }), 0);
|
|
177
|
+
return result;
|
|
178
|
+
};
|
|
179
|
+
h.w.waitForFileChooser = ((original) => (tabId) => original(tabId, 200))(h.w.waitForFileChooser);
|
|
180
|
+
return h;
|
|
181
|
+
}
|
|
182
|
+
const setFiles = (h) => h.calls.filter((c) => c.method === "DOM.setFileInputFiles").map((c) => c.params);
|
|
183
|
+
|
|
184
|
+
test("upload resolves labels and single-input wrappers to the file input without clicking", async () => {
|
|
185
|
+
for (const selector of ["#file", "#label", "#drop"]) {
|
|
186
|
+
const h = uploadWorker();
|
|
187
|
+
const result = await h.w.dispatch("page.upload", { selector, paths: ["/a.txt", "/b.txt"], background: true });
|
|
188
|
+
assert.equal(result.mode, "file-input", selector);
|
|
189
|
+
assert.equal(setFiles(h).length, 1);
|
|
190
|
+
assert.equal(h.inputs().length, 0, "no click for a real file input");
|
|
191
|
+
}
|
|
192
|
+
});
|
|
193
|
+
|
|
194
|
+
test("upload finds file inputs inside same-origin iframes and enforces single-file inputs", async () => {
|
|
195
|
+
const h = uploadWorker();
|
|
196
|
+
const result = await h.w.dispatch("page.upload", { selector: "#frame-file", paths: ["/a.txt"], background: true });
|
|
197
|
+
assert.equal(result.mode, "file-input");
|
|
198
|
+
assert.equal(result.inFrame, true);
|
|
199
|
+
await assert.rejects(uploadWorker().w.dispatch("page.upload", { selector: "#frame-file", paths: ["/a.txt", "/b.txt"], background: true }), /accepts one file but 2 paths/);
|
|
200
|
+
});
|
|
201
|
+
|
|
202
|
+
test("upload buttons open an intercepted file chooser with a trusted click on visible pages", async () => {
|
|
203
|
+
const h = uploadWorker();
|
|
204
|
+
const result = await h.w.dispatch("page.upload", { selector: "#button", paths: ["/a.txt"], background: true });
|
|
205
|
+
assert.equal(result.mode, "file-chooser");
|
|
206
|
+
assert.equal(result.trigger, "chrome");
|
|
207
|
+
assert.deepEqual(clone(setFiles(h)), [{ backendNodeId: 42, files: ["/a.txt"] }]);
|
|
208
|
+
assert.deepEqual(h.calls.filter((c) => c.method === "Page.setInterceptFileChooserDialog").map((c) => c.params.enabled), [true, false]);
|
|
209
|
+
assert.ok(h.inputs().some((c) => c.params.type === "mousePressed"));
|
|
210
|
+
});
|
|
211
|
+
|
|
212
|
+
test("hidden pages use a user-gesture element.click() for upload buttons unless domFallback:false", async () => {
|
|
213
|
+
const h = uploadWorker({ visibility: "hidden" });
|
|
214
|
+
const result = await h.w.dispatch("page.upload", { selector: "#button", paths: ["/a.txt"], background: true });
|
|
215
|
+
assert.equal(result.trigger, "activation-click");
|
|
216
|
+
assert.match(result.reason, /hidden/);
|
|
217
|
+
assert.equal(h.inputs().length, 0);
|
|
218
|
+
const strict = uploadWorker({ visibility: "hidden" });
|
|
219
|
+
await assert.rejects(strict.w.dispatch("page.upload", { selector: "#button", paths: ["/a.txt"], background: true, domFallback: false }), /is hidden/);
|
|
220
|
+
assert.equal(setFiles(strict).length, 0);
|
|
221
|
+
});
|
|
222
|
+
|
|
223
|
+
test("chooser failures attach nothing and always disable interception", async () => {
|
|
224
|
+
const single = uploadWorker({ chooserMode: "selectSingle" });
|
|
225
|
+
await assert.rejects(single.w.dispatch("page.upload", { selector: "#button", paths: ["/a.txt", "/b.txt"], background: true }), /accepts one file/);
|
|
226
|
+
assert.equal(setFiles(single).length, 0);
|
|
227
|
+
const none = uploadWorker({ opensChooser: false });
|
|
228
|
+
await assert.rejects(none.w.dispatch("page.upload", { selector: "#button", paths: ["/a.txt"], background: true }), /no file chooser opened/);
|
|
229
|
+
assert.deepEqual(none.calls.filter((c) => c.method === "Page.setInterceptFileChooserDialog").map((c) => c.params.enabled), [true, false]);
|
|
230
|
+
assert.equal(none.calls.at(-1).method, "Runtime.releaseObject");
|
|
231
|
+
});
|
|
232
|
+
|
|
233
|
+
test("file chooser events attached by targetId route to the right tab", async () => {
|
|
234
|
+
const h = worker();
|
|
235
|
+
vm.runInContext('attachedTabs.set(5, { debuggee: { targetId: "T5" } })', h.w);
|
|
236
|
+
const pending = h.w.waitForFileChooser(5, 1000);
|
|
237
|
+
h.w.handleDebuggerEvent({ targetId: "T5" }, "Page.fileChooserOpened", { backendNodeId: 7, mode: "selectSingle" });
|
|
238
|
+
assert.deepEqual(clone(await pending.promise), { backendNodeId: 7, mode: "selectSingle" });
|
|
239
|
+
});
|
|
240
|
+
|
|
241
|
+
// ---- screenshot pruning ----
|
|
242
|
+
const pruneSource = indexSource.slice(indexSource.indexOf("const SCREENSHOT_NAME_RE"), indexSource.indexOf("// End screenshot pruning."));
|
|
243
|
+
const prune = vm.runInNewContext(`${stripTypeScriptTypes(pruneSource)}; ({ screenshotFilesToPrune, pruneScreenshotDir })`, {
|
|
244
|
+
readdir: fs.promises.readdir, unlink: fs.promises.unlink, join: path.join, Date,
|
|
245
|
+
});
|
|
246
|
+
const DAY = 24 * 60 * 60 * 1000;
|
|
247
|
+
const stamp = (ms) => new Date(ms).toISOString().replace(/[:.]/g, "-");
|
|
248
|
+
|
|
249
|
+
test("screenshot pruning keeps the newest 20 captures and anything inside the retention window", () => {
|
|
250
|
+
const now = Date.parse("2026-09-26T12:00:00.000Z");
|
|
251
|
+
const names = [];
|
|
252
|
+
for (let i = 0; i < 25; i++) names.push(`${stamp(now - (30 + i) * DAY)}.png`);
|
|
253
|
+
const fullPage = stamp(now - 60 * DAY);
|
|
254
|
+
names.push(`${fullPage}-tile0.png`, `${fullPage}-tile1.png`, `${fullPage}.png.json`);
|
|
255
|
+
names.push(`${stamp(now - DAY)}.jpeg`, "my-notes.png", "2026-01-01.png", "keep.json");
|
|
256
|
+
const pruned = prune.screenshotFilesToPrune(names, now);
|
|
257
|
+
// 25 old singles + 1 full-page capture + 1 recent = 27 captures; the newest 20 survive. The recent
|
|
258
|
+
// capture is newest, so 19 old singles survive and 6 old singles + the full-page capture go.
|
|
259
|
+
assert.equal(pruned.length, 6 + 3);
|
|
260
|
+
for (const suffix of ["-tile0.png", "-tile1.png", ".png.json"]) assert.ok(pruned.includes(`${fullPage}${suffix}`));
|
|
261
|
+
for (const name of ["my-notes.png", "2026-01-01.png", "keep.json", `${stamp(now - DAY)}.jpeg`]) assert.ok(!pruned.includes(name), name);
|
|
262
|
+
|
|
263
|
+
const recent = Array.from({ length: 30 }, (_, i) => `${stamp(now - i * 60_000)}.png`);
|
|
264
|
+
assert.equal(prune.screenshotFilesToPrune(recent, now).length, 0, "nothing inside the retention window is removed");
|
|
265
|
+
assert.equal(prune.screenshotFilesToPrune(names, now, 0).length, 0, "retentionDays:0 disables pruning");
|
|
266
|
+
assert.equal(prune.screenshotFilesToPrune(names, now, 55).length, 3, "a longer window keeps more");
|
|
267
|
+
});
|
|
268
|
+
|
|
269
|
+
test("pruneScreenshotDir deletes only eligible files and never throws", async () => {
|
|
270
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "pi-chrome-shots-"));
|
|
271
|
+
const now = Date.now();
|
|
272
|
+
for (let i = 0; i < 22; i++) fs.writeFileSync(path.join(dir, `${stamp(now - (10 + i) * DAY)}.png`), "x");
|
|
273
|
+
fs.writeFileSync(path.join(dir, "mine.png"), "x");
|
|
274
|
+
assert.equal(await prune.pruneScreenshotDir(dir), 2);
|
|
275
|
+
assert.equal(fs.readdirSync(dir).length, 21);
|
|
276
|
+
assert.ok(fs.existsSync(path.join(dir, "mine.png")));
|
|
277
|
+
assert.equal(await prune.pruneScreenshotDir(path.join(dir, "missing")), 0);
|
|
278
|
+
fs.rmSync(dir, { recursive: true, force: true });
|
|
279
|
+
});
|
|
@@ -17,6 +17,14 @@ function harness({ tag = "DIV", editable = true, initial = "", nodeId = 7 } = {}
|
|
|
17
17
|
isContentEditable: editable, isConnected: true, textContent: initial,
|
|
18
18
|
scrollIntoView() {}, getBoundingClientRect: () => ({ left: 10, top: 10, width: 200, height: 60 }),
|
|
19
19
|
contains: (el) => el === element,
|
|
20
|
+
querySelectorAll: () => [],
|
|
21
|
+
listeners: [], events: [],
|
|
22
|
+
addEventListener(type, fn) { this.listeners.push({ type, fn }); },
|
|
23
|
+
dispatchEvent(event) {
|
|
24
|
+
this.events.push(event.type);
|
|
25
|
+
for (const l of this.listeners.splice(0)) if (l.type === event.type) l.fn(event); else this.listeners.push(l);
|
|
26
|
+
return true;
|
|
27
|
+
},
|
|
20
28
|
};
|
|
21
29
|
const selection = {
|
|
22
30
|
removeAllRanges() { selectAll = false; },
|
|
@@ -33,6 +41,9 @@ function harness({ tag = "DIV", editable = true, initial = "", nodeId = 7 } = {}
|
|
|
33
41
|
__PI_CHROME_STATE__: { elements: { "el-1": element } },
|
|
34
42
|
});
|
|
35
43
|
page.window = page;
|
|
44
|
+
page.document.defaultView = page;
|
|
45
|
+
page.document.querySelectorAll = () => [];
|
|
46
|
+
element.ownerDocument = page.document;
|
|
36
47
|
const listener = { addListener() {}, removeListener() {} };
|
|
37
48
|
const chrome = {
|
|
38
49
|
runtime: { id: "test", getManifest: () => ({ version: "0.0.0" }), onInstalled: listener, onStartup: listener },
|
|
@@ -71,8 +82,15 @@ function harness({ tag = "DIV", editable = true, initial = "", nodeId = 7 } = {}
|
|
|
71
82
|
return { exceptionDetails: { text: error.message, exception: { description: error.message } } };
|
|
72
83
|
}
|
|
73
84
|
}
|
|
85
|
+
if (method === "Runtime.callFunctionOn" && params.objectId === "upload-object") {
|
|
86
|
+
const fn = vm.runInContext(`(${params.functionDeclaration})`, page);
|
|
87
|
+
return { result: { value: clone(fn.call(element) ?? null) } };
|
|
88
|
+
}
|
|
74
89
|
if (method === "DOM.requestNode") return h.nodeResult;
|
|
75
|
-
if (method === "DOM.setFileInputFiles")
|
|
90
|
+
if (method === "DOM.setFileInputFiles") {
|
|
91
|
+
files.push(...params.files);
|
|
92
|
+
if (h.nativeChange) { element.dispatchEvent({ type: "input" }); element.dispatchEvent({ type: "change" }); }
|
|
93
|
+
}
|
|
76
94
|
if (method === "Input.insertText") element.textContent += params.text;
|
|
77
95
|
if (method === "Input.dispatchKeyEvent" && params.type === "keyDown") {
|
|
78
96
|
if (params.key === "Delete") {
|
|
@@ -88,6 +106,7 @@ function harness({ tag = "DIV", editable = true, initial = "", nodeId = 7 } = {}
|
|
|
88
106
|
element.textContent = params.text;
|
|
89
107
|
return { input: "dom-fallback" };
|
|
90
108
|
};
|
|
109
|
+
page.Event = class { constructor(type) { this.type = type; } };
|
|
91
110
|
const h = {
|
|
92
111
|
worker, chrome, page, calls, element, selected, files, tabLookups, failures: new Map(), nodeResult: { nodeId },
|
|
93
112
|
call: (action, params = {}) => worker.dispatch(`page.${action}`, { targetId: "2", background: true, ...params }),
|
|
@@ -100,6 +119,7 @@ for (const [name, nodeResult] of [["node ID", { nodeId: 7 }], ["zero node ID", {
|
|
|
100
119
|
test(`upload accepts ${name} and releases the remote object`, async () => {
|
|
101
120
|
const h = harness({ tag: "INPUT", editable: false });
|
|
102
121
|
h.element.type = "file";
|
|
122
|
+
h.element.multiple = true;
|
|
103
123
|
h.nodeResult = nodeResult;
|
|
104
124
|
const result = await h.call("upload", { uid: "el-1", paths: ["/tmp/a.txt", "/tmp/b.txt"] });
|
|
105
125
|
const target = nodeResult?.nodeId ? { nodeId: 7 } : { objectId: "upload-object" };
|
|
@@ -112,6 +132,17 @@ for (const [name, nodeResult] of [["node ID", { nodeId: 7 }], ["zero node ID", {
|
|
|
112
132
|
});
|
|
113
133
|
}
|
|
114
134
|
|
|
135
|
+
test("upload never duplicates the change event Chrome already fired", async () => {
|
|
136
|
+
for (const nativeChange of [true, false]) {
|
|
137
|
+
const h = harness({ tag: "INPUT", editable: false });
|
|
138
|
+
h.element.type = "file";
|
|
139
|
+
h.nativeChange = nativeChange;
|
|
140
|
+
const result = await h.call("upload", { selector: "#target", paths: ["/tmp/a.txt"] });
|
|
141
|
+
assert.equal(result.events, nativeChange ? "native" : "dispatched");
|
|
142
|
+
assert.deepEqual(h.element.events.filter((t) => t === "change"), ["change"], `exactly one change (native=${nativeChange})`);
|
|
143
|
+
}
|
|
144
|
+
});
|
|
145
|
+
|
|
115
146
|
test("upload falls back on requestNode failure, not on a failed file attachment", async () => {
|
|
116
147
|
const h = harness({ tag: "INPUT", editable: false });
|
|
117
148
|
h.element.type = "file";
|
|
@@ -124,7 +155,7 @@ test("upload falls back on requestNode failure, not on a failed file attachment"
|
|
|
124
155
|
await assert.rejects(h.call("upload", { selector: "#target", paths: ["/tmp/a.txt"] }), /attachment denied/);
|
|
125
156
|
assert.equal(h.commands("DOM.setFileInputFiles").length, 1, "no object-ID retry after an uncertain file attachment");
|
|
126
157
|
assert.equal(h.commands("Runtime.releaseObject").length, 1, "cleanup still runs on error");
|
|
127
|
-
assert.equal(h.commands("Runtime.callFunctionOn").length, 0, "no notification after failed attachment");
|
|
158
|
+
assert.equal(h.commands("Runtime.callFunctionOn").filter((c) => /dispatchEvent/.test(c.params.functionDeclaration)).length, 0, "no notification after failed attachment");
|
|
128
159
|
});
|
|
129
160
|
|
|
130
161
|
test("upload releases valid node-path references on failure and tolerates notification/cleanup failure", async () => {
|
|
@@ -134,14 +165,16 @@ test("upload releases valid node-path references on failure and tolerates notifi
|
|
|
134
165
|
await assert.rejects(h.call("upload", { selector: "#target", paths: ["/tmp/a.txt"] }), /file inaccessible/);
|
|
135
166
|
assert.equal(h.commands("Runtime.releaseObject").length, 1);
|
|
136
167
|
h.failures.delete("DOM.setFileInputFiles");
|
|
137
|
-
for (const method of ["DOM.enable", "Runtime.
|
|
168
|
+
for (const method of ["DOM.enable", "Runtime.releaseObject"]) h.failures.set(method, "optional step failed");
|
|
138
169
|
const result = await h.call("upload", { selector: "#target", paths: ["/tmp/a.txt"] });
|
|
139
170
|
assert.equal(result.input, "chrome");
|
|
140
171
|
});
|
|
141
172
|
|
|
142
|
-
test("upload rejects non-
|
|
173
|
+
test("upload rejects non-trigger elements, missing targets/paths, and stale UIDs without attaching anything", async () => {
|
|
143
174
|
const h = harness();
|
|
144
|
-
|
|
175
|
+
h.worker.waitForFileChooser = () => ({ promise: Promise.reject(new Error("chrome.upload: no file chooser opened after clicking the target.")), cancel() {} });
|
|
176
|
+
await assert.rejects(h.call("upload", { selector: "#target", paths: ["/tmp/a.txt"] }), /no file chooser opened/);
|
|
177
|
+
assert.deepEqual(h.commands("Page.setInterceptFileChooserDialog").map((c) => c.params.enabled), [true, false], "intercept is always disabled again");
|
|
145
178
|
await assert.rejects(h.call("upload", { paths: ["/tmp/a.txt"] }), /selector or uid required/);
|
|
146
179
|
await assert.rejects(h.call("upload", { selector: "#target", paths: [] }), /no file paths/);
|
|
147
180
|
h.element.tagName = "INPUT";
|
|
@@ -158,7 +191,7 @@ for (const target of [{ selector: "#target" }, { uid: "el-1" }, {}]) {
|
|
|
158
191
|
const h = harness();
|
|
159
192
|
const result = await h.call("type", { ...target, text: caption });
|
|
160
193
|
assert.deepEqual(h.commands("Input.insertText").map((c) => c.params), [{ text: caption }]);
|
|
161
|
-
assert.equal(h.commands("Input.dispatchKeyEvent").length, 0);
|
|
194
|
+
assert.equal(h.commands("Input.dispatchKeyEvent").filter((c) => c.params.key !== "End").length, 0);
|
|
162
195
|
assert.equal(h.element.textContent, caption);
|
|
163
196
|
assert.equal(result.input, "chrome");
|
|
164
197
|
assert.equal(result.length, caption.length);
|
|
@@ -59,6 +59,7 @@ function harness(element, { platform = "MacIntel", tabs = {} } = {}) {
|
|
|
59
59
|
calls.push({ method, params: clone(params) });
|
|
60
60
|
if (method === "Input.dispatchKeyEvent") {
|
|
61
61
|
if ((params.commands || []).includes("selectAll")) { element.selectionStart = 0; element.selectionEnd = element.value.length; }
|
|
62
|
+
else if ((params.commands || []).includes("moveToEndOfDocument") && typeof element.value === "string") { element.selectionStart = element.selectionEnd = element.value.length; }
|
|
62
63
|
else if (params.type === "keyDown" && params.key === "Delete") {
|
|
63
64
|
if (element.selectionEnd > element.selectionStart) splice("");
|
|
64
65
|
} else if (params.type === "keyDown" && params.text && params.key !== "Enter") splice(params.text);
|
|
@@ -68,9 +69,29 @@ function harness(element, { platform = "MacIntel", tabs = {} } = {}) {
|
|
|
68
69
|
return { worker, calls, element, listeners, fire: (tabId, info) => { for (const fn of [...listeners]) fn(tabId, info); } };
|
|
69
70
|
}
|
|
70
71
|
|
|
71
|
-
test("
|
|
72
|
+
test("a targeted chrome_type moves the caret to the end after its focus click, so it appends", async () => {
|
|
72
73
|
const h = harness(makeInput({ value: "why do flamingos stand", caret: 7 }));
|
|
73
|
-
const result = await h.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: "
|
|
74
|
+
const result = await h.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: " still" });
|
|
75
|
+
assert.equal(h.element.value, "why do flamingos stand still");
|
|
76
|
+
assert.equal(result.insertedAt, "caret-end");
|
|
77
|
+
const moves = h.calls.filter((c) => (c.params.commands || []).includes("moveToEndOfDocument"));
|
|
78
|
+
assert.equal(moves.length, 1);
|
|
79
|
+
assert.equal(h.calls.indexOf(moves[0]) > h.calls.findIndex((c) => c.method === "Input.dispatchMouseEvent" && c.params.type === "mouseReleased"), true, "caret moves after the focus click");
|
|
80
|
+
});
|
|
81
|
+
|
|
82
|
+
test("replace=true does not move the caret first; untargeted typing keeps the user's caret", async () => {
|
|
83
|
+
const replace = harness(makeInput({ value: "old", caret: 1 }));
|
|
84
|
+
await replace.worker.dispatch("page.type", { targetId: "2", selector: "#q", text: "new", replace: true });
|
|
85
|
+
assert.equal(replace.calls.some((c) => (c.params.commands || []).includes("moveToEndOfDocument")), false);
|
|
86
|
+
const focused = harness(makeInput({ value: "abc", caret: 1 }));
|
|
87
|
+
await focused.worker.dispatch("page.type", { targetId: "2", text: "X" });
|
|
88
|
+
assert.equal(focused.calls.some((c) => (c.params.commands || []).includes("moveToEndOfDocument")), false);
|
|
89
|
+
assert.equal(focused.element.value, "aXbc");
|
|
90
|
+
});
|
|
91
|
+
|
|
92
|
+
test("typing into the middle of a focused prefilled field reports the splice instead of a bare count", async () => {
|
|
93
|
+
const h = harness(makeInput({ value: "why do flamingos stand", caret: 7 }));
|
|
94
|
+
const result = await h.worker.dispatch("page.type", { targetId: "2", text: "cats " });
|
|
74
95
|
assert.equal(h.element.value, "why do cats flamingos stand");
|
|
75
96
|
assert.equal(result.valueBefore, "why do flamingos stand");
|
|
76
97
|
assert.equal(result.valueAfter, "why do cats flamingos stand");
|