pi-chrome 0.15.47 → 0.15.49
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +18 -0
- package/README.md +15 -1
- package/SECURITY.md +1 -1
- package/docs/ARCHITECTURE.md +16 -4
- package/docs/COMPARISON.md +1 -1
- package/extensions/chrome-profile-bridge/browser-extension/manifest.json +1 -1
- package/extensions/chrome-profile-bridge/browser-extension/service_worker.js +161 -91
- package/extensions/chrome-profile-bridge/index.ts +97 -106
- package/package.json +2 -2
- package/test-suite/README.md +18 -0
- package/test-suite/challenges/21-keyboard-modifiers.html +5 -2
- package/test-suite/challenges/43-hard-background.html +42 -0
- package/test-suite/challenges/44-input-reliability.html +105 -0
- package/test-suite/manifest.json +119 -2
- package/test-suite/unit/automation-target.test.mjs +1 -1
- package/test-suite/unit/background-policy.test.mjs +489 -0
- package/test-suite/unit/input-reliability.test.mjs +401 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,24 @@
|
|
|
2
2
|
|
|
3
3
|
All notable user-facing changes to `pi-chrome`.
|
|
4
4
|
|
|
5
|
+
## 0.15.49 — 2026-09-10
|
|
6
|
+
|
|
7
|
+
- **Validation scope.** Node regression suites passed. Live browser validation remains incomplete: an input attempt encountered `Input.dispatchMouseEvent: Detached while handling command.`; a subsequent retest was blocked by a disconnected companion. No live-browser pass is claimed for these changes.
|
|
8
|
+
|
|
9
|
+
- **Upload node fallback.** When Chrome cannot convert a file input's remote object to a DOM node ID, use that same `objectId` directly. Release the remote object after success or failure and reject stale snapshot UIDs. No native file picker or new permissions.
|
|
10
|
+
- **Native rich-editor insertion.** `chrome_type` and `chrome_fill` use one CDP `Input.insertText` for focused contenteditables. Inputs/textareas keep individual key events; `perCharacter:true` preserves that path for rich editors needing `keydown` events. Results report `typing` as `insertText`, `keys`, or `none`. Existing DOM-fallback controls and background policy remain intact.
|
|
11
|
+
- **Complete rich-editor fill and single Enter.** Select all requested contenteditable contents before deletion, not only one paragraph. `pressEnter` sends one Enter instead of two; Enter after type/fill stays pinned to the resolved tab.
|
|
12
|
+
- **Shift-only typing.** Shift+a, Shift+1, and other US-layout printable chords now carry shifted text. Ctrl/Meta/Alt shortcuts still suppress literal insertion.
|
|
13
|
+
- **Input regressions.** Added worker fault-injection tests and challenge 44. Challenge 16 explicitly tests per-character typing; challenge 21 waits for Shift release before grading.
|
|
14
|
+
|
|
15
|
+
## 0.15.48 — 2026-09-09
|
|
16
|
+
|
|
17
|
+
- **Existing background mode is now hard background.** `/chrome background on` (still the default) overrides per-call foreground requests, keeps new tabs inactive, and blocks `chrome_tab activate`. Use the existing `/chrome background off` for foreground/watch mode; no new command or lock state.
|
|
18
|
+
- **Screenshots without tab activation.** PNG/JPEG and full-page tiles use CDP instead of `captureVisibleTab`. Debugger/capture failures never fall back to switching tabs. Old companions reject background creation/capture with a reload instruction. Screenshot tools now require debugger access.
|
|
19
|
+
- **Trusted input preserved.** Background policy does not replace Chrome input with synthetic events. It blocks explicit focus/activation, not page/native/Chrome/OS side effects; inactive-page rendering and focus-gated workflows can still vary by environment.
|
|
20
|
+
- **Regression coverage.** Added policy/worker/screenshot unit tests and challenge 43 for inactive-tab visibility plus trusted input. Full-page capture restores both scroll axes best-effort after success or failure.
|
|
21
|
+
- **Live validation and unresolved limitation.** Chrome 152/macOS checks passed for inactive tab creation, blocked activation, background PNG/JPEG/full-page capture, and scroll restoration. The trusted-click check encountered debugger detachment, then a visibility failure on retry; the cause remains unresolved and human interference was not ruled out. This release does not promise zero focus changes during trusted input. Chrome-behind-another-app and macOS Spaces behavior remain unverified.
|
|
22
|
+
|
|
5
23
|
## 0.15.47 — 2026-09-09
|
|
6
24
|
|
|
7
25
|
- **Bounded session cleanup.** On exit, Pi waits up to two seconds for cleanup before stopping the bridge. Reload preserves browser resources; revoke remains non-blocking.
|
package/README.md
CHANGED
|
@@ -74,6 +74,12 @@ Second doctor run should show all checks passing.
|
|
|
74
74
|
|
|
75
75
|
Tool parameters and gotchas are documented inline in Pi.
|
|
76
76
|
|
|
77
|
+
### Typing into rich editors
|
|
78
|
+
|
|
79
|
+
`chrome_type` and `chrome_fill` use one native CDP `Input.insertText` operation for focused contenteditables. This avoids per-character delays for long text and preserves Unicode/newlines. Ordinary inputs and textareas retain individual key events.
|
|
80
|
+
|
|
81
|
+
For an editor that needs a `keydown` for every character, pass `perCharacter:true`. Bulk insertion still uses Chrome's input system, but does not emit per-character key events or a clipboard `paste` event. Use `includeSnapshot:true` to verify the result; `chrome_fill` still honors `domFallback:false` when synthetic fallback is unwanted.
|
|
82
|
+
|
|
77
83
|
---
|
|
78
84
|
|
|
79
85
|
## Safety model
|
|
@@ -108,13 +114,21 @@ Security details: [`SECURITY.md`](./SECURITY.md). Architecture details: [`docs/A
|
|
|
108
114
|
/chrome status # connection + auth + background state
|
|
109
115
|
/chrome authorize [duration]
|
|
110
116
|
/chrome revoke
|
|
111
|
-
/chrome background on # default:
|
|
117
|
+
/chrome background on # default: hard background policy
|
|
112
118
|
/chrome background off # foreground/watch mode
|
|
113
119
|
/chrome background status
|
|
114
120
|
```
|
|
115
121
|
|
|
116
122
|
If loaded extension is older than installed `pi-chrome`, `/chrome doctor` tells you to reload it from `chrome://extensions`.
|
|
117
123
|
|
|
124
|
+
### Background policy
|
|
125
|
+
|
|
126
|
+
`/chrome background on` is enforced, not an overridable default. Per-call `background:false` cannot bring Chrome forward, new tabs stay inactive, and `chrome_tab activate` is blocked. Use the existing `/chrome background off` for foreground/watch mode; per-call `background:true` still works when that mode is off.
|
|
127
|
+
|
|
128
|
+
Screenshots use CDP without activating background tabs. Debugger/capture failures return errors, never an activation fallback. Reload both Pi and the Chrome companion after upgrading; old companions reject background tab creation/screenshots rather than silently switching tabs.
|
|
129
|
+
|
|
130
|
+
This prevents explicit pi-chrome focus/activation, not every Chrome/OS side effect. Trusted input, page popups, native prompts, debugger banners, and macOS Spaces can still affect focus. Inactive pages may throttle rendering or reject focus-gated actions. See [scope and risks](./docs/ARCHITECTURE.md#scope-and-risks).
|
|
131
|
+
|
|
118
132
|
---
|
|
119
133
|
|
|
120
134
|
## Limits
|
package/SECURITY.md
CHANGED
|
@@ -27,7 +27,7 @@ The Chrome extension under `extensions/chrome-profile-bridge/browser-extension/`
|
|
|
27
27
|
- Loopback bridge only. No remote port. No telemetry.
|
|
28
28
|
- Chrome real input layer for interactive controls.
|
|
29
29
|
- Chrome control locked by default; `/chrome authorize` unlocks current Pi session after terminal confirmation, `/chrome revoke` locks it again.
|
|
30
|
-
-
|
|
30
|
+
- Hard background mode is on by default: tools cannot override it to explicitly focus windows or activate tabs. `/chrome background off` allows foreground/watch mode. This is not a security sandbox: trusted input, page scripts, native prompts, and Chrome/OS behavior can still affect focus.
|
|
31
31
|
|
|
32
32
|
## Custom ports
|
|
33
33
|
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -46,14 +46,26 @@ To point pi-chrome at an existing tab, pass `targetId`, `urlIncludes`, or `title
|
|
|
46
46
|
|
|
47
47
|
## Background mode
|
|
48
48
|
|
|
49
|
-
|
|
49
|
+
The existing background setting is a hard session policy, enabled by default. No separate lock/unlock command is needed.
|
|
50
50
|
|
|
51
51
|
```text
|
|
52
|
-
/chrome background on #
|
|
53
|
-
/chrome background off # foreground/watch mode
|
|
52
|
+
/chrome background on # enforce no explicit window focus/tab activation
|
|
53
|
+
/chrome background off # allow foreground/watch mode
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
-
|
|
56
|
+
- With background on, per-call `background:false` and legacy `foreground:true` cannot override the policy. `chrome_tab activate` errors with instructions to ask the user to turn background off.
|
|
57
|
+
- With background off, calls may focus Chrome; per-call `background:true` still avoids explicit focus/tab activation.
|
|
58
|
+
- Policy is applied in `authorizedBridgeSend` for every tool, including `chrome_launch(url)`, tab creation, and tools without a background parameter. Each session sends its own effective `background`/`foreground` flags through the shared bridge.
|
|
59
|
+
- All worker window-focus/tab-activation writes go through a guarded helper. Background tab creation uses `active:false`; implicit automation windows remain `focused:false`.
|
|
60
|
+
- Screenshots use CDP `Page.captureScreenshot` with `fromSurface:true` and `captureBeyondViewport:false`. They target a tab, not whichever tab happens to be visible. There is no `captureVisibleTab`/activation fallback on debugger or capture failure. PNG/JPEG output is unchanged; full-page capture retains tiles plus a JSON manifest and restores scroll position best-effort, including on failure.
|
|
61
|
+
- Background tab creation and screenshots use internal `tab.new.background` / `page.screenshot.background` wire actions. Old companions reject them before changing tabs; Pi reports a reload instruction instead of retrying an unsafe legacy action. No capability-probe race or extra round trip is needed.
|
|
62
|
+
- Real CDP input and existing explicit DOM-fallback controls are unchanged. Background mode never silently substitutes synthetic input to avoid focus.
|
|
63
|
+
|
|
64
|
+
### Scope and risks
|
|
65
|
+
|
|
66
|
+
This is a policy against **explicit pi-chrome focus/activation**, not an OS focus sandbox. Page scripts (`window.open`, `window.focus`), trusted input, native dialogs, debugger banners, Chrome window/Spaces behavior, and closing an active tab can still change focus or selection. Other sessions and human actions remain independent. Requests already dispatched before a mode change keep their earlier policy.
|
|
67
|
+
|
|
68
|
+
Inactive/minimized tabs can throttle timers or rendering, and clipboard/fullscreen/other focus-gated workflows may fail. Screenshots now require debugger attachment, which can conflict with DevTools or other extensions; hidden-tab rendering can differ or be unavailable. No automatic foreground retry is allowed. Full-page capture temporarily scrolls the target page. Reload both Pi and the Chrome companion after upgrading, and live-test tab selection, OS focus, and screenshot fidelity on supported Chrome/OS versions.
|
|
57
69
|
|
|
58
70
|
## Authorization
|
|
59
71
|
|
package/docs/COMPARISON.md
CHANGED
|
@@ -134,7 +134,7 @@ If your threat model excludes extensions with broad permissions, neither approac
|
|
|
134
134
|
|
|
135
135
|
## Public benchmarks worth knowing (for axis 2 / axis 3 comparison)
|
|
136
136
|
|
|
137
|
-
Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **
|
|
137
|
+
Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **43 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering trusted input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, iframes, file uploads, strict-CSP screenshot fallback and CDP eval/snapshot bypass, dynamic waits, tab lifecycle, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count, with `core`, `conditional`, and `quality` gate buckets. That's **driver-level** grading.
|
|
138
138
|
|
|
139
139
|
For **agent-level** comparison (axis 2), the public benchmarks worth citing:
|
|
140
140
|
|
|
@@ -681,7 +681,13 @@ function cdpKeyInfo(key, shifted) {
|
|
|
681
681
|
};
|
|
682
682
|
if (SPECIAL[key]) return { key, ...SPECIAL[key] };
|
|
683
683
|
if (key.length === 1) {
|
|
684
|
-
|
|
684
|
+
// Explicit Shift chords need shifted text as well as a modifier bit. CDP does
|
|
685
|
+
// not derive printable text from code/windowsVirtualKeyCode for us.
|
|
686
|
+
const SHIFTED = {
|
|
687
|
+
"`": "~", "1": "!", "2": "@", "3": "#", "4": "$", "5": "%", "6": "^", "7": "&", "8": "*", "9": "(", "0": ")",
|
|
688
|
+
"-": "_", "=": "+", "[": "{", "]": "}", "\\": "|", ";": ":", "'": "\"", ",": "<", ".": ">", "/": "?",
|
|
689
|
+
};
|
|
690
|
+
const ch = shifted ? (/^[a-z]$/.test(key) ? key.toUpperCase() : SHIFTED[key] || key) : key;
|
|
685
691
|
const layout = usKeyLayoutForChar(ch);
|
|
686
692
|
return { key: ch, code: layout.code, windowsVirtualKeyCode: layout.keyCode, text: ch };
|
|
687
693
|
}
|
|
@@ -745,7 +751,7 @@ async function domClickFallback(tabId, params, cause) {
|
|
|
745
751
|
|
|
746
752
|
async function chromeInputClick(params) {
|
|
747
753
|
const tab = await getTabByParams(params);
|
|
748
|
-
|
|
754
|
+
await bringToFront(tab, params);
|
|
749
755
|
try {
|
|
750
756
|
await attachDebugger(tab.id);
|
|
751
757
|
const resolved = await resolveTargetInTab(tab.id, params);
|
|
@@ -782,7 +788,7 @@ async function chromeInputClick(params) {
|
|
|
782
788
|
|
|
783
789
|
async function chromeInputHover(params) {
|
|
784
790
|
const tab = await getTabByParams(params);
|
|
785
|
-
|
|
791
|
+
await bringToFront(tab, params);
|
|
786
792
|
await attachDebugger(tab.id);
|
|
787
793
|
const resolved = await resolveTargetInTab(tab.id, params);
|
|
788
794
|
const point = resolved.rect ? pickInsideRect(resolved.rect) : { x: resolved.x, y: resolved.y };
|
|
@@ -793,7 +799,7 @@ async function chromeInputHover(params) {
|
|
|
793
799
|
|
|
794
800
|
async function chromeInputKey(params) {
|
|
795
801
|
const tab = await getTabByParams(params);
|
|
796
|
-
|
|
802
|
+
await bringToFront(tab, params);
|
|
797
803
|
await attachDebugger(tab.id);
|
|
798
804
|
const key = String(params.key || "");
|
|
799
805
|
if (!key) throw new Error("chrome.key: missing key");
|
|
@@ -809,13 +815,14 @@ async function chromeInputKey(params) {
|
|
|
809
815
|
await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyDown", key: m.key, code: m.code, windowsVirtualKeyCode: m.vk, modifiers: modBits });
|
|
810
816
|
await sleep(rng(6, 18));
|
|
811
817
|
}
|
|
812
|
-
const info = cdpKeyInfo(key);
|
|
813
|
-
//
|
|
814
|
-
|
|
818
|
+
const info = cdpKeyInfo(key, mods.shiftKey);
|
|
819
|
+
// Ctrl/Meta/Alt chords must not insert literal text (e.g. Cmd+V). Shift alone
|
|
820
|
+
// still types: Shift+a -> A, Shift+1 -> !, and Shift+Enter carries a newline.
|
|
821
|
+
const shortcut = !!(mods.ctrlKey || mods.metaKey || mods.altKey);
|
|
815
822
|
await cdp(tab.id, "Input.dispatchKeyEvent", {
|
|
816
|
-
type:
|
|
823
|
+
type: shortcut ? "rawKeyDown" : "keyDown", key: info.key, code: info.code,
|
|
817
824
|
windowsVirtualKeyCode: info.windowsVirtualKeyCode, nativeVirtualKeyCode: info.windowsVirtualKeyCode,
|
|
818
|
-
text:
|
|
825
|
+
text: shortcut ? "" : info.text, unmodifiedText: shortcut ? "" : info.text, modifiers: modBits,
|
|
819
826
|
});
|
|
820
827
|
await sleep(rng(25, 90));
|
|
821
828
|
await cdp(tab.id, "Input.dispatchKeyEvent", {
|
|
@@ -829,9 +836,52 @@ async function chromeInputKey(params) {
|
|
|
829
836
|
return { input: "chrome", key: info.key, modifiers: mods };
|
|
830
837
|
}
|
|
831
838
|
|
|
839
|
+
// Read the actual focused editor, not a role=textbox lookalike. For fill, select
|
|
840
|
+
// the requested editor's entire contents: triple-click only selects a paragraph.
|
|
841
|
+
// Selection uses the DOM; deletion and insertion still use Chrome's input layer.
|
|
842
|
+
async function contentEditableInTab(tabId, selectAllParams = null) {
|
|
843
|
+
const results = await executeScriptTimed({
|
|
844
|
+
target: { tabId, frameIds: [0] },
|
|
845
|
+
world: "MAIN",
|
|
846
|
+
func: (selector, uid, selectAll) => {
|
|
847
|
+
const active = document.activeElement;
|
|
848
|
+
if (selectAll) {
|
|
849
|
+
const state = window.__PI_CHROME_STATE__;
|
|
850
|
+
const el = uid ? state?.elements?.[uid] : document.querySelector(selector);
|
|
851
|
+
if (uid && (!el || !el.isConnected)) throw new Error(`snapshot uid ${uid} is stale; refresh chrome_snapshot`);
|
|
852
|
+
if (!el?.isContentEditable) return false;
|
|
853
|
+
if (!active?.isContentEditable || !(el === active || el.contains(active) || active.contains(el))) {
|
|
854
|
+
throw new Error("chrome.fill: requested contenteditable is not focused");
|
|
855
|
+
}
|
|
856
|
+
const selection = window.getSelection();
|
|
857
|
+
if (!selection) throw new Error("Could not select contenteditable contents");
|
|
858
|
+
const range = document.createRange();
|
|
859
|
+
range.selectNodeContents(el);
|
|
860
|
+
selection.removeAllRanges();
|
|
861
|
+
selection.addRange(range);
|
|
862
|
+
}
|
|
863
|
+
return active?.isContentEditable === true;
|
|
864
|
+
},
|
|
865
|
+
args: [selectAllParams?.selector ?? null, selectAllParams?.uid ?? null, selectAllParams !== null],
|
|
866
|
+
}, `inspect contenteditable in tab ${tabId}`);
|
|
867
|
+
return results?.[0]?.result === true;
|
|
868
|
+
}
|
|
869
|
+
|
|
870
|
+
async function typeTextInTab(tabId, text, perCharacter) {
|
|
871
|
+
if (!text) return "none";
|
|
872
|
+
if (!perCharacter && await contentEditableInTab(tabId)) {
|
|
873
|
+
// One native edit avoids per-character delays and rich-editor render races.
|
|
874
|
+
// Do not retry as keystrokes if insertion fails: it may already have applied.
|
|
875
|
+
await cdp(tabId, "Input.insertText", { text });
|
|
876
|
+
return "insertText";
|
|
877
|
+
}
|
|
878
|
+
for (const ch of Array.from(text)) await cdpTypeChar(tabId, ch);
|
|
879
|
+
return "keys";
|
|
880
|
+
}
|
|
881
|
+
|
|
832
882
|
async function chromeInputType(params) {
|
|
833
883
|
const tab = await getTabByParams(params);
|
|
834
|
-
|
|
884
|
+
await bringToFront(tab, params);
|
|
835
885
|
await attachDebugger(tab.id);
|
|
836
886
|
if (params.selector || params.uid) {
|
|
837
887
|
// Focus target by clicking it first.
|
|
@@ -844,12 +894,9 @@ async function chromeInputType(params) {
|
|
|
844
894
|
await sleep(rng(50, 120));
|
|
845
895
|
}
|
|
846
896
|
const text = String(params.text || "");
|
|
847
|
-
|
|
848
|
-
if (params.pressEnter) {
|
|
849
|
-
|
|
850
|
-
await chromeInputKey({ ...params, key: "Enter" });
|
|
851
|
-
}
|
|
852
|
-
return { input: "chrome", length: text.length };
|
|
897
|
+
const typing = await typeTextInTab(tab.id, text, params.perCharacter);
|
|
898
|
+
if (params.pressEnter) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" });
|
|
899
|
+
return { input: "chrome", length: text.length, typing };
|
|
853
900
|
}
|
|
854
901
|
|
|
855
902
|
async function domFillFallback(tabId, params, cause) {
|
|
@@ -894,7 +941,7 @@ async function domFillFallback(tabId, params, cause) {
|
|
|
894
941
|
|
|
895
942
|
async function chromeInputFill(params) {
|
|
896
943
|
const tab = await getTabByParams(params);
|
|
897
|
-
|
|
944
|
+
await bringToFront(tab, params);
|
|
898
945
|
try {
|
|
899
946
|
await attachDebugger(tab.id);
|
|
900
947
|
if (!(params.selector || params.uid)) throw new Error("chrome.fill: selector or uid required");
|
|
@@ -908,14 +955,15 @@ async function chromeInputFill(params) {
|
|
|
908
955
|
await cdp(tab.id, "Input.dispatchMouseEvent", { type: "mouseReleased", x: point.x, y: point.y, button: "left", buttons: 0, clickCount: i, pointerType: "mouse" });
|
|
909
956
|
await sleep(rng(20, 60));
|
|
910
957
|
}
|
|
958
|
+
await contentEditableInTab(tab.id, params);
|
|
911
959
|
// Delete selection.
|
|
912
960
|
await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyDown", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 });
|
|
913
961
|
await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyUp", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 });
|
|
914
962
|
await sleep(rng(20, 60));
|
|
915
963
|
const text = String(params.text || "");
|
|
916
|
-
|
|
917
|
-
if (params.submit) await chromeInputKey({ ...params, key: "Enter" });
|
|
918
|
-
return { input: "chrome", length: text.length };
|
|
964
|
+
const typing = await typeTextInTab(tab.id, text, params.perCharacter);
|
|
965
|
+
if (params.submit) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" });
|
|
966
|
+
return { input: "chrome", length: text.length, typing };
|
|
919
967
|
} catch (error) {
|
|
920
968
|
if (params.domFallback === false) throw error;
|
|
921
969
|
return domFillFallback(tab.id, params, error);
|
|
@@ -924,7 +972,7 @@ async function chromeInputFill(params) {
|
|
|
924
972
|
|
|
925
973
|
async function chromeInputScroll(params) {
|
|
926
974
|
const tab = await getTabByParams(params);
|
|
927
|
-
|
|
975
|
+
await bringToFront(tab, params);
|
|
928
976
|
await attachDebugger(tab.id);
|
|
929
977
|
const resolved = (params.selector || params.uid) ? await resolveTargetInTab(tab.id, params) : { x: 100, y: 100, rect: null };
|
|
930
978
|
const x = resolved.rect ? resolved.rect.left + Math.min(resolved.rect.width, 800) / 2 : resolved.x;
|
|
@@ -974,7 +1022,7 @@ async function chromeInputScroll(params) {
|
|
|
974
1022
|
|
|
975
1023
|
async function chromeInputTap(params) {
|
|
976
1024
|
const tab = await getTabByParams(params);
|
|
977
|
-
|
|
1025
|
+
await bringToFront(tab, params);
|
|
978
1026
|
await attachDebugger(tab.id);
|
|
979
1027
|
const resolved = (params.selector || params.uid || (typeof params.x === "number" && typeof params.y === "number"))
|
|
980
1028
|
? await resolveTargetInTab(tab.id, params)
|
|
@@ -990,7 +1038,7 @@ async function chromeInputTap(params) {
|
|
|
990
1038
|
|
|
991
1039
|
async function chromeInputDrag(params) {
|
|
992
1040
|
const tab = await getTabByParams(params);
|
|
993
|
-
|
|
1041
|
+
await bringToFront(tab, params);
|
|
994
1042
|
await attachDebugger(tab.id);
|
|
995
1043
|
const from = await resolveTargetInTab(tab.id, { selector: params.fromSelector ?? null, uid: params.fromUid ?? null, x: params.fromX ?? null, y: params.fromY ?? null });
|
|
996
1044
|
const to = await resolveTargetInTab(tab.id, { selector: params.toSelector ?? null, uid: params.toUid ?? null, x: params.toX ?? null, y: params.toY ?? null });
|
|
@@ -1015,7 +1063,7 @@ async function chromeInputDrag(params) {
|
|
|
1015
1063
|
|
|
1016
1064
|
async function chromeInputUpload(params) {
|
|
1017
1065
|
const tab = await getTabByParams(params);
|
|
1018
|
-
|
|
1066
|
+
await bringToFront(tab, params);
|
|
1019
1067
|
await attachDebugger(tab.id);
|
|
1020
1068
|
if (!(params.selector || params.uid)) throw new Error("chrome.upload: selector or uid required");
|
|
1021
1069
|
const paths = Array.isArray(params.paths) ? params.paths.map(String) : [];
|
|
@@ -1024,25 +1072,31 @@ async function chromeInputUpload(params) {
|
|
|
1024
1072
|
const selector = ${JSON.stringify(params.selector ?? null)};
|
|
1025
1073
|
const uid = ${JSON.stringify(params.uid ?? null)};
|
|
1026
1074
|
const state = window.__PI_CHROME_STATE__;
|
|
1027
|
-
const el = uid
|
|
1075
|
+
const el = uid ? state?.elements?.[uid] : (selector ? document.querySelector(selector) : null);
|
|
1076
|
+
if (uid && (!el || !el.isConnected)) throw new Error("snapshot uid " + uid + " is stale; refresh chrome_snapshot");
|
|
1028
1077
|
if (!el || el.tagName !== "INPUT" || el.type !== "file") throw new Error("Target must be <input type=file>");
|
|
1029
1078
|
el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" });
|
|
1030
1079
|
return el;
|
|
1031
1080
|
})()`;
|
|
1032
1081
|
const evaluated = await cdp(tab.id, "Runtime.evaluate", { expression, objectGroup: "pi-chrome-upload", includeCommandLineAPI: false, returnByValue: false });
|
|
1033
|
-
if (evaluated.exceptionDetails) throw new Error(evaluated.exceptionDetails
|
|
1082
|
+
if (evaluated.exceptionDetails) throw new Error(cdpExceptionText(evaluated.exceptionDetails) || "Could not resolve file input");
|
|
1034
1083
|
const objectId = evaluated.result?.objectId;
|
|
1035
1084
|
if (!objectId) throw new Error("Could not resolve file input object");
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
|
|
1041
|
-
objectId
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1085
|
+
try {
|
|
1086
|
+
await cdp(tab.id, "DOM.enable", {}).catch(() => undefined);
|
|
1087
|
+
// Some DOM agents return nodeId:0 (or reject conversion) for a valid remote
|
|
1088
|
+
// element. CDP accepts that same objectId directly, before any file mutation.
|
|
1089
|
+
const requested = await cdp(tab.id, "DOM.requestNode", { objectId }).catch(() => null);
|
|
1090
|
+
const target = requested?.nodeId ? { nodeId: requested.nodeId } : { objectId };
|
|
1091
|
+
await cdp(tab.id, "DOM.setFileInputFiles", { ...target, files: paths });
|
|
1092
|
+
await cdp(tab.id, "Runtime.callFunctionOn", {
|
|
1093
|
+
objectId,
|
|
1094
|
+
functionDeclaration: `function() { this.dispatchEvent(new Event("input", { bubbles: true })); this.dispatchEvent(new Event("change", { bubbles: true })); return this.files ? this.files.length : 0; }`,
|
|
1095
|
+
returnByValue: true,
|
|
1096
|
+
}).catch(() => undefined);
|
|
1097
|
+
} finally {
|
|
1098
|
+
await cdp(tab.id, "Runtime.releaseObject", { objectId }).catch(() => undefined);
|
|
1099
|
+
}
|
|
1046
1100
|
return { input: "chrome", uploaded: paths.map((path) => ({ path })) };
|
|
1047
1101
|
}
|
|
1048
1102
|
// ===============================================================
|
|
@@ -1208,22 +1262,29 @@ async function dispatch(action, params) {
|
|
|
1208
1262
|
extensionVersion: chrome.runtime.getManifest().version,
|
|
1209
1263
|
bridgeUrl: BRIDGE_URL,
|
|
1210
1264
|
userAgent: navigator.userAgent,
|
|
1265
|
+
capabilities: { hardBackground: true },
|
|
1211
1266
|
};
|
|
1212
1267
|
case "tab.list": {
|
|
1213
1268
|
const tabs = await chrome.tabs.query({});
|
|
1214
1269
|
return Promise.all(tabs.map(formatTab));
|
|
1215
1270
|
}
|
|
1271
|
+
case "tab.new.background":
|
|
1272
|
+
case "page.screenshot.background":
|
|
1273
|
+
// Older workers reject these action names before touching tabs. Do not replace this with
|
|
1274
|
+
// a capability probe followed by an old action: a reload/profile change can race the probe.
|
|
1275
|
+
return dispatch(action.slice(0, -".background".length), { ...params, background: true, foreground: false });
|
|
1216
1276
|
case "tab.new": {
|
|
1217
1277
|
// Every Pi-opened tab must join a tab group. There is intentionally no opt-out: an ungrouped
|
|
1218
1278
|
// Pi-created tab is easy to lose among user tabs. If grouping fails after creation, close the
|
|
1219
1279
|
// tab best-effort before surfacing the error so tab.new never leaves an ungrouped Pi tab.
|
|
1220
1280
|
const groupTitle = params.groupTitle || "Pi";
|
|
1221
1281
|
const existingGroup = await findGroupRecordByTitle(groupTitle);
|
|
1222
|
-
const createParams = { url: params.url || "about:blank", active:
|
|
1282
|
+
const createParams = { url: params.url || "about:blank", active: foregroundRequested(params) };
|
|
1223
1283
|
if (existingGroup && typeof existingGroup.windowId === "number") createParams.windowId = existingGroup.windowId;
|
|
1224
1284
|
const tab = await chrome.tabs.create(createParams);
|
|
1225
1285
|
await trackSessionTab(sessionKeyOf(params), tab.id, true);
|
|
1226
1286
|
try {
|
|
1287
|
+
await bringToFront(tab, params);
|
|
1227
1288
|
return await groupTab(tab, groupTitle, params.groupColor);
|
|
1228
1289
|
} catch (error) {
|
|
1229
1290
|
if (typeof tab.id === "number") await chrome.tabs.remove(tab.id).catch(() => {});
|
|
@@ -1231,12 +1292,14 @@ async function dispatch(action, params) {
|
|
|
1231
1292
|
}
|
|
1232
1293
|
}
|
|
1233
1294
|
case "tab.activate": {
|
|
1295
|
+
if (!foregroundRequested(params)) {
|
|
1296
|
+
throw new Error("Tab activation is blocked by background mode. Ask the user to run /chrome background off to allow foreground work.");
|
|
1297
|
+
}
|
|
1234
1298
|
// Management actions never auto-create an automation target (createOwnedTarget:false): with
|
|
1235
1299
|
// no explicit target they act on an owned target if one exists, else error — they must never
|
|
1236
1300
|
// fall back to (or spawn a tab just to touch) the user's active tab.
|
|
1237
1301
|
const tab = await getTabByParams(params, { createOwnedTarget: false });
|
|
1238
|
-
await
|
|
1239
|
-
return formatTab(await chrome.tabs.update(tab.id, { active: true }));
|
|
1302
|
+
return formatTab(await bringToFront(tab, params));
|
|
1240
1303
|
}
|
|
1241
1304
|
case "tab.group": {
|
|
1242
1305
|
const tab = await getTabByParams(params, { createOwnedTarget: false });
|
|
@@ -1292,7 +1355,7 @@ async function dispatch(action, params) {
|
|
|
1292
1355
|
// Poll from the service worker via CDP (bypasses CSP). The old approach ran the polling
|
|
1293
1356
|
// loop in-page with new Function() for expression checks, which fails under strict CSP.
|
|
1294
1357
|
const tab = await getTabByParams(params);
|
|
1295
|
-
|
|
1358
|
+
await bringToFront(tab, params);
|
|
1296
1359
|
const timeoutMs = params.timeoutMs || 10000;
|
|
1297
1360
|
const intervalMs = params.intervalMs || 250;
|
|
1298
1361
|
const started = Date.now();
|
|
@@ -1316,7 +1379,7 @@ async function dispatch(action, params) {
|
|
|
1316
1379
|
return executeInTab(params, probePage, []);
|
|
1317
1380
|
case "page.navigate": {
|
|
1318
1381
|
const tab = await getTabByParams(params);
|
|
1319
|
-
|
|
1382
|
+
await bringToFront(tab, params);
|
|
1320
1383
|
if (params.initScript) {
|
|
1321
1384
|
// Register a one-shot document_start content script. We register, navigate, wait, then unregister.
|
|
1322
1385
|
await registerInitScript(tab.id, params.initScript);
|
|
@@ -1477,7 +1540,7 @@ const HELPER_FUNCS = [
|
|
|
1477
1540
|
|
|
1478
1541
|
async function executeInTab(params, func, args) {
|
|
1479
1542
|
const tab = await getTabByParams(params);
|
|
1480
|
-
|
|
1543
|
+
await bringToFront(tab, params);
|
|
1481
1544
|
|
|
1482
1545
|
// Phase 1: define the helpers and the action function as page globals via CDP
|
|
1483
1546
|
// Runtime.evaluate. This bypasses page CSP (no `eval`/`new Function`), which is the
|
|
@@ -1544,7 +1607,7 @@ function piEvalStringify(v) {
|
|
|
1544
1607
|
// pages that ship `script-src 'self'` without `'unsafe-eval'` (which blocks `eval`/`new Function`).
|
|
1545
1608
|
async function evaluateInTab(params) {
|
|
1546
1609
|
const tab = await getTabByParams(params);
|
|
1547
|
-
|
|
1610
|
+
await bringToFront(tab, params);
|
|
1548
1611
|
const expression = String(params.expression ?? "");
|
|
1549
1612
|
const stringifySrc = `(${piEvalStringify.toString()})`;
|
|
1550
1613
|
// Wrap the user expression so the result is run through piEvalStringify in-page before it
|
|
@@ -1591,7 +1654,7 @@ async function withOptionalSnapshot(params, actionFn) {
|
|
|
1591
1654
|
// It shares window.__PI_CHROME_STATE__ (same el- uid scheme) with the CDP-injected input helpers.
|
|
1592
1655
|
async function snapshotInTab(params) {
|
|
1593
1656
|
const tab = await getTabByParams(params);
|
|
1594
|
-
|
|
1657
|
+
await bringToFront(tab, params);
|
|
1595
1658
|
const args = [
|
|
1596
1659
|
params.maxElements || 80,
|
|
1597
1660
|
params.containingText ?? null,
|
|
@@ -1635,7 +1698,7 @@ async function snapshotInTab(params) {
|
|
|
1635
1698
|
async function inspectInTab(params) {
|
|
1636
1699
|
if (!params.uid && !params.selector) throw new Error("chrome_inspect requires uid or selector");
|
|
1637
1700
|
const tab = await getTabByParams(params);
|
|
1638
|
-
|
|
1701
|
+
await bringToFront(tab, params);
|
|
1639
1702
|
const args = [params.uid ?? null, params.selector ?? null, params.scrollIntoView === true];
|
|
1640
1703
|
await executeScriptTimed({
|
|
1641
1704
|
target: { tabId: tab.id, frameIds: [0] },
|
|
@@ -1705,9 +1768,15 @@ if (chrome.webNavigation && chrome.webNavigation.onCommitted) {
|
|
|
1705
1768
|
});
|
|
1706
1769
|
}
|
|
1707
1770
|
|
|
1708
|
-
|
|
1771
|
+
function foregroundRequested(params) {
|
|
1772
|
+
// Fail quiet when unspecified, and let background veto even a contradictory foreground flag.
|
|
1773
|
+
return params?.foreground === true && params.background !== true;
|
|
1774
|
+
}
|
|
1775
|
+
|
|
1776
|
+
async function bringToFront(tab, params) {
|
|
1777
|
+
if (!foregroundRequested(params)) return tab;
|
|
1709
1778
|
await chrome.windows.update(tab.windowId, { focused: true });
|
|
1710
|
-
|
|
1779
|
+
return chrome.tabs.update(tab.id, { active: true });
|
|
1711
1780
|
}
|
|
1712
1781
|
|
|
1713
1782
|
function waitForTabComplete(tabId, timeoutMs) {
|
|
@@ -1727,50 +1796,51 @@ function waitForTabComplete(tabId, timeoutMs) {
|
|
|
1727
1796
|
});
|
|
1728
1797
|
}
|
|
1729
1798
|
|
|
1730
|
-
async function
|
|
1731
|
-
const
|
|
1732
|
-
if (params.foreground) await bringToFront(tab);
|
|
1733
|
-
let previousActiveId;
|
|
1734
|
-
if (!tab.active) {
|
|
1735
|
-
const activeBefore = await chrome.tabs.query({ active: true, windowId: tab.windowId });
|
|
1736
|
-
previousActiveId = activeBefore[0]?.id;
|
|
1737
|
-
await chrome.tabs.update(tab.id, { active: true });
|
|
1738
|
-
}
|
|
1799
|
+
async function captureTabScreenshot(tabId, params) {
|
|
1800
|
+
const format = params.format || "png";
|
|
1739
1801
|
try {
|
|
1740
|
-
|
|
1741
|
-
|
|
1742
|
-
|
|
1743
|
-
|
|
1744
|
-
|
|
1745
|
-
|
|
1802
|
+
await attachDebugger(tabId);
|
|
1803
|
+
const captureParams = { format, fromSurface: true, captureBeyondViewport: false };
|
|
1804
|
+
if (format === "jpeg" && params.quality !== undefined) captureParams.quality = params.quality;
|
|
1805
|
+
const result = await cdp(tabId, "Page.captureScreenshot", captureParams);
|
|
1806
|
+
if (typeof result?.data !== "string" || !result.data) throw new Error("CDP returned no screenshot data");
|
|
1807
|
+
return `data:image/${format};base64,${result.data}`;
|
|
1808
|
+
} catch (error) {
|
|
1809
|
+
// captureVisibleTab requires activation and can race with the user switching tabs. Never
|
|
1810
|
+
// use it as a fallback, even when debugger attachment or background rendering fails.
|
|
1811
|
+
throw new Error(`Chrome screenshot via CDP failed; no tab-activation fallback was attempted. ${error?.message || error}`);
|
|
1812
|
+
}
|
|
1813
|
+
}
|
|
1814
|
+
|
|
1815
|
+
async function takeScreenshot(params) {
|
|
1816
|
+
const tab = await bringToFront(await getTabByParams(params), params);
|
|
1817
|
+
if (params.fullPage) {
|
|
1818
|
+
// Preserve the existing tile + manifest contract. Every tile captures the same resolved tab,
|
|
1819
|
+
// without activation; selector/title changes during capture must not retarget later tiles.
|
|
1820
|
+
const targetParams = { ...params, targetId: tab.id, foreground: false };
|
|
1821
|
+
const tiles = await executeInTab(targetParams, captureFullPageTiles, []);
|
|
1822
|
+
const captured = [];
|
|
1823
|
+
try {
|
|
1746
1824
|
for (const tile of tiles.tiles) {
|
|
1747
|
-
await executeInTab(
|
|
1748
|
-
|
|
1749
|
-
await
|
|
1750
|
-
const dataUrl = await chrome.tabs.captureVisibleTab(tab.windowId, {
|
|
1751
|
-
format: params.format || "png",
|
|
1752
|
-
quality: params.format === "jpeg" ? params.quality : undefined,
|
|
1753
|
-
});
|
|
1754
|
-
captured.push({ y: tile.y, dataUrl });
|
|
1825
|
+
await executeInTab(targetParams, scrollToY, [tile.scrollY]);
|
|
1826
|
+
await sleep(120); // Let scroll/lazy-load handlers settle.
|
|
1827
|
+
captured.push({ y: tile.y, dataUrl: await captureTabScreenshot(tab.id, params) });
|
|
1755
1828
|
}
|
|
1756
|
-
|
|
1757
|
-
|
|
1758
|
-
|
|
1759
|
-
|
|
1760
|
-
dimensions: { width: tiles.width, height: tiles.height, viewportHeight: tiles.viewportHeight, dpr: tiles.dpr },
|
|
1761
|
-
tiles: captured,
|
|
1762
|
-
};
|
|
1763
|
-
}
|
|
1764
|
-
const dataUrl = await chrome.tabs.captureVisibleTab(tab.windowId, {
|
|
1765
|
-
format: params.format || "png",
|
|
1766
|
-
quality: params.format === "jpeg" ? params.quality : undefined,
|
|
1767
|
-
});
|
|
1768
|
-
return { dataUrl, tab: await formatTab(tab) };
|
|
1769
|
-
} finally {
|
|
1770
|
-
if (previousActiveId !== undefined && previousActiveId !== tab.id) {
|
|
1771
|
-
await chrome.tabs.update(previousActiveId, { active: true }).catch(() => undefined);
|
|
1829
|
+
} finally {
|
|
1830
|
+
// A failed tile must not strand the page at a new scroll position. Restore both axes
|
|
1831
|
+
// best-effort, without masking the capture error if the tab/debugger is gone.
|
|
1832
|
+
await executeInTab(targetParams, scrollToY, [tiles.originalScrollY, tiles.originalScrollX]).catch(() => undefined);
|
|
1772
1833
|
}
|
|
1834
|
+
return {
|
|
1835
|
+
fullPage: true,
|
|
1836
|
+
method: "cdp",
|
|
1837
|
+
tab: await formatTab(tab),
|
|
1838
|
+
dimensions: { width: tiles.width, height: tiles.height, viewportHeight: tiles.viewportHeight, dpr: tiles.dpr },
|
|
1839
|
+
tiles: captured,
|
|
1840
|
+
};
|
|
1773
1841
|
}
|
|
1842
|
+
const dataUrl = await captureTabScreenshot(tab.id, params);
|
|
1843
|
+
return { dataUrl, method: "cdp", tab: await formatTab(tab) };
|
|
1774
1844
|
}
|
|
1775
1845
|
|
|
1776
1846
|
// ---------------------------------------------------------------------------
|
|
@@ -2197,8 +2267,7 @@ function probePage() {
|
|
|
2197
2267
|
}
|
|
2198
2268
|
|
|
2199
2269
|
function captureFullPageTiles() {
|
|
2200
|
-
// Returns the
|
|
2201
|
-
// in the SW. We just report the scroll positions and metrics.
|
|
2270
|
+
// Returns the plan for CDP tile capture in the worker: scroll positions and page metrics.
|
|
2202
2271
|
const html = document.documentElement;
|
|
2203
2272
|
const body = document.body;
|
|
2204
2273
|
const width = Math.max(html.scrollWidth, body ? body.scrollWidth : 0, innerWidth);
|
|
@@ -2206,17 +2275,18 @@ function captureFullPageTiles() {
|
|
|
2206
2275
|
const viewportHeight = innerHeight;
|
|
2207
2276
|
const dpr = window.devicePixelRatio || 1;
|
|
2208
2277
|
const originalScrollY = scrollY;
|
|
2278
|
+
const originalScrollX = scrollX;
|
|
2209
2279
|
const tiles = [];
|
|
2210
2280
|
let y = 0;
|
|
2211
2281
|
while (y < height) {
|
|
2212
2282
|
tiles.push({ y, scrollY: y });
|
|
2213
2283
|
y += viewportHeight;
|
|
2214
2284
|
}
|
|
2215
|
-
return { width, height, viewportHeight, dpr, originalScrollY, tiles };
|
|
2285
|
+
return { width, height, viewportHeight, dpr, originalScrollY, originalScrollX, tiles };
|
|
2216
2286
|
}
|
|
2217
2287
|
|
|
2218
|
-
function scrollToY(y) {
|
|
2219
|
-
window.scrollTo({ top: y, left:
|
|
2288
|
+
function scrollToY(y, x = 0) {
|
|
2289
|
+
window.scrollTo({ top: y, left: x, behavior: "instant" });
|
|
2220
2290
|
return { scrollY };
|
|
2221
2291
|
}
|
|
2222
2292
|
|