pi-chrome 0.15.47 → 0.15.49

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,24 @@
2
2
 
3
3
  All notable user-facing changes to `pi-chrome`.
4
4
 
5
+ ## 0.15.49 — 2026-09-10
6
+
7
+ - **Validation scope.** Node regression suites passed. Live browser validation remains incomplete: an input attempt encountered `Input.dispatchMouseEvent: Detached while handling command.`; a subsequent retest was blocked by a disconnected companion. No live-browser pass is claimed for these changes.
8
+
9
+ - **Upload node fallback.** When Chrome cannot convert a file input's remote object to a DOM node ID, use that same `objectId` directly. Release the remote object after success or failure and reject stale snapshot UIDs. No native file picker or new permissions.
10
+ - **Native rich-editor insertion.** `chrome_type` and `chrome_fill` use one CDP `Input.insertText` for focused contenteditables. Inputs/textareas keep individual key events; `perCharacter:true` preserves that path for rich editors needing `keydown` events. Results report `typing` as `insertText`, `keys`, or `none`. Existing DOM-fallback controls and background policy remain intact.
11
+ - **Complete rich-editor fill and single Enter.** Select all requested contenteditable contents before deletion, not only one paragraph. `pressEnter` sends one Enter instead of two; Enter after type/fill stays pinned to the resolved tab.
12
+ - **Shift-only typing.** Shift+a, Shift+1, and other US-layout printable chords now carry shifted text. Ctrl/Meta/Alt shortcuts still suppress literal insertion.
13
+ - **Input regressions.** Added worker fault-injection tests and challenge 44. Challenge 16 explicitly tests per-character typing; challenge 21 waits for Shift release before grading.
14
+
15
+ ## 0.15.48 — 2026-09-09
16
+
17
+ - **Existing background mode is now hard background.** `/chrome background on` (still the default) overrides per-call foreground requests, keeps new tabs inactive, and blocks `chrome_tab activate`. Use the existing `/chrome background off` for foreground/watch mode; no new command or lock state.
18
+ - **Screenshots without tab activation.** PNG/JPEG and full-page tiles use CDP instead of `captureVisibleTab`. Debugger/capture failures never fall back to switching tabs. Old companions reject background creation/capture with a reload instruction. Screenshot tools now require debugger access.
19
+ - **Trusted input preserved.** Background policy does not replace Chrome input with synthetic events. It blocks explicit focus/activation, not page/native/Chrome/OS side effects; inactive-page rendering and focus-gated workflows can still vary by environment.
20
+ - **Regression coverage.** Added policy/worker/screenshot unit tests and challenge 43 for inactive-tab visibility plus trusted input. Full-page capture restores both scroll axes best-effort after success or failure.
21
+ - **Live validation and unresolved limitation.** Chrome 152/macOS checks passed for inactive tab creation, blocked activation, background PNG/JPEG/full-page capture, and scroll restoration. The trusted-click check encountered debugger detachment, then a visibility failure on retry; the cause remains unresolved and human interference was not ruled out. This release does not promise zero focus changes during trusted input. Chrome-behind-another-app and macOS Spaces behavior remain unverified.
22
+
5
23
  ## 0.15.47 — 2026-09-09
6
24
 
7
25
  - **Bounded session cleanup.** On exit, Pi waits up to two seconds for cleanup before stopping the bridge. Reload preserves browser resources; revoke remains non-blocking.
package/README.md CHANGED
@@ -74,6 +74,12 @@ Second doctor run should show all checks passing.
74
74
 
75
75
  Tool parameters and gotchas are documented inline in Pi.
76
76
 
77
+ ### Typing into rich editors
78
+
79
+ `chrome_type` and `chrome_fill` use one native CDP `Input.insertText` operation for focused contenteditables. This avoids per-character delays for long text and preserves Unicode/newlines. Ordinary inputs and textareas retain individual key events.
80
+
81
+ For an editor that needs a `keydown` for every character, pass `perCharacter:true`. Bulk insertion still uses Chrome's input system, but does not emit per-character key events or a clipboard `paste` event. Use `includeSnapshot:true` to verify the result; `chrome_fill` still honors `domFallback:false` when synthetic fallback is unwanted.
82
+
77
83
  ---
78
84
 
79
85
  ## Safety model
@@ -108,13 +114,21 @@ Security details: [`SECURITY.md`](./SECURITY.md). Architecture details: [`docs/A
108
114
  /chrome status # connection + auth + background state
109
115
  /chrome authorize [duration]
110
116
  /chrome revoke
111
- /chrome background on # default: don't steal focus
117
+ /chrome background on # default: hard background policy
112
118
  /chrome background off # foreground/watch mode
113
119
  /chrome background status
114
120
  ```
115
121
 
116
122
  If loaded extension is older than installed `pi-chrome`, `/chrome doctor` tells you to reload it from `chrome://extensions`.
117
123
 
124
+ ### Background policy
125
+
126
+ `/chrome background on` is enforced, not an overridable default. Per-call `background:false` cannot bring Chrome forward, new tabs stay inactive, and `chrome_tab activate` is blocked. Use the existing `/chrome background off` for foreground/watch mode; per-call `background:true` still works when that mode is off.
127
+
128
+ Screenshots use CDP without activating background tabs. Debugger/capture failures return errors, never an activation fallback. Reload both Pi and the Chrome companion after upgrading; old companions reject background tab creation/screenshots rather than silently switching tabs.
129
+
130
+ This prevents explicit pi-chrome focus/activation, not every Chrome/OS side effect. Trusted input, page popups, native prompts, debugger banners, and macOS Spaces can still affect focus. Inactive pages may throttle rendering or reject focus-gated actions. See [scope and risks](./docs/ARCHITECTURE.md#scope-and-risks).
131
+
118
132
  ---
119
133
 
120
134
  ## Limits
package/SECURITY.md CHANGED
@@ -27,7 +27,7 @@ The Chrome extension under `extensions/chrome-profile-bridge/browser-extension/`
27
27
  - Loopback bridge only. No remote port. No telemetry.
28
28
  - Chrome real input layer for interactive controls.
29
29
  - Chrome control locked by default; `/chrome authorize` unlocks current Pi session after terminal confirmation, `/chrome revoke` locks it again.
30
- - Run-in-background optional; tab/window focus is observable by default (the user can see Pi acting).
30
+ - Hard background mode is on by default: tools cannot override it to explicitly focus windows or activate tabs. `/chrome background off` allows foreground/watch mode. This is not a security sandbox: trusted input, page scripts, native prompts, and Chrome/OS behavior can still affect focus.
31
31
 
32
32
  ## Custom ports
33
33
 
@@ -46,14 +46,26 @@ To point pi-chrome at an existing tab, pass `targetId`, `urlIncludes`, or `title
46
46
 
47
47
  ## Background mode
48
48
 
49
- By default, chrome calls run in background so Chrome does not steal focus.
49
+ The existing background setting is a hard session policy, enabled by default. No separate lock/unlock command is needed.
50
50
 
51
51
  ```text
52
- /chrome background on # background mode
53
- /chrome background off # foreground/watch mode
52
+ /chrome background on # enforce no explicit window focus/tab activation
53
+ /chrome background off # allow foreground/watch mode
54
54
  ```
55
55
 
56
- Per-call `background: false` brings Chrome forward for that action. Per-call `background: true` forces background.
56
+ - With background on, per-call `background:false` and legacy `foreground:true` cannot override the policy. `chrome_tab activate` errors with instructions to ask the user to turn background off.
57
+ - With background off, calls may focus Chrome; per-call `background:true` still avoids explicit focus/tab activation.
58
+ - Policy is applied in `authorizedBridgeSend` for every tool, including `chrome_launch(url)`, tab creation, and tools without a background parameter. Each session sends its own effective `background`/`foreground` flags through the shared bridge.
59
+ - All worker window-focus/tab-activation writes go through a guarded helper. Background tab creation uses `active:false`; implicit automation windows remain `focused:false`.
60
+ - Screenshots use CDP `Page.captureScreenshot` with `fromSurface:true` and `captureBeyondViewport:false`. They target a tab, not whichever tab happens to be visible. There is no `captureVisibleTab`/activation fallback on debugger or capture failure. PNG/JPEG output is unchanged; full-page capture retains tiles plus a JSON manifest and restores scroll position best-effort, including on failure.
61
+ - Background tab creation and screenshots use internal `tab.new.background` / `page.screenshot.background` wire actions. Old companions reject them before changing tabs; Pi reports a reload instruction instead of retrying an unsafe legacy action. No capability-probe race or extra round trip is needed.
62
+ - Real CDP input and existing explicit DOM-fallback controls are unchanged. Background mode never silently substitutes synthetic input to avoid focus.
63
+
64
+ ### Scope and risks
65
+
66
+ This is a policy against **explicit pi-chrome focus/activation**, not an OS focus sandbox. Page scripts (`window.open`, `window.focus`), trusted input, native dialogs, debugger banners, Chrome window/Spaces behavior, and closing an active tab can still change focus or selection. Other sessions and human actions remain independent. Requests already dispatched before a mode change keep their earlier policy.
67
+
68
+ Inactive/minimized tabs can throttle timers or rendering, and clipboard/fullscreen/other focus-gated workflows may fail. Screenshots now require debugger attachment, which can conflict with DevTools or other extensions; hidden-tab rendering can differ or be unavailable. No automatic foreground retry is allowed. Full-page capture temporarily scrolls the target page. Reload both Pi and the Chrome companion after upgrading, and live-test tab selection, OS focus, and screenshot fidelity on supported Chrome/OS versions.
57
69
 
58
70
  ## Authorization
59
71
 
@@ -134,7 +134,7 @@ If your threat model excludes extensions with broad permissions, neither approac
134
134
 
135
135
  ## Public benchmarks worth knowing (for axis 2 / axis 3 comparison)
136
136
 
137
- Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **42 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering trusted input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, iframes, file uploads, strict-CSP screenshot fallback and CDP eval/snapshot bypass, dynamic waits, tab lifecycle, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count, with `core`, `conditional`, and `quality` gate buckets. That's **driver-level** grading.
137
+ Pi-chrome itself ships a benchmark suite ([`../test-suite/`](../test-suite)) of **43 primitive challenges** plus **4 hermetic BrowserGym-style long-horizon tasks** covering trusted input, pointer humanization, keyboard fidelity, drag/drop, Shadow DOM, iframes, file uploads, strict-CSP screenshot fallback and CDP eval/snapshot bypass, dynamic waits, tab lifecycle, network observability, fingerprint leaks, and agent-safety honeypots. Scoring tracks expected outcomes per challenge instead of raw PASS count, with `core`, `conditional`, and `quality` gate buckets. That's **driver-level** grading.
138
138
 
139
139
  For **agent-level** comparison (axis 2), the public benchmarks worth citing:
140
140
 
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "manifest_version": 3,
3
3
  "name": "Pi Chrome Connector",
4
- "version": "0.15.47",
4
+ "version": "0.15.49",
5
5
  "description": "Lets Pi control tabs in Chrome via a local connector at 127.0.0.1.",
6
6
  "permissions": [
7
7
  "tabs",
@@ -681,7 +681,13 @@ function cdpKeyInfo(key, shifted) {
681
681
  };
682
682
  if (SPECIAL[key]) return { key, ...SPECIAL[key] };
683
683
  if (key.length === 1) {
684
- const ch = key;
684
+ // Explicit Shift chords need shifted text as well as a modifier bit. CDP does
685
+ // not derive printable text from code/windowsVirtualKeyCode for us.
686
+ const SHIFTED = {
687
+ "`": "~", "1": "!", "2": "@", "3": "#", "4": "$", "5": "%", "6": "^", "7": "&", "8": "*", "9": "(", "0": ")",
688
+ "-": "_", "=": "+", "[": "{", "]": "}", "\\": "|", ";": ":", "'": "\"", ",": "<", ".": ">", "/": "?",
689
+ };
690
+ const ch = shifted ? (/^[a-z]$/.test(key) ? key.toUpperCase() : SHIFTED[key] || key) : key;
685
691
  const layout = usKeyLayoutForChar(ch);
686
692
  return { key: ch, code: layout.code, windowsVirtualKeyCode: layout.keyCode, text: ch };
687
693
  }
@@ -745,7 +751,7 @@ async function domClickFallback(tabId, params, cause) {
745
751
 
746
752
  async function chromeInputClick(params) {
747
753
  const tab = await getTabByParams(params);
748
- if (params.foreground) await bringToFront(tab);
754
+ await bringToFront(tab, params);
749
755
  try {
750
756
  await attachDebugger(tab.id);
751
757
  const resolved = await resolveTargetInTab(tab.id, params);
@@ -782,7 +788,7 @@ async function chromeInputClick(params) {
782
788
 
783
789
  async function chromeInputHover(params) {
784
790
  const tab = await getTabByParams(params);
785
- if (params.foreground) await bringToFront(tab);
791
+ await bringToFront(tab, params);
786
792
  await attachDebugger(tab.id);
787
793
  const resolved = await resolveTargetInTab(tab.id, params);
788
794
  const point = resolved.rect ? pickInsideRect(resolved.rect) : { x: resolved.x, y: resolved.y };
@@ -793,7 +799,7 @@ async function chromeInputHover(params) {
793
799
 
794
800
  async function chromeInputKey(params) {
795
801
  const tab = await getTabByParams(params);
796
- if (params.foreground) await bringToFront(tab);
802
+ await bringToFront(tab, params);
797
803
  await attachDebugger(tab.id);
798
804
  const key = String(params.key || "");
799
805
  if (!key) throw new Error("chrome.key: missing key");
@@ -809,13 +815,14 @@ async function chromeInputKey(params) {
809
815
  await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyDown", key: m.key, code: m.code, windowsVirtualKeyCode: m.vk, modifiers: modBits });
810
816
  await sleep(rng(6, 18));
811
817
  }
812
- const info = cdpKeyInfo(key);
813
- // When modifiers are active, browsers usually emit "rawKeyDown" (no text) so chords like Cmd+V don't insert the literal char.
814
- const downType = modBits ? "rawKeyDown" : "keyDown";
818
+ const info = cdpKeyInfo(key, mods.shiftKey);
819
+ // Ctrl/Meta/Alt chords must not insert literal text (e.g. Cmd+V). Shift alone
820
+ // still types: Shift+a -> A, Shift+1 -> !, and Shift+Enter carries a newline.
821
+ const shortcut = !!(mods.ctrlKey || mods.metaKey || mods.altKey);
815
822
  await cdp(tab.id, "Input.dispatchKeyEvent", {
816
- type: downType, key: info.key, code: info.code,
823
+ type: shortcut ? "rawKeyDown" : "keyDown", key: info.key, code: info.code,
817
824
  windowsVirtualKeyCode: info.windowsVirtualKeyCode, nativeVirtualKeyCode: info.windowsVirtualKeyCode,
818
- text: modBits ? "" : info.text, unmodifiedText: modBits ? "" : info.text, modifiers: modBits,
825
+ text: shortcut ? "" : info.text, unmodifiedText: shortcut ? "" : info.text, modifiers: modBits,
819
826
  });
820
827
  await sleep(rng(25, 90));
821
828
  await cdp(tab.id, "Input.dispatchKeyEvent", {
@@ -829,9 +836,52 @@ async function chromeInputKey(params) {
829
836
  return { input: "chrome", key: info.key, modifiers: mods };
830
837
  }
831
838
 
839
+ // Read the actual focused editor, not a role=textbox lookalike. For fill, select
840
+ // the requested editor's entire contents: triple-click only selects a paragraph.
841
+ // Selection uses the DOM; deletion and insertion still use Chrome's input layer.
842
+ async function contentEditableInTab(tabId, selectAllParams = null) {
843
+ const results = await executeScriptTimed({
844
+ target: { tabId, frameIds: [0] },
845
+ world: "MAIN",
846
+ func: (selector, uid, selectAll) => {
847
+ const active = document.activeElement;
848
+ if (selectAll) {
849
+ const state = window.__PI_CHROME_STATE__;
850
+ const el = uid ? state?.elements?.[uid] : document.querySelector(selector);
851
+ if (uid && (!el || !el.isConnected)) throw new Error(`snapshot uid ${uid} is stale; refresh chrome_snapshot`);
852
+ if (!el?.isContentEditable) return false;
853
+ if (!active?.isContentEditable || !(el === active || el.contains(active) || active.contains(el))) {
854
+ throw new Error("chrome.fill: requested contenteditable is not focused");
855
+ }
856
+ const selection = window.getSelection();
857
+ if (!selection) throw new Error("Could not select contenteditable contents");
858
+ const range = document.createRange();
859
+ range.selectNodeContents(el);
860
+ selection.removeAllRanges();
861
+ selection.addRange(range);
862
+ }
863
+ return active?.isContentEditable === true;
864
+ },
865
+ args: [selectAllParams?.selector ?? null, selectAllParams?.uid ?? null, selectAllParams !== null],
866
+ }, `inspect contenteditable in tab ${tabId}`);
867
+ return results?.[0]?.result === true;
868
+ }
869
+
870
+ async function typeTextInTab(tabId, text, perCharacter) {
871
+ if (!text) return "none";
872
+ if (!perCharacter && await contentEditableInTab(tabId)) {
873
+ // One native edit avoids per-character delays and rich-editor render races.
874
+ // Do not retry as keystrokes if insertion fails: it may already have applied.
875
+ await cdp(tabId, "Input.insertText", { text });
876
+ return "insertText";
877
+ }
878
+ for (const ch of Array.from(text)) await cdpTypeChar(tabId, ch);
879
+ return "keys";
880
+ }
881
+
832
882
  async function chromeInputType(params) {
833
883
  const tab = await getTabByParams(params);
834
- if (params.foreground) await bringToFront(tab);
884
+ await bringToFront(tab, params);
835
885
  await attachDebugger(tab.id);
836
886
  if (params.selector || params.uid) {
837
887
  // Focus target by clicking it first.
@@ -844,12 +894,9 @@ async function chromeInputType(params) {
844
894
  await sleep(rng(50, 120));
845
895
  }
846
896
  const text = String(params.text || "");
847
- for (const ch of Array.from(text)) await cdpTypeChar(tab.id, ch);
848
- if (params.pressEnter) {
849
- await cdpTypeChar(tab.id, "\r").catch(() => undefined);
850
- await chromeInputKey({ ...params, key: "Enter" });
851
- }
852
- return { input: "chrome", length: text.length };
897
+ const typing = await typeTextInTab(tab.id, text, params.perCharacter);
898
+ if (params.pressEnter) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" });
899
+ return { input: "chrome", length: text.length, typing };
853
900
  }
854
901
 
855
902
  async function domFillFallback(tabId, params, cause) {
@@ -894,7 +941,7 @@ async function domFillFallback(tabId, params, cause) {
894
941
 
895
942
  async function chromeInputFill(params) {
896
943
  const tab = await getTabByParams(params);
897
- if (params.foreground) await bringToFront(tab);
944
+ await bringToFront(tab, params);
898
945
  try {
899
946
  await attachDebugger(tab.id);
900
947
  if (!(params.selector || params.uid)) throw new Error("chrome.fill: selector or uid required");
@@ -908,14 +955,15 @@ async function chromeInputFill(params) {
908
955
  await cdp(tab.id, "Input.dispatchMouseEvent", { type: "mouseReleased", x: point.x, y: point.y, button: "left", buttons: 0, clickCount: i, pointerType: "mouse" });
909
956
  await sleep(rng(20, 60));
910
957
  }
958
+ await contentEditableInTab(tab.id, params);
911
959
  // Delete selection.
912
960
  await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyDown", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 });
913
961
  await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyUp", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 });
914
962
  await sleep(rng(20, 60));
915
963
  const text = String(params.text || "");
916
- for (const ch of Array.from(text)) await cdpTypeChar(tab.id, ch);
917
- if (params.submit) await chromeInputKey({ ...params, key: "Enter" });
918
- return { input: "chrome", length: text.length };
964
+ const typing = await typeTextInTab(tab.id, text, params.perCharacter);
965
+ if (params.submit) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" });
966
+ return { input: "chrome", length: text.length, typing };
919
967
  } catch (error) {
920
968
  if (params.domFallback === false) throw error;
921
969
  return domFillFallback(tab.id, params, error);
@@ -924,7 +972,7 @@ async function chromeInputFill(params) {
924
972
 
925
973
  async function chromeInputScroll(params) {
926
974
  const tab = await getTabByParams(params);
927
- if (params.foreground) await bringToFront(tab);
975
+ await bringToFront(tab, params);
928
976
  await attachDebugger(tab.id);
929
977
  const resolved = (params.selector || params.uid) ? await resolveTargetInTab(tab.id, params) : { x: 100, y: 100, rect: null };
930
978
  const x = resolved.rect ? resolved.rect.left + Math.min(resolved.rect.width, 800) / 2 : resolved.x;
@@ -974,7 +1022,7 @@ async function chromeInputScroll(params) {
974
1022
 
975
1023
  async function chromeInputTap(params) {
976
1024
  const tab = await getTabByParams(params);
977
- if (params.foreground) await bringToFront(tab);
1025
+ await bringToFront(tab, params);
978
1026
  await attachDebugger(tab.id);
979
1027
  const resolved = (params.selector || params.uid || (typeof params.x === "number" && typeof params.y === "number"))
980
1028
  ? await resolveTargetInTab(tab.id, params)
@@ -990,7 +1038,7 @@ async function chromeInputTap(params) {
990
1038
 
991
1039
  async function chromeInputDrag(params) {
992
1040
  const tab = await getTabByParams(params);
993
- if (params.foreground) await bringToFront(tab);
1041
+ await bringToFront(tab, params);
994
1042
  await attachDebugger(tab.id);
995
1043
  const from = await resolveTargetInTab(tab.id, { selector: params.fromSelector ?? null, uid: params.fromUid ?? null, x: params.fromX ?? null, y: params.fromY ?? null });
996
1044
  const to = await resolveTargetInTab(tab.id, { selector: params.toSelector ?? null, uid: params.toUid ?? null, x: params.toX ?? null, y: params.toY ?? null });
@@ -1015,7 +1063,7 @@ async function chromeInputDrag(params) {
1015
1063
 
1016
1064
  async function chromeInputUpload(params) {
1017
1065
  const tab = await getTabByParams(params);
1018
- if (params.foreground) await bringToFront(tab);
1066
+ await bringToFront(tab, params);
1019
1067
  await attachDebugger(tab.id);
1020
1068
  if (!(params.selector || params.uid)) throw new Error("chrome.upload: selector or uid required");
1021
1069
  const paths = Array.isArray(params.paths) ? params.paths.map(String) : [];
@@ -1024,25 +1072,31 @@ async function chromeInputUpload(params) {
1024
1072
  const selector = ${JSON.stringify(params.selector ?? null)};
1025
1073
  const uid = ${JSON.stringify(params.uid ?? null)};
1026
1074
  const state = window.__PI_CHROME_STATE__;
1027
- const el = uid && state && state.elements ? state.elements[uid] : (selector ? document.querySelector(selector) : null);
1075
+ const el = uid ? state?.elements?.[uid] : (selector ? document.querySelector(selector) : null);
1076
+ if (uid && (!el || !el.isConnected)) throw new Error("snapshot uid " + uid + " is stale; refresh chrome_snapshot");
1028
1077
  if (!el || el.tagName !== "INPUT" || el.type !== "file") throw new Error("Target must be <input type=file>");
1029
1078
  el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" });
1030
1079
  return el;
1031
1080
  })()`;
1032
1081
  const evaluated = await cdp(tab.id, "Runtime.evaluate", { expression, objectGroup: "pi-chrome-upload", includeCommandLineAPI: false, returnByValue: false });
1033
- if (evaluated.exceptionDetails) throw new Error(evaluated.exceptionDetails.text || "Could not resolve file input");
1082
+ if (evaluated.exceptionDetails) throw new Error(cdpExceptionText(evaluated.exceptionDetails) || "Could not resolve file input");
1034
1083
  const objectId = evaluated.result?.objectId;
1035
1084
  if (!objectId) throw new Error("Could not resolve file input object");
1036
- await cdp(tab.id, "DOM.enable", {}).catch(() => undefined);
1037
- const requested = await cdp(tab.id, "DOM.requestNode", { objectId });
1038
- if (!requested.nodeId) throw new Error("Could not resolve file input node");
1039
- await cdp(tab.id, "DOM.setFileInputFiles", { nodeId: requested.nodeId, files: paths });
1040
- await cdp(tab.id, "Runtime.callFunctionOn", {
1041
- objectId,
1042
- functionDeclaration: `function() { this.dispatchEvent(new Event("input", { bubbles: true })); this.dispatchEvent(new Event("change", { bubbles: true })); return this.files ? this.files.length : 0; }`,
1043
- returnByValue: true,
1044
- }).catch(() => undefined);
1045
- await cdp(tab.id, "Runtime.releaseObject", { objectId }).catch(() => undefined);
1085
+ try {
1086
+ await cdp(tab.id, "DOM.enable", {}).catch(() => undefined);
1087
+ // Some DOM agents return nodeId:0 (or reject conversion) for a valid remote
1088
+ // element. CDP accepts that same objectId directly, before any file mutation.
1089
+ const requested = await cdp(tab.id, "DOM.requestNode", { objectId }).catch(() => null);
1090
+ const target = requested?.nodeId ? { nodeId: requested.nodeId } : { objectId };
1091
+ await cdp(tab.id, "DOM.setFileInputFiles", { ...target, files: paths });
1092
+ await cdp(tab.id, "Runtime.callFunctionOn", {
1093
+ objectId,
1094
+ functionDeclaration: `function() { this.dispatchEvent(new Event("input", { bubbles: true })); this.dispatchEvent(new Event("change", { bubbles: true })); return this.files ? this.files.length : 0; }`,
1095
+ returnByValue: true,
1096
+ }).catch(() => undefined);
1097
+ } finally {
1098
+ await cdp(tab.id, "Runtime.releaseObject", { objectId }).catch(() => undefined);
1099
+ }
1046
1100
  return { input: "chrome", uploaded: paths.map((path) => ({ path })) };
1047
1101
  }
1048
1102
  // ===============================================================
@@ -1208,22 +1262,29 @@ async function dispatch(action, params) {
1208
1262
  extensionVersion: chrome.runtime.getManifest().version,
1209
1263
  bridgeUrl: BRIDGE_URL,
1210
1264
  userAgent: navigator.userAgent,
1265
+ capabilities: { hardBackground: true },
1211
1266
  };
1212
1267
  case "tab.list": {
1213
1268
  const tabs = await chrome.tabs.query({});
1214
1269
  return Promise.all(tabs.map(formatTab));
1215
1270
  }
1271
+ case "tab.new.background":
1272
+ case "page.screenshot.background":
1273
+ // Older workers reject these action names before touching tabs. Do not replace this with
1274
+ // a capability probe followed by an old action: a reload/profile change can race the probe.
1275
+ return dispatch(action.slice(0, -".background".length), { ...params, background: true, foreground: false });
1216
1276
  case "tab.new": {
1217
1277
  // Every Pi-opened tab must join a tab group. There is intentionally no opt-out: an ungrouped
1218
1278
  // Pi-created tab is easy to lose among user tabs. If grouping fails after creation, close the
1219
1279
  // tab best-effort before surfacing the error so tab.new never leaves an ungrouped Pi tab.
1220
1280
  const groupTitle = params.groupTitle || "Pi";
1221
1281
  const existingGroup = await findGroupRecordByTitle(groupTitle);
1222
- const createParams = { url: params.url || "about:blank", active: true };
1282
+ const createParams = { url: params.url || "about:blank", active: foregroundRequested(params) };
1223
1283
  if (existingGroup && typeof existingGroup.windowId === "number") createParams.windowId = existingGroup.windowId;
1224
1284
  const tab = await chrome.tabs.create(createParams);
1225
1285
  await trackSessionTab(sessionKeyOf(params), tab.id, true);
1226
1286
  try {
1287
+ await bringToFront(tab, params);
1227
1288
  return await groupTab(tab, groupTitle, params.groupColor);
1228
1289
  } catch (error) {
1229
1290
  if (typeof tab.id === "number") await chrome.tabs.remove(tab.id).catch(() => {});
@@ -1231,12 +1292,14 @@ async function dispatch(action, params) {
1231
1292
  }
1232
1293
  }
1233
1294
  case "tab.activate": {
1295
+ if (!foregroundRequested(params)) {
1296
+ throw new Error("Tab activation is blocked by background mode. Ask the user to run /chrome background off to allow foreground work.");
1297
+ }
1234
1298
  // Management actions never auto-create an automation target (createOwnedTarget:false): with
1235
1299
  // no explicit target they act on an owned target if one exists, else error — they must never
1236
1300
  // fall back to (or spawn a tab just to touch) the user's active tab.
1237
1301
  const tab = await getTabByParams(params, { createOwnedTarget: false });
1238
- await chrome.windows.update(tab.windowId, { focused: true });
1239
- return formatTab(await chrome.tabs.update(tab.id, { active: true }));
1302
+ return formatTab(await bringToFront(tab, params));
1240
1303
  }
1241
1304
  case "tab.group": {
1242
1305
  const tab = await getTabByParams(params, { createOwnedTarget: false });
@@ -1292,7 +1355,7 @@ async function dispatch(action, params) {
1292
1355
  // Poll from the service worker via CDP (bypasses CSP). The old approach ran the polling
1293
1356
  // loop in-page with new Function() for expression checks, which fails under strict CSP.
1294
1357
  const tab = await getTabByParams(params);
1295
- if (params.foreground) await bringToFront(tab);
1358
+ await bringToFront(tab, params);
1296
1359
  const timeoutMs = params.timeoutMs || 10000;
1297
1360
  const intervalMs = params.intervalMs || 250;
1298
1361
  const started = Date.now();
@@ -1316,7 +1379,7 @@ async function dispatch(action, params) {
1316
1379
  return executeInTab(params, probePage, []);
1317
1380
  case "page.navigate": {
1318
1381
  const tab = await getTabByParams(params);
1319
- if (params.foreground) await bringToFront(tab);
1382
+ await bringToFront(tab, params);
1320
1383
  if (params.initScript) {
1321
1384
  // Register a one-shot document_start content script. We register, navigate, wait, then unregister.
1322
1385
  await registerInitScript(tab.id, params.initScript);
@@ -1477,7 +1540,7 @@ const HELPER_FUNCS = [
1477
1540
 
1478
1541
  async function executeInTab(params, func, args) {
1479
1542
  const tab = await getTabByParams(params);
1480
- if (params.foreground) await bringToFront(tab);
1543
+ await bringToFront(tab, params);
1481
1544
 
1482
1545
  // Phase 1: define the helpers and the action function as page globals via CDP
1483
1546
  // Runtime.evaluate. This bypasses page CSP (no `eval`/`new Function`), which is the
@@ -1544,7 +1607,7 @@ function piEvalStringify(v) {
1544
1607
  // pages that ship `script-src 'self'` without `'unsafe-eval'` (which blocks `eval`/`new Function`).
1545
1608
  async function evaluateInTab(params) {
1546
1609
  const tab = await getTabByParams(params);
1547
- if (params.foreground) await bringToFront(tab);
1610
+ await bringToFront(tab, params);
1548
1611
  const expression = String(params.expression ?? "");
1549
1612
  const stringifySrc = `(${piEvalStringify.toString()})`;
1550
1613
  // Wrap the user expression so the result is run through piEvalStringify in-page before it
@@ -1591,7 +1654,7 @@ async function withOptionalSnapshot(params, actionFn) {
1591
1654
  // It shares window.__PI_CHROME_STATE__ (same el- uid scheme) with the CDP-injected input helpers.
1592
1655
  async function snapshotInTab(params) {
1593
1656
  const tab = await getTabByParams(params);
1594
- if (params.foreground) await bringToFront(tab);
1657
+ await bringToFront(tab, params);
1595
1658
  const args = [
1596
1659
  params.maxElements || 80,
1597
1660
  params.containingText ?? null,
@@ -1635,7 +1698,7 @@ async function snapshotInTab(params) {
1635
1698
  async function inspectInTab(params) {
1636
1699
  if (!params.uid && !params.selector) throw new Error("chrome_inspect requires uid or selector");
1637
1700
  const tab = await getTabByParams(params);
1638
- if (params.foreground) await bringToFront(tab);
1701
+ await bringToFront(tab, params);
1639
1702
  const args = [params.uid ?? null, params.selector ?? null, params.scrollIntoView === true];
1640
1703
  await executeScriptTimed({
1641
1704
  target: { tabId: tab.id, frameIds: [0] },
@@ -1705,9 +1768,15 @@ if (chrome.webNavigation && chrome.webNavigation.onCommitted) {
1705
1768
  });
1706
1769
  }
1707
1770
 
1708
- async function bringToFront(tab) {
1771
+ function foregroundRequested(params) {
1772
+ // Fail quiet when unspecified, and let background veto even a contradictory foreground flag.
1773
+ return params?.foreground === true && params.background !== true;
1774
+ }
1775
+
1776
+ async function bringToFront(tab, params) {
1777
+ if (!foregroundRequested(params)) return tab;
1709
1778
  await chrome.windows.update(tab.windowId, { focused: true });
1710
- await chrome.tabs.update(tab.id, { active: true });
1779
+ return chrome.tabs.update(tab.id, { active: true });
1711
1780
  }
1712
1781
 
1713
1782
  function waitForTabComplete(tabId, timeoutMs) {
@@ -1727,50 +1796,51 @@ function waitForTabComplete(tabId, timeoutMs) {
1727
1796
  });
1728
1797
  }
1729
1798
 
1730
- async function takeScreenshot(params) {
1731
- const tab = await getTabByParams(params);
1732
- if (params.foreground) await bringToFront(tab);
1733
- let previousActiveId;
1734
- if (!tab.active) {
1735
- const activeBefore = await chrome.tabs.query({ active: true, windowId: tab.windowId });
1736
- previousActiveId = activeBefore[0]?.id;
1737
- await chrome.tabs.update(tab.id, { active: true });
1738
- }
1799
+ async function captureTabScreenshot(tabId, params) {
1800
+ const format = params.format || "png";
1739
1801
  try {
1740
- if (params.fullPage) {
1741
- // Tile-stitched full page capture: scroll, capture, paste, repeat.
1742
- const tiles = await executeInTab({ ...params, foreground: false }, captureFullPageTiles, []);
1743
- // captureFullPageTiles only computes scroll positions / metrics; we capture per scroll here
1744
- // (chrome.tabs.captureVisibleTab can't be called from MAIN world).
1745
- const captured = [];
1802
+ await attachDebugger(tabId);
1803
+ const captureParams = { format, fromSurface: true, captureBeyondViewport: false };
1804
+ if (format === "jpeg" && params.quality !== undefined) captureParams.quality = params.quality;
1805
+ const result = await cdp(tabId, "Page.captureScreenshot", captureParams);
1806
+ if (typeof result?.data !== "string" || !result.data) throw new Error("CDP returned no screenshot data");
1807
+ return `data:image/${format};base64,${result.data}`;
1808
+ } catch (error) {
1809
+ // captureVisibleTab requires activation and can race with the user switching tabs. Never
1810
+ // use it as a fallback, even when debugger attachment or background rendering fails.
1811
+ throw new Error(`Chrome screenshot via CDP failed; no tab-activation fallback was attempted. ${error?.message || error}`);
1812
+ }
1813
+ }
1814
+
1815
+ async function takeScreenshot(params) {
1816
+ const tab = await bringToFront(await getTabByParams(params), params);
1817
+ if (params.fullPage) {
1818
+ // Preserve the existing tile + manifest contract. Every tile captures the same resolved tab,
1819
+ // without activation; selector/title changes during capture must not retarget later tiles.
1820
+ const targetParams = { ...params, targetId: tab.id, foreground: false };
1821
+ const tiles = await executeInTab(targetParams, captureFullPageTiles, []);
1822
+ const captured = [];
1823
+ try {
1746
1824
  for (const tile of tiles.tiles) {
1747
- await executeInTab({ ...params, foreground: false }, scrollToY, [tile.scrollY]);
1748
- // Small settle delay; many sites have on-scroll animations / lazy-load.
1749
- await sleep(120);
1750
- const dataUrl = await chrome.tabs.captureVisibleTab(tab.windowId, {
1751
- format: params.format || "png",
1752
- quality: params.format === "jpeg" ? params.quality : undefined,
1753
- });
1754
- captured.push({ y: tile.y, dataUrl });
1825
+ await executeInTab(targetParams, scrollToY, [tile.scrollY]);
1826
+ await sleep(120); // Let scroll/lazy-load handlers settle.
1827
+ captured.push({ y: tile.y, dataUrl: await captureTabScreenshot(tab.id, params) });
1755
1828
  }
1756
- await executeInTab({ ...params, foreground: false }, scrollToY, [tiles.originalScrollY]);
1757
- return {
1758
- fullPage: true,
1759
- tab: await formatTab(tab),
1760
- dimensions: { width: tiles.width, height: tiles.height, viewportHeight: tiles.viewportHeight, dpr: tiles.dpr },
1761
- tiles: captured,
1762
- };
1763
- }
1764
- const dataUrl = await chrome.tabs.captureVisibleTab(tab.windowId, {
1765
- format: params.format || "png",
1766
- quality: params.format === "jpeg" ? params.quality : undefined,
1767
- });
1768
- return { dataUrl, tab: await formatTab(tab) };
1769
- } finally {
1770
- if (previousActiveId !== undefined && previousActiveId !== tab.id) {
1771
- await chrome.tabs.update(previousActiveId, { active: true }).catch(() => undefined);
1829
+ } finally {
1830
+ // A failed tile must not strand the page at a new scroll position. Restore both axes
1831
+ // best-effort, without masking the capture error if the tab/debugger is gone.
1832
+ await executeInTab(targetParams, scrollToY, [tiles.originalScrollY, tiles.originalScrollX]).catch(() => undefined);
1772
1833
  }
1834
+ return {
1835
+ fullPage: true,
1836
+ method: "cdp",
1837
+ tab: await formatTab(tab),
1838
+ dimensions: { width: tiles.width, height: tiles.height, viewportHeight: tiles.viewportHeight, dpr: tiles.dpr },
1839
+ tiles: captured,
1840
+ };
1773
1841
  }
1842
+ const dataUrl = await captureTabScreenshot(tab.id, params);
1843
+ return { dataUrl, method: "cdp", tab: await formatTab(tab) };
1774
1844
  }
1775
1845
 
1776
1846
  // ---------------------------------------------------------------------------
@@ -2197,8 +2267,7 @@ function probePage() {
2197
2267
  }
2198
2268
 
2199
2269
  function captureFullPageTiles() {
2200
- // Returns the *plan* for tile capture; the actual chrome.tabs.captureVisibleTab calls happen
2201
- // in the SW. We just report the scroll positions and metrics.
2270
+ // Returns the plan for CDP tile capture in the worker: scroll positions and page metrics.
2202
2271
  const html = document.documentElement;
2203
2272
  const body = document.body;
2204
2273
  const width = Math.max(html.scrollWidth, body ? body.scrollWidth : 0, innerWidth);
@@ -2206,17 +2275,18 @@ function captureFullPageTiles() {
2206
2275
  const viewportHeight = innerHeight;
2207
2276
  const dpr = window.devicePixelRatio || 1;
2208
2277
  const originalScrollY = scrollY;
2278
+ const originalScrollX = scrollX;
2209
2279
  const tiles = [];
2210
2280
  let y = 0;
2211
2281
  while (y < height) {
2212
2282
  tiles.push({ y, scrollY: y });
2213
2283
  y += viewportHeight;
2214
2284
  }
2215
- return { width, height, viewportHeight, dpr, originalScrollY, tiles };
2285
+ return { width, height, viewportHeight, dpr, originalScrollY, originalScrollX, tiles };
2216
2286
  }
2217
2287
 
2218
- function scrollToY(y) {
2219
- window.scrollTo({ top: y, left: 0, behavior: "instant" });
2288
+ function scrollToY(y, x = 0) {
2289
+ window.scrollTo({ top: y, left: x, behavior: "instant" });
2220
2290
  return { scrollY };
2221
2291
  }
2222
2292