pi-chrome 0.15.48 → 0.15.51
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +21 -0
- package/README.md +63 -103
- package/extensions/chrome-profile-bridge/browser-extension/manifest.json +1 -1
- package/extensions/chrome-profile-bridge/browser-extension/service_worker.js +81 -27
- package/extensions/chrome-profile-bridge/index.ts +14 -16
- package/package.json +2 -2
- package/test-suite/README.md +9 -0
- package/test-suite/challenges/21-keyboard-modifiers.html +5 -2
- package/test-suite/challenges/44-input-reliability.html +105 -0
- package/test-suite/manifest.json +44 -2
- package/test-suite/unit/background-policy.test.mjs +20 -0
- package/test-suite/unit/chrome-command.test.mjs +163 -0
- package/test-suite/unit/input-reliability.test.mjs +401 -0
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,27 @@
|
|
|
2
2
|
|
|
3
3
|
All notable user-facing changes to `pi-chrome`.
|
|
4
4
|
|
|
5
|
+
## 0.15.51 — 2026-09-10
|
|
6
|
+
|
|
7
|
+
- **Fewer Chrome commands.** Removed `/chrome status`; use bare `/chrome` for the quick connection, authorization, and background dashboard plus controls. The dashboard remains lightweight and does not run page probes.
|
|
8
|
+
- **Complete Doctor report.** `/chrome doctor` now includes authorization and background state alongside connection, version, page checks, and troubleshooting hints, even when Chrome is offline or outdated.
|
|
9
|
+
- **Command regressions.** Added tests for command dispatch/completion, lightweight dashboard behavior, Doctor state reporting, and failure paths. `/chrome background status` remains available.
|
|
10
|
+
|
|
11
|
+
## 0.15.50 — 2026-09-10
|
|
12
|
+
|
|
13
|
+
- **README clarity.** Lead with workflow examples, correct setup ordering and platform-specific onboarding instructions, and clarify privacy guidance. Describe background behavior and the on/off/status controls directly.
|
|
14
|
+
- **Documentation-only release.** Browser behavior and permissions are unchanged. The companion version matches the npm package version.
|
|
15
|
+
|
|
16
|
+
## 0.15.49 — 2026-09-10
|
|
17
|
+
|
|
18
|
+
- **Validation scope.** Node regression suites passed. Live browser validation remains incomplete: an input attempt encountered `Input.dispatchMouseEvent: Detached while handling command.`; a subsequent retest was blocked by a disconnected companion. No live-browser pass is claimed for these changes.
|
|
19
|
+
|
|
20
|
+
- **Upload node fallback.** When Chrome cannot convert a file input's remote object to a DOM node ID, use that same `objectId` directly. Release the remote object after success or failure and reject stale snapshot UIDs. No native file picker or new permissions.
|
|
21
|
+
- **Native rich-editor insertion.** `chrome_type` and `chrome_fill` use one CDP `Input.insertText` for focused contenteditables. Inputs/textareas keep individual key events; `perCharacter:true` preserves that path for rich editors needing `keydown` events. Results report `typing` as `insertText`, `keys`, or `none`. Existing DOM-fallback controls and background policy remain intact.
|
|
22
|
+
- **Complete rich-editor fill and single Enter.** Select all requested contenteditable contents before deletion, not only one paragraph. `pressEnter` sends one Enter instead of two; Enter after type/fill stays pinned to the resolved tab.
|
|
23
|
+
- **Shift-only typing.** Shift+a, Shift+1, and other US-layout printable chords now carry shifted text. Ctrl/Meta/Alt shortcuts still suppress literal insertion.
|
|
24
|
+
- **Input regressions.** Added worker fault-injection tests and challenge 44. Challenge 16 explicitly tests per-character typing; challenge 21 waits for Shift release before grading.
|
|
25
|
+
|
|
5
26
|
## 0.15.48 — 2026-09-09
|
|
6
27
|
|
|
7
28
|
- **Existing background mode is now hard background.** `/chrome background on` (still the default) overrides per-call foreground requests, keeps new tabs inactive, and blocks `chrome_tab activate`. Use the existing `/chrome background off` for foreground/watch mode; no new command or lock state.
|
package/README.md
CHANGED
|
@@ -1,151 +1,111 @@
|
|
|
1
1
|
# pi-chrome
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Give Pi the Chrome you're already signed into.**
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Debug your app, inspect signed-in dashboards, and capture screenshots using your existing Chrome profile—without setting up a separate automation browser.
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
You: "Find my open GitHub PR tab, summarize review state, and screenshot failing CI."
|
|
9
|
-
Agent: chrome_tab(list) → chrome_snapshot(uid:…) → chrome_screenshot(...)
|
|
10
|
-
✓ 3 reviewers, 1 change requested, CI red on iOS. Saved → .pi/chrome-screenshots/ci.png
|
|
11
|
-
You: [keeps coding — agent never asked you to log in]
|
|
12
|
-
```
|
|
7
|
+
Built for the [Pi coding agent](https://pi.dev).
|
|
13
8
|
|
|
14
|
-
|
|
9
|
+
## What you can do
|
|
15
10
|
|
|
16
|
-
|
|
11
|
+
Try prompts like these after setup:
|
|
17
12
|
|
|
18
|
-
|
|
13
|
+
| Use case | Ask Pi |
|
|
14
|
+
| --- | --- |
|
|
15
|
+
| **Debug a signed-in app** | “Reproduce the filter bug in my staging app. Inspect captured console and network errors, then save a screenshot.” |
|
|
16
|
+
| **Understand an existing page** | “Find my open dashboard tab and summarize what's on the page. Don't change anything.” |
|
|
17
|
+
| **Create evidence for a PR** | “On my local app, capture the empty, loading, and populated states of this feature for my PR.” |
|
|
19
18
|
|
|
20
|
-
|
|
21
|
-
pi install npm:pi-chrome
|
|
22
|
-
```
|
|
19
|
+
Pi gets tools to inspect pages, click, type, fill forms, scroll, upload files, capture screenshots, and inspect captured console logs and `fetch`/`XMLHttpRequest` responses. You describe the task; Pi handles the agent loop.
|
|
23
20
|
|
|
24
|
-
|
|
21
|
+
**Best fit:** interactive workflows in the Chrome profile you already use. For deterministic CI tests, consider a test framework such as Playwright; for fleets of isolated browsers, consider a hosted browser service. See [more workflows](./docs/EXAMPLES.md) and [browser-tool comparisons](./docs/COMPARISON.md).
|
|
25
22
|
|
|
26
|
-
|
|
27
|
-
/chrome onboard
|
|
28
|
-
```
|
|
23
|
+
## Quick start
|
|
29
24
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
1. Turn on **Developer mode**.
|
|
33
|
-
2. Click **Load unpacked**.
|
|
34
|
-
3. Open path field with **Cmd+Shift+G** on macOS or **Ctrl+L** on Windows/Linux.
|
|
35
|
-
4. Paste copied path.
|
|
36
|
-
5. Press Enter.
|
|
25
|
+
**Requirements:** [Pi](https://pi.dev) and Google Chrome. Setup includes a one-time manual installation of the bundled Chrome companion extension.
|
|
37
26
|
|
|
38
|
-
|
|
27
|
+
> **Trust and privacy:** The companion has broad browser permissions and runs in your real Chrome profile. Review [its source](./extensions/chrome-profile-bridge/browser-extension/) before loading it, and authorize only tasks you trust. The browser bridge is local, but page content returned to Pi may be sent to your configured model provider.
|
|
39
28
|
|
|
40
|
-
|
|
41
|
-
/reload
|
|
42
|
-
```
|
|
29
|
+
### 1. Install and load the Pi package
|
|
43
30
|
|
|
44
|
-
|
|
31
|
+
In your terminal:
|
|
45
32
|
|
|
46
|
-
```
|
|
47
|
-
|
|
33
|
+
```bash
|
|
34
|
+
pi install npm:pi-chrome
|
|
48
35
|
```
|
|
49
36
|
|
|
50
|
-
|
|
37
|
+
Start Pi with `pi`. If Pi is already running, run `/reload` in that session **before** using the `/chrome` commands.
|
|
51
38
|
|
|
52
|
-
|
|
53
|
-
✓ Chrome is connected (...)
|
|
54
|
-
```
|
|
39
|
+
### 2. Connect Chrome
|
|
55
40
|
|
|
56
|
-
|
|
41
|
+
In Pi:
|
|
57
42
|
|
|
58
43
|
```text
|
|
59
|
-
/chrome
|
|
60
|
-
/chrome doctor
|
|
44
|
+
/chrome onboard
|
|
61
45
|
```
|
|
62
46
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
---
|
|
47
|
+
The setup dialog shows the companion extension's folder path.
|
|
66
48
|
|
|
67
|
-
|
|
49
|
+
- **macOS:** after confirmation, Pi opens `chrome://extensions`, reveals the companion folder in Finder, and copies its path to your clipboard.
|
|
50
|
+
- **Windows/Linux:** copy the folder path shown in the dialog and open `chrome://extensions` manually. Automatic desktop opening and clipboard setup are currently macOS-only.
|
|
68
51
|
|
|
69
|
-
|
|
70
|
-
- Click, type, fill forms, scroll, drag, tap, and upload files.
|
|
71
|
-
- Capture screenshots for bugs, PRs, and demos.
|
|
72
|
-
- Inspect console logs and captured `fetch`/`XMLHttpRequest` responses.
|
|
73
|
-
- Manage tabs without taking over your active window.
|
|
52
|
+
In Chrome:
|
|
74
53
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
54
|
+
1. Turn on **Developer mode**.
|
|
55
|
+
2. Click **Load unpacked**.
|
|
56
|
+
3. Select the companion folder shown by `/chrome onboard`. On macOS, press **Cmd+Shift+G** in the folder picker and paste the copied path.
|
|
78
57
|
|
|
79
|
-
|
|
58
|
+
### 3. Authorize and verify
|
|
80
59
|
|
|
81
|
-
|
|
60
|
+
In Pi:
|
|
82
61
|
|
|
83
62
|
```text
|
|
84
|
-
/chrome authorize
|
|
85
|
-
/chrome
|
|
86
|
-
/chrome authorize indefinite
|
|
87
|
-
/chrome revoke # lock again
|
|
88
|
-
/chrome status
|
|
63
|
+
/chrome authorize
|
|
64
|
+
/chrome doctor
|
|
89
65
|
```
|
|
90
66
|
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
- Extension runs in your real Chrome profile and has broad tab/scripting permissions. Install only from trusted package source.
|
|
94
|
-
- Pi side binds to `127.0.0.1:17318` only; no default network exposure.
|
|
95
|
-
- Bridge rejects browser-origin command requests, so ordinary web pages cannot drive it through CORS.
|
|
96
|
-
- Each Pi session gets its own automation target; user tabs/windows are not closed by cleanup.
|
|
97
|
-
- `/chrome revoke` closes only calling session's automation target.
|
|
98
|
-
|
|
99
|
-
Security details: [`SECURITY.md`](./SECURITY.md). Architecture details: [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md).
|
|
67
|
+
Approve the authorization prompt for a task you trust. The default authorization lasts **15 minutes**. Doctor should report `✓ Chrome is connected (...)`; follow its instructions if any checks fail.
|
|
100
68
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
## Commands
|
|
69
|
+
Then try a read-only first task:
|
|
104
70
|
|
|
105
71
|
```text
|
|
106
|
-
|
|
107
|
-
/chrome doctor # connectivity + version + eval checks
|
|
108
|
-
/chrome status # connection + auth + background state
|
|
109
|
-
/chrome authorize [duration]
|
|
110
|
-
/chrome revoke
|
|
111
|
-
/chrome background on # default: hard background policy
|
|
112
|
-
/chrome background off # foreground/watch mode
|
|
113
|
-
/chrome background status
|
|
72
|
+
List my open Chrome tabs without navigating, clicking, or changing anything.
|
|
114
73
|
```
|
|
115
74
|
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
### Background policy
|
|
119
|
-
|
|
120
|
-
`/chrome background on` is enforced, not an overridable default. Per-call `background:false` cannot bring Chrome forward, new tabs stay inactive, and `chrome_tab activate` is blocked. Use the existing `/chrome background off` for foreground/watch mode; per-call `background:true` still works when that mode is off.
|
|
121
|
-
|
|
122
|
-
Screenshots use CDP without activating background tabs. Debugger/capture failures return errors, never an activation fallback. Reload both Pi and the Chrome companion after upgrading; old companions reject background tab creation/screenshots rather than silently switching tabs.
|
|
75
|
+
Run `/chrome revoke` when finished. Use `/chrome authorize` again whenever you want to grant access for another task or session.
|
|
123
76
|
|
|
124
|
-
|
|
77
|
+
## Safety
|
|
125
78
|
|
|
126
|
-
|
|
79
|
+
- **Per-session approval.** Pi's Chrome tools require `/chrome authorize`. `/chrome revoke` locks them and requests cleanup of that session's owned automation tabs. Cleanup preserves existing user tabs.
|
|
80
|
+
- **Separate targets by default.** Page actions without an explicit target use a session-owned automation window or tab. The agent can deliberately target an existing tab when your task calls for it.
|
|
81
|
+
- **Local transport, not a sandbox.** The bridge binds to `127.0.0.1:17318` and rejects browser-origin command requests. It does not authenticate arbitrary non-browser local callers; it is not protection against hostile processes on your machine.
|
|
82
|
+
- **Background mode:** `/chrome background on` (default) blocks pi-chrome tools from directly bringing Chrome to the front or switching your selected tab. Use `/chrome background off` to allow those actions, and `/chrome background status` to check the setting.
|
|
127
83
|
|
|
128
|
-
|
|
84
|
+
### Limits
|
|
129
85
|
|
|
130
|
-
|
|
86
|
+
This is browser automation, not full OS control. Native Chrome/OS dialogs, password-manager prompts, passkeys/security keys/biometrics, CAPTCHA challenges, cross-origin iframe DOM access, rich multitouch/stylus gestures, and arbitrary desktop apps are outside its reliable tool surface. Some workflows need human assistance.
|
|
131
87
|
|
|
132
|
-
|
|
88
|
+
If page inspection or evaluation is blocked, use screenshots and coordinate input where possible. Background pages can throttle rendering or reject focus-gated actions. See the [FAQ](./docs/FAQ.md) for details.
|
|
133
89
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
---
|
|
90
|
+
## Commands
|
|
137
91
|
|
|
138
|
-
|
|
92
|
+
```text
|
|
93
|
+
/chrome # quick connection/auth/background dashboard and controls
|
|
94
|
+
/chrome onboard # one-time companion setup
|
|
95
|
+
/chrome authorize # authorize this Pi session for 15 minutes
|
|
96
|
+
/chrome authorize 30m # choose a duration
|
|
97
|
+
/chrome authorize indefinite # no time limit; revoke when finished
|
|
98
|
+
/chrome revoke # lock tools and request session cleanup
|
|
99
|
+
/chrome doctor # full diagnostics, including authorization/background state
|
|
100
|
+
/chrome background on # default: block explicit focus/tab activation
|
|
101
|
+
/chrome background off # allow foreground/watch mode
|
|
102
|
+
/chrome background status
|
|
103
|
+
```
|
|
139
104
|
|
|
140
|
-
|
|
141
|
-
- FAQ: [`docs/FAQ.md`](./docs/FAQ.md)
|
|
142
|
-
- Comparison: [`docs/COMPARISON.md`](./docs/COMPARISON.md)
|
|
143
|
-
- Security: [`SECURITY.md`](./SECURITY.md)
|
|
144
|
-
- Benchmark suite: [`test-suite/README.md`](./test-suite/README.md)
|
|
145
|
-
- Architecture: [`docs/ARCHITECTURE.md`](./docs/ARCHITECTURE.md)
|
|
105
|
+
Bare `/chrome` checks the connection without running page probes. Use `/chrome doctor` for version and page checks, troubleshooting hints, and authorization/background state.
|
|
146
106
|
|
|
147
|
-
|
|
107
|
+
Tool parameters are documented inline in Pi. See [architecture](./docs/ARCHITECTURE.md) for target ownership, screenshot behavior, and background-policy details.
|
|
148
108
|
|
|
149
|
-
|
|
109
|
+
### Updating and troubleshooting
|
|
150
110
|
|
|
151
|
-
|
|
111
|
+
After `pi update npm:pi-chrome`, run `/reload` in Pi and reload **Pi Chrome Connector** in `chrome://extensions`. Run `/chrome doctor` to check the connection and companion version.
|
|
@@ -681,7 +681,13 @@ function cdpKeyInfo(key, shifted) {
|
|
|
681
681
|
};
|
|
682
682
|
if (SPECIAL[key]) return { key, ...SPECIAL[key] };
|
|
683
683
|
if (key.length === 1) {
|
|
684
|
-
|
|
684
|
+
// Explicit Shift chords need shifted text as well as a modifier bit. CDP does
|
|
685
|
+
// not derive printable text from code/windowsVirtualKeyCode for us.
|
|
686
|
+
const SHIFTED = {
|
|
687
|
+
"`": "~", "1": "!", "2": "@", "3": "#", "4": "$", "5": "%", "6": "^", "7": "&", "8": "*", "9": "(", "0": ")",
|
|
688
|
+
"-": "_", "=": "+", "[": "{", "]": "}", "\\": "|", ";": ":", "'": "\"", ",": "<", ".": ">", "/": "?",
|
|
689
|
+
};
|
|
690
|
+
const ch = shifted ? (/^[a-z]$/.test(key) ? key.toUpperCase() : SHIFTED[key] || key) : key;
|
|
685
691
|
const layout = usKeyLayoutForChar(ch);
|
|
686
692
|
return { key: ch, code: layout.code, windowsVirtualKeyCode: layout.keyCode, text: ch };
|
|
687
693
|
}
|
|
@@ -809,13 +815,14 @@ async function chromeInputKey(params) {
|
|
|
809
815
|
await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyDown", key: m.key, code: m.code, windowsVirtualKeyCode: m.vk, modifiers: modBits });
|
|
810
816
|
await sleep(rng(6, 18));
|
|
811
817
|
}
|
|
812
|
-
const info = cdpKeyInfo(key);
|
|
813
|
-
//
|
|
814
|
-
|
|
818
|
+
const info = cdpKeyInfo(key, mods.shiftKey);
|
|
819
|
+
// Ctrl/Meta/Alt chords must not insert literal text (e.g. Cmd+V). Shift alone
|
|
820
|
+
// still types: Shift+a -> A, Shift+1 -> !, and Shift+Enter carries a newline.
|
|
821
|
+
const shortcut = !!(mods.ctrlKey || mods.metaKey || mods.altKey);
|
|
815
822
|
await cdp(tab.id, "Input.dispatchKeyEvent", {
|
|
816
|
-
type:
|
|
823
|
+
type: shortcut ? "rawKeyDown" : "keyDown", key: info.key, code: info.code,
|
|
817
824
|
windowsVirtualKeyCode: info.windowsVirtualKeyCode, nativeVirtualKeyCode: info.windowsVirtualKeyCode,
|
|
818
|
-
text:
|
|
825
|
+
text: shortcut ? "" : info.text, unmodifiedText: shortcut ? "" : info.text, modifiers: modBits,
|
|
819
826
|
});
|
|
820
827
|
await sleep(rng(25, 90));
|
|
821
828
|
await cdp(tab.id, "Input.dispatchKeyEvent", {
|
|
@@ -829,6 +836,49 @@ async function chromeInputKey(params) {
|
|
|
829
836
|
return { input: "chrome", key: info.key, modifiers: mods };
|
|
830
837
|
}
|
|
831
838
|
|
|
839
|
+
// Read the actual focused editor, not a role=textbox lookalike. For fill, select
|
|
840
|
+
// the requested editor's entire contents: triple-click only selects a paragraph.
|
|
841
|
+
// Selection uses the DOM; deletion and insertion still use Chrome's input layer.
|
|
842
|
+
async function contentEditableInTab(tabId, selectAllParams = null) {
|
|
843
|
+
const results = await executeScriptTimed({
|
|
844
|
+
target: { tabId, frameIds: [0] },
|
|
845
|
+
world: "MAIN",
|
|
846
|
+
func: (selector, uid, selectAll) => {
|
|
847
|
+
const active = document.activeElement;
|
|
848
|
+
if (selectAll) {
|
|
849
|
+
const state = window.__PI_CHROME_STATE__;
|
|
850
|
+
const el = uid ? state?.elements?.[uid] : document.querySelector(selector);
|
|
851
|
+
if (uid && (!el || !el.isConnected)) throw new Error(`snapshot uid ${uid} is stale; refresh chrome_snapshot`);
|
|
852
|
+
if (!el?.isContentEditable) return false;
|
|
853
|
+
if (!active?.isContentEditable || !(el === active || el.contains(active) || active.contains(el))) {
|
|
854
|
+
throw new Error("chrome.fill: requested contenteditable is not focused");
|
|
855
|
+
}
|
|
856
|
+
const selection = window.getSelection();
|
|
857
|
+
if (!selection) throw new Error("Could not select contenteditable contents");
|
|
858
|
+
const range = document.createRange();
|
|
859
|
+
range.selectNodeContents(el);
|
|
860
|
+
selection.removeAllRanges();
|
|
861
|
+
selection.addRange(range);
|
|
862
|
+
}
|
|
863
|
+
return active?.isContentEditable === true;
|
|
864
|
+
},
|
|
865
|
+
args: [selectAllParams?.selector ?? null, selectAllParams?.uid ?? null, selectAllParams !== null],
|
|
866
|
+
}, `inspect contenteditable in tab ${tabId}`);
|
|
867
|
+
return results?.[0]?.result === true;
|
|
868
|
+
}
|
|
869
|
+
|
|
870
|
+
async function typeTextInTab(tabId, text, perCharacter) {
|
|
871
|
+
if (!text) return "none";
|
|
872
|
+
if (!perCharacter && await contentEditableInTab(tabId)) {
|
|
873
|
+
// One native edit avoids per-character delays and rich-editor render races.
|
|
874
|
+
// Do not retry as keystrokes if insertion fails: it may already have applied.
|
|
875
|
+
await cdp(tabId, "Input.insertText", { text });
|
|
876
|
+
return "insertText";
|
|
877
|
+
}
|
|
878
|
+
for (const ch of Array.from(text)) await cdpTypeChar(tabId, ch);
|
|
879
|
+
return "keys";
|
|
880
|
+
}
|
|
881
|
+
|
|
832
882
|
async function chromeInputType(params) {
|
|
833
883
|
const tab = await getTabByParams(params);
|
|
834
884
|
await bringToFront(tab, params);
|
|
@@ -844,12 +894,9 @@ async function chromeInputType(params) {
|
|
|
844
894
|
await sleep(rng(50, 120));
|
|
845
895
|
}
|
|
846
896
|
const text = String(params.text || "");
|
|
847
|
-
|
|
848
|
-
if (params.pressEnter) {
|
|
849
|
-
|
|
850
|
-
await chromeInputKey({ ...params, key: "Enter" });
|
|
851
|
-
}
|
|
852
|
-
return { input: "chrome", length: text.length };
|
|
897
|
+
const typing = await typeTextInTab(tab.id, text, params.perCharacter);
|
|
898
|
+
if (params.pressEnter) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" });
|
|
899
|
+
return { input: "chrome", length: text.length, typing };
|
|
853
900
|
}
|
|
854
901
|
|
|
855
902
|
async function domFillFallback(tabId, params, cause) {
|
|
@@ -908,14 +955,15 @@ async function chromeInputFill(params) {
|
|
|
908
955
|
await cdp(tab.id, "Input.dispatchMouseEvent", { type: "mouseReleased", x: point.x, y: point.y, button: "left", buttons: 0, clickCount: i, pointerType: "mouse" });
|
|
909
956
|
await sleep(rng(20, 60));
|
|
910
957
|
}
|
|
958
|
+
await contentEditableInTab(tab.id, params);
|
|
911
959
|
// Delete selection.
|
|
912
960
|
await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyDown", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 });
|
|
913
961
|
await cdp(tab.id, "Input.dispatchKeyEvent", { type: "keyUp", key: "Delete", code: "Delete", windowsVirtualKeyCode: 46 });
|
|
914
962
|
await sleep(rng(20, 60));
|
|
915
963
|
const text = String(params.text || "");
|
|
916
|
-
|
|
917
|
-
if (params.submit) await chromeInputKey({ ...params, key: "Enter" });
|
|
918
|
-
return { input: "chrome", length: text.length };
|
|
964
|
+
const typing = await typeTextInTab(tab.id, text, params.perCharacter);
|
|
965
|
+
if (params.submit) await chromeInputKey({ ...params, targetId: tab.id, key: "Enter" });
|
|
966
|
+
return { input: "chrome", length: text.length, typing };
|
|
919
967
|
} catch (error) {
|
|
920
968
|
if (params.domFallback === false) throw error;
|
|
921
969
|
return domFillFallback(tab.id, params, error);
|
|
@@ -1024,25 +1072,31 @@ async function chromeInputUpload(params) {
|
|
|
1024
1072
|
const selector = ${JSON.stringify(params.selector ?? null)};
|
|
1025
1073
|
const uid = ${JSON.stringify(params.uid ?? null)};
|
|
1026
1074
|
const state = window.__PI_CHROME_STATE__;
|
|
1027
|
-
const el = uid
|
|
1075
|
+
const el = uid ? state?.elements?.[uid] : (selector ? document.querySelector(selector) : null);
|
|
1076
|
+
if (uid && (!el || !el.isConnected)) throw new Error("snapshot uid " + uid + " is stale; refresh chrome_snapshot");
|
|
1028
1077
|
if (!el || el.tagName !== "INPUT" || el.type !== "file") throw new Error("Target must be <input type=file>");
|
|
1029
1078
|
el.scrollIntoView({ block: "center", inline: "center", behavior: "instant" });
|
|
1030
1079
|
return el;
|
|
1031
1080
|
})()`;
|
|
1032
1081
|
const evaluated = await cdp(tab.id, "Runtime.evaluate", { expression, objectGroup: "pi-chrome-upload", includeCommandLineAPI: false, returnByValue: false });
|
|
1033
|
-
if (evaluated.exceptionDetails) throw new Error(evaluated.exceptionDetails
|
|
1082
|
+
if (evaluated.exceptionDetails) throw new Error(cdpExceptionText(evaluated.exceptionDetails) || "Could not resolve file input");
|
|
1034
1083
|
const objectId = evaluated.result?.objectId;
|
|
1035
1084
|
if (!objectId) throw new Error("Could not resolve file input object");
|
|
1036
|
-
|
|
1037
|
-
|
|
1038
|
-
|
|
1039
|
-
|
|
1040
|
-
|
|
1041
|
-
objectId
|
|
1042
|
-
|
|
1043
|
-
|
|
1044
|
-
|
|
1045
|
-
|
|
1085
|
+
try {
|
|
1086
|
+
await cdp(tab.id, "DOM.enable", {}).catch(() => undefined);
|
|
1087
|
+
// Some DOM agents return nodeId:0 (or reject conversion) for a valid remote
|
|
1088
|
+
// element. CDP accepts that same objectId directly, before any file mutation.
|
|
1089
|
+
const requested = await cdp(tab.id, "DOM.requestNode", { objectId }).catch(() => null);
|
|
1090
|
+
const target = requested?.nodeId ? { nodeId: requested.nodeId } : { objectId };
|
|
1091
|
+
await cdp(tab.id, "DOM.setFileInputFiles", { ...target, files: paths });
|
|
1092
|
+
await cdp(tab.id, "Runtime.callFunctionOn", {
|
|
1093
|
+
objectId,
|
|
1094
|
+
functionDeclaration: `function() { this.dispatchEvent(new Event("input", { bubbles: true })); this.dispatchEvent(new Event("change", { bubbles: true })); return this.files ? this.files.length : 0; }`,
|
|
1095
|
+
returnByValue: true,
|
|
1096
|
+
}).catch(() => undefined);
|
|
1097
|
+
} finally {
|
|
1098
|
+
await cdp(tab.id, "Runtime.releaseObject", { objectId }).catch(() => undefined);
|
|
1099
|
+
}
|
|
1046
1100
|
return { input: "chrome", uploaded: paths.map((path) => ({ path })) };
|
|
1047
1101
|
}
|
|
1048
1102
|
// ===============================================================
|
|
@@ -992,7 +992,11 @@ Usage rules:
|
|
|
992
992
|
// Shared handlers, dispatched by the unified /chrome command below.
|
|
993
993
|
const doctorHandler = async (ctx: ExtensionContext) => {
|
|
994
994
|
ctx.ui.notify("Checking pi-chrome…", "info");
|
|
995
|
-
const lines: string[] = [
|
|
995
|
+
const lines: string[] = [
|
|
996
|
+
`pi-chrome v${PI_CHROME_VERSION}`,
|
|
997
|
+
`• Authorization: ${authSummary()}.`,
|
|
998
|
+
`• Background: ${backgroundEnabled ? "on (hard)" : "off (foreground/watch mode)"}.`,
|
|
999
|
+
];
|
|
996
1000
|
const status = bridge.status();
|
|
997
1001
|
const roleLabel = status.mode === "client" ? "sharing another pi session's connection" : "running the Chrome connection for this machine";
|
|
998
1002
|
lines.push(`• This pi session is ${roleLabel}.`);
|
|
@@ -1144,8 +1148,7 @@ Usage rules:
|
|
|
1144
1148
|
);
|
|
1145
1149
|
};
|
|
1146
1150
|
|
|
1147
|
-
//
|
|
1148
|
-
// picker and as the body of /chrome status.
|
|
1151
|
+
// Lightweight connection/auth/background header for the bare-/chrome picker. No page probes.
|
|
1149
1152
|
const statusSummary = async (): Promise<string> => {
|
|
1150
1153
|
const parts: string[] = [];
|
|
1151
1154
|
try {
|
|
@@ -1163,11 +1166,6 @@ Usage rules:
|
|
|
1163
1166
|
return parts.join(" · ");
|
|
1164
1167
|
};
|
|
1165
1168
|
|
|
1166
|
-
const statusHandler = async (ctx: ExtensionContext) => {
|
|
1167
|
-
ctx.ui.notify("Checking Chrome connection…", "info");
|
|
1168
|
-
ctx.ui.notify(await statusSummary(), "info");
|
|
1169
|
-
};
|
|
1170
|
-
|
|
1171
1169
|
const openAuthorizeMenu = async (ctx: ExtensionContext): Promise<void> => {
|
|
1172
1170
|
while (true) {
|
|
1173
1171
|
const choice = await ctx.ui.select("Authorize Chrome control", [
|
|
@@ -1225,7 +1223,7 @@ Usage rules:
|
|
|
1225
1223
|
|
|
1226
1224
|
pi.registerCommand("chrome", {
|
|
1227
1225
|
description:
|
|
1228
|
-
"All pi-chrome controls in one place.\n /chrome authorize [15m|30m|<minutes>|indefinite] — allow this Pi session to use chrome_* tools.\n /chrome revoke — lock Chrome control.\n /chrome
|
|
1226
|
+
"All pi-chrome controls in one place.\n /chrome authorize [15m|30m|<minutes>|indefinite] — allow this Pi session to use chrome_* tools.\n /chrome revoke — lock Chrome control.\n /chrome doctor — full health check plus authorization and background state.\n /chrome onboard — install the Chrome companion extension.\n /chrome background [on|off|status|toggle] — enforce no explicit focus/tab activation, or allow foreground/watch mode.\nRun with no arguments for an interactive picker that shows current state.",
|
|
1229
1227
|
getArgumentCompletions: (prefix) => {
|
|
1230
1228
|
const raw = prefix;
|
|
1231
1229
|
const trimmedRight = raw.replace(/\s+$/, "");
|
|
@@ -1244,8 +1242,7 @@ Usage rules:
|
|
|
1244
1242
|
candidates = [
|
|
1245
1243
|
{ fullValue: "authorize", label: "authorize", description: "Allow this Pi session to use chrome_* tools." },
|
|
1246
1244
|
{ fullValue: "revoke", label: "revoke", description: "Lock Chrome control for this Pi session." },
|
|
1247
|
-
{ fullValue: "
|
|
1248
|
-
{ fullValue: "doctor", label: "doctor", description: "Full health check. Tells you if Chrome is connected and what's wrong if it isn't." },
|
|
1245
|
+
{ fullValue: "doctor", label: "doctor", description: "Full diagnostics: connection, version, page checks, authorization, and background state." },
|
|
1249
1246
|
{ fullValue: "onboard", label: "onboard", description: "Install the Chrome companion extension (first-time setup)." },
|
|
1250
1247
|
{ fullValue: "background", label: "background", description: "Enforce hard background or allow foreground/watch mode." },
|
|
1251
1248
|
];
|
|
@@ -1279,7 +1276,6 @@ Usage rules:
|
|
|
1279
1276
|
switch (head) {
|
|
1280
1277
|
case "authorize": return authorizeHandler(ctx, subArgs);
|
|
1281
1278
|
case "revoke": return revokeHandler(ctx);
|
|
1282
|
-
case "status": return statusHandler(ctx);
|
|
1283
1279
|
case "doctor": return doctorHandler(ctx);
|
|
1284
1280
|
case "onboard": return onboardHandler(ctx);
|
|
1285
1281
|
case "background":
|
|
@@ -1292,7 +1288,7 @@ Usage rules:
|
|
|
1292
1288
|
return;
|
|
1293
1289
|
}
|
|
1294
1290
|
default:
|
|
1295
|
-
ctx.ui.notify(`Unknown subcommand '${head}'.
|
|
1291
|
+
ctx.ui.notify(`Unknown subcommand '${head}'. Run /chrome for current state and controls, or try: /chrome authorize | revoke | doctor | onboard | background.`, "warning");
|
|
1296
1292
|
}
|
|
1297
1293
|
},
|
|
1298
1294
|
});
|
|
@@ -1564,12 +1560,13 @@ Usage rules:
|
|
|
1564
1560
|
name: "chrome_type",
|
|
1565
1561
|
label: "Chrome Type",
|
|
1566
1562
|
description:
|
|
1567
|
-
"Focus an optional snapshot uid or CSS selector, then type
|
|
1563
|
+
"Focus an optional snapshot uid or CSS selector, then type using Chrome's real input. Contenteditables use one native text insertion; other fields use key events. Set perCharacter=true for editors needing individual keydown events. Pass includeSnapshot=true to verify after typing.",
|
|
1568
1564
|
promptSnippet: "Type text into Chrome, optionally focusing a snapshot uid or selector first.",
|
|
1569
1565
|
parameters: Type.Object({
|
|
1570
1566
|
text: Type.String(),
|
|
1571
1567
|
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
|
|
1572
1568
|
selector: Type.Optional(Type.String({ description: "CSS selector to focus before typing." })),
|
|
1569
|
+
perCharacter: Type.Optional(Type.Boolean({ default: false, description: "Send individual key events even in contenteditables. Default: one native text insertion for contenteditables; key events for other fields." })),
|
|
1573
1570
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, include a fresh chrome_snapshot result after typing." })),
|
|
1574
1571
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
|
1575
1572
|
pressEnter: Type.Optional(Type.Boolean()),
|
|
@@ -1595,12 +1592,13 @@ Usage rules:
|
|
|
1595
1592
|
name: "chrome_fill",
|
|
1596
1593
|
label: "Chrome Fill",
|
|
1597
1594
|
description:
|
|
1598
|
-
"Set the full value of a text input, textarea, or contenteditable
|
|
1595
|
+
"Set the full value of a text input, textarea, or contenteditable using Chrome click/select/delete/type input. Contenteditables use one native text insertion; perCharacter=true retains individual keydown events. Accepts a snapshot uid or CSS selector. Pass includeSnapshot=true to verify after filling.",
|
|
1599
1596
|
promptSnippet: "Fill a Chrome form field by snapshot uid or selector, optionally returning a fresh snapshot.",
|
|
1600
1597
|
parameters: Type.Object({
|
|
1601
1598
|
text: Type.String(),
|
|
1602
1599
|
uid: Type.Optional(Type.String({ description: "Stable element uid from chrome_snapshot." })),
|
|
1603
1600
|
selector: Type.Optional(Type.String({ description: "CSS selector to fill if uid is omitted." })),
|
|
1601
|
+
perCharacter: Type.Optional(Type.Boolean({ default: false, description: "Send individual key events even in contenteditables. Default: one native text insertion for contenteditables; key events for other fields." })),
|
|
1604
1602
|
submit: Type.Optional(Type.Boolean({ description: "If true, press Enter after filling." })),
|
|
1605
1603
|
domFallback: Type.Optional(Type.Boolean({ description: "If true (default), fall back to DOM value-setting if Chrome's CDP input path is blocked by another extension overlay or debugger failure." })),
|
|
1606
1604
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, include a fresh chrome_snapshot result after filling." })),
|
|
@@ -1636,7 +1634,7 @@ Usage rules:
|
|
|
1636
1634
|
ctrlKey: Type.Optional(Type.Boolean()),
|
|
1637
1635
|
altKey: Type.Optional(Type.Boolean()),
|
|
1638
1636
|
metaKey: Type.Optional(Type.Boolean()),
|
|
1639
|
-
}, { description: "Modifier keys to hold while pressing the key (
|
|
1637
|
+
}, { description: "Modifier keys to hold while pressing the key. Shift alone types the shifted US-layout character (a → A, 1 → !); Ctrl/Meta/Alt chords do not insert literal text." })),
|
|
1640
1638
|
includeSnapshot: Type.Optional(Type.Boolean({ description: "If true, include a fresh chrome_snapshot result after the keypress." })),
|
|
1641
1639
|
maxElements: Type.Optional(Type.Number({ default: MAX_ELEMENTS, description: "Max elements in the included snapshot." })),
|
|
1642
1640
|
targetId: Type.Optional(Type.String()),
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-chrome",
|
|
3
|
-
"version": "0.15.
|
|
3
|
+
"version": "0.15.51",
|
|
4
4
|
"scripts": {
|
|
5
|
-
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && node test-suite/unit/session-cleanup.test.mjs && node test-suite/unit/background-policy.test.mjs",
|
|
5
|
+
"test": "node test-suite/unit/csp-eval.test.mjs && node test-suite/unit/automation-target.test.mjs && node test-suite/unit/session-cleanup.test.mjs && node test-suite/unit/background-policy.test.mjs && node test-suite/unit/input-reliability.test.mjs && node test-suite/unit/chrome-command.test.mjs",
|
|
6
6
|
"version": "node scripts/sync-manifest-version.js",
|
|
7
7
|
"prepublishOnly": "node scripts/sync-manifest-version.js"
|
|
8
8
|
},
|
package/test-suite/README.md
CHANGED
|
@@ -177,6 +177,15 @@ The dashboard renders this from `manifest.json`. In brief:
|
|
|
177
177
|
41. explicit tab lifecycle
|
|
178
178
|
42. strict CSP eval/snapshot via CDP (regression guard for the CSP bypass)
|
|
179
179
|
43. hard background: inactive-tab visibility and trusted input
|
|
180
|
+
44. input reliability: native rich-editor insertion, full replacement, per-character override, Shift chords, single Enter, and file upload
|
|
181
|
+
|
|
182
|
+
### Input regressions (16, 21, 31, 44)
|
|
183
|
+
|
|
184
|
+
Challenge 44 exercises `chrome_type`/`chrome_fill` bulk insertion into contenteditables, including long Unicode text and multiple paragraphs. It also checks `perCharacter:true`, Shift-only printable keys, one Enter per `pressEnter`, and real file contents. Read `window.__inputFixture.caption` and follow its manifest recipe. Use `domFallback:false` to ensure failures are not hidden by synthetic events.
|
|
185
|
+
|
|
186
|
+
Challenge 16 intentionally requests `perCharacter:true`: its grader requires a caret update for each keystroke. Challenge 21 focuses its field before sending Shift+a and grades after Shift release, not during the earlier input event. Challenge 31 retains standalone upload coverage.
|
|
187
|
+
|
|
188
|
+
`npm test` also fault-injects zero/missing/rejected upload node IDs, attachment/cleanup failures, stale UIDs, insertion failures, shortcut suppression, and authorization/option forwarding. These mocked tests cannot prove `isTrusted` or real browser selection behavior; run the browser challenges too.
|
|
180
189
|
|
|
181
190
|
### Hard-background regression (43)
|
|
182
191
|
|
|
@@ -30,9 +30,12 @@ for (const name of ["keydown","keypress","keyup","input"]) {
|
|
|
30
30
|
});
|
|
31
31
|
}
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
33
|
+
// input fires before keyup. Grade only when Shift is released so a valid chord
|
|
34
|
+
// cannot fail merely because its release events have not happened yet.
|
|
35
|
+
t.addEventListener("keyup", (event) => {
|
|
36
|
+
if (event.key !== "Shift") return;
|
|
35
37
|
const bad = [];
|
|
38
|
+
if (t.value !== "A") bad.push(`value=${JSON.stringify(t.value)} (need 'A')`);
|
|
36
39
|
const downShift = log.find(e => e.name === "keydown" && e.key === "Shift");
|
|
37
40
|
const downA = log.find(e => e.name === "keydown" && e.code === "KeyA");
|
|
38
41
|
const upShift = log.find(e => e.name === "keyup" && e.key === "Shift");
|