browser-pilot-cli 0.1.1 → 0.1.3

package/README.md

@@ -54,6 +54,7 @@ The agent will use `bp` commands automatically. Your real login sessions are pre
 - **CLI-native** — Any agent with bash access can use it. No MCP protocol, no SDK integration needed
 - **Auto-snapshot** — Every action returns page state with numbered `[ref]` elements, so the agent always knows what's on screen
 - **Lightweight** — 78KB npm package. No bundled Chromium (unlike Playwright's 400MB+)
+- **Rich editor support** — Works with contenteditable editors (Draft.js, ProseMirror, Quill, Slate) and Shadow DOM elements out of the box
 
 ## Comparison
 
@@ -127,7 +128,7 @@ Popup windows (target="_blank", window.open) are auto-detected. Run `bp tabs` to
 | `bp net` | List recent requests (`--url`, `--method`, `--status`, `--type`) |
 | `bp net show <id>` | Full request/response details (`--save <file>`) |
 | `bp net block <pattern>` | Block requests matching URL pattern |
-| `bp net mock <pattern>` | Mock responses (`--body`, `--file`, `--status`) |
+| `bp net mock <pattern>` | Mock responses (`--body`, `--file`) |
 | `bp net headers <pattern> <header...>` | Add/override request headers |
 | `bp net rules` | List active interception rules |
 | `bp net remove [id]` | Remove rule(s) (`--all`) |
@@ -149,13 +150,18 @@ Action commands return a snapshot of interactive elements, each with a `[ref]` n
 
 ```
 [1] link "Home"
-[2] textbox "Search"
-[3] button "Submit"
+[2] textbox "Search"                ← <input>, <textarea>, or contenteditable
+[3] textbox ""                      ← unnamed input (still interactive)
+[4] combobox ""                     ← <select> dropdown
+[5] spinbutton "Quantity"           ← <input type="number">
+[6] button "Submit"
+[7] checkbox "Agree" checked
+[8] slider "Volume"                 ← <input type="range">
 ```
 
 Use the number in subsequent commands: `bp click 1`, `bp type 2 "hello"`.
 
-Refs are scoped to the current page — they refresh automatically after every action.
+Refs are scoped to the current page — they refresh automatically after every action. Elements inside Shadow DOM are included automatically.
 
 ## Output
 
@@ -197,6 +203,22 @@ bp click 5                        # click "Search by image"
 bp upload ~/Downloads/photo.jpg    # auto-finds file input, triggers upload
 ```
 
+## Rich Text Editors & Shadow DOM
+
+`bp type` works with contenteditable-based editors (Draft.js, ProseMirror, Quill, Slate, Lexical). They appear as `textbox` in snapshots:
+
+```bash
+bp type 3 "new content" --clear     # replace content in a rich text editor
+```
+
+Shadow DOM elements are traversed automatically — no special commands needed. Elements inside open shadow roots (even deeply nested) appear in snapshots and can be clicked/typed normally.
+
+For `<select>` dropdowns (shown as `combobox`), use `bp eval`:
+
+```bash
+bp eval 'document.querySelector("select").value = "opt2"; document.querySelector("select").dispatchEvent(new Event("change",{bubbles:true}))'
+```
+
 ## Network Interception
 
 Monitor, block, and mock HTTP requests:
@@ -213,7 +235,7 @@ bp net block "*ads*"
 
 # Mock API responses
 bp net mock "*api/data*" --body '{"ok":true}'
-bp net mock "*api/users*" --file mock.json --status 200
+bp net mock "*api/users*" --file mock.json
 
 # Override request headers
 bp net headers "*api*" "Authorization:Bearer test123"
@@ -225,6 +247,23 @@ bp net remove --all                    # clear all rules
 bp net clear                           # clear captured request log
 ```
 
+## Testing
+
+121 tests across 4 Playwright projects, adapted from Playwright/Puppeteer test fixtures and real-world sites:
+
+```bash
+npm test                    # core + compat + network (105 tests, ~2min)
+npm run test:integration    # real-site tests against the-internet.herokuapp.com
+npm run test:all            # all 121 tests
+```
+
+| Project | Tests | Coverage |
+|---------|-------|----------|
+| **core** | 41 | lifecycle, nav, click, type, press, eval, screenshot, pdf, cookies, frames, upload, auth, tabs, dialogs |
+| **compat** | 46 | contenteditable (5 variants), Shadow DOM (3 levels), input types, scrollable, overlay, select |
+| **network** | 19 | request monitoring, block, mock, headers, rule management |
+| **integration** | 15 | checkboxes, dropdown, key presses, dynamic controls, Shadow DOM, nested frames, TinyMCE, large DOM |
+
 ## Requirements
 
 - Chrome 144+ / Edge / Brave (any Chromium-based browser)

package/dist/cli.js

@@ -549,7 +549,8 @@ Workflow:
   bp open <url>                       # navigate \u2014 returns snapshot with [ref] numbers
   bp click <ref>                      # interact \u2014 returns updated snapshot
   bp type <ref> <text>                # input text \u2014 returns updated snapshot
-  bp press <key>                      # keyboard \u2014 returns updated snapshot
+  bp keyboard <text>                  # type via keyboard events (Google Docs etc.)
+  bp press <key>                      # press key \u2014 returns updated snapshot
   bp eval <js>                        # run JavaScript (escape hatch for anything)
 
 Refs:
@@ -563,12 +564,17 @@ Output:
   Actions return: {"ok":true, "title":"...", "url":"...", "elements":[...]}
   Errors return:  {"ok":false, "error":"...", "hint":"..."}
 
+Canvas editors (Google Docs, Sheets, Figma):
+  bp keyboard "text" --click ".editor"               # click to focus, then type
+  bp keyboard "text" --clear                          # select all + delete, then type
+  bp press Meta+b                                     # toggle bold (works in Docs)
+
 Edge cases:
-  bp upload <ref> <filepath>                         # file input upload
-  bp auth <user> <pass>                              # HTTP Basic Auth
-  bp frame                                           # list iframes
-  bp frame 1                                         # eval in iframe context
-  bp frame 0                                         # back to top frame
+  bp upload <filepath>                                # file input upload (auto-detect)
+  bp auth <user> <pass>                               # HTTP Basic Auth
+  bp frame                                            # list iframes
+  bp frame 1                                          # eval in iframe context
+  bp frame 0                                          # back to top frame
   Dialogs (alert/confirm) are auto-handled by the daemon.
 
 Eval (replaces scroll, back, forward, extract, etc.):
@@ -808,6 +814,73 @@ program.command("type <ref> <text>").description("Type text into element and ret
     emitSnapshot(await snap(transport, sessionId, state.activeTargetId, limit));
   });
 }));
+async function typeViaKeyboard(t, sid, text) {
+  for (const char of text) {
+    if (char === "\n") {
+      await dispatchKey(t, sid, "Enter");
+    } else if (char === "	") {
+      await dispatchKey(t, sid, "Tab");
+    } else if (char.charCodeAt(0) >= 32 && char.charCodeAt(0) <= 126) {
+      const code = char === " " ? "Space" : `Key${char.toUpperCase()}`;
+      const keyCode = char.toUpperCase().charCodeAt(0);
+      await t.send("Input.dispatchKeyEvent", { type: "keyDown", key: char, code, windowsVirtualKeyCode: keyCode, text: char }, sid);
+      await t.send("Input.dispatchKeyEvent", { type: "keyUp", key: char, code, windowsVirtualKeyCode: keyCode }, sid);
+    } else {
+      await t.send("Input.insertText", { text: char }, sid);
+    }
+  }
+}
+program.command("keyboard <text>").description("Type text via keyboard events (for canvas editors like Google Docs)").option("-c, --clear", "select all + delete before typing").option("-s, --submit", "press Enter after typing").option("-d, --delay <ms>", "delay between keystrokes in ms").option("--click <selector>", "click element by CSS selector first to focus it").option("-l, --limit <n>", "max elements in snapshot", "50").addHelpText("after", `
+Unlike 'type', this does not target a specific element. It sends real
+keyboard events to whatever is currently focused \u2014 works with canvas-based
+editors (Google Docs, Google Sheets, Figma) that don't expose DOM inputs.
+
+Use --click to focus an element before typing (sends a real CDP mouse click).
+
+Examples:
+  bp keyboard "hello world"
+  bp keyboard "new content" --clear
+  bp keyboard "search query" --submit
+  bp keyboard "Hello Docs!" --click ".kix-appview-editor"
+  bp keyboard "slow typing" --delay 50`).action(action(async (text, opts) => {
+  const limit = parseLimit(opts.limit);
+  await withPilot(async ({ transport, sessionId, state }) => {
+    if (opts.click) {
+      const { result } = await transport.send("Runtime.evaluate", {
+        expression: `JSON.stringify((function(){var el=document.querySelector(${JSON.stringify(opts.click)});if(!el)return null;el.scrollIntoView({block:'center'});var r=el.getBoundingClientRect();return{x:r.x+r.width/2,y:r.y+r.height/2}})())`,
+        returnByValue: true
+      }, sessionId);
+      const coords = result.value ? JSON.parse(result.value) : null;
+      if (!coords) throw new Error(`Element not found: ${opts.click}`);
+      await dispatchClick(transport, sessionId, coords.x, coords.y);
+      await new Promise((r) => setTimeout(r, 300));
+    }
+    if (opts.clear) {
+      await dispatchKey(transport, sessionId, "Meta+a");
+      await dispatchKey(transport, sessionId, "Delete");
+    }
+    if (opts.delay) {
+      const delay = parseInt(opts.delay, 10);
+      for (const char of text) {
+        if (char === "\n") {
+          await dispatchKey(transport, sessionId, "Enter");
+        } else if (char.charCodeAt(0) >= 32 && char.charCodeAt(0) <= 126) {
+          const code = char === " " ? "Space" : `Key${char.toUpperCase()}`;
+          const keyCode = char.toUpperCase().charCodeAt(0);
+          await transport.send("Input.dispatchKeyEvent", { type: "keyDown", key: char, code, windowsVirtualKeyCode: keyCode, text: char }, sessionId);
+          await transport.send("Input.dispatchKeyEvent", { type: "keyUp", key: char, code, windowsVirtualKeyCode: keyCode }, sessionId);
+        } else {
+          await transport.send("Input.insertText", { text: char }, sessionId);
+        }
+        await new Promise((r) => setTimeout(r, delay));
+      }
+    } else {
+      await typeViaKeyboard(transport, sessionId, text);
+    }
+    if (opts.submit) await dispatchKey(transport, sessionId, "Enter");
+    emitSnapshot(await snap(transport, sessionId, state.activeTargetId, limit));
+  });
+}));
 program.command("press <key>").description("Press key combo (e.g. Enter, Escape, Control+a) and return snapshot").option("-l, --limit <n>", "max elements in snapshot", "50").addHelpText("after", "\nKeys: Enter, Escape, Tab, Space, Backspace, Delete,\n      ArrowUp, ArrowDown, ArrowLeft, ArrowRight,\n      Home, End, PageUp, PageDown\nModifiers: Control (Ctrl), Shift, Alt, Meta (Cmd)\n\nExamples:\n  bp press Enter\n  bp press Escape\n  bp press Control+a\n  bp press Meta+c").action(action(async (key, opts) => {
   const limit = parseLimit(opts.limit);
   await withPilot(async ({ transport, sessionId, state }) => {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "browser-pilot-cli",
-  "version": "0.1.1",
+  "version": "0.1.3",
   "description": "CLI tool to control your browser via Chrome DevTools Protocol",
   "type": "module",
   "bin": {

package/playwright.config.ts

@@ -1,36 +0,0 @@
-import { defineConfig } from '@playwright/test';
-
-export default defineConfig({
-  testDir: './tests',
-  timeout: 30_000,
-  retries: 0,
-  workers: 1, // bp uses a single browser session — must be serial
-  globalSetup: './tests/global-setup.ts',
-  globalTeardown: './tests/global-teardown.ts',
-  use: {
-    baseURL: 'http://127.0.0.1:18274',
-  },
-  webServer: {
-    command: 'node tests/server.mjs 18274',
-    port: 18274,
-    reuseExistingServer: true,
-  },
-  projects: [
-    {
-      name: 'core',
-      testMatch: /core\.spec/,
-    },
-    {
-      name: 'compat',
-      testMatch: /fill|click|snapshot/,
-    },
-    {
-      name: 'network',
-      testMatch: /network\.spec/,
-    },
-    {
-      name: 'integration',
-      testMatch: /real-site\.spec/,
-    },
-  ],
-});

package/test-results/.last-run.json

@@ -1,4 +0,0 @@
-{
-  "status": "passed",
-  "failedTests": []
-}