browser-pilot-cli 0.1.0 → 0.1.1

package/README.md

@@ -1,89 +1,94 @@
 # browser-pilot
 
-CLI tool that gives LLM agents control of your real browser — with your logins, cookies, and extensions intact.
+Give your AI agent control of your real browser — with your logins, cookies, and extensions intact. No extension needed.
 
 ```bash
-bp open https://github.com       # navigate — returns interactive elements
-bp click 3                        # click [3] — returns updated page
-bp type 5 "hello" --submit        # type + Enter — returns updated page
-bp eval "document.title"          # run any JavaScript
+npm install -g browser-pilot-cli
 ```
 
+## Agent Setup
+
+### 1. Enable Chrome remote debugging (one-time)
+
+Open `chrome://inspect/#remote-debugging` in Chrome (144+) and click Allow. No command-line flags, no restart needed.
+
+> Chrome 136 disabled the old `--remote-debugging-port` flag for security. Chrome 144 introduced this new UI toggle as the replacement — browser-pilot uses this.
+
+### 2. Install the plugin for your agent
+
+**Claude Code:**
+```
+/plugin marketplace add https://github.com/relixiaobo/browser-pilot.git
+/plugin install browser-pilot@browser-pilot-marketplace
+```
+
+**Codex CLI:**
+```bash
+npx skills add relixiaobo/browser-pilot
+```
+
+**OpenClaw:**
+```bash
+cp -r plugin/skills/browser-pilot ~/.agents/skills/
+```
+
+**Cursor / VS Code Copilot:**
+```bash
+npx skills add relixiaobo/browser-pilot
+```
+
+### 3. Use it
+
+Just tell your agent what you want to do:
+
+- "Open GitHub and check my notifications"
+- "Go to Hacker News and summarize the top 5 posts"
+- "Fill out the form on this page"
+
+The agent will use `bp` commands automatically. Your real login sessions are preserved — no need to re-authenticate.
+
+## Why browser-pilot?
+
+- **No extension required** — Uses Chrome 144's native remote debugging toggle, not the Extension Debugger API
+- **Real login sessions** — Operates your actual browser profile. Cookies, extensions, logins all intact
+- **CLI-native** — Any agent with bash access can use it. No MCP protocol, no SDK integration needed
+- **Auto-snapshot** — Every action returns page state with numbered `[ref]` elements, so the agent always knows what's on screen
+- **Lightweight** — 78KB npm package. No bundled Chromium (unlike Playwright's 400MB+)
+
 ## Comparison
 
 | | browser-pilot | Playwright MCP | Chrome DevTools MCP | browser-use |
 |---|---|---|---|---|
 | **Interface** | CLI (bash) | MCP protocol | MCP protocol | Python SDK |
 | **Login session reuse** | Yes | No | Depends | Yes |
+| **Extension required** | No | No | No | No |
 | **Element refs** | Numbered (accessibility tree) | Named refs (ARIA) | CSS selectors | Numbered (DOM) |
 | **Auto-snapshot after action** | Yes | Yes | No | Yes |
 | **Network interception** | Yes (block/mock/headers) | Yes | Yes | No |
-| **Drag and drop** | No | Yes | Yes | No |
-| **Mobile emulation** | No | Yes | Yes | Yes |
 | **Multi-browser** | Chromium-only | Chromium + Firefox + WebKit | Chromium-only | Chromium-only |
-| **Connection model** | Daemon (one-time Allow) | MCP server | MCP server | Python process |
 | **Dialog auto-handling** | Yes | Yes | No | Yes |
 | **JSON output** | Default | MCP structured | MCP structured | Python objects |
-| **Visual indicator** | Pulsing glow | None | None | None |
 | **File upload** | Auto-detect input | Yes | No | Yes |
-| **Autonomous agent** | No (tool, not agent) | No (tool) | No (tool) | Yes (plans + executes) |
-
-**browser-pilot is best when:**
-- You need your existing login sessions (paywalled content, internal tools)
-- Your LLM has bash access but no MCP support
-- You want every action to return page state automatically
-
-**Use Playwright MCP when:**
-- You need multi-browser or mobile emulation
-- Your LLM supports MCP natively
-- You don't need existing login sessions
-
-**Use browser-use when:**
-- You want an autonomous agent that plans and executes multi-step tasks
-- You're building in Python
-
-## Quick Start
-
-```bash
-# Install
-npm install -g browser-pilot
-
-# Enable debugging in Chrome (one-time)
-# Open chrome://inspect/#remote-debugging → toggle ON
-
-# Connect (click Allow in Chrome's dialog)
-bp connect
-
-# Use
-bp open https://github.com
-bp click 3
-bp type 5 "hello" --submit
-bp eval "document.title"
-bp screenshot page.png
-bp disconnect
-```
 
 ## How It Works
 
 ```
-LLM (bash tool)
+Agent (bash tool)
   │  bp open / bp click / bp eval ...
   ▼
 CLI Process ──── HTTP/Unix Socket ──── Daemon Process (persistent)
                                            │
-                                           │  WebSocket (CDP, one-time Allow)
+                                           │  WebSocket (CDP)
                                            ▼
                                        Chrome (your browser, your profile)
                                        ├── Your windows (untouched)
-                                       └── Pilot window (bp operates here)
+                                       └── Pilot window (agent operates here)
 ```
 
-The daemon maintains a single CDP WebSocket connection. Chrome's "Allow" dialog appears once per session. A pulsing blue glow around the Pilot window indicates the agent is active.
+The daemon maintains a single CDP WebSocket connection. A pulsing blue glow around the Pilot window indicates the agent is active.
 
 ## Commands
 
-Run `bp --help` for full details including workflow, refs, and eval examples.
-
 ### Core Loop
 
 | Command | Returns | Description |
@@ -222,9 +227,9 @@ bp net clear                           # clear captured request log
 
 ## Requirements
 
-- Chrome / Chromium / Edge / Brave (any Chromium-based browser)
+- Chrome 144+ / Edge / Brave (any Chromium-based browser)
 - Node.js >= 18
-- Chrome remote debugging enabled (`chrome://inspect/#remote-debugging`)
+- Remote debugging enabled (`chrome://inspect/#remote-debugging`)
 
 ## License
 

package/dist/cli.js

@@ -245,6 +245,14 @@ var SET_VALUE = `function(text, clear) {
   this.dispatchEvent(new Event('input',  {bubbles:true}));
   this.dispatchEvent(new Event('change', {bubbles:true}));
 }`;
+var IS_CONTENTEDITABLE = `function() {
+  return this.isContentEditable && !(this instanceof HTMLInputElement) && !(this instanceof HTMLTextAreaElement);
+}`;
+var CONTENTEDITABLE_CLEAR = `function() {
+  this.focus();
+  document.execCommand('selectAll');
+  document.execCommand('delete');
+}`;
 var PAGE_INFO = `JSON.stringify({title:document.title,url:location.href})`;
 var PAGE_DIMENSIONS = `JSON.stringify({
   width:  Math.max(document.documentElement.scrollWidth,  document.documentElement.clientWidth),
@@ -431,6 +439,17 @@ var INTERACTIVE_ROLES = /* @__PURE__ */ new Set([
   "menuitemradio",
   "tab"
 ]);
+var ALLOW_EMPTY_NAME = /* @__PURE__ */ new Set([
+  "textbox",
+  "searchbox",
+  "combobox",
+  "listbox",
+  "checkbox",
+  "radio",
+  "spinbutton",
+  "slider",
+  "switch"
+]);
 function saveRefs(targetId, entries) {
   writeFileSync2(REFS_FILE, JSON.stringify({ targetId, entries }), { mode: 384 });
 }
@@ -466,16 +485,20 @@ async function takeSnapshot(transport, sessionId, targetId, limit = 50) {
     if (!node) return;
     if (!node.ignored) {
       const role = node.role?.value;
-      if (role && INTERACTIVE_ROLES.has(role) && node.backendDOMNodeId !== void 0) {
-        const props = Object.fromEntries(
-          (node.properties || []).map((p) => [p.name, p.value?.value])
-        );
+      const props = Object.fromEntries(
+        (node.properties || []).map((p) => [p.name, p.value?.value])
+      );
+      const isEditable = props.editable === "richtext";
+      const isInteractive = role && (INTERACTIVE_ROLES.has(role) || isEditable);
+      if (isInteractive && node.backendDOMNodeId !== void 0) {
         const name = node.name?.value || "";
         const value = node.value?.value;
-        if (!props.disabled && (name || value) && refs.length < limit) {
+        const effectiveRole = isEditable && role === "generic" ? "textbox" : role;
+        const hasIdentity = name || value || ALLOW_EMPTY_NAME.has(effectiveRole) || isEditable;
+        if (!props.disabled && hasIdentity && refs.length < limit) {
           const checked = props.checked === "true" || props.checked === true ? true : void 0;
-          refs.push({ backendNodeId: node.backendDOMNodeId, role, name });
-          elements.push({ ref: refs.length, backendNodeId: node.backendDOMNodeId, role, name, value, checked });
+          refs.push({ backendNodeId: node.backendDOMNodeId, role: effectiveRole, name });
+          elements.push({ ref: refs.length, backendNodeId: node.backendDOMNodeId, role: effectiveRole, name, value, checked });
         }
       }
     }
@@ -747,11 +770,36 @@ program.command("type <ref> <text>").description("Type text into element and ret
   await withPilot(async ({ transport, sessionId, state }) => {
     const objectId = await resolveTarget(transport, sessionId, ref, state.activeTargetId);
     try {
-      await transport.send("Runtime.callFunctionOn", {
+      const { result: ceResult } = await transport.send("Runtime.callFunctionOn", {
         objectId,
-        functionDeclaration: SET_VALUE,
-        arguments: [{ value: text }, { value: !!opts.clear }]
+        functionDeclaration: IS_CONTENTEDITABLE,
+        returnByValue: true
       }, sessionId);
+      if (ceResult.value) {
+        if (opts.clear) {
+          await transport.send("Runtime.callFunctionOn", {
+            objectId,
+            functionDeclaration: CONTENTEDITABLE_CLEAR
+          }, sessionId);
+        } else {
+          await transport.send("Runtime.callFunctionOn", {
+            objectId,
+            functionDeclaration: `function() {
+                this.focus();
+                const sel = window.getSelection();
+                sel.selectAllChildren(this);
+                sel.collapseToEnd();
+              }`
+          }, sessionId);
+        }
+        await transport.send("Input.insertText", { text }, sessionId);
+      } else {
+        await transport.send("Runtime.callFunctionOn", {
+          objectId,
+          functionDeclaration: SET_VALUE,
+          arguments: [{ value: text }, { value: !!opts.clear }]
+        }, sessionId);
+      }
       if (opts.submit) await dispatchKey(transport, sessionId, "Enter");
     } finally {
       await transport.send("Runtime.releaseObject", { objectId }, sessionId).catch(() => {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "browser-pilot-cli",
-  "version": "0.1.0",
+  "version": "0.1.1",
   "description": "CLI tool to control your browser via Chrome DevTools Protocol",
   "type": "module",
   "bin": {
@@ -11,13 +11,16 @@
     "build": "tsup",
     "dev": "tsup --watch",
     "start": "node dist/cli.js",
-    "test": "bash tests/run.sh"
+    "test": "tsup && npx playwright test --project core --project compat --project network",
+    "test:integration": "tsup && npx playwright test --project integration",
+    "test:all": "tsup && npx playwright test"
   },
   "dependencies": {
     "commander": "^13.1.0",
     "ws": "^8.18.0"
   },
   "devDependencies": {
+    "@playwright/test": "^1.59.1",
     "@types/node": "^22.0.0",
     "@types/ws": "^8.5.0",
     "tsup": "^8.0.0",

package/playwright.config.ts

@@ -0,0 +1,36 @@
+import { defineConfig } from '@playwright/test';
+
+export default defineConfig({
+  testDir: './tests',
+  timeout: 30_000,
+  retries: 0,
+  workers: 1, // bp uses a single browser session — must be serial
+  globalSetup: './tests/global-setup.ts',
+  globalTeardown: './tests/global-teardown.ts',
+  use: {
+    baseURL: 'http://127.0.0.1:18274',
+  },
+  webServer: {
+    command: 'node tests/server.mjs 18274',
+    port: 18274,
+    reuseExistingServer: true,
+  },
+  projects: [
+    {
+      name: 'core',
+      testMatch: /core\.spec/,
+    },
+    {
+      name: 'compat',
+      testMatch: /fill|click|snapshot/,
+    },
+    {
+      name: 'network',
+      testMatch: /network\.spec/,
+    },
+    {
+      name: 'integration',
+      testMatch: /real-site\.spec/,
+    },
+  ],
+});

package/test-results/.last-run.json

@@ -0,0 +1,4 @@
+{
+  "status": "passed",
+  "failedTests": []
+}