browser-pilot-cli 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,444 +1,355 @@
1
- # browser-pilot
1
+ # Browser Pilot
2
2
 
3
- Give your AI agent control of your real browser — with your logins, cookies, and extensions intact. No extension needed.
3
+ Give an AI Agent control of the user's real Chrome browser, including existing
4
+ logins, cookies, profiles, and extensions. No browser extension is required.
4
5
 
5
6
  ```bash
6
- npm install -g browser-pilot-cli
7
+ npm install -g browser-pilot-cli@latest
8
+ bp --version
7
9
  ```
8
10
 
9
- ## Agent Setup
11
+ Native releases support Apple Silicon macOS, x64 Linux, and x64 Windows. Intel
12
+ Mac is not supported.
10
13
 
11
- ### 1. Enable Chrome remote debugging (one-time)
14
+ ## Agent Setup
12
15
 
13
- Open `chrome://inspect/#remote-debugging` in Chrome (144+) and enable remote
14
- debugging. No command-line flags or restart are needed. Chrome may show its
15
- separate Allow dialog when `bp connect` requests the actual connection.
16
+ ### 1. Enable Chrome remote debugging
16
17
 
17
- > Chrome 136 disabled the old `--remote-debugging-port` flag for security. Chrome 144 introduced this new UI toggle as the replacement — browser-pilot uses this.
18
+ Open `chrome://inspect/#remote-debugging` in Chrome 144 or newer and enable
19
+ remote debugging. Browser Pilot does not use command-line debugging flags or a
20
+ Chrome extension.
18
21
 
19
- Check every installed supported browser and its exact setup state with:
22
+ Inspect installed browsers without requesting authorization:
20
23
 
21
24
  ```bash
22
25
  bp browsers
23
26
  ```
24
27
 
25
- Discovery is passive: `bp browsers`, bridge startup, and `initialize` never open
26
- a Chrome WebSocket and never display the Allow dialog.
28
+ Connect when browser work is needed:
27
29
 
28
- Use `bp connect --browser <id|product|channel>` for an explicit choice. Product
29
- embedders can pass the same selector to `browser-pilot bridge --stdio
30
- --browser <selector>`; otherwise selection follows a stable platform order.
31
- One connection covers every live Chrome Profile on that endpoint. When several
32
- Profiles are open, `bp connect` lists them without creating a Pilot window; use
33
- `bp profiles` and `bp profile <index>`, or pass `--profile` to `bp open --new`.
30
+ ```bash
31
+ bp connect
32
+ ```
34
33
 
35
- ### 2. Install the plugin for your agent
34
+ Chrome may display one Allow dialog for the browser connection. Browser
35
+ discovery and background-service startup are passive. Concurrent clients share
36
+ one pending connection request, so they do not create an authorization loop.
36
37
 
37
- **Claude Code:**
38
- ```
38
+ ### 2. Install the Agent skill
39
+
40
+ Claude Code:
41
+
42
+ ```text
39
43
  /plugin marketplace add https://github.com/relixiaobo/browser-pilot.git
40
44
  /plugin install browser-pilot@browser-pilot-marketplace
41
45
  ```
42
46
 
43
- **Codex CLI:**
47
+ Codex, Cursor, or VS Code Copilot:
48
+
44
49
  ```bash
45
- npx skills add relixiaobo/browser-pilot
50
+ npx skills add relixiaobo/browser-pilot --skill browser-pilot
46
51
  ```
47
52
 
48
- **OpenClaw:**
53
+ OpenClaw or another Agent Skills-compatible host:
54
+
49
55
  ```bash
50
56
  cp -r plugin/skills/browser-pilot ~/.agents/skills/
51
57
  ```
52
58
 
53
- **Cursor / VS Code Copilot:**
54
- ```bash
55
- npx skills add relixiaobo/browser-pilot
59
+ ### 3. Ask for the browser task
60
+
61
+ - "Fill in the form I already opened."
62
+ - "Check my GitHub notifications."
63
+ - "Download the latest invoice and summarize it."
64
+ - "Open this report in a new tab and save it as PDF."
65
+
66
+ The Agent uses its existing shell tool to run `bp`. Browser Pilot does not add
67
+ native Agent tools, an MCP server, an SDK, or a persistent client protocol.
68
+
69
+ ## Public Interface
70
+
71
+ Browser Pilot has one Agent integration surface:
72
+
73
+ ```text
74
+ Agent -> Browser Pilot skill -> existing shell/command runner
75
+ -> bp CLI -> shared background service -> Chrome
56
76
  ```
57
77
 
58
- ### 3. Use it
78
+ The skill provides progressive disclosure and browser-specific operating
79
+ guidance. The CLI provides stable JSON, errors, state, files, waiting, and
80
+ recovery. The background service is private implementation detail shared by all
81
+ compatible CLI installations for the current OS user.
59
82
 
60
- Just tell your agent what you want to do:
83
+ This same path supports both installation modes:
61
84
 
62
- - "Open GitHub and check my notifications"
63
- - "Go to Hacker News and summarize the top 5 posts"
64
- - "Fill out the form on this page"
85
+ - An Agent installs `browser-pilot-cli` itself.
86
+ - A product bundles a pinned CLI and places its directory on the Agent command
87
+ environment's `PATH`.
65
88
 
66
- The agent will use `bp` commands automatically. Your real login sessions are preserved — no need to re-authenticate.
89
+ Tenon, OpenClaw, Codex, and other shell-capable Agents need no Browser
90
+ Pilot-specific runtime integration. See
91
+ [the embedding reference](plugin/skills/browser-pilot/references/embedding.md).
67
92
 
68
- ## Why browser-pilot?
93
+ ## Browser Scope
69
94
 
70
- - **No extension required** Uses Chrome 144's native remote debugging toggle, not the Extension Debugger API
71
- - **Real login sessions** — Operates your actual browser profile. Cookies, extensions, logins all intact
72
- - **Multi-Profile routing** — Lists user tabs across live Chrome Profiles and creates managed work only in a resolved Profile
73
- - **CLI-native** — Any agent with bash access can use it. No MCP protocol, no SDK integration needed
74
- - **Auto-snapshot** — Every action returns page state with numbered `[ref]` elements, so the agent always knows what's on screen
75
- - **Adaptive page views** — Bounded read, text search, DOM metadata, page geometry, and annotated screenshots let an agent choose the smallest useful representation
76
- - **Verified page primitives** — Scroll and dropdown tools return fresh state and typed evidence instead of requiring ad hoc JavaScript
77
- - **Lightweight npm install** — About 200KB as an npm tarball. No bundled Chromium (unlike Playwright's 400MB+)
78
- - **Rich editor support** — Works with contenteditable editors (Draft.js, ProseMirror, Quill, Slate) and Shadow DOM elements out of the box
95
+ `bp tabs` exposes all eligible page tabs on the authorized browser endpoint:
79
96
 
80
- ## Comparison
97
+ - Browser Pilot-created managed tabs;
98
+ - eligible popups from managed tabs;
99
+ - eligible tabs the user opened in any live Chrome profile.
81
100
 
82
- | | browser-pilot | Playwright MCP | Chrome DevTools MCP | browser-use |
83
- |---|---|---|---|---|
84
- | **Interface** | CLI (bash) | MCP protocol | MCP protocol | Python SDK |
85
- | **Login session reuse** | Yes | No | Depends | Yes |
86
- | **Extension required** | No | No | No | No |
87
- | **Element refs** | Numbered (accessibility tree) | Named refs (ARIA) | CSS selectors | Numbered (DOM) |
88
- | **Auto-snapshot after action** | Yes | Yes | No | Yes |
89
- | **Network interception** | Yes (block/mock/headers) | Yes | Yes | No |
90
- | **Multi-browser** | Chromium-only | Chromium + Firefox + WebKit | Chromium-only | Chromium-only |
91
- | **Dialog handling** | Explicit | Automatic | Manual | Automatic |
92
- | **JSON output** | Default | MCP structured | MCP structured | Python objects |
93
- | **File upload** | Auto-detect input | Yes | No | Yes |
101
+ The Agent can therefore fill a form or inspect a page the user already opened.
102
+ Browser-internal and extension-owned pages are excluded. Existing Chrome tab
103
+ groups do not limit tab control, whether expanded or collapsed, but Browser
104
+ Pilot cannot inspect or manage tab groups without an extension.
94
105
 
95
- ## How It Works
106
+ Browser Pilot does not implement task-intent permission prompts. Invoking the
107
+ tool gives the Agent the full eligible browser capability surface. An Agent
108
+ host may apply its own policy or approval UX, but that is outside the Browser
109
+ Pilot contract.
96
110
 
97
- ```
98
- Agent (bash tool)
99
- │ bp open / bp click / bp eval ...
100
-
101
- CLI Process ──── HTTP/Unix Socket ──── Daemon Process (persistent)
102
-
103
- │ WebSocket (CDP)
104
-
105
- Chrome (your browser, your profile)
106
- ├── Your windows (untouched)
107
- └── Pilot window (agent operates here)
111
+ User-opened tabs remain open when Agent state is released. `bp close --all`
112
+ closes only Browser Pilot-created tabs. `bp close` explicitly closes the
113
+ selected tab, including a user tab.
114
+
115
+ ## Chrome Profiles
116
+
117
+ One authorized browser connection can expose several live Chrome profiles.
118
+
119
+ ```bash
120
+ bp profiles
121
+ bp profiles --identify
122
+ bp profile 2
123
+ bp open https://example.com --new --profile 2
108
124
  ```
109
125
 
110
- The Broker maintains isolated state for each Agent. A private connection
111
- supervisor owns the Broker's single browser-level CDP WebSocket and proxies it
112
- to the daemon. It also owns only Browser Pilot-created tabs so it can clean them
113
- after a Broker crash without touching user-opened tabs. A pulsing blue glow
114
- around a Pilot window indicates Agent activity.
126
+ `bp profiles` is passive and includes representative tabs. Use
127
+ `--identify` only when account-aware identity is needed; it briefly opens and
128
+ closes a visible `chrome://version` page per unidentified profile and accepts
129
+ identity only after exact profile-path verification. Browser Pilot reports
130
+ verified profile/account names rather than forcing the Agent to speak in
131
+ ambiguous "Profile 1" labels.
115
132
 
116
- ## Platform Evolution
133
+ Profile selection controls where new managed tabs are created. It does not
134
+ grant or restrict access to existing eligible tabs.
117
135
 
118
- Browser Pilot now routes direct `bp` commands and embedded clients through the
119
- same Agent-neutral, multi-client Browser Broker.
120
- The approved architecture and execution plans are:
136
+ ## Multi-Agent State
121
137
 
122
- - [Platform specification](docs/architecture/browser-pilot-platform-spec.md)
123
- - [Universal Agent integration plan](docs/plans/universal-agent-integration.md)
124
- - [Browser capability and reliability plan](docs/plans/browser-capability-evolution.md)
125
- - [Stdio bridge integration contract](docs/integration/stdio-bridge.md)
126
- - [Stdio black-box conformance suite](docs/integration/stdio-conformance.md)
127
- - [Reference consumer adapters](https://github.com/relixiaobo/browser-pilot/tree/main/examples/adapters),
128
- including Tenon and OpenClaw lifecycle mappings
129
-
130
- The public integration direction remains CLI-only: one-shot commands for direct
131
- Agent use and a persistent `bridge --stdio` mode for products that embed the
132
- official executable. Browser Pilot will not require an extension, Native SDK,
133
- or MCP server. The repository's consumer adapters are source examples built
134
- only on the stdio contract; they are intentionally excluded from the npm
135
- package and do not create a second public library API.
136
-
137
- The dedicated Pilot window remains the default managed tab set for independent
138
- Agent work. The Broker architecture also includes all eligible user tabs in the
139
- Agent's inventory without a separate grant step, so an Agent can operate a page
140
- the user already opened. Invoking or exposing the tool is the authorization
141
- boundary; products may apply their own approval UX or remove operations when
142
- launching the bridge. Bulk cleanup remains limited to managed tabs, and user
143
- tabs remain open when a session ends.
144
-
145
- The `browser-pilot bridge --stdio` transport, Broker lifecycle, browser tool
146
- dispatch, event replay, protected Artifacts, protocol 1.1 transport limit
147
- negotiation, and protocol 1.2 Chrome Profile routing are implemented. Browser
148
- disconnect and explicit reconnect handling, scoped
149
- download Artifacts, Workspace resource isolation, and typed watchdog events for
150
- stalled navigation, selected-frame detach, pending dialogs, and repeated
151
- browser-observable no-progress actions are also implemented. Bounded page
152
- search, DOM metadata lookup, scrolling, native/ARIA dropdown operations,
153
- Observation page geometry, and viewport screenshot annotations adapt the most
154
- useful browser-use inspection patterns to the same scoped Broker contract. An
155
- internal managed-target janitor is the sole owner of the browser-level CDP
156
- connection, proxies daemon traffic over private IPC, and closes only
157
- Broker-created tabs and their managed popup descendants if the daemon exits or
158
- crashes. It does not persist target IDs or close user tabs. Broker startup and
159
- browser discovery are passive. Only `bp connect` or the versioned
160
- `browser.connect` tool requests Chrome authorization; concurrent clients share
161
- one in-flight request, and a failed or dropped connection is never retried by a
162
- timer.
163
- Compatible Agent products reuse one per-user Broker through protocol
164
- negotiation. Protected shutdown cannot terminate live embedded clients, and an
165
- incompatible product must explicitly select a separate `BROWSER_PILOT_HOME`.
166
- Global npm installation, local npm/npx use, and product-bundled absolute-path
167
- launches are covered by distribution black-box tests. Native self-contained
168
- release artifacts use one executable for the public CLI and its private Broker
169
- and janitor roles, with no runtime download or system-Node fallback.
170
-
171
- ## Installation and Embedding
172
-
173
- Agent-managed installation remains supported:
138
+ Every independent Agent must use one stable key across all short-lived CLI
139
+ processes:
174
140
 
175
141
  ```bash
176
- npm install -g browser-pilot-cli
177
- bp connect
142
+ export BROWSER_PILOT_CLIENT_KEY=product.install.agent-01
178
143
  bp tabs
144
+ bp snapshot
179
145
  ```
180
146
 
181
- A project can pin Browser Pilot locally and run the same CLI without a global
182
- installation:
147
+ Or pass `--client-key` before the command. The key preserves the Agent's
148
+ selected profile, tab, frame, refs, auth, network rules, downloads, and command
149
+ history. Different keys are isolated and cannot overwrite one another.
150
+
151
+ Browser Pilot keeps this state in background-service memory, not global
152
+ `state.json` or `refs.json` files. `bp disconnect` releases only the current
153
+ key. It stops the service only when no other Agent is using it and the invoking
154
+ executable owns that service process.
155
+
156
+ Compatible global, local, and product-bundled CLIs reuse the same per-user
157
+ service through protocol negotiation. Executable identity is required only for
158
+ stopping the whole service. Use a distinct absolute `BROWSER_PILOT_HOME` only
159
+ for a deliberately incompatible isolated installation.
160
+
161
+ ## Core Workflow
183
162
 
184
163
  ```bash
185
- npm install --save-exact browser-pilot-cli@0.3.0
186
- npx --no-install browser-pilot tabs
164
+ bp tabs # find existing user or managed tabs
165
+ bp tab 2 # select by current one-based index
166
+ bp open https://example.com --new # create independent managed work
167
+ bp snapshot # get controls with numbered refs
168
+ bp read main --limit 10000 # get bounded readable content
169
+ bp search "invoice total" # find text with nearby context
170
+ bp click 3 # act on a fresh ref
171
+ bp type 5 "hello" --clear # replace input and verify readback
172
+ bp press Enter # press a key and return fresh state
187
173
  ```
188
174
 
189
- Agent products should pin either the npm package or a native archive from the
190
- GitHub release. Launch an absolute path directly with `bridge --stdio`; do not
191
- invoke a shell, import `src/*`, or depend on Broker locator/socket files. The
192
- npm form launches `node_modules/browser-pilot-cli/dist/cli.js` with the
193
- product's pinned Node runtime. The native archive launches its single
194
- `browser-pilot` executable and does not require Node to be installed.
195
-
196
- Each native archive contains `manifest.json`, per-file `SHA256SUMS`, licenses,
197
- and an adjacent archive checksum. The manifest reports the actual signature
198
- kind: Developer ID or ad-hoc on macOS, Authenticode or unsigned on Windows, and
199
- unsigned on Linux. Release automation never labels an artifact signed when its
200
- signing credentials were unavailable.
201
-
202
- All three forms use the same protocol and compatible per-user Broker. Use a
203
- distinct absolute `BROWSER_PILOT_HOME` only when a product deliberately needs
204
- an incompatible isolated Broker.
205
-
206
- ## Commands
207
-
208
- ### Core Loop
209
-
210
- | Command | Returns | Description |
211
- |---------|---------|-------------|
212
- | `bp open <url>` | snapshot | Navigate to URL |
213
- | `bp open <url> --new --profile <selector>` | snapshot | Create managed work in one live Chrome Profile |
214
- | `bp snapshot` | snapshot | Get interactive elements |
215
- | `bp read [selector]` | text | Get bounded readable page/region content |
216
- | `bp search <text>` | matches | Find bounded visible text and nearby context |
217
- | `bp find <selector>` | elements | Inspect bounded DOM metadata and requested attributes |
218
- | `bp scroll [direction]` | snapshot | Scroll page/element/text and return fresh state |
219
- | `bp dropdown <target>` | options | List native or exposed ARIA dropdown options |
220
- | `bp select <target> <option>` | snapshot | Select and verify a dropdown option |
221
- | `bp click <ref>` | snapshot | Click element by ref number (`--double`, `--right`) |
222
- | `bp click --xy x,y` | snapshot | Click at viewport coordinates (canvas, maps) |
223
- | `bp locate <selector>` | coords | Get element center x,y + size (for `click --xy`) |
224
- | `bp type <ref> <text>` | snapshot | Type into element (`--clear`, `--submit`) |
225
- | `bp keyboard <text>` | snapshot | Type via keyboard events (`--click`, `--clear`) |
226
- | `bp press <key>` | snapshot | Press key (Enter, Escape, Control+a, Meta+c) |
227
- | `bp eval [js]` | value | Run JavaScript (escape hatch for anything) |
228
-
229
- ### Utilities
230
-
231
- | Command | Description |
232
- |---------|-------------|
233
- | `bp screenshot [file]` | Capture screenshot (`--full`, `--selector`, `--annotate [refs]`) |
234
- | `bp pdf [file]` | Save page as PDF (`--landscape`) |
235
- | `bp cookies [domain]` | View cookies (includes HttpOnly) |
236
-
237
- ### Edge Cases
238
-
239
- | Command | Description |
240
- |---------|-------------|
241
- | `bp upload <filepath>` | Upload file (auto-finds `<input type="file">`) |
242
- | `bp auth <user> <pass>` | Set HTTP Basic Auth credentials (`--clear`) |
243
- | `bp frame [index]` | List or switch iframe context (0 = top) |
244
- | `bp dialogs` | List pending JavaScript dialogs |
245
- | `bp dialog <id> --accept\|--dismiss` | Explicitly respond to a dialog (`--prompt`) |
246
-
247
- Dialogs remain pending until explicitly accepted or dismissed. One-shot CLI
248
- dialogs are scoped to the compatibility Workspace and isolated from embedded
249
- Broker clients.
250
-
251
- Run `bp tabs` to list Pilot-managed tabs, their popups, and eligible tabs the
252
- user opened elsewhere across all live Profiles in the same browser endpoint.
253
- Each JSON entry includes its opaque `profileContextId`; `bp tab <n>` switches
254
- control to any listed tab.
255
-
256
- ### Network
257
-
258
- | Command | Description |
259
- |---------|-------------|
260
- | `bp net` | List recent requests (`--url`, `--method`, `--status`, `--type`) |
261
- | `bp net show <id>` | Full request/response details (`--save <file>`) |
262
- | `bp net block <pattern>` | Block requests matching URL pattern |
263
- | `bp net mock <pattern>` | Mock responses (`--body`, `--file`) |
264
- | `bp net headers <pattern> <header...>` | Add/override request headers |
265
- | `bp net rules` | List active interception rules |
266
- | `bp net remove [id]` | Remove rule(s) (`--all`) |
267
- | `bp net clear` | Clear captured request log |
268
-
269
- ### Session
270
-
271
- | Command | Description |
272
- |---------|-------------|
273
- | `bp connect` | Connect to Chrome; create a Pilot window immediately only when Profile routing is unambiguous |
274
- | `bp disconnect` | Close the CLI Pilot window; stop the daemon when no embedded client is live |
275
- | `bp profiles` | List live Chrome Profile contexts and representative tabs |
276
- | `bp profile <index\|id\|label\|name>` | Select a Profile for subsequent managed tabs |
277
- | `bp tabs` | List all controllable tabs in the current browser |
278
- | `bp tab <n>` | Switch to any listed managed or user tab |
279
- | `bp close` | Close the current tab (`--all` closes Pilot-managed tabs only) |
280
-
281
- ## Refs
282
-
283
- Action commands return a snapshot of interactive elements, each with a `[ref]` number:
175
+ Action commands return fresh page state and typed browser-visible evidence.
176
+ Refs belong to the current tab, frame, and page state. Refresh them after
177
+ navigation, document changes, tab/frame switches, reconnect, or `stale_ref`.
284
178
 
285
- ```
286
- [1] link "Home"
287
- [2] textbox "Search" ← <input>, <textarea>, or contenteditable
288
- [3] textbox "" unnamed input (still interactive)
289
- [4] combobox "" ← <select> dropdown
290
- [5] spinbutton "Quantity" ← <input type="number">
291
- [6] button "Submit"
292
- [7] checkbox "Agree" checked
293
- [8] slider "Volume" ← <input type="range">
179
+ Use dedicated page primitives before JavaScript:
180
+
181
+ ```bash
182
+ bp find ".result" --attributes href
183
+ bp scroll --to-text "Payment details"
184
+ bp dropdown 4
185
+ bp select 4 "United States"
186
+ bp screenshot page.png --annotate 1,3,8
294
187
  ```
295
188
 
296
- Use the number in subsequent commands: `bp click 1`, `bp type 2 "hello"`.
189
+ `bp eval` remains an escape hatch when no dedicated command represents the
190
+ operation.
297
191
 
298
- Refs are scoped to the current page — they refresh automatically after every
299
- action. Elements inside Shadow DOM are included automatically. Snapshot JSON
300
- may also include a `page` block with viewport/document size, scroll position,
301
- remaining pixels, and scroll percentages.
192
+ ## Async Work and Recovery
302
193
 
303
- Across one-shot CLI processes, active target, frame, and refs live only in the
304
- daemon's keyed compatibility Workspace and renewable Lease. They are never
305
- written to `state.json` or `refs.json`; after Lease expiry the Agent must
306
- observe again. `bp disconnect` explicitly clears the compatibility Workspace.
194
+ Wait for browser-visible state instead of using shell sleeps:
307
195
 
308
- ## Output
196
+ ```bash
197
+ bp --timeout 30000 wait --url "*/complete"
198
+ bp --timeout 30000 wait --text "Payment received"
199
+ bp --timeout 30000 wait --selector ".result"
200
+ bp --timeout 30000 wait --dialog
201
+ bp --timeout 120000 wait --download
202
+ bp --timeout 30000 wait --popup
203
+ ```
309
204
 
310
- **JSON by default** when piped (for LLM/script consumption). Human-readable when run in a terminal.
205
+ Inspect service and command state after a timeout, interrupted shell call, or
206
+ lost result:
311
207
 
312
- ```json
313
- {"ok":true,"title":"Example","url":"https://example.com","page":{"viewportWidth":1280,"viewportHeight":720,"documentWidth":1280,"documentHeight":1800,"scrollX":0,"scrollY":0,"pixelsAbove":0,"pixelsBelow":1080,"pixelsLeft":0,"pixelsRight":0,"scrollPercentX":0,"scrollPercentY":0},"elements":[{"ref":1,"role":"link","name":"More info"}]}
208
+ ```bash
209
+ bp status
210
+ bp commands --status accepted,dispatched,unknown_outcome
211
+ bp command <command-id>
212
+ bp cancel <command-id>
314
213
  ```
315
214
 
316
- Errors include hints:
317
- ```json
318
- {"ok":false, "error":"Ref [99] not found.", "hint":"Run 'bp snapshot' to refresh element refs."}
319
- ```
215
+ Attach a stable request identity before the first attempt when the caller may
216
+ need to recover the same intended operation:
320
217
 
321
- Force human output: `bp --human open https://example.com`
218
+ ```bash
219
+ bp --request-id task-42-submit click 7
220
+ ```
322
221
 
323
- `bp screenshot [file]`, `bp pdf [file]`, and `bp net show <id> --save <file>`
324
- return an absolute local path. Screenshot and PDF bytes are first protected
325
- Artifacts, explicitly exported to that path, then released from Broker storage.
326
- `bp upload <file>` similarly imports a protected copy and releases it after the
327
- browser receives the file; the source file is never removed.
222
+ Reuse that ID only for recovery of the same intended call. Use a new ID for a
223
+ new action. Browser Pilot deduplicates stable requests and attempts
224
+ cancellation on CLI `SIGINT`/`SIGTERM`, but a dispatched mutation can still
225
+ have an `unknown_outcome`. Inspect the current page before retrying.
328
226
 
329
- After `bp snapshot`, `bp screenshot page.png --annotate` draws up to the first
330
- 200 current refs on a viewport image; pass a comma-separated list such as
331
- `--annotate 1,3,8` to limit it. Annotation does not inject elements into the
332
- page, runs in an isolated JavaScript world, and cannot be combined with full-page
333
- or selector capture.
227
+ Machine errors always include stable `code` and `retryable` fields. Branch on
228
+ `code`, not English text.
334
229
 
335
- ## Eval
230
+ ## Files, Screenshots, PDFs, and Downloads
336
231
 
337
- `eval` is the escape hatch anything JavaScript can do:
232
+ Use explicit task-owned paths or set a default directory:
338
233
 
339
234
  ```bash
340
- bp eval "history.back()" # go back
341
- bp eval "history.forward()" # go forward
342
- bp eval "location.reload()" # reload
343
- bp eval "document.querySelector('h1').textContent" # extract text
344
- bp eval "document.querySelector('div').innerHTML" # extract HTML
345
- bp eval "JSON.stringify(localStorage)" # read storage
346
- echo 'complex js here' | bp eval # stdin for complex JS
235
+ export BROWSER_PILOT_OUTPUT_DIR=/absolute/task/output
236
+ bp screenshot
237
+ bp screenshot full.png --full
238
+ bp pdf report.pdf --landscape
239
+ bp downloads
240
+ bp download 1 invoice.pdf
241
+ bp net show 12 --save response.json
347
242
  ```
348
243
 
349
- Prefer `bp read`, `bp search`, `bp find`, `bp scroll`, `bp dropdown`, and
350
- `bp select` for their bounded output, scoped state, and verification. Use
351
- `eval` only when no dedicated command represents the operation.
244
+ File results return an absolute `file` path, MIME type, byte size, and image
245
+ dimensions when available. An Agent can open that path with its normal image or
246
+ file-reading capability. CLI output remains bounded JSON and never embeds large
247
+ binary payloads.
352
248
 
353
- ## File Upload
249
+ Browser Pilot copies uploads and completed downloads into protected temporary
250
+ storage while processing them. It never removes the source upload or Chrome's
251
+ original download.
354
252
 
355
- `bp upload` auto-detects `<input type="file">` on the page:
253
+ ## Command Groups
356
254
 
357
- ```bash
358
- bp open https://images.google.com
359
- bp click 5 # click "Search by image"
360
- bp upload ~/Downloads/photo.jpg # auto-finds file input, triggers upload
361
- ```
255
+ | Area | Commands |
256
+ |------|----------|
257
+ | Setup and state | `browsers`, `connect`, `status`, `disconnect` |
258
+ | Profiles and tabs | `profiles`, `profile`, `tabs`, `tab`, `open`, `close` |
259
+ | Observation | `snapshot`, `read`, `search`, `find`, `locate` |
260
+ | Interaction | `click`, `type`, `keyboard`, `press`, `scroll`, `dropdown`, `select`, `eval` |
261
+ | Async and recovery | `wait`, `commands`, `command`, `cancel` |
262
+ | Page context | `frame`, `dialogs`, `dialog` |
263
+ | Files and media | `upload`, `downloads`, `download`, `screenshot`, `pdf` |
264
+ | Browser data | `cookies`, `auth`, `net` |
362
265
 
363
- ## Rich Text Editors & Shadow DOM
266
+ Run `bp --help` or `bp <command> --help` for exact options.
364
267
 
365
- `bp type` works with contenteditable-based editors (Draft.js, ProseMirror, Quill, Slate, Lexical). They appear as `textbox` in snapshots:
268
+ ## How It Works
366
269
 
367
- ```bash
368
- bp type 3 "new content" --clear # replace content in a rich text editor
270
+ ```text
271
+ short-lived bp CLI processes
272
+ |
273
+ | private local HTTP/Unix socket
274
+ v
275
+ per-user Browser Pilot service
276
+ |
277
+ | one supervised CDP WebSocket per browser endpoint
278
+ v
279
+ Chrome profiles and eligible tabs
369
280
  ```
370
281
 
371
- For canvas-based editors (Google Docs, Google Sheets, Figma), use `bp keyboard` which sends real keyboard events:
282
+ The service serializes actions per physical tab while allowing independent
283
+ tabs to progress concurrently. Each Agent receives isolated logical state and
284
+ opaque tab/ref addresses. The same physical user tab can be controlled by only
285
+ one Agent at a time.
372
286
 
373
- ```bash
374
- bp keyboard "Hello Docs!" --click ".kix-appview-editor" # Google Docs
375
- bp press Meta+b # toggle bold
376
- bp keyboard "bold text"
377
- bp click --xy 400,300 # click canvas area
378
- ```
287
+ An internal connection supervisor owns Browser Pilot-created tabs so it can
288
+ clean them up after a service crash without touching user-opened tabs. Browser
289
+ disconnect invalidates old tab, frame, and ref identities before reconnect.
379
290
 
380
- Shadow DOM elements are traversed automatically — no special commands needed. Elements inside open shadow roots (even deeply nested) appear in snapshots and can be clicked/typed normally.
291
+ See:
381
292
 
382
- For `<select>` dropdowns (shown as `combobox`), use `bp eval`:
293
+ - [Current architecture](docs/architecture/browser-pilot-platform-spec.md)
294
+ - [Universal Agent integration plan](docs/plans/universal-agent-integration.md)
295
+ - [Browser capability evolution](docs/plans/browser-capability-evolution.md)
296
+ - [Chrome profile routing](docs/plans/profile-context-routing.md)
383
297
 
384
- ```bash
385
- bp eval 'document.querySelector("select").value = "opt2"; document.querySelector("select").dispatchEvent(new Event("change",{bubbles:true}))'
386
- ```
298
+ ## Comparison
387
299
 
388
- ## Network Interception
300
+ | | Browser Pilot | browser-use | Playwright MCP | Chrome DevTools MCP |
301
+ |---|---|---|---|---|
302
+ | Agent interface | Skill + CLI | Python library/Agent runtime | MCP tools | MCP tools |
303
+ | User's signed-in Chrome | Yes | Yes | Usually isolated browser | Depends on launch mode |
304
+ | Browser extension | No | No | No | No |
305
+ | Eligible user tabs | Yes | Browser-session dependent | Browser-session dependent | Yes |
306
+ | Progressive Agent guidance | Skill references | Library/Agent prompts | Tool descriptions | Tool descriptions |
307
+ | Action verification | Typed evidence + fresh state | State/history checks | Fresh snapshots | Manual inspection |
308
+ | Async recovery | Wait + command status + request IDs | Agent/controller lifecycle | Host/MCP lifecycle | Host/MCP lifecycle |
309
+ | Local media results | Paths + metadata | Library objects/paths | MCP image/resource results | MCP results |
310
+ | Network interception | Observe, block, mock, headers | Limited by controller | Supported | Supported |
311
+
312
+ Browser Pilot adopts several strong browser-use ideas without adopting its
313
+ Python Agent runtime: adaptive DOM/accessibility observation, fresh state after
314
+ actions, input readback, obstruction checks, stale-state invalidation,
315
+ watchdogs, and bounded outputs. The Agent interface remains the CLI.
316
+
317
+ ## Product Embedding
318
+
319
+ Bundle the native CLI or `browser-pilot-cli` plus a pinned Node runtime. Verify
320
+ the versioned release index and checksums. Put the bundled command directory on
321
+ the Agent's `PATH`, install the complete skill, and inject:
322
+
323
+ ```text
324
+ BROWSER_PILOT_CLIENT_KEY=<stable logical Agent identity>
325
+ BROWSER_PILOT_OUTPUT_DIR=<absolute task-owned directory>
326
+ ```
389
327
 
390
- Monitor, block, and mock HTTP requests:
328
+ Do not map every CLI command into a native tool. The Agent already has a shell
329
+ tool, the skill controls context growth, and `bp --help` provides runtime
330
+ discovery. Agent-managed and product-bundled installations therefore exercise
331
+ the exact same implementation and guidance.
391
332
 
392
- ```bash
393
- # Monitor traffic
394
- bp net # list recent requests
395
- bp net --url "*api*" --method POST # filter by URL and method
396
- bp net show 3 # full details + response body
397
-
398
- # Block requests
399
- bp net block "*tracking*" # block analytics/tracking
400
- bp net block "*ads*"
401
-
402
- # Mock API responses
403
- bp net mock "*api/data*" --body '{"ok":true}'
404
- bp net mock "*api/users*" --file mock.json
405
-
406
- # Override request headers
407
- bp net headers "*api*" "Authorization:Bearer test123"
408
-
409
- # Manage rules
410
- bp net rules # list active rules
411
- bp net remove <id-from-bp-net-rules> # rule IDs are opaque
412
- bp net remove --all # clear all rules
413
- bp net clear # clear captured request log
414
- ```
333
+ Each release ships a plugin archive, native archives, checksums, and a release
334
+ index binding the CLI version, compatible skill range, protocol range, and
335
+ supported assets. No `darwin-x64` artifact is produced.
415
336
 
416
337
  ## Testing
417
338
 
418
- Local release tests are deterministic. Real-site checks run separately as
419
- non-blocking canaries so a third-party outage cannot fail the release gate:
420
-
421
339
  ```bash
422
- npm test # unit + local core, compat, and network gates
423
- npm run test:capabilities # isolated-Chrome quantitative capability gate
424
- npm run test:host-acceptance # two embedded hosts + CLI + Artifacts + reconnect
425
- npm run test:distribution # packed global/local/product-bundled launch modes
426
- npm run test:canary # real-site drift report; non-blocking
427
- npm run test:canary:strict # fail on drift or unavailability
428
- npm run test:integration # compatibility alias for strict canaries
429
- npm run test:all # release gates plus non-blocking canaries
340
+ npm run test:unit # TypeScript build and all Node tests
341
+ npm test # unit tests plus local Playwright gates
342
+ npm run test:capabilities # isolated-Chrome quantitative capability gate
343
+ npm run test:distribution # packed global/local/bundled CLI verification
344
+ npm run test:canary # non-blocking real-site drift report
345
+ npm run test:canary:strict
430
346
  ```
431
347
 
432
- Automated browser gates, including Playwright and stdio conformance, start their
433
- own headless Chrome with a temporary profile, random debugging port, and
434
- isolated Broker home. Teardown stops the Broker and Chrome and removes those
435
- files. Tests fail closed if that isolation is absent; they never attach to the
436
- user's Chrome by default. A maintainer can explicitly opt into a destructive
437
- manual compatibility run against the user's browser by setting
438
- `BROWSER_PILOT_TEST_USER_CHROME=1`.
348
+ Automated browser tests use isolated temporary Chrome profiles and Broker homes.
349
+ They never attach to the user's Chrome unless a maintainer explicitly opts into
350
+ a controlled manual test.
439
351
 
440
- Maintainers can build and verify a native artifact with an official,
441
- SEA-capable Node 22.17.0 runtime:
352
+ Build and verify a native artifact with an SEA-capable Node runtime:
442
353
 
443
354
  ```bash
444
355
  BROWSER_PILOT_SEA_NODE=/absolute/path/to/node npm run build:standalone
@@ -446,16 +357,21 @@ npm run verify:standalone
446
357
  npm run package:standalone
447
358
  ```
448
359
 
449
- The canary report is written to
450
- `test-results/real-site-canary/report.json`. It distinguishes semantic drift,
451
- third-party unavailability, and runner errors. Set
452
- `BROWSER_PILOT_CANARY_REPORT` to use another report path.
360
+ The root `package.json` version is the release source of truth. The npm version
361
+ lifecycle synchronizes the lockfile, plugin manifests, skill compatibility
362
+ range, and marketplace entry:
363
+
364
+ ```bash
365
+ npm version "$BROWSER_PILOT_RELEASE_VERSION" --no-git-tag-version
366
+ npm run release:check-version
367
+ ```
453
368
 
454
369
  ## Requirements
455
370
 
456
- - Chrome 144+ / Edge / Brave (any Chromium-based browser)
457
- - Node.js >= 18
458
- - Remote debugging enabled (`chrome://inspect/#remote-debugging`)
371
+ - Chrome 144+, Edge, Brave, or another supported Chromium browser
372
+ - Node.js 18 or newer for the npm distribution
373
+ - Remote debugging enabled at `chrome://inspect/#remote-debugging`
374
+ - Apple Silicon macOS, x64 Linux, or x64 Windows for native releases
459
375
 
460
376
  ## License
461
377