browser-pilot-cli 0.1.6 → 0.2.1

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025-2026 Browser Pilot contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -10,10 +10,25 @@ npm install -g browser-pilot-cli
 
 ### 1. Enable Chrome remote debugging (one-time)
 
-Open `chrome://inspect/#remote-debugging` in Chrome (144+) and click Allow. No command-line flags, no restart needed.
+Open `chrome://inspect/#remote-debugging` in Chrome (144+) and enable remote
+debugging. No command-line flags or restart are needed. Chrome may show its
+separate Allow dialog when `bp connect` requests the actual connection.
 
 > Chrome 136 disabled the old `--remote-debugging-port` flag for security. Chrome 144 introduced this new UI toggle as the replacement — browser-pilot uses this.
 
+Check every installed supported browser and its exact setup state with:
+
+```bash
+bp browsers
+```
+
+Discovery is passive: `bp browsers`, bridge startup, and `initialize` never open
+a Chrome WebSocket and never display the Allow dialog.
+
+Use `bp connect --browser <id|product|channel>` for an explicit choice. Product
+embedders can pass the same selector to `browser-pilot bridge --stdio
+--browser <selector>`; otherwise selection follows a stable platform order.
+
 ### 2. Install the plugin for your agent
 
 **Claude Code:**
@@ -53,7 +68,9 @@ The agent will use `bp` commands automatically. Your real login sessions are pre
 - **Real login sessions** — Operates your actual browser profile. Cookies, extensions, logins all intact
 - **CLI-native** — Any agent with bash access can use it. No MCP protocol, no SDK integration needed
 - **Auto-snapshot** — Every action returns page state with numbered `[ref]` elements, so the agent always knows what's on screen
-- **Lightweight** — 78KB npm package. No bundled Chromium (unlike Playwright's 400MB+)
+- **Adaptive page views** — Bounded read, text search, DOM metadata, page geometry, and annotated screenshots let an agent choose the smallest useful representation
+- **Verified page primitives** — Scroll and dropdown tools return fresh state and typed evidence instead of requiring ad hoc JavaScript
+- **Lightweight npm install** — About 200KB as an npm tarball. No bundled Chromium (unlike Playwright's 400MB+)
 - **Rich editor support** — Works with contenteditable editors (Draft.js, ProseMirror, Quill, Slate) and Shadow DOM elements out of the box
 
 ## Comparison
@@ -67,7 +84,7 @@ The agent will use `bp` commands automatically. Your real login sessions are pre
 | **Auto-snapshot after action** | Yes | Yes | No | Yes |
 | **Network interception** | Yes (block/mock/headers) | Yes | Yes | No |
 | **Multi-browser** | Chromium-only | Chromium + Firefox + WebKit | Chromium-only | Chromium-only |
-| **Dialog auto-handling** | Yes | Yes | No | Yes |
+| **Dialog handling** | Explicit | Automatic | Manual | Automatic |
 | **JSON output** | Default | MCP structured | MCP structured | Python objects |
 | **File upload** | Auto-detect input | Yes | No | Yes |
 
@@ -86,7 +103,100 @@ CLI Process ──── HTTP/Unix Socket ──── Daemon Process (persisten
                                        └── Pilot window (agent operates here)
 ```
 
-The daemon maintains a single CDP WebSocket connection. A pulsing blue glow around the Pilot window indicates the agent is active.
+The Broker maintains isolated state for each Agent. A private connection
+supervisor owns the Broker's single browser-level CDP WebSocket and proxies it
+to the daemon. It also owns only Browser Pilot-created tabs so it can clean them
+after a Broker crash without touching user-opened tabs. A pulsing blue glow
+around a Pilot window indicates Agent activity.
+
+## Platform Evolution
+
+Browser Pilot now routes direct `bp` commands and embedded clients through the
+same Agent-neutral, multi-client Browser Broker.
+The approved architecture and execution plans are:
+
+- [Platform specification](docs/architecture/browser-pilot-platform-spec.md)
+- [Universal Agent integration plan](docs/plans/universal-agent-integration.md)
+- [Browser capability and reliability plan](docs/plans/browser-capability-evolution.md)
+- [Stdio bridge integration contract](docs/integration/stdio-bridge.md)
+- [Stdio black-box conformance suite](docs/integration/stdio-conformance.md)
+- [Reference consumer adapters](https://github.com/relixiaobo/browser-pilot/tree/main/examples/adapters),
+  including Tenon and OpenClaw lifecycle mappings
+
+The public integration direction remains CLI-only: one-shot commands for direct
+Agent use and a persistent `bridge --stdio` mode for products that embed the
+official executable. Browser Pilot will not require an extension, Native SDK,
+or MCP server. The repository's consumer adapters are source examples built
+only on the stdio contract; they are intentionally excluded from the npm
+package and do not create a second public library API.
+
+The dedicated Pilot window remains the default managed tab set for independent
+Agent work. The Broker architecture also includes all eligible user tabs in the
+Agent's inventory without a separate grant step, so an Agent can operate a page
+the user already opened. Invoking or exposing the tool is the authorization
+boundary; products may apply their own approval UX or remove operations when
+launching the bridge. Bulk cleanup remains limited to managed tabs, and user
+tabs remain open when a session ends.
+
+The `browser-pilot bridge --stdio` transport, Broker lifecycle, browser tool
+dispatch, event replay, protected Artifacts, and protocol 1.1 transport limit
+negotiation are implemented. Browser disconnect and explicit reconnect handling, scoped
+download Artifacts, Workspace resource isolation, and typed watchdog events for
+stalled navigation, selected-frame detach, pending dialogs, and repeated
+browser-observable no-progress actions are also implemented. Bounded page
+search, DOM metadata lookup, scrolling, native/ARIA dropdown operations,
+Observation page geometry, and viewport screenshot annotations adapt the most
+useful browser-use inspection patterns to the same scoped Broker contract. An
+internal managed-target janitor is the sole owner of the browser-level CDP
+connection, proxies daemon traffic over private IPC, and closes only
+Broker-created tabs and their managed popup descendants if the daemon exits or
+crashes. It does not persist target IDs or close user tabs. Broker startup and
+browser discovery are passive. Only `bp connect` or the versioned
+`browser.connect` tool requests Chrome authorization; concurrent clients share
+one in-flight request, and a failed or dropped connection is never retried by a
+timer.
+Compatible Agent products reuse one per-user Broker through protocol
+negotiation. Protected shutdown cannot terminate live embedded clients, and an
+incompatible product must explicitly select a separate `BROWSER_PILOT_HOME`.
+Global npm installation, local npm/npx use, and product-bundled absolute-path
+launches are covered by distribution black-box tests. Native self-contained
+release artifacts use one executable for the public CLI and its private Broker
+and janitor roles, with no runtime download or system-Node fallback.
+
+## Installation and Embedding
+
+Agent-managed installation remains supported:
+
+```bash
+npm install -g browser-pilot-cli
+bp connect
+bp tabs
+```
+
+A project can pin Browser Pilot locally and run the same CLI without a global
+installation:
+
+```bash
+npm install --save-exact browser-pilot-cli@0.2.1
+npx --no-install browser-pilot tabs
+```
+
+Agent products should pin either the npm package or a native archive from the
+GitHub release. Launch an absolute path directly with `bridge --stdio`; do not
+invoke a shell, import `src/*`, or depend on Broker locator/socket files. The
+npm form launches `node_modules/browser-pilot-cli/dist/cli.js` with the
+product's pinned Node runtime. The native archive launches its single
+`browser-pilot` executable and does not require Node to be installed.
+
+Each native archive contains `manifest.json`, per-file `SHA256SUMS`, licenses,
+and an adjacent archive checksum. The manifest reports the actual signature
+kind: Developer ID or ad-hoc on macOS, Authenticode or unsigned on Windows, and
+unsigned on Linux. Release automation never labels an artifact signed when its
+signing credentials were unavailable.
+
+All three forms use the same protocol and compatible per-user Broker. Use a
+distinct absolute `BROWSER_PILOT_HOME` only when a product deliberately needs
+an incompatible isolated Broker.
 
 ## Commands
 
@@ -96,8 +206,15 @@ The daemon maintains a single CDP WebSocket connection. A pulsing blue glow arou
 |---------|---------|-------------|
 | `bp open <url>` | snapshot | Navigate to URL |
 | `bp snapshot` | snapshot | Get interactive elements |
+| `bp read [selector]` | text | Get bounded readable page/region content |
+| `bp search <text>` | matches | Find bounded visible text and nearby context |
+| `bp find <selector>` | elements | Inspect bounded DOM metadata and requested attributes |
+| `bp scroll [direction]` | snapshot | Scroll page/element/text and return fresh state |
+| `bp dropdown <target>` | options | List native or exposed ARIA dropdown options |
+| `bp select <target> <option>` | snapshot | Select and verify a dropdown option |
 | `bp click <ref>` | snapshot | Click element by ref number (`--double`, `--right`) |
-| `bp click 0 --xy x,y` | snapshot | Click at viewport coordinates (canvas, maps) |
+| `bp click --xy x,y` | snapshot | Click at viewport coordinates (canvas, maps) |
+| `bp locate <selector>` | coords | Get element center x,y + size (for `click --xy`) |
 | `bp type <ref> <text>` | snapshot | Type into element (`--clear`, `--submit`) |
 | `bp keyboard <text>` | snapshot | Type via keyboard events (`--click`, `--clear`) |
 | `bp press <key>` | snapshot | Press key (Enter, Escape, Control+a, Meta+c) |
@@ -107,7 +224,7 @@ The daemon maintains a single CDP WebSocket connection. A pulsing blue glow arou
 
 | Command | Description |
 |---------|-------------|
-| `bp screenshot [file]` | Capture screenshot (`--full`, `--selector`) |
+| `bp screenshot [file]` | Capture screenshot (`--full`, `--selector`, `--annotate [refs]`) |
 | `bp pdf [file]` | Save page as PDF (`--landscape`) |
 | `bp cookies [domain]` | View cookies (includes HttpOnly) |
 
@@ -118,10 +235,16 @@ The daemon maintains a single CDP WebSocket connection. A pulsing blue glow arou
 | `bp upload <filepath>` | Upload file (auto-finds `<input type="file">`) |
 | `bp auth <user> <pass>` | Set HTTP Basic Auth credentials (`--clear`) |
 | `bp frame [index]` | List or switch iframe context (0 = top) |
+| `bp dialogs` | List pending JavaScript dialogs |
+| `bp dialog <id> --accept\|--dismiss` | Explicitly respond to a dialog (`--prompt`) |
 
-Dialogs (`alert`/`confirm`/`prompt`) are auto-handled by the daemon.
+Dialogs remain pending until explicitly accepted or dismissed. One-shot CLI
+dialogs are scoped to the compatibility Workspace and isolated from embedded
+Broker clients.
 
-Popup windows (target="_blank", window.open) are auto-detected. Run `bp tabs` to see and switch to them.
+Run `bp tabs` to list Pilot-managed tabs, their popups, and eligible tabs the
+user opened elsewhere in the same browser. `bp tab <n>` switches control to any
+listed tab.
 
 ### Network
 
@@ -141,10 +264,10 @@ Popup windows (target="_blank", window.open) are auto-detected. Run `bp tabs` to
 | Command | Description |
 |---------|-------------|
 | `bp connect` | Connect to Chrome, create pilot window |
-| `bp disconnect` | Close pilot window, stop daemon |
-| `bp tabs` | List pilot tabs (auto-adopts popups) |
-| `bp tab <n>` | Switch tab |
-| `bp close` | Close current tab (`--all`) |
+| `bp disconnect` | Close the CLI Pilot window; stop the daemon when no embedded client is live |
+| `bp tabs` | List all controllable tabs in the current browser |
+| `bp tab <n>` | Switch to any listed managed or user tab |
+| `bp close` | Close the current tab (`--all` closes Pilot-managed tabs only) |
 
 ## Refs
 
@@ -163,14 +286,22 @@ Action commands return a snapshot of interactive elements, each with a `[ref]` n
 
 Use the number in subsequent commands: `bp click 1`, `bp type 2 "hello"`.
 
-Refs are scoped to the current page — they refresh automatically after every action. Elements inside Shadow DOM are included automatically.
+Refs are scoped to the current page — they refresh automatically after every
+action. Elements inside Shadow DOM are included automatically. Snapshot JSON
+may also include a `page` block with viewport/document size, scroll position,
+remaining pixels, and scroll percentages.
+
+Across one-shot CLI processes, active target, frame, and refs live only in the
+daemon's keyed compatibility Workspace and renewable Lease. They are never
+written to `state.json` or `refs.json`; after Lease expiry the Agent must
+observe again. `bp disconnect` explicitly clears the compatibility Workspace.
 
 ## Output
 
 **JSON by default** when piped (for LLM/script consumption). Human-readable when run in a terminal.
 
 ```json
-{"ok":true, "title":"Example", "url":"https://example.com", "elements":[{"ref":1, "role":"link", "name":"More info"}]}
+{"ok":true,"title":"Example","url":"https://example.com","page":{"viewportWidth":1280,"viewportHeight":720,"documentWidth":1280,"documentHeight":1800,"scrollX":0,"scrollY":0,"pixelsAbove":0,"pixelsBelow":1080,"pixelsLeft":0,"pixelsRight":0,"scrollPercentX":0,"scrollPercentY":0},"elements":[{"ref":1,"role":"link","name":"More info"}]}
 ```
 
 Errors include hints:
@@ -180,6 +311,18 @@ Errors include hints:
 
 Force human output: `bp --human open https://example.com`
 
+`bp screenshot [file]`, `bp pdf [file]`, and `bp net show <id> --save <file>`
+return an absolute local path. Screenshot and PDF bytes are first protected
+Artifacts, explicitly exported to that path, then released from Broker storage.
+`bp upload <file>` similarly imports a protected copy and releases it after the
+browser receives the file; the source file is never removed.
+
+After `bp snapshot`, `bp screenshot page.png --annotate` draws up to the first
+200 current refs on a viewport image; pass a comma-separated list such as
+`--annotate 1,3,8` to limit it. Annotation does not inject elements into the
+page, runs in an isolated JavaScript world, and cannot be combined with full-page
+or selector capture.
+
 ## Eval
 
 `eval` is the escape hatch — anything JavaScript can do:
@@ -188,13 +331,16 @@ Force human output: `bp --human open https://example.com`
 bp eval "history.back()"                            # go back
 bp eval "history.forward()"                         # go forward
 bp eval "location.reload()"                         # reload
-bp eval "window.scrollBy(0, 500)"                   # scroll down
 bp eval "document.querySelector('h1').textContent"   # extract text
 bp eval "document.querySelector('div').innerHTML"    # extract HTML
 bp eval "JSON.stringify(localStorage)"               # read storage
 echo 'complex js here' | bp eval                    # stdin for complex JS
 ```
 
+Prefer `bp read`, `bp search`, `bp find`, `bp scroll`, `bp dropdown`, and
+`bp select` for their bounded output, scoped state, and verification. Use
+`eval` only when no dedicated command represents the operation.
+
 ## File Upload
 
 `bp upload` auto-detects `<input type="file">` on the page:
@@ -219,7 +365,7 @@ For canvas-based editors (Google Docs, Google Sheets, Figma), use `bp keyboard`
 bp keyboard "Hello Docs!" --click ".kix-appview-editor"   # Google Docs
 bp press Meta+b                                            # toggle bold
 bp keyboard "bold text"
-bp click 0 --xy 400,300                                    # click canvas area
+bp click --xy 400,300                                    # click canvas area
 ```
 
 Shadow DOM elements are traversed automatically — no special commands needed. Elements inside open shadow roots (even deeply nested) appear in snapshots and can be clicked/typed normally.
@@ -253,27 +399,47 @@ bp net headers "*api*" "Authorization:Bearer test123"
 
 # Manage rules
 bp net rules                           # list active rules
-bp net remove 2                        # remove rule #2
+bp net remove <id-from-bp-net-rules>   # rule IDs are opaque
 bp net remove --all                    # clear all rules
 bp net clear                           # clear captured request log
 ```
 
 ## Testing
 
-121 tests across 4 Playwright projects, adapted from Playwright/Puppeteer test fixtures and real-world sites:
+Local release tests are deterministic. Real-site checks run separately as
+non-blocking canaries so a third-party outage cannot fail the release gate:
+
+```bash
+npm test                         # unit + local core, compat, and network gates
+npm run test:capabilities        # isolated-Chrome quantitative capability gate
+npm run test:distribution        # packed global/local/product-bundled launch modes
+npm run test:canary              # real-site drift report; non-blocking
+npm run test:canary:strict       # fail on drift or unavailability
+npm run test:integration         # compatibility alias for strict canaries
+npm run test:all                 # release gates plus non-blocking canaries
+```
+
+Automated browser gates, including Playwright and stdio conformance, start their
+own headless Chrome with a temporary profile, random debugging port, and
+isolated Broker home. Teardown stops the Broker and Chrome and removes those
+files. Tests fail closed if that isolation is absent; they never attach to the
+user's Chrome by default. A maintainer can explicitly opt into a destructive
+manual compatibility run against the user's browser by setting
+`BROWSER_PILOT_TEST_USER_CHROME=1`.
+
+Maintainers can build and verify a native artifact with an official,
+SEA-capable Node 22.17.0 runtime:
 
 ```bash
-npm test                    # core + compat + network (105 tests, ~2min)
-npm run test:integration    # real-site tests against the-internet.herokuapp.com
-npm run test:all            # all 121 tests
+BROWSER_PILOT_SEA_NODE=/absolute/path/to/node npm run build:standalone
+npm run verify:standalone
+npm run package:standalone
 ```
 
-| Project | Tests | Coverage |
-|---------|-------|----------|
-| **core** | 41 | lifecycle, nav, click, type, press, eval, screenshot, pdf, cookies, frames, upload, auth, tabs, dialogs |
-| **compat** | 46 | contenteditable (5 variants), Shadow DOM (3 levels), input types, scrollable, overlay, select |
-| **network** | 19 | request monitoring, block, mock, headers, rule management |
-| **integration** | 15 | checkboxes, dropdown, key presses, dynamic controls, Shadow DOM, nested frames, TinyMCE, large DOM |
+The canary report is written to
+`test-results/real-site-canary/report.json`. It distinguishes semantic drift,
+third-party unavailability, and runner errors. Set
+`BROWSER_PILOT_CANARY_REPORT` to use another report path.
 
 ## Requirements