browser-pilot-cli 0.4.0 → 0.5.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,694 +0,0 @@
1
- # Stdio Bridge Integration Contract
2
-
3
- Status: browser `tools/call`, scoped Observations, target inventory, Artifacts,
4
- command recovery, event delivery, browser reconnect, and scoped downloads
5
- implemented.
6
-
7
- This document describes the Agent-neutral process boundary. Tenon, OpenClaw,
8
- and other Agent hosts use the same executable and protocol. No consumer imports
9
- Browser Pilot source files, links a Native SDK, installs an extension, or needs
10
- an MCP transport.
11
-
12
- ## Process Launch
13
-
14
- Resolve an exact Browser Pilot executable version from one of these locations:
15
-
16
- 1. The product's bundled dependency.
17
- 2. A project-local npm dependency.
18
- 3. A compatible global installation selected explicitly by the host.
19
-
20
- For release bundles, verify the versioned release index first. It binds the npm
21
- package, Agent plugin/skill compatible CLI range and tested version, protocol
22
- range, supported native assets, and adjacent SHA-256 files. Native releases support `darwin-arm64`,
23
- `linux-x64`, and `win32-x64`; Intel Mac is not supported.
24
-
25
- Launch it directly, without a shell:
26
-
27
- ```text
28
- browser-pilot bridge --stdio
29
- ```
30
-
31
- The three supported layouts are behaviorally equivalent:
32
-
33
- - global npm: the host explicitly resolves the installed `browser-pilot` bin;
34
- - local npm/npx: the host pins `browser-pilot-cli` and resolves its local bin or
35
- absolute `dist/cli.js` path;
36
- - native bundle: the host pins and launches the release archive's absolute
37
- `browser-pilot` executable path.
38
-
39
- The npm layout requires Node 18 or newer and starts the adjacent packaged
40
- `daemon.js` and `managed-target-janitor.js` files as private processes. A native
41
- release is a Node SEA executable with its runtime and dependencies embedded. It
42
- respawns that same executable for private Broker and janitor roles; reserved
43
- `--browser-pilot-internal=*` arguments are not a Host API. It never downloads a
44
- runtime or falls back to a system Node installation.
45
-
46
- Native archives contain `manifest.json`, `SHA256SUMS`, dependency/runtime
47
- licenses, and an adjacent archive checksum. Verify the archive checksum before
48
- unpacking and the manifest-listed file hashes before launch. The manifest's
49
- `signature.kind` is evidence, not an aspirational channel label: `developer_id`
50
- or `adhoc` on macOS, `authenticode` or `unsigned` on Windows, and `unsigned` on
51
- Linux. Hosts that require a trusted publisher must reject ad-hoc or unsigned
52
- artifacts themselves.
53
-
54
- The host may add `--browser <id|product|channel>` to make its initial browser
55
- preference explicit. Without it, Browser Pilot deterministically selects the
56
- first candidate with a recorded debugging endpoint, then stable platform order.
57
- The running Broker retains that
58
- choice in memory; it does not write target or profile selection state to disk.
59
-
60
- Create pipes for stdin, stdout, and stderr. Stdout is protocol-only. Treat each
61
- stderr line as a diagnostic and never parse it as protocol data.
62
-
63
- The executable owns Broker discovery and startup. Hosts must not read PID
64
- files, locator metadata, Unix socket paths, or Windows named-pipe names. Two
65
- products that launch concurrently serialize through the per-user startup lock,
66
- then both reuse the winning compatible Broker. A dead locator is recovered on
67
- the next launch. Broker startup and browser discovery are passive. Only an
68
- explicit `browser.connect` tool call requests Chrome authorization, and
69
- concurrent calls for the same browser share one in-flight WebSocket attempt. A
70
- ready live process with an
71
- unresponsive endpoint is not killed or replaced; launch returns
72
- `browser_disconnected` with `restart_unresponsive_broker` remediation.
73
-
74
- The default is one shared Broker per OS user. Protocol-compatible clients reuse
75
- it even when different Agent products or product versions launched their
76
- bridges. If a product must run an incompatible Browser Pilot version, set an
77
- absolute, product-owned `BROWSER_PILOT_HOME` before process launch. This creates
78
- a deliberately isolated locator, endpoint, Artifact store, and version history
79
- on macOS, Linux, and Windows. Do not silently choose an isolated home after
80
- `protocol_incompatible`: shared-browser coordination and cleanup semantics must
81
- remain an explicit host decision.
82
-
83
- ## Framing
84
-
85
- The transport is JSON-RPC 2.0 over newline-delimited UTF-8 JSON:
86
-
87
- - one complete JSON object per line;
88
- - ordinary requests are dispatched in input order;
89
- - request IDs are non-empty strings or safe integers;
90
- - notifications have no response;
91
- - the default input limit is 1 MiB per line and the default result limit is
92
- 4 MiB; protocol 1.1 clients may negotiate lower per-Connection limits;
93
- - invalid JSON, invalid UTF-8, invalid envelopes, and oversized input terminate
94
- that bridge process after an error response with `id: null` when possible;
95
- - output observes stream backpressure;
96
- - at most 256 ordinary calls and 16 out-of-band control calls are pending per
97
- bridge process; saturation returns retryable `result_too_large` without
98
- dispatching the rejected call;
99
- - EOF releases every Lease owned by that bridge Connection;
100
- - `shutdown` exits only the bridge process, not the shared per-user daemon.
101
-
102
- Hosts must not send concurrent writes that interleave bytes. They may pipeline
103
- complete lines and must correlate responses by JSON-RPC ID. `commands/get`,
104
- `commands/cancel`, and dialog list/respond calls may overtake a pending
105
- `tools/call`, so those responses can arrive earlier. Dialog control needs this
106
- exception because a browser dialog can pause the command that caused it.
107
-
108
- ## Initialization
109
-
110
- `initialize` must be the first successful request on a bridge Connection:
111
-
112
- ```json
113
- {"jsonrpc":"2.0","id":1,"method":"initialize","params":{"client":{"id":"com.example.agent","name":"Example Agent","version":"2.0.0","instanceId":"install:01J..."},"protocol":{"min":{"major":1,"minor":1},"max":{"major":1,"minor":3}},"requestedCapabilities":["browser.control","workspace.manage","observation.read","action.input","artifact.read","event.read"],"launchMode":"embedded","limits":{"maxMessageBytes":1048576,"maxResultBytes":4194304}}}
114
- ```
115
-
116
- The response returns the selected protocol, supported and granted
117
- capabilities, executable and service versions, a process-stable Broker
118
- identity, a Connection ID, browser candidates, and negotiated limits. Each
119
- candidate has a stable `id`, product/channel and `userDataRoot`, aggregate
120
- `state`, separate `processState`, `remoteDebuggingState`, and
121
- `authorizationState`, plus structured `remediation` when action is needed.
122
- Initialization succeeds when no candidate is ready, so a host can present setup
123
- state and poll `browser.discover`. Discovery is passive and reports
124
- authorization as `unknown` until an explicit connection succeeds or fails.
125
- Workspace and Lease creation also remain passive; call `browser.connect` once
126
- before target tools. Branch on structured `error.data.code`; never branch on
127
- English error messages.
128
-
129
- Protocol 1.0 uses the service limits returned by `initialize`. Protocol 1.1 may
130
- send `limits.maxMessageBytes` and `limits.maxResultBytes`; each must be from
131
- 64 KiB through 1 GiB. The selected value is the smaller client/service maximum
132
- and applies only to that bridge Connection. `maxArtifactBytes` and
133
- `eventJournalSize` remain service resource limits, not client preferences.
134
-
135
- Protocol 1.2 adds connection-scoped Chrome Profile routing. It exposes
136
- `browser.profiles.list`, `browser.profiles.select`, and `profileContextId` on
137
- tab inventory and new-target operations. Protocol 1.0/1.1 clients retain their
138
- existing tools; the Broker hides 1.2-only operations and rejects a manually
139
- constructed Profile call with `protocol_incompatible`.
140
-
141
- Protocol 1.3 adds the explicit `browser.profiles.identify` mutation and
142
- structured Profile identity fields. Browser discovery returns `userDataRoot`
143
- instead of the legacy candidate field `profile`, and tab inventory returns
144
- `selected` instead of `active`. `selected` means the target selected by this
145
- Lease; it does not describe Chrome foreground focus. Protocol 1.2 clients keep
146
- the old field vocabulary and do not see the identity tool.
147
-
148
- The initialize request and response use the service's fixed bootstrap limits.
149
- The bridge switches limits only after the successful response has been written,
150
- and waits for that switch before parsing a pipelined next line. Oversized
151
- responses become `result_too_large`; oversized best-effort notifications are
152
- dropped and remain recoverable through `events/poll`.
153
-
154
- An already-running incompatible daemon is never replaced. Initialization
155
- returns `protocol_incompatible` with remediation to use a compatible executable
156
- or a deliberate `BROWSER_PILOT_HOME` isolation boundary. Executable versions
157
- and protocol versions are independent; successful protocol negotiation is what
158
- allows embedded clients from different products or releases to share a Broker.
159
-
160
- ## Lifecycle Methods
161
-
162
- The implemented lifecycle surface is:
163
-
164
- ```text
165
- initialize
166
- tools/list
167
- tools/call
168
- commands/get
169
- commands/cancel
170
- workspaces/create
171
- workspaces/get
172
- workspaces/release
173
- leases/create
174
- leases/heartbeat
175
- leases/release
176
- events/poll
177
- artifacts/get
178
- artifacts/export
179
- artifacts/import
180
- artifacts/retain
181
- artifacts/release
182
- shutdown
183
- ```
184
-
185
- `workspaces/create` accepts an optional `browserId`. Protocol 1.1 clients may
186
- also provide `clientKey` to make active Workspace creation idempotent within
187
- their Principal; reusing a key for another browser fails. Without a browser ID,
188
- the Broker prefers its first ready connection and otherwise uses its selected
189
- stable-order browser binding. It returns a Workspace, its default logical
190
- ManagedTabSet, the complete `managedTabSets` array, and an `eventCursor`.
191
- `workspaces/get` also returns the Workspace's transient
192
- `selectedProfileContextId` when one is selected and the current cursor as a
193
- recovery baseline. Creating a Workspace does not itself create a browser
194
- connection or window. Call the Workspace-scoped `browser.connect` tool to
195
- request Chrome authorization; the first managed navigation then creates the
196
- dedicated browser window in its resolved Profile context.
197
-
198
- `leases/create` accepts a `workspaceId` and optional `ttlMs`. Protocol 1.1
199
- clients may also provide a `clientKey`; repeating that key on the same live
200
- Connection and Workspace returns the same Lease and renews its TTL. The default
201
- Lease is 30 seconds, with a supported range of 1 second through 5 minutes. Heartbeat
202
- well before `expiresAt`. A Lease belongs to its live Connection and cannot be
203
- continued by a replacement bridge process. A replacement Connection from the
204
- same ClientPrincipal may obtain a new Lease for a still-active Workspace.
205
-
206
- Workspace identity belongs to `client.id + client.instanceId` for the lifetime
207
- of the daemon. It is transient and is never restored from disk after Broker
208
- restart. A Workspace released twice returns the same successful release result
209
- while its bounded tombstone remains available.
210
-
211
- ## Tool Calls
212
-
213
- `tools/list` returns only operations implemented by the running Broker and
214
- allowed by negotiated capabilities. Call those tools with one uniform envelope:
215
-
216
- ```json
217
- {"jsonrpc":"2.0","id":8,"method":"tools/call","params":{"name":"browser.observe","arguments":{"limit":50},"workspaceId":"workspace:...","leaseId":"lease:...","targetId":"target:...","commandId":"command:...","idempotencyKey":"observe:01J...","deadlineMs":30000}}
218
- ```
219
-
220
- Connection tools omit Workspace fields. Workspace tools require
221
- `workspaceId + leaseId`; target tools additionally require the opaque
222
- `targetId` returned by `browser.tabs.list` or `browser.open`. Raw CDP target and
223
- session IDs are never public inputs. The Broker validates the negotiated
224
- capability, Lease ownership, tool schema, and target control before dispatch.
225
-
226
- Each tool's `sensitivity.input` and `sensitivity.output` arrays list the classes
227
- that may occur anywhere in that direction. Specific content-bearing schema
228
- nodes carry `x-browser-pilot-sensitivity`, for example:
229
-
230
- ```json
231
- {"type":"string","maxLength":1000000,"x-browser-pilot-sensitivity":["browser_data"]}
232
- ```
233
-
234
- Use the field annotation when mapping `browser.read.result.text`, Observation
235
- element values, auth fields, cookie values, network headers/bodies, upload
236
- Artifact references, prompt text, or eval values into the Agent runtime. Keep
237
- the annotation attached to derived model content; do not infer sensitivity from
238
- field names or English tool descriptions. The annotation does not wrap or alter
239
- the value. Artifact descriptors and BrowserEvents provide their actual runtime
240
- `sensitivity` and should be propagated directly.
241
-
242
- Every `tools/call` response is a Command outcome:
243
-
244
- ```json
245
- {"command":{"id":"command:...","status":"completed","method":"browser.observe","browserConnectionGeneration":3,"idempotencyKey":"observe:01J...","acceptedAt":1,"deadlineAt":30001,"dispatchedAt":2,"completedAt":20,"mutating":false},"result":{"workspaceId":"workspace:...","leaseId":"lease:...","targetId":"target:...","url":"https://example.com","observationId":"observation:...","title":"Example","elements":[],"truncated":false,"truncationReasons":[]}}
246
- ```
247
-
248
- The manifest's output schema describes the inner `result`. Clients should pass
249
- a unique `commandId` when they may need cancellation; otherwise the Broker
250
- generates one. `idempotencyKey` is optional, but retries must reuse it or reuse
251
- the same caller-supplied `commandId`. Reusing a key for different arguments is
252
- rejected. A duplicate completed call returns the recorded result; a duplicate
253
- that is still accepted or dispatched returns its current Command without
254
- dispatching again.
255
-
256
- `commands/get` and `commands/cancel` accept `commandId` plus `workspaceId` for
257
- Workspace commands. They are authorized by Principal ownership rather than the
258
- original Lease, so a replacement bridge from the same product instance can
259
- inspect a known outcome. Cancellation while accepted prevents browser dispatch.
260
- Cancellation after dispatch is best-effort and sets `cancellationRequested`;
261
- it never claims rollback. A mutating command whose deadline elapses after
262
- browser dispatch becomes `unknown_outcome` and is never automatically replayed.
263
- Known tool failures are stored as a completed Command with a nested JSON-RPC
264
- `error`; the original call also returns that error normally.
265
-
266
- ### Chrome Profile routing
267
-
268
- One connected Chrome endpoint may expose ordinary tabs from several live
269
- Profiles. Browser Pilot authorizes the endpoint once; it does not reconnect or
270
- show another Chrome Allow dialog per Profile. `browser.tabs.list` returns every
271
- eligible managed and user-opened tab across those contexts, each with an opaque
272
- `profileContextId`. Existing tabs can be selected and controlled immediately;
273
- Profile selection is routing for new managed targets, not a permission grant.
274
- Browser-internal and extension-owned pages are excluded from ordinary tab and
275
- Profile representative inventory.
276
-
277
- Protocol 1.2+ hosts should use this flow when creating independent work:
278
-
279
- 1. Call `browser.profiles.list`. It is passive and returns neutral labels,
280
- bounded representative tabs, counts, and the Workspace selection.
281
- 2. On protocol 1.3, if account-aware identity is needed, call
282
- `browser.profiles.identify`. It briefly creates visible `chrome://version`
283
- targets, verifies each reported path as an immediate child of the connected
284
- `userDataRoot`, reads bounded Chrome Local State metadata, closes every
285
- temporary target, and caches the result for one connection generation.
286
- `refresh: true` explicitly re-probes.
287
- 3. If there is one Profile, `browser.open` can infer it. If there are several
288
- and no current target or Workspace selection, ask the user which Profile to
289
- use.
290
- 4. Call `browser.profiles.select` with the returned opaque ID, or pass that ID
291
- directly as `browser.open.arguments.profileContextId`.
292
- 5. Treat IDs and verified identity as scoped to the current browser connection
293
- generation. After reconnect, relist Profiles, tabs, frames, and Observations.
294
-
295
- Profile results use `identityStatus: unidentified | verified | unavailable`.
296
- Only verified results may include `profileName`, `accountName`, `accountEmail`,
297
- and `profileDirectory`; unavailable results include a stable
298
- `identityErrorCode` rather than a guessed account. Current codes are
299
- `profile_path_unavailable`, `profile_path_unverified`,
300
- `local_state_unavailable`, and `profile_metadata_missing`. Never expose or persist raw
301
- CDP context IDs. New-target routing uses an explicit call argument, the Lease's logical selected
302
- target, the Workspace selection, or the only available context, in that order.
303
- `browser.profiles.select` clears only the logical selected-target anchor so its
304
- selection applies to the next new managed target; it does not release control,
305
- activate a user tab, or change Chrome focus.
306
-
307
- With multiple unanchored contexts, `browser.open` fails before browser dispatch
308
- with `profile_selection_required`. Relist on `profile_context_stale`; on
309
- `profile_context_unavailable`, follow its setup remediation. An
310
- `unknown_outcome` during Profile-window creation means Chrome may have created
311
- a target that Browser Pilot could not prove absent; list tabs and Profiles and
312
- inspect current state before deciding whether to retry.
313
-
314
- `browser.click` returns a new Observation plus bounded evidence. For example,
315
- a checkbox may return:
316
-
317
- ```json
318
- {"action":"click","status":"verified","kind":"checkbox","effects":["checked_changed","focus_changed"],"checked":true,"focused":true}
319
- ```
320
-
321
- Effects can also include `selected_changed`, `pressed_changed`,
322
- `expanded_changed`, `navigation`, `document_changed`, `dialog_opened`, and
323
- `popup_opened`. A semantic control that does not reach its expected state uses
324
- `mismatch`; coordinate clicks and ref clicks with no conclusive observable
325
- effect use `unavailable`. Treat evidence as browser-level execution evidence,
326
- then inspect the returned Observation to decide whether the task itself
327
- succeeded.
328
-
329
- Every Observation produced by the built-in Browser tools also includes a
330
- bounded `hints` array. Each item is discriminated by `code` and includes
331
- `source`, `confidence`, and a stable `recommendedAction`:
332
-
333
- ```json
334
- {"hints":[{"code":"modal_overlay","source":"observation","confidence":"strong","recommendedAction":"resolve_overlay_first","blocking":true,"refs":[2,5]}]}
335
- ```
336
-
337
- Current codes are `autocomplete`, `modal_overlay`, `filter_controls`,
338
- `authentication_surface`, `access_blocked`, `download`, and
339
- `repeated_action`. Observation hints may contain only refs from the returned
340
- Observation. Authentication state is `present` on the first observed auth
341
- surface, `entered` when a session moves onto one, and `left` when it moves off
342
- one. A session or selected-frame change resets that transition baseline.
343
-
344
- Hints are advisory browser evidence. Use the recommended action as the next
345
- inspection strategy, not as permission, proof of task success, or a reason to
346
- skip normal error handling. Unknown hint codes and unknown response fields must
347
- be ignored. The field is additive for protocol 1.0 and 1.1 and introduces no
348
- new capability; its data is covered by the tool or event's existing
349
- sensitivity and granted capability.
350
-
351
- ### Observation representations
352
-
353
- The browser tools expose complementary bounded representations rather than one
354
- unbounded page dump:
355
-
356
- - `browser.observe` creates semantic elements and scoped refs for actions.
357
- - `browser.read` returns broad readable text from the page or one CSS region.
358
- - `browser.search` returns visible text matches, nearby context, and viewport
359
- geometry, including open Shadow DOM, with at most 200 results.
360
- - `browser.elements.find` returns bounded safe metadata for a CSS query,
361
- optionally across open Shadow DOM. It never exposes DOM/CDP handles and its
362
- results are not action refs.
363
- - `browser.capture` may draw Observation ref boxes for spatial reasoning without
364
- injecting an overlay into the page.
365
-
366
- Observations may include an additive `page` object containing viewport and
367
- document dimensions, scroll coordinates, pixels remaining in each direction,
368
- and horizontal/vertical scroll percentages. Hosts must tolerate its absence
369
- when talking to an older compatible Broker.
370
-
371
- `browser.scroll` moves the page, an Observation ref, a CSS-selected container,
372
- or a visible text match, then returns a fresh Observation and typed evidence.
373
- `browser.dropdown.options` enumerates native or currently exposed ARIA options.
374
- `browser.dropdown.select` verifies native value changes or uses fresh
375
- Observation refs for custom controls. These tools reuse `observation.read` and
376
- `action.input`; they are additive manifest entries, not a protocol-version
377
- fork. Discover them through `tools/list`.
378
-
379
- `browser.type` and `browser.keyboard` also return bounded evidence after the
380
- page has had time to apply a controlled-field update:
381
-
382
- ```json
383
- {"action":"type","status":"verified","kind":"input","sensitive":false,"beforeLength":3,"expectedLength":7,"afterLength":7}
384
- ```
385
-
386
- Use `verification: "require_exact"` when a mismatch must fail the command.
387
- Native text controls and contenteditable receive Chrome editing events;
388
- date/time/color/range controls use their native value setter and emit
389
- `beforeinput` plus `input`, but not an early `change`. Framework rollback,
390
- browser value sanitization, canceled `beforeinput`, and editor interception are
391
- reported from the effective value after the action. Password evidence never
392
- contains the value itself.
393
-
394
- `browser.type` and `browser.keyboard` are guarded composite actions. Browser
395
- Pilot pins the target/session/frame/loader/Document identity and, once input
396
- focus is established, the deep active element. It checks them between
397
- select-all, deletion, individual keyboard characters, and submit. If a change
398
- is detected before the first browser mutation, the call fails with retryable
399
- `action_not_verified`. If one or more steps already ran, it fails with
400
- `unknown_outcome`, for example:
401
-
402
- ```json
403
- {"code":"unknown_outcome","retryable":true,"context":{"action":"keyboard","step":"type_character:4","reason":"focus_changed","dispatchedSteps":4,"remainingStepsStopped":true}}
404
- ```
405
-
406
- Reasons are `target_changed`, `session_changed`, `frame_changed`,
407
- `loader_changed`, `document_changed`, or `focus_changed`. On either failure,
408
- list or switch to the intended target if necessary, create a fresh Observation,
409
- and decide from current page state whether to continue. Never replay the whole
410
- mutation automatically after `unknown_outcome`.
411
-
412
- `browser.press` returns `action: "press"` with bounded effects such as
413
- `value_changed`, `checked_changed`, `selected_changed`, `focus_changed`,
414
- `navigation`, `document_changed`, `dialog_opened`, or `popup_opened`. It returns
415
- `unavailable` with `no_observable_effect` when the key dispatch completed but no
416
- supported effect was visible.
417
-
418
- `browser.upload` reads the file input after assignment. It returns `verified`
419
- only when one file with the expected browser filename remains selected,
420
- `mismatch` when the page cleared or replaced the selection, and `unavailable`
421
- when the target could no longer be read. Evidence contains counts and a filename
422
- match boolean, never the Broker storage path.
423
-
424
- Inventory includes every eligible ordinary user tab plus the Workspace's
425
- managed tabs. A physical tab can be controlled by only one Lease at a time.
426
- Commands for the same physical tab share one actor even though each Workspace
427
- has a different opaque target ID. A controlling Lease can call
428
- `browser.tabs.release` for one target without closing it. After that command
429
- completes and emits `target_control.released`, another Lease may use its own
430
- opaque ID to acquire the tab. This two-step handoff never accepts a destination
431
- Lease ID and never silently steals control. Releasing a Workspace closes managed
432
- tabs but leaves user tabs open.
433
-
434
- ## Events
435
-
436
- Every event is first committed to a bounded per-Workspace journal and then
437
- offered as this best-effort notification:
438
-
439
- ```json
440
- {"jsonrpc":"2.0","method":"events/event","params":{"event":{"id":"event:...","sequence":7,"timestamp":1,"workspaceId":"workspace:...","browserConnectionGeneration":3,"leaseId":"lease:...","targetId":"target:...","type":"dialog","payloadVersion":1,"sensitivity":"browser_data","payload":{"dialogId":"dialog:...","state":"opened","type":"confirm","message":"Continue?","url":"https://example.com"}}}}
441
- ```
442
-
443
- Notifications can interleave with responses and can be dropped under transport
444
- backpressure or a disconnected bridge. Treat them as a low-latency signal, not
445
- as the recovery record. Track the last fully processed cursor and poll:
446
-
447
- ```json
448
- {"jsonrpc":"2.0","id":20,"method":"events/poll","params":{"workspaceId":"workspace:...","cursor":"cursor:6","limit":100}}
449
- ```
450
-
451
- The result contains ordered `events`, `nextCursor`, and `hasMore`. Continue until
452
- `hasMore` is false. If the cursor predates the retained journal, the Broker
453
- returns `cursor_expired` with `earliestCursor` and `latestCursor`; rebuild tab
454
- and observation state, then call `workspaces/get` to establish a new baseline.
455
- Never infer delivery from notification arrival alone, and never advance the
456
- stored cursor past an event the host has not processed.
457
-
458
- Current producers cover command status, Lease expiry, navigation, document
459
- change, target attach/detach, target control, managed popups, dialogs, downloads,
460
- browser connection loss/restoration, network request/response metadata, and
461
- observation invalidation. They also include `watchdog.navigation_stalled`,
462
- `watchdog.frame_detached`, `watchdog.dialog_unhandled`, and
463
- `watchdog.no_progress`. Network and watchdog events never contain credentials,
464
- typed values, refs, headers, request/response bodies, or raw CDP IDs; retrieve
465
- sensitive detail explicitly through the scoped tools.
466
-
467
- `network.response` adds an `access_blocked` hint only for a main-document 403 or
468
- 429, not for failed images, scripts, XHR, or fetches. The threshold
469
- `watchdog.no_progress` event and the action result that reaches the threshold
470
- both contain the same `repeated_action` hint. The reason is either repeated
471
- observable failure or `stagnant_page`, where the same action signature was
472
- repeated while the bounded page fingerprint stayed unchanged. Fingerprints and
473
- action parameters never leave the Broker. Change representation and strategy
474
- instead of automatically repeating the same action or navigation.
475
-
476
- `connection.lost` keeps the last connection generation and causes later browser
477
- tools to fail with retryable `browser_disconnected`. The daemon passively
478
- refreshes the selected browser endpoint metadata and does not select a Profile
479
- context or open a WebSocket. The host calls `browser.connect` when it is
480
- ready to request reconnection. `connection.restored` then carries a strictly
481
- newer generation. Existing Workspaces
482
- and active Leases remain valid, but old target IDs, frame IDs, CDP sessions,
483
- Observations, and refs do not. Poll through the restoration event, call
484
- `browser.tabs.list`, and rebuild target state. Never retry a mutating command whose
485
- recorded status is `unknown_outcome`.
486
-
487
- Every event has a top-level `browserConnectionGeneration`. A delayed ordinary
488
- event from an older connection is discarded before journaling. Historical
489
- `command.status` events and reconnect cleanup events remain in the journal with
490
- their original generation, so consumers can finish old Command state machines
491
- without treating cleanup as current browser state. Commands accepted against an
492
- old generation are fenced before dispatch; commands already dispatched when the
493
- connection changes follow the `unknown_outcome` rule above.
494
-
495
- JavaScript dialogs remain pending. Use `browser.dialogs.list`, then call
496
- `browser.dialogs.respond` with the returned `dialogId`, target, and explicit
497
- `accept` or `dismiss` action. Browser Pilot never auto-accepts a dialog.
498
-
499
- Watchdog events are advisory browser-state signals, not autonomous recovery:
500
-
501
- - `watchdog.navigation_stalled` means `Page.navigate` was dispatched but the
502
- page did not become interactive within 30 seconds. The command is
503
- `unknown_outcome`; observe the target before deciding whether to navigate
504
- again.
505
- - `watchdog.frame_detached` carries the opaque selected `frameId`. Browser Pilot
506
- clears that selection and invalidates the Observation; list frames and create
507
- a fresh Observation before continuing.
508
- - `watchdog.dialog_unhandled` is emitted once when a dialog remains pending for
509
- 15 seconds. It does not accept or dismiss the dialog.
510
- - `watchdog.no_progress` is emitted once after three consecutive completed
511
- actions on the same Lease and target either have browser-observable mismatch
512
- or repeat the same action while the bounded page state remains stagnant.
513
- Observable success resets the mismatch detector; a changed action or page
514
- state resets the repetition detector. Navigation, frame/session changes, and
515
- cleanup reset both. Coordinate/canvas actions whose effects cannot be observed
516
- do not count as mismatch-only failures.
517
-
518
- Browser Pilot never automatically retries, repeats, stops, accepts, or dismisses
519
- an action because of a watchdog event. The host or Agent must inspect current
520
- state and decide. Timers and streaks are in-memory transient state and are
521
- removed on dialog close, session loss, Lease release, Workspace release, or
522
- browser disconnect as applicable.
523
-
524
- ## Artifacts
525
-
526
- `browser.capture` and `browser.pdf` return Artifact descriptors, never base64 or
527
- an internal path. Use `artifacts/get` with the owning active Workspace and Lease
528
- to obtain a protected local path, or `artifacts/export` with an absolute path to
529
- copy the file to client-owned storage. Export does not overwrite by default.
530
-
531
- Artifacts expire after 15 minutes by default. `artifacts/retain` extends that
532
- to the retained TTL; `artifacts/release` removes the bytes immediately.
533
- Workspace release and Broker shutdown also remove owned temporary bytes.
534
- Directories use mode `0700` and files use `0600` where POSIX permissions apply.
535
-
536
- Large screenshots default to a model-sized preview. Pass
537
- `includeOriginal: true` to receive both the original descriptor and a preview
538
- descriptor whose `previewOf` points to the original. The adapter reads the
539
- selected file and converts it to its Agent runtime's native image/file content.
540
-
541
- For spatial reasoning, a viewport-only `browser.capture` call may provide:
542
-
543
- ```json
544
- {"annotations":{"observationId":"observation:...","refs":[1,3,8]}}
545
- ```
546
-
547
- Omitting `refs` requests up to the first 200 refs in that Observation. The
548
- Broker revalidates each ref, clips live geometry to the viewport, draws on an
549
- unattached canvas in a Chrome isolated world, and returns `annotationCount`.
550
- Screenshot bytes are not evaluated in the page's default world. Annotation
551
- cannot be combined with `fullPage` or `selector`. A page can still move itself
552
- between geometry sampling and capture, so hosts should use a recent Observation
553
- and treat the image as visual context rather than durable element identity.
554
-
555
- For upload, first call `artifacts/import` with the owning Workspace, active
556
- Lease, and an absolute client-authorized path:
557
-
558
- ```json
559
- {"jsonrpc":"2.0","id":30,"method":"artifacts/import","params":{"workspaceId":"workspace:...","leaseId":"lease:...","path":"/absolute/path/resume.pdf","mimeType":"application/pdf"}}
560
- ```
561
-
562
- The Broker copies the source into protected storage and returns an
563
- `upload_input` Artifact descriptor. Pass only its `artifactId` to
564
- `browser.upload`; optionally identify a file input with
565
- `observationId + ref` or `inputIndex`. Output Artifacts such as screenshots and
566
- downloads are rejected as upload inputs. Workspace cleanup deletes the imported
567
- copy but never deletes the client-owned source file.
568
-
569
- Downloads have no path-returning tool. Once a controlled target session is
570
- attached, the Broker enables browser-level completion events while preserving
571
- Chrome's default download behavior, and emits `download` events with one of
572
- these states:
573
-
574
- - `capture_unavailable`: Chrome rejected browser-level completion events;
575
- - `started`: includes an opaque `downloadId`, bounded URL, and suggested filename;
576
- - `completed`: includes the Workspace-owned `download` Artifact descriptor;
577
- - `failed` or `cancelled`: includes a stable, bounded reason and byte metadata.
578
-
579
- These four lifecycle states also carry a `download` hint. `started` recommends
580
- waiting, `completed` recommends inspecting the Artifact and includes only its
581
- public `artifactId`, and terminal failures recommend inspecting the bounded
582
- failure reason. `capture_unavailable` is setup state and does not claim that a
583
- download started.
584
-
585
- The private CDP GUID and Chrome source path never appear in protocol output. On
586
- `completed`, use the descriptor's `id` with `artifacts/get` or
587
- `artifacts/export` exactly as for screenshots and PDFs. A session-scoped
588
- `Page.downloadWillBegin` event establishes Workspace/Lease/target ownership;
589
- the browser-level completion event is ignored unless that ownership already
590
- exists. Browser Pilot enables those completion events once per browser
591
- connection generation and Chrome Profile context with
592
- `Browser.setDownloadBehavior({ behavior: "default", eventsEnabled: true })`.
593
- It never supplies `downloadPath`, so Chrome continues to use the user's selected
594
- download directory.
595
-
596
- The Broker validates the completed absolute regular-file path and copies it to
597
- protected Artifact storage. Releasing or expiring the Artifact, Lease, or
598
- Workspace deletes only that copy. The original Chrome download is never moved
599
- or deleted. Size, concurrency, or Artifact quota failures reject only the
600
- Artifact copy and do not call `Browser.cancelDownload`. Downloads from tabs
601
- without an attached controlled session remain ordinary Chrome downloads and
602
- produce no Browser Pilot event.
603
-
604
- ## Cleanup
605
-
606
- Normal EOF and `shutdown` release Connection-owned Leases immediately. A killed
607
- bridge is recovered by Lease expiry; stale Connection records and inactive
608
- Workspaces are also reclaimed by bounded daemon sweeps. Releasing a Workspace
609
- will eventually close only its Broker-managed targets. It must never close a
610
- user tab merely because a client disconnected or a Workspace expired.
611
-
612
- The daemon also starts a private managed-target janitor that solely owns the
613
- browser-level CDP WebSocket and proxies daemon CDP traffic over private IPC. It
614
- creates and tracks only Broker-managed targets and their managed popup
615
- descendants. Daemon EOF, including an ungraceful process exit, makes the janitor
616
- perform bounded cleanup without consulting disk or grouping tabs by browser
617
- window. This internal process is not a public SDK, tool, or transport; embedded
618
- clients still launch only `browser-pilot bridge --stdio`.
619
-
620
- The stdio `shutdown` method exits only that bridge and disconnects its Broker
621
- Connection. The human `bp disconnect` command separately releases the one-shot
622
- compatibility Workspace and requests daemon shutdown. The daemon accepts that
623
- request only when the Broker process identity and executable installation
624
- identity still match, `embeddedConnections` is zero, and no other active Lease
625
- remains. Otherwise it returns
626
- `protocol_incompatible` or retryable `broker_in_use`; it never evicts an Agent
627
- product to complete an upgrade.
628
-
629
- Owner-only `broker-locator.json` contains current executable and protocol
630
- metadata. Owner-only `broker-versions.json` retains at most current and previous
631
- executable installation metadata. Neither file contains browser selection,
632
- tabs, target IDs, Workspaces, Leases, refs, Artifacts, credentials, network
633
- rules, or command results. Rollback selection and preservation of old binaries
634
- belong to the installer or embedding product, not to the Broker.
635
-
636
- The current lifecycle runtime uses finite limits for live Connections,
637
- per-Principal Workspaces, per-Connection Leases, and retained terminal records.
638
- It does not persist target mappings, refs, cookies, credentials, network bodies,
639
- or command state.
640
-
641
- ## Reference Consumer Adapters
642
-
643
- The source repository includes
644
- [`examples/adapters`](https://github.com/relixiaobo/browser-pilot/tree/main/examples/adapters)
645
- implementations for a
646
- Pi-style Tenon tool runtime and an OpenClaw dispatcher tool. They are host-side
647
- examples, not npm package files, a Native SDK, or an additional supported
648
- protocol surface. Production consumers should copy or port the small adapter
649
- pattern into their own integration and continue treating this document plus the
650
- runtime manifest as authoritative.
651
-
652
- The shared example launches an absolute executable path without a shell,
653
- negotiates protocol 1.1 through 1.3, discovers tools at runtime, and maps a host work scope
654
- to a Workspace plus its active invocation to a renewable Lease. It never stores
655
- a selected tab, frame, Observation, ref, credential, rule, Artifact, or event
656
- cursor on disk. Host tool-call IDs become Command and idempotency identities;
657
- transport loss after dispatch of a mutating operation is returned as
658
- non-retryable `unknown_outcome`.
659
-
660
- For event recovery, call `pollEvents`, completely process the returned ordered
661
- events, and only then call `acknowledgeEvents(nextCursor)`. A second poll before
662
- acknowledgement intentionally starts from the same last processed cursor. On
663
- `cursor_expired`, first rebuild tab and Observation state, then call
664
- `resetEventCursor` to use `workspaces/get` as the new live baseline. Best-effort
665
- notifications never advance the cursor.
666
-
667
- Image Artifacts are read from their protected path, converted to the host's
668
- native image content, and released. PDF and other file results require an
669
- absolute host-owned output directory; they are exported there before the
670
- protected Artifact is released. Cleanup invalidates local host contexts even if
671
- the explicit Broker release fails, so Lease expiry remains a bounded fallback
672
- and stale tools cannot continue issuing calls.
673
-
674
- ## Current Release Gate
675
-
676
- `tools/list` is generated from the canonical schemas used for argument and
677
- result validation, and production filtering prevents unwired tools from being
678
- advertised. The current bridge supports discovery, connect, Profile routing, open, all-tab
679
- inventory, observe/read/search/find, scroll and dropdown actions, annotated
680
- screenshots, core actions, scoped frames, explicit dialogs, cookies, auth,
681
- network observation/rules, eval, PDF, protected upload import, scoped download
682
- Artifacts, Artifact access, command recovery, browser reconnect, and event
683
- replay. Broker startup serialization, executable/protocol negotiation,
684
- live-client shutdown protection, bounded version history, and deliberate
685
- version isolation are covered by process tests. Packed global npm, local
686
- npm/npx, and product-bundled absolute-path launches are also exercised as
687
- black-box installs. Native artifact verification covers manifest hashes,
688
- version output, stdio startup, the private Broker role, protected disconnect,
689
- and the private janitor role without attaching to the user's browser.
690
-
691
- Before shipping an adapter, run the black-box
692
- [stdio conformance suite](stdio-conformance.md) against the exact bundled launch
693
- command. The suite exercises only the public NDJSON protocol and creates no
694
- user-tab control state.