browser-pilot-cli 0.1.6 → 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,614 @@
1
+ # Stdio Bridge Integration Contract
2
+
3
+ Status: browser `tools/call`, scoped Observations, target inventory, Artifacts,
4
+ command recovery, event delivery, browser reconnect, and scoped downloads
5
+ implemented.
6
+
7
+ This document describes the Agent-neutral process boundary. Tenon, OpenClaw,
8
+ and other Agent hosts use the same executable and protocol. No consumer imports
9
+ Browser Pilot source files, links a Native SDK, installs an extension, or needs
10
+ an MCP transport.
11
+
12
+ ## Process Launch
13
+
14
+ Resolve an exact Browser Pilot executable version from one of these locations:
15
+
16
+ 1. The product's bundled dependency.
17
+ 2. A project-local npm dependency.
18
+ 3. A compatible global installation selected explicitly by the host.
19
+
20
+ Launch it directly, without a shell:
21
+
22
+ ```text
23
+ browser-pilot bridge --stdio
24
+ ```
25
+
26
+ The three supported layouts are behaviorally equivalent:
27
+
28
+ - global npm: the host explicitly resolves the installed `browser-pilot` bin;
29
+ - local npm/npx: the host pins `browser-pilot-cli` and resolves its local bin or
30
+ absolute `dist/cli.js` path;
31
+ - native bundle: the host pins and launches the release archive's absolute
32
+ `browser-pilot` executable path.
33
+
34
+ The npm layout requires Node 18 or newer and starts the adjacent packaged
35
+ `daemon.js` and `managed-target-janitor.js` files as private processes. A native
36
+ release is a Node SEA executable with its runtime and dependencies embedded. It
37
+ respawns that same executable for private Broker and janitor roles; reserved
38
+ `--browser-pilot-internal=*` arguments are not a Host API. It never downloads a
39
+ runtime or falls back to a system Node installation.
40
+
41
+ Native archives contain `manifest.json`, `SHA256SUMS`, dependency/runtime
42
+ licenses, and an adjacent archive checksum. Verify the archive checksum before
43
+ unpacking and the manifest-listed file hashes before launch. The manifest's
44
+ `signature.kind` is evidence, not an aspirational channel label: `developer_id`
45
+ or `adhoc` on macOS, `authenticode` or `unsigned` on Windows, and `unsigned` on
46
+ Linux. Hosts that require a trusted publisher must reject ad-hoc or unsigned
47
+ artifacts themselves.
48
+
49
+ The host may add `--browser <id|product|channel>` to make its initial browser
50
+ preference explicit. Without it, Browser Pilot deterministically selects the
51
+ first candidate with a recorded debugging endpoint, then stable platform order.
52
+ The running Broker retains that
53
+ choice in memory; it does not write target or profile selection state to disk.
54
+
55
+ Create pipes for stdin, stdout, and stderr. Stdout is protocol-only. Treat each
56
+ stderr line as a diagnostic and never parse it as protocol data.
57
+
58
+ The executable owns Broker discovery and startup. Hosts must not read PID
59
+ files, locator metadata, Unix socket paths, or Windows named-pipe names. Two
60
+ products that launch concurrently serialize through the per-user startup lock,
61
+ then both reuse the winning compatible Broker. A dead locator is recovered on
62
+ the next launch. Broker startup and browser discovery are passive. Only an
63
+ explicit `browser.connect` tool call requests Chrome authorization, and
64
+ concurrent calls for the same browser share one in-flight WebSocket attempt. A
65
+ ready live process with an
66
+ unresponsive endpoint is not killed or replaced; launch returns
67
+ `browser_disconnected` with `restart_unresponsive_broker` remediation.
68
+
69
+ The default is one shared Broker per OS user. Protocol-compatible clients reuse
70
+ it even when different Agent products or product versions launched their
71
+ bridges. If a product must run an incompatible Browser Pilot version, set an
72
+ absolute, product-owned `BROWSER_PILOT_HOME` before process launch. This creates
73
+ a deliberately isolated locator, endpoint, Artifact store, and version history
74
+ on macOS, Linux, and Windows. Do not silently choose an isolated home after
75
+ `protocol_incompatible`: shared-browser coordination and cleanup semantics must
76
+ remain an explicit host decision.
77
+
78
+ ## Framing
79
+
80
+ The transport is JSON-RPC 2.0 over newline-delimited UTF-8 JSON:
81
+
82
+ - one complete JSON object per line;
83
+ - ordinary requests are dispatched in input order;
84
+ - request IDs are non-empty strings or safe integers;
85
+ - notifications have no response;
86
+ - the default input limit is 1 MiB per line and the default result limit is
87
+ 4 MiB; protocol 1.1 clients may negotiate lower per-Connection limits;
88
+ - invalid JSON, invalid UTF-8, invalid envelopes, and oversized input terminate
89
+ that bridge process after an error response with `id: null` when possible;
90
+ - output observes stream backpressure;
91
+ - at most 256 ordinary calls and 16 out-of-band control calls are pending per
92
+ bridge process; saturation returns retryable `result_too_large` without
93
+ dispatching the rejected call;
94
+ - EOF releases every Lease owned by that bridge Connection;
95
+ - `shutdown` exits only the bridge process, not the shared per-user daemon.
96
+
97
+ Hosts must not send concurrent writes that interleave bytes. They may pipeline
98
+ complete lines and must correlate responses by JSON-RPC ID. `commands/get`,
99
+ `commands/cancel`, and dialog list/respond calls may overtake a pending
100
+ `tools/call`, so those responses can arrive earlier. Dialog control needs this
101
+ exception because a browser dialog can pause the command that caused it.
102
+
103
+ ## Initialization
104
+
105
+ `initialize` must be the first successful request on a bridge Connection:
106
+
107
+ ```json
108
+ {"jsonrpc":"2.0","id":1,"method":"initialize","params":{"client":{"id":"com.example.agent","name":"Example Agent","version":"2.0.0","instanceId":"install:01J..."},"protocol":{"min":{"major":1,"minor":1},"max":{"major":1,"minor":1}},"requestedCapabilities":["browser.control","workspace.manage","observation.read","action.input","artifact.read","event.read"],"launchMode":"embedded","limits":{"maxMessageBytes":1048576,"maxResultBytes":4194304}}}
109
+ ```
110
+
111
+ The response returns the selected protocol, supported and granted
112
+ capabilities, executable and service versions, a process-stable Broker
113
+ identity, a Connection ID, browser candidates, and negotiated limits. Each
114
+ candidate has a stable `id`, product/channel/profile identity, aggregate
115
+ `state`, separate `processState`, `remoteDebuggingState`, and
116
+ `authorizationState`, plus structured `remediation` when action is needed.
117
+ Initialization succeeds when no candidate is ready, so a host can present setup
118
+ state and poll `browser.discover`. Discovery is passive and reports
119
+ authorization as `unknown` until an explicit connection succeeds or fails.
120
+ Workspace and Lease creation also remain passive; call `browser.connect` once
121
+ before target tools. Branch on structured `error.data.code`; never branch on
122
+ English error messages.
123
+
124
+ Protocol 1.0 uses the service limits returned by `initialize`. Protocol 1.1 may
125
+ send `limits.maxMessageBytes` and `limits.maxResultBytes`; each must be from
126
+ 64 KiB through 1 GiB. The selected value is the smaller client/service maximum
127
+ and applies only to that bridge Connection. `maxArtifactBytes` and
128
+ `eventJournalSize` remain service resource limits, not client preferences.
129
+
130
+ The initialize request and response use the service's fixed bootstrap limits.
131
+ The bridge switches limits only after the successful response has been written,
132
+ and waits for that switch before parsing a pipelined next line. Oversized
133
+ responses become `result_too_large`; oversized best-effort notifications are
134
+ dropped and remain recoverable through `events/poll`.
135
+
136
+ An already-running incompatible daemon is never replaced. Initialization
137
+ returns `protocol_incompatible` with remediation to use a compatible executable
138
+ or a deliberate `BROWSER_PILOT_HOME` isolation boundary. Executable versions
139
+ and protocol versions are independent; successful protocol negotiation is what
140
+ allows embedded clients from different products or releases to share a Broker.
141
+
142
+ ## Lifecycle Methods
143
+
144
+ The implemented lifecycle surface is:
145
+
146
+ ```text
147
+ initialize
148
+ tools/list
149
+ tools/call
150
+ commands/get
151
+ commands/cancel
152
+ workspaces/create
153
+ workspaces/get
154
+ workspaces/release
155
+ leases/create
156
+ leases/heartbeat
157
+ leases/release
158
+ events/poll
159
+ artifacts/get
160
+ artifacts/export
161
+ artifacts/import
162
+ artifacts/retain
163
+ artifacts/release
164
+ shutdown
165
+ ```
166
+
167
+ `workspaces/create` accepts an optional `browserId`. Protocol 1.1 clients may
168
+ also provide `clientKey` to make active Workspace creation idempotent within
169
+ their Principal; reusing a key for another browser fails. Without a browser ID,
170
+ the Broker prefers its first ready connection and otherwise uses its selected
171
+ stable-order browser binding. It returns a Workspace, its default logical
172
+ ManagedTabSet, and an `eventCursor`. `workspaces/get` returns the current cursor
173
+ as a recovery baseline. Creating a Workspace does not itself create a browser
174
+ connection or window. Call the Workspace-scoped `browser.connect` tool to
175
+ request Chrome authorization; the first managed navigation then creates the
176
+ dedicated browser window.
177
+
178
+ `leases/create` accepts a `workspaceId` and optional `ttlMs`. Protocol 1.1
179
+ clients may also provide a `clientKey`; repeating that key on the same live
180
+ Connection and Workspace returns the same Lease and renews its TTL. The default
181
+ Lease is 30 seconds, with a supported range of 1 second through 5 minutes. Heartbeat
182
+ well before `expiresAt`. A Lease belongs to its live Connection and cannot be
183
+ continued by a replacement bridge process. A replacement Connection from the
184
+ same ClientPrincipal may obtain a new Lease for a still-active Workspace.
185
+
186
+ Workspace identity belongs to `client.id + client.instanceId` for the lifetime
187
+ of the daemon. It is transient and is never restored from disk after Broker
188
+ restart. A Workspace released twice returns the same successful release result
189
+ while its bounded tombstone remains available.
190
+
191
+ ## Tool Calls
192
+
193
+ `tools/list` returns only operations implemented by the running Broker and
194
+ allowed by negotiated capabilities. Call those tools with one uniform envelope:
195
+
196
+ ```json
197
+ {"jsonrpc":"2.0","id":8,"method":"tools/call","params":{"name":"browser.observe","arguments":{"limit":50},"workspaceId":"workspace:...","leaseId":"lease:...","targetId":"target:...","commandId":"command:...","idempotencyKey":"observe:01J...","deadlineMs":30000}}
198
+ ```
199
+
200
+ Connection tools omit Workspace fields. Workspace tools require
201
+ `workspaceId + leaseId`; target tools additionally require the opaque
202
+ `targetId` returned by `browser.tabs.list` or `browser.open`. Raw CDP target and
203
+ session IDs are never public inputs. The Broker validates the negotiated
204
+ capability, Lease ownership, tool schema, and target control before dispatch.
205
+
206
+ Each tool's `sensitivity.input` and `sensitivity.output` arrays list the classes
207
+ that may occur anywhere in that direction. Specific content-bearing schema
208
+ nodes carry `x-browser-pilot-sensitivity`, for example:
209
+
210
+ ```json
211
+ {"type":"string","maxLength":1000000,"x-browser-pilot-sensitivity":["browser_data"]}
212
+ ```
213
+
214
+ Use the field annotation when mapping `browser.read.result.text`, Observation
215
+ element values, auth fields, cookie values, network headers/bodies, upload
216
+ Artifact references, prompt text, or eval values into the Agent runtime. Keep
217
+ the annotation attached to derived model content; do not infer sensitivity from
218
+ field names or English tool descriptions. The annotation does not wrap or alter
219
+ the value. Artifact descriptors and BrowserEvents provide their actual runtime
220
+ `sensitivity` and should be propagated directly.
221
+
222
+ Every `tools/call` response is a Command outcome:
223
+
224
+ ```json
225
+ {"command":{"id":"command:...","status":"completed","method":"browser.observe","browserConnectionGeneration":3,"idempotencyKey":"observe:01J...","acceptedAt":1,"deadlineAt":30001,"dispatchedAt":2,"completedAt":20,"mutating":false},"result":{"workspaceId":"workspace:...","leaseId":"lease:...","targetId":"target:...","url":"https://example.com","observationId":"observation:...","title":"Example","elements":[],"truncated":false,"truncationReasons":[]}}
226
+ ```
227
+
228
+ The manifest's output schema describes the inner `result`. Clients should pass
229
+ a unique `commandId` when they may need cancellation; otherwise the Broker
230
+ generates one. `idempotencyKey` is optional, but retries must reuse it or reuse
231
+ the same caller-supplied `commandId`. Reusing a key for different arguments is
232
+ rejected. A duplicate completed call returns the recorded result; a duplicate
233
+ that is still accepted or dispatched returns its current Command without
234
+ dispatching again.
235
+
236
+ `commands/get` and `commands/cancel` accept `commandId` plus `workspaceId` for
237
+ Workspace commands. They are authorized by Principal ownership rather than the
238
+ original Lease, so a replacement bridge from the same product instance can
239
+ inspect a known outcome. Cancellation while accepted prevents browser dispatch.
240
+ Cancellation after dispatch is best-effort and sets `cancellationRequested`;
241
+ it never claims rollback. A mutating command whose deadline elapses after
242
+ browser dispatch becomes `unknown_outcome` and is never automatically replayed.
243
+ Known tool failures are stored as a completed Command with a nested JSON-RPC
244
+ `error`; the original call also returns that error normally.
245
+
246
+ `browser.click` returns a new Observation plus bounded evidence. For example,
247
+ a checkbox may return:
248
+
249
+ ```json
250
+ {"action":"click","status":"verified","kind":"checkbox","effects":["checked_changed","focus_changed"],"checked":true,"focused":true}
251
+ ```
252
+
253
+ Effects can also include `selected_changed`, `pressed_changed`,
254
+ `expanded_changed`, `navigation`, `document_changed`, `dialog_opened`, and
255
+ `popup_opened`. A semantic control that does not reach its expected state uses
256
+ `mismatch`; coordinate clicks and ref clicks with no conclusive observable
257
+ effect use `unavailable`. Treat evidence as browser-level execution evidence,
258
+ then inspect the returned Observation to decide whether the task itself
259
+ succeeded.
260
+
261
+ Every Observation produced by the built-in Browser tools also includes a
262
+ bounded `hints` array. Each item is discriminated by `code` and includes
263
+ `source`, `confidence`, and a stable `recommendedAction`:
264
+
265
+ ```json
266
+ {"hints":[{"code":"modal_overlay","source":"observation","confidence":"strong","recommendedAction":"resolve_overlay_first","blocking":true,"refs":[2,5]}]}
267
+ ```
268
+
269
+ Current codes are `autocomplete`, `modal_overlay`, `filter_controls`,
270
+ `authentication_surface`, `access_blocked`, `download`, and
271
+ `repeated_action`. Observation hints may contain only refs from the returned
272
+ Observation. Authentication state is `present` on the first observed auth
273
+ surface, `entered` when a session moves onto one, and `left` when it moves off
274
+ one. A session or selected-frame change resets that transition baseline.
275
+
276
+ Hints are advisory browser evidence. Use the recommended action as the next
277
+ inspection strategy, not as permission, proof of task success, or a reason to
278
+ skip normal error handling. Unknown hint codes and unknown response fields must
279
+ be ignored. The field is additive for protocol 1.0 and 1.1 and introduces no
280
+ new capability; its data is covered by the tool or event's existing
281
+ sensitivity and granted capability.
282
+
283
+ ### Observation representations
284
+
285
+ The browser tools expose complementary bounded representations rather than one
286
+ unbounded page dump:
287
+
288
+ - `browser.observe` creates semantic elements and scoped refs for actions.
289
+ - `browser.read` returns broad readable text from the page or one CSS region.
290
+ - `browser.search` returns visible text matches, nearby context, and viewport
291
+ geometry, including open Shadow DOM, with at most 200 results.
292
+ - `browser.elements.find` returns bounded safe metadata for a CSS query,
293
+ optionally across open Shadow DOM. It never exposes DOM/CDP handles and its
294
+ results are not action refs.
295
+ - `browser.capture` may draw Observation ref boxes for spatial reasoning without
296
+ injecting an overlay into the page.
297
+
298
+ Observations may include an additive `page` object containing viewport and
299
+ document dimensions, scroll coordinates, pixels remaining in each direction,
300
+ and horizontal/vertical scroll percentages. Hosts must tolerate its absence
301
+ when talking to an older compatible Broker.
302
+
303
+ `browser.scroll` moves the page, an Observation ref, a CSS-selected container,
304
+ or a visible text match, then returns a fresh Observation and typed evidence.
305
+ `browser.dropdown.options` enumerates native or currently exposed ARIA options.
306
+ `browser.dropdown.select` verifies native value changes or uses fresh
307
+ Observation refs for custom controls. These tools reuse `observation.read` and
308
+ `action.input`; they are additive manifest entries, not a protocol-version
309
+ fork. Discover them through `tools/list`.
310
+
311
+ `browser.type` and `browser.keyboard` also return bounded evidence after the
312
+ page has had time to apply a controlled-field update:
313
+
314
+ ```json
315
+ {"action":"type","status":"verified","kind":"input","sensitive":false,"beforeLength":3,"expectedLength":7,"afterLength":7}
316
+ ```
317
+
318
+ Use `verification: "require_exact"` when a mismatch must fail the command.
319
+ Native text controls and contenteditable receive Chrome editing events;
320
+ date/time/color/range controls use their native value setter and emit
321
+ `beforeinput` plus `input`, but not an early `change`. Framework rollback,
322
+ browser value sanitization, canceled `beforeinput`, and editor interception are
323
+ reported from the effective value after the action. Password evidence never
324
+ contains the value itself.
325
+
326
+ `browser.type` and `browser.keyboard` are guarded composite actions. Browser
327
+ Pilot pins the target/session/frame/loader/Document identity and, once input
328
+ focus is established, the deep active element. It checks them between
329
+ select-all, deletion, individual keyboard characters, and submit. If a change
330
+ is detected before the first browser mutation, the call fails with retryable
331
+ `action_not_verified`. If one or more steps already ran, it fails with
332
+ `unknown_outcome`, for example:
333
+
334
+ ```json
335
+ {"code":"unknown_outcome","retryable":true,"context":{"action":"keyboard","step":"type_character:4","reason":"focus_changed","dispatchedSteps":4,"remainingStepsStopped":true}}
336
+ ```
337
+
338
+ Reasons are `target_changed`, `session_changed`, `frame_changed`,
339
+ `loader_changed`, `document_changed`, or `focus_changed`. On either failure,
340
+ list or switch to the intended target if necessary, create a fresh Observation,
341
+ and decide from current page state whether to continue. Never replay the whole
342
+ mutation automatically after `unknown_outcome`.
343
+
344
+ `browser.press` returns `action: "press"` with bounded effects such as
345
+ `value_changed`, `checked_changed`, `selected_changed`, `focus_changed`,
346
+ `navigation`, `document_changed`, `dialog_opened`, or `popup_opened`. It returns
347
+ `unavailable` with `no_observable_effect` when the key dispatch completed but no
348
+ supported effect was visible.
349
+
350
+ `browser.upload` reads the file input after assignment. It returns `verified`
351
+ only when one file with the expected browser filename remains selected,
352
+ `mismatch` when the page cleared or replaced the selection, and `unavailable`
353
+ when the target could no longer be read. Evidence contains counts and a filename
354
+ match boolean, never the Broker storage path.
355
+
356
+ Inventory includes every eligible ordinary user tab plus the Workspace's
357
+ managed tabs. A physical tab can be controlled by only one Lease at a time.
358
+ Commands for the same physical tab share one actor even though each Workspace
359
+ has a different opaque target ID. A controlling Lease can call
360
+ `browser.tabs.release` for one target without closing it. After that command
361
+ completes and emits `target_control.released`, another Lease may use its own
362
+ opaque ID to acquire the tab. This two-step handoff never accepts a destination
363
+ Lease ID and never silently steals control. Releasing a Workspace closes managed
364
+ tabs but leaves user tabs open.
365
+
366
+ ## Events
367
+
368
+ Every event is first committed to a bounded per-Workspace journal and then
369
+ offered as this best-effort notification:
370
+
371
+ ```json
372
+ {"jsonrpc":"2.0","method":"events/event","params":{"event":{"id":"event:...","sequence":7,"timestamp":1,"workspaceId":"workspace:...","browserConnectionGeneration":3,"leaseId":"lease:...","targetId":"target:...","type":"dialog","payloadVersion":1,"sensitivity":"browser_data","payload":{"dialogId":"dialog:...","state":"opened","type":"confirm","message":"Continue?","url":"https://example.com"}}}}
373
+ ```
374
+
375
+ Notifications can interleave with responses and can be dropped under transport
376
+ backpressure or a disconnected bridge. Treat them as a low-latency signal, not
377
+ as the recovery record. Track the last fully processed cursor and poll:
378
+
379
+ ```json
380
+ {"jsonrpc":"2.0","id":20,"method":"events/poll","params":{"workspaceId":"workspace:...","cursor":"cursor:6","limit":100}}
381
+ ```
382
+
383
+ The result contains ordered `events`, `nextCursor`, and `hasMore`. Continue until
384
+ `hasMore` is false. If the cursor predates the retained journal, the Broker
385
+ returns `cursor_expired` with `earliestCursor` and `latestCursor`; rebuild tab
386
+ and observation state, then call `workspaces/get` to establish a new baseline.
387
+ Never infer delivery from notification arrival alone, and never advance the
388
+ stored cursor past an event the host has not processed.
389
+
390
+ Current producers cover command status, Lease expiry, navigation, document
391
+ change, target attach/detach, target control, managed popups, dialogs, downloads,
392
+ browser connection loss/restoration, network request/response metadata, and
393
+ observation invalidation. They also include `watchdog.navigation_stalled`,
394
+ `watchdog.frame_detached`, `watchdog.dialog_unhandled`, and
395
+ `watchdog.no_progress`. Network and watchdog events never contain credentials,
396
+ typed values, refs, headers, request/response bodies, or raw CDP IDs; retrieve
397
+ sensitive detail explicitly through the scoped tools.
398
+
399
+ `network.response` adds an `access_blocked` hint only for a main-document 403 or
400
+ 429, not for failed images, scripts, XHR, or fetches. The threshold
401
+ `watchdog.no_progress` event and the action result that reaches the threshold
402
+ both contain the same `repeated_action` hint. The reason is either repeated
403
+ observable failure or `stagnant_page`, where the same action signature was
404
+ repeated while the bounded page fingerprint stayed unchanged. Fingerprints and
405
+ action parameters never leave the Broker. Change representation and strategy
406
+ instead of automatically repeating the same action or navigation.
407
+
408
+ `connection.lost` keeps the last connection generation and causes later browser
409
+ tools to fail with retryable `browser_disconnected`. The daemon passively
410
+ refreshes the originally selected profile's endpoint metadata and does not
411
+ switch profiles or open a WebSocket. The host calls `browser.connect` when it is
412
+ ready to request reconnection. `connection.restored` then carries a strictly
413
+ newer generation. Existing Workspaces
414
+ and active Leases remain valid, but old target IDs, frame IDs, CDP sessions,
415
+ Observations, and refs do not. Poll through the restoration event, call
416
+ `browser.tabs.list`, and rebuild target state. Never retry a mutating command whose
417
+ recorded status is `unknown_outcome`.
418
+
419
+ Every event has a top-level `browserConnectionGeneration`. A delayed ordinary
420
+ event from an older connection is discarded before journaling. Historical
421
+ `command.status` events and reconnect cleanup events remain in the journal with
422
+ their original generation, so consumers can finish old Command state machines
423
+ without treating cleanup as current browser state. Commands accepted against an
424
+ old generation are fenced before dispatch; commands already dispatched when the
425
+ connection changes follow the `unknown_outcome` rule above.
426
+
427
+ JavaScript dialogs remain pending. Use `browser.dialogs.list`, then call
428
+ `browser.dialogs.respond` with the returned `dialogId`, target, and explicit
429
+ `accept` or `dismiss` action. Browser Pilot never auto-accepts a dialog.
430
+
431
+ Watchdog events are advisory browser-state signals, not autonomous recovery:
432
+
433
+ - `watchdog.navigation_stalled` means `Page.navigate` was dispatched but the
434
+ page did not become interactive within 30 seconds. The command is
435
+ `unknown_outcome`; observe the target before deciding whether to navigate
436
+ again.
437
+ - `watchdog.frame_detached` carries the opaque selected `frameId`. Browser Pilot
438
+ clears that selection and invalidates the Observation; list frames and create
439
+ a fresh Observation before continuing.
440
+ - `watchdog.dialog_unhandled` is emitted once when a dialog remains pending for
441
+ 15 seconds. It does not accept or dismiss the dialog.
442
+ - `watchdog.no_progress` is emitted once after three consecutive completed
443
+ actions on the same Lease and target either have browser-observable mismatch
444
+ or repeat the same action while the bounded page state remains stagnant.
445
+ Observable success resets the mismatch detector; a changed action or page
446
+ state resets the repetition detector. Navigation, frame/session changes, and
447
+ cleanup reset both. Coordinate/canvas actions whose effects cannot be observed
448
+ do not count as mismatch-only failures.
449
+
450
+ Browser Pilot never automatically retries, repeats, stops, accepts, or dismisses
451
+ an action because of a watchdog event. The host or Agent must inspect current
452
+ state and decide. Timers and streaks are in-memory transient state and are
453
+ removed on dialog close, session loss, Lease release, Workspace release, or
454
+ browser disconnect as applicable.
455
+
456
+ ## Artifacts
457
+
458
+ `browser.capture` and `browser.pdf` return Artifact descriptors, never base64 or
459
+ an internal path. Use `artifacts/get` with the owning active Workspace and Lease
460
+ to obtain a protected local path, or `artifacts/export` with an absolute path to
461
+ copy the file to client-owned storage. Export does not overwrite by default.
462
+
463
+ Artifacts expire after 15 minutes by default. `artifacts/retain` extends that
464
+ to the retained TTL; `artifacts/release` removes the bytes immediately.
465
+ Workspace release and Broker shutdown also remove owned temporary bytes.
466
+ Directories use mode `0700` and files use `0600` where POSIX permissions apply.
467
+
468
+ Large screenshots default to a model-sized preview. Pass
469
+ `includeOriginal: true` to receive both the original descriptor and a preview
470
+ descriptor whose `previewOf` points to the original. The adapter reads the
471
+ selected file and converts it to its Agent runtime's native image/file content.
472
+
473
+ For spatial reasoning, a viewport-only `browser.capture` call may provide:
474
+
475
+ ```json
476
+ {"annotations":{"observationId":"observation:...","refs":[1,3,8]}}
477
+ ```
478
+
479
+ Omitting `refs` requests up to the first 200 refs in that Observation. The
480
+ Broker revalidates each ref, clips live geometry to the viewport, draws on an
481
+ unattached canvas in a Chrome isolated world, and returns `annotationCount`.
482
+ Screenshot bytes are not evaluated in the page's default world. Annotation
483
+ cannot be combined with `fullPage` or `selector`. A page can still move itself
484
+ between geometry sampling and capture, so hosts should use a recent Observation
485
+ and treat the image as visual context rather than durable element identity.
486
+
487
+ For upload, first call `artifacts/import` with the owning Workspace, active
488
+ Lease, and an absolute client-authorized path:
489
+
490
+ ```json
491
+ {"jsonrpc":"2.0","id":30,"method":"artifacts/import","params":{"workspaceId":"workspace:...","leaseId":"lease:...","path":"/absolute/path/resume.pdf","mimeType":"application/pdf"}}
492
+ ```
493
+
494
+ The Broker copies the source into protected storage and returns an
495
+ `upload_input` Artifact descriptor. Pass only its `artifactId` to
496
+ `browser.upload`; optionally identify a file input with
497
+ `observationId + ref` or `inputIndex`. Output Artifacts such as screenshots and
498
+ downloads are rejected as upload inputs. Workspace cleanup deletes the imported
499
+ copy but never deletes the client-owned source file.
500
+
501
+ Downloads have no path-returning tool. Once a controlled target session is
502
+ attached, the Broker configures target-session download capture and emits
503
+ `download` events with one of these states:
504
+
505
+ - `capture_unavailable`: that Chrome version rejected target-session isolation;
506
+ - `started`: includes an opaque `downloadId`, bounded URL, and suggested filename;
507
+ - `completed`: includes the Workspace-owned `download` Artifact descriptor;
508
+ - `failed` or `cancelled`: includes a stable, bounded reason and byte metadata.
509
+
510
+ These four lifecycle states also carry a `download` hint. `started` recommends
511
+ waiting, `completed` recommends inspecting the Artifact and includes only its
512
+ public `artifactId`, and terminal failures recommend inspecting the bounded
513
+ failure reason. `capture_unavailable` is setup state and does not claim that a
514
+ download started.
515
+
516
+ The private CDP GUID and staging path never appear in protocol output. On
517
+ `completed`, use the descriptor's `id` with `artifacts/get` or `artifacts/export`
518
+ exactly as for screenshots and PDFs. Staging directories are separate per CDP
519
+ session, use mode `0700`, enforce concurrency and byte quotas, and are removed on
520
+ session, Lease, or Workspace release. The session setting is restored to
521
+ `default` before an attached session is retired. Browser Pilot never calls browser-wide
522
+ `Browser.setDownloadBehavior`; unsupported target-session capture remains
523
+ unavailable instead of redirecting unrelated user downloads.
524
+
525
+ ## Cleanup
526
+
527
+ Normal EOF and `shutdown` release Connection-owned Leases immediately. A killed
528
+ bridge is recovered by Lease expiry; stale Connection records and inactive
529
+ Workspaces are also reclaimed by bounded daemon sweeps. Releasing a Workspace
530
+ will eventually close only its Broker-managed targets. It must never close a
531
+ user tab merely because a client disconnected or a Workspace expired.
532
+
533
+ The daemon also starts a private managed-target janitor that solely owns the
534
+ browser-level CDP WebSocket and proxies daemon CDP traffic over private IPC. It
535
+ creates and tracks only Broker-managed targets and their managed popup
536
+ descendants. Daemon EOF, including an ungraceful process exit, makes the janitor
537
+ perform bounded cleanup without consulting disk or grouping tabs by browser
538
+ window. This internal process is not a public SDK, tool, or transport; embedded
539
+ clients still launch only `browser-pilot bridge --stdio`.
540
+
541
+ The stdio `shutdown` method exits only that bridge and disconnects its Broker
542
+ Connection. The human `bp disconnect` command separately releases the one-shot
543
+ compatibility Workspace and requests daemon shutdown. The daemon accepts that
544
+ request only when the Broker process identity and executable installation
545
+ identity still match and `embeddedConnections` is zero. Otherwise it returns
546
+ `protocol_incompatible` or retryable `broker_in_use`; it never evicts an Agent
547
+ product to complete an upgrade.
548
+
549
+ Owner-only `broker-locator.json` contains current executable and protocol
550
+ metadata. Owner-only `broker-versions.json` retains at most current and previous
551
+ executable installation metadata. Neither file contains browser selection,
552
+ tabs, target IDs, Workspaces, Leases, refs, Artifacts, credentials, network
553
+ rules, or command results. Rollback selection and preservation of old binaries
554
+ belong to the installer or embedding product, not to the Broker.
555
+
556
+ The current lifecycle runtime uses finite limits for live Connections,
557
+ per-Principal Workspaces, per-Connection Leases, and retained terminal records.
558
+ It does not persist target mappings, refs, cookies, credentials, network bodies,
559
+ or command state.
560
+
561
+ ## Reference Consumer Adapters
562
+
563
+ The source repository includes
564
+ [`examples/adapters`](https://github.com/relixiaobo/browser-pilot/tree/main/examples/adapters)
565
+ implementations for a
566
+ Pi-style Tenon tool runtime and an OpenClaw dispatcher tool. They are host-side
567
+ examples, not npm package files, a Native SDK, or an additional supported
568
+ protocol surface. Production consumers should copy or port the small adapter
569
+ pattern into their own integration and continue treating this document plus the
570
+ runtime manifest as authoritative.
571
+
572
+ The shared example launches an absolute executable path without a shell,
573
+ negotiates protocol 1.1, discovers tools at runtime, and maps a host work scope
574
+ to a Workspace plus its active invocation to a renewable Lease. It never stores
575
+ an active tab, frame, Observation, ref, credential, rule, Artifact, or event
576
+ cursor on disk. Host tool-call IDs become Command and idempotency identities;
577
+ transport loss after dispatch of a mutating operation is returned as
578
+ non-retryable `unknown_outcome`.
579
+
580
+ For event recovery, call `pollEvents`, completely process the returned ordered
581
+ events, and only then call `acknowledgeEvents(nextCursor)`. A second poll before
582
+ acknowledgement intentionally starts from the same last processed cursor. On
583
+ `cursor_expired`, first rebuild tab and Observation state, then call
584
+ `resetEventCursor` to use `workspaces/get` as the new live baseline. Best-effort
585
+ notifications never advance the cursor.
586
+
587
+ Image Artifacts are read from their protected path, converted to the host's
588
+ native image content, and released. PDF and other file results require an
589
+ absolute host-owned output directory; they are exported there before the
590
+ protected Artifact is released. Cleanup invalidates local host contexts even if
591
+ the explicit Broker release fails, so Lease expiry remains a bounded fallback
592
+ and stale tools cannot continue issuing calls.
593
+
594
+ ## Current Release Gate
595
+
596
+ `tools/list` is generated from the canonical schemas used for argument and
597
+ result validation, and production filtering prevents unwired tools from being
598
+ advertised. The current bridge supports discovery, connect, open, all-tab
599
+ inventory, observe/read/search/find, scroll and dropdown actions, annotated
600
+ screenshots, core actions, scoped frames, explicit dialogs, cookies, auth,
601
+ network observation/rules, eval, PDF, protected upload import, scoped download
602
+ Artifacts, Artifact access, command recovery, browser reconnect, and event
603
+ replay. Broker startup serialization, executable/protocol negotiation,
604
+ live-client shutdown protection, bounded version history, and deliberate
605
+ version isolation are covered by process tests. Packed global npm, local
606
+ npm/npx, and product-bundled absolute-path launches are also exercised as
607
+ black-box installs. Native artifact verification covers manifest hashes,
608
+ version output, stdio startup, the private Broker role, protected disconnect,
609
+ and the private janitor role without attaching to the user's browser.
610
+
611
+ Before shipping an adapter, run the black-box
612
+ [stdio conformance suite](stdio-conformance.md) against the exact bundled launch
613
+ command. The suite exercises only the public NDJSON protocol and creates no
614
+ user-tab control state.
@@ -0,0 +1,51 @@
1
+ # Stdio Conformance Suite
2
+
3
+ The stdio conformance suite verifies an embedded Browser Pilot executable as a
4
+ black box. It launches the exact command supplied by the integrator, exchanges
5
+ only newline-delimited JSON-RPC 2.0 over stdin/stdout, and imports no Browser
6
+ Pilot source or protocol modules.
7
+
8
+ Build and test the repository executable:
9
+
10
+ ```bash
11
+ npm run build
12
+ node scripts/run-stdio-conformance.mjs
13
+ ```
14
+
15
+ An installed package also exposes the same runner as
16
+ `browser-pilot-conformance`.
17
+
18
+ Test an executable or product-bundled adapter with its exact arguments. The
19
+ command after `--` is passed directly to the operating system without a shell:
20
+
21
+ ```bash
22
+ node scripts/run-stdio-conformance.mjs -- \
23
+ /absolute/path/to/browser-pilot bridge --stdio
24
+ ```
25
+
26
+ Use `--report <path>` to write the same JSON report printed to stdout, and
27
+ `--timeout-ms <value>` to change the per-operation timeout. A passed suite exits
28
+ with status 0, a conformance failure exits with status 1, and invalid runner
29
+ arguments exit with status 2.
30
+
31
+ The suite creates one transient Workspace, one renewable Lease, and one managed
32
+ `about:blank` target. It does not enumerate or control user tabs. It validates:
33
+
34
+ - initialization and capability negotiation;
35
+ - tool manifest discovery;
36
+ - Workspace, Lease, and heartbeat lifecycle;
37
+ - managed target creation and inventory;
38
+ - a bounded Observation;
39
+ - screenshot creation, protected Artifact access, and release;
40
+ - event cursor polling;
41
+ - explicit target, Lease, Workspace, and bridge cleanup.
42
+
43
+ Failures trigger best-effort cleanup of only the resources created by the
44
+ suite. The versioned report contains check outcomes, counts, timings, version
45
+ metadata, and a bounded error summary. It intentionally omits opaque resource
46
+ IDs, Artifact paths, browser content, and bridge stderr text.
47
+
48
+ The repository tests exercise the runner against an independent fake stdio
49
+ executable, so normal unit tests do not connect to Chrome. Running the suite
50
+ against the real executable requires a ready supported browser and may start or
51
+ reuse the per-user Browser Pilot daemon.