pi-agent-browser-native 0.9.1 → 0.9.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (42) hide show
  1. package/CHANGELOG.md +26 -0
  2. package/README.md +35 -8
  3. package/dist/extensions/agent-browser/index.js +39 -16
  4. package/dist/extensions/agent-browser/lib/batch-lifecycle.js +6 -3
  5. package/dist/extensions/agent-browser/lib/browser-transcript.js +2 -1
  6. package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +4 -0
  7. package/dist/extensions/agent-browser/lib/native-output.js +2 -1
  8. package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +2 -8
  9. package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +12 -2
  10. package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +62 -10
  11. package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +18 -12
  12. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +33 -24
  13. package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +111 -53
  14. package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +6 -2
  15. package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +32 -20
  16. package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +37 -32
  17. package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +30 -11
  18. package/dist/extensions/agent-browser/lib/orchestration/output-file.js +3 -1
  19. package/dist/extensions/agent-browser/lib/page-target-validation.js +10 -3
  20. package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +2 -1
  21. package/dist/extensions/agent-browser/lib/playbook.js +7 -6
  22. package/dist/extensions/agent-browser/lib/process-environment.js +12 -3
  23. package/dist/extensions/agent-browser/lib/process.js +6 -6
  24. package/dist/extensions/agent-browser/lib/read-confirmation.js +56 -16
  25. package/dist/extensions/agent-browser/lib/results/confirmation.js +3 -0
  26. package/dist/extensions/agent-browser/lib/results/envelope.js +5 -1
  27. package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +10 -1
  28. package/dist/extensions/agent-browser/lib/results/presentation/common.js +1 -0
  29. package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +34 -1
  30. package/dist/extensions/agent-browser/lib/results/presentation/registry.js +5 -1
  31. package/dist/extensions/agent-browser/lib/results/presentation.js +15 -2
  32. package/dist/extensions/agent-browser/lib/results/selector-recovery.js +5 -5
  33. package/dist/extensions/agent-browser/lib/runtime.js +5 -1
  34. package/dist/extensions/agent-browser/lib/session-page-state.js +35 -16
  35. package/docs/ARCHITECTURE.md +10 -9
  36. package/docs/COMMAND_REFERENCE.md +37 -13
  37. package/docs/RELEASE.md +6 -5
  38. package/docs/REQUIREMENTS.md +3 -1
  39. package/docs/SESSION_STORAGE.md +1 -1
  40. package/docs/SUPPORT_MATRIX.md +12 -3
  41. package/docs/TOOL_CONTRACT.md +29 -20
  42. package/package.json +4 -4
@@ -28,6 +28,8 @@ When upstream ships a new `agent-browser` or the inventory changes:
28
28
 
29
29
  ## Current browser contract
30
30
 
31
+ Native text and artifact ownership: the registered artifact owner covers empty-row screenshot alignment, nonscreenshot placeholders and bail truncation; the upgrade owner covers one-copy failed explicit text. The text/presentation owners cover cookie/storage format redaction across content, details, complete spills and exports, effective raw/stdin batch intent and native JSON all-entry maps. Real-native cases exercise screenshot paths, explicit/shorthand sensitive-key storage reads, empty-name cookies and CLI true/false helper continuity over opposite env/config defaults. Debug false creates no native log with clean or opposite defaults; later calls retain native inheritance. Native unescaped multiline all-entry/raw-batch text retains the documented heuristic ceiling; structured JSON provides the unambiguous storage boundary.
32
+
31
33
  The compact direct/code/loader surface replaces public multimode inputs while retaining native commands and advanced outcomes. This section describes the implementation contract; dated evidence below does not qualify new Pi 1.0 code.
32
34
 
33
35
  | Area | Contract and verification target |
@@ -40,6 +42,7 @@ The compact direct/code/loader surface replaces public multimode inputs while re
40
42
  | RQ-0142 code | Fresh permissioned Node child over persistent parent-selected browser; explicit JSON/current-cell image output, visible failures/counts, ordinary auth/config/environment, ordered Pi intent/completion transitions, child cancellation/reaping, no legacy public execution. |
41
43
  | Hotpath boundaries | Unrelated tools/harmless Bash return before prompt policy. Latest raw user intent is seeded on start/tree, updated by `message_end` and refreshed after native content replacement. Captured admission uses a validated append-only prefix; one queued replay restores branch/lease state and folds per-session launch settings once. Per-row renderer memoization skips unchanged formatting. No broad config/trust/history cache, new wait/timer or budget increase. |
42
44
  | Shared ordering | Actual socket/namespace/session lock spans helper/action or entire cell; sorted identity sets for replacement/cleanup and namespace barrier for direct close-all. Requires every participating Pi to run the updated package; no human/third-party/older-version guarantee or transaction. |
45
+ | Lightpanda engine | Chrome remains default; separately installed Lightpanda is opt-in. `--engine` uses existing managed fresh-launch recovery; effective CLI/env/config Lightpanda omits wrapper Chrome profile/executable defaults while preserving native choices. Positively identified images are text-rendered with unknown geometry, not visual/coordinate parity. No automatic fallback. `test/agent-browser.runtime.test.ts`, `test/agent-browser.shared-defaults.test.ts`, and `test/agent-browser.screenshot-geometry.test.ts` own the regressions; [limits](COMMAND_REFERENCE.md#lightpanda-engine). |
43
46
  | Observation and geometry | Canonical bounded JSON/prose/code/native `structuredContent` observations expose exact recovery and receipts, with native namespace/output-schema metadata and canonical returned `isError`; complete observation spills preserve large output. Image dimensions/CSS viewport/DPR/crop/scroll/frame evidence is measured or explicitly unknown; no viewport-coordinate assumptions for full-page/element images. |
44
47
  | Pi integration | Pi 1.0 floor with independently qualified official/fork graphs. Five advanced direct tools use native `defaultActive: false`; a narrow additive initial-resume fallback preserves available transcript selections without overriding explicit/defaultTools/filtered host selection. Loader/tree/reload retain native history; public `pi:instruction-groups` registration supplies one full `browser` instruction source. Managed hosts own deferred delivery; unmanaged/stock hosts receive identical full eager guidance even for generic prompts. No permission widening or legacy discovery metadata. `test/agent-browser.discovery.test.ts` covers full-text parity, dynamic ownership, and startup/late registration; tool-surface tests preserve host selections, with pipeline coverage in `test/agent-browser.pi-pipeline.test.ts`. No checkpoint or fork metadata/revision APIs, provider payload edits or image-detail overrides. |
45
48
 
@@ -81,7 +84,7 @@ Current summary:
81
84
  | RQ-0142 | Persistent `agent_browser_code` uses the retained permissioned child and the same native executor as direct calls. Browser/auth state persists; JS state resets. Selected JSON/images, handled-failure receipts/counts, bounded IPC/deadline, shared whole-cell ordering, and ordered Pi transition replay replace isolated finally-close execution. The optional host `beforeExecute` callback retains outer call identity and serial dispatch. Strict old lease readers exist only for upgrade cleanup. | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#agent_browser_code), [`ARCHITECTURE.md`](ARCHITECTURE.md#persistent-browser-code), `test/agent-browser.script.test.ts`, `test/agent-browser.tool-surface.test.ts`, `test/agent-browser.execution-lock.test.ts`, `test/agent-browser.execution-order.test.ts` |
82
85
  | RQ-0143 | Browser-backed calls require a stable upstream version at or above the configured 0.35.0 floor, report minimum/observed version mismatch before launch, and keep stateless/local diagnostics available; the recommended target and minimum are shared by runtime, docs verification, and package contents. | [`scripts/agent-browser-target.mjs`](../scripts/agent-browser-target.mjs), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0360-rebaseline), `test/agent-browser.upstream-version.test.ts` |
83
86
  | RQ-0144 | Managed restore keys are scoped to both checkout generation and Pi transcript so upstream 0.33.2's newest-file-wins loading cannot let concurrent chats clobber or inherit cookies/storage. Fresh rotations, reload, restart, and `/resume` of one transcript retain continuity; after a confirmed inactive daemon, a branch-restored compatible owned session retains its pending reopen through daemon-starting non-page calls and transcript replay, then reopens the complete recorded URL (including its fragment) before current-page operations such as `get url` or `reload`, verifies the actual target and invalidates old refs. This reloads a URL, not forms, JavaScript memory, or history; live wrong-tab recovery never auto-navigates. Passive and explicit upstream configs pass through, while explicit config selection suppresses the wrapper's automatic restore injection. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.managed-session-restore.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
84
- | RQ-0145 | Mega-dogfood correctness closes false-positive success paths: URL QA makes matched post-clear page-error evidence non-pass as unverified, counts definitely new errors separately, and waits 150 ms for immediate post-load diagnostics, stale pre-existing artifacts and missing/stale recordings finalized by restart fail verification, aliased destinations (including dangling symlinks, full macOS/Windows Unicode folds, final short/long diff output flags, and same-call `outputPath` aliases) and active recording-path reuse through artifacts, path-bearing HAR stops, long/short or reordered explicit download waits, interspersed-global command forms, Electron cleanup output, or normalized output writes is rejected through an atomic process-wide, transcript-backed namespace/session reservation index independent of bounded manifest eviction; cross-branch close tombstones survive reload and each direct or ordered nested close retires exact identity at that lifecycle point while terminal closes clear attached/ref/page/route state, later lifecycle-proven browser launches—including failed post-close rows and `record stop`—can reactivate/rebuild it without stale pre-close `about:blank` recovery while bounded launch evidence survives failed-step presentation and transcript replay, explicitly non-launching diagnostics preserve the close, missing evidence stays conservatively active even on the first managed call, every successful direct/nested close clears wrapper trace/profiler ownership before later successful rows can rebuild it, and namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, only the newest pending recording path remains authoritative even across same-timestamp restart rows, current recording transitions replay directly without artifact-manifest compatibility inference, definitive direct/batch no-recording stop failures retire stale state at their ordered step, recording starts after nested close are rejected, pending rows coalesce into terminal batch outcomes without retaining an intermediate close-abandoned duplicate after a later saved stop, and any later same-session failure retains exact cleanup actions, same-page getter refs are refreshed, locator select resolves exact current native controls, no-movement scroll fails honestly, and existing owned cleanup preserves failure evidence. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), `test/agent-browser.extension-input-modes.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.presentation-artifacts-batch.test.ts`, `test/agent-browser.recording-reservations.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.script.test.ts` |
87
+ | RQ-0145 | Mega-dogfood correctness closes false-positive success paths: URL QA makes matched post-clear page-error evidence non-pass as unverified, counts definitely new errors separately, and waits 150 ms for immediate post-load diagnostics, stale pre-existing artifacts and missing/stale recordings finalized by restart fail verification, aliased destinations (including dangling symlinks, full macOS/Windows Unicode folds, final short/long diff output flags, and same-call `outputPath` aliases) and active recording-path reuse through artifacts, path-bearing HAR stops, long/short or reordered explicit download waits, interspersed-global command forms, Electron cleanup output, or normalized output writes is rejected through an atomic process-wide, transcript-backed namespace/session reservation index independent of bounded manifest eviction; cross-branch close tombstones survive reload and each direct or ordered nested close retires exact identity at that lifecycle point while terminal closes clear attached/ref/page/route state, later lifecycle-proven browser launches—including failed post-close rows and `record stop`—can reactivate/rebuild it without stale pre-close `about:blank` recovery while bounded launch evidence survives failed-step presentation and transcript replay, explicitly non-launching diagnostics preserve the close, missing evidence stays conservatively active even on the first managed call, every successful direct/nested close clears wrapper trace/profiler ownership before later successful rows can rebuild it, and namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, only the newest pending recording path remains authoritative even across same-timestamp restart rows, current recording transitions replay directly without artifact-manifest compatibility inference, definitive direct/batch no-recording stop failures retire stale state at their ordered step, recording starts after nested close are rejected, rejected or pending restarts keep the active take's reservation and stop action unless a terminal previous-take receipt or native `restarted: true` acknowledges replacement, pending rows coalesce into terminal batch outcomes without retaining an intermediate close-abandoned duplicate after a later saved stop, and any later same-session failure retains exact cleanup actions, same-page getter refs are refreshed, locator select resolves exact current native controls, no-movement scroll fails honestly, and existing owned cleanup preserves failure evidence. | [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), `test/agent-browser.extension-input-modes.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.presentation-artifacts-batch.test.ts`, `test/agent-browser.recording-reservations.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.script.test.ts` |
85
88
  | RQ-0146 | Upstream `agent-browser 0.34.0` rebaseline adds sticky `--pin-tab` / `--no-pin-tab` (`AGENT_BROWSER_PIN_TAB`) for shared CDP/auto-connect sessions, CDP target ids as tab refs, and `tab_gone` recovery data (`data.targetId`, optional `data.lastUrl`). The wrapper parses those flags as optional global booleans rather than launch-scoped, classifies `tab_gone` as `failureCategory: "tab-gone"` with `list-tabs-after-tab-gone` / `open-tab-after-tab-gone`, and shows `targetId` in `tab list` presentation. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0340-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), `test/agent-browser.results.test.ts`, `test/agent-browser.runtime.test.ts` |
86
89
  | RQ-0147 | Local Chrome composes stock `--no-startup-window` with native launch defaults while headless remains the default; URL-less open uses native lazy URL reads across direct/batch/code paths. Existing root/owned daemon probes and explicit-local session info gate bootstrap, without tab cleanup or a launcher/state journal. See `test/agent-browser.stock-startup.test.ts` for stock browser PID/page/storage checks. Native follow-ups preserve profiled and other launch-configured browser pages by omitting wrapper compatibility launch arguments while the daemon is active; daemon inspection reapplies the retained compatibility user agent only when relaunch is required. Caller `--args` retain their values after the startup default and native precedence; config, file access, and explicit local navigation remain native-owned, and cancellation wins over policy validation before spawn. | [`docs/ARCHITECTURE.md`](ARCHITECTURE.md), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.runtime.test.ts`, `test/agent-browser.process.test.ts`, `test/agent-browser.real-upstream-contract.test.ts` |
87
90
  | RQ-0148 | Upstream 0.35.0 adds Linux Chromium private CA trust and the `protected-vercel-deployments` skill. The wrapper parses CA set/clear globals, requires fresh managed launches, suppresses automatic managed restore for CA-enabled sessions, passes caller certificate paths upstream unchanged, and uses 0.35.0 as its runtime floor. | [`docs/COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#upstream-0350-rebaseline), [`docs/TOOL_CONTRACT.md`](TOOL_CONTRACT.md), `test/agent-browser.runtime.test.ts`, `test/agent-browser.managed-session-restore.test.ts`, `test/agent-browser.page-target-validation.test.ts` |
@@ -95,6 +98,8 @@ Browser journal storage (RQ-0153) uses one canonical small event/reducer, once-s
95
98
 
96
99
  QA truthfulness (#121, RQ-0145) is covered by the registered-tool controls in `test/agent-browser.extension-input-modes.test.ts` and the strict `contract suite matches QA non-pass after same-URL error-buffer rollover` case in `test/agent-browser.real-upstream-contract.test.ts`. The real fixture fills the native error FIFO with identical rows, then verifies that a new document actually throws again at the same URL and that QA is always non-pass. Matched rows fail as an unverified check, not an application-error assertion. Empty/cleared and explicit `checkErrors: false` controls preserve valid passes; the wrapper does not fix upstream `errors --clear` or accept a warning-only pass as resolution.
97
100
 
101
+ Explicit native text preserves standalone and compound/raw-or-stdin batch output without changing caller input. `test/agent-browser.text-output.test.ts` covers opaque JSON/confirmation-like page strings, complete whitespace-preserving stdout and redacted subprocess/presentation spills, text-only output export success/failure notices, nonzero failure and strict malformed JSON; the `real native explicit text` case in `test/agent-browser.real-upstream-contract.test.ts` proves native URL reads, both diff orders, row boolean operands and last-wins outer selection. The same native case protects empty stdin-row skipping, later successful rows, Unicode results, bail behavior and original row positions in text errors; malformed shapes still fail before dispatch. Failed-exit stderr is primary while zero-exit malformed JSON remains strict, covered by the registered `failed-json`/`strict-json` cases. Text contains no trusted row, confirmation, lifecycle, page-target or artifact receipts; use JSON for those fields. See [the output contract](TOOL_CONTRACT.md#wrapper-json).
102
+
98
103
  Browser evidence (RQ-0152) is covered by `test/agent-browser.browser-evidence.test.ts`, `test/agent-browser.read-confirmation.test.ts`, `test/agent-browser.recording-receipts.test.ts` and `test/agent-browser.recording-recovery.test.ts`, alongside existing presentation, reservation/replay and lifecycle checks. They cover cold/shared/unknown/invalid/timeout reads, native-default and shared confirmation routing, capability absence and script identity boundaries, separate daemon/browser/Pi identity, sparse/legacy capture metrics, native failure retention, stale/same-path mismatches, one-query stop recovery, error exports and artifact aliases. Full live identity, receipt lookup and browser-independent native read/confirm handling require companion upstream fixes; missing fields remain unknown, and native receipts do not survive daemon exit. These source checks do not replace paired native/Pi runtime verification.
99
104
 
100
105
  Contributor fixes #133/#152 remove unused prompt suffix entries without changing runtime guidance and diagnose bare `--no-sandbox` only in the command slot or navigation option positions. Native `--args` values and literal operands remain intact; batch checks use raw effective rows without treating row-local `--args` as a launch setting. `test/agent-browser.chromium-args.test.ts` covers pre-dispatch rejection, literal/flag-value controls, inspection, raw/stdin batch precedence and dispatcher outcomes. Existing prompt/grammar checks remain; source checks do not replace native-product gates.
@@ -115,6 +120,10 @@ Batch/ref fidelity (#122/#123/#125/#126): tab recovery uses verified native sele
115
120
 
116
121
  Overlay recovery (RQ-0073, #147) recommends only `inspect-overlay-state` after an upstream covered-click rejection, preserving `upstream-error` and the exact session/namespace. Direct, semantic, raw `find` (including `nth` and default-click), and failed native-batch paths share the matcher; hover and generic errors do not qualify. `test/agent-browser.overlay-click-recovery.test.ts` covers presentation and error envelopes, `test/agent-browser.pi-pipeline.test.ts` covers persisted errors and parseable JSON, and the opt-in `test/agent-browser.overlay-real-upstream.test.ts` checks actual covered clicks, executable inspection, and unchanged target/blocker click counters. This is not a fix for silent input dispatch (#62).
117
122
 
123
+ Lightpanda support preserves native command access rather than promising Chrome parity. Registered-tool image controls cover native metadata in direct/JSON, batch and selected code attachments plus standalone browser-brand evidence when legacy/attached native output omits or mislabels the engine. Verified image bytes remain unchanged and coordinate claims stay unknown even when dimensions match synthetic browser geometry. Root-default controls cover CLI/env/native-config precedence and untouched caller settings; engine planner controls preserve fresh/new/explicit launches. See [engine limits](COMMAND_REFERENCE.md#lightpanda-engine). Release evidence below remains separate from these focused regressions.
124
+
125
+ Confirmation-policy continuity uses existing canonical session page events, resolved before helpers. The focused native nextAction case proves exact denial, a useful confirmed audit followed immediately by title/body reads across resume, bare getter/PID continuity, and denied recording-restart sentinel/original-take preservation; the shared-defaults owner covers selected-string/empty/unset precedence, code/advanced routing, ordered pending-batch targets/replay with a gated URL helper, failed replacement cleanup/recovery, off-branch cleanup, close and identity isolation. Strict native guarded-control markers reuse the same transcript reducer with compact command/action provenance, not command replay; exact decisions avoid helper-slot replacement without claiming native ID validation or browser independence. Read-confirmation tests cover forged/foreign/first-nondecision controls, raw-argv-over-stdin decisions, later unknown-page checks, ordered confirmed batch effects, truthful requested/early-helper/post-command-helper pending copy, and completed click/tab/failed-inner/close reconciliation through real code intent/finish replay, cold-resume covers pending helper exposure/replay, and Electron host tests cover ambient/retained policy. Definite no-admission attempts restore prior policy while interrupted work retains intent. Pending close does not retire ownership or restore state. Native policy changes still restart normally; pending commands remain native-owned. Read-confirmation regressions also cover ordered confirmed snapshots, later pending/failed/invalidation rows, one-use freshness consumption through direct/code/batch resume, semantic role click/check/fill and unique role/label select after snapshot approval with unchanged payloads, absent/ambiguous select targets and drift, warm/no-resume direct and semantic select success on unchanged path/hash routes for caller-owned and freshly established wrapper-owned managed sessions versus refusal after a live-observed complete-URL change, including hash-only routes, without a gated-snapshot loop or changing ordinary fragment-insensitive ref/tab comparisons, plus live/resumed attachment retirement after confirmed close. Registered confirmation tests also cover direct/batch tab transport metadata, exact approval/denial, visible conditional continuation, malformed/mismatched extras and page/eval/HTTP provenance rejection; actual SDK/native proof uses the real authenticated-tab transport with a dummy loopback token. Presentation tests distinguish direct/nested pending snapshot controls from complete empty snapshots and preserve redacted recovery actions and explicit JSON. Click-dispatch owner tests and local native SDK proof cover timer cancellation, pending-safe marker/listener expiry without an extra helper, and native ref/XPath click dispatch when `evaluate` guards optional diagnostic probes. The patched recording-restart/navigation subcase has a separate test-only opt-in; stock upstream still runs the remaining confirmation checks. See [real upstream qualification](RELEASE.md#real-upstream-contract-validation).
126
+
118
127
  Shared native defaults are covered by `test/agent-browser.shared-defaults.test.ts`: environment/file/argv precedence, caller-owned reuse across independent Pi contexts, quit ownership, native helper idle settings, unchanged implicit cleanup, and an opt-in real local-profile fixture proving shared state and code follows the selected ordinary browser rather than a disposable isolated session. Code tests cover the fresh child language boundary and persistent browser behavior; the local Electron handoff regression in `test/agent-browser.extension-electron-lifecycle.test.ts` keeps generated launch ownership despite a native session environment default; native config validation uses upstream rather than a second schema. See [shared browser defaults](../README.md#shared-browser-defaults). These tests do not promise permanent third-party authentication.
119
128
 
120
129
  Browser correctness regressions cover credential-bearing filtered network details/exports, native generated-Blob downloads with actual saved bytes, accessible-name click target identity and abstention, smooth CSS-container movement, and parseable early-result JSON. Native fixture coverage complements deterministic dispatch/cleanup and output-shape checks; release qualification remains separate.
@@ -152,7 +161,7 @@ Re-run the gates below before each release. Dated outcomes in this table are his
152
161
  | Configured-source lifecycle | `npm run verify -- lifecycle` (`scripts/verify-lifecycle.mjs`) drives `/reload`, closes and relaunches Pi with the same exact `--session-id`, checks the JSONL session header id, session continuity, slash-command sentinel tokens (`v1` before reload and `v2` after full relaunch), persisted spill reachability, and native returned failure status and JSON preservation for a QA reclassification with a fake upstream on `PATH`. Default Pi model is `zai/glm-5.2`; default per-step wait is **180000 ms** (`DEFAULT_TIMEOUT_MS`); override model with `--model <id>` and waits with `--timeout-ms <ms>`. Passthrough flags in [`scripts/project.mjs`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/scripts/project.mjs): `--keep-artifacts`, `--model`, `--verbose`, and `--timeout-ms` plus a value (for example `npm run verify -- lifecycle --model openai-codex/gpt-5.5:minimal --keep-artifacts --verbose --timeout-ms 600000`). | **Pi 0.84.4 on 2026-09-01:** pass; `/reload`, exact-session relaunch, managed continuity, persisted spill reachability, and Pi failure status all verified. |
153
162
  | Quick isolated Pi smoke | `pi --approve --no-extensions --no-skills -e .` from trusted repo root; checkout-native browser tools only. | **Earlier 0.36.0 / package 0.6.2 / Pi 0.84.4:** isolated checkout tmux smoke passed on 2026-09-01; native `agent_browser` completed version, localhost open, WebMCP list/invoke, visible-text verification, and close. |
154
163
 
155
- Runtime floor note: package metadata requires Node 24.21.0 or newer, keeps optional Pi core package peer ranges wildcard per installed Pi package docs, pins direct Pi validation dependencies to 1.0.0, and makes `pi --version` below 1.0.0 a setup failure through `pi-agent-browser-doctor` / `npm run doctor`. The Pi floor follows the native tool-history/prompt APIs and includes no older-host compatibility shim.
164
+ Runtime floor note: package metadata requires Node 24.21.0 or newer, keeps optional Pi core package peer ranges wildcard per installed Pi package docs, keeps direct Pi development dependencies locked as reproducible snapshots while qualification resolves the latest stable official Pi and maintained fork main independently, and makes `pi --version` below 1.0.0 a setup failure through `pi-agent-browser-doctor` / `npm run doctor`. The Pi floor follows the native tool-history/prompt APIs and includes no older-host compatibility shim.
156
165
 
157
166
  ## Baseline checklist by inventory section
158
167
 
@@ -163,7 +172,7 @@ Runtime floor note: package metadata requires Node 24.21.0 or newer, keeps optio
163
172
  | Sessions, state, tabs, frames, dialogs, and windows | 28 canonical tokens from baseline section `state-tabs-frames-dialogs`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands), stateful workflow notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Stateful summaries redact credentials while preserving restore identifiers, every session/state list row, and ordinary `cookies set --curl` file operands in invocation echoes and failed batch diagnostics; explicit targets and paths pass through. Runtime retains state artifact verification, sessionless local command planning, automatic managed restore, native-CDP-ID tab recovery across duplicate URLs and transcript replay, reached failed-batch target invalidation/live verification, active-target refresh after tab close, and close alias cleanup. | Extension tab/ref tests, real-upstream stable-id/label tab lifecycle, runtime session/resume tests, presentation redaction tests, lifecycle harness. | Supported. External profile/auth state remains operator-owned. |
164
173
  | Network, storage, artifacts, diagnostics, and performance | Canonical tokens from baseline section `network-storage-artifacts-diagnostics`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage), diagnostic sections, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Thin passthrough plus native-owned click/download execution for all downloads (including loopback, Blob exports, and redirects), compact diagnostics with identical credential redaction for page/origin/URL-filtered details and exports, route-mock warnings, useful-but-redacted storage output, stream idempotency normalization, FPS/cursor/contact-sheet-aware recording destinations/pinning and contact-sheet image verification, truthful ref-refresh and missing-ffmpeg warnings, positive WebMCP navigation hints and catalog updates, sensitive-data redaction, timeout bounds, and cleanup-pair guidance. | Fake non-core matrix and safe real-upstream coverage for network/HAR, diff, trace/profiler, console/errors/highlight, stream, vitals, React missing-renderer, native FPS recording path/page checks and WebMCP navigation hints. | Supported. Environment-sensitive operations need suitable local/browser state. |
165
174
  | Batch, auth, confirmations, setup, dashboard, devices, and AI commands | Canonical tokens from baseline section `batch-auth-setup-ai`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup), README security notes, release docs. | Native-tool batch stdin, generated QA/lookup batch plans, auth/confirmation redaction, sessionless local auth/setup/dashboard/doctor/plugin planning, upgrade-only native text normalization with error precedence and explicit-JSON results, plugin list/show JSON envelope normalization, bare-`mcp` validation with `mcp --help` preserved, timeout/cleanup guidance. | Parser/runtime plugin and MCP unit coverage; fake-upstream plugin list/show and MCP help/blocking coverage; registered upgrade text/success/nonzero/timeout/abort/JSON controls in `test/agent-browser.upgrade-output.test.ts`; real-upstream plugin list shape probe; structured input-mode tests. | Supported. Interactive side-effecting setup/auth/chat remains upstream-owned. `plugin` is local/sessionless; `mcp` is external-client-only except help; `auth login --credential-provider` resolves credentials via a plugin; `install --with-deps` failures remain upstream-owned. |
166
- | Global flags, config, providers, policy, and environment | Canonical tokens from baseline section `options-and-env`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment), README provider/setup notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode), architecture/runtime docs. | Runtime preserves caller-requested JSON for early filters, scroll results, and preparation failures without output-file prose (help/version remains native text), and handles command discovery, value-flag prevalidation, launch-scoped flags, redacted echoes, fresh-session recovery hints, explicit sessions, provider/device launch-scoping, parent env forwarding with consistent managed idle-timeout/socket/operation-timeout overrides on top-level and helper subprocesses, subprocess completion, and package-owned Pi-scoped config for optional companion features. | Runtime tests for flags/planning/redaction/session behavior; process/ref tests for env consistency and stdio-linger completion; config/web-search/CLI tests; fake provider/specialized-skill matrix; package doctor. | Supported. Provider clouds, iOS/Appium, proxies, profiles, and credentials require external setup. |
175
+ | Global flags, config, providers, policy, and environment | Canonical tokens from baseline section `options-and-env`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment), README provider/setup notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode), architecture/runtime docs. | Runtime preserves caller-requested JSON for early filters, scroll results, and preparation failures without output-file prose (help/version remains native text), and handles command discovery, value-flag prevalidation, launch-scoped flags, redacted echoes, fresh-session recovery hints, explicit sessions, provider/device launch-scoping, parent env forwarding with consistent managed idle-timeout/socket/operation-timeout overrides and per-call explicit policy/confirmation context on top-level and helper subprocesses, subprocess completion, and package-owned Pi-scoped config for optional companion features. | Runtime tests for flags/planning/redaction/session behavior; process/ref tests for env consistency and stdio-linger completion; config/web-search/CLI tests; fake provider/specialized-skill matrix; package doctor. | Supported. Provider clouds, iOS/Appium, proxies, profiles, and credentials require external setup. |
167
176
 
168
177
  ## Follow-up decision after closure
169
178
 
@@ -168,15 +168,21 @@ Use exact labels from the latest snapshot. Return to the model when a fresh obse
168
168
 
169
169
  `screenshot --if-changed` and `--threshold <0-1>` preserve native `changed`, `revision`, `pixelChangeRatio`, and `threshold`. An unchanged response has no path, artifact, or image attachment—even if the caller requested an existing file. It succeeds as an observation, not proof that an evidence file was saved.
170
170
 
171
- `record start` / `restart` accept `--cursor`, `--contact-sheet`, and `--contact-sheet-threshold <n>`. Reported `contactSheetPath` values become image artifacts: pending during capture, checked on disk after stop, and attached when verified. The contact-sheet destination shares the active video's reservation and transcript lifecycle; same-call/batch artifact and `outputPath` aliases are rejected before dispatch. A successful direct `record restart` retains the prior take's known contact-sheet path from the existing reservation, checks it on disk, and labels it unverified because native restart omits terminal sheet evidence. Use `record stop` then a new start when verified completion of both artifacts is required. Video receipt handling remains separate. Native `--input-mode <mode>` is a sticky, live-changeable session setting, not a launch flag; per-action pointer options pass through. See the [command reference](COMMAND_REFERENCE.md#upstream-0381-rebaseline).
171
+ `record start` / `restart` accept `--cursor`, `--contact-sheet`, and `--contact-sheet-threshold <n>`. Reported `contactSheetPath` values become image artifacts: pending during capture, checked on disk after stop, and attached when verified. The contact-sheet destination shares the active video's reservation and transcript lifecycle; same-call/batch artifact and `outputPath` aliases are rejected before dispatch. A successful direct `record restart` retains the prior take's known contact-sheet path from the existing reservation, checks it on disk, and labels it unverified because native restart omits terminal sheet evidence. Use `record stop` then a new start when verified completion of both artifacts is required. Video receipt handling remains separate. Without a terminal `previousRecording` receipt, only native `restarted: true` retires the prior take; rejected or pending restarts preserve its reservation and exact stop action. Native `--input-mode <mode>` is a sticky, live-changeable session setting, not a launch flag; per-action pointer options pass through. See the [command reference](COMMAND_REFERENCE.md#upstream-0381-rebaseline).
172
172
 
173
173
  <a id="wrapper-json"></a>
174
174
 
175
175
  ## Wrapper `--json`
176
176
 
177
+ Native text cookie reads redact `name=value`, including empty names; storage reads redact `key: value`, including empty and colon-containing keys, with the existing benign-value policy. A standalone get-by-key (`storage local|session get <key>` or native shorthand `storage local|session <key>`) redacts the entire value, including embedded newlines, when that policy requires it, even if invocation display redaction changes the supplied key. Content, details, spills and exports use the same redacted text. Effective batch argv takes precedence over stdin when selecting these format heuristics; text never establishes per-row provenance. Native all-entry storage and raw-batch output do not escape multiline values: a secret continuation can look identical to a separate benign entry or ordinary page output. Text redaction cannot resolve that ambiguity; use structured JSON for an unambiguous sensitive-storage boundary. Native JSON all-entry maps retain names and benign values while redacting other values.
178
+
177
179
  When explicitly requested, visible JSON remains parseable for early preparation results too: snapshot/network filters, scroll results, and preparation failures use the same JSON envelope as ordinary execution. Diagnostic metadata remains in `details`; `outputPath` notices do not append prose to JSON. Help/version stays native text.
178
180
 
179
- The extension always plans normal browser commands with `--json` prepended in `effectiveArgs` so upstream returns structured JSON for presentation and `details`. Omit `--json` in caller `args` for ordinary prose. Include it when you need the visible tool text as a parseable JSON envelope; structured `details` remain available in both modes. Plain-text inspection (`--help`, `--version`) keeps its own output shape. Read-only skills and local/setup commands such as auth/profile/setup, `session list`, and syntactically local state lifecycle operations skip implicit session injection as documented under `sessionMode`. Upstream session/state rows and targets remain visible, and state/config/path operations pass through unchanged.
181
+ The extension prepends `--json` to normal browser commands when the caller does not select an output mode, so upstream returns structured JSON for presentation and `details`. Omit `--json` in caller `args` for ordinary prose. Include it when you need the visible tool text as a parseable JSON envelope; structured `details` remain available in both modes. Plain-text inspection (`--help`, `--version`) keeps its own output shape. Read-only skills and local/setup commands such as auth/profile/setup, `session list`, and syntactically local state lifecycle operations skip implicit session injection as documented under `sessionMode`. Upstream session/state rows and targets remain visible, and state/config/path operations pass through unchanged.
182
+
183
+ Explicit `--json false` selects native text, including native batch output. Optional booleans use upstream's last-occurrence-wins semantics; `--json` and `--json true` select JSON. Caller argv/stdin, config, and environment remain unchanged. Native text is retained as a redacted string, including leading/trailing whitespace and line breaks, even when it resembles JSON or a confirmation banner. Inline data, complete spills and output exports preserve that whitespace. Success means the process completed successfully; nonzero exit, stderr errors, timeout, cancellation, and spawn failure remain failures. Text does not supply trusted batch rows, pending confirmations, lifecycle, page-target or artifact receipts. URL-shaped page content cannot replace the session target; only structured native observations can establish it. Saved-output notices stay prose and never parse native text as JSON, including when an export fails. Native confirmation enforcement and visible confirm/deny instructions remain upstream-owned; the wrapper never auto-confirms. Use JSON mode when structured evidence is required. Helpers that need structured state continue requesting and strictly parsing JSON. Explicit `--action-policy`, `--confirm-actions`, `--debug` and `--no-auto-dialog` selections reach those helpers within the current call, including CLI boolean `false` over true config/environment defaults; the established `--confirm-actions` setting is retained in the existing session lifecycle state for same-identity helpers and follow-ups, including exact confirm/deny actions, code/advanced calls and resume. First launch and new explicit `--confirm-actions` / `--config` selections use native argv → environment → config precedence. Bare follow-ups retain selected strings, including explicit `""`, over ambient environment/discovered-config edits. Native unset is not pinned and continues native default resolution. Native compound confirmations remain pending until every required action is decided; pending navigation never becomes an observed page target, including native batch rows. Completed non-read confirmation results establish their resulting page target before ordinary follow-ups. Successful close retires the setting; replacement cleanup uses the old identity's setting and preserves continuity if close fails; Electron launch and host helpers use the same selected setting. Definitely unestablished attempts (wrapper rejection, native CLI parse/config failure or local inspection) restore the prior setting when no earlier browser helper could have established the request; unknown/interrupted dispatch retains selected intent. Other settings remain call-scoped and must stay consistent to avoid native daemon replacement.
184
+
185
+ Empty stdin batch rows (`[]`) pass through and are skipped by native execution; `[""]` remains a failing command. Native bail behavior and later reached rows remain intact. When an upstream process exits nonzero with stderr instead of an envelope, stderr is the primary error and `failureCategory` is `upstream-error`; `details.parseError` retains secondary diagnostics. Zero-exit malformed machine output still fails strict JSON parsing.
180
186
 
181
187
  ## Experimental WebMCP
182
188
 
@@ -223,10 +229,11 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
223
229
  - Do not assume Playwright selector dialects such as text=Close or button:has-text('Close') are supported wrapper syntax unless current upstream agent-browser behavior has been verified.
224
230
  - For authenticated or user-specific content explicitly requested by the user, such as feeds, inboxes, account pages, or private dashboards, use a real profile only when the user/config asks for it or profiles have been inspected; do not assume --profile Default exists on every machine. Do not use a real profile for public pages just because they are dashboards. Treat visible page content from real profiles as model-visible transcript data. On macOS, copied Chrome profiles may omit encrypted cookies, so profile selection alone is not proof of authentication; verify the target page and use a user-approved headed login once when needed. Use --auto-connect only if profile-based reuse is unavailable or the task is specifically about attaching to a running debug-enabled browser. If profile/user-data-dir resolution fails, stop retrying opens, run profiles and/or doctor through agent_browser, then report what the user needs to configure.
225
231
  - Ordinary bare calls share one native browser within your root Pi parent/subagent group; unrelated roots browse independently. Coordinate multi-call navigation inside your group with existing subagent/intercom tools. Explicit native session defaults still win. Pi quit does not close group browsers; never close an unrelated browser.
226
- - When using launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), put them on the first command for that session. If you intentionally use an explicit --session, keep using that same explicit session for follow-ups.
232
+ - When using launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --engine, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), put them on the first command for that session. If you intentionally use an explicit --session, keep using that same explicit session for follow-ups.
227
233
  - Cooperating Pi processes serialize each complete browser operation and code cell by native socket context, namespace, and session, including target/ref helpers and state updates. Different identities remain concurrent; namespace-wide close waits for matching operations. All participating Pi instances must load this release; humans, third-party clients, and older extensions are outside that coordination. It is not a transaction or rollback. For raw batches whose later content depends on navigation, use exact batch --bail or split the calls.
228
234
  - After a successful `connect`, `--cdp`, or enabled `--auto-connect` call, verify with get url and keep using the resulting session without repeating the attach flag. The wrapper remembers that attachment across active-branch reload/resume and live-checks the URL before later page reads/interactions because an attached browser can drift externally; caller config, file access, launch arguments, and environment pass through unchanged. A successful close clears the marker. When several named sessions share one Chrome, pass --pin-tab once (AGENT_BROWSER_PIN_TAB) so a closed bound tab fails as tab_gone instead of acting on a neighbor; recover with tab new or tab list. --no-pin-tab turns the sticky pin off. tab list includes each tab's CDP targetId, accepted as a tab ref.
229
- - If you already used the implicit session and now need launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), retry with top-level sessionMode set to fresh or pass an explicit --session for the new launch; never pass --session-mode inside args. After a successful unnamed fresh launch, later auto calls follow that new session.
235
+ - If you already used the implicit session and now need launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --ca-cert, --no-ca-cert, --enable, --engine, --executable-path, --webgpu, --no-webmcp, --init-script, --idle-timeout, --args, --user-agent, --headed, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), retry with top-level sessionMode set to fresh or pass an explicit --session for the new launch; never pass --session-mode inside args. After a successful unnamed fresh launch, later auto calls follow that new session.
236
+ - Chrome is the default. Use separately installed Lightpanda only when explicitly selected with --engine lightpanda, environment or native config; --engine is launch-scoped, so use sessionMode:fresh to change a managed browser. Lightpanda text-rendered screenshots do not prove graphical layout/visibility or mouse coordinates. Chrome profiles, headed mode, multi-tab, WebGPU and native WebMCP are not parity capabilities; choose Chrome explicitly when needed, never assume automatic fallback.
230
237
  - For WebGPU pages, use args ["--webgpu", "open", "<url>"] on a fresh local browser launch; use doctor --webgpu (or --headed on Linux/Windows capture paths) to prove rendering before trusting a non-black screenshot. WebGPU cannot be combined with --cdp, --auto-connect, or provider launches unless --webgpu false overrides an enabled config/environment default.
231
238
  - For experimental WebMCP page tools, use webmcp list, then webmcp invoke <tool> with --params and optional --frame/--detach/--timeout; use webmcp result or cancel for detached calls. Locally managed Chrome enables WebMCP by default; a positive navigation hint means the page has tools to list. --no-webmcp is launch-scoped and requires a fresh session; invoke/result/cancel can mutate or navigate, so refresh snapshot refs afterward.
232
239
  - For --allowed-domains, use a fresh local Chrome context. Upstream rejects CDP/auto-connect, profiles, restore/state replay, direct-page providers, iOS/Safari, and startup/profile Chrome args because they cannot guarantee containment; Chromium also disables RTCPeerConnection while the allowlist is active.
@@ -238,7 +245,7 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
238
245
  - For non-core families, pass current upstream commands through the native tool directly: network requests, network route <url>, network har start/stop [path], diff snapshot, diff screenshot --baseline <file>, diff url <u1> <u2>, trace start, trace stop [path], profiler start, profiler stop [path], record start <path>, record stop, console/errors [--clear], highlight <selector>, inspect, clipboard read, clipboard write <text>, clipboard copy/paste, stream enable/disable/status, dashboard start/stop, device list for iOS simulator inventory, and chat <message>. For compact network requests output, prefer details.nextActions for request detail, route-mock diagnostics, actionable failed-request agent_browser_network_source, filtering, clearing the aggregate buffer before repro, or HAR capture follow-ups instead of guessing request-id syntax. Artifact-producing commands report details.artifacts and verification state; long-running starts such as stream, dashboard, trace/profiler, and record should be paired with the matching stop/disable command when the task is done; stream enable already-enabled outcomes are treated as idempotent success with status/disable follow-ups.
239
246
  - For Electron desktop apps, enable agent_browser_electron through agent_browser_tools for wrapper-owned discovery, isolated launch, status, compact probe, and cleanup: list first, treat likely-sensitive annotations as hints rather than enforcement, launch with the default snapshot handoff unless handoff: "tabs" is the safer diagnostic starting point, use agent_browser_electron probe or snapshot -i/agent_browser_qa attached:true for current-session state, and always cleanup the returned launchId when done. agent_browser_electron launch uses an isolated temporary profile; it does not reuse the app's normal signed-in profile or attach to an already-running authenticated app. For signed-in local app state, host-launch the normal app with --remote-debugging-port when appropriate, then use raw args connect <port|url>; after connect, run get url to verify the active target before page-content reads, inspect tab list, select the stable tab id such as tab t2, verify it again with get url, then run a condition wait or snapshot -i before using refs. close commands (`close`, `quit`, or `exit`) only close the browser/CDP session; leave manually launched app shutdown, profile cleanup, and explicit artifacts to the host owner.
240
247
  - For provider or specialized app workflows, load version-matched upstream guidance with skills get agentcore|electron|slack|dogfood|vercel-sandbox|derive-client through the native tool; add --full when you need references/templates, and use skills get --all only for broad skill audits. Use derive-client when recording HAR traffic to generate a standalone API client; prefer network har start (text bodies by default) or network har start --content all|none before multi-step capture. For accessibility audits use a11y or a11y --tags wcag2a,wcag2aa (CDP browsers only). Hosted sandbox workflows should use upstream @agent-browser/sandbox helpers outside this wrapper. Provider launches such as -p ios, --provider browserbase/kernel/browseruse/browserless/agentcore, and iOS --device are upstream-owned setup paths; use sessionMode fresh when switching providers and expect external credentials or local Appium/Xcode setup to be required.
241
- - For dialogs and frames, use dialog status/accept/dismiss and frame <selector|main> through native args; dialog commands and eval snippets that look like alert/confirm/prompt/dialog triggers are shorter-bounded than normal browser calls, and timed-out dialog-like interactions may add inspect-dialog-after-timeout, dismiss-dialog-after-timeout, or recover-fresh-session-after-dialog-timeout nextActions. When --confirm-actions produces a pending confirmation, use details.nextActions or exact confirm <id> / deny <id> calls instead of inventing ids.
248
+ - For dialogs and frames, use dialog status/accept/dismiss and frame <selector|main> through native args; dialog commands and eval snippets that look like alert/confirm/prompt/dialog triggers are shorter-bounded than normal browser calls, and timed-out dialog-like interactions may add inspect-dialog-after-timeout, dismiss-dialog-after-timeout, or recover-fresh-session-after-dialog-timeout nextActions. Set --confirm-actions on the session's first command; selected strings, including an explicit empty string, follow same-identity helpers and resumed follow-ups until successful close. Native unset keeps ordinary native defaults. Changing a running daemon's policy can discard pending confirmations. When it produces a pending confirmation, including a cold-reopen helper, follow the exact details.nextActions or confirm <id> / deny <id> calls without repeating the policy or inventing ids. Matching native guarded controls avoid page helpers that could replace the pending action; this is not native ID-validation proof or browser independence. In native batch, only a matching first effective decision gets that helper suppression; later page/ref checks remain. Nested decisions remain pending; failed inner actions fail, and completed browser actions reconcile their page state. A pending tab_new transport may show a validated after_confirmation open request and conditional guidance: follow it in the same session and new tab only after successful tab approval, never after denial. The extension does not execute that continuation. Confirmed full snapshots install returned refs; a final confirmed capture supplies the next same-identity call's freshness sample once, including semantic role click/check/fill and unique accessible select resolution, with target/membership/invalidation checks intact. On this one use, live verification compares the complete capture URL, including fragments, and rejects refs when it changes—including wrapper-owned managed sessions and warm sessions without resume. Ordinary ref/tab URL comparisons remain fragment-insensitive. Pending snapshots display confirmation required, not an empty page. Intervening calls or later rows consume it; DOM-only renames on the same page after that sample are not resampled on this one use.
242
249
  - If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or agent_browser_qa attached:true; for disappearance checks, use wait --fn predicates instead of stale upstream-help examples like wait <selector> --state hidden. Use agent_browser_electron probe/status for wrapper-owned launch health or target mismatch. Fixed waits are a last resort: their duration is positional (wait <ms>, not wait --time <ms>). Use explicit --timeout or top-level timeoutMs for legitimately slow waits, and treat a successful payload like "waited":"timeout" as elapsed time only—verify completion with an observed condition, fresh snapshot, or screenshot.
243
250
  - For feed, timeline, or inbox reading tasks, focus on the main timeline/list region and read the first item there rather than unrelated composer or sidebar content.
244
251
  - For read-only browsing tasks, use read <url> for documentation or other unstructured text without requiring a Chrome page, or read with no URL for rendered active-tab DOM. Prefer the current snapshot, structured ref labels, getters, or scoped eval --stdin when you need interactive structure or targeted page state. Only click into media viewers, detail routes, or new pages when the current view does not contain the needed information.
@@ -246,7 +253,7 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
246
253
  - On dashboards with nested scroll containers, verify scroll with a screenshot or fresh snapshot -i; if the viewport did not move, details.data.scrolled may be false/noMovement true and you should prefer scrollintoview <@ref> or target the actual scrollable region with scroll <selector> <dir> [px|percent]. For native selects, use select <selector> <value...> (or agent_browser_action select) instead of clicking option refs; for custom comboboxes, a click/agent_browser_action may only focus the field, so re-snapshot and use keyboard type <text> for focused input, press ArrowDown or press Enter, or visible option refs. Raw type requires both <selector> and <text>.
247
254
  - When using eval --stdin, scope checks and actions to the target element or route whenever possible instead of relying on broad page-wide text heuristics.
248
255
  - When using eval --stdin for extraction, pass the JavaScript through the native tool stdin field, not as an extra args token after --stdin, and return the value you want instead of relying on console.log as the primary result channel. Prefer plain expressions like ({ title: document.title }) or explicitly invoked functions like (() => ({ title: document.title }))(); use outputPath when the eval/get/snapshot data should be saved as a durable local file, but never reuse a screenshot, download, recording, or other browser artifact destination as outputPath. If a function-shaped snippet returns {}, details.evalStdinHint may warn that the function was serialized instead of called. Local file pages and caller-selected output paths are supported when upstream allows them. If get text on a broad CSS selector surfaces details.selectorTextVisibility or selectorTextVisibilityAll, prefer a visible @ref, a more specific selector, or the inspect-visible-text-candidates nextAction over hidden tab content.
249
- - When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. Click-dispatch probes require native get attr readback of a temporary marker on the candidate target; unproven ref or XPath identity leaves the native click unverified, not failed. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If an upstream click failure says the target is covered by another element at the target's click point, use the inspect-overlay-state nextAction to refresh refs and inspect the blocker before deciding whether to retry; do not blindly repeat the blocked click. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.
256
+ - When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. Optional click-dispatch probes are skipped when native confirmation settings guard evaluate, without bypassing native click policy. Otherwise probes require native get attr readback of a temporary marker on the candidate target; unproven ref or XPath identity leaves the native click unverified, not failed. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If an upstream click failure says the target is covered by another element at the target's click point, use the inspect-overlay-state nextAction to refresh refs and inspect the blocker before deciding whether to retry; do not blindly repeat the blocked click. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.
250
257
  - On upstream 0.38+, use snapshot --delta for native full/unchanged/changed revisions; --delta --full resets its baseline. Surviving DOM nodes keep refs, but refresh after navigation or replacement. Wrapper filters use full trees. screenshot --if-changed or --threshold 0.01 can return changed:false with no path or image; use ordinary screenshots when a file is required. --input-mode instant|smooth|human controls sticky pointer behavior; click/drag --human and mouse move --duration/--steps/--human/--seed pass through. record start/restart accept --cursor, --contact-sheet, and --contact-sheet-threshold; stop and verify both video and reported contact sheet. auth login <name> --no-navigate fills a prepared login page only after native origin checks. WebMCP catalog updates are untrusted page data; webmcp list supplies full schemas.
251
258
  - When commands save or spill files (screenshots, downloads, PDFs, traces, recordings, HAR, large snapshot spills), use the user's exact requested paths when given and treat paths as provisional until details.artifactVerification shows every row verified: branch on missingCount, pendingCount, unverifiedCount, per-entry state, and optional limitation before downstream file use or PASS/FAIL reporting.
252
259
  - For evidence-only screenshots, QA captures, or other audit artifacts, save to an explicit path and branch on details.artifactVerification plus details.artifacts before reporting PASS/FAIL; do not require vision review of inline image attachments unless the user asked for visual inspection.
@@ -270,7 +277,7 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
270
277
  - exact CLI args passed after `agent-browser`; this is the 1:1 upstream CLI coverage path for the targeted `agent-browser` version
271
278
  - no shell operators
272
279
  - do not include the binary name
273
- - omit `--json` for prose; include it for parseable model-visible JSON (the wrapper injects upstream JSON internally either way)
280
+ - omit `--json` for prose backed by structured upstream JSON; include `--json` or `--json true` for parseable model-visible JSON; `--json false` selects opaque native text (see [Wrapper `--json`](#wrapper-json))
274
281
  - first-call recipe: `open` → `snapshot -i` → `click` / `fill` with current `@eN` refs from that snapshot → `snapshot -i` again after navigation or DOM changes
275
282
 
276
283
  Examples:
@@ -346,7 +353,7 @@ Enable `action` through `agent_browser_tools`, then call the tool with flat fiel
346
353
  | `role`, `name` | Accessible role and optional name; role may replace locator value and must agree if both are given |
347
354
  | `session` | Explicit native session; preserved in compiled commands and recovery |
348
355
 
349
- Locator click/check/fill compiles to native `find`; direct targets compile to their ordinary command. Select uses native `select <selector> <value...>`. Accessible select resolves exactly one current visible combobox/listbox through a fresh snapshot; missing or ambiguous targets fail before action. Use `role: "combobox"` or `"listbox"` plus `name`, or label plus option `values`. No native `find ... select` capability is assumed.
356
+ Locator click/check/fill compiles to native `find`; direct targets compile to their ordinary command. Select uses native `select <selector> <value...>`. Accessible select resolves exactly one current visible combobox/listbox through a fresh snapshot or the eligible one-use confirmed capture; missing or ambiguous targets fail before action. Use `role: "combobox"` or `"listbox"` plus `name`, or label plus option `values`. No native `find ... select` capability is assumed.
350
357
 
351
358
  ```json
352
359
  { "action": "fill", "locator": "label", "value": "Email", "text": "user@example.com" }
@@ -480,7 +487,7 @@ Example:
480
487
 
481
488
  ### `sessionMode`
482
489
 
483
- A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` selects a caller-owned browser for direct, code, action, QA, and source tools. Per-call flags still win. Like literal `--session`, this selection takes precedence over `fresh`; it reports `usedImplicitSession: false` and is not closed when Pi quits. Without a selected native session, attachment, or explicit fresh launch, auto calls select a stable root-Pi named session and root-specific native restore key. `PI_SUBAGENT_ROOT_SESSION_ID` carries the parent identity to descendants; ordinary new/fork/clone roots differ, while resume/cwd changes keep identity. Root sessions report `usedImplicitSession: false` and survive parent/child quit. Global/override package profile names with `policy: "always"` and executable defaults bootstrap only inactive automatic roots. Active roots preserve native launch settings/current restore key, including matching root-name follow-ups; unrelated explicit browsers are unchanged. Profile paths and project-only profile config remain advisory. Native restore covers cookies and web storage, not newly created IndexedDB or memory-only authentication. See the README for communal bootstrap and persistent-profile fallback. Active older/fresh managed sessions retain the lifecycle below. `agent_browser_qa` with `attached: true` can inspect this shared current session using the same live target checks. See [shared browser defaults](../README.md#shared-browser-defaults).
490
+ A native `session` default from user/project JSON or `AGENT_BROWSER_SESSION` selects a caller-owned browser for direct, code, action, QA, and source tools. Per-call flags still win. Like literal `--session`, this selection takes precedence over `fresh`; it reports `usedImplicitSession: false` and is not closed when Pi quits. Without a selected native session, attachment, or explicit fresh launch, auto calls select a stable root-Pi named session and root-specific native restore key. `PI_SUBAGENT_ROOT_SESSION_ID` carries the parent identity to descendants; ordinary new/fork/clone roots differ, while resume/cwd changes keep identity. Root sessions report `usedImplicitSession: false` and survive parent/child quit. Global/override package profile names with `policy: "always"` and executable defaults bootstrap only inactive automatic roots whose effective engine is Chrome. Lightpanda receives no wrapper-supplied Chrome profile/executable defaults; caller native args, environment and config remain authoritative. Active roots preserve native launch settings/current restore key, including matching root-name follow-ups; unrelated explicit browsers are unchanged. Profile paths and project-only profile config remain advisory. Native restore covers cookies and web storage, not newly created IndexedDB or memory-only authentication. See the README for communal bootstrap and persistent-profile fallback. Active older/fresh managed sessions retain the lifecycle below. `agent_browser_qa` with `attached: true` can inspect this shared current session using the same live target checks. See [shared browser defaults](../README.md#shared-browser-defaults).
484
491
 
485
492
  - type: `"auto" | "fresh"`
486
493
  - optional
@@ -490,7 +497,7 @@ Behavior:
490
497
  - if `args` already include `--session` (including argv compiled from `agent_browser_action.session`), upstream session choice wins
491
498
  - `"auto"` prepends the current extension-managed active session when appropriate
492
499
  - after resume, a confirmed inactive wrapper-owned daemon with automatic managed restore enabled reopens its complete recorded URL, including its fragment, before the first current-page operation, including `get url` and `reload`. Non-page calls such as `tab list` or explicit HTTP `read <url>` can start a daemon without consuming that pending reopen, even across branch/reload replay. The wrapper verifies the observed tab and invalidates old refs; native `open` resets frame scope. This reloads the page with restored cookies/storage, not unsaved forms, JavaScript memory, or history. Live wrong-tab recovery does not navigate. Explicit URL reads, URL `a11y`/`vitals`, `diff url`, `window new`, URL-bearing recording commands, and explicit navigation/context changes do not require the old tab. Caller-owned/attached and restore-disabled sessions are not auto-reopened
493
- - `"fresh"` rotates that managed session to a fresh upstream launch so startup-scoped flags like `--profile`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--headed`, `--enable`, `-p` / `--provider`, or iOS `--device` apply and later default calls follow the new browser; `--idle-timeout` must equal the Pi process's configured managed idle timeout or the wrapper rejects it with restart guidance
500
+ - `"fresh"` rotates that managed session to a fresh upstream launch so startup-scoped flags like `--engine`, `--profile`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--no-webmcp`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--headed`, `--enable`, `-p` / `--provider`, or iOS `--device` apply and later default calls follow the new browser; `--idle-timeout` must equal the Pi process's configured managed idle timeout or the wrapper rejects it with restart guidance
494
501
  - upstream `--webgpu` is a launch-scoped optional boolean: both enabled and explicit `false` values require a fresh managed launch once an implicit session exists; enabled WebGPU is local-launch-only and upstream rejects combinations with CDP, auto-connect, or providers
495
502
  - upstream `--no-webmcp` is also launch-scoped for bare/`true` and explicit `false` values because it selects whether locally managed Chrome enables the experimental feature
496
503
  - upstream restore sessions may periodically save cookies/localStorage while open; an explicit `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` passes through unchanged when a daemon launches (`30000` upstream default, `0` disables periodic saves but keeps native close saves), while wrapper-owned headed launches default it to `0` when unset to avoid upstream 0.33.2's visible temporary collector tabs; direct window close can lose newer state because headed browsers are exempt from idle shutdown, the effective interval persists across resume and changing it in either direction on a running wrapper-owned headed daemon requires close plus a fresh launch, and those restore files remain upstream-owned rather than wrapper artifacts
@@ -503,14 +510,14 @@ Recommended use:
503
510
 
504
511
  ## Wrapper behavior
505
512
 
506
- Caller `args` should omit `--json`; the wrapper prepends it for normal execution so `details` and presentation stay structured. See [Wrapper `--json`](#wrapper-json).
513
+ Caller `args` may omit `--json` for structured details and prose; explicit `--json false` selects opaque native text. See [Wrapper `--json`](#wrapper-json).
507
514
 
508
515
  The extension should:
509
- - inject `--json`
516
+ - inject `--json` when no caller output mode is selected
510
517
  - invoke `agent-browser` directly, not through a shell
511
- - parse JSON output into tool details
518
+ - parse JSON-mode output into tool details
512
519
  - handle observed JSON result shapes, including the array returned by `batch --json`
513
- - allow plain-text output for native inspection calls and valid sessionless `upgrade` commands; all other commands retain JSON envelope validation
520
+ - allow plain-text output for native inspection calls, valid sessionless `upgrade` commands and explicit `--json false`; JSON-mode browser commands retain strict envelope validation
514
521
  - support those inspection calls unconditionally so the tool contract stays local and predictable
515
522
 
516
523
  <!-- agent-browser-playbook:start inspection -->
@@ -540,10 +547,12 @@ Inner code calls use `modelVisible: false` internally: no full model prose rende
540
547
 
541
548
  ### Image observations
542
549
 
543
- `imageObservations[]` carries verified capture metadata: `path`, optional `id` (cell-local code handle), `mimeType`, optional image `pixels` dimensions, `capture` (`viewport`, `full-page`, `element`, or `unknown`), and `geometry`. `geometry.status` is `measured` or `unknown`, with a `reason` and available before/after samples. Samples include CSS viewport/document dimensions, scroll offsets, device pixel ratio, visual viewport, frame evidence, and a uniquely measured CSS element box when available.
550
+ `imageObservations[]` carries verified capture metadata: `path`, optional `id` (cell-local code handle), `mimeType`, optional image `pixels` dimensions, `capture` (`viewport`, `full-page`, `element`, or `unknown`), optional `rendering: "text"` for positively identified Lightpanda text representations, and `geometry`. `geometry.status` is `measured` or `unknown`, with a `reason` and available before/after samples. Samples include CSS viewport/document dimensions, scroll offsets, device pixel ratio, visual viewport, frame evidence, and a uniquely measured CSS element box when available.
544
551
 
545
552
  A measured mapping requires matching pre/post samples and image dimensions that exactly match the CSS crop at the sampled DPR. It includes document-space `crop: { x, y, width, height }` and independent `pixelsPerCssPixel: { x, y }`. This is bracketed measurement, not an atomic capture guarantee. Viewport crop starts at sampled document scroll; full-page crop starts at document origin. Scrolled/ambiguous/non-CSS element captures, child-frame uncertainty, changed geometry, zoom/offset visual viewports, and captures without samples (including native batch captures) remain unknown rather than borrowing final-page geometry.
546
553
 
554
+ Lightpanda renders page text into an image rather than capturing graphical layout. Native engine metadata labels verified direct/batch/code images; standalone samples also recognize its browser brand when native engine metadata is absent or describes a CDP attachment as Chrome. Identified images remain attached but always have unknown geometry with no crop/scale mapping, even if synthetic dimensions match. Missing provenance does not justify coordinates. See [engine limits](COMMAND_REFERENCE.md#lightpanda-engine).
555
+
547
556
  Native mouse coordinates are **current viewport CSS pixels**. For measured geometry, original image pixel `(px, py)` maps to document CSS `(crop.x + px / scale.x, crop.y + py / scale.y)`; subtract current document scroll for viewport coordinates only after rechecking frame/scroll context. Prefer current refs/locators. Pi resize notes map sent-image pixels to original-image pixels only, not CSS coordinates. No provider image-detail override or model-specific coordinate rewrite is applied.
548
557
 
549
558
  Conditional `changed: false` captures have no new image/artifact attachment. A path in JSON is not vision input. Ordinary screenshot attachments use existing verification/size checks; code explicitly selects a handle from its current call. Unknown geometry permits inspection, but cannot justify coordinate input.
@@ -632,7 +641,7 @@ For `batch`, each `batchSteps[]` entry can carry its own `nextActions` for that
632
641
 
633
642
  `pageChangeSummary` is an optional compact summary for mutation-prone and artifact-producing commands. It includes `changeType` (`"navigation"`, `"mutation"`, `"artifact"`, or `"confirmation"`), `observed`, `command`, a readable `summary`, optional `title`/`url`, optional `artifactCount` or `savedFilePath`, and `nextActionIds`. `observed: false` means upstream dispatched a mutation-capable action but the wrapper did not observe an application change; standalone results also append a visible `Action dispatched; application change unverified` warning. The wrapper maintains explicit command/subcommand capability checks through `isPageChangeSummaryCommand` in `extensions/agent-browser/lib/command-taxonomy.ts`: those commands still emit a `mutation`-typed summary when upstream JSON lacks navigation metadata, as long as no stronger signal (artifact, saved path, navigation fields, or pending confirmation) applies. That capability is independent from `invalidatesBatchRefs` and `triggersPostMutationSnapshot`, so artifact summaries like `download` / `screenshot` and guarded-but-non-invalidating `fill` are documented directly in the capability table instead of implied by broad set spreading. Commands outside that set omit `pageChangeSummary` unless the parsed payload shows navigation, a confirmation prompt, saved files, or artifacts—including read-only inspection commands, which normally have no summary unless one of those signals appears. For `batch`, the top-level summary favors artifact rollups when any step produced artifacts; otherwise it synthesizes an observed-or-unverified summary from step evidence. Visible batch output promotes dispatch-only mutation evidence before step details and states that fixed waits are not postconditions. Agents should verify URL/text/state or an external receipt for important mutations before continuing.
634
643
 
635
- `clickDispatch` may appear after a **top-level non-Electron** direct `click` when the wrapper installed a target-specific DOM-event probe, upstream reported success, and the post-click probe found no trusted DOM event reached the resolved target. Target-specific probes cover `xpath=` targets and role-gated `@e…` refs when the latest wrapper-tracked snapshot has role/name metadata; eligible ref roles are `button`, `checkbox`, `menuitem`, `radio`, `switch`, and `tab`, and the role/name must be unique in both the saved snapshot and the live candidates. Before trusting either a ref or XPath candidate, the wrapper sets a per-probe temporary DOM attribute and requires native `get attr <original-selector> <marker>` readback to match it. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and leaves the native click to run without dispatch verification. This does not implement a new accessible-name algorithm or promise that every ref is observable. Duplicate-name refs pass through without a probe because snapshot order can change and is not proof of target identity. Raw `find … click` locator calls, including compiled `agent_browser_action` clicks that still execute as upstream `find`, are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. It does **not** take a fresh pre-click snapshot because that could recycle upstream refs before the intended click. The wrapper does **not** replay clicks in-page. On a miss it marks the tool failed, appends `Click dispatch diagnostic: …`, and sets `clickDispatch.status` to `"no-native-event-observed"` with `reason: "native-click-produced-no-target-dom-event"`, `nativeEventCount`, and a redacted `target` descriptor (`kind: "xpath"` plus `selector`, or `kind: "accessible"` plus `refId`, `role`, and redacted `name`). `details.nextActions` gains `inspect-click-dispatch-miss` (`snapshot -i`) and `retry-click-after-dispatch-miss` (same upstream click argv, session-prefixed when applicable). If a local static fixture must be exercised despite this diagnostic, a caller may explicitly run a programmatic activation via `eval --stdin` such as `document.querySelector(...).click()`, but that emits an untrusted scripted event and is only a debugging/workaround path; it must not be used as proof that real user-like clicking works or to bypass prompt stop boundaries. This diagnostic is only for standalone top-level direct click calls; `find` locator clicks and native batch/QA click steps remain upstream-owned behavior.
644
+ `clickDispatch` may appear after a **top-level non-Electron** direct `click` when the wrapper installed a target-specific DOM-event probe, upstream reported success, and the post-click probe found no trusted DOM event reached the resolved target. Target-specific probes cover `xpath=` targets and role-gated `@e…` refs when the latest wrapper-tracked snapshot has role/name metadata; eligible ref roles are `button`, `checkbox`, `menuitem`, `radio`, `switch`, and `tab`, and the role/name must be unique in both the saved snapshot and the live candidates. Before trusting either a ref or XPath candidate, the wrapper sets a per-probe temporary DOM attribute and requires native `get attr <original-selector> <marker>` readback to match it. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and leaves the native click to run without dispatch verification. Probe listeners and the window marker self-expire after the call's process-timeout budget plus the existing two-second cleanup grace when a pending decision prevents normal cleanup; normal collection cancels the timer. Expiry dispatches no browser helper and missing/expired probes cannot diagnose a dispatch failure. This does not implement a new accessible-name algorithm or promise that every ref is observable. Duplicate-name refs pass through without a probe because snapshot order can change and is not proof of target identity. Raw `find … click` locator calls, including compiled `agent_browser_action` clicks that still execute as upstream `find`, are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. It does **not** take a fresh pre-click snapshot because that could recycle upstream refs before the intended click. The wrapper does **not** replay clicks in-page. On a miss it marks the tool failed, appends `Click dispatch diagnostic: …`, and sets `clickDispatch.status` to `"no-native-event-observed"` with `reason: "native-click-produced-no-target-dom-event"`, `nativeEventCount`, and a redacted `target` descriptor (`kind: "xpath"` plus `selector`, or `kind: "accessible"` plus `refId`, `role`, and redacted `name`). `details.nextActions` gains `inspect-click-dispatch-miss` (`snapshot -i`) and `retry-click-after-dispatch-miss` (same upstream click argv, session-prefixed when applicable). If a local static fixture must be exercised despite this diagnostic, a caller may explicitly run a programmatic activation via `eval --stdin` such as `document.querySelector(...).click()`, but that emits an untrusted scripted event and is only a debugging/workaround path; it must not be used as proof that real user-like clicking works or to bypass prompt stop boundaries. This diagnostic is only for standalone top-level direct click calls; `find` locator clicks and native batch/QA click steps remain upstream-owned behavior.
636
645
 
637
646
  `promptGuard` may appear on wrapper-blocked calls only for concrete, machine-checkable prompt requirements. `reason: "requested-artifacts-missing-before-close"` blocks `close` / `quit` / `exit` when the prompt used a direct screenshot/recording creation phrase with a destination such as `here`, `at`, `as`, or `to` and the session artifact manifest has not verified that exact path; a destination heading can carry that intent across contiguous plain or Markdown-bulleted path-only list lines. Bare, review-only, fenced-reference, conditional, permissive/uncertain, directly negated, and Pi clipboard/attachment image/video paths are treated as input, not output requirements, while subordinate requirements such as “do not close until you save” remain output intent. A recording-availability qualifier may precede, appear within, or follow its path/list and is scoped through the next path boundary; if the same path appears more than once, any required occurrence takes precedence. Markdown-link destinations resolve to the destination path. Explicitly optional artifacts are not close requirements. The classifier is deliberately conservative, so use wording such as `Save a screenshot here: <path>` when machine-enforced close blocking matters. Optional recording paths are only required when recording appears available. The wrapper does **not** parse broad user/business intent such as “do not place the order” or “do not post anything” into click/key blocks; agents must follow those instructions themselves. Prompt guards return `failureCategory: "policy-blocked"` and `validationError` text instead of invoking upstream.
638
647
 
@@ -784,11 +793,11 @@ Matching requires the actual namespace/session, expected path and known recordin
784
793
 
785
794
  ## Browser-independent read confirmations
786
795
 
787
- `details.readConfirmation` stores only native control-response provenance for an actual explicit URL read: `{ id, sessionName, namespace?, source: "native-explicit-url-read", state: "pending" | "cleared", capabilities? }`. Page content, nested JSON text and bare DOM reads do not establish this provenance. It is replayed in existing per-session transcript state on resume/branch changes and retired by successful confirmation/denial or session close.
796
+ `details.readConfirmation` retains native control-response provenance in the existing per-session transcript state: `{ id, sessionName, namespace?, source, state: "pending" | "cleared", capabilities?, command?, action?, refSnapshotFresh?: true }`. `source: "native-guarded-action"` carries only the compact originating command and native action, never argv or a command replay blob. `source: "native-explicit-url-read"` remains specific to explicit URL reads. Page content, eval results and JSON-looking text cannot establish either provenance.
788
797
 
789
- Routing uses the actual native session, including `default` when the original read did not allocate a managed browser. Explicit caller session/namespace choices still win, and code calls cannot borrow another session's confirmation. Legacy native prompts retain this routing. Only a control response with `capabilities.readRequiresConfirmation: true` also proves native explicit-ID matching and enables matching confirm/deny without page helpers or managed-browser replacement. Legacy/unproven and DOM confirmations retain normal page checks. The advertised capability requires native ID validation before consuming a pending action, so a stale read ID cannot approve a later DOM action. A failed or expired confirmation returns exact session-status guidance, not an automatic retry of another ID.
798
+ Routing preserves the actual native session, including `default` for an unallocated explicit read, through resume/branch replay. Explicit caller identities and the code cell's fixed identity still win. An exact pending ID/canonical-identity match for a genuine guarded control suppresses page/pinning helpers that could replace the native pending slot; it neither proves native ID validation nor grants browser independence. Completed guarded actions and confirmed native batch rows use ordinary page/ref reconciliation, including click/eval/tab effects and successful close retirement through direct and code replay. Only a matching first effective guarded batch decision suppresses pre-command helpers; raw arguments still take precedence over stdin, and later page/ref/unknown-target checks remain. Requested-action pending is not labelled as a hidden helper; helper-origin pending retains its own diagnostic, and an early helper decision explicitly states that the requested command was not dispatched. Only explicit URL-read `capabilities.readRequiresConfirmation: true` grants browser-independent confirm/deny and promises native ID validation. Legacy URL reads and unrelated/nonmatching decisions keep normal checks. Failed or expired confirmations offer exact session inspection, never automatic approval of another ID. A genuine `tab_new` control may include the paired transport fields `after_confirmation: ["open", <original requested URL>]` and nonempty `guidance`; other extra fields or mismatched/malformed continuations do not grant confirmation provenance. The original URL is read with ordinary command operand handling, including preceding labels and headers. Valid conditional guidance stays model-visible and redacted: approval creates the blank/labeled tab, and only a separate caller-requested navigation follows; denial must not navigate. These fields are not stored as command replay, automatically executed, or accepted from page/eval/HTTP content.
790
799
 
791
- A confirmed HTTP failure is a tool failure regardless of that capability, including inside a batch. A pending confirmation is not success even when the native outer envelope says otherwise. If a new DOM confirmation replaces a pending read, the old read marker clears and the new confirmation keeps its own approve/deny actions and normal page checks.
800
+ A failed inner native confirmed action is a tool failure, including inside a batch, even when the outer confirmation succeeds. Nested confirmations stay pending and retain the original compact command provenance. A new native guarded control replaces the previous marker and supplies its own exact actions. A completed confirmed full snapshot installs returned refs in direct and ordered batch results; pending, denied, failed or partial captures do not grant freshness. When it is the final completed row, `refSnapshotFresh: true` allows the next same-identity call to reuse that sample once instead of requesting another gated freshness snapshot. Semantic role click/check/fill and unique role/label select use that same eligible capture with its editable metadata and unchanged action payloads. The call consumes the marker, including across direct/code/batch replay; any intervening same-identity call consumes it too. Pending snapshot controls use the ordinary confirmation-required presentation and executable actions; they are not empty-page observations. Target, membership, invalidation and native generation checks remain. On this one use, required live verification compares the complete capture URL, including fragments, and rejects refs when it changes—including wrapper-owned managed sessions and warm sessions without resume. Ordinary ref/tab URL comparisons remain fragment-insensitive. A DOM-only rename on the same page after the approved sample is not resampled on this one use; later ref operations retain ordinary freshness checks. Confirmed close also retires attachment state on live and resumed paths. If a cold managed reopen helper is pending, the result exposes that decision with `agentBrowserStarted: false` for the requested getter, unknown page target and invalid refs; it does not dispatch the getter or replay open. Successful confirm can then reconcile the native resulting target before immediate title/body inspection.
792
801
 
793
802
  ## High-value result rendering
794
803
 
@@ -855,7 +864,7 @@ If `agent-browser` is not on `PATH`, fail with a message that:
855
864
  - keep wrapper-spawned commands bounded by clamping `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented default of 25 seconds while the default wrapper child-process watchdog is 35 seconds (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides it, and top-level `timeoutMs` overrides it per call for browser CLI subprocesses). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a per-call subprocess watchdog from the requested command duration plus a small grace window. Dialog commands use `PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS` (default 5000 ms), and click/tap/find refs or tokens plus `eval --stdin` snippets whose text looks like alert/confirm/prompt/dialog triggers use `PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS` (default 8000 ms). Timed-out advanced QA or native batch calls may add `details.timeoutPartialProgress` and visible `Timeout partial progress` evidence with per-step status, retry payloads, current page title/URL, and declared artifact path checks; timed-out dialog-like commands may add dialog status/dismiss/fresh-session recovery next actions
856
865
  - interactive or long-running upstream families such as `chat` without a prompt, `dashboard start`, `stream enable`, `trace start`, `profiler start`, `record start`, `inspect`, `install`, `upgrade`, `doctor --fix`, and `confirm-interactive` are passed through thinly but remain bounded by the same wrapper timeout/session planning rules; prefer explicit arguments, single-shot `chat <message>`, non-interactive flags like `doctor --offline --quick` or `doctor --json`, and cleanup pairs such as `dashboard stop`, `stream disable`, `trace stop`, `profiler stop`, and `record stop`
857
866
  - treat successful plain-text inspection commands like `--help` and `--version` as stateless: do not inject the implicit managed session and do not let those calls claim the managed-session slot
858
- - if startup-scoped flags like `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--enable`, `-p` / `--provider`, or iOS `--device` target the current active managed session, return a validation error with a structured recovery hint that recommends `sessionMode: "fresh"`; when the call explicitly names the current managed session, remove that `--session` from the recovery payload so rotation is actionable. If daemon inspection proves an explicitly targeted older wrapper-owned session active, reject its startup-scoped flags with close-first or fresh-rotation guidance before spawn
867
+ - if startup-scoped flags like `--engine`, `--profile`, `--args`, `--user-agent`, `--executable-path`, `--ca-cert`, `--no-ca-cert`, `--webgpu`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--enable`, `-p` / `--provider`, or iOS `--device` target the current active managed session, return a validation error with a structured recovery hint that recommends `sessionMode: "fresh"`; when the call explicitly names the current managed session, remove that `--session` from the recovery payload so rotation is actionable. If daemon inspection proves an explicitly targeted older wrapper-owned session active, reject its startup-scoped flags with close-first or fresh-rotation guidance before spawn
859
868
  - for direct headless local Chrome launches to `chat.com` / `chatgpt.com` / `chat.openai.com` or `dash.cloudflare.com`, allow a narrow compatibility fallback that injects a normal Chrome `--user-agent` only when the caller did not explicitly provide one and did not choose raw Chrome arguments, headed, CDP, auto-connect, provider-backed, custom-UA, or non-Chrome behavior through argv or matching upstream environment. Wrapper-managed sessions retain that wrapper-owned user agent as per-session state across follow-up calls, failed replacement closes, and branch reload/resume. Active daemons omit both launch forms so upstream does not replace a launch-configured browser; a session proven inactive receives the retained compatibility launch values, including a fixed, comma-safe Chrome launch argument because the per-page override does not propagate to new tabs or SSO popups.
860
869
 
861
870
  ## Shared execution coordination
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-agent-browser-native",
3
- "version": "0.9.1",
3
+ "version": "0.9.3",
4
4
  "description": "pi extension that exposes agent-browser as a native tool for browser automation",
5
5
  "type": "module",
6
6
  "author": "Mitch Fultz (https://github.com/fitchmultz)",
@@ -68,9 +68,9 @@
68
68
  "typebox": "*"
69
69
  },
70
70
  "devDependencies": {
71
- "@earendil-works/pi-ai": "1.0.0",
72
- "@earendil-works/pi-coding-agent": "1.0.0",
73
- "@earendil-works/pi-tui": "1.0.0",
71
+ "@earendil-works/pi-ai": "1.0.2",
72
+ "@earendil-works/pi-coding-agent": "1.0.2",
73
+ "@earendil-works/pi-tui": "1.0.2",
74
74
  "@types/cross-spawn": "6.0.6",
75
75
  "@types/node": "24.19.1",
76
76
  "@types/which": "2.0.2",