pi-agent-browser-native 0.2.71 → 0.2.74

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/CHANGELOG.md +46 -0
  2. package/README.md +14 -12
  3. package/dist/extensions/agent-browser/index.js +104 -16
  4. package/dist/extensions/agent-browser/lib/argv-grammar.js +124 -0
  5. package/dist/extensions/agent-browser/lib/command-taxonomy.js +12 -1
  6. package/dist/extensions/agent-browser/lib/electron/cdp.js +2 -2
  7. package/dist/extensions/agent-browser/lib/electron/launch.js +48 -12
  8. package/dist/extensions/agent-browser/lib/input-modes/params.js +96 -98
  9. package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +88 -2
  10. package/dist/extensions/agent-browser/lib/managed-session-capabilities.js +22 -0
  11. package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +432 -0
  12. package/dist/extensions/agent-browser/lib/managed-session-restore.js +367 -0
  13. package/dist/extensions/agent-browser/lib/managed-session-snapshots.js +367 -0
  14. package/dist/extensions/agent-browser/lib/managed-session-state-policy.js +589 -0
  15. package/dist/extensions/agent-browser/lib/managed-session-storage.js +299 -0
  16. package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +35 -0
  17. package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +9 -2
  18. package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +40 -22
  19. package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +15 -6
  20. package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +54 -33
  21. package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +182 -0
  22. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/direct-anchor-download.js +1 -1
  23. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/network-page-filter.js +1 -1
  24. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +1 -1
  25. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +1 -1
  26. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +625 -429
  27. package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +136 -56
  28. package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +28 -40
  29. package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +102 -19
  30. package/dist/extensions/agent-browser/lib/orchestration/output-file.js +13 -1
  31. package/dist/extensions/agent-browser/lib/playbook.js +10 -9
  32. package/dist/extensions/agent-browser/lib/process-identity.js +82 -0
  33. package/dist/extensions/agent-browser/lib/process.js +270 -34
  34. package/dist/extensions/agent-browser/lib/results/artifact-manifest.js +5 -3
  35. package/dist/extensions/agent-browser/lib/results/categories.js +21 -2
  36. package/dist/extensions/agent-browser/lib/results/presentation/common.js +2 -1
  37. package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +80 -12
  38. package/dist/extensions/agent-browser/lib/results/presentation/managed-list-filter.js +42 -0
  39. package/dist/extensions/agent-browser/lib/results/recovery-actions.js +7 -0
  40. package/dist/extensions/agent-browser/lib/runtime.js +85 -85
  41. package/dist/extensions/agent-browser/lib/session-page-state.js +48 -17
  42. package/dist/extensions/agent-browser/lib/temp.js +13 -25
  43. package/docs/ARCHITECTURE.md +9 -8
  44. package/docs/COMMAND_REFERENCE.md +97 -32
  45. package/docs/ELECTRON.md +10 -10
  46. package/docs/RELEASE.md +3 -2
  47. package/docs/SUPPORT_MATRIX.md +22 -19
  48. package/docs/TOOL_CONTRACT.md +31 -28
  49. package/docs/platform-smoke.md +5 -5
  50. package/package.json +1 -1
  51. package/platform-smoke.config.mjs +3 -1
  52. package/scripts/agent-browser-capability-baseline.mjs +45 -3
@@ -126,7 +126,7 @@ For link and button text, use the **exact** visible label from the latest `snaps
126
126
 
127
127
  ## Wrapper `--json`
128
128
 
129
- The extension always plans normal browser commands with `--json` prepended in `effectiveArgs` so upstream returns structured JSON for presentation and `details`. **Do not** include `--json` in caller `args`; it is unnecessary and can confuse planning or transcript hooks that treat caller-requested JSON differently. Plain-text inspection (`--help`, `--version`) keeps its own output shape. Read-only skills and local/setup commands such as `skills list` / `skills get` / `skills path`, local auth profile management (`auth save/list/show/delete/remove`), `profiles`, `dashboard`, `device list`, `doctor`, `install`, `upgrade`, `session id`, `session info`, `session list`, `plugin add/list/show/run`, `mcp --help`, and targeted/all local saved-state maintenance including `state clear --all`, `state clear -a`, and named `state clear <session-name>` skip implicit session injection as documented under `sessionMode`; bare `mcp` server calls are rejected before spawn because they are for external MCP clients.
129
+ The extension always plans normal browser commands with `--json` prepended in `effectiveArgs` so upstream returns structured JSON for presentation and `details`. **Do not** include `--json` in caller `args`; it is unnecessary and can confuse planning or transcript hooks that treat caller-requested JSON differently. Plain-text inspection (`--help`, `--version`) keeps its own output shape. Read-only skills and local/setup commands such as `skills list` / `skills get` / `skills path`, local auth profile management (`auth save/list/show/delete/remove`), `profiles`, `dashboard`, `device list`, `doctor`, `install`, `upgrade`, `session id`, `session info`, `session list` (with wrapper-managed rows removed), `plugin add/list/show/run`, `mcp --help`, and caller-owned saved-state inspection/targeted maintenance (`state list/show/rename` and named `state clear <caller-owned-name>`) skip implicit session injection as documented under `sessionMode`; broad clear/clean and managed-state targets are rejected before spawn; bare `mcp` server calls are rejected before spawn because they are for external MCP clients.
130
130
 
131
131
  ## Headed and local fixture limits
132
132
 
@@ -135,7 +135,7 @@ The extension always plans normal browser commands with `--json` prepended in `e
135
135
  - `--executable-path <path>` selects a custom Chromium-compatible browser executable when upstream can launch it. Use it with `sessionMode: "fresh"` when switching from an already-active implicit session. For non-Chrome Chromium login state, use a full profile/user-data directory path only when upstream accepts it, or attach to a debug-enabled running browser with `--auto-connect` / `connect` when appropriate.
136
136
  - `--allowed-domains <list>` is launch-scoped under the wrapper and requires a fresh local Chrome context. As of upstream 0.32.0, it contains workers and popups and disables Chromium `RTCPeerConnection`; upstream rejects combinations with CDP/auto-connect, profiles, restore/state replay, direct-page providers, iOS/Safari, and startup/profile Chrome args because those paths cannot guarantee containment.
137
137
  - A successful headed call proves only that upstream accepted and ran the browser command. The wrapper currently has no portable contract field that proves the OS window is visible on the user's desktop. When visibility matters, collect independent evidence such as `screenshot`, `tab list`, `get url`, or `snapshot -i`, and treat “user cannot see the browser” as a display/provider/session setup issue until proven otherwise.
138
- - `localhost` / `127.0.0.1` URLs are resolved by the browser host, which may differ from the shell or Pi process that started a temporary server. Errors such as `net::ERR_EMPTY_RESPONSE` on local ports are not reliable page-render evidence; they can mean the browser cannot reach the host loopback. Use an environment-specific host-reachable address when available, or fall back to `file://` only for static fixtures.
138
+ - `localhost` / `127.0.0.1` URLs are resolved by the browser host, which may differ from the shell or Pi process that started a temporary server. Errors such as `net::ERR_EMPTY_RESPONSE` on local ports are not reliable page-render evidence; they can mean the browser cannot reach the host loopback. Use an environment-specific host-reachable HTTP(S) address. The native wrapper blocks follow-up inspection and interaction on `file://` pages to protect authenticated local browser state.
139
139
  - `file://` pages do not provide HTTP headers and can differ from HTTP pages for MIME handling, CORS, storage, and debugger/script behavior. If `eval --stdin` returns `null` or otherwise fails to prove DOM state on a `file://` page, first confirm the script was passed through the native tool `stdin` field (not as a third `args` item after `--stdin`), then treat that verification as inconclusive and use `snapshot -i`, `get text` from current refs, screenshots, or a reachable HTTP fixture instead.
140
140
  - Temporary HTTP servers launched outside the tool are host-owned. The native tool does not allocate ports, track background server PIDs, or clean them up; use a harness or shell cleanup for those processes.
141
141
 
@@ -151,17 +151,18 @@ The extension always plans normal browser commands with `--json` prepended in `e
151
151
  - For authenticated or user-specific content explicitly requested by the user, such as feeds, inboxes, account pages, or private dashboards, use a real profile only when the user/config asks for it or profiles have been inspected; do not assume --profile Default exists on every machine. Do not use a real profile for public pages just because they are dashboards. Treat visible page content from real profiles as model-visible transcript data; use --auto-connect only if profile-based reuse is unavailable or the task is specifically about attaching to a running debug-enabled browser. If profile/user-data-dir resolution fails, stop retrying opens, run profiles and/or doctor through agent_browser, then report what the user needs to configure.
152
152
  - Do not invent fixed explicit session names for routine tasks. Use the implicit session unless you truly need multiple isolated browser sessions in the same conversation.
153
153
  - When using launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --enable, --executable-path, --webgpu, --init-script, --idle-timeout, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), put them on the first command for that session. If you intentionally use an explicit --session, keep using that same explicit session for follow-ups.
154
+ - Caller-owned explicit sessions are serialized per effective canonical namespace/session inside this extension while live URL checks, semantic-action snapshots, and the requested command run. For raw batches whose later content step depends on navigation, use exact batch --bail or split the calls; unsafe continue-after-navigation-failure shapes are rejected before the batch runs.
154
155
  - If you already used the implicit session and now need launch-scoped flags (--auto-connect, --allowed-domains, --namespace, --cdp, --enable, --executable-path, --webgpu, --init-script, --idle-timeout, --device, --profile, --provider, -p, --session-name, --restore, --restore-save, --restore-check-url, --restore-check-text, --restore-check-fn, --state), retry with top-level sessionMode set to fresh or pass an explicit --session for the new launch; never pass --session-mode inside args. After a successful unnamed fresh launch, later auto calls follow that new session.
155
156
  - For WebGPU pages, use args ["--webgpu", "open", "<url>"] on a fresh local browser launch; use doctor --webgpu (or --headed on Linux/Windows capture paths) to prove rendering before trusting a non-black screenshot. WebGPU cannot be combined with --cdp, --auto-connect, or provider launches unless --webgpu false overrides an enabled config/environment default.
156
157
  - For --allowed-domains, use a fresh local Chrome context. Upstream rejects CDP/auto-connect, profiles, restore/state replay, direct-page providers, iOS/Safari, and startup/profile Chrome args because they cannot guarantee containment; Chromium also disables RTCPeerConnection while the allowlist is active.
157
158
  - For React introspection, launch the page with --enable react-devtools before first navigation, then use react tree, react inspect <fiberId>, sourceLookup candidates for local UI source hints, react renders start/stop, or react suspense; sourceLookup is experimental and reports confidence/evidence instead of guaranteed DOM-to-file mappings. For failed fetches and APIs, networkSourceLookup (experimental) correlates failed network requests with initiator metadata and bounded workspace URL literals—candidates only, not definitive blame. Use vitals [url] for Core Web Vitals and hydration timing, and pushstate <url> for client-side SPA navigation.
158
159
  - For first-navigation setup, use open without a URL plus network route --resource-type <csv>, cookies set --curl <file>, or --init-script/--enable before navigate/opening the target page.
159
- - For stateful browser context work, prefer purpose-specific page actions before dumping browser data: use auth save --password-stdin with the tool stdin field for credentials, auth list/show/delete/remove for local auth-profile maintenance, auth login when you need the browser to fill a saved profile, state save/load for portable test state, state list/show/rename/clear/clear -a/clean for saved-state lifecycle cleanup, cookies get/set/clear and storage local|session only when the task needs those values, and expect cookie/storage/auth/state summaries to redact credential-like fields while allowing benign primitive storage values when useful for local QA.
160
+ - For stateful browser context work, prefer purpose-specific page actions before dumping browser data: use auth save --password-stdin with the tool stdin field for credentials, auth list/show/delete/remove for local auth-profile maintenance, auth login when you need the browser to fill a saved profile, state save/load for portable test state, state list/show/rename and targeted `state clear <caller-owned-name>` for saved-state lifecycle cleanup (the native wrapper blocks broad clear/clean and managed targets), cookies get/set/clear and storage local|session only when the task needs those values, and expect cookie/storage/auth/state summaries to redact credential-like fields while allowing benign primitive storage values when useful for local QA.
160
161
  - Upstream restore sessions periodically autosave cookies and localStorage while the browser stays open, including page-driven background changes; AGENT_BROWSER_AUTOSAVE_INTERVAL_MS controls the interval (30000 by default; 0 disables periodic saves but keeps save-on-close), while the never value for --restore-save disables automatic saves for that restore session.
161
162
  - For batch chains that touch cookies, storage, auth, or other secret-bearing commands, use details.batchSteps for per-step artifacts, categories, spill paths, and full structured errors; top-level details.data on batch is only a compact redacted step matrix (success, argv-redacted command, redacted result or scrubbed error text) built from the same presentation rules as standalone calls.
162
163
  - For non-core families, pass current upstream commands through the native tool directly: network requests, network route <url>, network har start/stop [path], diff snapshot, diff screenshot --baseline <file>, diff url <u1> <u2>, trace start, trace stop [path], profiler start, profiler stop [path], record start <path>, record stop, console/errors [--clear], highlight <selector>, inspect, clipboard read, clipboard write <text>, clipboard copy/paste, stream enable/disable/status, dashboard start/stop, device list for iOS simulator inventory, and chat <message>. For compact network requests output, prefer details.nextActions for request detail, route-mock diagnostics, actionable failed-request networkSourceLookup, filtering, clearing the aggregate buffer before repro, or HAR capture follow-ups instead of guessing request-id syntax. Artifact-producing commands report details.artifacts and verification state; long-running starts such as stream, dashboard, trace/profiler, and record should be paired with the matching stop/disable command when the task is done; stream enable already-enabled outcomes are treated as idempotent success with status/disable follow-ups.
163
- - For Electron desktop apps, prefer top-level electron for wrapper-owned discovery, isolated launch, status, compact probe, and cleanup: list first, treat likely-sensitive annotations as hints rather than enforcement, launch with the default snapshot handoff unless handoff: "tabs" is the safer diagnostic starting point, use electron.probe or snapshot -i/qa.attached for current-session state, and always cleanup the returned launchId when done. electron.launch uses an isolated temporary profile; it does not reuse the app's normal signed-in profile or attach to an already-running authenticated app. For signed-in local app state, host-launch the normal app with --remote-debugging-port when appropriate, then use raw args connect <port|url>; after connect, inspect tab list, select the stable tab id such as tab t2, then run a condition wait or snapshot -i before using refs. close commands (`close`, `quit`, or `exit`) only close the browser/CDP session; leave manually launched app shutdown, profile cleanup, and explicit artifacts to the host owner.
164
- - For provider or specialized app workflows, load version-matched upstream guidance with skills get agentcore|electron|slack|dogfood|vercel-sandbox through the native tool; add --full when you need references/templates, and use skills get --all only for broad skill audits. Hosted sandbox workflows should use upstream @agent-browser/sandbox helpers outside this wrapper. Provider launches such as -p ios, --provider browserbase/kernel/browseruse/browserless/agentcore, and iOS --device are upstream-owned setup paths; use sessionMode fresh when switching providers and expect external credentials or local Appium/Xcode setup to be required.
164
+ - For Electron desktop apps, prefer top-level electron for wrapper-owned discovery, isolated launch, status, compact probe, and cleanup: list first, treat likely-sensitive annotations as hints rather than enforcement, launch with the default snapshot handoff unless handoff: "tabs" is the safer diagnostic starting point, use electron.probe or snapshot -i/qa.attached for current-session state, and always cleanup the returned launchId when done. electron.launch uses an isolated temporary profile; it does not reuse the app's normal signed-in profile or attach to an already-running authenticated app. For signed-in local app state, host-launch the normal app with --remote-debugging-port when appropriate, then use raw args connect <port|url>; after connect, run get url to verify the active target before page-content reads, inspect tab list, select the stable tab id such as tab t2, verify it again with get url, then run a condition wait or snapshot -i before using refs. close commands (`close`, `quit`, or `exit`) only close the browser/CDP session; leave manually launched app shutdown, profile cleanup, and explicit artifacts to the host owner.
165
+ - For provider or specialized app workflows, load version-matched upstream guidance with skills get agentcore|electron|slack|dogfood|vercel-sandbox|derive-client through the native tool; add --full when you need references/templates, and use skills get --all only for broad skill audits. Use derive-client when recording HAR traffic to generate a standalone API client; prefer network har start (text bodies by default) or network har start --content all|none before multi-step capture. For accessibility audits use a11y or a11y --tags wcag2a,wcag2aa (CDP browsers only). Hosted sandbox workflows should use upstream @agent-browser/sandbox helpers outside this wrapper. Provider launches such as -p ios, --provider browserbase/kernel/browseruse/browserless/agentcore, and iOS --device are upstream-owned setup paths; use sessionMode fresh when switching providers and expect external credentials or local Appium/Xcode setup to be required.
165
166
  - For dialogs and frames, use dialog status/accept/dismiss and frame <selector|main> through native args; dialog commands and eval snippets that look like alert/confirm/prompt/dialog triggers are shorter-bounded than normal browser calls, and timed-out dialog-like interactions may add inspect-dialog-after-timeout, dismiss-dialog-after-timeout, or recover-fresh-session-after-dialog-timeout nextActions. When --confirm-actions produces a pending confirmation, use details.nextActions or exact confirm <id> / deny <id> calls instead of inventing ids.
166
167
  - If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks, use wait --fn predicates instead of stale upstream-help examples like wait <selector> --state hidden. Use electron.probe/status for wrapper-owned launch health or target mismatch. Fixed waits are a last resort: use explicit --timeout or top-level timeoutMs for legitimately slow waits, and treat a successful payload like "waited":"timeout" as elapsed time only—verify completion with an observed condition, fresh snapshot, or screenshot.
167
168
  - For feed, timeline, or inbox reading tasks, focus on the main timeline/list region and read the first item there rather than unrelated composer or sidebar content.
@@ -169,7 +170,7 @@ The extension always plans normal browser commands with `--json` prepended in `e
169
170
  - For downloads, prefer download <selector> <path> when an element click should save a file; simple loopback anchor downloads are saved to the requested path when the wrapper can resolve an HTTP(S) href. Do not rely on click alone when you need the downloaded file on disk.
170
171
  - On dashboards with nested scroll containers, verify scroll with a screenshot or fresh snapshot -i; if the viewport did not move, details.data.scrolled may be false/noMovement true and you should prefer scrollintoview <@ref> or target the actual scrollable region with scroll <selector> <dir> [px|percent]. For native selects, use select <selector> <value...> (or semanticAction/job select) instead of clicking option refs; for custom comboboxes, a click/semanticAction may only focus the field, so re-snapshot and fall back to type, press Enter/arrow keys, or visible option refs.
171
172
  - When using eval --stdin, scope checks and actions to the target element or route whenever possible instead of relying on broad page-wide text heuristics.
172
- - When using eval --stdin for extraction, pass the JavaScript through the native tool stdin field, not as an extra args token after --stdin, and return the value you want instead of relying on console.log as the primary result channel. Prefer plain expressions like ({ title: document.title }) or explicitly invoked functions like (() => ({ title: document.title }))(); use outputPath when the eval/get/snapshot data should be saved as a durable local file. If a function-shaped snippet returns {}, details.evalStdinHint may warn that the function was serialized instead of called. On file:// pages, when upstream JSON returns result: null for non-trivial stdin, details.evalResultWarning may append Eval result warning without failing the tool—treat that as inconclusive DOM verification. If get text on a broad CSS selector surfaces details.selectorTextVisibility or selectorTextVisibilityAll, prefer a visible @ref, a more specific selector, or the inspect-visible-text-candidates nextAction over hidden tab content.
173
+ - When using eval --stdin for extraction, pass the JavaScript through the native tool stdin field, not as an extra args token after --stdin, and return the value you want instead of relying on console.log as the primary result channel. Prefer plain expressions like ({ title: document.title }) or explicitly invoked functions like (() => ({ title: document.title }))(); use outputPath when the eval/get/snapshot data should be saved as a durable local file. If a function-shaped snippet returns {}, details.evalStdinHint may warn that the function was serialized instead of called. The native wrapper blocks follow-up inspection, scripting, and interaction on file:// pages to protect authenticated local browser state; use a reachable HTTP(S) fixture instead. If get text on a broad CSS selector surfaces details.selectorTextVisibility or selectorTextVisibilityAll, prefer a visible @ref, a more specific selector, or the inspect-visible-text-candidates nextAction over hidden tab content.
173
174
  - When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.
174
175
  - When commands save or spill files (screenshots, downloads, PDFs, traces, recordings, HAR, large snapshot spills), use the user's exact requested paths when given and treat paths as provisional until details.artifactVerification shows every row verified: branch on missingCount, pendingCount, unverifiedCount, per-entry state, and optional limitation before downstream file use or PASS/FAIL reporting.
175
176
  - For evidence-only screenshots, QA captures, or other audit artifacts, save to an explicit path and branch on details.artifactVerification plus details.artifacts before reporting PASS/FAIL; do not require vision review of inline image attachments unless the user asked for visual inspection.
@@ -227,7 +228,7 @@ Examples:
227
228
  - supported actions: `click`, `fill`, `check`, `select`
228
229
  - supported locators for `click` / `fill` / `check`: `role`, `text`, `label`, `placeholder`, `alt`, `title`, `testid`
229
230
  - optional `selector` is accepted for direct `click`, `check`, and `fill` targets (including current `@refs`); do not combine it with `locator`, `value`, `role`, or `name`. For `fill`, `text` is still required.
230
- - `semanticAction` does not expose `uncheck` while upstream `find ... uncheck` is not runtime-supported; use raw `args: ["uncheck", <selector-or-ref>]` after a stable selector or current snapshot ref
231
+ - `semanticAction` does not expose `uncheck` because upstream `find` actions are only `click, fill, check, hover, text`; use raw `args: ["uncheck", <selector-or-ref>]` after a stable selector or current snapshot ref
231
232
  - for locator actions, `value` is the locator argument (for example ARIA role token `"button"`, label text, or visible substring), must be a non-empty string after trim; for `locator: "role"`, callers may provide `role` instead of redundant `value`
232
233
  - `fill` requires non-empty `text` (compiled as the trailing value argument to `find`)
233
234
  - `select` requires non-empty `selector` plus either `value` (single option value) or `values` (non-empty array of option values). `select` does not accept `locator`, `role`, `name`, or `text`; upstream `find` does not expose a verified `select` action, so role/name/label dropdown targeting must first be resolved to a stable selector or current `@ref`.
@@ -246,7 +247,7 @@ Compilation (then `--json` and session handling apply like any other call):
246
247
  | `select` + `selector` + `value` / `values` | `["select",<selector>,<value...>]` |
247
248
  | any supported action + `session` | prepends `["--session",<session>]` before the compiled argv |
248
249
 
249
- When `semanticAction` compiles successfully, `details.compiledSemanticAction` echoes `{ action, locator, args }` for `find` actions, `{ action, selector, args }` for direct selector/ref click/check/fill actions, or `{ action: "select", selector, values, args }` for `select`, with `args` redacted the same way as other invocation details. Expect it on the initial wrapper validation return (when that path still builds the early `details` object) and on the unified result after `agent-browser` runs. It is omitted when the call used `args` only, when compilation never produced argv, and on some in-`execute` error returns that attach a slimmer `details` shape before the unified merge (for example certain session-plan, stdin-contract, tab-pinning, or missing-binary guard paths); compare `extensions/agent-browser/index.ts` where `compiledSemanticAction` is assigned. For active sessions, role/name `click`, `check`, and guarded `fill` semantic actions may be resolved through one fresh `snapshot -i` to a current visible `@ref` before execution; fill only resolves when one exact editable `combobox`, `searchbox`, or `textbox` ref matches. This avoids hidden duplicate matches stealing an upstream `find` action. In that case `details.compiledSemanticAction` still records the original semantic target while `details.effectiveArgs` shows the executed ref action.
250
+ When `semanticAction` compiles successfully, `details.compiledSemanticAction` echoes `{ action, locator, args }` for `find` actions, `{ action, selector, args }` for direct selector/ref click/check/fill actions, or `{ action: "select", selector, values, args }` for `select`, with `args` redacted the same way as other invocation details. Expect it on the initial wrapper validation return (when that path still builds the early `details` object) and on the unified result after `agent-browser` runs. It is omitted when the call used `args` only, when compilation never produced argv, and on some in-`execute` error returns that attach a slimmer `details` shape before the unified merge (for example certain session-plan, stdin-contract, tab-pinning, or missing-binary guard paths); compare `extensions/agent-browser/index.ts` where `compiledSemanticAction` is assigned. For active sessions, role/name `click`, `check`, and guarded `fill` semantic actions may be resolved through one fresh `snapshot -i` to a current visible `@ref` before execution; for a caller-owned explicit `semanticAction.session`, the wrapper first verifies the live URL, and the same process-local per-session critical section covers that probe, snapshot, and main action; fill only resolves when one exact editable `combobox`, `searchbox`, or `textbox` ref matches. This avoids hidden duplicate matches stealing an upstream `find` action. In that case `details.compiledSemanticAction` still records the original semantic target while `details.effectiveArgs` shows the executed ref action.
250
251
 
251
252
  If a raw `find` or compiled `semanticAction` fails with `failureCategory: "selector-not-found"`, the wrapper may run one fresh session-scoped `snapshot -i` and add visible `Current snapshot ref fallback` plus `details.visibleRefFallback` when that snapshot contains exact role/name matches for the failed target. Non-fill matches can also add `try-current-visible-ref` / `try-current-visible-ref-N` next actions. The matcher is bounded to current snapshot refs and exact normalized role/name matches: role locators require `--name`, text-click falls back only to exact-name `button`/`link` refs, label-fill to exact-name `textbox`, and placeholder-fill to exact-name `searchbox`/`textbox`. It never fuzzy-matches names such as prefixes; when several exact refs match, each action carries safety copy telling agents to inspect the snapshot and choose only if unambiguous. For post-failure `fill` matches, `visibleRefFallback.candidates[].args` and `visibleRefFallback.target.text` are omitted so recovery details do not repeat the fill text.
252
253
 
@@ -277,7 +278,7 @@ Examples:
277
278
  - optional; mutually exclusive with `args`, `semanticAction`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron`
278
279
  - top-level tool input only; do not nest `job` inside `batch` stdin
279
280
  - constrained orchestration only: every step compiles to existing upstream `batch` argv and the compiled plan is echoed as `details.compiledJob`
280
- - optional `failFast` boolean; defaults to `true`, compiling to upstream `batch --bail` so later mutating job steps do not run after an earlier required step fails. Set `failFast: false` only when you explicitly want upstream batch's continue-after-error behavior.
281
+ - optional `failFast` boolean; defaults to `true`, compiling to upstream `batch --bail` so later mutating job steps do not run after an earlier required step fails. Set `failFast: false` only when you explicitly want upstream batch's continue-after-error behavior and every later step remains safe if an earlier navigation fails; otherwise the wrapper may require fail-fast behavior or split calls.
281
282
  - there is no separate reusable named “browser recipe” extension surface above `job`, `qa`, and raw `batch` yet; the closed `RQ-0068` decision, evidence bar, and revisit criteria are in [`ARCHITECTURE.md`](ARCHITECTURE.md#no-reusable-recipe-layer-yet) and [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
282
283
  - supported steps (`open.loadState` can emit an extra readiness row immediately after the `open`; otherwise each row becomes one upstream `batch` step except paced `type`, which expands to existing focus/keyboard/wait/press rows; `click` / `fill` accept either `selector` or semantic locator fields, while `type` / `select` pass `selector` through as the same argv token shape standalone upstream commands would use, including `@refs`):
283
284
  - `open` with `url`; optional `loadState` (`domcontentloaded`, `load`, or `networkidle`) inserts `wait --load <state>` immediately after the open when the next step needs page-readiness evidence
@@ -338,7 +339,7 @@ On app pages that expose a native dropdown, add a `select` step such as `{ "acti
338
339
 
339
340
  `assertUrl` compiles to upstream `wait --url <url-or-pattern>` for exact URLs and glob-style patterns, preserving query strings and literal `?` in exact URLs and delegating `*` / `**` matching to the current upstream matcher.
340
341
 
341
- Use raw `args` plus `stdin` for upstream `batch` when a flow needs commands, flags, stdin forms, or failure policies outside this constrained schema. `job.failFast: false` keeps the constrained schema but removes `--bail` when collecting later diagnostic artifacts is more important than stopping on the first failure.
342
+ Use raw `args` plus `stdin` for upstream `batch` when a flow needs commands, flags, stdin forms, or failure policies outside this constrained schema. `job.failFast: false` keeps the constrained schema but removes `--bail` for continue-after-error diagnostics. If a later content step assumes navigation succeeded, keep the default fail-fast behavior or split the flow: the wrapper rejects non-bail batches when any failed transition could leave a local or unverified page active.
342
343
 
343
344
  Because `job` still executes as upstream `batch` with generated stdin, the same wrapper page-scoped `@e…` preflight applies: if you pass `@refs` in `click`/`fill`/`select` selectors after an `open`, non-form `click`, or another step that can navigate or mutate the page, split the work across tool calls or switch to raw `batch` and insert your own `snapshot -i` rows between steps—the constrained `job` vocabulary emits a `snapshot` step only when you include `{ action: "snapshot" }` explicitly. Multiple same-snapshot `fill @e…` rows may run before the first click/submit-style step. Raw `args:["batch"]` stdin can also batch native form-control rows (`check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, and `select` combobox refs) before that click.
344
345
 
@@ -379,21 +380,21 @@ Action schemas:
379
380
  | Action | Fields | Behavior |
380
381
  | --- | --- | --- |
381
382
  | `list` | `query?`, `maxResults?` | Scans supported platform app locations for Electron evidence and returns bounded app metadata in `details.electron.apps`. Likely-sensitive app annotations are advisory metadata only. Does not spawn upstream `agent-browser`; list output also warns that later wrapper launches are isolated and will not read existing signed-in desktop state. |
382
- | `launch` | exactly one of `appPath`, `appName`, `bundleId`, or `executablePath`; optional `appArgs`, `handoff`, `targetType`, `timeoutMs`, `allow`, `deny` | Resolves and verifies an Electron target, launches it with a wrapper-owned isolated profile and OS-chosen CDP port, attaches through upstream `connect` using `sessionMode: "fresh"`, and records `details.electron.launch`. It does not reuse the app's normal signed-in profile or attach to an already-running authenticated app; when signed-in local app state is the goal, use a host debug-port launch plus raw `connect` instead. |
383
+ | `launch` | exactly one of `appPath`, `appName`, `bundleId`, or `executablePath`; optional `appArgs`, `handoff`, `targetType`, `timeoutMs`, `allow`, `deny` | Resolves and verifies an Electron target, launches it with a wrapper-owned isolated profile and OS-chosen CDP port, attaches through upstream `connect` using `sessionMode: "fresh"`, and records `details.electron.launch`. It does not reuse the app's normal signed-in profile or attach to an already-running authenticated app; when signed-in local app state is the goal, use a host debug-port launch plus raw `connect` instead. Pi cancellation before spawn returns `failureCategory: "aborted"` without opening the app; cancellation during readiness polling or post-attach handoff closes the managed session, stops the process, and removes its isolated profile. If any wrapper validation, policy, or live-URL handoff check fails after launch, the wrapper immediately cleans the new process/profile and retains a partial record only when cleanup could not finish. |
383
384
  | `status` | optional `launchId` or `all`, optional `timeoutMs` | Inspects wrapper-tracked launches, debug-port liveness, and current CDP targets without mutating the app. With neither `launchId` nor `all`, selects the single active wrapper launch when unambiguous. Runtime-owned launches remain visible by `launchId` after a Pi `session_tree` branch switch. Current branch-visible launches survive `/reload`; off-branch owned launches are cleaned on reload. |
384
385
  | `cleanup` | optional `launchId` or `all`, optional `timeoutMs` | Closes the tracked upstream session when present, stops only the wrapper-tracked process, verifies debug-port shutdown, removes the wrapper-created `userDataDir`, and marks records cleaned or partial. Cleanup is serialized with managed-session browser work, and a successful managed-session close step clears live/restore managed-session state even when host process/profile cleanup remains partial. |
385
- | `probe` | optional `launchId`, optional `timeoutMs` | Runs bounded current-session or launch-scoped state reads (`get title`, `get url`, focused-element `eval --stdin`, `tab list`, compact `snapshot -i`) and reports `details.electron.probe`. Without `launchId`, requires an active attached managed session; with `launchId`, it resolves the tracked launch session, including runtime-owned off-branch launch records, and can report mismatch guidance. |
386
+ | `probe` | optional `launchId`, optional `timeoutMs` | Runs bounded current-session or launch-scoped state reads (`get url`, then `get title`, focused-element `eval --stdin`, `tab list`, compact `snapshot -i`) and reports `details.electron.probe`; current-managed probes also persist top-level `details.namespace`, `sessionTabTarget`, and `refSnapshot` so reload/branch replay keeps the same namespaced page identity. Without `launchId`, requires an active attached managed session; with `launchId`, it resolves the tracked launch session, including runtime-owned off-branch launch records, and can report mismatch guidance. If every read fails, the tool fails as `upstream-error`; tracked local-file targets fail validation before probe helpers run, and the live URL is rechecked before any title/content helper. |
386
387
 
387
388
  Validation and defaults:
388
389
 
389
390
  - `launch` requires exactly one target field. `list` accepts only `query` and `maxResults`; `probe` accepts only `launchId` and `timeoutMs` beyond `action`; `status` / `cleanup` accept only `launchId`, `all`, and `timeoutMs`.
390
391
  - Host install discovery (`electron.list` and resolving `appName` / `bundleId` through `discoverElectronApps` in `extensions/agent-browser/lib/electron/discovery.ts`) runs on **macOS** and **Linux** only. On **Windows** (and any other platform), `list` returns `platform: "unsupported"` with an empty `apps` array, and `launch` cannot resolve purely name-based targets without a prior scan—use `executablePath` or a host `appPath` that resolves to a verifiable Electron binary (`inspectElectronExecutablePath` still gates Windows executables).
391
- - `launch.handoff` defaults to `"snapshot"`; supported values are `"connect"`, `"tabs"`, and `"snapshot"`. `"connect"` stops after attach, `"tabs"` adds a session-scoped `tab list` and is the safer diagnostic starting point when you do not want to capture refs/content yet, and `"snapshot"` adds `tab list` plus `snapshot -i` so current refs are immediately available. If the first Electron snapshot returns no refs while the app is still settling, the wrapper retries briefly before reporting no refs and tells agents to run `snapshot -i` once more before treating the UI as unusable.
392
+ - `launch.handoff` defaults to `"snapshot"`; supported values are `"connect"`, `"tabs"`, and `"snapshot"`. `"connect"` stops after attach, `"tabs"` verifies the live URL before a session-scoped `tab list` and is the safer diagnostic starting point when you do not want to capture refs/content yet, and `"snapshot"` verifies the live URL before `tab list` plus `snapshot -i` so current refs are immediately available. If the first Electron snapshot returns no refs while the app is still settling, the wrapper retries briefly before reporting no refs and tells agents to run `snapshot -i` once more before treating the UI as unusable.
392
393
  - `launch.targetType` defaults to `"page"`; supported values are `"page"`, `"webview"`, and `"any"`. When a matching CDP target exposes a WebSocket URL, launch connects to that target; otherwise it falls back to the browser port.
393
394
  - `appArgs` are passed to the Electron app, but wrapper-owned lifecycle/debug flags are rejected (`--user-data-dir`, `--remote-debugging-port`, `--remote-debugging-address`, `--remote-debugging-pipe`, and `--`).
394
395
  - `allow` and `deny` are optional caller-owned policy lists. Entries match app name, bundle id, desktop id, app path, or executable path by substring. If `allow` is set, the target must match it; `deny` wins on conflict. With neither list, launch is permitted.
395
396
  - `electron.status` / `electron.cleanup` accept optional `all` only as the boolean literal `true` to include every wrapper-tracked launch; `all` and `launchId` cannot both be set. Status and cleanup use the same runtime wrapper-tracked scope: current branch-visible records plus still-owned off-branch records. Default no-argument status/cleanup is intentionally ambiguous when more than one active launch is in that merged scope; pass `launchId` or `all: true`.
396
- - `electron.launch` `timeoutMs` is an optional positive integer for the host CDP readiness window. The launcher normalizes missing or non-positive values to **15000 ms** and **caps** at **120000 ms** (`normalizeTimeoutMs` in `extensions/agent-browser/lib/electron/launch.ts`). Other actions use `timeoutMs` differently: **`status`** forwards it only to managed-session `get title` / `get url` reads for mismatch diagnostics (when omitted, those subprocess reads use the normal `runAgentBrowserProcess` default from `getAgentBrowserProcessTimeoutMs`), while localhost CDP liveness in `inspectElectronLaunchStatus` uses a fixed **1000 ms** fetch budget per HTTP probe (`ELECTRON_STATUS_FETCH_TIMEOUT_MS` in `extensions/agent-browser/lib/electron/cleanup.ts`). **`cleanup`** applies one budget to upstream managed-session `close` plus host process exit, debug-port verification, and temp profile removal (`cleanupTrackedElectronHostLaunches` in `extensions/agent-browser/lib/orchestration/electron-host/index.ts`); when omitted it defaults to **`PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS`** else **5000 ms** (`getImplicitSessionCloseTimeoutMs` in `extensions/agent-browser/lib/runtime.ts`). **`probe`** forwards it to each bounded upstream read (`get title`, `get url`, `eval --stdin`, `tab list`, `snapshot -i`); when omitted, those reads use the same `runAgentBrowserProcess` default as other tool calls.
397
+ - `electron.launch` `timeoutMs` is an optional positive integer for the host CDP readiness window. The launcher normalizes missing or non-positive values to **15000 ms** and **caps** at **120000 ms** (`normalizeTimeoutMs` in `extensions/agent-browser/lib/electron/launch.ts`). Other actions use `timeoutMs` differently: **`status`** forwards it only to managed-session `get url`, then `get title` reads for mismatch diagnostics (when omitted, those subprocess reads use the normal `runAgentBrowserProcess` default from `getAgentBrowserProcessTimeoutMs`), while localhost CDP liveness in `inspectElectronLaunchStatus` uses a fixed **1000 ms** fetch budget per HTTP probe (`ELECTRON_STATUS_FETCH_TIMEOUT_MS` in `extensions/agent-browser/lib/electron/cleanup.ts`). **`cleanup`** applies one budget to upstream managed-session `close` plus host process exit, debug-port verification, and temp profile removal (`cleanupTrackedElectronHostLaunches` in `extensions/agent-browser/lib/orchestration/electron-host/index.ts`); when omitted it defaults to **`PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS`** else **5000 ms** (`getImplicitSessionCloseTimeoutMs` in `extensions/agent-browser/lib/runtime.ts`). **`probe`** forwards it to each bounded upstream read (`get url`, `get title`, `eval --stdin`, `tab list`, `snapshot -i`); when omitted, those reads use the same `runAgentBrowserProcess` default as other tool calls.
397
398
  - Non-Electron targets are rejected as a correctness failure; the wrapper does not blindly launch arbitrary executables as Electron.
398
399
 
399
400
  Safety defaults and ownership:
@@ -612,7 +613,7 @@ Behavior:
612
613
  - `"fresh"` rotates that managed session to a fresh upstream launch so startup-scoped flags like `--profile`, `--executable-path`, `--webgpu`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--idle-timeout`, `--enable`, `-p` / `--provider`, or iOS `--device` apply and later default calls follow the new browser; `--idle-timeout` must equal the Pi process's configured managed idle timeout or the wrapper rejects it with restart guidance
613
614
  - upstream `--webgpu` is a launch-scoped optional boolean: both enabled and explicit `false` values require a fresh managed launch once an implicit session exists; enabled WebGPU is local-launch-only and upstream rejects combinations with CDP, auto-connect, or providers
614
615
  - upstream restore sessions may periodically save cookies/localStorage while open; `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` passes through unchanged (`30000` default, `0` disables periodic saves but keeps save-on-close), and those restore files remain upstream-owned rather than wrapper artifacts
615
- - sessionless paths skip that injection even under `"auto"`: plain-text `--help` / `-h` / `--version` / `-V` (see the generated inspection playbook fragment below), read-only `skills list`, `skills get …`, and `skills path …`, local auth profile management (`auth save/list/show/delete/remove`), local/setup commands (`profiles`, `dashboard start/stop`, `device list`, `doctor`, `install`, `upgrade`, `session id`, `session info`, `session list`, `plugin add/list/show/run`, `mcp --help`), and targeted/all local saved-state maintenance (`state list/show`, `state clear --all`, `state clear -a`, `state clear <session-name>`, `state clean --older-than <days>`, `state rename`) keep `effectiveArgs` free of the implicit managed `--session` unless the caller supplied `--session` explicitly; successful results therefore omit `usedImplicitSession` and the extension-managed `sessionName` for those calls, while bare `mcp` server calls are rejected before spawn and root `session`, untargeted `state clear`, bare `state clean`, browser-backed `auth login`, and `state save/load` keep normal managed-session injection (`extensions/agent-browser/lib/command-policy.ts`, `needsManagedSession`; `extensions/agent-browser/lib/runtime.ts`, `buildExecutionPlan`)
616
+ - sessionless paths skip that injection even under `"auto"`: plain-text `--help` / `-h` / `--version` / `-V` (see the generated inspection playbook fragment below), read-only `skills list`, `skills get …`, and `skills path …`, local auth profile management (`auth save/list/show/delete/remove`), local/setup commands (`profiles`, `dashboard start/stop`, `device list`, `doctor`, `install`, `upgrade`, `session id`, `session info`, `session list` (with wrapper-managed rows removed), `plugin add/list/show/run`, `mcp --help`), and caller-owned local saved-state maintenance (`state list/show`, targeted `state clear <caller-owned-name>`, and `state rename`) keep `effectiveArgs` free of the implicit managed `--session` unless the caller supplied `--session` explicitly; broad clear/clean and managed-state targets are rejected before spawn, while successful allowed results omit `usedImplicitSession` and the extension-managed `sessionName` for those calls, while bare `mcp` server calls are rejected before spawn and root `session`, untargeted `state clear`, bare `state clean`, browser-backed `auth login`, and `state save/load` keep normal managed-session injection (`extensions/agent-browser/lib/command-policy.ts`, `needsManagedSession`; `extensions/agent-browser/lib/runtime.ts`, `buildExecutionPlan`)
616
617
 
617
618
  Recommended use:
618
619
  - use `"auto"` for the common browse/snapshot/click flow inside one `pi` session
@@ -713,7 +714,7 @@ Ref preflight details (command taxonomy in `extensions/agent-browser/lib/command
713
714
 
714
715
  **Presentation redaction (implementation map):** Successful non-`batch` tool calls and each successful `batchSteps[]` row run upstream `data` through `redactPresentationData` in `extensions/agent-browser/lib/results/presentation/diagnostics.ts`: `cookies` still walk objects/arrays and replace case-insensitive `value` keys with `"[REDACTED]"`; `storage` redacts values when the key or value looks credential-like (token, cookie, auth, secret, JWT, bearer/basic credential, high-entropy token-like string, or nested sensitive JSON) but keeps low-risk primitive QA values such as booleans, numbers, and short strings visible. Redacted storage entries add `valueRedacted` plus `valueRedactionReason` in `details.data`; diagnostic formatters mirror the same decision. Every other command’s payload is recursively scrubbed with `redactStructuredPresentationValue`, which redacts known sensitive key names and applies string-level sensitivity heuristics so network, diff, trace/profiler, stream, dashboard, chat, and other structured results do not echo bearer tokens, proxy credentials, or similar fields verbatim into `details.data`. Echoed `command` arrays in `details` and in batch roll-ups use `redactInvocationArgs` from `extensions/agent-browser/lib/runtime.ts` to mask trailing values for sensitive global flags (including `--body`, `--headers`, `--password`, and `--proxy`), preserve the special positional rules for `cookies set`, `storage local|session set`, and `set credentials`, and scrub other argv tokens for URLs and inline secrets. Failed batch steps additionally run `redactExactValues` on structured step errors so literals taken from that step’s argv (cookie value, storage set value, `--password` / `--password=` tokens) cannot reappear inside formatted error blobs.
715
716
 
716
- `nextActions` is an optional machine-readable list of exact native `agent_browser` follow-ups. Each entry includes `tool: "agent_browser"`, an `id`, a short `reason`, optional `safety`, and either `params` (`args`, optional `stdin`, optional `sessionMode`, optional `networkSourceLookup`, optional `electron`) or an `artifactPath` for saved-file workflows. Agents should prefer these payloads over prose when present. When a result ran under upstream `--namespace <name>`, follow-up `params.args` preserve that namespace so retries and diagnostics target the same daemon/restore-state space. Tab/session recovery id strings are centralized in `AGENT_BROWSER_RECOVERY_NEXT_ACTION_IDS`, while rich-input focus/click recovery ids are centralized in `AGENT_BROWSER_RICH_INPUT_RECOVERY_NEXT_ACTION_IDS` plus `getAgentBrowserRichInputRecoveryNextActionId(s)` in `extensions/agent-browser/lib/results/recovery-actions.ts`; docs and tests mirror those registries/helpers rather than inventing recovery ids in prose. Current recommendations include: timed-out jobs/batches with a retryable read-only/idempotent first incomplete step → `retry-timeout-step`, while timed-out flows whose first incomplete step may be mutating → `inspect-current-page-after-timeout` (`snapshot -i`) before splitting the remaining work into shorter batches; raw `connect` success → session-scoped `list-connected-session-tabs` only, then the agent should inspect/select a stable `tab t<N>` target and run `snapshot -i` explicitly; `snapshot` failures whose upstream error says `No active page` and whose wrapper result has a known session → `list-tabs-after-no-active-page` only, because this path has no wrapper-observed safe tab id to select atomically; browser profile/user-data-dir resolution failures → `inspect-browser-profiles` (`profiles`) and `run-agent-browser-doctor` (`doctor`) before retrying opens; Electron launches → wrapper-tracked `electron.status` / `electron.probe` / `electron.cleanup` actions plus session-scoped tab/snapshot inspection when attached; Electron status/probe mismatch diagnostics → `reattach-electron-launch` plus fresh tab/snapshot inspection; Electron post-command health failures → status/probe/cleanup for the same `launchId`; Electron or contenteditable fill verification mismatches → `inspect-after-fill-verification` and `verify-filled-value`; Electron same-URL ref freshness warnings → `refresh-electron-refs-after-rerender`; packaged-Electron `sourceLookup` no-candidate diagnostics → session snapshot, launch probe, and tab list; Electron cleanup partial failures → status plus retry-cleanup for the same wrapper-owned `launchId`; `open` success → `snapshot -i`; mutating/navigation commands (see `buildAgentBrowserNextActions` in source for the exact command set) → `snapshot -i`; stale refs and selector failures → `snapshot -i` via `refresh-interactive-refs` (prefixed with `--session <name>` when the failed call ran in a named or managed session); selector misses with exact current snapshot role/name matches → direct ref retries via `try-current-visible-ref` or bounded `try-current-visible-ref-N` for non-fill targets; semantic `fill` selector misses with exact current editable refs → `focus-current-editable-ref` / `click-current-editable-ref` or numbered variants that do not include fill text or submit; unknown getter shortcuts such as `title` / `url` → exact read-only retries like `get title` / `get url` with ids `use-get-title` / `use-get-url`; compact `network requests` results with safe request IDs → bounded read-only request detail, `networkSourceLookup`, path filter, or HAR-capture follow-ups; semantic `selector-not-found` failures that compiled from `semanticAction` may append `try-button-name-candidate` or `try-link-name-candidate` after presentation `nextActions` only for the bounded click pair enumerated under `semanticAction`; semantic `stale-ref` failures that compiled from `semanticAction` `find` argv may also include `retry-semantic-action-after-stale-ref` after that snapshot step; successful snapshots or qualifying same-URL non-Electron top-level clicks (see `overlayBlockers` below) with snapshot evidence of likely overlay/banner/dialog close controls may append `inspect-overlay-state` and bounded `try-overlay-blocker-candidate-*` entries; successful top-level `scroll` calls whose pre/post viewport and sampled scroll-container positions do not change may append `inspect-after-noop-scroll` and `verify-noop-scroll-visually`; explicit combobox-targeted actions that focus a combobox without visible options may append `inspect-focused-combobox`, `try-open-combobox-with-arrow`, and `try-open-combobox-with-enter`; `get text <selector>` calls with hidden/multiple CSS matches may append `inspect-visible-text-candidates` with a read-only `eval --stdin` probe (each prefixed with `--session <name>` when `details.sessionName` is set, same `sessionPrefixArgs` rule as other session-scoped follow-ups); confirmations → exact `confirm <id>` and `deny <id>` choices; generic tab drift → `list-tabs-for-recovery` with `tab list` first, then select or confirm the stable target before running `snapshot -i`; about:blank or tab-drift recovery with a wrapper-known target → `list-tabs-for-about-blank-recovery` or `list-tabs-for-tab-drift-recovery`, plus `select-intended-tab-after-drift` and `snapshot-after-tab-recovery` when the wrapper already observed the stable `t<N>` tab id; `wait --text` assertion failures → `inspect-after-text-assertion-failure` with a read-only snapshot; download verification failures or missing successful download artifacts → `wait --download [path]`; saved artifacts → the artifact path to inspect/consume after checking `artifactVerification`/metadata; missing non-download artifacts → `verify-artifact-path` so agents do not trust an absent file. When nothing applies, the field is omitted.
717
+ `nextActions` is an optional machine-readable list of exact native `agent_browser` follow-ups. Each entry includes `tool: "agent_browser"`, an `id`, a short `reason`, optional `safety`, and either `params` (`args`, optional `stdin`, optional `sessionMode`, optional `networkSourceLookup`, optional `electron`) or an `artifactPath` for saved-file workflows. Agents should prefer these payloads over prose when present. When a result ran under upstream `--namespace <name>`, follow-up `params.args` preserve that namespace so retries and diagnostics target the same daemon/restore-state space. Tab/session recovery id strings are centralized in `AGENT_BROWSER_RECOVERY_NEXT_ACTION_IDS`, while rich-input focus/click recovery ids are centralized in `AGENT_BROWSER_RICH_INPUT_RECOVERY_NEXT_ACTION_IDS` plus `getAgentBrowserRichInputRecoveryNextActionId(s)` in `extensions/agent-browser/lib/results/recovery-actions.ts`; docs and tests mirror those registries/helpers rather than inventing recovery ids in prose. Current recommendations include: timed-out jobs/batches with a retryable read-only/idempotent first incomplete step → `retry-timeout-step`, while timed-out flows whose first incomplete step may be mutating → `inspect-current-page-after-timeout` (`snapshot -i`) before splitting the remaining work into shorter batches; raw `connect` success → session-scoped `verify-connected-session-url` (`get url`) plus `list-connected-session-tabs`; page-content reads remain blocked until a non-file target is verified, after which the agent can select/confirm a stable `tab t<N>`, verify it with `get url`, and run `snapshot -i`; `snapshot` failures whose upstream error says `No active page` and whose wrapper result has a known session → `list-tabs-after-no-active-page` only, because this path has no wrapper-observed safe tab id to select atomically; browser profile/user-data-dir resolution failures → `inspect-browser-profiles` (`profiles`) and `run-agent-browser-doctor` (`doctor`) before retrying opens; Electron launches → wrapper-tracked `electron.status` / `electron.probe` / `electron.cleanup` actions plus session-scoped tab/snapshot inspection when attached; Electron status/probe mismatch diagnostics → `reattach-electron-launch` plus fresh tab/snapshot inspection; Electron post-command health failures → status/probe/cleanup for the same `launchId`; Electron or contenteditable fill verification mismatches → `inspect-after-fill-verification` and `verify-filled-value`; Electron same-URL ref freshness warnings → `refresh-electron-refs-after-rerender`; packaged-Electron `sourceLookup` no-candidate diagnostics → session snapshot, launch probe, and tab list; Electron cleanup partial failures → status plus retry-cleanup for the same wrapper-owned `launchId`; `open` success → `snapshot -i`; mutating/navigation commands (see `buildAgentBrowserNextActions` in source for the exact command set) → `snapshot -i`; stale refs and selector failures → `snapshot -i` via `refresh-interactive-refs` (prefixed with `--session <name>` when the failed call ran in a named or managed session); selector misses with exact current snapshot role/name matches → direct ref retries via `try-current-visible-ref` or bounded `try-current-visible-ref-N` for non-fill targets; semantic `fill` selector misses with exact current editable refs → `focus-current-editable-ref` / `click-current-editable-ref` or numbered variants that do not include fill text or submit; unknown getter shortcuts such as `title` / `url` → exact read-only retries like `get title` / `get url` with ids `use-get-title` / `use-get-url`; compact `network requests` results with safe request IDs → bounded read-only request detail, `networkSourceLookup`, path filter, or HAR-capture follow-ups; semantic `selector-not-found` failures that compiled from `semanticAction` may append `try-button-name-candidate` or `try-link-name-candidate` after presentation `nextActions` only for the bounded click pair enumerated under `semanticAction`; semantic `stale-ref` failures that compiled from `semanticAction` `find` argv may also include `retry-semantic-action-after-stale-ref` after that snapshot step; successful snapshots or qualifying same-URL non-Electron top-level clicks (see `overlayBlockers` below) with snapshot evidence of likely overlay/banner/dialog close controls may append `inspect-overlay-state` and bounded `try-overlay-blocker-candidate-*` entries; successful top-level `scroll` calls whose pre/post viewport and sampled scroll-container positions do not change may append `inspect-after-noop-scroll` and `verify-noop-scroll-visually`; explicit combobox-targeted actions that focus a combobox without visible options may append `inspect-focused-combobox`, `try-open-combobox-with-arrow`, and `try-open-combobox-with-enter`; `get text <selector>` calls with hidden/multiple CSS matches may append `inspect-visible-text-candidates` with a read-only `eval --stdin` probe (each prefixed with `--session <name>` when `details.sessionName` is set, same `sessionPrefixArgs` rule as other session-scoped follow-ups); confirmations → exact `confirm <id>` and `deny <id>` choices; generic tab drift → `list-tabs-for-recovery` with `tab list` first, then select or confirm the stable target before running `snapshot -i`; about:blank or tab-drift recovery with a wrapper-known target → `list-tabs-for-about-blank-recovery` or `list-tabs-for-tab-drift-recovery`, plus `select-intended-tab-after-drift` and `snapshot-after-tab-recovery` when the wrapper already observed the stable `t<N>` tab id; `wait --text` assertion failures → `inspect-after-text-assertion-failure` with a read-only snapshot; download verification failures or missing successful download artifacts → `wait --download [path]`; saved artifacts → the artifact path to inspect/consume after checking `artifactVerification`/metadata; missing non-download artifacts → `verify-artifact-path` so agents do not trust an absent file. When nothing applies, the field is omitted.
717
718
 
718
719
  **Unknown-command getter hints (failure presentation):** `buildErrorPresentation` in `extensions/agent-browser/lib/results/presentation/errors.ts` only runs this path when upstream error text (after model-facing redaction) matches `unknown command`, `unknown subcommand`, or `unrecognized command` (case-insensitive) **and** the failed invocation’s primary command token is one of `attr`, `count`, `html`, `text`, `title`, `url`, or `value`. Visible text then includes a grouped-`get` hint line plus per-token guidance (`get text <selector>`, `get html …`, `get attr …`, `get count …`, `get value …`, `get title`, `get url`). Machine `nextActions` with ids `use-get-title` / `use-get-url` are emitted only for `title` / `url`, with `params.args` optionally prefixed by `--session <name>` when the failed call targeted a named session. If the error string already contains `Agent-browser hint:` from selector recovery (stale-ref or unsupported selector dialect appendages), the getter block is skipped so two stacked `Agent-browser hint:` headers are not emitted.
719
720
 
@@ -780,10 +781,11 @@ Implementation and precedence:
780
781
  - Shared machine-readable types are centralized in `extensions/agent-browser/lib/results/contracts.ts` (including re-exports such as `AgentBrowserNextAction` from `next-actions.ts`). Classifiers live in `categories.ts` (`classifyAgentBrowserSuccessCategory`, `classifyAgentBrowserFailureCategory`, `buildAgentBrowserResultCategoryDetails`—the last prefers an explicit `failureCategory` when the caller already knows the bucket, otherwise it runs the classifier). Generic follow-up assembly lives in `action-recommendations.ts` (`buildAgentBrowserNextActions`). Tab/session recovery ids live in `recovery-actions.ts` (`AGENT_BROWSER_RECOVERY_NEXT_ACTION_IDS`, `AGENT_BROWSER_RICH_INPUT_RECOVERY_NEXT_ACTION_IDS`, `getAgentBrowserRichInputRecoveryNextActionId`, `getAgentBrowserRichInputRecoveryNextActionIds`, `buildRecoveryNextActions`) and session-aware wrappers live in `recovery-next-actions.ts`. Selector miss and rich-input diagnostic shapes/actions live in `selector-recovery.ts`. Failed upstream `network requests` rows flow through `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `network.ts` for QA analysis (`analyzeQaPresetResults` in `extensions/agent-browser/index.ts`) and for actionable-vs-benign lines plus request-specific nextActions in `network requests` presentation (`extensions/agent-browser/lib/results/presentation/diagnostics.ts`).
781
782
  - Artifact verification: `ArtifactVerificationSummary` / `ArtifactVerificationEntry` types live in `contracts.ts`. `buildArtifactVerificationSummary`, `getArtifactVerificationEntry`, and `getManifestVerificationEntry` in `presentation/artifacts.ts` merge each resolved file artifact with manifest rows whose `storageScope` is not `explicit-path` (those rows duplicate file artifacts) and whose `path` is in the current result’s spill path set. Presentation then fails closed with `failureCategory: "artifact-missing"` when any resolved non-pending file artifact has `exists: false`; pending recording-start artifacts and non-explicit spill rows do not trigger this failure. If no hard-missing file artifact exists, successful presentation merges run `classifyPresentationSuccessCategory`, which can still force `successCategory: "artifact-unverified"` for unverified counts before delegating to `classifyAgentBrowserSuccessCategory`.
782
783
  - Inner success categories (`classifyAgentBrowserSuccessCategory` in `categories.ts`, after verification counts are clear): if `inspection` is true → `"inspection"`; else if any non-pending artifact lacks confirmed on-disk presence (`exists !== true`) and was not upgraded to an `artifact-missing` failure → `"artifact-unverified"`; else if there is a `savedFile` or any `artifacts` → `"artifact-saved"`; else → `"completed"`.
783
- - Failure: the classifier walks a single ordered chain (first match wins): `confirmation-required` → `timeout` → `missing-binary` → `parse-failure` → `aborted` → `policy-blocked` → `cleanup-failed` → `tab-drift` → `stale-ref` (including “unknown ref” text and a narrow `@eN` plus “element not found” heuristic) → `selector-unsupported` → `selector-not-found` → `download-not-verified` (download / wait-download style failures) → `validation-error` when a wrapper `validationError` is present default `upstream-error`. Wrapper-known missing artifact checks pass an explicit `artifact-missing` category rather than relying on this text classifier.
784
+ - Failure: the classifier walks a single ordered chain (first match wins): explicit `options.confirmationRequired` → upstream locator-detail misses (`selector-not-found`, including 0.32.4+ `Names seen:` / `No element found: getByRole(...)` / `Element not found: … Verify the selector, role, or name`) → text-derived `confirmation-required` → `timeout` → `missing-binary` → `parse-failure` → `aborted` → `policy-blocked` → `cleanup-failed` → explicit `options.validationError` → `tab-drift` → `stale-ref` (including “unknown ref” text and a narrow `@eN` plus “element not found” heuristic) → `selector-unsupported` → other `selector-not-found` shapes → `download-not-verified` (download / wait-download style failures) → default `upstream-error`. Locator-detail misses are classified before text-derived confirmation/timeout so an accessible name containing those phrases cannot suppress selector recovery. Wrapper-known missing artifact checks pass an explicit `artifact-missing` category rather than relying on this text classifier.
784
785
  - The main tool implementation merges these fields into Pi-facing `details` from `extensions/agent-browser/index.ts` and from `extensions/agent-browser/lib/results/presentation.ts` for presentation-time failures.
785
786
 
786
787
  Additional structured fields can appear when relevant:
788
+ - `sessionTabTargetUnknown: true` after a spawned `connect`, `state load`, history navigation, or tab-selection/close call changes the active page without a trustworthy observed target. The marker is persisted and restored across branch/reload replay, clears stale refs and tab pinning, and blocks page inspection until `get url` or an explicit safe navigation observes a target; `tab list` and close remain available.
787
789
  - `compiledSemanticAction` when the call used `semanticAction` and the result includes the unified `details` merge: `{ action, locator, args }` for `find` actions or `{ action: "select", selector, values, args }` for `select`, with the same redaction rules as `args` / `effectiveArgs`; omitted for plain `args`/`job` calls and omitted on some early error returns that omit this field (see the `semanticAction` section above)
788
790
  - `compiledJob` when the call used `job` or the job-backed `qa` preset: by default `{ args: ["batch", "--bail"], failFast: true, stdin, steps: [{ action, args }] }`; with `failFast: false`, `{ args: ["batch"], failFast: false, stdin, steps: [{ action, args }] }`. Step args are redacted the same way as other invocation details. Semantic `job` click/fill steps appear here as their compiled upstream `find … click|fill …` argv, not as the input object.
789
791
  - `compiledQaPreset` when the call used `qa`: the compiled job fields plus the QA `checks` object. `args` is `batch --bail` and `failFast` is `true` for QA presets. `checks.attached` is `true` for current-session QA, `checks.url` is present only for URL-opening QA, and `checks.diagnosticsResetAtStart` is `true` only for URL-opening QA because `qa.attached` preserves existing session diagnostics.
@@ -795,7 +797,7 @@ Additional structured fields can appear when relevant:
795
797
  - `compiledElectron` when the call used `electron`: redacted action plan for `list`, `launch`, `status`, `cleanup`, or `probe`.
796
798
  - `electron` when the call used `electron`: action-specific lifecycle, discovery, probe, and cleanup data; see the `electron` section below.
797
799
  - `batchFailure` and `batchSteps` for `batch` rendering, including mixed-success runs
798
- - `navigationSummary` for navigation-style commands like ref-backed or href-bearing `click`, `back`, `forward`, and `reload`; helper probes use `get url` / `get title`. CSS selector clicks without upstream href/navigation fields skip this helper to avoid perturbing click dispatch; verify important CSS-click state with an explicit follow-up read.
800
+ - `navigationSummary` for navigation-style commands like ref-backed or href-bearing `click`, `back`, `forward`, and `reload`; helper probes run `get url` first and run `get title` only after a non-file URL is verified. A local-file result stops before title and reclassifies an implicit transition as a validation failure. CSS selector clicks without upstream href/navigation fields skip this helper to avoid perturbing click dispatch; verify important CSS-click state with an explicit follow-up read.
799
801
  - `pageChangeSummary` for compact mutation/artifact/navigation summaries on commands that can change browser state
800
802
  - `clickDispatch` when a top-level non-Electron direct `click` reported upstream success but the target-specific probe found no trusted event reached the resolved target; shape follows `ClickDispatchDiagnostic` in `extensions/agent-browser/lib/orchestration/browser-run/types.ts`
801
803
  - `promptGuard` when the requested-artifact-before-close guard blocks browser close before required prompt artifact paths are verified; implementation lives in `extensions/agent-browser/lib/orchestration/browser-run/prompt-guards.ts`
@@ -812,17 +814,17 @@ Additional structured fields can appear when relevant:
812
814
  - `comboboxFocus` after a successful explicit combobox-targeted `click` / `fill` / `find … click|fill` (for example `semanticAction` with role `combobox`, including when that semantic action resolves through a current visible `@ref` before execution) when a read-only probe sees the active element is combobox-like, `aria-expanded` is explicitly present (`false` or `true`), and no visible `listbox` / `option` / menu option elements are open. Shape: `{ reason: "focused-combobox-without-visible-options", message, activeElement, visibleListboxCount, visibleOptionCount, recommendations }`; `activeElement` includes bounded role/tag/expanded/hasPopup/name metadata with normal text redaction. Visible text appends `Combobox diagnostic: focused combobox did not expose visible options`, and `details.nextActions` gains `inspect-focused-combobox` (`snapshot -i`), `try-open-combobox-with-arrow` (`press ArrowDown`), and `try-open-combobox-with-enter` (`press Enter`), session-prefixed when applicable. The diagnostic is deliberately gated to explicit combobox-targeted calls to avoid extra probes or false positives on ordinary clicks/textboxes.
813
815
  - `recordingDependencyWarning` after a successful `record start` or `record restart` when the wrapper cannot find an executable `ffmpeg` on the Pi process `PATH`. Shape: `{ reason: "ffmpeg-missing-for-recording", dependency: "ffmpeg", command, message, recommendations }`. Visible text appends `Recording dependency warning: ffmpeg not found on PATH`. This is a non-blocking preflight warning: upstream may start recording, but `record stop` needs `ffmpeg` to encode the WebM.
814
816
  - `selectorTextVisibility` after a **successful** upstream `get text <selector>` (standalone or inside a successful `batch`) when the wrapper’s follow-up probe finds a hazard: more than one DOM match (upstream reads the first `querySelectorAll` hit, which may be the wrong tab/panel), or the first match is hidden while at least one other match is visible (requires multiple DOM nodes so a visible peer exists; a lone hidden match is not flagged). The probe is a read-only `eval --stdin` script (`buildVisibleTextProbeScript` in `extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`) that counts matches, applies a small visibility heuristic (`display`/`visibility`/`opacity` plus non-zero client rects), may include a redacted `firstVisibleTextPreview`, and may include up to eight `visibleCandidates` entries (`index` in `querySelectorAll`, `tagName`, optional `role`, optional redacted `textPreview`). It is **not** run for simple id selectors, page-scoped `@e…` selectors, or when the selector string is withheld because `selectorMayExposeSensitiveLiteral` would risk echoing secrets in probe output. `details.selectorTextVisibility` mirrors the primary diagnostic (first sorted entry); when several selectors in one `batch` qualify, `selectorTextVisibilityAll` lists every diagnostic sorted so hidden-first cases precede generic multi-match ambiguity. Appended visible warning text names the matching `details.nextActions` id and may list visible candidate previews. Appended `details.nextActions` use ids `inspect-visible-text-candidates` and `inspect-visible-text-candidates-2`, … with the probe replayed via `eval --stdin` for each hazardous selector. If the probe still leaves more than one visible candidate, it is only ambiguity evidence; agents should narrow the selector, use a current visible `@ref`, or run a targeted visible-element `eval --stdin` rather than trusting the broad selector.
815
- - `electronGetTextScopeWarning` after a successful wrapper-tracked attached Electron `get text <selector>` (standalone or successful `batch`) when a broad non-ref CSS selector such as `body`, `html`, `main`, `div`, or `[role=application]` may read the whole app shell. Ordinary browser pages, including `file://` fixtures, do not qualify without wrapper-owned Electron launch provenance. Shape: `{ selector, summary, electronContext: { launchId?, sessionName?, url? } }`; multiple batched diagnostics use `electronGetTextScopeWarnings`. Visible text appends `Broad Electron get text selector warning`, and next actions use `snapshot-for-electron-text-scope` ids with session-scoped `snapshot -i` payloads.
817
+ - `electronGetTextScopeWarning` after a successful wrapper-tracked attached Electron `get text <selector>` (standalone or successful `batch`) when a broad non-ref CSS selector such as `body`, `html`, `main`, `div`, or `[role=application]` may read the whole app shell. Ordinary browser pages do not qualify without wrapper-owned Electron launch provenance, and local `file://` page follow-ups fail validation before this diagnostic. Shape: `{ selector, summary, electronContext: { launchId?, sessionName?, url? } }`; multiple batched diagnostics use `electronGetTextScopeWarnings`. Visible text appends `Broad Electron get text selector warning`, and next actions use `snapshot-for-electron-text-scope` ids with session-scoped `snapshot -i` payloads.
816
818
  - `evalStdinHint` after a successful `eval --stdin` when caller stdin (trimmed) looks function-shaped to the wrapper’s lightweight detector (in `extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`: leading `function` / `async function`, parenthesized arrow `(…) =>`, or a concise `name =>` / `async name =>` form) **and** upstream JSON `data` is an object whose `result` field is a plain empty object (`{}`). Arrays such as `[]` do not qualify. It includes `reason` and `suggestion`; visible output appends `Eval stdin hint` with the same guidance. This is a heuristic for the common mistake of returning a function object instead of invoking it or passing a plain expression, not a JavaScript parser or proof that the page returned no useful data. Before this diagnostic path runs, the wrapper also recovers the common malformed native-tool call `args: ["eval", "--stdin", "..."]` with no top-level `stdin` by moving trailing `args` tokens after `--stdin` into the process stdin stream.
817
819
  - `evalResultWarning` after a successful `eval --stdin` when the current or prior page URL is `file:` (from navigation summary, session tab target, or persisted session page state), upstream JSON `data.result` is strictly `null`, and stdin is non-empty and not a trivial literal `null`/`undefined`. Fields: `reason`, `suggestion`. Visible output appends `Eval result warning` without failing the tool. Use snapshot -i, ref-based getters, screenshots, or http(s) fixtures when file:// null results are inconclusive.
818
- - `timeoutPartialProgress` after `runAgentBrowserProcess` reports `timedOut` (wrapper child-process watchdog) when best-effort recovery finds useful context. `summary` is a short sentence counting recovered planned-step state and declared artifact paths, plus whether page context came from live session reads or only from a planned URL (when nothing in the plan declares an artifact path, the fraction may read `0/0` while `currentPage` can still carry session or planned URL context). `steps` lists planned argv from the compiled `job` or `qa` batch plan (`compiledJob` in `extensions/agent-browser/index.ts`, which is only populated for those top-level modes) or, when that object is absent, from the same JSON-array `batch` stdin the tool sends upstream—whether caller-authored or wrapper-generated for `sourceLookup` / `networkSourceLookup` (1-based indices; only JSON-array stdin whose elements are string[] argv arrays is parsed). Generated rows such as `open.loadState` waits may include `generatedFrom`. Each step includes `status` (`completed`, `failed`, `pending`, or `unknown`) and optional `reason`; the first incomplete step becomes `retryStep`, but `retry.args` and top-level `retry-timeout-step` are emitted only for read-only or idempotent commands such as waits, snapshots, screenshots, navigation, and diagnostics. Mutating steps such as clicks, fills, keyboard typing, presses, selects, or checks are still identified as the first incomplete step but omit executable retry args because they may already have run; when the timed-out session is still usable, `details.nextActions` can instead include `inspect-current-page-after-timeout` (`snapshot -i`) so the agent verifies current state before continuing with a shorter split flow. When a retryable step timed out during `sessionMode: "fresh"` and no live URL was recovered, `retry-timeout-step` uses top-level `sessionMode: "fresh"` instead of prefixing the abandoned generated session name. `currentPage` comes from session-scoped `get url` / `get title` when the session answers, otherwise a fallback URL may be inferred from the last `open` / `navigate` / `pushstate` step in the plan; `liveUrlRecovered` is true only when the wrapper recovered a live URL, so planned URLs are not treated as proof that the page actually opened. `openedButPostOpenTimedOut` is set when a live opened page was recovered and a later step appears to have timed out. `artifacts` covers declared output paths on `screenshot`, `pdf`, `download`, and `wait --download` steps (absolute path, existence, `state`, optional `sizeBytes`, `stepIndex`). Visible text repeats the same block under `Timeout partial progress`, applying URL and path-segment redaction; the prose `Planned steps` list shows at most six steps, then an omitted-count line when the plan is longer. This is recovery evidence only; missing entries do not prove the upstream step never ran or that no other side effects occurred.
820
+ - `timeoutPartialProgress` after `runAgentBrowserProcess` reports `timedOut` (wrapper child-process watchdog) when best-effort recovery finds useful context. `summary` is a short sentence counting recovered planned-step state and declared artifact paths, plus whether page context came from live session reads or only from a planned URL (when nothing in the plan declares an artifact path, the fraction may read `0/0` while `currentPage` can still carry session or planned URL context). `steps` lists planned argv from the compiled `job` or `qa` batch plan (`compiledJob` in `extensions/agent-browser/index.ts`, which is only populated for those top-level modes) or, when that object is absent, from the same JSON-array `batch` stdin the tool sends upstream—whether caller-authored or wrapper-generated for `sourceLookup` / `networkSourceLookup` (1-based indices; only JSON-array stdin whose elements are string[] argv arrays is parsed). Generated rows such as `open.loadState` waits may include `generatedFrom`. Each step includes `status` (`completed`, `failed`, `pending`, or `unknown`) and optional `reason`; the first incomplete step becomes `retryStep`, but `retry.args` and top-level `retry-timeout-step` are emitted only for read-only or idempotent commands such as waits, snapshots, screenshots, navigation, and diagnostics. Mutating steps such as clicks, fills, keyboard typing, presses, selects, or checks are still identified as the first incomplete step but omit executable retry args because they may already have run; when the timed-out session is still usable, `details.nextActions` can instead include `inspect-current-page-after-timeout` (`snapshot -i`) so the agent verifies current state before continuing with a shorter split flow. When a retryable step timed out during `sessionMode: "fresh"` and no live URL was recovered, `retry-timeout-step` uses top-level `sessionMode: "fresh"` instead of prefixing the abandoned generated session name. `currentPage` comes from session-scoped `get url` followed by `get title` only for a verified non-file URL when the session answers, otherwise a fallback URL may be inferred from the last `open` / `navigate` / `pushstate` step in the plan; `liveUrlRecovered` is true only when the wrapper recovered a live URL, so planned URLs are not treated as proof that the page actually opened. `openedButPostOpenTimedOut` is set when a live opened page was recovered and a later step appears to have timed out. `artifacts` covers declared output paths on `screenshot`, `pdf`, `download`, and `wait --download` steps (absolute path, existence, `state`, optional `sizeBytes`, `stepIndex`). Visible text repeats the same block under `Timeout partial progress`, applying URL and path-segment redaction; the prose `Planned steps` list shows at most six steps, then an omitted-count line when the plan is longer. This is recovery evidence only; missing entries do not prove the upstream step never ran or that no other side effects occurred.
819
821
  - `managedSessionOutcome` after a managed-session plan reaches process execution (`buildManagedSessionOutcome` / `formatManagedSessionOutcomeText` in `extensions/agent-browser/lib/orchestration/browser-run/session-state.ts`). Populated when `buildExecutionPlan` injects an extension-managed implicit or fresh `--session`, and also when a successful explicit `--session <current-wrapper-managed-session> close` closes the current managed session. It remains omitted for unrelated explicit user-managed sessions and for sessionless inspection/local paths that skip injection. Fields: `status` (`created`, `replaced`, `unchanged`, `closed`, `preserved`, or `abandoned`), `sessionMode`, `attemptedSessionName`, `previousSessionName`, `currentSessionName`, optional `currentSessionNamespace`, optional `replacedSessionName`, optional `replacedSessionNamespace`, `activeBefore`, `activeAfter`, `succeeded`, and `summary` (machine-oriented; may include generated session names). Use `currentSessionNamespace` with `currentSessionName` when following preserved-session recovery actions; retry-fresh actions stay in the attempted namespace. Model-visible echo: only when `sessionMode` is `"fresh"` **and** `succeeded` is false, the wrapper appends action-oriented `Managed session outcome` and `Recovery` lines without repeating generated session ids in visible prose; session names remain in `details.managedSessionOutcome`. Failed fresh launches may also append `details.nextActions` such as `run-agent-browser-doctor`, `verify-current-managed-session`, `snapshot-current-managed-session`, or `retry-fresh-managed-session`. When other trailing diagnostic prose is also emitted in the same result, that block is concatenated **after** semantic-action candidate lines, overlay/selector-visibility tails, eval hints/warnings, and `Timeout partial progress` (see `rawAppendedDiagnosticText` in `extensions/agent-browser/lib/orchestration/browser-run/final-result.ts`). For `"auto"` failures the same struct may appear on `details` without that extra line. When post-upstream analysis (for example **`qa`** preset failure) flips the overall tool result after a successful batch, or a fresh `job`/batch opens the requested page and then a later step fails, the managed-session transition still reflects that the fresh browser became current. The visible recovery says the fresh launch became current and points to `failureCategory` / `qaPreset` / `batchFailure` for the post-launch failure instead of telling the agent that the old session was preserved.
820
822
  - `imagePath` / `imagePaths` for Pi inline image attachments from the **`screenshot`** command (including batched screenshot steps). **`diff screenshot`** still records the diff output as an `image`-kind entry in `details.artifacts`, but it does **not** populate `imagePath` / `imagePaths` or attach an inline image: only plain `screenshot` is treated as a trusted live-capture path for automatic inlining (`isTrustedScreenshotOutput` in `extensions/agent-browser/lib/results/presentation/artifacts.ts`).
821
823
  - `artifacts` for saved files such as screenshots, `state save` outputs, `diff screenshot` diff images, PDFs, downloads, `wait --download` files, traces, CPU profiles, completed WebM recordings, path-bearing HAR captures, and future recording output paths reported by `record start` / `record restart`. Non-file URL payloads such as `data:` / `blob:` / `http(s):` values are not treated as verified local artifacts. For direct artifact commands and batch artifact steps, the wrapper creates parent directories for requested paths before spawning upstream. Each artifact includes the original saved or requested `path`, resolved `absolutePath`, `kind`/`artifactType`, optional `mediaType`, optional `extension`, best-effort disk metadata such as `exists` and `sizeBytes`, plus `requestedPath`, `status`, `cwd`, `session`, and `tempPath` when applicable. Pending `record start` / `record restart` artifacts use `status: "pending"`, omit `exists` rather than reporting false, and include `recordingState: "openRecording"` / `willExistOnStop: true`.
822
824
  - `savedFilePath` / `savedFile` for direct `download`, `pdf`, and `wait --download` saved-file workflows when a host file path is reported or wrapper-verified. Batch results preserve the same fields on the relevant `batchSteps` entry. These fields are metadata only until `artifactVerification` verifies the file. For simple loopback `download <selector> <path>` anchors with a non-ref selector, `details.downloadRecovery.method: "direct-anchor-fetch"` means the wrapper resolved the anchor URL in-session and saved the in-page HTTP(S) response directly to the requested path before using upstream's click/download fallback; non-loopback/profile downloads stay upstream-owned so external provider behavior is preserved.
823
825
  - `batchSteps[].artifacts` for per-step artifacts in `batch` output; top-level `artifacts` aggregates all step artifacts in order. `record restart` can include both the previous recording that was finalized by the restart and the new pending recording.
824
826
  - `artifactVerification` for a normalized verification summary on the unified result and on each successful `batchSteps[]` row (failed batch steps omit artifact rows). Top-level `batch` verification rolls up all step file artifacts; each step’s summary reflects that step’s nested tool presentation (including its spill paths and manifest slice). It reports `verified`, `verifiedCount`, `missingCount`, `pendingCount`, `unverifiedCount`, and `artifacts[]` entries with `path`, optional `absolutePath`, optional `requestedPath`, `kind` (a normal file artifact kind or `"spill"` for manifest-backed rows), optional `mediaType`, optional `exists`, optional `sizeBytes`, optional `status`, optional `retentionState` / `storageScope` on manifest-derived rows, `state` (`verified`, `missing`, `pending`, or `unverified`), and optional `limitation` (human-readable lifecycle or retention context, for example pending `record start` / `record restart`, missing or unverified files, ephemeral spill files, or evicted persisted spills). The summary `verified` boolean is true only when every entry is `verified`. `record start` / `record restart` are `pending` until `record stop`; `state load` may mention a path in command output but is not a saved artifact row.
825
- - `fullOutputPath` / `fullOutputPaths` when large snapshot output or other oversized tool output is compacted and spilled to a private file; persisted sessions keep that path under a private session-scoped artifact directory with a bounded per-session budget so it survives reload/resume without unbounded growth
827
+ - `fullOutputPath` / `fullOutputPaths` when parse-valid large snapshot output or other oversized tool output is compacted and spilled to a private file; persisted sessions keep that path under a private session-scoped artifact directory with a bounded per-session budget so it survives reload/resume without unbounded growth. Malformed oversized upstream output is discarded after parsing, is omitted from `details.stdout`, and reports `fullOutputUnavailable` instead of creating a parse-failure spill.
826
828
  - `artifactManifest` for a bounded, metadata-only inventory of recent session artifacts. Entries include path metadata, artifact `kind`, source `command`/`subcommand` when safe, `storageScope` (`persistent-session`, `process-temp`, or `explicit-path`), and `retentionState` (`live`, `ephemeral`, `missing`, or `evicted`). The default recent window is 100 entries and can be configured with `PI_AGENT_BROWSER_SESSION_ARTIFACT_MANIFEST_MAX_ENTRIES`. The manifest must not store command args, output contents, headers, DOM snapshots, or downloaded file contents.
827
829
  - `artifactRetentionSummary` with a concise count of live, evicted, ephemeral, and missing artifacts from the current manifest; results append this summary to model-facing text only when retention state affects recovery, such as spill files, ephemeral files, or evictions. Routine explicit saved files keep the summary in details to avoid noisy browsing transcripts.
828
830
  - `artifactCleanup` after a successful close command (`close`, `quit`, or `exit`) when `artifactManifest` exists and `entries` is non-empty. Fields: `owner: "host-file-tools"`, `summary` (same retention summary string as `artifactRetentionSummary` for that manifest), `note` explaining that browser close commands do not delete explicit screenshots/downloads/PDFs/traces/HAR/recordings, and `explicitArtifactPaths`: up to ten **distinct existing** paths taken from manifest rows with `storageScope: "explicit-path"` in encounter order (de-duplicated after checking the filesystem); deleted/stale explicit paths are skipped. When the recent window has no existing explicit rows—for example only spill/ephemeral inventory or explicit paths already deleted—the array is empty but `summary` / `note` still surface so agents know close is not file deletion. The visible close text stays compact and points operators to `details.artifactCleanup.explicitArtifactPaths` instead of listing paths inline. The native browser tool intentionally does not expose a delete operation for arbitrary user-chosen artifact paths; agents should inspect `artifactVerification` / manifest metadata, then remove files with normal host file tools when cleanup is required.
@@ -832,7 +834,7 @@ Additional structured fields can appear when relevant:
832
834
 
833
835
  When the tool echoes `args` or `effectiveArgs` back into Pi, sensitive values such as `--headers`, proxy credentials, and auth-bearing URL parameters should be redacted first.
834
836
 
835
- For oversized snapshots and other oversized tool outputs, details should switch to a compact metadata object and include `fullOutputPath` pointing at a private spill file with the full redacted upstream payload. The model-facing tool text should print the actual spill-file path when one exists instead of only saying to inspect a details key. Oversized batch/job/qa failures include bounded failed-step context inline before the preview so agents can see the failed assertion/error and failure category without opening the spill file. Persisted sessions should keep that spill file under a private session-scoped artifact directory so the path remains usable after reload/restart. The oldest persisted spill files are evicted as needed to stay within `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES` (default 32 MiB), and those evictions are reported as `artifactManifest.entries[].retentionState: "evicted"` instead of silently disappearing from the session inventory. This persisted-spill byte budget is separate from the recent metadata window controlled by `PI_AGENT_BROWSER_SESSION_ARTIFACT_MANIFEST_MAX_ENTRIES`.
837
+ For parse-valid oversized snapshots and other oversized tool outputs, details should switch to a compact metadata object and include `fullOutputPath` pointing at a private spill file with the full redacted upstream payload. Malformed oversized output is not safe to redact structurally, so its temporary subprocess spill is deleted and no durable `fullOutputPath` is returned. The model-facing tool text should print the actual spill-file path when one exists instead of only saying to inspect a details key. Oversized batch/job/qa failures include bounded failed-step context inline before the preview so agents can see the failed assertion/error and failure category without opening the spill file. Persisted sessions should keep that spill file under a private session-scoped artifact directory so the path remains usable after reload/restart. The oldest persisted spill files are evicted as needed to stay within `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES` (default 32 MiB), and those evictions are reported as `artifactManifest.entries[].retentionState: "evicted"` instead of silently disappearing from the session inventory. This persisted-spill byte budget is separate from the recent metadata window controlled by `PI_AGENT_BROWSER_SESSION_ARTIFACT_MANIFEST_MAX_ENTRIES`.
836
838
 
837
839
  ## High-value result rendering
838
840
 
@@ -845,7 +847,7 @@ Worth doing in v1:
845
847
  - file artifacts such as PDFs, downloads, `wait --download` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed WebM recordings, and path-bearing HAR captures → concise saved-path summaries plus metadata in `details.artifacts` and bounded recent metadata in `details.artifactManifest`; `record start` / `record restart` report recording lifecycle state and the future output path without adding a missing manifest entry, and `record restart` can also report the previous wrapper-known recording that was finalized by the restart; upstream needs `ffmpeg` on `PATH` for `record stop` to encode the WebM, and successful `record start` / `record restart` calls may also expose `details.recordingDependencyWarning` when the wrapper cannot find `ffmpeg`; direct saved-file workflows also expose `details.savedFilePath` / `details.savedFile`; large or binary artifacts are not inlined into model context; the recent manifest cap can age out explicit-file metadata but does not remove explicit saved files from disk
846
848
  - `diff screenshot` → same file-artifact pattern as above for the **diff** image path only (summary text uses “Saved diff image” only when the diff output exists; missing output says “Diff image reported; file not verified” and fails as `artifact-missing`); baseline paths and other fields stay in the structured payload but are not echoed as separate saved artifacts in the visible artifact block, and there is no Pi inline image attachment for the diff output
847
849
  - `state load` → completion text may mention the loaded path, but the wrapper does **not** treat that path as a new saved artifact (`artifacts` / `artifactManifest` stay unset) the way `state save` does
848
- - auth, cookies, storage, clipboard, dialog, frame, state, network, debug, diff, stream, dashboard, chat, and other structured results → concise summaries that avoid expanding secret-bearing payloads; credential-like keys, values, URLs, body snippets, bearer/basic credentials, clipboard write text, cookie values, and likely secret storage values are redacted before model-facing output and `details.data`, while benign primitive storage values may remain visible for local QA
850
+ - auth, cookies, storage, clipboard, dialog, frame, state, network, debug, diff, stream, dashboard, chat, and other structured results → concise summaries that avoid expanding secret-bearing payloads; `state show` exposes metadata only in visible text and redacts every cookie/localStorage/sessionStorage `value` in structured details; credential-like keys, values, URLs, body snippets, bearer/basic credentials, clipboard write text, cookie values, and likely secret storage values are redacted before model-facing output and `details.data`, while benign primitive storage values may remain visible for local QA
849
851
  - TUI display → custom `agent_browser` call/result rendering with colorized command/output text and a built-in-style collapsed view for long visible output; top-level native modes render as `agent_browser qa → batch --bail`, `agent_browser job → batch --bail` by default (`agent_browser job → batch` when `failFast:false`), or `agent_browser semanticAction → find …` so reviewers can see both the native input mode and compiled upstream command; failed results keep `resultCategory` / `failureCategory` visible before truncated output; `ctrl+o` expansion reveals the full rendered tool result without changing the model-facing content
850
852
  - snapshots → origin + ref count + main-content-first compact preview, with the raw snapshot spill path printed directly in content and kept in `details.fullOutputPath` plus `details.artifactManifest` when the inline result would otherwise be too large
851
853
  - oversized generic outputs such as large `eval --stdin` payloads → compact preview plus the actual spill file path instead of dumping the whole payload into model context
@@ -854,7 +856,7 @@ Worth doing in v1:
854
856
  - navigation actions like `click`, `back`, `forward`, and `reload` → lightweight post-action title/url summary when available
855
857
  - tab lists → compact summary/table
856
858
  - stream status → enabled/connected/port summary plus WebSocket URL and frame format when a port is known; `stream enable` errors that only say streaming is already enabled are normalized to a successful idempotent no-op with `details.data.alreadyEnabled: true` and status/disable nextActions; if the caller explicitly passed `--json`, visible text is valid JSON instead of a prose summary
857
- - diagnostic/status families (`session`, `session list`, `profiles`, `doctor`, `auth list`/`show`, `cookies`, `storage`, `dialog`, `frame`, `state`, `network requests`, `console`, `errors`, and dashboard start/stop/status outputs) → compact readable summaries with counts and stable fields; `doctor` renders status/check/fix rows even when upstream puts those fields at the top level of its JSON envelope; `session list` and `tab list` keep names/labels/active markers/titles/URLs readable instead of opaque generated ids only; `network requests` and `console` previews label their scope as the upstream session aggregate unless upstream or a URL-opening QA preset explicitly cleared/filtered the buffers first; network request lists include an actionable-vs-benign failed-request summary and mark low-impact browser icon failures separately; active route mocks can add failed/pending/CORS route diagnostics; `data:image` artifact request rows are hidden from compact previews while preserved in raw details; request-detail URLs from `network request` and fetched URLs from explicit `read <url>` remain diagnostic-only rather than session page targets; large log/request/error outputs use previews plus `fullOutputPath` spill files; sensitive nested auth/header/token fields are not expanded in the model-facing text
859
+ - diagnostic/status families (`session`, `session list`, `profiles`, `doctor`, `auth list`/`show`, `cookies`, `storage`, `dialog`, `frame`, `state`, `network requests`, `console`, `errors`, and dashboard start/stop/status outputs) → compact readable summaries with counts and stable fields; `doctor` renders status/check/fix rows even when upstream puts those fields at the top level of its JSON envelope; `session list` keeps caller-owned names/labels/active markers/titles/URLs readable while removing wrapper-managed `piab-*` rows, and `tab list` keeps its corresponding fields readable instead of opaque generated ids only; `network requests` and `console` previews label their scope as the upstream session aggregate unless upstream or a URL-opening QA preset explicitly cleared/filtered the buffers first; network request lists include an actionable-vs-benign failed-request summary and mark low-impact browser icon failures separately; active route mocks can add failed/pending/CORS route diagnostics; `data:image` artifact request rows are hidden from compact previews while preserved in raw details; request-detail URLs from `network request` and fetched URLs from explicit `read <url>` remain diagnostic-only rather than session page targets; large log/request/error outputs use previews plus `fullOutputPath` spill files; sensitive nested auth/header/token fields are not expanded in the model-facing text
858
860
  - trace/profiler owner conflicts → when the wrapper has observed one owner active for a session, block conflicting starts/stops with "wrapper believes ..." wording because upstream or external CLI use can desynchronize wrapper-local state
859
861
 
860
862
  ## Missing binary behavior
@@ -874,14 +876,15 @@ If `agent-browser` is not on `PATH`, fail with a message that:
874
876
  - close the active extension-managed session when the originating `pi` process quits, while leaving explicit caller-provided sessions alone
875
877
  - set one idle timeout on extension-managed sessions as a backstop for abnormal exits or cleanup failures, and pass that same `AGENT_BROWSER_IDLE_TIMEOUT_MS` to top-level commands plus every wrapper helper subprocess so upstream does not restart the background browser, reset the active tab, or discard current refs between a snapshot and action
876
878
  - clean up process-private temp spill artifacts on shutdown, while keeping persisted-session snapshot spill files in a private session-scoped artifact directory so `details.fullOutputPath` survives reload/restart and the oldest spill files are evicted if the per-session artifact budget is exceeded
877
- - reconstruct the current branch-visible extension-managed session, latest page-scoped refs, latest `artifactManifest`, and wrapper-tracked Electron launch records from the active transcript branch on `session_start` and Pi `session_tree` so later default calls keep following the active managed browser and can continue reporting artifact retention state; successful explicit wrapper-owned close rows and `electron.cleanup` managed-session steps are restore-visible close events
878
- - keep runtime cleanup ownership separate from branch-visible state: `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work; independent caller-owned explicit-session commands can still run in parallel, but a branch-state generation guard prevents stale completions from overwriting a newer branch restore. Extension-managed sessions and wrapper-launched Electron records owned by the current process remain eligible for quit/cleanup, and fresh-session allocation stays monotonic across branch restores, including auto rows and close rows that reference wrapper-generated fresh names
879
+ - reconstruct the current branch-visible extension-managed session, latest page-scoped refs, newest-revision aggregate `artifactManifest`, and wrapper-tracked Electron launch records from the active transcript branch on `session_start` and Pi `session_tree` so later default calls keep following the active managed browser and can continue reporting artifact retention state; successful explicit wrapper-owned close rows and `electron.cleanup` managed-session steps are restore-visible close events
880
+ - keep runtime cleanup ownership separate from branch-visible state: `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work; caller-owned explicit-session commands use separate process-local queues keyed by effective canonical namespace/session (explicit namespace argv wins over inherited `AGENT_BROWSER_NAMESPACE`, including an explicit empty default), so the live URL probe, preparation helpers, semantic snapshot, main command, and state commit for one identity cannot interleave while different identities remain concurrent. Namespace and session identity components are additionally Unicode-normalized and case-folded on macOS and Windows to match their case-insensitive daemon paths. Only the outer tool execution acquires that key; nested helpers run under it without re-entry. Policy, route, and artifact deltas survive unrelated managed-state commits, while a separate branch-restore generation guard prevents stale completions from overwriting a newer branch. Concurrent artifact-producing results carry a monotonic aggregate manifest revision so transcript replay selects the complete bounded manifest rather than whichever call happened to occupy the last row. Extension-managed sessions and wrapper-launched Electron records owned by the current process remain eligible for quit/cleanup, and fresh-session allocation stays monotonic across branch restores, including auto rows and close rows that reference wrapper-generated fresh names
879
881
  - when a close command or `electron.cleanup` successfully closes the current wrapper-managed session, clear live page/ref state, reserve the next generated fresh-session ordinal, and rotate the next default auto call to a fresh wrapper-generated session name rather than reusing the closed name
880
882
  - when `/reload` shuts down an extension instance, close off-branch owned managed sessions and off-branch owned Electron launches before clearing process-local ownership; preserve only the current branch-visible active managed session and active Electron launch plus its isolated `userDataDir` for reload continuity, and also persistently protect `userDataDir` paths when partial cleanup intentionally skips or fails profile removal so later temp cleanup, process exit, and stale temp-root pruning after restart do not violate Electron cleanup's safety decision; rebuild active branch state from the active branch on the next `session_start`
881
883
  - when an unnamed `sessionMode: "fresh"` launch succeeds, make it the new extension-managed session so later default calls keep using it
882
884
  - when an unnamed `sessionMode: "fresh"` launch fails or times out, preserve the previous managed session when one was active or report the attempted fresh session as abandoned when no managed session was active (`details.managedSessionOutcome`; visible `Managed session outcome: …` only when the final tool call used `sessionMode: "fresh"` and failed—see `#details`)
883
885
  - if that unnamed fresh launch replaced an already-active managed session, best-effort close the old managed session after the switch succeeds
884
- - treat explicit caller-provided `--session` choices as user-managed; `--session` alone isolates a live browser session but is not a persisted tab/auth restore mechanism after a close command (`close`, `quit`, or `exit`), so use `--session <id> --restore`, `--profile`, or `--state` when persisted auth/tab state is required
886
+ - treat explicit caller-provided non-managed `--session` choices outside the reserved `piab-*` prefix as user-managed; those names alone isolate a live browser session but are not a persisted tab/auth restore mechanism after a close command (`close`, `quit`, or `exit`), so use `--session <id> --restore`, `--profile`, or `--state` when persisted auth/tab state is required. Wrapper-owned managed sessions (the current/generated identity or an exact namespace/session in this extension instance's ownership records; foreign `piab-*` targets are rejected) set a Git-checkout-generation-stable `AGENT_BROWSER_RESTORE` env key automatically (disable with `PI_AGENT_BROWSER_MANAGED_SESSION_RESTORE=0`) so SSO cookies/localStorage/sessionStorage can survive idle shutdown and later chats in the same checkout generation without a manual `state save` flow. Any project/user/explicit upstream config discovered while planning blocks browser-backed native calls without parsing caller-selected content; accepted browser-backed subprocesses, including wrapper-owned closes, pin a process-private empty config (`0400` on POSIX) in the marked secure-temp lifecycle so later config creation cannot change the receiving browser. Sessionless local/setup commands retain upstream config behavior, and this restriction is separate from the package's Pi-scoped config. Raw batch argv, batch stdin containing nested `connect`/`batch`, and browser mutation flags/env such as extensions, init scripts, raw launch args, custom user agents/executables, proxies, plugins, WebGPU, profiles, parent env namespaces, CDP, providers, state, and containment also disable automatic managed restore. Wrapper-owned close canonicalizes upstream argv to the known namespace/session, discards caller config/restore globals, and preserves the live daemon's existing restore key rather than injecting one derived from a replacement checkout. A failed fresh call probes its exact daemon identity and remains owned when that daemon is live or uninspectable, so shutdown still closes it. After a wrapper-owned close succeeds, the wrapper persists the returned state path as an atomic record in a lockless convergent per-key ownership directory (`0700`, with `0600` records, on POSIX), retains the two newest proven snapshots for its exact restore key across Pi restarts, self-heals malformed regular records, removes additional proven snapshots older than 30 days, expires stale ownership-proven snapshots and empty manifests from other restore-key generations after 30 days only when a private lineage record proves the same canonical checkout path, and caps young close churn at 256 records per restore key without running namespace-wide `state clean`; unrecorded matching files and the current checkout key are untouched. Automatic restore requires a durable Git checkout generation and an absolute home root. The generation UUID lives in the checkout's Git admin directory and is combined with checkout-root plus Git-admin filesystem identity, so it follows a rename, changes on checkout replacement/copy, and is not migrated from cwd-only keys. On POSIX the wrapper canonicalizes and pins `HOME`, requires trusted non-writable owner ancestry plus stable device/inode/birth-time metadata for checkout and Git-admin directories, enforces owner-only mode `0700` without silently repairing insecure existing directories, and rejects symlinks/non-directories along `~/.agent-browser[/namespaces/<canonical>/state]/sessions` and its `.tmp` write area; Windows requires a valid 64-character hex `AGENT_BROWSER_ENCRYPTION_KEY`. Namespace identity uses upstream's lowercase sanitized form, and wrapper-owned subprocesses pin it (including an explicit empty default) so parent environment cannot redirect helpers or close. `electron.status` target reads and `electron.probe` hold the managed daemon-policy lock and apply the owned restore context to each underlying title/URL/focus/tab/snapshot read. When a managed session was launched with restore suppressed by an incompatible path or the explicit opt-out, `details.managedSessionRestoreDisabled: true` means later bare follow-ups do not inject wrapper restore. Same-process `session_tree` transitions retain the process-owned daemon-policy proof; reload, restart, and `/resume` restore the sticky flag but not that proof, so a still-live restore-disabled daemon fails closed until it is closed or a fresh identity is used. If inspection proves the prior daemon inactive, the next owned no-restore spawn records a null daemon policy so later follow-ups can reuse the replacement. A user-private immutable ticket-claim lock serializes policy inspection through the receiving same-identity spawn and bridges the pre-update v2 lock path; every lock winner re-inspects the live daemon with a fixed bounded timeout independent of shorter caller watchdog overrides, and abandoned v2 locks fail closed for manual repair. Before incompatible reuse, the wrapper inspects the actual same-identity daemon and fails closed when it retains any restore key, cannot be inspected, or reports restore-disabled policy without current-process provenance, including live daemons absent from transcript state and explicit restore-key sessions; close it first, retry without an explicit session using `sessionMode: "fresh"`, or use a distinct explicit session. Managed restore keys and key-bearing paths are redacted from visible text, structured details, JSON-mode output, and persisted tool rows. Browser access to `.agent-browser` local storage is rejected before spawn through command-specific input/output operands (including dash-prefixed values), every path-bearing upstream environment mirror (state/profile/config, executable/extension/init-script, action-policy, artifact, skills, and socket paths), encoded, nested-file-scheme, Windows-aliased, or symlinked paths (including nonexistent descendants of symlinked directories), protected top-level `outputPath`, content-returning local-URL calls, all follow-ups on local file pages, and persisted unverified top-level or batch tab/attachment/script/state-load transitions, plus raw batch command strings recursively tokenized on literal ASCII spaces exactly like upstream. Electron probes and later capture from a wrapper-tracked protected target use the same guard. Raw artifact destinations use the same command parser as preparation and are checked before it creates parent directories. Enabled `--allow-file-access` argv/env and file-access-enabling or protected-path raw Chrome values are rejected. Every subprocess clears `AGENT_BROWSER_ARGS`, removes caller file-access occurrences, and adds canonical `--args "" --allow-file-access false` defaults so unsupported equals forms and project/user config cannot re-enable it; an explicit validated safe CLI `--args` value may follow and override the empty raw-args default. Post-transition summaries, including forced live probes after arbitrary `eval`, verify URL before title and fail implicit transitions to local file pages. Failed or unexecuted navigation attempts remain unverified, and stale concurrent completions serialize only the authoritative page state; transcript replay gives `sessionTabTargetUnknown` precedence over any inconsistent stale target/ref fields. `get url`, `tab list`, non-content `tab <id>` selection, safe explicit navigation away, and session/tab close remain available; tab selection stays unverified until `get url` succeeds. `session list` and `state list` omit wrapper-managed rows; the `piab-*` live-session reservation is case-insensitive, and foreign managed live-session and restore/state references, broad state clearing/cleaning, and managed save/rename targets fail before spawn. Malformed oversized upstream output is discarded rather than persisted as a parse-failure spill. Checkout, storage, environment, managed-session ownership, and state-access policy is checked again after async config/socket setup immediately before spawn. POSIX socket storage is accepted only when the target is current-user-owned mode `0700`, its ancestry is trusted, and existing entries are current-user-owned regular files, sockets, or directories without symlinks; pre-existing unsafe modes are rejected rather than repaired. Identity and config globals use upstream's separated forms (`--session <name>`, `--namespace <name>`, `--config <path>`); the wrapper rejects leading identity equals forms that upstream 0.33.2 does not recognize, while equals-shaped tokens in command payload positions remain ordinary payload rather than silently selecting wrapper ownership or config. Native Windows command-first adaptation relocates only syntactically valid leading globals, rewrites valued `--restore <name>` as `--restore=<name>`, and leaves command-scoped or invalid leading input untouched. Optional global booleans also follow upstream's separated, last-occurrence-wins form: only exact lowercase `false` after the flag disables it, and an unsupported token such as `--auto-connect=false` does not override an earlier bare flag
887
+ - before a content-bearing read or interaction against a caller-owned explicit session, run a session-scoped `get url` probe and apply the local-state boundary to that live target. Missing or stale transcript page state is not accepted as proof; a failed or non-URL probe blocks the requested content command. The process-local effective-namespace/session queue keeps that probe atomic with semantic snapshot resolution and the main command inside one extension instance; direct CLI calls and other Pi processes remain outside this guarantee. Non-bail batch validation retains every possible pre-transition page state up to a fixed bound, so later content is rejected when any failed transition could expose local or unverified content and branching beyond that bound fails closed with exact `batch --bail` guidance; exact `batch --bail` or split calls remove that continuation path. Protected Windows paths include drive-relative forms such as `C:.agent-browser\\state\\...`. Nested `batch` steps are rejected, while raw batch command strings mirror upstream's ASCII-space tokenizer, including its single/double-quote and backslash handling, without splitting on other Unicode whitespace.
885
888
  - pass explicit `--profile` straight through to upstream `agent-browser`; no profile-cloning or isolation layer is added in v1
886
889
  <!-- agent-browser-playbook:start wrapper-tab-recovery -->
887
890
  <!-- Generated from extensions/agent-browser/lib/playbook.ts. Run `npm run docs -- playbook write` to update. -->
@@ -890,7 +893,7 @@ If `agent-browser` is not on `PATH`, fail with a message that:
890
893
  - For sessions with observed tab-drift risk, after a successful command on a known target tab, agent_browser also best-effort restores that intended tab if a restored/background tab steals focus after the command completes. Routine same-session commands skip this post-command tab-list probe.
891
894
  - If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
892
895
  <!-- agent-browser-playbook:end wrapper-tab-recovery -->
893
- - on local Unix launches, set a short private socket directory for wrapper-spawned `agent-browser` processes so extension-generated session names do not fail the upstream Unix socket-path length limit in longer cwd/session-name combinations
896
+ - on local Unix launches, set a short private socket directory for wrapper-spawned `agent-browser` processes so extension-generated session names do not fail the upstream Unix socket-path length limit in longer cwd/session-name combinations; require an absolute non-symlink directory owned by the current uid with mode `0700`, otherwise fail before spawn
894
897
  - keep wrapper-spawned commands bounded by clamping `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented default of 25 seconds while the default wrapper child-process watchdog is 35 seconds (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides it, and top-level `timeoutMs` overrides it per call for browser CLI subprocesses). Explicit `wait <ms>` or `wait --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a per-call subprocess watchdog from the requested wait duration plus a small grace window. Dialog commands use `PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS` (default 5000 ms), and click/tap/find refs or tokens plus `eval --stdin` snippets whose text looks like alert/confirm/prompt/dialog triggers use `PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS` (default 8000 ms). Timed-out compiled `job` / `qa` or caller `batch` calls may add `details.timeoutPartialProgress` and visible `Timeout partial progress` evidence with per-step status, retry payloads, current page title/URL, and declared artifact path checks; timed-out dialog-like commands may add dialog status/dismiss/fresh-session recovery next actions
895
898
  - interactive or long-running upstream families such as `chat` without a prompt, `dashboard start`, `stream enable`, `trace start`, `profiler start`, `record start`, `inspect`, `install`, `upgrade`, `doctor --fix`, and `confirm-interactive` are passed through thinly but remain bounded by the same wrapper timeout/session planning rules; prefer explicit arguments, single-shot `chat <message>`, non-interactive flags like `doctor --offline --quick` or `doctor --json`, and cleanup pairs such as `dashboard stop`, `stream disable`, `trace stop`, `profiler stop`, and `record stop`
896
899
  - treat successful plain-text inspection commands like `--help` and `--version` as stateless: do not inject the implicit managed session and do not let those calls claim the managed-session slot
@@ -62,7 +62,7 @@ PLATFORM_SMOKE_MAC_PORT=22
62
62
  PLATFORM_SMOKE_UBUNTU_IMAGE="pi-agent-browser-native-platform:node24-agent-browser<baseline-version>"
63
63
 
64
64
  PLATFORM_SMOKE_WINDOWS_VM="pi-extension-windows-template"
65
- PLATFORM_SMOKE_WINDOWS_SNAPSHOT="crabbox-ready"
65
+ PLATFORM_SMOKE_WINDOWS_SNAPSHOT="crabbox-ready-ab-0.33.0"
66
66
  PLATFORM_SMOKE_WINDOWS_USER="<windows-ssh-user>"
67
67
  PLATFORM_SMOKE_WINDOWS_WORK_ROOT="C:\\crabbox\\pi-agent-browser-native"
68
68
 
@@ -72,7 +72,7 @@ PLATFORM_SMOKE_AUTH_ENV=""
72
72
 
73
73
  The Ubuntu target image is derived from `node:24-bookworm`, installs the `agent-browser` version from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs), installs Debian Chromium plus the upstream Linux WebGPU/Xvfb runtime packages (`libvulkan1`, `mesa-vulkan-drivers`, and `xvfb`) through apt, creates a non-root `circleci` user, and sets `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium`. Rebuild it after upstream rebaselining, or override `PLATFORM_SMOKE_UBUNTU_IMAGE` with an equivalent prepared local image. Do not install `agent-browser` ad hoc inside the Ubuntu smoke command; a missing tool is image/template drift.
74
74
 
75
- The configured upstream `agent-browser` baseline is imported from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs). Target-local browser suites verify that exact `agent-browser` version before running. Bake the exact upstream CLI and browser runtime into the Windows template/snapshot for speed and reproducibility; missing or stale Windows `agent-browser` / browser readiness is a blocked setup, not something the smoke command repairs. The Windows browser suite checks the preinstalled browser cache and prewarms one short local file URL before the extension harness runs.
75
+ The configured upstream `agent-browser` baseline is imported from [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs). Target-local browser suites verify that exact `agent-browser` version before running. Bake the exact upstream CLI and browser runtime into the Windows template/snapshot for speed and reproducibility; missing or stale Windows `agent-browser` / browser readiness is a blocked setup, not something the smoke command repairs. The Windows browser suite checks the preinstalled browser cache and prewarms one short loopback HTTP URL before the extension harness runs.
76
76
 
77
77
  ## Target setup expectations
78
78
 
@@ -80,11 +80,11 @@ Crabbox does not install project runtime tools. The macOS host, Ubuntu image, an
80
80
 
81
81
  - Node/npm at or above the configured Node major baseline in [`platform-smoke.config.mjs`](../platform-smoke.config.mjs).
82
82
  - Git and `tar`.
83
- - Upstream `agent-browser` matching this wrapper’s capability baseline. The Ubuntu target gets it from [`scripts/platform-smoke/linux-image/Dockerfile`](../scripts/platform-smoke/linux-image/Dockerfile); the Windows template gets it from the shared `pi-extension-windows-template` / `crabbox-ready` snapshot.
83
+ - Upstream `agent-browser` matching this wrapper’s capability baseline. The Ubuntu target gets it from [`scripts/platform-smoke/linux-image/Dockerfile`](../scripts/platform-smoke/linux-image/Dockerfile); the Windows template gets it from the shared `pi-extension-windows-template` snapshot named in [`platform-smoke.config.mjs`](../platform-smoke.config.mjs) (currently `crabbox-ready-ab-0.33.0`, a child of the shared `crabbox-ready` base used by other projects' linked clones).
84
84
  - Browser/runtime dependencies needed by upstream `agent-browser`.
85
85
  - Native PowerShell and OpenSSH Server on Windows.
86
86
 
87
- For Windows, reuse `pi-extension-windows-template` with the shared canonical `crabbox-ready` power-off snapshot configured in [`platform-smoke.config.mjs`](../platform-smoke.config.mjs). Do not create one-off project VMs or run tests directly on the source VM. If a reusable tool is missing, update the shared template, verify from a fresh SSH session, remove caches/secrets/checkouts, shut down cleanly, and promote a known-good power-off snapshot.
87
+ For Windows, reuse `pi-extension-windows-template` with the power-off snapshot configured in [`platform-smoke.config.mjs`](../platform-smoke.config.mjs). Do not create one-off project VMs or run tests directly on the source VM. If a reusable tool is missing, update the shared template from the current base snapshot, verify from a fresh SSH session, remove caches/secrets/checkouts, shut down cleanly, and promote a known-good power-off snapshot (a child snapshot is fine when other projects still hold linked clones of an older base).
88
88
 
89
89
  ## What the suites prove
90
90
 
@@ -106,7 +106,7 @@ Each required target runs `platform-build` and `browser-dogfood-smoke` on one Cr
106
106
 
107
107
  1. Run `npm ci` in the synced checkout if needed.
108
108
  2. Run the deterministic model-free browser smoke through `scripts/verify-agent-browser-dogfood.ts`.
109
- 3. Exercise native wrapper surfaces against the deterministic local file fixture from `scripts/verify-agent-browser-dogfood.ts`: top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close.
109
+ 3. Exercise native wrapper surfaces against the deterministic loopback HTTP fixture from `scripts/verify-agent-browser-dogfood.ts`: top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close.
110
110
  4. Persist the dogfood JSON report and stdout/stderr evidence.
111
111
  5. Fail on missing browser artifacts, failed tool calls, leaked secrets, or unclosed sessions.
112
112
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-agent-browser-native",
3
- "version": "0.2.71",
3
+ "version": "0.2.74",
4
4
  "description": "pi extension that exposes agent-browser as a native tool for browser automation",
5
5
  "type": "module",
6
6
  "author": "Mitch Fultz (https://github.com/fitchmultz)",
@@ -22,7 +22,9 @@ export default {
22
22
  ubuntuContainerImage: PLATFORM_SMOKE_UBUNTU_IMAGE,
23
23
  windowsParallels: {
24
24
  sourceVm: "pi-extension-windows-template",
25
- snapshot: "crabbox-ready",
25
+ // Shared template snapshot; refresh when the Windows image moves past agent-browser 0.33.0.
26
+ // Parent crabbox-ready remains for other projects' linked clones.
27
+ snapshot: "crabbox-ready-ab-0.33.0",
26
28
  },
27
29
  nodeValidationMajor: 22,
28
30
  agentBrowserVersion: PLATFORM_SMOKE_AGENT_BROWSER_VERSION,