pi-agent-browser-native 0.6.17 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (49) hide show
  1. package/CHANGELOG.md +19 -0
  2. package/README.md +55 -167
  3. package/dist/extensions/agent-browser/index.js +312 -223
  4. package/dist/extensions/agent-browser/lib/browser-transcript.js +33 -0
  5. package/dist/extensions/agent-browser/lib/execution-cwd.js +4 -2
  6. package/dist/extensions/agent-browser/lib/input-modes/job.js +2 -213
  7. package/dist/extensions/agent-browser/lib/input-modes/params.js +146 -142
  8. package/dist/extensions/agent-browser/lib/input-modes/script.js +64 -92
  9. package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +170 -24
  10. package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +13 -1
  11. package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +26 -13
  12. package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +21 -1
  13. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +4 -2
  14. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +1 -3
  15. package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +35 -20
  16. package/dist/extensions/agent-browser/lib/orchestration/browser-run/recording-recovery.js +3 -1
  17. package/dist/extensions/agent-browser/lib/orchestration/browser-run/screenshot-observation.js +79 -0
  18. package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +3 -4
  19. package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +19 -52
  20. package/dist/extensions/agent-browser/lib/orchestration/native-session-defaults.js +3 -3
  21. package/dist/extensions/agent-browser/lib/orchestration/output-file.js +3 -3
  22. package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +91 -278
  23. package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +9 -9
  24. package/dist/extensions/agent-browser/lib/playbook.js +46 -25
  25. package/dist/extensions/agent-browser/lib/process-identity.js +22 -6
  26. package/dist/extensions/agent-browser/lib/process.js +16 -13
  27. package/dist/extensions/agent-browser/lib/results/action-recommendations.js +2 -2
  28. package/dist/extensions/agent-browser/lib/results/next-actions.js +1 -2
  29. package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +25 -4
  30. package/dist/extensions/agent-browser/lib/results/presentation/batch.js +6 -2
  31. package/dist/extensions/agent-browser/lib/results/presentation/common.js +2 -0
  32. package/dist/extensions/agent-browser/lib/results/presentation/content.js +39 -0
  33. package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +3 -3
  34. package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +41 -1
  35. package/dist/extensions/agent-browser/lib/results/presentation.js +16 -11
  36. package/dist/extensions/agent-browser/lib/runtime.js +18 -6
  37. package/dist/extensions/agent-browser/lib/session-page-state.js +3 -4
  38. package/dist/extensions/agent-browser/lib/tool-surface.js +138 -0
  39. package/dist/extensions/agent-browser/script-worker.js +4 -1
  40. package/docs/ARCHITECTURE.md +46 -62
  41. package/docs/COMMAND_REFERENCE.md +77 -136
  42. package/docs/ELECTRON.md +51 -49
  43. package/docs/RELEASE.md +35 -21
  44. package/docs/REQUIREMENTS.md +18 -14
  45. package/docs/SUPPORT_MATRIX.md +33 -15
  46. package/docs/TOOL_CONTRACT.md +216 -405
  47. package/docs/platform-smoke.md +2 -2
  48. package/package.json +1 -1
  49. package/scripts/doctor.mjs +2 -2
@@ -10,13 +10,15 @@ Related docs:
10
10
 
11
11
  ## Purpose
12
12
 
13
+ Requires Pi 0.87.0 or newer; official and fork hosts are qualified independently.
14
+
13
15
  Provide a local, repo-readable command reference for the native `agent_browser` tool.
14
16
 
15
17
  This project intentionally blocks normal `agent-browser` bash usage in most agent sessions, so the agent still needs an accessible local equivalent of the upstream command surface. This document is the durable reference the agent can read inside the repository without calling the binary directly.
16
18
 
17
19
  After updating `pi-agent-browser-native`, fully quit and restart Pi before using the updated tools. `/reload` can retain previously loaded compiled JavaScript even after `dist/` is rebuilt, so it is not a reliable way to pick up package updates.
18
20
 
19
- SDK hosts can supply an awaited [`beforeExecute` callback](TOOL_CONTRACT.md#host-execution-hook) to save host state before ordinary or script-inner browser dispatch. This is a factory option, not a tool argument; normal installations do not need it.
21
+ SDK hosts can supply an awaited [`beforeExecute` callback](TOOL_CONTRACT.md#host-execution-hook) to save host state before ordinary or code-inner browser dispatch. This is a factory option, not a tool argument; normal installations do not need it.
20
22
 
21
23
  ## Upstream baseline
22
24
 
@@ -154,7 +156,7 @@ The 0.31.1 rebaseline is a React bugfix release: `react tree`, `react inspect <i
154
156
 
155
157
  The 0.31.0 rebaseline adds restore workflow and namespace/session lifecycle surfaces: `--restore [name]`, `--restore-save <policy>`, restore check flags, `--namespace <name>`, `session id`, and `session info`. The wrapper parses those globals, keeps `--namespace` before `--session`, carries namespace context through managed-session probes and state, and keeps `session id` / `session info` sessionless. Use `agent_browser` with `args: ["session", "id", "--scope", "worktree", "--prefix", "my-skill"]` to derive reusable session ids from inside Pi; use `--restore=<key>` when passing an explicit key that could be confused with a command word.
156
158
 
157
- Runtime probes retain the 0.30.1 `wait --url` fix: `wait --url "**/dashboard"` succeeds after a `pushstate /dashboard`, so `job.assertUrl` delegates exact and glob patterns to upstream `wait --url`. Upstream 0.32.4 aligns advertised `find` actions with the dispatcher: `click, fill, check, hover, text` only—use top-level `uncheck <selector-or-ref>` (and `type` / `focus`) instead of `find ... uncheck|type|focus`. One older caveat still stands: `wait <selector> --state hidden|detached` remains advertised by some help paths but fails at runtime, so keep `wait --fn` disappearance guidance.
159
+ Runtime probes retain the 0.30.1 `wait --url` fix: `wait --url "**/dashboard"` succeeds after a `pushstate /dashboard`, so URL assertions use native `wait --url` with exact or glob patterns. Upstream 0.32.4 aligns advertised `find` actions with the dispatcher: `click, fill, check, hover, text` only—use top-level `uncheck <selector-or-ref>` (and `type` / `focus`) instead of `find ... uncheck|type|focus`. One older caveat still stands: `wait <selector> --state hidden|detached` remains advertised by some help paths but fails at runtime, so keep `wait --fn` disappearance guidance.
158
160
 
159
161
  ### Upstream 0.29.1 rebaseline
160
162
 
@@ -188,80 +190,57 @@ The 0.27.3 rebaseline is an install-only compatibility update: upstream changed
188
190
 
189
191
  ## Core mental model
190
192
 
191
- Input mode chooser (one per call): **`script`** for one-shot loops, branches, or multi-page aggregation; **`args`** for the default open → snapshot -i → click/fill `@refs` flow; **`semanticAction`** for stable role/text/label targets; **`job`** / **`qa`** for multi-step checks; **`electron`** for desktop apps only; **`sourceLookup`** / **`networkSourceLookup`** are **experimental candidates-only** helpers (not authoritative mappings). Omit `--json` for ordinary prose; include it when you need parseable JSON text. Match link and button text to the latest snapshot (on `https://example.com/` the main link is `Learn more`, not legacy `More information...` copy). See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#input-mode-chooser) for snapshot variants (`-i` vs `--compact` vs full) and batching three or more getters.
193
+ Use `agent_browser` for one native command and `batch --bail` with JSON-array stdin for a fixed sequence. Use `agent_browser_code` for loops, branches, and aggregation over the same persistent browser. Return to the model when observation needs judgment. All native command families remain available; there is no public job DSL or isolated script input.
192
194
 
193
- Tool parameters (use exactly one of `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`):
195
+ `agent_browser` accepts `{ args, stdin?, outputPath?, timeoutMs?, sessionMode? }`. Tokens exclude the binary and shell syntax. Stdin is accepted for native batch, `eval --stdin`, and `auth save --password-stdin`. Omit `--json` for prose; include it for a parseable canonical observation. Help/version remains native text.
194
196
 
195
197
  ```json
196
- { "script": "const page = await browser({ args: ['get', 'title'] }); if (!page.ok) throw new Error(page.error); emit(page.data.title ?? page.data.result);" }
198
+ { "args": ["open", "https://example.com"] }
199
+ { "args": ["snapshot", "-i"] }
200
+ { "args": ["batch", "--bail"], "stdin": "[[\"get\",\"title\"],[\"get\",\"url\"]]" }
197
201
  ```
198
202
 
199
- ```json
200
- { "args": ["open", "https://example.com"], "sessionMode": "auto" }
201
- ```
203
+ Ordinary calls share the root Pi parent/subagent browser unless native config, an explicit identity, or an active managed/fresh browser selects otherwise. `sessionMode: "fresh"` intentionally rotates the managed browser; explicit native session choice still wins. Keep profile/provider/attachment launch flags on first launch. After a failed fresh call, inspect `managedSessionOutcome` rather than guessing which browser is current.
202
204
 
203
- ```json
204
- { "semanticAction": { "action": "click", "locator": "text", "value": "Submit" }, "sessionMode": "auto" }
205
- { "semanticAction": { "action": "fill", "selector": "@e1", "text": "prompt text" } }
206
- { "semanticAction": { "action": "select", "selector": "#flavor", "value": "chocolate" } }
207
- ```
205
+ A cooperating-process lock covers each direct helper/action sequence or whole code cell, keyed by actual socket context, canonical namespace, and session. Independent identities remain concurrent; direct namespace-wide close barriers matching work. Restart all participants onto the new version. This is not rollback or protection against human/external-client navigation, and separate tool calls still need coordination.
208
206
 
209
- ```json
210
- { "args": ["batch"], "stdin": "[[\"open\",\"https://example.com\"],[\"snapshot\",\"-i\"]]" }
211
- ```
207
+ `outputPath` saves result data or recording receipts, including failed recording evidence, through the existing writer. Keep it distinct from browser artifacts; aliasing fails without overwriting the artifact. `timeoutMs` bounds the requested subprocess; managed policy inspection has its separate budget. Unknown mutation outcomes require inspection before retry.
212
208
 
213
- ```json
214
- { "job": { "steps": [{ "action": "open", "url": "https://example.com" }, { "action": "assertText", "text": "Example Domain" }] } }
215
- ```
209
+ ### Persistent browser code
216
210
 
217
- ```json
218
- { "sourceLookup": { "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" } }
219
- ```
211
+ Call `agent_browser_code` with `{ code, session?, namespace?, timeoutMs?, outputPath? }`. Its async source receives `browser({ args, stdin?, timeoutMs? })`, `emit(JSON)`, and `emitImage(imageHandle)`. Check `result.success`, select useful data, and emit only images needed for vision.
220
212
 
221
213
  ```json
222
- { "networkSourceLookup": { "requestId": "req-1", "url": "/api/fail" } }
214
+ {
215
+ "code": "const titles = []; for (const url of ['https://example.com', 'https://example.org']) { const opened = await browser({ args: ['open', url] }); if (!opened.success) throw new Error(opened.error); const title = await browser({ args: ['get', 'title'] }); if (!title.success) throw new Error(title.error); titles.push({ url, title: title.data.title ?? title.data.result }); } emit(titles);"
216
+ }
223
217
  ```
224
218
 
225
- ```json
226
- { "electron": { "action": "list", "query": "code" } }
227
- { "electron": { "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" } }
228
- ```
219
+ The selected browser and authentication persist; JavaScript variables reset. Native batch, auth/profile/state/connect/close and ordinary config/environment remain usable. One cell cannot change its parent-selected session/namespace or issue `close --all`; do those directly outside code. No host APIs, imports, process, filesystem/network access, timers, or dynamic code generation are exposed by the fresh permissioned child.
220
+
221
+ Inner calls are serial even under `Promise.all`. Bounds are 25 calls, 64 KiB source/emitted JSON, 1 MiB per IPC message, 8 MiB cumulative IPC, and 120 seconds default/300 seconds maximum. `emitImage(shot.imageObservations[0])` selects a verified current-cell image; arbitrary paths are not handles. Up to eight images/20 MiB total are allowed under the configurable inline bound (5 MiB per image by default). Conditional unchanged screenshots have no image.
222
+
223
+ Handled failures remain in visible `failures` and `codeRun` counts. A completed source body does not prove all browser operations succeeded. Input-shape rejection, source exceptions, cancellation, and timeout fail the code result. Ordered Pi intent/completion entries preserve browser state across resume; interrupted pending work stays unknown until inspection. Code requires Pi persistence, but does not finally-close the browser. See [the code contract](TOOL_CONTRACT.md#agent_browser_code) and [0.7 migration](TOOL_CONTRACT.md#07-migration).
224
+
225
+ ### Advanced tools
229
226
 
230
- - `script`: one-shot sandboxed async JavaScript with `browser({ args, stdin?, timeoutMs? })` and `emit(value)`. Use it for loops, conditional page branches, or aggregation only; every invocation gets a unique non-profile session, clears ambient upstream launch controls, closes afterward, and has no host APIs or reusable-name/state surface.
231
- - `args`: exact `agent-browser` CLI tokens after the binary name. Omit when using `script`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron` instead (mutually exclusive).
232
- - `semanticAction`: optional shorthand for common `find` flows, direct selector/ref click/check/fill, and native dropdown `select`; compiles to upstream argv and is rejected together with `args`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron` on the same call.
233
- - `job`: optional constrained short-workflow schema; compiles to existing upstream `batch` args/stdin, defaults to `batch --bail` (`failFast: true`), and reports the compiled plan in `details.compiledJob`. Keep stateful jobs short around navigation, click, and rerender boundaries on dynamic apps.
234
- - `qa`: optional lightweight QA preset; compiles to the same fail-fast batch path and reports `details.compiledQaPreset` plus `details.qaPreset` pass/fail evidence.
235
- - `sourceLookup`: **EXPERIMENTAL — candidates only** for local UI-to-source hints; compiles to the same `batch` path, reports `details.compiledSourceLookup` and `details.sourceLookup`, and never reclassifies a fully successful upstream batch as failed the way `qa` can (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sourcelookup) and the longer notes below).
236
- - `networkSourceLookup`: **EXPERIMENTAL — candidates only** for failed request-to-source hints; compiles to generated `batch`, reports `details.compiledNetworkSourceLookup` and `details.networkSourceLookup`, and never assigns blame or edits files.
237
- - `electron`: optional Electron desktop-app shorthand. `list`, `status`, `cleanup`, and `probe` are wrapper-owned host/session helpers; `launch` starts a wrapper-owned isolated Electron profile and attaches through upstream `connect`.
238
- - `stdin`: top-level stdin is only for `batch`, `eval --stdin`, and `auth save --password-stdin`; other combinations are rejected before `agent-browser` is launched. `script` puts inner stdin on `browser({ stdin })`; `job`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` generate or manage their own input.
239
- - `outputPath`: optional wrapper-owned local file sink for successful results and recording receipts, including failed, pending, timed-out or recovered stops. Recording exports preserve an envelope with original attempt status and native receipt/verification (`details.outputFile.source: "recording-receipt"`); unrelated failed extractions remain unwritten. Use it for durable `eval`, `get`, `snapshot`, or diagnostic outputs, not as the destination for screenshots, downloads, recordings, or other browser artifacts; if the paths resolve to the same file, the browser artifact is preserved and the result-data write fails validation. If presentation compacted a large direct result, a result row, or the whole `batch`, the writer copies each full command-redacted pre-compaction value only from its matching live wrapper-manifest spill; if any required spill is unavailable or untrusted, it fails without writing compact metadata. `details.outputFile` reports the saved path and byte count. If caller argv includes upstream `--json`, the visible JSON content stays parseable, including early snapshot/network filters, scroll results, and preparation failures; metadata stays in `details` and the save notice is only in `details.outputFile`. Help/version keeps native text output.
240
- - `timeoutMs`: optional per-call wrapper subprocess watchdog override in milliseconds for the requested browser CLI process. Managed-session policy inspection can independently consume up to 35 seconds before that process; this preflight is intentionally not shortened by `timeoutMs` because a busy but valid daemon must remain distinguishable from an unverifiable one.
241
- - `sessionMode`:
242
- - Unconfigured ordinary calls select a root-Pi named browser with native restore; descendants share it, unrelated roots do not wait on each other, and Pi quit leaves it running. Existing managed sessions and explicit fresh launches retain their lifecycle.
243
- - Native configured `session` / `AGENT_BROWSER_SESSION` defaults select a caller-owned shared browser for ordinary calls, just like explicit `--session`, without requiring repeated flags. Such a selection wins over both modes and is not closed on Pi quit. See [shared browser defaults](../README.md#shared-browser-defaults).
244
- - `"auto"` reuses the extension-managed session when no native session is selected.
245
- - `"fresh"` rotates that managed session to a fresh upstream launch so launch-scoped flags (`--allowed-domains`, `--auto-connect`, `--args`, `--ca-cert`, `--no-ca-cert`, `--cdp`, `--enable`, `--executable-path`, `--webgpu`, `--init-script`, `--idle-timeout`, `--user-agent`, `--headed`, `--device`, `--namespace`, `--profile`, `--provider`, `-p`, `--restore`, `--restore-save`, `--restore-check-url`, `--restore-check-text`, `--restore-check-fn`, `--session-name`, `--state`) apply.
246
- - If a fresh launch fails or times out, read `details.managedSessionOutcome` for `preserved` vs `abandoned` (and related fields). A model-visible `Managed session outcome: …` line is appended for failing calls that used `sessionMode: "fresh"` and when automatic close of a replaced session fails; `"auto"` failures can still populate the struct without that extra line. If you explicitly close the current wrapper-managed session with `--session <name> close`, later default auto calls rotate to a new wrapper-generated session instead of reusing the closed name; repeated closes and branch restores keep those generated names monotonic.
247
-
248
- ### One-shot code mode
249
-
250
- Use `script` when a loop, an optional page branch, or multi-page aggregation would otherwise require several top-level tool calls and artifact reads. Source runs as an async JavaScript body. Call `await browser({ args, stdin?, timeoutMs? })`, check its `{ ok, data, details?, error?, failureCategory?, nextActions?, resultCategory, successCategory?, summary, text }` envelope, then call `emit(value)` with the one JSON-compatible value the model needs. Inner calls still traverse the ordinary wrapper executor, including policy checks, redaction, presentation, compact-spill rehydration, artifacts, and timeouts.
227
+ Call `agent_browser_tools {}` for inventory, then enable required capabilities:
251
228
 
252
229
  ```json
253
- {
254
- "script": "const titles = []; for (const url of ['https://example.com', 'https://example.org']) { const opened = await browser({ args: ['open', url] }); if (!opened.ok) throw new Error(opened.error); const title = await browser({ args: ['get', 'title'] }); if (!title.ok) throw new Error(title.error); titles.push({ url, title: title.data.title ?? title.data.result }); } emit(titles);"
255
- }
230
+ { "enable": ["action", "qa", "electron", "source", "network"] }
256
231
  ```
257
232
 
258
- The wrapper serializes inner calls, caps them at 25, caps source/final JSON at 64 KiB, defaults the whole script to 120 seconds, and rejects more than 300 seconds. Final data is redacted, compact-serialized, and byte-checked again before presentation so nesting cannot amplify small JSON into unbounded prose. Inner summaries/text are bounded, complete envelopes are checked against the IPC cap, and script-visible browser `nextActions` retain only policy-compatible calls after the isolated identity prefix is removed. It launches a separate permissioned Node child with no imports, process, filesystem, network, timers, dynamic code generation, or host object/function references. `Promise.all` is allowed for local orchestration but does not make browser calls concurrent. Pi approval covers the one visible top-level input and may therefore authorize all 25 inner calls. The collapsed Pi tool row shows a bounded terminal-safe source preview with line breaks marked as `↵`; expand the row to inspect the full terminal-safe source before approval. JavaScript CR/U+2028/U+2029 line terminators remain visible newlines, and removed terminal/directional/zero-width controls become visible markers.
259
-
260
- A script invocation uses a unique `piab-script-<uuid>` browser identity in an empty namespace with managed restore disabled. It cannot name or attach to sessions, pass `--config`, use profiles/providers/state/restore/raw launch mutation, issue lifecycle/sessionless/local commands, or nest `batch` or another top-level input mode. Every inner helper and cleanup process also clears ambient `AGENT_BROWSER_*` and standard proxy variables and uses an empty temporary native config to bypass HOME/project profile defaults before the wrapper reapplies its namespace, timeout, and compatibility values. It never replaces the implicit conversation browser and always closes its isolated session. Use ordinary `args` with a requested profile/attachment for authenticated state.
233
+ Activation is additive and uses native Pi selected-tool history. Omit CLI `--tools` for normal lazy activation, or include desired advanced tools in that explicit selection; excluded tools remain unavailable. The tools take flat input:
261
234
 
262
- Pi persistence is required because the extension appends a strict model-invisible cleanup lease before the first inner browser launch. Pi branch changes, quit, and reload abort the script and await normal isolated-session cleanup before state restoration continues. A failed close is retried on the active branch after restart and returns `failureCategory: "cleanup-failed"` plus exact `details.scriptSession.closeCommandArgs` / `close-script-session-after-cleanup-failure`. Any rejected inner policy/validation call fails the top-level result even if source consumes its envelope. See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#script) for the full schema, limits, result fields, rejected controls, and recovery semantics.
235
+ | Tool | Example |
236
+ | --- | --- |
237
+ | `agent_browser_action` | `{ "action": "select", "locator": "role", "role": "combobox", "name": "Flavor", "value": "chocolate" }` |
238
+ | `agent_browser_qa` | `{ "url": "https://example.com", "expectedText": "Example Domain" }` |
239
+ | `agent_browser_electron` | `{ "action": "list", "query": "code" }` |
240
+ | `agent_browser_source` | `{ "selector": "#save", "componentName": "SaveButton" }` |
241
+ | `agent_browser_network_source` | `{ "requestId": "req-1", "url": "/api/fail" }` |
263
242
 
264
- Code mode remains ad hoc: there is no name, registry, imported module, persistent workflow state, or versioned recipe. Keep recurring linear flows in `job`, `qa`, raw `batch`, or docs instead of building a script catalog.
243
+ Native commands remain in `agent_browser`; advanced objects are neither nested batch rows nor nested fields on that tool. If a recovery action names an inactive advanced tool, enable it before calling its exact flat payload.
265
244
 
266
245
  ### Debug, diff, stream, dashboard, and chat families
267
246
 
@@ -352,28 +331,22 @@ Examples:
352
331
  { "args": ["find", "role", "button", "click", "--name", "Close"] }
353
332
  { "args": ["find", "text", "Close", "click"] }
354
333
  { "args": ["find", "label", "Email", "fill", "user@example.com"] }
355
- { "semanticAction": { "action": "click", "locator": "role", "value": "button", "name": "Close" } }
356
- { "semanticAction": { "action": "click", "locator": "role", "role": "button", "name": "Continue without Signing In" } }
357
- { "semanticAction": { "action": "fill", "locator": "label", "value": "Email", "text": "user@example.com" } }
358
- { "semanticAction": { "action": "fill", "selector": "@e1", "text": "prompt text" } }
359
- { "semanticAction": { "action": "click", "selector": "#submit" } }
360
- { "semanticAction": { "action": "select", "selector": "#flavor", "value": "chocolate" } }
361
- { "semanticAction": { "action": "click", "locator": "text", "value": "Close", "session": "named-browser" } }
334
+ { "args": ["select", "#flavor", "chocolate"] }
362
335
  { "args": ["scrollintoview", "@e12"] }
363
336
  { "args": ["snapshot", "-i"] }
364
337
  ```
365
338
 
366
- The optional native `semanticAction` object is only a thin schema for common locator-based actions, direct selector/ref click/check/fill, and native dropdown selection; it compiles click/check/fill locator actions to existing upstream `find` commands, direct selector/ref actions to `click` / `check` / `fill`, and direct `action: "select"` to upstream `select <selector> <value...>`. Active-session role/name (`combobox` or `listbox`) and label select locators are resolved through one fresh snapshot to exactly one current visible ref before direct `select` execution; `details.compiledSemanticAction.args` reports that resolved `select @ref` argv (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction) for the full field rules). For `locator: "role"`, pass either `value: "button"` or `role: "button"`; if both are present they must match. It is a top-level alternative to `script`, `args`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron`, not a nested shape inside `batch` stdin arrays. Add `session` inside `semanticAction` when the shorthand should target a named upstream browser session; the compiled argv prepends `--session <name>` before `find`, direct selector/ref commands, or `select`, and fallback candidate actions preserve that prefix. For active sessions, role/name click/check/fill shorthands may resolve through the current `snapshot -i` refs before execution so hidden duplicate matches do not steal the action; fill only resolves when there is one exact editable current ref match. Inspect `details.effectiveArgs` when you need the exact executed argv. `semanticAction` does not expose `uncheck` because upstream `find` actions are only `click, fill, check, hover, text`; use raw `uncheck <selector-or-ref>` after choosing a stable selector or current snapshot ref. If a raw `find` or semantic action misses with `selector-not-found`, the wrapper may take one fresh snapshot and append `Current snapshot ref fallback` when that snapshot has exact visible role/name matches for the failed target. Non-fill matches can include direct `try-current-visible-ref*` next actions. Semantic click misses may also include `Agent-browser candidate fallbacks`; `details.nextActions` first recommends a fresh `snapshot -i` and may include bounded role/name retries such as `button`/`link` for a missed `text` click, each as a `try-*-candidate` entry carrying redacted `find role …` argv.
339
+ The optional `agent_browser_action` tool preserves stable accessible click/check/fill and unique visible dropdown selection. Enable `action`, then pass flat locator/selector fields. Role/name or label select resolves one current visible native combobox/listbox; it does not assume native `find ... select`. A `session` field targets a named browser. `compiledSemanticAction` remains the audit metadata name; `effectiveArgs` identifies the executed ref/command. `uncheck` remains native argv. Selector failures retain exact ref/role candidates and session-aware recovery.
367
340
 
368
341
  For desktop, contenteditable, or host-controlled rich inputs, treat a semantic `fill` miss or mismatch differently. Active-session role/name fills can execute through one exact current editable `combobox`, `searchbox`, or `textbox` ref before upstream `find` runs. If a later selector miss still finds an exact current editable ref (`searchbox` or `textbox`), `details.richInputRecovery` and visible `Rich input recovery` describe the candidate and append `focus-current-editable-ref*` / `click-current-editable-ref*` next actions. Those actions deliberately do **not** copy the fill text and never press `Enter` or submit. Direct `fill @ref <text>` on contenteditable refs may also append/prepend instead of replacing; when the latest snapshot proves the target is contenteditable, the wrapper verifies `get text` after a successful fill and appends `details.fillVerification` plus `inspect-after-fill-verification` / `verify-filled-value` if the visible text does not match. Use the safe ladder instead: refresh refs, choose the current editable `@ref`, focus or click it, then use `keyboard type` for framework-controlled editors that require real key events. `keyboard inserttext` is paste-like: it can change a DOM value without updating application state, so use it only when later application-state evidence proves the edit was accepted. Do not auto-submit unless the user flow explicitly calls for it.
369
342
 
370
343
  Do not assume Playwright selector dialects such as `text=Close` or `button:has-text('Close')` are supported wrapper syntax. In particular, current upstream can report successful `scrollintoview text=...` without moving the page, so the wrapper rejects that form before dispatch—directly or in an effective raw/stdin batch row—and shows executable `find text <label> hover` plus snapshot/ref recovery payloads in visible failure text and `details.nextActions`. `scrollintoview ... --help` and `-h` remain native help calls. Use `scrollintoview` with CSS, `xpath=...`, or a current `@e…` ref; use `find` for semantic text targets.
371
344
 
372
- Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `semanticAction` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
345
+ Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `agent_browser_action` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
373
346
 
374
347
  A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
375
348
 
376
- Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a `batch`/`job` tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close. A click that upstream rejects because another element covers the target's click point (`is covered by` … `at its click point`) remains `failureCategory: "upstream-error"`, but receives the same session-aware `inspect-overlay-state` snapshot action. This includes direct `click`, `semanticAction` click, raw `find` clicks (including `nth` and omitted default-click actions), and failed `batch`/`job` rows. That failed-click path does not retry the original click or synthesize a dismiss candidate before a fresh snapshot provides evidence; it is separate from silent input-dispatch failures.
349
+ Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a native batch tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close. A click that upstream rejects because another element covers the target's click point (`is covered by` … `at its click point`) remains `failureCategory: "upstream-error"`, but receives the same session-aware `inspect-overlay-state` snapshot action. This includes direct `click`, `agent_browser_action` click, raw `find` clicks (including `nth` and omitted default-click actions), and failed native batch rows. That failed-click path does not retry the original click or synthesize a dismiss candidate before a fresh snapshot provides evidence; it is separate from silent input-dispatch failures.
377
350
 
378
351
  ### Extract page data
379
352
 
@@ -408,65 +381,29 @@ On tabbed or hidden-DOM pages, `get text <selector>` reads the upstream-selected
408
381
 
409
382
  Use exact `batch --bail` when later steps should stop after the first failed command; omit it to continue after errors. `--bail=true` / `--bail=false` are unsupported: upstream treats them as raw command strings and ignores stdin. The wrapper returns shape guidance without running that ignored stdin. Tab recovery does not change caller flags, literal operands, or batch control flow; a failed wrapper tab selection stops before user commands. Both pinned and unpinned mixed failures retain per-step results and failure counts.
410
383
 
411
- For short constrained flows, use top-level `job` instead of hand-writing `batch` stdin. Supported job steps are `open`, `click`, `fill`, `type`, `select`, `wait`, `assertText`, `assertUrl`, `waitForDownload`, `snapshot`, and `screenshot`. `open` can include `loadState: "domcontentloaded" | "load" | "networkidle"` to insert a `wait --load …` row immediately after navigation before the next click/read step. `click` and `fill` accept either a stable `selector` or the same semantic locator fields as top-level `semanticAction` (`locator`, plus `role`/`name` or `value` as appropriate) and compile locator steps to upstream `find` argv. `type` focuses an optional selector, sends text through upstream keyboard typing, can insert `wait` rows via `delayMs` for human-paced input, and can append a final `press` key such as `Enter`; delayed typing is capped at 200 characters per step, and generated per-character rows are compacted in model-visible batch text while remaining available in `details.batchSteps`. `select` requires `selector` plus `value` or `values`, and compiles to upstream `select <selector> <value...>`. By default the wrapper compiles steps to upstream `batch --bail` so a failed setup/fill/assertion step stops later mutating clicks; set `failFast: false` only when you explicitly need continue-after-error diagnostics and those later steps remain safe if an earlier navigation fails; otherwise keep fail-fast or split navigation from content. The wrapper records `details.compiledJob.steps[]` plus `details.compiledJob.failFast`. There is still no separate first-class catalog of reusable named browser recipes above `job`, the `qa` preset, and raw `batch`; see [`ARCHITECTURE.md`](ARCHITECTURE.md#no-reusable-recipe-layer-yet) for the closed `RQ-0068` decision and revisit bar.
412
-
413
- `assertText` takes only `text`, not selector or locator fields. Clicks can stale subsequent `@refs`; split the job and take a fresh snapshot before using them.
414
-
415
- **Job navigation is explicit.** A `click` step (or other navigation-prone interaction) does not prove the next page loaded. The wrapper does not auto-insert `assertUrl` or `assertText` after clicks inside `job`; add those steps yourself with the exact URL, a `*` / `**` glob-style URL pattern, or on-page text you expect, especially after forms, checkout, tabs, or submit buttons, before screenshots or later steps. Exact and glob-style `assertUrl` values compile to `wait --url` unchanged, including query strings and literal `?`; upstream `agent-browser 0.31.1` matches `*` / `**` patterns against the full active URL. Do not put a whole dynamic checkout into one long job: split around login, sorting/cart mutations, checkout navigation, and final evidence capture so refs and app state can be rechecked between phases.
416
-
417
- ```json
418
- {
419
- "job": {
420
- "steps": [
421
- { "action": "open", "url": "https://example.com" },
422
- { "action": "assertText", "text": "Example Domain" },
423
- { "action": "screenshot", "path": ".dogfood/example.png" }
424
- ]
425
- }
426
- }
427
- ```
428
-
429
- Human-paced typing flow:
384
+ For fixed flows, keep native batch rows short around navigation and rerender boundaries. Preserve fail-fast behavior with `--bail` when later work depends on earlier success. Add `wait --url` and/or `wait --text` after navigation-prone clicks; a dispatch receipt does not prove navigation. Multiple same-snapshot fills can precede submit; split when refs must be refreshed.
430
385
 
431
386
  ```json
432
387
  {
433
- "job": {
434
- "steps": [
435
- { "action": "open", "url": "https://example.test/form" },
436
- { "action": "type", "selector": "#prompt", "text": "hello", "delayMs": 20, "press": "Enter" },
437
- { "action": "assertText", "text": "Submitted" }
438
- ]
439
- }
388
+ "args": ["batch", "--bail"],
389
+ "stdin": "[[\"open\",\"https://shop.example/checkout\"],[\"fill\",\"#email\",\"user@example.com\"],[\"click\",\"#continue\"],[\"wait\",\"--url\",\"**/shipping\"],[\"wait\",\"--text\",\"Shipping address\"],[\"screenshot\",\"/tmp/shipping.png\"]]"
440
390
  }
441
391
  ```
442
392
 
443
- Navigation-prone flow (open → fill → click → assert destination → screenshot):
393
+ Paced input uses explicit native rows, preserving the former job's behavior without assuming `--delay` equivalence:
444
394
 
445
395
  ```json
446
- {
447
- "job": {
448
- "steps": [
449
- { "action": "open", "url": "https://shop.example/checkout" },
450
- { "action": "fill", "selector": "#email", "text": "user@example.com" },
451
- { "action": "click", "selector": "#continue" },
452
- { "action": "assertUrl", "url": "**/shipping" },
453
- { "action": "assertText", "text": "Shipping address" },
454
- { "action": "screenshot", "path": ".dogfood/shipping.png" }
455
- ]
456
- }
457
- }
396
+ { "args": ["batch", "--bail"], "stdin": "[[\"focus\",\"#prompt\"],[\"keyboard\",\"type\",\"h\"],[\"wait\",\"20\"],[\"keyboard\",\"type\",\"i\"]]" }
458
397
  ```
459
398
 
460
- On app pages that expose a native dropdown, add a `select` step such as `{ "action": "select", "selector": "#flavor", "value": "chocolate" }` before the assertion that depends on it. Insert `{ "action": "snapshot" }` between mutation-prone steps when a later job row needs fresh `@refs`. On pages where stable CSS is not known, use semantic job steps such as `{ "action": "fill", "locator": "role", "role": "searchbox", "name": "Search", "text": "agent browser" }` and `{ "action": "click", "locator": "role", "role": "button", "name": "Search" }` instead of guessing selectors.
461
-
462
- Use raw `args: ["batch"]` with `stdin` when you need arbitrary upstream commands, flags, or batch failure policies outside the constrained schema. Do not pass `stdin` with `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`; those modes generate or manage their own input.
399
+ For longer text, code can construct the rows and call native batch; keep the former 200-character pacing bound when porting a job. Add a final `press` only when submission is intended. Fixed waits are not postconditions. Timeout partial progress remains available through the same batch path. See [all job step equivalents](TOOL_CONTRACT.md#07-migration).
463
400
 
464
- For quick smoke/QA checks, use top-level `qa`. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
401
+ For quick smoke/QA checks, enable `qa` and call `agent_browser_qa` with flat fields. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
465
402
 
466
- The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. Filtered data receives the same credential redaction as ordinary network diagnostics before entering `details.data` or an `outputPath` export. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
403
+ The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `agent_browser_network_source` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. Filtered data receives the same credential redaction as ordinary network diagnostics before entering `details.data` or an `outputPath` export. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
467
404
 
468
405
  ```json
469
- { "qa": { "url": "https://example.com", "expectedText": "Example Domain", "screenshotPath": ".dogfood/qa-example.png" } }
406
+ { "url": "https://example.com", "expectedText": "Example Domain", "screenshotPath": ".dogfood/qa-example.png" }
470
407
  ```
471
408
 
472
409
  Optional `loadState`, `checkNetwork`, `checkConsole`, and `checkErrors` default to `"domcontentloaded"`, `true`, `true`, and `true` for URL-opening QA; set a check to `false` to skip that diagnostic. For `qa.attached`, the diagnostic checks default to `false` because upstream buffers may predate the current check; opt in with `checkNetwork`, `checkConsole`, or `checkErrors` when preserved-buffer failures are desired. Omit `expectedText` and `expectedSelector` when you only need load plus diagnostics.
@@ -474,30 +411,32 @@ Optional `loadState`, `checkNetwork`, `checkConsole`, and `checkErrors` default
474
411
  For attached Electron or manually connected CDP sessions, use `qa.attached` after the session exists. It does not open a URL and rejects `sessionMode: "fresh"` because it checks the current managed session. Before running diagnostics, the wrapper requires a readable non-empty page URL on the attached session; missing URLs and read failures fail fast with recovery `nextActions` such as `tab list` and `snapshot -i` instead of running the full QA batch. `file:`, custom-scheme, and other attached targets are accepted. Unlike URL-opening QA, `qa.attached` preserves existing upstream network/console/page-error buffers; by default it does not inspect those buffers so stale rows do not false-fail a current-page smoke check. Set `checkNetwork`, `checkConsole`, or `checkErrors` to `true` to opt into preserved-buffer diagnostics; model-visible text and `details.compiledQaPreset.checks.diagnosticsResetAtStart` call out that preserved diagnostics may include earlier events.
475
412
 
476
413
  ```json
477
- { "qa": { "attached": true, "expectedText": "Explorer", "screenshotPath": ".dogfood/electron.png" } }
414
+ { "attached": true, "expectedText": "Explorer", "screenshotPath": ".dogfood/electron.png" }
478
415
  ```
479
416
 
480
- Use custom `job` or raw `batch` when you need a different check sequence.
417
+ Use native batch or code for a different check sequence.
481
418
 
482
419
  ### Electron desktop apps
483
420
 
421
+ Here `electron.launch` and similar labels refer to `agent_browser_electron { "action": "launch", ... }`; `qa.attached` refers to `agent_browser_qa { "attached": true, ... }`. These labels are not nested public input fields.
422
+
484
423
  Full public guide: [`ELECTRON.md`](ELECTRON.md). Use it as the entry point when Electron support is the task; this section keeps the inline workflow snippets for agents reading the broader command surface.
485
424
 
486
- Use top-level `electron` when the wrapper should discover, launch, attach to, probe, and clean up a desktop Electron app. The wrapper owns only launches it created. It uses an isolated temporary `userDataDir`, `--remote-debugging-port=0`, and safe launch defaults; it does **not** reuse the app's normal signed-in profile or attach to an already-running authenticated app. For already-authenticated desktop app content, do not stop at the isolated-launch warning: when host tools are available and the app is not already running, launch the normal app with a debug port (macOS example: `open -a Slack --args --remote-debugging-port=9222 --remote-allow-origins='*'`), verify the port, then attach with `{ "args": ["connect", "9222"], "sessionMode": "fresh" }`; if the app is already running without a debug port, ask before relaunching it. Remote debugging still exposes app content, so use caller-owned `allow` / `deny` lists for sensitive app policies when needed. `electron.list` may annotate common private-data apps as `[likely sensitive: …]`; this is advisory metadata only and does not block `launch` or replace caller policy.
425
+ Enable `electron` and use `agent_browser_electron` with flat fields when the wrapper should discover, launch, attach to, probe, and clean up a desktop Electron app. The wrapper owns only launches it created. It uses an isolated temporary `userDataDir`, `--remote-debugging-port=0`, and safe launch defaults; it does **not** reuse the app's normal signed-in profile or attach to an already-running authenticated app. For already-authenticated desktop app content, do not stop at the isolated-launch warning: when host tools are available and the app is not already running, launch the normal app with a debug port (macOS example: `open -a Slack --args --remote-debugging-port=9222 --remote-allow-origins='*'`), verify the port, then attach with `{ "args": ["connect", "9222"], "sessionMode": "fresh" }`; if the app is already running without a debug port, ask before relaunching it. Remote debugging still exposes app content, so use caller-owned `allow` / `deny` lists for sensitive app policies when needed. `electron.list` may annotate common private-data apps as `[likely sensitive: …]`; this is advisory metadata only and does not block `launch` or replace caller policy.
487
426
 
488
427
  Install scans for `electron.list` (and resolving `appName` / `bundleId` targets) are implemented for **macOS and Linux** hosts only. On **Windows**, `list` returns `platform: "unsupported"` with no apps, so prefer `executablePath` (or a host `appPath` that points at the real Electron `.exe`) when launching there—the wrapper still runs Electron evidence checks on that path before spawn.
489
428
 
490
- Typical lifecycle:
429
+ Call `agent_browser_electron` for lifecycle actions, and `agent_browser` for the native snapshot:
491
430
 
492
431
  ```json
493
- { "electron": { "action": "list", "query": "code" } }
494
- { "electron": { "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" } }
432
+ { "action": "list", "query": "code" }
433
+ { "action": "launch", "appName": "Visual Studio Code", "handoff": "snapshot" }
495
434
  { "args": ["snapshot", "-i"] }
496
- { "electron": { "action": "probe", "timeoutMs": 5000 } }
497
- { "electron": { "action": "cleanup", "launchId": "electron-…" } }
435
+ { "action": "probe", "timeoutMs": 5000 }
436
+ { "action": "cleanup", "launchId": "electron-…" }
498
437
  ```
499
438
 
500
- `electron.status` and `electron.cleanup` take either `launchId`, **`all: true`** (literal boolean) to walk every active wrapper-tracked launch (including dead, failed, or partial records, but excluding cleaned records), or neither when exactly one active launch exists—never both `launchId` and `all`. They can target the current branch-visible launch plus still-owned off-branch launch records by `launchId`; default no-arg calls are intentionally ambiguous when more than one active launch is owned. `/reload` preserves the current branch-visible active Electron launch and its isolated temp `userDataDir` for continuity, and cleans off-branch owned Electron launches. First reuse after reload/resume checks the live app's saved debug endpoint and the exact named upstream connection with `get cdp-url`, without reconnecting or resetting page refs; if cleanup is partial and skips or fails profile removal, the generic temp sweep preserves that `userDataDir` across reload, quit, later temp cleanup, process exit, and stale temp-root pruning after restart. `electron.list` has no configurable timeout and rejects both top-level and nested `timeoutMs`. For `electron.launch`, nested `timeoutMs` sets host CDP readiness polling to a **15s** default and **120s** cap after target discovery; upstream attach and handoff use separate subprocess budgets. Optional `timeoutMs` on **`status`** applies to each managed-session `get url` / `get title` read and any `get cdp-url` read needed to verify a restored connection (localhost CDP probes stay on a short fixed fetch budget). On **`cleanup`**, it is applied separately to upstream `close` and the initial host process-exit wait, not to the entire teardown; debug-port checks have fixed fetch budgets and profile removal has no configurable deadline; when omitted it follows the implicit session close default (**35s** unless `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` overrides). A successful managed-session close step retires that wrapper-managed session even when host process/profile cleanup remains partial. On **`probe`**, it bounds each underlying upstream read subprocess—omit it to use the normal tool subprocess default, or raise it on slow desktops.
439
+ `electron.status` and `electron.cleanup` take either `launchId`, **`all: true`** (literal boolean) to walk every active wrapper-tracked launch (including dead, failed, or partial records, but excluding cleaned records), or neither when exactly one active launch exists—never both `launchId` and `all`. They can target the current branch-visible launch plus still-owned off-branch launch records by `launchId`; default no-arg calls are intentionally ambiguous when more than one active launch is owned. `/reload` preserves the current branch-visible active Electron launch and its isolated temp `userDataDir` for continuity, and cleans off-branch owned Electron launches. First reuse after reload/resume checks the live app's saved debug endpoint and the exact named upstream connection with `get cdp-url`, without reconnecting or resetting page refs; if cleanup is partial and skips or fails profile removal, the generic temp sweep preserves that `userDataDir` across reload, quit, later temp cleanup, process exit, and stale temp-root pruning after restart. `agent_browser_electron` with `action: "list"` has no timeout field. For `action: "launch"`, flat `timeoutMs` sets host CDP readiness polling to a **15s** default and **120s** cap after target discovery; upstream attach and handoff use separate subprocess budgets. Optional `timeoutMs` on **`status`** applies to each managed-session `get url` / `get title` read and any `get cdp-url` read needed to verify a restored connection (localhost CDP probes stay on a short fixed fetch budget). On **`cleanup`**, it is applied separately to upstream `close` and the initial host process-exit wait, not to the entire teardown; debug-port checks have fixed fetch budgets and profile removal has no configurable deadline; when omitted it follows the implicit session close default (**35s** unless `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` overrides). A successful managed-session close step retires that wrapper-managed session even when host process/profile cleanup remains partial. On **`probe`**, it bounds each underlying upstream read subprocess—omit it to use the normal tool subprocess default, or raise it on slow desktops.
501
440
 
502
441
  Explicit-ID `electron.status` labels a cleaned record as historical while measuring PID/port liveness independently. `details.electron.statuses[].userDataDirState` freshly reports the tracked profile path as `present`, `absent` (only ENOENT), or `unknown` (other native `lstat` errors); dangling symlinks are present. This is not an audit of all app residue and does not change stored launch records or cleanup ownership.
503
442
 
@@ -518,16 +457,16 @@ A successful raw `connect` means the debug endpoint accepted the session, not th
518
457
 
519
458
  For current-session smoke checks after either path, use `qa.attached`; for compact state instead of separate title/url/focus/tab/snapshot calls, use `electron.probe`. `electron.probe.timeoutMs` bounds each underlying read subprocess; `electron.probe.launchId` ties the probe to a wrapper launch and can surface session or target mismatch guidance before you trust page refs. Electron status target reads and probe reads use the same daemon-policy lock and owned restore decision as ordinary managed commands. A probe reads and validates the live URL before title, focus, tab, or snapshot helpers. Electron `launch` snapshot/tabs handoff likewise validates the URL before tab/snapshot reads; handoff failure or cancellation closes the new managed session and host process/profile. Current-managed probe results persist their top-level namespace, tab target, and ref snapshot so Pi reload/branch replay restores the same page identity. For VS Code-style quick inputs, treat a successful `fill` as tentative: the wrapper may append `details.fillVerification` if `get value` still reads empty or different, and Electron `@e…` mutations can append `refresh-electron-refs-after-rerender` because same-URL UI rerenders commonly churn refs.
520
459
 
521
- For local app debugging, top-level `sourceLookup` can gather candidate component/file locations for a visible element from selector DOM hints, React DevTools inspection, and a bounded workspace component-name search rooted at the Pi session working directory (`maxWorkspaceFiles` defaults to 2000 and cannot exceed 5000; the scan records at most ten `workspace-search` candidates). With a `selector`, the wrapper runs `is visible` and, unless `includeDomHints` is `false`, `get html` so DOM data attributes and embedded source-like paths can become `dom-attribute` candidates. It reports evidence and confidence in `details.sourceLookup` instead of claiming a guaranteed source file. React hints require a session opened with `--enable react-devtools`. The `details.sourceLookup.status` field reads `unsupported` only when no candidates were collected **and** a `react` batch step failed (inspect errors, missing renderer, and similar); it reads `no-candidates` when the batch succeeded but nothing matched. If selector or workspace hints still yield candidates, `status` remains `candidates-found` even when React inspection failed. Unlike `qa`, the wrapper does not downgrade a **fully successful** upstream batch to `isError` solely because those statuses appear—though failed batch steps still produce normal tool errors. For wrapper-tracked packaged Electron sessions with no candidates, `details.sourceLookup.workspaceRoot` and optional `details.sourceLookup.electronContext` explain that the scan only covered the Pi tool cwd; installed app resources or `app.asar` bundles are outside that scan and are not unpacked. Those results may add `snapshot-electron-session`, `probe-electron-launch`, and `list-electron-tabs` next actions so you can inspect the live packaged app before deciding whether to change the workspace or app bundle.
460
+ For local app debugging, enable `source`; `agent_browser_source` can gather candidate component/file locations for a visible element from selector DOM hints, React DevTools inspection, and a bounded workspace component-name search rooted at the captured execution directory (`maxWorkspaceFiles` defaults to 2000 and cannot exceed 5000; the scan records at most ten `workspace-search` candidates). With a `selector`, the wrapper runs `is visible` and, unless `includeDomHints` is `false`, `get html` so DOM data attributes and embedded source-like paths can become `dom-attribute` candidates. It reports evidence and confidence in `details.sourceLookup` instead of claiming a guaranteed source file. React hints require a session opened with `--enable react-devtools`. The `details.sourceLookup.status` field reads `unsupported` only when no candidates were collected **and** a `react` batch step failed (inspect errors, missing renderer, and similar); it reads `no-candidates` when the batch succeeded but nothing matched. If selector or workspace hints still yield candidates, `status` remains `candidates-found` even when React inspection failed. Unlike `qa`, the wrapper does not downgrade a **fully successful** upstream batch to `isError` solely because those statuses appear—though failed batch steps still produce normal tool errors. For wrapper-tracked packaged Electron sessions with no candidates, `details.sourceLookup.workspaceRoot` and optional `details.sourceLookup.electronContext` explain that the scan only covered the captured execution directory; installed app resources or `app.asar` bundles are outside that scan and are not unpacked. Those results may add `snapshot-electron-session`, `probe-electron-launch`, and `list-electron-tabs` next actions so you can inspect the live packaged app before deciding whether to change the workspace or app bundle.
522
461
 
523
462
  ```json
524
- { "sourceLookup": { "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" } }
463
+ { "selector": "#save", "reactFiberId": "2", "componentName": "SaveButton" }
525
464
  ```
526
465
 
527
- Top-level `networkSourceLookup` does the same for failed browser requests. When `requestId` is set it adds `network request <requestId>`; when `filter` or `url` is set it also adds `network requests --filter …`, using `url` as the filter pattern when `filter` is omitted. Add `namespace` / `session` when the generated batch should target an explicit upstream namespace/session; `namespace: ""` explicitly selects the default namespace and overrides an ambient namespace. With `requestId` only, the compiled batch is just that request step; failed-request detection still walks the returned batch JSON and treats HTTP status ≥ 400, `failed: true`, or an `error` field as failure. When `filter` or `url` is present, the same heuristics apply but requests are correlated only if their URL matches that substring (either direction). Workspace URL literal search under the captured execution cwd reuses the `sourceLookup` scan rules (`maxWorkspaceFiles` defaults to 2000, hard cap 5000, at most ten `workspace-search` rows, up to eight URL/path needles from the query plus failed request URLs). It reports `details.networkSourceLookup.status` as `failed-requests-found`, `no-failed-requests`, or `no-candidates` and never assigns definitive blame. Request-detail URLs are diagnostic evidence, not active-tab evidence: standalone `network request …` and generated `networkSourceLookup` batches preserve the previous app page target and latest same-page `refSnapshot`.
466
+ Enable `network` and call `agent_browser_network_source` for failed browser requests. `requestId` adds `network request <requestId>`; `filter` or `url` adds `network requests --filter …`, with explicit filter winning. Optional `namespace` and `session` select the native target; `namespace: ""` overrides an ambient namespace. HTTP status ≥ 400, `failed: true`, and error fields identify failed rows. Filter/URL limits correlation by substring. Workspace URL-literal search uses the captured execution directory and the source tool's bounds: 2,000 files by default, 5,000 maximum, eight needles and ten matches. Results retain `details.networkSourceLookup` with status, failed requests, candidate evidence, and limitations. They never assign definitive blame or replace the active page target and same-page refs.
528
467
 
529
468
  ```json
530
- { "networkSourceLookup": { "requestId": "req-1", "url": "/api/fail" } }
469
+ { "requestId": "req-1", "url": "/api/fail" }
531
470
  ```
532
471
 
533
472
  ### Wait for page readiness or downloads
@@ -571,12 +510,14 @@ The upstream screenshot aliases are `screenshot --full` for full-page capture an
571
510
 
572
511
  Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. Native download always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper never substitutes a fetch of the anchor's `href`. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
573
512
 
513
+ Screenshot `imageObservations` includes verified pixel dimensions and capture kind. Matching pre/post CSS viewport, scroll, DPR, and crop samples may establish a measured document-space mapping; otherwise geometry remains unknown, including unmeasured batches and uncertain scrolled/iframe element captures. Native mouse uses current viewport CSS pixels. Account separately for crop origin, current scroll/frame, and any Pi resize before using coordinates; prefer refs. See [image geometry](TOOL_CONTRACT.md#image-observations). Code only attaches images selected with `emitImage` from a current-cell handle.
514
+
574
515
  For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
575
516
 
576
517
  Wrapper result rendering is metadata-first for saved files. Image MIME types come from a bounded header read for PNG, JPEG, GIF and WebP, never from a filename suffix; missing, unreadable, unknown or truncated headers omit `mediaType`. This identifies a format, not full image validity. Inline screenshots use the same byte check and existing size limit, so a PNG saved as `.webm` still attaches as `image/png`; other artifact kinds are not auto-inlined. An artifact-producing command fails as `artifact-missing` with artifact `status: "stale"` when the reported path's `mtimeMs` falls outside the command's bounded start/end window (with two seconds of filesystem precision tolerance), including a previous recording that `record restart` claims to finalize; clearly old or future-dated evidence is never accepted as a fresh capture. A batch, whether supplied through stdin arrays or argument command strings, must use distinct explicit artifact destinations; preflight canonicalizes existing path ancestry, compares existing file identities to catch hardlinks, and applies full Unicode plus platform case folding on macOS/Windows so aliases cannot satisfy another step's verification. The same preflight prevents `outputPath` from aliasing a same-call browser artifact, follows upstream's forward option consumption and final effective `-o` / `--output` for `diff screenshot`, and treats the optional path on `network har stop` as an artifact destination; upstream ignores positional paths on `network har start`. Outer CLI artifact parsing removes upstream global flags, so direct forms such as `record --json start <path>` and `pdf --quiet <path>` retain their native destinations. Native batch rows do not run that cleanup: `pdf --quick ignored.pdf` writes to the literal path `--quick`, and `download #link --quiet ignored.bin` writes to `--quiet`. Preflight, directory preparation, presentation, and timeout evidence use those same operands, not the ignored trailing tokens. Screenshot destination parsing mirrors upstream's exact flag matching and `[selector] [path]` positional order: `--` is positional, `true` / `false` after screenshot-only `--full` / `-f` remain positional, extra positionals are ignored after the path slot, selector-prefixed (`.`, `#`, `@`) or uppercase-extension single arguments remain selectors, and lowercase image extensions or slash-bearing arguments are paths. The wrapper deliberately keeps its existing slash-bearing hidden-workspace path normalization (for example `.dogfood/run/foo.png`) before launch. `wait --download` is observational and may verify a download that completed just before the wait began, so it is exempt from the command-window mtime gate; an explicit wait destination, in long `--download <path>` or short `-d <path>` form (including `wait --download --timeout 30000 capture.csv`), still participates in active-recording reservation preflight; the path is the next retained operand after the first timeout pair is removed; unsupported `--download=<path>` fails with split-argument guidance:
577
518
  - screenshots return a saved-path summary, visible artifact metadata, structured `details.artifacts` metadata, and an inline image attachment when safe; the visible block includes artifact type, requested path, absolute path, existence, size, cwd, session, and repair/copy status when applicable
578
519
  - downloads, PDFs, `wait --download` files, `state save` state files, diff screenshot output images, traces, CPU profiles, completed video recordings from `record stop`, and path-bearing HAR captures return concise saved-path summaries plus structured `details.artifacts` metadata without inlining large files
579
- - `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification; a legacy file without a terminal native receipt remains unverified, not saved; a missing or stale prior file fails as `artifact-missing` while the new recording remains visible as pending and the prior manifest row is retired. Within one Pi extension process, an unbounded transcript-backed index reserves active recording destinations independently of the bounded artifact manifest. Artifact lifecycle calls and result `outputPath` writes serialize around that global check; reservations use canonical namespace/session identity, survive manifest eviction and branch replay, and retire after direct, ordered nested-batch, fresh-replacement, script, Electron, or shutdown close; the newest pending row per identity is authoritative. Legacy batch replay retires a pending manifest only when the ordered close lifecycle leaves recording closed; a later successful browser reactivation plus `record start` keeps the new pending reservation. Lexical, hardlink, existing/dangling symlink, full Unicode-fold, and macOS/Windows case aliases are rejected, so `record restart` must use a distinct new path. Do not place `record start` or `record restart` after `close` / `quit` / `exit` in one batch: wrapper preflight rejects it because upstream can report success without starting a recording; split the close and recording into separate calls. A `No recording in progress` stop failure checks the matching native receipt once and preserves checked file metadata instead of assuming the path is missing; a later successful batch recording row opens its new pending path normally. Recovery offers an exact status query, and a stop only when the matching take is still current and pending. The target remains unverified until recording stops. Native 0.37 checks `ffmpeg` before starting; older supported natives may defer failure. If a successful start/restart reports pending output without `ffmpeg`, the wrapper appends `Recording dependency warning: ffmpeg not found on PATH` and `details.recordingDependencyWarning`; stop, check the result, then install the dependency before starting a new recording.
520
+ - `record start <path>` and `record restart <path>` report `successCategory: "artifact-pending"` and that output will be written on `record stop`; dispatched `record start` and URL-bearing `record restart` attempts append one `Page state:` warning on success or failure, describing conservative ref invalidation rather than an observed page change; explicit `--json` puts that warning in `warnings`. Only reached batch rows qualify, not preflight failures, missing binaries, help calls or unconfirmed planned rows — the wrapper invalidates the session’s prior ref snapshot (direct calls and batch steps alike, and even when the start fails with `Recording already active`, to protect older supported natives that can swap the page before that check), so old `@e…` refs fail as `stale-ref` until a fresh `snapshot -i` succeeds; `record restart <path> <url>` navigates the current page and invalidates refs the same way, while a restart without a URL, including FPS-only options, keeps the current page and refs; `details.artifacts` / `details.artifactVerification` mark that future file as `pending` with `recordingState: "openRecording"` and `willExistOnStop: true`, and `details.nextActions` includes exact `stop-pending-recording` args. When `record restart` returns a native `previousRecording`, that receipt's outcome and capture window control the previous artifact's verification; a legacy file without a terminal native receipt remains unverified, not saved; a missing or stale prior file fails as `artifact-missing` while the new recording remains visible as pending and the prior manifest row is retired. Within one Pi extension process, an unbounded transcript-backed index reserves active recording destinations independently of the bounded artifact manifest. Artifact lifecycle calls and result `outputPath` writes serialize around that global check; reservations use canonical namespace/session identity, survive manifest eviction and branch replay, and retire after direct, ordered nested-batch, fresh-replacement, code, Electron, or shutdown close; the newest pending row per identity is authoritative. Legacy batch replay retires a pending manifest only when the ordered close lifecycle leaves recording closed; a later successful browser reactivation plus `record start` keeps the new pending reservation. Lexical, hardlink, existing/dangling symlink, full Unicode-fold, and macOS/Windows case aliases are rejected, so `record restart` must use a distinct new path. Do not place `record start` or `record restart` after `close` / `quit` / `exit` in one batch: wrapper preflight rejects it because upstream can report success without starting a recording; split the close and recording into separate calls. A `No recording in progress` stop failure checks the matching native receipt once and preserves checked file metadata instead of assuming the path is missing; a later successful batch recording row opens its new pending path normally. Recovery offers an exact status query, and a stop only when the matching take is still current and pending. The target remains unverified until recording stops. Native 0.37 checks `ffmpeg` before starting; older supported natives may defer failure. If a successful start/restart reports pending output without `ffmpeg`, the wrapper appends `Recording dependency warning: ffmpeg not found on PATH` and `details.recordingDependencyWarning`; stop, check the result, then install the dependency before starting a new recording.
580
521
  - `batch` keeps each step's artifacts in `details.batchSteps[].artifacts`; top-level `details.artifacts` and `details.artifactManifest` coalesce an earlier pending recording into the later saved, missing, or stale terminal result for the same namespace/session identity; a successful later close retires an unfinalized recording as `close-abandoned`, uses `missing` only after a filesystem check proves absence (otherwise unverified), removes its stop action, and resets earlier ref/page/network-route batch state; a later successful `record stop` replaces that intermediate abandoned row with its verified saved artifact, and later rows—including failed rows—whose lifecycle reports a browser launch may rebuild state without triggering stale pre-close `about:blank` recovery; failed-step `batchSteps[]` retains only the bounded `lifecycle.effectiveLaunch.browserLaunched` boolean for replay, explicitly non-launching diagnostics leave the close terminal, missing lifecycle evidence remains conservatively active even on the first managed call, every successful close clears wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and any later same-session failure before recording stops keeps exact `stop-pending-recording` args alongside its normal recovery
581
522
 
582
523
  `diff screenshot` follows the file-artifact path above for the **diff** image: model-visible text and `details.artifacts` focus on that output, while baseline paths stay out of the artifact summary block, and Pi does **not** auto-inline the diff the way it inlines trusted `screenshot` captures. `state load` may print the loaded path in prose but does not add a saved-file artifact entry the way `state save` does.
@@ -655,7 +596,7 @@ If the result says `Pending confirmation id: c_8f3a1234`, choose one follow-up:
655
596
  { "args": ["deny", "c_8f3a1234"] }
656
597
  ```
657
598
 
658
- For a policy-required explicit URL read, use the returned actions: they name the actual native namespace/session, including `default`, and this routing survives resume/branch replay. The wrapper skips page helpers on matching confirm/deny only when the native read response also advertises `capabilities.readRequiresConfirmation: true`, which includes strict native ID matching. Legacy read prompts retain correct routing but normal page checks; DOM or page-content-shaped prompts never gain the exemption. Explicit caller identities and script isolation still win.
599
+ For a policy-required explicit URL read, use the returned actions: they name the actual native namespace/session, including `default`, and this routing survives resume/branch replay. The wrapper skips page helpers on matching confirm/deny only when the native read response also advertises `capabilities.readRequiresConfirmation: true`, which includes strict native ID matching. Legacy read prompts retain correct routing but normal page checks; DOM or page-content-shaped prompts never gain the exemption. Explicit caller identities and the code cell's fixed session/namespace still win.
659
600
 
660
601
  Confirmation context may be redacted when it contains credentials, tokens, cookies, or auth-bearing URLs. Replacements are marked `[REDACTED]` (URL-encoded in parsed URLs); ordinary technical phrases such as `bearer token` and `bearer authentication` stay intact outside credential fields and headers. URL scrubbing covers `code`, SAMLRequest, SAMLResponse, RelayState, `authorization_session_id`, and auth-context `state` / `nonce` while retaining ordinary non-auth query values and the spelling of URLs needing no redaction. Visible text, structured details, persisted spills, and `outputPath` exports use the same redaction; exact internal page-target URLs remain available to browser state logic. Use the id exactly as printed.
661
602
 
@@ -680,7 +621,7 @@ Operational notes:
680
621
  - Visible page content from real authenticated profiles is still model-visible and may persist in transcripts or saved artifacts. The wrapper redacts credential-like cookie/storage/auth data, not the ordinary page text you asked it to read.
681
622
  - `stdin` is accepted only for `batch`, `eval --stdin`, and `auth save --password-stdin`; other stdin-bearing calls are rejected before launch.
682
623
  - `auth list/show/save/login/delete` summaries avoid expanding profile secrets. Prefer `auth save --password-stdin` over `--password <value>`.
683
- - `session list` and `tab list` are formatted as compact field lists so caller-owned names, labels, active markers, page titles, and URLs are visible without relying on raw JSON. Wrapper-managed `piab-*` session rows are removed from `session list`.
624
+ - `session list` and `tab list` are formatted as compact field lists so caller-owned names, labels, active markers, page titles, and URLs are visible without relying on raw JSON. Wrapper-managed `piab-*` session rows remain visible in `session list`.
684
625
  - `state save <path>` is a verified file-artifact workflow; the wrapper creates missing parent directories before invoking upstream, then inspect `details.artifactVerification` before relying on the file. `state load <path>` is not treated as a newly saved artifact.
685
626
  - `cookies get` can expose real authenticated-profile cookies; prefer task-specific page actions and only inspect cookies when the user needs cookie data.
686
627
  - `storage local|session` summaries redact sensitive keys and likely secret values but may keep benign primitive local QA values visible, for example `theme: dark`; still avoid broad storage dumps unless necessary.
@@ -713,7 +654,7 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
713
654
 
714
655
  | Command | Purpose |
715
656
  | --- | --- |
716
- | `open [url]` | Launch the browser and optionally navigate. URL-less `open` stays on `about:blank` so agents can stage routes, cookies, or init scripts before first navigation. |
657
+ | `open [url]` | Navigate when given a URL. URL-less `open` runs native `get url`, launching if needed and otherwise retaining the current page. Use explicit `about:blank` when navigation to blank is intended. |
717
658
  | `open <url>` | Navigate to a URL; `goto <url>` and `navigate <url>` are equivalent navigation aliases when a URL is present. |
718
659
  | `read [url]` | Fetch agent-readable text from an explicit URL without requiring a Chrome page, or omit the URL to read rendered active-tab DOM. Supports `--raw`, `--require-md`, `--llms <index|full>`, `--outline`, `--filter <text>`, and `--timeout <ms>`. |
719
660
  | `click <sel>` | Click an element or `@ref`. |
@@ -752,7 +693,7 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
752
693
 
753
694
  On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
754
695
 
755
- Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, direct `semanticAction: { action: "select", selector, value|values }`, active-session semantic role/name or label select, or a `job` `select` step instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `semanticAction` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `semanticAction` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
696
+ Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, flat `agent_browser_action` select fields or active-session semantic role/name/label select instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `agent_browser_action` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `agent_browser_action` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
756
697
 
757
698
  ### Navigation
758
699
 
@@ -953,7 +894,7 @@ Long-running or lifecycle commands should be explicitly paired with cleanup call
953
894
  | `mcp` | Start a local MCP stdio server for external MCP clients; bare native-tool calls are rejected before spawn. External clients can opt into experimental page tools with `mcp --tools core,webmcp`. |
954
895
  | `profiles` | List available Chrome profiles. |
955
896
 
956
- When these commands are invoked through the native `agent_browser` tool, structured diagnostic/status outputs are rendered as compact summaries. `session list` and `state list` keep every upstream row and restore identifier visible. Explicit sessions, state/restore paths, broad state lifecycle commands, config, local files, and launch environment pass through unchanged. Local inspection/setup calls remain sessionless unless you explicitly pass `--session`; browser-backed or context-dependent calls keep normal managed-session behavior when no explicit session is supplied. List-like outputs such as sessions, Chrome profiles, auth profiles, network requests, console messages, and page errors include counts and key fields; large outputs are previewed with a `Full output path:` spill file instead of dumping the entire payload into context. For `network requests`, the wrapper shows a failed-request summary split into actionable versus benign low-impact rows, then status, method, URL, resource/mime type, request id, and, when the installed upstream output includes body-like fields, bounded redacted payload, response, and failure/error snippets. Safe request IDs also produce `details.nextActions` for exact request details, actionable failed-request source lookup candidates, filtered request lists, or starting HAR capture before a repro. If the same session has active wrapper-observed network routes, failed/pending/CORS-looking matched request rows add `details.networkRouteDiagnostics` and executable route-mock next actions before the generic request actions. `data:image` artifact rows are omitted from compact request previews but remain in raw `details.data.requests`. `network request <requestId>` can expose upstream full-detail body fields such as response bodies using the same bounded model-facing preview; its request URL stays diagnostic-only and does not overwrite `details.sessionTabTarget` for later ref guards. Clipboard failures that mention `NotAllowedError` or permission denial are usually browser/OS capability limits, not proof that a read, paste, or page mutation happened; prefer page-native reads (`snapshot -i`, `get text`, `eval --stdin`) or direct typing (`keyboard inserttext` / `keyboard type`) when the workflow allows it, and retry true clipboard flows only from an allowed profile/session on a normal `http(s)` page. Header, cookie, auth, token, and other secret-like fields are not expanded in model-facing text or `details.data`; low-risk primitive storage values may remain visible, while command echoes still redact `--body`, `--headers`, `--password`, proxy credentials, auth-bearing URLs, `clipboard write` text, cookie/storage set values, and bearer/basic credential text in positional arguments. Use upstream HAR or full raw details only when complete data is required.
897
+ When these commands are invoked through the native `agent_browser` tool, structured diagnostic/status outputs are rendered as compact summaries. `session list` and `state list` keep every upstream row and restore identifier visible. Explicit sessions, state/restore paths, broad state lifecycle commands, config, local files, and launch environment pass through unchanged. Local inspection/setup calls remain sessionless unless you explicitly pass `--session`; browser-backed or context-dependent calls keep normal managed-session behavior when no explicit session is supplied. List-like outputs such as sessions, Chrome profiles, auth profiles, network requests, console messages, and page errors include counts and key fields; large outputs are previewed with a `Full output path:` spill file instead of dumping the entire payload into context. For `network requests`, the wrapper shows a failed-request summary split into actionable versus benign low-impact rows, then status, method, URL, resource/mime type, request id, and, when the installed upstream output includes body-like fields, bounded redacted payload, response, and failure/error snippets. Safe request IDs also produce `details.nextActions` for exact request details, actionable failed-request source lookup candidates, filtered request lists, or starting HAR capture before a repro. If the same session has active wrapper-observed network routes, failed/pending/CORS-looking matched request rows add `details.networkRouteDiagnostics` and executable route-mock next actions before the generic request actions. `data:image` artifact rows are omitted from compact request previews but remain in raw `details.data.requests`. `network request <requestId>` can expose upstream full-detail body fields such as response bodies using the same bounded model-facing preview; its request URL stays diagnostic-only and does not overwrite `details.sessionTabTarget` for later ref guards. Clipboard failures that mention `NotAllowedError` or permission denial are usually browser/OS capability limits, not proof that a read, paste, or page mutation happened; prefer page-native reads (`snapshot -i`, `get text`, `eval --stdin`) or direct typing (`keyboard inserttext` / `keyboard type`) when the workflow allows it, and retry true clipboard flows only from an allowed profile/session on a normal `http(s)` page. Header, cookie, auth, token, and other secret-like fields are not expanded in model-facing text or `details.data`; low-risk primitive storage values may remain visible, while command echoes still redact `--body`, `--headers`, `--password`, proxy credentials, auth-bearing URLs, `clipboard write` text, cookie/storage set values, and bearer/basic credential text in positional arguments. Raw batch row strings receive the same command-aware redaction in visible output and ordered code journals without changing execution argv. Use upstream HAR or full raw details only when complete data is required.
957
898
 
958
899
  ## Optional package config and companion web search
959
900
 
@@ -1000,7 +941,7 @@ npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-conf
1000
941
  npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config browser executable set "/Applications/Brave Browser.app/Contents/MacOS/Brave Browser"
1001
942
  ```
1002
943
 
1003
- The optional `agent_browser_web_search` tool is available when Exa or Brave credentials are visible from startup config or trusted session config and the runtime config has not set `webSearch.enabled` to `false`. It is a separate custom tool, not an `agent_browser` input mode, and does not launch a browser. Prefer it for current/live external web facts and URL discovery; use `agent_browser` for browser interaction, screenshots, authenticated/profile pages, and DOM inspection after you have a target URL. Prefer it over driving public search-engine forms such as Google with browser `job`/`type` flows, which can redirect headless automation to anti-bot or CAPTCHA pages; do not attempt CAPTCHA bypass. Disable scope is explicit: `web-search disable --global` sets the normal user default, `web-search disable --project` disables it for one repo, and a `PI_AGENT_BROWSER_CONFIG` override containing `{ "version": 1, "webSearch": { "enabled": false } }` wins over both for a hard per-run disable. Loaded config may use plaintext, custom env aliases, interpolation literals, malformed-or-late-bound `$` values, and command-backed web-search keys; the resolved secret reaches the provider request while model-facing tool output and status text stay redacted. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
944
+ The optional `agent_browser_web_search` tool is available when Exa or Brave credentials are visible from startup config or trusted session config and the runtime config has not set `webSearch.enabled` to `false`. It is a separate custom tool, not an `agent_browser` input mode, and does not launch a browser. Prefer it for current/live external web facts and URL discovery; use `agent_browser` for browser interaction, screenshots, authenticated/profile pages, and DOM inspection after you have a target URL. Prefer it over driving public search-engine forms such as Google with browser typing flows, which can redirect headless automation to anti-bot or CAPTCHA pages; do not attempt CAPTCHA bypass. Disable scope is explicit: `web-search disable --global` sets the normal user default, `web-search disable --project` disables it for one repo, and a `PI_AGENT_BROWSER_CONFIG` override containing `{ "version": 1, "webSearch": { "enabled": false } }` wins over both for a hard per-run disable. Loaded config may use plaintext, custom env aliases, interpolation literals, malformed-or-late-bound `$` values, and command-backed web-search keys; the resolved secret reaches the provider request while model-facing tool output and status text stay redacted. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
1004
945
 
1005
946
  For Exa, effective search type precedence is per-call `searchType` → `webSearch.defaultSearchType` → `auto`, with regular `contents.highlights: true`. Typical latencies are `instant` ~250 ms, `fast` ~450 ms, `auto` ~1 second, `deep-lite` ~4 seconds, `deep` 4–15 seconds, and `deep-reasoning` 12–40 seconds. Prefer `deep-lite` for research before implementation, `deep` for hard multi-source work, and `deep-reasoning` only for exhaustive or still-thin research. Searches are serialized; do not launch several in parallel.
1006
947
 
@@ -1038,7 +979,7 @@ Example config:
1038
979
  }
1039
980
  ```
1040
981
 
1041
- Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions, script, or Electron launches. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
982
+ Browser defaults are scoped: global/override `defaultProfile` with `policy: "always"` and a Chrome profile name (not a path), plus `executablePath`, become native subprocess defaults only for automatic root browsers and their helpers. Roots have distinct named sessions and restore keys; parents and descendants share one root through `PI_SUBAGENT_ROOT_SESSION_ID`. These defaults never apply to unrelated explicit sessions or Electron launches. Code selecting the ordinary root browser uses the same defaults. Profile paths, other policies, and project-only guidance remain advisory. Native restore keeps cookies/localStorage/sessionStorage, not new IndexedDB, service workers, or memory-only grants; verify app access after a real restart. Avoid global native profile changes or a shared writable profile path; see [shared browser defaults](../README.md#shared-browser-defaults). Configure profile/executable guidance globally, in trusted project config, or through `PI_AGENT_BROWSER_CONFIG`. Ask the agent to run `profiles` and `doctor` when profile resolution fails, then use the reported Chrome profile directory name, a full profile/user-data directory path if upstream accepts one, or the configured `browser.executablePath` with top-level `sessionMode: "fresh"`.
1042
983
 
1043
984
  ## Important global flags, config, and environment
1044
985
 
@@ -1062,7 +1003,7 @@ Browser defaults are scoped: global/override `defaultProfile` with `policy: "alw
1062
1003
 
1063
1004
  ### Browser launch and runtime flags
1064
1005
 
1065
- The extension composes stock `--args --no-startup-window` for local Chrome startup without changing the headless default or external browser engines/attachments. Caller arguments retain native precedence. Plain active follow-ups do not repeat bootstrap settings; older custom-argument sessions can restart once when the new argument changes native launch identity. URL-less `open` runs native `get url` to lazily launch or preserve the active page, including batch/script calls; the result exposes the actual command and URL, not a fabricated launch receipt. See [the startup contract](TOOL_CONTRACT.md#headed-and-local-fixture-limits).
1006
+ The extension composes stock `--args --no-startup-window` for local Chrome startup without changing the headless default or external browser engines/attachments. Caller arguments retain native precedence. Plain active follow-ups do not repeat bootstrap settings; older custom-argument sessions can restart once when the new argument changes native launch identity. URL-less `open` runs native `get url` to lazily launch or preserve the active page, including batch/code calls; the result exposes the actual command and URL, not a fabricated launch receipt. See [the startup contract](TOOL_CONTRACT.md#headed-and-local-fixture-limits).
1066
1007
 
1067
1008
  - `--executable-path <path>`: custom Chromium-compatible browser executable, such as Brave/Edge/Arc/Vivaldi when upstream can launch that binary. Environment: `AGENT_BROWSER_EXECUTABLE_PATH`.
1068
1009
  - `--extension <path>`: load browser extensions; repeatable. Environment: `AGENT_BROWSER_EXTENSIONS`.
@@ -1083,7 +1024,7 @@ The extension composes stock `--args --no-startup-window` for local Chrome start
1083
1024
  - `--download-path <path>`: default browser download directory. Environment: `AGENT_BROWSER_DOWNLOAD_PATH`.
1084
1025
  - `--engine <name>`: browser engine, `chrome` by default or `lightpanda`. Environment: `AGENT_BROWSER_ENGINE`.
1085
1026
 
1086
- On Android/Termux, follow the README setup to install the packaged Linux-musl arm64 upstream binary, install Termux's `which`, and expose its launcher as `$PREFIX/bin/chromium`. Prefer that upstream system-browser discovery over ambient `AGENT_BROWSER_EXECUTABLE_PATH`: it survives isolated `HOME` values, works for ordinary calls and top-level `script`, and preserves the script security boundary that clears ambient launch controls and rejects inner `--executable-path` flags. Wrapper-generated Android managed identities use a compact 80-bit digest so ordinary namespaces and fresh rotations fit upstream's Unix socket path.
1027
+ On Android/Termux, follow the README setup to install the packaged Linux-musl arm64 upstream binary, install Termux's `which`, and expose its launcher as `$PREFIX/bin/chromium`. Prefer that upstream system-browser discovery over ambient `AGENT_BROWSER_EXECUTABLE_PATH`: it survives isolated `HOME` values and works for ordinary native calls. Code retains the selected browser's normal launch configuration. Wrapper-generated Android managed identities use a compact 80-bit digest so ordinary namespaces and fresh rotations fit upstream's Unix socket path.
1087
1028
 
1088
1029
  - `--no-auto-dialog`: disable automatic dismissal of alert/beforeunload dialogs. Environment: `AGENT_BROWSER_NO_AUTO_DIALOG`.
1089
1030
  - `--idle-timeout <ms>`: native background browser lifecycle setting (also accepts `10s`, `3m`, `1h`). Caller-owned sessions retain native idle policy; an explicit flag is carried to every helper in that call. Keep it consistent between calls to avoid a native daemon restart. Only wrapper-owned sessions receive the implicit timeout and numeric mismatch check against `PI_AGENT_BROWSER_IMPLICIT_SESSION_IDLE_TIMEOUT_MS`.
@@ -1120,7 +1061,7 @@ Standalone `agent-browser` looks for `agent-browser.json` in these locations, fr
1120
1061
  3. Environment variables, including `AGENT_BROWSER_CONFIG`.
1121
1062
  4. CLI flags.
1122
1063
 
1123
- Native `session` and `namespace` defaults are honored by ordinary tool calls before implicit-session generation; `sessionName` is only a legacy restore key. Per-call flags override environment, which overrides project/user JSON. `--config` or `AGENT_BROWSER_CONFIG` selects one file instead of merging the discovered files; per-call config also reaches helper subprocesses. Caller-owned native `AGENT_BROWSER_SOCKET_DIR` is honored unless the wrapper-specific socket override is set. Script alone bypasses these defaults with an empty temporary config.
1064
+ Native `session` and `namespace` defaults are honored by ordinary tool calls before implicit-session generation; `sessionName` is only a legacy restore key. Per-call flags override environment, which overrides project/user JSON. `--config` or `AGENT_BROWSER_CONFIG` selects one file instead of merging the discovered files; per-call config also reaches helper subprocesses. Caller-owned native `AGENT_BROWSER_SOCKET_DIR` is honored unless the wrapper-specific socket override is set. Code retains these native defaults while fixing the parent-selected session and namespace for the cell.
1124
1065
 
1125
1066
  Use separated `--config <path>` to load a specific upstream config; upstream 0.33.2 does not recognize `--config=<path>` as the global selector. Browser-backed and sessionless native calls preserve `--config`, `AGENT_BROWSER_CONFIG`, passive project/user config, and other upstream environment exactly as supplied. The Pi-scoped package config under `.pi/config/pi-agent-browser-native/` remains separate. Boolean flags accept optional `true` or `false` values, such as `--headed false`, `--webgpu false`, or `--no-webmcp false`, to override config. Browser extensions from user and project configs are merged rather than replaced.
1126
1067
 
@@ -1140,7 +1081,7 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
1140
1081
  - If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
1141
1082
  - If upstream reports tab_gone, the pinned bound tab is gone; use details.nextActions (tab list / tab new) instead of assuming another tab is yours.
1142
1083
  <!-- agent-browser-playbook:end wrapper-tab-recovery -->
1143
- - Wrapper-spawned commands clamp `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented 25-second default and use a 35-second child-process watchdog (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides the default 35s budget; top-level `timeoutMs` overrides it per browser CLI call). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a subprocess watchdog from the requested command duration plus a small grace window. Batch budgeting and timeout recovery read the same effective source as upstream: raw command strings when present, otherwise stdin rows. Dialog commands are additionally bounded to 5 seconds (`PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS`), and click/tap/find refs or tokens plus `eval --stdin` snippets that look like alert/confirm/prompt/dialog triggers are bounded to 8 seconds (`PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS`). A `session info` timeout returns only an exact-session `retry-session-info` status action, without page probes, liveness claims, or changes to existing page/ref state. When a browser-operation watchdog fires, `details.timeoutPartialProgress` may include a planned step list with per-step status (including `generatedFrom` labels for wrapper-inserted rows such as `open.loadState`) and a `retry-timeout-step` next action with a one-row native batch (`args: ["batch"]` plus `stdin`) only when the first incomplete step is read-only or idempotent, or `inspect-current-page-after-timeout` when the target is already verified but the incomplete step may be mutating and should not be blindly retried. If the target is unknown, standalone snapshots are removed and visible failure text plus `details.nextActions` show `verify-page-target-after-timeout`, including its session-scoped `batch --bail` args and short stdin for fail-fast `get url` then `snapshot -i`; dialog status/accept/dismiss actions remain allowed for blocking-dialog recovery. It also includes current page URL from best-effort session `get url`, followed by `get title` only after a URL is recovered (or a planned URL inferred from the step list when the session cannot answer), an `openedButPostOpenTimedOut` classification only when a live page URL was recovered before a later step hung, and declared artifact paths such as `screenshot`, `pdf`, `download`, or `wait --download` outputs with existence/state checks; the same evidence is appended under `Timeout partial progress` in visible text with URL/path redaction.
1084
+ - Wrapper-spawned commands clamp `AGENT_BROWSER_DEFAULT_TIMEOUT` to the upstream documented 25-second default and use a 35-second child-process watchdog (`PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS` overrides the default 35s budget; top-level `timeoutMs` overrides it per browser CLI call). Explicit `wait <ms>`, `wait --timeout <ms>`, and WebMCP `invoke` / `result --timeout <ms>` calls can exceed that default; when top-level `timeoutMs` is omitted, the wrapper derives a subprocess watchdog from the requested command duration plus a small grace window. Batch budgeting and timeout recovery read the same effective source as upstream: raw command strings when present, otherwise stdin rows. Dialog commands are additionally bounded to 5 seconds (`PI_AGENT_BROWSER_DIALOG_PROCESS_TIMEOUT_MS`), and click/tap/find refs or tokens plus `eval --stdin` snippets that look like alert/confirm/prompt/dialog triggers are bounded to 8 seconds (`PI_AGENT_BROWSER_DIALOG_TRIGGER_PROCESS_TIMEOUT_MS`). A `session info` timeout returns only an exact-session `retry-session-info` status action, without page probes, liveness claims, or changes to existing page/ref state. When a browser-operation watchdog fires, `details.timeoutPartialProgress` may include a planned step list with per-step status (including `generatedFrom` labels for generated QA rows) and a `retry-timeout-step` next action with a one-row native batch (`args: ["batch"]` plus `stdin`) only when the first incomplete step is read-only or idempotent, or `inspect-current-page-after-timeout` when the target is already verified but the incomplete step may be mutating and should not be blindly retried. If the target is unknown, standalone snapshots are removed and visible failure text plus `details.nextActions` show `verify-page-target-after-timeout`, including its session-scoped `batch --bail` args and short stdin for fail-fast `get url` then `snapshot -i`; dialog status/accept/dismiss actions remain allowed for blocking-dialog recovery. It also includes current page URL from best-effort session `get url`, followed by `get title` only after a URL is recovered (or a planned URL inferred from the step list when the session cannot answer), an `openedButPostOpenTimedOut` classification only when a live page URL was recovered before a later step hung, and declared artifact paths such as `screenshot`, `pdf`, `download`, or `wait --download` outputs with existence/state checks; the same evidence is appended under `Timeout partial progress` in visible text with URL/path redaction.
1144
1085
  - Oversized snapshots and oversized generic outputs may be compacted in tool content, with the full redacted output written to a spill file path shown directly in the tool result. Recent artifact metadata is bounded by `PI_AGENT_BROWSER_SESSION_ARTIFACT_MANIFEST_MAX_ENTRIES` (default 100); persisted spill files have a separate `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES` budget (default 32 MiB; `0` disables automatic eviction).
1145
1086
  - The wrapper keeps `--help` and `--version` stateless so they do not consume the implicit managed-session slot.
1146
1087