pi-agent-browser-native 0.6.14 → 0.6.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -2,6 +2,20 @@
2
2
 
3
3
  ## Unreleased
4
4
 
5
+ ## 0.6.15 - 2026-09-18
6
+
7
+ ### Fixed
8
+
9
+ - Apply ordinary credential redaction to page/origin/URL-filtered network results before structured details and `outputPath` exports.
10
+ - Leave every download's click and file transfer to native download, including loopback links, generated Blob exports, and redirects; remove the anchor-fetch shortcut that could save page HTML as an export.
11
+ - Verify click-probe candidate identity with native selector/ref attribute readback, including XPath scope; clean up and abstain when identity is unproven instead of falsely failing a completed click or recommending a duplicate action.
12
+ - Scroll CSS containers instantly before measuring movement so smooth-scroll styles cannot cause false no-movement failures.
13
+ - Preserve explicitly requested JSON for early snapshot/network filters, scroll results, and preparation failures, retaining metadata in `details` without output-file prose; help/version remains native text.
14
+
15
+ ### Tests
16
+
17
+ - Add native browser regressions for filtered credential redaction, generated Blob download bytes, accessible-name click identity, smooth container movement, and filtered snapshot JSON, alongside deterministic cleanup and early-result output checks.
18
+
5
19
  ## 0.6.14 - 2026-09-17
6
20
 
7
21
  ### Changed
package/README.md CHANGED
@@ -75,14 +75,14 @@ The result is optimized for agent work:
75
75
  |---|---|---|
76
76
  | Agents build fragile shell commands or repeat browser calls for loops and branches | Exposes `agent_browser` with one-shot sandboxed `script` orchestration, exact `args`, an optional `semanticAction` shorthand for common `find` flows and native `select`, constrained `job` / `qa` presets, experimental `sourceLookup` / `networkSourceLookup` that compile short workflows to `batch`, top-level `electron` for desktop lifecycle, plus controlled `stdin` and `sessionMode` | `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/input-modes/`, [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md) |
77
77
  | Page snapshots are too large or viewport-blind | Shows compact, main-content-first summaries, surfaces an `Omitted high-value controls` section (plus `details.data.highValueControlRefIds`) when dense pages or desktop host screens hide editables, named surfaces/tabs, primary action buttons, and high-signal named links such as repository results from the trimmed ref lists, supports wrapper-side `snapshot -i --search <text>` / `--filter role=<role>` to trim dense pages while preserving full `details.refSnapshot`, supports `snapshot --viewport` for scroll/viewport metadata, supports `snapshot --diff` for quick ref-map deltas versus the prior tracked snapshot, and stores full redacted output in spill files when needed | `extensions/agent-browser/lib/results/snapshot.ts`, `extensions/agent-browser/lib/orchestration/browser-run/prepare.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.extension-validation.test.ts` |
78
- | Screenshots/downloads get lost in text | Normalizes artifact paths, creates missing parent directories, saves simple loopback anchor downloads to the requested path when possible, and reports existence, size, cwd, session, and repair status | [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#download-screenshot-and-pdf-files) |
78
+ | Screenshots/downloads get lost in text | Normalizes artifact paths, creates missing parent directories, leaves click/download execution to native download, and reports existence, size, cwd, session, and repair status | [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#download-screenshot-and-pdf-files) |
79
79
  | Profile restores and tab drift confuse agents | Tracks managed sessions, keeps every upstream helper probe on the same idle-timeout launch configuration so the background browser is not restarted between a snapshot and action, re-selects target tabs after observed drift, live-verifies the active target and refreshes its title after successful tab selection or `tab close` (while retaining deliberate `about:blank` selections and post-close blank targets), rehydrates branch-backed session state on Pi session-tree changes, and pins later commands only for sessions with drift/restored-session risk | generated tab-recovery notes below; `test/agent-browser.extension-tab-recovery.test.ts` (drift and about:blank recovery), `test/agent-browser.extension-tabs.test.ts` (post-close target), `test/agent-browser.extension-ref-guards.test.ts` (snapshot/action environment and session-tree rehydration), `test/agent-browser.resume-state.test.ts` (persisted session / resume planning) |
80
80
  | Auth/profile workflows can leak secrets | Supports `auth save --password-stdin`, redacts sensitive args, SAML/OAuth-bearing URLs, stdout/stderr, details, and snapshot spills, and discards malformed oversized stdout instead of persisting a parse-failure spill | `test/agent-browser.extension-security-redaction.test.ts` |
81
81
  | Stateful cookies/storage/auth output bloats or leaks context | Presentation layer redacts `details.data` for cookies and credential-like storage values while keeping low-risk local QA values such as `theme: dark` readable; recursively scrubs other structured upstream JSON (network, diff, trace/profiler, stream, dashboard, chat, auth, dialog, frame, state, and similar) using sensitive key names plus string heuristics; masks sensitive argv flags and positionals; scrubs secrets from failed batch step errors; and exposes a compact redacted `batch` matrix on top-level `details.data` | `extensions/agent-browser/lib/results/presentation.ts`, `extensions/agent-browser/lib/results/presentation/diagnostics.ts`, `extensions/agent-browser/lib/runtime.ts`, `test/agent-browser.presentation-diagnostics.test.ts` |
82
82
  | Stale `@eN` refs fail mysteriously | Records per-session `details.refSnapshot`, rejects mismatched URLs / unknown refs / unsafe `batch` stdin ordering before spawn, adds recovery guidance to rerun `snapshot -i` or use stable `find` locators | `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/session-page-state.ts`, `test/agent-browser.session-page-state.test.ts`, `test/agent-browser.results.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.extension-semantic-recovery.test.ts` |
83
83
  | Agents need stable success/failure buckets | Exposes bounded `resultCategory`, `successCategory`, and `failureCategory` on tool `details` for branching without parsing prose; a `tool_result` hook also aligns real Pi `isError` semantics, naming `Pi tool isError: true` in prose output while preserving parseable caller-requested `--json` output | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), `extensions/agent-browser/lib/results/categories.ts`, `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/pi-tool-rendering.ts`, `test/agent-browser.results.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.pi-pipeline.test.ts` |
84
- | Clicks can report success without the page receiving the event | Top-level non-Electron direct `click` calls on `xpath=` targets or role-gated current `@e…` refs (`button`, `checkbox`, `menuitem`, `radio`, `switch`, `tab`) install a bounded target-specific DOM-event probe; eligible `@e…` refs require a unique role/name in both the latest snapshot and the live candidates. Duplicate-name refs pass through without a probe: their old ordinal cannot prove which element received the click. If upstream reports success but no trusted event reaches the resolved target, the wrapper fails the tool, exposes `details.clickDispatch`, and suggests explicit retry/inspect next actions (no in-page replay), including a nested-scroll `scrollintoview` action when the probe sees the target outside a scroll container or viewport. Unresolved locator clicks such as raw `find … click` are left upstream-owned to avoid false failures for frame-scoped targets. Other click results still expose `details.pageChangeSummary`; `observed: false` explicitly marks dispatch-only mutation summaries and adds a visible `Action dispatched; application change unverified` warning. Unchanged-URL clicks can surface evidence-backed `details.overlayBlockers` candidates. | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), `extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.ts`, `extensions/agent-browser/lib/results/presentation/navigation.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.extension-click-dispatch.test.ts` |
85
- | Dashboard scroll commands can look successful while nothing moves | Handles standard `scroll <dir> [px]` against the document first (including pages whose smooth-scroll CSS defeats upstream wheel timing), falls back upstream when the document cannot move, and samples viewport/containers around the fallback; unchanged positions fail as `upstream-error` with `details.scrollNoop`, visible recovery guidance, and exact snapshot/screenshot checks. Unsupported `scrollintoview text=...` fails before dispatch, including inside effective batch rows, and shows exact native `find text ... hover` and snapshot/ref recovery payloads; help remains native pass-through. | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#core-page-and-element-commands), `test/agent-browser.extension-validation.test.ts` |
84
+ | Clicks can report success without the page receiving the event | Top-level non-Electron direct `click` calls on `xpath=` targets or role-gated current `@e…` refs (`button`, `checkbox`, `menuitem`, `radio`, `switch`, `tab`) install a bounded target-specific DOM-event probe; eligible `@e…` refs require a unique role/name in both the latest snapshot and the live candidates. A per-probe temporary DOM marker must be read back by native `get attr <original-selector> <marker>` before the candidate can verify dispatch, including XPath scope. Missing or mismatched identity cleans up the probe and leaves the native click unverified, not failed. Duplicate-name refs pass through without a probe: their old ordinal cannot prove which element received the click. If upstream reports success but no trusted event reaches the resolved target, the wrapper fails the tool, exposes `details.clickDispatch`, and suggests explicit retry/inspect next actions (no in-page replay), including a nested-scroll `scrollintoview` action when the probe sees the target outside a scroll container or viewport. Unresolved locator clicks such as raw `find … click` are left upstream-owned to avoid false failures for frame-scoped targets. Other click results still expose `details.pageChangeSummary`; `observed: false` explicitly marks dispatch-only mutation summaries and adds a visible `Action dispatched; application change unverified` warning. Unchanged-URL clicks can surface evidence-backed `details.overlayBlockers` candidates. | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), `extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.ts`, `extensions/agent-browser/lib/results/presentation/navigation.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.extension-click-dispatch.test.ts` |
85
+ | Dashboard scroll commands can look successful while nothing moves | Handles standard `scroll <dir> [px]` against the document first (including pages whose smooth-scroll CSS defeats upstream wheel timing), falls back upstream when the document cannot move, and samples viewport/containers around the fallback; explicit CSS-container scrolling uses instant movement so smooth-scroll CSS cannot cause a premature no-movement failure; unchanged positions fail as `upstream-error` with `details.scrollNoop`, visible recovery guidance, and exact snapshot/screenshot checks. Unsupported `scrollintoview text=...` fails before dispatch, including inside effective batch rows, and shows exact native `find text ... hover` and snapshot/ref recovery payloads; help remains native pass-through. | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#core-page-and-element-commands), `test/agent-browser.extension-validation.test.ts` |
86
86
  | Dropdown/combobox clicks can focus or hit native option box-model errors | Adds first-class `select <selector> <value...>` paths through raw `args`, `job`, and `semanticAction`; semantic role/name or label select resolves exactly one current visible combobox/listbox ref before action. Custom combobox clicks still detect focused controls with explicit `aria-expanded` state but no visible options and return `details.comboboxFocus` plus exact recovery `nextActions` | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#core-page-and-element-commands), `extensions/agent-browser/lib/input-modes/semantic-action.ts`, `test/agent-browser.extension-input-modes.test.ts`, `test/agent-browser.extension-validation.test.ts` |
87
87
  | Recording workflows fail late when `ffmpeg` is missing or report stale lifecycle state | After successful `record start` / `record restart`, reports `successCategory: "artifact-pending"`, returns an exact `stop-pending-recording` action, warns when older natives report pending output without `ffmpeg`, and conservatively invalidates prior page-scoped `@e…` refs on every executed start attempt and URL-bearing restart to protect older supported natives. This is not evidence of a page change: 0.37 records the active page unless a URL is supplied; FPS-only calls keep the intended tab; an unbounded transcript-backed namespace/session index reserves active destinations across aliases, serializes artifact lifecycle and explicit wait/output writes, persists cross-branch close tombstones, retires every successful close path (including every matching namespace owner for `close --all`), rejects missing/stale restart output, coalesces terminal batch state, keeps only the newest pending path per identity, rejects recording starts after a nested close, folds Unicode path aliases, and retains exact cleanup actions with visible guidance on any later same-session failure | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#diff-debug-and-streaming), `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.presentation-artifacts-batch.test.ts` |
88
88
  | Upstream CLI drift can silently invalidate wrapper behavior | Publishes a repo-readable command reference, verifies it against the recommended 0.38.1 target, and probes browser-backed calls once per cwd/PATH so stable versions below the 0.35.0 floor fail before browser launch with installed/expected version evidence | `npm run verify` |
@@ -310,7 +310,7 @@ This adds agent guidance for signed-in/account-specific tasks; current releases
310
310
 
311
311
  You usually prompt the agent in natural language. These JSON snippets show the exact native tool shape the agent should use.
312
312
 
313
- Open a page and inspect it (first-call recipe: open → snapshot -i → interact with current `@refs` → snapshot -i after changes). Do not pass `--json` in `args`; the wrapper injects it.
313
+ Open a page and inspect it (first-call recipe: open → snapshot -i → interact with current `@refs` → snapshot -i after changes). Omit `--json` unless you need JSON text; structured details are always available.
314
314
 
315
315
  ```json
316
316
  { "args": ["open", "https://example.com"] }
@@ -395,7 +395,7 @@ Evaluate page JavaScript through stdin. Put the script in the top-level `stdin`
395
395
  { "args": ["eval", "--stdin"], "stdin": "({ title: document.title, url: location.href })", "outputPath": "logs/page-state.json" }
396
396
  ```
397
397
 
398
- Use `outputPath` when `eval`, `get`, `snapshot`, or another extraction should be saved as a durable workspace file. Recording results also export on failure or timeout: their JSON envelope retains the failed attempt, native receipt, verification and any recovery evidence rather than writing misleading bare success data. Keep it distinct from screenshot, download, recording, and other browser artifact destinations; preflight rejects known same-call aliases before browser activity, and the result writer preserves the browser artifact if an alias becomes apparent only afterward. The wrapper writes `details.data` when present, otherwise the model-facing text content. When presentation compacted a large direct result, a result row, or the whole `batch`, it instead reads the full command-redacted pre-compaction payload only from the corresponding live wrapper-managed spill recorded in `details.artifactManifest`; if any required spill is unavailable or untrusted, the call fails without writing compact metadata to the requested path. `details.outputFile` reports the saved path and byte count. Explicit upstream `--json` content stays parseable; in that case the save notice lives only in `details.outputFile`.
398
+ Use `outputPath` when `eval`, `get`, `snapshot`, or another extraction should be saved as a durable workspace file. Recording results also export on failure or timeout: their JSON envelope retains the failed attempt, native receipt, verification and any recovery evidence rather than writing misleading bare success data. Keep it distinct from screenshot, download, recording, and other browser artifact destinations; preflight rejects known same-call aliases before browser activity, and the result writer preserves the browser artifact if an alias becomes apparent only afterward. The wrapper writes `details.data` when present, otherwise the model-facing text content. When presentation compacted a large direct result, a result row, or the whole `batch`, it instead reads the full command-redacted pre-compaction payload only from the corresponding live wrapper-managed spill recorded in `details.artifactManifest`; if any required spill is unavailable or untrusted, the call fails without writing compact metadata to the requested path. `details.outputFile` reports the saved path and byte count. Explicit upstream `--json` content stays parseable, including early snapshot/network filters, scroll results, and preparation failures; metadata stays in `details` and the save notice lives only in `details.outputFile`. Help/version output remains native text.
399
399
 
400
400
  Recording receipts separate captured-frame rate and wall-clock capture duration from nominal/output FPS. `details.artifacts[].recording` includes native capture timestamps, received frame counts (not pixel-unique frames), encoded/written/held/dropped/skipped counts, and output duration. Missing native metrics stay unknown. Repeated, static, sparse or final-state-only frames cannot establish UI smoothness.
401
401
 
@@ -459,7 +459,7 @@ For supported upstream `find` flows, direct selector/ref `click` / `check` / `fi
459
459
  Typical pitfalls:
460
460
 
461
461
  - Supply **exactly one** of `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron` per call (not more, not none). Prefer `script` only for one-shot loops/branches/aggregation, `args` for routine browse; `semanticAction` for stable locators; `job`/`qa` for multi-step checks; `electron` for desktop apps; treat `sourceLookup` / `networkSourceLookup` as experimental candidates-only.
462
- - Do not pass `--json` in `args`; the wrapper injects it automatically.
462
+ - Omit `--json` for ordinary prose; include it when you need parseable JSON text.
463
463
  - `semanticAction` and `job` are **not** valid inside `batch` stdin; batch steps stay upstream argv string arrays (spell a `find` step as tokens there if you need it in a batch).
464
464
  - Commands or locators outside the supported shorthand still require explicit `args`. Common page getters are grouped under `get`: use `get title`, `get url`, or `get text <selector>` rather than shortcut commands such as `title` or `url`; unknown getter shortcuts can return read-only `details.nextActions` like `use-get-title`.
465
465
  - For `locator: "role"`, pass either `value: "button"` or `role: "button"`; if both are present they must match.
@@ -467,7 +467,7 @@ Typical pitfalls:
467
467
  - Do not reuse `@e…` refs across navigation or in-place rerenders. The wrapper records the latest snapshot refs per session and fails stale/recycled getter and mutation refs, including batched getters, before upstream can silently read or hit a different current-page element; use the session-aware `refresh-interactive-refs` next action.
468
468
  - If upstream classifies the failure as `stale-ref` and `details.compiledSemanticAction` is present for a compiled `find` action, `details.nextActions` may list `retry-semantic-action-after-stale-ref` after `refresh-interactive-refs`, carrying the same compiled `find` argv so you can retry the locator-stable target once it is safe to do so. `select` calls that used stale `@refs` only get refresh guidance; use a fresh snapshot or stable selector before retrying (contract in [`docs/TOOL_CONTRACT.md#semanticaction`](docs/TOOL_CONTRACT.md#semanticaction)).
469
469
  - If the failure is `selector-not-found`, the wrapper may take one fresh snapshot and add `Current snapshot ref fallback` when that snapshot has exact visible role/name matches for the failed `find` / `semanticAction` target. Non-fill targets can include direct `try-current-visible-ref*` next actions, and semantic click misses can still add bounded `Agent-browser candidate fallbacks` such as `button`/`link` role retries for `text` clicks. `semanticAction` does not expose `uncheck` while upstream `find ... uncheck` is not runtime-supported; use raw `args: ["uncheck", <selector-or-ref>]` after a stable selector or fresh snapshot ref. For semantic `fill` misses on desktop or host-controlled rich inputs, prefer `details.richInputRecovery`: refresh refs, choose the current editable `@ref`, focus or click it, then use `keyboard type` for framework-controlled editors that require real key events. `keyboard inserttext` is paste-like and can change a DOM value without updating application state, so use it only with separate application-state verification. Direct contenteditable fills are verified with `get text` when snapshot metadata proves the target is contenteditable; if replacement did not happen, `details.fillVerification` warns before any submit step. Those recovery nextActions do not copy the fill text and do not press `Enter` or submit; only submit when the user flow explicitly calls for it (same contract link).
470
- - A successful upstream `click` is not proof that the web app handled the event or changed state. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper may fail the tool with `details.clickDispatch` and a `Click dispatch diagnostic` line when upstream reported success but no trusted DOM event reached the resolved target. Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. `@e…` ref click probes are limited to current snapshot refs with accessible role `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. Duplicate-name refs remain upstream-owned; snapshot order is not target identity. Use the suggested `inspect-click-dispatch-miss` / `retry-click-after-dispatch-miss` next actions instead of assuming the click mutated the page; when `details.clickDispatch.scrollContainer` is present, use `scroll-target-into-view-after-dispatch-miss` first. When the task depends on a mutation, follow `inspect-after-mutation` / `pageChangeSummary` evidence with a wait, URL/text check, or fresh snapshot before trusting the result; if the target still did not change, retry with a current visible ref or stable selector and report the workflow issue instead of silently continuing. For static local fixtures where the user only needs to exercise app code, an explicit `eval --stdin` programmatic click such as `document.querySelector("#demo").click()` can be a diagnostic workaround, but treat it as an untrusted scripted activation rather than proof a real user click works, and never use it to bypass user instructions. Respect explicit user stop boundaries yourself: if the user says to stop before order/post/purchase/submit, gather evidence on that page and do not click the final action. The wrapper does not parse broad prompt text into business-intent action blocks; `details.promptGuard` is reserved for concrete artifact-before-close checks.
470
+ - A successful upstream `click` is not proof that the web app handled the event or changed state. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper may fail the tool with `details.clickDispatch` and a `Click dispatch diagnostic` line when upstream reported success but no trusted DOM event reached the resolved target. Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. `@e…` ref click probes are limited to current snapshot refs with accessible role `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. Ref and XPath candidates must also match native `get attr <original-selector> <marker>` readback of a per-probe temporary DOM marker. If identity is missing or mismatched, the wrapper cleans up and abstains from dispatch verification while the native click runs. Duplicate-name refs remain upstream-owned; snapshot order is not target identity. Use the suggested `inspect-click-dispatch-miss` / `retry-click-after-dispatch-miss` next actions instead of assuming the click mutated the page; when `details.clickDispatch.scrollContainer` is present, use `scroll-target-into-view-after-dispatch-miss` first. When the task depends on a mutation, follow `inspect-after-mutation` / `pageChangeSummary` evidence with a wait, URL/text check, or fresh snapshot before trusting the result; if the target still did not change, retry with a current visible ref or stable selector and report the workflow issue instead of silently continuing. For static local fixtures where the user only needs to exercise app code, an explicit `eval --stdin` programmatic click such as `document.querySelector("#demo").click()` can be a diagnostic workaround, but treat it as an untrusted scripted activation rather than proof a real user click works, and never use it to bypass user instructions. Respect explicit user stop boundaries yourself: if the user says to stop before order/post/purchase/submit, gather evidence on that page and do not click the final action. The wrapper does not parse broad prompt text into business-intent action blocks; `details.promptGuard` is reserved for concrete artifact-before-close checks.
471
471
  - A successful upstream `click` can deliver no input at all on some sites. Reproduced against `https://www.saucedemo.com/` with upstream `agent-browser` 0.34.0 and no wrapper involved: when every CLI invocation in the flow is spaced ~0.6s apart, the React add-to-cart click reports success while **zero** DOM events reach a capture-phase listener and the cart badge never updates (0/5 runs succeeded, versus 4/5 with no spacing). A single idle gap is harmless (5/5); only sustained spacing across the flow triggers it. In that state a scripted `document.querySelector(...).click()` still works and updates the badge, so the page and its handlers are fine and the input never arrives; retries, `scrollintoview`, headed mode, and re-navigation do not recover it. This looks site-specific — React TodoMVC and react.dev were unaffected under the same pacing and command count. Wrapper calls do more per-command work than raw CLI calls, so they sit in the slow regime more often. Running the interaction steps as one `batch` keeps them inside a single upstream process and succeeded 5/5; prefer `batch` for consecutive real-app click flows, and treat a click whose `pageChangeSummary`/`navigationSummary` shows no change as unproven. When a `wait --url` or `assertUrl` times out after a suspected missed click, use the `fresh-session-after-url-wait-timeout` next action (`sessionMode: "fresh"` + `open about:blank`): replace about:blank with the target URL and replay the flow as one batch in a fresh session instead of retrying the wait.
472
472
  - A successful `snapshot -i` can surface `Possible overlay blockers` immediately when refs already contain strong dialog/alertdialog evidence plus close/dismiss controls. If a **top-level** `@e…`/`ref=` click succeeds (unified command `click`, not a `batch` step), upstream reports `data.clicked`, and `details.navigationSummary.url` stays on the same tab URL under the same normalization as ref preflight (fragment-insensitive), the wrapper may take one extra `snapshot -i` and add `Possible overlay blockers` with `details.overlayBlockers` (`candidates`, `summary`, optional `snapshot` refresh for refs) plus session-aware `inspect-overlay-state` / bounded `try-overlay-blocker-candidate-*` next actions when that snapshot shows strong modal context (`dialog` / `alertdialog`) and close/dismiss-like controls. Page-wide words like privacy, sign in, or banner alone do not trigger this diagnostic. The unchanged-URL check compares the prior pinned tab target with `details.navigationSummary.url`; CSS selector clicks do not run this overlay probe. Also skipped when tab correction or about-blank recovery already ran on that result. Separately, when upstream rejects a click because another element covers the target's click point (`is covered by` … `at its click point`), the wrapper keeps `failureCategory: "upstream-error"` and adds only the session-aware `inspect-overlay-state` snapshot action; it does not retry the blocked click or guess a dismiss control without fresh snapshot evidence. This covers direct `click`, `semanticAction` click, raw `find` clicks (including `nth` and an omitted action that defaults to click), and failed `batch`/`job` rows; it does not diagnose silent input-dispatch failures.
473
473
  - If `get text <selector>` reads a non-ref, non-simple-id CSS selector with multiple matches or a hidden first match while visible matches exist, including successful `batch` steps, the wrapper may add `Selector text visibility warning`, `details.selectorTextVisibility` (plus `selectorTextVisibilityAll` for multiple batched warnings), and `inspect-visible-text-candidates` next actions; the warning names the matching `details.nextActions` id. Prefer a visible `@ref`, a scoped selector, or a targeted `eval --stdin` over hidden tab content.
@@ -570,7 +570,7 @@ For local app debugging, `sourceLookup` can gather candidate component/file loca
570
570
 
571
571
  This is an experiment, not a guarantee. React hints require a session opened with `--enable react-devtools`, and many builds do not expose useful sourcemap/source metadata; `status: "no-candidates"` is common when nothing matched, and `status: "unsupported"` only when no candidates were found **and** a compiled `react` batch step failed (if DOM or workspace search still produced candidates, you get `candidates-found` instead). For wrapper-tracked packaged Electron apps, a no-candidate result includes `details.sourceLookup.workspaceRoot`, optional `details.sourceLookup.electronContext`, limitations explaining that the scan is limited to the Pi cwd and does not unpack app bundles/`app.asar`, plus Electron snapshot/probe/tab next actions when a launch is known.
572
572
 
573
- `networkSourceLookup` is the matching failed-request experiment. It runs `network request <id>` when `requestId` is present and/or `network requests --filter …` when `filter` or `url` is present (`url` supplies the filter pattern when `filter` is omitted); add `namespace` / `session` when the generated batch should target an explicit upstream namespace/session (`namespace: ""` explicitly selects the default namespace and overrides an ambient namespace). It merges failed-request rows from the batch JSON with initiator-style hints and a bounded workspace literal scan (`maxWorkspaceFiles` defaults to 2000, cap 5000), surfaces everything under `details.networkSourceLookup`, and avoids automatic blame or edits. Compact `network requests` results with safe request IDs also add `details.nextActions` for request details, bounded `networkSourceLookup` on actionable failures, path filtering, diagnostic-buffer clearing before a repro, or HAR capture so agents can branch without guessing request-id syntax. For noisy aggregate buffers, wrapper-side `network requests --current-page` / `--current-origin` keeps only rows matching the active page origin, while `--current-url` keeps exact active-document URL rows and reports counts in `details.networkRequestsPageFilter`. When the wrapper has seen `network route` in the same session, pending fetch/XHR rows or CORS-looking errors that match the route surface `details.networkRouteDiagnostics` plus executable follow-ups to inspect the request or start HAR capture; same-origin/CORS-correct fixture retry guidance stays in prose. Network diagnostics are read-only for wrapper page state: request URLs in `network request` or generated `networkSourceLookup` batches do not replace the session’s active page target or invalidate page-scoped refs from the app page.
573
+ `networkSourceLookup` is the matching failed-request experiment. It runs `network request <id>` when `requestId` is present and/or `network requests --filter …` when `filter` or `url` is present (`url` supplies the filter pattern when `filter` is omitted); add `namespace` / `session` when the generated batch should target an explicit upstream namespace/session (`namespace: ""` explicitly selects the default namespace and overrides an ambient namespace). It merges failed-request rows from the batch JSON with initiator-style hints and a bounded workspace literal scan (`maxWorkspaceFiles` defaults to 2000, cap 5000), surfaces everything under `details.networkSourceLookup`, and avoids automatic blame or edits. Compact `network requests` results with safe request IDs also add `details.nextActions` for request details, bounded `networkSourceLookup` on actionable failures, path filtering, diagnostic-buffer clearing before a repro, or HAR capture so agents can branch without guessing request-id syntax. For noisy aggregate buffers, wrapper-side `network requests --current-page` / `--current-origin` keeps only rows matching the active page origin, while `--current-url` keeps exact active-document URL rows and reports counts in `details.networkRequestsPageFilter`. Filtered results use the same credential redaction as ordinary network diagnostics before populating `details.data` or writing `outputPath`. When the wrapper has seen `network route` in the same session, pending fetch/XHR rows or CORS-looking errors that match the route surface `details.networkRouteDiagnostics` plus executable follow-ups to inspect the request or start HAR capture; same-origin/CORS-correct fixture retry guidance stays in prose. Network diagnostics are read-only for wrapper page state: request URLs in `network request` or generated `networkSourceLookup` batches do not replace the session’s active page target or invalidate page-scoped refs from the app page.
574
574
 
575
575
  ```json
576
576
  { "networkSourceLookup": { "requestId": "req-1", "url": "/api/fail" } }
@@ -583,7 +583,7 @@ For asynchronous exports, use the export control's current snapshot ref (for exa
583
583
  { "args": ["wait", "--download", "/tmp/report.csv"] }
584
584
  ```
585
585
 
586
- When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute a different path in the final report. The wrapper creates missing parent directories for direct artifact paths such as `state save`, screenshots, PDFs, downloads, and `wait --download`. For simple loopback `download <selector> <path>` anchor links with HTTP(S) `href`, it can save the in-page response directly to the requested path before falling back to upstream click/download behavior; non-loopback/profile downloads stay upstream-owned. With current upstream `agent-browser`, treat `details.savedFilePath` as upstream-reported metadata and confirm `details.artifacts[].exists` / `details.artifactVerification.verified` before relying on the requested `wait --download <path>` file being present on disk; non-file download payloads such as `data:` URLs are not verified local artifacts.
586
+ When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute a different path in the final report. The wrapper creates missing parent directories for direct artifact paths such as `state save`, screenshots, PDFs, downloads, and `wait --download`. Native `download <selector> <path>` always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper does not fetch an anchor's `href` instead of clicking it. With current upstream `agent-browser`, treat `details.savedFilePath` as upstream-reported metadata and confirm `details.artifacts[].exists` / `details.artifactVerification.verified` before relying on the requested `wait --download <path>` file being present on disk; non-file download payloads such as `data:` URLs are not verified local artifacts.
587
587
 
588
588
  For evidence-only screenshots or QA captures, branch on `details.artifactVerification` and `details.artifacts` before reporting PASS/FAIL; a pre-existing path from an artifact-producing command that was not updated during the command fails as `status: "stale"` instead of being accepted as fresh evidence (`wait --download` remains observational). Inline image attachments are optional when size limits allow—do not require vision review unless the user asked for visual inspection. If the latest prompt names exact required artifact paths, browser close can be blocked with `details.promptGuard` until those artifacts are saved and verified.
589
589
 
@@ -31,6 +31,9 @@ function getClickDispatchProbeTarget(commandTokens, refSnapshot) {
31
31
  function getEvalResultRecord(data) {
32
32
  return isRecord(data) && isRecord(data.result) ? data.result : undefined;
33
33
  }
34
+ function getClickDispatchIdentityAttribute(probe) {
35
+ return `data-pi-click-dispatch-${probe.marker.toLowerCase()}`;
36
+ }
34
37
  function buildClickDispatchProbeInstallScript(probe) {
35
38
  const target = probe.target;
36
39
  const resolveTarget = target.kind === "selector"
@@ -124,8 +127,14 @@ const listeners = eventTypes.map((type) => {
124
127
  document.addEventListener(type, listener, true);
125
128
  return [type, listener];
126
129
  });
127
- state.cleanup = () => listeners.forEach(([type, listener]) => document.removeEventListener(type, listener, true));
130
+ const identityAttribute = ${JSON.stringify(getClickDispatchIdentityAttribute(probe))};
131
+ state.removeIdentityMarker = () => element.removeAttribute(identityAttribute);
132
+ state.cleanup = () => {
133
+ state.removeIdentityMarker();
134
+ listeners.forEach(([type, listener]) => document.removeEventListener(type, listener, true));
135
+ };
128
136
  window[marker] = state;
137
+ element.setAttribute(identityAttribute, marker);
129
138
  return { status: "installed", marker, target: state.target };
130
139
  })()`;
131
140
  }
@@ -200,9 +209,33 @@ export async function prepareClickDispatchProbe(options) {
200
209
  if (!target)
201
210
  return undefined;
202
211
  const probe = { marker: `${CLICK_DISPATCH_MARKER_PREFIX}${Date.now().toString(36)}_${Math.random().toString(36).slice(2)}`, target };
203
- const installData = await runSessionCommandData({ args: ["eval", "--stdin"], cwd: options.cwd, namespace: options.namespace, sessionName: options.sessionName, signal: options.signal, stdin: buildClickDispatchProbeInstallScript(probe) });
204
- const installResult = getEvalResultRecord(installData);
205
- return installResult?.status === "installed" ? probe : undefined;
212
+ let prepared = false;
213
+ try {
214
+ const installData = await runSessionCommandData({ args: ["eval", "--stdin"], cwd: options.cwd, namespace: options.namespace, sessionName: options.sessionName, signal: options.signal, stdin: buildClickDispatchProbeInstallScript(probe) });
215
+ if (getEvalResultRecord(installData)?.status !== "installed")
216
+ return undefined;
217
+ // Name matching and in-page XPath only find candidates. Native resolution
218
+ // must prove identity, including frame scope, before missing events mean failure.
219
+ const selector = target.kind === "accessible" ? `@${target.refId}` : target.kind === "xpath" ? `xpath=${target.selector}` : target.selector;
220
+ const identity = await runSessionCommandData({ args: ["get", "attr", selector, getClickDispatchIdentityAttribute(probe)], cwd: options.cwd, namespace: options.namespace, sessionName: options.sessionName, signal: options.signal });
221
+ if (!isRecord(identity) || identity.value !== probe.marker)
222
+ return undefined;
223
+ const removed = await runSessionCommandData({ args: ["eval", "--stdin"], cwd: options.cwd, namespace: options.namespace, sessionName: options.sessionName, signal: options.signal, stdin: `(() => {
224
+ const state = window[${JSON.stringify(probe.marker)}];
225
+ if (!state || typeof state.removeIdentityMarker !== "function") return { status: "probe-missing" };
226
+ state.removeIdentityMarker();
227
+ return { status: "identity-marker-removed" };
228
+ })()` });
229
+ if (getEvalResultRecord(removed)?.status !== "identity-marker-removed")
230
+ return undefined;
231
+ prepared = !options.signal?.aborted;
232
+ return prepared ? probe : undefined;
233
+ }
234
+ finally {
235
+ // Even a lost/aborted install response may have left listeners in-page.
236
+ if (!prepared)
237
+ await cleanupClickDispatchProbe({ ...options, probe });
238
+ }
206
239
  }
207
240
  function getClickDispatchScrollContainerDiagnostic(result) {
208
241
  const target = isRecord(result.target) ? result.target : undefined;
@@ -1,12 +1,12 @@
1
1
  import { runAgentBrowserProcess, withAttachedBrowserSessionContext, withChromeStartupArgs } from "../../process.js";
2
2
  import { isRecord } from "../../parsing.js";
3
3
  import { collectNativeWebMcp, getNativeWebMcpCatalog } from "../../webmcp-observation.js";
4
- import { redactSensitiveValue } from "../../runtime.js";
4
+ import { isPlainTextInspectionArgs, redactSensitiveValue } from "../../runtime.js";
5
5
  import { formatWebMcpCatalogUpdate } from "../../results/presentation/common.js";
6
6
  import { withOwnedManagedSessionContext } from "../../managed-session-restore.js";
7
7
  import { cleanupClickDispatchProbe } from "./click-dispatch.js";
8
8
  import { applyBrowserRunStatePatch, getSessionContextKey } from "./session-state.js";
9
- import { buildMissingBinaryFailureResult } from "./final-result.js";
9
+ import { buildJsonVisibleContent, buildMissingBinaryFailureResult } from "./final-result.js";
10
10
  import { prepareBrowserRun } from "./prepare.js";
11
11
  import { processBrowserOutput } from "./process-output.js";
12
12
  export { closeManagedSession } from "./managed-session-daemon-policy.js";
@@ -38,7 +38,17 @@ async function runAgentBrowserToolInContext(options) {
38
38
  const preparedResult = await prepareBrowserRun(options);
39
39
  applyBrowserRunStatePatch(options.state, preparedResult.kind === "ready" ? preparedResult.prepared.statePatch : preparedResult.statePatch);
40
40
  if (preparedResult.kind === "early-result") {
41
- return preparedResult.result;
41
+ const result = preparedResult.result;
42
+ if (options.input.toolArgs.includes("--json") && !isPlainTextInspectionArgs(options.input.toolArgs)) {
43
+ const details = isRecord(result.details) ? result.details : {};
44
+ const summary = result.content.filter(item => item.type === "text").map(item => item.text).join("\n");
45
+ result.content = buildJsonVisibleContent({
46
+ error: result.isError ? details.validationError ?? summary : null,
47
+ presentation: { content: result.content, data: details.data, summary },
48
+ succeeded: result.isError !== true,
49
+ });
50
+ }
51
+ return result;
42
52
  }
43
53
  const { prepared } = preparedResult;
44
54
  const ownedManagedSession = prepared.ownedManagedSessionContext;
@@ -1,6 +1,7 @@
1
1
  import { isRecord } from "../../../parsing.js";
2
2
  import { buildAgentBrowserResultCategoryDetails } from "../../../results/categories.js";
3
- import { redactSensitiveText } from "../../../runtime.js";
3
+ import { redactPresentationData } from "../../../results/presentation/diagnostics.js";
4
+ import { redactInvocationArgs, redactSensitiveText } from "../../../runtime.js";
4
5
  import { buildSessionDetailFields, runSessionCommandData } from "../session-state.js";
5
6
  function parseNetworkRequestsPageFilterRequest(commandTokens) {
6
7
  if (commandTokens[0] !== "network" || commandTokens[1] !== "requests")
@@ -95,9 +96,9 @@ export async function tryNetworkRequestsPageFilter(options) {
95
96
  args: options.redactedArgs,
96
97
  command: "network",
97
98
  compatibilityWorkaround: options.compatibilityWorkaround,
98
- data: filtered.data,
99
+ data: redactPresentationData({ command: "network", subcommand: "requests" }, filtered.data),
99
100
  effectiveArgs: options.effectiveArgs,
100
- networkRequestsPageFilter: { cleanArgs: request.cleanArgs, currentUrl: redactSensitiveText(currentUrl), matchedRows: filtered.matchedRows, mode: request.mode, totalRows: filtered.totalRows },
101
+ networkRequestsPageFilter: { cleanArgs: redactInvocationArgs(request.cleanArgs), currentUrl: redactSensitiveText(currentUrl), matchedRows: filtered.matchedRows, mode: request.mode, totalRows: filtered.totalRows },
101
102
  sessionMode: options.sessionMode,
102
103
  ...buildAgentBrowserResultCategoryDetails({ args: options.effectiveArgs, command: "network", succeeded: true }),
103
104
  ...buildSessionDetailFields(options.sessionName, options.usedImplicitSession, options.namespace, options.managedSessionRestoreDisabled()),
@@ -37,8 +37,7 @@ function buildContainerScrollScript(request) {
37
37
  return Number.isFinite(pixels) && pixels > 0 ? Math.floor(pixels) : Math.max(1, Math.floor(extent * 0.8));
38
38
  };
39
39
  const delta = parseAmount() * (direction === "up" || direction === "left" ? -1 : 1);
40
- if (axis === "x") element.scrollLeft += delta;
41
- else element.scrollTop += delta;
40
+ element.scrollBy({ [axis === "x" ? "left" : "top"]: delta, behavior: "instant" });
42
41
  const after = { scrollLeft: element.scrollLeft, scrollTop: element.scrollTop, scrollHeight: element.scrollHeight, scrollWidth: element.scrollWidth, clientHeight: element.clientHeight, clientWidth: element.clientWidth };
43
42
  const moved = before.scrollLeft !== after.scrollLeft || before.scrollTop !== after.scrollTop;
44
43
  return { status: moved ? "scrolled" : "no-movement", selector, direction, amount: amountToken || undefined, before, after };
@@ -9,7 +9,6 @@ import { cleanupElectronLaunchResources } from "../../electron/cleanup.js";
9
9
  import { launchElectronApp } from "../../electron/launch.js";
10
10
  import { pathExists } from "../../fs-utils.js";
11
11
  import { getCompiledSemanticActionSessionPrefix } from "../../input-modes/semantic-action.js";
12
- import { tryDirectAnchorDownload } from "./prepare/direct-anchor-download.js";
13
12
  import { tryNetworkRequestsPageFilter } from "./prepare/network-page-filter.js";
14
13
  import { tryContainerScroll, tryPageScrollTo } from "./prepare/scroll-shims.js";
15
14
  import { trySnapshotFilter } from "./prepare/snapshot-filter.js";
@@ -1005,22 +1004,6 @@ export async function prepareBrowserRun(options) {
1005
1004
  if (pageScrollTo)
1006
1005
  return { kind: "early-result", statePatch, result: pageScrollTo };
1007
1006
  }
1008
- const directAnchorDownload = await tryDirectAnchorDownload({
1009
- artifactManifest: state.artifactManifest,
1010
- commandTokens,
1011
- compatibilityWorkaround,
1012
- cwd,
1013
- effectiveArgs: redactedEffectiveArgs,
1014
- managedSessionRestoreDisabled,
1015
- redactedArgs,
1016
- sessionMode,
1017
- namespace: executionPlan.namespace,
1018
- sessionName: executionPlan.sessionName,
1019
- signal,
1020
- usedImplicitSession: executionPlan.usedImplicitSession,
1021
- });
1022
- if (directAnchorDownload)
1023
- return { kind: "early-result", statePatch: { ...statePatch, artifactManifest: directAnchorDownload.artifactManifest ?? statePatch.artifactManifest }, result: directAnchorDownload.result };
1024
1007
  const processArgs = executionPlan.effectiveArgs;
1025
1008
  const processStdin = preparedArgs.stdin ?? runtimeToolStdin;
1026
1009
  const clickDispatchProbe = compiledElectron === undefined
@@ -7,7 +7,7 @@ export function buildInstalledDocsGuideline(paths) {
7
7
  return `For detailed agent_browser docs, read targeted sections: ${paths.readmePath} (setup), ${paths.commandReferencePath} (commands), ${paths.toolContractPath} (result/details). Do not load the full command reference unless needed.`;
8
8
  }
9
9
  export const QUICK_START_GUIDELINES = [
10
- `Quick start mental model: use exactly one of script (one-shot JavaScript orchestration), args (exact agent-browser CLI args after the binary), semanticAction (a thin shorthand compiled to find argv for locator actions, direct selector/ref click/check/fill, or select argv for native dropdowns), job (a constrained short-workflow schema compiled to batch --bail by default; set failFast:false only when later diagnostics remain safe if an earlier navigation fails), qa (a lightweight fail-fast QA preset built on batch --bail with bounded visible expected-text checks, including qa.attached for current sessions), electron (desktop Electron list/launch/status/cleanup/probe), or the experimental sourceLookup / networkSourceLookup helpers (candidates only; each compiled to batch); stdin is only for batch, eval --stdin, auth save --password-stdin, and wrapper-generated batch stdin from job, qa, sourceLookup, or networkSourceLookup, and is rejected with electron; sessionMode=fresh switches the extension-managed pi-scoped session to a fresh upstream launch when you need new launch-scoped flags (${LAUNCH_SCOPED_FLAG_LABEL}) to apply. Use outputPath for durable eval/get/snapshot captures. Do not pass --json in args; the wrapper injects it.`,
10
+ `Quick start mental model: use exactly one of script (one-shot JavaScript orchestration), args (exact agent-browser CLI args after the binary), semanticAction (a thin shorthand compiled to find argv for locator actions, direct selector/ref click/check/fill, or select argv for native dropdowns), job (a constrained short-workflow schema compiled to batch --bail by default; set failFast:false only when later diagnostics remain safe if an earlier navigation fails), qa (a lightweight fail-fast QA preset built on batch --bail with bounded visible expected-text checks, including qa.attached for current sessions), electron (desktop Electron list/launch/status/cleanup/probe), or the experimental sourceLookup / networkSourceLookup helpers (candidates only; each compiled to batch); stdin is only for batch, eval --stdin, auth save --password-stdin, and wrapper-generated batch stdin from job, qa, sourceLookup, or networkSourceLookup, and is rejected with electron; sessionMode=fresh switches the extension-managed pi-scoped session to a fresh upstream launch when you need new launch-scoped flags (${LAUNCH_SCOPED_FLAG_LABEL}) to apply. Use outputPath for durable eval/get/snapshot captures. Omit --json unless you need JSON text; the wrapper otherwise injects it for structured details.`,
11
11
  "Use script only when loops, conditional page branches, or multi-page aggregation would otherwise require several top-level calls: call await browser({ args, stdin?, timeoutMs? }), check each returned { ok, data, error, details, failureCategory, nextActions, resultCategory, summary, text } envelope, and call emit(value) for the one final JSON value. Script runs in a one-shot isolated session without profiles, attachments, imports, host filesystem/network/process access, caller session controls, or inherited agent-browser launch/proxy settings; inner browser calls and their response envelopes are serialized and bounded, compatible suggested next actions can be passed back to browser(), and the wrapper always closes that session. One top-level approval can authorize up to 25 inner calls, so inspect the full source before approval. Use args/job/qa for ordinary linear work.",
12
12
  "There is no first-class reusable named browser recipe runtime: script is ad hoc source for one call only, with no saved names, registry, or versioned workflow state. Keep recurring flows in documentation examples, job, qa, or raw batch (closed RQ-0068; see docs/ARCHITECTURE.md#no-reusable-recipe-layer-yet).",
13
13
  "Common first calls (first-call recipe): { args: [\"open\", \"<url>\"] } → { args: [\"snapshot\", \"-i\"] } → { args: [\"click\", \"@eN\"] } or { args: [\"fill\", \"@eN\", \"<text>\"] } using @refs and visible labels from that snapshot, then { args: [\"snapshot\", \"-i\"] } after navigation or DOM changes. On https://example.com/ the main link label is Learn more (use exact snapshot text, not guessed link copy).",
@@ -15,7 +15,7 @@ export const QUICK_START_GUIDELINES = [
15
15
  `Common advanced calls: { args: ["batch", "--bail"], stdin: "[[\"open\",\"https://example.com\"],[\"snapshot\",\"-i\"]]" }, { job: { steps: [{ action: "open", url: "https://example.com" }, { action: "assertText", text: "Example Domain" }, { action: "screenshot", path: ".dogfood/example.png" }] } }, { qa: { url: "https://example.com", expectedText: "Example Domain", screenshotPath: ".dogfood/qa-example.png" } } (example.com smoke only; elsewhere match exact visible text from snapshot -i), { electron: { action: "list", query: "code" } }, { electron: { action: "launch", appName: "Visual Studio Code", handoff: "snapshot" } }, { electron: { action: "probe" } }, { qa: { attached: true, expectedText: "Explorer" } }, { args: ["eval", "--stdin"], stdin: "document.title", outputPath: "logs/page-title.json" }, { args: ["auth", "save", "name", "--password-stdin"], stdin: "<password from user-approved secret source>" }, { args: ["--profile", "Default", "open", "https://example.com/account"], sessionMode: "fresh" }, and { args: ["open", "--enable", "react-devtools", "https://example.com"], sessionMode: "fresh" }. For app pages with a native dropdown, job steps can include { action: "select", selector: "#flavor", value: "chocolate" } before the dependent assertion; for locator-friendly pages, job click/fill steps can use semantic locator fields such as { action: "fill", locator: "role", role: "searchbox", name: "Search", text: "agent browser" }; for human-paced input, job type steps can use { action: "type", selector: "#prompt", text: "hello", delayMs: 20, press: "Enter" }; delayed typing is capped at 200 characters per step, and generated per-character rows are compacted in visible batch prose while full rows remain in details.batchSteps.`,
16
16
  "Constrained job navigation is explicit only: click (and select/submit flows that may navigate) does not prove the next page loaded; add an assertUrl that does not already match the starting page and/or assertText for new page state after navigation-prone steps before screenshot or later interactions. assertText takes only text, not selector or locator fields. Clicks can stale subsequent @refs: split the job and re-snapshot before using those refs. Keep jobs short around navigation, click, and rerender boundaries on dynamic React/product apps; avoid a whole checkout in one job. If a long job times out and details.timeoutPartialProgress shows a mutating incomplete step, inspect current page state and continue with a shorter job or single action instead of blindly retrying the mutating step. Example: { job: { steps: [{ action: \"open\", url: \"https://shop.example/checkout\" }, { action: \"fill\", selector: \"#email\", text: \"user@example.com\" }, { action: \"click\", selector: \"#continue\" }, { action: \"assertUrl\", url: \"**/shipping\" }, { action: \"assertText\", text: \"Shipping address\" }, { action: \"screenshot\", path: \".dogfood/shipping.png\" }] } }. Top-level click may add pageChangeSummary hints, but job never auto-inserts post-click asserts.",
17
17
  "High-value command reference: click <selector> --new-tab opens link-like targets in a new tab; select <selector> <value...> changes native dropdown values; wrapper-handled scroll <dir> [px|percent] and scroll to end/top target document scrolling before upstream fallback, while scroll <selector> <dir> [px|percent] targets nested scrollers; download <selector> <path> saves a file triggered by a click; read [url] returns agent-readable text (explicit URLs and all-read batches skip browser helpers and managed replacement, including timeout recovery; omit the URL for verified active-tab DOM); get title/url need no selector; get text/html/value/count <selector> and get attr <selector> <name> read elements/page state (use body for whole-page text/html); screenshot [selector] [path] captures a page or element image; pdf <path> saves a PDF; tab list and tab <tab-id-or-label> inspect or recover the active tab; react tree, react inspect <fiberId>, react renders start/stop, and react suspense introspect React after --enable react-devtools; vitals [url] measures Core Web Vitals; pushstate <url> performs SPA navigation; tap <selector> and swipe <direction> [distance] support iOS/provider touch flows.",
18
- "For artifact-producing commands, read the visible artifact block and details.artifactVerification before using files: check requested path, absolute path, existence, size bytes, artifact kind, optional mediaType, status, optional limitation, and verified/missing/pending/unverified counts. details.artifacts contains per-file metadata; record start rows are pending/openRecording until record stop finalizes the target. Recording receipts separate captured-frame rate/wall duration from nominal FPS and held/encoded frames; unknown native metrics stay unknown, and sparse or final-only frames cannot prove smoothness. A stop timeout gets one bounded session-info receipt query; follow its exact status/stop actions. Distinct outputPath exports retain failed attempts too. Current upstream records the active page unless a URL is supplied. To support older natives, the wrapper conservatively blocks prior @e… refs after every dispatched start attempt and URL-bearing restart, even on failure; this does not prove the page changed. Take a fresh snapshot before continuing. A restart with only --fps keeps the page and refs; --fps <n> selects 1–60 fps (default 30). The wrapper creates parent directories for direct artifact paths and can save simple loopback HTTP(S) anchor downloads directly to the requested path before upstream download fallback. Browser close does not delete explicit saved files; if close reports details.artifactCleanup, use host file tools to remove paths listed in explicitArtifactPaths (when non-empty) after inspection. If close fails with details.promptGuard.reason=requested-artifacts-missing-before-close, save the exact required artifact path before closing. A bare inbound image/video path is not a requested output artifact and does not block close. For annotated screenshots inside batch, put --annotate in top-level args (for example { args: [\"--annotate\", \"batch\"], stdin: \"[[\\\"screenshot\\\",\\\"/tmp/page.png\\\"]]\" }) rather than inside the screenshot step; if annotation labels crowd a dense page, use a scoped or non-annotated screenshot plus snapshot refs instead.",
18
+ "For artifact-producing commands, read the visible artifact block and details.artifactVerification before using files: check requested path, absolute path, existence, size bytes, artifact kind, optional mediaType, status, optional limitation, and verified/missing/pending/unverified counts. details.artifacts contains per-file metadata; record start rows are pending/openRecording until record stop finalizes the target. Recording receipts separate captured-frame rate/wall duration from nominal FPS and held/encoded frames; unknown native metrics stay unknown, and sparse or final-only frames cannot prove smoothness. A stop timeout gets one bounded session-info receipt query; follow its exact status/stop actions. Distinct outputPath exports retain failed attempts too. Current upstream records the active page unless a URL is supplied. To support older natives, the wrapper conservatively blocks prior @e… refs after every dispatched start attempt and URL-bearing restart, even on failure; this does not prove the page changed. Take a fresh snapshot before continuing. A restart with only --fps keeps the page and refs; --fps <n> selects 1–60 fps (default 30). The wrapper creates parent directories for direct artifact paths; native download owns the click and download, including loopback links, generated Blob exports, and redirects. Browser close does not delete explicit saved files; if close reports details.artifactCleanup, use host file tools to remove paths listed in explicitArtifactPaths (when non-empty) after inspection. If close fails with details.promptGuard.reason=requested-artifacts-missing-before-close, save the exact required artifact path before closing. A bare inbound image/video path is not a requested output artifact and does not block close. For annotated screenshots inside batch, put --annotate in top-level args (for example { args: [\"--annotate\", \"batch\"], stdin: \"[[\\\"screenshot\\\",\\\"/tmp/page.png\\\"]]\" }) rather than inside the screenshot step; if annotation labels crowd a dense page, use a scoped or non-annotated screenshot plus snapshot refs instead.",
19
19
  "Use session info for one read-only preflight distinguishing daemon PID/activity from native browser liveness, Chrome PID, exact profile, tabs and native ownership; Pi cleanup ownership is separate and absent fields stay unknown. For policy-required URL reads, use the exact confirm/deny actions and session they return. Only native explicit-read provenance plus capabilities.readRequiresConfirmation enables browserless confirmation; legacy/DOM confirmations retain normal page checks.",
20
20
  "When failure output shows Next actions, prefer those exact native agent_browser follow-up payloads over guessed commands. The same actions are available in details.nextActions to callers that expose structured details; short stdin is shown inline, while long stdin stays details-only.",
21
21
  ];
@@ -50,11 +50,11 @@ export const SHARED_BROWSER_PLAYBOOK_GUIDELINES = [
50
50
  "If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks, use wait --fn predicates instead of stale upstream-help examples like wait <selector> --state hidden. Use electron.probe/status for wrapper-owned launch health or target mismatch. Fixed waits are a last resort: their duration is positional (wait <ms>, not wait --time <ms>). Use explicit --timeout or top-level timeoutMs for legitimately slow waits, and treat a successful payload like \"waited\":\"timeout\" as elapsed time only—verify completion with an observed condition, fresh snapshot, or screenshot.",
51
51
  "For feed, timeline, or inbox reading tasks, focus on the main timeline/list region and read the first item there rather than unrelated composer or sidebar content.",
52
52
  "For read-only browsing tasks, use read <url> for documentation or other unstructured text without requiring a Chrome page, or read with no URL for rendered active-tab DOM. Prefer the current snapshot, structured ref labels, getters, or scoped eval --stdin when you need interactive structure or targeted page state. Only click into media viewers, detail routes, or new pages when the current view does not contain the needed information.",
53
- "For downloads, prefer download <selector> <path> when an element click should save a file; simple loopback anchor downloads are saved to the requested path when the wrapper can resolve an HTTP(S) href. Do not rely on click alone when you need the downloaded file on disk.",
53
+ "For downloads, prefer download <selector> <path> when an element click should save a file; native download owns the click and download, including loopback links, generated Blob exports, and redirects. Do not rely on click alone when you need the downloaded file on disk.",
54
54
  "On dashboards with nested scroll containers, verify scroll with a screenshot or fresh snapshot -i; if the viewport did not move, details.data.scrolled may be false/noMovement true and you should prefer scrollintoview <@ref> or target the actual scrollable region with scroll <selector> <dir> [px|percent]. For native selects, use select <selector> <value...> (or semanticAction/job select) instead of clicking option refs; for custom comboboxes, a click/semanticAction may only focus the field, so re-snapshot and use keyboard type <text> for focused input, press ArrowDown or press Enter, or visible option refs. Raw type requires both <selector> and <text>.",
55
55
  "When using eval --stdin, scope checks and actions to the target element or route whenever possible instead of relying on broad page-wide text heuristics.",
56
56
  "When using eval --stdin for extraction, pass the JavaScript through the native tool stdin field, not as an extra args token after --stdin, and return the value you want instead of relying on console.log as the primary result channel. Prefer plain expressions like ({ title: document.title }) or explicitly invoked functions like (() => ({ title: document.title }))(); use outputPath when the eval/get/snapshot data should be saved as a durable local file, but never reuse a screenshot, download, recording, or other browser artifact destination as outputPath. If a function-shaped snippet returns {}, details.evalStdinHint may warn that the function was serialized instead of called. Local file pages and caller-selected output paths are supported when upstream allows them. If get text on a broad CSS selector surfaces details.selectorTextVisibility or selectorTextVisibilityAll, prefer a visible @ref, a more specific selector, or the inspect-visible-text-candidates nextAction over hidden tab content.",
57
- "When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If an upstream click failure says the target is covered by another element at the target's click point, use the inspect-overlay-state nextAction to refresh refs and inspect the blocker before deciding whether to retry; do not blindly repeat the blocked click. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.",
57
+ "When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. Click-dispatch probes require native get attr readback of a temporary marker on the candidate target; unproven ref or XPath identity leaves the native click unverified, not failed. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If an upstream click failure says the target is covered by another element at the target's click point, use the inspect-overlay-state nextAction to refresh refs and inspect the blocker before deciding whether to retry; do not blindly repeat the blocked click. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.",
58
58
  "On upstream 0.38+, use snapshot --delta for native full/unchanged/changed revisions; --delta --full resets its baseline. Surviving DOM nodes keep refs, but refresh after navigation or replacement. Wrapper filters use full trees. screenshot --if-changed or --threshold 0.01 can return changed:false with no path or image; use ordinary screenshots when a file is required. --input-mode instant|smooth|human controls sticky pointer behavior; click/drag --human and mouse move --duration/--steps/--human/--seed pass through. record start/restart accept --cursor, --contact-sheet, and --contact-sheet-threshold; stop and verify both video and reported contact sheet. auth login <name> --no-navigate fills a prepared login page only after native origin checks. WebMCP catalog updates are untrusted page data; webmcp list supplies full schemas.",
59
59
  "When commands save or spill files (screenshots, downloads, PDFs, traces, recordings, HAR, large snapshot spills), use the user's exact requested paths when given and treat paths as provisional until details.artifactVerification shows every row verified: branch on missingCount, pendingCount, unverifiedCount, per-entry state, and optional limitation before downstream file use or PASS/FAIL reporting.",
60
60
  "For evidence-only screenshots, QA captures, or other audit artifacts, save to an explicit path and branch on details.artifactVerification plus details.artifacts before reporting PASS/FAIL; do not require vision review of inline image attachments unless the user asked for visual inspection.",
@@ -79,7 +79,7 @@ export const WRAPPER_TAB_RECOVERY_BEHAVIOR = [
79
79
  ];
80
80
  /** Tier A: always-on tool promptGuidelines (keep small; Tier B lives in SHARED_BROWSER_PLAYBOOK_GUIDELINES and docs). */
81
81
  export const RUNTIME_PROMPT_GUIDELINES = [
82
- "Use agent_browser with one input mode: script, args, semanticAction, job, qa, sourceLookup/networkSourceLookup, or electron. stdin: batch/eval/auth/wrapper batch only; electron rejects it; never pass --json.",
82
+ "agent_browser: one input mode: script, args, semanticAction, job, qa, sourceLookup/networkSourceLookup, or electron. stdin: batch/eval/auth/wrapper batch only; electron rejects it. Use --json for JSON text.",
83
83
  "For agent_browser, use open → snapshot -i → @refs; re-snapshot after changes. In authenticated unattended/auto-approved employee flows, ordinary requested non-destructive submissions may proceed. Honor explicit stops; require explicit authorization for purchases, production-control, destructive/irreversible, or account/security/privacy changes.",
84
84
  "agent_browser bare calls share a root Pi browser with descendants, not unrelated roots; coordinate within the group. Native explicit sessions win. Use sessionMode=fresh for an intentional separate launch. Use requested/configured profiles only; verify auth after restart. Script is disposable.",
85
85
  "agent_browser: exact user paths; verify artifactVerification/artifacts before success claims. Save promptGuard-required files before close; ffmpeg before recording; close keeps files; waited:timeout proves nothing.",
@@ -37,7 +37,7 @@ Why:
37
37
  The extension should:
38
38
  - resolve `agent-browser` from `PATH`
39
39
  - invoke it with native Node `spawn` on POSIX and `cross-spawn` on Windows. The latter handles Windows `PATH`/`PATHEXT`, `.cmd` shims and argument escaping without PowerShell or wrapper-owned command reordering. Caller commands and helper probes retain empty operands, literal doublequotes, command/subcommand adjacency and explicit default `--namespace ""`; the selected shim still owns upstream architecture selection. Keep piped stdin/stdout/stderr and terminate the full Windows shell/agent-browser process tree with `taskkill /T /F` on timeout or abort before falling back to the direct child signal. Missing commands use the spawner's `ENOENT`, not a parsed shell-error string
40
- - inject `--json`
40
+ - inject `--json`; caller-requested JSON also uses the existing `buildJsonVisibleContent` envelope for early snapshot/network filters, scroll results, and preparation failures, retaining metadata in `details` and keeping output-file notices out of visible JSON. Help/version stays native text
41
41
  - complete each upstream invocation when the direct `agent-browser` child exits even if Node delays `"close"`: piped stdio can stay referenced by longer-lived descendant processes, so `runAgentBrowserProcess` watches `exit` and `close` together, leaves stdio intact during a short post-`exit` grace so normal `close` can still win, destroys streams only when the post-`exit` fallback fires, and prefers `close` codes then wrapper timeout (`124`) over signal-shaped `exit` codes (`watchSpawnedChildCompletion` / `resolveSpawnedChildExitCode` in `extensions/agent-browser/lib/process.ts`) so the tool cannot hang after the CLI process has already terminated
42
42
  - support optional stdin only for `eval --stdin`, `batch`, `auth save --password-stdin`, and wrapper-generated `batch` stdin from top-level `job`, `qa`, `sourceLookup`, or `networkSourceLookup`, rejecting other command/stdin combinations before launch; top-level `electron` never accepts caller `stdin` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron))
43
43
  - support optional top-level `outputPath` for successful browser results and failed/pending/recovered recording receipts. Recording exports retain success/error, original attempt, native data, artifacts and recovery provenance; they do not treat a written receipt as a successful video. Other successful results write `details.data` (or model-facing text when no structured data exists) to a caller-requested local file and reporting `details.outputFile`, without changing upstream argv semantics or overwriting a browser artifact when both destinations resolve to the same file. If presentation compacted direct data, a batch result row, or a whole batch, apply the relevant command-specific redactor before spilling and rehydrate each full pre-compaction value only from its matching live wrapper-manifest spill; fail without writing compact metadata when any required spill is unavailable or untrusted
@@ -186,7 +186,7 @@ Practical policy:
186
186
  - set an idle timeout on extension-managed sessions as a backstop for abnormal exits or cleanup failures, and apply that same `AGENT_BROWSER_IDLE_TIMEOUT_MS` value to every subprocess targeting that owned session (including wrapper helper snapshots, tab lists, and navigation-summary reads); caller-owned sessions instead retain native config/environment idle policy because changing the launch environment between calls can make upstream restart the background browser, discard the active tab, and invalidate fresh refs
187
187
  - for wrapper-owned implicit sessions only, set a transcript- and checkout-scoped `AGENT_BROWSER_RESTORE` key on compatible calls so cookies and web storage can survive idle shutdown, reload, and resume. Explicit caller sessions, restore/state choices, profiles, upstream config, file access, launch arguments, environment variables, local pages, output paths, and close arguments remain upstream-owned and pass through unchanged. `piab-*` names are not reserved; session/state lists and restore identifiers are not filtered or redacted. The wrapper validates only its automatic restore checkout/storage identity and coordinates same-daemon reuse so its own restore pools cannot mix. Ambiguous page-target transitions still require live `get url` verification before content calls. The current v3 ticket-claim lock is the only managed-daemon coordination protocol; no earlier lock bridge or compatibility path remains.
188
188
  - redact snapshot spill payloads before writing them, clean up process-private temp spill artifacts on shutdown, and keep persisted-session spill files in a private session-scoped artifact directory for `details.fullOutputPath` after reload/resume, with a 32 MiB per-session budget by default; `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES=0` skips persistent-spill enumeration and eviction without changing temporary spill cleanup
189
- - keep explicit screenshots, downloads, PDFs, traces, HAR captures, and recordings written to caller-chosen paths on disk after a successful upstream close command (`close`, `quit`, or `exit`); before artifact-producing commands run, create missing parent directories for requested host paths, and for simple loopback HTML anchor downloads with resolvable HTTP(S) hrefs the wrapper may save directly to the requested path before upstream fallback. When the bounded `details.artifactManifest` has entries, successful close commands also surface `details.artifactCleanup` and a compact `Artifact lifecycle` note pointing to structured explicit paths so operators remove files with normal host tools—the native tool does not delete arbitrary user paths (`extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`, `getArtifactCleanupGuidance`); contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), checklist `RQ-0079` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
189
+ - keep explicit screenshots, downloads, PDFs, traces, HAR captures, and recordings written to caller-chosen paths on disk after a successful upstream close command (`close`, `quit`, or `exit`); before artifact-producing commands run, create missing parent directories for requested host paths, while native download always owns the click and download, including loopback links, generated Blob exports, and redirects; never substitute an anchor-href fetch. When the bounded `details.artifactManifest` has entries, successful close commands also surface `details.artifactCleanup` and a compact `Artifact lifecycle` note pointing to structured explicit paths so operators remove files with normal host tools—the native tool does not delete arbitrary user paths (`extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`, `getArtifactCleanupGuidance`); contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), checklist `RQ-0079` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
190
190
  - reconstruct the current branch-visible extension-managed session, every transcript-proven still-active wrapper-owned managed identity, page-scoped refs, newest-revision aggregate artifact manifest, unbounded active-recording reservation events, and Electron launch records from the active transcript branch on `session_start` and `session_tree` so later default and explicit off-current calls keep following owned browsers after resume/reload or branch switching; restore also honors successful explicit `--session <wrapper-owned> close` rows, terminal nested-batch close outcomes even when aggregate artifact verification failed, and `electron.cleanup` managed-session steps so closed wrapper-owned sessions are not resurrected; a nested close invalidates the pre-close page target so a lifecycle-proven relaunch at `about:blank` is not treated as stale focus drift; explicit lifecycle evidence that a later diagnostic did not launch a browser preserves the terminal close, while any later row—including a failed row—whose lifecycle reports a browser launch keeps active/attached provenance; failed-step presentation persists only that bounded launch boolean so transcript replay reaches the same decision, missing lifecycle evidence remains conservatively active even on the first managed call, successful closes clear wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and recording starts after close are rejected before spawn
191
191
  - keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, script, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs, cursor/contact-sheet switches, and contact-sheet threshold values without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
192
192
  - keep process-owned cleanup registries for extension-managed sessions and wrapper-launched Electron records separate from the current branch-visible view; `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work, while caller-owned explicit-session commands are serialized by process-local queues keyed to effective canonical namespace/session across prepare helpers (explicit namespace argv overrides inherited `AGENT_BROWSER_NAMESPACE`, including an explicit empty default) and main execution. macOS and Windows additionally normalize and case-fold namespace and session components to match case-insensitive daemon identity. Different caller-owned identities remain concurrent, except namespace-scoped `close --all` drains and exclusively barriers managed plus matching caller-owned work before clearing global namespace state; nested helpers never re-enter the outer queue, policy/route/artifact deltas merge across unrelated managed-state commits, and a separate branch-restore generation guard prevents stale completions from overwriting newer branch-visible state; aggregate artifact results use monotonic revisions so transcript replay cannot lose a concurrently completed entry. Branch switches still must not drop resources the current Pi process owns and must keep fresh-session allocation monotonic
@@ -204,8 +204,9 @@ Practical policy:
204
204
  - keep a per-session `refSnapshot` aligned with the last successful `snapshot` (including refs merged from a successful `batch` by taking the last successful `snapshot` step in batch result order): restore it from persisted tool `details` when reloading, resuming, or moving to a different Pi session-tree branch, store bounded ref role/name metadata from the same snapshot for wrapper-side current-ref diagnostics, drop it on successful close commands (`close`, `quit`, or `exit`), replace it with a persisted `page-transition` invalidation after any upstream-executed `record start` attempt (direct or batch; older supported natives can swap the page before their already-active check, so failed starts count; the warning and persisted summary explicitly describe conservative ref invalidation, not an observed page change), a `record restart` with a URL operand, `window new`, `diff url`, or WebMCP `invoke` / `result` / `cancel` (these page-provided tools can mutate, rerender, or navigate; when a spawned `batch` yields no parseable result rows, for example after a wrapper timeout, planned transition steps still record the invalidation), or after a failed non-batch transition command (`eval`, `back`, `forward`, `reload`, `connect`, `state load`, `tab` selection) whose live URL re-verification probe observed the page (a failed transition can still have mutated or replaced the document before throwing, so keeping the verified URL must not keep the prior refs; transcript replay preserves the persisted invalidation summary), and refuse page-scoped `@e…` argv before spawn when the active tab URL no longer matches the snapshot URL, when a ref id was never in that snapshot, when the snapshot state is invalidated, or when a `batch` step would reuse `@e…` on a guarded getter or mutation step after an earlier invalidating step (including `record start`, URL-bearing `record restart`, and WebMCP `invoke` / `result` / `cancel`) without a later `snapshot` step in the same plan; batch steps come from the source upstream actually executes (raw batch argument strings exclusively when any exist — filtering only the exact `--bail` token like upstream — stdin only otherwise, via `getUpstreamEffectiveBatchSteps` in `extensions/agent-browser/lib/orchestration/batch-stdin.ts`); tab recovery (which leaves user argv/stdin and continue-on-error control flow unchanged), artifact/recording preflight, batch screenshot path preparation (parent directories are created for effective raw rows too, without rewriting raw strings), and stale-ref echo args use that same selection so pinning and preflights cannot act on upstream-ignored stdin, while the pre-spawn state-policy validator deliberately keeps scanning parseable stdin alongside argv as a fail-closed content superset and treats stdin parse failures as fatal only when upstream would actually read stdin (its raw-token filter also uses the exact `--bail` token only). Same-snapshot `fill @e…` rows are guarded but do not themselves set that invalidation latch, so ordinary form fills can precede a click/submit row in one batch—see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) for the agent-visible contract and failure text; typed per-session tab/ref/pinning state lives in `extensions/agent-browser/lib/session-page-state.ts` and is updated from `extensions/agent-browser/index.ts` after each tool result
205
205
  - native 0.37 owns first-document setup inheritance for new tabs; the wrapper does not copy headers, user agents or other browser setup itself. The page-summary formatter displays native positive WebMCP availability while retaining raw details; it adds no discovery probe
206
206
  - when a direct or batched WebMCP call returns `status: "pending"`, or `result` / `cancel` fails while that target is unknown, keep its tab target unknown and discard same-call snapshot evidence instead of treating an immediate post-dispatch URL probe as stable; `webmcp result` / `cancel`, `get url`, and explicit navigation remain available while unknown; replace the generic blocked snapshot action with `verify-page-target-after-pending-webmcp` (`get url`), and let a completed `batch --bail` use that verification before `snapshot -i` to re-establish both target and refs
207
- - for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`job`/`qa` click steps
207
+ - for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Ref and XPath candidates must additionally prove identity through a per-probe temporary DOM attribute read back by native `get attr <original-selector> <marker>`. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and lets native click run without dispatch verification; candidate naming heuristics alone are never proof. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`job`/`qa` click steps
208
208
  - derive narrow prompt guards only for concrete evidence invariants: explicitly requested screenshot/recording output paths block browser close until the artifact manifest verifies those paths, while bare inbound attachment paths remain inputs. The wrapper intentionally does not infer broad business/user intent from prompt text such as order/payment/post boundaries; agents must follow those instructions themselves. The artifact guard is bounded preflight policy (`details.promptGuard`, `failureCategory: "policy-blocked"`), not a reusable browser recipe layer
209
+ - keep the existing page/selector scroll API; explicit CSS-container movement uses `Element.scrollBy` with `behavior: "instant"` before measuring offsets so smooth-scroll CSS cannot cause a premature no-movement failure
209
210
  - reject direct and effective batch `scrollintoview text=...` / `scrollinto text=...` before dispatch because current upstream can falsely report success without movement, while leaving help forms untouched; return only native recovery (`find text ... hover` or fresh snapshot/ref), leaving CSS, XPath, and current-ref behavior upstream-owned
210
211
  - after successful `get text` on a qualifying non-ref CSS selector, optionally issue one read-only `eval --stdin` probe per selector when multiple DOM matches or a hidden first match with visible peers could misread tabbed or off-screen content; simple id selectors and sensitive-looking literals skip this probe. Merge `details.selectorTextVisibility` / `selectorTextVisibilityAll`, visible warning lines, and `inspect-visible-text-candidates*` next actions as documented in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) and `RQ-0074` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
211
212
  - caller-owned sessions honor native `AGENT_BROWSER_SOCKET_DIR` unless `PI_AGENT_BROWSER_SOCKET_DIR` overrides it, with the same socket integrity checks. For other local Unix launches, set a short private socket directory so extension-generated session names do not fail on the upstream Unix socket-path length limit; require the selected path to be absolute, owned by the current uid, mode `0700`, under checked ancestry, and free of symlink, foreign-owner, or special planted entries; reject pre-existing unsafe modes instead of repairing them, then recheck before spawn. The actual filesystem root `/` is supplied by the trusted operating environment: a directory without group/other write bits is accepted regardless of its reported owner, which can be unmapped in a Linux user namespace. Existing root-owned sticky-directory acceptance is unchanged. This boundary does not protect against whoever controls the root filesystem. Every non-root ancestor still needs trusted ownership and permissions, including the destination ancestry of root-owned aliases; a matching overflow UID is not trusted. Android/Termux uses `/data/data/<package>/piab`, treats the owner-only app-data directory as the trust anchor, permits the app's matching private uid/gid ancestry, compacts generated managed identities to one 80-bit digest so ordinary namespace plus fresh-session paths remain within the limit, and places policy-lock coordination under `os.tmpdir()` because Android `/tmp` is shell-owned and inaccessible
@@ -263,6 +264,7 @@ Upstream restore-state persistence remains upstream-owned. The wrapper passes an
263
264
  - tool registration and schema (including the optional `semanticAction` compilation path to upstream `find` or `select`)
264
265
  - subprocess execution and JSON parsing through `buildAgentBrowserProcessEnv` in `extensions/agent-browser/lib/process.ts`: copies the parent process environment so user-approved provider credentials and other runtime variables reach upstream, then applies wrapper overrides such as the managed socket directory and clamped default operation timeout
265
266
  - clear missing-binary errors
267
+ - filtered `network requests --current-page` / `--current-origin` / `--current-url` preparation results use the same `redactPresentationData` credential redactor as ordinary diagnostics before populating `details.data` or writing `outputPath`
266
268
  - compact result summaries, including presentation-time redaction: stateful browser-context commands (`auth`, `cookies`, `storage`, `dialog`, `frame`, `state`) use field-aware value redaction and compact formatters, while other structured upstream JSON (for example `network`, `diff`, `trace` / `profiler` / `record`, `console` / `errors` / `highlight` / `inspect` / `clipboard`, `stream`, `dashboard`, and `chat`) is passed through `redactPresentationData` in `extensions/agent-browser/lib/results/presentation.ts` so model-facing `details.data` and batch roll-ups stay compact and do not echo bearer tokens, proxy passwords, or similar fields verbatim; `redactInvocationArgs` in `extensions/agent-browser/lib/runtime.ts` masks trailing values for sensitive global flags such as `--body`, `--headers`, `--password`, and `--proxy`, preserves positional rules for `cookies set` and `storage local|session set`, and nested `batch` steps use the same argv and error-body scrubbing before echoing commands or errors. Shared URL redaction covers SAMLRequest/SAMLResponse/RelayState and auth-context-only state/nonce at model/persisted-artifact boundaries while retaining exact internal page-target URLs; snapshot spills are redacted before disk writes
267
269
  - bounded machine-readable outcome metadata on tool `details` (`resultCategory`, `successCategory`, `failureCategory`, optional `nextActions`, optional `pageChangeSummary` with observed-vs-dispatched evidence and per-step summaries on `batch`, optional `artifactVerification` with the same shape on each successful `batchSteps[]` row) so agents can branch without parsing prose; enums, classifier precedence, and generic follow-up payloads are implemented under `extensions/agent-browser/lib/results/` in focused modules (`contracts.ts` for shared types, `categories.ts` for `classifyAgentBrowserSuccessCategory` / `classifyAgentBrowserFailureCategory` / `buildAgentBrowserResultCategoryDetails`, `action-recommendations.ts` for `buildAgentBrowserNextActions`, `next-actions.ts` for the `AgentBrowserNextAction` shape and merge helpers, `recovery-actions.ts` for recovery id registries and `buildRecoveryNextActions`, `network.ts` for `classifyNetworkRequestFailure` / `summarizeNetworkFailures`, and related helpers). Per-session tab target, `refSnapshot` alignment, invalidation, and tab pinning observations flow through `extensions/agent-browser/lib/session-page-state.ts` from `extensions/agent-browser/index.ts`. Compact page-change summaries and artifact verification rollups are built in `extensions/agent-browser/lib/results/presentation.ts` (`buildPageChangeSummary`, `buildArtifactVerificationSummary`), and the human contract lives in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). Real Pi custom tools otherwise only mark a row failed when `execute` throws, so the extension also registers `pi.on("tool_result", …)` and patches `agent_browser` results whose `details.resultCategory` is `failure` to set `isError: true`. Prose results also receive a short category notice, while caller-requested `--json` results with parseable JSON content keep that text unchanged so JSONL transcripts, UI affordances, and the machine-readable contract stay aligned for wrapper-side reclassifications such as `qa-failure` (`buildAgentBrowserToolResultPatch` in `extensions/agent-browser/lib/pi-tool-rendering.ts`; transcript semantics in the same contract doc)
268
270
  - inline screenshots/images for the plain `screenshot` command; other image-like saves (for example `diff screenshot`) still appear in `details.artifacts` and summaries but are not auto-inlined as Pi image attachments (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details))
@@ -188,7 +188,7 @@ The 0.27.3 rebaseline is an install-only compatibility update: upstream changed
188
188
 
189
189
  ## Core mental model
190
190
 
191
- Input mode chooser (one per call): **`script`** for one-shot loops, branches, or multi-page aggregation; **`args`** for the default open → snapshot -i → click/fill `@refs` flow; **`semanticAction`** for stable role/text/label targets; **`job`** / **`qa`** for multi-step checks; **`electron`** for desktop apps only; **`sourceLookup`** / **`networkSourceLookup`** are **experimental candidates-only** helpers (not authoritative mappings). Do not pass `--json` in `args`—the wrapper injects it. Match link and button text to the latest snapshot (on `https://example.com/` the main link is `Learn more`, not legacy `More information...` copy). See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#input-mode-chooser) for snapshot variants (`-i` vs `--compact` vs full) and batching three or more getters.
191
+ Input mode chooser (one per call): **`script`** for one-shot loops, branches, or multi-page aggregation; **`args`** for the default open → snapshot -i → click/fill `@refs` flow; **`semanticAction`** for stable role/text/label targets; **`job`** / **`qa`** for multi-step checks; **`electron`** for desktop apps only; **`sourceLookup`** / **`networkSourceLookup`** are **experimental candidates-only** helpers (not authoritative mappings). Omit `--json` for ordinary prose; include it when you need parseable JSON text. Match link and button text to the latest snapshot (on `https://example.com/` the main link is `Learn more`, not legacy `More information...` copy). See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#input-mode-chooser) for snapshot variants (`-i` vs `--compact` vs full) and batching three or more getters.
192
192
 
193
193
  Tool parameters (use exactly one of `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`):
194
194
 
@@ -236,7 +236,7 @@ Tool parameters (use exactly one of `script`, `args`, `semanticAction`, `job`, `
236
236
  - `networkSourceLookup`: **EXPERIMENTAL — candidates only** for failed request-to-source hints; compiles to generated `batch`, reports `details.compiledNetworkSourceLookup` and `details.networkSourceLookup`, and never assigns blame or edits files.
237
237
  - `electron`: optional Electron desktop-app shorthand. `list`, `status`, `cleanup`, and `probe` are wrapper-owned host/session helpers; `launch` starts a wrapper-owned isolated Electron profile and attaches through upstream `connect`.
238
238
  - `stdin`: top-level stdin is only for `batch`, `eval --stdin`, and `auth save --password-stdin`; other combinations are rejected before `agent-browser` is launched. `script` puts inner stdin on `browser({ stdin })`; `job`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` generate or manage their own input.
239
- - `outputPath`: optional wrapper-owned local file sink for successful results and recording receipts, including failed, pending, timed-out or recovered stops. Recording exports preserve an envelope with original attempt status and native receipt/verification (`details.outputFile.source: "recording-receipt"`); unrelated failed extractions remain unwritten. Use it for durable `eval`, `get`, `snapshot`, or diagnostic outputs, not as the destination for screenshots, downloads, recordings, or other browser artifacts; if the paths resolve to the same file, the browser artifact is preserved and the result-data write fails validation. If presentation compacted a large direct result, a result row, or the whole `batch`, the writer copies each full command-redacted pre-compaction value only from its matching live wrapper-manifest spill; if any required spill is unavailable or untrusted, it fails without writing compact metadata. `details.outputFile` reports the saved path and byte count. If caller argv includes upstream `--json`, the visible JSON content stays parseable and the save notice is only in `details.outputFile`.
239
+ - `outputPath`: optional wrapper-owned local file sink for successful results and recording receipts, including failed, pending, timed-out or recovered stops. Recording exports preserve an envelope with original attempt status and native receipt/verification (`details.outputFile.source: "recording-receipt"`); unrelated failed extractions remain unwritten. Use it for durable `eval`, `get`, `snapshot`, or diagnostic outputs, not as the destination for screenshots, downloads, recordings, or other browser artifacts; if the paths resolve to the same file, the browser artifact is preserved and the result-data write fails validation. If presentation compacted a large direct result, a result row, or the whole `batch`, the writer copies each full command-redacted pre-compaction value only from its matching live wrapper-manifest spill; if any required spill is unavailable or untrusted, it fails without writing compact metadata. `details.outputFile` reports the saved path and byte count. If caller argv includes upstream `--json`, the visible JSON content stays parseable, including early snapshot/network filters, scroll results, and preparation failures; metadata stays in `details` and the save notice is only in `details.outputFile`. Help/version keeps native text output.
240
240
  - `timeoutMs`: optional per-call wrapper subprocess watchdog override in milliseconds for the requested browser CLI process. Managed-session policy inspection can independently consume up to 35 seconds before that process; this preflight is intentionally not shortened by `timeoutMs` because a busy but valid daemon must remain distinguishable from an unverifiable one.
241
241
  - `sessionMode`:
242
242
  - Unconfigured ordinary calls select a root-Pi named browser with native restore; descendants share it, unrelated roots do not wait on each other, and Pi quit leaves it running. Existing managed sessions and explicit fresh launches retain their lifecycle.
@@ -371,7 +371,7 @@ Do not assume Playwright selector dialects such as `text=Close` or `button:has-t
371
371
 
372
372
  Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `semanticAction` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
373
373
 
374
- A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
374
+ A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
375
375
 
376
376
  Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a `batch`/`job` tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close. A click that upstream rejects because another element covers the target's click point (`is covered by` … `at its click point`) remains `failureCategory: "upstream-error"`, but receives the same session-aware `inspect-overlay-state` snapshot action. This includes direct `click`, `semanticAction` click, raw `find` clicks (including `nth` and omitted default-click actions), and failed `batch`/`job` rows. That failed-click path does not retry the original click or synthesize a dismiss candidate before a fresh snapshot provides evidence; it is separate from silent input-dispatch failures.
377
377
 
@@ -463,7 +463,7 @@ Use raw `args: ["batch"]` with `stdin` when you need arbitrary upstream commands
463
463
 
464
464
  For quick smoke/QA checks, use top-level `qa`. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
465
465
 
466
- The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
466
+ The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. Filtered data receives the same credential redaction as ordinary network diagnostics before entering `details.data` or an `outputPath` export. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
467
467
 
468
468
  ```json
469
469
  { "qa": { "url": "https://example.com", "expectedText": "Example Domain", "screenshotPath": ".dogfood/qa-example.png" } }
@@ -569,7 +569,7 @@ A successful wait-based download renders a readable summary such as `Download co
569
569
 
570
570
  The upstream screenshot aliases are `screenshot --full` for full-page capture and `screenshot --annotate` for labeled screenshots. Annotated screenshots can be noisy on dense pages because labels overlap real content; when labels obscure evidence, capture a scoped element screenshot, take a non-annotated screenshot, or use `snapshot -i` high-value refs as the machine-readable map. When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute another path in the final report. When the latest prompt names exact required screenshot paths, `close` / `quit` / `exit` can be blocked with `details.promptGuard.reason: "requested-artifacts-missing-before-close"` until those paths appear as verified explicit artifacts.
571
571
 
572
- Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. For simple loopback HTML anchors with `href` and a non-ref selector, the wrapper first reads the resolved anchor URL and saves the in-page credentialed response directly to the requested host path, avoiding upstream random-name download spills in local fixture tests; non-loopback/profile downloads still use upstream fallback. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
572
+ Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. Native download always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper never substitutes a fetch of the anchor's `href`. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
573
573
 
574
574
  For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
575
575
 
@@ -750,7 +750,7 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
750
750
  | `tap <selector>` | Touch-oriented tap alias for iOS/provider workflows. |
751
751
  | `swipe <direction> [distance]` | Touch-oriented swipe for iOS/provider workflows. |
752
752
 
753
- On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
753
+ On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
754
754
 
755
755
  Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, direct `semanticAction: { action: "select", selector, value|values }`, active-session semantic role/name or label select, or a `job` `select` step instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `semanticAction` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `semanticAction` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
756
756
 
package/docs/RELEASE.md CHANGED
@@ -47,7 +47,7 @@ npm run verify -- pre-pr
47
47
  3. `npm run verify -- package-pi`, which first validates package contents via `npm pack --json --dry-run` and then smoke-loads the packed package in Pi isolation
48
48
  4. `npm run smoke:platform:doctor` and the full Crabbox matrix from [`platform-smoke.md`](platform-smoke.md): macOS SSH, Ubuntu local-container, and native Windows Parallels targets running fast target-local `platform-build` plus `browser-dogfood-smoke`
49
49
 
50
- The 0.6.14 release uses the documented [local macOS and hosted native-Windows alternatives](platform-smoke.md#alternate-native-transports) instead of SSH and Parallels. Run and inspect those target-local suites separately alongside the remaining release checks. The default composed gate is unchanged; alternate evidence must not be labeled a passing Crabbox matrix.
50
+ The 0.6.15 release uses the documented [local macOS and hosted native-Windows alternatives](platform-smoke.md#alternate-native-transports) instead of SSH and Parallels. Run and inspect those target-local suites separately alongside the remaining release checks. The default composed gate is unchanged; alternate evidence must not be labeled a passing Crabbox matrix.
51
51
 
52
52
  `npm publish` runs npm’s `prepublishOnly` script from `package.json`, which executes the same `npm run verify -- release` gate and then `npm pack --dry-run`. That concatenated gate is everything in the default `npm run verify` step (generated playbook drift, clean `dist/` build, TypeScript, the unit/fake suite, generated command-reference blocks, and live upstream command-reference sampling against the targeted `agent-browser` on `PATH`), the configured-source lifecycle harness, the packaged Pi smoke in `package-pi`, and the release-blocking Crabbox platform matrix. Using `npm publish --ignore-scripts` skips that contract intentionally.
53
53
 
@@ -71,7 +71,7 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
71
71
  - Supported actions, locators, exclusivity rules, when `details.compiledSemanticAction` appears, and bounded `try-*-candidate` follow-ups on `selector-not-found` (specific action/locator pairs only; see contract) are specified in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction), with workflow examples in [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md).
72
72
  - Constrained `job` remains a thin batch compiler, but its `click`/`fill` steps may use the same semantic locator fields as `semanticAction` so short workflows can avoid brittle selectors without adding a reusable recipe runtime, and `type` steps may expand to a bounded set of existing upstream focus/keyboard/wait/press rows for human-paced input while compacting model-visible batch text. `job` must default to fail-fast (`batch --bail`) so later mutating steps do not run after an earlier required step fails; `failFast: false` is the explicit opt-out.
73
73
  - `qa` must fail fast on failed readiness/text/selector assertions so missing expected text cannot burn the wrapper watchdog before reporting `qa-failure`. `qa.attached` must never erase existing session diagnostics; URL-opening `qa` may clear buffers to scope a fresh page load. Attached QA preserves buffers, reports that scope in `details.compiledQaPreset.checks.diagnosticsResetAtStart`, and defaults diagnostic checks off unless the caller opts into preserved-buffer failure checks.
74
- - Direct artifact workflows must create missing parent directories before spawning upstream and must verify saved files before downstream use. Simple loopback HTTP(S) anchor downloads may be saved directly by the wrapper to the requested path when that avoids upstream random-name download behavior without bypassing authenticated browser credentials. `outputPath` may write a successful result payload to a caller-requested local file and must report `details.outputFile`; it must never overwrite a screenshot, download, recording, or other browser artifact from the same result, including through a filesystem alias. Missing non-pending artifacts, including diff screenshot outputs, must never use saved/verified wording; `record start` future files are pending/open until `record stop`, not missing.
74
+ - Direct artifact workflows must create missing parent directories before spawning upstream and must verify saved files before downstream use. Native download must own the click and download, including loopback links, generated Blob exports, and redirects; the wrapper must not substitute an anchor-href fetch. `outputPath` may write a successful result payload to a caller-requested local file and must report `details.outputFile`; it must never overwrite a screenshot, download, recording, or other browser artifact from the same result, including through a filesystem alias. Missing non-pending artifacts, including diff screenshot outputs, must never use saved/verified wording; `record start` future files are pending/open until `record stop`, not missing.
75
75
 
76
76
  ### Documentation standard
77
77
 
@@ -95,6 +95,8 @@ Overlay recovery (RQ-0073, #147) recommends only `inspect-overlay-state` after a
95
95
 
96
96
  Shared native defaults are covered by `test/agent-browser.shared-defaults.test.ts`: environment/file/argv precedence, caller-owned reuse across independent Pi contexts, quit ownership, native helper idle settings, unchanged implicit cleanup, and an opt-in real local-profile fixture proving shared state and disposable-script separation. `test/agent-browser.script.test.ts` checks empty native config on helpers and cleanup; the local Electron handoff regression in `test/agent-browser.extension-electron-lifecycle.test.ts` keeps generated launch ownership despite a native session environment default; native config validation uses upstream rather than a second schema. See [shared browser defaults](../README.md#shared-browser-defaults). These tests do not promise permanent third-party authentication.
97
97
 
98
+ Browser correctness regressions cover credential-bearing filtered network details/exports, native generated-Blob downloads with actual saved bytes, accessible-name click target identity and abstention, smooth CSS-container movement, and parseable early-result JSON. Native fixture coverage complements deterministic dispatch/cleanup and output-shape checks; release qualification remains separate.
99
+
98
100
  ## Verification evidence
99
101
 
100
102
  **0.38.1 PR evidence (2026-09-17, local macOS):** the full unit/fake suite passed 1,146 tests with 27 opt-in skips; the final review fixes then passed typecheck and 64 affected tests. Generated docs, live command-reference sampling, and the 132-file packaged Pi smoke passed. The real-upstream contract and 75 native batch/fidelity checks passed, as did all eight deterministic dogfood flows. Interactive compiled-checkout Pi smoke covered example.com and react.dev, delta/ref follow-ups, conditional screenshots, and a recording batch whose persisted result had two verified artifacts and no pending artifacts. Separate native probes confirmed helper-consumed WebMCP catalog updates remain visible. These checks do not qualify release-only platform/lifecycle gates; the hosted native-Windows result is recorded on the PR.
@@ -124,11 +126,11 @@ Runtime floor note: package metadata keeps optional Pi core package peer ranges
124
126
  | Baseline section | Baseline items | Documentation | Runtime handling | Test coverage | Validation status |
125
127
  | --- | --- | --- | --- | --- | --- |
126
128
  | Built-in skills | 19 canonical tokens from baseline section `skills`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#built-in-skills). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#built-in-skills), generated baseline block, README proof section, release docs. | `needsManagedSession` keeps read-only skills inspection sessionless while preserving thin upstream passthrough; upstream `@agent-browser/sandbox` remains external package guidance, not a bundled wrapper dependency. | Runtime and extension-validation skills/provider matrix; real-upstream inspection/skills group. | Supported. |
127
- | Core page, element, navigation, and extraction commands | Canonical tokens from baseline section `core-commands`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), README quick start. | Thin passthrough with wrapper-owned JSON/session planning, ref guidance, artifact verification, page-change summaries, click-dispatch diagnostics, inspection-only recovery for upstream overlay-blocked clicks, deterministic document scroll before wheel/no-op diagnostics, shorthand compilers, and redaction. | Real-upstream core matrix (including snapshot-ref select) plus fake core matrix for passthrough, ordering, diagnostics, and compiler validation. | Supported. Upstream semantics remain upstream-owned. |
129
+ | Core page, element, navigation, and extraction commands | Canonical tokens from baseline section `core-commands`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#core-page-and-element-commands), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), README quick start. | Thin passthrough with wrapper-owned JSON/session planning, ref guidance, artifact verification, page-change summaries, click-dispatch diagnostics only after native temporary-marker target identity readback (otherwise abstaining), inspection-only recovery for upstream overlay-blocked clicks, deterministic document scroll before wheel/no-op diagnostics and instant CSS-container movement, shorthand compilers, and redaction. | Real-upstream core matrix (including snapshot-ref select) plus fake core matrix for passthrough, ordering, diagnostics, and compiler validation. | Supported. Upstream semantics remain upstream-owned. |
128
130
  | Sessions, state, tabs, frames, dialogs, and windows | 28 canonical tokens from baseline section `state-tabs-frames-dialogs`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#session-state-frames-dialogs-windows-and-inspection-commands), stateful workflow notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Stateful summaries redact credentials while preserving restore identifiers and every session/state list row; explicit targets and paths pass through. Runtime retains state artifact verification, sessionless local command planning, automatic managed restore, tab target pinning, active-target refresh after tab close, and close alias cleanup. | Extension tab/ref tests, real-upstream stable-id/label tab lifecycle, runtime session/resume tests, presentation redaction tests, lifecycle harness. | Supported. External profile/auth state remains operator-owned. |
129
- | Network, storage, artifacts, diagnostics, and performance | Canonical tokens from baseline section `network-storage-artifacts-diagnostics`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage), diagnostic sections, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Thin passthrough plus compact diagnostics, route-mock warnings, useful-but-redacted storage output, stream idempotency normalization, FPS/cursor/contact-sheet-aware recording destinations/pinning and contact-sheet image verification, truthful ref-refresh and missing-ffmpeg warnings, positive WebMCP navigation hints and catalog updates, sensitive-data redaction, timeout bounds, and cleanup-pair guidance. | Fake non-core matrix and safe real-upstream coverage for network/HAR, diff, trace/profiler, console/errors/highlight, stream, vitals, React missing-renderer, native FPS recording path/page checks and WebMCP navigation hints. | Supported. Environment-sensitive operations need suitable local/browser state. |
131
+ | Network, storage, artifacts, diagnostics, and performance | Canonical tokens from baseline section `network-storage-artifacts-diagnostics`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#page-state-finding-mouse-settings-network-and-storage), diagnostic sections, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). | Thin passthrough plus native-owned click/download execution for all downloads (including loopback, Blob exports, and redirects), compact diagnostics with identical credential redaction for page/origin/URL-filtered details and exports, route-mock warnings, useful-but-redacted storage output, stream idempotency normalization, FPS/cursor/contact-sheet-aware recording destinations/pinning and contact-sheet image verification, truthful ref-refresh and missing-ffmpeg warnings, positive WebMCP navigation hints and catalog updates, sensitive-data redaction, timeout bounds, and cleanup-pair guidance. | Fake non-core matrix and safe real-upstream coverage for network/HAR, diff, trace/profiler, console/errors/highlight, stream, vitals, React missing-renderer, native FPS recording path/page checks and WebMCP navigation hints. | Supported. Environment-sensitive operations need suitable local/browser state. |
130
132
  | Batch, auth, confirmations, setup, dashboard, devices, and AI commands | Canonical tokens from baseline section `batch-auth-setup-ai`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#batch-auth-confirmations-sessions-chat-dashboard-devices-and-setup), README security notes, release docs. | Native-tool batch stdin, generated `job`/`qa`/lookup batch plans, auth/confirmation redaction, sessionless local auth/setup/dashboard/doctor/plugin planning, upgrade-only native text normalization with error precedence and explicit-JSON results, plugin list/show JSON envelope normalization, bare-`mcp` validation with `mcp --help` preserved, timeout/cleanup guidance. | Parser/runtime plugin and MCP unit coverage; fake-upstream plugin list/show and MCP help/blocking coverage; registered upgrade text/success/nonzero/timeout/abort/JSON controls in `test/agent-browser.upgrade-output.test.ts`; real-upstream plugin list shape probe; structured input-mode tests. | Supported. Interactive side-effecting setup/auth/chat remains upstream-owned. `plugin` is local/sessionless; `mcp` is external-client-only except help; `auth login --credential-provider` resolves credentials via a plugin; `install --with-deps` failures remain upstream-owned. |
131
- | Global flags, config, providers, policy, and environment | Canonical tokens from baseline section `options-and-env`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment), README provider/setup notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode), architecture/runtime docs. | Runtime handles command discovery, value-flag prevalidation, launch-scoped flags, redacted echoes, fresh-session recovery hints, explicit sessions, provider/device launch-scoping, parent env forwarding with consistent managed idle-timeout/socket/operation-timeout overrides on top-level and helper subprocesses, subprocess completion, and package-owned Pi-scoped config for optional companion features. | Runtime tests for flags/planning/redaction/session behavior; process/ref tests for env consistency and stdio-linger completion; config/web-search/CLI tests; fake provider/specialized-skill matrix; package doctor. | Supported. Provider clouds, iOS/Appium, proxies, profiles, and credentials require external setup. |
133
+ | Global flags, config, providers, policy, and environment | Canonical tokens from baseline section `options-and-env`; see [`scripts/agent-browser-capability-baseline.mjs`](../scripts/agent-browser-capability-baseline.mjs) and generated [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment). | [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#important-global-flags-config-and-environment), README provider/setup notes, [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#sessionmode), architecture/runtime docs. | Runtime preserves caller-requested JSON for early filters, scroll results, and preparation failures without output-file prose (help/version remains native text), and handles command discovery, value-flag prevalidation, launch-scoped flags, redacted echoes, fresh-session recovery hints, explicit sessions, provider/device launch-scoping, parent env forwarding with consistent managed idle-timeout/socket/operation-timeout overrides on top-level and helper subprocesses, subprocess completion, and package-owned Pi-scoped config for optional companion features. | Runtime tests for flags/planning/redaction/session behavior; process/ref tests for env consistency and stdio-linger completion; config/web-search/CLI tests; fake provider/specialized-skill matrix; package doctor. | Supported. Provider clouds, iOS/Appium, proxies, profiles, and credentials require external setup. |
132
134
 
133
135
  ## Follow-up decision after closure
134
136
 
@@ -138,6 +140,6 @@ The compact decision index lives at [`docs/support-notes.md`](https://github.com
138
140
  | --- | --- | --- |
139
141
  | Native structured input modes (`script`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, `semanticAction`) | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
140
142
  | Electron lifecycle, manual CDP attach, desktop readiness, and tab/surface recovery | [`ELECTRON.md`](ELECTRON.md), [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#electron-desktop-apps) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
141
- | Ref lifecycle, click dispatch (unique role/name or XPath targets only; native duplicate-name mutation regression), selector recovery, rich inputs, and dense snapshots | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#selector-strategy), README pitfalls | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
143
+ | Ref lifecycle, click dispatch (eligible ref or XPath candidates require native marker identity readback; unproven identity abstains), selector recovery, rich inputs, and dense snapshots | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#selector-strategy), README pitfalls | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
142
144
  | Diagnostics, artifacts, QA/network classification, timeout recovery, scroll/combobox/recording guidance | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md), [`RELEASE.md`](RELEASE.md) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
143
145
  | Package config and optional web search, including the default Exa mode and bounded filters | [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#optional-companion-web-search), README optional package config, [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md#optional-package-config-and-companion-web-search) | [`docs/support-notes.md`](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/docs/support-notes.md) |
@@ -137,7 +137,7 @@ Use exactly one top-level input per call:
137
137
  | When you need | Use | Notes |
138
138
  | --- | --- | --- |
139
139
  | One-shot loops, conditional page branches, or multi-page aggregation | `script` | Sandboxed async JavaScript with `browser()` and `emit()`; unique isolated session, bounded calls/output/time, always closed. Not for profiles, attachments, persistent auth, or reusable named recipes. |
140
- | Routine browse, click, fill, screenshots, upstream commands | `args` | Default path: `open` → `snapshot -i` → `click`/`fill` `@eN` → `snapshot -i` after navigation or DOM changes. Do not pass `--json`; the wrapper injects it (see [Wrapper `--json`](#wrapper-json)). |
140
+ | Routine browse, click, fill, screenshots, upstream commands | `args` | Default path: `open` → `snapshot -i` → `click`/`fill` `@eN` → `snapshot -i` after navigation or DOM changes. Omit `--json` unless you need JSON text (see [Wrapper `--json`](#wrapper-json)). |
141
141
  | Stable visible role/text/label/placeholder targets | `semanticAction` | Compiles to upstream `find` or `select`; optional `session` for a named upstream browser. |
142
142
  | Short multi-step smoke or evidence flows | `job` or `qa` | Both compile to `batch`; `qa` may reclassify diagnostics as failure. |
143
143
  | Desktop Electron apps (list/launch/probe/cleanup) | `electron` | Wrapper-owned lifecycle; not for ordinary websites. |
@@ -164,7 +164,9 @@ For link and button text, use the **exact** visible label from the latest `snaps
164
164
 
165
165
  ## Wrapper `--json`
166
166
 
167
- The extension always plans normal browser commands with `--json` prepended in `effectiveArgs` so upstream returns structured JSON for presentation and `details`. **Do not** include `--json` in caller `args`; it is unnecessary and can confuse planning or transcript hooks that treat caller-requested JSON differently. Plain-text inspection (`--help`, `--version`) keeps its own output shape. Read-only skills and local/setup commands such as auth/profile/setup, `session list`, and syntactically local state lifecycle operations skip implicit session injection as documented under `sessionMode`. Upstream session/state rows and targets remain visible, and state/config/path operations pass through unchanged.
167
+ When explicitly requested, visible JSON remains parseable for early preparation results too: snapshot/network filters, scroll results, and preparation failures use the same JSON envelope as ordinary execution. Diagnostic metadata remains in `details`; `outputPath` notices do not append prose to JSON. Help/version stays native text.
168
+
169
+ The extension always plans normal browser commands with `--json` prepended in `effectiveArgs` so upstream returns structured JSON for presentation and `details`. Omit `--json` in caller `args` for ordinary prose. Include it when you need the visible tool text as a parseable JSON envelope; structured `details` remain available in both modes. Plain-text inspection (`--help`, `--version`) keeps its own output shape. Read-only skills and local/setup commands such as auth/profile/setup, `session list`, and syntactically local state lifecycle operations skip implicit session injection as documented under `sessionMode`. Upstream session/state rows and targets remain visible, and state/config/path operations pass through unchanged.
168
170
 
169
171
  ## Experimental WebMCP
170
172
 
@@ -229,11 +231,11 @@ URL-less `open` is executed as native `get url`, including effective raw/stdin b
229
231
  - If a session lands on the wrong page or tab, an interaction changes origin unexpectedly, or an open call returns blocked, blank, or otherwise unexpected results, use tab list / tab <tab-id-or-label> / snapshot -i to recover state before retrying different URLs or fallback strategies. For headed demos, put --headed on the first launch with sessionMode=fresh and verify with screenshot/tab/get-url evidence because tool success cannot prove the OS window is visible to the user. For desktop readiness, prefer real conditions first: wait --text, wait --url, wait --fn, wait --load <state>, wait --download, or qa.attached; for disappearance checks, use wait --fn predicates instead of stale upstream-help examples like wait <selector> --state hidden. Use electron.probe/status for wrapper-owned launch health or target mismatch. Fixed waits are a last resort: their duration is positional (wait <ms>, not wait --time <ms>). Use explicit --timeout or top-level timeoutMs for legitimately slow waits, and treat a successful payload like "waited":"timeout" as elapsed time only—verify completion with an observed condition, fresh snapshot, or screenshot.
230
232
  - For feed, timeline, or inbox reading tasks, focus on the main timeline/list region and read the first item there rather than unrelated composer or sidebar content.
231
233
  - For read-only browsing tasks, use read <url> for documentation or other unstructured text without requiring a Chrome page, or read with no URL for rendered active-tab DOM. Prefer the current snapshot, structured ref labels, getters, or scoped eval --stdin when you need interactive structure or targeted page state. Only click into media viewers, detail routes, or new pages when the current view does not contain the needed information.
232
- - For downloads, prefer download <selector> <path> when an element click should save a file; simple loopback anchor downloads are saved to the requested path when the wrapper can resolve an HTTP(S) href. Do not rely on click alone when you need the downloaded file on disk.
234
+ - For downloads, prefer download <selector> <path> when an element click should save a file; native download owns the click and download, including loopback links, generated Blob exports, and redirects. Do not rely on click alone when you need the downloaded file on disk.
233
235
  - On dashboards with nested scroll containers, verify scroll with a screenshot or fresh snapshot -i; if the viewport did not move, details.data.scrolled may be false/noMovement true and you should prefer scrollintoview <@ref> or target the actual scrollable region with scroll <selector> <dir> [px|percent]. For native selects, use select <selector> <value...> (or semanticAction/job select) instead of clicking option refs; for custom comboboxes, a click/semanticAction may only focus the field, so re-snapshot and use keyboard type <text> for focused input, press ArrowDown or press Enter, or visible option refs. Raw type requires both <selector> and <text>.
234
236
  - When using eval --stdin, scope checks and actions to the target element or route whenever possible instead of relying on broad page-wide text heuristics.
235
237
  - When using eval --stdin for extraction, pass the JavaScript through the native tool stdin field, not as an extra args token after --stdin, and return the value you want instead of relying on console.log as the primary result channel. Prefer plain expressions like ({ title: document.title }) or explicitly invoked functions like (() => ({ title: document.title }))(); use outputPath when the eval/get/snapshot data should be saved as a durable local file, but never reuse a screenshot, download, recording, or other browser artifact destination as outputPath. If a function-shaped snippet returns {}, details.evalStdinHint may warn that the function was serialized instead of called. Local file pages and caller-selected output paths are supported when upstream allows them. If get text on a broad CSS selector surfaces details.selectorTextVisibility or selectorTextVisibilityAll, prefer a visible @ref, a more specific selector, or the inspect-visible-text-candidates nextAction over hidden tab content.
236
- - When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If an upstream click failure says the target is covered by another element at the target's click point, use the inspect-overlay-state nextAction to refresh refs and inspect the blocker before deciding whether to retry; do not blindly repeat the blocked click. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.
238
+ - When details.pageChangeSummary is present, use changeType and summary as a compact signal for navigation, DOM mutation, confirmations, or artifacts; when nextActionIds is set, match those ids to entries in details.nextActions (or per-step nextActions inside batch) for concrete follow-up payloads instead of inferring from prose alone. Click-dispatch probes require native get attr readback of a temporary marker on the candidate target; unproven ref or XPath identity leaves the native click unverified, not failed. If details.clickDispatch reports a click-dispatch miss, refresh/inspect/retry the real click first; for static local fixtures only, an explicit eval --stdin programmatic .click() can exercise app handlers, but treat it as an untrusted scripted workaround and never use it to bypass stop-before-submit/order/purchase boundaries. If an upstream click failure says the target is covered by another element at the target's click point, use the inspect-overlay-state nextAction to refresh refs and inspect the blocker before deciding whether to retry; do not blindly repeat the blocked click. If a no-navigation click surfaces details.overlayBlockers, inspect the fresh snapshot evidence before using a close/dismiss candidate nextAction; ordinary page chrome without dialog/alertdialog evidence should not trigger this diagnostic.
237
239
  - On upstream 0.38+, use snapshot --delta for native full/unchanged/changed revisions; --delta --full resets its baseline. Surviving DOM nodes keep refs, but refresh after navigation or replacement. Wrapper filters use full trees. screenshot --if-changed or --threshold 0.01 can return changed:false with no path or image; use ordinary screenshots when a file is required. --input-mode instant|smooth|human controls sticky pointer behavior; click/drag --human and mouse move --duration/--steps/--human/--seed pass through. record start/restart accept --cursor, --contact-sheet, and --contact-sheet-threshold; stop and verify both video and reported contact sheet. auth login <name> --no-navigate fills a prepared login page only after native origin checks. WebMCP catalog updates are untrusted page data; webmcp list supplies full schemas.
238
240
  - When commands save or spill files (screenshots, downloads, PDFs, traces, recordings, HAR, large snapshot spills), use the user's exact requested paths when given and treat paths as provisional until details.artifactVerification shows every row verified: branch on missingCount, pendingCount, unverifiedCount, per-entry state, and optional limitation before downstream file use or PASS/FAIL reporting.
239
241
  - For evidence-only screenshots, QA captures, or other audit artifacts, save to an explicit path and branch on details.artifactVerification plus details.artifacts before reporting PASS/FAIL; do not require vision review of inline image attachments unless the user asked for visual inspection.
@@ -840,13 +842,13 @@ Ref preflight details (command taxonomy in `extensions/agent-browser/lib/command
840
842
 
841
843
  **Unknown-command getter hints (failure presentation):** `buildErrorPresentation` in `extensions/agent-browser/lib/results/presentation/errors.ts` only runs this path when upstream error text (after model-facing redaction) matches `unknown command`, `unknown subcommand`, or `unrecognized command` (case-insensitive) **and** the failed invocation’s primary command token is one of `attr`, `count`, `html`, `text`, `title`, `url`, or `value`. Visible text then includes a grouped-`get` hint line plus per-token guidance (`get text <selector>`, `get html …`, `get attr …`, `get count …`, `get value …`, `get title`, `get url`). Machine `nextActions` with ids `use-get-title` / `use-get-url` are emitted only for `title` / `url`, with `params.args` optionally prefixed by `--session <name>` when the failed call targeted a named session. If the error string already contains `Agent-browser hint:` from selector recovery (stale-ref or unsupported selector dialect appendages), the getter block is skipped so two stacked `Agent-browser hint:` headers are not emitted.
842
844
 
843
- For `network requests`, `details.nextActions` is bounded to one selected safe request ID, preferring actionable failed rows, then API/fetch-like rows, then benign failed rows, then the first request with a safe ID. Detail/filter/HAR actions use `params.args` and preserve known `--namespace <name>` / `--session <name>` prefixes when the current presentation has `details.namespace` / `details.sessionName`; source-candidate actions use `params.networkSourceLookup` with the selected `requestId` plus `namespace` / `session` when known and are only emitted for actionable failed rows that the failed-request analyzer can correlate. A `clear-network-requests-before-repro` action can run `network requests --clear` so the next reproduction starts with a current-page-focused diagnostic buffer. Wrapper-side `network requests --current-page` / `--current-origin` renders only rows whose URL matches the active page origin, while `--current-url` renders exact active-document URL matches; those wrapper-only flags are removed before upstream spawn and reported in `details.networkRequestsPageFilter`. URLs and query strings are not copied into action params; path filters are skipped when they look sensitive or too large. If the wrapper has observed `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows also add `details.networkRouteDiagnostics[]` with `{ reason, routePattern, mode, requestId?, requestUrl?, summary }` and prepend executable route-mock next actions (`inspect-routed-network-request`, `start-network-har-capture-for-route-mock`) before generic request follow-ups; same-origin/CORS fixture retry guidance stays in visible prose. The route tracker is wrapper-session-local, updated on successful `network route`/`network unroute`, and cleared when that session closes or is replaced.
845
+ For `network requests`, `details.nextActions` is bounded to one selected safe request ID, preferring actionable failed rows, then API/fetch-like rows, then benign failed rows, then the first request with a safe ID. Detail/filter/HAR actions use `params.args` and preserve known `--namespace <name>` / `--session <name>` prefixes when the current presentation has `details.namespace` / `details.sessionName`; source-candidate actions use `params.networkSourceLookup` with the selected `requestId` plus `namespace` / `session` when known and are only emitted for actionable failed rows that the failed-request analyzer can correlate. A `clear-network-requests-before-repro` action can run `network requests --clear` so the next reproduction starts with a current-page-focused diagnostic buffer. Wrapper-side `network requests --current-page` / `--current-origin` renders only rows whose URL matches the active page origin, while `--current-url` renders exact active-document URL matches; those wrapper-only flags are removed before upstream spawn and reported in `details.networkRequestsPageFilter`. Filtered results pass through the same `redactPresentationData` credential redactor as ordinary network diagnostics before entering `details.data` or an `outputPath` export. URLs and query strings are not copied into action params; path filters are skipped when they look sensitive or too large. If the wrapper has observed `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows also add `details.networkRouteDiagnostics[]` with `{ reason, routePattern, mode, requestId?, requestUrl?, summary }` and prepend executable route-mock next actions (`inspect-routed-network-request`, `start-network-har-capture-for-route-mock`) before generic request follow-ups; same-origin/CORS fixture retry guidance stays in visible prose. The route tracker is wrapper-session-local, updated on successful `network route`/`network unroute`, and cleared when that session closes or is replaced.
844
846
 
845
847
  For `batch`, each `batchSteps[]` entry can carry its own `nextActions` for that step’s success or failure. Top-level `details.nextActions` on a failed batch duplicates `batchFailure.failedStep.nextActions` so callers can read one aggregate object. On a fully successful batch, top-level `nextActions` may still list artifact follow-ups derived from the combined step artifacts.
846
848
 
847
849
  `pageChangeSummary` is an optional compact summary for mutation-prone and artifact-producing commands. It includes `changeType` (`"navigation"`, `"mutation"`, `"artifact"`, or `"confirmation"`), `observed`, `command`, a readable `summary`, optional `title`/`url`, optional `artifactCount` or `savedFilePath`, and `nextActionIds`. `observed: false` means upstream dispatched a mutation-capable action but the wrapper did not observe an application change; standalone results also append a visible `Action dispatched; application change unverified` warning. The wrapper maintains explicit command/subcommand capability checks through `isPageChangeSummaryCommand` in `extensions/agent-browser/lib/command-taxonomy.ts`: those commands still emit a `mutation`-typed summary when upstream JSON lacks navigation metadata, as long as no stronger signal (artifact, saved path, navigation fields, or pending confirmation) applies. That capability is independent from `invalidatesBatchRefs` and `triggersPostMutationSnapshot`, so artifact summaries like `download` / `screenshot` and guarded-but-non-invalidating `fill` are documented directly in the capability table instead of implied by broad set spreading. Commands outside that set omit `pageChangeSummary` unless the parsed payload shows navigation, a confirmation prompt, saved files, or artifacts—including read-only inspection commands, which normally have no summary unless one of those signals appears. For `batch`, the top-level summary favors artifact rollups when any step produced artifacts; otherwise it synthesizes an observed-or-unverified summary from step evidence. Visible batch output promotes dispatch-only mutation evidence before step details and states that fixed waits are not postconditions. Agents should verify URL/text/state or an external receipt for important mutations before continuing.
848
850
 
849
- `clickDispatch` may appear after a **top-level non-Electron** direct `click` when the wrapper installed a target-specific DOM-event probe, upstream reported success, and the post-click probe found no trusted DOM event reached the resolved target. Target-specific probes cover `xpath=` targets and role-gated `@e…` refs when the latest wrapper-tracked snapshot has role/name metadata; eligible ref roles are `button`, `checkbox`, `menuitem`, `radio`, `switch`, and `tab`, and the role/name must be unique in both the saved snapshot and the live candidates. Duplicate-name refs pass through without a probe because snapshot order can change and is not proof of target identity. Raw `find … click` locator calls, including compiled `semanticAction` clicks that still execute as upstream `find`, are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. It does **not** take a fresh pre-click snapshot because that could recycle upstream refs before the intended click. The wrapper does **not** replay clicks in-page. On a miss it marks the tool failed, appends `Click dispatch diagnostic: …`, and sets `clickDispatch.status` to `"no-native-event-observed"` with `reason: "native-click-produced-no-target-dom-event"`, `nativeEventCount`, and a redacted `target` descriptor (`kind: "xpath"` plus `selector`, or `kind: "accessible"` plus `refId`, `role`, and redacted `name`). `details.nextActions` gains `inspect-click-dispatch-miss` (`snapshot -i`) and `retry-click-after-dispatch-miss` (same upstream click argv, session-prefixed when applicable). If a local static fixture must be exercised despite this diagnostic, a caller may explicitly run a programmatic activation via `eval --stdin` such as `document.querySelector(...).click()`, but that emits an untrusted scripted event and is only a debugging/workaround path; it must not be used as proof that real user-like clicking works or to bypass prompt stop boundaries. This diagnostic is only for standalone top-level direct click calls; `find` locator clicks and `batch`/`job`/`qa` click steps remain upstream-owned behavior.
851
+ `clickDispatch` may appear after a **top-level non-Electron** direct `click` when the wrapper installed a target-specific DOM-event probe, upstream reported success, and the post-click probe found no trusted DOM event reached the resolved target. Target-specific probes cover `xpath=` targets and role-gated `@e…` refs when the latest wrapper-tracked snapshot has role/name metadata; eligible ref roles are `button`, `checkbox`, `menuitem`, `radio`, `switch`, and `tab`, and the role/name must be unique in both the saved snapshot and the live candidates. Before trusting either a ref or XPath candidate, the wrapper sets a per-probe temporary DOM attribute and requires native `get attr <original-selector> <marker>` readback to match it. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and leaves the native click to run without dispatch verification. This does not implement a new accessible-name algorithm or promise that every ref is observable. Duplicate-name refs pass through without a probe because snapshot order can change and is not proof of target identity. Raw `find … click` locator calls, including compiled `semanticAction` clicks that still execute as upstream `find`, are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. It does **not** take a fresh pre-click snapshot because that could recycle upstream refs before the intended click. The wrapper does **not** replay clicks in-page. On a miss it marks the tool failed, appends `Click dispatch diagnostic: …`, and sets `clickDispatch.status` to `"no-native-event-observed"` with `reason: "native-click-produced-no-target-dom-event"`, `nativeEventCount`, and a redacted `target` descriptor (`kind: "xpath"` plus `selector`, or `kind: "accessible"` plus `refId`, `role`, and redacted `name`). `details.nextActions` gains `inspect-click-dispatch-miss` (`snapshot -i`) and `retry-click-after-dispatch-miss` (same upstream click argv, session-prefixed when applicable). If a local static fixture must be exercised despite this diagnostic, a caller may explicitly run a programmatic activation via `eval --stdin` such as `document.querySelector(...).click()`, but that emits an untrusted scripted event and is only a debugging/workaround path; it must not be used as proof that real user-like clicking works or to bypass prompt stop boundaries. This diagnostic is only for standalone top-level direct click calls; `find` locator clicks and `batch`/`job`/`qa` click steps remain upstream-owned behavior.
850
852
 
851
853
  `promptGuard` may appear on wrapper-blocked calls only for concrete, machine-checkable prompt requirements. `reason: "requested-artifacts-missing-before-close"` blocks `close` / `quit` / `exit` when the prompt used a direct screenshot/recording creation phrase with a destination such as `here`, `at`, `as`, or `to` and the session artifact manifest has not verified that exact path; a destination heading can carry that intent across contiguous plain or Markdown-bulleted path-only list lines. Bare, review-only, fenced-reference, conditional, permissive/uncertain, directly negated, and Pi clipboard/attachment image/video paths are treated as input, not output requirements, while subordinate requirements such as “do not close until you save” remain output intent. A recording-availability qualifier may precede, appear within, or follow its path/list and is scoped through the next path boundary; if the same path appears more than once, any required occurrence takes precedence. Markdown-link destinations resolve to the destination path. Explicitly optional artifacts are not close requirements. The classifier is deliberately conservative, so use wording such as `Save a screenshot here: <path>` when machine-enforced close blocking matters. Optional recording paths are only required when recording appears available. The wrapper does **not** parse broad user/business intent such as “do not place the order” or “do not post anything” into click/key blocks; agents must follow those instructions themselves. Prompt guards return `failureCategory: "policy-blocked"` and `validationError` text instead of invoking upstream.
852
854
 
@@ -938,7 +940,7 @@ Additional structured fields can appear when relevant:
938
940
  - `networkRequestsPageFilter` after wrapper-side `network requests --current-page`, `--current-origin`, or `--current-url`. Shape: `{ cleanArgs, currentUrl, mode, matchedRows, totalRows }`; the visible rows and `details.data.requests` / `items` / `entries` are filtered while the active session page target is read with `get url`.
939
941
  - `richInputRecovery` after a raw `find` or compiled `semanticAction` `fill` fails with `selector-not-found` and the same current-ref diagnostic finds exact editable `searchbox` / `textbox` candidates. Shape follows `RichInputRecoveryDiagnostic` in `extensions/agent-browser/lib/results/selector-recovery.ts`: `{ candidates, inputMethodHint, nextActionIds, summary, target }`, where each candidate has `ref`, `role`, `name`, `focusArgs`, `clickArgs`, and `reason`. Visible text appends `Rich input recovery`, and `details.nextActions` gains ids from `getAgentBrowserRichInputRecoveryNextActionIds`: `focus-current-editable-ref` / `click-current-editable-ref` (or numbered variants). These actions are bounded to focus/click/inspect-style recovery: they do not include the fill text, do not press `Enter`, and do not submit. After the right current editable ref is focused, use `keyboard type` for framework-controlled editors that require real key events. Use paste-like `keyboard inserttext` only with separate application-state verification, and submit only when explicitly required by the flow.
940
942
  - unsupported `scrollintoview text=<label>` / `scrollinto text=<label>` fails before upstream dispatch, directly or inside an effective raw/stdin batch row, because current upstream can report success without moving the page; help forms pass through unchanged. Visible failure text and `details.nextActions` both return the session-scoped `scroll-semantic-text-target` (`find text <label> hover`) when hover side effects are acceptable, plus `refresh-refs-for-scroll-target` (`snapshot -i`) before `scrollintoview <@ref>`. CSS, `xpath=...`, and current `@e…` targets remain native pass-through.
941
- - `scrollPage` when the wrapper moves `document.scrollingElement` directly for `scroll <up|down|left|right> [px|percent]` or `scroll to end|top`; it temporarily disables smooth scrolling so immediate before/after offsets are reliable, returns `{ request, result }`, and includes `exitCode: 0` on success. Directional document no-movement falls through to upstream wheel behavior so nested panes still work. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled and report `details.scrollContainer`. All scroll helper shims are skipped when startup-scoped flags are present so the requested browser/profile launches before any helper command.
943
+ - `scrollPage` when the wrapper moves `document.scrollingElement` directly for `scroll <up|down|left|right> [px|percent]` or `scroll to end|top`; it temporarily disables smooth scrolling so immediate before/after offsets are reliable, returns `{ request, result }`, and includes `exitCode: 0` on success. Directional document no-movement falls through to upstream wheel behavior so nested panes still work. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` before measuring movement so smooth-scroll CSS cannot cause a premature no-movement failure, and report `details.scrollContainer`. All scroll helper shims are skipped when startup-scoped flags are present so the requested browser/profile launches before any helper command.
942
944
  - `scrollNoop` after a nominally successful large **top-level** upstream scroll fallback on an existing or fresh managed session when wrapper-side read-only probes before and after the command show no change in `window.scrollX` / `window.scrollY` and no change in the sampled prominent scrollable containers. The wrapper reclassifies this outcome as `failureCategory: "upstream-error"` rather than claiming the page scrolled. To avoid pre-launching a session without caller startup state, this probe is skipped for small pixel scrolls, calls that would create a managed session only for the probe, and invocations with startup-scoped flags such as `--profile`, `--state`, `--restore`, `--namespace`, `--session-name`, `--cdp`, providers, init scripts, or similar launch settings. Shape: `{ reason: "no-observed-scroll-position-change", message, before, after, recommendations }`; `before` / `after` include viewport dimensions, document scroll dimensions, and up to ten sampled container descriptors plus scroll offsets. Container descriptors use only sample index, tag name, and ARIA role; DOM ids/classes are intentionally not stored. This diagnostic is conservative evidence that the page-level scroll likely missed a nested pane, not proof that every app-specific region is unchanged. Visible text starts with `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, sets `details.data.scrolled` to `false` / `details.data.noMovement` to `true`, and `details.nextActions` gains `inspect-after-noop-scroll` (`snapshot -i`) plus `verify-noop-scroll-visually` (`screenshot`), session-prefixed when applicable.
943
945
  - `comboboxFocus` after a successful explicit combobox-targeted `click` / `fill` / `find … click|fill` (for example `semanticAction` with role `combobox`, including when that semantic action resolves through a current visible `@ref` before execution) when a read-only probe sees the active element is combobox-like, `aria-expanded` is explicitly present (`false` or `true`), and no visible `listbox` / `option` / menu option elements are open. Shape: `{ reason: "focused-combobox-without-visible-options", message, activeElement, visibleListboxCount, visibleOptionCount, recommendations }`; `activeElement` includes bounded role/tag/expanded/hasPopup/name metadata with normal text redaction. Visible text appends `Combobox diagnostic: focused combobox did not expose visible options`, and `details.nextActions` gains `inspect-focused-combobox` (`snapshot -i`), `try-open-combobox-with-arrow` (`press ArrowDown`), and `try-open-combobox-with-enter` (`press Enter`), session-prefixed when applicable. The diagnostic is deliberately gated to explicit combobox-targeted calls to avoid extra probes or false positives on ordinary clicks/textboxes.
944
946
  - `recordingDependencyWarning` after a successful `record start` or `record restart` when the wrapper cannot find an executable `ffmpeg` on the Pi process `PATH`. Shape: `{ reason: "ffmpeg-missing-for-recording", dependency: "ffmpeg", command, message, recommendations }`. Visible text appends `Recording dependency warning: ffmpeg not found on PATH`. This is a non-blocking warning after native success, not a preflight or encoding check. Native 0.37 validates ffmpeg at startup; older supported natives may defer failure. Treat the pending output as unverified, stop and check its result, then install ffmpeg before starting a new recording.
@@ -951,10 +953,10 @@ Additional structured fields can appear when relevant:
951
953
  - `managedSessionHeadedAutosaveDisabled: true` is the narrower compatibility marker that the targeted session uses the wrapper's default interval `0`, rather than an explicit caller interval. It accompanies `managedSessionHeadedAutosaveInterval: "0"` on active rows and remains omitted for explicitly configured autosave.
952
954
  - `managedSessionOutcome` after a managed-session plan reaches process execution (`buildManagedSessionOutcome` / `formatManagedSessionOutcomeText` in `extensions/agent-browser/lib/orchestration/browser-run/session-state.ts`). Populated when `buildExecutionPlan` injects an extension-managed implicit or fresh `--session`, and also when a successful explicit `--session <current-wrapper-managed-session> close` closes the current managed session. It remains omitted for unrelated explicit user-managed sessions and for sessionless inspection/local paths that skip injection. Successful nested-batch lifecycle rows are evaluated in order: a terminal close reports and replays `status: "closed"` even when aggregate artifact verification makes the tool result fail; a later lifecycle-proven browser launch (including a post-close `record stop`) keeps the session active, an explicitly non-launching diagnostic leaves it closed, and an unknown row stays conservatively active even when the failed batch was the first managed call. Fields: `status` (`created`, `replaced`, `unchanged`, `closed`, `preserved`, or `abandoned`), `sessionMode`, `attemptedSessionName`, `previousSessionName`, `currentSessionName`, optional `currentSessionNamespace`, optional `replacedSessionName`, optional `replacedSessionNamespace`, optional `replacedSessionClosed` (false means automatic close failed and the previous session remains wrapper-owned/restorable for explicit cleanup), `activeBefore`, `activeAfter`, `succeeded`, and `summary` (machine-oriented; may include generated session names). Use `currentSessionNamespace` with `currentSessionName` when following preserved-session recovery actions; retry-fresh actions stay in the attempted namespace. Model-visible echo: when `sessionMode` is `"fresh"` **and** `succeeded` is false, or when `replacedSessionClosed` is false after a replacement, the wrapper appends action-oriented `Managed session outcome` and `Recovery` lines without repeating generated session ids in visible prose; session names remain in `details.managedSessionOutcome`. Failed fresh launches may also append `details.nextActions` such as `run-agent-browser-doctor`, `verify-current-managed-session`, `snapshot-current-managed-session`, or `retry-fresh-managed-session`. When other trailing diagnostic prose is also emitted in the same result, that block is concatenated **after** semantic-action candidate lines, overlay/selector-visibility tails, eval hints/warnings, and `Timeout partial progress` (see `rawAppendedDiagnosticText` in `extensions/agent-browser/lib/orchestration/browser-run/final-result.ts`). For `"auto"` failures the same struct may appear on `details` without that extra line. When post-upstream analysis (for example **`qa`** preset failure) flips the overall tool result after a successful batch, or a fresh `job`/batch opens the requested page and then a later step fails, the managed-session transition still reflects that the fresh browser became current. The visible recovery says the fresh launch became current and points to `failureCategory` / `qaPreset` / `batchFailure` for the post-launch failure instead of telling the agent that the old session was preserved.
953
955
  - `imagePath` / `imagePaths` for Pi inline image attachments from the **`screenshot`** command (including batched screenshot steps). **`diff screenshot`** still records the diff output as an `image`-kind entry in `details.artifacts`, but it does **not** populate `imagePath` / `imagePaths` or attach an inline image: only plain `screenshot` is treated as a trusted live-capture path for automatic inlining (`isTrustedScreenshotOutput` in `extensions/agent-browser/lib/results/presentation/artifacts.ts`).
954
- - `artifacts` for saved files such as screenshots, `state save` outputs, `diff screenshot` diff images, PDFs, downloads, `wait --download` / `wait -d` files, traces, CPU profiles, completed video recordings, path-bearing HAR captures, and future recording output paths reported by `record start` / `record restart`. Non-file URL payloads such as `data:` / `blob:` / `http(s):` values are not treated as verified local artifacts. For direct artifact commands and batch artifact steps, the wrapper creates parent directories for requested paths before spawning upstream. Filesystem `mkdir` failures at this shared preparation boundary return `validation-error`, `agentBrowserStarted: false`, the attempted directory and `verify-artifact-path` guidance. Raw batch strings are never rewritten; use absolute artifact paths because the daemon's cwd may differ from Pi's. Each artifact includes the original saved or requested `path`, resolved `absolutePath`, `kind`/`artifactType`, optional `mediaType`, optional `extension`, best-effort disk metadata such as `exists`, `sizeBytes`, and `updatedAtMs`, plus `requestedPath`, `status`, `cwd`, `session`, `namespace`, and `tempPath` when applicable. `requestedPath` is retained only when known from the caller, separately from reported/resolved locations; a differing screenshot report remains in `tempPath` and is displayed as `Reported path`, whether it is a temporary file or a canonical path alias. Ordinary file `mediaType` values come from bounded PNG/JPEG/GIF/WebP header recognition, not suffixes; unknown, missing, unreadable or truncated headers leave it undefined. Header recognition is not full-file format validation. Inline screenshot attachments use the same byte classifier and existing size limit. Direct-anchor downloads retain their response Content-Type metadata. For commands that create/update artifacts, a path that existed but was not updated during this command uses `status: "stale"`; observational `wait --download` may accept a file completed just before the wait began. Pending `record start` / `record restart` artifacts use `status: "pending"`, omit `exists` rather than reporting false, and include `recordingState: "openRecording"` / `willExistOnStop: true`. Within one Pi extension process, the wrapper keeps an unbounded transcript-backed active-recording reservation index separate from the bounded artifact manifest, keyed by canonical namespace plus session; still-live process-owned reservations survive branch switches, while known closures are appended after tree navigation and during shutdown/reload so a close on one branch cannot be resurrected after returning to an older branch. Persisted active reservations require absolute storage paths and cwd; their display paths may remain relative. If a journal append fails, the next serialized browser boundary, tree navigation, or shutdown retries all current reservations and known closures. `recordingPersistenceWarning` and visible warning text remain present while restart protection is not durable; successful recovery is quiet and cleanup still runs. Artifact lifecycle calls, explicit `wait --download <path>` / `wait -d <path>` destinations, and result `outputPath` writes serialize around the global destination check/update, every successful direct, ordered nested-batch, fresh-replacement, script, Electron, or shutdown close retires only its exact identity at that lifecycle point, and destination reuse is rejected through lexical, existing or dangling symlink, hardlink, full Unicode-fold, or macOS/Windows case aliases. Batch preflight rejects `record start` / `record restart` after a close row because upstream can report a recording that did not start; split those operations into separate calls. A `No recording in progress` stop failure, direct or nested, first checks the matching native receipt once. Retirement preserves the receipt and freshly checked file metadata; it never turns an existing file into a missing file merely because no recording is active. A later successful batch recording row opens its new pending path normally. Batch preflight applies the same distinct-destination rule to the steps upstream will execute: raw argument command strings exclusively when any exist, stdin arrays only otherwise; upstream-ignored stdin rows cannot fail artifact preflight, add pending recordings, or create parent directories. Parent directories are prepared for the effective steps in both modes; raw argument strings are never rewritten, so the screenshot absolute-path normalization and tracked path request apply to stdin rows only. Outer CLI globals are removed before artifact parsing, but native batch row operands stay literal: `pdf --quick ignored.pdf` targets `--quick`, not `ignored.pdf`. Reservation checks, preparation, and requested-path presentation follow that same distinction. Recording path/URL consumers skip complete numeric `--fps` pairs without rewriting argv; native still validates rate, format and extra arguments. FPS-only calls keep the intended pinned tab.
956
+ - `artifacts` for saved files such as screenshots, `state save` outputs, `diff screenshot` diff images, PDFs, downloads, `wait --download` / `wait -d` files, traces, CPU profiles, completed video recordings, path-bearing HAR captures, and future recording output paths reported by `record start` / `record restart`. Non-file URL payloads such as `data:` / `blob:` / `http(s):` values are not treated as verified local artifacts. For direct artifact commands and batch artifact steps, the wrapper creates parent directories for requested paths before spawning upstream. Filesystem `mkdir` failures at this shared preparation boundary return `validation-error`, `agentBrowserStarted: false`, the attempted directory and `verify-artifact-path` guidance. Raw batch strings are never rewritten; use absolute artifact paths because the daemon's cwd may differ from Pi's. Each artifact includes the original saved or requested `path`, resolved `absolutePath`, `kind`/`artifactType`, optional `mediaType`, optional `extension`, best-effort disk metadata such as `exists`, `sizeBytes`, and `updatedAtMs`, plus `requestedPath`, `status`, `cwd`, `session`, `namespace`, and `tempPath` when applicable. `requestedPath` is retained only when known from the caller, separately from reported/resolved locations; a differing screenshot report remains in `tempPath` and is displayed as `Reported path`, whether it is a temporary file or a canonical path alias. Ordinary file `mediaType` values come from bounded PNG/JPEG/GIF/WebP header recognition, not suffixes; unknown, missing, unreadable or truncated headers leave it undefined. Header recognition is not full-file format validation. Inline screenshot attachments use the same byte classifier and existing size limit. For commands that create/update artifacts, a path that existed but was not updated during this command uses `status: "stale"`; observational `wait --download` may accept a file completed just before the wait began. Pending `record start` / `record restart` artifacts use `status: "pending"`, omit `exists` rather than reporting false, and include `recordingState: "openRecording"` / `willExistOnStop: true`. Within one Pi extension process, the wrapper keeps an unbounded transcript-backed active-recording reservation index separate from the bounded artifact manifest, keyed by canonical namespace plus session; still-live process-owned reservations survive branch switches, while known closures are appended after tree navigation and during shutdown/reload so a close on one branch cannot be resurrected after returning to an older branch. Persisted active reservations require absolute storage paths and cwd; their display paths may remain relative. If a journal append fails, the next serialized browser boundary, tree navigation, or shutdown retries all current reservations and known closures. `recordingPersistenceWarning` and visible warning text remain present while restart protection is not durable; successful recovery is quiet and cleanup still runs. Artifact lifecycle calls, explicit `wait --download <path>` / `wait -d <path>` destinations, and result `outputPath` writes serialize around the global destination check/update, every successful direct, ordered nested-batch, fresh-replacement, script, Electron, or shutdown close retires only its exact identity at that lifecycle point, and destination reuse is rejected through lexical, existing or dangling symlink, hardlink, full Unicode-fold, or macOS/Windows case aliases. Batch preflight rejects `record start` / `record restart` after a close row because upstream can report a recording that did not start; split those operations into separate calls. A `No recording in progress` stop failure, direct or nested, first checks the matching native receipt once. Retirement preserves the receipt and freshly checked file metadata; it never turns an existing file into a missing file merely because no recording is active. A later successful batch recording row opens its new pending path normally. Batch preflight applies the same distinct-destination rule to the steps upstream will execute: raw argument command strings exclusively when any exist, stdin arrays only otherwise; upstream-ignored stdin rows cannot fail artifact preflight, add pending recordings, or create parent directories. Parent directories are prepared for the effective steps in both modes; raw argument strings are never rewritten, so the screenshot absolute-path normalization and tracked path request apply to stdin rows only. Outer CLI globals are removed before artifact parsing, but native batch row operands stay literal: `pdf --quick ignored.pdf` targets `--quick`, not `ignored.pdf`. Reservation checks, preparation, and requested-path presentation follow that same distinction. Recording path/URL consumers skip complete numeric `--fps` pairs without rewriting argv; native still validates rate, format and extra arguments. FPS-only calls keep the intended pinned tab.
955
957
 
956
958
  Recording destinations are reserved within one Pi process, not across processes. Use unique paths for concurrent Pi processes: different explicit sessions can overwrite one file even when both `record stop` results are verified. Upstream’s same-session `record start` guard does not reserve the filename across other sessions.
957
- - `savedFilePath` / `savedFile` for direct `download`, `pdf`, and `wait --download` / `wait -d` saved-file workflows when a host file path is reported or wrapper-verified. Batch results preserve the same fields on the relevant `batchSteps` entry. These fields are metadata only until `artifactVerification` verifies the file. For simple loopback `download <selector> <path>` anchors with a non-ref selector, `details.downloadRecovery.method: "direct-anchor-fetch"` means the wrapper resolved the anchor URL in-session and saved the in-page HTTP(S) response directly to the requested path before using upstream's click/download fallback; non-loopback/profile downloads stay upstream-owned so external provider behavior is preserved.
959
+ - `savedFilePath` / `savedFile` for direct `download`, `pdf`, and `wait --download` / `wait -d` saved-file workflows when a host file path is reported or wrapper-verified. Batch results preserve the same fields on the relevant `batchSteps` entry. These fields are metadata only until `artifactVerification` verifies the file. Native `download <selector> <path>` always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper does not substitute a fetch of an anchor's `href`.
958
960
  - `batchSteps[].artifacts` for per-step artifacts in `batch` output; top-level `artifacts` and `artifactManifest` coalesce an earlier pending recording into the later saved, missing, or stale terminal result for the same namespace/session identity. `record restart` includes both the previous recording it finalized (or an explicit missing/stale failure) and the new pending recording; missing/stale terminal rows retire the prior pending manifest row. A successful later `close` / `quit` / `exit` retires an earlier unfinalized pending recording as `subcommand: "close-abandoned"`, clears its stop action, and leaves its file unverified. Batch presentation checks the path: only a confirmed absent file becomes `status: "missing"`; a present or inaccessible file stays unverified. It updates aggregate verification/manifest state consistently; a later successful `record stop` replaces that intermediate abandoned row with its saved artifact. Close also resets ref/page/network-route state produced by earlier rows; later lifecycle-proven browser launches, including `record stop`, can rebuild that state without triggering stale pre-close `about:blank` recovery, explicitly non-launching diagnostics cannot, and unknown later rows stay conservatively active. Per-step history remains unchanged. When any later call on the same namespace/session fails while a recording remains pending, `nextActions` combines its normal recovery with exact `stop-pending-recording` args and visible cleanup guidance; the same applies at top level when a later batch step fails. After reload in a non-Git checkout or with managed restore disabled, a live daemon without current-instance provenance cannot accept a stop. A tracked Electron attachment can rebuild that proof through the live debug-endpoint check described above; generic restore-disabled sessions cannot. That policy refusal includes `managedSessionCleanupOnlyReason: "restore-disabled-daemon-without-provenance"` plus the exact `sessionName`/`namespace`, including on implicit calls. It replaces the impossible stop with `close-pending-recording`, an exact close without `sessionMode: "fresh"`. Close retires the recording as `close-abandoned`; any file it leaves is unverified. Same-instance recordings and supported durable-Git reloads still use stop and normal WebM verification.
959
961
  - `artifactVerification` for a normalized verification summary on the unified result and on each successful `batchSteps[]` row and on failed recording rows whose receipt identifies an artifact. Top-level `batch` verification rolls up all step file artifacts; each step’s summary reflects that step’s nested tool presentation (including its spill paths and manifest slice). It reports `verified`, `verifiedCount`, `missingCount`, `pendingCount`, `unverifiedCount`, and `artifacts[]` entries with `path`, optional `absolutePath`, optional `requestedPath`, `kind` (a normal file artifact kind or `"spill"` for manifest-backed rows), optional `mediaType`, optional `exists`, optional `sizeBytes`, optional `updatedAtMs`, optional `status`, optional `retentionState` / `storageScope` on manifest-derived rows, `state` (`verified`, `missing`, `pending`, or `unverified`), and optional `limitation` (human-readable lifecycle or retention context, for example pending `record start` / `record restart`, missing, stale, or otherwise unverified files, ephemeral spill files, or evicted persisted spills). The summary `verified` boolean is true only when every entry is `verified`. `record start` / `record restart` are `pending` until `record stop`; `state load` may mention a path in command output but is not a saved artifact row.
960
962
  - `fullOutputPath` / `fullOutputPaths` when parse-valid large snapshot output or other oversized tool output is compacted and spilled to a private file; persisted sessions keep that path under a private session-scoped artifact directory for reload/resume, with a per-session byte budget by default; `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES=0` disables automatic eviction. Malformed oversized upstream output is discarded after parsing, is omitted from `details.stdout`, and reports `fullOutputUnavailable` instead of creating a parse-failure spill.
@@ -38,7 +38,7 @@ npm run verify -- platform-smoke run --target ubuntu --suite platform-build
38
38
 
39
39
  ## Alternate native transports
40
40
 
41
- For the 0.6.14 release, local macOS execution replaces localhost SSH, and the [Native Windows workflow](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/windows-native.yml) runs the existing Windows suites on a GitHub-hosted Windows runner. These are native platform checks, not Crabbox SSH or Parallels passes. The default `release` and `prepublishOnly` commands still select the Crabbox matrix; record alternate suite evidence separately rather than reporting those commands as passed.
41
+ For the 0.6.15 release, local macOS execution replaces localhost SSH, and the [Native Windows workflow](https://github.com/fitchmultz/pi-agent-browser-native/blob/main/.github/workflows/windows-native.yml) runs the existing Windows suites on a GitHub-hosted Windows runner. These are native platform checks, not Crabbox SSH or Parallels passes. The default `release` and `prepublishOnly` commands still select the Crabbox matrix; record alternate suite evidence separately rather than reporting those commands as passed.
42
42
 
43
43
  On macOS, run the unchanged commands returned by `buildPlatformBuildCommand('macos', 'pi-agent-browser-native', 24)` and `buildBrowserDogfoodCommand('macos', '0.38.1', true)` in [`scripts/platform-smoke/targets.mjs`](../scripts/platform-smoke/targets.mjs), serially in a clean private source copy. Use private HOME, npm and Pi settings, and headless browser profiles. No Remote Login or host security change is required.
44
44
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-agent-browser-native",
3
- "version": "0.6.14",
3
+ "version": "0.6.15",
4
4
  "description": "pi extension that exposes agent-browser as a native tool for browser automation",
5
5
  "type": "module",
6
6
  "author": "Mitch Fultz (https://github.com/fitchmultz)",
@@ -1,142 +0,0 @@
1
- import { mkdir, stat, writeFile } from "node:fs/promises";
2
- import { dirname, resolve } from "node:path";
3
- import { isRecord } from "../../../parsing.js";
4
- import { buildAgentBrowserResultCategoryDetails } from "../../../results/categories.js";
5
- import { formatSessionArtifactRetentionSummary, mergeSessionArtifactManifest } from "../../../results/artifact-manifest.js";
6
- import { redactSensitiveText } from "../../../runtime.js";
7
- import { buildSessionDetailFields, runSessionCommandData } from "../session-state.js";
8
- const DIRECT_ANCHOR_DOWNLOAD_MAX_BYTES = 2 * 1024 * 1024;
9
- function getDirectDownloadRequest(commandTokens) {
10
- if (commandTokens[0] !== "download" || commandTokens.length !== 3)
11
- return undefined;
12
- const selector = commandTokens[1];
13
- const path = commandTokens[2];
14
- if (!selector || !path || selector.startsWith("@"))
15
- return undefined;
16
- return { path, selector };
17
- }
18
- function buildAnchorDownloadProbe(selector) {
19
- return `(async () => {\n const selector = ${JSON.stringify(selector)};\n const maxBytes = ${DIRECT_ANCHOR_DOWNLOAD_MAX_BYTES};\n const isLoopbackHttpUrl = (url) => (url.protocol === "http:" || url.protocol === "https:") && (url.hostname === "localhost" || url.hostname === "127.0.0.1" || url.hostname === "::1" || url.hostname === "[::1]");\n const element = document.querySelector(selector);\n const anchor = element?.closest?.("a[href]");\n const pageUrl = location.href;\n const page = new URL(pageUrl);\n if (!anchor) return { status: "no-anchor", pageUrl };\n const href = anchor.href;\n const anchorUrl = new URL(href, pageUrl);\n if (!isLoopbackHttpUrl(page)) return { download: anchor.getAttribute("download") || "", href, pageUrl, status: "not-loopback-page" };\n if (anchorUrl.origin !== page.origin) return { download: anchor.getAttribute("download") || "", href, pageUrl, status: "not-same-origin" };\n if (!isLoopbackHttpUrl(anchorUrl)) return { download: anchor.getAttribute("download") || "", href, pageUrl, status: "not-loopback-href" };\n const response = await fetch(anchorUrl.href, { credentials: "include", redirect: "manual" });\n if (!response.ok) return { download: anchor.getAttribute("download") || "", href, pageUrl, responseUrl: response.url, status: "fetch-failed", statusCode: response.status };\n const responseUrl = new URL(response.url);\n if (!isLoopbackHttpUrl(responseUrl) || responseUrl.origin !== page.origin) return { download: anchor.getAttribute("download") || "", href, pageUrl, responseUrl: response.url, status: "not-loopback-response" };\n const buffer = await response.arrayBuffer();\n if (buffer.byteLength > maxBytes) return { download: anchor.getAttribute("download") || "", href, pageUrl, responseUrl: response.url, sizeBytes: buffer.byteLength, status: "too-large" };\n const bytes = new Uint8Array(buffer);\n let binary = "";\n for (let index = 0; index < bytes.length; index += 32768) binary += String.fromCharCode(...bytes.subarray(index, index + 32768));\n return { bodyBase64: btoa(binary), contentType: response.headers.get("content-type") || "", download: anchor.getAttribute("download") || "", href, pageUrl, responseUrl: response.url, sizeBytes: buffer.byteLength, status: "fetched-anchor" };\n})()`;
20
- }
21
- function isLoopbackHttpUrl(url) {
22
- return (url.protocol === "http:" || url.protocol === "https:") && (url.hostname === "localhost" || url.hostname === "127.0.0.1" || url.hostname === "::1" || url.hostname === "[::1]");
23
- }
24
- export async function tryDirectAnchorDownload(options) {
25
- const request = getDirectDownloadRequest(options.commandTokens);
26
- if (!request || !options.sessionName)
27
- return undefined;
28
- try {
29
- const probeData = await runSessionCommandData({
30
- args: ["eval", "--stdin"],
31
- cwd: options.cwd,
32
- namespace: options.namespace,
33
- sessionName: options.sessionName,
34
- signal: options.signal,
35
- stdin: buildAnchorDownloadProbe(request.selector),
36
- });
37
- const probe = isRecord(probeData) && isRecord(probeData.result) ? probeData.result : probeData;
38
- if (!isRecord(probe) || probe.status !== "fetched-anchor" || typeof probe.href !== "string" || typeof probe.pageUrl !== "string" || typeof probe.bodyBase64 !== "string")
39
- return undefined;
40
- const href = new URL(probe.href);
41
- const pageUrl = new URL(probe.pageUrl);
42
- const responseUrl = typeof probe.responseUrl === "string" ? new URL(probe.responseUrl) : href;
43
- if (!isLoopbackHttpUrl(pageUrl) || !isLoopbackHttpUrl(href) || !isLoopbackHttpUrl(responseUrl) || href.origin !== pageUrl.origin || responseUrl.origin !== pageUrl.origin)
44
- return undefined;
45
- const body = Buffer.from(probe.bodyBase64, "base64");
46
- if (body.byteLength > DIRECT_ANCHOR_DOWNLOAD_MAX_BYTES)
47
- return undefined;
48
- if (typeof probe.sizeBytes === "number" && probe.sizeBytes !== body.byteLength)
49
- return undefined;
50
- const absolutePath = resolve(options.cwd, request.path);
51
- await mkdir(dirname(absolutePath), { recursive: true });
52
- await writeFile(absolutePath, body);
53
- const fileStat = await stat(absolutePath);
54
- const mediaType = typeof probe.contentType === "string" && probe.contentType.length > 0 ? probe.contentType : undefined;
55
- const manifestEntry = {
56
- absolutePath,
57
- command: "download",
58
- createdAtMs: Date.now(),
59
- cwd: options.cwd,
60
- exists: true,
61
- kind: "download",
62
- mediaType,
63
- path: absolutePath,
64
- requestedPath: request.path,
65
- retentionState: "live",
66
- session: options.sessionName,
67
- sizeBytes: fileStat.size,
68
- storageScope: "explicit-path",
69
- };
70
- const artifactManifest = mergeSessionArtifactManifest({ base: options.artifactManifest, entries: [manifestEntry] });
71
- const artifactRetentionSummary = artifactManifest ? formatSessionArtifactRetentionSummary(artifactManifest) : undefined;
72
- const artifact = {
73
- absolutePath,
74
- artifactType: "download",
75
- command: "download",
76
- cwd: options.cwd,
77
- exists: true,
78
- kind: "download",
79
- mediaType,
80
- path: absolutePath,
81
- requestedPath: request.path,
82
- session: options.sessionName,
83
- sizeBytes: fileStat.size,
84
- status: "saved",
85
- };
86
- const artifactVerification = {
87
- artifacts: [{
88
- absolutePath,
89
- exists: true,
90
- kind: "download",
91
- mediaType,
92
- path: absolutePath,
93
- requestedPath: request.path,
94
- sizeBytes: fileStat.size,
95
- state: "verified",
96
- status: "saved",
97
- }],
98
- missingCount: 0,
99
- pendingCount: 0,
100
- unverifiedCount: 0,
101
- verified: true,
102
- verifiedCount: 1,
103
- };
104
- const savedFile = { command: "download", kind: "download", metadata: { download: probe.download, href: redactSensitiveText(href.href), method: "direct-anchor-fetch" }, path: absolutePath };
105
- return {
106
- artifactManifest,
107
- result: {
108
- content: [{
109
- type: "text",
110
- text: [
111
- `Download completed: ${absolutePath}`,
112
- `Requested path: ${request.path}`,
113
- `Source: ${redactSensitiveText(href.href)}`,
114
- `Size: ${fileStat.size} bytes`,
115
- "Method: direct anchor fetch before upstream download fallback.",
116
- ].join("\n"),
117
- }],
118
- details: {
119
- args: options.redactedArgs,
120
- artifactManifest,
121
- artifactRetentionSummary,
122
- artifacts: [artifact],
123
- artifactVerification,
124
- command: "download",
125
- compatibilityWorkaround: options.compatibilityWorkaround,
126
- downloadRecovery: { href: redactSensitiveText(href.href), method: "direct-anchor-fetch", selector: request.selector },
127
- effectiveArgs: options.effectiveArgs,
128
- savedFile,
129
- savedFilePath: absolutePath,
130
- sessionMode: options.sessionMode,
131
- ...buildAgentBrowserResultCategoryDetails({ artifacts: [artifact], args: options.effectiveArgs, command: "download", savedFile, succeeded: true }),
132
- ...buildSessionDetailFields(options.sessionName, options.usedImplicitSession, options.namespace, options.managedSessionRestoreDisabled()),
133
- summary: `Download completed: ${absolutePath}`,
134
- },
135
- isError: false,
136
- },
137
- };
138
- }
139
- catch {
140
- return undefined;
141
- }
142
- }