pi-agent-browser-native 0.6.13 → 0.6.15

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. package/CHANGELOG.md +29 -0
  2. package/README.md +17 -15
  3. package/dist/extensions/agent-browser/index.js +7 -4
  4. package/dist/extensions/agent-browser/lib/argv-grammar.js +1 -0
  5. package/dist/extensions/agent-browser/lib/command-taxonomy.js +4 -0
  6. package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +14 -3
  7. package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +38 -5
  8. package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +30 -4
  9. package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +15 -4
  10. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/network-page-filter.js +4 -3
  11. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +1 -2
  12. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +3 -0
  13. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +2 -17
  14. package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +14 -3
  15. package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +5 -2
  16. package/dist/extensions/agent-browser/lib/playbook.js +6 -5
  17. package/dist/extensions/agent-browser/lib/recording-reservations.js +8 -1
  18. package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +15 -1
  19. package/dist/extensions/agent-browser/lib/results/presentation/batch.js +18 -9
  20. package/dist/extensions/agent-browser/lib/results/presentation/common.js +3 -0
  21. package/dist/extensions/agent-browser/lib/results/presentation/registry.js +1 -1
  22. package/dist/extensions/agent-browser/lib/results/presentation.js +7 -3
  23. package/dist/extensions/agent-browser/lib/results/snapshot-refs.js +14 -2
  24. package/dist/extensions/agent-browser/lib/results/snapshot.js +12 -5
  25. package/dist/extensions/agent-browser/lib/runtime.js +3 -1
  26. package/dist/extensions/agent-browser/lib/session-page-state.js +7 -3
  27. package/dist/extensions/agent-browser/lib/webmcp-observation.js +28 -0
  28. package/dist/scripts/agent-browser-target.mjs +1 -1
  29. package/docs/ARCHITECTURE.md +12 -4
  30. package/docs/COMMAND_REFERENCE.md +73 -20
  31. package/docs/ELECTRON.md +1 -1
  32. package/docs/RELEASE.md +4 -3
  33. package/docs/REQUIREMENTS.md +1 -1
  34. package/docs/SUPPORT_MATRIX.md +15 -9
  35. package/docs/TOOL_CONTRACT.md +24 -12
  36. package/docs/platform-smoke.md +2 -2
  37. package/package.json +1 -1
  38. package/scripts/agent-browser-capability-baseline.mjs +34 -3
  39. package/scripts/agent-browser-target.mjs +1 -1
  40. package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/direct-anchor-download.js +0 -142
@@ -37,7 +37,7 @@ Why:
37
37
  The extension should:
38
38
  - resolve `agent-browser` from `PATH`
39
39
  - invoke it with native Node `spawn` on POSIX and `cross-spawn` on Windows. The latter handles Windows `PATH`/`PATHEXT`, `.cmd` shims and argument escaping without PowerShell or wrapper-owned command reordering. Caller commands and helper probes retain empty operands, literal doublequotes, command/subcommand adjacency and explicit default `--namespace ""`; the selected shim still owns upstream architecture selection. Keep piped stdin/stdout/stderr and terminate the full Windows shell/agent-browser process tree with `taskkill /T /F` on timeout or abort before falling back to the direct child signal. Missing commands use the spawner's `ENOENT`, not a parsed shell-error string
40
- - inject `--json`
40
+ - inject `--json`; caller-requested JSON also uses the existing `buildJsonVisibleContent` envelope for early snapshot/network filters, scroll results, and preparation failures, retaining metadata in `details` and keeping output-file notices out of visible JSON. Help/version stays native text
41
41
  - complete each upstream invocation when the direct `agent-browser` child exits even if Node delays `"close"`: piped stdio can stay referenced by longer-lived descendant processes, so `runAgentBrowserProcess` watches `exit` and `close` together, leaves stdio intact during a short post-`exit` grace so normal `close` can still win, destroys streams only when the post-`exit` fallback fires, and prefers `close` codes then wrapper timeout (`124`) over signal-shaped `exit` codes (`watchSpawnedChildCompletion` / `resolveSpawnedChildExitCode` in `extensions/agent-browser/lib/process.ts`) so the tool cannot hang after the CLI process has already terminated
42
42
  - support optional stdin only for `eval --stdin`, `batch`, `auth save --password-stdin`, and wrapper-generated `batch` stdin from top-level `job`, `qa`, `sourceLookup`, or `networkSourceLookup`, rejecting other command/stdin combinations before launch; top-level `electron` never accepts caller `stdin` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#electron))
43
43
  - support optional top-level `outputPath` for successful browser results and failed/pending/recovered recording receipts. Recording exports retain success/error, original attempt, native data, artifacts and recovery provenance; they do not treat a written receipt as a successful video. Other successful results write `details.data` (or model-facing text when no structured data exists) to a caller-requested local file and reporting `details.outputFile`, without changing upstream argv semantics or overwriting a browser artifact when both destinations resolve to the same file. If presentation compacted direct data, a batch result row, or a whole batch, apply the relevant command-specific redactor before spilling and rehydrate each full pre-compaction value only from its matching live wrapper-manifest spill; fail without writing compact metadata when any required spill is unavailable or untrusted
@@ -158,6 +158,12 @@ An explicit native `session info` call is the preflight: daemon activity/PID is
158
158
 
159
159
  `lib/read-confirmation.ts` recognizes only native structured control responses for actual explicit URL reads. Ordered pending/cleared `readConfirmation` markers live in the existing `SessionPageState` and tool-result transcript, including resume/branch replay. They preserve actual session/default routing even for legacy prompts. Only native `capabilities.readRequiresConfirmation: true`, which promises exact ID validation, allows matching confirm/deny to skip page helpers and managed replacement. Explicit caller identities win, DOM/unknown-origin confirmations keep their prior checks, and response content cannot manufacture provenance. There is no separate confirmation store or wrapper authorization system.
160
160
 
161
+ ### Native observation state
162
+
163
+ Native `snapshot --delta` owns its revision baseline. Nested full responses are projected onto the existing tree/ref readers; partial responses stay native in presentation and trigger one ordinary full snapshot read for the existing `SessionPageState` ref guard. This does not advance the native delta baseline or add a second tree/revision cache. Failed ref refreshes invalidate refs. Batch folding selects the last reached snapshot unless a later transition invalidates it; transcript replay persists the resulting complete refs as before. Wrapper-only filters request full trees.
164
+
165
+ Conditional screenshots keep upstream `changed: false` pathless: no destination repair, artifact, or old image attachment. Recording cursor/contact-sheet flags are skipped by the shared path/URL reader; native contact-sheet paths use existing image metadata, verification, and attachment handling, with the companion path on the existing video reservation and transcript entry. Preflight derives only the documented native filename for alias checks; artifact reporting uses the actual native path. `--input-mode` is a value-taking global, not launch-scoped. Native WebMCP catalog updates remain untrusted page data in ordinary presentations. A call-scoped `AsyncLocalStorage` collects the latest native update across helper and main responses: native discovery emits updates once, so preflight reads must not consume them silently. The final tool boundary exposes `details.webMcpCatalog` and any helper-only prose, without mutating native result data or storing a cross-call catalog.
166
+
161
167
  ### Explicit upstream sessions and fresh launches
162
168
 
163
169
  If the caller provides `--session`, `--profile`, `--cdp`, or similar upstream flags, the extension should respect them with minimal interference.
@@ -180,9 +186,9 @@ Practical policy:
180
186
  - set an idle timeout on extension-managed sessions as a backstop for abnormal exits or cleanup failures, and apply that same `AGENT_BROWSER_IDLE_TIMEOUT_MS` value to every subprocess targeting that owned session (including wrapper helper snapshots, tab lists, and navigation-summary reads); caller-owned sessions instead retain native config/environment idle policy because changing the launch environment between calls can make upstream restart the background browser, discard the active tab, and invalidate fresh refs
181
187
  - for wrapper-owned implicit sessions only, set a transcript- and checkout-scoped `AGENT_BROWSER_RESTORE` key on compatible calls so cookies and web storage can survive idle shutdown, reload, and resume. Explicit caller sessions, restore/state choices, profiles, upstream config, file access, launch arguments, environment variables, local pages, output paths, and close arguments remain upstream-owned and pass through unchanged. `piab-*` names are not reserved; session/state lists and restore identifiers are not filtered or redacted. The wrapper validates only its automatic restore checkout/storage identity and coordinates same-daemon reuse so its own restore pools cannot mix. Ambiguous page-target transitions still require live `get url` verification before content calls. The current v3 ticket-claim lock is the only managed-daemon coordination protocol; no earlier lock bridge or compatibility path remains.
182
188
  - redact snapshot spill payloads before writing them, clean up process-private temp spill artifacts on shutdown, and keep persisted-session spill files in a private session-scoped artifact directory for `details.fullOutputPath` after reload/resume, with a 32 MiB per-session budget by default; `PI_AGENT_BROWSER_SESSION_ARTIFACT_MAX_BYTES=0` skips persistent-spill enumeration and eviction without changing temporary spill cleanup
183
- - keep explicit screenshots, downloads, PDFs, traces, HAR captures, and recordings written to caller-chosen paths on disk after a successful upstream close command (`close`, `quit`, or `exit`); before artifact-producing commands run, create missing parent directories for requested host paths, and for simple loopback HTML anchor downloads with resolvable HTTP(S) hrefs the wrapper may save directly to the requested path before upstream fallback. When the bounded `details.artifactManifest` has entries, successful close commands also surface `details.artifactCleanup` and a compact `Artifact lifecycle` note pointing to structured explicit paths so operators remove files with normal host tools—the native tool does not delete arbitrary user paths (`extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`, `getArtifactCleanupGuidance`); contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), checklist `RQ-0079` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
189
+ - keep explicit screenshots, downloads, PDFs, traces, HAR captures, and recordings written to caller-chosen paths on disk after a successful upstream close command (`close`, `quit`, or `exit`); before artifact-producing commands run, create missing parent directories for requested host paths, while native download always owns the click and download, including loopback links, generated Blob exports, and redirects; never substitute an anchor-href fetch. When the bounded `details.artifactManifest` has entries, successful close commands also surface `details.artifactCleanup` and a compact `Artifact lifecycle` note pointing to structured explicit paths so operators remove files with normal host tools—the native tool does not delete arbitrary user paths (`extensions/agent-browser/lib/orchestration/browser-run/diagnostics.ts`, `getArtifactCleanupGuidance`); contract in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details), checklist `RQ-0079` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
184
190
  - reconstruct the current branch-visible extension-managed session, every transcript-proven still-active wrapper-owned managed identity, page-scoped refs, newest-revision aggregate artifact manifest, unbounded active-recording reservation events, and Electron launch records from the active transcript branch on `session_start` and `session_tree` so later default and explicit off-current calls keep following owned browsers after resume/reload or branch switching; restore also honors successful explicit `--session <wrapper-owned> close` rows, terminal nested-batch close outcomes even when aggregate artifact verification failed, and `electron.cleanup` managed-session steps so closed wrapper-owned sessions are not resurrected; a nested close invalidates the pre-close page target so a lifecycle-proven relaunch at `about:blank` is not treated as stale focus drift; explicit lifecycle evidence that a later diagnostic did not launch a browser preserves the terminal close, while any later row—including a failed row—whose lifecycle reports a browser launch keeps active/attached provenance; failed-step presentation persists only that bounded launch boolean so transcript replay reaches the same decision, missing lifecycle evidence remains conservatively active even on the first managed call, successful closes clear wrapper trace/profiler ownership before ordered later successful rows can rebuild it, namespace-scoped `close --all` clears all matching managed/attached/page/ref/route/trace/recording ownership, and recording starts after close are rejected before spawn
185
- - keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, script, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
191
+ - keep active recording destination reservations separate from the bounded metadata-only artifact manifest. The same journal retains native recording IDs and start windows; native receipts, including failures, flow through existing artifact metadata and verification. A stop timeout/no-recording result gets one two-second native `session info` query matching namespace/session, ID/path or effective batch start window. Recovery needs terminal native encoder success plus a matching verified file; filesystem presence alone is insufficient, old same-path receipts cannot verify a new take, and freshness starts at capture rather than status-query time. Original attempts and unrelated batch failures remain visible. No second ledger, recorder, sidecar or runtime ffprobe is added. The process-wide map is keyed by canonical namespace/session identity, rebuilt from append-only branch events, and retained for still-live process-owned recordings across branch switches. Shutdown/reload appends both terminal tombstones and still-live reservations onto the current branch so restart cannot resurrect a cross-branch close or lose a live-daemon reservation. One artifact lifecycle/output queue makes global destination preflight and reservation updates atomic across otherwise-concurrent caller-owned session queues. Every successful direct, ordered nested-batch, managed replacement, script, Electron, or shutdown close retires its exact identity at that lifecycle point; only the newest pending recording path remains authoritative across current transition replay (including same-timestamp restart rows), and recording starts after a nested close are rejected because upstream can falsely report success. Existing and dangling symlink ancestry, hardlink inode identity, full Unicode/platform case folding, and same-call `outputPath` comparison prevent alias reuse. One shared command-token projection mirrors upstream's full-argv global cleanup before artifact, recording, and presentation parsing; wait-download detection removes only the first timeout pair, follows upstream long/short mode precedence, and accepts both `--download` and `-d` wherever download mode wins; screenshot destinations use upstream's exact-flag, selector-prefix, case-sensitive extension, slash-path, and second-positional rules, while retaining the wrapper's intentional slash-bearing hidden-workspace path normalization. Recording path/URL consumers share a command-local reader that skips complete numeric `--fps` pairs, cursor/contact-sheet switches, and contact-sheet threshold values without rewriting argv or treating them as outer globals; native owns rate, format and extra-argument validation. Current recording transitions are replayed directly; artifact manifests are not treated as reservation events
186
192
  - keep process-owned cleanup registries for extension-managed sessions and wrapper-launched Electron records separate from the current branch-visible view; `session_tree` restore and wrapper-owned browser commands are serialized with managed-session work, while caller-owned explicit-session commands are serialized by process-local queues keyed to effective canonical namespace/session across prepare helpers (explicit namespace argv overrides inherited `AGENT_BROWSER_NAMESPACE`, including an explicit empty default) and main execution. macOS and Windows additionally normalize and case-fold namespace and session components to match case-insensitive daemon identity. Different caller-owned identities remain concurrent, except namespace-scoped `close --all` drains and exclusively barriers managed plus matching caller-owned work before clearing global namespace state; nested helpers never re-enter the outer queue, policy/route/artifact deltas merge across unrelated managed-state commits, and a separate branch-restore generation guard prevents stale completions from overwriting newer branch-visible state; aggregate artifact results use monotonic revisions so transcript replay cannot lose a concurrently completed entry. Branch switches still must not drop resources the current Pi process owns and must keep fresh-session allocation monotonic
187
193
  - record successful `connect`, `--cdp`, enabled `--auto-connect`, environment-configured CDP/auto-connect, and wrapper Electron attachment identities in branch-visible state. First-use and later content-bearing calls live-check `get url` because attached targets can drift outside Pi. Caller config, file access, launch arguments, and environment pass through unchanged; only wrapper-injected compatibility launch arguments are omitted on active attachments. A terminal successful close removes the marker; a close followed by a later step whose lifecycle reports a browser launch preserves it, while a non-launching diagnostic such as `stream status` leaves the close terminal
188
194
  - when a successful close targets the current extension-managed session, including an explicit `--session <current> close` or an `electron.cleanup` managed-session step, clear page/ref state, mark that session inactive, untrack cleanup ownership, and rotate the next default auto call to a fresh wrapper-generated session name rather than reusing the closed name
@@ -198,8 +204,9 @@ Practical policy:
198
204
  - keep a per-session `refSnapshot` aligned with the last successful `snapshot` (including refs merged from a successful `batch` by taking the last successful `snapshot` step in batch result order): restore it from persisted tool `details` when reloading, resuming, or moving to a different Pi session-tree branch, store bounded ref role/name metadata from the same snapshot for wrapper-side current-ref diagnostics, drop it on successful close commands (`close`, `quit`, or `exit`), replace it with a persisted `page-transition` invalidation after any upstream-executed `record start` attempt (direct or batch; older supported natives can swap the page before their already-active check, so failed starts count; the warning and persisted summary explicitly describe conservative ref invalidation, not an observed page change), a `record restart` with a URL operand, `window new`, `diff url`, or WebMCP `invoke` / `result` / `cancel` (these page-provided tools can mutate, rerender, or navigate; when a spawned `batch` yields no parseable result rows, for example after a wrapper timeout, planned transition steps still record the invalidation), or after a failed non-batch transition command (`eval`, `back`, `forward`, `reload`, `connect`, `state load`, `tab` selection) whose live URL re-verification probe observed the page (a failed transition can still have mutated or replaced the document before throwing, so keeping the verified URL must not keep the prior refs; transcript replay preserves the persisted invalidation summary), and refuse page-scoped `@e…` argv before spawn when the active tab URL no longer matches the snapshot URL, when a ref id was never in that snapshot, when the snapshot state is invalidated, or when a `batch` step would reuse `@e…` on a guarded getter or mutation step after an earlier invalidating step (including `record start`, URL-bearing `record restart`, and WebMCP `invoke` / `result` / `cancel`) without a later `snapshot` step in the same plan; batch steps come from the source upstream actually executes (raw batch argument strings exclusively when any exist — filtering only the exact `--bail` token like upstream — stdin only otherwise, via `getUpstreamEffectiveBatchSteps` in `extensions/agent-browser/lib/orchestration/batch-stdin.ts`); tab recovery (which leaves user argv/stdin and continue-on-error control flow unchanged), artifact/recording preflight, batch screenshot path preparation (parent directories are created for effective raw rows too, without rewriting raw strings), and stale-ref echo args use that same selection so pinning and preflights cannot act on upstream-ignored stdin, while the pre-spawn state-policy validator deliberately keeps scanning parseable stdin alongside argv as a fail-closed content superset and treats stdin parse failures as fatal only when upstream would actually read stdin (its raw-token filter also uses the exact `--bail` token only). Same-snapshot `fill @e…` rows are guarded but do not themselves set that invalidation latch, so ordinary form fills can precede a click/submit row in one batch—see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) for the agent-visible contract and failure text; typed per-session tab/ref/pinning state lives in `extensions/agent-browser/lib/session-page-state.ts` and is updated from `extensions/agent-browser/index.ts` after each tool result
199
205
  - native 0.37 owns first-document setup inheritance for new tabs; the wrapper does not copy headers, user agents or other browser setup itself. The page-summary formatter displays native positive WebMCP availability while retaining raw details; it adds no discovery probe
200
206
  - when a direct or batched WebMCP call returns `status: "pending"`, or `result` / `cancel` fails while that target is unknown, keep its tab target unknown and discard same-call snapshot evidence instead of treating an immediate post-dispatch URL probe as stable; `webmcp result` / `cancel`, `get url`, and explicit navigation remain available while unknown; replace the generic blocked snapshot action with `verify-page-target-after-pending-webmcp` (`get url`), and let a completed `batch --bail` use that verification before `snapshot -i` to re-establish both target and refs
201
- - for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`job`/`qa` click steps
207
+ - for top-level non-Electron direct `click` commands with an eligible target, install a bounded in-page target-specific event probe before upstream runs; if upstream reports success but no trusted pointer/mouse/click event reached the resolved target, fail the tool and report `details.clickDispatch` with explicit retry/inspect next actions (the wrapper does not replay clicks in-page). The probe covers `xpath=` targets and current `@e…` / `ref=` refs whose latest stored `refSnapshot.refs` role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`; it requires a unique role/name in the saved snapshot and the live candidates instead of taking a fresh pre-click snapshot that could recycle upstream refs. Ref and XPath candidates must additionally prove identity through a per-probe temporary DOM attribute read back by native `get attr <original-selector> <marker>`. Missing or mismatched identity, including XPath/frame scope mismatches, cleans up the probe and lets native click run without dispatch verification; candidate naming heuristics alone are never proof. Duplicate-name refs pass through without a probe: their old ordinal is not target identity. The probe is intentionally skipped for CSS selector clicks, unresolved `find … click` locators, and `batch`/`job`/`qa` click steps
202
208
  - derive narrow prompt guards only for concrete evidence invariants: explicitly requested screenshot/recording output paths block browser close until the artifact manifest verifies those paths, while bare inbound attachment paths remain inputs. The wrapper intentionally does not infer broad business/user intent from prompt text such as order/payment/post boundaries; agents must follow those instructions themselves. The artifact guard is bounded preflight policy (`details.promptGuard`, `failureCategory: "policy-blocked"`), not a reusable browser recipe layer
209
+ - keep the existing page/selector scroll API; explicit CSS-container movement uses `Element.scrollBy` with `behavior: "instant"` before measuring offsets so smooth-scroll CSS cannot cause a premature no-movement failure
203
210
  - reject direct and effective batch `scrollintoview text=...` / `scrollinto text=...` before dispatch because current upstream can falsely report success without movement, while leaving help forms untouched; return only native recovery (`find text ... hover` or fresh snapshot/ref), leaving CSS, XPath, and current-ref behavior upstream-owned
204
211
  - after successful `get text` on a qualifying non-ref CSS selector, optionally issue one read-only `eval --stdin` probe per selector when multiple DOM matches or a hidden first match with visible peers could misread tabbed or off-screen content; simple id selectors and sensitive-looking literals skip this probe. Merge `details.selectorTextVisibility` / `selectorTextVisibilityAll`, visible warning lines, and `inspect-visible-text-candidates*` next actions as documented in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) and `RQ-0074` in [`SUPPORT_MATRIX.md`](SUPPORT_MATRIX.md)
205
212
  - caller-owned sessions honor native `AGENT_BROWSER_SOCKET_DIR` unless `PI_AGENT_BROWSER_SOCKET_DIR` overrides it, with the same socket integrity checks. For other local Unix launches, set a short private socket directory so extension-generated session names do not fail on the upstream Unix socket-path length limit; require the selected path to be absolute, owned by the current uid, mode `0700`, under checked ancestry, and free of symlink, foreign-owner, or special planted entries; reject pre-existing unsafe modes instead of repairing them, then recheck before spawn. The actual filesystem root `/` is supplied by the trusted operating environment: a directory without group/other write bits is accepted regardless of its reported owner, which can be unmapped in a Linux user namespace. Existing root-owned sticky-directory acceptance is unchanged. This boundary does not protect against whoever controls the root filesystem. Every non-root ancestor still needs trusted ownership and permissions, including the destination ancestry of root-owned aliases; a matching overflow UID is not trusted. Android/Termux uses `/data/data/<package>/piab`, treats the owner-only app-data directory as the trust anchor, permits the app's matching private uid/gid ancestry, compacts generated managed identities to one 80-bit digest so ordinary namespace plus fresh-session paths remain within the limit, and places policy-lock coordination under `os.tmpdir()` because Android `/tmp` is shell-owned and inaccessible
@@ -257,6 +264,7 @@ Upstream restore-state persistence remains upstream-owned. The wrapper passes an
257
264
  - tool registration and schema (including the optional `semanticAction` compilation path to upstream `find` or `select`)
258
265
  - subprocess execution and JSON parsing through `buildAgentBrowserProcessEnv` in `extensions/agent-browser/lib/process.ts`: copies the parent process environment so user-approved provider credentials and other runtime variables reach upstream, then applies wrapper overrides such as the managed socket directory and clamped default operation timeout
259
266
  - clear missing-binary errors
267
+ - filtered `network requests --current-page` / `--current-origin` / `--current-url` preparation results use the same `redactPresentationData` credential redactor as ordinary diagnostics before populating `details.data` or writing `outputPath`
260
268
  - compact result summaries, including presentation-time redaction: stateful browser-context commands (`auth`, `cookies`, `storage`, `dialog`, `frame`, `state`) use field-aware value redaction and compact formatters, while other structured upstream JSON (for example `network`, `diff`, `trace` / `profiler` / `record`, `console` / `errors` / `highlight` / `inspect` / `clipboard`, `stream`, `dashboard`, and `chat`) is passed through `redactPresentationData` in `extensions/agent-browser/lib/results/presentation.ts` so model-facing `details.data` and batch roll-ups stay compact and do not echo bearer tokens, proxy passwords, or similar fields verbatim; `redactInvocationArgs` in `extensions/agent-browser/lib/runtime.ts` masks trailing values for sensitive global flags such as `--body`, `--headers`, `--password`, and `--proxy`, preserves positional rules for `cookies set` and `storage local|session set`, and nested `batch` steps use the same argv and error-body scrubbing before echoing commands or errors. Shared URL redaction covers SAMLRequest/SAMLResponse/RelayState and auth-context-only state/nonce at model/persisted-artifact boundaries while retaining exact internal page-target URLs; snapshot spills are redacted before disk writes
261
269
  - bounded machine-readable outcome metadata on tool `details` (`resultCategory`, `successCategory`, `failureCategory`, optional `nextActions`, optional `pageChangeSummary` with observed-vs-dispatched evidence and per-step summaries on `batch`, optional `artifactVerification` with the same shape on each successful `batchSteps[]` row) so agents can branch without parsing prose; enums, classifier precedence, and generic follow-up payloads are implemented under `extensions/agent-browser/lib/results/` in focused modules (`contracts.ts` for shared types, `categories.ts` for `classifyAgentBrowserSuccessCategory` / `classifyAgentBrowserFailureCategory` / `buildAgentBrowserResultCategoryDetails`, `action-recommendations.ts` for `buildAgentBrowserNextActions`, `next-actions.ts` for the `AgentBrowserNextAction` shape and merge helpers, `recovery-actions.ts` for recovery id registries and `buildRecoveryNextActions`, `network.ts` for `classifyNetworkRequestFailure` / `summarizeNetworkFailures`, and related helpers). Per-session tab target, `refSnapshot` alignment, invalidation, and tab pinning observations flow through `extensions/agent-browser/lib/session-page-state.ts` from `extensions/agent-browser/index.ts`. Compact page-change summaries and artifact verification rollups are built in `extensions/agent-browser/lib/results/presentation.ts` (`buildPageChangeSummary`, `buildArtifactVerificationSummary`), and the human contract lives in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details). Real Pi custom tools otherwise only mark a row failed when `execute` throws, so the extension also registers `pi.on("tool_result", …)` and patches `agent_browser` results whose `details.resultCategory` is `failure` to set `isError: true`. Prose results also receive a short category notice, while caller-requested `--json` results with parseable JSON content keep that text unchanged so JSONL transcripts, UI affordances, and the machine-readable contract stay aligned for wrapper-side reclassifications such as `qa-failure` (`buildAgentBrowserToolResultPatch` in `extensions/agent-browser/lib/pi-tool-rendering.ts`; transcript semantics in the same contract doc)
262
270
  - inline screenshots/images for the plain `screenshot` command; other image-like saves (for example `diff screenshot`) still appear in `details.artifacts` and summaries but are not auto-inlined as Pi image attachments (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details))
@@ -22,15 +22,36 @@ SDK hosts can supply an awaited [`beforeExecute` callback](TOOL_CONTRACT.md#host
22
22
 
23
23
  <!-- agent-browser-capability-baseline:start upstream-baseline -->
24
24
  <!-- Generated from scripts/agent-browser-capability-baseline.mjs. Run `npm run docs -- command-reference write` to update. Do not edit manually. -->
25
- This reference is baselined to the locally installed `agent-browser 0.37.0` command/help surface, audited against vercel-labs/agent-browser@471ab3852b47b98847f1d9c855c272bb62d0d50b. Upstream `agent-browser` remains the source of truth for command semantics; this file is the local fallback for Pi agent sessions where direct binary help is blocked or discouraged.
25
+ This reference is baselined to the locally installed `agent-browser 0.38.1` command/help surface, audited against vercel-labs/agent-browser@aff6125c023b810ea3f2e5deec5379e9a4270bdc. Upstream `agent-browser` remains the source of truth for command semantics; this file is the local fallback for Pi agent sessions where direct binary help is blocked or discouraged.
26
26
 
27
27
  The lightweight drift check is `npm run verify -- command-reference`. Run it whenever the installed upstream `agent-browser` version changes or this reference is edited.
28
28
 
29
29
  <!-- agent-browser-capability-baseline:end upstream-baseline -->
30
30
 
31
+ ### Upstream 0.38.1 rebaseline
32
+
33
+ The recommended release keeps the stable 0.35.0 floor. New options require an upstream release that provides them; the wrapper adds no version-specific emulation.
34
+
35
+ - `snapshot --delta` returns a nested `snapshot` object: `kind: full` includes `tree`, `refs`, and `revision`; `unchanged` includes `baseRevision` and `revision`; `delta` also includes ref `changes` and an exact `treeChange` line splice. `snapshot --delta --full` resets the native baseline. The wrapper preserves this payload and shows revisions rather than reporting an empty page. Partial results trigger a full native snapshot read for wrapper ref checks without advancing the delta baseline; there is no wrapper revision cache. Wrapper-only search/filter/viewport/diff options use a full tree instead of native delta output.
36
+ - Surviving DOM elements retain refs across same-document snapshots. Navigation and replaced page/iframe documents invalidate them. Keep refreshing after page changes; persistent refs do not remove page-target checks or same-batch mutation boundaries.
37
+ - `screenshot --if-changed` suppresses unchanged captures. `screenshot --threshold <0-1>` implies it and tolerates that changed-pixel ratio (default 0). Native `changed: false` omits the path: no new file or inline image is reported, even with an explicit destination. Use an ordinary screenshot when every call must save evidence.
38
+ - `--input-mode <mode>` sets sticky session pointer behavior (`instant`, `smooth`, or `human`) and can change on a live session. `click <sel> --human` and `drag <src> <dst> --human` override an action. `mouse move <x> <y> --duration <ms> --steps <n> --human --seed <n>` controls timed, reproducible paths (1–240 movement events); browser response time can extend the requested duration.
39
+ - `record start` / `record restart` accept `--cursor`, `--contact-sheet`, and `--contact-sheet-threshold <n>` (0–1, default 0.05, implies contact sheet). These options are not path/URL operands. Contact sheets use `<video-stem>.contact-sheet.png`, contain timestamped visual changes, and remain pending until stop; reported sheets join `details.artifacts` as images and are attached after verification. The sheet's destination is reserved alongside its video until stop/close, including transcript replay and same-batch/output-path alias checks. After direct restart, the previous take's known sheet is shown with disk evidence but remains unverified because native restart omits its terminal receipt; use stop then start when both completed artifacts must be verified. The 0.38.1 cursor overlay stays synchronized with drags and timed mouse movement. Screenshots during recording include the overlay; snapshots do not.
40
+ - `auth login <name> --no-navigate` uses the prepared top-level login page, checks its origin against the effective credential URL, and permits submit-triggered navigation. Saved and plugin credentials support the same username/password/submit selector overrides; use native auth rather than implementing a second login flow.
41
+ - Native WebMCP discovery emits bounded catalog updates on discovery/change, including names, descriptions, origin, and frame IDs. The wrapper surfaces these as untrusted page data and keeps `data.webmcp`; `details.webMcpCatalog` also preserves updates consumed by helper reads during that call. It never reuses a prior call's catalog. Request full schemas with `webmcp list [tool] [--frame <frame-id>]`; for one relevant tool, use `webmcp list set_message --frame <frame-id>`. Upstream owns CDP recovery, normalized dropdown labels, recording frame pacing, and 0.37.1 Windows headless desktop isolation.
42
+
43
+ Examples through `agent_browser`:
44
+
45
+ ```json
46
+ { "args": ["snapshot", "-i", "--delta"] }
47
+ { "args": ["screenshot", "--if-changed", "--threshold", "0.01", "shots/current.png"] }
48
+ { "args": ["--input-mode", "human", "click", "@e4"] }
49
+ { "args": ["record", "start", "demo.webm", "--cursor", "--contact-sheet"] }
50
+ ```
51
+
31
52
  ### Upstream 0.37.0 rebaseline
32
53
 
33
- The recommended release keeps the stable 0.35.0 floor and no upper version cap.
54
+ This release keeps the stable 0.35.0 floor and no upper version cap.
34
55
 
35
56
  - `record start` / `record restart` accept command-local `--fps <n>` before, between or after path/URL operands (1–60, default 30). WebM uses VP8/libvpx; MP4 uses H.264/libx264. Other extensions are handed to ffmpeg; extensionless paths are rejected. Native startup validates the path, rate and ffmpeg availability.
36
57
  - Recording uses the current active page without replacing its DOM/JavaScript state unless a URL is supplied. The wrapper still conservatively requires a fresh snapshot after dispatched starts and URL-bearing restarts to protect older supported natives; this is not evidence that a page changed. FPS alone neither chooses another tab nor makes a restart invalidate refs.
@@ -167,7 +188,7 @@ The 0.27.3 rebaseline is an install-only compatibility update: upstream changed
167
188
 
168
189
  ## Core mental model
169
190
 
170
- Input mode chooser (one per call): **`script`** for one-shot loops, branches, or multi-page aggregation; **`args`** for the default open → snapshot -i → click/fill `@refs` flow; **`semanticAction`** for stable role/text/label targets; **`job`** / **`qa`** for multi-step checks; **`electron`** for desktop apps only; **`sourceLookup`** / **`networkSourceLookup`** are **experimental candidates-only** helpers (not authoritative mappings). Do not pass `--json` in `args`—the wrapper injects it. Match link and button text to the latest snapshot (on `https://example.com/` the main link is `Learn more`, not legacy `More information...` copy). See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#input-mode-chooser) for snapshot variants (`-i` vs `--compact` vs full) and batching three or more getters.
191
+ Input mode chooser (one per call): **`script`** for one-shot loops, branches, or multi-page aggregation; **`args`** for the default open → snapshot -i → click/fill `@refs` flow; **`semanticAction`** for stable role/text/label targets; **`job`** / **`qa`** for multi-step checks; **`electron`** for desktop apps only; **`sourceLookup`** / **`networkSourceLookup`** are **experimental candidates-only** helpers (not authoritative mappings). Omit `--json` for ordinary prose; include it when you need parseable JSON text. Match link and button text to the latest snapshot (on `https://example.com/` the main link is `Learn more`, not legacy `More information...` copy). See [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#input-mode-chooser) for snapshot variants (`-i` vs `--compact` vs full) and batching three or more getters.
171
192
 
172
193
  Tool parameters (use exactly one of `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`):
173
194
 
@@ -215,7 +236,7 @@ Tool parameters (use exactly one of `script`, `args`, `semanticAction`, `job`, `
215
236
  - `networkSourceLookup`: **EXPERIMENTAL — candidates only** for failed request-to-source hints; compiles to generated `batch`, reports `details.compiledNetworkSourceLookup` and `details.networkSourceLookup`, and never assigns blame or edits files.
216
237
  - `electron`: optional Electron desktop-app shorthand. `list`, `status`, `cleanup`, and `probe` are wrapper-owned host/session helpers; `launch` starts a wrapper-owned isolated Electron profile and attaches through upstream `connect`.
217
238
  - `stdin`: top-level stdin is only for `batch`, `eval --stdin`, and `auth save --password-stdin`; other combinations are rejected before `agent-browser` is launched. `script` puts inner stdin on `browser({ stdin })`; `job`, `qa`, `sourceLookup`, `networkSourceLookup`, and `electron` generate or manage their own input.
218
- - `outputPath`: optional wrapper-owned local file sink for successful results and recording receipts, including failed, pending, timed-out or recovered stops. Recording exports preserve an envelope with original attempt status and native receipt/verification (`details.outputFile.source: "recording-receipt"`); unrelated failed extractions remain unwritten. Use it for durable `eval`, `get`, `snapshot`, or diagnostic outputs, not as the destination for screenshots, downloads, recordings, or other browser artifacts; if the paths resolve to the same file, the browser artifact is preserved and the result-data write fails validation. If presentation compacted a large direct result, a result row, or the whole `batch`, the writer copies each full command-redacted pre-compaction value only from its matching live wrapper-manifest spill; if any required spill is unavailable or untrusted, it fails without writing compact metadata. `details.outputFile` reports the saved path and byte count. If caller argv includes upstream `--json`, the visible JSON content stays parseable and the save notice is only in `details.outputFile`.
239
+ - `outputPath`: optional wrapper-owned local file sink for successful results and recording receipts, including failed, pending, timed-out or recovered stops. Recording exports preserve an envelope with original attempt status and native receipt/verification (`details.outputFile.source: "recording-receipt"`); unrelated failed extractions remain unwritten. Use it for durable `eval`, `get`, `snapshot`, or diagnostic outputs, not as the destination for screenshots, downloads, recordings, or other browser artifacts; if the paths resolve to the same file, the browser artifact is preserved and the result-data write fails validation. If presentation compacted a large direct result, a result row, or the whole `batch`, the writer copies each full command-redacted pre-compaction value only from its matching live wrapper-manifest spill; if any required spill is unavailable or untrusted, it fails without writing compact metadata. `details.outputFile` reports the saved path and byte count. If caller argv includes upstream `--json`, the visible JSON content stays parseable, including early snapshot/network filters, scroll results, and preparation failures; metadata stays in `details` and the save notice is only in `details.outputFile`. Help/version keeps native text output.
219
240
  - `timeoutMs`: optional per-call wrapper subprocess watchdog override in milliseconds for the requested browser CLI process. Managed-session policy inspection can independently consume up to 35 seconds before that process; this preflight is intentionally not shortened by `timeoutMs` because a busy but valid daemon must remain distinguishable from an unverifiable one.
220
241
  - `sessionMode`:
221
242
  - Unconfigured ordinary calls select a root-Pi named browser with native restore; descendants share it, unrelated roots do not wait on each other, and Pi quit leaves it running. Existing managed sessions and explicit fresh launches retain their lifecycle.
@@ -350,7 +371,7 @@ Do not assume Playwright selector dialects such as `text=Close` or `button:has-t
350
371
 
351
372
  Treat `@eN`, `eN`, and `ref=eN` selector refs as page-scoped. Ref-looking fill/type text, select values, paths, and keyboard/mouse data remain literal. After a successful `snapshot`, the wrapper records the latest refs and page target for that session; getter or mutation ref commands such as `get text @e4`, `click @e4`, `select @e5 chocolate`, or batch steps with old refs fail with `failureCategory: "stale-ref"` when the page target changed or the ref is absent from the latest same-page snapshot. If a session `snapshot -i` fails with `No active page`, the wrapper invalidates prior refs for that session; later mutation-prone `@e…` calls fail before upstream until a successful fresh `snapshot -i` records refs again. Inside `batch` stdin JSON, the wrapper also walks steps in order before spawn: steps whose first token can navigate or mutate set a latch; a later step whose first token is `snapshot` clears that latch for following rows; guarded steps that still mention `@e…` after an uncleared latch fail with the same `stale-ref` bucket without launching upstream. Same-snapshot form fills and native form-control steps are allowed before a click or submit step, so `fill`, `check`/`uncheck` checkbox or radio refs, checkbox/radio `click`/`tap` refs, `select` combobox refs, then a final submit `click` can run from one snapshot. Split dynamic or autosubmit forms with a fresh snapshot if a control interaction rerenders the targets. Follow the `refresh-interactive-refs` next action (it includes `--session <name>` when needed) and prefer stable `find` or `semanticAction` locators when navigation or rerendering is likely. Contract detail: [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details) (`refSnapshot`, `refSnapshotInvalidation`).
352
373
 
353
- A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
374
+ A successful `click` result means upstream reported a target, not that the app definitely handled the event. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper installs a bounded target-specific DOM-event probe when it can; when upstream reports success but no trusted event reaches the resolved target, it fails the tool and exposes `details.clickDispatch` plus a `Click dispatch diagnostic` line with explicit retry/inspect next actions (no in-page click replay). Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. Direct `@e…` click probes are role-gated to current snapshot refs whose accessible role is `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, with a unique role/name in both the saved snapshot and the live candidates. For both ref and XPath candidates, the wrapper requires native `get attr <original-selector> <marker>` to read back a per-probe temporary DOM marker on that candidate. Missing or mismatched identity (including frame/XPath scope mismatches) cleans up the probe and lets the native click run without dispatch verification. Duplicate-name refs pass through without a probe because their old ordinal does not prove target identity. If the probe evidence shows the target is outside a nested scroll container or viewport, `details.clickDispatch.scrollContainer` and `scroll-target-into-view-after-dispatch-miss` point to `scrollintoview <target>` before retry. When the workflow depends on a mutation, use `details.pageChangeSummary`, a wait, URL/text extraction, or a fresh `snapshot -i` before trusting the state; if nothing changed, retry with a current visible ref or stable selector and report the workflow issue. For static local fixtures or debugging where the user explicitly accepts scripted activation, `eval --stdin` can call `document.querySelector(...).click()` to exercise inline handlers and app code; treat that as an untrusted programmatic event, not as evidence that CDP/user-like clicking works. Respect explicit user stop boundaries yourself: if the user says to stop before a final order, post, purchase, or submit action, gather evidence from that page and do not click the final action or use scripted activation to bypass the stop. The wrapper does not infer broad business intent from prompt text; `details.promptGuard` is reserved for concrete artifact-before-close checks. `press`, `key`, `keydown`, and `keyup` accept exactly one key token; focus or click the target first, then run `press Enter` or another single-key command.
354
375
 
355
376
  Successful `snapshot -i` results can also surface `Possible overlay blockers` when their own refs already show dialog/alertdialog context plus close/dismiss controls, so agents can detect likely obstruction before clicking. When a **top-level** `@e…`/`ref=` click succeeds (not a `click` hidden inside a `batch`/`job` tool call—the unified command must be `click`), the upstream payload includes `data.clicked`, no `details.clickDispatch` diagnostic fired for the same result, and the wrapper sees `details.navigationSummary.url` unchanged after the same normalization it uses for ref guards (**`#fragment` ignored**), it may run one extra `snapshot -i` and surface `Possible overlay blockers` plus `details.overlayBlockers` (`candidates`, `summary`, and a `snapshot` map that can refresh `refSnapshot`) when that snapshot shows strong modal context (`dialog` / `alertdialog`) **and** up to three close/dismiss-like controls; page-wide words such as privacy, sign in, or banner alone do not trigger it. The URL check compares the session’s prior pinned tab target to `details.navigationSummary.url`. CSS selector clicks do not run this overlay probe. The diagnostic is skipped if the wrapper already applied tab-focus correction or about-blank recovery on that result. Appended `inspect-overlay-state` / `try-overlay-blocker-candidate-*` entries in `details.nextActions` preserve namespace/session context (`--namespace <namespace> --session <name>` when namespaced, otherwise `--session <name>` when the session is named), same as other session-scoped follow-ups. Treat `inspect-overlay-state` as the safe first follow-up; only use a `try-overlay-blocker-candidate-*` next action when the candidate is clearly the control you intend to close. A click that upstream rejects because another element covers the target's click point (`is covered by` … `at its click point`) remains `failureCategory: "upstream-error"`, but receives the same session-aware `inspect-overlay-state` snapshot action. This includes direct `click`, `semanticAction` click, raw `find` clicks (including `nth` and omitted default-click actions), and failed `batch`/`job` rows. That failed-click path does not retry the original click or synthesize a dismiss candidate before a fresh snapshot provides evidence; it is separate from silent input-dispatch failures.
356
377
 
@@ -442,7 +463,7 @@ Use raw `args: ["batch"]` with `stdin` when you need arbitrary upstream commands
442
463
 
443
464
  For quick smoke/QA checks, use top-level `qa`. It clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks expected text/selector, then inspects fresh network requests, console messages, and page errors only if preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped diagnostic output. If final page-error rows match a nonempty post-clear baseline, QA is non-pass with “page-error check could not be verified”: identical new errors can replace old rows in the native FIFO buffer without changing the result. Those matches are not proof of new application errors. Definitely new rows, including errors after an empty baseline, count separately as page errors; a clean final buffer can pass. Explicit `checkErrors: false` omits this check for a more limited smoke test. The preset compiles to `batch --bail` so a missing text/selector assertion fails crisply instead of letting slower diagnostics burn the wrapper watchdog. Expected text compiles to bounded visible-text `wait --fn … --timeout 5000` predicates after load so dense pages can pass on visible headings/copy without dumping `body` text; missing text reports a crisp QA failure. The readiness wait defaults to `loadState: "domcontentloaded"`; set `loadState` to `"load"` or `"networkidle"` only when that stricter state is useful and the site is not expected to keep background requests alive. QA network diagnostics classify failed requests by likely impact and list failed rows first in the network preview: actionable document/script/API-style failures fail the preset, while common low-impact browser icon misses such as `favicon.ico` are surfaced as warnings (`qaPreset.warnings`) so they do not fail an otherwise healthy page. Successful QA with no failed checks returns compact model-visible prose (page URL/title when known, checks run, optional screenshot verification) while keeping the full step matrix in `details.qaPreset` and `details.batchSteps`. Failed QA presets report `details.resultCategory: "failure"`, `failureCategory: "qa-failure"`, show compact failure reasons with the full matrix in `details.batchSteps`, and real Pi sessions treat the diagnostic as a failed tool result. Prose output also gets a model-visible result-category line including `Pi tool isError: true`; caller-requested `--json` output keeps the JSON string parseable and relies on the patched `isError` plus `details` fields.
444
465
 
445
- The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
466
+ The same classification drives plain `network requests` presentation: when any row counts as failed (HTTP status ≥ 400, `failed: true`, or a string `error`), model-facing text starts with a line like `Network failure summary: 0 actionable, 1 benign low-impact (1 total).`, and each preview line can end with an impact tag such as `[benign: low-impact browser icon asset]` or `[actionable: document, script, API, or non-benign request failure]`. When safe request IDs are present, `details.nextActions` adds bounded read-only follow-ups such as `network request <id>`, `networkSourceLookup` for actionable failed rows, `network requests --filter <path>`, `network requests --clear` before a repro, and `network har start`; prefer those payloads over rebuilding request-id commands from prose. For aggregate buffers, the wrapper accepts `network requests --current-page` / `--current-origin` to render only rows matching the active page origin, or `--current-url` for exact active document URL matching; it strips those wrapper-only flags before upstream spawn and reports counts in `details.networkRequestsPageFilter`. Filtered data receives the same credential redaction as ordinary network diagnostics before entering `details.data` or an `outputPath` export. If the wrapper has seen a prior `network route` in the same session, matching failed, pending, or CORS-looking fetch/XHR rows add `details.networkRouteDiagnostics` plus executable route-mock follow-ups (`inspect-routed-network-request` and `start-network-har-capture-for-route-mock`) so agents do not mistake an unfulfilled mock for a fulfilled mock; same-origin/CORS fixture retry guidance stays in visible prose. `network requests` also hides `data:image` screenshot/artifact noise from the compact preview by default while preserving raw rows in `details.data.requests`. Rules live in `classifyNetworkRequestFailure` / `summarizeNetworkFailures` in `extensions/agent-browser/lib/results/network.ts`; QA aggregation is `analyzeQaPresetResults` in `extensions/agent-browser/lib/input-modes/job.ts`.
446
467
 
447
468
  ```json
448
469
  { "qa": { "url": "https://example.com", "expectedText": "Example Domain", "screenshotPath": ".dogfood/qa-example.png" } }
@@ -476,7 +497,7 @@ Typical lifecycle:
476
497
  { "electron": { "action": "cleanup", "launchId": "electron-…" } }
477
498
  ```
478
499
 
479
- `electron.status` and `electron.cleanup` take either `launchId`, **`all: true`** (literal boolean) to walk every active wrapper-tracked launch (including dead, failed, or partial records, but excluding cleaned records), or neither when exactly one active launch exists—never both `launchId` and `all`. They can target the current branch-visible launch plus still-owned off-branch launch records by `launchId`; default no-arg calls are intentionally ambiguous when more than one active launch is owned. `/reload` preserves the current branch-visible active Electron launch and its isolated temp `userDataDir` for continuity, and cleans off-branch owned Electron launches. First reuse after reload/resume checks the live app's saved debug endpoint and the exact named upstream connection with `get cdp-url`, without reconnecting or resetting page refs; if cleanup is partial and skips or fails profile removal, the generic temp sweep preserves that `userDataDir` across reload, quit, later temp cleanup, process exit, and stale temp-root pruning after restart. `electron.list` has no configurable timeout and rejects both top-level and nested `timeoutMs`. For `electron.launch`, nested `timeoutMs` sets host CDP readiness polling to a **15s** default and **120s** cap after target discovery; upstream attach and handoff use separate subprocess budgets. Optional `timeoutMs` on **`status`** applies to each managed-session `get url` / `get title` read and any `get cdp-url` read needed to verify a restored connection (localhost CDP probes stay on a short fixed fetch budget). On **`cleanup`**, it is applied separately to upstream `close` and the initial host process-exit wait, not to the entire teardown; debug-port checks have fixed fetch budgets and profile removal has no configurable deadline; when omitted it follows the implicit session close default (**5s** unless `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` overrides). A successful managed-session close step retires that wrapper-managed session even when host process/profile cleanup remains partial. On **`probe`**, it bounds each underlying upstream read subprocess—omit it to use the normal tool subprocess default, or raise it on slow desktops.
500
+ `electron.status` and `electron.cleanup` take either `launchId`, **`all: true`** (literal boolean) to walk every active wrapper-tracked launch (including dead, failed, or partial records, but excluding cleaned records), or neither when exactly one active launch exists—never both `launchId` and `all`. They can target the current branch-visible launch plus still-owned off-branch launch records by `launchId`; default no-arg calls are intentionally ambiguous when more than one active launch is owned. `/reload` preserves the current branch-visible active Electron launch and its isolated temp `userDataDir` for continuity, and cleans off-branch owned Electron launches. First reuse after reload/resume checks the live app's saved debug endpoint and the exact named upstream connection with `get cdp-url`, without reconnecting or resetting page refs; if cleanup is partial and skips or fails profile removal, the generic temp sweep preserves that `userDataDir` across reload, quit, later temp cleanup, process exit, and stale temp-root pruning after restart. `electron.list` has no configurable timeout and rejects both top-level and nested `timeoutMs`. For `electron.launch`, nested `timeoutMs` sets host CDP readiness polling to a **15s** default and **120s** cap after target discovery; upstream attach and handoff use separate subprocess budgets. Optional `timeoutMs` on **`status`** applies to each managed-session `get url` / `get title` read and any `get cdp-url` read needed to verify a restored connection (localhost CDP probes stay on a short fixed fetch budget). On **`cleanup`**, it is applied separately to upstream `close` and the initial host process-exit wait, not to the entire teardown; debug-port checks have fixed fetch budgets and profile removal has no configurable deadline; when omitted it follows the implicit session close default (**35s** unless `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` overrides). A successful managed-session close step retires that wrapper-managed session even when host process/profile cleanup remains partial. On **`probe`**, it bounds each underlying upstream read subprocess—omit it to use the normal tool subprocess default, or raise it on slow desktops.
480
501
 
481
502
  Explicit-ID `electron.status` labels a cleaned record as historical while measuring PID/port liveness independently. `details.electron.statuses[].userDataDirState` freshly reports the tracked profile path as `present`, `absent` (only ENOENT), or `unknown` (other native `lstat` errors); dangling symlinks are present. This is not an audit of all app residue and does not change stored launch records or cleanup ownership.
482
503
 
@@ -548,7 +569,7 @@ A successful wait-based download renders a readable summary such as `Download co
548
569
 
549
570
  The upstream screenshot aliases are `screenshot --full` for full-page capture and `screenshot --annotate` for labeled screenshots. Annotated screenshots can be noisy on dense pages because labels overlap real content; when labels obscure evidence, capture a scoped element screenshot, take a non-annotated screenshot, or use `snapshot -i` high-value refs as the machine-readable map. When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute another path in the final report. When the latest prompt names exact required screenshot paths, `close` / `quit` / `exit` can be blocked with `details.promptGuard.reason: "requested-artifacts-missing-before-close"` until those paths appear as verified explicit artifacts.
550
571
 
551
- Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. For simple loopback HTML anchors with `href` and a non-ref selector, the wrapper first reads the resolved anchor URL and saves the in-page credentialed response directly to the requested host path, avoiding upstream random-name download spills in local fixture tests; non-loopback/profile downloads still use upstream fallback. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
572
+ Prefer `download <selector> <path>` when the target element itself is the downloadable link/control. Native download always owns the click and download, including loopback links, generated Blob exports, and redirects; the wrapper never substitutes a fetch of the anchor's `href`. Use `click` plus `wait --download [path]` when a previous action starts the download indirectly.
552
573
 
553
574
  For evidence-only screenshots, QA captures, or audit artifacts, save to an explicit path and branch on `details.artifactVerification` plus `details.artifacts` before reporting PASS/FAIL. Inline image attachments are optional convenience when size limits allow; do not require vision review unless the user asked for visual inspection.
554
575
 
@@ -722,14 +743,14 @@ Skill-source debugging note: upstream honors `AGENT_BROWSER_SKILLS_DIR` as an ov
722
743
  | `screenshot [selector] [path]` | Take a full-page or element-scoped screenshot; a single selector-like argument scopes, while a path-like argument saves to that path. |
723
744
  | `screenshot [path]` | Take a screenshot and optionally save it to a path. |
724
745
  | `pdf <path>` | Save the page as a PDF. |
725
- | `snapshot` | Print an accessibility tree with refs for AI interaction. Common options include `snapshot --interactive`, `snapshot --urls`, `snapshot --compact`, `snapshot --depth <n>`, `snapshot --selector <sel>`, and `snapshot --cursor` / `snapshot -C` for cursor/focus context when upstream returns it. |
746
+ | `snapshot` | Print an accessibility tree with refs for AI interaction. Common options include `snapshot --interactive`, `snapshot --urls`, `snapshot --compact`, `snapshot --depth <n>`, `snapshot --selector <sel>`, and native `snapshot --delta` / `snapshot --delta --full`. Cursor-interactive elements are included by default; `snapshot --cursor` / `snapshot -C` are deprecated no-ops. |
726
747
  | `eval <js>` | Run JavaScript. Use `eval --stdin` through this wrapper for larger snippets, or `eval -b <base64>` for shell-escaping-safe one-liners. |
727
748
  | `connect <port|url>` | Connect to a browser through CDP. |
728
749
  | `close [--all]` | Close the current browser or all sessions; `quit` and `exit` are upstream close aliases. |
729
750
  | `tap <selector>` | Touch-oriented tap alias for iOS/provider workflows. |
730
751
  | `swipe <direction> [distance]` | Touch-oriented swipe for iOS/provider workflows. |
731
752
 
732
- On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
753
+ On dashboards and other apps with nested scroll containers, `scroll <dir> [px]` can miss because a page-level wheel does not move the document or the intended pane. Without startup-scoped launch flags, the wrapper first applies ordinary `scroll <up|down|left|right> [px|percent]` directly to `document.scrollingElement` with smooth scrolling temporarily disabled; successful movement reports `details.scrollPage`. If the document cannot move, it falls back to upstream wheel behavior. For large fallback calls on an existing or fresh managed session, the wrapper samples viewport and prominent scroll-container positions before and after the command; when nothing changes it reclassifies the nominal upstream success as `failureCategory: "upstream-error"`, prepends `Scroll completed with no observed movement`, appends `Scroll diagnostic: no observed scroll movement`, exposes `details.scrollNoop`, marks `details.data.scrolled: false`, and adds exact `details.nextActions` for a fresh `snapshot -i` and screenshot. Explicit CSS-container calls `scroll <selector> <up|down|left|right> [px|percent]` remain wrapper-handled, use `Element.scrollBy` with `behavior: "instant"` so smooth-scroll CSS cannot delay the measured movement, and report `details.scrollContainer`; `scroll to end` / `scroll to top` report `details.scrollPage`. Calls with startup-scoped flags skip all helper shims so the requested launch configuration runs first. Use these paths before repeating page scrolls; when you need a specific element, prefer `scrollintoview <@ref>` or target the actual scrollable region. Do not pass `text=...` to `scrollintoview`: the wrapper rejects that upstream false-success path and returns `scroll-semantic-text-target` (`find text ... hover`) plus `refresh-refs-for-scroll-target` (`snapshot -i`) actions.
733
754
 
734
755
  Comboboxes vary by app. For native `<select>` controls, prefer raw `select <selector> <value...>`, direct `semanticAction: { action: "select", selector, value|values }`, active-session semantic role/name or label select, or a `job` `select` step instead of clicking option refs; native option refs can be non-boxed in CDP and fail before a real selection. A `click` or `semanticAction` role/name click may focus a searchable custom combobox without opening its option list. For explicit combobox-targeted actions such as `semanticAction` role `combobox`, the wrapper checks whether a combobox-like element is focused, has explicit `aria-expanded` state, and has no visible listbox/options open; this still applies when the semantic action first resolves to a current visible `@ref` before execution. When that happens it appends `Combobox diagnostic: focused combobox did not expose visible options`, exposes `details.comboboxFocus`, and adds exact `details.nextActions` for a fresh `snapshot -i`, `press ArrowDown`, and `press Enter`. Use those instead of assuming click alone expanded the control. To search the focused input, use `keyboard type <text>`; raw `type` requires both a selector and text. Reserve visible option refs for custom comboboxes after a fresh snapshot shows the intended option.
735
756
 
@@ -835,10 +856,11 @@ With `--pin-tab`, a closed bound tab fails as `tab_gone` (`data.targetId`, optio
835
856
  | `snapshot -i` / `snapshot --interactive` | Include only interactive elements. |
836
857
  | `snapshot -i --urls` | Include only interactive elements and link hrefs. |
837
858
  | `snapshot -u` / `snapshot --urls` | Include href URLs for link elements. |
838
- | `snapshot -C` / `snapshot --cursor` | Include cursor/focus context when upstream provides it. |
859
+ | `snapshot -C` / `snapshot --cursor` | Deprecated no-ops; cursor-interactive elements are included by default. |
839
860
  | `snapshot -c` / `snapshot --compact` | Remove empty structural elements. |
840
861
  | `snapshot -d <n>` / `snapshot --depth <n>` | Limit tree depth. |
841
862
  | `snapshot -s <sel>` / `snapshot --selector <sel>` | Scope to a CSS selector. |
863
+ | `snapshot --delta` / `snapshot --delta --full` | Native revision-relative observation or a forced full baseline; see [0.38.1](#upstream-0381-rebaseline). |
842
864
 
843
865
  When a snapshot is too large for inline output, the Pi wrapper renders a compact view before spilling the full redacted snapshot to `details.fullOutputPath`. Compact snapshots are main-content-first, but dense pages and desktop host screens can still hide actionable controls in omitted content; scan `Omitted high-value controls` before opening the spill file. That bounded section favors editable/searchbox/textbox/combobox controls, named tab/surface controls, primary action buttons, and named action links such as row/navigation links and repository-style result links, then includes other useful controls such as checkboxes, radios, options, and menuitems that were not already listed under key refs or other refs. When that section appears, `details.data.highValueControlRefIds` repeats the same visible ref ids for programmatic follow-up alongside fields such as `previewMode`, `previewSections`, and counts on `details.data` (see [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#details)).
844
866
 
@@ -903,7 +925,7 @@ Long-running or lifecycle commands should be explicitly paired with cleanup call
903
925
  | --- | --- |
904
926
  | `batch [--bail] ["cmd" ...]` | Execute multiple commands sequentially from args or stdin. |
905
927
  | `auth save <name> [opts]` | Save an auth profile. Full credential form: `auth save <name> --url <url> --username <user> --password <pass>`; selector override form: `auth save <name> --username-selector <s> --password-selector <s> --submit-selector <s>`. Prefer `auth save <name> --password-stdin` with the tool `stdin` field; avoid putting passwords in `args`. |
906
- | `auth login <name>` | Login using saved credentials. |
928
+ | `auth login <name>` | Login using saved credentials; add `--no-navigate` for a prepared, same-origin login page. |
907
929
  | `auth list` | List saved auth profiles. |
908
930
  | `auth show <name>` | Show auth profile metadata. |
909
931
  | `auth delete <name>` | Delete an auth profile; `auth remove <name>` is the upstream alias. |
@@ -1127,14 +1149,14 @@ Other useful environment variables include `AGENT_BROWSER_DEFAULT_TIMEOUT`, `AGE
1127
1149
  <!-- agent-browser-capability-baseline:start capability-token-baseline -->
1128
1150
  <!-- Generated from scripts/agent-browser-capability-baseline.mjs. Run `npm run docs -- command-reference write` to update. Do not edit manually. -->
1129
1151
  <details>
1130
- <summary>Generated verifier capability baseline for agent-browser 0.37.0</summary>
1152
+ <summary>Generated verifier capability baseline for agent-browser 0.38.1</summary>
1131
1153
 
1132
1154
  This generated block is review data for maintainers. The human-authored reference sections above remain the readable command guide.
1133
1155
 
1134
1156
  #### Source evidence
1135
1157
  - repository: `vercel-labs/agent-browser`
1136
- - upstream HEAD: `471ab3852b47b98847f1d9c855c272bb62d0d50b`
1137
- - upstream package version: `0.37.0`
1158
+ - upstream HEAD: `aff6125c023b810ea3f2e5deec5379e9a4270bdc`
1159
+ - upstream package version: `0.38.1`
1138
1160
  - inspected: `agent-browser --version`
1139
1161
  - inspected: `agent-browser --help`
1140
1162
  - inspected: `selected agent-browser <command> --help output`
@@ -1163,6 +1185,9 @@ This generated block is review data for maintainers. The human-authored referenc
1163
1185
  - inspected: `cli/src/native/daemon.rs`
1164
1186
  - inspected: `cli/src/native/element.rs`
1165
1187
  - inspected: `cli/src/native/recording.rs`
1188
+ - inspected: `cli/src/native/snapshot.rs`
1189
+ - inspected: `cli/src/native/interaction.rs`
1190
+ - inspected: `cli/src/native/screenshot.rs`
1166
1191
  - inspected: `cli/src/native/stream/cdp_loop.rs`
1167
1192
  - inspected: `cli/src/native/stream/dashboard.rs`
1168
1193
  - inspected: `cli/src/native/test_fixtures/webmcp_frame_probe.html`
@@ -1209,6 +1234,7 @@ This generated block is review data for maintainers. The human-authored referenc
1209
1234
  - get help: `agent-browser get --help`
1210
1235
  - is help: `agent-browser is --help`
1211
1236
  - mouse help: `agent-browser mouse --help`
1237
+ - drag help: `agent-browser drag --help`
1212
1238
  - set help: `agent-browser set --help`
1213
1239
  - tab help: `agent-browser tab --help`
1214
1240
  - snapshot help: `agent-browser snapshot --help`
@@ -1252,11 +1278,11 @@ This generated block is review data for maintainers. The human-authored referenc
1252
1278
 
1253
1279
  #### Inventory sections
1254
1280
  - Built-in skills: 19 human-doc token(s), 24 upstream token(s)
1255
- - Core page, element, navigation, and extraction commands: 82 human-doc token(s), 85 upstream token(s)
1281
+ - Core page, element, navigation, and extraction commands: 89 human-doc token(s), 94 upstream token(s)
1256
1282
  - Sessions, state, tabs, frames, dialogs, and windows: 28 human-doc token(s), 26 upstream token(s)
1257
- - Network, storage, artifacts, diagnostics, and performance: 58 human-doc token(s), 68 upstream token(s)
1258
- - Batch, auth, confirmations, setup, dashboard, devices, and AI commands: 36 human-doc token(s), 40 upstream token(s)
1259
- - Global flags, config, providers, policy, and environment: 152 human-doc token(s), 119 upstream token(s)
1283
+ - Network, storage, artifacts, diagnostics, and performance: 62 human-doc token(s), 71 upstream token(s)
1284
+ - Batch, auth, confirmations, setup, dashboard, devices, and AI commands: 37 human-doc token(s), 41 upstream token(s)
1285
+ - Global flags, config, providers, policy, and environment: 153 human-doc token(s), 120 upstream token(s)
1260
1286
 
1261
1287
  #### Human-authored doc tokens required
1262
1288
  ##### Built-in skills
@@ -1327,6 +1353,13 @@ This generated block is review data for maintainers. The human-authored referenc
1327
1353
  - `screenshot [path]`
1328
1354
  - `screenshot --full`
1329
1355
  - `screenshot --annotate`
1356
+ - `screenshot --if-changed`
1357
+ - `screenshot --threshold <0-1>`
1358
+ - `snapshot --delta`
1359
+ - `snapshot --delta --full`
1360
+ - `click <sel> --human`
1361
+ - `drag <src> <dst> --human`
1362
+ - `mouse move <x> <y> --duration <ms> --steps <n> --human --seed <n>`
1330
1363
  - `pdf <path>`
1331
1364
  - `snapshot`
1332
1365
  - `snapshot --cursor`
@@ -1410,6 +1443,7 @@ This generated block is review data for maintainers. The human-authored referenc
1410
1443
  - `cookies set --curl <file>`
1411
1444
  - `storage <local|session>`
1412
1445
  - `webmcp list`
1446
+ - `webmcp list [tool] [--frame <frame-id>]`
1413
1447
  - `webmcp invoke <tool>`
1414
1448
  - `webmcp invoke <tool> --params <json|@file>`
1415
1449
  - `webmcp invoke <tool> --frame <frame-id>`
@@ -1429,6 +1463,9 @@ This generated block is review data for maintainers. The human-authored referenc
1429
1463
  - `record start <path> [url]`
1430
1464
  - `record restart <path> [url]`
1431
1465
  - `--fps <n>`
1466
+ - `--cursor`
1467
+ - `--contact-sheet`
1468
+ - `--contact-sheet-threshold <n>`
1432
1469
  - `record stop`
1433
1470
  - `console [--clear]`
1434
1471
  - `errors [--clear]`
@@ -1461,6 +1498,7 @@ This generated block is review data for maintainers. The human-authored referenc
1461
1498
  - `auth save <name> --username-selector <s> --password-selector <s> --submit-selector <s>`
1462
1499
  - `auth save <name> --password-stdin`
1463
1500
  - `auth login <name>`
1501
+ - `auth login <name> --no-navigate`
1464
1502
  - `auth list`
1465
1503
  - `auth show <name>`
1466
1504
  - `auth delete <name>`
@@ -1564,6 +1602,7 @@ This generated block is review data for maintainers. The human-authored referenc
1564
1602
  - `AGENT_BROWSER_DOWNLOAD_PATH`
1565
1603
  - `--engine <name>`
1566
1604
  - `AGENT_BROWSER_ENGINE`
1605
+ - `--input-mode <mode>`
1567
1606
  - `--no-auto-dialog`
1568
1607
  - `AGENT_BROWSER_NO_AUTO_DIALOG`
1569
1608
  - `--json`
@@ -1755,6 +1794,15 @@ This generated block is review data for maintainers. The human-authored referenc
1755
1794
  - keyboard help: `inserttext <text>`
1756
1795
  - screenshot help: `--full, -f`
1757
1796
  - screenshot help: `--annotate`
1797
+ - screenshot help: `--if-changed`
1798
+ - screenshot help: `--threshold <0-1>`
1799
+ - snapshot help: `--delta`
1800
+ - snapshot help: `--full`
1801
+ - click help: `--human`
1802
+ - drag help: `--human`
1803
+ - mouse help: `--duration <ms>`
1804
+ - mouse help: `--steps <n>`
1805
+ - mouse help: `--seed <n>`
1758
1806
  - find help: `role <role>`
1759
1807
  - find help: `testid <id>`
1760
1808
  - tap help: `tap <selector>`
@@ -1799,7 +1847,7 @@ This generated block is review data for maintainers. The human-authored referenc
1799
1847
  - root help: `cookies [get|set|clear]`
1800
1848
  - root help: `cookies set --curl <file>`
1801
1849
  - root help: `storage <local|session>`
1802
- - root help: `webmcp list`
1850
+ - root help: `webmcp list [tool]`
1803
1851
  - root help: `webmcp invoke <tool>`
1804
1852
  - root help: `--params <json|@file>`
1805
1853
  - root help: `--frame <frame-id>`
@@ -1855,6 +1903,9 @@ This generated block is review data for maintainers. The human-authored referenc
1855
1903
  - profiler help: `--categories <list>`
1856
1904
  - record help: `record restart <path.webm|path.mp4> [url] [--fps <n>]`
1857
1905
  - record help: `--fps <n>`
1906
+ - record help: `--cursor`
1907
+ - record help: `--contact-sheet`
1908
+ - record help: `--contact-sheet-threshold <n>`
1858
1909
  - console help: `--clear`
1859
1910
  - errors help: `--clear`
1860
1911
 
@@ -1895,6 +1946,7 @@ This generated block is review data for maintainers. The human-authored referenc
1895
1946
  - root help: `plugin show <name>`
1896
1947
  - root help: `plugin run <name> <type>`
1897
1948
  - auth help: `--credential-provider <p>`
1949
+ - auth help: `--no-navigate`
1898
1950
  - mcp help: `agent_browser_open`
1899
1951
  - mcp help: `--tools`
1900
1952
  - plugin help: `Add a plugin from npm or GitHub`
@@ -1968,6 +2020,7 @@ This generated block is review data for maintainers. The human-authored referenc
1968
2020
  - root help: `AGENT_BROWSER_DOWNLOAD_PATH`
1969
2021
  - root help: `--engine <name>`
1970
2022
  - root help: `AGENT_BROWSER_ENGINE`
2023
+ - root help: `--input-mode <mode>`
1971
2024
  - root help: `--no-auto-dialog`
1972
2025
  - root help: `AGENT_BROWSER_NO_AUTO_DIALOG`
1973
2026
  - root help: `--json`
package/docs/ELECTRON.md CHANGED
@@ -210,7 +210,7 @@ On Pi `quit`, active wrapper-owned Electron launches are best-effort cleaned. On
210
210
  | --- | --- | --- |
211
211
  | `launch` | Host-side wait for `DevToolsActivePort` and CDP readiness | **15 s**, hard-capped at **120 s** (`normalizeTimeoutMs` in `extensions/agent-browser/lib/electron/launch.ts`) |
212
212
  | `status` | Each optional managed-session `get url` / `get title` subprocess, including `get cdp-url` when verifying a restored connection | Normal wrapper subprocess budget (**35 s**, or `PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS`); localhost CDP probes use **1000 ms** each (`ELECTRON_CDP_FETCH_TIMEOUT_MS` in `extensions/agent-browser/lib/electron/cdp.ts`) |
213
- | `cleanup` | Applied separately to managed-session `close` and the initial tracked-process exit wait; not a deadline for debug-port checks or profile removal | `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` when set, else **5000 ms** (`getImplicitSessionCloseTimeoutMs` in `extensions/agent-browser/lib/runtime.ts`, passed through `cleanupTrackedElectronHostLaunches` in `extensions/agent-browser/lib/orchestration/electron-host/index.ts`) |
213
+ | `cleanup` | Applied separately to managed-session `close` and the initial tracked-process exit wait; not a deadline for debug-port checks or profile removal | `PI_AGENT_BROWSER_IMPLICIT_SESSION_CLOSE_TIMEOUT_MS` when set, else **35000 ms** (`getImplicitSessionCloseTimeoutMs` in `extensions/agent-browser/lib/runtime.ts`, passed through `cleanupTrackedElectronHostLaunches` in `extensions/agent-browser/lib/orchestration/electron-host/index.ts`) |
214
214
  | `probe` | **Each** upstream read: optional `get cdp-url` verification, then `get url`, `get title`, focused `eval --stdin`, `tab list`, and `snapshot -i` | Same wrapper subprocess default (**35 s**, or `PI_AGENT_BROWSER_PROCESS_TIMEOUT_MS`, from `getAgentBrowserProcessTimeoutMs` in `extensions/agent-browser/lib/process.ts`) |
215
215
 
216
216
  ## `qa.attached` — current-session smoke check
package/docs/RELEASE.md CHANGED
@@ -47,7 +47,7 @@ npm run verify -- pre-pr
47
47
  3. `npm run verify -- package-pi`, which first validates package contents via `npm pack --json --dry-run` and then smoke-loads the packed package in Pi isolation
48
48
  4. `npm run smoke:platform:doctor` and the full Crabbox matrix from [`platform-smoke.md`](platform-smoke.md): macOS SSH, Ubuntu local-container, and native Windows Parallels targets running fast target-local `platform-build` plus `browser-dogfood-smoke`
49
49
 
50
- The 0.6.13 release uses the documented [local macOS and hosted native-Windows alternatives](platform-smoke.md#alternate-native-transports) instead of SSH and Parallels. Run and inspect those target-local suites separately alongside the remaining release checks. The default composed gate is unchanged; alternate evidence must not be labeled a passing Crabbox matrix.
50
+ The 0.6.15 release uses the documented [local macOS and hosted native-Windows alternatives](platform-smoke.md#alternate-native-transports) instead of SSH and Parallels. Run and inspect those target-local suites separately alongside the remaining release checks. The default composed gate is unchanged; alternate evidence must not be labeled a passing Crabbox matrix.
51
51
 
52
52
  `npm publish` runs npm’s `prepublishOnly` script from `package.json`, which executes the same `npm run verify -- release` gate and then `npm pack --dry-run`. That concatenated gate is everything in the default `npm run verify` step (generated playbook drift, clean `dist/` build, TypeScript, the unit/fake suite, generated command-reference blocks, and live upstream command-reference sampling against the targeted `agent-browser` on `PATH`), the configured-source lifecycle harness, the packaged Pi smoke in `package-pi`, and the release-blocking Crabbox platform matrix. Using `npm publish --ignore-scripts` skips that contract intentionally.
53
53
 
@@ -269,8 +269,9 @@ That npm script sets `PI_AGENT_BROWSER_REAL_UPSTREAM=1` for the test process. To
269
269
 
270
270
  This suite requires the installed stable `agent-browser --version` to meet the minimum in `scripts/agent-browser-target.mjs`; the output-shape fixture and command-reference verifier remain aligned to the current recommended baseline. It serves fixture pages from localhost and checks stable `details`/`data` keys via `test/fixtures/agent-browser-real-output-shapes.json`. Coverage groups:
271
271
 
272
- - **Inspection and skills (stateless JSON):** `--version`, `--help`, `snapshot --help`, `skills list`, `skills get … --full` (including `webmcp-gen` on the 0.37.0 target), `skills path …` (no managed `sessionName` / `usedImplicitSession`).
273
- - **WebMCP target contract:** on the 0.37.0 target, `webmcp list`, `invoke` with params/frame selection, detached `result` / `cancel`, ref invalidation after page tools, and a separate fresh launch with `--no-webmcp` returning an empty list.
272
+ - **Inspection and skills (stateless JSON):** `--version`, `--help`, `snapshot --help`, `skills list`, `skills get … --full` (including `webmcp-gen` on the 0.38.1 target), `skills path …` (no managed `sessionName` / `usedImplicitSession`).
273
+ - **Native 0.38 observation/recording:** full/unchanged delta snapshots, persistent refs, partial batch ref follow-ups, conditional screenshot suppression, sticky pointer mode and timed mouse movement, cursor recording and verified contact-sheet output. These options are skipped when explicitly validating the older stable floor.
274
+ - **WebMCP target contract:** on the 0.38.1 target, `webmcp list`, `invoke` with params/frame selection, detached `result` / `cancel`, ref invalidation after page tools, and a separate fresh launch with `--no-webmcp` returning an empty list.
274
275
  - **Navigation metadata and native tab setup:** plain pages stay quiet; a page with tools retains positive native `data.webmcp` and shows the `webmcp list` hint. New tabs inherit configured headers before their first request, and clearing headers removes them on the next tab. No wrapper inheritance code is involved.
275
276
  - **Recording FPS paths and pinning:** the opt-in `test/agent-browser.batch-fidelity.test.ts` native case checks direct/raw/stdin preflight before dispatch, actual start/restart destinations and pinned targets with leading `--fps`, conservative start-ref protection, FPS-only restart continuity, and valid 12-fps WebM output after explicitly logged 12-second fixture captures. Run it with `PI_AGENT_BROWSER_REAL_UPSTREAM=1`; it requires native 0.37 or newer. Keep focused older-native controls for fresh-page starts and the start-then-ref batch latch when changing that policy.
276
277
  - **Managed session core and safe diagnostic matrix:** fresh `open` on the contract fixture, then implicit reuse across `eval --stdin`, `snapshot -i`, interaction commands (`click`, `dblclick`, `fill`, `type`, `type --clear --delay`, `focus`, `keyboard` with `type` / `inserttext`, `press`, `hover`, `check`, `uncheck`, `select`, failed `select` no-match, `upload`, `drag`, `mouse`, `scroll`, off-viewport click, `scrollintoview`, `wait` on selectors in the main frame and a selected iframe), extraction (`get` variants, `is` variants, `find label … fill` via native `<label>`, `aria-label`, and `aria-labelledby`, inline `eval`), file outputs (`screenshot`, `pdf`), navigation (`back`, `forward`, `reload`, `tab list`, another `open` to the same fixture), `batch` stdin, `pushstate`, `vitals … --json`, network route/requests/HAR, diff snapshot/screenshot/url, trace/profiler, console/errors/highlight, stream enable/status/disable, and `cookies set --curl`.
@@ -71,7 +71,7 @@ Define the product requirements and constraints for `pi-agent-browser-native`.
71
71
  - Supported actions, locators, exclusivity rules, when `details.compiledSemanticAction` appears, and bounded `try-*-candidate` follow-ups on `selector-not-found` (specific action/locator pairs only; see contract) are specified in [`TOOL_CONTRACT.md`](TOOL_CONTRACT.md#semanticaction), with workflow examples in [`COMMAND_REFERENCE.md`](COMMAND_REFERENCE.md).
72
72
  - Constrained `job` remains a thin batch compiler, but its `click`/`fill` steps may use the same semantic locator fields as `semanticAction` so short workflows can avoid brittle selectors without adding a reusable recipe runtime, and `type` steps may expand to a bounded set of existing upstream focus/keyboard/wait/press rows for human-paced input while compacting model-visible batch text. `job` must default to fail-fast (`batch --bail`) so later mutating steps do not run after an earlier required step fails; `failFast: false` is the explicit opt-out.
73
73
  - `qa` must fail fast on failed readiness/text/selector assertions so missing expected text cannot burn the wrapper watchdog before reporting `qa-failure`. `qa.attached` must never erase existing session diagnostics; URL-opening `qa` may clear buffers to scope a fresh page load. Attached QA preserves buffers, reports that scope in `details.compiledQaPreset.checks.diagnosticsResetAtStart`, and defaults diagnostic checks off unless the caller opts into preserved-buffer failure checks.
74
- - Direct artifact workflows must create missing parent directories before spawning upstream and must verify saved files before downstream use. Simple loopback HTTP(S) anchor downloads may be saved directly by the wrapper to the requested path when that avoids upstream random-name download behavior without bypassing authenticated browser credentials. `outputPath` may write a successful result payload to a caller-requested local file and must report `details.outputFile`; it must never overwrite a screenshot, download, recording, or other browser artifact from the same result, including through a filesystem alias. Missing non-pending artifacts, including diff screenshot outputs, must never use saved/verified wording; `record start` future files are pending/open until `record stop`, not missing.
74
+ - Direct artifact workflows must create missing parent directories before spawning upstream and must verify saved files before downstream use. Native download must own the click and download, including loopback links, generated Blob exports, and redirects; the wrapper must not substitute an anchor-href fetch. `outputPath` may write a successful result payload to a caller-requested local file and must report `details.outputFile`; it must never overwrite a screenshot, download, recording, or other browser artifact from the same result, including through a filesystem alias. Missing non-pending artifacts, including diff screenshot outputs, must never use saved/verified wording; `record start` future files are pending/open until `record stop`, not missing.
75
75
 
76
76
  ### Documentation standard
77
77