pi-agent-browser-native 0.2.77 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +154 -0
- package/README.md +69 -35
- package/dist/extensions/agent-browser/index.js +911 -121
- package/dist/extensions/agent-browser/lib/argv-descriptor.js +35 -8
- package/dist/extensions/agent-browser/lib/argv-grammar.js +48 -7
- package/dist/extensions/agent-browser/lib/batch-lifecycle.js +71 -0
- package/dist/extensions/agent-browser/lib/command-policy.js +1 -6
- package/dist/extensions/agent-browser/lib/command-taxonomy.js +35 -8
- package/dist/extensions/agent-browser/lib/config-policy.js +0 -5
- package/dist/extensions/agent-browser/lib/config.js +0 -6
- package/dist/extensions/agent-browser/lib/electron/cdp.js +6 -7
- package/dist/extensions/agent-browser/lib/electron/cleanup.js +0 -7
- package/dist/extensions/agent-browser/lib/electron/discovery.js +0 -7
- package/dist/extensions/agent-browser/lib/electron/launch.js +0 -7
- package/dist/extensions/agent-browser/lib/input-modes/electron.js +0 -5
- package/dist/extensions/agent-browser/lib/input-modes/job.js +61 -9
- package/dist/extensions/agent-browser/lib/input-modes/lookups.js +2 -7
- package/dist/extensions/agent-browser/lib/input-modes/params.js +23 -29
- package/dist/extensions/agent-browser/lib/input-modes/script.js +462 -0
- package/dist/extensions/agent-browser/lib/input-modes/semantic-action.js +51 -17
- package/dist/extensions/agent-browser/lib/input-modes/shared.js +0 -5
- package/dist/extensions/agent-browser/lib/input-modes/types.js +0 -5
- package/dist/extensions/agent-browser/lib/json-schema.js +0 -5
- package/dist/extensions/agent-browser/lib/launch-scoped-flags.js +12 -5
- package/dist/extensions/agent-browser/lib/managed-session-capabilities.js +0 -2
- package/dist/extensions/agent-browser/lib/managed-session-policy-lock.js +3 -4
- package/dist/extensions/agent-browser/lib/managed-session-restore.js +55 -48
- package/dist/extensions/agent-browser/lib/managed-session-snapshots.js +2 -9
- package/dist/extensions/agent-browser/lib/managed-session-state-policy.js +48 -36
- package/dist/extensions/agent-browser/lib/managed-session-storage.js +50 -29
- package/dist/extensions/agent-browser/lib/navigation-policy.js +0 -5
- package/dist/extensions/agent-browser/lib/orchestration/batch-stdin.js +26 -5
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/artifact-paths.js +110 -30
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.js +2 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/diagnostics.js +27 -26
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/final-result.js +58 -33
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/index.js +2 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/managed-session-daemon-policy.js +22 -14
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/direct-anchor-download.js +1 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/network-page-filter.js +1 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/scroll-shims.js +1 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare/snapshot-filter.js +1 -1
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/prepare.js +148 -34
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/process-output.js +225 -61
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-state.js +74 -30
- package/dist/extensions/agent-browser/lib/orchestration/electron-host/index.js +47 -26
- package/dist/extensions/agent-browser/lib/orchestration/input-plan.js +40 -19
- package/dist/extensions/agent-browser/lib/orchestration/output-file.js +42 -1
- package/dist/extensions/agent-browser/lib/orchestration/script-mode.js +299 -0
- package/dist/extensions/agent-browser/lib/parsing.js +0 -7
- package/dist/extensions/agent-browser/lib/pi-tool-rendering.js +36 -11
- package/dist/extensions/agent-browser/lib/playbook.js +20 -24
- package/dist/extensions/agent-browser/lib/process-environment.js +14 -0
- package/dist/extensions/agent-browser/lib/process-identity.js +4 -9
- package/dist/extensions/agent-browser/lib/process.js +131 -48
- package/dist/extensions/agent-browser/lib/prompt-policy.js +171 -27
- package/dist/extensions/agent-browser/lib/recording-reservations.js +183 -0
- package/dist/extensions/agent-browser/lib/results/action-recommendations.js +63 -13
- package/dist/extensions/agent-browser/lib/results/artifact-manifest.js +68 -11
- package/dist/extensions/agent-browser/lib/results/categories.js +7 -9
- package/dist/extensions/agent-browser/lib/results/confirmation.js +0 -7
- package/dist/extensions/agent-browser/lib/results/contracts.js +0 -7
- package/dist/extensions/agent-browser/lib/results/editable-ref-evidence.js +0 -7
- package/dist/extensions/agent-browser/lib/results/envelope.js +0 -7
- package/dist/extensions/agent-browser/lib/results/network.js +0 -7
- package/dist/extensions/agent-browser/lib/results/next-actions.js +19 -35
- package/dist/extensions/agent-browser/lib/results/presentation/artifacts.js +85 -44
- package/dist/extensions/agent-browser/lib/results/presentation/batch.js +66 -11
- package/dist/extensions/agent-browser/lib/results/presentation/browser-profile-recovery.js +0 -5
- package/dist/extensions/agent-browser/lib/results/presentation/common.js +18 -5
- package/dist/extensions/agent-browser/lib/results/presentation/content.js +0 -5
- package/dist/extensions/agent-browser/lib/results/presentation/diagnostics.js +7 -6
- package/dist/extensions/agent-browser/lib/results/presentation/errors.js +2 -1
- package/dist/extensions/agent-browser/lib/results/presentation/large-output.js +0 -5
- package/dist/extensions/agent-browser/lib/results/presentation/managed-list-filter.js +0 -5
- package/dist/extensions/agent-browser/lib/results/presentation/navigation.js +26 -16
- package/dist/extensions/agent-browser/lib/results/presentation/registry.js +58 -13
- package/dist/extensions/agent-browser/lib/results/presentation/semantic-action.js +1 -15
- package/dist/extensions/agent-browser/lib/results/presentation/skills.js +0 -5
- package/dist/extensions/agent-browser/lib/results/presentation.js +6 -8
- package/dist/extensions/agent-browser/lib/results/recovery-actions.js +2 -7
- package/dist/extensions/agent-browser/lib/results/recovery-next-actions.js +0 -7
- package/dist/extensions/agent-browser/lib/results/selector-recovery.js +51 -15
- package/dist/extensions/agent-browser/lib/results/snapshot-high-value-controls.js +13 -14
- package/dist/extensions/agent-browser/lib/results/snapshot-refs.js +0 -7
- package/dist/extensions/agent-browser/lib/results/snapshot-segments.js +0 -7
- package/dist/extensions/agent-browser/lib/results/snapshot-spill.js +0 -7
- package/dist/extensions/agent-browser/lib/results/snapshot.js +0 -7
- package/dist/extensions/agent-browser/lib/results/text.js +0 -7
- package/dist/extensions/agent-browser/lib/runtime.js +119 -52
- package/dist/extensions/agent-browser/lib/session-page-state.js +62 -17
- package/dist/extensions/agent-browser/lib/string-enum-schema.js +0 -5
- package/dist/extensions/agent-browser/lib/temp.js +0 -7
- package/dist/extensions/agent-browser/lib/upstream-version.js +14 -0
- package/dist/extensions/agent-browser/lib/web-search.js +0 -5
- package/dist/extensions/agent-browser/script-worker.js +169 -0
- package/dist/scripts/agent-browser-target.mjs +3 -0
- package/docs/ARCHITECTURE.md +50 -31
- package/docs/COMMAND_REFERENCE.md +97 -41
- package/docs/ELECTRON.md +3 -19
- package/docs/RELEASE.md +8 -20
- package/docs/REQUIREMENTS.md +8 -6
- package/docs/SUPPORT_MATRIX.md +32 -25
- package/docs/TOOL_CONTRACT.md +107 -64
- package/docs/platform-smoke.md +1 -1
- package/package.json +7 -6
- package/platform-smoke.config.mjs +2 -2
- package/scripts/agent-browser-capability-baseline.mjs +24 -6
- package/scripts/agent-browser-target.mjs +3 -0
- package/scripts/build.mjs +41 -0
- package/scripts/doctor.mjs +8 -7
- package/scripts/platform-smoke/browser-dogfood-windows.ps1 +9 -3
- package/scripts/platform-smoke/targets.mjs +12 -6
- package/dist/extensions/agent-browser/lib/electron/text.js +0 -13
- package/dist/extensions/agent-browser/lib/input-modes.js +0 -10
- package/dist/extensions/agent-browser/lib/orchestration/browser-run/session-artifacts.js +0 -5
- package/dist/extensions/agent-browser/lib/results/artifact-state.js +0 -13
- package/dist/extensions/agent-browser/lib/results.js +0 -14
package/README.md
CHANGED
|
@@ -71,7 +71,7 @@ The result is optimized for agent work:
|
|
|
71
71
|
|
|
72
72
|
| Pain | Native wrapper capability | Proof surface |
|
|
73
73
|
|---|---|---|
|
|
74
|
-
| Agents build fragile shell commands | Exposes `agent_browser` with exact `args`, an optional `semanticAction` shorthand for common `find` flows and native `select`, constrained `job` / `qa` presets, experimental `sourceLookup` / `networkSourceLookup` that compile short workflows to `batch`, top-level `electron` for desktop lifecycle, plus controlled `stdin` and `sessionMode` | `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/input-modes/`, [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md) |
|
|
74
|
+
| Agents build fragile shell commands or repeat browser calls for loops and branches | Exposes `agent_browser` with one-shot sandboxed `script` orchestration, exact `args`, an optional `semanticAction` shorthand for common `find` flows and native `select`, constrained `job` / `qa` presets, experimental `sourceLookup` / `networkSourceLookup` that compile short workflows to `batch`, top-level `electron` for desktop lifecycle, plus controlled `stdin` and `sessionMode` | `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/input-modes/`, [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md) |
|
|
75
75
|
| Page snapshots are too large or viewport-blind | Shows compact, main-content-first summaries, surfaces an `Omitted high-value controls` section (plus `details.data.highValueControlRefIds`) when dense pages or desktop host screens hide editables, named surfaces/tabs, primary action buttons, and high-signal named links such as repository results from the trimmed ref lists, supports wrapper-side `snapshot -i --search <text>` / `--filter role=<role>` to trim dense pages while preserving full `details.refSnapshot`, supports `snapshot --viewport` for scroll/viewport metadata, supports `snapshot --diff` for quick ref-map deltas versus the prior tracked snapshot, and stores full raw output in spill files when needed | `extensions/agent-browser/lib/results/snapshot.ts`, `extensions/agent-browser/lib/orchestration/browser-run/prepare.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.extension-validation.test.ts` |
|
|
76
76
|
| Screenshots/downloads get lost in text | Normalizes artifact paths, creates missing parent directories, saves simple loopback anchor downloads to the requested path when possible, and reports existence, size, cwd, session, and repair status | [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#download-screenshot-and-pdf-files) |
|
|
77
77
|
| Profile restores and tab drift confuse agents | Tracks managed sessions, keeps every upstream helper probe on the same idle-timeout launch configuration so the background browser is not restarted between a snapshot and action, re-selects target tabs after observed drift, refreshes the active target after `tab close`, rehydrates branch-backed session state on Pi session-tree changes, and pins later commands only for sessions with drift/restored-session risk | generated tab-recovery notes below; `test/agent-browser.extension-tab-recovery.test.ts` (drift and about:blank recovery), `test/agent-browser.extension-tabs.test.ts` (post-close target), `test/agent-browser.extension-ref-guards.test.ts` (snapshot/action environment and session-tree rehydration), `test/agent-browser.resume-state.test.ts` (persisted session / resume planning) |
|
|
@@ -80,16 +80,16 @@ The result is optimized for agent work:
|
|
|
80
80
|
| Stale `@eN` refs fail mysteriously | Records per-session `details.refSnapshot`, rejects mismatched URLs / unknown refs / unsafe `batch` stdin ordering before spawn, adds recovery guidance to rerun `snapshot -i` or use stable `find` locators | `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/session-page-state.ts`, `test/agent-browser.session-page-state.test.ts`, `test/agent-browser.results.test.ts`, `test/agent-browser.extension-ref-guards.test.ts`, `test/agent-browser.extension-semantic-recovery.test.ts` |
|
|
81
81
|
| Agents need stable success/failure buckets | Exposes bounded `resultCategory`, `successCategory`, and `failureCategory` on tool `details` for branching without parsing prose; a `tool_result` hook also aligns real Pi `isError` semantics, naming `Pi tool isError: true` in prose output while preserving parseable caller-requested `--json` output | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), `extensions/agent-browser/lib/results/categories.ts`, `extensions/agent-browser/index.ts`, `extensions/agent-browser/lib/pi-tool-rendering.ts`, `test/agent-browser.results.test.ts`, `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.pi-pipeline.test.ts` |
|
|
82
82
|
| Clicks can report success without the page receiving the event | Top-level non-Electron direct `click` calls on `xpath=` targets or role-gated current `@e…` refs (`button`, `checkbox`, `menuitem`, `radio`, `switch`, `tab`) install a bounded target-specific DOM-event probe; eligible `@e…` refs use the latest snapshot role/name metadata, and duplicate-name refs use snapshot-order `duplicateIndex` rather than requiring a unique name. If upstream reports success but no trusted event reaches the resolved target, the wrapper fails the tool, exposes `details.clickDispatch`, and suggests explicit retry/inspect next actions (no in-page replay), including a nested-scroll `scrollintoview` action when the probe sees the target outside a scroll container or viewport. Unresolved locator clicks such as raw `find … click` are left upstream-owned to avoid false failures for frame-scoped targets. Other click results still expose `details.pageChangeSummary`, and unchanged-URL clicks can surface evidence-backed `details.overlayBlockers` candidates. | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), `extensions/agent-browser/lib/orchestration/browser-run/click-dispatch.ts`, `extensions/agent-browser/lib/results/presentation/navigation.ts`, `test/agent-browser.presentation.test.ts`, `test/agent-browser.extension-click-dispatch.test.ts` |
|
|
83
|
-
| Dashboard scroll commands can look successful while nothing moves | Handles standard `scroll <dir> [px]` against the document first (including pages whose smooth-scroll CSS defeats upstream wheel timing), falls back upstream when the document cannot move, and samples viewport/containers around the fallback; unchanged positions
|
|
84
|
-
| Dropdown/combobox clicks can focus or hit native option box-model errors | Adds first-class `select <selector> <value...>` paths through raw `args`, `
|
|
85
|
-
| Recording workflows fail late when `ffmpeg` is missing | After successful `record start` / `record restart`, warns when `ffmpeg` is not on `
|
|
86
|
-
|
|
|
83
|
+
| Dashboard scroll commands can look successful while nothing moves | Handles standard `scroll <dir> [px]` against the document first (including pages whose smooth-scroll CSS defeats upstream wheel timing), falls back upstream when the document cannot move, and samples viewport/containers around the fallback; unchanged positions fail as `upstream-error` with `details.scrollNoop`, visible recovery guidance, and exact snapshot/screenshot checks | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#core-page-and-element-commands), `test/agent-browser.extension-validation.test.ts` |
|
|
84
|
+
| Dropdown/combobox clicks can focus or hit native option box-model errors | Adds first-class `select <selector> <value...>` paths through raw `args`, `job`, and `semanticAction`; semantic role/name or label select resolves exactly one current visible combobox/listbox ref before action. Custom combobox clicks still detect focused controls with explicit `aria-expanded` state but no visible options and return `details.comboboxFocus` plus exact recovery `nextActions` | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#core-page-and-element-commands), `extensions/agent-browser/lib/input-modes/semantic-action.ts`, `test/agent-browser.extension-input-modes.test.ts`, `test/agent-browser.extension-validation.test.ts` |
|
|
85
|
+
| Recording workflows fail late when `ffmpeg` is missing or report stale lifecycle state | After successful `record start` / `record restart`, reports `successCategory: "artifact-pending"`, returns an exact `stop-pending-recording` action, warns when `ffmpeg` is unavailable, and tells agents that `record start` switches to a fresh active page whose in-page state does not carry over while invalidating prior page-scoped `@e…` refs on every executed start attempt (even a failed already-active one) and on URL-bearing `record restart` (stale-ref until a fresh snapshot); an unbounded transcript-backed namespace/session index reserves active destinations across aliases, serializes artifact lifecycle and explicit wait/output writes, persists cross-branch close tombstones, retires every successful close path (including every matching namespace owner for `close --all`), rejects missing/stale restart output, coalesces terminal batch state, keeps only the newest pending path per identity, rejects recording starts after a nested close, folds Unicode path aliases, and retains exact cleanup actions with visible guidance on any later same-session failure | [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#details), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#diff-debug-and-streaming), `test/agent-browser.extension-validation.test.ts`, `test/agent-browser.presentation-artifacts-batch.test.ts` |
|
|
86
|
+
| Upstream CLI drift can silently invalidate wrapper behavior | Publishes a repo-readable command reference, verifies it against the target, and probes browser-backed calls once per cwd/PATH so anything other than exact `agent-browser 0.34.0` fails before browser launch with installed/expected version evidence | `npm run verify` |
|
|
87
87
|
| Desktop Electron apps need discovery, CDP attach, and safe teardown | Top-level `electron` runs host `list` / isolated `launch` (temp profile, OS-chosen debug port) / `status` / `probe` / `cleanup`, merges `launchId` plus managed `sessionName`, supports `handoff` `snapshot` / `tabs` / `connect`, and surfaces mismatch and post-command health guidance; wrapper cleanup applies only to launches it created | `extensions/agent-browser/lib/electron/discovery.ts`, `launch.ts`, `cleanup.ts`, [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#electron), [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#electron-desktop-apps) |
|
|
88
88
|
| Agents need bundled `skills` text and local setup/status commands without touching the live session | Treats `skills list`, `skills get …`, `skills path …`, local auth profile management (`auth save/list/show/delete/remove`), `profiles`, `dashboard`, `device list`, `doctor`, `install`, `upgrade`, `session list` (with wrapper-managed rows hidden), `session id`, `session info`, `plugin add/list/show/run`, `mcp --help`, and caller-owned saved-state inspection/targeted maintenance (`state list/show/rename` or named clear) as sessionless reads/actions: no implicit managed `--session` under default `sessionMode: "auto"`; broad clear/clean and managed-state targets are rejected before spawn (same session-ownership goal as plain-text `--help` / `--version`), while bare `mcp` server calls are blocked and provider/browser-backed workflows stay thin passthroughs that require upstream setup and credentials | [`docs/COMMAND_REFERENCE.md`](docs/COMMAND_REFERENCE.md#built-in-skills), `extensions/agent-browser/lib/command-policy.ts`, `extensions/agent-browser/lib/runtime.ts` |
|
|
89
89
|
|
|
90
90
|
## Fastest way to try it
|
|
91
91
|
|
|
92
|
-
Use Pi 0.
|
|
92
|
+
Use Pi 0.84.0 or newer. This package keeps optional Pi core imports as wildcard `peerDependencies` because Pi package docs require the host Pi install to provide those packages, pins its direct Pi validation dependencies to 0.84.0, and makes older hosts a setup failure through `pi-agent-browser-doctor`. Version 0.3.0 intentionally provides no compatibility shims for older Pi releases.
|
|
93
93
|
|
|
94
94
|
Install upstream `agent-browser` first and make sure it is on `PATH`:
|
|
95
95
|
|
|
@@ -103,9 +103,28 @@ Optional external tools unlock the full command surface:
|
|
|
103
103
|
| `agent-browser` | All browser automation through this extension | See upstream install docs |
|
|
104
104
|
| `ffmpeg` | `record stop` WebM encoding after `record start` / `record restart` | `brew install ffmpeg` or `brew install ffmpeg-full` |
|
|
105
105
|
|
|
106
|
-
Keep both binaries on `PATH`. `record start` can begin without a file on disk, but `record stop` needs `ffmpeg` to encode the WebM.
|
|
106
|
+
Keep both binaries on `PATH`. This package targets exactly `agent-browser 0.34.0`; browser-backed calls fail fast on a different installed version while local inspection/setup commands remain available for diagnosis. `record start` can begin without a file on disk, but `record stop` needs `ffmpeg` to encode the WebM.
|
|
107
107
|
|
|
108
|
-
|
|
108
|
+
### Android / Termux
|
|
109
|
+
|
|
110
|
+
Android support currently uses Termux's system Chromium rather than Chrome for Testing. Upstream issue [vercel-labs/agent-browser#1587](https://github.com/vercel-labs/agent-browser/issues/1587) tracks native Android packaging; until upstream ships an Android launcher, install the packaged Linux-musl arm64 binary without lifecycle scripts and point the global command at it:
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
pkg install tur-repo x11-repo
|
|
114
|
+
pkg install chromium ffmpeg which
|
|
115
|
+
npm install -g --ignore-scripts agent-browser@0.34.0
|
|
116
|
+
ln -sfn "$(npm root -g)/agent-browser/bin/agent-browser-linux-musl-arm64" \
|
|
117
|
+
"$(npm prefix -g)/bin/agent-browser"
|
|
118
|
+
ln -sfn "$PREFIX/lib/chromium/chromium-launcher.sh" "$PREFIX/bin/chromium"
|
|
119
|
+
agent-browser --version
|
|
120
|
+
which chromium
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
The `which` package and launcher symlink satisfy upstream's existing Linux system-browser discovery even when tests or Pi sessions isolate `HOME`. This also lets isolated `script` calls launch without forbidden `--executable-path` overrides.
|
|
124
|
+
|
|
125
|
+
Reapply the musl command symlink after reinstalling or upgrading upstream until #1587 is resolved. The wrapper uses Termux-private socket/policy storage, compact 80-bit managed identities so ordinary namespaces and fresh rotations fit the Unix socket-path limit, Termux's `ps`, and Android app-sandbox trust rules automatically. Headless browser flows, managed restore, namespaced sessions, `script`, `qa`, `job`, screenshots, and recording are locally validated; Electron desktop discovery/lifecycle is not applicable to Android apps. Android remains outside the release-blocking Crabbox macOS/Ubuntu/native-Windows matrix until a repeatable Android provider target is added.
|
|
126
|
+
|
|
127
|
+
The native tool also gives agents absolute installed-package doc paths in its compact runtime guidance. Raw `args` are the 1:1 upstream CLI coverage path for the targeted `agent-browser` release; `script` adds bounded one-shot orchestration, while typed modes such as `semanticAction`, `job`, `qa`, source lookups, and Electron lifecycle helpers are reliability shorthands layered on top. Agents should read `README.md` for setup/dependencies, `docs/COMMAND_REFERENCE.md` for targeted command workflows, and `docs/TOOL_CONTRACT.md` for result/detail contracts only when deeper guidance is needed.
|
|
109
128
|
|
|
110
129
|
Then install this Pi package:
|
|
111
130
|
|
|
@@ -125,7 +144,7 @@ For a one-off trial that does not touch your configured Pi extensions:
|
|
|
125
144
|
pi --no-extensions -e npm:pi-agent-browser-native
|
|
126
145
|
```
|
|
127
146
|
|
|
128
|
-
Pi 0.
|
|
147
|
+
Pi 0.84.0+ may ask whether to trust the current project before loading project-local instructions, settings, or resources. This extension treats its own project-local package config as developer-trusted by default; use `--no-approve` when you intentionally want Pi and this extension to ignore project-local inputs for that run.
|
|
129
148
|
|
|
130
149
|
For a specific published version:
|
|
131
150
|
|
|
@@ -166,6 +185,8 @@ The doctor checks:
|
|
|
166
185
|
|
|
167
186
|
It does **not** edit Pi settings and does **not** run upstream `agent-browser doctor --fix`.
|
|
168
187
|
|
|
188
|
+
Pi hosts that run as uid 0 should set `PI_AGENT_BROWSER_SOCKET_DIR` to a short absolute directory under private root-owned ancestry, create it with mode `0700`, and keep it owned by the Pi user. The extension validates that directory and forwards it as upstream `AGENT_BROWSER_SOCKET_DIR`; ambient upstream socket overrides remain ignored.
|
|
189
|
+
|
|
169
190
|
## Optional package config and web search
|
|
170
191
|
|
|
171
192
|
`pi-agent-browser-native` also reads package-owned config under Pi-scoped paths:
|
|
@@ -234,7 +255,7 @@ printf '%s' "$EXA_API_KEY" | npm exec --yes --package pi-agent-browser-native@la
|
|
|
234
255
|
npm exec --yes --package pi-agent-browser-native@latest -- pi-agent-browser-config web-search set-command "op read 'op://Private/Brave Search/API Key'" --provider brave --global
|
|
235
256
|
```
|
|
236
257
|
|
|
237
|
-
Config merges in this order: global → project → `PI_AGENT_BROWSER_CONFIG` override. Under Pi 0.
|
|
258
|
+
Config merges in this order: global → project → `PI_AGENT_BROWSER_CONFIG` override. Under Pi 0.84.0+, the globally installed or CLI-loaded extension still loads project-local `.pi/config/pi-agent-browser-native/config.json` when Pi trust allows that project layer; it skips that project layer when Pi reports the project is untrusted or when Pi is launched with `--no-approve`. `webSearch.enabled` is evaluated after the loaded layers merge. Use `web-search disable --global` for a user default, `web-search disable --project` for one repo, and a `PI_AGENT_BROWSER_CONFIG` override with `{ "webSearch": { "enabled": false } }` when web search must stay off even if project config exists. Loaded config may use plaintext, custom environment aliases, interpolation literals, malformed-or-late-bound `$` values, and `!command` credential sources; the resolved secret is passed to the provider request while tool content, details, status output, and docs examples stay redacted. `web-search set-key`, `set-command`, and `clear` require `--provider`; `set-env` infers Exa/Brave from `EXA_API_KEY` or `BRAVE_API_KEY` unless you pass `--provider`.
|
|
238
259
|
|
|
239
260
|
For Exa, the tool defaults to `searchType: "auto"` with `contents.highlights: true`. Agents may pass `searchType` (`fast`, `instant`, `deep-lite`, `deep`, or `deep-reasoning`) only when the task needs that latency/depth tradeoff; structured output schemas are intentionally not exposed yet.
|
|
240
261
|
|
|
@@ -268,6 +289,8 @@ Watch a browser window during a demo or QA run by adding upstream's global `--he
|
|
|
268
289
|
{ "args": ["screenshot", "/tmp/agent-browser-headed-check.png"] }
|
|
269
290
|
```
|
|
270
291
|
|
|
292
|
+
For wrapper-owned headed launches, the extension disables upstream 0.33.2 periodic restore autosave by default because its multi-origin collector opens visible temporary tabs and can delay daemon policy inspection. The extension records the effective launch-time interval and reapplies it to every helper and follow-up subprocess, including still-owned off-current sessions (also after failed replacement cleanup), Electron cleanup closes, and reload/resume, so the receiving daemon does not see changing configuration. Native `close` still saves, but upstream exempts headed browsers from idle shutdown, so closing the window by hand can lose newer state. Set `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` before launch when periodic preservation matters; changing it on a running wrapper-owned headed daemon is rejected until you close that session and launch fresh.
|
|
293
|
+
|
|
271
294
|
Render a WebGPU page by enabling upstream's WebGPU launch preset on a fresh local browser:
|
|
272
295
|
|
|
273
296
|
```json
|
|
@@ -321,7 +344,7 @@ Evaluate page JavaScript through stdin. Put the script in the top-level `stdin`
|
|
|
321
344
|
{ "args": ["eval", "--stdin"], "stdin": "({ title: document.title, url: location.href })", "outputPath": "logs/page-state.json" }
|
|
322
345
|
```
|
|
323
346
|
|
|
324
|
-
Use `outputPath` when `eval`, `get`, `snapshot`, or another extraction should be saved as a durable workspace file. The wrapper writes `details.data` when present, otherwise the model-facing text content, and returns `details.outputFile` with the saved path and byte count. Explicit upstream `--json` content stays parseable; in that case the save notice lives only in `details.outputFile`.
|
|
347
|
+
Use `outputPath` when `eval`, `get`, `snapshot`, or another extraction should be saved as a durable workspace file. Keep it distinct from screenshot, download, recording, and other browser artifact destinations; preflight rejects known same-call aliases before browser activity, and the result writer preserves the browser artifact if an alias becomes apparent only afterward. The wrapper writes `details.data` when present, otherwise the model-facing text content, and returns `details.outputFile` with the saved path and byte count. Explicit upstream `--json` content stays parseable; in that case the save notice lives only in `details.outputFile`.
|
|
325
348
|
|
|
326
349
|
Extract several known refs or selectors in one `batch` call instead of many serial getter calls:
|
|
327
350
|
|
|
@@ -341,6 +364,25 @@ Download a file from a known link or control:
|
|
|
341
364
|
{ "args": ["download", "@e5", "/tmp/report.pdf"] }
|
|
342
365
|
```
|
|
343
366
|
|
|
367
|
+
### One-shot code mode (`script`)
|
|
368
|
+
|
|
369
|
+
Use top-level `script` when the browser work needs a loop, a conditional page branch, or multi-page aggregation that would otherwise require several `agent_browser` calls. The source is an async JavaScript body with only two task-specific globals:
|
|
370
|
+
|
|
371
|
+
- `await browser({ args, stdin?, timeoutMs? })` runs one ordinary native browser call through the same validation, policy, redaction, presentation, artifact, and timeout pipeline as top-level `args`. It resolves to `{ ok, data, details?, error?, failureCategory?, nextActions?, resultCategory, successCategory?, summary, text }`; check `ok` before consuming `data`. Script-visible browser `nextActions` keep only policy-compatible calls with the wrapper-owned isolated identity prefix removed, so their `params` can be passed back to `browser()`.
|
|
372
|
+
- `emit(value)` adds a JSON-compatible output value. One emission is returned directly; multiple emissions are returned as an array. With no emission, the async body’s return value is used; when it returns nothing, `details.data` is omitted.
|
|
373
|
+
|
|
374
|
+
```json
|
|
375
|
+
{
|
|
376
|
+
"script": "const rows = [];\nfor (const page of [1, 2]) {\n const opened = await browser({ args: [\"open\", `https://news.ycombinator.com/news?p=${page}`] });\n if (!opened.ok) throw new Error(opened.error);\n const extracted = await browser({ args: [\"eval\", \"--stdin\"], stdin: \"({ hasBanner: Boolean(document.querySelector('[role=dialog]')), rows: [...document.querySelectorAll('tr.athing')].slice(0, 30).map(row => ({ id: row.id, title: row.querySelector('.titleline > a')?.textContent ?? '' })) })\" });\n if (!extracted.ok) throw new Error(extracted.error);\n if (extracted.data.result.hasBanner) {\n const dismissed = await browser({ args: [\"click\", \"[role=dialog] button\"] });\n if (!dismissed.ok) throw new Error(dismissed.error);\n }\n rows.push(...extracted.data.result.rows);\n}\nemit(rows);"
|
|
377
|
+
}
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
This mode is intentionally one-shot, not a reusable recipe runtime. Each invocation gets a unique non-profile browser session, never touches the implicit conversation session, serializes inner calls, and closes the isolated session in `finally`. It rejects caller `--session` / `--namespace`, browser lifecycle and attachment commands, persistent launch/profile/restore controls, nested `batch`, local/sessionless commands, and every other top-level input mode. Every script-owned helper and cleanup subprocess also clears ambient `AGENT_BROWSER_*` and standard proxy variables before the wrapper reapplies its own isolated-session controls, so shell defaults cannot attach, restore, or select a profile behind the script’s back. The sandbox has no imports, `require`, process, filesystem, network, timers, dynamic code generation, or host object/function references.
|
|
381
|
+
|
|
382
|
+
Limits are fixed: 25 attempted `browser()` calls, 64 KiB source, 64 KiB final emitted JSON, a 120-second default timeout, and a 300-second hard timeout ceiling. Final data is redacted, serialized compactly, and checked again before presentation; unsafe depth or post-redaction growth becomes a structured validation failure rather than unbounded prose. One approved top-level `agent_browser` call can authorize all 25 inner calls, so inspect the visible script source before approving it: the collapsed Pi tool row shows a bounded terminal-safe preview with source line breaks marked as `↵`, and expanding that row shows the full terminal-safe source with JavaScript line terminators preserved as visible newlines and removed controls marked visibly. The extension rehydrates only wrapper-verified parse-valid compact-result spills before returning inner `data`; ordinary result redaction still applies. Inner `summary` and `text` are bounded, and a complete envelope that still exceeds the IPC message cap becomes a handleable `upstream-error` browser result instead of breaking the sandbox bridge. Pi session persistence is required so the wrapper can append a model-invisible cleanup lease before the first browser launch and retry a failed close after restart. A rejected inner policy/validation call fails the top-level result even when source handles its returned envelope; an uncaught source exception returns `failureCategory: "script-error"`; a failed cleanup overrides any script outcome with `failureCategory: "cleanup-failed"`, `details.scriptSession.closeCommandArgs`, and an exact `close-script-session-after-cleanup-failure` next action. Compact prose confirms a successful isolated-session close after browser-bearing runs. Pi branch changes, quit, and reload abort active scripts, wait for isolated-session cleanup, and reap the sandbox child before restoring branch-visible state.
|
|
383
|
+
|
|
384
|
+
Use normal `args`, `job`, or `qa` for linear work. Use ordinary profile/attached flows for authenticated browser state. Do not store code mode source under a name or treat it as shared workflow configuration.
|
|
385
|
+
|
|
344
386
|
### Locator shorthand (`semanticAction`)
|
|
345
387
|
|
|
346
388
|
For supported upstream `find` flows, direct selector/ref `click` / `check` / `fill`, and native dropdown selection you can omit hand-built `args` and pass a top-level `semanticAction` object instead. The wrapper compiles locator actions to the same `find` argv upstream already understands, direct selector/ref actions to matching upstream commands, or `action: "select"` to upstream `select <selector> <value...>`; compiled argv is echoed as `details.compiledSemanticAction` when the unified result includes that field. Full field rules live in [`docs/TOOL_CONTRACT.md#semanticaction`](docs/TOOL_CONTRACT.md#semanticaction).
|
|
@@ -357,23 +399,24 @@ For supported upstream `find` flows, direct selector/ref `click` / `check` / `fi
|
|
|
357
399
|
|
|
358
400
|
Typical pitfalls:
|
|
359
401
|
|
|
360
|
-
- Supply **exactly one** of `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron` per call (not more, not none). Prefer `args` for routine browse; `semanticAction` for stable locators; `job`/`qa` for multi-step checks; `electron` for desktop apps; treat `sourceLookup` / `networkSourceLookup` as experimental candidates-only.
|
|
402
|
+
- Supply **exactly one** of `script`, `args`, `semanticAction`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron` per call (not more, not none). Prefer `script` only for one-shot loops/branches/aggregation, `args` for routine browse; `semanticAction` for stable locators; `job`/`qa` for multi-step checks; `electron` for desktop apps; treat `sourceLookup` / `networkSourceLookup` as experimental candidates-only.
|
|
361
403
|
- Do not pass `--json` in `args`; the wrapper injects it automatically.
|
|
362
404
|
- `semanticAction` and `job` are **not** valid inside `batch` stdin; batch steps stay upstream argv string arrays (spell a `find` step as tokens there if you need it in a batch).
|
|
363
405
|
- Commands or locators outside the supported shorthand still require explicit `args`. Common page getters are grouped under `get`: use `get title`, `get url`, or `get text <selector>` rather than shortcut commands such as `title` or `url`; unknown getter shortcuts can return read-only `details.nextActions` like `use-get-title`.
|
|
364
406
|
- For `locator: "role"`, pass either `value: "button"` or `role: "button"`; if both are present they must match.
|
|
365
407
|
- Use `semanticAction.session` to target a named upstream browser session; the wrapper prepends `--session <name>` before the compiled `find` or `select` argv and keeps that prefix on retry/candidate actions. In active sessions, role/name click/check/fill shorthands may resolve through the current `snapshot -i` refs before execution so hidden duplicate matches do not steal the action; fill only resolves when the current snapshot has one exact editable ref match. `details.effectiveArgs` shows the exact executed argv.
|
|
366
|
-
- Do not reuse `@e…` refs across navigation. The wrapper records the latest snapshot refs per session and fails
|
|
408
|
+
- Do not reuse `@e…` refs across navigation or in-place rerenders. The wrapper records the latest snapshot refs per session and fails stale/recycled getter and mutation refs, including batched getters, before upstream can silently read or hit a different current-page element; use the session-aware `refresh-interactive-refs` next action.
|
|
367
409
|
- If upstream classifies the failure as `stale-ref` and `details.compiledSemanticAction` is present for a compiled `find` action, `details.nextActions` may list `retry-semantic-action-after-stale-ref` after `refresh-interactive-refs`, carrying the same compiled `find` argv so you can retry the locator-stable target once it is safe to do so. `select` calls that used stale `@refs` only get refresh guidance; use a fresh snapshot or stable selector before retrying (contract in [`docs/TOOL_CONTRACT.md#semanticaction`](docs/TOOL_CONTRACT.md#semanticaction)).
|
|
368
410
|
- If the failure is `selector-not-found`, the wrapper may take one fresh snapshot and add `Current snapshot ref fallback` when that snapshot has exact visible role/name matches for the failed `find` / `semanticAction` target. Non-fill targets can include direct `try-current-visible-ref*` next actions, and semantic click misses can still add bounded `Agent-browser candidate fallbacks` such as `button`/`link` role retries for `text` clicks. `semanticAction` does not expose `uncheck` while upstream `find ... uncheck` is not runtime-supported; use raw `args: ["uncheck", <selector-or-ref>]` after a stable selector or fresh snapshot ref. For semantic `fill` misses on desktop or host-controlled rich inputs, prefer `details.richInputRecovery`: refresh refs, choose the current editable `@ref`, focus or click it, then use `keyboard inserttext` or `keyboard type` with the intended text. Direct contenteditable fills are verified with `get text` when snapshot metadata proves the target is contenteditable; if replacement did not happen, `details.fillVerification` warns before any submit step. Those recovery nextActions do not copy the fill text and do not press `Enter` or submit; only submit when the user flow explicitly calls for it (same contract link).
|
|
369
411
|
- A successful upstream `click` is not proof that the web app handled the event or changed state. For top-level non-Electron direct clicks on `xpath=` targets and eligible current `@e…` refs, the wrapper may fail the tool with `details.clickDispatch` and a `Click dispatch diagnostic` line when upstream reported success but no trusted DOM event reached the resolved target. Raw `find … click` locator calls are not probed because the wrapper has no concrete element before upstream resolves the locator, and document-level probes can falsely fail frame-scoped clicks. `@e…` ref click probes are limited to current snapshot refs with accessible role `button`, `checkbox`, `menuitem`, `radio`, `switch`, or `tab`, using duplicate-name snapshot order when needed. Use the suggested `inspect-click-dispatch-miss` / `retry-click-after-dispatch-miss` next actions instead of assuming the click mutated the page; when `details.clickDispatch.scrollContainer` is present, use `scroll-target-into-view-after-dispatch-miss` first. When the task depends on a mutation, follow `inspect-after-mutation` / `pageChangeSummary` evidence with a wait, URL/text check, or fresh snapshot before trusting the result; if the target still did not change, retry with a current visible ref or stable selector and report the workflow issue instead of silently continuing. For static local fixtures where the user only needs to exercise app code, an explicit `eval --stdin` programmatic click such as `document.querySelector("#demo").click()` can be a diagnostic workaround, but treat it as an untrusted scripted activation rather than proof a real user click works, and never use it to bypass user instructions. Respect explicit user stop boundaries yourself: if the user says to stop before order/post/purchase/submit, gather evidence on that page and do not click the final action. The wrapper does not parse broad prompt text into business-intent action blocks; `details.promptGuard` is reserved for concrete artifact-before-close checks.
|
|
412
|
+
- A successful upstream `click` can deliver no input at all on some sites. Reproduced against `https://www.saucedemo.com/` with upstream `agent-browser` 0.34.0 and no wrapper involved: when every CLI invocation in the flow is spaced ~0.6s apart, the React add-to-cart click reports success while **zero** DOM events reach a capture-phase listener and the cart badge never updates (0/5 runs succeeded, versus 4/5 with no spacing). A single idle gap is harmless (5/5); only sustained spacing across the flow triggers it. In that state a scripted `document.querySelector(...).click()` still works and updates the badge, so the page and its handlers are fine and the input never arrives; retries, `scrollintoview`, headed mode, and re-navigation do not recover it. This looks site-specific — React TodoMVC and react.dev were unaffected under the same pacing and command count. Wrapper calls do more per-command work than raw CLI calls, so they sit in the slow regime more often. Running the interaction steps as one `batch` keeps them inside a single upstream process and succeeded 5/5; prefer `batch` for consecutive real-app click flows, and treat a click whose `pageChangeSummary`/`navigationSummary` shows no change as unproven. When a `wait --url` or `assertUrl` times out after a suspected missed click, use the `fresh-session-after-url-wait-timeout` next action (`sessionMode: "fresh"` + `open about:blank`): replace about:blank with the target URL and replay the flow as one batch in a fresh session instead of retrying the wait.
|
|
370
413
|
- A successful `snapshot -i` can surface `Possible overlay blockers` immediately when refs already contain strong dialog/alertdialog evidence plus close/dismiss controls. If a **top-level** `@e…`/`ref=` click succeeds (unified command `click`, not a `batch` step), upstream reports `data.clicked`, and `details.navigationSummary.url` stays on the same tab URL under the same normalization as ref preflight (fragment-insensitive), the wrapper may take one extra `snapshot -i` and add `Possible overlay blockers` with `details.overlayBlockers` (`candidates`, `summary`, optional `snapshot` refresh for refs) plus session-aware `inspect-overlay-state` / bounded `try-overlay-blocker-candidate-*` next actions when that snapshot shows strong modal context (`dialog` / `alertdialog`) and close/dismiss-like controls. Page-wide words like privacy, sign in, or banner alone do not trigger this diagnostic. The unchanged-URL check compares the prior pinned tab target with `details.navigationSummary.url`; CSS selector clicks do not run this overlay probe. Also skipped when tab correction or about-blank recovery already ran on that result.
|
|
371
414
|
- If `get text <selector>` reads a non-ref, non-simple-id CSS selector with multiple matches or a hidden first match while visible matches exist, including successful `batch` steps, the wrapper may add `Selector text visibility warning`, `details.selectorTextVisibility` (plus `selectorTextVisibilityAll` for multiple batched warnings), and `inspect-visible-text-candidates` next actions; the warning names the matching `details.nextActions` id. Prefer a visible `@ref`, a scoped selector, or a targeted `eval --stdin` over hidden tab content.
|
|
372
415
|
- In wrapper-tracked attached Electron sessions, broad selectors such as `body`, `html`, `main`, or `[role=application]` may read the whole app shell. The wrapper may add `Broad Electron get text selector warning`, `details.electronGetTextScopeWarning`, and `snapshot-for-electron-text-scope`; ordinary browser pages do not qualify without Electron launch provenance, and local `file://` page follow-ups are blocked before this diagnostic. Prefer `snapshot -i`, a current `@ref`, or a narrower panel selector.
|
|
373
416
|
|
|
374
417
|
### Constrained browser jobs
|
|
375
418
|
|
|
376
|
-
For short repeatable workflows, pass a top-level `job` instead of hand-writing `batch` stdin. Keep dynamic app jobs short around navigation, click, and rerender boundaries; avoid packing a whole checkout into one job. The wrapper only supports constrained steps (`open`, `click`, `fill`, `type`, `select`, `wait`, `assertText`, `assertUrl`, `waitForDownload`, `snapshot`, and `screenshot`), compiles them to existing upstream `batch` commands, and echoes the compiled commands as `details.compiledJob` for auditability. `open` steps can include `loadState` (`domcontentloaded`, `load`, or `networkidle`) to insert a readiness wait before the next step. `click` and `fill` steps can use either CSS `selector` or semantic locator fields (`locator`, `role`/`value`, optional `name`) so a job can express flows like role/name search without brittle selectors. `type` can use `selector`, `text`, optional `delayMs` for per-character pacing, and optional `press` for a final key such as `Enter`; paced type compiles to existing `focus`, `keyboard type`, `wait`, and `press` batch rows, is capped at 200 characters per delayed step, and compacts model-visible batch text while full rows remain in `details.batchSteps`. The same compile path backs top-level `qa`, so long `qa` runs surface the same timeout evidence shape. If a long `job`, `qa`, or `batch` hits the wrapper watchdog, `details.timeoutPartialProgress` may recover per-step status (`completed`, `failed`, `pending`, or `unknown`), current page URL plus a title only after a non-file URL is verified, declared artifact paths that already exist on disk, and either a `retry-timeout-step` next action for the first incomplete read-only or idempotent step or `inspect-current-page-after-timeout` when the first incomplete step may be mutating and needs state inspection before a shorter follow-up flow (see [`docs/TOOL_CONTRACT.md#details`](docs/TOOL_CONTRACT.md#details)). There is no separate catalog of reusable named browser recipes above `job`, `qa`, and raw `batch`; see [`docs/ARCHITECTURE.md#no-reusable-recipe-layer-yet`](docs/ARCHITECTURE.md#no-reusable-recipe-layer-yet) for the closed `RQ-0068` decision and when to revisit it.
|
|
419
|
+
For short repeatable workflows, pass a top-level `job` instead of hand-writing `batch` stdin. Keep dynamic app jobs short around navigation, click, and rerender boundaries; avoid packing a whole checkout into one job. The wrapper only supports constrained steps (`open`, `click`, `fill`, `type`, `select`, `wait`, `assertText`, `assertUrl`, `waitForDownload`, `snapshot`, and `screenshot`), compiles them to existing upstream `batch` commands, and echoes the compiled commands as `details.compiledJob` for auditability. `open` steps can include `loadState` (`domcontentloaded`, `load`, or `networkidle`) to insert a readiness wait before the next step. `click` and `fill` steps can use either CSS `selector` or semantic locator fields (`locator`, `role`/`value`, optional `name`) so a job can express flows like role/name search without brittle selectors. `type` can use `selector`, `text`, optional `delayMs` for per-character pacing, and optional `press` for a final key such as `Enter`; paced type compiles to existing `focus`, `keyboard type`, `wait`, and `press` batch rows, is capped at 200 characters per delayed step, and compacts model-visible batch text while full rows remain in `details.batchSteps`. The same compile path backs top-level `qa`, so long `qa` runs surface the same timeout evidence shape. If a long `job`, `qa`, or `batch` hits the wrapper watchdog, `details.timeoutPartialProgress` may recover per-step status (`completed`, `failed`, `pending`, or `unknown`), current page URL plus a title only after a non-file URL is verified, declared artifact paths that already exist on disk, and either a `retry-timeout-step` next action for the first incomplete read-only or idempotent step or `inspect-current-page-after-timeout` when the first incomplete step may be mutating and needs state inspection before a shorter follow-up flow (see [`docs/TOOL_CONTRACT.md#details`](docs/TOOL_CONTRACT.md#details)). There is no separate catalog of reusable named browser recipes above one-shot ad hoc `script`, `job`, `qa`, and raw `batch`; `script` has no names, registry, or persistent workflow state; see [`docs/ARCHITECTURE.md#no-reusable-recipe-layer-yet`](docs/ARCHITECTURE.md#no-reusable-recipe-layer-yet) for the closed `RQ-0068` decision and when to revisit it.
|
|
377
420
|
|
|
378
421
|
**Navigation inside `job` is explicit.** A successful `click` does not prove the next page loaded; add `assertUrl` and/or `assertText` after navigation-prone clicks (forms, checkout, tabs, submit buttons) before screenshots or steps that assume the new page. `assertUrl` accepts exact URLs and `*` / `**` glob-style patterns and now compiles directly to upstream `wait --url` for both forms.
|
|
379
422
|
|
|
@@ -406,7 +449,7 @@ For short repeatable workflows, pass a top-level `job` instead of hand-writing `
|
|
|
406
449
|
|
|
407
450
|
On app pages that expose a native dropdown, add a `select` step such as `{ "action": "select", "selector": "#flavor", "value": "chocolate" }` before the assertion that depends on it. On locator-friendly pages, use semantic job steps such as `{ "action": "fill", "locator": "role", "role": "searchbox", "name": "Search", "text": "agent browser" }` and `{ "action": "click", "locator": "role", "role": "button", "name": "Search" }`.
|
|
408
451
|
|
|
409
|
-
Use raw `args`/`stdin` when you need full upstream `batch` power, custom flags, or commands outside the constrained job schema. Do not pass `stdin` with `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`;
|
|
452
|
+
Use raw `args`/`stdin` when you need full upstream `batch` power, custom flags, or commands outside the constrained job schema. Do not pass top-level `stdin` with `script`, `job`, `qa`, `sourceLookup`, `networkSourceLookup`, or `electron`; script puts inner stdin on `browser({ stdin })`, while the other modes generate or manage their own input.
|
|
410
453
|
|
|
411
454
|
### Electron desktop apps
|
|
412
455
|
|
|
@@ -440,7 +483,7 @@ After either path, use `qa: { "attached": true, ... }` for a current-session smo
|
|
|
440
483
|
|
|
441
484
|
### Lightweight QA preset
|
|
442
485
|
|
|
443
|
-
For a quick smoke/QA pass, use top-level `qa`. It compiles to the same batch path as `job` and uses `batch --bail` so failed readiness/text/selector assertions stop before slower diagnostics can burn the wrapper watchdog. The URL form clears enabled network/console
|
|
486
|
+
For a quick smoke/QA pass, use top-level `qa`. It compiles to the same batch path as `job` and uses `batch --bail` so failed readiness/text/selector assertions stop before slower diagnostics can burn the wrapper watchdog. The URL form clears enabled network/console buffers and snapshots any page-error residue after the unreliable upstream clear, then opens the target URL and gives immediate post-load console/page-error callbacks a bounded 150 ms settle, waits for page readiness, checks optional expected text or selector, inspects fresh network requests, console messages, and page errors when preceding assertions pass, and can capture an evidence screenshot. Successful reset rows are labeled as reset-scoped output. Only page-error residue still present after the clear can be ignored when it remains unchanged; a matching error that reappears after a successful clear still fails the target page. Expected text is checked with bounded visible-text `wait --fn … --timeout 5000` predicates after the requested load state so dense pages can pass on visible headings/copy and missing text becomes crisp QA evidence. The attached form (`qa: { "attached": true }`) runs checks against the current managed session, such as an attached Electron app, rejects `url`, and deliberately preserves existing diagnostics instead of clearing evidence; its diagnostic reads default off so stale buffers do not fail a current-page smoke unless `checkNetwork`, `checkConsole`, or `checkErrors` is explicitly `true`. `loadState` defaults to `"domcontentloaded"`; set it to `"load"` or `"networkidle"` only when the stricter state is useful and the site is not expected to keep background requests alive. For URL-opening QA, `checkNetwork`, `checkConsole`, and `checkErrors` default to true; set one to `false` to skip that diagnostic read. Network failures are classified by likely impact and failed rows are listed first in network previews: actionable document/script/API-style failures still fail QA, while some low-impact browser icon asset misses (for example certain `favicon` or `apple-touch-icon` paths when upstream marks the row failed and resource metadata looks image-like) surface only as warnings instead of failing an otherwise healthy smoke check (`details.qaPreset.warnings`, with human-readable `details.qaPreset.summary` when the preset still passes). Exact predicates live in [`docs/TOOL_CONTRACT.md`](docs/TOOL_CONTRACT.md#qa) and `classifyNetworkRequestFailure` in `extensions/agent-browser/lib/results/network.ts`.
|
|
444
487
|
|
|
445
488
|
```json
|
|
446
489
|
{
|
|
@@ -464,7 +507,7 @@ For local app debugging, `sourceLookup` can gather candidate component/file loca
|
|
|
464
507
|
|
|
465
508
|
This is an experiment, not a guarantee. React hints require a session opened with `--enable react-devtools`, and many builds do not expose useful sourcemap/source metadata; `status: "no-candidates"` is common when nothing matched, and `status: "unsupported"` only when no candidates were found **and** a compiled `react` batch step failed (if DOM or workspace search still produced candidates, you get `candidates-found` instead). For wrapper-tracked packaged Electron apps, a no-candidate result includes `details.sourceLookup.workspaceRoot`, optional `details.sourceLookup.electronContext`, limitations explaining that the scan is limited to the Pi cwd and does not unpack app bundles/`app.asar`, plus Electron snapshot/probe/tab next actions when a launch is known.
|
|
466
509
|
|
|
467
|
-
`networkSourceLookup` is the matching failed-request experiment. It runs `network request <id>` when `requestId` is present and/or `network requests --filter …` when `filter` or `url` is present (`url` supplies the filter pattern when `filter` is omitted); add `namespace` / `session` when the generated batch should target an explicit upstream namespace/session. It merges failed-request rows from the batch JSON with initiator-style hints and a bounded workspace literal scan (`maxWorkspaceFiles` defaults to 2000, cap 5000), surfaces everything under `details.networkSourceLookup`, and avoids automatic blame or edits. Compact `network requests` results with safe request IDs also add `details.nextActions` for request details, bounded `networkSourceLookup` on actionable failures, path filtering, diagnostic-buffer clearing before a repro, or HAR capture so agents can branch without guessing request-id syntax. For noisy aggregate buffers, wrapper-side `network requests --current-page` / `--current-origin` keeps only rows matching the active page origin, while `--current-url` keeps exact active-document URL rows and reports counts in `details.networkRequestsPageFilter`. When the wrapper has seen `network route` in the same session, pending fetch/XHR rows or CORS-looking errors that match the route surface `details.networkRouteDiagnostics` plus executable follow-ups to inspect the request or start HAR capture; same-origin/CORS-correct fixture retry guidance stays in prose. Network diagnostics are read-only for wrapper page state: request URLs in `network request` or generated `networkSourceLookup` batches do not replace the session’s active page target or invalidate page-scoped refs from the app page.
|
|
510
|
+
`networkSourceLookup` is the matching failed-request experiment. It runs `network request <id>` when `requestId` is present and/or `network requests --filter …` when `filter` or `url` is present (`url` supplies the filter pattern when `filter` is omitted); add `namespace` / `session` when the generated batch should target an explicit upstream namespace/session (`namespace: ""` explicitly selects the default namespace and overrides an ambient namespace). It merges failed-request rows from the batch JSON with initiator-style hints and a bounded workspace literal scan (`maxWorkspaceFiles` defaults to 2000, cap 5000), surfaces everything under `details.networkSourceLookup`, and avoids automatic blame or edits. Compact `network requests` results with safe request IDs also add `details.nextActions` for request details, bounded `networkSourceLookup` on actionable failures, path filtering, diagnostic-buffer clearing before a repro, or HAR capture so agents can branch without guessing request-id syntax. For noisy aggregate buffers, wrapper-side `network requests --current-page` / `--current-origin` keeps only rows matching the active page origin, while `--current-url` keeps exact active-document URL rows and reports counts in `details.networkRequestsPageFilter`. When the wrapper has seen `network route` in the same session, pending fetch/XHR rows or CORS-looking errors that match the route surface `details.networkRouteDiagnostics` plus executable follow-ups to inspect the request or start HAR capture; same-origin/CORS-correct fixture retry guidance stays in prose. Network diagnostics are read-only for wrapper page state: request URLs in `network request` or generated `networkSourceLookup` batches do not replace the session’s active page target or invalidate page-scoped refs from the app page.
|
|
468
511
|
|
|
469
512
|
```json
|
|
470
513
|
{ "networkSourceLookup": { "requestId": "req-1", "url": "/api/fail" } }
|
|
@@ -479,7 +522,7 @@ For asynchronous exports, click first and then wait for the download:
|
|
|
479
522
|
|
|
480
523
|
When a user gives exact artifact paths for screenshots, recordings, downloads, PDFs, traces, or HAR files, use those paths or explicitly report why the artifact was unavailable; do not silently substitute a different path in the final report. The wrapper creates missing parent directories for direct artifact paths such as `state save`, screenshots, PDFs, downloads, and `wait --download`. For simple loopback `download <selector> <path>` anchor links with HTTP(S) `href`, it can save the in-page response directly to the requested path before falling back to upstream click/download behavior; non-loopback/profile downloads stay upstream-owned. With current upstream `agent-browser`, treat `details.savedFilePath` as upstream-reported metadata and confirm `details.artifacts[].exists` / `details.artifactVerification.verified` before relying on the requested `wait --download <path>` file being present on disk; non-file download payloads such as `data:` URLs are not verified local artifacts.
|
|
481
524
|
|
|
482
|
-
For evidence-only screenshots or QA captures, branch on `details.artifactVerification` and `details.artifacts` before reporting PASS/FAIL;
|
|
525
|
+
For evidence-only screenshots or QA captures, branch on `details.artifactVerification` and `details.artifacts` before reporting PASS/FAIL; a pre-existing path from an artifact-producing command that was not updated during the command fails as `status: "stale"` instead of being accepted as fresh evidence (`wait --download` remains observational). Inline image attachments are optional when size limits allow—do not require vision review unless the user asked for visual inspection. If the latest prompt names exact required artifact paths, browser close can be blocked with `details.promptGuard` until those artifacts are saved and verified.
|
|
483
526
|
|
|
484
527
|
Artifact cleanup is host-owned, not a browser command. Close commands (`close`, `quit`, or `exit`) shut down the browser session but do **not** delete explicit screenshots, downloads, PDFs, traces, HAR files, or recordings saved to paths you chose. When the session’s non-empty `details.artifactManifest` is in scope, a successful close command appends a compact `Artifact lifecycle` note and sets `details.artifactCleanup` with the same retention summary as `details.artifactRetentionSummary`, a fixed `note` about host-owned cleanup, and `explicitArtifactPaths`: up to ten distinct paths from manifest rows whose `storageScope` is `explicit-path` (this list can be empty if the recent window only holds spills or other non-explicit inventory). Remove any listed paths with normal file tools after inspection.
|
|
485
528
|
|
|
@@ -507,15 +550,15 @@ The wrapper does not clone profiles or hide what upstream Chrome/Chromium profil
|
|
|
507
550
|
Use these rules:
|
|
508
551
|
|
|
509
552
|
- Use public/temp profiles for tests and examples.
|
|
510
|
-
- Do not assume `--profile Default` is correct. Ask the agent to run `profiles` to list Chrome profile directory names, then `doctor` if profile/user-data-dir resolution still fails.
|
|
553
|
+
- Do not assume `--profile Default` is correct. Ask the agent to run `profiles` to list Chrome profile directory names, then `doctor` if profile/user-data-dir resolution still fails. On macOS, a copied Chrome profile may omit Keychain-encrypted cookies, so profile selection is not proof that the target page is authenticated; verify the page and use a user-approved headed login once when needed.
|
|
511
554
|
- For non-Chrome Chromium browsers such as Brave, Edge, Arc, or Vivaldi, use `--executable-path <path>` when upstream can launch that executable. If you need that browser's existing login state, use the browser's real profile/user-data directory path when upstream accepts it, or attach with `--auto-connect` / `connect` to a debug-enabled running browser when appropriate.
|
|
512
555
|
- Use `sessionMode: "fresh"` when switching from public browsing to `--allowed-domains`, `--profile`, `--executable-path`, `--webgpu`, `--restore`, `--restore-save`, restore check flags, `--namespace`, `--session-name`, `--cdp`, `--state`, `--auto-connect`, `--init-script`, `--enable`, `-p` / `--provider`, or iOS `--device`.
|
|
513
556
|
- Use `--session` when you want to manage a live upstream session name yourself. For CDP, connect once, verify with `get url`, keep using that session without repeating `--cdp`, and close it explicitly when done. The wrapper preserves the established attachment across follow-ups instead of resending local-launch defaults, and live-checks the URL before later page reads or interactions because an attached browser can change tabs outside Pi.
|
|
514
|
-
- Do not treat an arbitrary `--session` name alone as persisted auth after `close`, `quit`, or `exit`. Wrapper-owned managed sessions automatically set a Git-checkout-generation-stable `AGENT_BROWSER_RESTORE` key so cookies/localStorage/sessionStorage survive browser relaunches across Pi chats in the same checkout; the key follows a renamed checkout but changes when that path is replaced or copied, and automatic restore fails closed outside a Git checkout. The wrapper combines the checkout-root and Git-admin filesystem identities with a generation UUID in the Git admin directory and never adopts older cwd-only keys; a bare caller `--session` name does not get that injection, and the wrapper reserves `piab-*` names case-insensitively so another Pi process cannot attach to a managed authenticated browser through a case alias. Disable with `PI_AGENT_BROWSER_MANAGED_SESSION_RESTORE=0`. For explicit non-managed sessions use `--session <id> --restore`, `--profile`, or `--state`. SSO/2FA such as Okta Touch ID may still need one human approval (often `--headed` the first time); after that, managed restore should keep the session without a manual `state save` dance. Any upstream `agent-browser.json` / `AGENT_BROWSER_CONFIG` / `--config` discovered while planning blocks browser-backed native calls without reading it, while accepted browser-backed spawns pin a process-private empty config to close config-creation races. This is separate from this package's trusted Pi-scoped config; sessionless local/setup commands retain upstream config behavior. Raw batch argv, batch stdin containing nested `connect`/`batch`, browser mutation flag, or matching launch-mutation env disables automatic managed restore rather than risking restored auth in a caller-customized or attached browser. Every accepted browser-backed subprocess, including wrapper-owned close, pins `AGENT_BROWSER_CONFIG` to that process-private empty config (`0400` on POSIX) in the marked secure-temp lifecycle so a project or user config created between planning and spawn cannot change the browser. A user-private immutable ticket-claim lock serializes each same-identity daemon inspection through the receiving spawn and bridges the pre-update v2 lock path; every lock winner re-inspects the live daemon, and abandoned v2 locks fail closed rather than being reclaimed unsafely. POSIX process identity probes use absolute `/bin/ps` then `/usr/bin/ps` paths. Before an incompatible call, the wrapper inspects the actual same-identity daemon and blocks when it retains any restore key, cannot be inspected, or reports restore-disabled policy without current-process provenance, including daemons missing from transcript state and sessions launched with explicit restore keys. Same-process `session_tree` transitions retain recorded provenance; extension reload, restart, and `/resume` deliberately do not trust transcript-only provenance for a still-live restore-disabled daemon, so close it first, omit the explicit session and use `sessionMode: "fresh"`, or choose a distinct explicit session. If inspection instead proves the old daemon inactive, the next owned no-restore spawn records that null policy so subsequent follow-ups remain usable. Wrapper-owned subprocesses pin the canonical namespace, including an explicit empty default, so a parent `AGENT_BROWSER_NAMESPACE` cannot redirect close or helper calls; Electron status target reads and current-managed probes also acquire the same daemon-policy lock, verify the live URL before title/content reads, and apply the same restore decision to every underlying read. A probe whose reads all fail is an `upstream-error`, not a successful empty partial result. Current-managed probe results persist their namespace and ref state for Pi reload/branch replay. Upstream restore files live under `~/.agent-browser/` and are plaintext unless you set `AGENT_BROWSER_ENCRYPTION_KEY`; on POSIX the wrapper canonicalizes and pins `HOME` after caller env merging, requires owner-trusted non-writable ancestry, requires stable device/inode/birth-time metadata for both checkout and Git-admin directories, enforces mode `0700` without silently tightening unsafe existing directories, and rejects symlinks/non-directories along the exact restore `sessions` path and its `.tmp` write area before automatic managed restore. Windows automatic managed restore requires an absolute `USERPROFILE` and the documented 64-character hex `AGENT_BROWSER_ENCRYPTION_KEY` because POSIX mode checks cannot verify profile ACLs. Wrapper-owned close commands discard caller config/restore globals, preserve the live daemon's existing restore key instead of injecting one derived from a possibly replaced checkout, and record a returned old-generation snapshot against that observed wrapper key. If a fresh command starts agent-browser but then fails, the wrapper probes that exact identity and retains a live or uninspectable daemon for shutdown cleanup instead of abandoning it. After a wrapper-owned managed session closes successfully, the wrapper persists the returned state path as an atomic record in a lockless convergent per-key ownership directory (`0700`, with `0600` records, on POSIX), keeps the two newest proven snapshots for its exact restore key across Pi restarts, self-heals malformed regular records, removes additional proven snapshots older than 30 days, expires ownership-proven snapshots and empty manifests from older restore-key generations only when a private lineage record proves the same canonical checkout path, after 30 days, and caps young close churn at 256 records per restore key; unrecorded matching files and the current checkout key remain untouched. Managed restore keys and key-bearing paths are redacted from tool output and transcripts. `session list` and `state list` hide wrapper-managed rows; cross-checkout managed `--restore` / `--state` / state-file access, broad `state clear`, `state clean`, and managed save/rename targets are rejected before spawn. Browser access to `.agent-browser` storage is blocked through command-specific file operands (including dash-prefixed values), every path-bearing upstream environment mirror (including state/profile/config, executable/extension/init-script, action-policy, artifact, skills, and socket paths), encoded, nested-file-scheme, Windows-aliased, or symlinked targets (including not-yet-created descendants of symlinked directories), content-returning local-URL commands, protected artifact destinations and top-level `outputPath`, local-page follow-ups, and persisted unverified top-level or batch tab/attachment/script/state-load transitions. Raw batch command strings are split on literal ASCII spaces exactly like upstream and inspected recursively just like batch stdin arrays; Electron launch handoffs, probes, and later capture share the same boundary: snapshot/tabs handoff and probes verify the live URL before tab/title/content helpers, and cancellation during handoff closes the managed session plus process/profile. The wrapper rejects enabled `--allow-file-access` argv/env plus file-access-enabling or protected-path `--args` / `AGENT_BROWSER_ARGS` values, removes caller file-access occurrences, clears raw-args env, and adds canonical `--args "" --allow-file-access false` defaults on local-browser spawns so project/user config cannot silently preserve local-page filesystem access; attached-session follow-ups omit those launch-only flags; an explicit safe CLI `--args` value may still override the empty default. Post-transition summaries, including after arbitrary `eval`, verify the live URL before title and fail implicit transitions to local file pages; failed navigation attempts remain unverified, and stale concurrent completions cannot overwrite newer unknown page state. `get url`, `tab list`, non-content `tab <id>` selection, explicit safe navigation away, and session/tab close remain available for recovery; tab selection stays unverified until `get url` succeeds. The wrapper repeats checkout, storage, environment, managed-session ownership, and managed-state access validation after async config/socket setup immediately before spawn. On POSIX the selected daemon socket directory must be absolute, current-user-owned, mode `0700`, under trusted ancestry, and free of symlink, foreign-owner, or special planted entries. Pre-existing unsafe modes are rejected rather than repaired, and the check is repeated immediately before spawn. On native Windows, command-first launcher reordering moves only syntactically valid leading globals, rewrites a valued `--restore <name>` as `--restore=<name>` to preserve upstream optional-value semantics, and leaves invalid or command-scoped leading tokens untouched. Upstream periodically saves restore-enabled cookies/localStorage while the browser is open; `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` defaults to `30000`, `0` disables periodic saves but keeps save-on-close, and the `never` value for `--restore-save` disables automatic saves for that restore session.
|
|
557
|
+
- Do not treat an arbitrary `--session` name alone as persisted auth after `close`, `quit`, or `exit`. Wrapper-owned managed sessions automatically set a Pi-transcript- and Git-checkout-generation-scoped `AGENT_BROWSER_RESTORE` key so cookies/localStorage/sessionStorage survive browser relaunches, reloads, and `/resume` for that transcript. Different Pi chats use different restore pools because upstream 0.33.2 loads the newest file for a key regardless of browser-session suffix; this prevents concurrent chats from clobbering or inheriting each other's state. The Git generation identity survives a checkout rename, but the full key also binds the transcript's cwd-derived managed-session base name, so renaming the checkout or running the same transcript from a different working directory starts a fresh key fail-closed (re-authenticate once); the key also changes when that path is replaced/copied or the Pi transcript changes, and automatic restore fails closed outside a Git checkout. The wrapper combines the checkout-root and Git-admin filesystem identities, a generation UUID in the Git admin directory, and the transcript's cwd-derived managed-session base name, and never adopts older cwd-only keys; a bare caller `--session` name does not get that injection, and the wrapper reserves `piab-*` names case-insensitively so another Pi process cannot attach to a managed authenticated browser through a case alias. Disable with `PI_AGENT_BROWSER_MANAGED_SESSION_RESTORE=0`. For explicit non-managed sessions use `--session <id> --restore`, `--profile`, or `--state`. SSO/2FA such as Okta Touch ID may still need one human approval (often `--headed` the first time); after that, managed restore should keep the session without a manual `state save` dance. Passive project/user upstream `agent-browser.json` files are ignored because accepted browser-backed spawns pin a process-private empty config to close config-creation races. Explicit `AGENT_BROWSER_CONFIG` or `--config` overrides still block browser-backed native calls without reading caller-selected content. This is separate from this package's trusted Pi-scoped config; sessionless local/setup commands retain upstream config behavior. Raw batch argv, batch stdin containing nested `connect`/`batch`, browser mutation flag, or matching launch-mutation env disables automatic managed restore rather than risking restored auth in a caller-customized or attached browser. Every accepted browser-backed subprocess, including wrapper-owned close, pins `AGENT_BROWSER_CONFIG` to that process-private empty config (`0400` on POSIX) in the marked secure-temp lifecycle so a project or user config created between planning and spawn cannot change the browser. A user-private immutable ticket-claim lock serializes each same-identity daemon inspection through the receiving spawn and bridges the pre-update v2 lock path; every lock winner re-inspects the live daemon, and abandoned v2 locks fail closed rather than being reclaimed unsafely. POSIX process identity probes use absolute `/bin/ps` then `/usr/bin/ps` paths; Android/Termux uses the `ps` binary beside the running Node executable because Android has neither path. Before an incompatible call, the wrapper inspects the actual same-identity daemon and blocks when it retains any restore key, cannot be inspected, or reports restore-disabled policy without current-process provenance, including daemons missing from transcript state and sessions launched with explicit restore keys. Same-process `session_tree` transitions retain recorded provenance; extension reload, restart, and `/resume` deliberately do not trust transcript-only provenance for a still-live restore-disabled daemon, so close it first, omit the explicit session and use `sessionMode: "fresh"`, or choose a distinct explicit session. If inspection instead proves the old daemon inactive, the next owned no-restore spawn records that null policy so subsequent follow-ups remain usable. Wrapper-owned subprocesses pin the canonical namespace, including an explicit empty default, so a parent `AGENT_BROWSER_NAMESPACE` cannot redirect close or helper calls; Electron status target reads and current-managed probes also acquire the same daemon-policy lock, verify the live URL before title/content reads, and apply the same restore decision to every underlying read. A probe whose reads all fail is an `upstream-error`, not a successful empty partial result. Current-managed probe results persist their namespace and ref state for Pi reload/branch replay. Upstream restore files live under `~/.agent-browser/` and are plaintext unless you set `AGENT_BROWSER_ENCRYPTION_KEY`; on POSIX the wrapper canonicalizes and pins `HOME` after caller env merging, requires owner-trusted non-writable ancestry, requires stable device/inode/birth-time metadata for both checkout and Git-admin directories (Android uses stable device/inode identity plus the generation UUID because its Node runtime reports mutable ctime as birth time), enforces mode `0700` without silently tightening unsafe existing directories, and rejects symlinks/non-directories along the exact restore `sessions` path and its `.tmp` write area before automatic managed restore. Windows automatic managed restore requires an absolute `USERPROFILE` and the documented 64-character hex `AGENT_BROWSER_ENCRYPTION_KEY` because POSIX mode checks cannot verify profile ACLs. Wrapper-owned close commands discard caller config/restore globals, preserve the live daemon's existing restore key instead of injecting one derived from a possibly replaced checkout, and record a returned old-generation snapshot against that observed wrapper key. If a fresh command starts agent-browser but then fails, the wrapper probes that exact identity and retains a live or uninspectable daemon for shutdown cleanup instead of abandoning it. After a wrapper-owned managed session closes successfully, the wrapper persists the returned state path as an atomic record in a lockless convergent per-key ownership directory (`0700`, with `0600` records, on POSIX), keeps the two newest proven snapshots for its exact restore key across Pi restarts, self-heals malformed regular records, removes additional proven snapshots older than 30 days, expires ownership-proven snapshots and empty manifests from older restore-key generations only when a private lineage record proves the same canonical checkout path, after 30 days, and caps young close churn at 256 records per restore key; unrecorded matching files and the current checkout key remain untouched. Managed restore keys and key-bearing paths are redacted from tool output and transcripts. `session list` and `state list` hide wrapper-managed rows; cross-checkout managed `--restore` / `--state` / state-file access, broad `state clear`, `state clean`, and managed save/rename targets are rejected before spawn. Browser access to `.agent-browser` storage is blocked through command-specific file operands (including dash-prefixed values), every path-bearing upstream environment mirror (including state/profile/config, executable/extension/init-script, action-policy, artifact, skills, and socket paths), encoded, nested-file-scheme, Windows-aliased, or symlinked targets (including not-yet-created descendants of symlinked directories), content-returning local-URL commands, protected artifact destinations and top-level `outputPath`, local-page follow-ups, and persisted unverified top-level or batch tab/attachment/script/state-load transitions. Raw batch command strings are split on literal ASCII spaces exactly like upstream and inspected recursively just like batch stdin arrays; Electron launch handoffs, probes, and later capture share the same boundary: snapshot/tabs handoff and probes verify the live URL before tab/title/content helpers, and cancellation during handoff closes the managed session plus process/profile. The wrapper rejects enabled `--allow-file-access` argv/env plus file-access-enabling or protected-path `--args` / `AGENT_BROWSER_ARGS` values, removes caller file-access occurrences, clears raw-args env, and adds canonical `--allow-file-access false` on local-browser spawns. Routine HTTP(S) work relies on the protected empty config and cleared raw-args environment instead of sending `--args ""`, which `agent-browser 0.34.0` treats as a launch override that can replace a profiled browser with `about:blank`. Local-file navigation is limited to wrapper-managed local browsers; caller-owned and attached browsers are blocked because their file-access launch provenance is unknown. A fixed non-empty `--args` value is added only when a wrapper user-agent compatibility session is launching or its daemon is verified inactive; active and attached follow-ups omit launch-only flags. Explicit validated safe CLI `--args` and `--user-agent` remain usable but are launch-scoped, so changing either on an active managed session requires `sessionMode: "fresh"`. Post-transition summaries, including after arbitrary `eval`, verify the live URL before title and fail implicit transitions to local file pages; failed navigation attempts remain unverified, and stale concurrent completions cannot overwrite newer unknown page state. `get url`, `tab list`, non-content `tab <id>` selection, explicit safe navigation away, and session/tab close remain available for recovery; tab selection stays unverified until `get url` succeeds. The wrapper repeats checkout, storage, environment, managed-session ownership, and managed-state access validation after async config/socket setup immediately before spawn. On POSIX the selected daemon socket directory must be absolute, current-user-owned, mode `0700`, under trusted ancestry, and free of symlink, foreign-owner, or special planted entries. On Android/Termux the wrapper anchors that trust at the private `/data/data/<package>` app sandbox and uses a short socket root there instead of inaccessible `/tmp`. Pre-existing unsafe modes are rejected rather than repaired, and the check is repeated immediately before spawn. On native Windows, command-first launcher reordering moves only syntactically valid leading globals, rewrites a valued `--restore <name>` as `--restore=<name>` to preserve upstream optional-value semantics, and leaves invalid or command-scoped leading tokens untouched. Upstream periodically saves restore-enabled cookies/localStorage while the browser is open; `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` defaults to `30000`, `0` disables periodic saves but keeps save-on-close, and the `never` value for `--restore-save` disables automatic saves for that restore session.
|
|
515
558
|
- Caller-owned explicit sessions are live-checked with `get url` before content-bearing reads or interactions. Missing or stale transcript page state is not treated as proof of a safe target; if the live URL cannot be verified, the requested content command does not run. Calls to the same effective canonical namespace/session are serialized inside one extension instance; explicit namespace argv overrides `AGENT_BROWSER_NAMESPACE`, including an explicit empty default from that probe through any semantic-action snapshot and the requested command, while different caller-owned sessions remain independent. Raw non-bail batches are rejected when a failed navigation could expose prior local or unverified page content; use exact `batch --bail` or split navigation from content. Protected Windows paths include drive-relative forms such as `C:.agent-browser\\state\\...`. Nested `batch` steps are rejected, and raw batch command strings mirror upstream's ASCII-space tokenizer, including its single/double-quote and backslash handling, without splitting on other Unicode whitespace.
|
|
516
559
|
- Prefer page actions and storage checks over cookie dumps. `cookies get` can expose real profile cookies.
|
|
517
560
|
- Prefer `auth save --password-stdin` over putting passwords in `args`; the wrapper only accepts caller `stdin` for `batch`, `eval --stdin`, and `auth save --password-stdin` (top-level `job` and `qa` compile to `batch` and supply their own stdin).
|
|
518
|
-
- Use `state save <path>` / `state load <path>` for portable test state. `state save` is reported as a file artifact with verification metadata; if an upstream-successful artifact command reports a non-pending file path that the wrapper cannot find
|
|
561
|
+
- Use `state save <path>` / `state load <path>` for portable test state. `state save` is reported as a file artifact with verification metadata; if an upstream-successful artifact command reports a non-pending file path that the wrapper cannot find or did not update during this command, the tool fails with `failureCategory: "artifact-missing"` instead of treating missing/stale evidence as durable. `state load` may mention a path but is not treated as a newly saved artifact.
|
|
519
562
|
- Treat `cookies get`, `storage local|session`, `state show`, and `auth show` output as sensitive. `state show` is presented as saved-state metadata only, and cookie/localStorage/sessionStorage values are redacted from structured details. The native presentation summarizes and redacts credential-like values while allowing benign primitive storage values to aid local QA, but avoid requesting broad dumps unless the task needs them.
|
|
520
563
|
- Use `dialog status`, `dialog accept [text]`, `dialog dismiss`, and `frame <selector|main>` through native `args`; dialog commands use a shorter wrapper timeout and timed-out interactions add `inspect-dialog-after-timeout` / `dismiss-dialog-after-timeout` / fresh-session recovery actions so a blocking alert/prompt does not burn the full default watchdog. Use exact `confirm <id>` / `deny <id>` next actions for guarded-action confirmations.
|
|
521
564
|
|
|
@@ -606,17 +649,6 @@ npm run verify -- pre-pr
|
|
|
606
649
|
|
|
607
650
|
That mode composes the full default gate with `npm run verify -- package`, so package contents and forbidden repo-only files are checked without launching Pi lifecycle, Crabbox, or live dogfood flows. Package modes build through npm `prepare`; lifecycle and startup-profile build in their focused scripts; default and platform-target build before consuming `dist/`, so clean checkouts do not validate stale or missing compiled output. The same `prepare` script owns GitHub/source installs; when Pi installs with `npm install --omit=dev`, it installs the source-build dev dependencies with lifecycle scripts disabled before building the ignored `dist/` entrypoint that Pi loads.
|
|
608
651
|
|
|
609
|
-
The deterministic agent-efficiency benchmark’s **standalone JSON/Markdown accounting run** is not part of default or pre-PR `npm run verify` (only `npm run verify -- benchmark` or `npm run benchmark:agent-browser` invokes the script). The full unit suite still exercises `test/agent-browser.efficiency-benchmark.test.ts`. Use the script before and after agent-facing abstractions to prove call-count, output-size, stale-ref, artifact, failure-category coverage, success-rate, and elapsed-time effects before changing the wrapper UX:
|
|
610
|
-
|
|
611
|
-
```bash
|
|
612
|
-
npm run benchmark:agent-browser
|
|
613
|
-
npm run verify -- benchmark
|
|
614
|
-
```
|
|
615
|
-
|
|
616
|
-
Save a JSON baseline (for example before changing playbook or wrapper behavior), then compare later runs: `npm run benchmark:agent-browser -- --json > /tmp/agent-browser-benchmark.json` and `npm run benchmark:agent-browser -- --compare /tmp/agent-browser-benchmark.json`.
|
|
617
|
-
|
|
618
|
-
It does not launch a browser or mutate local profiles; it models representative raw workflows and provides a stable baseline for later comparisons.
|
|
619
|
-
|
|
620
652
|
The opt-in startup profiler measures only the package extension entrypoint import plus factory registration in fresh Node processes. It intentionally does **not** launch Pi, tmux, mise, npm, browsers, or `agent-browser`; full Pi TUI ready-prompt profiling proved too invasive for routine verification on the operator machine. Run it after package entrypoint, generated runtime, or top-level import changes:
|
|
621
653
|
|
|
622
654
|
```bash
|
|
@@ -640,7 +672,7 @@ A deterministic host-only live-browser wrapper smoke is available without an LLM
|
|
|
640
672
|
npm run verify -- dogfood
|
|
641
673
|
```
|
|
642
674
|
|
|
643
|
-
That mode drives the native wrapper through top-level `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close against a deterministic local fixture. It complements, but does not replace, the interactive Pi/tmux release dogfood in [`docs/RELEASE.md`](docs/RELEASE.md#pre-release-checks).
|
|
675
|
+
That mode clean-builds the package, then drives the native wrapper through top-level `script` branching/aggregation, `qa`, `semanticAction`, constrained `job`, screenshot artifact verification, and session close against a deterministic local fixture. It complements, but does not replace, the interactive Pi/tmux release dogfood in [`docs/RELEASE.md`](docs/RELEASE.md#pre-release-checks).
|
|
644
676
|
|
|
645
677
|
Cross-platform release coverage uses Crabbox to run macOS, Ubuntu Linux, and native Windows target suites; see [`docs/platform-smoke.md`](docs/platform-smoke.md) for the required matrix, standalone coverage (`npm run smoke:platform:all` and per-target `smoke:platform:macos` / `:ubuntu` / `:windows-native`), and artifact/lease inspection. The release gate is:
|
|
646
678
|
|
|
@@ -704,7 +736,7 @@ Configured-source lifecycle validation:
|
|
|
704
736
|
npm run verify -- lifecycle
|
|
705
737
|
```
|
|
706
738
|
|
|
707
|
-
The harness defaults to Pi model `zai/glm-5.2` and **180000 ms** per-step tmux waits; pass `--model <id>` and/or `--timeout-ms <ms>` after `lifecycle` when you need different settings (see [Configured-source lifecycle validation](docs/RELEASE.md#configured-source-lifecycle-validation) in `docs/RELEASE.md`). It launches Pi
|
|
739
|
+
The harness defaults to Pi model `zai/glm-5.2` and **180000 ms** per-step tmux waits; pass `--model <id>` and/or `--timeout-ms <ms>` after `lifecycle` when you need different settings (see [Configured-source lifecycle validation](docs/RELEASE.md#configured-source-lifecycle-validation) in `docs/RELEASE.md`). It launches the supported Pi runtime with `--approve` and a deterministic `--session-id`, drives `/reload`, closes Pi, relaunches the exact same session, asserts the JSONL header id, and checks managed-session continuity, compiled-entrypoint pickup after process restart, persisted spill reachability, and real Pi `tool_result` failure-patch behavior.
|
|
708
740
|
|
|
709
741
|
Use lifecycle validation when testing `/reload`, exact-session relaunch, `/resume`, managed-session continuity, or persisted artifact behavior. Branch-backed state and `session_tree` cleanup ownership are covered by focused extension harness tests. Maintainers must run the lifecycle harness before every publish; see [Pre-release checks](docs/RELEASE.md#pre-release-checks).
|
|
710
742
|
|
|
@@ -735,6 +767,7 @@ These calls return plain text and stay stateless: the extension does not inject
|
|
|
735
767
|
- After the wrapper observes tab-drift risk for a session (for example open correction, overlapping stale opens, or resumed session state), later active-tab commands best-effort pin that tab inside the same upstream invocation. Routine same-session commands are not preflighted with tab list just because a target tab or ref snapshot is known.
|
|
736
768
|
- For sessions with observed tab-drift risk, after a successful command on a known target tab, agent_browser also best-effort restores that intended tab if a restored/background tab steals focus after the command completes. Routine same-session commands skip this post-command tab-list probe.
|
|
737
769
|
- If a known session target unexpectedly reports about:blank, agent_browser best-effort re-selects the prior intended target when it still exists; if recovery fails, it records the observed about:blank target and reports exact recovery guidance instead of treating the prior page as active.
|
|
770
|
+
- If upstream reports tab_gone, the pinned bound tab is gone; use details.nextActions (tab list / tab new) instead of assuming another tab is yours.
|
|
738
771
|
<!-- agent-browser-playbook:end wrapper-tab-recovery -->
|
|
739
772
|
|
|
740
773
|
## Project map
|
|
@@ -745,7 +778,8 @@ These calls return plain text and stay stateless: the extension does not inject
|
|
|
745
778
|
| `extensions/agent-browser/lib/runtime.ts` | Argv parsing, session planning, redaction, and execution-plan helpers (pure planning; subprocess wiring lives beside the entrypoint) |
|
|
746
779
|
| `extensions/agent-browser/lib/results/` | Model-facing result rendering and error guidance |
|
|
747
780
|
| `extensions/agent-browser/lib/playbook.ts` | Canonical generated agent/browser guidance |
|
|
748
|
-
| `scripts/agent-browser-
|
|
781
|
+
| `scripts/agent-browser-target.mjs` | Canonical target upstream version shared by runtime and build-time checks |
|
|
782
|
+
| `scripts/agent-browser-capability-baseline.mjs` | Help samples and doc/token inventory for drift checks; imports the canonical target version |
|
|
749
783
|
| `scripts/check-command-reference-baseline.mjs` | Regenerates or verifies HTML-bounded baseline blocks in `docs/COMMAND_REFERENCE.md` (via `npm run docs -- command-reference …`) |
|
|
750
784
|
| `docs/COMMAND_REFERENCE.md` | Repo-readable native command reference |
|
|
751
785
|
| `docs/TOOL_CONTRACT.md` | Tool parameters, result shape, and behavior contract |
|