agent-browser 0.31.1 → 0.32.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +55 -5
- package/bin/agent-browser-darwin-arm64 +0 -0
- package/bin/agent-browser-darwin-x64 +0 -0
- package/bin/agent-browser-linux-arm64 +0 -0
- package/bin/agent-browser-linux-musl-arm64 +0 -0
- package/bin/agent-browser-linux-musl-x64 +0 -0
- package/bin/agent-browser-linux-x64 +0 -0
- package/bin/agent-browser-win32-x64.exe +0 -0
- package/package.json +1 -1
- package/scripts/check-version-sync.js +11 -0
- package/scripts/sync-version.js +14 -0
- package/skill-data/core/SKILL.md +12 -2
- package/skill-data/core/references/commands.md +6 -2
- package/skill-data/core/references/session-management.md +1 -1
- package/skill-data/core/references/trust-boundaries.md +1 -0
- package/skill-data/core/references/webgpu.md +118 -0
package/README.md
CHANGED
|
@@ -517,7 +517,7 @@ Common tools include:
|
|
|
517
517
|
- `agent_browser_eval`
|
|
518
518
|
- `agent_browser_close`
|
|
519
519
|
|
|
520
|
-
Each tool has typed fields such as `url`, `selector`, `text`, `key`, and `
|
|
520
|
+
Each tool has typed fields such as `url`, `selector`, `text`, `key`, `session`, and `allowedDomains`, so MCP clients show meaningful approval prompts instead of raw command arrays. The common `allowedDomains` array maps to `--allowed-domains` and activates the same WebRTC containment and launch-mode restrictions. Each tool also accepts `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally.
|
|
521
521
|
|
|
522
522
|
Example MCP client config:
|
|
523
523
|
|
|
@@ -689,6 +689,8 @@ agent-browser --session "$SESSION" --restore open twitter.com
|
|
|
689
689
|
agent-browser --session "$SESSION" --restore --restore-check-text Dashboard open twitter.com
|
|
690
690
|
```
|
|
691
691
|
|
|
692
|
+
State is saved when the browser closes (explicit `close`, idle timeout, or daemon shutdown) and also periodically while the browser is open, so a browser window you close by hand still leaves a recent save behind. Periodic autosave waits for commands to settle, then saves at most once per `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` (default 30000; set to `0` to save only on close). Idle sessions keep saving on the same interval, so changes the page makes on its own (token refreshes, background requests) are captured too. It respects the `--restore-save` policy.
|
|
693
|
+
|
|
692
694
|
### State Encryption
|
|
693
695
|
|
|
694
696
|
Encrypt saved session data at rest with AES-256-GCM:
|
|
@@ -705,6 +707,7 @@ agent-browser --session secure --restore open example.com
|
|
|
705
707
|
| --------------------------------- | -------------------------------------------------- |
|
|
706
708
|
| `AGENT_BROWSER_RESTORE` | Auto-save/load state persistence name |
|
|
707
709
|
| `AGENT_BROWSER_RESTORE_SAVE` | Restore save policy: `auto`, `always`, or `never` |
|
|
710
|
+
| `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` | Min ms between periodic autosaves (default: 30000, 0 disables) |
|
|
708
711
|
| `AGENT_BROWSER_NAMESPACE` | Namespace for daemon sockets and restore state |
|
|
709
712
|
| `AGENT_BROWSER_SESSION_NAME` | Legacy auto-save/load state persistence name |
|
|
710
713
|
| `AGENT_BROWSER_ENCRYPTION_KEY` | 64-char hex key for AES-256-GCM encryption |
|
|
@@ -717,7 +720,7 @@ agent-browser includes security features for safe AI agent deployments. All feat
|
|
|
717
720
|
- **Authentication Vault**: Store credentials locally (always encrypted), reference by name. The LLM never sees passwords. `auth login` navigates with `load` and then waits for login form selectors to appear (SPA-friendly, timeout follows the default action timeout). A key is auto-generated at `~/.agent-browser/.encryption-key` if `AGENT_BROWSER_ENCRYPTION_KEY` is not set: `echo "pass" | agent-browser auth save github --url https://github.com/login --username user --password-stdin` then `agent-browser auth login github`
|
|
718
721
|
- **Plugin System**: Extend agent-browser with external executable plugins. Plugins run out-of-process over the `agent-browser.plugin.v1` stdio JSON protocol and declare capabilities such as `credential.read`, `browser.provider`, `launch.mutate`, or `command.run`.
|
|
719
722
|
- **Content Boundary Markers**: Wrap page output in delimiters so LLMs can distinguish tool output from untrusted content: `--content-boundaries`
|
|
720
|
-
- **Domain Allowlist**: Restrict navigation to trusted domains (wildcards like `*.example.com` also match the bare domain): `--allowed-domains "example.com,*.example.com"`. Sub-resource requests (scripts, images, fetch)
|
|
723
|
+
- **Domain Allowlist**: Restrict navigation to trusted domains (wildcards like `*.example.com` also match the bare domain): `--allowed-domains "example.com,*.example.com"`. Sub-resource requests (scripts, images, fetch), WebSocket/EventSource connections, and `sendBeacon` calls to non-allowed domains are blocked. WebRTC peer connections are disabled in supported Chromium sessions while the allowlist is active to prevent STUN, TURN, and DNS traffic from bypassing HTTP interception. Dedicated and shared workers are guarded with a bootstrap wrapper; if a page CSP forbids that wrapper, the worker fails closed rather than running without the allowlist guard. Pre-existing CDP sessions, auto-connect, Chrome profiles, direct-page provider plugins, agent-browser restore or state-file replay, raw Chrome args that select profiles, restore sessions, or open startup pages, iOS, and Safari reject this option because agent-browser cannot install equivalent containment before page scripts run. Include any CDN domains your target pages depend on (e.g., `*.cdn.example.com`).
|
|
721
724
|
- **Action Policy**: Gate destructive actions with a static policy file: `--action-policy ./policy.json`
|
|
722
725
|
- **Action Confirmation**: Require explicit approval for sensitive action categories: `--confirm-actions eval,download`
|
|
723
726
|
- **Output Length Limits**: Prevent context flooding: `--max-output 50000`
|
|
@@ -726,7 +729,7 @@ agent-browser includes security features for safe AI agent deployments. All feat
|
|
|
726
729
|
| ----------------------------------- | ---------------------------------------- |
|
|
727
730
|
| `AGENT_BROWSER_CONTENT_BOUNDARIES` | Wrap page output in boundary markers |
|
|
728
731
|
| `AGENT_BROWSER_MAX_OUTPUT` | Max characters for page output |
|
|
729
|
-
| `AGENT_BROWSER_ALLOWED_DOMAINS` | Comma-separated allowed domain patterns
|
|
732
|
+
| `AGENT_BROWSER_ALLOWED_DOMAINS` | Comma-separated allowed domain patterns; requires a fresh controllable browser context without profile/session startup args, restore/state replay, or direct-page provider plugins |
|
|
730
733
|
| `AGENT_BROWSER_ACTION_POLICY` | Path to action policy JSON file |
|
|
731
734
|
| `AGENT_BROWSER_CONFIRM_ACTIONS` | Action categories requiring confirmation |
|
|
732
735
|
| `AGENT_BROWSER_CONFIRM_INTERACTIVE` | Enable interactive confirmation prompts |
|
|
@@ -900,13 +903,14 @@ This is useful for multimodal AI models that can reason about visual layout, unl
|
|
|
900
903
|
| `--screenshot-quality <n>` | JPEG quality 0-100 (or `AGENT_BROWSER_SCREENSHOT_QUALITY` env) |
|
|
901
904
|
| `--screenshot-format <fmt>` | Screenshot format: `png`, `jpeg` (or `AGENT_BROWSER_SCREENSHOT_FORMAT` env) |
|
|
902
905
|
| `--headed` | Show browser window (not headless) (or `AGENT_BROWSER_HEADED` env) |
|
|
906
|
+
| `--webgpu` | Enable WebGPU; SwiftShader software Vulkan on Linux, no GPU required (or `AGENT_BROWSER_WEBGPU` env) |
|
|
903
907
|
| `--cdp <port\|url>` | Connect via Chrome DevTools Protocol (port or WebSocket URL) |
|
|
904
908
|
| `--auto-connect` | Auto-discover and connect to running Chrome (or `AGENT_BROWSER_AUTO_CONNECT` env) |
|
|
905
909
|
| `--color-scheme <scheme>` | Color scheme: `dark`, `light`, `no-preference` (or `AGENT_BROWSER_COLOR_SCHEME` env) |
|
|
906
910
|
| `--download-path <path>` | Default download directory (or `AGENT_BROWSER_DOWNLOAD_PATH` env) |
|
|
907
911
|
| `--content-boundaries` | Wrap page output in boundary markers for LLM safety (or `AGENT_BROWSER_CONTENT_BOUNDARIES` env) |
|
|
908
912
|
| `--max-output <chars>` | Truncate page output to N characters (or `AGENT_BROWSER_MAX_OUTPUT` env) |
|
|
909
|
-
| `--allowed-domains <list>` | Comma-separated allowed domain patterns (or `AGENT_BROWSER_ALLOWED_DOMAINS` env) |
|
|
913
|
+
| `--allowed-domains <list>` | Comma-separated allowed domain patterns; also disables WebRTC peer connections in supported Chromium sessions and rejects CDP, auto-connect, Chrome profiles, restore/state replay, direct-page provider plugins, unsafe startup `--args`, iOS, and Safari (or `AGENT_BROWSER_ALLOWED_DOMAINS` env) |
|
|
910
914
|
| `--action-policy <path>` | Path to action policy JSON file (or `AGENT_BROWSER_ACTION_POLICY` env) |
|
|
911
915
|
| `--confirm-actions <list>` | Action categories requiring confirmation (or `AGENT_BROWSER_CONFIRM_ACTIONS` env) |
|
|
912
916
|
| `--confirm-interactive` | Interactive confirmation prompts; auto-denies if stdin is not a TTY (or `AGENT_BROWSER_CONFIRM_INTERACTIVE` env) |
|
|
@@ -1148,8 +1152,41 @@ agent-browser open example.com --headed
|
|
|
1148
1152
|
|
|
1149
1153
|
This opens a visible browser window instead of running headless.
|
|
1150
1154
|
|
|
1155
|
+
On Linux hosts with no display (servers, containers), `--headed` still works: when `DISPLAY` is unset and Xvfb is installed, agent-browser starts a private virtual display for the browser and cleans it up on close (opt out with `AGENT_BROWSER_NO_XVFB=1`). Needed for [WebGPU screenshots](#webgpu), and useful for extensions that misbehave headless.
|
|
1156
|
+
|
|
1151
1157
|
> **Note:** Browser extensions work in both headed and headless mode (Chrome's `--headless=new`).
|
|
1152
1158
|
|
|
1159
|
+
## WebGPU
|
|
1160
|
+
|
|
1161
|
+
Headless Chrome does not expose WebGPU by default, so pages using it (three.js `WebGPURenderer`, Babylon.js, etc.) silently render black. The `--webgpu` flag enables a launch preset that makes WebGPU work, including in GPU-less containers and CI:
|
|
1162
|
+
|
|
1163
|
+
```bash
|
|
1164
|
+
agent-browser --webgpu open https://my-webgpu-app.example.com
|
|
1165
|
+
agent-browser screenshot app.png
|
|
1166
|
+
```
|
|
1167
|
+
|
|
1168
|
+
On macOS and Windows this uses the hardware Metal/D3D backend. On Linux it routes WebGPU through SwiftShader's software Vulkan (no GPU needed), which requires the system Vulkan loader and Mesa ICD:
|
|
1169
|
+
|
|
1170
|
+
```bash
|
|
1171
|
+
apt-get install -y libvulkan1 mesa-vulkan-drivers
|
|
1172
|
+
```
|
|
1173
|
+
|
|
1174
|
+
One upstream caveat: headless Chrome cannot capture WebGPU canvas presentation in screenshots on Windows and Linux (rendering and in-page readbacks work; the capture is black). Screenshots of WebGPU pages work headless on macOS; on Windows run `--headed` in a logged-in desktop session; on Linux just add `--headed` — when no `DISPLAY` is set and Xvfb is installed, agent-browser starts a private virtual display automatically (opt out with `AGENT_BROWSER_NO_XVFB=1`).
|
|
1175
|
+
|
|
1176
|
+
Verify the full pipeline (adapter, render pass, and screenshot capture) with:
|
|
1177
|
+
|
|
1178
|
+
```bash
|
|
1179
|
+
agent-browser doctor --webgpu
|
|
1180
|
+
```
|
|
1181
|
+
|
|
1182
|
+
Notes for WebGPU pages:
|
|
1183
|
+
|
|
1184
|
+
- WebGPU only exists in secure contexts (`https://`, `http://localhost`, or `file://`).
|
|
1185
|
+
- three.js `WebGPURenderer` initializes asynchronously and silently falls back to WebGL2 when no adapter is available — wait for the app to render its first frame before taking a screenshot.
|
|
1186
|
+
- To prefer a real GPU on Linux instead of SwiftShader, override both the Vulkan driver and the adapter with `--args "--use-vulkan=native,--use-webgpu-adapter=default"` (user args win over the preset; `--use-webgpu-adapter` alone still enumerates only SwiftShader).
|
|
1187
|
+
|
|
1188
|
+
See the [WebGPU docs page](https://agent-browser.dev/webgpu) for the full platform matrix and container recipe.
|
|
1189
|
+
|
|
1153
1190
|
## Authenticated Sessions
|
|
1154
1191
|
|
|
1155
1192
|
Use `--headers` to set HTTP headers for a specific origin, enabling authentication without login flows:
|
|
@@ -1217,10 +1254,23 @@ const result = await withAgentBrowserSandbox(async (sandbox) => {
|
|
|
1217
1254
|
});
|
|
1218
1255
|
```
|
|
1219
1256
|
|
|
1220
|
-
Install `@agent-browser/sandbox` and `@vercel/sandbox` in the consuming app. See the [sandbox helper example](examples/sandbox/) for minimal
|
|
1257
|
+
Install `@agent-browser/sandbox` and `@vercel/sandbox` in the consuming app. See the [sandbox helper example](examples/sandbox/) for minimal Vercel Sandbox usage, or the [environments example](examples/environments/) for a full UI demo with a deploy-to-Vercel button.
|
|
1221
1258
|
|
|
1222
1259
|
Fresh Vercel and Eve sandboxes install Chromium system dependencies by default. Pass `installSystemDependencies: false` only when your sandbox image already includes those libraries.
|
|
1223
1260
|
|
|
1261
|
+
### Eve extension
|
|
1262
|
+
|
|
1263
|
+
Give an [eve](https://eve.dev) agent the full browser tool set by mounting the [`@agent-browser/eve`](packages/@agent-browser/eve/) extension:
|
|
1264
|
+
|
|
1265
|
+
```typescript
|
|
1266
|
+
// agent/extensions/browser.ts
|
|
1267
|
+
import browser from "@agent-browser/eve";
|
|
1268
|
+
|
|
1269
|
+
export default browser({});
|
|
1270
|
+
```
|
|
1271
|
+
|
|
1272
|
+
This composes ~20 namespaced tools into the agent — `browser__navigate`, `browser__snapshot`, `browser__click`, `browser__fill`, `browser__find`, `browser__screenshot`, and more — all running agent-browser inside the agent's sandbox. agent-browser installs automatically on first use; pre-install it in `agent/sandbox.ts` with the `@agent-browser/eve/sandbox` helpers to bake the cost into the sandbox template instead. Configuration (domain allowlists, output limits, session naming) and per-tool overrides are covered in the [package README](packages/@agent-browser/eve/README.md), and the [eve example](examples/eve/) is a complete app with the extension mounted.
|
|
1273
|
+
|
|
1224
1274
|
### Serverless (AWS Lambda)
|
|
1225
1275
|
|
|
1226
1276
|
```typescript
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/package.json
CHANGED
|
@@ -44,6 +44,11 @@ if (!sandboxVersionMatch) {
|
|
|
44
44
|
|
|
45
45
|
const sandboxRuntimeVersion = sandboxVersionMatch[1];
|
|
46
46
|
|
|
47
|
+
// Read Eve package version
|
|
48
|
+
const evePkg = JSON.parse(readFileSync(join(rootDir, 'packages/@agent-browser/eve/package.json'), 'utf-8'));
|
|
49
|
+
const eveVersion = evePkg.version;
|
|
50
|
+
const eveSandboxDependency = evePkg.dependencies?.['@agent-browser/sandbox'];
|
|
51
|
+
|
|
47
52
|
const mismatches = [];
|
|
48
53
|
if (packageVersion !== cargoVersion) {
|
|
49
54
|
mismatches.push(` cli/Cargo.toml: ${cargoVersion}`);
|
|
@@ -57,6 +62,12 @@ if (packageVersion !== sandboxVersion) {
|
|
|
57
62
|
if (packageVersion !== sandboxRuntimeVersion) {
|
|
58
63
|
mismatches.push(` packages/@agent-browser/sandbox/src/version.ts: ${sandboxRuntimeVersion}`);
|
|
59
64
|
}
|
|
65
|
+
if (packageVersion !== eveVersion) {
|
|
66
|
+
mismatches.push(` packages/@agent-browser/eve/package.json: ${eveVersion}`);
|
|
67
|
+
}
|
|
68
|
+
if (eveSandboxDependency !== 'workspace:^') {
|
|
69
|
+
mismatches.push(` packages/@agent-browser/eve dependency @agent-browser/sandbox: ${eveSandboxDependency}`);
|
|
70
|
+
}
|
|
60
71
|
|
|
61
72
|
if (mismatches.length > 0) {
|
|
62
73
|
console.error('Version mismatch detected!');
|
package/scripts/sync-version.js
CHANGED
|
@@ -68,6 +68,20 @@ if (sandboxPkg.version !== version) {
|
|
|
68
68
|
console.log(` packages/@agent-browser/sandbox/package.json already up to date`);
|
|
69
69
|
}
|
|
70
70
|
|
|
71
|
+
// Update packages/@agent-browser/eve/package.json (version + workspace sandbox dependency)
|
|
72
|
+
const evePkgPath = join(rootDir, "packages", "@agent-browser", "eve", "package.json");
|
|
73
|
+
const evePkg = JSON.parse(readFileSync(evePkgPath, "utf-8"));
|
|
74
|
+
const eveSandboxDependency = "workspace:^";
|
|
75
|
+
if (evePkg.version !== version || evePkg.dependencies["@agent-browser/sandbox"] !== eveSandboxDependency) {
|
|
76
|
+
const oldVersion = evePkg.version;
|
|
77
|
+
evePkg.version = version;
|
|
78
|
+
evePkg.dependencies["@agent-browser/sandbox"] = eveSandboxDependency;
|
|
79
|
+
writeFileSync(evePkgPath, JSON.stringify(evePkg, null, 2) + "\n");
|
|
80
|
+
console.log(` Updated packages/@agent-browser/eve/package.json: ${oldVersion} -> ${version}`);
|
|
81
|
+
} else {
|
|
82
|
+
console.log(` packages/@agent-browser/eve/package.json already up to date`);
|
|
83
|
+
}
|
|
84
|
+
|
|
71
85
|
// Update package runtime version constant
|
|
72
86
|
const sandboxVersionPath = join(
|
|
73
87
|
rootDir,
|
package/skill-data/core/SKILL.md
CHANGED
|
@@ -58,7 +58,11 @@ agent-browser mcp --tools all
|
|
|
58
58
|
agent-browser mcp --tools core,network,react
|
|
59
59
|
```
|
|
60
60
|
|
|
61
|
-
Configure the MCP client to launch `agent-browser` with `["mcp"]`. The server defaults to MCP protocol 2025-11-25 and accepts older supported client protocol versions during initialization. The default tools profile is `core`, which keeps MCP context small for everyday browser automation. Use `--tools all` for the full typed CLI parity surface, or combine profiles with commas, such as `--tools core,network,react`. Profiles are `core`, `network`, `state`, `debug`, `tabs`, `react`, `mobile`, and `all`; the `debug` profile includes plugin registry and command.run tools. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the tool `session` argument or `AGENT_BROWSER_SESSION` to isolate browser sessions.
|
|
61
|
+
Configure the MCP client to launch `agent-browser` with `["mcp"]`. The server defaults to MCP protocol 2025-11-25 and accepts older supported client protocol versions during initialization. The default tools profile is `core`, which keeps MCP context small for everyday browser automation. Use `--tools all` for the full typed CLI parity surface, or combine profiles with commas, such as `--tools core,network,react`. Profiles are `core`, `network`, `state`, `debug`, `tabs`, `react`, `mobile`, and `all`; the `debug` profile includes plugin registry and command.run tools. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. The common `allowedDomains` array maps to `--allowed-domains` and activates the same WebRTC containment and launch-mode restrictions. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the tool `session` argument or `AGENT_BROWSER_SESSION` to isolate browser sessions.
|
|
62
|
+
|
|
63
|
+
## Eve agent integration
|
|
64
|
+
|
|
65
|
+
For Eve agents, mount the `@agent-browser/eve` extension instead of hand-writing browser tools. It adds namespaced tools such as `browser__navigate`, `browser__snapshot`, `browser__click`, `browser__fill`, `browser__find`, and `browser__screenshot`, all backed by agent-browser running inside the Eve sandbox. The sandbox bootstrap helpers (`installAgentBrowser`, `agentBrowserRevalidationKey`) ship with the same package under `@agent-browser/eve/sandbox`, so `agent/sandbox.ts` needs no extra dependency.
|
|
62
66
|
|
|
63
67
|
## Reading a page
|
|
64
68
|
|
|
@@ -105,6 +109,8 @@ agent-browser get count ".item" # count matching elements
|
|
|
105
109
|
|
|
106
110
|
Use `read [url]` when you need to consume documentation or other text pages rather than interact with a rendered UI. Omit the URL to read the rendered DOM of the active tab in the current browser session, including browser auth state and client-side updates. Explicit URL reads send `Accept: text/markdown`, try the same URL with `.md` appended when the first response is not markdown, walk ancestor paths toward `/` to find the nearest `llms.txt` for a matching docs link, print markdown/plain text when available, and fall back to readable text extracted from HTML without launching Chrome. Add `--filter <text>` to narrow a page to matching heading sections, `--outline` for compact headings on one page, `--llms index` for a compact nearest-ancestor `llms.txt` link list, and `--llms full` only when you explicitly need `llms-full.txt`. With `--llms` or `--require-md`, omitting the URL uses the active tab URL because those modes depend on HTTP resources. With `--llms` or `--outline`, `--filter <text>` narrows links, sections, or headings. Add `--require-md` when you specifically want to verify markdown negotiation, `--raw` when you need the response body unchanged, and `--json` when you need metadata such as `source` and `contentType`. Global safeguards such as `--allowed-domains`, `--content-boundaries`, and `--max-output` also apply to read fetches and output.
|
|
107
111
|
|
|
112
|
+
For sessions that handle sensitive data, use `--allowed-domains` to restrict navigations and page-initiated network traffic. Supported Chromium sessions also disable `RTCPeerConnection` while the allowlist is active so WebRTC STUN, TURN, and related DNS traffic cannot bypass the HTTP filter. Dedicated and shared workers are guarded with a bootstrap wrapper; if a page CSP forbids that wrapper, the worker fails closed rather than running without the allowlist guard. Pre-existing CDP sessions, auto-connect, Chrome profiles, direct-page provider plugins, agent-browser restore or state-file replay, raw Chrome args that select profiles, restore sessions, or open startup pages, iOS, and Safari reject this option because agent-browser cannot install equivalent containment before page scripts run. This is browser-level containment, not an operating-system firewall; see [Trust boundaries](references/trust-boundaries.md) for deployment guidance.
|
|
113
|
+
|
|
108
114
|
## Interacting
|
|
109
115
|
|
|
110
116
|
```bash
|
|
@@ -228,7 +234,7 @@ SESSION="$(agent-browser session id --scope worktree --prefix my-app)"
|
|
|
228
234
|
agent-browser --session "$SESSION" --restore open https://app.example.com
|
|
229
235
|
```
|
|
230
236
|
|
|
231
|
-
`--restore` with no value uses the current `--session` as the persistence key. Agent skills should prefer this over hand-built state file paths. Use `--restore-save auto` by default so a failed restore does not overwrite the previous known-good state.
|
|
237
|
+
`--restore` with no value uses the current `--session` as the persistence key. Agent skills should prefer this over hand-built state file paths. Use `--restore-save auto` by default so a failed restore does not overwrite the previous known-good state. State is saved on close and also periodically while the browser is open (at most once per `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS`, default 30000), so state survives even if the user closes the browser window by hand.
|
|
232
238
|
|
|
233
239
|
```bash
|
|
234
240
|
agent-browser --session "$SESSION" --restore --restore-check-text Dashboard open https://app.example.com
|
|
@@ -400,6 +406,8 @@ EOF
|
|
|
400
406
|
|
|
401
407
|
**Cross-origin iframe not accessible** Cross-origin iframes that block accessibility tree access are silently skipped. Use `frame "#iframe"` to switch into them explicitly if the parent opts in, otherwise the iframe's contents aren't available via snapshot — fall back to `eval` in the iframe's origin or use the `--headers` flag to satisfy CORS.
|
|
402
408
|
|
|
409
|
+
**WebGPU page renders black in screenshots** Headless Chrome doesn't expose WebGPU by default; three.js `WebGPURenderer` then silently falls back or renders nothing. Relaunch with the `--webgpu` flag, wait for the app's first rendered frame, then screenshot. On Linux install `libvulkan1 mesa-vulkan-drivers` first. If it's still black on Windows/Linux, that's an upstream headless-capture limitation: add `--headed` (needs a logged-in desktop on Windows; on Linux agent-browser starts a private virtual display automatically when Xvfb is installed — never wrap in `xvfb-run`, which kills the display when the CLI exits while the browser lives on). Verify with `agent-browser doctor --webgpu`. See [references/webgpu.md](references/webgpu.md).
|
|
410
|
+
|
|
403
411
|
**Authentication expires mid-workflow** Use `--session <id> --restore` so your session survives browser restarts. Check `agent-browser session info --json` if restore fails. See [references/session-management.md](references/session-management.md) and [references/authentication.md](references/authentication.md).
|
|
404
412
|
|
|
405
413
|
## Global flags worth knowing
|
|
@@ -408,6 +416,7 @@ EOF
|
|
|
408
416
|
--session <name> # isolated browser session
|
|
409
417
|
--json # JSON output (for machine parsing)
|
|
410
418
|
--headed # show the window (default is headless)
|
|
419
|
+
--webgpu # enable WebGPU (software Vulkan on Linux, no GPU needed)
|
|
411
420
|
--auto-connect # connect to an already-running Chrome
|
|
412
421
|
--cdp <port> # connect to a specific CDP port
|
|
413
422
|
--profile <name|path> # use a Chrome profile (login state survives)
|
|
@@ -466,4 +475,5 @@ That pulls in:
|
|
|
466
475
|
- `references/profiling.md` — Chrome DevTools tracing and profiling
|
|
467
476
|
- `references/video-recording.md` — video capture options
|
|
468
477
|
- `references/proxy-support.md` — proxy configuration
|
|
478
|
+
- `references/webgpu.md` — screenshots/video of WebGPU pages (three.js, Babylon.js), Linux/CI setup
|
|
469
479
|
- `templates/*` — starter shell scripts for auth, capture, form automation
|
|
@@ -358,14 +358,15 @@ Common tools include:
|
|
|
358
358
|
- `agent_browser_eval`
|
|
359
359
|
- `agent_browser_close`
|
|
360
360
|
|
|
361
|
-
Tool calls use the same config files and environment variables as the CLI. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the `session` tool argument or `AGENT_BROWSER_SESSION` to isolate browser state.
|
|
361
|
+
Tool calls use the same config files and environment variables as the CLI. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. The common `allowedDomains` array maps to `--allowed-domains` and activates the same WebRTC containment and launch-mode restrictions. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the `session` tool argument or `AGENT_BROWSER_SESSION` to isolate browser state.
|
|
362
362
|
|
|
363
363
|
## Global Options
|
|
364
364
|
|
|
365
365
|
```bash
|
|
366
366
|
agent-browser --session <name> ... # Isolated browser session
|
|
367
367
|
agent-browser --json ... # JSON output for parsing
|
|
368
|
-
agent-browser --headed ... # Show browser window (not headless)
|
|
368
|
+
agent-browser --headed ... # Show browser window (not headless; on displayless Linux an Xvfb display starts automatically)
|
|
369
|
+
agent-browser --webgpu ... # Enable WebGPU (SwiftShader software Vulkan on Linux, no GPU needed)
|
|
369
370
|
agent-browser --cdp <port> ... # Connect via Chrome DevTools Protocol
|
|
370
371
|
agent-browser -p <provider> ... # Browser provider or configured provider plugin
|
|
371
372
|
agent-browser --proxy <url> ... # Use proxy server
|
|
@@ -449,9 +450,12 @@ AGENT_BROWSER_EXTENSIONS="/ext1,/ext2" # Comma-separated extension paths
|
|
|
449
450
|
AGENT_BROWSER_INIT_SCRIPTS="/a.js,/b.js" # Comma-separated init script paths
|
|
450
451
|
AGENT_BROWSER_ENABLE="react-devtools" # Comma-separated built-in init script features
|
|
451
452
|
AGENT_BROWSER_HIDE_SCROLLBARS="false" # Keep native scrollbars visible in headless Chromium screenshots
|
|
453
|
+
AGENT_BROWSER_WEBGPU="1" # Enable the WebGPU launch preset (see references/webgpu.md)
|
|
454
|
+
AGENT_BROWSER_NO_XVFB="1" # Disable automatic Xvfb for headed mode on displayless Linux
|
|
452
455
|
AGENT_BROWSER_PROVIDER="browserbase" # Browser provider or configured provider plugin
|
|
453
456
|
AGENT_BROWSER_STREAM_PORT="9223" # Override WebSocket streaming port (default: OS-assigned)
|
|
454
457
|
AGENT_BROWSER_CONFIG="./agent-browser.json" # Custom config file
|
|
455
458
|
AGENT_BROWSER_CDP="9222" # Connect daemon to CDP port or WebSocket URL
|
|
459
|
+
AGENT_BROWSER_ALLOWED_DOMAINS="example.com" # Restrict network domains; requires a fresh controllable browser context without profile/session startup args, restore/state replay, or direct-page provider plugins
|
|
456
460
|
AGENT_BROWSER_PLUGINS='[{"name":"vault","command":"agent-browser-plugin-vault","capabilities":["credential.read"]},{"name":"stealth","command":"agent-browser-plugin-stealth","capabilities":["launch.mutate"]}]'
|
|
457
461
|
```
|
|
@@ -57,7 +57,7 @@ SESSION="$(agent-browser session id --scope worktree --prefix next-dev-loop)"
|
|
|
57
57
|
agent-browser --session "$SESSION" --restore open https://app.example.com/dashboard
|
|
58
58
|
```
|
|
59
59
|
|
|
60
|
-
State is loaded before navigation and saved on close, daemon shutdown, idle timeout, and compatible relaunch. The default save policy is `--restore-save auto`, which skips auto-save if restore failed or validation failed.
|
|
60
|
+
State is loaded before navigation and saved on close, daemon shutdown, idle timeout, and compatible relaunch. It is also saved periodically while the browser is open (after commands settle, at most once per `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS`, default 30000; set to `0` to save only on close), so a browser window the user closes by hand still leaves a recent save behind. Idle sessions keep saving on the same interval, capturing changes the page makes on its own such as token refreshes. The default save policy is `--restore-save auto`, which skips auto-save if restore failed or validation failed; `never` disables periodic autosave too.
|
|
61
61
|
|
|
62
62
|
```bash
|
|
63
63
|
agent-browser --session "$SESSION" --restore --restore-check-url "**/dashboard" open https://app.example.com/dashboard
|
|
@@ -45,6 +45,7 @@ The hook in particular exposes `window.__REACT_DEVTOOLS_GLOBAL_HOOK__` to every
|
|
|
45
45
|
|
|
46
46
|
## Network interception and automation artifacts
|
|
47
47
|
|
|
48
|
+
- `--allowed-domains` blocks non-allowlisted HTTP traffic, WebSocket and EventSource connections, and `sendBeacon` calls. It also disables `RTCPeerConnection` for supported Chromium sessions because STUN, TURN, and related DNS traffic do not pass through CDP HTTP interception. Dedicated and shared workers are guarded with a bootstrap wrapper; if a page CSP forbids that wrapper, the worker fails closed rather than running without the allowlist guard. Locally launched Chrome additionally disables non-proxied WebRTC UDP. Pre-existing CDP sessions, auto-connect, Chrome profiles, direct-page provider plugins, agent-browser restore or state-file replay, raw Chrome args that select profiles, restore sessions, or open startup pages, iOS, and Safari reject this option because agent-browser cannot install equivalent containment before page scripts run. Treat this as browser-level containment and combine it with host or container egress controls when you need an operating-system security boundary.
|
|
48
49
|
- `network route` can fail or mock requests. Treat it the way you treat production traffic manipulation — confirm with the user before using it against anything other than a dev server.
|
|
49
50
|
- `har start` / `har stop` records every request and response body to disk, including auth headers and bearer tokens. Don't share HAR files without redaction.
|
|
50
51
|
- Screenshots and videos can accidentally capture secrets (auto-filled form fields, visible tokens in URL bars, etc.). Review before sending.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# WebGPU
|
|
2
|
+
|
|
3
|
+
Screenshots and video of WebGPU pages (three.js `WebGPURenderer`, Babylon.js, raw WebGPU) in headless Chrome. Without setup this is a silent failure: the page loads, the screenshot succeeds, and the canvas is black.
|
|
4
|
+
|
|
5
|
+
## Quick start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
agent-browser --webgpu open https://my-webgpu-app.example.com
|
|
9
|
+
# wait for the app to render (see "Timing" below)
|
|
10
|
+
agent-browser screenshot app.png
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`--webgpu` (or `AGENT_BROWSER_WEBGPU=1`, or `"webgpu": true` in agent-browser.json) applies a launch preset:
|
|
14
|
+
|
|
15
|
+
- everywhere: `--enable-unsafe-webgpu` (WebGPU is hidden in headless/blocklisted environments by default)
|
|
16
|
+
- Linux only: `--enable-features=Vulkan --use-angle=vulkan --use-vulkan=swiftshader --use-webgpu-adapter=swiftshader --disable-vulkan-surface` — routes WebGPU through SwiftShader's software Vulkan, so it works with no GPU (containers, CI)
|
|
17
|
+
|
|
18
|
+
macOS uses the hardware Metal backend; Windows uses D3D. Nothing extra to install on either.
|
|
19
|
+
|
|
20
|
+
## Platform matrix (verified)
|
|
21
|
+
|
|
22
|
+
| Platform | WebGPU rendering (headless) | Screenshots of WebGPU canvases |
|
|
23
|
+
|---|---|---|
|
|
24
|
+
| macOS | works | works headless |
|
|
25
|
+
| Windows | works (hardware D3D) | **headless captures black** — use `--headed` on a logged-in desktop |
|
|
26
|
+
| Linux | works (SwiftShader Vulkan) | headless capture not supported upstream — add `--headed` (virtual display starts automatically) |
|
|
27
|
+
|
|
28
|
+
The Windows/Linux screenshot gap is an upstream headless-Chrome limitation: WebGPU canvas *presentation* never reaches the headless compositor, even though rendering itself works (verified by pixel readback). It is not an agent-browser or flag problem — no known flag combination fixes it. Rendering, `eval`-based pixel readbacks, and compute all work headless everywhere.
|
|
29
|
+
|
|
30
|
+
On Linux, `--headed` is all you need even on displayless servers and containers: when no `DISPLAY` is set and Xvfb is installed (`apt-get install -y xvfb`), agent-browser starts a private virtual display for the browser and tears it down with it. Set `AGENT_BROWSER_NO_XVFB=1` to opt out.
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
agent-browser --webgpu --headed open https://my-webgpu-app.example.com
|
|
34
|
+
agent-browser screenshot app.png # real WebGPU pixels, no display hardware
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
On Windows, the session must run headed in a logged-in desktop session (an ssh/Session-0 context is not enough — schedule the launch on the interactive desktop, e.g. `schtasks /IT`, then drive it from anywhere).
|
|
38
|
+
|
|
39
|
+
## Verify the pipeline
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
agent-browser doctor --webgpu
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
This launches a scratch session with the preset and pixel-checks two stages separately:
|
|
46
|
+
|
|
47
|
+
1. **render** — requests an adapter (with retries; a cold Chrome returns null while the GPU process starts), clears an offscreen texture to red through a real render pass, and reads the buffer back. Proves WebGPU works at all, and reports the adapter (e.g. `nvidia ampere`, `apple metal-3`, `google swiftshader`).
|
|
48
|
+
2. **screenshot** — decodes an actual screenshot of a presenting canvas. Proves the capture path. Expected to fail headless on Windows/Linux (see matrix); the failure message says so and points at `--headed`.
|
|
49
|
+
|
|
50
|
+
Add `--headed` (`agent-browser doctor --webgpu --headed`) to validate the capture path itself — on displayless Linux the probe starts its own Xvfb, so both checks should pass.
|
|
51
|
+
|
|
52
|
+
## Linux / containers / CI
|
|
53
|
+
|
|
54
|
+
The SwiftShader Vulkan path needs the system Vulkan loader and Mesa ICD. Without them `requestAdapter()` returns null (or fails with "A valid external Instance reference no longer exists"):
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
apt-get install -y libvulkan1 mesa-vulkan-drivers
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Container recipe (Debian/Ubuntu base; xvfb needed only for the screenshot path). Verified with both Chrome for Testing and Debian's `chromium` package (set `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium` for the latter — useful on ARM64, where Chrome for Testing has no Linux builds):
|
|
61
|
+
|
|
62
|
+
```dockerfile
|
|
63
|
+
FROM node:22-bookworm-slim
|
|
64
|
+
RUN apt-get update && apt-get install -y \
|
|
65
|
+
ca-certificates libvulkan1 mesa-vulkan-drivers xvfb xauth \
|
|
66
|
+
&& rm -rf /var/lib/apt/lists/*
|
|
67
|
+
RUN npm install -g agent-browser \
|
|
68
|
+
&& agent-browser install # downloads Chrome for Testing
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
No real GPU or `/dev/dri` is required. To prefer a real GPU on a Linux machine that has working hardware Vulkan, override both the Vulkan driver and the adapter — the preset pins `--use-vulkan=swiftshader`, so overriding only the adapter still enumerates SwiftShader (user `--args` win over the preset):
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
agent-browser --webgpu --args "--use-vulkan=native,--use-webgpu-adapter=default" open ...
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Secure contexts
|
|
78
|
+
|
|
79
|
+
`navigator.gpu` only exists in secure contexts. `https://`, `http://localhost`, and `file://` qualify; a plain `http://` LAN address or `data:` URL does not — WebGPU will be `undefined` there no matter which flags are set.
|
|
80
|
+
|
|
81
|
+
## Timing: don't screenshot too early
|
|
82
|
+
|
|
83
|
+
WebGPU apps initialize asynchronously. A screenshot taken at `load` captures a blank canvas with no error anywhere. In particular:
|
|
84
|
+
|
|
85
|
+
- **three.js `WebGPURenderer`**: `renderer.init()` is async; the first frame lands only after it resolves. Also note three.js **silently falls back to WebGL2** when it can't get a WebGPU adapter — the page "works" but you're not testing WebGPU (and on old setups the WebGL fallback itself may be black).
|
|
86
|
+
- Wait for an app-specific signal before capturing: a canvas with content, a "ready" DOM marker, or simply a rendered-frame check:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
agent-browser wait --fn "window.__appReady === true"
|
|
90
|
+
# or generically: give the render loop a frame or two
|
|
91
|
+
agent-browser eval "new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))"
|
|
92
|
+
agent-browser screenshot app.png
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
To check which backend a three.js app actually got:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
agent-browser eval "document.querySelector('canvas').getContext('webgpu') ? 'webgpu' : 'webgl-fallback'"
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Reading pixels back inside the page
|
|
102
|
+
|
|
103
|
+
If you `eval` your own WebGPU readback, don't snapshot the canvas (`drawImage(webgpuCanvas, ...)`) — it depends on presentation timing and reads transparent black on Windows even when rendering works. Render to an offscreen texture and read it back deterministically:
|
|
104
|
+
|
|
105
|
+
```js
|
|
106
|
+
const tex = device.createTexture({ size: [w, h], format: 'rgba8unorm',
|
|
107
|
+
usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC });
|
|
108
|
+
// ...render to tex, then:
|
|
109
|
+
encoder.copyTextureToBuffer({ texture: tex }, { buffer, bytesPerRow }, [w, h]);
|
|
110
|
+
device.queue.submit([encoder.finish()]);
|
|
111
|
+
await buffer.mapAsync(GPUMapMode.READ);
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
This works headless on every platform (it's how `doctor --webgpu` proves rendering).
|
|
115
|
+
|
|
116
|
+
## Performance expectations
|
|
117
|
+
|
|
118
|
+
SwiftShader is a CPU rasterizer. Simple scenes render fine; heavy three.js scenes are single-digit FPS. For screenshots that's usually irrelevant; for smooth video capture of complex scenes, use hardware (macOS/Windows, or Linux with `--use-vulkan=native,--use-webgpu-adapter=default` and real Vulkan drivers).
|