agent-browser 0.31.1 → 0.32.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -517,7 +517,7 @@ Common tools include:
517
517
  - `agent_browser_eval`
518
518
  - `agent_browser_close`
519
519
 
520
- Each tool has typed fields such as `url`, `selector`, `text`, `key`, and `session`, so MCP clients show meaningful approval prompts instead of raw command arrays. Each tool also accepts `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally.
520
+ Each tool has typed fields such as `url`, `selector`, `text`, `key`, `session`, and `allowedDomains`, so MCP clients show meaningful approval prompts instead of raw command arrays. The common `allowedDomains` array maps to `--allowed-domains` and activates the same WebRTC containment and launch-mode restrictions. Each tool also accepts `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally.
521
521
 
522
522
  Example MCP client config:
523
523
 
@@ -689,6 +689,8 @@ agent-browser --session "$SESSION" --restore open twitter.com
689
689
  agent-browser --session "$SESSION" --restore --restore-check-text Dashboard open twitter.com
690
690
  ```
691
691
 
692
+ State is saved when the browser closes (explicit `close`, idle timeout, or daemon shutdown) and also periodically while the browser is open, so a browser window you close by hand still leaves a recent save behind. Periodic autosave waits for commands to settle, then saves at most once per `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` (default 30000; set to `0` to save only on close). Idle sessions keep saving on the same interval, so changes the page makes on its own (token refreshes, background requests) are captured too. It respects the `--restore-save` policy.
693
+
692
694
  ### State Encryption
693
695
 
694
696
  Encrypt saved session data at rest with AES-256-GCM:
@@ -705,6 +707,7 @@ agent-browser --session secure --restore open example.com
705
707
  | --------------------------------- | -------------------------------------------------- |
706
708
  | `AGENT_BROWSER_RESTORE` | Auto-save/load state persistence name |
707
709
  | `AGENT_BROWSER_RESTORE_SAVE` | Restore save policy: `auto`, `always`, or `never` |
710
+ | `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS` | Min ms between periodic autosaves (default: 30000, 0 disables) |
708
711
  | `AGENT_BROWSER_NAMESPACE` | Namespace for daemon sockets and restore state |
709
712
  | `AGENT_BROWSER_SESSION_NAME` | Legacy auto-save/load state persistence name |
710
713
  | `AGENT_BROWSER_ENCRYPTION_KEY` | 64-char hex key for AES-256-GCM encryption |
@@ -717,7 +720,7 @@ agent-browser includes security features for safe AI agent deployments. All feat
717
720
  - **Authentication Vault**: Store credentials locally (always encrypted), reference by name. The LLM never sees passwords. `auth login` navigates with `load` and then waits for login form selectors to appear (SPA-friendly, timeout follows the default action timeout). A key is auto-generated at `~/.agent-browser/.encryption-key` if `AGENT_BROWSER_ENCRYPTION_KEY` is not set: `echo "pass" | agent-browser auth save github --url https://github.com/login --username user --password-stdin` then `agent-browser auth login github`
718
721
  - **Plugin System**: Extend agent-browser with external executable plugins. Plugins run out-of-process over the `agent-browser.plugin.v1` stdio JSON protocol and declare capabilities such as `credential.read`, `browser.provider`, `launch.mutate`, or `command.run`.
719
722
  - **Content Boundary Markers**: Wrap page output in delimiters so LLMs can distinguish tool output from untrusted content: `--content-boundaries`
720
- - **Domain Allowlist**: Restrict navigation to trusted domains (wildcards like `*.example.com` also match the bare domain): `--allowed-domains "example.com,*.example.com"`. Sub-resource requests (scripts, images, fetch) and WebSocket/EventSource connections to non-allowed domains are also blocked. Include any CDN domains your target pages depend on (e.g., `*.cdn.example.com`).
723
+ - **Domain Allowlist**: Restrict navigation to trusted domains (wildcards like `*.example.com` also match the bare domain): `--allowed-domains "example.com,*.example.com"`. Sub-resource requests (scripts, images, fetch), WebSocket/EventSource connections, and `sendBeacon` calls to non-allowed domains are blocked. WebRTC peer connections are disabled in supported Chromium sessions while the allowlist is active to prevent STUN, TURN, and DNS traffic from bypassing HTTP interception. Dedicated and shared workers are guarded with a bootstrap wrapper; if a page CSP forbids that wrapper, the worker fails closed rather than running without the allowlist guard. Pre-existing CDP sessions, auto-connect, Chrome profiles, direct-page provider plugins, agent-browser restore or state-file replay, raw Chrome args that select profiles, restore sessions, or open startup pages, iOS, and Safari reject this option because agent-browser cannot install equivalent containment before page scripts run. Include any CDN domains your target pages depend on (e.g., `*.cdn.example.com`).
721
724
  - **Action Policy**: Gate destructive actions with a static policy file: `--action-policy ./policy.json`
722
725
  - **Action Confirmation**: Require explicit approval for sensitive action categories: `--confirm-actions eval,download`
723
726
  - **Output Length Limits**: Prevent context flooding: `--max-output 50000`
@@ -726,7 +729,7 @@ agent-browser includes security features for safe AI agent deployments. All feat
726
729
  | ----------------------------------- | ---------------------------------------- |
727
730
  | `AGENT_BROWSER_CONTENT_BOUNDARIES` | Wrap page output in boundary markers |
728
731
  | `AGENT_BROWSER_MAX_OUTPUT` | Max characters for page output |
729
- | `AGENT_BROWSER_ALLOWED_DOMAINS` | Comma-separated allowed domain patterns |
732
+ | `AGENT_BROWSER_ALLOWED_DOMAINS` | Comma-separated allowed domain patterns; requires a fresh controllable browser context without profile/session startup args, restore/state replay, or direct-page provider plugins |
730
733
  | `AGENT_BROWSER_ACTION_POLICY` | Path to action policy JSON file |
731
734
  | `AGENT_BROWSER_CONFIRM_ACTIONS` | Action categories requiring confirmation |
732
735
  | `AGENT_BROWSER_CONFIRM_INTERACTIVE` | Enable interactive confirmation prompts |
@@ -900,13 +903,14 @@ This is useful for multimodal AI models that can reason about visual layout, unl
900
903
  | `--screenshot-quality <n>` | JPEG quality 0-100 (or `AGENT_BROWSER_SCREENSHOT_QUALITY` env) |
901
904
  | `--screenshot-format <fmt>` | Screenshot format: `png`, `jpeg` (or `AGENT_BROWSER_SCREENSHOT_FORMAT` env) |
902
905
  | `--headed` | Show browser window (not headless) (or `AGENT_BROWSER_HEADED` env) |
906
+ | `--webgpu` | Enable WebGPU; SwiftShader software Vulkan on Linux, no GPU required (or `AGENT_BROWSER_WEBGPU` env) |
903
907
  | `--cdp <port\|url>` | Connect via Chrome DevTools Protocol (port or WebSocket URL) |
904
908
  | `--auto-connect` | Auto-discover and connect to running Chrome (or `AGENT_BROWSER_AUTO_CONNECT` env) |
905
909
  | `--color-scheme <scheme>` | Color scheme: `dark`, `light`, `no-preference` (or `AGENT_BROWSER_COLOR_SCHEME` env) |
906
910
  | `--download-path <path>` | Default download directory (or `AGENT_BROWSER_DOWNLOAD_PATH` env) |
907
911
  | `--content-boundaries` | Wrap page output in boundary markers for LLM safety (or `AGENT_BROWSER_CONTENT_BOUNDARIES` env) |
908
912
  | `--max-output <chars>` | Truncate page output to N characters (or `AGENT_BROWSER_MAX_OUTPUT` env) |
909
- | `--allowed-domains <list>` | Comma-separated allowed domain patterns (or `AGENT_BROWSER_ALLOWED_DOMAINS` env) |
913
+ | `--allowed-domains <list>` | Comma-separated allowed domain patterns; also disables WebRTC peer connections in supported Chromium sessions and rejects CDP, auto-connect, Chrome profiles, restore/state replay, direct-page provider plugins, unsafe startup `--args`, iOS, and Safari (or `AGENT_BROWSER_ALLOWED_DOMAINS` env) |
910
914
  | `--action-policy <path>` | Path to action policy JSON file (or `AGENT_BROWSER_ACTION_POLICY` env) |
911
915
  | `--confirm-actions <list>` | Action categories requiring confirmation (or `AGENT_BROWSER_CONFIRM_ACTIONS` env) |
912
916
  | `--confirm-interactive` | Interactive confirmation prompts; auto-denies if stdin is not a TTY (or `AGENT_BROWSER_CONFIRM_INTERACTIVE` env) |
@@ -1148,8 +1152,41 @@ agent-browser open example.com --headed
1148
1152
 
1149
1153
  This opens a visible browser window instead of running headless.
1150
1154
 
1155
+ On Linux hosts with no display (servers, containers), `--headed` still works: when `DISPLAY` is unset and Xvfb is installed, agent-browser starts a private virtual display for the browser and cleans it up on close (opt out with `AGENT_BROWSER_NO_XVFB=1`). Needed for [WebGPU screenshots](#webgpu), and useful for extensions that misbehave headless.
1156
+
1151
1157
  > **Note:** Browser extensions work in both headed and headless mode (Chrome's `--headless=new`).
1152
1158
 
1159
+ ## WebGPU
1160
+
1161
+ Headless Chrome does not expose WebGPU by default, so pages using it (three.js `WebGPURenderer`, Babylon.js, etc.) silently render black. The `--webgpu` flag enables a launch preset that makes WebGPU work, including in GPU-less containers and CI:
1162
+
1163
+ ```bash
1164
+ agent-browser --webgpu open https://my-webgpu-app.example.com
1165
+ agent-browser screenshot app.png
1166
+ ```
1167
+
1168
+ On macOS and Windows this uses the hardware Metal/D3D backend. On Linux it routes WebGPU through SwiftShader's software Vulkan (no GPU needed), which requires the system Vulkan loader and Mesa ICD:
1169
+
1170
+ ```bash
1171
+ apt-get install -y libvulkan1 mesa-vulkan-drivers
1172
+ ```
1173
+
1174
+ One upstream caveat: headless Chrome cannot capture WebGPU canvas presentation in screenshots on Windows and Linux (rendering and in-page readbacks work; the capture is black). Screenshots of WebGPU pages work headless on macOS; on Windows run `--headed` in a logged-in desktop session; on Linux just add `--headed` — when no `DISPLAY` is set and Xvfb is installed, agent-browser starts a private virtual display automatically (opt out with `AGENT_BROWSER_NO_XVFB=1`).
1175
+
1176
+ Verify the full pipeline (adapter, render pass, and screenshot capture) with:
1177
+
1178
+ ```bash
1179
+ agent-browser doctor --webgpu
1180
+ ```
1181
+
1182
+ Notes for WebGPU pages:
1183
+
1184
+ - WebGPU only exists in secure contexts (`https://`, `http://localhost`, or `file://`).
1185
+ - three.js `WebGPURenderer` initializes asynchronously and silently falls back to WebGL2 when no adapter is available — wait for the app to render its first frame before taking a screenshot.
1186
+ - To prefer a real GPU on Linux instead of SwiftShader, override both the Vulkan driver and the adapter with `--args "--use-vulkan=native,--use-webgpu-adapter=default"` (user args win over the preset; `--use-webgpu-adapter` alone still enumerates only SwiftShader).
1187
+
1188
+ See the [WebGPU docs page](https://agent-browser.dev/webgpu) for the full platform matrix and container recipe.
1189
+
1153
1190
  ## Authenticated Sessions
1154
1191
 
1155
1192
  Use `--headers` to set HTTP headers for a specific origin, enabling authentication without login flows:
@@ -1217,10 +1254,23 @@ const result = await withAgentBrowserSandbox(async (sandbox) => {
1217
1254
  });
1218
1255
  ```
1219
1256
 
1220
- Install `@agent-browser/sandbox` and `@vercel/sandbox` in the consuming app. See the [sandbox helper example](examples/sandbox/) for minimal Eve and Vercel Sandbox usage, or the [environments example](examples/environments/) for a full UI demo with a deploy-to-Vercel button.
1257
+ Install `@agent-browser/sandbox` and `@vercel/sandbox` in the consuming app. See the [sandbox helper example](examples/sandbox/) for minimal Vercel Sandbox usage, or the [environments example](examples/environments/) for a full UI demo with a deploy-to-Vercel button.
1221
1258
 
1222
1259
  Fresh Vercel and Eve sandboxes install Chromium system dependencies by default. Pass `installSystemDependencies: false` only when your sandbox image already includes those libraries.
1223
1260
 
1261
+ ### Eve extension
1262
+
1263
+ Give an [eve](https://eve.dev) agent the full browser tool set by mounting the [`@agent-browser/eve`](packages/@agent-browser/eve/) extension:
1264
+
1265
+ ```typescript
1266
+ // agent/extensions/browser.ts
1267
+ import browser from "@agent-browser/eve";
1268
+
1269
+ export default browser({});
1270
+ ```
1271
+
1272
+ This composes ~20 namespaced tools into the agent — `browser__navigate`, `browser__snapshot`, `browser__click`, `browser__fill`, `browser__find`, `browser__screenshot`, and more — all running agent-browser inside the agent's sandbox. agent-browser installs automatically on first use; pre-install it in `agent/sandbox.ts` with the `@agent-browser/eve/sandbox` helpers to bake the cost into the sandbox template instead. Configuration (domain allowlists, output limits, session naming) and per-tool overrides are covered in the [package README](packages/@agent-browser/eve/README.md), and the [eve example](examples/eve/) is a complete app with the extension mounted.
1273
+
1224
1274
  ### Serverless (AWS Lambda)
1225
1275
 
1226
1276
  ```typescript
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
Binary file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-browser",
3
- "version": "0.31.1",
3
+ "version": "0.32.0",
4
4
  "description": "Browser automation CLI for AI agents",
5
5
  "type": "module",
6
6
  "engines": {
@@ -44,6 +44,11 @@ if (!sandboxVersionMatch) {
44
44
 
45
45
  const sandboxRuntimeVersion = sandboxVersionMatch[1];
46
46
 
47
+ // Read Eve package version
48
+ const evePkg = JSON.parse(readFileSync(join(rootDir, 'packages/@agent-browser/eve/package.json'), 'utf-8'));
49
+ const eveVersion = evePkg.version;
50
+ const eveSandboxDependency = evePkg.dependencies?.['@agent-browser/sandbox'];
51
+
47
52
  const mismatches = [];
48
53
  if (packageVersion !== cargoVersion) {
49
54
  mismatches.push(` cli/Cargo.toml: ${cargoVersion}`);
@@ -57,6 +62,12 @@ if (packageVersion !== sandboxVersion) {
57
62
  if (packageVersion !== sandboxRuntimeVersion) {
58
63
  mismatches.push(` packages/@agent-browser/sandbox/src/version.ts: ${sandboxRuntimeVersion}`);
59
64
  }
65
+ if (packageVersion !== eveVersion) {
66
+ mismatches.push(` packages/@agent-browser/eve/package.json: ${eveVersion}`);
67
+ }
68
+ if (eveSandboxDependency !== 'workspace:^') {
69
+ mismatches.push(` packages/@agent-browser/eve dependency @agent-browser/sandbox: ${eveSandboxDependency}`);
70
+ }
60
71
 
61
72
  if (mismatches.length > 0) {
62
73
  console.error('Version mismatch detected!');
@@ -68,6 +68,20 @@ if (sandboxPkg.version !== version) {
68
68
  console.log(` packages/@agent-browser/sandbox/package.json already up to date`);
69
69
  }
70
70
 
71
+ // Update packages/@agent-browser/eve/package.json (version + workspace sandbox dependency)
72
+ const evePkgPath = join(rootDir, "packages", "@agent-browser", "eve", "package.json");
73
+ const evePkg = JSON.parse(readFileSync(evePkgPath, "utf-8"));
74
+ const eveSandboxDependency = "workspace:^";
75
+ if (evePkg.version !== version || evePkg.dependencies["@agent-browser/sandbox"] !== eveSandboxDependency) {
76
+ const oldVersion = evePkg.version;
77
+ evePkg.version = version;
78
+ evePkg.dependencies["@agent-browser/sandbox"] = eveSandboxDependency;
79
+ writeFileSync(evePkgPath, JSON.stringify(evePkg, null, 2) + "\n");
80
+ console.log(` Updated packages/@agent-browser/eve/package.json: ${oldVersion} -> ${version}`);
81
+ } else {
82
+ console.log(` packages/@agent-browser/eve/package.json already up to date`);
83
+ }
84
+
71
85
  // Update package runtime version constant
72
86
  const sandboxVersionPath = join(
73
87
  rootDir,
@@ -58,7 +58,11 @@ agent-browser mcp --tools all
58
58
  agent-browser mcp --tools core,network,react
59
59
  ```
60
60
 
61
- Configure the MCP client to launch `agent-browser` with `["mcp"]`. The server defaults to MCP protocol 2025-11-25 and accepts older supported client protocol versions during initialization. The default tools profile is `core`, which keeps MCP context small for everyday browser automation. Use `--tools all` for the full typed CLI parity surface, or combine profiles with commas, such as `--tools core,network,react`. Profiles are `core`, `network`, `state`, `debug`, `tabs`, `react`, `mobile`, and `all`; the `debug` profile includes plugin registry and command.run tools. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the tool `session` argument or `AGENT_BROWSER_SESSION` to isolate browser sessions.
61
+ Configure the MCP client to launch `agent-browser` with `["mcp"]`. The server defaults to MCP protocol 2025-11-25 and accepts older supported client protocol versions during initialization. The default tools profile is `core`, which keeps MCP context small for everyday browser automation. Use `--tools all` for the full typed CLI parity surface, or combine profiles with commas, such as `--tools core,network,react`. Profiles are `core`, `network`, `state`, `debug`, `tabs`, `react`, `mobile`, and `all`; the `debug` profile includes plugin registry and command.run tools. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. The common `allowedDomains` array maps to `--allowed-domains` and activates the same WebRTC containment and launch-mode restrictions. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the tool `session` argument or `AGENT_BROWSER_SESSION` to isolate browser sessions.
62
+
63
+ ## Eve agent integration
64
+
65
+ For Eve agents, mount the `@agent-browser/eve` extension instead of hand-writing browser tools. It adds namespaced tools such as `browser__navigate`, `browser__snapshot`, `browser__click`, `browser__fill`, `browser__find`, and `browser__screenshot`, all backed by agent-browser running inside the Eve sandbox. The sandbox bootstrap helpers (`installAgentBrowser`, `agentBrowserRevalidationKey`) ship with the same package under `@agent-browser/eve/sandbox`, so `agent/sandbox.ts` needs no extra dependency.
62
66
 
63
67
  ## Reading a page
64
68
 
@@ -105,6 +109,8 @@ agent-browser get count ".item" # count matching elements
105
109
 
106
110
  Use `read [url]` when you need to consume documentation or other text pages rather than interact with a rendered UI. Omit the URL to read the rendered DOM of the active tab in the current browser session, including browser auth state and client-side updates. Explicit URL reads send `Accept: text/markdown`, try the same URL with `.md` appended when the first response is not markdown, walk ancestor paths toward `/` to find the nearest `llms.txt` for a matching docs link, print markdown/plain text when available, and fall back to readable text extracted from HTML without launching Chrome. Add `--filter <text>` to narrow a page to matching heading sections, `--outline` for compact headings on one page, `--llms index` for a compact nearest-ancestor `llms.txt` link list, and `--llms full` only when you explicitly need `llms-full.txt`. With `--llms` or `--require-md`, omitting the URL uses the active tab URL because those modes depend on HTTP resources. With `--llms` or `--outline`, `--filter <text>` narrows links, sections, or headings. Add `--require-md` when you specifically want to verify markdown negotiation, `--raw` when you need the response body unchanged, and `--json` when you need metadata such as `source` and `contentType`. Global safeguards such as `--allowed-domains`, `--content-boundaries`, and `--max-output` also apply to read fetches and output.
107
111
 
112
+ For sessions that handle sensitive data, use `--allowed-domains` to restrict navigations and page-initiated network traffic. Supported Chromium sessions also disable `RTCPeerConnection` while the allowlist is active so WebRTC STUN, TURN, and related DNS traffic cannot bypass the HTTP filter. Dedicated and shared workers are guarded with a bootstrap wrapper; if a page CSP forbids that wrapper, the worker fails closed rather than running without the allowlist guard. Pre-existing CDP sessions, auto-connect, Chrome profiles, direct-page provider plugins, agent-browser restore or state-file replay, raw Chrome args that select profiles, restore sessions, or open startup pages, iOS, and Safari reject this option because agent-browser cannot install equivalent containment before page scripts run. This is browser-level containment, not an operating-system firewall; see [Trust boundaries](references/trust-boundaries.md) for deployment guidance.
113
+
108
114
  ## Interacting
109
115
 
110
116
  ```bash
@@ -228,7 +234,7 @@ SESSION="$(agent-browser session id --scope worktree --prefix my-app)"
228
234
  agent-browser --session "$SESSION" --restore open https://app.example.com
229
235
  ```
230
236
 
231
- `--restore` with no value uses the current `--session` as the persistence key. Agent skills should prefer this over hand-built state file paths. Use `--restore-save auto` by default so a failed restore does not overwrite the previous known-good state.
237
+ `--restore` with no value uses the current `--session` as the persistence key. Agent skills should prefer this over hand-built state file paths. Use `--restore-save auto` by default so a failed restore does not overwrite the previous known-good state. State is saved on close and also periodically while the browser is open (at most once per `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS`, default 30000), so state survives even if the user closes the browser window by hand.
232
238
 
233
239
  ```bash
234
240
  agent-browser --session "$SESSION" --restore --restore-check-text Dashboard open https://app.example.com
@@ -400,6 +406,8 @@ EOF
400
406
 
401
407
  **Cross-origin iframe not accessible** Cross-origin iframes that block accessibility tree access are silently skipped. Use `frame "#iframe"` to switch into them explicitly if the parent opts in, otherwise the iframe's contents aren't available via snapshot — fall back to `eval` in the iframe's origin or use the `--headers` flag to satisfy CORS.
402
408
 
409
+ **WebGPU page renders black in screenshots** Headless Chrome doesn't expose WebGPU by default; three.js `WebGPURenderer` then silently falls back or renders nothing. Relaunch with the `--webgpu` flag, wait for the app's first rendered frame, then screenshot. On Linux install `libvulkan1 mesa-vulkan-drivers` first. If it's still black on Windows/Linux, that's an upstream headless-capture limitation: add `--headed` (needs a logged-in desktop on Windows; on Linux agent-browser starts a private virtual display automatically when Xvfb is installed — never wrap in `xvfb-run`, which kills the display when the CLI exits while the browser lives on). Verify with `agent-browser doctor --webgpu`. See [references/webgpu.md](references/webgpu.md).
410
+
403
411
  **Authentication expires mid-workflow** Use `--session <id> --restore` so your session survives browser restarts. Check `agent-browser session info --json` if restore fails. See [references/session-management.md](references/session-management.md) and [references/authentication.md](references/authentication.md).
404
412
 
405
413
  ## Global flags worth knowing
@@ -408,6 +416,7 @@ EOF
408
416
  --session <name> # isolated browser session
409
417
  --json # JSON output (for machine parsing)
410
418
  --headed # show the window (default is headless)
419
+ --webgpu # enable WebGPU (software Vulkan on Linux, no GPU needed)
411
420
  --auto-connect # connect to an already-running Chrome
412
421
  --cdp <port> # connect to a specific CDP port
413
422
  --profile <name|path> # use a Chrome profile (login state survives)
@@ -466,4 +475,5 @@ That pulls in:
466
475
  - `references/profiling.md` — Chrome DevTools tracing and profiling
467
476
  - `references/video-recording.md` — video capture options
468
477
  - `references/proxy-support.md` — proxy configuration
478
+ - `references/webgpu.md` — screenshots/video of WebGPU pages (three.js, Babylon.js), Linux/CI setup
469
479
  - `templates/*` — starter shell scripts for auth, capture, form automation
@@ -358,14 +358,15 @@ Common tools include:
358
358
  - `agent_browser_eval`
359
359
  - `agent_browser_close`
360
360
 
361
- Tool calls use the same config files and environment variables as the CLI. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the `session` tool argument or `AGENT_BROWSER_SESSION` to isolate browser state.
361
+ Tool calls use the same config files and environment variables as the CLI. Each tool accepts typed arguments plus `extraArgs` for advanced CLI flags and exact CLI parity. The common `allowedDomains` array maps to `--allowed-domains` and activates the same WebRTC containment and launch-mode restrictions. Tool discovery is paginated and includes read-only/open-world annotations so modern MCP clients can load the large typed surface incrementally. Use the `session` tool argument or `AGENT_BROWSER_SESSION` to isolate browser state.
362
362
 
363
363
  ## Global Options
364
364
 
365
365
  ```bash
366
366
  agent-browser --session <name> ... # Isolated browser session
367
367
  agent-browser --json ... # JSON output for parsing
368
- agent-browser --headed ... # Show browser window (not headless)
368
+ agent-browser --headed ... # Show browser window (not headless; on displayless Linux an Xvfb display starts automatically)
369
+ agent-browser --webgpu ... # Enable WebGPU (SwiftShader software Vulkan on Linux, no GPU needed)
369
370
  agent-browser --cdp <port> ... # Connect via Chrome DevTools Protocol
370
371
  agent-browser -p <provider> ... # Browser provider or configured provider plugin
371
372
  agent-browser --proxy <url> ... # Use proxy server
@@ -449,9 +450,12 @@ AGENT_BROWSER_EXTENSIONS="/ext1,/ext2" # Comma-separated extension paths
449
450
  AGENT_BROWSER_INIT_SCRIPTS="/a.js,/b.js" # Comma-separated init script paths
450
451
  AGENT_BROWSER_ENABLE="react-devtools" # Comma-separated built-in init script features
451
452
  AGENT_BROWSER_HIDE_SCROLLBARS="false" # Keep native scrollbars visible in headless Chromium screenshots
453
+ AGENT_BROWSER_WEBGPU="1" # Enable the WebGPU launch preset (see references/webgpu.md)
454
+ AGENT_BROWSER_NO_XVFB="1" # Disable automatic Xvfb for headed mode on displayless Linux
452
455
  AGENT_BROWSER_PROVIDER="browserbase" # Browser provider or configured provider plugin
453
456
  AGENT_BROWSER_STREAM_PORT="9223" # Override WebSocket streaming port (default: OS-assigned)
454
457
  AGENT_BROWSER_CONFIG="./agent-browser.json" # Custom config file
455
458
  AGENT_BROWSER_CDP="9222" # Connect daemon to CDP port or WebSocket URL
459
+ AGENT_BROWSER_ALLOWED_DOMAINS="example.com" # Restrict network domains; requires a fresh controllable browser context without profile/session startup args, restore/state replay, or direct-page provider plugins
456
460
  AGENT_BROWSER_PLUGINS='[{"name":"vault","command":"agent-browser-plugin-vault","capabilities":["credential.read"]},{"name":"stealth","command":"agent-browser-plugin-stealth","capabilities":["launch.mutate"]}]'
457
461
  ```
@@ -57,7 +57,7 @@ SESSION="$(agent-browser session id --scope worktree --prefix next-dev-loop)"
57
57
  agent-browser --session "$SESSION" --restore open https://app.example.com/dashboard
58
58
  ```
59
59
 
60
- State is loaded before navigation and saved on close, daemon shutdown, idle timeout, and compatible relaunch. The default save policy is `--restore-save auto`, which skips auto-save if restore failed or validation failed.
60
+ State is loaded before navigation and saved on close, daemon shutdown, idle timeout, and compatible relaunch. It is also saved periodically while the browser is open (after commands settle, at most once per `AGENT_BROWSER_AUTOSAVE_INTERVAL_MS`, default 30000; set to `0` to save only on close), so a browser window the user closes by hand still leaves a recent save behind. Idle sessions keep saving on the same interval, capturing changes the page makes on its own such as token refreshes. The default save policy is `--restore-save auto`, which skips auto-save if restore failed or validation failed; `never` disables periodic autosave too.
61
61
 
62
62
  ```bash
63
63
  agent-browser --session "$SESSION" --restore --restore-check-url "**/dashboard" open https://app.example.com/dashboard
@@ -45,6 +45,7 @@ The hook in particular exposes `window.__REACT_DEVTOOLS_GLOBAL_HOOK__` to every
45
45
 
46
46
  ## Network interception and automation artifacts
47
47
 
48
+ - `--allowed-domains` blocks non-allowlisted HTTP traffic, WebSocket and EventSource connections, and `sendBeacon` calls. It also disables `RTCPeerConnection` for supported Chromium sessions because STUN, TURN, and related DNS traffic do not pass through CDP HTTP interception. Dedicated and shared workers are guarded with a bootstrap wrapper; if a page CSP forbids that wrapper, the worker fails closed rather than running without the allowlist guard. Locally launched Chrome additionally disables non-proxied WebRTC UDP. Pre-existing CDP sessions, auto-connect, Chrome profiles, direct-page provider plugins, agent-browser restore or state-file replay, raw Chrome args that select profiles, restore sessions, or open startup pages, iOS, and Safari reject this option because agent-browser cannot install equivalent containment before page scripts run. Treat this as browser-level containment and combine it with host or container egress controls when you need an operating-system security boundary.
48
49
  - `network route` can fail or mock requests. Treat it the way you treat production traffic manipulation — confirm with the user before using it against anything other than a dev server.
49
50
  - `har start` / `har stop` records every request and response body to disk, including auth headers and bearer tokens. Don't share HAR files without redaction.
50
51
  - Screenshots and videos can accidentally capture secrets (auto-filled form fields, visible tokens in URL bars, etc.). Review before sending.
@@ -0,0 +1,118 @@
1
+ # WebGPU
2
+
3
+ Screenshots and video of WebGPU pages (three.js `WebGPURenderer`, Babylon.js, raw WebGPU) in headless Chrome. Without setup this is a silent failure: the page loads, the screenshot succeeds, and the canvas is black.
4
+
5
+ ## Quick start
6
+
7
+ ```bash
8
+ agent-browser --webgpu open https://my-webgpu-app.example.com
9
+ # wait for the app to render (see "Timing" below)
10
+ agent-browser screenshot app.png
11
+ ```
12
+
13
+ `--webgpu` (or `AGENT_BROWSER_WEBGPU=1`, or `"webgpu": true` in agent-browser.json) applies a launch preset:
14
+
15
+ - everywhere: `--enable-unsafe-webgpu` (WebGPU is hidden in headless/blocklisted environments by default)
16
+ - Linux only: `--enable-features=Vulkan --use-angle=vulkan --use-vulkan=swiftshader --use-webgpu-adapter=swiftshader --disable-vulkan-surface` — routes WebGPU through SwiftShader's software Vulkan, so it works with no GPU (containers, CI)
17
+
18
+ macOS uses the hardware Metal backend; Windows uses D3D. Nothing extra to install on either.
19
+
20
+ ## Platform matrix (verified)
21
+
22
+ | Platform | WebGPU rendering (headless) | Screenshots of WebGPU canvases |
23
+ |---|---|---|
24
+ | macOS | works | works headless |
25
+ | Windows | works (hardware D3D) | **headless captures black** — use `--headed` on a logged-in desktop |
26
+ | Linux | works (SwiftShader Vulkan) | headless capture not supported upstream — add `--headed` (virtual display starts automatically) |
27
+
28
+ The Windows/Linux screenshot gap is an upstream headless-Chrome limitation: WebGPU canvas *presentation* never reaches the headless compositor, even though rendering itself works (verified by pixel readback). It is not an agent-browser or flag problem — no known flag combination fixes it. Rendering, `eval`-based pixel readbacks, and compute all work headless everywhere.
29
+
30
+ On Linux, `--headed` is all you need even on displayless servers and containers: when no `DISPLAY` is set and Xvfb is installed (`apt-get install -y xvfb`), agent-browser starts a private virtual display for the browser and tears it down with it. Set `AGENT_BROWSER_NO_XVFB=1` to opt out.
31
+
32
+ ```bash
33
+ agent-browser --webgpu --headed open https://my-webgpu-app.example.com
34
+ agent-browser screenshot app.png # real WebGPU pixels, no display hardware
35
+ ```
36
+
37
+ On Windows, the session must run headed in a logged-in desktop session (an ssh/Session-0 context is not enough — schedule the launch on the interactive desktop, e.g. `schtasks /IT`, then drive it from anywhere).
38
+
39
+ ## Verify the pipeline
40
+
41
+ ```bash
42
+ agent-browser doctor --webgpu
43
+ ```
44
+
45
+ This launches a scratch session with the preset and pixel-checks two stages separately:
46
+
47
+ 1. **render** — requests an adapter (with retries; a cold Chrome returns null while the GPU process starts), clears an offscreen texture to red through a real render pass, and reads the buffer back. Proves WebGPU works at all, and reports the adapter (e.g. `nvidia ampere`, `apple metal-3`, `google swiftshader`).
48
+ 2. **screenshot** — decodes an actual screenshot of a presenting canvas. Proves the capture path. Expected to fail headless on Windows/Linux (see matrix); the failure message says so and points at `--headed`.
49
+
50
+ Add `--headed` (`agent-browser doctor --webgpu --headed`) to validate the capture path itself — on displayless Linux the probe starts its own Xvfb, so both checks should pass.
51
+
52
+ ## Linux / containers / CI
53
+
54
+ The SwiftShader Vulkan path needs the system Vulkan loader and Mesa ICD. Without them `requestAdapter()` returns null (or fails with "A valid external Instance reference no longer exists"):
55
+
56
+ ```bash
57
+ apt-get install -y libvulkan1 mesa-vulkan-drivers
58
+ ```
59
+
60
+ Container recipe (Debian/Ubuntu base; xvfb needed only for the screenshot path). Verified with both Chrome for Testing and Debian's `chromium` package (set `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium` for the latter — useful on ARM64, where Chrome for Testing has no Linux builds):
61
+
62
+ ```dockerfile
63
+ FROM node:22-bookworm-slim
64
+ RUN apt-get update && apt-get install -y \
65
+ ca-certificates libvulkan1 mesa-vulkan-drivers xvfb xauth \
66
+ && rm -rf /var/lib/apt/lists/*
67
+ RUN npm install -g agent-browser \
68
+ && agent-browser install # downloads Chrome for Testing
69
+ ```
70
+
71
+ No real GPU or `/dev/dri` is required. To prefer a real GPU on a Linux machine that has working hardware Vulkan, override both the Vulkan driver and the adapter — the preset pins `--use-vulkan=swiftshader`, so overriding only the adapter still enumerates SwiftShader (user `--args` win over the preset):
72
+
73
+ ```bash
74
+ agent-browser --webgpu --args "--use-vulkan=native,--use-webgpu-adapter=default" open ...
75
+ ```
76
+
77
+ ## Secure contexts
78
+
79
+ `navigator.gpu` only exists in secure contexts. `https://`, `http://localhost`, and `file://` qualify; a plain `http://` LAN address or `data:` URL does not — WebGPU will be `undefined` there no matter which flags are set.
80
+
81
+ ## Timing: don't screenshot too early
82
+
83
+ WebGPU apps initialize asynchronously. A screenshot taken at `load` captures a blank canvas with no error anywhere. In particular:
84
+
85
+ - **three.js `WebGPURenderer`**: `renderer.init()` is async; the first frame lands only after it resolves. Also note three.js **silently falls back to WebGL2** when it can't get a WebGPU adapter — the page "works" but you're not testing WebGPU (and on old setups the WebGL fallback itself may be black).
86
+ - Wait for an app-specific signal before capturing: a canvas with content, a "ready" DOM marker, or simply a rendered-frame check:
87
+
88
+ ```bash
89
+ agent-browser wait --fn "window.__appReady === true"
90
+ # or generically: give the render loop a frame or two
91
+ agent-browser eval "new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))"
92
+ agent-browser screenshot app.png
93
+ ```
94
+
95
+ To check which backend a three.js app actually got:
96
+
97
+ ```bash
98
+ agent-browser eval "document.querySelector('canvas').getContext('webgpu') ? 'webgpu' : 'webgl-fallback'"
99
+ ```
100
+
101
+ ## Reading pixels back inside the page
102
+
103
+ If you `eval` your own WebGPU readback, don't snapshot the canvas (`drawImage(webgpuCanvas, ...)`) — it depends on presentation timing and reads transparent black on Windows even when rendering works. Render to an offscreen texture and read it back deterministically:
104
+
105
+ ```js
106
+ const tex = device.createTexture({ size: [w, h], format: 'rgba8unorm',
107
+ usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC });
108
+ // ...render to tex, then:
109
+ encoder.copyTextureToBuffer({ texture: tex }, { buffer, bytesPerRow }, [w, h]);
110
+ device.queue.submit([encoder.finish()]);
111
+ await buffer.mapAsync(GPUMapMode.READ);
112
+ ```
113
+
114
+ This works headless on every platform (it's how `doctor --webgpu` proves rendering).
115
+
116
+ ## Performance expectations
117
+
118
+ SwiftShader is a CPU rasterizer. Simple scenes render fine; heavy three.js scenes are single-digit FPS. For screenshots that's usually irrelevant; for smooth video capture of complex scenes, use hardware (macOS/Windows, or Linux with `--use-vulkan=native,--use-webgpu-adapter=default` and real Vulkan drivers).