dsh-x-core 0.1.21 → 0.1.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/assets/dsh-preview.png +0 -0
  2. package/assets/dsh.ico +0 -0
  3. package/assets/icon.ico +0 -0
  4. package/assets/tray.ico +0 -0
  5. package/assets/tray.png +0 -0
  6. package/core-npm.js +201 -183
  7. package/desktop-plugins.js +86 -0
  8. package/desktop.js +200 -0
  9. package/filemanager-base.js +7 -8
  10. package/filemanager-linux.js +124 -0
  11. package/filemanager.js +4 -7
  12. package/native-notify.js +74 -0
  13. package/node/node +2 -1
  14. package/package.json +1 -1
  15. package/packs/dsh-x-workbench/README.md +61 -0
  16. package/packs/dsh-x-workbench/dspack.json +4 -0
  17. package/packs/dsh-x-workbench/home/skills/agent-browser/LICENSE-APACHE-2.0.txt +202 -0
  18. package/packs/dsh-x-workbench/home/skills/agent-browser/NOTICE.md +6 -0
  19. package/packs/dsh-x-workbench/home/skills/agent-browser/SKILL.md +697 -0
  20. package/packs/dsh-x-workbench/home/skills/agent-browser/references/authentication.md +313 -0
  21. package/packs/dsh-x-workbench/home/skills/agent-browser/references/commands.md +300 -0
  22. package/packs/dsh-x-workbench/home/skills/agent-browser/references/profiling.md +120 -0
  23. package/packs/dsh-x-workbench/home/skills/agent-browser/references/proxy-support.md +194 -0
  24. package/packs/dsh-x-workbench/home/skills/agent-browser/references/session-management.md +201 -0
  25. package/packs/dsh-x-workbench/home/skills/agent-browser/references/snapshot-refs.md +229 -0
  26. package/packs/dsh-x-workbench/home/skills/agent-browser/references/video-recording.md +173 -0
  27. package/packs/dsh-x-workbench/home/skills/agent-browser/templates/authenticated-session.sh +105 -0
  28. package/packs/dsh-x-workbench/home/skills/agent-browser/templates/capture-workflow.sh +69 -0
  29. package/packs/dsh-x-workbench/home/skills/agent-browser/templates/form-automation.sh +62 -0
  30. package/packs/dsh-x-workbench/home/skills/electron/LICENSE-APACHE-2.0.txt +202 -0
  31. package/packs/dsh-x-workbench/home/skills/electron/NOTICE.md +6 -0
  32. package/packs/dsh-x-workbench/home/skills/electron/SKILL.md +243 -0
  33. package/packs/dsh-x-workbench/manifest.json +35 -0
  34. package/perf/patch-hooks.mjs +14 -1
  35. package/perf/plugin-profile.mjs +204 -0
  36. package/perf/register.mjs +4 -0
  37. package/platform.js +2 -1
  38. package/public/booting.html +475 -0
  39. package/public/favicon.ico +0 -0
  40. package/public/favicon.png +0 -0
  41. package/public/icon.png +0 -0
  42. package/public/index.html +9277 -8173
  43. package/public/internal-tabs.js +45 -41
  44. package/public/launcher.css +285 -98
  45. package/public/mascot.js +1 -1
  46. package/public/theme.css +33 -6
  47. package/public/water.js +1 -1
  48. package/registry.js +38 -3
  49. package/server.js +1060 -119
  50. package/settings.js +78 -1
@@ -0,0 +1,697 @@
1
+ ---
2
+ name: agent-browser
3
+ description: Browser automation CLI for AI agents. Use when the user needs to interact with websites, including navigating pages, filling forms, clicking buttons, taking screenshots, extracting data, testing web apps, or automating any browser task. Triggers include requests to "open a website", "fill out a form", "click a button", "take a screenshot", "scrape data from a page", "test this web app", "login to a site", "automate browser actions", or any task requiring programmatic web interaction.
4
+ allowed-tools: Bash(npx agent-browser:*), Bash(agent-browser:*)
5
+ ---
6
+
7
+ <!--
8
+ Derived from vercel-labs/agent-browser (skills/agent-browser/SKILL.md).
9
+ Copyright 2025 Vercel Inc. Licensed under Apache-2.0.
10
+ Modified by ZCode: local integration, formatting and adaptations.
11
+ See THIRD-PARTY-NOTICES.md in the repository root for license and provenance.
12
+ -->
13
+
14
+ # Browser Automation with agent-browser
15
+
16
+ The CLI uses Chrome/Chromium via CDP directly. Install via `npm i -g agent-browser`, `brew install agent-browser`, or `cargo install agent-browser`. Run `agent-browser install` to download Chrome. Run `agent-browser upgrade` to update to the latest version.
17
+
18
+ ## Core Workflow
19
+
20
+ Every browser automation follows this pattern:
21
+
22
+ 1. **Navigate**: `agent-browser open <url>`
23
+ 2. **Snapshot**: `agent-browser snapshot -i` (get element refs like `@e1`, `@e2`)
24
+ 3. **Interact**: Use refs to click, fill, select
25
+ 4. **Re-snapshot**: After navigation or DOM changes, get fresh refs
26
+
27
+ ```bash
28
+ agent-browser open https://example.com/form
29
+ agent-browser snapshot -i
30
+ # Output: @e1 [input type="email"], @e2 [input type="password"], @e3 [button] "Submit"
31
+
32
+ agent-browser fill @e1 "user@example.com"
33
+ agent-browser fill @e2 "password123"
34
+ agent-browser click @e3
35
+ agent-browser wait --load networkidle
36
+ agent-browser snapshot -i # Check result
37
+ ```
38
+
39
+ ## Command Chaining
40
+
41
+ Commands can be chained with `&&` in a single shell invocation. The browser persists between commands via a background daemon, so chaining is safe and more efficient than separate calls.
42
+
43
+ ```bash
44
+ # Chain open + wait + snapshot in one call
45
+ agent-browser open https://example.com && agent-browser wait --load networkidle && agent-browser snapshot -i
46
+
47
+ # Chain multiple interactions
48
+ agent-browser fill @e1 "user@example.com" && agent-browser fill @e2 "password123" && agent-browser click @e3
49
+
50
+ # Navigate and capture
51
+ agent-browser open https://example.com && agent-browser wait --load networkidle && agent-browser screenshot page.png
52
+ ```
53
+
54
+ **When to chain:** Use `&&` when you don't need to read the output of an intermediate command before proceeding (e.g., open + wait + screenshot). Run commands separately when you need to parse the output first (e.g., snapshot to discover refs, then interact using those refs).
55
+
56
+ ## Handling Authentication
57
+
58
+ When automating a site that requires login, choose the approach that fits:
59
+
60
+ **Option 1: Import auth from the user's browser (fastest for one-off tasks)**
61
+
62
+ ```bash
63
+ # Connect to the user's running Chrome (they're already logged in)
64
+ agent-browser --auto-connect state save ./auth.json
65
+ # Use that auth state
66
+ agent-browser --state ./auth.json open https://app.example.com/dashboard
67
+ ```
68
+
69
+ State files contain session tokens in plaintext -- add to `.gitignore` and delete when no longer needed. Set `AGENT_BROWSER_ENCRYPTION_KEY` for encryption at rest.
70
+
71
+ **Option 2: Persistent profile (simplest for recurring tasks)**
72
+
73
+ ```bash
74
+ # First run: login manually or via automation
75
+ agent-browser --profile ~/.myapp open https://app.example.com/login
76
+ # ... fill credentials, submit ...
77
+
78
+ # All future runs: already authenticated
79
+ agent-browser --profile ~/.myapp open https://app.example.com/dashboard
80
+ ```
81
+
82
+ **Option 3: Session name (auto-save/restore cookies + localStorage)**
83
+
84
+ ```bash
85
+ agent-browser --session-name myapp open https://app.example.com/login
86
+ # ... login flow ...
87
+ agent-browser close # State auto-saved
88
+
89
+ # Next time: state auto-restored
90
+ agent-browser --session-name myapp open https://app.example.com/dashboard
91
+ ```
92
+
93
+ **Option 4: Auth vault (credentials stored encrypted, login by name)**
94
+
95
+ ```bash
96
+ echo "$PASSWORD" | agent-browser auth save myapp --url https://app.example.com/login --username user --password-stdin
97
+ agent-browser auth login myapp
98
+ ```
99
+
100
+ `auth login` navigates with `load` and then waits for login form selectors to appear before filling/clicking, which is more reliable on delayed SPA login screens.
101
+
102
+ **Option 5: State file (manual save/load)**
103
+
104
+ ```bash
105
+ # After logging in:
106
+ agent-browser state save ./auth.json
107
+ # In a future session:
108
+ agent-browser state load ./auth.json
109
+ agent-browser open https://app.example.com/dashboard
110
+ ```
111
+
112
+ See [references/authentication.md](references/authentication.md) for OAuth, 2FA, cookie-based auth, and token refresh patterns.
113
+
114
+ ## Essential Commands
115
+
116
+ ```bash
117
+ # Navigation
118
+ agent-browser open <url> # Navigate (aliases: goto, navigate)
119
+ agent-browser close # Close browser
120
+
121
+ # Snapshot
122
+ agent-browser snapshot -i # Interactive elements with refs (recommended)
123
+ agent-browser snapshot -s "#selector" # Scope to CSS selector
124
+
125
+ # Interaction (use @refs from snapshot)
126
+ agent-browser click @e1 # Click element
127
+ agent-browser click @e1 --new-tab # Click and open in new tab
128
+ agent-browser fill @e2 "text" # Clear and type text
129
+ agent-browser type @e2 "text" # Type without clearing
130
+ agent-browser select @e1 "option" # Select dropdown option
131
+ agent-browser check @e1 # Check checkbox
132
+ agent-browser press Enter # Press key
133
+ agent-browser keyboard type "text" # Type at current focus (no selector)
134
+ agent-browser keyboard inserttext "text" # Insert without key events
135
+ agent-browser scroll down 500 # Scroll page
136
+ agent-browser scroll down 500 --selector "div.content" # Scroll within a specific container
137
+
138
+ # Get information
139
+ agent-browser get text @e1 # Get element text
140
+ agent-browser get url # Get current URL
141
+ agent-browser get title # Get page title
142
+ agent-browser get cdp-url # Get CDP WebSocket URL
143
+
144
+ # Wait
145
+ agent-browser wait @e1 # Wait for element
146
+ agent-browser wait --load networkidle # Wait for network idle
147
+ agent-browser wait --url "**/page" # Wait for URL pattern
148
+ agent-browser wait 2000 # Wait milliseconds
149
+ agent-browser wait --text "Welcome" # Wait for text to appear (substring match)
150
+ agent-browser wait --fn "!document.body.innerText.includes('Loading...')" # Wait for text to disappear
151
+ agent-browser wait "#spinner" --state hidden # Wait for element to disappear
152
+
153
+ # Downloads
154
+ agent-browser download @e1 ./file.pdf # Click element to trigger download
155
+ agent-browser wait --download ./output.zip # Wait for any download to complete
156
+ agent-browser --download-path ./downloads open <url> # Set default download directory
157
+
158
+ # Network
159
+ agent-browser network requests # Inspect tracked requests
160
+ agent-browser network requests --type xhr,fetch # Filter by resource type
161
+ agent-browser network requests --method POST # Filter by HTTP method
162
+ agent-browser network requests --status 2xx # Filter by status (200, 2xx, 400-499)
163
+ agent-browser network request <requestId> # View full request/response detail
164
+ agent-browser network route "**/api/*" --abort # Block matching requests
165
+ agent-browser network har start # Start HAR recording
166
+ agent-browser network har stop ./capture.har # Stop and save HAR file
167
+
168
+ # Viewport & Device Emulation
169
+ agent-browser set viewport 1920 1080 # Set viewport size (default: 1280x720)
170
+ agent-browser set viewport 1920 1080 2 # 2x retina (same CSS size, higher res screenshots)
171
+ agent-browser set device "iPhone 14" # Emulate device (viewport + user agent)
172
+
173
+ # Capture
174
+ agent-browser screenshot # Screenshot to temp dir
175
+ agent-browser screenshot --full # Full page screenshot
176
+ agent-browser screenshot --annotate # Annotated screenshot with numbered element labels
177
+ agent-browser screenshot --screenshot-dir ./shots # Save to custom directory
178
+ agent-browser screenshot --screenshot-format jpeg --screenshot-quality 80
179
+ agent-browser pdf output.pdf # Save as PDF
180
+
181
+ # Clipboard
182
+ agent-browser clipboard read # Read text from clipboard
183
+ agent-browser clipboard write "Hello, World!" # Write text to clipboard
184
+ agent-browser clipboard copy # Copy current selection
185
+ agent-browser clipboard paste # Paste from clipboard
186
+
187
+ # Diff (compare page states)
188
+ agent-browser diff snapshot # Compare current vs last snapshot
189
+ agent-browser diff snapshot --baseline before.txt # Compare current vs saved file
190
+ agent-browser diff screenshot --baseline before.png # Visual pixel diff
191
+ agent-browser diff url <url1> <url2> # Compare two pages
192
+ agent-browser diff url <url1> <url2> --wait-until networkidle # Custom wait strategy
193
+ agent-browser diff url <url1> <url2> --selector "#main" # Scope to element
194
+ ```
195
+
196
+ ## Batch Execution
197
+
198
+ Execute multiple commands in a single invocation by piping a JSON array of string arrays to `batch`. This avoids per-command process startup overhead when running multi-step workflows.
199
+
200
+ ```bash
201
+ echo '[
202
+ ["open", "https://example.com"],
203
+ ["snapshot", "-i"],
204
+ ["click", "@e1"],
205
+ ["screenshot", "result.png"]
206
+ ]' | agent-browser batch --json
207
+
208
+ # Stop on first error
209
+ agent-browser batch --bail < commands.json
210
+ ```
211
+
212
+ Use `batch` when you have a known sequence of commands that don't depend on intermediate output. Use separate commands or `&&` chaining when you need to parse output between steps (e.g., snapshot to discover refs, then interact).
213
+
214
+ ## Common Patterns
215
+
216
+ ### Form Submission
217
+
218
+ ```bash
219
+ agent-browser open https://example.com/signup
220
+ agent-browser snapshot -i
221
+ agent-browser fill @e1 "Jane Doe"
222
+ agent-browser fill @e2 "jane@example.com"
223
+ agent-browser select @e3 "California"
224
+ agent-browser check @e4
225
+ agent-browser click @e5
226
+ agent-browser wait --load networkidle
227
+ ```
228
+
229
+ ### Authentication with Auth Vault (Recommended)
230
+
231
+ ```bash
232
+ # Save credentials once (encrypted with AGENT_BROWSER_ENCRYPTION_KEY)
233
+ # Recommended: pipe password via stdin to avoid shell history exposure
234
+ echo "pass" | agent-browser auth save github --url https://github.com/login --username user --password-stdin
235
+
236
+ # Login using saved profile (LLM never sees password)
237
+ agent-browser auth login github
238
+
239
+ # List/show/delete profiles
240
+ agent-browser auth list
241
+ agent-browser auth show github
242
+ agent-browser auth delete github
243
+ ```
244
+
245
+ `auth login` waits for username/password/submit selectors before interacting, with a timeout tied to the default action timeout.
246
+
247
+ ### Authentication with State Persistence
248
+
249
+ ```bash
250
+ # Login once and save state
251
+ agent-browser open https://app.example.com/login
252
+ agent-browser snapshot -i
253
+ agent-browser fill @e1 "$USERNAME"
254
+ agent-browser fill @e2 "$PASSWORD"
255
+ agent-browser click @e3
256
+ agent-browser wait --url "**/dashboard"
257
+ agent-browser state save auth.json
258
+
259
+ # Reuse in future sessions
260
+ agent-browser state load auth.json
261
+ agent-browser open https://app.example.com/dashboard
262
+ ```
263
+
264
+ ### Session Persistence
265
+
266
+ ```bash
267
+ # Auto-save/restore cookies and localStorage across browser restarts
268
+ agent-browser --session-name myapp open https://app.example.com/login
269
+ # ... login flow ...
270
+ agent-browser close # State auto-saved to ~/.agent-browser/sessions/
271
+
272
+ # Next time, state is auto-loaded
273
+ agent-browser --session-name myapp open https://app.example.com/dashboard
274
+
275
+ # Encrypt state at rest
276
+ export AGENT_BROWSER_ENCRYPTION_KEY=$(openssl rand -hex 32)
277
+ agent-browser --session-name secure open https://app.example.com
278
+
279
+ # Manage saved states
280
+ agent-browser state list
281
+ agent-browser state show myapp-default.json
282
+ agent-browser state clear myapp
283
+ agent-browser state clean --older-than 7
284
+ ```
285
+
286
+ ### Working with Iframes
287
+
288
+ Iframe content is automatically inlined in snapshots. Refs inside iframes carry frame context, so you can interact with them directly.
289
+
290
+ ```bash
291
+ agent-browser open https://example.com/checkout
292
+ agent-browser snapshot -i
293
+ # @e1 [heading] "Checkout"
294
+ # @e2 [Iframe] "payment-frame"
295
+ # @e3 [input] "Card number"
296
+ # @e4 [input] "Expiry"
297
+ # @e5 [button] "Pay"
298
+
299
+ # Interact directly — no frame switch needed
300
+ agent-browser fill @e3 "4111111111111111"
301
+ agent-browser fill @e4 "12/28"
302
+ agent-browser click @e5
303
+
304
+ # To scope a snapshot to one iframe:
305
+ agent-browser frame @e2
306
+ agent-browser snapshot -i # Only iframe content
307
+ agent-browser frame main # Return to main frame
308
+ ```
309
+
310
+ ### Data Extraction
311
+
312
+ ```bash
313
+ agent-browser open https://example.com/products
314
+ agent-browser snapshot -i
315
+ agent-browser get text @e5 # Get specific element text
316
+ agent-browser get text body > page.txt # Get all page text
317
+
318
+ # JSON output for parsing
319
+ agent-browser snapshot -i --json
320
+ agent-browser get text @e1 --json
321
+ ```
322
+
323
+ ### Parallel Sessions
324
+
325
+ ```bash
326
+ agent-browser --session site1 open https://site-a.com
327
+ agent-browser --session site2 open https://site-b.com
328
+
329
+ agent-browser --session site1 snapshot -i
330
+ agent-browser --session site2 snapshot -i
331
+
332
+ agent-browser session list
333
+ ```
334
+
335
+ ### Connect to Existing Chrome
336
+
337
+ ```bash
338
+ # Auto-discover running Chrome with remote debugging enabled
339
+ agent-browser --auto-connect open https://example.com
340
+ agent-browser --auto-connect snapshot
341
+
342
+ # Or with explicit CDP port
343
+ agent-browser --cdp 9222 snapshot
344
+ ```
345
+
346
+ Auto-connect discovers Chrome via `DevToolsActivePort`, common debugging ports (9222, 9229), and falls back to a direct WebSocket connection if HTTP-based CDP discovery fails.
347
+
348
+ ### Color Scheme (Dark Mode)
349
+
350
+ ```bash
351
+ # Persistent dark mode via flag (applies to all pages and new tabs)
352
+ agent-browser --color-scheme dark open https://example.com
353
+
354
+ # Or via environment variable
355
+ AGENT_BROWSER_COLOR_SCHEME=dark agent-browser open https://example.com
356
+
357
+ # Or set during session (persists for subsequent commands)
358
+ agent-browser set media dark
359
+ ```
360
+
361
+ ### Viewport & Responsive Testing
362
+
363
+ ```bash
364
+ # Set a custom viewport size (default is 1280x720)
365
+ agent-browser set viewport 1920 1080
366
+ agent-browser screenshot desktop.png
367
+
368
+ # Test mobile-width layout
369
+ agent-browser set viewport 375 812
370
+ agent-browser screenshot mobile.png
371
+
372
+ # Retina/HiDPI: same CSS layout at 2x pixel density
373
+ # Screenshots stay at logical viewport size, but content renders at higher DPI
374
+ agent-browser set viewport 1920 1080 2
375
+ agent-browser screenshot retina.png
376
+
377
+ # Device emulation (sets viewport + user agent in one step)
378
+ agent-browser set device "iPhone 14"
379
+ agent-browser screenshot device.png
380
+ ```
381
+
382
+ The `scale` parameter (3rd argument) sets `window.devicePixelRatio` without changing CSS layout. Use it when testing retina rendering or capturing higher-resolution screenshots.
383
+
384
+ ### Visual Browser (Debugging)
385
+
386
+ ```bash
387
+ agent-browser --headed open https://example.com
388
+ agent-browser highlight @e1 # Highlight element
389
+ agent-browser inspect # Open Chrome DevTools for the active page
390
+ agent-browser record start demo.webm # Record session
391
+ agent-browser profiler start # Start Chrome DevTools profiling
392
+ agent-browser profiler stop trace.json # Stop and save profile (path optional)
393
+ ```
394
+
395
+ Use `AGENT_BROWSER_HEADED=1` to enable headed mode via environment variable. Browser extensions work in both headed and headless mode.
396
+
397
+ ### Local Files (PDFs, HTML)
398
+
399
+ ```bash
400
+ # Open local files with file:// URLs
401
+ agent-browser --allow-file-access open file:///path/to/document.pdf
402
+ agent-browser --allow-file-access open file:///path/to/page.html
403
+ agent-browser screenshot output.png
404
+ ```
405
+
406
+ ### iOS Simulator (Mobile Safari)
407
+
408
+ ```bash
409
+ # List available iOS simulators
410
+ agent-browser device list
411
+
412
+ # Launch Safari on a specific device
413
+ agent-browser -p ios --device "iPhone 16 Pro" open https://example.com
414
+
415
+ # Same workflow as desktop - snapshot, interact, re-snapshot
416
+ agent-browser -p ios snapshot -i
417
+ agent-browser -p ios tap @e1 # Tap (alias for click)
418
+ agent-browser -p ios fill @e2 "text"
419
+ agent-browser -p ios swipe up # Mobile-specific gesture
420
+
421
+ # Take screenshot
422
+ agent-browser -p ios screenshot mobile.png
423
+
424
+ # Close session (shuts down simulator)
425
+ agent-browser -p ios close
426
+ ```
427
+
428
+ **Requirements:** macOS with Xcode, Appium (`npm install -g appium && appium driver install xcuitest`)
429
+
430
+ **Real devices:** Works with physical iOS devices if pre-configured. Use `--device "<UDID>"` where UDID is from `xcrun xctrace list devices`.
431
+
432
+ ## Security
433
+
434
+ All security features are opt-in. By default, agent-browser imposes no restrictions on navigation, actions, or output.
435
+
436
+ ### Content Boundaries (Recommended for AI Agents)
437
+
438
+ Enable `--content-boundaries` to wrap page-sourced output in markers that help LLMs distinguish tool output from untrusted page content:
439
+
440
+ ```bash
441
+ export AGENT_BROWSER_CONTENT_BOUNDARIES=1
442
+ agent-browser snapshot
443
+ # Output:
444
+ # --- AGENT_BROWSER_PAGE_CONTENT nonce=<hex> origin=https://example.com ---
445
+ # [accessibility tree]
446
+ # --- END_AGENT_BROWSER_PAGE_CONTENT nonce=<hex> ---
447
+ ```
448
+
449
+ ### Domain Allowlist
450
+
451
+ Restrict navigation to trusted domains. Wildcards like `*.example.com` also match the bare domain `example.com`. Sub-resource requests, WebSocket, and EventSource connections to non-allowed domains are also blocked. Include CDN domains your target pages depend on:
452
+
453
+ ```bash
454
+ export AGENT_BROWSER_ALLOWED_DOMAINS="example.com,*.example.com"
455
+ agent-browser open https://example.com # OK
456
+ agent-browser open https://malicious.com # Blocked
457
+ ```
458
+
459
+ ### Action Policy
460
+
461
+ Use a policy file to gate destructive actions:
462
+
463
+ ```bash
464
+ export AGENT_BROWSER_ACTION_POLICY=./policy.json
465
+ ```
466
+
467
+ Example `policy.json`:
468
+
469
+ ```json
470
+ { "default": "deny", "allow": ["navigate", "snapshot", "click", "scroll", "wait", "get"] }
471
+ ```
472
+
473
+ Auth vault operations (`auth login`, etc.) bypass action policy but domain allowlist still applies.
474
+
475
+ ### Output Limits
476
+
477
+ Prevent context flooding from large pages:
478
+
479
+ ```bash
480
+ export AGENT_BROWSER_MAX_OUTPUT=50000
481
+ ```
482
+
483
+ ## Diffing (Verifying Changes)
484
+
485
+ Use `diff snapshot` after performing an action to verify it had the intended effect. This compares the current accessibility tree against the last snapshot taken in the session.
486
+
487
+ ```bash
488
+ # Typical workflow: snapshot -> action -> diff
489
+ agent-browser snapshot -i # Take baseline snapshot
490
+ agent-browser click @e2 # Perform action
491
+ agent-browser diff snapshot # See what changed (auto-compares to last snapshot)
492
+ ```
493
+
494
+ For visual regression testing or monitoring:
495
+
496
+ ```bash
497
+ # Save a baseline screenshot, then compare later
498
+ agent-browser screenshot baseline.png
499
+ # ... time passes or changes are made ...
500
+ agent-browser diff screenshot --baseline baseline.png
501
+
502
+ # Compare staging vs production
503
+ agent-browser diff url https://staging.example.com https://prod.example.com --screenshot
504
+ ```
505
+
506
+ `diff snapshot` output uses `+` for additions and `-` for removals, similar to git diff. `diff screenshot` produces a diff image with changed pixels highlighted in red, plus a mismatch percentage.
507
+
508
+ ## Timeouts and Slow Pages
509
+
510
+ The default timeout is 25 seconds. This can be overridden with the `AGENT_BROWSER_DEFAULT_TIMEOUT` environment variable (value in milliseconds). For slow websites or large pages, use explicit waits instead of relying on the default timeout:
511
+
512
+ ```bash
513
+ # Wait for network activity to settle (best for slow pages)
514
+ agent-browser wait --load networkidle
515
+
516
+ # Wait for a specific element to appear
517
+ agent-browser wait "#content"
518
+ agent-browser wait @e1
519
+
520
+ # Wait for a specific URL pattern (useful after redirects)
521
+ agent-browser wait --url "**/dashboard"
522
+
523
+ # Wait for a JavaScript condition
524
+ agent-browser wait --fn "document.readyState === 'complete'"
525
+
526
+ # Wait a fixed duration (milliseconds) as a last resort
527
+ agent-browser wait 5000
528
+ ```
529
+
530
+ When dealing with consistently slow websites, use `wait --load networkidle` after `open` to ensure the page is fully loaded before taking a snapshot. If a specific element is slow to render, wait for it directly with `wait <selector>` or `wait @ref`.
531
+
532
+ ## Session Management and Cleanup
533
+
534
+ When running multiple agents or automations concurrently, always use named sessions to avoid conflicts:
535
+
536
+ ```bash
537
+ # Each agent gets its own isolated session
538
+ agent-browser --session agent1 open site-a.com
539
+ agent-browser --session agent2 open site-b.com
540
+
541
+ # Check active sessions
542
+ agent-browser session list
543
+ ```
544
+
545
+ Always close your browser session when done to avoid leaked processes:
546
+
547
+ ```bash
548
+ agent-browser close # Close default session
549
+ agent-browser --session agent1 close # Close specific session
550
+ ```
551
+
552
+ If a previous session was not closed properly, the daemon may still be running. Use `agent-browser close` to clean it up before starting new work.
553
+
554
+ To auto-shutdown the daemon after a period of inactivity (useful for ephemeral/CI environments):
555
+
556
+ ```bash
557
+ AGENT_BROWSER_IDLE_TIMEOUT_MS=60000 agent-browser open example.com
558
+ ```
559
+
560
+ ## Ref Lifecycle (Important)
561
+
562
+ Refs (`@e1`, `@e2`, etc.) are invalidated when the page changes. Always re-snapshot after:
563
+
564
+ - Clicking links or buttons that navigate
565
+ - Form submissions
566
+ - Dynamic content loading (dropdowns, modals)
567
+
568
+ ```bash
569
+ agent-browser click @e5 # Navigates to new page
570
+ agent-browser snapshot -i # MUST re-snapshot
571
+ agent-browser click @e1 # Use new refs
572
+ ```
573
+
574
+ ## Annotated Screenshots (Vision Mode)
575
+
576
+ Use `--annotate` to take a screenshot with numbered labels overlaid on interactive elements. Each label `[N]` maps to ref `@eN`. This also caches refs, so you can interact with elements immediately without a separate snapshot.
577
+
578
+ ```bash
579
+ agent-browser screenshot --annotate
580
+ # Output includes the image path and a legend:
581
+ # [1] @e1 button "Submit"
582
+ # [2] @e2 link "Home"
583
+ # [3] @e3 textbox "Email"
584
+ agent-browser click @e2 # Click using ref from annotated screenshot
585
+ ```
586
+
587
+ Use annotated screenshots when:
588
+
589
+ - The page has unlabeled icon buttons or visual-only elements
590
+ - You need to verify visual layout or styling
591
+ - Canvas or chart elements are present (invisible to text snapshots)
592
+ - You need spatial reasoning about element positions
593
+
594
+ ## Semantic Locators (Alternative to Refs)
595
+
596
+ When refs are unavailable or unreliable, use semantic locators:
597
+
598
+ ```bash
599
+ agent-browser find text "Sign In" click
600
+ agent-browser find label "Email" fill "user@test.com"
601
+ agent-browser find role button click --name "Submit"
602
+ agent-browser find placeholder "Search" type "query"
603
+ agent-browser find testid "submit-btn" click
604
+ ```
605
+
606
+ ## JavaScript Evaluation (eval)
607
+
608
+ Use `eval` to run JavaScript in the browser context. **Shell quoting can corrupt complex expressions** -- use `--stdin` or `-b` to avoid issues.
609
+
610
+ ```bash
611
+ # Simple expressions work with regular quoting
612
+ agent-browser eval 'document.title'
613
+ agent-browser eval 'document.querySelectorAll("img").length'
614
+
615
+ # Complex JS: use --stdin with heredoc (RECOMMENDED)
616
+ agent-browser eval --stdin <<'EVALEOF'
617
+ JSON.stringify(
618
+ Array.from(document.querySelectorAll("img"))
619
+ .filter(i => !i.alt)
620
+ .map(i => ({ src: i.src.split("/").pop(), width: i.width }))
621
+ )
622
+ EVALEOF
623
+
624
+ # Alternative: base64 encoding (avoids all shell escaping issues)
625
+ agent-browser eval -b "$(echo -n 'Array.from(document.querySelectorAll("a")).map(a => a.href)' | base64)"
626
+ ```
627
+
628
+ **Why this matters:** When the shell processes your command, inner double quotes, `!` characters (history expansion), backticks, and `$()` can all corrupt the JavaScript before it reaches agent-browser. The `--stdin` and `-b` flags bypass shell interpretation entirely.
629
+
630
+ **Rules of thumb:**
631
+
632
+ - Single-line, no nested quotes -> regular `eval 'expression'` with single quotes is fine
633
+ - Nested quotes, arrow functions, template literals, or multiline -> use `eval --stdin <<'EVALEOF'`
634
+ - Programmatic/generated scripts -> use `eval -b` with base64
635
+
636
+ ## Configuration File
637
+
638
+ Create `agent-browser.json` in the project root for persistent settings:
639
+
640
+ ```json
641
+ {
642
+ "headed": true,
643
+ "proxy": "http://localhost:8080",
644
+ "profile": "./browser-data"
645
+ }
646
+ ```
647
+
648
+ Priority (lowest to highest): `~/.agent-browser/config.json` < `./agent-browser.json` < env vars < CLI flags. Use `--config <path>` or `AGENT_BROWSER_CONFIG` env var for a custom config file (exits with error if missing/invalid). All CLI options map to camelCase keys (e.g., `--executable-path` -> `"executablePath"`). Boolean flags accept `true`/`false` values (e.g., `--headed false` overrides config). Extensions from user and project configs are merged, not replaced.
649
+
650
+ ## Deep-Dive Documentation
651
+
652
+ | Reference | When to Use |
653
+ | -------------------------------------------------------------------- | --------------------------------------------------------- |
654
+ | [references/commands.md](references/commands.md) | Full command reference with all options |
655
+ | [references/snapshot-refs.md](references/snapshot-refs.md) | Ref lifecycle, invalidation rules, troubleshooting |
656
+ | [references/session-management.md](references/session-management.md) | Parallel sessions, state persistence, concurrent scraping |
657
+ | [references/authentication.md](references/authentication.md) | Login flows, OAuth, 2FA handling, state reuse |
658
+ | [references/video-recording.md](references/video-recording.md) | Recording workflows for debugging and documentation |
659
+ | [references/profiling.md](references/profiling.md) | Chrome DevTools profiling for performance analysis |
660
+ | [references/proxy-support.md](references/proxy-support.md) | Proxy configuration, geo-testing, rotating proxies |
661
+
662
+ ## Browser Engine Selection
663
+
664
+ Use `--engine` to choose a local browser engine. The default is `chrome`.
665
+
666
+ ```bash
667
+ # Use Lightpanda (fast headless browser, requires separate install)
668
+ agent-browser --engine lightpanda open example.com
669
+
670
+ # Via environment variable
671
+ export AGENT_BROWSER_ENGINE=lightpanda
672
+ agent-browser open example.com
673
+
674
+ # With custom binary path
675
+ agent-browser --engine lightpanda --executable-path /path/to/lightpanda open example.com
676
+ ```
677
+
678
+ Supported engines:
679
+
680
+ - `chrome` (default) -- Chrome/Chromium via CDP
681
+ - `lightpanda` -- Lightpanda headless browser via CDP (10x faster, 10x less memory than Chrome)
682
+
683
+ Lightpanda does not support `--extension`, `--profile`, `--state`, or `--allow-file-access`. Install Lightpanda from https://lightpanda.io/docs/open-source/installation.
684
+
685
+ ## Ready-to-Use Templates
686
+
687
+ | Template | Description |
688
+ | ------------------------------------------------------------------------ | ----------------------------------- |
689
+ | [templates/form-automation.sh](templates/form-automation.sh) | Form filling with validation |
690
+ | [templates/authenticated-session.sh](templates/authenticated-session.sh) | Login once, reuse state |
691
+ | [templates/capture-workflow.sh](templates/capture-workflow.sh) | Content extraction with screenshots |
692
+
693
+ ```bash
694
+ ./templates/form-automation.sh https://example.com/form
695
+ ./templates/authenticated-session.sh https://app.example.com/login
696
+ ./templates/capture-workflow.sh https://example.com ./output
697
+ ```