@divebell/agent-browser 0.33.1-divebell.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +201 -0
- package/README.md +1831 -0
- package/bin/agent-browser-darwin-arm64 +0 -0
- package/bin/agent-browser-darwin-x64 +0 -0
- package/bin/agent-browser-linux-arm64 +0 -0
- package/bin/agent-browser-linux-musl-arm64 +0 -0
- package/bin/agent-browser-linux-musl-x64 +0 -0
- package/bin/agent-browser-linux-x64 +0 -0
- package/bin/agent-browser-win32-x64.exe +0 -0
- package/bin/agent-browser.js +120 -0
- package/cli/src/native/a11y/LICENSE-axe-core-THIRD-PARTY.txt +66 -0
- package/cli/src/native/a11y/LICENSE-axe-core.txt +362 -0
- package/package.json +61 -0
- package/scripts/build-all-platforms.sh +85 -0
- package/scripts/check-version-sync.js +81 -0
- package/scripts/copy-native.js +36 -0
- package/scripts/postinstall.js +321 -0
- package/scripts/sync-version.js +125 -0
- package/scripts/windows-debug/provision.sh +220 -0
- package/scripts/windows-debug/run.sh +92 -0
- package/scripts/windows-debug/start.sh +43 -0
- package/scripts/windows-debug/stop.sh +28 -0
- package/scripts/windows-debug/sync.sh +27 -0
- package/skill-data/agentcore/SKILL.md +115 -0
- package/skill-data/core/SKILL.md +518 -0
- package/skill-data/core/references/authentication.md +380 -0
- package/skill-data/core/references/commands.md +511 -0
- package/skill-data/core/references/profiling.md +120 -0
- package/skill-data/core/references/proxy-support.md +194 -0
- package/skill-data/core/references/session-management.md +180 -0
- package/skill-data/core/references/snapshot-refs.md +219 -0
- package/skill-data/core/references/trust-boundaries.md +51 -0
- package/skill-data/core/references/video-recording.md +175 -0
- package/skill-data/core/references/webgpu.md +118 -0
- package/skill-data/core/templates/authenticated-session.sh +105 -0
- package/skill-data/core/templates/capture-workflow.sh +69 -0
- package/skill-data/core/templates/form-automation.sh +62 -0
- package/skill-data/derive-client/SKILL.md +86 -0
- package/skill-data/dogfood/SKILL.md +220 -0
- package/skill-data/dogfood/references/issue-taxonomy.md +109 -0
- package/skill-data/dogfood/templates/dogfood-report-template.md +53 -0
- package/skill-data/electron/SKILL.md +236 -0
- package/skill-data/slack/SKILL.md +285 -0
- package/skill-data/slack/references/slack-tasks.md +348 -0
- package/skill-data/slack/templates/slack-report-template.md +163 -0
- package/skill-data/vercel-sandbox/SKILL.md +213 -0
- package/skills/agent-browser/SKILL.md +51 -0
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# Video Recording
|
|
2
|
+
|
|
3
|
+
Capture browser automation as video for debugging, documentation, or verification.
|
|
4
|
+
|
|
5
|
+
**Related**: [commands.md](commands.md) for full command reference, [SKILL.md](../SKILL.md) for quick start.
|
|
6
|
+
|
|
7
|
+
## Contents
|
|
8
|
+
|
|
9
|
+
- [Basic Recording](#basic-recording)
|
|
10
|
+
- [Recording Commands](#recording-commands)
|
|
11
|
+
- [Use Cases](#use-cases)
|
|
12
|
+
- [Best Practices](#best-practices)
|
|
13
|
+
- [Output Format](#output-format)
|
|
14
|
+
- [Limitations](#limitations)
|
|
15
|
+
|
|
16
|
+
## Basic Recording
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
# Launch the browser, then start recording
|
|
20
|
+
agent-browser open https://example.com
|
|
21
|
+
agent-browser record start ./demo.webm
|
|
22
|
+
|
|
23
|
+
# Perform actions
|
|
24
|
+
agent-browser snapshot -i
|
|
25
|
+
agent-browser click @e1
|
|
26
|
+
agent-browser fill @e2 "test input"
|
|
27
|
+
|
|
28
|
+
# Stop and save
|
|
29
|
+
agent-browser record stop
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
## Recording Commands
|
|
33
|
+
|
|
34
|
+
```bash
|
|
35
|
+
# Launch a session first
|
|
36
|
+
agent-browser open
|
|
37
|
+
|
|
38
|
+
# Start recording to file
|
|
39
|
+
agent-browser record start ./output.webm
|
|
40
|
+
|
|
41
|
+
# Stop current recording
|
|
42
|
+
agent-browser record stop
|
|
43
|
+
|
|
44
|
+
# Restart with new file (stops current + starts new)
|
|
45
|
+
agent-browser record restart ./take2.webm
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
## Use Cases
|
|
49
|
+
|
|
50
|
+
### Debugging Failed Automation
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
#!/bin/bash
|
|
54
|
+
# Record automation for debugging
|
|
55
|
+
|
|
56
|
+
# Run your automation
|
|
57
|
+
agent-browser open https://app.example.com
|
|
58
|
+
agent-browser record start ./debug-$(date +%Y%m%d-%H%M%S).webm
|
|
59
|
+
agent-browser snapshot -i
|
|
60
|
+
agent-browser click @e1 || {
|
|
61
|
+
echo "Click failed - check recording"
|
|
62
|
+
agent-browser record stop
|
|
63
|
+
exit 1
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
agent-browser record stop
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
### Documentation Generation
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
#!/bin/bash
|
|
73
|
+
# Record workflow for documentation
|
|
74
|
+
|
|
75
|
+
agent-browser open https://app.example.com/login
|
|
76
|
+
agent-browser record start ./docs/how-to-login.webm
|
|
77
|
+
agent-browser wait 1000 # Pause for visibility
|
|
78
|
+
|
|
79
|
+
agent-browser snapshot -i
|
|
80
|
+
agent-browser fill @e1 "demo@example.com"
|
|
81
|
+
agent-browser wait 500
|
|
82
|
+
|
|
83
|
+
agent-browser fill @e2 "password"
|
|
84
|
+
agent-browser wait 500
|
|
85
|
+
|
|
86
|
+
agent-browser click @e3
|
|
87
|
+
agent-browser wait --load networkidle
|
|
88
|
+
agent-browser wait 1000 # Show result
|
|
89
|
+
|
|
90
|
+
agent-browser record stop
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### CI/CD Test Evidence
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
#!/bin/bash
|
|
97
|
+
# Record E2E test runs for CI artifacts
|
|
98
|
+
|
|
99
|
+
TEST_NAME="${1:-e2e-test}"
|
|
100
|
+
RECORDING_DIR="./test-recordings"
|
|
101
|
+
mkdir -p "$RECORDING_DIR"
|
|
102
|
+
|
|
103
|
+
agent-browser open
|
|
104
|
+
agent-browser record start "$RECORDING_DIR/$TEST_NAME-$(date +%s).webm"
|
|
105
|
+
|
|
106
|
+
# Run test
|
|
107
|
+
if run_e2e_test; then
|
|
108
|
+
echo "Test passed"
|
|
109
|
+
else
|
|
110
|
+
echo "Test failed - recording saved"
|
|
111
|
+
fi
|
|
112
|
+
|
|
113
|
+
agent-browser record stop
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
## Best Practices
|
|
117
|
+
|
|
118
|
+
### 1. Add Pauses for Clarity
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
# Slow down for human viewing
|
|
122
|
+
agent-browser click @e1
|
|
123
|
+
agent-browser wait 500 # Let viewer see result
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### 2. Use Descriptive Filenames
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
# Include context in filename
|
|
130
|
+
agent-browser record start ./recordings/login-flow-2024-01-15.webm
|
|
131
|
+
agent-browser record start ./recordings/checkout-test-run-42.webm
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### 3. Handle Recording in Error Cases
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
#!/bin/bash
|
|
138
|
+
set -e
|
|
139
|
+
|
|
140
|
+
cleanup() {
|
|
141
|
+
agent-browser record stop 2>/dev/null || true
|
|
142
|
+
agent-browser close 2>/dev/null || true
|
|
143
|
+
}
|
|
144
|
+
trap cleanup EXIT
|
|
145
|
+
|
|
146
|
+
agent-browser open
|
|
147
|
+
agent-browser record start ./automation.webm
|
|
148
|
+
# ... automation steps ...
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### 4. Combine with Screenshots
|
|
152
|
+
|
|
153
|
+
```bash
|
|
154
|
+
# Record video AND capture key frames
|
|
155
|
+
agent-browser open https://example.com
|
|
156
|
+
agent-browser record start ./flow.webm
|
|
157
|
+
agent-browser screenshot ./screenshots/step1-homepage.png
|
|
158
|
+
|
|
159
|
+
agent-browser click @e1
|
|
160
|
+
agent-browser screenshot ./screenshots/step2-after-click.png
|
|
161
|
+
|
|
162
|
+
agent-browser record stop
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
## Output Format
|
|
166
|
+
|
|
167
|
+
- Default format: WebM (VP8/VP9 codec)
|
|
168
|
+
- Compatible with all modern browsers and video players
|
|
169
|
+
- Compressed but high quality
|
|
170
|
+
|
|
171
|
+
## Limitations
|
|
172
|
+
|
|
173
|
+
- Recording adds slight overhead to automation
|
|
174
|
+
- Large recordings can consume significant disk space
|
|
175
|
+
- Some headless environments may have codec limitations
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# WebGPU
|
|
2
|
+
|
|
3
|
+
Screenshots and video of WebGPU pages (three.js `WebGPURenderer`, Babylon.js, raw WebGPU) in headless Chrome. Without setup this is a silent failure: the page loads, the screenshot succeeds, and the canvas is black.
|
|
4
|
+
|
|
5
|
+
## Quick start
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
agent-browser --webgpu open https://my-webgpu-app.example.com
|
|
9
|
+
# wait for the app to render (see "Timing" below)
|
|
10
|
+
agent-browser screenshot app.png
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`--webgpu` (or `AGENT_BROWSER_WEBGPU=1`, or `"webgpu": true` in agent-browser.json) applies a launch preset:
|
|
14
|
+
|
|
15
|
+
- everywhere: `--enable-unsafe-webgpu` (WebGPU is hidden in headless/blocklisted environments by default)
|
|
16
|
+
- Linux only: `--enable-features=Vulkan --use-angle=vulkan --use-vulkan=swiftshader --use-webgpu-adapter=swiftshader --disable-vulkan-surface` — routes WebGPU through SwiftShader's software Vulkan, so it works with no GPU (containers, CI)
|
|
17
|
+
|
|
18
|
+
macOS uses the hardware Metal backend; Windows uses D3D. Nothing extra to install on either.
|
|
19
|
+
|
|
20
|
+
## Platform matrix (verified)
|
|
21
|
+
|
|
22
|
+
| Platform | WebGPU rendering (headless) | Screenshots of WebGPU canvases |
|
|
23
|
+
|---|---|---|
|
|
24
|
+
| macOS | works | works headless |
|
|
25
|
+
| Windows | works (hardware D3D) | **headless captures black** — use `--headed` on a logged-in desktop |
|
|
26
|
+
| Linux | works (SwiftShader Vulkan) | headless capture not supported upstream — add `--headed` (virtual display starts automatically) |
|
|
27
|
+
|
|
28
|
+
The Windows/Linux screenshot gap is an upstream headless-Chrome limitation: WebGPU canvas *presentation* never reaches the headless compositor, even though rendering itself works (verified by pixel readback). It is not an agent-browser or flag problem — no known flag combination fixes it. Rendering, `eval`-based pixel readbacks, and compute all work headless everywhere.
|
|
29
|
+
|
|
30
|
+
On Linux, `--headed` is all you need even on displayless servers and containers: when no `DISPLAY` is set and Xvfb is installed (`apt-get install -y xvfb`), agent-browser starts a private virtual display for the browser and tears it down with it. Set `AGENT_BROWSER_NO_XVFB=1` to opt out.
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
agent-browser --webgpu --headed open https://my-webgpu-app.example.com
|
|
34
|
+
agent-browser screenshot app.png # real WebGPU pixels, no display hardware
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
On Windows, the session must run headed in a logged-in desktop session (an ssh/Session-0 context is not enough — schedule the launch on the interactive desktop, e.g. `schtasks /IT`, then drive it from anywhere).
|
|
38
|
+
|
|
39
|
+
## Verify the pipeline
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
agent-browser doctor --webgpu
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
This launches a scratch session with the preset and pixel-checks two stages separately:
|
|
46
|
+
|
|
47
|
+
1. **render** — requests an adapter (with retries; a cold Chrome returns null while the GPU process starts), clears an offscreen texture to red through a real render pass, and reads the buffer back. Proves WebGPU works at all, and reports the adapter (e.g. `nvidia ampere`, `apple metal-3`, `google swiftshader`).
|
|
48
|
+
2. **screenshot** — decodes an actual screenshot of a presenting canvas. Proves the capture path. Expected to fail headless on Windows/Linux (see matrix); the failure message says so and points at `--headed`.
|
|
49
|
+
|
|
50
|
+
Add `--headed` (`agent-browser doctor --webgpu --headed`) to validate the capture path itself — on displayless Linux the probe starts its own Xvfb, so both checks should pass.
|
|
51
|
+
|
|
52
|
+
## Linux / containers / CI
|
|
53
|
+
|
|
54
|
+
The SwiftShader Vulkan path needs the system Vulkan loader and Mesa ICD. Without them `requestAdapter()` returns null (or fails with "A valid external Instance reference no longer exists"):
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
apt-get install -y libvulkan1 mesa-vulkan-drivers
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Container recipe (Debian/Ubuntu base; xvfb needed only for the screenshot path). Verified with both Chrome for Testing and Debian's `chromium` package (set `AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium` for the latter — useful on ARM64, where Chrome for Testing has no Linux builds):
|
|
61
|
+
|
|
62
|
+
```dockerfile
|
|
63
|
+
FROM node:22-bookworm-slim
|
|
64
|
+
RUN apt-get update && apt-get install -y \
|
|
65
|
+
ca-certificates libvulkan1 mesa-vulkan-drivers xvfb xauth \
|
|
66
|
+
&& rm -rf /var/lib/apt/lists/*
|
|
67
|
+
RUN npm install -g agent-browser \
|
|
68
|
+
&& agent-browser install # downloads Chrome for Testing
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
No real GPU or `/dev/dri` is required. To prefer a real GPU on a Linux machine that has working hardware Vulkan, override both the Vulkan driver and the adapter — the preset pins `--use-vulkan=swiftshader`, so overriding only the adapter still enumerates SwiftShader (user `--args` win over the preset):
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
agent-browser --webgpu --args "--use-vulkan=native,--use-webgpu-adapter=default" open ...
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Secure contexts
|
|
78
|
+
|
|
79
|
+
`navigator.gpu` only exists in secure contexts. `https://`, `http://localhost`, and `file://` qualify; a plain `http://` LAN address or `data:` URL does not — WebGPU will be `undefined` there no matter which flags are set.
|
|
80
|
+
|
|
81
|
+
## Timing: don't screenshot too early
|
|
82
|
+
|
|
83
|
+
WebGPU apps initialize asynchronously. A screenshot taken at `load` captures a blank canvas with no error anywhere. In particular:
|
|
84
|
+
|
|
85
|
+
- **three.js `WebGPURenderer`**: `renderer.init()` is async; the first frame lands only after it resolves. Also note three.js **silently falls back to WebGL2** when it can't get a WebGPU adapter — the page "works" but you're not testing WebGPU (and on old setups the WebGL fallback itself may be black).
|
|
86
|
+
- Wait for an app-specific signal before capturing: a canvas with content, a "ready" DOM marker, or simply a rendered-frame check:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
agent-browser wait --fn "window.__appReady === true"
|
|
90
|
+
# or generically: give the render loop a frame or two
|
|
91
|
+
agent-browser eval "new Promise(r => requestAnimationFrame(() => requestAnimationFrame(r)))"
|
|
92
|
+
agent-browser screenshot app.png
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
To check which backend a three.js app actually got:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
agent-browser eval "document.querySelector('canvas').getContext('webgpu') ? 'webgpu' : 'webgl-fallback'"
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## Reading pixels back inside the page
|
|
102
|
+
|
|
103
|
+
If you `eval` your own WebGPU readback, don't snapshot the canvas (`drawImage(webgpuCanvas, ...)`) — it depends on presentation timing and reads transparent black on Windows even when rendering works. Render to an offscreen texture and read it back deterministically:
|
|
104
|
+
|
|
105
|
+
```js
|
|
106
|
+
const tex = device.createTexture({ size: [w, h], format: 'rgba8unorm',
|
|
107
|
+
usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC });
|
|
108
|
+
// ...render to tex, then:
|
|
109
|
+
encoder.copyTextureToBuffer({ texture: tex }, { buffer, bytesPerRow }, [w, h]);
|
|
110
|
+
device.queue.submit([encoder.finish()]);
|
|
111
|
+
await buffer.mapAsync(GPUMapMode.READ);
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
This works headless on every platform (it's how `doctor --webgpu` proves rendering).
|
|
115
|
+
|
|
116
|
+
## Performance expectations
|
|
117
|
+
|
|
118
|
+
SwiftShader is a CPU rasterizer. Simple scenes render fine; heavy three.js scenes are single-digit FPS. For screenshots that's usually irrelevant; for smooth video capture of complex scenes, use hardware (macOS/Windows, or Linux with `--use-vulkan=native,--use-webgpu-adapter=default` and real Vulkan drivers).
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# Template: Authenticated Session Workflow
|
|
3
|
+
# Purpose: Login once, save state, reuse for subsequent runs
|
|
4
|
+
# Usage: ./authenticated-session.sh <login-url> [state-file]
|
|
5
|
+
#
|
|
6
|
+
# RECOMMENDED: Use the auth vault instead of this template:
|
|
7
|
+
# echo "<pass>" | agent-browser auth save myapp --url <login-url> --username <user> --password-stdin
|
|
8
|
+
# agent-browser auth login myapp
|
|
9
|
+
# The auth vault stores credentials securely and the LLM never sees passwords.
|
|
10
|
+
#
|
|
11
|
+
# Environment variables:
|
|
12
|
+
# APP_USERNAME - Login username/email
|
|
13
|
+
# APP_PASSWORD - Login password
|
|
14
|
+
#
|
|
15
|
+
# Two modes:
|
|
16
|
+
# 1. Discovery mode (default): Shows form structure so you can identify refs
|
|
17
|
+
# 2. Login mode: Performs actual login after you update the refs
|
|
18
|
+
#
|
|
19
|
+
# Setup steps:
|
|
20
|
+
# 1. Run once to see form structure (discovery mode)
|
|
21
|
+
# 2. Update refs in LOGIN FLOW section below
|
|
22
|
+
# 3. Set APP_USERNAME and APP_PASSWORD
|
|
23
|
+
# 4. Delete the DISCOVERY section
|
|
24
|
+
|
|
25
|
+
set -euo pipefail
|
|
26
|
+
|
|
27
|
+
LOGIN_URL="${1:?Usage: $0 <login-url> [state-file]}"
|
|
28
|
+
STATE_FILE="${2:-./auth-state.json}"
|
|
29
|
+
|
|
30
|
+
echo "Authentication workflow: $LOGIN_URL"
|
|
31
|
+
|
|
32
|
+
# ================================================================
|
|
33
|
+
# SAVED STATE: Skip login if valid saved state exists
|
|
34
|
+
# ================================================================
|
|
35
|
+
if [[ -f "$STATE_FILE" ]]; then
|
|
36
|
+
echo "Loading saved state from $STATE_FILE..."
|
|
37
|
+
if agent-browser --state "$STATE_FILE" open "$LOGIN_URL" 2>/dev/null; then
|
|
38
|
+
agent-browser wait --load networkidle
|
|
39
|
+
|
|
40
|
+
CURRENT_URL=$(agent-browser get url)
|
|
41
|
+
if [[ "$CURRENT_URL" != *"login"* ]] && [[ "$CURRENT_URL" != *"signin"* ]]; then
|
|
42
|
+
echo "Session restored successfully"
|
|
43
|
+
agent-browser snapshot -i
|
|
44
|
+
exit 0
|
|
45
|
+
fi
|
|
46
|
+
echo "Session expired, performing fresh login..."
|
|
47
|
+
agent-browser close 2>/dev/null || true
|
|
48
|
+
else
|
|
49
|
+
echo "Failed to load state, re-authenticating..."
|
|
50
|
+
fi
|
|
51
|
+
rm -f "$STATE_FILE"
|
|
52
|
+
fi
|
|
53
|
+
|
|
54
|
+
# ================================================================
|
|
55
|
+
# DISCOVERY MODE: Shows form structure (delete after setup)
|
|
56
|
+
# ================================================================
|
|
57
|
+
echo "Opening login page..."
|
|
58
|
+
agent-browser open "$LOGIN_URL"
|
|
59
|
+
agent-browser wait --load networkidle
|
|
60
|
+
|
|
61
|
+
echo ""
|
|
62
|
+
echo "Login form structure:"
|
|
63
|
+
echo "---"
|
|
64
|
+
agent-browser snapshot -i
|
|
65
|
+
echo "---"
|
|
66
|
+
echo ""
|
|
67
|
+
echo "Next steps:"
|
|
68
|
+
echo " 1. Note the refs: username=@e?, password=@e?, submit=@e?"
|
|
69
|
+
echo " 2. Update the LOGIN FLOW section below with your refs"
|
|
70
|
+
echo " 3. Set: export APP_USERNAME='...' APP_PASSWORD='...'"
|
|
71
|
+
echo " 4. Delete this DISCOVERY MODE section"
|
|
72
|
+
echo ""
|
|
73
|
+
agent-browser close
|
|
74
|
+
exit 0
|
|
75
|
+
|
|
76
|
+
# ================================================================
|
|
77
|
+
# LOGIN FLOW: Uncomment and customize after discovery
|
|
78
|
+
# ================================================================
|
|
79
|
+
# : "${APP_USERNAME:?Set APP_USERNAME environment variable}"
|
|
80
|
+
# : "${APP_PASSWORD:?Set APP_PASSWORD environment variable}"
|
|
81
|
+
#
|
|
82
|
+
# agent-browser open "$LOGIN_URL"
|
|
83
|
+
# agent-browser wait --load networkidle
|
|
84
|
+
# agent-browser snapshot -i
|
|
85
|
+
#
|
|
86
|
+
# # Fill credentials (update refs to match your form)
|
|
87
|
+
# agent-browser fill @e1 "$APP_USERNAME"
|
|
88
|
+
# agent-browser fill @e2 "$APP_PASSWORD"
|
|
89
|
+
# agent-browser click @e3
|
|
90
|
+
# agent-browser wait --load networkidle
|
|
91
|
+
#
|
|
92
|
+
# # Verify login succeeded
|
|
93
|
+
# FINAL_URL=$(agent-browser get url)
|
|
94
|
+
# if [[ "$FINAL_URL" == *"login"* ]] || [[ "$FINAL_URL" == *"signin"* ]]; then
|
|
95
|
+
# echo "Login failed - still on login page"
|
|
96
|
+
# agent-browser screenshot /tmp/login-failed.png
|
|
97
|
+
# agent-browser close
|
|
98
|
+
# exit 1
|
|
99
|
+
# fi
|
|
100
|
+
#
|
|
101
|
+
# # Save state for future runs
|
|
102
|
+
# echo "Saving state to $STATE_FILE"
|
|
103
|
+
# agent-browser state save "$STATE_FILE"
|
|
104
|
+
# echo "Login successful"
|
|
105
|
+
# agent-browser snapshot -i
|
|
@@ -0,0 +1,69 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# Template: Content Capture Workflow
|
|
3
|
+
# Purpose: Extract content from web pages (text, screenshots, PDF)
|
|
4
|
+
# Usage: ./capture-workflow.sh <url> [output-dir]
|
|
5
|
+
#
|
|
6
|
+
# Outputs:
|
|
7
|
+
# - page-full.png: Full page screenshot
|
|
8
|
+
# - page-structure.txt: Page element structure with refs
|
|
9
|
+
# - page-text.txt: All text content
|
|
10
|
+
# - page.pdf: PDF version
|
|
11
|
+
#
|
|
12
|
+
# Optional: Load auth state for protected pages
|
|
13
|
+
|
|
14
|
+
set -euo pipefail
|
|
15
|
+
|
|
16
|
+
TARGET_URL="${1:?Usage: $0 <url> [output-dir]}"
|
|
17
|
+
OUTPUT_DIR="${2:-.}"
|
|
18
|
+
|
|
19
|
+
echo "Capturing: $TARGET_URL"
|
|
20
|
+
mkdir -p "$OUTPUT_DIR"
|
|
21
|
+
|
|
22
|
+
# Optional: Load authentication state
|
|
23
|
+
# if [[ -f "./auth-state.json" ]]; then
|
|
24
|
+
# echo "Loading authentication state..."
|
|
25
|
+
# agent-browser state load "./auth-state.json"
|
|
26
|
+
# fi
|
|
27
|
+
|
|
28
|
+
# Navigate to target
|
|
29
|
+
agent-browser open "$TARGET_URL"
|
|
30
|
+
agent-browser wait --load networkidle
|
|
31
|
+
|
|
32
|
+
# Get metadata
|
|
33
|
+
TITLE=$(agent-browser get title)
|
|
34
|
+
URL=$(agent-browser get url)
|
|
35
|
+
echo "Title: $TITLE"
|
|
36
|
+
echo "URL: $URL"
|
|
37
|
+
|
|
38
|
+
# Capture full page screenshot
|
|
39
|
+
agent-browser screenshot --full "$OUTPUT_DIR/page-full.png"
|
|
40
|
+
echo "Saved: $OUTPUT_DIR/page-full.png"
|
|
41
|
+
|
|
42
|
+
# Get page structure with refs
|
|
43
|
+
agent-browser snapshot -i > "$OUTPUT_DIR/page-structure.txt"
|
|
44
|
+
echo "Saved: $OUTPUT_DIR/page-structure.txt"
|
|
45
|
+
|
|
46
|
+
# Extract all text content
|
|
47
|
+
agent-browser get text body > "$OUTPUT_DIR/page-text.txt"
|
|
48
|
+
echo "Saved: $OUTPUT_DIR/page-text.txt"
|
|
49
|
+
|
|
50
|
+
# Save as PDF
|
|
51
|
+
agent-browser pdf "$OUTPUT_DIR/page.pdf"
|
|
52
|
+
echo "Saved: $OUTPUT_DIR/page.pdf"
|
|
53
|
+
|
|
54
|
+
# Optional: Extract specific elements using refs from structure
|
|
55
|
+
# agent-browser get text @e5 > "$OUTPUT_DIR/main-content.txt"
|
|
56
|
+
|
|
57
|
+
# Optional: Handle infinite scroll pages
|
|
58
|
+
# for i in {1..5}; do
|
|
59
|
+
# agent-browser scroll down 1000
|
|
60
|
+
# agent-browser wait 1000
|
|
61
|
+
# done
|
|
62
|
+
# agent-browser screenshot --full "$OUTPUT_DIR/page-scrolled.png"
|
|
63
|
+
|
|
64
|
+
# Cleanup
|
|
65
|
+
agent-browser close
|
|
66
|
+
|
|
67
|
+
echo ""
|
|
68
|
+
echo "Capture complete:"
|
|
69
|
+
ls -la "$OUTPUT_DIR"
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
#!/bin/bash
|
|
2
|
+
# Template: Form Automation Workflow
|
|
3
|
+
# Purpose: Fill and submit web forms with validation
|
|
4
|
+
# Usage: ./form-automation.sh <form-url>
|
|
5
|
+
#
|
|
6
|
+
# This template demonstrates the snapshot-interact-verify pattern:
|
|
7
|
+
# 1. Navigate to form
|
|
8
|
+
# 2. Snapshot to get element refs
|
|
9
|
+
# 3. Fill fields using refs
|
|
10
|
+
# 4. Submit and verify result
|
|
11
|
+
#
|
|
12
|
+
# Customize: Update the refs (@e1, @e2, etc.) based on your form's snapshot output
|
|
13
|
+
|
|
14
|
+
set -euo pipefail
|
|
15
|
+
|
|
16
|
+
FORM_URL="${1:?Usage: $0 <form-url>}"
|
|
17
|
+
|
|
18
|
+
echo "Form automation: $FORM_URL"
|
|
19
|
+
|
|
20
|
+
# Step 1: Navigate to form
|
|
21
|
+
agent-browser open "$FORM_URL"
|
|
22
|
+
agent-browser wait --load networkidle
|
|
23
|
+
|
|
24
|
+
# Step 2: Snapshot to discover form elements
|
|
25
|
+
echo ""
|
|
26
|
+
echo "Form structure:"
|
|
27
|
+
agent-browser snapshot -i
|
|
28
|
+
|
|
29
|
+
# Step 3: Fill form fields (customize these refs based on snapshot output)
|
|
30
|
+
#
|
|
31
|
+
# Common field types:
|
|
32
|
+
# agent-browser fill @e1 "John Doe" # Text input
|
|
33
|
+
# agent-browser fill @e2 "user@example.com" # Email input
|
|
34
|
+
# agent-browser fill @e3 "SecureP@ss123" # Password input
|
|
35
|
+
# agent-browser select @e4 "Option Value" # Dropdown
|
|
36
|
+
# agent-browser check @e5 # Checkbox
|
|
37
|
+
# agent-browser click @e6 # Radio button
|
|
38
|
+
# agent-browser fill @e7 "Multi-line text" # Textarea
|
|
39
|
+
# agent-browser upload @e8 /path/to/file.pdf # File upload
|
|
40
|
+
#
|
|
41
|
+
# Uncomment and modify:
|
|
42
|
+
# agent-browser fill @e1 "Test User"
|
|
43
|
+
# agent-browser fill @e2 "test@example.com"
|
|
44
|
+
# agent-browser click @e3 # Submit button
|
|
45
|
+
|
|
46
|
+
# Step 4: Wait for submission
|
|
47
|
+
# agent-browser wait --load networkidle
|
|
48
|
+
# agent-browser wait --url "**/success" # Or wait for redirect
|
|
49
|
+
|
|
50
|
+
# Step 5: Verify result
|
|
51
|
+
echo ""
|
|
52
|
+
echo "Result:"
|
|
53
|
+
agent-browser get url
|
|
54
|
+
agent-browser snapshot -i
|
|
55
|
+
|
|
56
|
+
# Optional: Capture evidence
|
|
57
|
+
agent-browser screenshot /tmp/form-result.png
|
|
58
|
+
echo "Screenshot saved: /tmp/form-result.png"
|
|
59
|
+
|
|
60
|
+
# Cleanup
|
|
61
|
+
agent-browser close
|
|
62
|
+
echo "Done"
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: derive-client
|
|
3
|
+
description: Reverse-engineer a website's internal API by recording browser traffic into a HAR file, then generate a standalone client or CLI that calls the endpoints directly, with no browser needed after the first recording. Use when asked to "derive a client", "build a CLI for <site>", "reverse engineer this site's API", "record network requests", "turn this site into an API", or when the same site will be automated repeatedly and direct HTTP calls would beat driving the browser every time.
|
|
4
|
+
allowed-tools: Bash(agent-browser:*), Bash(npx agent-browser:*)
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Derive an API client from a recorded session
|
|
8
|
+
|
|
9
|
+
Driving a browser is the right tool for the first visit and the wrong tool for the hundredth. This skill records a site's network traffic once while you use it, then turns the captured requests into a standalone client (script, CLI, or library) that talks to the site's internal API directly.
|
|
10
|
+
|
|
11
|
+
The recording alone contains everything needed: agent-browser embeds text response bodies (JSON/HTML/JS) in the HAR by default, so endpoint shapes can be studied offline after the browser is closed.
|
|
12
|
+
|
|
13
|
+
## Workflow
|
|
14
|
+
|
|
15
|
+
```
|
|
16
|
+
1. Record Start HAR capture, drive the flows you want in the client
|
|
17
|
+
2. Identify Find the real API endpoints among the noise
|
|
18
|
+
3. Extract Pull request shapes, response schemas, and auth material
|
|
19
|
+
4. Generate Write the client, one function per flow
|
|
20
|
+
5. Verify Call every endpoint for real before declaring done
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
## 1. Record
|
|
24
|
+
|
|
25
|
+
```bash
|
|
26
|
+
agent-browser network har start # embeds text response bodies by default
|
|
27
|
+
# ... drive the site: search, open a detail page, paginate, etc. ...
|
|
28
|
+
agent-browser network har stop /tmp/site.har
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
- Exercise **every flow the client should support**, and run each one at least twice with different inputs (two search terms, two detail pages). Diffing the recorded URLs reveals which parts are parameters.
|
|
32
|
+
- If the site needs login, log in **before** starting the HAR so credentials don't land in the recording unnecessarily. The session cookies are exported separately in step 3.
|
|
33
|
+
- `--content all` embeds binary bodies too (base64); `--content none` disables embedding. Per-body cap is 2 MB.
|
|
34
|
+
|
|
35
|
+
While the session is still open, `agent-browser network requests` and `network request <id>` give the same data interactively — but only the HAR survives navigation and browser close, so prefer it for anything multi-page.
|
|
36
|
+
|
|
37
|
+
## 2. Identify endpoints
|
|
38
|
+
|
|
39
|
+
Query the HAR with `jq`:
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
# All JSON API calls: method, URL, status
|
|
43
|
+
jq -r '.log.entries[]
|
|
44
|
+
| select(.response.content.mimeType | test("json"))
|
|
45
|
+
| "\(.request.method) \(.response.status) \(.request.url)"' /tmp/site.har
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Ignore analytics and infrastructure noise: telemetry endpoints (`/collect`, `/track`, `/beacon`, `/log`), third-party domains (google-analytics, segment, sentry, datadog, intercom, hotjar), and static assets. The real API is usually first-party, JSON, and correlates with the actions you performed.
|
|
49
|
+
|
|
50
|
+
## 3. Extract shapes and auth
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
# Full detail for one endpoint: request headers, POST body, response body
|
|
54
|
+
jq '.log.entries[] | select(.request.url | test("api/search"))
|
|
55
|
+
| {request: {method: .request.method, headers: .request.headers,
|
|
56
|
+
postData: .request.postData.text},
|
|
57
|
+
response: .response.content.text}' /tmp/site.har
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
- **Response schema**: read `.response.content.text` — this is the real payload, use it to derive types.
|
|
61
|
+
- **Auth**: compare request headers across endpoints. Look for `authorization`, `cookie`, `x-csrf-token`, `x-api-key`, and site-specific `x-*` headers. Replay only the ones that matter — test by omission in step 5.
|
|
62
|
+
- **Cookies**: export the live session with `agent-browser cookies get --json > cookies.json` for the client to load at runtime. Never hardcode cookie values into generated source.
|
|
63
|
+
|
|
64
|
+
## 4. Generate the client
|
|
65
|
+
|
|
66
|
+
- One function per recorded flow (`search(query)`, `getItem(id)`), typed from the observed response bodies.
|
|
67
|
+
- Auth material (cookies, bearer tokens) loads from a file or environment variable, with a clear error telling the user to re-run the browser login when it expires.
|
|
68
|
+
- Reproduce the headers the API actually requires — some sites 403 without a matching `user-agent`, `referer`, or `x-requested-with`.
|
|
69
|
+
- Keep pagination, sort, and filter parameters that appeared in the recorded query strings as function options.
|
|
70
|
+
|
|
71
|
+
## 5. Verify
|
|
72
|
+
|
|
73
|
+
Call every generated function against the live API and compare the response shape with the recording. Common failures:
|
|
74
|
+
|
|
75
|
+
| Symptom | Cause | Fix |
|
|
76
|
+
|---------|-------|-----|
|
|
77
|
+
| 401/403 | Expired or missing session | Re-login via agent-browser, re-export cookies |
|
|
78
|
+
| 403/419 on writes | CSRF token is per-session or per-form | Fetch the token endpoint first, or keep that flow browser-driven |
|
|
79
|
+
| Works then breaks | Signed/expiring request params | Fall back to the browser for that step; derive the rest |
|
|
80
|
+
| Different shape than HAR | A/B tests or geo-dependent responses | Re-record and treat the union as optional fields |
|
|
81
|
+
|
|
82
|
+
## Caveats
|
|
83
|
+
|
|
84
|
+
- Internal APIs are unversioned and change without notice — keep the HAR so the client can be re-derived.
|
|
85
|
+
- Respect the site's terms of service and rate limits; add delays for bulk fetching.
|
|
86
|
+
- HAR files contain live session credentials (cookies, tokens, POST bodies). Treat them like secrets: keep them out of version control and delete them when done.
|