@testdriverai/agent 7.11.202-test → 7.11.203-test
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ai/skills/testdriver-bring-your-own-machine/SKILL.md +320 -0
- package/ai/skills/testdriver-customizing-devices/SKILL.md +1 -1
- package/ai/skills/testdriver-self-hosted/SKILL.md +16 -7
- package/docs/_data/examples-manifest.json +40 -40
- package/docs/bring-your-own-machine.mdx +321 -0
- package/docs/customizing-devices.mdx +1 -1
- package/docs/docs.json +2 -1
- package/docs/examples/assert.mdx +1 -1
- package/docs/examples/element-not-found.mdx +1 -1
- package/docs/examples/findall-coffee-icons.mdx +1 -1
- package/docs/examples/hover-image.mdx +1 -1
- package/docs/examples/hover-text-with-description.mdx +1 -1
- package/docs/examples/hover-text.mdx +1 -1
- package/docs/examples/installer.mdx +1 -1
- package/docs/examples/launch-vscode-linux.mdx +1 -1
- package/docs/examples/parse.mdx +1 -1
- package/docs/examples/press-keys.mdx +1 -1
- package/docs/examples/scroll-keyboard.mdx +1 -1
- package/docs/examples/scroll.mdx +1 -1
- package/docs/examples/type.mdx +1 -1
- package/docs/self-hosted.mdx +16 -7
- package/package.json +1 -1
|
@@ -0,0 +1,320 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: testdriver:bring-your-own-machine
|
|
3
|
+
description: Run the TestDriver runner on a laptop, desktop, or VM you control and drive it from your tests.
|
|
4
|
+
---
|
|
5
|
+
<!-- Generated from bring-your-own-machine.mdx. DO NOT EDIT. -->
|
|
6
|
+
|
|
7
|
+
TestDriver normally provisions a cloud sandbox for you. You can instead run the **runner** on a machine you own: a developer laptop, an office Mac, a lab Windows box, or a VM in your own data center. Your tests then drive that machine instead of a cloud sandbox.
|
|
8
|
+
|
|
9
|
+
This is useful when the machine under test cannot be replicated in the cloud: specific hardware, a licensed desktop application, a device plugged into USB, or an app that only reaches an internal network.
|
|
10
|
+
|
|
11
|
+
```mermaid
|
|
12
|
+
graph LR
|
|
13
|
+
A[Your Test] --> B[TestDriver API]
|
|
14
|
+
B --> C[Ably Channel]
|
|
15
|
+
C --> D[Runner on your machine]
|
|
16
|
+
D --> C
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
<Note>
|
|
20
|
+
The runner only makes **outbound** connections, to the TestDriver API and to Ably. You do not need to open an inbound port or expose the machine to the internet.
|
|
21
|
+
</Note>
|
|
22
|
+
|
|
23
|
+
## Choose your path
|
|
24
|
+
|
|
25
|
+
| Your machine | Use |
|
|
26
|
+
|---|---|
|
|
27
|
+
| Windows EC2 instance in your AWS account | [AWS Setup Guide](/aws-setup). Instances are spawned and provisioned automatically. |
|
|
28
|
+
| Any other machine (macOS, Linux, Windows, on-prem VM) | This guide. You start the runner yourself. |
|
|
29
|
+
|
|
30
|
+
<Tip>
|
|
31
|
+
If you want unlimited execution, assisted setup, and custom VM images, see [Self-Hosted (Enterprise)](/self-hosted).
|
|
32
|
+
</Tip>
|
|
33
|
+
|
|
34
|
+
## Prerequisites
|
|
35
|
+
|
|
36
|
+
Every machine that runs the runner needs:
|
|
37
|
+
|
|
38
|
+
- **Node.js 18 or newer**
|
|
39
|
+
- **Python 3** with `pyautogui` and `Pillow`
|
|
40
|
+
- **A real, unlocked desktop session.** The runner drives the mouse and keyboard, so the screen must be awake and not locked. Disable screen savers, sleep, and lock timers.
|
|
41
|
+
- **Google Chrome** on the `PATH`, if your tests launch a browser
|
|
42
|
+
- Outbound HTTPS and WebSocket access to `api.testdriver.ai` and `*.ably.io`
|
|
43
|
+
|
|
44
|
+
Install the Python dependencies:
|
|
45
|
+
|
|
46
|
+
<Tabs>
|
|
47
|
+
<Tab title="macOS">
|
|
48
|
+
```bash
|
|
49
|
+
brew install python
|
|
50
|
+
pip3 install pyautogui Pillow
|
|
51
|
+
```
|
|
52
|
+
</Tab>
|
|
53
|
+
<Tab title="Linux">
|
|
54
|
+
```bash
|
|
55
|
+
sudo apt-get install -y python3-pip python3-xlib
|
|
56
|
+
pip3 install pyautogui Pillow
|
|
57
|
+
```
|
|
58
|
+
</Tab>
|
|
59
|
+
<Tab title="Windows">
|
|
60
|
+
```powershell
|
|
61
|
+
pip install pyautogui Pillow
|
|
62
|
+
```
|
|
63
|
+
</Tab>
|
|
64
|
+
</Tabs>
|
|
65
|
+
|
|
66
|
+
### Platform notes
|
|
67
|
+
|
|
68
|
+
<AccordionGroup>
|
|
69
|
+
<Accordion title="macOS: grant Accessibility and Screen Recording permissions">
|
|
70
|
+
macOS blocks synthetic input and screen capture until you allow it explicitly. Without this the runner connects but every click and screenshot silently fails.
|
|
71
|
+
|
|
72
|
+
Open **System Settings → Privacy & Security** and add your terminal application (Terminal, iTerm, or whichever app starts the runner) to **both**:
|
|
73
|
+
|
|
74
|
+
- **Accessibility** (lets the runner move the mouse and type)
|
|
75
|
+
- **Screen Recording** (lets the runner take screenshots)
|
|
76
|
+
|
|
77
|
+
Quit and reopen the terminal after granting permissions, otherwise the old process keeps the old denial.
|
|
78
|
+
|
|
79
|
+
On Retina displays, screenshots are captured at physical pixels and downscaled to logical points so that coordinates returned by the AI line up with where clicks actually land. No configuration is needed.
|
|
80
|
+
</Accordion>
|
|
81
|
+
|
|
82
|
+
<Accordion title="Linux: provide a display">
|
|
83
|
+
The runner needs an X11 display. On a desktop machine, set `DISPLAY` to your existing session. On a headless server, start a virtual framebuffer first:
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
sudo apt-get install -y xvfb xfce4 xfce4-terminal dbus-x11 wmctrl
|
|
87
|
+
Xvfb :0 -screen 0 1366x768x24 &
|
|
88
|
+
export DISPLAY=:0
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
To watch the session, add a VNC server and noVNC:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
sudo apt-get install -y x11vnc novnc websockify
|
|
95
|
+
x11vnc -display :0 -forever -shared &
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Disable screen blanking so the desktop stays visible:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
xset s off && xset s noblank && xset -dpms
|
|
102
|
+
```
|
|
103
|
+
</Accordion>
|
|
104
|
+
|
|
105
|
+
<Accordion title="Windows: keep the console session active">
|
|
106
|
+
Run the runner in a console or RDP session that stays signed in. If the session locks or disconnects, the desktop is torn down and automation stops. Configure the machine to auto-logon and disable the lock screen timeout.
|
|
107
|
+
</Accordion>
|
|
108
|
+
</AccordionGroup>
|
|
109
|
+
|
|
110
|
+
## Step 1: Install the runner
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
npm install -g @testdriverai/runner
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
This installs two commands. You use `testdriver-sandbox-agent`, which connects your machine to a TestDriver session.
|
|
117
|
+
|
|
118
|
+
Verify the install:
|
|
119
|
+
|
|
120
|
+
```bash
|
|
121
|
+
testdriver-sandbox-agent --help || echo "installed"
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
## Step 2: Register the machine
|
|
125
|
+
|
|
126
|
+
Ask the API for a session bound to your machine's IP address:
|
|
127
|
+
|
|
128
|
+
```bash
|
|
129
|
+
curl -X POST https://api.testdriver.ai/api/v7/sandbox/authenticate \
|
|
130
|
+
-H "Content-Type: application/json" \
|
|
131
|
+
-d '{
|
|
132
|
+
"apiKey": "your_api_key",
|
|
133
|
+
"ip": "192.168.1.50",
|
|
134
|
+
"os": "linux",
|
|
135
|
+
"keepAlive": 900000
|
|
136
|
+
}'
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
The response contains the sandbox ID and the Ably credentials the runner needs:
|
|
140
|
+
|
|
141
|
+
```json
|
|
142
|
+
{
|
|
143
|
+
"success": true,
|
|
144
|
+
"sandboxId": "sb-69cce96e-1e0",
|
|
145
|
+
"agent": {
|
|
146
|
+
"token": {
|
|
147
|
+
"token": "aFC5YQ.Ki9G9...",
|
|
148
|
+
"keyName": "aFC5YQ.ZIdTww",
|
|
149
|
+
"issued": 1791908438832,
|
|
150
|
+
"expires": 1791930038832,
|
|
151
|
+
"capability": "{\"testdriver:...:sb-69cce96e-1e0\":[\"history\",\"presence\",\"publish\",\"subscribe\"]}",
|
|
152
|
+
"clientId": "agent-sb-69cce96e-1e0"
|
|
153
|
+
},
|
|
154
|
+
"channel": "testdriver:...:sb-69cce96e-1e0"
|
|
155
|
+
}
|
|
156
|
+
}
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
<Note>
|
|
160
|
+
Pass `os: "linux"` on macOS. The value only describes the session to the API; the runner detects the real platform itself.
|
|
161
|
+
</Note>
|
|
162
|
+
|
|
163
|
+
### Write the config file
|
|
164
|
+
|
|
165
|
+
The runner reads its credentials from a JSON file on disk. Copy `sandboxId` and the whole `agent` block into it:
|
|
166
|
+
|
|
167
|
+
| Platform | Config file path |
|
|
168
|
+
|---|---|
|
|
169
|
+
| macOS / Linux | `/tmp/testdriver-agent.json` |
|
|
170
|
+
| Windows | `C:\Windows\Temp\testdriver-agent.json` |
|
|
171
|
+
|
|
172
|
+
```json /tmp/testdriver-agent.json
|
|
173
|
+
{
|
|
174
|
+
"sandboxId": "sb-69cce96e-1e0",
|
|
175
|
+
"apiRoot": "https://api.testdriver.ai",
|
|
176
|
+
"apiKey": "your_api_key",
|
|
177
|
+
"ably": {
|
|
178
|
+
"token": {
|
|
179
|
+
"token": "aFC5YQ.Ki9G9...",
|
|
180
|
+
"keyName": "aFC5YQ.ZIdTww",
|
|
181
|
+
"issued": 1791908438832,
|
|
182
|
+
"expires": 1791930038832,
|
|
183
|
+
"capability": "{\"testdriver:...:sb-69cce96e-1e0\":[\"history\",\"presence\",\"publish\",\"subscribe\"]}",
|
|
184
|
+
"clientId": "agent-sb-69cce96e-1e0"
|
|
185
|
+
},
|
|
186
|
+
"channel": "testdriver:...:sb-69cce96e-1e0"
|
|
187
|
+
}
|
|
188
|
+
}
|
|
189
|
+
```
|
|
190
|
+
|
|
191
|
+
| Field | Source | Purpose |
|
|
192
|
+
|---|---|---|
|
|
193
|
+
| `sandboxId` | `sandboxId` from the response | Identifies this session. Your test attaches to it. |
|
|
194
|
+
| `apiRoot` | The API you called | Where the runner uploads screenshots |
|
|
195
|
+
| `apiKey` | Your team API key | Authenticates those uploads |
|
|
196
|
+
| `ably.token` | `agent.token` from the response, copied verbatim | Realtime credentials |
|
|
197
|
+
| `ably.channel` | `agent.channel` from the response | Channel the runner listens on |
|
|
198
|
+
|
|
199
|
+
To put the file somewhere else, set `CONFIG_PATH` to its full path before starting the runner.
|
|
200
|
+
|
|
201
|
+
<Warning>
|
|
202
|
+
This file holds a session token and your API key. Keep it readable only by the user running the runner, do not commit it, and delete it when the machine is no longer in use.
|
|
203
|
+
</Warning>
|
|
204
|
+
|
|
205
|
+
## Step 3: Start the runner
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
testdriver-sandbox-agent
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
You should see the runner connect and announce itself:
|
|
212
|
+
|
|
213
|
+
```
|
|
214
|
+
[agent] TestDriver Sandbox Agent starting...
|
|
215
|
+
[agent] Sandbox ID: sb-69cce96e-1e0
|
|
216
|
+
[agent] Transport: Ably
|
|
217
|
+
[agent] Automation module initialized
|
|
218
|
+
[realtime] Publishing control: type=runner.ready
|
|
219
|
+
[agent] Agent ready — listening for commands via Ably
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
Leave this process running. On Linux, set `DISPLAY` first:
|
|
223
|
+
|
|
224
|
+
```bash
|
|
225
|
+
DISPLAY=:0 testdriver-sandbox-agent
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
## Step 4: Point your test at the machine
|
|
229
|
+
|
|
230
|
+
Pass the machine's `ip` and the sandbox ID from step 2:
|
|
231
|
+
|
|
232
|
+
```javascript tests/my-machine.test.mjs
|
|
233
|
+
import { describe, expect, it } from "vitest";
|
|
234
|
+
import { TestDriver } from "testdriverai/vitest/hooks";
|
|
235
|
+
|
|
236
|
+
describe("My machine", () => {
|
|
237
|
+
it("runs against my own hardware", async (context) => {
|
|
238
|
+
const testdriver = TestDriver(context, { newSandbox: false });
|
|
239
|
+
|
|
240
|
+
await testdriver.connect({
|
|
241
|
+
ip: process.env.TD_MACHINE_IP,
|
|
242
|
+
sandboxId: process.env.TD_SANDBOX_ID,
|
|
243
|
+
});
|
|
244
|
+
|
|
245
|
+
const button = await testdriver.find("Sign In button");
|
|
246
|
+
await button.click();
|
|
247
|
+
|
|
248
|
+
const result = await testdriver.assert("the dashboard is visible");
|
|
249
|
+
expect(result).toBeTruthy();
|
|
250
|
+
});
|
|
251
|
+
});
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
Run it:
|
|
255
|
+
|
|
256
|
+
```bash
|
|
257
|
+
TD_MACHINE_IP=192.168.1.50 TD_SANDBOX_ID=sb-69cce96e-1e0 vitest run
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
<Note>
|
|
261
|
+
`newSandbox: false` stops the SDK from provisioning a cloud sandbox. Without it, your test runs in the cloud and ignores your machine.
|
|
262
|
+
</Note>
|
|
263
|
+
|
|
264
|
+
<Note>
|
|
265
|
+
Each config file covers one session. The Ably token in it expires (see its `expires` field), so repeat steps 2 and 3 to get fresh credentials before the next run.
|
|
266
|
+
</Note>
|
|
267
|
+
|
|
268
|
+
## Logs
|
|
269
|
+
|
|
270
|
+
| Platform | Runner log |
|
|
271
|
+
|---|---|
|
|
272
|
+
| macOS / Linux | `/tmp/testdriver-agent.log` |
|
|
273
|
+
| Windows | `C:\Windows\Temp\testdriver-agent.log` |
|
|
274
|
+
|
|
275
|
+
The runner also writes a status file next to the log (`testdriver-agent-status.json`) with the sandbox ID, process ID, detected IP, and start time.
|
|
276
|
+
|
|
277
|
+
## Security
|
|
278
|
+
|
|
279
|
+
- **Use a dedicated machine or user account.** The runner can click anything on the desktop and run shell commands sent by your tests. Do not point it at a machine holding personal credentials or production access.
|
|
280
|
+
- **Treat the config file as a secret.** It contains a session token and your API key.
|
|
281
|
+
- **Do not expose VNC to the internet.** If you add VNC for debugging, bind it to localhost or your VPN and tunnel in over SSH.
|
|
282
|
+
- **Rotate your API key** if a machine is decommissioned or shared.
|
|
283
|
+
|
|
284
|
+
## Troubleshooting
|
|
285
|
+
|
|
286
|
+
<AccordionGroup>
|
|
287
|
+
<Accordion title="Sandbox has expired and is no longer available">
|
|
288
|
+
The session ended, which happens when the runner stops or your test disconnects. Repeat step 2 to get a fresh sandbox ID and token, update the config file, restart the runner, and use the new ID in your test.
|
|
289
|
+
</Accordion>
|
|
290
|
+
|
|
291
|
+
<Accordion title="Runner starts but clicks and screenshots do nothing (macOS)">
|
|
292
|
+
Accessibility or Screen Recording permission is missing. Add your terminal application to both lists under **System Settings → Privacy & Security**, then fully quit and reopen the terminal.
|
|
293
|
+
</Accordion>
|
|
294
|
+
|
|
295
|
+
<Accordion title="Waiting for config file">
|
|
296
|
+
The runner cannot find `/tmp/testdriver-agent.json` (or `C:\Windows\Temp\testdriver-agent.json`). Write the config file on the same machine and readable by the same user that starts the runner. To use a different location, set `CONFIG_PATH`.
|
|
297
|
+
</Accordion>
|
|
298
|
+
|
|
299
|
+
<Accordion title="Test provisions a cloud sandbox instead of using my machine">
|
|
300
|
+
Pass `newSandbox: false` to `TestDriver()` and make sure both `ip` and `sandboxId` reach `connect()`. If either is undefined, the SDK falls back to cloud provisioning.
|
|
301
|
+
</Accordion>
|
|
302
|
+
|
|
303
|
+
<Accordion title="Invalid API key">
|
|
304
|
+
Confirm `TD_API_KEY` matches the environment in `TD_API_ROOT`. Keys are scoped per environment. Get your key at [console.testdriver.ai/team](https://console.testdriver.ai/team).
|
|
305
|
+
</Accordion>
|
|
306
|
+
|
|
307
|
+
<Accordion title="Screenshots are blank or show a frozen desktop (Linux)">
|
|
308
|
+
`DISPLAY` points at a display with no desktop running, or the screen blanked. Start a desktop on that display and disable blanking with `xset s off && xset s noblank && xset -dpms`.
|
|
309
|
+
</Accordion>
|
|
310
|
+
</AccordionGroup>
|
|
311
|
+
|
|
312
|
+
## Environment variables
|
|
313
|
+
|
|
314
|
+
| Variable | Default | Purpose |
|
|
315
|
+
|---|---|---|
|
|
316
|
+
| `TD_API_KEY` | Required | Your team API key |
|
|
317
|
+
| `TD_API_ROOT` | `https://api.testdriver.ai` | API server to register with |
|
|
318
|
+
| `CONFIG_PATH` | `/tmp/testdriver-agent.json` | Override the runner config file location |
|
|
319
|
+
| `DISPLAY` | `:0` | X11 display (Linux) |
|
|
320
|
+
| `SENTRY_DSN` | Unset | Error reporting endpoint |
|
|
@@ -123,7 +123,7 @@ const testdriver = TestDriver(context, {
|
|
|
123
123
|
|
|
124
124
|
### IP Target
|
|
125
125
|
|
|
126
|
-
If self-hosting TestDriver, use `ip` to specify the device IP. See [
|
|
126
|
+
If self-hosting TestDriver, use `ip` to specify the device IP. See [Run on Your Own Machine](/bring-your-own-machine) for details.
|
|
127
127
|
|
|
128
128
|
```javascript
|
|
129
129
|
const testdriver = TestDriver(context, {
|
|
@@ -48,13 +48,22 @@ Run test sandboxes on your AWS infrastructure while using TestDriver's hosted da
|
|
|
48
48
|
**Bring your own Gemini API key.** TestDriver uses Google Gemini for vision and reasoning. When supplying your own Gemini key, it must be a [Gemini Enterprise Agent Platform express mode API key](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/start/api-keys?usertype=expressmode) — standard Google AI Studio (Generative Language API) keys are not supported for self-hosted deployments.
|
|
49
49
|
</Note>
|
|
50
50
|
|
|
51
|
-
<
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
51
|
+
<CardGroup cols={2}>
|
|
52
|
+
<Card
|
|
53
|
+
title="AWS Setup Guide"
|
|
54
|
+
icon="aws"
|
|
55
|
+
href="/aws-setup"
|
|
56
|
+
>
|
|
57
|
+
Step-by-step instructions for deploying TestDriver on your AWS infrastructure using CloudFormation.
|
|
58
|
+
</Card>
|
|
59
|
+
<Card
|
|
60
|
+
title="Run on Your Own Machine"
|
|
61
|
+
icon="display"
|
|
62
|
+
href="/bring-your-own-machine"
|
|
63
|
+
>
|
|
64
|
+
Run the runner on a laptop, lab machine, or on-prem VM and drive it from your tests.
|
|
65
|
+
</Card>
|
|
66
|
+
</CardGroup>
|
|
58
67
|
|
|
59
68
|
### Air-Gapped Deployment
|
|
60
69
|
|
|
@@ -2,16 +2,16 @@
|
|
|
2
2
|
"$schema": "./examples-manifest.schema.json",
|
|
3
3
|
"examples": {
|
|
4
4
|
"assert.test.mjs": {
|
|
5
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
6
|
-
"lastUpdated": "2026-10-
|
|
5
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ce699d28997901467cb",
|
|
6
|
+
"lastUpdated": "2026-10-13T00:35:49.925Z"
|
|
7
7
|
},
|
|
8
8
|
"drag-and-drop.test.mjs": {
|
|
9
9
|
"url": "https://console.testdriver.ai/runs/69a62b3aaa712ecd3dea730a/69a62b42fc0ac3cc632a918b",
|
|
10
10
|
"lastUpdated": "2026-03-03T00:32:25.275Z"
|
|
11
11
|
},
|
|
12
12
|
"exec-pwsh.test.mjs": {
|
|
13
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
14
|
-
"lastUpdated": "2026-10-
|
|
13
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cd71adbeb4ea97f70e2",
|
|
14
|
+
"lastUpdated": "2026-10-13T00:36:06.587Z"
|
|
15
15
|
},
|
|
16
16
|
"match-image.test.mjs": {
|
|
17
17
|
"url": "https://console-test.testdriver.ai/runs/69c8738614b73310c7839412/69c8738c14b73310c783941d",
|
|
@@ -22,78 +22,78 @@
|
|
|
22
22
|
"lastUpdated": "2026-03-03T00:32:25.282Z"
|
|
23
23
|
},
|
|
24
24
|
"hover-text-with-description.test.mjs": {
|
|
25
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
26
|
-
"lastUpdated": "2026-10-
|
|
25
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cda1adbeb4ea97f70e4",
|
|
26
|
+
"lastUpdated": "2026-10-13T00:36:09.473Z"
|
|
27
27
|
},
|
|
28
28
|
"windows-installer.test.mjs": {
|
|
29
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
30
|
-
"lastUpdated": "2026-10-
|
|
29
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cdc1adbeb4ea97f70e5",
|
|
30
|
+
"lastUpdated": "2026-10-13T00:39:09.052Z"
|
|
31
31
|
},
|
|
32
32
|
"exec-output.test.mjs": {
|
|
33
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
34
|
-
"lastUpdated": "2026-10-
|
|
33
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cdd1adbeb4ea97f70e6",
|
|
34
|
+
"lastUpdated": "2026-10-13T00:36:41.224Z"
|
|
35
35
|
},
|
|
36
36
|
"chrome-extension.test.mjs": {
|
|
37
37
|
"url": "https://console-test.testdriver.ai/runs/6a87988fb7cddebeab9bfbea/6a8798913062df0fb9b85592",
|
|
38
38
|
"lastUpdated": "2026-08-21T00:15:47.131Z"
|
|
39
39
|
},
|
|
40
40
|
"launch-vscode-linux.test.mjs": {
|
|
41
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
42
|
-
"lastUpdated": "2026-10-
|
|
41
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cd81adbeb4ea97f70e3",
|
|
42
|
+
"lastUpdated": "2026-10-13T00:36:36.587Z"
|
|
43
43
|
},
|
|
44
44
|
"hover-image.test.mjs": {
|
|
45
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
46
|
-
"lastUpdated": "2026-10-
|
|
45
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cdf5a09b43706d8d5a8",
|
|
46
|
+
"lastUpdated": "2026-10-13T00:35:42.733Z"
|
|
47
47
|
},
|
|
48
48
|
"installer.test.mjs": {
|
|
49
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
50
|
-
"lastUpdated": "2026-10-
|
|
49
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ce15a09b43706d8d5aa",
|
|
50
|
+
"lastUpdated": "2026-10-13T00:35:44.622Z"
|
|
51
51
|
},
|
|
52
52
|
"type.test.mjs": {
|
|
53
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
54
|
-
"lastUpdated": "2026-10-
|
|
53
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ce299d28997901467c9",
|
|
54
|
+
"lastUpdated": "2026-10-13T00:36:46.048Z"
|
|
55
55
|
},
|
|
56
56
|
"press-keys.test.mjs": {
|
|
57
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
58
|
-
"replayUrl": "https://console-test.testdriver.ai/replay/
|
|
59
|
-
"embedUrl": "https://console-test.testdriver.ai/replay/
|
|
60
|
-
"lastUpdated": "2026-10-
|
|
57
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ce499d28997901467ca",
|
|
58
|
+
"replayUrl": "https://console-test.testdriver.ai/replay/6acd7edd1adbeb4ea97f7120?share=TSZUSVjpnSPLXWr9U5k2A",
|
|
59
|
+
"embedUrl": "https://console-test.testdriver.ai/replay/6acd7edd1adbeb4ea97f7120?share=TSZUSVjpnSPLXWr9U5k2A&embed=true",
|
|
60
|
+
"lastUpdated": "2026-10-13T00:44:18.082Z"
|
|
61
61
|
},
|
|
62
62
|
"scroll-keyboard.test.mjs": {
|
|
63
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
64
|
-
"lastUpdated": "2026-10-
|
|
63
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ce91adbeb4ea97f70e7",
|
|
64
|
+
"lastUpdated": "2026-10-13T00:36:52.511Z"
|
|
65
65
|
},
|
|
66
66
|
"scroll.test.mjs": {
|
|
67
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
68
|
-
"lastUpdated": "2026-10-
|
|
67
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cef99d28997901467ce",
|
|
68
|
+
"lastUpdated": "2026-10-13T00:36:28.649Z"
|
|
69
69
|
},
|
|
70
70
|
"scroll-until-image.test.mjs": {
|
|
71
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
72
|
-
"lastUpdated": "2026-10-
|
|
71
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ceb99d28997901467cc",
|
|
72
|
+
"lastUpdated": "2026-10-13T00:35:55.512Z"
|
|
73
73
|
},
|
|
74
74
|
"prompt.test.mjs": {
|
|
75
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
76
|
-
"lastUpdated": "2026-10-
|
|
75
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cec1adbeb4ea97f70e8",
|
|
76
|
+
"lastUpdated": "2026-10-13T00:36:55.557Z"
|
|
77
77
|
},
|
|
78
78
|
"focus-window.test.mjs": {
|
|
79
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
80
|
-
"lastUpdated": "2026-10-
|
|
79
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cee99d28997901467cd",
|
|
80
|
+
"lastUpdated": "2026-10-13T00:36:56.825Z"
|
|
81
81
|
},
|
|
82
82
|
"captcha-api.test.mjs": {
|
|
83
83
|
"url": "https://console.testdriver.ai/runs/698f7df69e27ce1528d7d087/698f7fb0d3b320ad547d9d44",
|
|
84
84
|
"lastUpdated": "2026-02-13T19:55:05.951Z"
|
|
85
85
|
},
|
|
86
86
|
"element-not-found.test.mjs": {
|
|
87
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
88
|
-
"lastUpdated": "2026-10-
|
|
87
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cf199d28997901467cf",
|
|
88
|
+
"lastUpdated": "2026-10-13T00:36:30.227Z"
|
|
89
89
|
},
|
|
90
90
|
"formatted-logging.test.mjs": {
|
|
91
91
|
"url": "https://console-test.testdriver.ai/runs/69c8738614b73310c7839412/69c873a714b73310c7839450",
|
|
92
92
|
"lastUpdated": "2026-03-29T00:36:10.628Z"
|
|
93
93
|
},
|
|
94
94
|
"hover-text.test.mjs": {
|
|
95
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
96
|
-
"lastUpdated": "2026-10-
|
|
95
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cf399d28997901467d0",
|
|
96
|
+
"lastUpdated": "2026-10-13T00:36:32.092Z"
|
|
97
97
|
},
|
|
98
98
|
"no-provision.test.mjs": {
|
|
99
99
|
"url": "https://console.testdriver.ai/runs/69a62b3aaa712ecd3dea730a/69a62b7706a177a05bccd1cf",
|
|
@@ -138,12 +138,12 @@
|
|
|
138
138
|
"lastUpdated": "2026-02-13T19:55:05.953Z"
|
|
139
139
|
},
|
|
140
140
|
"findall-coffee-icons.test.mjs": {
|
|
141
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
142
|
-
"lastUpdated": "2026-10-
|
|
141
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7ce75a09b43706d8d5ab",
|
|
142
|
+
"lastUpdated": "2026-10-13T00:35:51.544Z"
|
|
143
143
|
},
|
|
144
144
|
"parse.test.mjs": {
|
|
145
|
-
"url": "https://console-test.testdriver.ai/runs/
|
|
146
|
-
"lastUpdated": "2026-10-
|
|
145
|
+
"url": "https://console-test.testdriver.ai/runs/6acd7cd599d28997901467c6/6acd7cf499d28997901467d1",
|
|
146
|
+
"lastUpdated": "2026-10-13T00:36:33.621Z"
|
|
147
147
|
},
|
|
148
148
|
"flake-diffthreshold-001.test.mjs": {
|
|
149
149
|
"url": "https://console.testdriver.ai/runs/69a62b3aaa712ecd3dea730a/69a62bcafc0ac3cc632a91aa",
|
|
@@ -0,0 +1,321 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Run on Your Own Machine"
|
|
3
|
+
sidebarTitle: "Your Own Machine"
|
|
4
|
+
description: "Run the TestDriver runner on a laptop, desktop, or VM you control and drive it from your tests."
|
|
5
|
+
icon: "display"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
TestDriver normally provisions a cloud sandbox for you. You can instead run the **runner** on a machine you own: a developer laptop, an office Mac, a lab Windows box, or a VM in your own data center. Your tests then drive that machine instead of a cloud sandbox.
|
|
9
|
+
|
|
10
|
+
This is useful when the machine under test cannot be replicated in the cloud: specific hardware, a licensed desktop application, a device plugged into USB, or an app that only reaches an internal network.
|
|
11
|
+
|
|
12
|
+
```mermaid
|
|
13
|
+
graph LR
|
|
14
|
+
A[Your Test] --> B[TestDriver API]
|
|
15
|
+
B --> C[Ably Channel]
|
|
16
|
+
C --> D[Runner on your machine]
|
|
17
|
+
D --> C
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
<Note>
|
|
21
|
+
The runner only makes **outbound** connections, to the TestDriver API and to Ably. You do not need to open an inbound port or expose the machine to the internet.
|
|
22
|
+
</Note>
|
|
23
|
+
|
|
24
|
+
## Choose your path
|
|
25
|
+
|
|
26
|
+
| Your machine | Use |
|
|
27
|
+
|---|---|
|
|
28
|
+
| Windows EC2 instance in your AWS account | [AWS Setup Guide](/aws-setup). Instances are spawned and provisioned automatically. |
|
|
29
|
+
| Any other machine (macOS, Linux, Windows, on-prem VM) | This guide. You start the runner yourself. |
|
|
30
|
+
|
|
31
|
+
<Tip>
|
|
32
|
+
If you want unlimited execution, assisted setup, and custom VM images, see [Self-Hosted (Enterprise)](/self-hosted).
|
|
33
|
+
</Tip>
|
|
34
|
+
|
|
35
|
+
## Prerequisites
|
|
36
|
+
|
|
37
|
+
Every machine that runs the runner needs:
|
|
38
|
+
|
|
39
|
+
- **Node.js 18 or newer**
|
|
40
|
+
- **Python 3** with `pyautogui` and `Pillow`
|
|
41
|
+
- **A real, unlocked desktop session.** The runner drives the mouse and keyboard, so the screen must be awake and not locked. Disable screen savers, sleep, and lock timers.
|
|
42
|
+
- **Google Chrome** on the `PATH`, if your tests launch a browser
|
|
43
|
+
- Outbound HTTPS and WebSocket access to `api.testdriver.ai` and `*.ably.io`
|
|
44
|
+
|
|
45
|
+
Install the Python dependencies:
|
|
46
|
+
|
|
47
|
+
<Tabs>
|
|
48
|
+
<Tab title="macOS">
|
|
49
|
+
```bash
|
|
50
|
+
brew install python
|
|
51
|
+
pip3 install pyautogui Pillow
|
|
52
|
+
```
|
|
53
|
+
</Tab>
|
|
54
|
+
<Tab title="Linux">
|
|
55
|
+
```bash
|
|
56
|
+
sudo apt-get install -y python3-pip python3-xlib
|
|
57
|
+
pip3 install pyautogui Pillow
|
|
58
|
+
```
|
|
59
|
+
</Tab>
|
|
60
|
+
<Tab title="Windows">
|
|
61
|
+
```powershell
|
|
62
|
+
pip install pyautogui Pillow
|
|
63
|
+
```
|
|
64
|
+
</Tab>
|
|
65
|
+
</Tabs>
|
|
66
|
+
|
|
67
|
+
### Platform notes
|
|
68
|
+
|
|
69
|
+
<AccordionGroup>
|
|
70
|
+
<Accordion title="macOS: grant Accessibility and Screen Recording permissions">
|
|
71
|
+
macOS blocks synthetic input and screen capture until you allow it explicitly. Without this the runner connects but every click and screenshot silently fails.
|
|
72
|
+
|
|
73
|
+
Open **System Settings → Privacy & Security** and add your terminal application (Terminal, iTerm, or whichever app starts the runner) to **both**:
|
|
74
|
+
|
|
75
|
+
- **Accessibility** (lets the runner move the mouse and type)
|
|
76
|
+
- **Screen Recording** (lets the runner take screenshots)
|
|
77
|
+
|
|
78
|
+
Quit and reopen the terminal after granting permissions, otherwise the old process keeps the old denial.
|
|
79
|
+
|
|
80
|
+
On Retina displays, screenshots are captured at physical pixels and downscaled to logical points so that coordinates returned by the AI line up with where clicks actually land. No configuration is needed.
|
|
81
|
+
</Accordion>
|
|
82
|
+
|
|
83
|
+
<Accordion title="Linux: provide a display">
|
|
84
|
+
The runner needs an X11 display. On a desktop machine, set `DISPLAY` to your existing session. On a headless server, start a virtual framebuffer first:
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
sudo apt-get install -y xvfb xfce4 xfce4-terminal dbus-x11 wmctrl
|
|
88
|
+
Xvfb :0 -screen 0 1366x768x24 &
|
|
89
|
+
export DISPLAY=:0
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
To watch the session, add a VNC server and noVNC:
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
sudo apt-get install -y x11vnc novnc websockify
|
|
96
|
+
x11vnc -display :0 -forever -shared &
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Disable screen blanking so the desktop stays visible:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
xset s off && xset s noblank && xset -dpms
|
|
103
|
+
```
|
|
104
|
+
</Accordion>
|
|
105
|
+
|
|
106
|
+
<Accordion title="Windows: keep the console session active">
|
|
107
|
+
Run the runner in a console or RDP session that stays signed in. If the session locks or disconnects, the desktop is torn down and automation stops. Configure the machine to auto-logon and disable the lock screen timeout.
|
|
108
|
+
</Accordion>
|
|
109
|
+
</AccordionGroup>
|
|
110
|
+
|
|
111
|
+
## Step 1: Install the runner
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
npm install -g @testdriverai/runner
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
This installs two commands. You use `testdriver-sandbox-agent`, which connects your machine to a TestDriver session.
|
|
118
|
+
|
|
119
|
+
Verify the install:
|
|
120
|
+
|
|
121
|
+
```bash
|
|
122
|
+
testdriver-sandbox-agent --help || echo "installed"
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## Step 2: Register the machine
|
|
126
|
+
|
|
127
|
+
Ask the API for a session bound to your machine's IP address:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
curl -X POST https://api.testdriver.ai/api/v7/sandbox/authenticate \
|
|
131
|
+
-H "Content-Type: application/json" \
|
|
132
|
+
-d '{
|
|
133
|
+
"apiKey": "your_api_key",
|
|
134
|
+
"ip": "192.168.1.50",
|
|
135
|
+
"os": "linux",
|
|
136
|
+
"keepAlive": 900000
|
|
137
|
+
}'
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
The response contains the sandbox ID and the Ably credentials the runner needs:
|
|
141
|
+
|
|
142
|
+
```json
|
|
143
|
+
{
|
|
144
|
+
"success": true,
|
|
145
|
+
"sandboxId": "sb-69cce96e-1e0",
|
|
146
|
+
"agent": {
|
|
147
|
+
"token": {
|
|
148
|
+
"token": "aFC5YQ.Ki9G9...",
|
|
149
|
+
"keyName": "aFC5YQ.ZIdTww",
|
|
150
|
+
"issued": 1791908438832,
|
|
151
|
+
"expires": 1791930038832,
|
|
152
|
+
"capability": "{\"testdriver:...:sb-69cce96e-1e0\":[\"history\",\"presence\",\"publish\",\"subscribe\"]}",
|
|
153
|
+
"clientId": "agent-sb-69cce96e-1e0"
|
|
154
|
+
},
|
|
155
|
+
"channel": "testdriver:...:sb-69cce96e-1e0"
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
<Note>
|
|
161
|
+
Pass `os: "linux"` on macOS. The value only describes the session to the API; the runner detects the real platform itself.
|
|
162
|
+
</Note>
|
|
163
|
+
|
|
164
|
+
### Write the config file
|
|
165
|
+
|
|
166
|
+
The runner reads its credentials from a JSON file on disk. Copy `sandboxId` and the whole `agent` block into it:
|
|
167
|
+
|
|
168
|
+
| Platform | Config file path |
|
|
169
|
+
|---|---|
|
|
170
|
+
| macOS / Linux | `/tmp/testdriver-agent.json` |
|
|
171
|
+
| Windows | `C:\Windows\Temp\testdriver-agent.json` |
|
|
172
|
+
|
|
173
|
+
```json /tmp/testdriver-agent.json
|
|
174
|
+
{
|
|
175
|
+
"sandboxId": "sb-69cce96e-1e0",
|
|
176
|
+
"apiRoot": "https://api.testdriver.ai",
|
|
177
|
+
"apiKey": "your_api_key",
|
|
178
|
+
"ably": {
|
|
179
|
+
"token": {
|
|
180
|
+
"token": "aFC5YQ.Ki9G9...",
|
|
181
|
+
"keyName": "aFC5YQ.ZIdTww",
|
|
182
|
+
"issued": 1791908438832,
|
|
183
|
+
"expires": 1791930038832,
|
|
184
|
+
"capability": "{\"testdriver:...:sb-69cce96e-1e0\":[\"history\",\"presence\",\"publish\",\"subscribe\"]}",
|
|
185
|
+
"clientId": "agent-sb-69cce96e-1e0"
|
|
186
|
+
},
|
|
187
|
+
"channel": "testdriver:...:sb-69cce96e-1e0"
|
|
188
|
+
}
|
|
189
|
+
}
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
| Field | Source | Purpose |
|
|
193
|
+
|---|---|---|
|
|
194
|
+
| `sandboxId` | `sandboxId` from the response | Identifies this session. Your test attaches to it. |
|
|
195
|
+
| `apiRoot` | The API you called | Where the runner uploads screenshots |
|
|
196
|
+
| `apiKey` | Your team API key | Authenticates those uploads |
|
|
197
|
+
| `ably.token` | `agent.token` from the response, copied verbatim | Realtime credentials |
|
|
198
|
+
| `ably.channel` | `agent.channel` from the response | Channel the runner listens on |
|
|
199
|
+
|
|
200
|
+
To put the file somewhere else, set `CONFIG_PATH` to its full path before starting the runner.
|
|
201
|
+
|
|
202
|
+
<Warning>
|
|
203
|
+
This file holds a session token and your API key. Keep it readable only by the user running the runner, do not commit it, and delete it when the machine is no longer in use.
|
|
204
|
+
</Warning>
|
|
205
|
+
|
|
206
|
+
## Step 3: Start the runner
|
|
207
|
+
|
|
208
|
+
```bash
|
|
209
|
+
testdriver-sandbox-agent
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
You should see the runner connect and announce itself:
|
|
213
|
+
|
|
214
|
+
```
|
|
215
|
+
[agent] TestDriver Sandbox Agent starting...
|
|
216
|
+
[agent] Sandbox ID: sb-69cce96e-1e0
|
|
217
|
+
[agent] Transport: Ably
|
|
218
|
+
[agent] Automation module initialized
|
|
219
|
+
[realtime] Publishing control: type=runner.ready
|
|
220
|
+
[agent] Agent ready — listening for commands via Ably
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
Leave this process running. On Linux, set `DISPLAY` first:
|
|
224
|
+
|
|
225
|
+
```bash
|
|
226
|
+
DISPLAY=:0 testdriver-sandbox-agent
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
## Step 4: Point your test at the machine
|
|
230
|
+
|
|
231
|
+
Pass the machine's `ip` and the sandbox ID from step 2:
|
|
232
|
+
|
|
233
|
+
```javascript tests/my-machine.test.mjs
|
|
234
|
+
import { describe, expect, it } from "vitest";
|
|
235
|
+
import { TestDriver } from "testdriverai/vitest/hooks";
|
|
236
|
+
|
|
237
|
+
describe("My machine", () => {
|
|
238
|
+
it("runs against my own hardware", async (context) => {
|
|
239
|
+
const testdriver = TestDriver(context, { newSandbox: false });
|
|
240
|
+
|
|
241
|
+
await testdriver.connect({
|
|
242
|
+
ip: process.env.TD_MACHINE_IP,
|
|
243
|
+
sandboxId: process.env.TD_SANDBOX_ID,
|
|
244
|
+
});
|
|
245
|
+
|
|
246
|
+
const button = await testdriver.find("Sign In button");
|
|
247
|
+
await button.click();
|
|
248
|
+
|
|
249
|
+
const result = await testdriver.assert("the dashboard is visible");
|
|
250
|
+
expect(result).toBeTruthy();
|
|
251
|
+
});
|
|
252
|
+
});
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Run it:
|
|
256
|
+
|
|
257
|
+
```bash
|
|
258
|
+
TD_MACHINE_IP=192.168.1.50 TD_SANDBOX_ID=sb-69cce96e-1e0 vitest run
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
<Note>
|
|
262
|
+
`newSandbox: false` stops the SDK from provisioning a cloud sandbox. Without it, your test runs in the cloud and ignores your machine.
|
|
263
|
+
</Note>
|
|
264
|
+
|
|
265
|
+
<Note>
|
|
266
|
+
Each config file covers one session. The Ably token in it expires (see its `expires` field), so repeat steps 2 and 3 to get fresh credentials before the next run.
|
|
267
|
+
</Note>
|
|
268
|
+
|
|
269
|
+
## Logs
|
|
270
|
+
|
|
271
|
+
| Platform | Runner log |
|
|
272
|
+
|---|---|
|
|
273
|
+
| macOS / Linux | `/tmp/testdriver-agent.log` |
|
|
274
|
+
| Windows | `C:\Windows\Temp\testdriver-agent.log` |
|
|
275
|
+
|
|
276
|
+
The runner also writes a status file next to the log (`testdriver-agent-status.json`) with the sandbox ID, process ID, detected IP, and start time.
|
|
277
|
+
|
|
278
|
+
## Security
|
|
279
|
+
|
|
280
|
+
- **Use a dedicated machine or user account.** The runner can click anything on the desktop and run shell commands sent by your tests. Do not point it at a machine holding personal credentials or production access.
|
|
281
|
+
- **Treat the config file as a secret.** It contains a session token and your API key.
|
|
282
|
+
- **Do not expose VNC to the internet.** If you add VNC for debugging, bind it to localhost or your VPN and tunnel in over SSH.
|
|
283
|
+
- **Rotate your API key** if a machine is decommissioned or shared.
|
|
284
|
+
|
|
285
|
+
## Troubleshooting
|
|
286
|
+
|
|
287
|
+
<AccordionGroup>
|
|
288
|
+
<Accordion title="Sandbox has expired and is no longer available">
|
|
289
|
+
The session ended, which happens when the runner stops or your test disconnects. Repeat step 2 to get a fresh sandbox ID and token, update the config file, restart the runner, and use the new ID in your test.
|
|
290
|
+
</Accordion>
|
|
291
|
+
|
|
292
|
+
<Accordion title="Runner starts but clicks and screenshots do nothing (macOS)">
|
|
293
|
+
Accessibility or Screen Recording permission is missing. Add your terminal application to both lists under **System Settings → Privacy & Security**, then fully quit and reopen the terminal.
|
|
294
|
+
</Accordion>
|
|
295
|
+
|
|
296
|
+
<Accordion title="Waiting for config file">
|
|
297
|
+
The runner cannot find `/tmp/testdriver-agent.json` (or `C:\Windows\Temp\testdriver-agent.json`). Write the config file on the same machine and readable by the same user that starts the runner. To use a different location, set `CONFIG_PATH`.
|
|
298
|
+
</Accordion>
|
|
299
|
+
|
|
300
|
+
<Accordion title="Test provisions a cloud sandbox instead of using my machine">
|
|
301
|
+
Pass `newSandbox: false` to `TestDriver()` and make sure both `ip` and `sandboxId` reach `connect()`. If either is undefined, the SDK falls back to cloud provisioning.
|
|
302
|
+
</Accordion>
|
|
303
|
+
|
|
304
|
+
<Accordion title="Invalid API key">
|
|
305
|
+
Confirm `TD_API_KEY` matches the environment in `TD_API_ROOT`. Keys are scoped per environment. Get your key at [console.testdriver.ai/team](https://console.testdriver.ai/team).
|
|
306
|
+
</Accordion>
|
|
307
|
+
|
|
308
|
+
<Accordion title="Screenshots are blank or show a frozen desktop (Linux)">
|
|
309
|
+
`DISPLAY` points at a display with no desktop running, or the screen blanked. Start a desktop on that display and disable blanking with `xset s off && xset s noblank && xset -dpms`.
|
|
310
|
+
</Accordion>
|
|
311
|
+
</AccordionGroup>
|
|
312
|
+
|
|
313
|
+
## Environment variables
|
|
314
|
+
|
|
315
|
+
| Variable | Default | Purpose |
|
|
316
|
+
|---|---|---|
|
|
317
|
+
| `TD_API_KEY` | Required | Your team API key |
|
|
318
|
+
| `TD_API_ROOT` | `https://api.testdriver.ai` | API server to register with |
|
|
319
|
+
| `CONFIG_PATH` | `/tmp/testdriver-agent.json` | Override the runner config file location |
|
|
320
|
+
| `DISPLAY` | `:0` | X11 display (Linux) |
|
|
321
|
+
| `SENTRY_DSN` | Unset | Error reporting endpoint |
|
|
@@ -123,7 +123,7 @@ const testdriver = TestDriver(context, {
|
|
|
123
123
|
|
|
124
124
|
### IP Target
|
|
125
125
|
|
|
126
|
-
If self-hosting TestDriver, use `ip` to specify the device IP. See [
|
|
126
|
+
If self-hosting TestDriver, use `ip` to specify the device IP. See [Run on Your Own Machine](/bring-your-own-machine) for details.
|
|
127
127
|
|
|
128
128
|
```javascript
|
|
129
129
|
const testdriver = TestDriver(context, {
|
package/docs/docs.json
CHANGED
package/docs/examples/assert.mdx
CHANGED
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* assert.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7ce699d28997901467cb/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* element-not-found.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cf199d28997901467cf/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -18,7 +18,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
18
18
|
|
|
19
19
|
{/* findall-coffee-icons.test.mjs output */}
|
|
20
20
|
<iframe
|
|
21
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
21
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7ce75a09b43706d8d5ab/replay"
|
|
22
22
|
width="100%"
|
|
23
23
|
height="600"
|
|
24
24
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* hover-image.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cdf5a09b43706d8d5a8/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -18,7 +18,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
18
18
|
|
|
19
19
|
{/* hover-text-with-description.test.mjs output */}
|
|
20
20
|
<iframe
|
|
21
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
21
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cda1adbeb4ea97f70e4/replay"
|
|
22
22
|
width="100%"
|
|
23
23
|
height="600"
|
|
24
24
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* hover-text.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cf399d28997901467d0/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* installer.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7ce15a09b43706d8d5aa/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* launch-vscode-linux.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cd81adbeb4ea97f70e3/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
package/docs/examples/parse.mdx
CHANGED
|
@@ -18,7 +18,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
18
18
|
|
|
19
19
|
{/* parse.test.mjs output */}
|
|
20
20
|
<iframe
|
|
21
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
21
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cf499d28997901467d1/replay"
|
|
22
22
|
width="100%"
|
|
23
23
|
height="600"
|
|
24
24
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* press-keys.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://console-test.testdriver.ai/replay/
|
|
15
|
+
src="https://console-test.testdriver.ai/replay/6acd7edd1adbeb4ea97f7120?share=TSZUSVjpnSPLXWr9U5k2A&embed=true"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* scroll-keyboard.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7ce91adbeb4ea97f70e7/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
package/docs/examples/scroll.mdx
CHANGED
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* scroll.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7cef99d28997901467ce/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
package/docs/examples/type.mdx
CHANGED
|
@@ -12,7 +12,7 @@ Watch this test execute in a real sandbox environment:
|
|
|
12
12
|
|
|
13
13
|
{/* type.test.mjs output */}
|
|
14
14
|
<iframe
|
|
15
|
-
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/
|
|
15
|
+
src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/6acd7ce299d28997901467c9/replay"
|
|
16
16
|
width="100%"
|
|
17
17
|
height="390"
|
|
18
18
|
style={{ border: "1px solid #333", borderRadius: "8px" }}
|
package/docs/self-hosted.mdx
CHANGED
|
@@ -50,13 +50,22 @@ Run test sandboxes on your AWS infrastructure while using TestDriver's hosted da
|
|
|
50
50
|
**Bring your own Gemini API key.** TestDriver uses Google Gemini for vision and reasoning. When supplying your own Gemini key, it must be a [Gemini Enterprise Agent Platform express mode API key](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/start/api-keys?usertype=expressmode) — standard Google AI Studio (Generative Language API) keys are not supported for self-hosted deployments.
|
|
51
51
|
</Note>
|
|
52
52
|
|
|
53
|
-
<
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
53
|
+
<CardGroup cols={2}>
|
|
54
|
+
<Card
|
|
55
|
+
title="AWS Setup Guide"
|
|
56
|
+
icon="aws"
|
|
57
|
+
href="/aws-setup"
|
|
58
|
+
>
|
|
59
|
+
Step-by-step instructions for deploying TestDriver on your AWS infrastructure using CloudFormation.
|
|
60
|
+
</Card>
|
|
61
|
+
<Card
|
|
62
|
+
title="Run on Your Own Machine"
|
|
63
|
+
icon="display"
|
|
64
|
+
href="/bring-your-own-machine"
|
|
65
|
+
>
|
|
66
|
+
Run the runner on a laptop, lab machine, or on-prem VM and drive it from your tests.
|
|
67
|
+
</Card>
|
|
68
|
+
</CardGroup>
|
|
60
69
|
|
|
61
70
|
### Air-Gapped Deployment
|
|
62
71
|
|