testdriverai 7.2.19 → 7.2.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +48 -282
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,314 +1,80 @@
|
|
|
1
|
-
<
|
|
1
|
+
<div align="center">
|
|
2
|
+
<a href="https://testdriver.ai">
|
|
3
|
+
<img width="250" alt="TestDriver.ai Icon" src="https://github.com/user-attachments/assets/c591b39d-6e17-454b-b36d-370ae8618840" />
|
|
4
|
+
</a>
|
|
5
|
+
</div>
|
|
2
6
|
|
|
3
|
-
|
|
7
|
+
<h1 align="center">Computer-Use SDK for E2E QA Testing</h1>
|
|
4
8
|
|
|
5
|
-
|
|
9
|
+
<div align="center">The TestDriver SDK is a JS plugin for vitest that makes it easy to spawn ephemeral devices and use vision-based LLMs to construct detemanistic and reliable tests.
|
|
6
10
|
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
# Install via NPM
|
|
10
|
-
|
|
11
|
-
[Follow the instructions on our docs for more.](https://docs.testdriver.ai/overview/quickstart).
|
|
12
|
-
|
|
13
|
-
## v7 SDK - Progressive Disclosure
|
|
14
|
-
|
|
15
|
-
TestDriver v7 introduces **three levels of API** to match your experience level:
|
|
16
|
-
|
|
17
|
-
### 🟢 Beginner: Presets (Zero Config)
|
|
18
|
-
|
|
19
|
-
```javascript
|
|
20
|
-
import { test } from 'vitest';
|
|
21
|
-
import { chromePreset } from 'testdriverai/presets';
|
|
22
|
-
|
|
23
|
-
test('login test', async (context) => {
|
|
24
|
-
const { client } = await chromePreset(context, {
|
|
25
|
-
url: 'https://myapp.com'
|
|
26
|
-
});
|
|
27
|
-
|
|
28
|
-
await client.find('Login button').click();
|
|
29
|
-
});
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
**Built-in presets:** Chrome, VS Code, Electron, and create your own!
|
|
33
|
-
|
|
34
|
-
### 🟡 Intermediate: Hooks (Flexible)
|
|
35
|
-
|
|
36
|
-
```javascript
|
|
37
|
-
import { test } from 'vitest';
|
|
38
|
-
import { useTestDriver, useDashcam } from 'testdriverai/vitest/hooks';
|
|
39
|
-
|
|
40
|
-
test('my test', async (context) => {
|
|
41
|
-
const client = useTestDriver(context, { os: 'linux' });
|
|
42
|
-
const dashcam = useDashcam(context, client, {
|
|
43
|
-
autoStart: true,
|
|
44
|
-
autoStop: true
|
|
45
|
-
});
|
|
11
|
+
<br />
|
|
12
|
+
<br />
|
|
46
13
|
|
|
47
|
-
|
|
48
|
-
});
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
**Automatic lifecycle management** - no more forgetting cleanup!
|
|
52
|
-
|
|
53
|
-
### 🔴 Advanced: Core Classes (Full Control)
|
|
54
|
-
|
|
55
|
-
```javascript
|
|
56
|
-
import { test } from 'vitest';
|
|
57
|
-
import { TestDriver, Dashcam } from 'testdriverai/core';
|
|
58
|
-
|
|
59
|
-
test('my test', async () => {
|
|
60
|
-
const client = new TestDriver(apiKey, { os: 'linux' });
|
|
61
|
-
const dashcam = new Dashcam(client);
|
|
62
|
-
|
|
63
|
-
await client.auth();
|
|
64
|
-
await client.connect();
|
|
65
|
-
await dashcam.start();
|
|
66
|
-
|
|
67
|
-
await client.find('button').click();
|
|
68
|
-
|
|
69
|
-
await dashcam.stop();
|
|
70
|
-
await client.disconnect();
|
|
71
|
-
});
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
**Full manual control** for advanced scenarios.
|
|
75
|
-
|
|
76
|
-
📖 **Learn more:** [MIGRATION.md](./docs/MIGRATION.md) | [PRESETS.md](./docs/PRESETS.md) | [HOOKS.md](./docs/HOOKS.md)
|
|
77
|
-
|
|
78
|
-
# About
|
|
79
|
-
|
|
80
|
-
TestDriver isn't like any test framework you've used before. TestDriver is an OS Agent for QA. TestDriver uses AI vision along with mouse and keyboard emulation to control the entire desktop. It's more like a QA employee than a test framework. This kind of black-box testing has some major advantages:
|
|
81
|
-
|
|
82
|
-
- **Easier set up:** No need to add test IDs or craft complex selectors
|
|
83
|
-
- **Less Maintenance:** Tests don't break when code changes
|
|
84
|
-
- **More Power:** TestDriver can test any application and control any OS setting
|
|
85
|
-
|
|
86
|
-
### Demo (Playing Balatro Desktop)
|
|
87
|
-
|
|
88
|
-
https://github.com/user-attachments/assets/7cb9ee5a-0d05-4ff0-a4fa-084bcee12e98
|
|
89
|
-
|
|
90
|
-
# Examples
|
|
91
|
-
|
|
92
|
-
- Test any user flow on any website in any browser
|
|
93
|
-
- Clone, build, and test any desktop app
|
|
94
|
-
- Render multiple browser windows and popups like 3rd party auth
|
|
95
|
-
- Test `<canvas>`, `<iframe>`, and `<video>` tags with ease
|
|
96
|
-
- Use file selectors to upload files to the browser
|
|
97
|
-
- Test chrome extensions
|
|
98
|
-
- Test integrations between applications
|
|
99
|
-
- Integrates into CI/CD via GitHub Actions ($)
|
|
100
|
-
|
|
101
|
-
Check out [the docs](https://docs.testdriver.ai/).
|
|
102
|
-
|
|
103
|
-
# Workflow
|
|
104
|
-
|
|
105
|
-
1. Tell TestDriver what to do in natural language on your local machine using `npm i testdriverai -g`
|
|
106
|
-
2. TestDriver looks at the screen and uses mouse and keyboard emulation to accomplish the goal
|
|
107
|
-
3. Run TestDriver tests on our test infrastructure
|
|
108
|
-
|
|
109
|
-
# Quickstart
|
|
110
|
-
|
|
111
|
-
## Initialize TestDriver
|
|
112
|
-
|
|
113
|
-
In your project directory:
|
|
114
|
-
|
|
115
|
-
```sh
|
|
116
|
-
npx testdriverai@latest init
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
## Teach TestDriver a test
|
|
120
|
-
|
|
121
|
-
Let's show TestDriver what we want to test. Run the following command:
|
|
122
|
-
|
|
123
|
-
```sh
|
|
124
|
-
npx testdriverai@latest .testdriver/test.yaml
|
|
125
|
-
```
|
|
14
|
+
[🚀 **Quick Start**](#-quick-start) • [📖 **Documentation**](https://docs.testdriver.ai) • [💻 **Examples**](https://github.com/testdriverai/testdriverai/tree/main/test/testdriver) • [📖 **Pricing**](https://docs.testdriver.ai) • [💬 **Discord**](https://discord.com/invite/cWDFW8DzPm) • [🌐 **Website**](https://testdriver.ai)
|
|
126
15
|
|
|
127
|
-
|
|
16
|
+
</div>
|
|
128
17
|
|
|
129
|
-
|
|
18
|
+
<img width="1490" height="854" alt="image" src="https://github.com/user-attachments/assets/36e426cc-e740-426f-b9f6-6e8565e66ad6" />
|
|
130
19
|
|
|
131
|
-
|
|
20
|
+
---
|
|
132
21
|
|
|
133
|
-
##
|
|
22
|
+
## Why TestDriver?
|
|
134
23
|
|
|
135
|
-
|
|
24
|
+
Don't ship bugs because flows are too hard to test. TestDriver helps engineering teams easily test, debug, and monitor E2E flows that are hard or impossible to cover with other tools like:
|
|
136
25
|
|
|
137
|
-
|
|
26
|
+
*Third-Party Web Apps* • *Desktop Apps* • *VS Code Extensions* • *Chrome Extensions* • *AI Chatbots* • *OAuth Flows* • *PDF Content* • *Spelling & Grammar* • *File System & Uploads* • *OS Accessibility* • *Visual Content* • *`<iframe>`* • *`<canvas>`* • *`<video>`*
|
|
138
27
|
|
|
139
|
-
|
|
140
|
-
> Click on sign up
|
|
141
|
-
TestDriver Generates a Test
|
|
142
|
-
TestDriver will look at your screen and generate a test script. TestDriver can see the screen, control the mouse, keyboard, and more!
|
|
143
|
-
TestDriver can only see your primary display!
|
|
144
|
-
To navigate to testdriver.ai, we need to focus on the
|
|
145
|
-
Google Chrome application, click on the search bar, type
|
|
146
|
-
the URL, and then press Enter.
|
|
28
|
+
## Example
|
|
147
29
|
|
|
148
|
-
|
|
30
|
+
```js
|
|
31
|
+
// Click on the new text document
|
|
32
|
+
await testdriver.find("New text document").mouseDown();
|
|
149
33
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
3. Type "testdriver.ai".
|
|
153
|
-
4. Press Enter.
|
|
34
|
+
// Drag the "New Text Document" icon to the "Recycle Bin"
|
|
35
|
+
await testdriver.find("Recycle Bin icon").mouseUp();
|
|
154
36
|
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
commands:
|
|
159
|
-
- command: focus-application
|
|
160
|
-
name: Google Chrome
|
|
161
|
-
- command: hover-text
|
|
162
|
-
text: Search Google or type a URL
|
|
163
|
-
description: main google search
|
|
164
|
-
action: click
|
|
165
|
-
|
|
166
|
-
After this, we will type the URL and press Enter.
|
|
167
|
-
```
|
|
168
|
-
|
|
169
|
-
## TestDriver executes the test script
|
|
170
|
-
|
|
171
|
-
TestDriver will execute the commands found in the yml codeblocks of the response.
|
|
172
|
-
|
|
173
|
-
See the yml TestDriver generated? That's our own schema. You can learn more about it in the [reference](https://docs.testdriver.ai/getting-started/editing).
|
|
174
|
-
|
|
175
|
-
> Take your hands off the mouse and keyboard while TestDriver executes! TestDriver is not a fan of backseat drivers.
|
|
176
|
-
|
|
177
|
-
## Keep going!
|
|
178
|
-
|
|
179
|
-
Feel free to ask TestDriver to perform some more tasks. Every time you prompt TestDriver it will look at your screen and generate more test step to complete your goal.
|
|
180
|
-
|
|
181
|
-
```sh
|
|
182
|
-
> navigate to airbnb.com
|
|
183
|
-
> search for destinations in austin tx
|
|
184
|
-
> click check in
|
|
185
|
-
> select august 8
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
If something didn't work, you can use `/undo` to remove all of the test steps added since the last prompt.
|
|
189
|
-
|
|
190
|
-
## Test the test locally
|
|
191
|
-
|
|
192
|
-
Now it's time to make sure the test plan works before we deploy it. Use testdriver run to run the test file you just created with /save .
|
|
193
|
-
|
|
194
|
-
```sh
|
|
195
|
-
npx testdriverai@latest run testdriver/test.yaml
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
Make sure to reset the test state!
|
|
199
|
-
|
|
200
|
-
## Deploy
|
|
201
|
-
|
|
202
|
-
Now it's time to deploy your test using our GitHub action! `testdriver init` already did the work for you and will start triggering tests once you commit the new files to your repository.
|
|
203
|
-
|
|
204
|
-
```sh
|
|
205
|
-
git add .
|
|
206
|
-
git commit -am "Add TestDriver tests"
|
|
207
|
-
gh pr create --web
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
Your test will run on every commit and the results will be posted as a Dashcam.io video within your GitHub summary! Learn more about deploying on CI [here](https://docs.testdriver.ai/action/setup).
|
|
211
|
-
|
|
212
|
-
## Using as a Module
|
|
213
|
-
|
|
214
|
-
TestDriver can also be used programmatically as a Node.js module. This is useful when you want to integrate TestDriver into your own applications or customize the test file paths.
|
|
215
|
-
|
|
216
|
-
### Custom Test File Paths
|
|
217
|
-
|
|
218
|
-
By default, TestDriver looks for test files at `testdriver/testdriver.yaml` relative to the current working directory. You can customize this:
|
|
219
|
-
|
|
220
|
-
```javascript
|
|
221
|
-
const TestDriverAgent = require("testdriverai");
|
|
222
|
-
|
|
223
|
-
// Option 1: Set default via environment variable
|
|
224
|
-
const agent1 = new TestDriverAgent({
|
|
225
|
-
TD_DEFAULT_TEST_FILE: "my-tests/integration.yaml",
|
|
226
|
-
});
|
|
227
|
-
|
|
228
|
-
// Option 2: Explicitly specify test file
|
|
229
|
-
const agent2 = new TestDriverAgent(
|
|
230
|
-
{},
|
|
231
|
-
{
|
|
232
|
-
args: ["path/to/specific/test.yaml"],
|
|
233
|
-
},
|
|
234
|
-
);
|
|
235
|
-
|
|
236
|
-
// Option 3: Custom working directory + relative path
|
|
237
|
-
const agent3 = new TestDriverAgent(
|
|
238
|
-
{ TD_DEFAULT_TEST_FILE: "tests/smoke.yaml" },
|
|
239
|
-
{ options: { workingDir: "/path/to/your/project" } },
|
|
37
|
+
// Assert "New Text Document" icon is not on the Desktop
|
|
38
|
+
const result = await testdriver.assert(
|
|
39
|
+
'the "New Text Document" icon is not visible on the Desktop'
|
|
240
40
|
);
|
|
241
|
-
|
|
242
|
-
// Run the test
|
|
243
|
-
await agent1.run();
|
|
41
|
+
expect(result).toBeTruthy();
|
|
244
42
|
```
|
|
245
43
|
|
|
246
|
-
|
|
44
|
+
[See Full Example](https://github.com/testdriverai/testdriverai/blob/main/test/testdriver/drag-and-drop.test.mjs) • [Browse All Examples](https://github.com/testdriverai/testdriverai/tree/main/test/testdriver)
|
|
247
45
|
|
|
248
|
-
|
|
46
|
+
---
|
|
249
47
|
|
|
250
|
-
|
|
251
|
-
export TD_DEFAULT_TEST_FILE="custom/path/test.yaml"
|
|
252
|
-
node your-script.js
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
## MCP Server for AI Agents
|
|
48
|
+
## 🚀 Quick Start
|
|
256
49
|
|
|
257
|
-
|
|
50
|
+
### Step 1: Create a TestDriver Account
|
|
258
51
|
|
|
259
|
-
|
|
52
|
+
<a href="https://app.testdriver.ai/team"><img src="https://img.shields.io/badge/Sign_Up-Free_Account-blue?style=for-the-badge" alt="Sign Up"/></a>
|
|
260
53
|
|
|
261
|
-
|
|
262
|
-
2. **User describes** what they want to test
|
|
263
|
-
3. **AI explores** the application using TestDriver commands
|
|
264
|
-
4. **AI generates** Vitest test code from successful interactions
|
|
54
|
+
*No credit card required!*
|
|
265
55
|
|
|
266
|
-
###
|
|
56
|
+
### Step 2: Initialize Your Project
|
|
267
57
|
|
|
268
58
|
```bash
|
|
269
|
-
|
|
270
|
-
npm install && npm run build
|
|
271
|
-
npm run deploy # Install to ~/.mcp/testdriver
|
|
272
|
-
```
|
|
273
|
-
|
|
274
|
-
### Configuration
|
|
275
|
-
|
|
276
|
-
Add to your MCP client configuration:
|
|
277
|
-
|
|
278
|
-
```json
|
|
279
|
-
{
|
|
280
|
-
"servers": {
|
|
281
|
-
"testdriverai": {
|
|
282
|
-
"type": "stdio",
|
|
283
|
-
"command": "node",
|
|
284
|
-
"args": ["/path/to/cli/mcp-server/dist/index.js"]
|
|
285
|
-
}
|
|
286
|
-
}
|
|
287
|
-
}
|
|
59
|
+
npx testdriverai@beta init
|
|
288
60
|
```
|
|
289
61
|
|
|
290
|
-
|
|
62
|
+
This will:
|
|
63
|
+
- Create a project folder
|
|
64
|
+
- Install dependencies (Vitest + TestDriver)
|
|
65
|
+
- Set up your API key
|
|
66
|
+
- Generate an example test
|
|
291
67
|
|
|
292
|
-
|
|
293
|
-
User: "Create a test that logs into the app"
|
|
68
|
+
### Step 3: Run Your First Test
|
|
294
69
|
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
AI: [Finds username field: await testdriver_find({ description: "username field" })]
|
|
298
|
-
AI: [Clicks and types: await testdriver_type({ text: "test_user" })]
|
|
299
|
-
AI: [Finds password field, enters password]
|
|
300
|
-
AI: [Clicks login button]
|
|
301
|
-
AI: [Asserts login succeeded]
|
|
302
|
-
AI: [Generates Vitest test file from these steps]
|
|
303
|
-
AI: [Saves test/login.test.mjs]
|
|
70
|
+
```bash
|
|
71
|
+
vitest run
|
|
304
72
|
```
|
|
305
73
|
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
- **Code generation** - AI translates interactions into proper Vitest code
|
|
312
|
-
|
|
313
|
-
See [mcp-server/TEST_CREATION_GUIDE.md](mcp-server/TEST_CREATION_GUIDE.md) for the complete guide.
|
|
74
|
+
Watch as TestDriver:
|
|
75
|
+
1. Spawns a cloud sandbox
|
|
76
|
+
2. Launches Chrome
|
|
77
|
+
3. Runs your test using AI vision
|
|
78
|
+
4. Returns results with video replay
|
|
314
79
|
|
|
80
|
+
<a href="https://docs.testdriver.ai/v7/quickstart"><img src="https://img.shields.io/badge/📖_Read_Full_Quickstart-4A90E2?style=for-the-badge" alt="Full Quickstart"/></a>
|