testdriverai 7.2.19 → 7.2.21

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +48 -282
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -1,314 +1,80 @@
1
- <a href="https://testdriver.ai"><img src="https://github.com/dashcamio/testdriver/assets/318295/2a0ad981-8504-46f0-ad97-60cb6c26f1e7"/></a>
1
+ <div align="center">
2
+ <a href="https://testdriver.ai">
3
+ <img width="250" alt="TestDriver.ai Icon" src="https://github.com/user-attachments/assets/c591b39d-6e17-454b-b36d-370ae8618840" />
4
+ </a>
5
+ </div>
2
6
 
3
- # TestDriver.ai
7
+ <h1 align="center">Computer-Use SDK for E2E QA Testing</h1>
4
8
 
5
- Automate and scale QA with computer-use agents.
9
+ <div align="center">The TestDriver SDK is a JS plugin for vitest that makes it easy to spawn ephemeral devices and use vision-based LLMs to construct detemanistic and reliable tests.
6
10
 
7
- [Docs](https://docs.testdriver.ai) | [Website](https://testdriver.ai) | [GitHub Action](https://github.com/marketplace/actions/testdriver-ai) | [Join our Discord](https://discord.com/invite/cWDFW8DzPm)
8
-
9
- # Install via NPM
10
-
11
- [Follow the instructions on our docs for more.](https://docs.testdriver.ai/overview/quickstart).
12
-
13
- ## v7 SDK - Progressive Disclosure
14
-
15
- TestDriver v7 introduces **three levels of API** to match your experience level:
16
-
17
- ### 🟢 Beginner: Presets (Zero Config)
18
-
19
- ```javascript
20
- import { test } from 'vitest';
21
- import { chromePreset } from 'testdriverai/presets';
22
-
23
- test('login test', async (context) => {
24
- const { client } = await chromePreset(context, {
25
- url: 'https://myapp.com'
26
- });
27
-
28
- await client.find('Login button').click();
29
- });
30
- ```
31
-
32
- **Built-in presets:** Chrome, VS Code, Electron, and create your own!
33
-
34
- ### 🟡 Intermediate: Hooks (Flexible)
35
-
36
- ```javascript
37
- import { test } from 'vitest';
38
- import { useTestDriver, useDashcam } from 'testdriverai/vitest/hooks';
39
-
40
- test('my test', async (context) => {
41
- const client = useTestDriver(context, { os: 'linux' });
42
- const dashcam = useDashcam(context, client, {
43
- autoStart: true,
44
- autoStop: true
45
- });
11
+ <br />
12
+ <br />
46
13
 
47
- await client.find('button').click();
48
- });
49
- ```
50
-
51
- **Automatic lifecycle management** - no more forgetting cleanup!
52
-
53
- ### 🔴 Advanced: Core Classes (Full Control)
54
-
55
- ```javascript
56
- import { test } from 'vitest';
57
- import { TestDriver, Dashcam } from 'testdriverai/core';
58
-
59
- test('my test', async () => {
60
- const client = new TestDriver(apiKey, { os: 'linux' });
61
- const dashcam = new Dashcam(client);
62
-
63
- await client.auth();
64
- await client.connect();
65
- await dashcam.start();
66
-
67
- await client.find('button').click();
68
-
69
- await dashcam.stop();
70
- await client.disconnect();
71
- });
72
- ```
73
-
74
- **Full manual control** for advanced scenarios.
75
-
76
- 📖 **Learn more:** [MIGRATION.md](./docs/MIGRATION.md) | [PRESETS.md](./docs/PRESETS.md) | [HOOKS.md](./docs/HOOKS.md)
77
-
78
- # About
79
-
80
- TestDriver isn't like any test framework you've used before. TestDriver is an OS Agent for QA. TestDriver uses AI vision along with mouse and keyboard emulation to control the entire desktop. It's more like a QA employee than a test framework. This kind of black-box testing has some major advantages:
81
-
82
- - **Easier set up:** No need to add test IDs or craft complex selectors
83
- - **Less Maintenance:** Tests don't break when code changes
84
- - **More Power:** TestDriver can test any application and control any OS setting
85
-
86
- ### Demo (Playing Balatro Desktop)
87
-
88
- https://github.com/user-attachments/assets/7cb9ee5a-0d05-4ff0-a4fa-084bcee12e98
89
-
90
- # Examples
91
-
92
- - Test any user flow on any website in any browser
93
- - Clone, build, and test any desktop app
94
- - Render multiple browser windows and popups like 3rd party auth
95
- - Test `<canvas>`, `<iframe>`, and `<video>` tags with ease
96
- - Use file selectors to upload files to the browser
97
- - Test chrome extensions
98
- - Test integrations between applications
99
- - Integrates into CI/CD via GitHub Actions ($)
100
-
101
- Check out [the docs](https://docs.testdriver.ai/).
102
-
103
- # Workflow
104
-
105
- 1. Tell TestDriver what to do in natural language on your local machine using `npm i testdriverai -g`
106
- 2. TestDriver looks at the screen and uses mouse and keyboard emulation to accomplish the goal
107
- 3. Run TestDriver tests on our test infrastructure
108
-
109
- # Quickstart
110
-
111
- ## Initialize TestDriver
112
-
113
- In your project directory:
114
-
115
- ```sh
116
- npx testdriverai@latest init
117
- ```
118
-
119
- ## Teach TestDriver a test
120
-
121
- Let's show TestDriver what we want to test. Run the following command:
122
-
123
- ```sh
124
- npx testdriverai@latest .testdriver/test.yaml
125
- ```
14
+ [🚀 **Quick Start**](#-quick-start) • [📖 **Documentation**](https://docs.testdriver.ai) • [💻 **Examples**](https://github.com/testdriverai/testdriverai/tree/main/test/testdriver) • [📖 **Pricing**](https://docs.testdriver.ai) • [💬 **Discord**](https://discord.com/invite/cWDFW8DzPm) • [🌐 **Website**](https://testdriver.ai)
126
15
 
127
- ## Reset the test state
16
+ </div>
128
17
 
129
- TestDriver best practice is to start instructing TestDriver with your app in it's initial state. For browsers, this means creating a new tab with the website you want to test.
18
+ <img width="1490" height="854" alt="image" src="https://github.com/user-attachments/assets/36e426cc-e740-426f-b9f6-6e8565e66ad6" />
130
19
 
131
- If you have multiple monitors, make sure you do this on your primary display.
20
+ ---
132
21
 
133
- ## Instruct TestDriver
22
+ ## Why TestDriver?
134
23
 
135
- Now, just tell TestDriver what you want it to do. For now, stick with single commands like "click sign up" and "scroll down."
24
+ Don't ship bugs because flows are too hard to test. TestDriver helps engineering teams easily test, debug, and monitor E2E flows that are hard or impossible to cover with other tools like:
136
25
 
137
- Later, try to perform higher level objectives like "complete the onboarding."
26
+ *Third-Party Web Apps* *Desktop Apps* *VS Code Extensions* • *Chrome Extensions* • *AI Chatbots* • *OAuth Flows* • *PDF Content* • *Spelling & Grammar* • *File System & Uploads* • *OS Accessibility* • *Visual Content* • *`<iframe>`* • *`<canvas>`* • *`<video>`*
138
27
 
139
- ```yaml
140
- > Click on sign up
141
- TestDriver Generates a Test
142
- TestDriver will look at your screen and generate a test script. TestDriver can see the screen, control the mouse, keyboard, and more!
143
- TestDriver can only see your primary display!
144
- To navigate to testdriver.ai, we need to focus on the
145
- Google Chrome application, click on the search bar, type
146
- the URL, and then press Enter.
28
+ ## Example
147
29
 
148
- Here are the steps:
30
+ ```js
31
+ // Click on the new text document
32
+ await testdriver.find("New text document").mouseDown();
149
33
 
150
- 1. Focus on the Google Chrome application.
151
- 2. Click on the search bar.
152
- 3. Type "testdriver.ai".
153
- 4. Press Enter.
34
+ // Drag the "New Text Document" icon to the "Recycle Bin"
35
+ await testdriver.find("Recycle Bin icon").mouseUp();
154
36
 
155
- Let's start with focusing on the Google Chrome application
156
- and clicking on the search bar.
157
-
158
- commands:
159
- - command: focus-application
160
- name: Google Chrome
161
- - command: hover-text
162
- text: Search Google or type a URL
163
- description: main google search
164
- action: click
165
-
166
- After this, we will type the URL and press Enter.
167
- ```
168
-
169
- ## TestDriver executes the test script
170
-
171
- TestDriver will execute the commands found in the yml codeblocks of the response.
172
-
173
- See the yml TestDriver generated? That's our own schema. You can learn more about it in the [reference](https://docs.testdriver.ai/getting-started/editing).
174
-
175
- > Take your hands off the mouse and keyboard while TestDriver executes! TestDriver is not a fan of backseat drivers.
176
-
177
- ## Keep going!
178
-
179
- Feel free to ask TestDriver to perform some more tasks. Every time you prompt TestDriver it will look at your screen and generate more test step to complete your goal.
180
-
181
- ```sh
182
- > navigate to airbnb.com
183
- > search for destinations in austin tx
184
- > click check in
185
- > select august 8
186
- ```
187
-
188
- If something didn't work, you can use `/undo` to remove all of the test steps added since the last prompt.
189
-
190
- ## Test the test locally
191
-
192
- Now it's time to make sure the test plan works before we deploy it. Use testdriver run to run the test file you just created with /save .
193
-
194
- ```sh
195
- npx testdriverai@latest run testdriver/test.yaml
196
- ```
197
-
198
- Make sure to reset the test state!
199
-
200
- ## Deploy
201
-
202
- Now it's time to deploy your test using our GitHub action! `testdriver init` already did the work for you and will start triggering tests once you commit the new files to your repository.
203
-
204
- ```sh
205
- git add .
206
- git commit -am "Add TestDriver tests"
207
- gh pr create --web
208
- ```
209
-
210
- Your test will run on every commit and the results will be posted as a Dashcam.io video within your GitHub summary! Learn more about deploying on CI [here](https://docs.testdriver.ai/action/setup).
211
-
212
- ## Using as a Module
213
-
214
- TestDriver can also be used programmatically as a Node.js module. This is useful when you want to integrate TestDriver into your own applications or customize the test file paths.
215
-
216
- ### Custom Test File Paths
217
-
218
- By default, TestDriver looks for test files at `testdriver/testdriver.yaml` relative to the current working directory. You can customize this:
219
-
220
- ```javascript
221
- const TestDriverAgent = require("testdriverai");
222
-
223
- // Option 1: Set default via environment variable
224
- const agent1 = new TestDriverAgent({
225
- TD_DEFAULT_TEST_FILE: "my-tests/integration.yaml",
226
- });
227
-
228
- // Option 2: Explicitly specify test file
229
- const agent2 = new TestDriverAgent(
230
- {},
231
- {
232
- args: ["path/to/specific/test.yaml"],
233
- },
234
- );
235
-
236
- // Option 3: Custom working directory + relative path
237
- const agent3 = new TestDriverAgent(
238
- { TD_DEFAULT_TEST_FILE: "tests/smoke.yaml" },
239
- { options: { workingDir: "/path/to/your/project" } },
37
+ // Assert "New Text Document" icon is not on the Desktop
38
+ const result = await testdriver.assert(
39
+ 'the "New Text Document" icon is not visible on the Desktop'
240
40
  );
241
-
242
- // Run the test
243
- await agent1.run();
41
+ expect(result).toBeTruthy();
244
42
  ```
245
43
 
246
- ### Environment Variables
44
+ [See Full Example](https://github.com/testdriverai/testdriverai/blob/main/test/testdriver/drag-and-drop.test.mjs) • [Browse All Examples](https://github.com/testdriverai/testdriverai/tree/main/test/testdriver)
247
45
 
248
- You can also set the default test file path using environment variables:
46
+ ---
249
47
 
250
- ```bash
251
- export TD_DEFAULT_TEST_FILE="custom/path/test.yaml"
252
- node your-script.js
253
- ```
254
-
255
- ## MCP Server for AI Agents
48
+ ## 🚀 Quick Start
256
49
 
257
- TestDriver includes a Model Context Protocol (MCP) server that enables AI agents to **interactively create Vitest test files**.
50
+ ### Step 1: Create a TestDriver Account
258
51
 
259
- ### How It Works
52
+ <a href="https://app.testdriver.ai/team"><img src="https://img.shields.io/badge/Sign_Up-Free_Account-blue?style=for-the-badge" alt="Sign Up"/></a>
260
53
 
261
- 1. **AI agent connects** to a persistent TestDriver sandbox
262
- 2. **User describes** what they want to test
263
- 3. **AI explores** the application using TestDriver commands
264
- 4. **AI generates** Vitest test code from successful interactions
54
+ *No credit card required!*
265
55
 
266
- ### Quick Start
56
+ ### Step 2: Initialize Your Project
267
57
 
268
58
  ```bash
269
- cd mcp-server
270
- npm install && npm run build
271
- npm run deploy # Install to ~/.mcp/testdriver
272
- ```
273
-
274
- ### Configuration
275
-
276
- Add to your MCP client configuration:
277
-
278
- ```json
279
- {
280
- "servers": {
281
- "testdriverai": {
282
- "type": "stdio",
283
- "command": "node",
284
- "args": ["/path/to/cli/mcp-server/dist/index.js"]
285
- }
286
- }
287
- }
59
+ npx testdriverai@beta init
288
60
  ```
289
61
 
290
- ### Example Workflow
62
+ This will:
63
+ - Create a project folder
64
+ - Install dependencies (Vitest + TestDriver)
65
+ - Set up your API key
66
+ - Generate an example test
291
67
 
292
- ```
293
- User: "Create a test that logs into the app"
68
+ ### Step 3: Run Your First Test
294
69
 
295
- AI: [Connects to sandbox with user's API key]
296
- AI: [Takes screenshot to see login page]
297
- AI: [Finds username field: await testdriver_find({ description: "username field" })]
298
- AI: [Clicks and types: await testdriver_type({ text: "test_user" })]
299
- AI: [Finds password field, enters password]
300
- AI: [Clicks login button]
301
- AI: [Asserts login succeeded]
302
- AI: [Generates Vitest test file from these steps]
303
- AI: [Saves test/login.test.mjs]
70
+ ```bash
71
+ vitest run
304
72
  ```
305
73
 
306
- ### Key Features
307
-
308
- - **Persistent sandbox** - Connection stays alive throughout test creation
309
- - **Live debugger URL** - User can watch the AI test in real-time
310
- - **Full SDK access** - All v7 SDK methods available
311
- - **Code generation** - AI translates interactions into proper Vitest code
312
-
313
- See [mcp-server/TEST_CREATION_GUIDE.md](mcp-server/TEST_CREATION_GUIDE.md) for the complete guide.
74
+ Watch as TestDriver:
75
+ 1. Spawns a cloud sandbox
76
+ 2. Launches Chrome
77
+ 3. Runs your test using AI vision
78
+ 4. Returns results with video replay
314
79
 
80
+ <a href="https://docs.testdriver.ai/v7/quickstart"><img src="https://img.shields.io/badge/📖_Read_Full_Quickstart-4A90E2?style=for-the-badge" alt="Full Quickstart"/></a>
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "testdriverai",
3
- "version": "7.2.19",
3
+ "version": "7.2.21",
4
4
  "description": "Next generation autonomous AI agent for end-to-end testing of web & desktop",
5
5
  "main": "sdk.js",
6
6
  "exports": {