@testdriverai/agent 7.11.229-test → 7.11.232-test
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/_data/examples-manifest.json +16 -16
- package/docs/_scripts/generate-examples.js +38 -45
- package/docs/agent.mdx +174 -93
- package/docs/assert.mdx +73 -269
- package/docs/aws-setup.mdx +196 -314
- package/docs/bring-your-own-machine.mdx +139 -206
- package/docs/cache.mdx +144 -151
- package/docs/caching.mdx +99 -89
- package/docs/captcha.mdx +126 -111
- package/docs/ci-cd.mdx +154 -543
- package/docs/click.mdx +74 -229
- package/docs/client.mdx +177 -231
- package/docs/copilot/auto-healing.mdx +86 -370
- package/docs/copilot/running-tests.mdx +121 -930
- package/docs/customizing-devices.mdx +74 -251
- package/docs/dashcam.mdx +72 -405
- package/docs/debugging-with-screenshots.mdx +105 -371
- package/docs/docs.json +79 -22
- package/docs/double-click.mdx +48 -68
- package/docs/errors.mdx +102 -176
- package/docs/events.mdx +170 -327
- package/docs/examples/{hover-text-with-description.mdx → add-to-cart.mdx} +30 -35
- package/docs/examples/assert.mdx +28 -41
- package/docs/examples/click-button.mdx +71 -0
- package/docs/examples/{hover-image.mdx → click-icon.mdx} +32 -28
- package/docs/examples/element-not-found.mdx +27 -26
- package/docs/examples/findall-coffee-icons.mdx +28 -32
- package/docs/examples/installer.mdx +61 -25
- package/docs/examples/launch-vscode-linux.mdx +28 -51
- package/docs/examples/navigate-and-scroll.mdx +82 -0
- package/docs/examples/parse.mdx +24 -28
- package/docs/examples/press-keys.mdx +28 -25
- package/docs/examples/scroll.mdx +25 -27
- package/docs/examples/type.mdx +26 -21
- package/docs/exec.mdx +92 -256
- package/docs/extract.mdx +69 -170
- package/docs/find.mdx +140 -736
- package/docs/focus-application.mdx +69 -246
- package/docs/generating-tests.mdx +103 -137
- package/docs/guide/best-practices-polling.mdx +65 -141
- package/docs/hosted.mdx +80 -144
- package/docs/hover.mdx +62 -235
- package/docs/locating-elements.mdx +74 -369
- package/docs/making-assertions.mdx +116 -74
- package/docs/mcp-setup.mdx +80 -63
- package/docs/mouse-down.mdx +48 -125
- package/docs/mouse-up.mdx +47 -129
- package/docs/options.mdx +75 -48
- package/docs/parse.mdx +95 -184
- package/docs/performing-actions.mdx +104 -94
- package/docs/press-keys.mdx +59 -299
- package/docs/provision.mdx +137 -182
- package/docs/quickstart-claude.mdx +85 -0
- package/docs/quickstart-github.mdx +62 -32
- package/docs/quickstart-manual.mdx +60 -33
- package/docs/quickstart-vscode.mdx +97 -0
- package/docs/quickstart.mdx +26 -9
- package/docs/redraw.mdx +101 -165
- package/docs/reusable-code.mdx +97 -200
- package/docs/right-click.mdx +47 -89
- package/docs/screenshot.mdx +47 -201
- package/docs/screenshots.mdx +87 -137
- package/docs/scroll.mdx +71 -154
- package/docs/secrets.mdx +96 -91
- package/docs/self-hosted.mdx +83 -142
- package/docs/snippets/enterprise.mdx +15 -0
- package/docs/styles.css +68 -0
- package/docs/test-results-json.mdx +167 -204
- package/docs/type.mdx +56 -304
- package/docs/variables.mdx +121 -81
- package/docs/wait.mdx +33 -31
- package/docs/what-is-testdriver.mdx +80 -32
- package/examples/{hover-text-with-description.test.mjs → add-to-cart.test.mjs} +4 -5
- package/examples/assert.test.mjs +3 -20
- package/examples/{hover-text.test.mjs → click-button.test.mjs} +3 -4
- package/examples/{hover-image.test.mjs → click-icon.test.mjs} +5 -6
- package/examples/element-not-found.test.mjs +3 -5
- package/examples/findall-coffee-icons.test.mjs +3 -3
- package/examples/installer.test.mjs +1 -1
- package/examples/launch-vscode-linux.test.mjs +6 -33
- package/examples/{scroll-keyboard.test.mjs → navigate-and-scroll.test.mjs} +4 -5
- package/examples/parse.test.mjs +1 -2
- package/examples/press-keys.test.mjs +4 -4
- package/examples/scroll.test.mjs +3 -6
- package/examples/type.test.mjs +5 -2
- package/lib/provision.js +6 -2
- package/package.json +1 -1
- package/sdk.d.ts +4 -2
- package/sdk.js +1 -1
- package/docs/claude-mcp-plugin.mdx +0 -244
- package/docs/elements.mdx +0 -606
- package/docs/examples/ai.mdx +0 -78
- package/docs/examples/captcha-api.mdx +0 -92
- package/docs/examples/chrome-extension.mdx +0 -156
- package/docs/examples/drag-and-drop.mdx +0 -100
- package/docs/examples/formatted-logging.mdx +0 -74
- package/docs/examples/hover-text.mdx +0 -70
- package/docs/examples/match-image.mdx +0 -97
- package/docs/examples/scroll-keyboard.mdx +0 -80
- package/docs/examples/scroll-until-text.mdx +0 -109
|
@@ -1,179 +1,103 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "
|
|
3
|
-
sidebarTitle: "Waiting for
|
|
4
|
-
description: "
|
|
2
|
+
title: "Wait for elements"
|
|
3
|
+
sidebarTitle: "Waiting for elements"
|
|
4
|
+
description: "Wait for elements with find() and a timeout instead of fixed wait() delays."
|
|
5
5
|
icon: "hourglass-half"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
## When to Use `wait()` vs `find()` with Timeout
|
|
11
|
-
|
|
12
|
-
`wait()` is useful for **simple pauses** — after actions, for animations, or for state changes to settle:
|
|
8
|
+
To wait for something to appear, give [`find()`](/find) a `timeout`. Don't pause with [`wait()`](/wait) and hope it's there.
|
|
13
9
|
|
|
14
10
|
```javascript
|
|
15
|
-
await testdriver.find('
|
|
16
|
-
await testdriver.wait(2000); // Wait for animation to complete
|
|
11
|
+
const confirmation = await testdriver.find('Order confirmation heading', { timeout: 30000 });
|
|
17
12
|
```
|
|
18
13
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
// ✅ GOOD: Polls until the element appears (up to 30s)
|
|
23
|
-
const element = await testdriver.find('success message', { timeout: 30000 });
|
|
24
|
-
|
|
25
|
-
// ❌ BAD: Arbitrary wait then hope the element is there
|
|
26
|
-
await testdriver.wait(5000);
|
|
27
|
-
const element = await testdriver.find('success message');
|
|
28
|
-
```
|
|
14
|
+
- `find()` returns as soon as the element appears, so fast runs stay fast.
|
|
15
|
+
- It keeps checking until the timeout, so slow runs still pass.
|
|
16
|
+
- A fixed `wait()` always takes its full duration, and may still be too short.
|
|
29
17
|
|
|
30
|
-
##
|
|
18
|
+
## How find() waits
|
|
31
19
|
|
|
32
|
-
|
|
20
|
+
`find()` waits by default. It checks the screen, and if the element isn't there, it tries again:
|
|
33
21
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
4. **Hard to debug**: When tests fail, you don't know if it was a timing issue or actual failure
|
|
22
|
+
- It waits 5 seconds between checks.
|
|
23
|
+
- It stops after `timeout` milliseconds. The default is `10000` (10 seconds).
|
|
24
|
+
- `timeout: 0` checks once and returns right away.
|
|
38
25
|
|
|
39
|
-
|
|
26
|
+
If the element never appears, `find()` doesn't throw. It returns an element whose `found()` is `false`. Calling `click()` or another action on that element throws `ElementNotFoundError`.
|
|
40
27
|
|
|
41
|
-
|
|
28
|
+
## Wait for a slow page
|
|
42
29
|
|
|
43
|
-
|
|
30
|
+
Raise the `timeout` for steps that take longer than 10 seconds, such as a page load or a file upload:
|
|
44
31
|
|
|
45
|
-
```javascript
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
const button = await testdriver.find("Submit button");
|
|
49
|
-
|
|
50
|
-
// ✅ CORRECT: Polling with find()
|
|
51
|
-
let button;
|
|
52
|
-
for (let i = 0; i < 10; i++) {
|
|
53
|
-
try {
|
|
54
|
-
button = await testdriver.find("Submit button");
|
|
55
|
-
if (button.found()) break;
|
|
56
|
-
} catch (e) {
|
|
57
|
-
if (i === 9) throw e; // Re-throw on last attempt
|
|
58
|
-
}
|
|
59
|
-
await new Promise(resolve => setTimeout(resolve, 1000));
|
|
60
|
-
}
|
|
61
|
-
```
|
|
32
|
+
```javascript tests/checkout.test.mjs
|
|
33
|
+
import { expect, it } from 'vitest';
|
|
34
|
+
import { TestDriver } from 'testdriverai/vitest/hooks';
|
|
62
35
|
|
|
63
|
-
|
|
36
|
+
it('places an order', async (context) => {
|
|
37
|
+
const testdriver = TestDriver(context);
|
|
38
|
+
await testdriver.provision.chrome({ url: 'https://testdriver-sandbox.vercel.app/login' });
|
|
64
39
|
|
|
65
|
-
|
|
40
|
+
// ...sign in and fill the cart
|
|
66
41
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
for (let i = 0; i < maxAttempts; i++) {
|
|
70
|
-
try {
|
|
71
|
-
const element = await testdriver.find(description);
|
|
72
|
-
if (element.found()) {
|
|
73
|
-
return element;
|
|
74
|
-
}
|
|
75
|
-
} catch (e) {
|
|
76
|
-
if (i === maxAttempts - 1) throw e;
|
|
77
|
-
}
|
|
78
|
-
await new Promise(resolve => setTimeout(resolve, delayMs));
|
|
79
|
-
}
|
|
80
|
-
throw new Error(`Element not found after ${maxAttempts} attempts: ${description}`);
|
|
81
|
-
}
|
|
42
|
+
const placeOrderButton = await testdriver.find('Place order button');
|
|
43
|
+
await placeOrderButton.click();
|
|
82
44
|
|
|
83
|
-
//
|
|
84
|
-
const
|
|
85
|
-
|
|
45
|
+
// Payment can take a while, so wait up to 30 seconds
|
|
46
|
+
const confirmation = await testdriver.find('Order confirmation heading', { timeout: 30000 });
|
|
47
|
+
expect(confirmation.found()).toBe(true);
|
|
48
|
+
});
|
|
86
49
|
```
|
|
87
50
|
|
|
88
|
-
##
|
|
51
|
+
## Handle an element that may not appear
|
|
89
52
|
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
- **After navigation**: Waiting for a new page to load
|
|
93
|
-
- **After user action**: Waiting for UI updates (form submission, modal opening, etc.)
|
|
94
|
-
- **Dynamic content**: Waiting for AJAX-loaded elements
|
|
95
|
-
- **State transitions**: Waiting for loading spinners to disappear or success messages to appear
|
|
96
|
-
|
|
97
|
-
## Example: Complete Login Flow
|
|
53
|
+
For optional elements, like a cookie banner, use a short `timeout` and check `found()`:
|
|
98
54
|
|
|
99
55
|
```javascript
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
// Helper function
|
|
103
|
-
async function waitForElement(testdriver, description, maxAttempts = 10, delayMs = 1000) {
|
|
104
|
-
for (let i = 0; i < maxAttempts; i++) {
|
|
105
|
-
try {
|
|
106
|
-
const element = await testdriver.find(description);
|
|
107
|
-
if (element.found()) return element;
|
|
108
|
-
} catch (e) {
|
|
109
|
-
if (i === maxAttempts - 1) throw e;
|
|
110
|
-
}
|
|
111
|
-
await new Promise(resolve => setTimeout(resolve, delayMs));
|
|
112
|
-
}
|
|
113
|
-
throw new Error(`Element not found after ${maxAttempts} attempts: ${description}`);
|
|
114
|
-
}
|
|
56
|
+
const cookieBanner = await testdriver.find('Cookie consent banner', { timeout: 5000 });
|
|
115
57
|
|
|
116
|
-
|
|
117
|
-
const
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
// Wait for login page to load
|
|
122
|
-
const emailField = await waitForElement(testdriver, "Email input field");
|
|
123
|
-
await emailField.click();
|
|
124
|
-
await testdriver.type("user@example.com");
|
|
125
|
-
|
|
126
|
-
const passwordField = await testdriver.find("Password input field");
|
|
127
|
-
await passwordField.click();
|
|
128
|
-
await testdriver.type("password123");
|
|
129
|
-
|
|
130
|
-
const loginButton = await testdriver.find("Login button");
|
|
131
|
-
await loginButton.click();
|
|
132
|
-
|
|
133
|
-
// Wait for dashboard to load after login
|
|
134
|
-
await waitForElement(testdriver, "Dashboard welcome message");
|
|
135
|
-
|
|
136
|
-
// Verify login successful
|
|
137
|
-
const isLoggedIn = await testdriver.assert("user is logged in to dashboard");
|
|
138
|
-
expect(isLoggedIn).toBeTruthy();
|
|
139
|
-
});
|
|
58
|
+
if (cookieBanner.found()) {
|
|
59
|
+
const acceptButton = await testdriver.find('Accept button in the cookie banner');
|
|
60
|
+
await acceptButton.click();
|
|
61
|
+
}
|
|
140
62
|
```
|
|
141
63
|
|
|
142
|
-
##
|
|
64
|
+
## Wait for an element to disappear
|
|
143
65
|
|
|
144
|
-
|
|
66
|
+
`find()` waits for elements to appear, not to go away. To wait for a spinner to disappear, check once per loop with `timeout: 0`:
|
|
145
67
|
|
|
146
68
|
```javascript
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
const
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
} catch {
|
|
154
|
-
console.log("No cookie dialog found, continuing...");
|
|
69
|
+
const deadline = Date.now() + 30000;
|
|
70
|
+
|
|
71
|
+
while (Date.now() < deadline) {
|
|
72
|
+
const spinner = await testdriver.find('Loading spinner', { timeout: 0 });
|
|
73
|
+
if (!spinner.found()) break;
|
|
74
|
+
await testdriver.wait(2000);
|
|
155
75
|
}
|
|
156
76
|
```
|
|
157
77
|
|
|
158
|
-
##
|
|
78
|
+
## When to use wait()
|
|
159
79
|
|
|
160
|
-
|
|
80
|
+
`wait(ms)` pauses for a fixed time. It defaults to 3000 ms. Use it only when there's nothing on screen to wait for, such as letting an animation finish:
|
|
161
81
|
|
|
162
82
|
```javascript
|
|
163
|
-
|
|
164
|
-
await waitForElement(testdriver, "Success message", 10, 300);
|
|
165
|
-
|
|
166
|
-
// Patient polling for slow operations (check every 2s, up to 20 seconds)
|
|
167
|
-
await waitForElement(testdriver, "Processing complete indicator", 10, 2000);
|
|
83
|
+
await testdriver.wait(1000);
|
|
168
84
|
```
|
|
169
85
|
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
|
|
86
|
+
`waitForText()` and `waitForImage()` are deprecated. Use `find()` with a `timeout` instead.
|
|
87
|
+
|
|
88
|
+
## Next steps
|
|
89
|
+
|
|
90
|
+
<CardGroup cols={2}>
|
|
91
|
+
<Card title="find()" icon="magnifying-glass" href="/find">
|
|
92
|
+
Every find() option, including timeout.
|
|
93
|
+
</Card>
|
|
94
|
+
<Card title="Locate elements" icon="crosshairs" href="/locating-elements">
|
|
95
|
+
Write descriptions that find the right element.
|
|
96
|
+
</Card>
|
|
97
|
+
<Card title="Debug a failing test" icon="bug" href="/debugging-with-screenshots">
|
|
98
|
+
Find out why an element wasn't found.
|
|
99
|
+
</Card>
|
|
100
|
+
<Card title="Redraw detection" icon="arrows-rotate" href="/redraw">
|
|
101
|
+
Wait for the screen to settle after each action.
|
|
102
|
+
</Card>
|
|
103
|
+
</CardGroup>
|
package/docs/hosted.mdx
CHANGED
|
@@ -1,157 +1,93 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Hosted"
|
|
2
|
+
title: "Hosted plans"
|
|
3
3
|
sidebarTitle: "Hosted"
|
|
4
|
-
description: "
|
|
4
|
+
description: "Run tests on TestDriver's cloud sandboxes, billed by testing minutes, with no infrastructure to manage."
|
|
5
5
|
icon: "cloud"
|
|
6
6
|
mode: "wide"
|
|
7
7
|
---
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
import { EnterpriseBadge, EnterpriseCta } from '/snippets/enterprise.mdx';
|
|
10
10
|
|
|
11
|
-
|
|
12
|
-
- **Free Tier** — Start with a limited preview at no cost.
|
|
13
|
-
- **Pay As You Go** — Pay only for the device-seconds that you use.
|
|
11
|
+
By default, your tests run on TestDriver's cloud. Each test gets a fresh sandbox, and you don't manage any machines.
|
|
14
12
|
|
|
15
|
-
|
|
13
|
+
- Hosted is the default. When you follow the [Quickstart](/quickstart), you're using it.
|
|
14
|
+
- You're billed in **testing minutes**: the time a sandbox is running.
|
|
15
|
+
- Your plan sets how many sandboxes can run at the same time.
|
|
16
16
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
17
|
+
## Plans
|
|
18
|
+
|
|
19
|
+
| | Free | Pro | Team |
|
|
20
|
+
| --- | --- | --- | --- |
|
|
21
|
+
| **Price** | $0 | $20 per seat per month | $600 per month |
|
|
22
|
+
| **Testing minutes included** | 30 | 120 | 10,000 |
|
|
23
|
+
| **Extra minutes** | Not available | $0.14 per minute | $0.14 per minute |
|
|
24
|
+
| **Sandboxes at once** | 2 | 4 | 8 |
|
|
25
|
+
| **Users** | Unlimited | One per seat | 5 |
|
|
26
|
+
| **Test analytics** | No | Yes | Yes |
|
|
27
|
+
| **CPU, RAM, and network profiles** | No | Yes | Yes |
|
|
28
|
+
| **Support** | Community | Community | Private |
|
|
29
|
+
|
|
30
|
+
<EnterpriseCta feature="Windows sandboxes, custom screen resolutions, and self-hosting" plural />
|
|
31
|
+
|
|
32
|
+
Pro starts with a 14-day free trial. Start a plan from the [TestDriver console](https://console.testdriver.ai/checkout/pro).
|
|
33
|
+
|
|
34
|
+
On the Free plan, new sandboxes are refused once you've used your 30 minutes. Upgrade to keep testing.
|
|
35
|
+
|
|
36
|
+
<Note>
|
|
37
|
+
If you install TestDriver from GitHub Marketplace, GitHub bills you instead. The **Open Source** plan is free with 300 testing minutes per month. The **Team** plan is $20 per seat per month with 120 testing minutes per seat.
|
|
38
|
+
</Note>
|
|
39
|
+
|
|
40
|
+
## How testing minutes are counted
|
|
41
|
+
|
|
42
|
+
Testing minutes run while a sandbox is running. That includes sandboxes started by:
|
|
43
|
+
|
|
44
|
+
- Your test runs, locally and in CI.
|
|
45
|
+
- The TestDriver agent, when it reviews a pull request or writes a test.
|
|
46
|
+
|
|
47
|
+
A sandbox kept running with [`keepAlive`](/customizing-devices#keep-a-sandbox-alive) uses minutes until it shuts down. Keep `keepAlive` as short as you need.
|
|
48
|
+
|
|
49
|
+
## Concurrency
|
|
50
|
+
|
|
51
|
+
Each running sandbox uses one of your plan's slots. A slot frees up when its sandbox shuts down.
|
|
52
|
+
|
|
53
|
+
When every slot is busy, a new test waits for a free slot instead of failing. The SDK retries every 10 seconds, for up to 60 seconds by default. Set `TD_CONCURRENCY_MAX_WAIT` to change the wait. See [Concurrency limits](/ci-cd#concurrency-limits).
|
|
54
|
+
|
|
55
|
+
To stay within your slots, limit how many test files Vitest runs at once:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
npx vitest run --maxWorkers=4
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
In GitHub Actions, a `concurrency` group stops two runs on the same branch from competing for slots. A new push cancels the older run:
|
|
62
|
+
|
|
63
|
+
```yaml .github/workflows/testdriver.yml
|
|
64
|
+
concurrency:
|
|
65
|
+
group: ${{ github.workflow }}-${{ github.ref }}
|
|
66
|
+
cancel-in-progress: true
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## When to self-host
|
|
70
|
+
|
|
71
|
+
Consider [self-hosting](/self-hosted) when you need to:
|
|
72
|
+
|
|
73
|
+
- Run tests on your own infrastructure, without per-minute billing.
|
|
74
|
+
- Use your own AI provider keys.
|
|
75
|
+
- Run custom VM images with your software preinstalled.
|
|
76
|
+
- Run more tests in parallel than the hosted plans allow.
|
|
77
|
+
|
|
78
|
+
## Next steps
|
|
79
|
+
|
|
80
|
+
<CardGroup cols={2}>
|
|
81
|
+
<Card title="Quickstart" icon="play" href="/quickstart">
|
|
82
|
+
Run your first test on a hosted sandbox.
|
|
25
83
|
</Card>
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
**$20/month per user**
|
|
29
|
-
|
|
30
|
-
- 2 Concurrent Sandboxes
|
|
31
|
-
- 120 Minutes Included
|
|
32
|
-
- Overage: $0.001/second
|
|
33
|
-
- Test Recordings
|
|
34
|
-
- Community Support
|
|
84
|
+
<Card title="Run tests in CI/CD" icon="code-branch" href="/ci-cd">
|
|
85
|
+
Run your tests on every push and pull request.
|
|
35
86
|
</Card>
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
- 10,000 Minutes Included
|
|
42
|
-
- Overage: $0.001/second
|
|
43
|
-
- 5 Team Users
|
|
44
|
-
- Test Recordings
|
|
45
|
-
- Private Support
|
|
46
|
-
- Test Analytics
|
|
47
|
-
- CPU, RAM, & Network Profiles
|
|
87
|
+
<Card title="Self-hosted" icon="server" href="/self-hosted">
|
|
88
|
+
Run sandboxes on your own infrastructure.
|
|
89
|
+
</Card>
|
|
90
|
+
<Card title="Run tests" icon="play" href="/copilot/running-tests">
|
|
91
|
+
Run tests locally and in parallel.
|
|
48
92
|
</Card>
|
|
49
93
|
</CardGroup>
|
|
50
|
-
|
|
51
|
-
## Get Started
|
|
52
|
-
Hosted is the default when you follow the Quickstart guide.
|
|
53
|
-
<Card
|
|
54
|
-
title="Try the Quickstart"
|
|
55
|
-
icon="play"
|
|
56
|
-
href="/quickstart"
|
|
57
|
-
>
|
|
58
|
-
Set your API key and start testing in minutes.
|
|
59
|
-
</Card>
|
|
60
|
-
|
|
61
|
-
## Parallel Testing Limits
|
|
62
|
-
|
|
63
|
-
Your account has a set number of **license slots** that determine how many devices can run simultaneously. You can view your available slots in the [TestDriver Dashboard](https://console.testdriver.ai).
|
|
64
|
-
|
|
65
|
-
<Info>
|
|
66
|
-
**When is a slot in use?** A license slot is occupied when a test client is connected. As soon as your device is destroyed the slot becomes available immediately.
|
|
67
|
-
</Info>
|
|
68
|
-
|
|
69
|
-
## Avoiding Slot Conflicts
|
|
70
|
-
|
|
71
|
-
To prevent tests from failing due to exceeding your license slot limit, we recommend two key configurations:
|
|
72
|
-
|
|
73
|
-
<AccordionGroup>
|
|
74
|
-
<Accordion title="Set Maximum Concurrency in Vitest">
|
|
75
|
-
Limit concurrent tests to match your available license slots:
|
|
76
|
-
|
|
77
|
-
```javascript vitest.config.mjs
|
|
78
|
-
import { defineConfig } from 'vitest/config';
|
|
79
|
-
import TestDriver from 'testdriverai/vitest';
|
|
80
|
-
|
|
81
|
-
export default defineConfig({
|
|
82
|
-
test: {
|
|
83
|
-
testTimeout: 900000,
|
|
84
|
-
hookTimeout: 900000,
|
|
85
|
-
maxConcurrency: 5, // Set to your license slot limit
|
|
86
|
-
reporters: ['default', TestDriver()],
|
|
87
|
-
setupFiles: ['testdriverai/vitest/setup'],
|
|
88
|
-
},
|
|
89
|
-
});
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
<Tip>
|
|
93
|
-
Check your slot count at [console.testdriver.ai](https://console.testdriver.ai) and set `maxConcurrency` to that number or lower.
|
|
94
|
-
</Tip>
|
|
95
|
-
</Accordion>
|
|
96
|
-
|
|
97
|
-
<Accordion title="Use GitHub Concurrency Keys">
|
|
98
|
-
Prevent multiple workflow runs from competing for the same slots by using [GitHub's concurrency controls](https://docs.github.com/actions/writing-workflows/choosing-what-your-workflow-does/control-the-concurrency-of-workflows-and-jobs):
|
|
99
|
-
|
|
100
|
-
```yaml .github/workflows/test.yml
|
|
101
|
-
name: Tests
|
|
102
|
-
|
|
103
|
-
on:
|
|
104
|
-
push:
|
|
105
|
-
branches: [main]
|
|
106
|
-
pull_request:
|
|
107
|
-
|
|
108
|
-
# Prevent concurrent runs from competing for license slots
|
|
109
|
-
concurrency:
|
|
110
|
-
group: ${{ github.workflow }}-${{ github.ref }}
|
|
111
|
-
cancel-in-progress: true
|
|
112
|
-
|
|
113
|
-
jobs:
|
|
114
|
-
test:
|
|
115
|
-
runs-on: ubuntu-latest
|
|
116
|
-
steps:
|
|
117
|
-
- uses: actions/checkout@v4
|
|
118
|
-
|
|
119
|
-
- name: Setup Node.js
|
|
120
|
-
uses: actions/setup-node@v4
|
|
121
|
-
with:
|
|
122
|
-
node-version: '20'
|
|
123
|
-
|
|
124
|
-
- name: Install dependencies
|
|
125
|
-
run: npm install
|
|
126
|
-
|
|
127
|
-
- name: Run tests
|
|
128
|
-
run: vitest run
|
|
129
|
-
env:
|
|
130
|
-
TD_API_KEY: ${{ secrets.TD_API_KEY }}
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
The `concurrency` block ensures:
|
|
134
|
-
- Only one workflow run per branch runs at a time
|
|
135
|
-
- New pushes cancel in-progress runs on the same branch
|
|
136
|
-
- Different branches/PRs can run in parallel (up to your slot limit)
|
|
137
|
-
</Accordion>
|
|
138
|
-
</AccordionGroup>
|
|
139
|
-
|
|
140
|
-
## When to Consider Self-Hosted
|
|
141
|
-
|
|
142
|
-
Hosted is perfect for getting started and for teams that want zero infrastructure management. However, you might consider [Self-Hosted](/self-hosted) if you:
|
|
143
|
-
|
|
144
|
-
- Want to escape per-second billing with a flat license fee
|
|
145
|
-
- Require greater concurrency than offered in Cloud plans
|
|
146
|
-
- Need full control over your infrastructure and privacy
|
|
147
|
-
- Want to use your own AI API keys
|
|
148
|
-
- Require custom hardware configurations
|
|
149
|
-
- Have high test volumes that make self-hosting more economical
|
|
150
|
-
|
|
151
|
-
<Card
|
|
152
|
-
title="Explore Self-Hosted"
|
|
153
|
-
icon="server"
|
|
154
|
-
href="/self-hosted"
|
|
155
|
-
>
|
|
156
|
-
Learn about self-hosting for unlimited test execution at a flat rate.
|
|
157
|
-
</Card>
|