@testdriverai/agent 7.11.240-test → 7.11.241-test

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/ai/skills/testdriver-agent/SKILL.md +174 -91
  2. package/ai/skills/testdriver-assert/SKILL.md +73 -269
  3. package/ai/skills/testdriver-aws-setup/SKILL.md +194 -312
  4. package/ai/skills/testdriver-bring-your-own-machine/SKILL.md +137 -204
  5. package/ai/skills/testdriver-cache/SKILL.md +142 -149
  6. package/ai/skills/testdriver-caching/SKILL.md +97 -87
  7. package/ai/skills/testdriver-captcha/SKILL.md +126 -111
  8. package/ai/skills/testdriver-ci-cd/SKILL.md +153 -542
  9. package/ai/skills/testdriver-click/SKILL.md +74 -229
  10. package/ai/skills/testdriver-client/SKILL.md +175 -229
  11. package/ai/skills/testdriver-customizing-devices/SKILL.md +72 -250
  12. package/ai/skills/testdriver-dashcam/SKILL.md +71 -404
  13. package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +103 -369
  14. package/ai/skills/testdriver-double-click/SKILL.md +46 -67
  15. package/ai/skills/testdriver-errors/SKILL.md +102 -176
  16. package/ai/skills/testdriver-events/SKILL.md +170 -327
  17. package/ai/skills/testdriver-exec/SKILL.md +92 -256
  18. package/ai/skills/testdriver-extract/SKILL.md +69 -170
  19. package/ai/skills/testdriver-find/SKILL.md +140 -736
  20. package/ai/skills/testdriver-focus-application/SKILL.md +69 -246
  21. package/ai/skills/testdriver-generating-tests/SKILL.md +102 -136
  22. package/ai/skills/testdriver-hosted/SKILL.md +79 -143
  23. package/ai/skills/testdriver-hover/SKILL.md +62 -235
  24. package/ai/skills/testdriver-locating-elements/SKILL.md +71 -366
  25. package/ai/skills/testdriver-making-assertions/SKILL.md +114 -72
  26. package/ai/skills/testdriver-mcp-setup/SKILL.md +79 -62
  27. package/ai/skills/testdriver-mouse-down/SKILL.md +46 -124
  28. package/ai/skills/testdriver-mouse-up/SKILL.md +45 -128
  29. package/ai/skills/testdriver-options/SKILL.md +73 -46
  30. package/ai/skills/testdriver-parse/SKILL.md +95 -184
  31. package/ai/skills/testdriver-performing-actions/SKILL.md +101 -91
  32. package/ai/skills/testdriver-press-keys/SKILL.md +59 -299
  33. package/ai/skills/testdriver-provision/SKILL.md +135 -179
  34. package/ai/skills/testdriver-quickstart/SKILL.md +24 -8
  35. package/ai/skills/testdriver-quickstart-claude/SKILL.md +84 -0
  36. package/ai/skills/testdriver-quickstart-github/SKILL.md +62 -32
  37. package/ai/skills/testdriver-quickstart-manual/SKILL.md +59 -32
  38. package/ai/skills/testdriver-quickstart-vscode/SKILL.md +96 -0
  39. package/ai/skills/testdriver-redraw/SKILL.md +100 -164
  40. package/ai/skills/testdriver-reusable-code/SKILL.md +95 -199
  41. package/ai/skills/testdriver-right-click/SKILL.md +45 -88
  42. package/ai/skills/testdriver-screenshot/SKILL.md +47 -201
  43. package/ai/skills/testdriver-screenshots/SKILL.md +86 -136
  44. package/ai/skills/testdriver-scroll/SKILL.md +71 -154
  45. package/ai/skills/testdriver-secrets/SKILL.md +94 -90
  46. package/ai/skills/testdriver-self-hosted/SKILL.md +82 -140
  47. package/ai/skills/testdriver-test-results-json/SKILL.md +165 -202
  48. package/ai/skills/testdriver-type/SKILL.md +56 -304
  49. package/ai/skills/testdriver-variables/SKILL.md +119 -80
  50. package/ai/skills/testdriver-wait/SKILL.md +31 -29
  51. package/ai/skills/testdriver-what-is-testdriver/SKILL.md +80 -32
  52. package/package.json +1 -1
@@ -1,40 +1,34 @@
1
1
  ---
2
2
  name: testdriver:ci-cd
3
- description: Run TestDriver tests in CI/CD with parallel execution and cross-platform support
3
+ description: Run TestDriver tests on every push and pull request in GitHub Actions or any CI provider.
4
4
  ---
5
5
  <!-- Generated from ci-cd.mdx. DO NOT EDIT. -->
6
6
 
7
- TestDriver integrates with the common CI providers. It lets you do automated end-to-end tests on each push and pull request.
7
+ import { EnterpriseBadge, EnterpriseCta } from '/snippets/enterprise.mdx';
8
8
 
9
- ## Authentication
9
+ TestDriver tests are ordinary Vitest files, so any CI provider can run them with `npx vitest run`. Each test starts its own cloud sandbox, and every run is recorded in the [TestDriver console](https://console.testdriver.ai).
10
10
 
11
- On **GitHub Actions, use OIDC** through the published `testdriverai/action`. Then
12
- there is no `TD_API_KEY` secret to store, copy, or change. The action shows that the
13
- workflow runs in your org. TestDriver changes that proof for your
14
- team's key at run time. See the GitHub Actions tab below.
11
+ You need two things in CI:
15
12
 
16
- For other CI providers (or self-hosted runners without OIDC), use a
17
- stored API key from [console.testdriver.ai/settings](https://console.testdriver.ai/settings).
18
- Add it as a `TD_API_KEY` secret in the settings of your CI provider.
13
+ - **Authentication.** On GitHub Actions, use the TestDriver action with OIDC. Elsewhere, store your API key as a secret named `TD_API_KEY`.
14
+ - **Your test command,** `npx vitest run`.
19
15
 
20
- <Note>
21
- Never commit your API key in the code. Always use OIDC or the secrets management of your CI provider.
22
- </Note>
16
+ ## GitHub Actions
23
17
 
24
- ## CI Provider Examples
18
+ The [`testdriverai/action`](https://github.com/testdriverai/action) authenticates with GitHub's OIDC token, so you don't store an API key in GitHub. It exchanges the token for your team's key and exports `TD_API_KEY` for later steps. It doesn't run tests itself.
25
19
 
26
- <Tabs>
27
- <Tab title="GitHub Actions">
28
- ### Authenticate with OIDC via `testdriverai/action` (recommended)
20
+ <Steps>
21
+ <Step title="Authorize the TestDriver GitHub app">
22
+
23
+ Install the [TestDriver GitHub app](/quickstart-github) on your organization. This links the organization to your TestDriver team. If you authorized the app before OIDC support was added, authorize it again once.
29
24
 
30
- Use the published [`testdriverai/action`](https://github.com/testdriverai/action). It makes the OIDC token, changes it for your team's API key, and exports `TD_API_KEY` for the steps that come after. **There is no `TD_API_KEY` secret to store or change.**
25
+ </Step>
26
+ <Step title="Add the workflow">
31
27
 
32
- <Note>
33
- Do this one time: authorize the [TestDriver GitHub App](https://console.testdriver.ai) for your org. Then the org → team binding exists. If your org authorized the App before OIDC support was available, authorize it again one time. If the App is not authorized, the action fails with a console link. (Or it uses the `api-key` secret if you give one.)
34
- </Note>
28
+ `npx testdriverai init` writes this file for you:
35
29
 
36
30
  ```yaml .github/workflows/testdriver.yml
37
- name: TestDriver Tests
31
+ name: TestDriver tests
38
32
 
39
33
  on:
40
34
  push:
@@ -46,7 +40,7 @@ Add it as a `TD_API_KEY` secret in the settings of your CI provider.
46
40
  test:
47
41
  runs-on: ubuntu-latest
48
42
  permissions:
49
- id-token: write # REQUIRED to mint an OIDC token
43
+ id-token: write # lets the job mint an OIDC token
50
44
  contents: read
51
45
 
52
46
  steps:
@@ -60,260 +54,74 @@ Add it as a `TD_API_KEY` secret in the settings of your CI provider.
60
54
  - run: npm ci
61
55
 
62
56
  - name: Authenticate to TestDriver
63
- uses: testdriverai/action@stable # pin @stable / @canary / @test to your SDK channel
57
+ uses: testdriverai/action@stable
64
58
  with:
65
- api-key: ${{ secrets.TD_API_KEY }} # optional fallback if OIDC isn't set up
59
+ api-key: ${{ secrets.TD_API_KEY }} # optional fallback
66
60
 
67
- - name: Run TestDriver tests
68
- run: vitest run
61
+ - name: Run tests
62
+ run: npx vitest run
69
63
  ```
70
64
 
71
- ### Stored-key fallback
72
-
73
- Only if you can't use OIDC (e.g. self-hosted runners without an OIDC provider). Add the key as a secret and pass it via `env`:
74
-
75
- 1. Navigate to your GitHub repository
76
- 2. Go to **Settings** → **Secrets and variables** → **Actions**
77
- 3. Click **New repository secret**
78
- 4. Name: `TD_API_KEY`, Value: your API key
79
- 5. Click **Add secret**
65
+ </Step>
66
+ </Steps>
80
67
 
81
- ### Basic Workflow
68
+ The job needs `id-token: write`, or the action can't mint the OIDC token.
82
69
 
83
- Create `.github/workflows/testdriver.yml`:
70
+ The `api-key` input is a fallback. The action uses it only when OIDC authentication fails, for example on a fork or before the app is authorized. Leave the secret unset if you don't need it.
84
71
 
85
- ```yaml .github/workflows/testdriver.yml
86
- name: TestDriver Tests
87
-
88
- on:
89
- push:
90
- branches: [main]
91
- pull_request:
92
- branches: [main]
93
-
94
- jobs:
95
- test:
96
- runs-on: ubuntu-latest
97
-
98
- steps:
99
- - uses: actions/checkout@v4
100
-
101
- - uses: actions/setup-node@v4
102
- with:
103
- node-version: '20'
104
- cache: 'npm'
105
-
106
- - run: npm ci
107
-
108
- - name: Run TestDriver tests
109
- env:
110
- TD_API_KEY: ${{ secrets.TD_API_KEY }}
111
- run: vitest --run
112
- ```
72
+ <AccordionGroup>
73
+ <Accordion title="Action inputs">
113
74
 
114
- ### Parallel Execution
75
+ | Input | Default | Description |
76
+ | --- | --- | --- |
77
+ | `channel` | `stable` | TestDriver release channel: `stable`, `canary`, `test`, or `dev`. Selects the API the action talks to. |
78
+ | `api-root` | | An explicit API URL. Overrides `channel`. |
79
+ | `api-key` | | A TestDriver API key to use if OIDC authentication fails. |
115
80
 
116
- Use matrix strategy to run tests in parallel:
81
+ </Accordion>
82
+ <Accordion title="Use a stored key instead of OIDC">
117
83
 
118
- ```yaml .github/workflows/testdriver-parallel.yml
119
- name: TestDriver Tests (Parallel)
84
+ If OIDC isn't available, skip the action and pass the key directly:
120
85
 
121
- on: [push, pull_request]
86
+ 1. Copy an API key from the [API keys page](https://console.testdriver.ai/team/api-keys).
87
+ 2. In your repository, go to **Settings → Secrets and variables → Actions** and add a secret named `TD_API_KEY`.
88
+ 3. Pass it to the test step:
122
89
 
123
- jobs:
124
- test:
125
- runs-on: ubuntu-latest
126
- strategy:
127
- fail-fast: false
128
- matrix:
129
- shard: [1, 2, 3, 4]
130
-
131
- steps:
132
- - uses: actions/checkout@v4
133
- - uses: actions/setup-node@v4
134
- with:
135
- node-version: '20'
136
- cache: 'npm'
137
- - run: npm ci
138
- - name: Run tests (shard ${{ matrix.shard }}/4)
139
- env:
140
- TD_API_KEY: ${{ secrets.TD_API_KEY }}
141
- run: vitest --run --shard=${{ matrix.shard }}/4
90
+ ```yaml
91
+ - name: Run tests
92
+ run: npx vitest run
93
+ env:
94
+ TD_API_KEY: ${{ secrets.TD_API_KEY }}
142
95
  ```
143
96
 
144
- ### Multi-Platform Testing
97
+ </Accordion>
98
+ </AccordionGroup>
145
99
 
146
- ```yaml .github/workflows/testdriver-multiplatform.yml
147
- name: TestDriver Tests (Multi-Platform)
100
+ ## Other CI providers
148
101
 
149
- on: [push, pull_request]
102
+ Copy an API key from the [API keys page](https://console.testdriver.ai/team/api-keys). Store it as a secret named `TD_API_KEY` in your CI provider, then use the config for your provider.
150
103
 
151
- jobs:
152
- test:
153
- runs-on: ubuntu-latest
154
- strategy:
155
- fail-fast: false
156
- matrix:
157
- td-os: [linux, windows]
158
-
159
- steps:
160
- - uses: actions/checkout@v4
161
- - uses: actions/setup-node@v4
162
- with:
163
- node-version: '20'
164
- cache: 'npm'
165
- - run: npm ci
166
- - name: Run tests on ${{ matrix.td-os }}
167
- env:
168
- TD_API_KEY: ${{ secrets.TD_API_KEY }}
169
- TD_OS: ${{ matrix.td-os }}
170
- run: vitest --run
171
- ```
172
- </Tab>
104
+ <Warning>
105
+ Never commit your API key to the repository. Keep it in your CI provider's secret store.
106
+ </Warning>
173
107
 
108
+ <Tabs>
174
109
  <Tab title="GitLab CI">
175
- ### Adding Secrets
176
-
177
- 1. Go to your GitLab project
178
- 2. Navigate to **Settings** → **CI/CD** → **Variables**
179
- 3. Click **Add variable**
180
- 4. Key: `TD_API_KEY`, Value: your API key
181
- 5. Check **Mask variable** and click **Add variable**
182
110
 
183
- ### Basic Pipeline
184
-
185
- Create `.gitlab-ci.yml`:
111
+ Add `TD_API_KEY` under **Settings → CI/CD → Variables**, with **Mask variable** checked. GitLab exposes project variables to every job.
186
112
 
187
113
  ```yaml .gitlab-ci.yml
188
- stages:
189
- - test
190
-
191
114
  testdriver:
192
- stage: test
193
115
  image: node:20
194
- cache:
195
- paths:
196
- - node_modules/
197
116
  script:
198
117
  - npm ci
199
- - vitest --run
200
- variables:
201
- TD_API_KEY: $TD_API_KEY
202
- ```
203
-
204
- ### Parallel Execution
205
-
206
- ```yaml .gitlab-ci.yml
207
- stages:
208
- - test
209
-
210
- .testdriver-base:
211
- stage: test
212
- image: node:20
213
- cache:
214
- paths:
215
- - node_modules/
216
- before_script:
217
- - npm ci
218
- variables:
219
- TD_API_KEY: $TD_API_KEY
220
-
221
- testdriver-shard-1:
222
- extends: .testdriver-base
223
- script:
224
- - vitest --run --shard=1/4
225
-
226
- testdriver-shard-2:
227
- extends: .testdriver-base
228
- script:
229
- - vitest --run --shard=2/4
230
-
231
- testdriver-shard-3:
232
- extends: .testdriver-base
233
- script:
234
- - vitest --run --shard=3/4
235
-
236
- testdriver-shard-4:
237
- extends: .testdriver-base
238
- script:
239
- - vitest --run --shard=4/4
118
+ - npx vitest run
240
119
  ```
241
120
 
242
- ### Multi-Platform Testing
243
-
244
- ```yaml .gitlab-ci.yml
245
- stages:
246
- - test
247
-
248
- .testdriver-base:
249
- stage: test
250
- image: node:20
251
- cache:
252
- paths:
253
- - node_modules/
254
- before_script:
255
- - npm ci
256
- variables:
257
- TD_API_KEY: $TD_API_KEY
258
-
259
- testdriver-linux:
260
- extends: .testdriver-base
261
- variables:
262
- TD_OS: linux
263
- script:
264
- - vitest --run
265
-
266
- testdriver-windows:
267
- extends: .testdriver-base
268
- variables:
269
- TD_OS: windows
270
- script:
271
- - vitest --run
272
- ```
273
121
  </Tab>
274
-
275
122
  <Tab title="CircleCI">
276
- ### Adding Secrets
277
-
278
- 1. Go to your CircleCI project
279
- 2. Click **Project Settings** → **Environment Variables**
280
- 3. Click **Add Environment Variable**
281
- 4. Name: `TD_API_KEY`, Value: your API key
282
-
283
- ### Basic Config
284
-
285
- Create `.circleci/config.yml`:
286
-
287
- ```yaml .circleci/config.yml
288
- version: 2.1
289
-
290
- jobs:
291
- test:
292
- docker:
293
- - image: cimg/node:20.0
294
- steps:
295
- - checkout
296
- - restore_cache:
297
- keys:
298
- - npm-deps-{{ checksum "package-lock.json" }}
299
- - run: npm ci
300
- - save_cache:
301
- key: npm-deps-{{ checksum "package-lock.json" }}
302
- paths:
303
- - node_modules
304
- - run:
305
- name: Run TestDriver tests
306
- command: vitest --run
307
- environment:
308
- TD_API_KEY: ${TD_API_KEY}
309
-
310
- workflows:
311
- test:
312
- jobs:
313
- - test
314
- ```
315
123
 
316
- ### Parallel Execution
124
+ Add `TD_API_KEY` under **Project Settings → Environment Variables**. CircleCI exposes project variables to every job.
317
125
 
318
126
  ```yaml .circleci/config.yml
319
127
  version: 2.1
@@ -321,24 +129,11 @@ Add it as a `TD_API_KEY` secret in the settings of your CI provider.
321
129
  jobs:
322
130
  test:
323
131
  docker:
324
- - image: cimg/node:20.0
325
- parallelism: 4
132
+ - image: cimg/node:lts
326
133
  steps:
327
134
  - checkout
328
- - restore_cache:
329
- keys:
330
- - npm-deps-{{ checksum "package-lock.json" }}
331
135
  - run: npm ci
332
- - save_cache:
333
- key: npm-deps-{{ checksum "package-lock.json" }}
334
- paths:
335
- - node_modules
336
- - run:
337
- name: Run TestDriver tests
338
- command: |
339
- vitest --run --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
340
- environment:
341
- TD_API_KEY: ${TD_API_KEY}
136
+ - run: npx vitest run
342
137
 
343
138
  workflows:
344
139
  test:
@@ -346,92 +141,17 @@ Add it as a `TD_API_KEY` secret in the settings of your CI provider.
346
141
  - test
347
142
  ```
348
143
 
349
- ### Multi-Platform Testing
350
-
351
- ```yaml .circleci/config.yml
352
- version: 2.1
353
-
354
- jobs:
355
- test:
356
- docker:
357
- - image: cimg/node:20.0
358
- parameters:
359
- td-os:
360
- type: string
361
- steps:
362
- - checkout
363
- - run: npm ci
364
- - run:
365
- name: Run TestDriver tests on << parameters.td-os >>
366
- command: vitest --run
367
- environment:
368
- TD_API_KEY: ${TD_API_KEY}
369
- TD_OS: << parameters.td-os >>
370
-
371
- workflows:
372
- test:
373
- jobs:
374
- - test:
375
- td-os: linux
376
- - test:
377
- td-os: windows
378
- ```
379
144
  </Tab>
380
-
381
145
  <Tab title="Azure Pipelines">
382
- ### Adding Secrets
383
-
384
- 1. Go to your Azure DevOps project
385
- 2. Navigate to **Pipelines** → **Library** → **Variable groups**
386
- 3. Create a new variable group or edit existing
387
- 4. Add variable: `TD_API_KEY` with your API key
388
- 5. Click the lock icon to make it secret
389
-
390
- ### Basic Pipeline
391
-
392
- Create `azure-pipelines.yml`:
393
-
394
- ```yaml azure-pipelines.yml
395
- trigger:
396
- - main
397
-
398
- pool:
399
- vmImage: 'ubuntu-latest'
400
-
401
- steps:
402
- - task: NodeTool@0
403
- inputs:
404
- versionSpec: '20.x'
405
- displayName: 'Setup Node.js'
406
146
 
407
- - script: npm ci
408
- displayName: 'Install dependencies'
409
-
410
- - script: vitest --run
411
- displayName: 'Run TestDriver tests'
412
- env:
413
- TD_API_KEY: $(TD_API_KEY)
414
- ```
415
-
416
- ### Parallel Execution
147
+ Add `TD_API_KEY` as a pipeline variable and mark it secret. Azure doesn't expose secret variables to scripts automatically, so map it with `env`.
417
148
 
418
149
  ```yaml azure-pipelines.yml
419
150
  trigger:
420
151
  - main
421
152
 
422
153
  pool:
423
- vmImage: 'ubuntu-latest'
424
-
425
- strategy:
426
- matrix:
427
- shard1:
428
- SHARD: '1/4'
429
- shard2:
430
- SHARD: '2/4'
431
- shard3:
432
- SHARD: '3/4'
433
- shard4:
434
- SHARD: '4/4'
154
+ vmImage: ubuntu-latest
435
155
 
436
156
  steps:
437
157
  - task: NodeTool@0
@@ -439,248 +159,139 @@ Add it as a `TD_API_KEY` secret in the settings of your CI provider.
439
159
  versionSpec: '20.x'
440
160
 
441
161
  - script: npm ci
442
- displayName: 'Install dependencies'
162
+ displayName: Install dependencies
443
163
 
444
- - script: vitest --run --shard=$(SHARD)
445
- displayName: 'Run TestDriver tests'
164
+ - script: npx vitest run
165
+ displayName: Run TestDriver tests
446
166
  env:
447
167
  TD_API_KEY: $(TD_API_KEY)
448
168
  ```
449
169
 
450
- ### Multi-Platform Testing
451
-
452
- ```yaml azure-pipelines.yml
453
- trigger:
454
- - main
455
-
456
- pool:
457
- vmImage: 'ubuntu-latest'
458
-
459
- strategy:
460
- matrix:
461
- linux:
462
- TD_OS: 'linux'
463
- windows:
464
- TD_OS: 'windows'
465
-
466
- steps:
467
- - task: NodeTool@0
468
- inputs:
469
- versionSpec: '20.x'
470
-
471
- - script: npm ci
472
- displayName: 'Install dependencies'
473
-
474
- - script: vitest --run
475
- displayName: 'Run TestDriver tests on $(TD_OS)'
476
- env:
477
- TD_API_KEY: $(TD_API_KEY)
478
- TD_OS: $(TD_OS)
479
- ```
480
170
  </Tab>
481
-
482
171
  <Tab title="Jenkins">
483
- ### Adding Secrets
484
-
485
- 1. Go to **Manage Jenkins** → **Credentials**
486
- 2. Select the appropriate domain
487
- 3. Click **Add Credentials**
488
- 4. Kind: **Secret text**
489
- 5. ID: `td-api-key`, Secret: your API key
490
-
491
- ### Basic Pipeline
492
172
 
493
- Create `Jenkinsfile`:
173
+ Add a **Secret text** credential with the ID `td-api-key` under **Manage Jenkins → Credentials**.
494
174
 
495
175
  ```groovy Jenkinsfile
496
176
  pipeline {
497
- agent {
498
- docker {
499
- image 'node:20'
500
- }
501
- }
502
-
177
+ agent { docker { image 'node:20' } }
178
+
503
179
  environment {
504
180
  TD_API_KEY = credentials('td-api-key')
505
181
  }
506
-
182
+
507
183
  stages {
508
- stage('Install') {
509
- steps {
510
- sh 'npm ci'
511
- }
512
- }
513
-
514
184
  stage('Test') {
515
185
  steps {
516
- sh 'vitest --run'
186
+ sh 'npm ci'
187
+ sh 'npx vitest run'
517
188
  }
518
189
  }
519
190
  }
520
191
  }
521
192
  ```
522
193
 
523
- ### Parallel Execution
194
+ </Tab>
195
+ </Tabs>
524
196
 
525
- ```groovy Jenkinsfile
526
- pipeline {
527
- agent none
528
-
529
- environment {
530
- TD_API_KEY = credentials('td-api-key')
531
- }
532
-
533
- stages {
534
- stage('Test') {
535
- parallel {
536
- stage('Shard 1') {
537
- agent { docker { image 'node:20' } }
538
- steps {
539
- sh 'npm ci'
540
- sh 'vitest --run --shard=1/4'
541
- }
542
- }
543
- stage('Shard 2') {
544
- agent { docker { image 'node:20' } }
545
- steps {
546
- sh 'npm ci'
547
- sh 'vitest --run --shard=2/4'
548
- }
549
- }
550
- stage('Shard 3') {
551
- agent { docker { image 'node:20' } }
552
- steps {
553
- sh 'npm ci'
554
- sh 'vitest --run --shard=3/4'
555
- }
556
- }
557
- stage('Shard 4') {
558
- agent { docker { image 'node:20' } }
559
- steps {
560
- sh 'npm ci'
561
- sh 'vitest --run --shard=4/4'
562
- }
563
- }
564
- }
565
- }
566
- }
567
- }
568
- ```
197
+ ## Split tests across jobs
569
198
 
570
- ### Multi-Platform Testing
199
+ Vitest runs test files in parallel within one job. To spread a large suite across several jobs, use Vitest's `--shard` flag:
571
200
 
572
- ```groovy Jenkinsfile
573
- pipeline {
574
- agent none
575
-
576
- environment {
577
- TD_API_KEY = credentials('td-api-key')
578
- }
579
-
580
- stages {
581
- stage('Test') {
582
- parallel {
583
- stage('Linux') {
584
- agent { docker { image 'node:20' } }
585
- environment {
586
- TD_OS = 'linux'
587
- }
588
- steps {
589
- sh 'npm ci'
590
- sh 'vitest --run'
591
- }
592
- }
593
- stage('Windows') {
594
- agent { docker { image 'node:20' } }
595
- environment {
596
- TD_OS = 'windows'
597
- }
598
- steps {
599
- sh 'npm ci'
600
- sh 'vitest --run'
601
- }
602
- }
603
- }
604
- }
605
- }
606
- }
607
- ```
608
- </Tab>
609
- </Tabs>
201
+ ```yaml .github/workflows/testdriver.yml
202
+ jobs:
203
+ test:
204
+ runs-on: ubuntu-latest
205
+ permissions:
206
+ id-token: write
207
+ contents: read
208
+ strategy:
209
+ fail-fast: false
210
+ matrix:
211
+ shard: [1, 2, 3, 4]
610
212
 
611
- ## Reading Platform in Tests
612
-
613
- When using multi-platform testing, read the `TD_OS` environment variable in your test:
614
-
615
- ```javascript tests/cross-platform.test.mjs
616
- import { describe, expect, it } from "vitest";
617
- import { TestDriver } from "testdriverai/vitest/hooks";
618
-
619
- describe("Cross-platform tests", () => {
620
- it("should work on both Linux and Windows", async (context) => {
621
- const os = process.env.TD_OS || 'linux';
622
-
623
- const testdriver = TestDriver(context, {
624
- os: os // 'linux' or 'windows'
625
- });
626
-
627
- await testdriver.provision.chrome({
628
- url: 'https://example.com',
629
- });
630
-
631
- const result = await testdriver.assert("the page loaded successfully");
632
- expect(result).toBeTruthy();
633
- });
634
- });
213
+ steps:
214
+ - uses: actions/checkout@v4
215
+ - uses: actions/setup-node@v4
216
+ with:
217
+ node-version: '20'
218
+ cache: 'npm'
219
+ - run: npm ci
220
+ - uses: testdriverai/action@stable
221
+ - run: npx vitest run --shard=${{ matrix.shard }}/4
635
222
  ```
636
223
 
637
- ## Concurrency limits
224
+ Other providers work the same way. Run `npx vitest run --shard=<index>/<total>` in each parallel job.
225
+
226
+ ## Run on Linux and Windows
638
227
 
639
- Your plan allows a fixed number of sandboxes running at once. When a test asks for
640
- a sandbox and you're already at that limit, the request is **queued** rather than
641
- failed immediately: the SDK waits for a slot to free up, retrying every 10 seconds,
642
- then proceeds automatically once one opens. This is what lets a parallel CI matrix
643
- (many jobs starting at once) work on a plan with fewer slots than jobs — the extra
644
- jobs simply wait their turn instead of erroring.
228
+ Set `TD_OS` to choose the sandbox's operating system. TestDriver reads it when it creates the sandbox, so your tests don't need any changes:
645
229
 
646
- By default the SDK waits up to **60 seconds** for a slot before giving up with a
647
- concurrency-limit error. Control that ceiling with `TD_CONCURRENCY_MAX_WAIT`:
230
+ ```yaml
231
+ strategy:
232
+ fail-fast: false
233
+ matrix:
234
+ td-os: [linux, windows]
235
+
236
+ steps:
237
+ # ...checkout, setup-node, npm ci, testdriverai/action
238
+ - run: npx vitest run
239
+ env:
240
+ TD_OS: ${{ matrix.td-os }}
241
+ ```
242
+
243
+ Linux is the default.
244
+
245
+ <EnterpriseCta feature="Windows sandboxes" plural />
648
246
 
649
- | Value | Behavior |
650
- | ----- | -------- |
651
- | _unset_ | Wait up to **60 seconds** (the default). |
652
- | `TD_CONCURRENCY_MAX_WAIT=300` | Wait up to **300 seconds** (5 minutes) before giving up. |
653
- | `TD_CONCURRENCY_MAX_WAIT=0` | **Don't queue** — fail on the first denial. |
247
+ An `os` option passed to `TestDriver()` takes priority over `TD_OS`. To branch on the platform inside a test, read `process.env.TD_OS`:
654
248
 
655
- The value is **in seconds** (fractional values are allowed and rounded to the
656
- nearest millisecond). Any invalid or negative value falls back to the 60-second
657
- default. The wait applies per sandbox request, across both the initial allocation
658
- and the realtime slot-approval handshake.
249
+ ```javascript
250
+ const isWindows = process.env.TD_OS === 'windows';
251
+
252
+ await testdriver.exec({
253
+ language: isWindows ? 'pwsh' : 'sh',
254
+ code: isWindows ? 'Get-ChildItem $HOME' : 'ls ~',
255
+ });
256
+ ```
257
+
258
+ ## Concurrency limits
259
+
260
+ Your plan sets how many sandboxes can run at once. When a job asks for a sandbox at that limit, it waits for a free slot instead of failing.
261
+
262
+ - The SDK retries every 10 seconds.
263
+ - By default, it gives up after 60 seconds with a concurrency-limit error.
264
+ - Set `TD_CONCURRENCY_MAX_WAIT` (in seconds) to wait longer, or to `0` to fail on the first refusal.
659
265
 
660
266
  ```yaml
661
- # Example: a large parallel matrix that may queue for a while.
662
- # Give each job up to 5 minutes to acquire a slot before failing.
663
- - name: Run TestDriver tests
267
+ - run: npx vitest run
664
268
  env:
665
- TD_API_KEY: ${{ secrets.TD_API_KEY }}
666
- TD_CONCURRENCY_MAX_WAIT: "300"
667
- run: vitest run
269
+ TD_CONCURRENCY_MAX_WAIT: '300' # wait up to 5 minutes for a slot
668
270
  ```
669
271
 
670
- <Tip>
671
- Raise `TD_CONCURRENCY_MAX_WAIT` when you run more parallel jobs than your plan has
672
- slots and would rather they queue than fail. Set it to `0` when you'd prefer a job
673
- to **fail fast** on a busy account (e.g. a quick smoke test that shouldn't sit
674
- waiting). When jobs routinely give up waiting, that's the signal to
675
- [add more slots](https://console.testdriver.ai/checkout/pro).
676
- </Tip>
272
+ Raise the wait when you run more parallel jobs than your plan has slots. If jobs regularly give up waiting, run fewer at once with `--maxWorkers` or add slots to your plan.
273
+
274
+ ## View results
677
275
 
678
- ## Viewing Results
276
+ Each run is recorded in the [TestDriver console](https://console.testdriver.ai), with a video replay, logs, and pass or fail status. When a run finishes, the SDK prints a link to it:
679
277
 
680
- All test runs are automatically recorded and visible in your TestDriver dashboard at [console.testdriver.ai](https://console.testdriver.ai):
278
+ ```text
279
+ TESTDRIVER_RUN_URL=https://console.testdriver.ai/runs/abc123
280
+ ```
681
281
 
682
- - All test runs with pass/fail status
683
- - Video replays of each test
684
- - Error messages and screenshots on failure
685
- - Git commit and branch information
686
- - Duration trends over time
282
+ ## Next steps
283
+
284
+ <CardGroup cols={2}>
285
+ <Card title="Run tests" icon="play" href="/copilot/running-tests">
286
+ Run tests locally, in parallel, and watch them live.
287
+ </Card>
288
+ <Card title="Add TestDriver to GitHub" icon="github" href="/quickstart-github">
289
+ Get an AI code review on every pull request.
290
+ </Card>
291
+ <Card title="Hosted plans" icon="cloud" href="/hosted">
292
+ Compare plans, testing minutes, and concurrency.
293
+ </Card>
294
+ <Card title="Speed up reruns with caching" icon="brain" href="/caching">
295
+ Reuse element locations from passing runs.
296
+ </Card>
297
+ </CardGroup>