testdriverai 7.2.3 → 7.2.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (142) hide show
  1. package/.github/workflows/publish.yaml +15 -7
  2. package/.github/workflows/testdriver.yml +163 -0
  3. package/.testdriver/last-sandbox +7 -0
  4. package/agent/events.js +1 -0
  5. package/agent/index.js +99 -163
  6. package/agent/lib/sandbox.js +11 -1
  7. package/agents.md +393 -0
  8. package/bin/testdriverai.js +8 -0
  9. package/debug/01-table-initial.png +0 -0
  10. package/debug/02-after-ai-explore.png +0 -0
  11. package/debug/02-after-scroll.png +0 -0
  12. package/debugger/index.html +37 -0
  13. package/docs/docs.json +93 -125
  14. package/docs/v7/_drafts/architecture.mdx +1 -26
  15. package/docs/v7/_drafts/caching.mdx +2 -2
  16. package/docs/v7/{getting-started → _drafts}/installation.mdx +0 -66
  17. package/docs/v7/{features/coverage.mdx → _drafts/powerful.mdx} +1 -90
  18. package/docs/v7/_drafts/quick-start-test-recording.mdx +0 -1
  19. package/docs/v7/{features → _drafts}/scalable.mdx +126 -4
  20. package/docs/v7/_drafts/screenshot.mdx +155 -0
  21. package/docs/v7/_drafts/test-recording.mdx +0 -6
  22. package/docs/v7/_drafts/writing-tests.mdx +25 -0
  23. package/docs/v7/{api/act.mdx → ai.mdx} +28 -27
  24. package/docs/v7/{api/assert.mdx → assert.mdx} +3 -3
  25. package/docs/v7/aws-setup.mdx +338 -0
  26. package/docs/v7/caching.mdx +128 -0
  27. package/docs/v7/ci-cd.mdx +605 -0
  28. package/docs/v7/{api/click.mdx → click.mdx} +4 -4
  29. package/docs/v7/cloud.mdx +120 -0
  30. package/docs/v7/customizing-devices.mdx +129 -0
  31. package/docs/v7/{api/doubleClick.mdx → double-click.mdx} +5 -5
  32. package/docs/v7/enterprise.mdx +135 -0
  33. package/docs/v7/examples.mdx +5 -0
  34. package/docs/v7/{api/exec.mdx → exec.mdx} +3 -3
  35. package/docs/v7/{api/find.mdx → find.mdx} +17 -21
  36. package/docs/v7/{api/focusApplication.mdx → focus-application.mdx} +3 -3
  37. package/docs/v7/generating-tests.mdx +32 -0
  38. package/docs/v7/{api/hover.mdx → hover.mdx} +3 -3
  39. package/docs/v7/locating-elements.mdx +71 -0
  40. package/docs/v7/making-assertions.mdx +32 -0
  41. package/docs/v7/{api/mouseDown.mdx → mouse-down.mdx} +7 -7
  42. package/docs/v7/{api/mouseUp.mdx → mouse-up.mdx} +8 -8
  43. package/docs/v7/performing-actions.mdx +51 -0
  44. package/docs/v7/{api/pressKeys.mdx → press-keys.mdx} +3 -3
  45. package/docs/v7/quickstart.mdx +162 -0
  46. package/docs/v7/reusable-code.mdx +240 -0
  47. package/docs/v7/{api/rightClick.mdx → right-click.mdx} +5 -5
  48. package/docs/v7/running-tests.mdx +181 -0
  49. package/docs/v7/{api/scroll.mdx → scroll.mdx} +3 -3
  50. package/docs/v7/secrets.mdx +115 -0
  51. package/docs/v7/self-hosted.mdx +66 -0
  52. package/docs/v7/{api/type.mdx → type.mdx} +3 -3
  53. package/docs/v7/variables.mdx +111 -0
  54. package/docs/v7/waiting-for-elements.mdx +66 -0
  55. package/docs/v7/what-is-testdriver.mdx +54 -0
  56. package/interfaces/cli/commands/init.js +33 -19
  57. package/interfaces/cli/lib/base.js +24 -0
  58. package/interfaces/cli.js +8 -1
  59. package/interfaces/logger.js +8 -3
  60. package/interfaces/vitest-plugin.mjs +16 -71
  61. package/lib/sentry.js +343 -0
  62. package/lib/vitest/hooks.mjs +81 -81
  63. package/package.json +4 -3
  64. package/sdk-log-formatter.js +41 -0
  65. package/sdk.d.ts +22 -9
  66. package/sdk.js +344 -100
  67. package/test/manual/reconnect-provision.test.mjs +49 -0
  68. package/test/manual/reconnect-signin.test.mjs +41 -0
  69. package/test/testdriver/act.test.mjs +30 -0
  70. package/test/testdriver/ai.test.mjs +30 -0
  71. package/test/testdriver/assert.test.mjs +1 -1
  72. package/test/testdriver/hover-text.test.mjs +1 -1
  73. package/test/testdriver/setup/testHelpers.mjs +8 -119
  74. package/test/testdriver/windows-installer.test.mjs +61 -0
  75. package/tests/example.test.js +33 -0
  76. package/tests/login.js +28 -0
  77. package/tests/table-sort-enrollments.test.mjs +72 -0
  78. package/tests/table-sort-experiment.test.mjs +42 -0
  79. package/tests/table-sort-setup.test.mjs +59 -0
  80. package/vitest.config.mjs +3 -1
  81. package/agent/lib/cache.js +0 -142
  82. package/docs/v7/api/assertions.mdx +0 -403
  83. package/docs/v7/features/ai-native.mdx +0 -413
  84. package/docs/v7/features/application-logs.mdx +0 -353
  85. package/docs/v7/features/browser-logs.mdx +0 -414
  86. package/docs/v7/features/cache-management.mdx +0 -402
  87. package/docs/v7/features/continuous-testing.mdx +0 -346
  88. package/docs/v7/features/data-driven-testing.mdx +0 -441
  89. package/docs/v7/features/easy-to-write.mdx +0 -280
  90. package/docs/v7/features/enterprise.mdx +0 -656
  91. package/docs/v7/features/fast.mdx +0 -406
  92. package/docs/v7/features/managed-sandboxes.mdx +0 -384
  93. package/docs/v7/features/network-monitoring.mdx +0 -568
  94. package/docs/v7/features/parallel-execution.mdx +0 -381
  95. package/docs/v7/features/powerful.mdx +0 -531
  96. package/docs/v7/features/sandbox-customization.mdx +0 -229
  97. package/docs/v7/features/stable.mdx +0 -473
  98. package/docs/v7/features/system-performance.mdx +0 -616
  99. package/docs/v7/features/test-analytics.mdx +0 -373
  100. package/docs/v7/features/test-cases.mdx +0 -393
  101. package/docs/v7/features/test-replays.mdx +0 -408
  102. package/docs/v7/features/test-reports.mdx +0 -308
  103. package/docs/v7/getting-started/debugging-tests.mdx +0 -382
  104. package/docs/v7/getting-started/quickstart.mdx +0 -90
  105. package/docs/v7/getting-started/running-tests.mdx +0 -173
  106. package/docs/v7/getting-started/setting-up-in-ci.mdx +0 -612
  107. package/docs/v7/getting-started/writing-tests.mdx +0 -534
  108. package/docs/v7/overview/what-is-testdriver.mdx +0 -386
  109. package/docs/v7/presets/chrome-extension.mdx +0 -248
  110. package/docs/v7/presets/chrome.mdx +0 -300
  111. package/docs/v7/presets/electron.mdx +0 -460
  112. package/docs/v7/presets/vscode.mdx +0 -417
  113. package/docs/v7/presets/webapp.mdx +0 -393
  114. /package/docs/v7/{commands → _drafts/commands}/assert.mdx +0 -0
  115. /package/docs/v7/{commands → _drafts/commands}/exec.mdx +0 -0
  116. /package/docs/v7/{commands → _drafts/commands}/focus-application.mdx +0 -0
  117. /package/docs/v7/{commands → _drafts/commands}/hover-image.mdx +0 -0
  118. /package/docs/v7/{commands → _drafts/commands}/hover-text.mdx +0 -0
  119. /package/docs/v7/{commands → _drafts/commands}/if.mdx +0 -0
  120. /package/docs/v7/{commands → _drafts/commands}/match-image.mdx +0 -0
  121. /package/docs/v7/{commands → _drafts/commands}/press-keys.mdx +0 -0
  122. /package/docs/v7/{commands → _drafts/commands}/remember.mdx +0 -0
  123. /package/docs/v7/{commands → _drafts/commands}/run.mdx +0 -0
  124. /package/docs/v7/{commands → _drafts/commands}/scroll-until-image.mdx +0 -0
  125. /package/docs/v7/{commands → _drafts/commands}/scroll-until-text.mdx +0 -0
  126. /package/docs/v7/{commands → _drafts/commands}/scroll.mdx +0 -0
  127. /package/docs/v7/{commands → _drafts/commands}/type.mdx +0 -0
  128. /package/docs/v7/{commands → _drafts/commands}/wait-for-image.mdx +0 -0
  129. /package/docs/v7/{commands → _drafts/commands}/wait-for-text.mdx +0 -0
  130. /package/docs/v7/{commands → _drafts/commands}/wait.mdx +0 -0
  131. /package/docs/v7/{getting-started → _drafts}/configuration.mdx +0 -0
  132. /package/docs/v7/{features → _drafts}/observable.mdx +0 -0
  133. /package/docs/v7/{platforms → _drafts/platforms}/linux.mdx +0 -0
  134. /package/docs/v7/{platforms → _drafts/platforms}/macos.mdx +0 -0
  135. /package/docs/v7/{platforms → _drafts/platforms}/windows.mdx +0 -0
  136. /package/docs/v7/{playwright.mdx → _drafts/playwright.mdx} +0 -0
  137. /package/docs/v7/{overview → _drafts}/readme.mdx +0 -0
  138. /package/docs/v7/{features → _drafts}/reports.mdx +0 -0
  139. /package/docs/v7/{api/client.mdx → client.mdx} +0 -0
  140. /package/docs/v7/{api/dashcam.mdx → dashcam.mdx} +0 -0
  141. /package/docs/v7/{api/elements.mdx → elements.mdx} +0 -0
  142. /package/docs/v7/{api/sandbox.mdx → sandbox.mdx} +0 -0
@@ -1,346 +0,0 @@
1
- ---
2
- title: "Continuous Testing"
3
- description: "Seamless CI/CD integration for automated testing at scale"
4
- icon: "arrows-spin"
5
- ---
6
-
7
- TestDriver integrates seamlessly with all major CI/CD platforms, enabling automated testing from small projects to enterprise test suites with thousands of tests.
8
-
9
- ## Works with Vitest
10
-
11
- Full integration with Vitest's powerful features:
12
-
13
- ```javascript vitest.config.mjs
14
- import { defineConfig } from 'vitest/config';
15
-
16
- export default defineConfig({
17
- test: {
18
- testTimeout: 120000,
19
- hookTimeout: 120000,
20
- maxConcurrency: 10, // Run 10 tests in parallel
21
- minThreads: 2,
22
- maxThreads: 10,
23
- },
24
- });
25
- ```
26
-
27
- <CardGroup cols={2}>
28
- <Card title="Parallel Execution" icon="layer-group">
29
- Run multiple tests simultaneously for maximum throughput:
30
-
31
- ```bash
32
- # Run all tests in parallel
33
- npx vitest run
34
-
35
- # Control concurrency
36
- npx vitest run --max-concurrency=5
37
- ```
38
- </Card>
39
-
40
- <Card title="Watch Mode" icon="eye">
41
- Instant feedback during development:
42
-
43
- ```bash
44
- # Run in watch mode
45
- npx vitest
46
-
47
- # Only run changed tests
48
- npx vitest --changed
49
- ```
50
- </Card>
51
- </CardGroup>
52
-
53
- [See complete Vitest integration guide](/v7/guides/vitest)
54
-
55
- ## CI/CD Compatible
56
-
57
- TestDriver integrates seamlessly with all major CI/CD platforms:
58
-
59
- <Tabs>
60
- <Tab title="GitHub Actions">
61
- ```yaml .github/workflows/test.yml
62
- name: E2E Tests
63
-
64
- on: [push, pull_request]
65
-
66
- jobs:
67
- test:
68
- runs-on: ubuntu-latest
69
- steps:
70
- - uses: actions/checkout@v3
71
- - uses: actions/setup-node@v3
72
- - run: npm install
73
- - run: npx vitest run
74
- env:
75
- TD_API_KEY: ${{ secrets.TD_API_KEY }}
76
- ```
77
-
78
- [GitHub Actions guide](/v7/guides/ci-cd/github-actions)
79
- </Tab>
80
-
81
- <Tab title="GitLab CI">
82
- ```yaml .gitlab-ci.yml
83
- test:
84
- image: node:20
85
- script:
86
- - npm install
87
- - npx vitest run
88
- variables:
89
- TD_API_KEY: $TD_API_KEY
90
- ```
91
-
92
- [GitLab CI guide](/v7/guides/ci-cd/gitlab)
93
- </Tab>
94
-
95
- <Tab title="CircleCI">
96
- ```yaml .circleci/config.yml
97
- version: 2.1
98
- jobs:
99
- test:
100
- docker:
101
- - image: cimg/node:20.0
102
- steps:
103
- - checkout
104
- - run: npm install
105
- - run: npx vitest run
106
- ```
107
-
108
- [CircleCI guide](/v7/guides/ci-cd/circleci)
109
- </Tab>
110
-
111
- <Tab title="Jenkins">
112
- ```groovy Jenkinsfile
113
- pipeline {
114
- agent any
115
- stages {
116
- stage('Test') {
117
- steps {
118
- sh 'npm install'
119
- sh 'npx vitest run'
120
- }
121
- }
122
- }
123
- }
124
- ```
125
-
126
- [Jenkins guide](/v7/guides/ci-cd/jenkins)
127
- </Tab>
128
- </Tabs>
129
-
130
- ## JUnit Reports
131
-
132
- Generate JUnit XML reports for test result aggregation:
133
-
134
- ```bash
135
- # Generate JUnit report
136
- npx vitest run --reporter=junit --outputFile=test-results.xml
137
- ```
138
-
139
- ```xml test-results.xml
140
- <?xml version="1.0" encoding="UTF-8"?>
141
- <testsuites name="vitest tests" tests="12" failures="0" errors="0" time="125.3">
142
- <testsuite name="login.test.js" tests="3" failures="0" time="31.2">
143
- <testcase name="user can login" time="12.4" />
144
- <testcase name="invalid credentials show error" time="9.8" />
145
- <testcase name="password reset works" time="9.0" />
146
- </testsuite>
147
- <!-- More test suites -->
148
- </testsuites>
149
- ```
150
-
151
- <Check>
152
- JUnit reports integrate with Jenkins, Azure DevOps, TeamCity, and other CI platforms for test result visualization.
153
- </Check>
154
-
155
- ## Scale to Thousands of Tests
156
-
157
- TestDriver handles large test suites efficiently:
158
-
159
- <Card title="Enterprise Test Suite Example" icon="building">
160
- ```
161
- Project: E-commerce Platform
162
- Total tests: 3,847
163
- Test files: 412
164
- Parallel runners: 20
165
- Total duration: 18 minutes
166
-
167
- Breakdown:
168
- - Unit tests: 2,134 (3 min)
169
- - Integration: 891 (7 min)
170
- - E2E (TestDriver): 822 (18 min)
171
-
172
- Cost per run: $4.32
173
- Runs per day: 50
174
- Monthly cost: $6,480
175
- ```
176
-
177
- With caching enabled:
178
- ```
179
- E2E duration: 8 minutes (2.25x faster)
180
- Cost per run: $1.89 (56% reduction)
181
- Monthly cost: $2,835 (saves $3,645/month)
182
- ```
183
- </Card>
184
-
185
- ## Test Sharding
186
-
187
- Distribute tests across multiple machines:
188
-
189
- <Tabs>
190
- <Tab title="GitHub Actions Matrix">
191
- ```yaml .github/workflows/test.yml
192
- strategy:
193
- matrix:
194
- shard: [1, 2, 3, 4, 5]
195
- steps:
196
- - run: npx vitest run --shard=${{ matrix.shard }}/5
197
- ```
198
-
199
- Runs 5 parallel jobs, each handling 1/5 of tests.
200
- </Tab>
201
-
202
- <Tab title="GitLab CI Parallel">
203
- ```yaml .gitlab-ci.yml
204
- test:
205
- parallel: 5
206
- script:
207
- - npx vitest run --shard=$CI_NODE_INDEX/$CI_NODE_TOTAL
208
- ```
209
- </Tab>
210
-
211
- <Tab title="Manual Sharding">
212
- ```bash
213
- # Machine 1
214
- npx vitest run --shard=1/4
215
-
216
- # Machine 2
217
- npx vitest run --shard=2/4
218
-
219
- # Machine 3
220
- npx vitest run --shard=3/4
221
-
222
- # Machine 4
223
- npx vitest run --shard=4/4
224
- ```
225
- </Tab>
226
- </Tabs>
227
-
228
- ## Team Collaboration
229
-
230
- Built for teams with multiple developers:
231
-
232
- <AccordionGroup>
233
- <Accordion title="Shared Test Cache">
234
- Tests automatically benefit from each other's cache entries:
235
-
236
- ```javascript
237
- // Developer A runs test
238
- await testdriver.find('submit button').click();
239
- // Creates cache entry
240
-
241
- // Developer B runs test later
242
- await testdriver.find('submit button').click();
243
- // Uses Developer A's cache ⚡
244
- ```
245
-
246
- Per-team cache sharing accelerates everyone's tests.
247
- </Accordion>
248
-
249
- <Accordion title="Dashcam Replays">
250
- Share test replays with your team:
251
-
252
- ```javascript
253
- test('shareable replay', async (context) => {
254
- const { testdriver, dashcam } = await chrome(context, { url });
255
-
256
- await testdriver.find('button').click();
257
-
258
- // Share replay URL with team
259
- console.log('Replay:', dashcam.url);
260
- // https://console.testdriver.ai/dashcam/abc123
261
- });
262
- ```
263
-
264
- View replays at [app.testdriver.ai](https://app.testdriver.ai)
265
- </Accordion>
266
-
267
- <Accordion title="Sandbox Management">
268
- Long-running sandboxes for team debugging:
269
-
270
- ```bash
271
- # Spawn a sandbox for the team
272
- testdriver sandbox spawn --timeout 7200000
273
-
274
- # Team members can connect
275
- export TESTDRIVER_SANDBOX_ID=i-0abc123def
276
- npx vitest run
277
- ```
278
-
279
- [Learn about sandbox management](/agents.md#sandbox-management)
280
- </Accordion>
281
- </AccordionGroup>
282
-
283
- ## Cost Optimization
284
-
285
- TestDriver scales cost-effectively:
286
-
287
- <Card title="Pricing Model" icon="dollar-sign">
288
- - **API calls:** Pay per AI vision analysis
289
- - **Sandbox time:** Pay per minute of VM usage
290
- - **Caching:** Reduces both costs dramatically
291
- - **Shared cache:** Team benefits from collective cache
292
-
293
- **Example costs:**
294
- - Single test run: $0.04 - $0.12
295
- - 100-test suite: $4 - $12 (first run)
296
- - Cached suite: $0.50 - $2 (90%+ savings)
297
- </Card>
298
-
299
- ## Enterprise Scale
300
-
301
- TestDriver supports the largest testing operations:
302
-
303
- - **Unlimited tests** in enterprise plans
304
- - **Unlimited sandbox hours** for large suites
305
- - **Unlimited team members** for collaboration
306
- - **Priority infrastructure** for faster execution
307
- - **Dedicated support** for optimization help
308
- - **Custom SLA** for guaranteed uptime
309
-
310
- [Contact sales](https://testdriver.ai/enterprise) for enterprise pricing.
311
-
312
- ## Learn More
313
-
314
- <CardGroup cols={2}>
315
- <Card
316
- title="Vitest Integration"
317
- icon="flask-vial"
318
- href="/v7/guides/vitest"
319
- >
320
- Complete Vitest guide
321
- </Card>
322
-
323
- <Card
324
- title="CI/CD Setup"
325
- icon="arrows-spin"
326
- href="/v7/guides/ci-cd/overview"
327
- >
328
- CI/CD integration guides
329
- </Card>
330
-
331
- <Card
332
- title="Performance Optimization"
333
- icon="gauge-high"
334
- href="/v7/guides/performance"
335
- >
336
- Optimize test performance
337
- </Card>
338
-
339
- <Card
340
- title="Self-Hosting"
341
- icon="server"
342
- href="/v7/guides/self-hosting"
343
- >
344
- Run on your infrastructure
345
- </Card>
346
- </CardGroup>