testdriverai 7.1.4 → 7.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/.github/workflows/acceptance.yaml +81 -0
  2. package/.github/workflows/publish.yaml +44 -0
  3. package/agent/index.js +18 -19
  4. package/agent/interface.js +4 -0
  5. package/agent/lib/commands.js +321 -121
  6. package/agent/lib/redraw.js +99 -39
  7. package/agent/lib/sandbox.js +98 -6
  8. package/agent/lib/sdk.js +25 -0
  9. package/agent/lib/system.js +2 -1
  10. package/agent/lib/validation.js +6 -6
  11. package/docs/docs.json +211 -101
  12. package/docs/snippets/tests/type-repeated-replay.mdx +1 -1
  13. package/docs/v7/_drafts/caching-selectors.mdx +24 -0
  14. package/docs/v7/api/act.mdx +1 -1
  15. package/docs/v7/api/assert.mdx +1 -1
  16. package/docs/v7/api/assertions.mdx +7 -7
  17. package/docs/v7/api/elements.mdx +78 -0
  18. package/docs/v7/api/find.mdx +38 -0
  19. package/docs/v7/api/focusApplication.mdx +2 -2
  20. package/docs/v7/api/hover.mdx +2 -2
  21. package/docs/v7/features/ai-native.mdx +57 -71
  22. package/docs/v7/features/application-logs.mdx +353 -0
  23. package/docs/v7/features/browser-logs.mdx +414 -0
  24. package/docs/v7/features/cache-management.mdx +402 -0
  25. package/docs/v7/features/continuous-testing.mdx +346 -0
  26. package/docs/v7/features/coverage.mdx +508 -0
  27. package/docs/v7/features/data-driven-testing.mdx +441 -0
  28. package/docs/v7/features/easy-to-write.mdx +2 -73
  29. package/docs/v7/features/enterprise.mdx +155 -39
  30. package/docs/v7/features/fast.mdx +63 -81
  31. package/docs/v7/features/managed-sandboxes.mdx +384 -0
  32. package/docs/v7/features/network-monitoring.mdx +568 -0
  33. package/docs/v7/features/observable.mdx +3 -22
  34. package/docs/v7/features/parallel-execution.mdx +381 -0
  35. package/docs/v7/features/powerful.mdx +1 -1
  36. package/docs/v7/features/reports.mdx +414 -0
  37. package/docs/v7/features/sandbox-customization.mdx +229 -0
  38. package/docs/v7/features/scalable.mdx +217 -2
  39. package/docs/v7/features/stable.mdx +106 -147
  40. package/docs/v7/features/system-performance.mdx +616 -0
  41. package/docs/v7/features/test-analytics.mdx +373 -0
  42. package/docs/v7/features/test-cases.mdx +393 -0
  43. package/docs/v7/features/test-replays.mdx +408 -0
  44. package/docs/v7/features/test-reports.mdx +308 -0
  45. package/docs/v7/getting-started/{running-and-debugging.mdx → debugging-tests.mdx} +12 -142
  46. package/docs/v7/getting-started/quickstart.mdx +22 -305
  47. package/docs/v7/getting-started/running-tests.mdx +173 -0
  48. package/docs/v7/overview/what-is-testdriver.mdx +2 -14
  49. package/docs/v7/presets/chrome-extension.mdx +147 -122
  50. package/interfaces/cli/commands/init.js +3 -3
  51. package/interfaces/cli/lib/base.js +3 -2
  52. package/interfaces/logger.js +0 -2
  53. package/interfaces/shared-test-state.mjs +0 -5
  54. package/interfaces/vitest-plugin.mjs +70 -50
  55. package/lib/core/Dashcam.js +60 -85
  56. package/lib/vitest/hooks.mjs +42 -50
  57. package/package.json +1 -1
  58. package/sdk-log-formatter.js +350 -175
  59. package/sdk.d.ts +36 -3
  60. package/sdk.js +431 -116
  61. package/setup/aws/cloudformation.yaml +2 -2
  62. package/setup/aws/self-hosted.yml +1 -1
  63. package/test/testdriver/chrome-extension.test.mjs +55 -72
  64. package/test/testdriver/element-not-found.test.mjs +2 -1
  65. package/test/testdriver/hover-image.test.mjs +1 -1
  66. package/test/testdriver/scroll-until-text.test.mjs +10 -6
  67. package/test/testdriver/setup/lifecycleHelpers.mjs +19 -24
  68. package/test/testdriver/setup/testHelpers.mjs +18 -23
  69. package/vitest.config.mjs +3 -3
  70. package/.github/workflows/linux-tests.yml +0 -28
  71. package/docs/v7/getting-started/generating-tests.mdx +0 -525
  72. package/test/testdriver/auto-cache-key-demo.test.mjs +0 -56
@@ -0,0 +1,384 @@
1
+ ---
2
+ title: "Managed Sandboxes"
3
+ description: "Isolated cloud environments automatically provisioned for every test"
4
+ icon: "box"
5
+ ---
6
+
7
+ TestDriver automatically provisions isolated cloud sandboxes for every test run, eliminating the need to manage browsers, VMs, or infrastructure.
8
+
9
+ ## Automatic Provisioning
10
+
11
+ Every test gets its own fresh sandbox environment:
12
+
13
+ ```javascript
14
+ import { test } from 'vitest';
15
+ import { chrome } from 'testdriverai/presets';
16
+
17
+ test('automatic sandbox', async (context) => {
18
+ // TestDriver automatically provisions a sandbox with Chrome
19
+ const { testdriver } = await chrome(context, {
20
+ url: 'https://example.com'
21
+ });
22
+
23
+ // Test runs in an isolated cloud environment
24
+ await testdriver.find('button').click();
25
+ await testdriver.assert('page loaded');
26
+
27
+ // Sandbox automatically cleaned up after test
28
+ });
29
+ ```
30
+
31
+ <Check>
32
+ No setup required - sandboxes are created on-demand and destroyed after each test.
33
+ </Check>
34
+
35
+ ## What's Included
36
+
37
+ Each sandbox comes pre-configured with everything you need:
38
+
39
+ <CardGroup cols={2}>
40
+ <Card title="Operating Systems" icon="desktop">
41
+ - Ubuntu Linux (default)
42
+ - Windows 10/11
43
+ - macOS (coming soon)
44
+ </Card>
45
+
46
+ <Card title="Browsers" icon="globe">
47
+ - Google Chrome
48
+ - Chromium
49
+ - Firefox (beta)
50
+ - Edge (beta)
51
+ </Card>
52
+
53
+ <Card title="Desktop Apps" icon="window">
54
+ - Any Linux application
55
+ - Windows applications
56
+ - Electron apps
57
+ </Card>
58
+
59
+ <Card title="Development Tools" icon="code">
60
+ - VS Code
61
+ - Node.js
62
+ - Python
63
+ - Common dev dependencies
64
+ </Card>
65
+ </CardGroup>
66
+
67
+ ## Sandbox Configuration
68
+
69
+ Customize sandbox settings per test:
70
+
71
+ <Tabs>
72
+ <Tab title="Operating System">
73
+ ```javascript
74
+ import { chrome } from 'testdriverai/presets';
75
+
76
+ test('choose OS', async (context) => {
77
+ const { testdriver } = await chrome(context, {
78
+ url: 'https://example.com',
79
+ os: 'windows' // 'linux', 'windows', or 'mac'
80
+ });
81
+
82
+ // Test runs on Windows sandbox
83
+ });
84
+ ```
85
+ </Tab>
86
+
87
+ <Tab title="Screen Size">
88
+ ```javascript
89
+ import { chrome } from 'testdriverai/presets';
90
+
91
+ test('custom resolution', async (context) => {
92
+ const { testdriver } = await chrome(context, {
93
+ url: 'https://example.com',
94
+ width: 1920,
95
+ height: 1080,
96
+ maximized: true
97
+ });
98
+
99
+ // Browser opens at 1920x1080 resolution
100
+ });
101
+ ```
102
+ </Tab>
103
+
104
+ <Tab title="Browser Options">
105
+ ```javascript
106
+ import { chrome } from 'testdriverai/presets';
107
+
108
+ test('browser settings', async (context) => {
109
+ const { testdriver } = await chrome(context, {
110
+ url: 'https://example.com',
111
+ guest: true, // Incognito/private mode
112
+ headless: false, // Show browser UI
113
+ locale: 'en-US', // Browser language
114
+ timezone: 'America/New_York'
115
+ });
116
+ });
117
+ ```
118
+ </Tab>
119
+
120
+ <Tab title="Timeout Settings">
121
+ ```javascript
122
+ import { chrome } from 'testdriverai/presets';
123
+
124
+ test('longer timeout', async (context) => {
125
+ const { testdriver } = await chrome(context, {
126
+ url: 'https://example.com',
127
+ timeout: 180000 // 3 minutes (default: 120s)
128
+ });
129
+
130
+ // Sandbox stays alive for up to 3 minutes
131
+ });
132
+ ```
133
+ </Tab>
134
+ </Tabs>
135
+
136
+ ## Persistent Sandboxes
137
+
138
+ Keep a sandbox alive across multiple tests for faster execution:
139
+
140
+ ```javascript
141
+ import { test, beforeAll, afterAll } from 'vitest';
142
+ import { chrome } from 'testdriverai/presets';
143
+
144
+ let testdriver;
145
+
146
+ beforeAll(async () => {
147
+ // Create persistent sandbox
148
+ const result = await chrome({
149
+ url: 'https://example.com',
150
+ timeout: 600000 // 10 minutes
151
+ });
152
+ testdriver = result.testdriver;
153
+ });
154
+
155
+ test('first test', async () => {
156
+ // Reuses existing sandbox
157
+ await testdriver.find('login button').click();
158
+ });
159
+
160
+ test('second test', async () => {
161
+ // Same sandbox, faster execution
162
+ await testdriver.find('dashboard').click();
163
+ });
164
+
165
+ afterAll(async () => {
166
+ // Clean up sandbox
167
+ await testdriver.close();
168
+ });
169
+ ```
170
+
171
+ <Tip>
172
+ Persistent sandboxes are faster for test suites but share state between tests. Use fresh sandboxes when test isolation is critical.
173
+ </Tip>
174
+
175
+ ## Sandbox Lifecycle
176
+
177
+ Understanding how sandboxes are managed:
178
+
179
+ <Steps>
180
+ <Step title="Provisioning">
181
+ When you call a preset (e.g., `chrome()`), TestDriver:
182
+ - Allocates a cloud VM
183
+ - Installs the OS and browser
184
+ - Configures network and display
185
+ - Returns ready-to-use testdriver instance
186
+
187
+ **Time:** 5-15 seconds
188
+ </Step>
189
+
190
+ <Step title="Active Testing">
191
+ Your test runs in the isolated environment:
192
+ - Full browser/desktop control
193
+ - Network access
194
+ - File system access
195
+ - Video recording (Dashcam)
196
+
197
+ **Time:** Duration of your test
198
+ </Step>
199
+
200
+ <Step title="Cleanup">
201
+ After the test completes:
202
+ - Dashcam replay saved
203
+ - Screenshots captured
204
+ - Sandbox terminated
205
+ - Resources released
206
+
207
+ **Time:** Automatic
208
+ </Step>
209
+ </Steps>
210
+
211
+ ## Reconnect to Sandboxes
212
+
213
+ Debug failed tests by reconnecting to existing sandboxes:
214
+
215
+ ```javascript
216
+ import { test } from 'vitest';
217
+ import { TestDriver } from 'testdriverai';
218
+
219
+ test('reconnectable sandbox', async () => {
220
+ const testdriver = new TestDriver({
221
+ apiKey: process.env.TD_API_KEY
222
+ });
223
+
224
+ const instance = await testdriver.connect();
225
+ console.log('Sandbox ID:', instance.instanceId);
226
+
227
+ // Save this ID for debugging
228
+ // export TESTDRIVER_SANDBOX_ID=i-0abc123def
229
+
230
+ await testdriver.find('button').click();
231
+ });
232
+ ```
233
+
234
+ Then reconnect later:
235
+
236
+ ```bash
237
+ # Set sandbox ID
238
+ export TESTDRIVER_SANDBOX_ID=i-0abc123def
239
+
240
+ # Run debugging session
241
+ node debug-script.js
242
+ ```
243
+
244
+ <Card title="Debugging Guide" icon="bug" href="/v7/guides/debugging">
245
+ Complete guide to debugging with sandbox reconnection
246
+ </Card>
247
+
248
+ ## Network Isolation
249
+
250
+ Each sandbox is network-isolated by default:
251
+
252
+ ```javascript
253
+ import { chrome } from 'testdriverai/presets';
254
+
255
+ test('isolated network', async (context) => {
256
+ const { testdriver } = await chrome(context, {
257
+ url: 'https://example.com'
258
+ });
259
+
260
+ // Sandbox has its own IP address
261
+ // No shared cookies or cache with other tests
262
+ // Clean slate for every test run
263
+ });
264
+ ```
265
+
266
+ <Check>
267
+ Network isolation prevents test interference and ensures consistent results across runs.
268
+ </Check>
269
+
270
+ ## Resource Limits
271
+
272
+ Understanding sandbox constraints:
273
+
274
+ <AccordionGroup>
275
+ <Accordion title="CPU & Memory">
276
+ - **CPU:** 2 vCPUs per sandbox
277
+ - **Memory:** 4GB RAM
278
+ - **Disk:** 20GB storage
279
+
280
+ Sufficient for most testing scenarios including heavy web apps and desktop applications.
281
+ </Accordion>
282
+
283
+ <Accordion title="Network Bandwidth">
284
+ - **Speed:** 100 Mbps connection
285
+ - **Timeout:** 30s per network request
286
+ - **Proxies:** Supported via environment variables
287
+
288
+ Fast enough for realistic load times and asset downloads.
289
+ </Accordion>
290
+
291
+ <Accordion title="Time Limits">
292
+ - **Default:** 2 minutes per test
293
+ - **Maximum:** 30 minutes (configurable)
294
+ - **Idle timeout:** 5 minutes of inactivity
295
+
296
+ Customize with the `timeout` option in preset configuration.
297
+ </Accordion>
298
+
299
+ <Accordion title="Concurrent Sandboxes">
300
+ - **Free tier:** 2 concurrent sandboxes
301
+ - **Pro tier:** 10 concurrent sandboxes
302
+ - **Enterprise:** Unlimited concurrent sandboxes
303
+
304
+ Run tests in parallel for faster CI/CD pipelines.
305
+ </Accordion>
306
+ </AccordionGroup>
307
+
308
+ ## Cost & Billing
309
+
310
+ Sandboxes are billed based on usage:
311
+
312
+ <Card title="Pricing Model" icon="dollar-sign">
313
+ **Sandbox time:** Charged per minute of VM runtime
314
+
315
+ - **Provisioning:** ~10 seconds (included)
316
+ - **Active testing:** Pay per minute
317
+ - **Cleanup:** Automatic (included)
318
+
319
+ **Example costs:**
320
+ - 1-minute test: $0.02
321
+ - 5-minute test: $0.10
322
+ - 100 tests @ 2min each: $4.00
323
+
324
+ **Optimization tips:**
325
+ - Use persistent sandboxes for test suites
326
+ - Enable caching to reduce test duration
327
+ - Run tests in parallel to maximize efficiency
328
+ </Card>
329
+
330
+ ## Self-Hosted Sandboxes
331
+
332
+ For enterprise customers who need on-premise infrastructure:
333
+
334
+ ```yaml docker-compose.yml
335
+ version: '3'
336
+ services:
337
+ testdriver-runner:
338
+ image: testdriver/runner:latest
339
+ environment:
340
+ - TD_API_KEY=${TD_API_KEY}
341
+ - TD_RUNNER_MODE=self-hosted
342
+ volumes:
343
+ - /var/run/docker.sock:/var/run/docker.sock
344
+ ```
345
+
346
+ <Card title="Self-Hosting Guide" icon="server" href="/v7/guides/self-hosting">
347
+ Complete guide to running TestDriver on your own infrastructure
348
+ </Card>
349
+
350
+ ## Learn More
351
+
352
+ <CardGroup cols={2}>
353
+ <Card
354
+ title="Debugging Guide"
355
+ icon="bug"
356
+ href="/v7/guides/debugging"
357
+ >
358
+ Debug tests with sandbox reconnection
359
+ </Card>
360
+
361
+ <Card
362
+ title="Self-Hosting"
363
+ icon="server"
364
+ href="/v7/guides/self-hosting"
365
+ >
366
+ Run sandboxes on your infrastructure
367
+ </Card>
368
+
369
+ <Card
370
+ title="Performance Optimization"
371
+ icon="gauge-high"
372
+ href="/v7/guides/performance"
373
+ >
374
+ Optimize sandbox usage and costs
375
+ </Card>
376
+
377
+ <Card
378
+ title="Enterprise Features"
379
+ icon="building"
380
+ href="/v7/features/enterprise"
381
+ >
382
+ Enterprise sandbox capabilities
383
+ </Card>
384
+ </CardGroup>