testdriverai 7.1.3 → 7.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (78) hide show
  1. package/.github/workflows/acceptance.yaml +81 -0
  2. package/.github/workflows/publish.yaml +44 -0
  3. package/.github/workflows/test-init.yml +145 -0
  4. package/agent/index.js +18 -19
  5. package/agent/lib/commander.js +2 -2
  6. package/agent/lib/commands.js +324 -124
  7. package/agent/lib/redraw.js +99 -39
  8. package/agent/lib/sandbox.js +98 -6
  9. package/agent/lib/sdk.js +25 -0
  10. package/agent/lib/system.js +2 -1
  11. package/agent/lib/validation.js +6 -6
  12. package/docs/docs.json +211 -101
  13. package/docs/snippets/tests/type-repeated-replay.mdx +1 -1
  14. package/docs/v7/_drafts/caching-selectors.mdx +24 -0
  15. package/docs/v7/_drafts/migration.mdx +3 -3
  16. package/docs/v7/api/act.mdx +2 -2
  17. package/docs/v7/api/assert.mdx +2 -2
  18. package/docs/v7/api/assertions.mdx +21 -21
  19. package/docs/v7/api/elements.mdx +78 -0
  20. package/docs/v7/api/find.mdx +38 -0
  21. package/docs/v7/api/focusApplication.mdx +2 -2
  22. package/docs/v7/api/hover.mdx +2 -2
  23. package/docs/v7/features/ai-native.mdx +57 -71
  24. package/docs/v7/features/application-logs.mdx +353 -0
  25. package/docs/v7/features/browser-logs.mdx +414 -0
  26. package/docs/v7/features/cache-management.mdx +402 -0
  27. package/docs/v7/features/continuous-testing.mdx +346 -0
  28. package/docs/v7/features/coverage.mdx +508 -0
  29. package/docs/v7/features/data-driven-testing.mdx +441 -0
  30. package/docs/v7/features/easy-to-write.mdx +2 -73
  31. package/docs/v7/features/enterprise.mdx +155 -39
  32. package/docs/v7/features/fast.mdx +63 -81
  33. package/docs/v7/features/managed-sandboxes.mdx +384 -0
  34. package/docs/v7/features/network-monitoring.mdx +568 -0
  35. package/docs/v7/features/observable.mdx +3 -22
  36. package/docs/v7/features/parallel-execution.mdx +381 -0
  37. package/docs/v7/features/powerful.mdx +1 -1
  38. package/docs/v7/features/reports.mdx +414 -0
  39. package/docs/v7/features/sandbox-customization.mdx +229 -0
  40. package/docs/v7/features/scalable.mdx +217 -2
  41. package/docs/v7/features/stable.mdx +106 -147
  42. package/docs/v7/features/system-performance.mdx +616 -0
  43. package/docs/v7/features/test-analytics.mdx +373 -0
  44. package/docs/v7/features/test-cases.mdx +393 -0
  45. package/docs/v7/features/test-replays.mdx +408 -0
  46. package/docs/v7/features/test-reports.mdx +308 -0
  47. package/docs/v7/getting-started/{running-and-debugging.mdx → debugging-tests.mdx} +12 -142
  48. package/docs/v7/getting-started/quickstart.mdx +22 -305
  49. package/docs/v7/getting-started/running-tests.mdx +173 -0
  50. package/docs/v7/overview/readme.mdx +1 -1
  51. package/docs/v7/overview/what-is-testdriver.mdx +2 -14
  52. package/docs/v7/presets/chrome-extension.mdx +147 -122
  53. package/interfaces/cli/commands/init.js +78 -20
  54. package/interfaces/cli/lib/base.js +3 -2
  55. package/interfaces/logger.js +0 -2
  56. package/interfaces/shared-test-state.mjs +0 -5
  57. package/interfaces/vitest-plugin.mjs +69 -42
  58. package/lib/core/Dashcam.js +65 -66
  59. package/lib/vitest/hooks.mjs +42 -50
  60. package/manual/test-init-command.js +223 -0
  61. package/package.json +2 -2
  62. package/schema.json +5 -5
  63. package/sdk-log-formatter.js +351 -176
  64. package/sdk.d.ts +8 -8
  65. package/sdk.js +436 -121
  66. package/setup/aws/cloudformation.yaml +2 -2
  67. package/setup/aws/self-hosted.yml +1 -1
  68. package/test/testdriver/chrome-extension.test.mjs +55 -72
  69. package/test/testdriver/element-not-found.test.mjs +2 -1
  70. package/test/testdriver/hover-image.test.mjs +1 -1
  71. package/test/testdriver/hover-text-with-description.test.mjs +0 -3
  72. package/test/testdriver/scroll-until-text.test.mjs +10 -6
  73. package/test/testdriver/setup/lifecycleHelpers.mjs +19 -24
  74. package/test/testdriver/setup/testHelpers.mjs +18 -23
  75. package/vitest.config.mjs +3 -3
  76. package/.github/workflows/linux-tests.yml +0 -28
  77. package/docs/v7/getting-started/generating-tests.mdx +0 -525
  78. package/test/testdriver/auto-cache-key-demo.test.mjs +0 -56
@@ -0,0 +1,441 @@
1
+ ---
2
+ title: "Data-Driven Testing"
3
+ description: "Build maintainable test suites with reusable code and dynamic test data"
4
+ icon: "table"
5
+ ---
6
+
7
+ Scale your test suite with reusable code patterns and dynamic data to reduce duplication and maintain consistency across thousands of tests.
8
+
9
+ ## Code Snippets for Reusability
10
+
11
+ Scale your test suite with reusable code snippets to reduce duplication and maintain consistency:
12
+
13
+ <Tabs>
14
+ <Tab title="Helper Functions">
15
+ ```javascript test/helpers/auth.js
16
+ // Reusable authentication helpers
17
+ export async function login(testdriver, { email, password }) {
18
+ await testdriver.find('email input').type(email);
19
+ await testdriver.find('password input').type(password);
20
+ await testdriver.find('login button').click();
21
+ await testdriver.assert('logged in successfully');
22
+ }
23
+
24
+ export async function logout(testdriver) {
25
+ await testdriver.find('user menu').click();
26
+ await testdriver.find('logout button').click();
27
+ }
28
+ ```
29
+
30
+ ```javascript test/checkout.test.js
31
+ import { login } from './helpers/auth.js';
32
+
33
+ test('checkout flow', async (context) => {
34
+ const { testdriver } = await chrome(context, { url });
35
+ await login(testdriver, {
36
+ email: 'user@example.com',
37
+ password: 'password123'
38
+ });
39
+ // Continue with checkout test
40
+ });
41
+ ```
42
+ </Tab>
43
+
44
+ <Tab title="Page Objects">
45
+ ```javascript test/pages/LoginPage.js
46
+ export class LoginPage {
47
+ constructor(testdriver) {
48
+ this.testdriver = testdriver;
49
+ }
50
+
51
+ async login(email, password) {
52
+ await this.testdriver.find('email input').type(email);
53
+ await this.testdriver.find('password input').type(password);
54
+ await this.testdriver.find('submit button').click();
55
+ }
56
+
57
+ async assertError(message) {
58
+ await this.testdriver.assert(`error message shows "${message}"`);
59
+ }
60
+ }
61
+ ```
62
+
63
+ ```javascript test/auth.test.js
64
+ import { LoginPage } from './pages/LoginPage.js';
65
+
66
+ test('invalid login shows error', async (context) => {
67
+ const { testdriver } = await chrome(context, { url });
68
+ const loginPage = new LoginPage(testdriver);
69
+
70
+ await loginPage.login('invalid@test.com', 'wrong');
71
+ await loginPage.assertError('Invalid credentials');
72
+ });
73
+ ```
74
+ </Tab>
75
+
76
+ <Tab title="Custom Commands">
77
+ ```javascript test/commands/navigation.js
78
+ export function addNavigationCommands(testdriver) {
79
+ testdriver.navigateTo = async (section) => {
80
+ await testdriver.find(`${section} nav link`).click();
81
+ await testdriver.assert(`${section} page is loaded`);
82
+ };
83
+
84
+ testdriver.searchFor = async (query) => {
85
+ await testdriver.find('search input').type(query);
86
+ await testdriver.find('search button').click();
87
+ await testdriver.assert('search results are displayed');
88
+ };
89
+
90
+ return testdriver;
91
+ }
92
+ ```
93
+
94
+ ```javascript test/search.test.js
95
+ import { addNavigationCommands } from './commands/navigation.js';
96
+
97
+ test('search functionality', async (context) => {
98
+ let { testdriver } = await chrome(context, { url });
99
+ testdriver = addNavigationCommands(testdriver);
100
+
101
+ await testdriver.searchFor('laptop');
102
+ // Custom command used
103
+ });
104
+ ```
105
+ </Tab>
106
+ </Tabs>
107
+
108
+ <Check>
109
+ Reusable snippets reduce test maintenance time by up to 70% in large test suites.
110
+ </Check>
111
+
112
+ ## Dynamic Variables for Data-Driven Tests
113
+
114
+ Scale your testing with dynamic data to cover more scenarios:
115
+
116
+ <Tabs>
117
+ <Tab title="Environment Variables">
118
+ ```javascript
119
+ import { test } from 'vitest';
120
+ import { chrome } from 'testdriverai/presets';
121
+
122
+ test('multi-environment testing', async (context) => {
123
+ const env = process.env.TEST_ENV || 'staging';
124
+ const urls = {
125
+ dev: 'https://dev.myapp.com',
126
+ staging: 'https://staging.myapp.com',
127
+ production: 'https://myapp.com'
128
+ };
129
+
130
+ const { testdriver } = await chrome(context, {
131
+ url: urls[env]
132
+ });
133
+
134
+ await testdriver.assert('app is running');
135
+ });
136
+ ```
137
+
138
+ ```bash
139
+ # Run against different environments
140
+ TEST_ENV=dev npx vitest run
141
+ TEST_ENV=staging npx vitest run
142
+ TEST_ENV=production npx vitest run
143
+ ```
144
+ </Tab>
145
+
146
+ <Tab title="Test Fixtures">
147
+ ```javascript test/fixtures/users.js
148
+ export const testUsers = [
149
+ { email: 'admin@test.com', role: 'admin' },
150
+ { email: 'user@test.com', role: 'user' },
151
+ { email: 'guest@test.com', role: 'guest' }
152
+ ];
153
+
154
+ export const products = [
155
+ { name: 'Laptop', price: 999 },
156
+ { name: 'Mouse', price: 29 },
157
+ { name: 'Keyboard', price: 89 }
158
+ ];
159
+ ```
160
+
161
+ ```javascript test/permissions.test.js
162
+ import { test } from 'vitest';
163
+ import { chrome } from 'testdriverai/presets';
164
+ import { testUsers } from './fixtures/users.js';
165
+
166
+ test.each(testUsers)('$role can access dashboard', async ({ email, role }, context) => {
167
+ const { testdriver } = await chrome(context, { url });
168
+
169
+ await testdriver.find('email input').type(email);
170
+ await testdriver.find('password input').type('password123');
171
+ await testdriver.find('login button').click();
172
+
173
+ if (role === 'admin') {
174
+ await testdriver.assert('admin panel is visible');
175
+ } else {
176
+ await testdriver.assert('user dashboard is visible');
177
+ }
178
+ });
179
+ ```
180
+ </Tab>
181
+
182
+ <Tab title="Dynamic Data Generation">
183
+ ```javascript
184
+ import { test } from 'vitest';
185
+ import { chrome } from 'testdriverai/presets';
186
+ import { faker } from '@faker-js/faker';
187
+
188
+ test('user registration with dynamic data', async (context) => {
189
+ const { testdriver } = await chrome(context, { url });
190
+
191
+ // Generate unique test data for each run
192
+ const userData = {
193
+ firstName: faker.person.firstName(),
194
+ lastName: faker.person.lastName(),
195
+ email: faker.internet.email(),
196
+ password: faker.internet.password({ length: 12 })
197
+ };
198
+
199
+ await testdriver.find('first name input').type(userData.firstName);
200
+ await testdriver.find('last name input').type(userData.lastName);
201
+ await testdriver.find('email input').type(userData.email);
202
+ await testdriver.find('password input').type(userData.password);
203
+ await testdriver.find('register button').click();
204
+
205
+ await testdriver.assert('registration successful');
206
+ console.log('Registered user:', userData.email);
207
+ });
208
+ ```
209
+
210
+ ```bash
211
+ npm install --save-dev @faker-js/faker
212
+ ```
213
+ </Tab>
214
+ </Tabs>
215
+
216
+ <Card title="Dynamic Variables Best Practices" icon="lightbulb">
217
+ - **Environment configs:** Store URLs, credentials, and settings in env vars
218
+ - **Test fixtures:** Maintain reusable test data in separate files
219
+ - **Data generators:** Use libraries like Faker for unique test data
220
+ - **Parameterization:** Test multiple scenarios with `test.each()`
221
+ - **CI/CD integration:** Pass dynamic values via environment variables
222
+ </Card>
223
+
224
+ ## Secure Secrets Management
225
+
226
+ Protect sensitive data in your tests with built-in secrets handling:
227
+
228
+ <Tabs>
229
+ <Tab title="Secret Type Option">
230
+ ```javascript
231
+ import { test } from 'vitest';
232
+ import { chrome } from 'testdriverai/presets';
233
+
234
+ test('login with secret password', async (context) => {
235
+ const { testdriver } = await chrome(context, { url });
236
+
237
+ await testdriver.find('email input').type('user@example.com');
238
+
239
+ // Password is never logged or cached
240
+ await testdriver.find('password input').type(process.env.TEST_PASSWORD, {
241
+ secret: true
242
+ });
243
+
244
+ await testdriver.find('login button').click();
245
+ await testdriver.assert('logged in successfully');
246
+ });
247
+ ```
248
+
249
+ <Check>
250
+ The `secret: true` option prevents sensitive data from appearing in logs, dashcam replays, or cache entries.
251
+ </Check>
252
+ </Tab>
253
+
254
+ <Tab title="Environment Variables">
255
+ ```javascript
256
+ // .env.test
257
+ TEST_PASSWORD=my-secret-password
258
+ TEST_API_KEY=sk-1234567890abcdef
259
+ TEST_CREDIT_CARD=4111111111111111
260
+
261
+ // test/auth.test.js
262
+ import { test } from 'vitest';
263
+ import { chrome } from 'testdriverai/presets';
264
+
265
+ test('secure authentication', async (context) => {
266
+ const { testdriver } = await chrome(context, { url });
267
+
268
+ // Load secrets from environment
269
+ await testdriver.find('password input').type(
270
+ process.env.TEST_PASSWORD,
271
+ { secret: true }
272
+ );
273
+
274
+ await testdriver.find('api key input').type(
275
+ process.env.TEST_API_KEY,
276
+ { secret: true }
277
+ );
278
+ });
279
+ ```
280
+
281
+ ```bash
282
+ # Install dotenv for local development
283
+ npm install --save-dev dotenv
284
+
285
+ # Load in test setup
286
+ import 'dotenv/config';
287
+ ```
288
+ </Tab>
289
+
290
+ <Tab title="CI/CD Secrets">
291
+ ```yaml .github/workflows/test.yml
292
+ name: E2E Tests
293
+
294
+ on: [push, pull_request]
295
+
296
+ jobs:
297
+ test:
298
+ runs-on: ubuntu-latest
299
+ steps:
300
+ - uses: actions/checkout@v3
301
+ - uses: actions/setup-node@v3
302
+ - run: npm install
303
+ - run: npx vitest run
304
+ env:
305
+ TD_API_KEY: ${{ secrets.TD_API_KEY }}
306
+ TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
307
+ TEST_API_KEY: ${{ secrets.TEST_API_KEY }}
308
+ ```
309
+
310
+ Configure secrets in your CI/CD platform:
311
+ - **GitHub:** Settings → Secrets and variables → Actions
312
+ - **GitLab:** Settings → CI/CD → Variables
313
+ - **CircleCI:** Project Settings → Environment Variables
314
+ </Tab>
315
+
316
+ <Tab title="Helper Functions">
317
+ ```javascript test/helpers/secrets.js
318
+ // Centralized secret handling
319
+ export function getSecret(key) {
320
+ const value = process.env[key];
321
+ if (!value) {
322
+ throw new Error(`Secret ${key} not found in environment`);
323
+ }
324
+ return value;
325
+ }
326
+
327
+ export async function typeSecret(testdriver, selector, secretKey) {
328
+ const value = getSecret(secretKey);
329
+ await testdriver.find(selector).type(value, { secret: true });
330
+ }
331
+ ```
332
+
333
+ ```javascript test/payment.test.js
334
+ import { typeSecret } from './helpers/secrets.js';
335
+
336
+ test('payment form', async (context) => {
337
+ const { testdriver } = await chrome(context, { url });
338
+
339
+ await typeSecret(testdriver, 'credit card input', 'TEST_CREDIT_CARD');
340
+ await typeSecret(testdriver, 'cvv input', 'TEST_CVV');
341
+
342
+ await testdriver.find('submit payment').click();
343
+ });
344
+ ```
345
+ </Tab>
346
+ </Tabs>
347
+
348
+ <Warning>
349
+ Never hardcode secrets in test files or commit them to version control. Always use environment variables or secure secret management systems.
350
+ </Warning>
351
+
352
+ ## Familiar Test Syntax
353
+
354
+ TestDriver works with the test frameworks you already know:
355
+
356
+ <Tabs>
357
+ <Tab title="Vitest">
358
+ ```javascript
359
+ import { test, describe, beforeAll, afterAll } from 'vitest';
360
+ import { chrome } from 'testdriverai/presets';
361
+
362
+ describe('My Feature', () => {
363
+ test('should work', async (context) => {
364
+ const { testdriver } = await chrome(context, { url });
365
+ await testdriver.find('button').click();
366
+ });
367
+ });
368
+ ```
369
+ </Tab>
370
+
371
+ <Tab title="Jest">
372
+ ```javascript
373
+ import { chrome } from 'testdriverai/presets';
374
+
375
+ describe('My Feature', () => {
376
+ test('should work', async () => {
377
+ const { testdriver } = await chrome({ url });
378
+ await testdriver.find('button').click();
379
+ });
380
+ });
381
+ ```
382
+ </Tab>
383
+
384
+ <Tab title="Mocha">
385
+ ```javascript
386
+ import { chrome } from 'testdriverai/presets';
387
+
388
+ describe('My Feature', function() {
389
+ it('should work', async function() {
390
+ const { testdriver } = await chrome(this, { url });
391
+ await testdriver.find('button').click();
392
+ });
393
+ });
394
+ ```
395
+ </Tab>
396
+ </Tabs>
397
+
398
+ ## Existing Systems Integration
399
+
400
+ Drop TestDriver into your current workflow without disruption:
401
+
402
+ ```javascript
403
+ import { test, expect } from 'vitest';
404
+ import { chrome } from 'testdriverai/presets';
405
+
406
+ test('integrates with existing assertions', async (context) => {
407
+ const { testdriver } = await chrome(context, { url });
408
+
409
+ // Use TestDriver's AI assertions
410
+ await testdriver.assert('welcome message is visible');
411
+
412
+ // Or use traditional assertions
413
+ const button = await testdriver.find('submit button');
414
+ expect(button.coordinates).toBeDefined();
415
+ expect(button.text).toContain('Submit');
416
+
417
+ // Mix and match as needed
418
+ const element = await testdriver.exec('js', 'document.title');
419
+ expect(element).toBe('My App');
420
+ });
421
+ ```
422
+
423
+ ## Learn More
424
+
425
+ <CardGroup cols={2}>
426
+ <Card
427
+ title="Vitest Integration"
428
+ icon="flask-vial"
429
+ href="/v7/guides/vitest"
430
+ >
431
+ Complete Vitest guide
432
+ </Card>
433
+
434
+ <Card
435
+ title="Continuous Testing"
436
+ icon="arrows-spin"
437
+ href="/v7/features/continuous-testing"
438
+ >
439
+ CI/CD integration and scaling
440
+ </Card>
441
+ </CardGroup>
@@ -1,5 +1,5 @@
1
1
  ---
2
- title: "Easy to Write"
2
+ title: "Natural Language Syntax"
3
3
  description: "Natural language testing that feels like describing what you want, not programming"
4
4
  icon: "feather"
5
5
  ---
@@ -61,7 +61,7 @@ All element methods are chainable:
61
61
 
62
62
  ## Familiar Syntax
63
63
 
64
- TestDriver integrates seamlessly with the test frameworks you already know and love:
64
+ TestDriver uses `vitest` to run tests with a familiar syntax you already know and love:
65
65
 
66
66
  <Tabs>
67
67
  <Tab title="Vitest">
@@ -78,36 +78,6 @@ TestDriver integrates seamlessly with the test frameworks you already know and l
78
78
  });
79
79
  ```
80
80
  </Tab>
81
-
82
- <Tab title="Jest">
83
- ```javascript
84
- import { chrome } from 'testdriverai/presets';
85
-
86
- test('my test', async () => {
87
- const { testdriver } = await chrome({
88
- url: 'https://example.com'
89
- });
90
-
91
- await testdriver.find('button').click();
92
- });
93
- ```
94
- </Tab>
95
-
96
- <Tab title="Mocha">
97
- ```javascript
98
- import { chrome } from 'testdriverai/presets';
99
-
100
- describe('My Suite', () => {
101
- it('should work', async function() {
102
- const { testdriver } = await chrome(this, {
103
- url: 'https://example.com'
104
- });
105
-
106
- await testdriver.find('button').click();
107
- });
108
- });
109
- ```
110
- </Tab>
111
81
  </Tabs>
112
82
 
113
83
  No need to learn a new DSL or configuration format. TestDriver works with your existing test infrastructure.
@@ -217,47 +187,6 @@ test('user can login', async () => {
217
187
  ```
218
188
  </CodeGroup>
219
189
 
220
- ## Progressive Complexity
221
-
222
- Choose your level of control:
223
-
224
- <CardGroup cols={3}>
225
- <Card title="Presets (Easiest)" icon="rocket">
226
- One-line setup for common scenarios:
227
-
228
- ```javascript
229
- const { testdriver } = await chrome(context, {
230
- url: 'https://example.com'
231
- });
232
- ```
233
-
234
- [Learn more](/v7/progressive-apis/PROVISION)
235
- </Card>
236
-
237
- <Card title="Hooks (Flexible)" icon="link">
238
- More control with composable hooks:
239
-
240
- ```javascript
241
- const client = useTestDriver(context);
242
- const dashcam = useDashcam(context, client);
243
- ```
244
-
245
- [Learn more](/v7/progressive-apis/HOOKS)
246
- </Card>
247
-
248
- <Card title="Core (Full Control)" icon="code">
249
- Manual control over everything:
250
-
251
- ```javascript
252
- const client = new TestDriver(apiKey);
253
- await client.connect();
254
- await client.find('button');
255
- ```
256
-
257
- [Learn more](/v7/progressive-apis/PROGRESSIVE_DISCLOSURE)
258
- </Card>
259
- </CardGroup>
260
-
261
190
  ## Zero Boilerplate
262
191
 
263
192
  Presets handle all the setup and teardown automatically: