testdriverai 7.2.21 → 7.2.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,1348 +0,0 @@
1
- # TestDriver SDK & Plugin Design - Optimal Architecture
2
-
3
- ## Executive Summary
4
-
5
- This document outlines the optimal SDK and plugin structure for TestDriver, designed to enable developers to add test automation to their applications with flexibility for both quick-start scenarios and advanced configurations.
6
-
7
- **Key Principles:**
8
- 1. **Progressive Disclosure**: Simple by default, powerful when needed
9
- 2. **Convention over Configuration**: Smart defaults for 90% use cases
10
- 3. **Composability**: Building blocks that can be mixed and matched
11
- 4. **Framework-First**: Vitest plugin as the primary interface, with standalone SDK support
12
- 5. **Type Safety**: Full TypeScript support for better DX
13
-
14
- ---
15
-
16
- ## Table of Contents
17
-
18
- 1. [Architecture Overview](#architecture-overview)
19
- 2. [Layer 1: Core SDK](#layer-1-core-sdk)
20
- 3. [Layer 2: Vitest Plugin](#layer-2-vitest-plugin)
21
- 4. [Layer 3: Presets & Helpers](#layer-3-presets--helpers)
22
- 5. [Configuration System](#configuration-system)
23
- 6. [Usage Examples](#usage-examples)
24
- 7. [Migration Path](#migration-path)
25
-
26
- ---
27
-
28
- ## Architecture Overview
29
-
30
- ```
31
- ┌─────────────────────────────────────────────────────────────┐
32
- │ Layer 3: Presets │
33
- │ Chrome Web Presets | VSCode Presets | Custom App Presets │
34
- └─────────────────────────────────────────────────────────────┘
35
-
36
- ┌─────────────────────────────────────────────────────────────┐
37
- │ Layer 2: Vitest Plugin │
38
- │ Auto-lifecycle | Test Recording | Dashcam Integration │
39
- └─────────────────────────────────────────────────────────────┘
40
-
41
- ┌─────────────────────────────────────────────────────────────┐
42
- │ Layer 1: Core SDK │
43
- │ TestDriver Client | Sandbox | Dashcam | Element API │
44
- └─────────────────────────────────────────────────────────────┘
45
- ```
46
-
47
- ---
48
-
49
- ## Layer 1: Core SDK
50
-
51
- ### 1.1 TestDriver Class (Core Client)
52
-
53
- **Purpose**: Low-level SDK for direct programmatic control
54
-
55
- ```typescript
56
- interface TestDriverOptions {
57
- apiKey?: string; // Can be omitted if using env var
58
- apiRoot?: string; // Default: production API
59
- os?: 'linux' | 'windows' | 'mac';
60
- resolution?: string;
61
- cache?: boolean;
62
- analytics?: boolean;
63
-
64
- // Advanced sandbox options
65
- sandbox?: {
66
- ami?: string;
67
- instance?: string;
68
- ip?: string;
69
- headless?: boolean;
70
- };
71
- }
72
-
73
- class TestDriver {
74
- constructor(apiKey?: string, options?: TestDriverOptions);
75
-
76
- // Core lifecycle
77
- async auth(): Promise<void>;
78
- async connect(options?: ConnectOptions): Promise<SandboxInstance>;
79
- async disconnect(): Promise<void>;
80
-
81
- // Element API (modern, recommended)
82
- async find(description: string): Promise<Element>;
83
- async findAll(description: string): Promise<Element[]>;
84
-
85
- // Element operations
86
- async click(x: number, y: number): Promise<void>;
87
- async type(text: string): Promise<void>;
88
- async pressKeys(keys: string[]): Promise<void>;
89
- async scroll(direction: Direction, amount: number): Promise<void>;
90
-
91
- // AI operations
92
- async ai(prompt: string): Promise<void>;
93
- async assert(expectation: string): Promise<boolean>;
94
- async remember(query: string): Promise<string>;
95
-
96
- // System operations
97
- async exec(shell: string, command: string, timeout?: number): Promise<string>;
98
- async focusApplication(name: string): Promise<void>;
99
-
100
- // State & utilities
101
- getSessionId(): string;
102
- getInstance(): SandboxInstance;
103
- getEmitter(): EventEmitter;
104
- }
105
- ```
106
-
107
- ### 1.2 Dashcam Module (Separate, Composable)
108
-
109
- ```typescript
110
- interface DashcamOptions {
111
- apiKey?: string;
112
- autoStart?: boolean;
113
- logs?: LogConfig[];
114
- }
115
-
116
- interface LogConfig {
117
- name: string;
118
- type: 'file' | 'stdout' | 'application';
119
- path?: string;
120
- application?: string;
121
- }
122
-
123
- class Dashcam {
124
- constructor(client: TestDriver, options?: DashcamOptions);
125
-
126
- async auth(apiKey?: string): Promise<void>;
127
- async addLog(config: LogConfig): Promise<void>;
128
- async start(): Promise<void>;
129
- async stop(): Promise<string | null>; // Returns replay URL
130
- async isRecording(): Promise<boolean>;
131
-
132
- // Convenience methods
133
- async addFileLog(path: string, name: string): Promise<void>;
134
- async addApplicationLog(app: string, name: string): Promise<void>;
135
- }
136
- ```
137
-
138
- ### 1.3 Element Class (Enhanced)
139
-
140
- ```typescript
141
- class Element {
142
- // Properties
143
- readonly description: string;
144
- readonly x: number | null;
145
- readonly y: number | null;
146
- readonly centerX: number | null;
147
- readonly centerY: number | null;
148
- readonly found: boolean;
149
-
150
- // Actions
151
- async click(): Promise<void>;
152
- async doubleClick(): Promise<void>;
153
- async rightClick(): Promise<void>;
154
- async hover(): Promise<void>;
155
- async dragTo(target: Element | {x: number, y: number}): Promise<void>;
156
-
157
- // Queries
158
- async getText(): Promise<string>;
159
- async isVisible(): Promise<boolean>;
160
- async waitUntilVisible(timeout?: number): Promise<void>;
161
-
162
- // Utilities
163
- found(): boolean;
164
- getDebugInfo(): DebugInfo;
165
- async saveScreenshot(path?: string): Promise<string>;
166
- }
167
- ```
168
-
169
- ---
170
-
171
- ## Layer 2: Vitest Plugin
172
-
173
- ### 2.1 Plugin Architecture
174
-
175
- The Vitest plugin provides automatic lifecycle management and test recording.
176
-
177
- ```typescript
178
- // vitest.config.ts
179
- import { defineConfig } from 'vitest/config';
180
- import testdriver from 'testdriverai/vitest';
181
-
182
- export default defineConfig({
183
- plugins: [
184
- testdriver({
185
- // API configuration
186
- apiKey: process.env.TD_API_KEY,
187
- apiRoot: process.env.TD_API_ROOT,
188
-
189
- // Lifecycle mode
190
- mode: 'auto', // 'auto' | 'manual' | 'hybrid'
191
-
192
- // Dashcam configuration
193
- dashcam: {
194
- enabled: true,
195
- scope: 'test', // 'test' | 'file' | 'suite' | 'run'
196
- autoStart: true,
197
- logs: [
198
- { type: 'file', path: '/tmp/app.log', name: 'App Log' }
199
- ]
200
- },
201
-
202
- // Application presets
203
- preset: 'chrome', // 'chrome' | 'vscode' | 'custom' | null
204
-
205
- // Custom launch configuration
206
- launch: {
207
- application: 'Google Chrome',
208
- url: 'https://myapp.com',
209
- waitFor: 'Login page is visible'
210
- },
211
-
212
- // Platform
213
- os: 'linux', // 'linux' | 'windows' | 'mac'
214
-
215
- // Test recording
216
- recording: {
217
- enabled: true,
218
- includeScreenshots: true,
219
- includeLogs: true
220
- }
221
- })
222
- ]
223
- });
224
- ```
225
-
226
- ### 2.2 Lifecycle Modes
227
-
228
- **Quick Comparison:**
229
-
230
- | Feature | Auto Mode | Hybrid Mode | Manual Mode |
231
- |---------|-----------|-------------|-------------|
232
- | **Who it's for** | Quick start, standard apps | Power users, custom apps | Full control, any framework |
233
- | **Setup complexity** | Minimal (1 line) | Medium (selective overrides) | High (all boilerplate) |
234
- | **Sandbox lifecycle** | ✅ Auto | ✅ Auto | ❌ You manage |
235
- | **App launch** | ✅ Auto (via preset) | ⚙️ You customize | ❌ You manage |
236
- | **Dashcam start/stop** | ✅ Auto (per test) | ⚙️ Customizable scope | ❌ You manage |
237
- | **Test recording** | ✅ Auto | ✅ Auto | ❌ You manage |
238
- | **Cleanup** | ✅ Auto | ✅ Auto | ❌ You manage |
239
- | **Code in tests** | Minimal | Medium | Maximum |
240
- | **Flexibility** | Low | High | Maximum |
241
-
242
- #### Auto Mode (Default - 90% Use Case)
243
- Handles everything automatically:
244
- - Creates sandbox before each test file
245
- - Authenticates dashcam
246
- - Launches application
247
- - Starts recording per test
248
- - Stops recording and uploads
249
- - Cleans up after tests
250
-
251
- ```typescript
252
- // tests/login.test.ts
253
- import { describe, it, expect } from 'vitest';
254
- import { useTestDriver } from 'testdriverai/vitest';
255
-
256
- describe('Login Flow', () => {
257
- const td = useTestDriver(); // Auto-configured!
258
-
259
- it('should login successfully', async () => {
260
- const email = await td.find('Email input');
261
- await email.click();
262
- await td.type('user@example.com');
263
-
264
- const submit = await td.find('Submit button');
265
- await submit.click();
266
-
267
- expect(await td.assert('Dashboard is visible')).toBe(true);
268
- });
269
- });
270
- ```
271
-
272
- #### Manual Mode (Advanced - Full Control)
273
- User manages all lifecycle:
274
-
275
- ```typescript
276
- import { describe, it, beforeAll, afterAll } from 'vitest';
277
- import { createTestDriver, createDashcam } from 'testdriverai';
278
-
279
- describe('Custom Lifecycle', () => {
280
- let td: TestDriver;
281
- let dashcam: Dashcam;
282
-
283
- beforeAll(async () => {
284
- td = createTestDriver();
285
- await td.auth();
286
- await td.connect();
287
-
288
- dashcam = createDashcam(td);
289
- await dashcam.auth();
290
- });
291
-
292
- beforeEach(async () => {
293
- await dashcam.start();
294
- });
295
-
296
- afterEach(async () => {
297
- const url = await dashcam.stop();
298
- console.log('Recording:', url);
299
- });
300
-
301
- afterAll(async () => {
302
- await td.disconnect();
303
- });
304
-
305
- it('custom test', async () => {
306
- // Your test
307
- });
308
- });
309
- ```
310
-
311
- #### Hybrid Mode (Recommended for Power Users)
312
-
313
- **What it is**: Start with auto mode's convenience, but selectively override specific behaviors where you need custom control.
314
-
315
- **When to use**: When you need the plugin to handle most lifecycle (sandbox creation, cleanup, test recording) but want to customize specific parts like:
316
- - Custom application launch sequence
317
- - Special dashcam configuration
318
- - Pre-test setup that's unique to your app
319
- - Different recording scope (file vs test)
320
-
321
- **How it works**: Use the convenience hooks (`useTestDriver`, `useDashcam`) but pass options to disable/customize specific auto behaviors.
322
-
323
- ```typescript
324
- import { describe, it, beforeAll } from 'vitest';
325
- import { useTestDriver, useDashcam } from 'testdriverai/vitest';
326
-
327
- describe('Hybrid Mode Example', () => {
328
- // Plugin handles: sandbox creation, auth, cleanup
329
- // We customize: launch and dashcam scope
330
- const td = useTestDriver({
331
- autoConnect: true, // ✅ Plugin auto-connects sandbox
332
- autoLaunch: false // ❌ We'll launch manually
333
- });
334
-
335
- const dashcam = useDashcam({
336
- scope: 'file', // Override: record entire file, not per-test
337
- logs: [
338
- { type: 'file', path: '/custom/path.log', name: 'My Log' }
339
- ]
340
- });
341
-
342
- beforeAll(async () => {
343
- // Custom launch sequence for our specific app
344
- await td.exec('sh', './scripts/setup-test-env.sh');
345
- await td.exec('sh', './scripts/start-app.sh');
346
- await td.wait(5000);
347
- });
348
-
349
- it('should work', async () => {
350
- // Test normally - dashcam auto-started at file level
351
- const btn = await td.find('Button');
352
- await btn.click();
353
- });
354
- });
355
- ```
356
-
357
- **Real-world example**: Testing a desktop app that needs special setup
358
-
359
- ```typescript
360
- describe('Electron App Tests', () => {
361
- const td = useTestDriver({
362
- autoConnect: true, // ✅ Let plugin handle sandbox
363
- autoLaunch: false, // ❌ Custom electron launch
364
- });
365
-
366
- const dashcam = useDashcam({
367
- scope: 'suite', // Override: one recording for entire suite
368
- logs: [
369
- { type: 'file', path: '/tmp/electron.log', name: 'App Log' },
370
- { type: 'file', path: '/tmp/renderer.log', name: 'Renderer Log' }
371
- ]
372
- });
373
-
374
- beforeAll(async () => {
375
- // Custom: Build and launch Electron app
376
- await td.exec('sh', 'npm run build');
377
- await td.exec('sh', 'npm run start', 60000);
378
- await td.wait(10000); // Wait for app to fully load
379
- });
380
-
381
- it('test 1', async () => {
382
- // Plugin auto-started dashcam at suite level
383
- // No need to manually start/stop per test
384
- const menu = await td.find('File menu');
385
- await menu.click();
386
- });
387
-
388
- it('test 2', async () => {
389
- // Same recording continues
390
- const newFile = await td.find('New File button');
391
- await newFile.click();
392
- });
393
-
394
- // Plugin auto-stops dashcam and cleans up after suite
395
- });
396
- ```
397
-
398
- **Comparison**:
399
-
400
- | What Happens | Auto Mode | Hybrid Mode | Manual Mode |
401
- |--------------|-----------|-------------|-------------|
402
- | Sandbox creation | ✅ Plugin | ✅ Plugin | ❌ You write |
403
- | Sandbox cleanup | ✅ Plugin | ✅ Plugin | ❌ You write |
404
- | App launch | ✅ Plugin (preset) | ⚙️ You customize | ❌ You write |
405
- | Dashcam start/stop | ✅ Plugin (per test) | ⚙️ You customize scope | ❌ You write |
406
- | Test recording | ✅ Plugin | ✅ Plugin | ❌ You write |
407
-
408
- ✅ = Automatic, ⚙️ = Customizable, ❌ = Your responsibility
409
-
410
- ### 2.3 Plugin Hooks System
411
-
412
- ```typescript
413
- // Expose hooks for custom behavior
414
- import { onBeforeConnect, onAfterConnect, onTestComplete } from 'testdriverai/vitest';
415
-
416
- onBeforeConnect(async (client) => {
417
- // Custom setup before sandbox connection
418
- console.log('Connecting to sandbox...');
419
- });
420
-
421
- onAfterConnect(async (client) => {
422
- // Custom setup after connection
423
- await client.exec('sh', 'install-dependencies.sh');
424
- });
425
-
426
- onTestComplete(async (client, result) => {
427
- // Custom cleanup or reporting
428
- if (result.status === 'failed') {
429
- await client.screenshot('failure.png');
430
- }
431
- });
432
- ```
433
-
434
- ---
435
-
436
- ## Layer 3: Presets & Helpers
437
-
438
- ### 3.1 Application Presets
439
-
440
- Presets are pre-configured bundles for common applications.
441
-
442
- ```typescript
443
- // Built-in presets
444
- import { chromePreset, vscodePreset, figmaPreset } from 'testdriverai/presets';
445
-
446
- // Chrome preset
447
- const chrome = chromePreset({
448
- url: 'https://myapp.com',
449
- waitFor: 'Page loaded',
450
- profile: 'guest', // 'guest' | 'default' | 'custom'
451
- windowSize: 'maximized',
452
- logs: true // Auto-configure console logs
453
- });
454
-
455
- // VSCode preset
456
- const vscode = vscodePreset({
457
- workspace: '/path/to/workspace',
458
- extensions: ['ms-python.python'],
459
- settings: {
460
- 'editor.fontSize': 14
461
- },
462
- logs: {
463
- extension: true,
464
- console: true
465
- }
466
- });
467
-
468
- // Use in plugin config
469
- testdriver({
470
- preset: chrome,
471
- // or
472
- preset: vscode
473
- })
474
- ```
475
-
476
- ### 3.2 Preset Definition API
477
-
478
- ```typescript
479
- interface ApplicationPreset {
480
- name: string;
481
-
482
- // Lifecycle hooks
483
- beforeConnect?: (client: TestDriver) => Promise<void>;
484
- afterConnect?: (client: TestDriver) => Promise<void>;
485
- beforeTest?: (client: TestDriver) => Promise<void>;
486
- afterTest?: (client: TestDriver) => Promise<void>;
487
-
488
- // Dashcam configuration
489
- dashcam?: {
490
- logs?: LogConfig[];
491
- scope?: 'test' | 'file' | 'suite';
492
- };
493
-
494
- // Launch configuration
495
- launch?: {
496
- command?: string;
497
- waitFor?: string;
498
- timeout?: number;
499
- };
500
- }
501
-
502
- // Example: Custom Slack preset
503
- export const slackPreset = (options: {
504
- workspace: string;
505
- autoLogin?: boolean;
506
- }): ApplicationPreset => ({
507
- name: 'Slack',
508
-
509
- async afterConnect(client) {
510
- // Launch Slack desktop app
511
- await client.exec('sh', 'open -a Slack');
512
- await client.wait(3000);
513
- },
514
-
515
- async beforeTest(client) {
516
- if (options.autoLogin) {
517
- await client.focusApplication('Slack');
518
- const workspace = await client.find('Workspace selector');
519
- await workspace.click();
520
- const ws = await client.find(options.workspace);
521
- await ws.click();
522
- }
523
- },
524
-
525
- dashcam: {
526
- logs: [
527
- {
528
- type: 'application',
529
- application: 'Slack',
530
- name: 'Slack Logs'
531
- }
532
- ],
533
- scope: 'test'
534
- }
535
- });
536
- ```
537
-
538
- ### 3.3 Helper Functions
539
-
540
- ```typescript
541
- // testdriverai/helpers
542
-
543
- // Authentication helpers
544
- export async function loginWithGoogle(
545
- client: TestDriver,
546
- email: string,
547
- password: string
548
- ): Promise<void>;
549
-
550
- export async function loginWithGithub(
551
- client: TestDriver,
552
- username: string,
553
- password: string
554
- ): Promise<void>;
555
-
556
- // Common workflows
557
- export async function uploadFile(
558
- client: TestDriver,
559
- filePath: string,
560
- inputDescription: string
561
- ): Promise<void>;
562
-
563
- export async function fillForm(
564
- client: TestDriver,
565
- formData: Record<string, string>
566
- ): Promise<void>;
567
-
568
- // Waiting utilities
569
- export async function waitForNavigation(
570
- client: TestDriver,
571
- expectedUrl: string,
572
- timeout?: number
573
- ): Promise<void>;
574
-
575
- export async function waitForElement(
576
- client: TestDriver,
577
- description: string,
578
- timeout?: number
579
- ): Promise<Element>;
580
-
581
- // Retry utilities
582
- export async function retryUntilSuccess<T>(
583
- fn: () => Promise<T>,
584
- options?: { maxAttempts?: number; delay?: number }
585
- ): Promise<T>;
586
- ```
587
-
588
- ---
589
-
590
- ## Configuration System
591
-
592
- ### 4.1 Configuration Hierarchy
593
-
594
- Configuration is resolved in this order (later overrides earlier):
595
-
596
- 1. Built-in defaults
597
- 2. `testdriver.config.ts` file
598
- 3. `vitest.config.ts` plugin options
599
- 4. Environment variables
600
- 5. Runtime options (passed to functions)
601
-
602
- ### 4.2 Shared Configuration File
603
-
604
- ```typescript
605
- // testdriver.config.ts
606
- import { defineConfig } from 'testdriverai';
607
- import { chromePreset } from 'testdriverai/presets';
608
-
609
- export default defineConfig({
610
- // Global defaults
611
- apiKey: process.env.TD_API_KEY,
612
- os: 'linux',
613
-
614
- // Dashcam defaults
615
- dashcam: {
616
- enabled: true,
617
- scope: 'test',
618
- apiKey: process.env.DASHCAM_API_KEY || process.env.TD_API_KEY
619
- },
620
-
621
- // Application preset
622
- preset: chromePreset({
623
- url: process.env.APP_URL || 'http://localhost:3000'
624
- }),
625
-
626
- // Per-environment overrides
627
- environments: {
628
- ci: {
629
- os: 'linux',
630
- dashcam: { enabled: true },
631
- recording: { includeScreenshots: true }
632
- },
633
- local: {
634
- os: 'mac',
635
- dashcam: { enabled: false },
636
- analytics: false
637
- }
638
- }
639
- });
640
- ```
641
-
642
- ### 4.3 Environment-Specific Config
643
-
644
- ```typescript
645
- // Automatically select based on NODE_ENV
646
- export default defineConfig({
647
- extends: './testdriver.config.base.ts',
648
-
649
- // Only applied when NODE_ENV=production
650
- production: {
651
- os: 'linux',
652
- dashcam: { enabled: true }
653
- },
654
-
655
- // Only applied when NODE_ENV=development
656
- development: {
657
- os: 'mac',
658
- dashcam: { enabled: false },
659
- cache: false // Disable cache in dev for accuracy
660
- }
661
- });
662
- ```
663
-
664
- ---
665
-
666
- ## Usage Examples
667
-
668
- ### 5.1 Quick Start (90% Use Case)
669
-
670
- **Goal**: Test a web app with minimal setup
671
-
672
- ```typescript
673
- // vitest.config.ts
674
- import { defineConfig } from 'vitest/config';
675
- import testdriver from 'testdriverai/vitest';
676
-
677
- export default defineConfig({
678
- plugins: [
679
- testdriver({
680
- preset: 'chrome', // That's it!
681
- launch: { url: 'https://myapp.com' }
682
- })
683
- ]
684
- });
685
-
686
- // tests/smoke.test.ts
687
- import { describe, it, expect } from 'vitest';
688
- import { useTestDriver } from 'testdriverai/vitest';
689
-
690
- describe('Smoke Tests', () => {
691
- const td = useTestDriver();
692
-
693
- it('homepage loads', async () => {
694
- expect(await td.assert('Homepage is visible')).toBe(true);
695
- });
696
-
697
- it('can login', async () => {
698
- const email = await td.find('Email field');
699
- await email.click();
700
- await td.type('test@example.com');
701
-
702
- const password = await td.find('Password field');
703
- await password.click();
704
- await td.type('password123');
705
-
706
- const submit = await td.find('Submit button');
707
- await submit.click();
708
-
709
- expect(await td.assert('Dashboard is visible')).toBe(true);
710
- });
711
- });
712
- ```
713
-
714
- ### 5.2 Hybrid Mode: Custom Application Setup
715
-
716
- **Goal**: Test VSCode extension with custom setup - plugin handles infrastructure, we customize launch
717
-
718
- ```typescript
719
- // vitest.config.ts
720
- import { defineConfig } from 'vitest/config';
721
- import testdriver from 'testdriverai/vitest';
722
-
723
- export default defineConfig({
724
- plugins: [
725
- testdriver({
726
- mode: 'hybrid', // Plugin handles sandbox + recording, we handle launch
727
- os: 'linux',
728
-
729
- dashcam: {
730
- scope: 'file', // One recording per file
731
- logs: [
732
- { type: 'file', path: '/tmp/vscode.log', name: 'VSCode Log' },
733
- { type: 'file', path: '/tmp/extension.log', name: 'Extension Log' }
734
- ]
735
- }
736
- })
737
- ]
738
- });
739
-
740
- // tests/extension.test.ts
741
- import { describe, beforeAll, it } from 'vitest';
742
- import { useTestDriver } from 'testdriverai/vitest';
743
-
744
- describe('VSCode Extension', () => {
745
- const td = useTestDriver({ autoLaunch: false }); // Disable auto-launch
746
-
747
- beforeAll(async () => {
748
- // Custom: Install extension
749
- await td.exec('sh', 'code --install-extension ./my-extension.vsix', 30000);
750
-
751
- // Custom: Launch VSCode with workspace
752
- await td.exec('sh', 'code /workspace');
753
- await td.wait(5000);
754
- });
755
-
756
- it('extension activates', async () => {
757
- await td.focusApplication('Code');
758
-
759
- // Open command palette
760
- await td.pressKeys(['ctrl', 'shift', 'p']);
761
-
762
- // Search for extension command
763
- await td.type('My Extension: Run');
764
- await td.pressKeys(['enter']);
765
-
766
- expect(await td.assert('Extension output is visible')).toBe(true);
767
- });
768
-
769
- // Plugin auto-handles: sandbox cleanup, dashcam stop, test recording
770
- });
771
- ```
772
-
773
- ### 5.3 Standalone SDK (No Plugin)
774
-
775
- **Goal**: Use SDK in non-Vitest context (scripts, other frameworks)
776
-
777
- ```typescript
778
- // scripts/test-production.ts
779
- import { TestDriver, Dashcam } from 'testdriverai';
780
-
781
- async function testProduction() {
782
- const td = new TestDriver(process.env.TD_API_KEY, {
783
- os: 'linux',
784
- resolution: '1920x1080'
785
- });
786
-
787
- await td.auth();
788
- await td.connect();
789
-
790
- const dashcam = new Dashcam(td);
791
- await dashcam.auth();
792
- await dashcam.start();
793
-
794
- try {
795
- // Launch browser
796
- await td.exec('sh', 'google-chrome --start-maximized https://myapp.com');
797
- await td.wait(3000);
798
-
799
- // Run test
800
- const loginBtn = await td.find('Login button');
801
- await loginBtn.click();
802
-
803
- const result = await td.assert('Login form is visible');
804
- console.log('Test passed:', result);
805
-
806
- } finally {
807
- const replayUrl = await dashcam.stop();
808
- console.log('Replay:', replayUrl);
809
- await td.disconnect();
810
- }
811
- }
812
-
813
- testProduction();
814
- ```
815
-
816
- ### 5.4 Multiple Applications in One Test
817
-
818
- **Goal**: Test integration between Chrome and VSCode
819
-
820
- ```typescript
821
- describe('Integration Test', () => {
822
- const td = useTestDriver({ autoLaunch: false });
823
-
824
- it('copy code from browser to vscode', async () => {
825
- // Launch Chrome
826
- await td.exec('sh', 'google-chrome https://github.com');
827
- await td.focusApplication('Google Chrome');
828
-
829
- // Find code snippet
830
- const code = await td.find('Code block');
831
- await code.click();
832
- await td.pressKeys(['ctrl', 'c']); // Copy
833
-
834
- // Switch to VSCode
835
- await td.exec('sh', 'code');
836
- await td.focusApplication('Code');
837
-
838
- // Paste code
839
- await td.pressKeys(['ctrl', 'v']);
840
-
841
- expect(await td.assert('Code is pasted in editor')).toBe(true);
842
- });
843
- });
844
- ```
845
-
846
- ### 5.5 Custom Preset Example
847
-
848
- **Goal**: Create reusable preset for internal tool
849
-
850
- ```typescript
851
- // presets/salesforce.ts
852
- import { ApplicationPreset } from 'testdriverai';
853
-
854
- export const salesforcePreset = (options: {
855
- instanceUrl: string;
856
- username?: string;
857
- password?: string;
858
- }): ApplicationPreset => ({
859
- name: 'Salesforce',
860
-
861
- async afterConnect(client) {
862
- // Launch Chrome with Salesforce URL
863
- const url = `${options.instanceUrl}/lightning`;
864
- await client.exec('sh', `google-chrome --start-maximized "${url}"`);
865
- await client.wait(3000);
866
- },
867
-
868
- async beforeTest(client) {
869
- // Auto-login if credentials provided
870
- if (options.username && options.password) {
871
- await client.focusApplication('Google Chrome');
872
-
873
- const usernameField = await client.find('Username field');
874
- await usernameField.click();
875
- await client.type(options.username);
876
-
877
- const passwordField = await client.find('Password field');
878
- await passwordField.click();
879
- await client.type(options.password);
880
-
881
- const loginBtn = await client.find('Log In button');
882
- await loginBtn.click();
883
-
884
- await client.wait(5000); // Wait for dashboard
885
- }
886
- },
887
-
888
- dashcam: {
889
- scope: 'test',
890
- logs: [
891
- {
892
- type: 'file',
893
- path: '/tmp/salesforce-console.log',
894
- name: 'Browser Console'
895
- }
896
- ]
897
- }
898
- });
899
-
900
- // Use it
901
- import { salesforcePreset } from './presets/salesforce';
902
-
903
- testdriver({
904
- preset: salesforcePreset({
905
- instanceUrl: 'https://mycompany.salesforce.com',
906
- username: process.env.SF_USERNAME,
907
- password: process.env.SF_PASSWORD
908
- })
909
- })
910
- ```
911
-
912
- ---
913
-
914
- ## Migration Path
915
-
916
- ### 6.1 Current State → Optimal State
917
-
918
- **Phase 1: Immediate Improvements (Week 1-2)**
919
- - ✅ Extract dashcam logic into separate `Dashcam` class
920
- - ✅ Create `useTestDriver()` hook for cleaner test code
921
- - ✅ Add preset system with `chromePreset` as first preset
922
- - ✅ Document lifecycle modes (auto/manual/hybrid)
923
-
924
- **Phase 2: Plugin Enhancement (Week 3-4)**
925
- - ✅ Implement auto-lifecycle mode in plugin
926
- - ✅ Add configuration file support (`testdriver.config.ts`)
927
- - ✅ Create helper function library
928
- - ✅ Add hooks system (onBeforeConnect, etc.)
929
-
930
- **Phase 3: Preset Ecosystem (Week 5-6)**
931
- - ✅ Build VSCode preset
932
- - ✅ Build Figma/design tool presets
933
- - ✅ Create preset builder API
934
- - ✅ Documentation for custom presets
935
-
936
- **Phase 4: DX Polish (Week 7-8)**
937
- - ✅ Full TypeScript definitions
938
- - ✅ Better error messages with context
939
- - ✅ CLI tool for generating presets
940
- - ✅ Example repository with common patterns
941
-
942
- ### 6.2 Backward Compatibility
943
-
944
- All current code continues to work:
945
-
946
- ```typescript
947
- // OLD WAY (still works)
948
- import { createTestClient, setupTest, teardownTest } from './testHelpers';
949
-
950
- beforeEach(async (ctx) => {
951
- testdriver = createTestClient({ task: ctx.task });
952
- await setupTest(testdriver);
953
- });
954
-
955
- // NEW WAY (recommended)
956
- import { useTestDriver } from 'testdriverai/vitest';
957
-
958
- const td = useTestDriver(); // Auto-configured
959
- ```
960
-
961
- ### 6.3 Deprecation Timeline
962
-
963
- - **Now - 3 months**: Both old and new APIs supported
964
- - **3-6 months**: Old API marked as deprecated (warnings)
965
- - **6+ months**: Old API removed (major version bump)
966
-
967
- ---
968
-
969
- ## Benefits Summary
970
-
971
- ### For 90% of Users (Quick Start)
972
- ```typescript
973
- // One line in config
974
- testdriver({ preset: 'chrome', launch: { url: 'https://myapp.com' } })
975
-
976
- // Clean test code
977
- const td = useTestDriver();
978
- await td.find('Button').click();
979
- ```
980
-
981
- **Benefits:**
982
- - ✅ Get started in <5 minutes
983
- - ✅ No boilerplate code
984
- - ✅ Automatic dashcam recording
985
- - ✅ Built-in test reporting
986
- - ✅ Smart defaults for everything
987
-
988
- ### For 10% of Users (Advanced)
989
- ```typescript
990
- // Full control when needed
991
- const td = createTestDriver();
992
- const dashcam = createDashcam(td, { logs: [...] });
993
- await customSetup();
994
- ```
995
-
996
- **Benefits:**
997
- - ✅ Granular control over lifecycle
998
- - ✅ Custom application support
999
- - ✅ Complex multi-app scenarios
1000
- - ✅ Reusable presets for internal tools
1001
- - ✅ Integration with any framework
1002
-
1003
- ### For SDK Maintainers
1004
- - ✅ Clear separation of concerns (SDK vs Plugin vs Presets)
1005
- - ✅ Easy to test each layer independently
1006
- - ✅ Simple to add new presets without touching core
1007
- - ✅ Better documentation structure
1008
- - ✅ Easier onboarding for contributors
1009
-
1010
- ---
1011
-
1012
- ## Appendix
1013
-
1014
- ### A. File Structure
1015
-
1016
- ```
1017
- testdriverai/
1018
- ├── src/
1019
- │ ├── core/
1020
- │ │ ├── TestDriver.ts # Core SDK class
1021
- │ │ ├── Element.ts # Element class
1022
- │ │ ├── Dashcam.ts # Dashcam module
1023
- │ │ └── types.ts # Core types
1024
- │ ├── vitest/
1025
- │ │ ├── plugin.ts # Vitest plugin
1026
- │ │ ├── hooks.ts # useTestDriver, etc.
1027
- │ │ ├── lifecycle.ts # Auto lifecycle
1028
- │ │ └── reporter.ts # Test recording
1029
- │ ├── presets/
1030
- │ │ ├── index.ts # Preset registry
1031
- │ │ ├── chrome.ts # Chrome preset
1032
- │ │ ├── vscode.ts # VSCode preset
1033
- │ │ ├── types.ts # Preset types
1034
- │ │ └── builder.ts # Preset builder
1035
- │ ├── helpers/
1036
- │ │ ├── auth.ts # Auth helpers
1037
- │ │ ├── forms.ts # Form helpers
1038
- │ │ ├── wait.ts # Wait utilities
1039
- │ │ └── retry.ts # Retry logic
1040
- │ └── config/
1041
- │ ├── loader.ts # Config file loader
1042
- │ ├── schema.ts # Config validation
1043
- │ └── defaults.ts # Default values
1044
- ├── examples/
1045
- │ ├── quickstart/
1046
- │ ├── advanced/
1047
- │ ├── custom-preset/
1048
- │ └── multi-app/
1049
- └── docs/
1050
- ├── getting-started.md
1051
- ├── vitest-plugin.md
1052
- ├── presets.md
1053
- └── api-reference.md
1054
- ```
1055
-
1056
- ### B. Package Exports
1057
-
1058
- ```json
1059
- {
1060
- "name": "testdriverai",
1061
- "exports": {
1062
- ".": "./dist/index.js",
1063
- "./vitest": "./dist/vitest/index.js",
1064
- "./presets": "./dist/presets/index.js",
1065
- "./helpers": "./dist/helpers/index.js",
1066
- "./config": "./dist/config/index.js"
1067
- }
1068
- }
1069
- ```
1070
-
1071
- ### C. TypeScript Definitions
1072
-
1073
- ```typescript
1074
- // Main exports
1075
- export { TestDriver, Dashcam, Element } from './core';
1076
- export type { TestDriverOptions, DashcamOptions, LogConfig } from './core/types';
1077
-
1078
- // Vitest exports
1079
- export { useTestDriver, useDashcam, onBeforeConnect } from './vitest';
1080
- export type { VitestPluginOptions } from './vitest/types';
1081
-
1082
- // Preset exports
1083
- export { chromePreset, vscodePreset, definePreset } from './presets';
1084
- export type { ApplicationPreset, PresetOptions } from './presets/types';
1085
-
1086
- // Helper exports
1087
- export * from './helpers';
1088
-
1089
- // Config exports
1090
- export { defineConfig } from './config';
1091
- export type { TestDriverConfig } from './config/types';
1092
- ```
1093
-
1094
- ---
1095
-
1096
- ---
1097
-
1098
- ## Design Principles: Avoiding Over-Abstraction
1099
-
1100
- ### Direct Access Always Available
1101
-
1102
- **Critical principle**: Every layer is **optional sugar** over the core SDK. You can always drop down to lower levels.
1103
-
1104
- ```typescript
1105
- // Layer 3: Preset (highest abstraction)
1106
- testdriver({ preset: 'chrome', launch: { url: 'https://myapp.com' } })
1107
-
1108
- // Layer 2: Plugin hook (medium abstraction)
1109
- const td = useTestDriver();
1110
-
1111
- // Layer 1: Direct SDK (no abstraction)
1112
- const td = new TestDriver(apiKey);
1113
- await td.auth();
1114
- await td.connect();
1115
-
1116
- // ALL THREE GIVE YOU THE SAME TestDriver INSTANCE
1117
- // You can call td.find(), td.exec(), etc. the same way
1118
- ```
1119
-
1120
- ### No Hidden Magic
1121
-
1122
- Every convenience method is just a thin wrapper that you could write yourself:
1123
-
1124
- ```typescript
1125
- // useTestDriver() is just:
1126
- export function useTestDriver(options = {}) {
1127
- let client;
1128
-
1129
- beforeAll(async () => {
1130
- client = new TestDriver(process.env.TD_API_KEY);
1131
- await client.auth();
1132
- await client.connect();
1133
- });
1134
-
1135
- afterAll(async () => {
1136
- await client.disconnect();
1137
- });
1138
-
1139
- return client; // Returns the actual SDK instance!
1140
- }
1141
-
1142
- // Dashcam is just a standalone class:
1143
- const dashcam = new Dashcam(td); // Takes TestDriver instance
1144
- await dashcam.start(); // Calls td.exec() under the hood
1145
- ```
1146
-
1147
- ### Escape Hatches Everywhere
1148
-
1149
- You can mix and match abstraction levels:
1150
-
1151
- ```typescript
1152
- describe('Mix and Match', () => {
1153
- // Use plugin for convenience
1154
- const td = useTestDriver();
1155
-
1156
- // But also create manual Dashcam for special config
1157
- let customDashcam;
1158
-
1159
- beforeAll(async () => {
1160
- customDashcam = new Dashcam(td, {
1161
- logs: [/* complex config */]
1162
- });
1163
- await customDashcam.auth();
1164
- });
1165
-
1166
- it('test', async () => {
1167
- // Start custom dashcam
1168
- await customDashcam.start();
1169
-
1170
- // Use td directly - it's just the SDK instance
1171
- await td.exec('sh', 'custom-command');
1172
- const el = await td.find('Button');
1173
- await el.click();
1174
-
1175
- // Stop custom dashcam
1176
- const url = await customDashcam.stop();
1177
- });
1178
- });
1179
- ```
1180
-
1181
- ### Programmatic Invocation Examples
1182
-
1183
- **1. Pure SDK - No Plugin/Preset (Zero Abstraction)**
1184
-
1185
- ```typescript
1186
- import { TestDriver, Dashcam } from 'testdriverai';
1187
-
1188
- const td = new TestDriver(process.env.TD_API_KEY, { os: 'linux' });
1189
- await td.auth();
1190
- await td.connect();
1191
-
1192
- const dashcam = new Dashcam(td);
1193
- await dashcam.auth();
1194
- await dashcam.addFileLog('/tmp/app.log', 'App Log');
1195
- await dashcam.start();
1196
-
1197
- // Do your testing
1198
- await td.exec('sh', 'google-chrome https://myapp.com');
1199
- const btn = await td.find('Login button');
1200
- await btn.click();
1201
-
1202
- const replayUrl = await dashcam.stop();
1203
- await td.disconnect();
1204
- ```
1205
-
1206
- **2. Programmatic with Helpers (Minimal Abstraction)**
1207
-
1208
- ```typescript
1209
- import { TestDriver, Dashcam } from 'testdriverai';
1210
- import { launchChrome, waitForElement } from 'testdriverai/helpers';
1211
-
1212
- const td = new TestDriver(apiKey);
1213
- await td.auth();
1214
- await td.connect();
1215
-
1216
- // Helpers are just functions, you can read their source
1217
- await launchChrome(td, 'https://myapp.com');
1218
- await waitForElement(td, 'Page loaded', 30000);
1219
-
1220
- // Still calling SDK directly
1221
- const el = await td.find('Button');
1222
- await el.click();
1223
- ```
1224
-
1225
- **3. Plugin Mode - Still Direct Access**
1226
-
1227
- ```typescript
1228
- import { useTestDriver } from 'testdriverai/vitest';
1229
-
1230
- describe('Tests', () => {
1231
- const td = useTestDriver(); // Returns TestDriver instance
1232
-
1233
- it('test', async () => {
1234
- // Access underlying SDK at any time
1235
- console.log(td.getSessionId()); // SDK method
1236
- console.log(td.getInstance()); // SDK method
1237
-
1238
- // Access internal agent if needed
1239
- const emitter = td.getEmitter();
1240
- emitter.on('command:start', (cmd) => {
1241
- console.log('Command:', cmd);
1242
- });
1243
-
1244
- // Call SDK methods directly
1245
- await td.exec('sh', 'custom-script.sh');
1246
- });
1247
- });
1248
- ```
1249
-
1250
- ### Implementation Strategy: Thin Wrappers Only
1251
-
1252
- ```typescript
1253
- // BAD: Thick abstraction that hides functionality
1254
- class ChromePresetManager {
1255
- private internalClient;
1256
- private internalDashcam;
1257
-
1258
- async click(selector) {
1259
- // Lost access to TestDriver!
1260
- }
1261
- }
1262
-
1263
- // GOOD: Thin wrapper that returns SDK instance
1264
- function chromePreset(options) {
1265
- return {
1266
- name: 'chrome',
1267
- async afterConnect(client) { // Receives TestDriver instance
1268
- // Just runs commands on the client
1269
- await client.exec('sh', `chrome ${options.url}`);
1270
- // Client is still fully accessible to user
1271
- }
1272
- };
1273
- }
1274
- ```
1275
-
1276
- ### Abstraction Layers Summary
1277
-
1278
- ```
1279
- ┌─────────────────────────────────────────────┐
1280
- │ Presets: chromePreset({ url }) │ ← Sugar
1281
- │ - Returns lifecycle hooks │
1282
- │ - Hooks receive TestDriver instance │
1283
- └─────────────────────────────────────────────┘
1284
-
1285
- ┌─────────────────────────────────────────────┐
1286
- │ Plugin Hooks: useTestDriver() │ ← Sugar
1287
- │ - Handles beforeAll/afterAll │
1288
- │ - Returns TestDriver instance │
1289
- └─────────────────────────────────────────────┘
1290
-
1291
- ┌─────────────────────────────────────────────┐
1292
- │ Helpers: launchChrome(td, url) │ ← Sugar
1293
- │ - Just functions that call TD methods │
1294
- │ - Fully transparent │
1295
- └─────────────────────────────────────────────┘
1296
-
1297
- ┌─────────────────────────────────────────────┐
1298
- │ Core SDK: new TestDriver() │ ← Reality
1299
- │ - Everything else is built on this │
1300
- │ - Always directly accessible │
1301
- │ - No hidden state │
1302
- └─────────────────────────────────────────────┘
1303
- ```
1304
-
1305
- ### Rules to Prevent Over-Abstraction
1306
-
1307
- 1. **Never hide the SDK instance** - All wrappers must expose the underlying TestDriver
1308
- 2. **No proxy patterns** - Don't intercept SDK method calls
1309
- 3. **Stateless helpers** - Helper functions take SDK instance as parameter
1310
- 4. **Transparent presets** - Presets are just lifecycle hooks with SDK access
1311
- 5. **Composable, not hierarchical** - Mix core SDK + helpers + plugin as needed
1312
-
1313
- ### When to Use What
1314
-
1315
- ```typescript
1316
- // Script or non-Vitest framework? Use core SDK
1317
- const td = new TestDriver(apiKey);
1318
-
1319
- // Want some helper functions? Import them
1320
- import { launchChrome } from 'testdriverai/helpers';
1321
- await launchChrome(td, url);
1322
-
1323
- // Using Vitest and want lifecycle management? Use plugin
1324
- const td = useTestDriver();
1325
- // Still get full SDK access!
1326
-
1327
- // Want preset configuration? Use it
1328
- testdriver({ preset: chromePreset({ url }) });
1329
- // Preset just runs commands on your SDK instance
1330
- ```
1331
-
1332
- ---
1333
-
1334
- ## Conclusion
1335
-
1336
- This design provides:
1337
-
1338
- 1. **Simple for beginners**: One-line setup for common cases
1339
- 2. **Powerful for experts**: Full control when needed - **no abstraction lock-in**
1340
- 3. **Maintainable**: Clear separation of concerns
1341
- 4. **Extensible**: Easy to add new presets and helpers
1342
- 5. **Type-safe**: Full TypeScript support
1343
- 6. **Well-documented**: Clear examples for every use case
1344
- 7. **Transparent**: Every layer is optional sugar over the core SDK
1345
-
1346
- The key insight is **progressive disclosure** - users start simple and only learn about advanced features when they need them. The preset system and helper functions cover the most common use cases, while the manual mode and hook system provide escape hatches for edge cases.
1347
-
1348
- **Most importantly**: You can always drop down to `new TestDriver()` and call methods directly. The layers don't hide functionality - they just reduce boilerplate for common patterns.