testdriverai 7.1.0 → 7.1.1

package/testdriver/acceptance-sdk/QUICK_REFERENCE.md

@@ -1,61 +0,0 @@
-# SDK Test Reporting - Quick Reference
-
-## 🚀 Commands
-
-```bash
-# Run all tests
-npm run test:sdk
-
-# View terminal summary
-npm run test:sdk:results
-
-# Open HTML report
-npm run test:sdk:report
-
-# Watch mode (dev)
-npm run test:sdk:watch
-
-# Interactive UI
-npm run test:sdk:ui
-```
-
-## 📊 What You Get
-
-### Locally
-
-1. **Console**: Verbose logs with full test output
-2. **Terminal Summary**: `npm run test:sdk:results` - Quick pass/fail counts
-3. **HTML Report**: `npm run test:sdk:report` - Interactive browser viewer
-
-### GitHub Actions
-
-1. **Step Summary**: Markdown tables in workflow summary page
-2. **Test Summary Action**: Badge counts and annotations
-3. **Artifacts**: Download junit.xml, results.json, and index.html
-
-## 📁 Output Files
-
-```
-test-results/
-├── junit.xml       # For CI/CD tools
-├── results.json    # Machine-readable
-└── index.html      # Interactive report
-```
-
-## ⚡ Quick Tips
-
-- **Debugging failures?** → `npm run test:sdk:report` (HTML has best error context)
-- **Quick status check?** → `npm run test:sdk:results` (terminal summary)
-- **PR review?** → Check GitHub Actions summary tab
-- **Need history?** → Download artifacts from GitHub Actions runs
-
-## 🔍 GitHub Summary Preview
-
-Every test run creates a summary with:
-
-- ✅ Pass/fail counts table
-- ❌ Failed test details with errors
-- ✅ List of all passing tests
-- ⏱️ Duration metrics
-
-Find it: Actions → Your workflow run → Summary tab

package/testdriver/acceptance-sdk/README.md

@@ -1,128 +0,0 @@
-# TestDriver SDK Acceptance Tests
-
-This directory contains acceptance tests for the TestDriver SDK using Vitest.
-
-## Running Tests
-
-### Run All Tests (Cross-Platform)
-
-```bash
-npm run test:sdk
-```
-
-### Run Platform-Specific Tests
-
-Use the `TEST_PLATFORM` environment variable to run tests for a specific platform:
-
-```bash
-# Run Windows-only tests
-npm run test:sdk:windows
-
-# Run macOS-only tests
-npm run test:sdk:mac
-
-# Run Linux-only tests
-npm run test:sdk:linux
-```
-
-Or set the environment variable directly:
-
-```bash
-TEST_PLATFORM=windows npm run test:sdk
-TEST_PLATFORM=mac npm run test:sdk
-```
-
-## Test Organization
-
-### Cross-Platform Tests
-
-Tests without a platform suffix run on all platforms:
-
-- `hover-text.test.mjs` - Runs everywhere
-- `scroll.test.mjs` - Runs everywhere
-- `screenshot.test.mjs` - Runs everywhere
-
-### Platform-Specific Tests
-
-Platform-specific tests use naming conventions:
-
-- `*.windows.test.mjs` - Windows-only tests (e.g., `exec-pwsh.windows.test.mjs`)
-- `*.mac.test.mjs` - macOS-only tests
-- `*.linux.test.mjs` - Linux-only tests
-
-### Conditional Test Skipping
-
-Some tests use `skipIf` to conditionally skip based on the platform:
-
-```javascript
-it.skipIf(() => testdriver.os === "linux")(
-  "should run on Windows/Mac",
-  async () => {
-    // This test will be skipped on Linux
-  },
-);
-```
-
-## Environment Variables
-
-- `TEST_PLATFORM` - Filter tests by platform (`windows`, `mac`, `linux`)
-- `TD_OS` - Override the sandbox OS (defaults to `linux`)
-- `TD_API_KEY` - Your TestDriver API key (required)
-- `TD_API_ROOT` - API endpoint (optional)
-- `DEBUG_ENV` - Show environment variable loading (optional)
-- `DEBUG_EVENTS` - Enable detailed event logging (optional)
-
-## Examples
-
-```bash
-# Run only Windows tests on a Windows sandbox
-TEST_PLATFORM=windows npm run test:sdk
-
-# Run all tests but use a Windows sandbox
-TD_OS=windows npm run test:sdk
-
-# Run with debugging enabled
-DEBUG_ENV=true DEBUG_EVENTS=true npm run test:sdk
-
-# Watch mode for development
-npm run test:sdk:watch
-
-# Generate coverage report
-npm run test:sdk:coverage
-```
-
-## Test Structure
-
-Each test follows this pattern:
-
-```javascript
-import { afterEach, beforeEach, describe, it } from "vitest";
-import {
-  createTestClient,
-  setupTest,
-  teardownTest,
-} from "./setup/testHelpers.mjs";
-
-describe("My Test", () => {
-  let testdriver;
-
-  beforeEach(async () => {
-    testdriver = createTestClient();
-    await setupTest(testdriver);
-  });
-
-  afterEach(async () => {
-    await teardownTest(testdriver);
-  });
-
-  it("should do something", async () => {
-    // Your test logic
-  });
-});
-```
-
-## See Also
-
-- [SDK README](../../SDK_README.md) - Full SDK documentation
-- [Quick Reference](./QUICK_REFERENCE.md) - SDK method quick reference
-- [Test Reporting](./TEST_REPORTING.md) - Test recording and reporting docs

package/testdriver/acceptance-sdk/TEST_REPORTING.md

@@ -1,245 +0,0 @@
-# TestDriver SDK Test Reporting
-
-This guide explains the enhanced test reporting system for the TestDriver SDK acceptance tests.
-
-## Overview
-
-The SDK tests use **Vitest** with multiple reporters to provide comprehensive test feedback:
-
-1. **Console Output** - Verbose, detailed logs during test execution
-2. **JUnit XML** - For CI/CD integration and third-party tools
-3. **JSON Results** - Machine-readable format for custom reporting
-4. **HTML Report** - Interactive browser-based test results viewer
-5. **GitHub Summary** - Markdown tables in GitHub Actions workflow summaries
-
-## Running Tests Locally
-
-### Basic Test Run
-
-```bash
-npm run test:sdk
-```
-
-This runs all SDK acceptance tests with verbose output.
-
-### Watch Mode (for development)
-
-```bash
-npm run test:sdk:watch
-```
-
-Re-runs tests automatically when files change.
-
-### Interactive UI
-
-```bash
-npm run test:sdk:ui
-```
-
-Opens Vitest's web UI for interactive test exploration.
-
-### View Results After Running Tests
-
-After running tests, you have several options to view results:
-
-#### Terminal Summary
-
-```bash
-npm run test:sdk:results
-```
-
-Displays a formatted summary in your terminal with:
-
-- ✅ Passed test count
-- ❌ Failed test count and error details
-- ⏱️ Test duration
-- 📁 File-by-file breakdown
-
-#### HTML Report (Best for detailed analysis)
-
-```bash
-npm run test:sdk:report
-```
-
-Opens the interactive HTML report in your browser showing:
-
-- Detailed test execution timeline
-- File-by-file results
-- Error stack traces with code context
-- Test duration metrics
-
-Or manually open: `test-results/index.html`
-
-## GitHub Actions Reporting
-
-When tests run in GitHub Actions, you get enhanced reporting automatically:
-
-### 📊 GitHub Step Summary
-
-The workflow generates a comprehensive summary visible in the Actions run:
-
-- **Overview Table**: Pass/fail counts, duration, and totals
-- **Failed Tests Section**: Each failure with error messages and stack traces
-- **Passed Tests Section**: List of all passing tests organized by file
-
-To view: Go to the Actions tab → Select your workflow run → Check the "Summary" section
-
-### 🧪 Test Summary Action
-
-The `test-summary/action` provides:
-
-- Test count badges
-- Duration metrics
-- Failure annotations in the Files Changed tab
-
-### 📦 Test Artifacts
-
-All test results are uploaded as artifacts (retained for 7 days):
-
-- `junit.xml` - JUnit format for third-party tools
-- `results.json` - Machine-readable JSON
-- `index.html` - Interactive HTML report (download and open locally)
-
-To download artifacts:
-
-1. Go to the workflow run
-2. Scroll to "Artifacts" section at the bottom
-3. Download `test-results.zip`
-
-## Test Output Files
-
-All test results are saved to the `test-results/` directory:
-
-```
-test-results/
-├── junit.xml       # JUnit XML format
-├── results.json    # Detailed JSON results
-└── index.html      # Interactive HTML report
-```
-
-Add this to your `.gitignore`:
-
-```
-test-results/
-```
-
-## Reporters Explained
-
-### 1. Verbose Reporter (Console)
-
-- Shows full test logs in real-time
-- Includes console.log output from tests
-- Color-coded pass/fail indicators
-- Stack traces for failures
-
-### 2. JUnit Reporter
-
-- Industry-standard XML format
-- Compatible with Jenkins, Azure DevOps, etc.
-- Used by `test-summary/action`
-
-### 3. JSON Reporter
-
-- Complete test results in JSON format
-- Programmatically parseable
-- Used by the custom results viewer script
-
-### 4. HTML Reporter
-
-- Interactive web-based viewer
-- Visual timeline of test execution
-- Filterable and searchable results
-- Best for debugging failures locally
-
-## Customizing Test Output
-
-### Run a Single Test File
-
-```bash
-npx vitest run testdriver/acceptance-sdk/assert.test.mjs
-```
-
-### Enable Even More Verbose Logging
-
-```bash
-VERBOSE=true LOGGING=true npm run test:sdk
-```
-
-### Change Parallelism
-
-Edit `vitest.config.mjs`:
-
-```javascript
-maxForks: 5, // Run 5 tests in parallel instead of 10
-```
-
-## Troubleshooting
-
-### "No test results found" error
-
-Make sure you've run the tests first:
-
-```bash
-npm run test:sdk
-```
-
-### HTML report won't open
-
-Manually navigate to and open `test-results/index.html` in your browser.
-
-### Tests timeout
-
-Increase timeout in `vitest.config.mjs`:
-
-```javascript
-testTimeout: 900000, // 15 minutes
-```
-
-## Best Practices
-
-1. **Use `test:sdk:results`** for quick terminal summaries
-2. **Use `test:sdk:report`** for deep debugging of failures
-3. **Check GitHub Summary** in PR reviews for test status
-4. **Download artifacts** from GitHub Actions for historical analysis
-5. **Run `test:sdk:watch`** during development for fast feedback
-
-## Example GitHub Summary Output
-
-```markdown
-# 🧪 TestDriver SDK Test Results
-
-## 📊 Overview
-
-| Metric      | Count   |
-| ----------- | ------- |
-| ✅ Passed   | 18      |
-| ❌ Failed   | 2       |
-| ⏭️ Skipped  | 0       |
-| 📝 Total    | 20      |
-| ⏱️ Duration | 145.23s |
-
-## ❌ Failed Tests
-
-### Assert Test > should assert the testdriver login page shows
-
-**File:** `testdriver/acceptance-sdk/assert.test.mjs`
-
-**Error:**
-```
-
-AssertionError: expected false to be truthy
-
-```
-
-## ✅ Passed Tests
-
-### type.test.mjs
-- ✅ should type text into input field
-- ✅ should clear and retype text
-
-### scroll.test.mjs
-- ✅ should scroll down the page
-- ✅ should scroll to specific element
-```
-
-This summary appears automatically in every GitHub Actions workflow run!

package/testdriver/acceptance-sdk/assert.test.mjs

@@ -1,26 +0,0 @@
-/**
- * TestDriver SDK - Assert Test (Vitest)
- * Converted from: testdriver/acceptance/assert.yaml
- */
-
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "../../src/vitest/hooks.mjs";
-
-describe("Assert Test", () => {
-  it("should assert the testdriver login page shows", async (context) => {
-    const testdriver = TestDriver(context, { headless: true, newSandbox: true });
-    
-    // provision.chrome() automatically calls ready() and starts dashcam
-    await testdriver.provision.chrome({
-      url: 'http://testdriver-sandbox.vercel.app/login',
-    });
-
-    // Assert the TestDriver.ai Sandbox login page is displayed
-    const result = await testdriver.assert(
-      "the TestDriver.ai Sandbox login page is displayed",
-    );
-
-    expect(result).toBeTruthy();
-  });
-});
-

package/testdriver/acceptance-sdk/hooks-example.test.mjs

@@ -1,38 +0,0 @@
-/**
- * TestDriver Hooks API Demo
- * Demonstrates the new React-style hooks for Vitest
- * 
- * This is the SIMPLEST way to use TestDriver with Vitest!
- */
-
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "../../src/vitest/hooks.mjs";
-
-describe("Hooks API Demo", () => {
-  it("should use hooks for automatic lifecycle management", async (context) => {
-    // ✨ One line gets you TestDriver + Dashcam with full auto-lifecycle!
-    const testdriver = TestDriver(context, { headless: true });
-    await testdriver.provision.chrome({ url: 'http://testdriver-sandbox.vercel.app/login' });
-    
-    // Everything is automatic:
-    // - Sandbox connection ✅
-    // - Dashcam recording ✅
-    // - Chrome launch ✅
-    // - Recording stop ✅
-    // - Cleanup ✅
-    
-    await testdriver.focusApplication("Google Chrome");
-
-    const signInButton = await testdriver.find(
-      "Sign In, black button below the password field",
-    );
-    await signInButton.click();
-
-    const result = await testdriver.assert(
-      "an error shows that fields are required",
-    );
-    expect(result).toBeTruthy();
-    
-    // That's it! Dashcam URL automatically registered with the test run
-  });
-});

package/testdriver/acceptance-sdk/presets-example.test.mjs

@@ -1,87 +0,0 @@
-/**
- * Examples demonstrating TestDriver Presets
- * 
- * Presets provide pre-configured setups for common applications,
- * reducing boilerplate and making tests easier to write.
- */
-
-import { test } from 'vitest';
-import { chromePreset, createPreset, electronPreset, vscodePreset } from '../../src/presets/index.mjs';
-
-test('Chrome preset - simple navigation', async (context) => {
-  const { browser } = await chromePreset(context, {
-    url: 'http://testdriver-sandbox.vercel.app/',
-  });
-
-  // Browser is already open and focused on the URL
-  const result = await browser.find('main heading text');
-  console.log('Found heading:', result);
-}, 60000);
-
-test('Chrome preset - with custom options', async (context) => {
-  const { browser, dashcam } = await chromePreset(context, {
-    url: 'https://google.com',
-    maximized: false,
-    guest: false,
-    dashcam: true, // Dashcam auto-starts and auto-stops
-  });
-
-  await browser.find('search box').type('testdriverai');
-  console.log('Dashcam recording:', dashcam.isRecording());
-}, 60000);
-
-test.skip('VS Code preset - basic usage', async (context) => {
-  const { vscode } = await vscodePreset(context, {
-    workspace: '/tmp/test-project',
-  });
-
-  // VS Code is already open with the workspace loaded
-  await vscode.find('File menu').click();
-  await vscode.find('New File').click();
-}, 60000);
-
-test.skip('VS Code preset - with extensions', async (context) => {
-  const { vscode } = await vscodePreset(context, {
-    workspace: '/tmp/test-project',
-    extensions: ['ms-python.python'],
-    dashcam: true,
-  });
-
-  // Extensions are pre-installed, dashcam auto-recording
-  await vscode.find('Python extension').click();
-}, 120000);
-
-test.skip('Custom preset - Electron app', async (context) => {
-  const { app } = await electronPreset(context, {
-    appPath: '/path/to/electron/app',
-    args: ['--enable-logging'],
-  });
-
-  await app.find('main window').click();
-}, 60000);
-
-test.skip('Custom preset - create your own', async (context) => {
-  // Create a custom preset for your application
-  const firefoxPreset = createPreset({
-    name: 'Firefox Browser',
-    defaults: { os: 'linux', dashcam: true },
-    async setup(context, client, dashcam, options) {
-      const { url } = options;
-      
-      // Launch Firefox
-      await client.exec('sh', `firefox "${url}" >/dev/null 2>&1 &`, 30000);
-      await client.focusApplication('Firefox');
-      
-      return {
-        browser: client,
-      };
-    },
-  });
-
-  // Use your custom preset
-  const { browser } = await firefoxPreset(context, {
-    url: 'https://example.com',
-  });
-
-  await browser.find('page content').click();
-}, 60000);