testdriverai 7.0.0 → 7.1.1

package/testdriver/acceptance-sdk/README.md

@@ -1,128 +0,0 @@
-# TestDriver SDK Acceptance Tests
-
-This directory contains acceptance tests for the TestDriver SDK using Vitest.
-
-## Running Tests
-
-### Run All Tests (Cross-Platform)
-
-```bash
-npm run test:sdk
-```
-
-### Run Platform-Specific Tests
-
-Use the `TEST_PLATFORM` environment variable to run tests for a specific platform:
-
-```bash
-# Run Windows-only tests
-npm run test:sdk:windows
-
-# Run macOS-only tests
-npm run test:sdk:mac
-
-# Run Linux-only tests
-npm run test:sdk:linux
-```
-
-Or set the environment variable directly:
-
-```bash
-TEST_PLATFORM=windows npm run test:sdk
-TEST_PLATFORM=mac npm run test:sdk
-```
-
-## Test Organization
-
-### Cross-Platform Tests
-
-Tests without a platform suffix run on all platforms:
-
-- `hover-text.test.mjs` - Runs everywhere
-- `scroll.test.mjs` - Runs everywhere
-- `screenshot.test.mjs` - Runs everywhere
-
-### Platform-Specific Tests
-
-Platform-specific tests use naming conventions:
-
-- `*.windows.test.mjs` - Windows-only tests (e.g., `exec-pwsh.windows.test.mjs`)
-- `*.mac.test.mjs` - macOS-only tests
-- `*.linux.test.mjs` - Linux-only tests
-
-### Conditional Test Skipping
-
-Some tests use `skipIf` to conditionally skip based on the platform:
-
-```javascript
-it.skipIf(() => testdriver.os === "linux")(
-  "should run on Windows/Mac",
-  async () => {
-    // This test will be skipped on Linux
-  },
-);
-```
-
-## Environment Variables
-
-- `TEST_PLATFORM` - Filter tests by platform (`windows`, `mac`, `linux`)
-- `TD_OS` - Override the sandbox OS (defaults to `linux`)
-- `TD_API_KEY` - Your TestDriver API key (required)
-- `TD_API_ROOT` - API endpoint (optional)
-- `DEBUG_ENV` - Show environment variable loading (optional)
-- `DEBUG_EVENTS` - Enable detailed event logging (optional)
-
-## Examples
-
-```bash
-# Run only Windows tests on a Windows sandbox
-TEST_PLATFORM=windows npm run test:sdk
-
-# Run all tests but use a Windows sandbox
-TD_OS=windows npm run test:sdk
-
-# Run with debugging enabled
-DEBUG_ENV=true DEBUG_EVENTS=true npm run test:sdk
-
-# Watch mode for development
-npm run test:sdk:watch
-
-# Generate coverage report
-npm run test:sdk:coverage
-```
-
-## Test Structure
-
-Each test follows this pattern:
-
-```javascript
-import { afterEach, beforeEach, describe, it } from "vitest";
-import {
-  createTestClient,
-  setupTest,
-  teardownTest,
-} from "./setup/testHelpers.mjs";
-
-describe("My Test", () => {
-  let testdriver;
-
-  beforeEach(async () => {
-    testdriver = createTestClient();
-    await setupTest(testdriver);
-  });
-
-  afterEach(async () => {
-    await teardownTest(testdriver);
-  });
-
-  it("should do something", async () => {
-    // Your test logic
-  });
-});
-```
-
-## See Also
-
-- [SDK README](../../SDK_README.md) - Full SDK documentation
-- [Quick Reference](./QUICK_REFERENCE.md) - SDK method quick reference
-- [Test Reporting](./TEST_REPORTING.md) - Test recording and reporting docs

package/testdriver/acceptance-sdk/TEST_REPORTING.md

@@ -1,245 +0,0 @@
-# TestDriver SDK Test Reporting
-
-This guide explains the enhanced test reporting system for the TestDriver SDK acceptance tests.
-
-## Overview
-
-The SDK tests use **Vitest** with multiple reporters to provide comprehensive test feedback:
-
-1. **Console Output** - Verbose, detailed logs during test execution
-2. **JUnit XML** - For CI/CD integration and third-party tools
-3. **JSON Results** - Machine-readable format for custom reporting
-4. **HTML Report** - Interactive browser-based test results viewer
-5. **GitHub Summary** - Markdown tables in GitHub Actions workflow summaries
-
-## Running Tests Locally
-
-### Basic Test Run
-
-```bash
-npm run test:sdk
-```
-
-This runs all SDK acceptance tests with verbose output.
-
-### Watch Mode (for development)
-
-```bash
-npm run test:sdk:watch
-```
-
-Re-runs tests automatically when files change.
-
-### Interactive UI
-
-```bash
-npm run test:sdk:ui
-```
-
-Opens Vitest's web UI for interactive test exploration.
-
-### View Results After Running Tests
-
-After running tests, you have several options to view results:
-
-#### Terminal Summary
-
-```bash
-npm run test:sdk:results
-```
-
-Displays a formatted summary in your terminal with:
-
-- ✅ Passed test count
-- ❌ Failed test count and error details
-- ⏱️ Test duration
-- 📁 File-by-file breakdown
-
-#### HTML Report (Best for detailed analysis)
-
-```bash
-npm run test:sdk:report
-```
-
-Opens the interactive HTML report in your browser showing:
-
-- Detailed test execution timeline
-- File-by-file results
-- Error stack traces with code context
-- Test duration metrics
-
-Or manually open: `test-results/index.html`
-
-## GitHub Actions Reporting
-
-When tests run in GitHub Actions, you get enhanced reporting automatically:
-
-### 📊 GitHub Step Summary
-
-The workflow generates a comprehensive summary visible in the Actions run:
-
-- **Overview Table**: Pass/fail counts, duration, and totals
-- **Failed Tests Section**: Each failure with error messages and stack traces
-- **Passed Tests Section**: List of all passing tests organized by file
-
-To view: Go to the Actions tab → Select your workflow run → Check the "Summary" section
-
-### 🧪 Test Summary Action
-
-The `test-summary/action` provides:
-
-- Test count badges
-- Duration metrics
-- Failure annotations in the Files Changed tab
-
-### 📦 Test Artifacts
-
-All test results are uploaded as artifacts (retained for 7 days):
-
-- `junit.xml` - JUnit format for third-party tools
-- `results.json` - Machine-readable JSON
-- `index.html` - Interactive HTML report (download and open locally)
-
-To download artifacts:
-
-1. Go to the workflow run
-2. Scroll to "Artifacts" section at the bottom
-3. Download `test-results.zip`
-
-## Test Output Files
-
-All test results are saved to the `test-results/` directory:
-
-```
-test-results/
-├── junit.xml       # JUnit XML format
-├── results.json    # Detailed JSON results
-└── index.html      # Interactive HTML report
-```
-
-Add this to your `.gitignore`:
-
-```
-test-results/
-```
-
-## Reporters Explained
-
-### 1. Verbose Reporter (Console)
-
-- Shows full test logs in real-time
-- Includes console.log output from tests
-- Color-coded pass/fail indicators
-- Stack traces for failures
-
-### 2. JUnit Reporter
-
-- Industry-standard XML format
-- Compatible with Jenkins, Azure DevOps, etc.
-- Used by `test-summary/action`
-
-### 3. JSON Reporter
-
-- Complete test results in JSON format
-- Programmatically parseable
-- Used by the custom results viewer script
-
-### 4. HTML Reporter
-
-- Interactive web-based viewer
-- Visual timeline of test execution
-- Filterable and searchable results
-- Best for debugging failures locally
-
-## Customizing Test Output
-
-### Run a Single Test File
-
-```bash
-npx vitest run testdriver/acceptance-sdk/assert.test.mjs
-```
-
-### Enable Even More Verbose Logging
-
-```bash
-VERBOSE=true LOGGING=true npm run test:sdk
-```
-
-### Change Parallelism
-
-Edit `vitest.config.mjs`:
-
-```javascript
-maxForks: 5, // Run 5 tests in parallel instead of 10
-```
-
-## Troubleshooting
-
-### "No test results found" error
-
-Make sure you've run the tests first:
-
-```bash
-npm run test:sdk
-```
-
-### HTML report won't open
-
-Manually navigate to and open `test-results/index.html` in your browser.
-
-### Tests timeout
-
-Increase timeout in `vitest.config.mjs`:
-
-```javascript
-testTimeout: 900000, // 15 minutes
-```
-
-## Best Practices
-
-1. **Use `test:sdk:results`** for quick terminal summaries
-2. **Use `test:sdk:report`** for deep debugging of failures
-3. **Check GitHub Summary** in PR reviews for test status
-4. **Download artifacts** from GitHub Actions for historical analysis
-5. **Run `test:sdk:watch`** during development for fast feedback
-
-## Example GitHub Summary Output
-
-```markdown
-# 🧪 TestDriver SDK Test Results
-
-## 📊 Overview
-
-| Metric      | Count   |
-| ----------- | ------- |
-| ✅ Passed   | 18      |
-| ❌ Failed   | 2       |
-| ⏭️ Skipped  | 0       |
-| 📝 Total    | 20      |
-| ⏱️ Duration | 145.23s |
-
-## ❌ Failed Tests
-
-### Assert Test > should assert the testdriver login page shows
-
-**File:** `testdriver/acceptance-sdk/assert.test.mjs`
-
-**Error:**
-```
-
-AssertionError: expected false to be truthy
-
-```
-
-## ✅ Passed Tests
-
-### type.test.mjs
-- ✅ should type text into input field
-- ✅ should clear and retype text
-
-### scroll.test.mjs
-- ✅ should scroll down the page
-- ✅ should scroll to specific element
-```
-
-This summary appears automatically in every GitHub Actions workflow run!

package/testdriver/acceptance-sdk/assert.test.mjs

@@ -1,44 +0,0 @@
-/**
- * TestDriver SDK - Assert Test (Vitest)
- * Converted from: testdriver/acceptance/assert.yaml
- */
-
-import { afterEach, beforeEach, describe, expect, it } from "vitest";
-import {
-  createTestClient,
-  setupTest,
-  teardownTest,
-} from "./setup/testHelpers.mjs";
-
-describe("Assert Test", () => {
-  let testdriver;
-
-  beforeEach(async (context) => {
-    // Create a new client for each test
-    testdriver = createTestClient({
-      task: context.task,
-    });
-
-    await setupTest(testdriver);
-  }, 600000);
-
-  afterEach(async (context) => {
-    // Teardown after each test, passing the individual test context
-    const sessionInfo = await teardownTest(testdriver, {
-      task: context.task,
-    });
-    console.log(
-      `[Test] Teardown complete, dashcam URL: ${sessionInfo.dashcamUrl}`,
-    );
-  });
-
-  it("should assert the testdriver login page shows", async () => {
-    // Assert the TestDriver.ai Sandbox login page is displayed
-    // const result = await testdriver.assert(
-    //   "the TestDriver.ai Sandbox login page is displayed",
-    // );
-
-    // expect(result).toBeTruthy();
-    return true;
-  });
-});

package/testdriver/acceptance-sdk/scroll-until-text.test.mjs

@@ -1,42 +0,0 @@
-/**
- * TestDriver SDK - Scroll Until Text Test (Vitest)
- * Converted from: testdriver/acceptance/scroll-until-text.yaml
- */
-
-import { afterEach, beforeEach, describe, expect, it } from "vitest";
-import {
-  createTestClient,
-  performLogin,
-  setupTest,
-  teardownTest,
-} from "./setup/testHelpers.mjs";
-
-describe("Scroll Until Text Test", () => {
-  let testdriver;
-
-  beforeEach(async (context) => {
-    testdriver = createTestClient({ task: context.task });
-
-    await setupTest(testdriver);
-  });
-
-  afterEach(async (context) => {
-    await teardownTest(testdriver, { task: context.task });
-  });
-
-  it('should scroll until "testdriver socks" appears', async () => {
-    // Perform login first
-    await performLogin(testdriver);
-
-    // Scroll until text appears
-    await testdriver.focusApplication("Google Chrome");
-    await testdriver.scrollUntilText("testdriver socks", "down");
-
-    // Assert testdriver socks appears on screen
-    await testdriver.focusApplication("Google Chrome");
-    const result = await testdriver.assert(
-      "TestDriver Socks appears on screen",
-    );
-    expect(result).toBeTruthy();
-  });
-});

package/testdriver/acceptance-sdk/setup/lifecycleHelpers.mjs

@@ -1,239 +0,0 @@
-/**
- * Lifecycle Helpers
- * Shared lifecycle hook functions (prerun, postrun)
- */
-
-/**
- * Authenticate dashcam with API key
- * @param {TestDriver} client - TestDriver client
- * @param {string} apiKey - Dashcam API key (default: 4e93d8bf-3886-4d26-a144-116c4063522d)
- */
-export async function authDashcam(
-  client,
-  apiKey = "4e93d8bf-3886-4d26-a144-116c4063522d",
-) {
-  if (client.os === "windows") {
-    const shell = "pwsh";
-
-    let debug = await client.exec(
-      shell,
-      `query session`,
-      40000,
-      true,
-    );
-
-    console.log("Debug version output:", debug);
-
-    let installDashcam = await client.exec(
-      shell,
-      `npm install dashcam@beta -g`,
-      120000,
-      true,
-    );
-    console.log("Install dashcam output:", installDashcam);
-
-    let dashcamVersion = await client.exec(shell, `npm ls dashcam -g`, 40000, true);
-    console.log("Dashcam version:", dashcamVersion);
-
-    const authOutput = await client.exec(shell, `dashcam auth ${apiKey}`, 120000, true);
-    console.log("Auth output:", authOutput);
-    return;
-  }
-  
-  const shell = "sh";
-  const authOutput = await client.exec(shell, `dashcam auth ${apiKey}`, 120000, true);
-  console.log("Auth output:", authOutput);
-}
-
-/**
- * Add log file tracking to dashcam
- * @param {TestDriver} client - TestDriver client
- * @param {string} logName - Name for the log in dashcam (default: "TestDriver Log")
- */
-export async function addDashcamLog(client, logName = "TestDriver Log") {
-  if (client.os === "windows") {
-    const shell = "pwsh";
-    const logPath = "C:\\Users\\testdriver\\Documents\\testdriver.log";
-    const createFileOutput = await client.exec(
-      shell,
-      `New-Item -ItemType File -Path "${logPath}" -Force`,
-      10000,
-      true,
-    );
-    console.log("Create log file output:", createFileOutput);
-    const addLogOutput = await client.exec(
-      shell,
-      `dashcam logs --add --type=file --file="${logPath}" --name="${logName}"`,
-      10000,
-      true,
-    );
-    console.log("Add log tracking output:", addLogOutput);
-    return;
-  }
-
-  const shell = "sh";
-  const logPath = "/tmp/testdriver.log";
-
-  // Create log file
-  await client.exec(shell, `touch ${logPath}`, 10000, true);
-
-  // Add log tracking
-  const addLogOutput = await client.exec(
-    shell,
-    `dashcam logs --add --type=file --file="${logPath}" --name="${logName}"`,
-    10000,
-    true,
-  );
-  console.log("Add log tracking output:", addLogOutput);
-}
-
-/**
- * Start dashcam recording
- * @param {TestDriver} client - TestDriver client
- */
-export async function startDashcam(client) {
-  if (client.os === "windows") {
-    console.log("Starting dashcam recording on Windows...");
-    
-    // Start dashcam using Start-Process without output redirection
-    // Let dashcam handle its own logging
-    const startScript = `Start-Process "cmd.exe" -ArgumentList "/c", "dashcam record"`;
-    
-    const startOutput = await client.exec("pwsh", startScript, 10000);
-    console.log("Start process output:", startOutput);
-    
-    console.log("✅ Dashcam recording started");
-    return;
-  }
-  
-  const shell = "sh";
-  await client.exec(shell, `dashcam record >/dev/null 2>&1 &`);
-}
-
-/**
- * Stop dashcam recording and retrieve URL
- * @param {TestDriver} client - TestDriver client
- * @returns {Promise<string|null>} Dashcam URL if available
- */
-export async function stopDashcam(client) {
-  console.log("🎬 Stopping dashcam and retrieving URL...");
-
-  if (client.os === "windows") {
-    console.log("Stopping dashcam process on Windows...");
-    
-    let stop = await client.exec(
-      "pwsh",
-      `Start-Process "cmd.exe" -ArgumentList "/c", "dashcam stop"`, 10000);
-
-
-    console.log("📤 Dashcam stop command output:", stop);
-
-    let urlData = stop;
-    
-    // Extract URL from output
-    if (urlData) {
-      const urlMatch = urlData.match(/https?:\/\/[^\s"',}]+/);
-      if (urlMatch) {
-        const url = urlMatch[0];
-        console.log("✅ Found dashcam URL:", url);
-        return url;
-      } else {
-        console.warn("⚠️  No URL found in dashcam config");
-      }
-    }
-    return null;
-  }
-  
-  const shell = "sh";
-  // On non-Windows, use regular stop command
-  const output = await client.exec(shell, "dashcam stop", 60000, false);
-  
-  console.log("📤 Dashcam command output:", output);
-
-  // Extract URL from output
-  if (output) {
-    const urlMatch = output.match(/https?:\/\/[^\s"']+/);
-    if (urlMatch) {
-      const url = urlMatch[0];
-      console.log("✅ Found dashcam URL:", url);
-      return url;
-    } else {
-      console.warn("⚠️  No URL found in dashcam output");
-    }
-  } else {
-    console.warn("⚠️  Dashcam command returned no output");
-  }
-
-  return null;
-}
-
-/**
- * Launch Chrome browser with guest mode
- * @param {TestDriver} client - TestDriver client
- * @param {string} url - URL to open (default: https://testdriver-sandbox.vercel.app/)
- */
-export async function launchChrome(
-  client,
-  url = "http://testdriver-sandbox.vercel.app/",
-) {
-  const shell = client.os === "windows" ? "pwsh" : "sh";
-
-  if (client.os === "windows") {
-    await client.exec(
-      "pwsh",
-      `Start-Process "C:/Program Files/Google/Chrome/Application/chrome.exe" -ArgumentList "--start-maximized", "--guest", "${url}"`,
-      30000,
-    );
-  } else {
-    await client.exec(
-      shell,
-      `google-chrome --start-maximized --disable-fre --no-default-browser-check --no-first-run --guest "${url}" >/dev/null 2>&1 &`,
-      30000,
-    );
-  }
-}
-
-/**
- * Wait for page to load by polling for text
- * @param {TestDriver} client - TestDriver client
- * @param {string} text - Text to wait for
- * @param {number} maxAttempts - Maximum number of attempts (default: 60)
- * @param {number} pollInterval - Interval between polls in ms (default: 5000)
- */
-export async function waitForPage(
-  client,
-  text,
-  maxAttempts = 60,
-  pollInterval = 5000,
-) {
-  console.log("Waiting for page to load, looking for text:", text);
-  let element;
-  for (let i = 0; i < maxAttempts; i++) {
-    element = await client.find(text);
-    if (element.found()) break;
-    await new Promise((resolve) => setTimeout(resolve, pollInterval));
-  }
-}
-
-/**
- * Run prerun lifecycle hooks
- * Implements lifecycle/prerun.yaml functionality
- * @param {TestDriver} client - TestDriver client
- */
-export async function runPrerun(client) {
-  await authDashcam(client);
-  await addDashcamLog(client);
-  await startDashcam(client);
-  await launchChrome(client);
-  await waitForPage(client, "TestDriver.ai Sandbox");
-}
-
-/**
- * Run postrun lifecycle hooks
- * Implements lifecycle/postrun.yaml functionality
- * @param {TestDriver} client - TestDriver client
- * @returns {Promise<string|null>} Dashcam URL if available
- */
-export async function runPostrun(client) {
-  return await stopDashcam(client);
-}