testdriverai 7.3.11 → 7.3.13

package/ai/skills/testdriver:testdriver-mechanic/SKILL.md

@@ -0,0 +1,165 @@
+---
+name: testdriver:testdriver-mechanic
+description: An expert at creating and refining automated tests using TestDriver.ai
+---
+
+# TestDriver Expert
+
+You are an expert at writing automated tests using the TestDriver library. Your goal is to create robust, reliable tests that verify the functionality of web applications. You work iteratively, verifying your progress at each step.
+
+TestDriver enables computer-use testing through natural language - controlling browsers, desktop apps, and more using AI vision.
+
+## Capabilities
+
+- **Test Creation**: You know how to build tests from scratch using TestDriver MCP tools.
+- **Visual Verification**: You examine screenshots after every action to understand the current context and verify that the test is performing as expected.
+- **Iterative Development**: You don't just write code once; you execute actions, check the screenshots, and refine until the task is fully complete.
+- **Code Generation**: You use `commit` to generate test files from successful command sequences.
+
+## Context and examples
+
+Use this agent when the user asks to:
+
+- "Write a test for X"
+- "Automate this workflow"
+- "Debug why this test is failing"
+- "Check if the login page works"
+
+## Workflow
+
+1. **Analyze**: Understand the user's requirements and the application under test.
+2. **Start Session**: Use `session_start` to provision sandbox and see initial state.
+3. **Explore & Record**: Use `find`, `click`, `type` etc. - examine screenshots after each action.
+4. **Check**: Use `check` after actions to verify they succeeded before moving on.
+5. **Assert**: Use `assert` to verify expected state (gets recorded in test file).
+6. **Commit**: Use `commit` to generate test file from recorded commands.
+7. **Verify**: Use `verify` to run the test from scratch.
+
+## Prerequisites
+
+### API Key Setup
+
+The user **must** have a TestDriver API key set in their environment:
+
+```bash
+# .env file
+TD_API_KEY=your_api_key_here
+```
+
+Get your API key at: **https://console.testdriver.ai/team**
+
+## MCP Tools Reference
+
+### Session Management
+
+| Tool | Description |
+|------|-------------|
+| `session_start` | Start sandbox with browser/app, returns initial screenshot |
+| `session_status` | Check session health, time remaining |
+| `session_extend` | Add more time before expiry |
+
+### Element Interaction
+
+| Tool | Description |
+|------|-------------|
+| `find` | Locate element by description, returns ref |
+| `click` | Click on element ref or coordinates |
+| `find_and_click` | Find and click in one action |
+| `type` | Type text into focused field |
+| `press_keys` | Press keyboard shortcuts |
+| `scroll` | Scroll page |
+
+### Verification
+
+| Tool | Description |
+|------|-------------|
+| `check` | AI analysis of whether action succeeded - use after every action |
+| `assert` | AI-powered boolean assertion (pass/fail for test files) |
+| `exec` | Execute code in sandbox |
+| `screenshot` | Capture screenshot |
+
+### Test Generation
+
+| Tool | Description |
+|------|-------------|
+| `commit` | Write commands to test file |
+| `verify` | Run test file |
+| `get_command_log` | View recorded commands |
+
+## Example Workflow
+
+```
+# 1. Start session
+session_start({ type: "chrome", url: "https://app.example.com/login" })
+→ Screenshot shows login page
+
+# 2. Fill in email
+find_and_click({ description: "email input field" })
+type({ text: "user@example.com" })
+
+# 3. Check if email was entered
+check({ task: "Was the email entered correctly?" })
+→ AI confirms email is in the field
+
+# 4. Fill in password
+find_and_click({ description: "password field" })
+type({ text: "secret123" })
+
+# 5. Submit login
+find_and_click({ description: "Sign In button" })
+
+# 6. Check if login succeeded
+check({ task: "Did the login complete successfully?" })
+→ AI analyzes screen and confirms dashboard is visible
+
+# 7. Assert success (for test file)
+assert({ assertion: "dashboard is visible" })
+
+# 8. Commit to file
+commit({ testFile: "tests/login.test.mjs", testName: "Login Flow" })
+
+# 9. Verify test runs
+verify({ testFile: "tests/login.test.mjs" })
+```
+
+## Visual Feedback
+
+Every tool returns a screenshot showing:
+
+- **Element highlights** - Blue box around found elements
+- **Click markers** - Red dot at click location
+- **Action status** - Success/failure
+- **Session info** - Time remaining
+
+## Best Practices
+
+1. **Use `check` after every action** - Verify actions succeeded before moving on
+2. **Work incrementally** - One action at a time, check the result
+3. **Be specific** - "blue Sign In button in header" > "button"
+4. **Use `check` for verification, `assert` for test files** - `check` gives AI analysis, `assert` gives boolean pass/fail
+5. **Watch session time** - Use `session_extend` if needed
+6. **Commit working sequences** - Don't lose progress
+
+## Error Recovery
+
+### Element Not Found
+
+1. Check screenshot to see actual screen state
+2. Adjust element description
+3. Add timeout: `find({ description: "...", timeout: 10000 })`
+4. Scroll if element is off-screen
+
+### Session Expired
+
+1. `session_start` again
+2. `verify` to replay committed test
+3. Continue from there
+
+## Tips for Agents
+
+1. **Use `check` after every action** - Verify actions succeeded before moving on
+2. **Examine every screenshot** - They show exactly what the sandbox sees
+3. **Be specific with descriptions** - More context = better element finding
+4. **Assert after major actions** - Catches issues early and gets recorded in test files
+5. **Commit often** - Preserve progress
+6. **Use `verify` to validate** - Ensures test works from scratch

package/ai/skills/testdriver:type/SKILL.md

@@ -0,0 +1,357 @@
+---
+name: testdriver:type
+description: Type text into focused input fields
+---
+<!-- Generated from type.mdx. DO NOT EDIT. -->
+
+## Overview
+
+Type text or numbers into the currently focused input field with optional delay between keystrokes.
+
+## Syntax
+
+```javascript
+await testdriver.type(text, options)
+```
+
+## Parameters
+
+<ParamField path="text" type="string | number" required>
+  Text to type (can be a string or number)
+</ParamField>
+
+<ParamField path="options" type="object | number">
+  Typing options (or legacy delay number)
+  
+  <Expandable title="properties">
+    <ParamField path="delay" type="number" default={250}>
+      Delay between keystrokes in milliseconds
+    </ParamField>
+    
+    <ParamField path="secret" type="boolean" default={false}>
+      If `true`, treats text as sensitive data (won't be logged or stored in debug info/dashcam)
+    </ParamField>
+  </Expandable>
+</ParamField>
+
+## Returns
+
+`Promise<void>`
+
+## Examples
+
+### Basic Typing
+
+```javascript
+// Type text
+await testdriver.type('hello@example.com');
+
+// Type numbers
+await testdriver.type(12345);
+
+// Type with custom delay (legacy syntax)
+await testdriver.type('slow typing', 500); // 500ms between each character
+
+// Type with options object
+await testdriver.type('text', { delay: 500 });
+```
+
+### Password/Secret Handling
+
+```javascript
+// ✅ SECURE - Mark as secret to prevent logging
+const passwordField = await testdriver.find('password input');
+await passwordField.click();
+await testdriver.type('MySecureP@ssw0rd', { secret: true });
+// Password NOT logged in dashcam or debug output
+
+// ❌ INSECURE - Password will be logged
+await testdriver.type('MySecureP@ssw0rd');
+// Password appears in logs, dashcam replay, and debug info
+
+// Use secret for any sensitive data
+await testdriver.find('api key input').click();
+await testdriver.type('sk-1234567890abcdef', { secret: true });
+
+await testdriver.find('credit card input').click();
+await testdriver.type('4111111111111111', { secret: true });
+```
+
+<Warning>
+  **Always use `secret: true` for passwords and sensitive data!**
+  
+  Without this option, typed text appears in:
+  - Dashcam video replays
+  - TestDriver logs
+  - Debug screenshots
+  - Error messages
+</Warning>
+
+### Form Filling
+
+```javascript
+// Focus field and type
+const emailField = await testdriver.find('email input');
+await emailField.click();
+await testdriver.type('user@example.com');
+
+// Tab to next field and type
+await testdriver.pressKeys(['tab']);
+await testdriver.type('John Doe');
+
+// Type password securely
+await testdriver.pressKeys(['tab']);
+await testdriver.type('MySecureP@ssw0rd', { secret: true });
+```
+
+### Clearing and Replacing Text
+
+```javascript
+const searchBox = await testdriver.find('search input');
+await searchBox.click();
+
+// Clear existing text
+await testdriver.pressKeys(['ctrl', 'a']); // Select all
+await testdriver.type('new search query');
+```
+
+## Best Practices
+
+<Check>
+  **Focus the field first**
+  
+  Always click the input field or navigate to it before typing:
+  
+  ```javascript
+  const input = await testdriver.find('username input');
+  await input.click();
+  await testdriver.type('testuser');
+  ```
+</Check>
+
+<Check>
+  **Use Tab for navigation**
+  
+  Navigate between fields using Tab instead of clicking each one:
+  
+  ```javascript
+  const firstField = await testdriver.find('first name');
+  await firstField.click();
+  await testdriver.type('John');
+  
+  await testdriver.pressKeys(['tab']);
+  await testdriver.type('Doe');
+  
+  await testdriver.pressKeys(['tab']);
+  await testdriver.type('john@example.com');
+  ```
+</Check>
+
+<Check>
+  **Clear fields before typing**
+  
+  Clear existing content to avoid appending:
+  
+  ```javascript
+  const input = await testdriver.find('search field');
+  await input.click();
+  await testdriver.pressKeys(['ctrl', 'a']); // Select all
+  await testdriver.type('new search');
+  ```
+</Check>
+
+<Warning>
+  **Field must be focused**
+  
+  Typing will only work if an input field is currently focused. If no field is focused, the text may be lost or trigger unexpected keyboard shortcuts.
+</Warning>
+
+## Use Cases
+
+<AccordionGroup>
+  <Accordion title="Login Forms">
+    ```javascript
+    await testdriver.focusApplication('Google Chrome');
+    
+    const usernameField = await testdriver.find('username input');
+    await usernameField.click();
+    await testdriver.type('testuser@example.com');
+    
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('MyP@ssword123', { secret: true });
+    
+    await testdriver.pressKeys(['enter']);
+    ```
+  </Accordion>
+  
+  <Accordion title="Search Fields">
+    ```javascript
+    const searchBox = await testdriver.find('search input');
+    await searchBox.click();
+    await testdriver.type('laptop computers');
+    await testdriver.pressKeys(['enter']);
+    
+    // Wait for results
+    await new Promise(r => setTimeout(r, 2000));
+    ```
+  </Accordion>
+  
+  <Accordion title="Multi-Field Forms">
+    ```javascript
+    // First field
+    const nameField = await testdriver.find('full name input');
+    await nameField.click();
+    await testdriver.type('Jane Smith');
+    
+    // Navigate with Tab
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('jane.smith@example.com');
+    
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('+1-555-0123');
+    
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('123 Main Street');
+    ```
+  </Accordion>
+  
+  <Accordion title="Text Editors">
+    ```javascript
+    const editor = await testdriver.find('text editor area');
+    await editor.click();
+    
+    await testdriver.type('# My Document', 100);
+    await testdriver.pressKeys(['enter', 'enter']);
+    await testdriver.type('This is the first paragraph.', 50);
+    ```
+  </Accordion>
+  
+  <Accordion title="Numeric Input">
+    ```javascript
+    const quantityField = await testdriver.find('quantity input');
+    await quantityField.click();
+    
+    // Clear field
+    await testdriver.pressKeys(['ctrl', 'a']);
+    
+    // Type number
+    await testdriver.type(5);
+    ```
+  </Accordion>
+</AccordionGroup>
+
+## Typing Speed
+
+Adjust the delay parameter based on your needs:
+
+```javascript
+// Fast typing (100ms delay)
+await testdriver.type('quick entry', 100);
+
+// Normal typing (250ms - default)
+await testdriver.type('standard speed');
+
+// Slow typing (500ms delay) - useful for fields with live validation
+await testdriver.type('slow and steady', 500);
+
+// Very slow (1000ms delay) - for problematic fields
+await testdriver.type('one by one', 1000);
+```
+
+<Note>
+  Some applications with live validation or autocomplete may require slower typing speeds to avoid race conditions.
+</Note>
+
+## Special Characters
+
+```javascript
+// Email addresses
+await testdriver.type('user@example.com');
+
+// URLs
+await testdriver.type('https://example.com/path?query=value');
+
+// Passwords with special characters
+await testdriver.type('P@ssw0rd!#$%');
+
+// Paths
+await testdriver.type('C:\\Users\\Documents\\file.txt');
+
+// Multi-line text (use pressKeys for Enter)
+await testdriver.type('Line 1');
+await testdriver.pressKeys(['enter']);
+await testdriver.type('Line 2');
+```
+
+## Complete Example
+
+```javascript
+import { beforeAll, afterAll, describe, it, expect } from 'vitest';
+import TestDriver from 'testdriverai';
+
+describe('Form Filling with Type', () => {
+  let testdriver;
+
+  beforeAll(async () => {
+    client = new TestDriver(process.env.TD_API_KEY);
+    await testdriver.auth();
+    await testdriver.connect();
+  });
+
+  afterAll(async () => {
+    await testdriver.disconnect();
+  });
+
+  it('should fill out registration form', async () => {
+    await testdriver.focusApplication('Google Chrome');
+    
+    // Email field
+    const emailField = await testdriver.find('email input field');
+    await emailField.click();
+    await testdriver.type('john.doe@example.com');
+    
+    // Tab through form fields
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('John');
+    
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('Doe');
+    
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('MySecureP@ssword123');
+    
+    await testdriver.pressKeys(['tab']);
+    await testdriver.type('MySecureP@ssword123'); // Confirm password
+    
+    // Verify fields were filled
+    const result = await testdriver.assert('all form fields are filled');
+    expect(result).toBeTruthy();
+  });
+
+  it('should update search query', async () => {
+    const searchBox = await testdriver.find('search input');
+    await searchBox.click();
+    
+    // Type initial search
+    await testdriver.type('laptops');
+    await testdriver.pressKeys(['enter']);
+    
+    await new Promise(r => setTimeout(r, 2000));
+    
+    // Update search
+    await searchBox.click();
+    await testdriver.pressKeys(['ctrl', 'a']); // Select all
+    await testdriver.type('gaming laptops');
+    await testdriver.pressKeys(['enter']);
+    
+    // Verify new search
+    await testdriver.assert('search results for "gaming laptops" are shown');
+  });
+});
+```
+
+## Related Methods
+
+- [`pressKeys()`](/v7/press-keys) - Press keyboard keys and shortcuts
+- [`find()`](/v7/find) - Locate input fields
+- [`click()`](/v7/click) - Focus input fields

package/ai/skills/testdriver:variables/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: testdriver:variables
+description: Use dynamic data and secure secrets in your tests
+---
+<!-- Generated from variables.mdx. DO NOT EDIT. -->
+
+Scale your testing with dynamic data and secure secrets management. Choose the right approach based on your testing needs.
+
+## Environment Variables
+
+Environment variables are ideal for **configuration that changes between environments** (dev, staging, production) or for **secrets that shouldn't be committed to code**. Use this approach when you need to run the same tests against different servers or with different credentials.
+
+```javascript
+import { test } from 'vitest';
+import { chrome } from 'testdriverai/presets';
+
+test('multi-environment testing', async (context) => {
+  const env = process.env.TEST_ENV || 'staging';
+  const urls = {
+    dev: 'https://dev.myapp.com',
+    staging: 'https://staging.myapp.com',
+    production: 'https://myapp.com'
+  };
+
+  const { testdriver } = await chrome(context, { 
+    url: urls[env] 
+  });
+
+  await testdriver.assert('app is running');
+});
+```
+
+```bash
+# Run against different environments
+TEST_ENV=dev vitest run
+TEST_ENV=staging vitest run
+TEST_ENV=production vitest run
+```
+
+## Test Fixtures
+
+Test fixtures work best when you have **structured, reusable test data** that needs to be shared across multiple tests. Use fixtures when testing different user roles, product catalogs, or any scenario where you want to parameterize tests with a known set of data.
+
+```javascript test/fixtures/users.js
+export const testUsers = [
+  { email: 'admin@test.com', role: 'admin' },
+  { email: 'user@test.com', role: 'user' },
+  { email: 'guest@test.com', role: 'guest' }
+];
+
+export const products = [
+  { name: 'Laptop', price: 999 },
+  { name: 'Mouse', price: 29 },
+  { name: 'Keyboard', price: 89 }
+];
+```
+
+```javascript test/permissions.test.js
+import { test } from 'vitest';
+import { chrome } from 'testdriverai/presets';
+import { testUsers } from './fixtures/users.js';
+
+test.each(testUsers)('$role can access dashboard', async ({ email, role }, context) => {
+  const { testdriver } = await chrome(context, { url });
+  
+  await testdriver.find('email input').type(email);
+  await testdriver.find('password input').type('password123');
+  await testdriver.find('login button').click();
+  
+  if (role === 'admin') {
+    await testdriver.assert('admin panel is visible');
+  } else {
+    await testdriver.assert('user dashboard is visible');
+  }
+});
+```
+
+## Dynamic Data Generation
+
+Dynamic data generation is perfect for **creating unique test data on each run**, avoiding conflicts with existing records, and **testing edge cases with realistic data**. Use libraries like Faker when you need fresh emails, names, or other data that won't collide with previous test runs.
+
+```javascript
+import { test } from 'vitest';
+import { chrome } from 'testdriverai/presets';
+import { faker } from '@faker-js/faker';
+
+test('user registration with dynamic data', async (context) => {
+  const { testdriver } = await chrome(context, { url });
+
+  // Generate unique test data for each run
+  const userData = {
+    firstName: faker.person.firstName(),
+    lastName: faker.person.lastName(),
+    email: faker.internet.email(),
+    password: faker.internet.password({ length: 12 })
+  };
+
+  await testdriver.find('first name input').type(userData.firstName);
+  await testdriver.find('last name input').type(userData.lastName);
+  await testdriver.find('email input').type(userData.email);
+  await testdriver.find('password input').type(userData.password);
+  await testdriver.find('register button').click();
+
+  await testdriver.assert('registration successful');
+  console.log('Registered user:', userData.email);
+});
+```
+
+```bash
+npm install --save-dev @faker-js/faker
+```

package/ai/skills/testdriver:waiting-for-elements/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: testdriver:waiting-for-elements
+description: Handle async operations and prevent flaky tests
+---
+<!-- Generated from waiting-for-elements.mdx. DO NOT EDIT. -->
+
+## Waiting for Elements
+
+Use the `timeout` option with `find()` to wait for elements that appear after async operations:
+
+```javascript
+// Wait up to 30 seconds for element to appear (polls every 5 seconds)
+const element = await testdriver.find('Loading complete indicator', { timeout: 30000 });
+await element.click();
+
+// Useful after actions that trigger loading states
+await testdriver.find('submit button').click();
+await testdriver.find('success message', { timeout: 15000 });
+
+// Short timeout for quick checks
+const toast = await testdriver.find('notification toast', { timeout: 5000 });
+```
+
+## Flake Prevention
+
+TestDriver automatically waits for the screen and network to stabilize after each action using **redraw detection**. This prevents flaky tests caused by animations, loading states, or dynamic content updates.
+
+<Note>
+  Redraw detection adds a small delay after each action but significantly reduces test flakiness.
+</Note>
+
+For example, when clicking a submit button that navigates to a new page:
+
+```javascript
+// Click submit - TestDriver automatically waits for the new page to load
+await testdriver.find('submit button').click();
+
+// By the time this runs, the page has fully loaded and stabilized
+await testdriver.assert('dashboard is displayed');
+await testdriver.find('welcome message');
+```
+
+Without redraw detection, you'd need manual waits or retries to handle the page transition. TestDriver handles this automatically by detecting when the screen stops changing and network requests complete.
+
+You can disable redraw detection or customize its behavior:
+
+```javascript
+// Disable redraw detection for faster tests (less reliable)
+const testdriver = TestDriver(context, { 
+  redraw: false 
+});
+```
+
+Here is an example of customizing redraw detection:
+
+```javascript
+// Fine-tune redraw detection
+const testdriver = TestDriver(context, { 
+  redraw: {
+    enabled: true,
+    diffThreshold: 0.1,      // Pixel difference threshold (0-1)
+    screenRedraw: true,      // Monitor screen changes
+    networkMonitor: true,    // Wait for network idle
+  }
+});
+```