testdriverai 7.3.1 → 7.3.3

package/docs/v7/ocr.mdx

@@ -0,0 +1,236 @@
+---
+title: "ocr()"
+sidebarTitle: "ocr"
+description: "Extract all visible text from the screen using OCR"
+icon: "text"
+---
+
+## Overview
+
+Extract all visible text from the current screen using Tesseract OCR. Returns structured data including each word's text content, bounding box coordinates, and confidence scores.
+
+This method runs OCR on-demand and returns the results immediately. It's useful for:
+- Verifying text content on screen
+- Finding elements by their text when visual matching alone isn't enough
+- Debugging what text TestDriver can "see"
+- Building custom text-based assertions
+
+<Note>
+  **Performance**: OCR runs server-side using Tesseract.js with a worker pool for fast extraction. A typical screenshot processes in 200-500ms.
+</Note>
+
+## Syntax
+
+```javascript
+const result = await testdriver.ocr()
+```
+
+## Parameters
+
+None.
+
+## Returns
+
+`Promise<OCRResult>` - Object containing extracted text data
+
+### OCRResult
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `words` | `OCRWord[]` | Array of extracted words with positions |
+| `fullText` | `string` | All text concatenated with spaces |
+| `confidence` | `number` | Overall OCR confidence (0-100) |
+| `imageWidth` | `number` | Width of the analyzed screenshot |
+| `imageHeight` | `number` | Height of the analyzed screenshot |
+
+### OCRWord
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `content` | `string` | The word's text content |
+| `confidence` | `number` | Confidence score for this word (0-100) |
+| `bbox.x0` | `number` | Left edge X coordinate |
+| `bbox.y0` | `number` | Top edge Y coordinate |
+| `bbox.x1` | `number` | Right edge X coordinate |
+| `bbox.y1` | `number` | Bottom edge Y coordinate |
+
+## Examples
+
+### Get All Text on Screen
+
+```javascript
+const result = await testdriver.ocr();
+console.log(result.fullText);
+// "Welcome to TestDriver Sign In Email Password Submit"
+
+console.log(`Found ${result.words.length} words with ${result.confidence}% confidence`);
+```
+
+### Check if Text Exists
+
+```javascript
+const result = await testdriver.ocr();
+
+// Check for error message
+const hasError = result.words.some(w => 
+  w.content.toLowerCase().includes('error')
+);
+
+if (hasError) {
+  console.log('Error message detected on screen');
+}
+```
+
+### Find and Click Text
+
+```javascript
+const result = await testdriver.ocr();
+
+// Find the "Submit" button text
+const submitWord = result.words.find(w => w.content === 'Submit');
+
+if (submitWord) {
+  // Calculate center of the word's bounding box
+  const x = Math.round((submitWord.bbox.x0 + submitWord.bbox.x1) / 2);
+  const y = Math.round((submitWord.bbox.y0 + submitWord.bbox.y1) / 2);
+  
+  // Click at those coordinates
+  await testdriver.click({ x, y });
+}
+```
+
+### Filter Words by Confidence
+
+```javascript
+const result = await testdriver.ocr();
+
+// Only use high-confidence words (90%+)
+const reliableWords = result.words.filter(w => w.confidence >= 90);
+
+console.log('High confidence words:', reliableWords.map(w => w.content));
+```
+
+### Build Custom Assertions
+
+```javascript
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+
+describe("Login Page", () => {
+  it("should show form labels", async (context) => {
+    const testdriver = TestDriver(context);
+    
+    await testdriver.provision.chrome({
+      url: 'https://myapp.com/login',
+    });
+
+    const result = await testdriver.ocr();
+    
+    // Assert expected labels are present
+    expect(result.fullText).toContain('Email');
+    expect(result.fullText).toContain('Password');
+    expect(result.fullText).toContain('Sign In');
+  });
+});
+```
+
+### Debug Screen Content
+
+```javascript
+// Useful for debugging what TestDriver can see
+const result = await testdriver.ocr();
+
+console.log('=== Screen Text ===');
+console.log(result.fullText);
+console.log('');
+
+console.log('=== Word Details ===');
+result.words.forEach((word, i) => {
+  console.log(`${i + 1}. "${word.content}" at (${word.bbox.x0}, ${word.bbox.y0}) - ${word.confidence}% confidence`);
+});
+```
+
+### Find Multiple Instances
+
+```javascript
+const result = await testdriver.ocr();
+
+// Find all instances of "Button" text
+const buttons = result.words.filter(w => 
+  w.content.toLowerCase() === 'button'
+);
+
+console.log(`Found ${buttons.length} buttons on screen`);
+
+buttons.forEach((btn, i) => {
+  console.log(`Button ${i + 1} at position (${btn.bbox.x0}, ${btn.bbox.y0})`);
+});
+```
+
+## How It Works
+
+1. TestDriver captures a screenshot of the current screen
+2. The image is sent to the TestDriver API
+3. Tesseract.js processes the image server-side with multiple workers
+4. The API returns structured data with text and positions
+5. Bounding box coordinates are scaled to match the original screen resolution
+
+<Note>
+  OCR works best with clear, readable text. Very small text, unusual fonts, or low-contrast text may have lower confidence scores or be missed entirely.
+</Note>
+
+## Best Practices
+
+<AccordionGroup>
+  <Accordion title="Use find() for element location">
+    For locating elements, prefer `find()` which uses AI vision. Use `ocr()` when you need raw text data or want to build custom text-based logic.
+    
+    ```javascript
+    // Prefer this for clicking elements
+    await testdriver.find("Submit button").click();
+    
+    // Use ocr() for text verification or custom logic
+    const result = await testdriver.ocr();
+    expect(result.fullText).toContain('Success');
+    ```
+  </Accordion>
+  
+  <Accordion title="Filter by confidence">
+    OCR can sometimes misread characters. Filter by confidence score when accuracy is critical.
+    
+    ```javascript
+    const result = await testdriver.ocr();
+    const reliable = result.words.filter(w => w.confidence >= 85);
+    ```
+  </Accordion>
+
+  <Accordion title="Handle case sensitivity">
+    Text matching should usually be case-insensitive since OCR capitalization can vary.
+    
+    ```javascript
+    const result = await testdriver.ocr();
+    const hasLogin = result.words.some(w => 
+      w.content.toLowerCase() === 'login'
+    );
+    ```
+  </Accordion>
+
+  <Accordion title="Wait for content to load">
+    If text isn't being found, the page may not be fully loaded. Add a wait or use `waitForText()`.
+    
+    ```javascript
+    // Wait for specific text to appear
+    await testdriver.waitForText("Welcome");
+    
+    // Then run OCR
+    const result = await testdriver.ocr();
+    ```
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+- [find()](/v7/find) - AI-powered element location
+- [assert()](/v7/assert) - Make AI-powered assertions about screen state
+- [waitForText()](/v7/waiting-for-elements) - Wait for text to appear on screen
+- [screenshot()](/v7/screenshot) - Capture screenshots

package/{ai/skills/testdriver:performing-actions/SKILL.md → docs/v7/performing-actions.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:performing-actions
-description: Click, type, hover, scroll and more with TestDriver
+title: "Performing Actions"
+description: "Click, type, hover, scroll and more with TestDriver"
+icon: "computer-mouse"
 ---
-<!-- Generated from performing-actions.mdx. DO NOT EDIT. -->
 
 ## Performing Actions
 

package/{ai/skills/testdriver:press-keys/SKILL.md → docs/v7/press-keys.mdx}

@@ -1,8 +1,9 @@
 ---
-name: testdriver:press-keys
-description: Press keyboard keys and shortcuts
+title: "pressKeys()"
+sidebarTitle: "pressKeys"
+description: "Press keyboard keys and shortcuts"
+icon: "keyboard"
 ---
-<!-- Generated from press-keys.mdx. DO NOT EDIT. -->
 
 ## Overview
 

package/{ai/skills/testdriver:quickstart/SKILL.md → docs/v7/quickstart.mdx}

@@ -1,8 +1,9 @@
 ---
-name: testdriver:quickstart
-description: Run your first computer-use test in minutes.
+title: "Quick Start"
+sidebarTitle: "Quickstart"
+description: "Run your first computer-use test in minutes."
+icon: "rocket"
 ---
-<!-- Generated from quickstart.mdx. DO NOT EDIT. -->
 
 TestDriver makes it easy to write automated computer-use tests for web browsers, desktop apps, and more. Follow the directions below to run your first TestDriver test.
 
@@ -14,27 +15,12 @@ TestDriver makes it easy to write automated computer-use tests for web browsers,
     Get started quickly with the TestDriver CLI.
 
     <Steps>
-      <Step title="Create a TestDriver Account">
-
-        You will need a TestDriver account to get an API key.
-
-        <Card
-          title="Get an API Key"
-          icon="user-plus"
-          href="https://console.testdriver.ai/team"
-          arrow
-          horizontal
-        >
-          Start with 60 free device minutes, no credit-card required!
-        </Card>
-
-      </Step>
       <Step title="Install TestDriver">
 
         Use `npx` to quickly set up an example project:
 
         ```bash
-        npx testdriverai@beta init
+        npx testdriverai init
         ```
 
         This will walk you through creating a new project folder, installing dependencies, and setting up your API key.
@@ -79,7 +65,7 @@ TestDriver makes it easy to write automated computer-use tests for web browsers,
         Install Vitest and TestDriver as dev dependencies:
 
         ```bash
-        npm install --save-dev vitest testdriverai@beta
+        npm install --save-dev vitest testdriverai
         ```
 
       </Step>

package/{ai/skills/testdriver:reusable-code/SKILL.md → docs/v7/reusable-code.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:reusable-code
-description: Build maintainable test suites with reusable code patterns
+title: "Reusable Code Snippets"
+description: "Build maintainable test suites with reusable code patterns"
+icon: "recycle"
 ---
-<!-- Generated from reusable-code.mdx. DO NOT EDIT. -->
 
 As your test suite grows, you'll want to extract common patterns into reusable code. This keeps tests DRY, readable, and easy to maintain.
 

package/{ai/skills/testdriver:right-click/SKILL.md → docs/v7/right-click.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:right-click
-description: Perform a right-click action to open context menus
+title: "rightClick"
+description: "Perform a right-click action to open context menus"
+icon: "bars"
 ---
-<!-- Generated from right-click.mdx. DO NOT EDIT. -->
 
 ## Overview
 

package/{ai/skills/testdriver:running-tests/SKILL.md → docs/v7/running-tests.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:running-tests
-description: Run TestDriver tests with Vitest test runner
+title: "Running Tests"
+description: "Run TestDriver tests with Vitest test runner"
+icon: "play"
 ---
-<!-- Generated from running-tests.mdx. DO NOT EDIT. -->
 
 Learn how to run TestDriver tests efficiently with Vitest's powerful test runner.
 
@@ -10,6 +10,10 @@ Learn how to run TestDriver tests efficiently with Vitest's powerful test runner
 
 TestDriver works with Vitest's powerful test runner.
 
+<Info>
+  Install Vitest globally for best results: `npm install vitest -g`
+</Info>
+
 ### Run All Tests
 
 ```bash

package/{ai/skills/testdriver:screenshot/SKILL.md → docs/v7/screenshot.mdx}

@@ -1,13 +1,18 @@
 ---
-name: testdriver:screenshot
-description: Capture and save screenshots during test execution
+title: "screenshot()"
+sidebarTitle: "screenshot"
+description: "Capture and save screenshots during test execution"
+icon: "camera"
 ---
-<!-- Generated from screenshot.mdx. DO NOT EDIT. -->
 
 ## Overview
 
 Capture a screenshot of the current screen and automatically save it to a local file. Screenshots are organized by test file for easy debugging and review.
 
+<Note>
+  **Automatic Screenshots (Default: Enabled)**: TestDriver automatically captures screenshots before and after every command (click, type, find, etc.). These are saved with descriptive filenames like `001-click-before-L42-submit-button.png` that include the line number from your test file. You can disable this with `autoScreenshots: false` in your TestDriver options.
+</Note>
+
 ## Syntax
 
 ```javascript
@@ -32,12 +37,32 @@ Screenshots are automatically saved to `.testdriver/screenshots/<test-file-name>
 .testdriver/
   screenshots/
     login.test/
-      screenshot-1737633600000.png
-      login-page.png
+      001-find-before-L15-email-input.png     # Auto: before find()
+      002-find-after-L15-email-input.png      # Auto: after find()
+      003-click-before-L16-email-input.png    # Auto: before click()
+      004-click-after-L16-email-input.png     # Auto: after click()
+      005-type-before-L17-userexamplecom.png  # Auto: before type()
+      006-type-after-L17-userexamplecom.png   # Auto: after type()
+      custom-screenshot.png                    # Manual: screenshot("custom-screenshot")
     checkout.test/
-      screenshot-1737633700000.png
+      001-find-before-L12-checkout-button.png
+      ...
 ```
 
+### Automatic Screenshot Naming
+
+When `autoScreenshots` is enabled (default), filenames follow this format:
+
+`<seq>-<action>-<phase>-L<line>-<description>.png`
+
+| Component | Description | Example |
+|-----------|-------------|---------|
+| `seq` | Sequential number (001, 002, ...) | `001` |
+| `action` | Command name | `click`, `type`, `find` |
+| `phase` | Before, after, or error | `before`, `after` |
+| `L<line>` | Line number from test file | `L42` |
+| `description` | Element description or action target | `submit-button` |
+
 <Note>
   The screenshot folder for each test file is automatically cleared when the test starts. This ensures you only see screenshots from the most recent test run.
 </Note>
@@ -107,9 +132,66 @@ describe("Login Flow", () => {
 });
 ```
 
+## Automatic Screenshots
+
+By default, TestDriver captures screenshots **automatically** before and after every command. This creates a complete visual timeline of your test execution without any additional code.
+
+### Enabling/Disabling
+
+```javascript
+// Auto-screenshots enabled by default
+const testdriver = TestDriver(context);
+
+// Explicitly disable if needed (not recommended)
+const testdriver = TestDriver(context, {
+  autoScreenshots: false
+});
+```
+
+### What Gets Captured
+
+Automatic screenshots are taken around these commands:
+- `find()` / `findAll()`
+- `click()` / `hover()` / `doubleClick()` / `rightClick()`
+- `type()` / `pressKeys()`
+- `scroll()` / `scrollUntilText()` / `scrollUntilImage()`
+- `waitForText()` / `waitForImage()`
+- `focusApplication()`
+- `assert()` / `extract()` / `exec()`
+
+### Example Output
+
+For this test code:
+
+```javascript
+// Line 15: Find email input
+const emailInput = await testdriver.find("email input");
+// Line 16: Click it
+await emailInput.click();
+// Line 17: Type email
+await testdriver.type("user@example.com");
+```
+
+TestDriver automatically saves:
+
+```
+001-find-before-L15-email-input.png
+002-find-after-L15-email-input.png
+003-click-before-L16-email-input.png
+004-click-after-L16-email-input.png
+005-type-before-L17-userexamplecom.png
+006-type-after-L17-userexamplecom.png
+```
+
+If an error occurs, the phase will be `error` instead of `after`.
+
 ## Best Practices
 
 <AccordionGroup>
+  <Accordion title="Let automatic screenshots do the work">
+    With `autoScreenshots: true` (default), you get comprehensive coverage without adding manual `screenshot()` calls. Only add manual screenshots for specific named checkpoints.
+  </Accordion>
+  
   <Accordion title="Use screenshots for debugging flaky tests">
     When a test fails intermittently, add screenshots at key steps to capture the actual screen state. This helps identify timing issues or unexpected UI states.
   </Accordion>

package/{ai/skills/testdriver:scroll/SKILL.md → docs/v7/scroll.mdx}

@@ -1,13 +1,29 @@
 ---
-name: testdriver:scroll
-description: Scroll pages and elements
+title: "scroll()"
+sidebarTitle: "scroll"
+description: "Scroll pages and elements"
+icon: "arrows-up-down"
 ---
-<!-- Generated from scroll.mdx. DO NOT EDIT. -->
 
 ## Overview
 
 Scroll the page or active element in any direction using mouse wheel or keyboard.
 
+<Warning>
+  **Focus Requirements**
+  
+  Scrolling requires the page or a frame to be focused. If an input field or other interactive element has focus, scroll commands may not work as expected. Before scrolling, ensure focus is on the page by:
+  - Clicking on a non-interactive area (e.g., page background)
+  - Pressing the Escape key to unfocus interactive elements
+  - Clicking outside of input fields or text areas
+  
+  **If scroll is still not working**, try using Page Down/Page Up keys directly:
+  ```javascript
+  await testdriver.pressKeys(['pagedown']); // Scroll down
+  await testdriver.pressKeys(['pageup']);   // Scroll up
+  ```
+</Warning>
+
 ## Syntax
 
 ```javascript
@@ -133,6 +149,27 @@ await testdriver.scrollUntilImage('loading spinner', 'down', 5000, 'keyboard', n
 
 ## Best Practices
 
+<Check>
+  **Ensure page has focus before scrolling**
+  
+  ```javascript
+  // After typing in an input, unfocus it first
+  await testdriver.find('email input').click();
+  await testdriver.type('user@example.com');
+  
+  // Click elsewhere or press Escape before scrolling
+  await testdriver.pressKeys(['escape']);
+  // Or click a non-interactive area
+  // await testdriver.find('page background').click();
+  
+  // Now scroll will work properly
+  await testdriver.scroll('down', 300);
+  
+  // If scroll still doesn't work, use Page Down directly
+  // await testdriver.pressKeys(['pagedown']);
+  ```
+</Check>
+
 <Check>
   **Choose the right scroll method**
   

package/{ai/skills/testdriver:secrets/SKILL.md → docs/v7/secrets.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:secrets
-description: Securely manage passwords and sensitive data in your tests
+title: "Using Secrets"
+description: "Securely manage passwords and sensitive data in your tests"
+icon: "key"
 ---
-<!-- Generated from secrets.mdx. DO NOT EDIT. -->
 
 Protect sensitive information like passwords, API keys, and tokens in your TestDriver tests.
 

package/{ai/skills/testdriver:self-hosted/SKILL.md → docs/v7/self-hosted.mdx}

@@ -1,8 +1,9 @@
 ---
-name: testdriver:self-hosted
-description: Unlimited test execution, complete privacy, and the ability to customize everything — all for a predictable flat license fee.
+title: "Self-Hosted"
+sidebarTitle: "Self-Hosted"
+description: "Unlimited test execution, complete privacy, and the ability to customize everything — all for a predictable flat license fee."
+icon: "server"
 ---
-<!-- Generated from self-hosted.mdx. DO NOT EDIT. -->
 
 Self-hosted pricing is based on **parallel test capacity**: the number of tests you can run simultaneously on **your infrastructure**. 
 

package/{ai/skills/testdriver:type/SKILL.md → docs/v7/type.mdx}

@@ -1,8 +1,9 @@
 ---
-name: testdriver:type
-description: Type text into focused input fields
+title: "type()"
+sidebarTitle: "type"
+description: "Type text into focused input fields"
+icon: "keyboard"
 ---
-<!-- Generated from type.mdx. DO NOT EDIT. -->
 
 ## Overview
 

package/{ai/skills/testdriver:variables/SKILL.md → docs/v7/variables.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:variables
-description: Use dynamic data and secure secrets in your tests
+title: "Using Variables"
+description: "Use dynamic data and secure secrets in your tests"
+icon: "square-root-variable"
 ---
-<!-- Generated from variables.mdx. DO NOT EDIT. -->
 
 Scale your testing with dynamic data and secure secrets management. Choose the right approach based on your testing needs.
 

package/{ai/skills/testdriver:waiting-for-elements/SKILL.md → docs/v7/waiting-for-elements.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:waiting-for-elements
-description: Handle async operations and prevent flaky tests
+title: "Waiting for Elements"
+description: "Handle async operations and prevent flaky tests"
+icon: "clock"
 ---
-<!-- Generated from waiting-for-elements.mdx. DO NOT EDIT. -->
 
 ## Waiting for Elements
 

package/{ai/skills/testdriver:what-is-testdriver/SKILL.md → docs/v7/what-is-testdriver.mdx}

@@ -1,8 +1,8 @@
 ---
-name: testdriver:what-is-testdriver
-description: Reliably test your most difficult user flows
+title: "What is TestDriver?"
+description: "Reliably test your most difficult user flows"
+icon: "circle-info"
 ---
-<!-- Generated from what-is-testdriver.mdx. DO NOT EDIT. -->
 
 ## The problem with modern testing tools