testdriverai 7.3.11 → 7.3.13

package/ai/skills/testdriver:mouse-up/SKILL.md

@@ -0,0 +1,164 @@
+---
+name: testdriver:mouse-up
+description: Release the mouse button
+---
+<!-- Generated from mouse-up.mdx. DO NOT EDIT. -->
+
+## Overview
+
+The `mouseUp()` method releases the mouse button, completing a drag operation or custom mouse gesture that was started with [`mouseDown()`](/v7/mouse-down). You can call it without parameters to release at the current mouse position.
+
+## Syntax
+
+```javascript
+// Release mouse button at current position
+await ai.mouseUp();
+```
+
+## Parameters
+
+None. The mouse button is released at the current cursor position.
+
+## Returns
+
+Returns a `Promise<void>` that resolves when the mouse button is released.
+
+## Examples
+
+### Complete Drag and Drop
+
+```javascript
+// Start dragging
+await ai.mouseDown('file to drag');
+
+// Move to drop location
+await ai.hover('target folder');
+
+// Complete the drop
+await ai.mouseUp();
+```
+
+### Selecting Multiple Files
+
+```javascript
+import { test } from 'vitest';
+import { vscode } from '@testdriver/sdk';
+
+test('selects range of files', async () => {
+  const { ai } = await vscode();
+  
+  // Click first file
+  await ai.click('first-file.js in explorer');
+  
+  // Hold shift and click last file
+  await ai.pressKeys('Shift');
+  await ai.mouseDown('last-file.js in explorer');
+  await ai.mouseUp();
+  
+  // Verify multiple files selected
+  const selectedCount = await ai.find('status bar showing file count');
+  expect(selectedCount.text).toContain('5 files');
+});
+```
+
+### Drawing Application
+
+```javascript
+test('draws a shape', async () => {
+  const { ai } = await chrome('https://drawing-app.example.com');
+  
+  // Select pencil tool
+  await ai.click('pencil tool');
+  
+  // Draw a line
+  await ai.mouseDown('canvas near top-left');
+  await ai.hover('canvas center');
+  await ai.hover('canvas bottom-right');
+  await ai.mouseUp();
+  
+  // Verify drawing exists
+  const strokes = await ai.exec('canvas.getContext("2d").getImageData(0,0,100,100)');
+  expect(strokes).toBeTruthy();
+});
+```
+
+### Resizing Window Panels
+
+```javascript
+test('resizes editor panel', async () => {
+  const { ai } = await vscode();
+  
+  // Grab the divider
+  await ai.mouseDown('panel resize divider');
+  
+  // Drag to new position
+  await ai.hover('position 400 pixels from left');
+  
+  // Release to complete resize
+  await ai.mouseUp();
+  
+  // Verify new panel size
+  const panel = await ai.find('editor panel');
+  expect(panel.width).toBeGreaterThan(350);
+});
+```
+
+### Drag to Reorder List Items
+
+```javascript
+import { test } from 'vitest';
+import { chrome } from '@testdriver/sdk';
+
+test('reorders tasks in list', async () => {
+  const { ai } = await chrome('https://todo-app.example.com');
+  
+  // Start dragging first task
+  await ai.mouseDown('drag handle on first task');
+  
+  // Move down to third position
+  await ai.hover('third task position');
+  
+  // Drop the task
+  await ai.mouseUp();
+  
+  // Verify new order
+  const thirdTask = await ai.find('third task in list');
+  expect(thirdTask.text).toContain('Original first task');
+});
+```
+
+### Text Selection with Mouse
+
+```javascript
+test('selects text with mouse drag', async () => {
+  const { ai } = await chrome('https://document.example.com');
+  
+  // Start selection at beginning of word
+  await ai.mouseDown('start of "TestDriver" word');
+  
+  // Drag to end of word
+  await ai.hover('end of "TestDriver" word');
+  
+  // Complete selection
+  await ai.mouseUp();
+  
+  // Verify selection
+  const selection = await ai.exec('window.getSelection().toString()');
+  expect(selection).toBe('TestDriver');
+});
+```
+
+## Important Notes
+
+- `mouseUp()` must be preceded by [`mouseDown()`](/v7/mouse-down) to have an effect
+- Releases the button at the current cursor position
+- Completes any drag or selection operation that was in progress
+- For simple clicks, use [`click()`](/v7/click) instead of mouseDown/mouseUp pair
+
+## Related Methods
+
+- [`mouseDown()`](/v7/mouse-down) - Press mouse button without releasing
+- [`hover()`](/v7/hover) - Move mouse to element
+- [`click()`](/v7/click) - Complete click (mouseDown + mouseUp)
+- [`doubleClick()`](/v7/double-click) - Double-click on element
+- [`rightClick()`](/v7/right-click) - Right-click for context menu

package/ai/skills/testdriver:ocr/SKILL.md

@@ -0,0 +1,235 @@
+---
+name: testdriver:ocr
+description: Extract all visible text from the screen using OCR
+---
+<!-- Generated from ocr.mdx. DO NOT EDIT. -->
+
+## Overview
+
+Extract all visible text from the current screen using Tesseract OCR. Returns structured data including each word's text content, bounding box coordinates, and confidence scores.
+
+This method runs OCR on-demand and returns the results immediately. It's useful for:
+- Verifying text content on screen
+- Finding elements by their text when visual matching alone isn't enough
+- Debugging what text TestDriver can "see"
+- Building custom text-based assertions
+
+<Note>
+  **Performance**: OCR runs server-side using Tesseract.js with a worker pool for fast extraction. A typical screenshot processes in 200-500ms.
+</Note>
+
+## Syntax
+
+```javascript
+const result = await testdriver.ocr()
+```
+
+## Parameters
+
+None.
+
+## Returns
+
+`Promise<OCRResult>` - Object containing extracted text data
+
+### OCRResult
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `words` | `OCRWord[]` | Array of extracted words with positions |
+| `fullText` | `string` | All text concatenated with spaces |
+| `confidence` | `number` | Overall OCR confidence (0-100) |
+| `imageWidth` | `number` | Width of the analyzed screenshot |
+| `imageHeight` | `number` | Height of the analyzed screenshot |
+
+### OCRWord
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `content` | `string` | The word's text content |
+| `confidence` | `number` | Confidence score for this word (0-100) |
+| `bbox.x0` | `number` | Left edge X coordinate |
+| `bbox.y0` | `number` | Top edge Y coordinate |
+| `bbox.x1` | `number` | Right edge X coordinate |
+| `bbox.y1` | `number` | Bottom edge Y coordinate |
+
+## Examples
+
+### Get All Text on Screen
+
+```javascript
+const result = await testdriver.ocr();
+console.log(result.fullText);
+// "Welcome to TestDriver Sign In Email Password Submit"
+
+console.log(`Found ${result.words.length} words with ${result.confidence}% confidence`);
+```
+
+### Check if Text Exists
+
+```javascript
+const result = await testdriver.ocr();
+
+// Check for error message
+const hasError = result.words.some(w => 
+  w.content.toLowerCase().includes('error')
+);
+
+if (hasError) {
+  console.log('Error message detected on screen');
+}
+```
+
+### Find and Click Text
+
+```javascript
+const result = await testdriver.ocr();
+
+// Find the "Submit" button text
+const submitWord = result.words.find(w => w.content === 'Submit');
+
+if (submitWord) {
+  // Calculate center of the word's bounding box
+  const x = Math.round((submitWord.bbox.x0 + submitWord.bbox.x1) / 2);
+  const y = Math.round((submitWord.bbox.y0 + submitWord.bbox.y1) / 2);
+  
+  // Click at those coordinates
+  await testdriver.click({ x, y });
+}
+```
+
+### Filter Words by Confidence
+
+```javascript
+const result = await testdriver.ocr();
+
+// Only use high-confidence words (90%+)
+const reliableWords = result.words.filter(w => w.confidence >= 90);
+
+console.log('High confidence words:', reliableWords.map(w => w.content));
+```
+
+### Build Custom Assertions
+
+```javascript
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+
+describe("Login Page", () => {
+  it("should show form labels", async (context) => {
+    const testdriver = TestDriver(context);
+    
+    await testdriver.provision.chrome({
+      url: 'https://myapp.com/login',
+    });
+
+    const result = await testdriver.ocr();
+    
+    // Assert expected labels are present
+    expect(result.fullText).toContain('Email');
+    expect(result.fullText).toContain('Password');
+    expect(result.fullText).toContain('Sign In');
+  });
+});
+```
+
+### Debug Screen Content
+
+```javascript
+// Useful for debugging what TestDriver can see
+const result = await testdriver.ocr();
+
+console.log('=== Screen Text ===');
+console.log(result.fullText);
+console.log('');
+
+console.log('=== Word Details ===');
+result.words.forEach((word, i) => {
+  console.log(`${i + 1}. "${word.content}" at (${word.bbox.x0}, ${word.bbox.y0}) - ${word.confidence}% confidence`);
+});
+```
+
+### Find Multiple Instances
+
+```javascript
+const result = await testdriver.ocr();
+
+// Find all instances of "Button" text
+const buttons = result.words.filter(w => 
+  w.content.toLowerCase() === 'button'
+);
+
+console.log(`Found ${buttons.length} buttons on screen`);
+
+buttons.forEach((btn, i) => {
+  console.log(`Button ${i + 1} at position (${btn.bbox.x0}, ${btn.bbox.y0})`);
+});
+```
+
+## How It Works
+
+1. TestDriver captures a screenshot of the current screen
+2. The image is sent to the TestDriver API
+3. Tesseract.js processes the image server-side with multiple workers
+4. The API returns structured data with text and positions
+5. Bounding box coordinates are scaled to match the original screen resolution
+
+<Note>
+  OCR works best with clear, readable text. Very small text, unusual fonts, or low-contrast text may have lower confidence scores or be missed entirely.
+</Note>
+
+## Best Practices
+
+<AccordionGroup>
+  <Accordion title="Use find() for element location">
+    For locating elements, prefer `find()` which uses AI vision. Use `ocr()` when you need raw text data or want to build custom text-based logic.
+    
+    ```javascript
+    // Prefer this for clicking elements
+    await testdriver.find("Submit button").click();
+    
+    // Use ocr() for text verification or custom logic
+    const result = await testdriver.ocr();
+    expect(result.fullText).toContain('Success');
+    ```
+  </Accordion>
+  
+  <Accordion title="Filter by confidence">
+    OCR can sometimes misread characters. Filter by confidence score when accuracy is critical.
+    
+    ```javascript
+    const result = await testdriver.ocr();
+    const reliable = result.words.filter(w => w.confidence >= 85);
+    ```
+  </Accordion>
+
+  <Accordion title="Handle case sensitivity">
+    Text matching should usually be case-insensitive since OCR capitalization can vary.
+    
+    ```javascript
+    const result = await testdriver.ocr();
+    const hasLogin = result.words.some(w => 
+      w.content.toLowerCase() === 'login'
+    );
+    ```
+  </Accordion>
+
+  <Accordion title="Wait for content to load">
+    If text isn't being found, the page may not be fully loaded. Add a wait or use `waitForText()`.
+    
+    ```javascript
+    // Wait for specific text to appear
+    await testdriver.waitForText("Welcome");
+    
+    // Then run OCR
+    const result = await testdriver.ocr();
+    ```
+  </Accordion>
+</AccordionGroup>
+
+## Related
+
+- [find()](/v7/find) - AI-powered element location
+- [assert()](/v7/assert) - Make AI-powered assertions about screen state
+- [waitForText()](/v7/waiting-for-elements) - Wait for text to appear on screen
+- [screenshot()](/v7/screenshot) - Capture screenshots

package/ai/skills/testdriver:performing-actions/SKILL.md

@@ -0,0 +1,51 @@
+---
+name: testdriver:performing-actions
+description: Click, type, hover, scroll and more with TestDriver
+---
+<!-- Generated from performing-actions.mdx. DO NOT EDIT. -->
+
+## Performing Actions
+
+TestDriver provides a variety of actions you can perform, like [clicking](/v7/click), [typing](/v7/type), [hovering](/v7/hover), and [scrolling](/v7/scroll). For a full list, see the [API Reference](/v7/click).
+
+```javascript
+// Clicking
+await testdriver.find('submit button').click();
+await testdriver.find('file item').doubleClick();
+await testdriver.find('text area').rightClick();
+
+// Typing
+await testdriver.find('email input').type('user@example.com');
+await testdriver.find('password input').type('secret', { secret: true });
+
+// Keyboard shortcuts
+await testdriver.pressKeys(['enter']);
+await testdriver.pressKeys(['ctrl', 'c']);
+
+// Hovering
+await testdriver.find('dropdown menu').hover();
+
+// Scrolling
+await testdriver.scroll('down', 500);
+await testdriver.scrollUntilText('Footer content');
+
+// Extracting information from screen
+const price = await testdriver.extract('the total price');
+const orderNumber = await testdriver.extract('the order confirmation number');
+```
+
+## Chaining Actions
+
+TestDriver supports method chaining for cleaner code:
+
+```javascript
+// Chain find() with actions
+const button = await testdriver.find('submit button').click();
+```
+
+Or save element reference for later use:
+
+```javascript
+const button = await testdriver.find('submit button');
+await button.click();
+```