@testdriverai/agent 7.8.0-canary.10

package/docs/v7/elements.mdx

@@ -0,0 +1,606 @@
+---
+title: "Element Finding"
+sidebarTitle: "Elements"
+description: "Locate and interact with UI elements using AI"
+icon: "mouse-pointer"
+---
+
+## Overview
+
+TestDriver's element finding system uses AI to locate elements on screen using natural language descriptions. The `find()` method returns an `Element` object that you can interact with.
+
+## Finding Elements
+
+### find()
+
+Locate an element on screen using a natural language description.
+
+```javascript
+const element = await testdriver.find(description)
+```
+
+**Parameters:**
+- `description` (string) - Natural language description of the element to find
+
+**Returns:** `Promise<Element>` - Element instance that has been located
+
+**Example:**
+```javascript
+// Find a button
+const submitButton = await testdriver.find('the submit button');
+
+// Find an input field with context
+const emailField = await testdriver.find('email input field in the login form');
+
+// Find an element by visual characteristics
+const redButton = await testdriver.find('red button in the top right corner');
+```
+
+<Tip>
+  Be specific in your descriptions. Include visual details, location context, or nearby text to improve accuracy.
+</Tip>
+
+## Element Class
+
+The `Element` class represents a located (or to-be-located) UI element. It provides methods for interaction and properties for element information.
+
+### Methods
+
+#### found()
+
+Check if the element was successfully located.
+
+```javascript
+element.found()
+```
+
+**Returns:** `boolean` - True if element coordinates were found
+
+**Example:**
+```javascript
+const element = await testdriver.find('login button');
+if (element.found()) {
+  await element.click();
+} else {
+  console.log('Element not found');
+}
+```
+
+#### find()
+
+Re-locate the element, optionally with a new description.
+
+```javascript
+await element.find(newDescription)
+```
+
+**Parameters:**
+- `newDescription` (string, optional) - New description to search for
+
+**Returns:** `Promise<Element>` - This element instance
+
+**Example:**
+```javascript
+// Re-locate if the UI changed
+const element = await testdriver.find('submit button');
+// ... page updates ...
+await element.find(); // Re-locate with same description
+
+// Or update the description
+await element.find('blue submit button'); // Now looking for blue button
+```
+
+#### click()
+
+Click on the element.
+
+```javascript
+await element.click(action)
+```
+
+**Parameters:**
+- `action` (string, optional) - Type of click: `'click'` (default), `'double-click'`, `'right-click'`, `'hover'`, `'mouseDown'`, `'mouseUp'`
+
+**Returns:** `Promise<void>`
+
+**Example:**
+```javascript
+const button = await testdriver.find('submit button');
+await button.click(); // Regular click
+
+const file = await testdriver.find('document.txt');
+await file.click('double-click'); // Double-click
+
+const menu = await testdriver.find('settings icon');
+await menu.click('right-click'); // Right-click
+```
+
+<Note>
+  The element must be found before clicking. The `find()` method automatically locates the element.
+</Note>
+
+#### hover()
+
+Hover over the element without clicking.
+
+```javascript
+await element.hover()
+```
+
+**Returns:** `Promise<void>`
+
+**Example:**
+```javascript
+const tooltip = await testdriver.find('info icon');
+await tooltip.hover();
+// Wait to see tooltip
+await new Promise(resolve => setTimeout(resolve, 1000));
+```
+
+#### doubleClick()
+
+Double-click on the element.
+
+```javascript
+await element.doubleClick()
+```
+
+**Returns:** `Promise<void>`
+
+**Example:**
+```javascript
+const file = await testdriver.find('README.txt file icon');
+await file.doubleClick();
+```
+
+#### rightClick()
+
+Right-click on the element to open context menu.
+
+```javascript
+await element.rightClick()
+```
+
+**Returns:** `Promise<void>`
+
+**Example:**
+```javascript
+const folder = await testdriver.find('Documents folder');
+await folder.rightClick();
+```
+
+#### mouseDown() / mouseUp()
+
+Press or release mouse button on the element (for drag operations).
+
+```javascript
+await element.mouseDown()
+await element.mouseUp()
+```
+
+**Returns:** `Promise<void>`
+
+**Example:**
+```javascript
+// Drag and drop
+const item = await testdriver.find('draggable item');
+await item.mouseDown();
+
+// Move to drop target (using coordinates or another element)
+const target = await testdriver.find('drop zone');
+await target.hover();
+await target.mouseUp();
+```
+
+### Properties
+
+Element properties provide additional information about located elements. Properties are available after a successful `find()` call.
+
+#### coordinates
+
+Get the element's coordinates object containing all position information.
+
+```javascript
+const coords = element.getCoordinates()
+// or access directly
+element.coordinates
+```
+
+**Returns:** `Object | null` - Coordinate object with `{ x, y, centerX, centerY }`
+
+**Example:**
+```javascript
+const button = await testdriver.find('submit button');
+const coords = button.coordinates;
+
+if (coords) {
+  console.log(`Top-left: (${coords.x}, ${coords.y})`);
+  console.log(`Center: (${coords.centerX}, ${coords.centerY})`);
+}
+```
+
+#### x, y, centerX, centerY
+
+Direct access to coordinate values. Always available after successful `find()`.
+
+```javascript
+element.x        // Top-left X coordinate (number)
+element.y        // Top-left Y coordinate (number)
+element.centerX  // Center X coordinate (number)
+element.centerY  // Center Y coordinate (number)
+```
+
+**Example:**
+```javascript
+const button = await testdriver.find('submit button');
+console.log(`Button at: (${button.x}, ${button.y})`);
+console.log(`Button center: (${button.centerX}, ${button.centerY})`);
+
+// Use for custom mouse operations
+await testdriver.click(button.centerX, button.centerY);
+```
+
+#### width, height
+
+Element dimensions in pixels. Available when AI detects element bounds.
+
+```javascript
+element.width   // Width in pixels (number | null)
+element.height  // Height in pixels (number | null)
+```
+
+**Example:**
+```javascript
+const button = await testdriver.find('submit button');
+
+if (button.width && button.height) {
+  console.log(`Button size: ${button.width}x${button.height}px`);
+  
+  // Check if button is large enough
+  if (button.width < 50) {
+    console.warn('Button might be too small');
+  }
+}
+```
+
+#### boundingBox
+
+Complete bounding box information including position and dimensions.
+
+```javascript
+element.boundingBox
+```
+
+**Returns:** `Object | null` - Bounding box with all dimension data
+
+```typescript
+{
+  x: number,        // Top-left X
+  y: number,        // Top-left Y
+  width: number,    // Width in pixels
+  height: number    // Height in pixels
+}
+```
+
+**Example:**
+```javascript
+const element = await testdriver.find('dialog box');
+
+if (element.boundingBox) {
+  const { x, y, width, height } = element.boundingBox;
+  console.log(`Dialog: ${width}x${height} at (${x}, ${y})`);
+  
+  // Calculate if element is in viewport
+  const rightEdge = x + width;
+  const bottomEdge = y + height;
+  console.log(`Element extends to (${rightEdge}, ${bottomEdge})`);
+}
+```
+
+#### screenshot
+
+Base64-encoded PNG screenshot of the screen when element was found. Only available in DEBUG mode or when an error occurs.
+
+```javascript
+element.screenshot
+```
+
+**Returns:** `string | null` - Base64-encoded PNG image
+
+**Example:**
+```javascript
+const element = await testdriver.find('error message');
+
+if (element.screenshot) {
+  // Save screenshot to file
+  const fs = require('fs');
+  const base64Data = element.screenshot.replace(/^data:image\/\w+;base64,/, '');
+  fs.writeFileSync('element-screenshot.png', Buffer.from(base64Data, 'base64'));
+  console.log('Screenshot saved');
+}
+```
+
+<Warning>
+  Screenshots can be large. They're automatically excluded from error messages to prevent memory issues.
+</Warning>
+
+#### text
+
+Text content extracted from the element by AI (if available).
+
+```javascript
+element.text
+```
+
+**Returns:** `string | null` - Element's text content
+
+**Example:**
+```javascript
+const message = await testdriver.find('notification message');
+
+if (message.text) {
+  console.log('Message says:', message.text);
+  
+  // Use text content in assertions
+  if (message.text.includes('success')) {
+    console.log('Success message detected');
+  }
+}
+
+// Another example - extracting button label
+const button = await testdriver.find('blue button');
+console.log('Button text:', button.text); // "Submit"
+```
+
+#### label
+
+Accessible label or name of the element (if available). Useful for verifying accessibility.
+
+```javascript
+element.label
+```
+
+**Returns:** `string | null` - Accessible label
+
+**Example:**
+```javascript
+const input = await testdriver.find('first input field');
+
+if (input.label) {
+  console.log('Input label:', input.label); // "Email Address"
+}
+```
+
+#### confidence
+
+AI confidence score for the element match (0-1, where 1 is perfect confidence).
+
+```javascript
+element.confidence
+```
+
+**Returns:** `number | null` - Confidence score between 0 and 1
+
+**Example:**
+```javascript
+const element = await testdriver.find('submit button');
+
+if (element.confidence !== null) {
+  const percentage = (element.confidence * 100).toFixed(1);
+  console.log(`Match confidence: ${percentage}%`);
+  
+  if (element.confidence < 0.8) {
+    console.warn('⚠️ Low confidence match - element might not be correct');
+  } else if (element.confidence > 0.95) {
+    console.log('✅ High confidence match');
+  }
+}
+```
+
+<Tip>
+  Confidence scores below 0.8 may indicate the element description was ambiguous or the wrong element was found.
+</Tip>
+
+### Property Availability
+
+| Property | When Available |
+|----------|---------------|
+| `x`, `y`, `centerX`, `centerY` | ✅ Always after successful `find()` |
+| `coordinates` | ✅ Always after successful `find()` |
+| `width`, `height` | ⚠️ When AI detects element bounds |
+| `boundingBox` | ⚠️ When AI detects element bounds |
+| `text` | ⚠️ When AI extracts text content |
+| `label` | ⚠️ When element has accessible label |
+| `confidence` | ✅ Always after AI element finding |
+| `screenshot` | ⚠️ Only in DEBUG mode or on errors |
+
+<Note>
+  Properties marked with ⚠️ may be `null` depending on what the AI could detect from the screenshot.
+</Note>
+
+## JSON Serialization
+
+Element objects can be safely serialized using `JSON.stringify()` for logging, debugging, and data storage. Circular references are automatically removed:
+
+```javascript
+const element = await testdriver.find('login button');
+
+// Safe to stringify - no circular reference errors!
+console.log(JSON.stringify(element, null, 2));
+```
+
+**Serialized output includes:**
+
+```json
+{
+  "description": "login button",
+  "coordinates": { "x": 100, "y": 200, "centerX": 150, "centerY": 225 },
+  "found": true,
+  "threshold": 0.01,
+  "x": 100,
+  "y": 200,
+  "cache": {
+    "hit": true,
+    "strategy": "pixel-diff",
+    "createdAt": "2025-12-09T10:30:00.000Z",
+    "diffPercent": 0.0023,
+    "imageUrl": "https://cache.testdriver.ai/..."
+  },
+  "similarity": 0.98,
+  "confidence": 0.95,
+  "selector": "button#login",
+  "aiResponse": "Found the blue login button in the center of the form..."
+}
+```
+
+**Serialized properties:**
+
+| Property | Type | Description |
+|----------|------|-------------|
+| `description` | string | Element search description |
+| `coordinates` | object | Full coordinate object `{x, y, centerX, centerY}` |
+| `found` | boolean | Whether element was located |
+| `threshold` | number | Cache threshold used for this find |
+| `x`, `y` | number | Top-left coordinates |
+| `cache.hit` | boolean | Whether cache was used |
+| `cache.strategy` | string | Cache strategy (e.g., "pixel-diff") |
+| `cache.createdAt` | string | ISO timestamp when cache was created |
+| `cache.diffPercent` | number | Pixel difference from cached image |
+| `cache.imageUrl` | string | URL to cached screenshot |
+| `similarity` | number | Similarity score (0-1) |
+| `confidence` | number | AI confidence score (0-1) |
+| `selector` | string | CSS/XPath selector if available |
+| `aiResponse` | string | AI's explanation of what it found |
+
+**Use cases:**
+
+```javascript
+// Debugging element detection
+const element = await testdriver.find('submit button');
+if (!element.found()) {
+  console.error('Element not found:', JSON.stringify(element, null, 2));
+}
+
+// Logging cache performance
+const data = JSON.parse(JSON.stringify(element));
+if (data.cache.hit) {
+  console.log(`Cache hit! Diff: ${(data.cache.diffPercent * 100).toFixed(2)}%`);
+}
+
+// Sharing element data across processes
+const elementData = JSON.stringify(element);
+// Send to another process, log to file, etc.
+```
+
+<Tip>
+  Use JSON serialization when you need to log element data or when debugging why an element wasn't found. The serialized output excludes large binary data (screenshots) and circular references.
+</Tip>
+
+## Examples
+
+### Basic Element Interaction
+
+```javascript
+// Find and click
+const submitButton = await testdriver.find('submit button');
+await submitButton.click();
+
+// Find, verify, then interact
+const emailInput = await testdriver.find('email input field');
+if (emailInput.found()) {
+  await emailInput.click();
+  await testdriver.type('user@example.com');
+}
+```
+
+### Working with Forms
+
+```javascript
+// Fill out a multi-field form
+const nameField = await testdriver.find('name input field');
+await nameField.click();
+await testdriver.type('John Doe');
+
+const emailField = await testdriver.find('email input field');
+await emailField.click();
+await testdriver.type('john@example.com');
+
+const submitButton = await testdriver.find('submit button');
+await submitButton.click();
+```
+
+### Conditional Interactions
+
+```javascript
+// Check if element exists before interacting
+const closeButton = await testdriver.find('close popup button');
+
+if (closeButton.found()) {
+  await closeButton.click();
+  console.log('Popup closed');
+} else {
+  console.log('No popup to close');
+}
+```
+
+### Re-locating Dynamic Elements
+
+```javascript
+// Element that moves or changes
+const notification = await testdriver.find('success notification');
+
+// Do something that might cause it to move
+await testdriver.scroll('down', 300);
+
+// Re-locate the element
+await notification.find();
+
+if (notification.found()) {
+  await notification.click();
+}
+```
+
+## Best Practices
+
+<AccordionGroup>
+  <Accordion title="Be specific with descriptions">
+    Include visual details, position context, and nearby text:
+    
+    ```javascript
+    // ❌ Too vague
+    await testdriver.find('button');
+    
+    // ✅ Specific
+    await testdriver.find('blue submit button below the email field');
+    ```
+  </Accordion>
+  
+  <Accordion title="Check if element was found">
+    Always verify elements were located before interacting:
+    
+    ```javascript
+    const element = await testdriver.find('submit button');
+    if (!element.found()) {
+      throw new Error('Submit button not found');
+    }
+    await element.click();
+    ```
+  </Accordion>
+
+  
+  <Accordion title="Reuse element references when possible">
+    If you need to interact with the same element multiple times, reuse the reference:
+    
+    ```javascript
+    const input = await testdriver.find('search input');
+    await input.click();
+    await testdriver.type('first search');
+    await testdriver.pressKeys(['enter']);
+    
+    // Re-use the same element reference
+    await input.click();
+    await testdriver.pressKeys(['ctrl', 'a']); // Select all
+    await testdriver.type('second search');
+    ```
+  </Accordion>
+</AccordionGroup>

package/docs/v7/enterprise.mdx

@@ -0,0 +1,9 @@
+---
+title: "Enterprise"
+sidebarTitle: "Enterprise"
+description: "Self-hosted enterprise deployments with assisted setup and dedicated support"
+icon: "building"
+redirect: "/v7/self-hosted"
+---
+
+This page has moved to [Self-Hosted](/v7/self-hosted).