testdriverai 7.2.9 → 7.2.10

package/agents.md

@@ -0,0 +1,393 @@
+# TestDriver Agent Guide
+
+This guide is designed for AI agents working with TestDriver. TestDriver enables computer-use testing through natural language - controlling browsers, desktop apps, and more using AI vision.
+
+## Quick Reference
+
+| Resource | Location |
+|----------|----------|
+| Code samples | `node_modules/testdriverai/test` |
+| TypeScript types | `node_modules/testdriverai/sdk.d.ts` |
+| Documentation | `node_modules/testdriverai/docs` |
+| API Key | [console.testdriver.ai/team](https://console.testdriver.ai/team) |
+
+## Prerequisites
+
+### API Key Setup
+
+The user **must** have a TestDriver API key set in their environment:
+
+```bash
+# .env file
+TD_API_KEY=your_api_key_here
+```
+
+Get your API key at: **https://console.testdriver.ai/team**
+
+### Test Runner
+
+TestDriver **only works with Vitest**. Tests must use the `.test.mjs` extension and import from vitest:
+
+```javascript
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+```
+
+## Basic Test Structure
+
+```javascript
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+
+describe("My Test Suite", () => {
+  it("should do something", async (context) => {
+    // Initialize TestDriver with options
+    const testdriver = TestDriver(context, { newSandbox: true, headless: false });
+    
+    // Start with provision - this launches the sandbox and browser
+    await testdriver.provision.chrome({
+      url: 'https://example.com',
+    });
+
+    // Find elements and interact
+    const button = await testdriver.find("Sign In button");
+    await button.click();
+
+    // Assert using natural language
+    const result = await testdriver.assert("the dashboard is visible");
+    expect(result).toBeTruthy();
+  });
+});
+```
+
+## Provisioning Options
+
+Most tests start with `testdriver.provision`. Choose the right one:
+
+### `provision.chrome` - Web Testing (Most Common)
+```javascript
+await testdriver.provision.chrome({
+  url: 'https://your-app.com',
+});
+```
+
+### `provision.installer` - Desktop App Testing
+```javascript
+// Download and install an application
+const filePath = await testdriver.provision.installer({
+  url: 'https://example.com/app.deb',  // or .msi, .exe, .sh
+  launch: true,  // auto-launch after install
+});
+```
+
+## Key SDK Methods
+
+The SDK has TypeScript types in `sdk.d.ts`. Key methods:
+
+| Method | Purpose |
+|--------|---------|
+| `find(description)` | Find element by natural language |
+| `findAll(description)` | Find all matching elements |
+| `assert(assertion)` | AI-powered assertion |
+| `type(text)` | Type text |
+| `pressKeys([keys])` | Press keyboard keys |
+| `scroll(direction)` | Scroll the page |
+| `exec(language, code)` | Execute code in sandbox |
+| `ai(task)` | AI exploratory loop (see note below) |
+
+### About `ai()` - Use for Exploration, Not Final Tests
+
+The `ai(task)` method lets the AI figure out how to accomplish a task autonomously. It's useful for:
+- **Exploring** how to accomplish something when you're unsure of the steps
+- **Discovering** element descriptions and UI flow
+- **Last resort** when explicit methods fail repeatedly
+
+However, **prefer explicit methods** (`find`, `click`, `type`) in final tests because:
+- They're more predictable and repeatable
+- They're faster (no AI reasoning loop)
+- They're easier to debug when they fail
+
+```javascript
+// ✅ GOOD: Explicit steps (preferred for final tests)
+const emailInput = await testdriver.find("email input field");
+await emailInput.click();
+await testdriver.type("user@example.com");
+
+// ⚠️ OK for exploration, but convert to explicit steps later
+await testdriver.ai("fill in the email field with user@example.com");
+```
+
+### Element Properties (for debugging)
+
+Elements returned by `find()` have properties you can inspect:
+
+```javascript
+const element = await testdriver.find("Sign In button");
+
+// Debugging properties
+console.log(element.x, element.y);           // coordinates
+console.log(element.centerX, element.centerY); // center coordinates  
+console.log(element.width, element.height);  // dimensions
+console.log(element.confidence);             // AI confidence score
+console.log(element.text);                   // detected text
+console.log(element.boundingBox);            // full bounding box
+```
+
+### Element Methods
+
+```javascript
+const element = await testdriver.find("button");
+await element.click();        // click
+await element.hover();        // hover
+await element.doubleClick();  // double-click
+await element.rightClick();   // right-click
+await element.mouseDown();    // press mouse down
+await element.mouseUp();      // release mouse
+element.found();              // check if found (boolean)
+```
+
+## Best Workflow: Two-File Pattern
+
+**The most efficient workflow for building tests uses two files.** This prevents having to restart from scratch when experimenting with new steps.
+
+### IMPORTANT: When to Use This Pattern
+
+- **If a working test already exists**: Only create an `experiment.test.mjs` file to add new steps. Do NOT recreate the stable file.
+- **If starting from scratch**: Start with a MINIMAL setup file, run it, verify it passes, THEN create the experiment file.
+
+### Step 1: Create a Minimal Setup File
+
+Start with the bare minimum - just provision and one assertion. **Do NOT call it "stable" yet** - name it `setup.test.mjs` until it's proven to work:
+
+```javascript
+/**
+ * Setup file - MINIMAL steps to get to starting state
+ * Only add more steps AFTER this passes!
+ */
+import { afterAll, describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+
+describe("Setup State", () => {
+
+  afterAll(async () => {
+    // DO NOT disconnect - keep sandbox alive for reconnect
+    console.log("Sandbox staying alive for 30 seconds (keepAlive)");
+  });
+
+  it("should set up the application state", async (context) => {
+    const testdriver = TestDriver(context, { 
+      newSandbox: true, 
+      headless: false 
+    });
+    
+    await testdriver.provision.chrome({
+      url: 'https://your-app.com/login',
+    });
+
+    // Start with just ONE assertion to verify we're on the right page
+    const result = await testdriver.assert("I can see the login page");
+    expect(result).toBeTruthy();
+    
+    console.log("✅ Setup ready - run experiment.test.mjs now");
+  });
+});
+```
+
+### Step 2: Run Setup File and Verify It Passes
+
+```bash
+vitest run tests/setup.test.mjs
+```
+
+**Only proceed to Step 3 if this passes!** If it fails, fix it first.
+
+### Step 3: Create Experiment File
+
+Only AFTER the setup file passes, create the experiment file.
+
+**CRITICAL: The experiment file must NOT call `provision`!** It reconnects to the existing sandbox where provision already ran:
+
+```javascript
+/**
+ * Experiment file - reconnects to existing sandbox
+ * Run AFTER setup.test.mjs passes (within 2 minutes)
+ */
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+
+describe("Experiment", () => {
+
+  it("should continue from existing state", async (context) => {
+    const testdriver = TestDriver(context, { 
+      newSandbox: true, 
+      headless: false,
+      reconnect: true  // ← Key: reconnects to last sandbox
+    });
+    
+    // NO provision here! The sandbox is already running from setup.test.mjs
+    
+    // Experiment with new steps here - try ONE thing at a time
+    const element = await testdriver.find("email input");
+    console.log("Found element:", element.found(), element.getCoordinates());
+    
+    await element.click();
+    await testdriver.type("user@example.com");
+    
+    // Assert after each major step
+    const result = await testdriver.assert("email is filled in");
+    console.log("Assertion result:", result);
+    expect(result).toBeTruthy();
+  });
+});
+```
+
+### Step 4: Iterate in Experiment File
+
+- Run experiment, see output, fix issues
+- Once steps work, move them to the setup file
+- Re-run setup file to verify it still passes
+- Repeat until complete
+
+### Running the Two-File Pattern
+
+```bash
+# Step 1: Run setup file (must pass first!)
+vitest run tests/setup.test.mjs
+
+# Step 2: Within 2 minutes, run experiment file
+vitest run tests/experiment.test.mjs
+```
+
+### After Experimentation: Combine and Rename
+
+**IMPORTANT:** Once the test is working, combine both files into a single, properly-named test file:
+
+1. **Merge the code** - Copy the working steps from `experiment.test.mjs` into `setup.test.mjs`
+2. **Remove reconnect options** - Delete `reconnect: true` since the final test runs standalone
+3. **Rename the file** - Give it a meaningful name like `login-flow.test.mjs` or `checkout-process.test.mjs`
+4. **Delete the experiment file** - Clean up `experiment.test.mjs`
+
+```javascript
+// Final combined test: login-flow.test.mjs
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/lib/vitest/hooks.mjs";
+
+describe("Login Flow", () => {
+  it("should log in and open settings", async (context) => {
+    const testdriver = TestDriver(context, { newSandbox: true, headless: false });
+    
+    await testdriver.provision.chrome({ url: 'https://your-app.com/login' });
+
+    // Steps from stable file
+    await testdriver.find("email input").click();
+    await testdriver.type("user@example.com");
+    await testdriver.find("password input").click();
+    await testdriver.type("password123");
+    await testdriver.find("Login button").click();
+    
+    // Steps from experiment file
+    const settingsBtn = await testdriver.find("Settings button");
+    await settingsBtn.click();
+    
+    const result = await testdriver.assert("Settings panel is open");
+    expect(result).toBeTruthy();
+  });
+});
+```
+
+## Recommended Development Workflow
+
+1. **Write a few steps** - Don't write the entire test at once
+2. **Run the test** - See what happens on the sandbox
+3. **Inspect outputs** - Use element properties to debug
+4. **Assert/expect** - Verify the step worked
+5. **Iterate** - Add more steps incrementally
+
+```javascript
+// Development workflow example
+it("should incrementally build test", async (context) => {
+  const testdriver = TestDriver(context, { newSandbox: true });
+  await testdriver.provision.chrome({ url: 'https://example.com' });
+
+  // Step 1: Find and inspect
+  const element = await testdriver.find("Some button");
+  console.log("Element found:", element.found());
+  console.log("Coordinates:", element.x, element.y);
+  console.log("Confidence:", element.confidence);
+  
+  // Step 2: Interact
+  await element.click();
+  
+  // Step 3: Assert and log
+  const result = await testdriver.assert("Something happened");
+  console.log("Assertion result:", result);
+  expect(result).toBeTruthy();
+  
+  // Then add more steps...
+});
+```
+
+## TestDriver Options Reference
+
+```javascript
+const testdriver = TestDriver(context, {
+  newSandbox: true,       // Create new sandbox (default: true)
+  headless: false,        // Run in headless mode (default: false)
+  reconnect: false,       // Reconnect to last sandbox (default: false)
+  keepAlive: 30000,       // Keep sandbox alive after test (default: 30000ms / 30 seconds)
+  os: 'linux',            // 'linux' | 'windows' (default: 'linux')
+  resolution: '1366x768', // Sandbox resolution
+  cache: true,            // Enable element caching (default: true)
+  cacheKey: 'my-test',    // Cache key for element finding
+});
+```
+
+## Common Patterns
+
+### Typing in Fields
+```javascript
+await testdriver.find("Email input").click();
+await testdriver.type("user@example.com");
+```
+
+### Keyboard Shortcuts
+```javascript
+await testdriver.pressKeys(["ctrl", "a"]);  // Select all
+await testdriver.pressKeys(["ctrl", "c"]);  // Copy
+await testdriver.pressKeys(["enter"]);       // Submit
+```
+
+### Waiting and Polling
+```javascript
+// Use timeout option to poll until element is found (retries every 5 seconds)
+const element = await testdriver.find("Loading complete indicator", { timeout: 30000 });
+await element.click();
+```
+
+### Scrolling
+```javascript
+await testdriver.scroll("down");
+await testdriver.scrollUntilText("Footer text");
+await testdriver.scrollUntilImage("Product image at bottom");
+```
+
+### Executing Code in Sandbox
+```javascript
+// JavaScript
+const result = await testdriver.exec("js", "return document.title", 5000);
+
+// Shell (Linux)
+const output = await testdriver.exec("sh", "ls -la", 5000);
+
+// PowerShell (Windows)
+const date = await testdriver.exec("pwsh", "Get-Date", 5000);
+```
+
+## Tips for Agents
+
+1. **Always check `sdk.d.ts`** for method signatures and types
+2. **Look at test samples** in `node_modules/testdriverai/test` for working examples
+3. **Use reconnect pattern** when iterating on test steps
+4. **Log element properties** to understand what the AI sees
+5. **Use `assert()` with specific, descriptive natural language**
+6. **Start simple** - get one step working before adding more

package/docs/docs.json

@@ -187,132 +187,100 @@
               }
             ]
           },
-        {
-  "version": "v7",
-  "groups": [
-    {
-      "group": "Getting Started",
-      "pages": [
-        "/v7/getting-started/quickstart",
-        "/v7/getting-started/writing-tests",
-        "/v7/getting-started/running-tests",
-        "/v7/getting-started/debugging-tests",
-        "/v7/getting-started/setting-up-in-ci"
-      ]
-    },
-    {
-      "group": "Examples",
-      "pages": [
-        "/v7/presets/chrome",
-        "/v7/presets/chrome-extension",
-        "/v7/presets/vscode",
-        "/v7/presets/electron"
-      ]
-    },
-    {
-      "group": "Guides",
-      "pages": [
-        {
-          "group": "Selectorless Testing",
-          "icon": "sparkles",
-          "pages": [
-            "/v7/features/easy-to-write",
-            "/v7/features/coverage",
-            "/v7/features/powerful",
-            "/v7/features/ai-native"
-          ]
-        },
-        {
-          "group": "Sandbox Environments",
-          "icon": "cube",
-          "pages": [
-            "/v7/features/managed-sandboxes",
-            "/v7/features/sandbox-customization",
-            "/v7/features/enterprise"
-          ]
-        },
-        {
-          "group": "Test Observability",
-          "icon": "microscope",
-          "pages": [
-            "/v7/features/application-logs",
-            "/v7/features/browser-logs",
-            "/v7/features/network-monitoring",
-            "/v7/features/system-performance"
-          ]
-        },
-        {
-          "group": "Performance & Reliability",
-          "icon": "gauge-high",
-          "pages": [
-            "/v7/features/fast",
-            "/v7/features/stable",
-            "/v7/features/parallel-execution",
-            "/v7/features/cache-management"
-          ]
-        },
-        {
-          "group": "Testing at Scale",
-          "icon": "arrow-up-right-dots",
-          "pages": [
-            "/v7/features/data-driven-testing",
-            "/v7/features/continuous-testing"
-          ]
-        },
-        {
-          "group": "Reports & Analytics",
-          "icon": "chart-line",
-          "pages": [
-            "/v7/features/test-reports",
-            "/v7/features/test-analytics",
-            "/v7/features/test-cases",
-            "/v7/features/test-replays"
-          ]
-        }
-      ]
-    },
-    {
-      "group": "Reference",
-      "icon": "book",
-      "pages": [
-        {
-          "group": "Actions",
-          "icon": "bolt",
-          "pages": [
-            "/v7/api/act",
-            "/v7/api/assert",
-            "/v7/api/assertions",
-            "/v7/api/click",
-            "/v7/api/doubleClick",
-            "/v7/api/exec",
-            "/v7/api/find",
-            "/v7/api/focusApplication",
-            "/v7/api/hover",
-            "/v7/api/mouseDown",
-            "/v7/api/mouseUp",
-            "/v7/api/pressKeys",
-            "/v7/api/rightClick",
-            "/v7/api/type",
-            "/v7/api/scroll"
-          ]
-        },
-        {
-          "group": "Client SDK",
-          "icon": "code",
-          "pages": [
-            "/v7/api/client",
-            "/v7/api/elements",
-            "/v7/api/sandbox",
-            "/v7/api/dashcam"
-          ]
-        }
-      ]
-    }
-  ]
-}
-
+          {
+            "version": "v7",
+            "groups": [
+              {
+                "group": "Overview",
+                "pages": [
+                  "/v7/what-is-testdriver",
+                  "/v7/quickstart",
+                  "/v7/examples",
+                  {
+                    "group": "Plans & Pricing",
+                    "icon": "server",
+                    "pages": [
+                      "/v7/cloud",
+                      {
+                        "group": "Self-Hosted",
+                        "icon": "server",
+                        "pages": [
+                          "/v7/self-hosted",
+                          "/v7/aws-setup"
+                        ]
+                      },
+                      "/v7/enterprise"
+                    ]
+                  }
+                ]
+              },
+              {
+                "group": "Creating Tests",
+                "pages": [
+                  "/v7/generating-tests",
+                  "/v7/locating-elements",
+                  "/v7/waiting-for-elements",
+                  "/v7/performing-actions",
+                  "/v7/making-assertions"
+                ]
+              },
+              {
+                "group": "Running Tests",
+                "pages": [
+                  "/v7/running-tests",
+                  "/v7/caching",
+                  "/v7/ci-cd"
+                ]
+              },
+              {
+                "group": "Scaling",
+                "pages": [
+                  "/v7/variables",
+                  "/v7/secrets",
+                  "/v7/reusable-code",
+                  "/v7/customizing-devices"
+                ]
+              },
+              {
+                "group": "Reference",
+                "icon": "book",
+                "pages": [
+                  {
+                    "group": "Actions",
+                    "icon": "bolt",
+                    "pages": [
+                      "/v7/ai",
+                      "/v7/assert",
+                      "/v7/click",
+                      "/v7/double-click",
+                      "/v7/exec",
+                      "/v7/find",
+                      "/v7/focus-application",
+                      "/v7/hover",
+                      "/v7/mouse-down",
+                      "/v7/mouse-up",
+                      "/v7/press-keys",
+                      "/v7/right-click",
+                      "/v7/type",
+                      "/v7/scroll"
+                    ]
+                  },
+                  {
+                    "group": "Client SDK",
+                    "icon": "code",
+                    "pages": [
+                      "/v7/client",
+                      "/v7/elements",
+                      "/v7/sandbox",
+                      "/v7/dashcam"
+                    ]
+                  }
+                ]
+              }
+            ]
+          }
         ]
-  },
+      },
   "logo": {
     "light": "/images/template/light.png",
     "dark": "/images/template/dark.png",

package/docs/v7/_drafts/caching.mdx

@@ -1,6 +1,6 @@
 ---
-title: "Caching"
-sidebarTitle: "Caching"
+title: "Caching Prompts"
+sidebarTitle: "Caching Prompts"
 description: "How TestDriver caches AI responses and element locations for faster tests"
 icon: "bolt"
 ---