@testdriverai/agent 7.8.0-canary.10

package/docs/v7/errors.mdx

@@ -0,0 +1,248 @@
+---
+title: "Errors"
+sidebarTitle: "Errors"
+description: "Custom error classes and error handling"
+icon: "triangle-exclamation"
+mode: "wide"
+---
+
+## Overview
+
+TestDriver provides custom error classes with rich debugging information. These are exported from the SDK and can be used for `instanceof` checks in your tests.
+
+```javascript
+import TestDriver, { ElementNotFoundError, AIError } from 'testdriverai';
+```
+
+## ElementNotFoundError
+
+Thrown when `find()` cannot locate an element on screen, or when calling `click()`/`hover()` on an unfound element.
+
+```javascript
+try {
+  await testdriver.find('nonexistent button').click();
+} catch (error) {
+  if (error instanceof ElementNotFoundError) {
+    console.log(error.description);       // "nonexistent button"
+    console.log(error.screenshotPath);    // path to debug screenshot
+    console.log(error.pixelDiffPath);     // path to pixel diff image
+  }
+}
+```
+
+### Properties
+
+<ParamField path="name" type="string">
+  Always `"ElementNotFoundError"`.
+</ParamField>
+
+<ParamField path="message" type="string">
+  Enhanced message with a debug block containing element description, cache status, similarity scores, and AI response details.
+</ParamField>
+
+<ParamField path="description" type="string">
+  The original element description passed to `find()`.
+</ParamField>
+
+<ParamField path="screenshotPath" type="string | null">
+  Absolute path to a debug screenshot saved at the time of failure. Written to `<os.tmpdir>/testdriver-debug/screenshot-<timestamp>.png`.
+</ParamField>
+
+<ParamField path="pixelDiffPath" type="string | null">
+  Absolute path to a pixel diff image showing the comparison between the cached and current screenshots. Written to `<os.tmpdir>/testdriver-debug/pixel-diff-error-<timestamp>.png`. Only present when cache was involved.
+</ParamField>
+
+<ParamField path="cachedImagePath" type="string | null">
+  URL to the cached image that was compared against.
+</ParamField>
+
+<ParamField path="aiResponse" type="object | null">
+  Sanitized AI response object. Large binary fields (`croppedImage`, `screenshot`, `pixelDiffImage`) are removed. Contains cache metadata like `similarity`, `cacheHit`, `cacheStrategy`, `cacheDiffPercent`, and `threshold`.
+</ParamField>
+
+<ParamField path="timestamp" type="string">
+  ISO 8601 timestamp of when the error was created.
+</ParamField>
+
+### Enhanced Message
+
+The error message is automatically enhanced with debugging information:
+
+```
+Element not found: "submit button"
+
+=== Element Debug Info ===
+Element: submit button
+Cache Hit: false
+Similarity: 0.23
+Cache Strategy: pixel-diff
+Threshold: 0.05
+AI Response Element: null
+```
+
+### Stack Trace Cleaning
+
+Stack traces are automatically cleaned to remove internal SDK frames (`Element.*`, `sdk.js` internals), showing only your test code for easier debugging.
+
+## AIError
+
+Thrown when `act()` exhausts all retry attempts.
+
+```javascript
+try {
+  await testdriver.act('perform complex workflow', { tries: 3 });
+} catch (error) {
+  if (error instanceof AIError) {
+    console.log(error.task);       // "perform complex workflow"
+    console.log(error.tries);     // 3
+    console.log(error.duration);  // 15234 (ms)
+    console.log(error.cause);     // underlying Error
+  }
+}
+```
+
+### Properties
+
+<ParamField path="name" type="string">
+  Always `"AIError"`.
+</ParamField>
+
+<ParamField path="message" type="string">
+  Enhanced message with execution details block.
+</ParamField>
+
+<ParamField path="task" type="string">
+  The task description passed to `act()`.
+</ParamField>
+
+<ParamField path="tries" type="number">
+  Number of attempts that were made.
+</ParamField>
+
+<ParamField path="maxTries" type="number">
+  Maximum number of attempts configured.
+</ParamField>
+
+<ParamField path="duration" type="number">
+  Total execution time in milliseconds.
+</ParamField>
+
+<ParamField path="cause" type="Error | undefined">
+  The underlying error that caused the final failure.
+</ParamField>
+
+<ParamField path="timestamp" type="string">
+  ISO 8601 timestamp of when the error was created.
+</ParamField>
+
+### Enhanced Message
+
+```
+AI failed: Element not found after 3 attempts
+
+=== AI Execution Details ===
+Task: perform complex workflow
+Tries: 3 / 3
+Duration: 15234ms
+Cause: ElementNotFoundError: Element not found: "submit button"
+```
+
+## Internal Error Classes
+
+These errors are used internally by the agent and are not exported, but may appear as the `cause` of an `AIError`:
+
+### MatchError
+
+Thrown when element matching fails (text, image, or assertion).
+
+| Property | Type | Description |
+|---|---|---|
+| `fatal` | `boolean` | If `true`, cannot be healed. Default: `false` |
+| `attachScreenshot` | `boolean` | Always `true` — a screenshot is attached to the error |
+
+### CommandError
+
+Thrown for invalid arguments or unsupported operations.
+
+| Property | Type | Description |
+|---|---|---|
+| `fatal` | `boolean` | Always `true` |
+| `attachScreenshot` | `boolean` | Always `false` |
+
+## Soft Assert Mode
+
+Inside `act()`, assertions run in **soft assert mode**. When an assertion fails, it returns the failure result instead of throwing, allowing the AI to process the failure and adjust its approach.
+
+```javascript
+// Inside act(), assertion failures don't throw
+await testdriver.act('verify the dashboard shows correct data', {
+  tries: 3,
+});
+// The AI can see assertion results and self-correct
+```
+
+This is automatic — you don't need to configure it. Regular `assert()` calls outside of `act()` will throw normally on failure.
+
+## Error Handling Patterns
+
+### Catching Specific Errors
+
+```javascript
+import TestDriver, { ElementNotFoundError, AIError } from 'testdriverai';
+
+try {
+  await testdriver.find('submit button').click();
+} catch (error) {
+  if (error instanceof ElementNotFoundError) {
+    // Element wasn't found — check screenshot for debugging
+    console.log('Debug screenshot:', error.screenshotPath);
+  } else if (error instanceof AIError) {
+    // AI exhausted retries
+    console.log(`Failed after ${error.tries} tries in ${error.duration}ms`);
+  } else {
+    throw error; // Unexpected error
+  }
+}
+```
+
+### Using Debug Screenshots
+
+```javascript
+try {
+  const el = await testdriver.find('checkout button');
+  await el.click();
+} catch (error) {
+  if (error instanceof ElementNotFoundError) {
+    // Screenshot of what the screen looked like
+    console.log('Screen:', error.screenshotPath);
+    // Pixel diff showing cache comparison
+    console.log('Diff:', error.pixelDiffPath);
+    // Full AI response metadata
+    console.log('AI:', JSON.stringify(error.aiResponse, null, 2));
+  }
+}
+```
+
+## Types
+
+```typescript
+class ElementNotFoundError extends Error {
+  name: 'ElementNotFoundError';
+  description: string;
+  screenshotPath: string | null;
+  pixelDiffPath: string | null;
+  cachedImagePath: string | null;
+  aiResponse: Record<string, any> | null;
+  timestamp: string;
+}
+
+class AIError extends Error {
+  name: 'AIError';
+  task: string;
+  tries: number;
+  maxTries: number;
+  duration: number;
+  cause?: Error;
+  timestamp: string;
+}
+```

package/docs/v7/events.mdx

@@ -0,0 +1,358 @@
+---
+title: "Events"
+sidebarTitle: "Events"
+description: "Listen to SDK lifecycle events with wildcard support"
+icon: "bolt"
+mode: "wide"
+---
+
+## Overview
+
+TestDriver uses [EventEmitter2](https://github.com/EventEmitter2/EventEmitter2) for its event system. Events use a colon-delimited namespace pattern and support wildcard listeners.
+
+Access the emitter through `testdriver.emitter`:
+
+```javascript
+testdriver.emitter.on('command:start', (data) => {
+  console.log(`Running: ${data.command}`);
+});
+```
+
+### Configuration
+
+The internal emitter is created with:
+
+```javascript
+new EventEmitter2({
+  wildcard: true,
+  delimiter: ':',
+  maxListeners: 20,
+  verboseMemoryLeak: false,
+  ignoreErrors: false,
+});
+```
+
+### Wildcard Listeners
+
+Use `*` to match a single level or `**` to match multiple levels:
+
+```javascript
+// Match all log events
+testdriver.emitter.on('log:*', (message) => {
+  console.log(message);
+});
+
+// Match all events in any namespace
+testdriver.emitter.on('**', (...args) => {
+  console.log('Event:', this.event, args);
+});
+```
+
+## Event Reference
+
+### Command Events
+
+Emitted during the execution of SDK commands (`click`, `type`, `find`, etc.).
+
+| Event | Payload |
+|---|---|
+| `command:start` | `{ command, depth, data, timestamp, sourcePosition }` |
+| `command:success` | `{ command, depth, data, duration, response, timestamp, sourcePosition }` |
+| `command:error` | `{ command, depth, data, error, duration, timestamp, sourcePosition }` |
+| `command:status` | `{ command, status: "executing", data, depth, timestamp }` |
+| `command:progress` | `{ command, status: "completed", timing, data, depth, timestamp }` |
+
+```javascript
+testdriver.emitter.on('command:start', ({ command, data }) => {
+  console.log(`Starting ${command}`, data);
+});
+
+testdriver.emitter.on('command:error', ({ command, error, duration }) => {
+  console.error(`${command} failed after ${duration}ms: ${error}`);
+});
+```
+
+### Step Events
+
+Emitted for each AI reasoning step within a command.
+
+| Event | Payload |
+|---|---|
+| `step:start` | `{ stepIndex, prompt, commandCount, timestamp, sourcePosition }` |
+| `step:success` | `{ stepIndex, prompt, commandCount, duration, timestamp, sourcePosition }` |
+| `step:error` | `{ stepIndex, prompt, error, duration?, timestamp, sourcePosition? }` |
+
+```javascript
+testdriver.emitter.on('step:start', ({ stepIndex, prompt }) => {
+  console.log(`Step ${stepIndex}: ${prompt}`);
+});
+```
+
+### Test Events
+
+Emitted when a test file execution starts.
+
+| Event | Payload |
+|---|---|
+| `test:start` | `{ filePath, timestamp }` |
+| `test:success` | *Emitted on test completion* |
+| `test:error` | *Emitted on test failure* |
+
+### Log Events
+
+Emitted for all log output from the SDK.
+
+| Event | Payload |
+|---|---|
+| `log:log` | `(message: string)` — general log message |
+| `log:warn` | `(message: string)` — warning message |
+| `log:debug` | `(message: string)` — debug output (only when `VERBOSE`/`DEBUG`/`TD_DEBUG` env set) |
+| `log:info` | `(message: string)` — informational message |
+| `log:error` | `(message: string)` — error message |
+| `log:narration` | `(message: string, overwrite?: boolean)` — in-place status line |
+| `log:markdown` | `(markdown: string)` — full static markdown content |
+| `log:markdown:start` | `(streamId: string)` — begin streaming markdown |
+| `log:markdown:chunk` | `(streamId: string, chunk: string)` — incremental chunk |
+| `log:markdown:end` | `(streamId: string)` — end streaming markdown |
+
+```javascript
+// Capture all logs
+testdriver.emitter.on('log:*', function (message) {
+  console.log(`[${this.event}]`, message);
+});
+```
+
+### Screen Capture Events
+
+Emitted during screenshot capture.
+
+| Event | Payload |
+|---|---|
+| `screen-capture:start` | `{ scale, silent, display }` |
+| `screen-capture:end` | `{ scale, silent, display }` |
+| `screen-capture:error` | `{ error, scale, silent, display }` |
+
+### Sandbox Events
+
+Emitted for sandbox WebSocket lifecycle and communication.
+
+| Event | Payload |
+|---|---|
+| `sandbox:connected` | *No payload* — WebSocket connection established |
+| `sandbox:authenticated` | `{ traceId }` — authentication successful |
+| `sandbox:error` | `(err: Error \| string)` — connection or sandbox error |
+| `sandbox:sent` | `(message: object)` — WebSocket message sent |
+| `sandbox:received` | *No payload* — successful message reply received |
+| `sandbox:progress` | `{ step, message }` — sandbox setup progress |
+
+```javascript
+testdriver.emitter.on('sandbox:connected', () => {
+  console.log('Connected to sandbox');
+});
+
+testdriver.emitter.on('sandbox:progress', ({ step, message }) => {
+  console.log(`Sandbox: [${step}] ${message}`);
+});
+```
+
+### Redraw Events
+
+Emitted during screen stability detection. See [Redraw](/v7/redraw) for more details.
+
+| Event | Payload |
+|---|---|
+| `redraw:status` | `{ redraw: { enabled, settled, hasChangedFromInitial, consecutiveFramesStable, diffFromInitial, diffFromLast, text }, network: { enabled, settled, rxBytes, txBytes, text }, timeout: { isTimeout, elapsed, max, text } }` |
+| `redraw:complete` | `{ screenSettled, hasChangedFromInitial, consecutiveFramesStable, networkSettled, isTimeout, timeElapsed }` |
+
+```javascript
+testdriver.emitter.on('redraw:complete', (result) => {
+  if (result.isTimeout) {
+    console.warn('Redraw timed out after', result.timeElapsed, 'ms');
+  }
+});
+```
+
+### File Events
+
+Emitted during file load/save operations in the agent.
+
+| Event | Payload |
+|---|---|
+| `file:start` | `{ operation: "load" \| "save" \| "run", filePath, timestamp }` |
+| `file:stop` | `{ operation, filePath, duration, success, sourceMap?, reason?, timestamp }` |
+| `file:load` | `{ filePath, size, timestamp }` |
+| `file:save` | `{ filePath, size, timestamp }` |
+| `file:diff` | `{ filePath, diff: { patches, sourceMaps, summary: { additions, deletions, modifications } }, timestamp }` |
+| `file:error` | `{ operation, filePath, error, duration?, timestamp }` |
+
+### Error Events
+
+Emitted for errors at various severity levels.
+
+| Event | Payload |
+|---|---|
+| `error:fatal` | `(error: string \| Error)` — terminates the process |
+| `error:general` | `(message: string)` — non-fatal error |
+| `error:sandbox` | `(err: Error \| string)` — sandbox/WebSocket error |
+
+```javascript
+testdriver.emitter.on('error:*', function (err) {
+  console.error(`[${this.event}]`, err);
+});
+```
+
+### SDK Events
+
+Emitted for API request lifecycle.
+
+| Event | Payload |
+|---|---|
+| `sdk:request` | `{ path }` — outgoing API request |
+| `sdk:response` | `{ path }` — API response received |
+| `sdk:retry` | `{ path, attempt, error, delayMs }` — request retry |
+
+### Other Events
+
+| Event | Payload |
+|---|---|
+| `exit` | `(exitCode: number)` — `0` for success, `1` for failure |
+| `status` | `(message: string)` — general status updates |
+| `mouse-click` | `{ x, y, button, click, double }` — mouse click performed |
+| `terminal:stdout` | Terminal stdout output |
+| `terminal:stderr` | Terminal stderr output |
+
+## Practical Examples
+
+### Custom Test Reporter
+
+```javascript
+const results = [];
+
+testdriver.emitter.on('command:success', ({ command, duration }) => {
+  results.push({ command, duration, status: 'pass' });
+});
+
+testdriver.emitter.on('command:error', ({ command, duration, error }) => {
+  results.push({ command, duration, status: 'fail', error });
+});
+
+// After test completes
+afterAll(() => {
+  console.table(results);
+});
+```
+
+### Progress Monitoring
+
+```javascript
+testdriver.emitter.on('step:start', ({ stepIndex, prompt }) => {
+  process.stdout.write(`\r  Step ${stepIndex}: ${prompt}`);
+});
+
+testdriver.emitter.on('command:progress', ({ command, timing }) => {
+  process.stdout.write(`\r  ${command} completed in ${timing}ms`);
+});
+```
+
+### Debug Logging
+
+```javascript
+// Log every event (verbose)
+testdriver.emitter.on('**', function (...args) {
+  console.debug(`[EVENT] ${this.event}`, ...args);
+});
+```
+
+## Types
+
+```typescript
+interface CommandStartEvent {
+  command: string;
+  depth: number;
+  data: Record<string, any>;
+  timestamp: number;
+  sourcePosition: SourcePosition;
+}
+
+interface CommandSuccessEvent {
+  command: string;
+  depth: number;
+  data: Record<string, any>;
+  duration: number;
+  response: any;
+  timestamp: number;
+  sourcePosition: SourcePosition;
+}
+
+interface CommandErrorEvent {
+  command: string;
+  depth: number;
+  data: Record<string, any>;
+  error: string;
+  duration: number;
+  timestamp: number;
+  sourcePosition: SourcePosition;
+}
+
+interface StepStartEvent {
+  stepIndex: number;
+  prompt: string;
+  commandCount: number;
+  timestamp: number;
+  sourcePosition: SourcePosition;
+}
+
+interface StepSuccessEvent {
+  stepIndex: number;
+  prompt: string;
+  commandCount: number;
+  duration: number;
+  timestamp: number;
+  sourcePosition: SourcePosition;
+}
+
+interface RedrawStatusEvent {
+  redraw: {
+    enabled: boolean;
+    settled: boolean;
+    hasChangedFromInitial: boolean;
+    consecutiveFramesStable: number;
+    diffFromInitial: number;
+    diffFromLast: number;
+    text: string;
+  };
+  network: {
+    enabled: boolean;
+    settled: boolean;
+    rxBytes: number;
+    txBytes: number;
+    text: string;
+  };
+  timeout: {
+    isTimeout: boolean;
+    elapsed: number;
+    max: number;
+    text: string;
+  };
+}
+
+interface RedrawCompleteEvent {
+  screenSettled: boolean;
+  hasChangedFromInitial: boolean;
+  consecutiveFramesStable: number;
+  networkSettled: boolean;
+  isTimeout: boolean;
+  timeElapsed: number;
+}
+
+interface SandboxProgressEvent {
+  step: string;
+  message: string;
+}
+
+interface SourcePosition {
+  file: string;
+  line: number;
+  column: number;
+}
+```

package/docs/v7/examples/ai.mdx

@@ -0,0 +1,72 @@
+---
+title: "AI Test Example"
+sidebarTitle: "AI"
+description: "Example test using the AI exploratory loop to automate complex multi-step interactions."
+icon: "wand-magic-sparkles"
+mode: "wide"
+---
+
+## Demo Test Run
+
+Watch this test execute in a real sandbox environment:
+
+{/* ai.test.mjs output */}
+<iframe 
+  src="https://api-test.testdriver.ai/api/v1/testdriver/testcase/69bc952cf14c7f71c46d6b90/replay" 
+  width="100%" 
+  height="390" 
+  style={{ border: "1px solid #333", borderRadius: "8px" }}
+  allow="fullscreen"
+/>
+
+## Source Code
+
+```javascript title="ai.test.mjs" {17}
+/**
+ * TestDriver SDK - AI Test
+ */
+
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/vitest/hooks";
+
+describe("AI Test", () => {
+  it("should use ai to search for testdriver on Google", async (context) => {
+    const testdriver = TestDriver(context, { ip: context.ip || process.env.TD_IP });
+    
+    // provision.chrome() automatically calls ready() and starts dashcam
+    await testdriver.provision.chrome({
+      url: 'https://duckduckgo.com',
+    });
+
+    // Use ai to search for testdriver
+    let aiRes = await testdriver.ai("click on the empty search box, type 'testdriver', and hit enter.");
+
+    console.log("AI response:", aiRes);
+
+    // Assert the search results are displayed
+    const result = await testdriver.assert(
+      "search results for testdriver are visible",
+    );
+
+    expect(result).toBeTruthy();
+  });
+});
+```
+
+## Running This Example
+
+```bash
+# Clone the TestDriver repository
+git clone https://github.com/testdriverai/testdriverai
+
+# Install dependencies
+cd testdriverai
+npm install
+
+# Run this specific example
+npx vitest run examples/ai.test.mjs
+```
+
+<Note>
+  Make sure you have `TD_API_KEY` set in your environment. Get one at [testdriver.ai](https://testdriver.ai).
+</Note>