testdriverai 7.2.80 → 7.2.82

package/ai/agents/testdriver.md

@@ -139,30 +139,32 @@ import { TestDriver } from "testdriverai/vitest/hooks";
 
 describe("My Test Suite", () => {
   it("should do something", async (context) => {
-    // Initialize TestDriver
+    // Initialize TestDriver - screenshots are captured automatically before/after each command
     const testdriver = TestDriver(context);
 
     // Start with provision - this launches the sandbox and browser
     await testdriver.provision.chrome({
       url: "https://example.com",
     });
-    await testdriver.screenshot(); // Capture initial page state
 
     // Find elements and interact
+    // Note: Screenshots are automatically captured before/after find() and click()
     const button = await testdriver.find("Sign In button");
-    await testdriver.screenshot(); // Capture before click
     await button.click();
     await testdriver.wait(2000); // Wait for state change
-    await testdriver.screenshot(); // Capture after click
 
     // Assert using natural language
-    await testdriver.screenshot(); // Capture before assertion
+    // Screenshots are automatically captured before/after assert()
     const result = await testdriver.assert("the dashboard is visible");
     expect(result).toBeTruthy();
   });
 });
 ```
 
+<Note>
+  **Automatic Screenshots**: TestDriver captures screenshots before and after every command by default. Screenshots are saved with descriptive names like `001-click-before-L42-submit-button.png` that include the line number from your test file.
+</Note>
+
 ## Provisioning Options
 
 Most tests start with `testdriver.provision`.
@@ -220,54 +222,21 @@ await element.mouseUp(); // release mouse
 element.found(); // check if found (boolean)
 ```
 
-### Screenshots for Debugging
-
-**Use `screenshot()` liberally throughout your tests** to capture the screen state at key moments. This makes debugging much easier when tests fail - you can see exactly what the screen looked like at each step.
+### Automatic Screenshots (Enabled by Default)
 
-```javascript
-// Capture a screenshot - saved to .testdriver/screenshots/<test-file>/
-const screenshotPath = await testdriver.screenshot();
-console.log("Screenshot saved to:", screenshotPath);
+TestDriver **automatically captures screenshots before and after every command** by default. This creates a complete visual timeline without any additional code. Screenshots are named with the line number from your test file, making it easy to trace issues:
 
-// Include mouse cursor in screenshot
-await testdriver.screenshot(1, false, true);
 ```
-
-**When to add screenshots:**
-- After provisioning (initial page load)
-- Before and after clicking important elements
-- After typing text into fields
-- Before assertions (to see what the AI is evaluating)
-- After any action that changes the page state
-- When debugging a flaky or failing test
-
-**⚠️ Important: Add delays before screenshots after actions**
-
-When you click or interact with an element that triggers a state change (page navigation, modal opening, content loading), **add a short delay before taking a screenshot** to allow the application state to update:
-
-```javascript
-await element.click();
-await testdriver.wait(2000); // Wait 2-3 seconds for state change
-await testdriver.screenshot(); // Now capture the updated state
+.testdriver/screenshots/login.test/
+  001-find-before-L15-email-input.png
+  002-find-after-L15-email-input.png
+  003-click-before-L16-email-input.png
+  004-click-after-L16-email-input.png
+  005-type-before-L17-userexamplecom.png
+  006-type-after-L17-userexamplecom.png
 ```
 
-This is especially important for:
-- Navigation clicks (page transitions)
-- Button clicks that open modals or dialogs
-- Form submissions
-- Actions that trigger AJAX requests or animations
-- Any interaction where visual feedback takes time to appear
-
-**Screenshot file organization:**
-
-```
-.testdriver/
-  screenshots/
-    login.test/           # Folder per test file
-      screenshot-1737633600000.png
-    checkout.test/
-      screenshot-1737633700000.png
-```
+**Filename format:** `<seq>-<action>-<phase>-L<line>-<description>.png`
 
 > **Note:** The screenshot folder for each test file is automatically cleared when the test starts.
 
@@ -295,30 +264,30 @@ session_start({ type: "chrome", url: "https://your-app.com/login", testFile: "te
 → Response includes: "ACTION REQUIRED: Append this code..."
 → ⚠️ IMMEDIATELY write to tests/login.test.mjs:
    await testdriver.provision.chrome({ url: "https://your-app.com/login" });
-   await testdriver.screenshot(); // Capture initial page state
 ```
 
 This provisions a sandbox with Chrome and navigates to your URL. You'll see a screenshot of the initial page.
 
+> **Note**: Screenshots are captured automatically before/after each command. The generated code no longer includes manual `screenshot()` calls.
+
 ### Step 2: Interact with the App
 
-Find elements and interact with them. **Write code to file after EACH action, including screenshots for debugging:**
+Find elements and interact with them. **Write code to file after EACH action:**
 
 ```
 find_and_click({ description: "email input field" })
 → Returns: screenshot with element highlighted
 → ⚠️ IMMEDIATELY append to test file:
    await testdriver.find("email input field").click();
-   await testdriver.wait(2000); // Wait for state change
-   await testdriver.screenshot(); // Capture after click
 
 type({ text: "user@example.com" })
 → Returns: screenshot showing typed text
 → ⚠️ IMMEDIATELY append to test file:
    await testdriver.type("user@example.com");
-   await testdriver.screenshot(); // Capture after typing
 ```
 
+> **Note**: Screenshots are automatically captured before/after each command. Each screenshot filename includes the line number (e.g., `001-click-before-L42-email-input.png`).
+
 ### Step 3: Verify Actions Succeeded (For Your Understanding)
 
 After actions, use `check` to verify they worked. This is for YOUR understanding - does NOT generate code:
@@ -336,7 +305,6 @@ Use `assert` for pass/fail conditions. This DOES generate code for the test file
 assert({ assertion: "the dashboard is visible" })
 → Returns: pass/fail with screenshot
 → ⚠️ IMMEDIATELY append to test file:
-   await testdriver.screenshot(); // Capture before assertion
    const assertResult = await testdriver.assert("the dashboard is visible");
    expect(assertResult).toBeTruthy();
 ```
@@ -372,28 +340,97 @@ Analyze the output, fix any issues, and iterate until the test passes.
 | `assert` | AI-powered boolean assertion - GENERATES CODE for test files |
 | `exec` | Execute JavaScript, shell, or PowerShell in sandbox |
 | `screenshot` | Capture screenshot - **only use when user explicitly asks** |
-| `list_local_screenshots` | List screenshots saved in `.testdriver` directory |
+| `list_local_screenshots` | List/filter screenshots by line, action, phase, regex, etc. |
 | `view_local_screenshot` | View a local screenshot (returns image to AI + displays to user) |
 
 ### Debugging with Local Screenshots
 
-After test runs (successful or failed), you can view saved screenshots to understand test behavior:
+After test runs (successful or failed), you can view saved screenshots to understand test behavior.
+
+**Screenshot filename format:** `<seq>-<action>-<phase>-L<line>-<description>.png`
+Example: `001-click-before-L42-submit-button.png`
 
-**1. List available screenshots:**
+**1. List all screenshots from a test:**
 
 ```
 list_local_screenshots({ directory: "login.test" })
 ```
 
-This returns all screenshots from the specified test file, sorted by modification time (newest first).
+**2. Filter by line number (find what happened at a specific line):**
+
+```
+// Find screenshots from line 42
+list_local_screenshots({ line: 42 })
+
+// Find screenshots from lines 10-20
+list_local_screenshots({ lineRange: { start: 10, end: 20 } })
+```
 
-**2. View specific screenshots:**
+**3. Filter by action type:**
 
 ```
-view_local_screenshot({ path: ".testdriver/screenshots/login.test/after-click.png" })
+// Find all click screenshots
+list_local_screenshots({ action: "click" })
+
+// Find all assertions
+list_local_screenshots({ action: "assert" })
 ```
 
-This displays the screenshot to both you (the AI) and the user via MCP App.
+**4. Filter by phase (before/after):**
+
+```
+// See state BEFORE actions (useful for debugging what was visible)
+list_local_screenshots({ phase: "before" })
+
+// See state AFTER actions (useful for verifying results)
+list_local_screenshots({ phase: "after" })
+```
+
+**5. Filter by regex pattern:**
+
+```
+// Find screenshots related to login
+list_local_screenshots({ pattern: "login|signin" })
+
+// Find button-related screenshots
+list_local_screenshots({ pattern: "button.*click" })
+```
+
+**6. Filter by sequence number:**
+
+```
+// Find screenshots 1-5 (first 5 actions)
+list_local_screenshots({ sequenceRange: { start: 1, end: 5 } })
+```
+
+**7. Sort results:**
+
+```
+// Sort by execution order (useful for understanding flow)
+list_local_screenshots({ sortBy: "sequence" })
+
+// Sort by line number (useful for tracing back to code)
+list_local_screenshots({ sortBy: "line" })
+
+// Sort by modified time (default - newest first)
+list_local_screenshots({ sortBy: "modified" })
+```
+
+**8. Combine filters:**
+
+```
+// Find click screenshots at line 42
+list_local_screenshots({ directory: "checkout.test", line: 42, action: "click" })
+
+// Find all "before" screenshots in lines 10-30
+list_local_screenshots({ lineRange: { start: 10, end: 30 }, phase: "before" })
+```
+
+**9. View a screenshot:**
+
+```
+view_local_screenshot({ path: ".testdriver/screenshots/login.test/001-click-before-L42-submit-button.png" })
+```
 
 **When to use screenshot viewing:**
 
@@ -402,17 +439,18 @@ This displays the screenshot to both you (the AI) and the user via MCP App.
 - **Comparing test runs** - View screenshots from multiple runs to identify flaky behavior
 - **Verifying test logic** - Before running a test, view screenshots from previous runs to understand the UI flow
 
-**Workflow example:**
+**Debugging workflow example:**
 
 ```
-# Test failed, let's debug
-list_local_screenshots({ directory: "checkout.test" })
+# Test failed at line 42, let's see what happened
+list_local_screenshots({ line: 42 })
 
-# View the last few screenshots to see what happened
-view_local_screenshot({ path: ".testdriver/screenshots/checkout.test/screenshot-1737633620000.png" })
-view_local_screenshot({ path: ".testdriver/screenshots/checkout.test/before-assertion.png" })
+# View the before/after state at that line
+view_local_screenshot({ path: ".testdriver/screenshots/checkout.test/005-click-before-L42-submit-button.png" })
+view_local_screenshot({ path: ".testdriver/screenshots/checkout.test/006-click-after-L42-submit-button.png" })
 
-# Analyze the UI state and update test code accordingly
+# Check what the screen looked like before the failing action
+list_local_screenshots({ directory: "checkout.test", phase: "before", limit: 10 })
 ```
 
 ### Tips for MCP Workflow
@@ -437,28 +475,28 @@ view_local_screenshot({ path: ".testdriver/screenshots/checkout.test/before-asse
 
 ```javascript
 // Development workflow example
+// Note: Screenshots are automatically captured before/after each command!
 it("should incrementally build test", async (context) => {
   const testdriver = TestDriver(context);
   await testdriver.provision.chrome({ url: "https://example.com" });
-  await testdriver.screenshot(); // Capture initial state
+  // Automatic screenshot: 001-provision-after-L3-chrome.png
 
   // Step 1: Find and inspect
   const element = await testdriver.find("Some button");
   console.log("Element found:", element.found());
   console.log("Coordinates:", element.x, element.y);
   console.log("Confidence:", element.confidence);
-  await testdriver.screenshot(); // Capture after find
+  // Automatic screenshot: 002-find-after-L7-some-button.png
 
   // Step 2: Interact
   await element.click();
-  await testdriver.wait(2000); // Wait for state change
-  await testdriver.screenshot(); // Capture after click
+  // Automatic screenshot: 003-click-after-L13-element.png
 
-  // Step 3: Assert and log
-  await testdriver.screenshot(); // Capture before assertion
+  // Step 3: Assert
   const result = await testdriver.assert("Something happened");
   console.log("Assertion result:", result);
   expect(result).toBeTruthy();
+  // Automatic screenshot: 004-assert-after-L17-something-happened.png
 
   // Then add more steps...
 });
@@ -476,6 +514,7 @@ const testdriver = TestDriver(context, {
   resolution: "1366x768", // Sandbox resolution
   cache: true, // Enable element caching (default: true)
   cacheKey: "my-test", // Cache key for element finding
+  autoScreenshots: true, // Capture screenshots before/after each command (default: true)
 });
 ```
 
@@ -550,36 +589,36 @@ const date = await testdriver.exec("pwsh", "Get-Date", 5000);
 
 ### Capturing Screenshots
 
-**Add screenshots liberally throughout your tests** for debugging. When a test fails, you'll have a visual trail showing exactly what happened at each step.
-
-```javascript
-// Basic screenshot - automatically saved to .testdriver/screenshots/<test-file>/
-await testdriver.screenshot();
-
-// Capture with mouse cursor visible
-await testdriver.screenshot(1, false, true);
+**Screenshots are captured automatically** before and after each SDK command (click, type, find, assert, etc.). Each screenshot filename includes:
+- Sequential number for chronological ordering
+- Action name (e.g., `click`, `find`, `assert`)
+- Phase (`before` or `after`)
+- Line number from your test file
+- Description from the command
 
-// Recommended pattern: screenshot after every significant action
-await testdriver.provision.chrome({ url: "https://example.com" });
-await testdriver.screenshot(); // After page load
+Example filenames:
+- `001-provision-after-L8-chrome.png`
+- `002-find-before-L12-login-button.png`
+- `003-click-after-L12-element.png`
 
-await testdriver.find("Login button").click();
-await testdriver.wait(2000); // Wait for state change
-await testdriver.screenshot(); // After click
+Screenshots are saved to `.testdriver/screenshots/<test-file>/`.
 
-await testdriver.type("user@example.com");
-await testdriver.screenshot(); // After typing
+To disable automatic screenshots:
+```javascript
+const testdriver = TestDriver(context, { autoScreenshots: false });
+```
 
-await testdriver.screenshot(); // Before assertion
-const result = await testdriver.assert("dashboard is visible");
+For manual screenshots (e.g., with mouse cursor visible):
+```javascript
+await testdriver.screenshot(1, false, true);
 ```
 
 ## Tips for Agents
 
 1. **⚠️ WRITE CODE IMMEDIATELY** - After EVERY successful MCP action, append the generated code to the test file RIGHT AWAY. Do NOT wait until the session ends.
 2. **⚠️ RUN TESTS YOURSELF** - Do NOT tell the user to run tests. YOU must run the tests using `npx vitest run <testFile> --reporter=dot`. Always use `--reporter=dot` for cleaner output. Analyze the output and iterate until the test passes. **Always share the test report link** (e.g., `https://app.testdriver.ai/projects/.../reports/...`) with the user after each run.
-3. **⚠️ ADD SCREENSHOTS LIBERALLY** - Include `await testdriver.screenshot()` throughout your tests: after provision, before/after clicks, after typing, and before assertions. This creates a visual trail that makes debugging failures much easier.
-4. **⚠️ USE SCREENSHOT VIEWING FOR DEBUGGING** - When tests fail, use `list_local_screenshots` and `view_local_screenshot` MCP commands to see exactly what the UI looked like. This is often faster than re-running the test.
+3. **Screenshots are automatic** - TestDriver captures screenshots before/after every command by default. Each screenshot filename includes the line number (e.g., `001-click-before-L42-submit-button.png`) making it easy to trace issues.
+4. **⚠️ USE SCREENSHOT VIEWING FOR DEBUGGING** - When tests fail, use `list_local_screenshots` and `view_local_screenshot` MCP commands to see exactly what the UI looked like. The filenames tell you which line of code triggered each screenshot.
 5. **⚠️ NEVER USE `.wait()`** - Do NOT use any `.wait()` method. Instead, use `find()` with a `timeout` option to poll for elements, or use `assert()` / `check()` to verify state. Explicit waits are flaky and slow.
 6. **Use MCP tools for development** - Build tests interactively with visual feedback
 7. **Always check `sdk.d.ts`** for method signatures and types when debugging generated tests

package/interfaces/cli/commands/init.js

@@ -8,6 +8,14 @@ const readline = require("readline");
 const os = require("os");
 const { execSync } = require("child_process");
 
+// Load .env file for CLI usage (TD_API_ROOT, etc.)
+require("dotenv").config();
+
+// API configuration
+const API_BASE_URL = process.env.TD_API_ROOT || "https://v6.testdriver.ai";
+const POLL_INTERVAL = 5000; // 5 seconds
+const POLL_TIMEOUT = 900000; // 15 minutes
+
 /**
  * Init command - scaffolds Vitest SDK example tests for TestDriver
  */
@@ -79,8 +87,33 @@ class InitCommand extends BaseCommand {
     }
 
     console.log(chalk.cyan("  Setting up your TestDriver API key...\n"));
+
+    // Ask user how they want to authenticate
+    const choice = await this.askChoice(
+      "  How would you like to authenticate?\n",
+      [
+        { key: "1", label: "Login with browser", description: "(recommended)" },
+        { key: "2", label: "Enter API key manually", description: "" },
+      ],
+    );
+
+    if (choice === "1") {
+      // Browser login flow
+      try {
+        const apiKey = await this.browserLogin();
+        if (apiKey) {
+          console.log(chalk.green("\n  ✓ Logged in successfully!\n"));
+          return apiKey;
+        }
+      } catch (error) {
+        console.log(chalk.yellow(`\n  ⚠️  Browser login failed: ${error.message}\n`));
+        console.log(chalk.gray("  Falling back to manual API key entry...\n"));
+      }
+    }
+
+    // Manual API key entry
     console.log(
-      chalk.gray("  Get your API key from: https://console.testdriver.ai/team"),
+      chalk.gray("  Get your API key from: https://console.testdriver.ai/team\n"),
     );
 
     // Ask if user wants to open the browser
@@ -89,7 +122,6 @@ class InitCommand extends BaseCommand {
     );
     if (shouldOpen) {
       try {
-        // Dynamic import for ES module
         const open = (await import("open")).default;
         await open("https://console.testdriver.ai/team");
         console.log(chalk.gray("  Opening browser...\n"));
@@ -119,6 +151,119 @@ class InitCommand extends BaseCommand {
     }
   }
 
+  /**
+   * Browser-based login flow using device code
+   * @returns {Promise<string>} The API key
+   */
+  async browserLogin() {
+    // Step 1: Create device code
+    process.stdout.write(chalk.gray("  Requesting authorization code..."));
+    
+    const createResponse = await fetch(`${API_BASE_URL}/auth/device/code`, {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+    });
+
+    if (!createResponse.ok) {
+      throw new Error("Failed to create device code");
+    }
+
+    const { device_code, verification_uri, expires_in, interval } = await createResponse.json();
+    console.log(chalk.green(" done\n"));
+
+    // Step 2: Open browser
+    console.log(chalk.cyan(`  Opening browser to authorize CLI...\n`));
+    console.log(chalk.gray(`  If browser doesn't open, visit:\n  ${verification_uri}\n`));
+
+    try {
+      const open = (await import("open")).default;
+      await open(verification_uri);
+    } catch (error) {
+      // Browser didn't open, user can use the URL manually
+    }
+
+    // Step 3: Poll for token
+    const pollInterval = (interval || 5) * 1000;
+    const timeout = (expires_in || 900) * 1000;
+    const startTime = Date.now();
+
+    process.stdout.write(chalk.gray("  Waiting for authorization..."));
+    
+    // Start spinner
+    const spinnerFrames = ["⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"];
+    let spinnerIndex = 0;
+    const spinnerInterval = setInterval(() => {
+      process.stdout.write(`\r  Waiting for authorization... ${spinnerFrames[spinnerIndex]}`);
+      spinnerIndex = (spinnerIndex + 1) % spinnerFrames.length;
+    }, 100);
+
+    try {
+      while (Date.now() - startTime < timeout) {
+        await this.sleep(pollInterval);
+
+        const tokenResponse = await fetch(`${API_BASE_URL}/auth/device/token`, {
+          method: "POST",
+          headers: { "Content-Type": "application/json" },
+          body: JSON.stringify({ deviceCode: device_code }),
+        });
+
+        const data = await tokenResponse.json();
+
+        if (tokenResponse.ok && data.apiKey) {
+          clearInterval(spinnerInterval);
+          process.stdout.write("\r  Waiting for authorization... " + chalk.green("✓") + "\n");
+          return data.apiKey;
+        }
+
+        if (data.error === "expired_token") {
+          clearInterval(spinnerInterval);
+          throw new Error("Authorization timed out. Please try again.");
+        }
+
+        // authorization_pending - continue polling
+      }
+
+      clearInterval(spinnerInterval);
+      throw new Error("Authorization timed out. Please try again.");
+    } catch (error) {
+      clearInterval(spinnerInterval);
+      process.stdout.write("\n");
+      throw error;
+    }
+  }
+
+  /**
+   * Ask user to choose from a list of options
+   */
+  async askChoice(question, options) {
+    return new Promise((resolve) => {
+      const rl = readline.createInterface({
+        input: process.stdin,
+        output: process.stdout,
+      });
+
+      console.log(question);
+      for (const opt of options) {
+        const desc = opt.description ? chalk.gray(` ${opt.description}`) : "";
+        console.log(`  ${chalk.cyan(opt.key)}. ${opt.label}${desc}`);
+      }
+      console.log("");
+
+      rl.question("  Enter choice [1]: ", (answer) => {
+        rl.close();
+        const normalized = answer.trim() || "1";
+        resolve(normalized);
+      });
+    });
+  }
+
+  /**
+   * Sleep for a given number of milliseconds
+   */
+  sleep(ms) {
+    return new Promise((resolve) => setTimeout(resolve, ms));
+  }
+
   /**
    * Prompt for hidden input (like password)
    */

package/interfaces/vitest-plugin.mjs

@@ -1,5 +1,6 @@
 import { execSync } from "child_process";
 import crypto from "crypto";
+import fs from "fs";
 import { createRequire } from "module";
 import path from "path";
 import { postOrUpdateTestResults } from "../lib/github-comment.mjs";
@@ -1201,6 +1202,49 @@ function getGitInfo() {
 // GitHub Comment Helper
 // ============================================================================
 
+/**
+ * Extract PR number from GitHub Actions environment
+ * Checks multiple sources: env vars, event file, and GITHUB_REF
+ * @returns {string|null} PR number or null if not found
+ */
+function extractPRNumber() {
+  // Try direct environment variables first
+  let prNumber =
+    process.env.GITHUB_PR_NUMBER ||
+    process.env.TD_GITHUB_PR ||
+    process.env.PR_NUMBER;
+
+  if (prNumber) {
+    return prNumber;
+  }
+
+  // Try to extract from GitHub Actions event path
+  if (process.env.GITHUB_EVENT_PATH) {
+    try {
+      const eventData = JSON.parse(
+        fs.readFileSync(process.env.GITHUB_EVENT_PATH, "utf8"),
+      );
+      if (eventData.pull_request?.number) {
+        return String(eventData.pull_request.number);
+      }
+    } catch (err) {
+      logger.debug("Could not read GitHub event file:", err.message);
+    }
+  }
+
+  // Try to extract from GITHUB_REF (refs/pull/123/merge or refs/pull/123/head)
+  if (process.env.GITHUB_REF) {
+    const match = process.env.GITHUB_REF.match(
+      /refs\/pull\/(\d+)\/(merge|head)/,
+    );
+    if (match) {
+      return match[1];
+    }
+  }
+
+  return null;
+}
+
 /**
  * Post GitHub comment with test results if enabled
  * Checks for GitHub token and PR number in environment variables
@@ -1220,7 +1264,7 @@ async function postGitHubCommentIfEnabled(testRunUrl, stats, completeData) {
 
     // Check if GitHub comment posting is enabled
     const githubToken = process.env.GITHUB_TOKEN || process.env.GH_TOKEN;
-    const prNumber = process.env.GITHUB_PR_NUMBER;
+    const prNumber = extractPRNumber();
     const commitSha = process.env.GITHUB_SHA || pluginState.gitInfo.commit;
 
     // Only post if we have a token and either a PR number or commit SHA

package/mcp-server/dist/server.mjs

@@ -1388,20 +1388,57 @@ server.registerTool("exec", {
         throw error;
     }
 });
+function parseScreenshotFilename(filename) {
+    // Match pattern: 001-click-before-L42-submit-button.png or 001-click-error-L42-submit-button.png
+    const match = filename.match(/^(\d+)-([a-z]+)-(before|after|error)-L(\d+)-(.+)\.png$/i);
+    if (match) {
+        return {
+            sequence: parseInt(match[1], 10),
+            action: match[2].toLowerCase(),
+            phase: match[3].toLowerCase(),
+            lineNumber: parseInt(match[4], 10),
+            description: match[5],
+        };
+    }
+    return {};
+}
 // List Local Screenshots - lists screenshots saved to .testdriver directory
 server.registerTool("list_local_screenshots", {
-    description: `List screenshots saved in the .testdriver directory.
+    description: `List and filter screenshots saved in the .testdriver directory.
+
+Screenshots from auto-screenshot feature use the format: <seq>-<action>-<phase>-L<line>-<description>.png
+Example: 001-click-before-L42-submit-button.png
 
-This tool helps you find screenshots that have been saved during test runs or via the screenshot tool.
-Screenshots are organized in subdirectories like 'mcp-screenshots' and 'screenshots'.
+This tool supports powerful filtering to find specific screenshots:
+- By test file (directory)
+- By line number or range
+- By action type (click, find, type, assert, etc.)
+- By phase (before/after/error - error screenshots are captured when actions fail)
+- By regex pattern on filename
+- By sequence number range
 
 Returns a list of screenshot paths that can be viewed with the 'view_local_screenshot' tool.`,
     inputSchema: z.object({
-        directory: z.string().optional().describe("Subdirectory to list (e.g., 'mcp-screenshots', 'screenshots'). If not provided, lists all subdirectories."),
+        directory: z.string().optional().describe("Test file or subdirectory to search (e.g., 'login.test', 'mcp-screenshots'). If not provided, searches all."),
+        line: z.number().optional().describe("Filter by exact line number from test file (e.g., 42 matches L42)"),
+        lineRange: z.object({
+            start: z.number().describe("Start line number (inclusive)"),
+            end: z.number().describe("End line number (inclusive)"),
+        }).optional().describe("Filter by line number range (e.g., { start: 10, end: 20 })"),
+        action: z.string().optional().describe("Filter by action type: click, find, type, assert, provision, scroll, hover, etc."),
+        phase: z.enum(["before", "after", "error"]).optional().describe("Filter by phase: 'before' (pre-action), 'after' (post-action), or 'error' (when action fails)"),
+        pattern: z.string().optional().describe("Regex pattern to match against filename (e.g., 'submit|login' or 'button.*click')"),
+        sequence: z.number().optional().describe("Filter by exact sequence number"),
+        sequenceRange: z.object({
+            start: z.number().describe("Start sequence (inclusive)"),
+            end: z.number().describe("End sequence (inclusive)"),
+        }).optional().describe("Filter by sequence range (e.g., { start: 1, end: 10 })"),
+        limit: z.number().optional().describe("Maximum number of results to return (default: 50)"),
+        sortBy: z.enum(["modified", "sequence", "line"]).optional().describe("Sort by: 'modified' (newest first), 'sequence' (execution order), or 'line' (line number). Default: 'modified'"),
     }),
 }, async (params) => {
     const startTime = Date.now();
-    logger.info("list_local_screenshots: Starting", { directory: params.directory });
+    logger.info("list_local_screenshots: Starting", { ...params });
     try {
         // Find .testdriver directory - check current working directory and common locations
         const possiblePaths = [
@@ -1420,6 +1457,16 @@ Returns a list of screenshot paths that can be viewed with the 'view_local_scree
             return createToolResult(false, "No .testdriver directory found. Screenshots are saved here during test runs.", { error: "Directory not found" });
         }
         const screenshots = [];
+        // Compile regex pattern if provided
+        let regexPattern = null;
+        if (params.pattern) {
+            try {
+                regexPattern = new RegExp(params.pattern, "i");
+            }
+            catch {
+                return createToolResult(false, `Invalid regex pattern: ${params.pattern}`, { error: "Invalid regex" });
+            }
+        }
         // Function to recursively find PNG files
         const findPngFiles = (dir) => {
             if (!fs.existsSync(dir))
@@ -1434,49 +1481,120 @@ Returns a list of screenshot paths that can be viewed with the 'view_local_scree
                     }
                 }
                 else if (entry.isFile() && entry.name.toLowerCase().endsWith(".png")) {
+                    const parsed = parseScreenshotFilename(entry.name);
+                    // Apply filters
+                    if (params.line !== undefined && parsed.lineNumber !== params.line)
+                        continue;
+                    if (params.lineRange && (parsed.lineNumber === undefined ||
+                        parsed.lineNumber < params.lineRange.start ||
+                        parsed.lineNumber > params.lineRange.end))
+                        continue;
+                    if (params.action && parsed.action !== params.action.toLowerCase())
+                        continue;
+                    if (params.phase && parsed.phase !== params.phase)
+                        continue;
+                    if (params.sequence !== undefined && parsed.sequence !== params.sequence)
+                        continue;
+                    if (params.sequenceRange && (parsed.sequence === undefined ||
+                        parsed.sequence < params.sequenceRange.start ||
+                        parsed.sequence > params.sequenceRange.end))
+                        continue;
+                    if (regexPattern && !regexPattern.test(entry.name))
+                        continue;
                     const stats = fs.statSync(fullPath);
                     screenshots.push({
                         path: fullPath,
                         name: entry.name,
                         modified: stats.mtime,
                         size: stats.size,
+                        parsed,
                     });
                 }
             }
         };
         findPngFiles(testdriverDir);
-        // Sort by modification time (newest first)
-        screenshots.sort((a, b) => b.modified.getTime() - a.modified.getTime());
+        // Sort based on sortBy parameter
+        const sortBy = params.sortBy || "modified";
+        if (sortBy === "modified") {
+            screenshots.sort((a, b) => b.modified.getTime() - a.modified.getTime());
+        }
+        else if (sortBy === "sequence") {
+            screenshots.sort((a, b) => (a.parsed.sequence ?? Infinity) - (b.parsed.sequence ?? Infinity));
+        }
+        else if (sortBy === "line") {
+            screenshots.sort((a, b) => (a.parsed.lineNumber ?? Infinity) - (b.parsed.lineNumber ?? Infinity));
+        }
         const duration = Date.now() - startTime;
         logger.info("list_local_screenshots: Completed", { count: screenshots.length, duration });
         if (screenshots.length === 0) {
-            return createToolResult(true, "No screenshots found in .testdriver directory.", {
+            const filters = [];
+            if (params.directory)
+                filters.push(`directory=${params.directory}`);
+            if (params.line)
+                filters.push(`line=${params.line}`);
+            if (params.lineRange)
+                filters.push(`lineRange=${params.lineRange.start}-${params.lineRange.end}`);
+            if (params.action)
+                filters.push(`action=${params.action}`);
+            if (params.phase)
+                filters.push(`phase=${params.phase}`);
+            if (params.pattern)
+                filters.push(`pattern=${params.pattern}`);
+            if (params.sequence)
+                filters.push(`sequence=${params.sequence}`);
+            if (params.sequenceRange)
+                filters.push(`sequenceRange=${params.sequenceRange.start}-${params.sequenceRange.end}`);
+            const filterMsg = filters.length > 0 ? ` with filters: ${filters.join(", ")}` : "";
+            return createToolResult(true, `No screenshots found in .testdriver directory${filterMsg}.`, {
                 action: "list_local_screenshots",
                 count: 0,
                 directory: testdriverDir,
+                filters: params,
                 duration
             });
         }
-        // Format the list for display
-        const screenshotList = screenshots.slice(0, 50).map((s, i) => {
+        const limit = params.limit || 50;
+        const limitedScreenshots = screenshots.slice(0, limit);
+        // Format the list for display with parsed info
+        const screenshotList = limitedScreenshots.map((s, i) => {
             const relativePath = path.relative(testdriverDir, s.path);
             const sizeKB = Math.round(s.size / 1024);
             const timeAgo = formatTimeAgo(s.modified);
-            return `${i + 1}. ${relativePath} (${sizeKB}KB, ${timeAgo})`;
+            // Add parsed info if available
+            const parts = [`${i + 1}. ${relativePath}`];
+            const meta = [];
+            if (s.parsed.lineNumber)
+                meta.push(`L${s.parsed.lineNumber}`);
+            if (s.parsed.action)
+                meta.push(s.parsed.action);
+            if (s.parsed.phase)
+                meta.push(s.parsed.phase);
+            meta.push(`${sizeKB}KB`);
+            meta.push(timeAgo);
+            parts.push(`(${meta.join(", ")})`);
+            return parts.join(" ");
         }).join("\n");
-        const message = screenshots.length > 50
-            ? `Found ${screenshots.length} screenshots (showing 50 most recent):\n\n${screenshotList}`
-            : `Found ${screenshots.length} screenshot(s):\n\n${screenshotList}`;
+        const message = screenshots.length > limit
+            ? `Found ${screenshots.length} screenshots (showing ${limit} results, sorted by ${sortBy}):\n\n${screenshotList}`
+            : `Found ${screenshots.length} screenshot(s) (sorted by ${sortBy}):\n\n${screenshotList}`;
         return createToolResult(true, message, {
             action: "list_local_screenshots",
             count: screenshots.length,
+            returned: limitedScreenshots.length,
             directory: testdriverDir,
-            screenshots: screenshots.slice(0, 50).map(s => ({
+            filters: params,
+            sortBy,
+            screenshots: limitedScreenshots.map(s => ({
                 path: s.path,
                 relativePath: path.relative(testdriverDir, s.path),
                 name: s.name,
                 modified: s.modified.toISOString(),
                 sizeBytes: s.size,
+                sequence: s.parsed.sequence,
+                action: s.parsed.action,
+                phase: s.parsed.phase,
+                lineNumber: s.parsed.lineNumber,
+                description: s.parsed.description,
             })),
             duration
         });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "testdriverai",
-  "version": "7.2.80",
+  "version": "7.2.82",
   "description": "Next generation autonomous AI agent for end-to-end testing of web & desktop",
   "main": "sdk.js",
   "types": "sdk.d.ts",
@@ -132,7 +132,7 @@
     "mocha": "^10.8.2",
     "node-addon-api": "^8.0.0",
     "prettier": "3.3.3",
-    "testdriverai": "^7.2.60",
+    "testdriverai": "^7.2.79",
     "vitest": "^4.0.18"
   },
   "optionalDependencies": {

package/sdk.d.ts

@@ -263,6 +263,13 @@ export interface TestDriverOptions {
   reconnect?: boolean;
   /** Enable/disable Dashcam video recording (default: true) */
   dashcam?: boolean;
+  /**
+   * Enable automatic screenshots before and after each command (default: true)
+   * Screenshots are saved to .testdriver/screenshots/<test>/ with descriptive filenames
+   * Format: <seq>-<action>-<phase>-L<line>-<description>.png
+   * Example: 001-click-before-L42-submit-button.png
+   */
+  autoScreenshots?: boolean;
   /** Redraw configuration for screen change detection */
   redraw?:
     | boolean

package/sdk.js

@@ -69,6 +69,53 @@ function getCallerFileHash() {
   }
 }
 
+/**
+ * Get detailed caller information including file path, line number, and column
+ * Used for automatic screenshot naming to identify which line of code triggered an action
+ * @param {number} [skipFrames=0] - Additional frames to skip in the stack trace
+ * @returns {{filePath: string|null, line: number|null, column: number|null, functionName: string|null}}
+ */
+function getCallerInfo(skipFrames = 0) {
+  const originalPrepareStackTrace = Error.prepareStackTrace;
+  try {
+    const err = new Error();
+    Error.prepareStackTrace = (_, stack) => stack;
+    const stack = err.stack;
+    Error.prepareStackTrace = originalPrepareStackTrace;
+
+    // Look for the first file that's not sdk.js, hooks.mjs, or node internals
+    let skipped = 0;
+    for (const callSite of stack) {
+      const fileName = callSite.getFileName();
+      if (
+        fileName &&
+        !fileName.includes("sdk.js") &&
+        !fileName.includes("hooks.mjs") &&
+        !fileName.includes("hooks.js") &&
+        !fileName.includes("node_modules") &&
+        !fileName.includes("node:internal") &&
+        fileName !== "evalmachine.<anonymous>"
+      ) {
+        if (skipped < skipFrames) {
+          skipped++;
+          continue;
+        }
+        return {
+          filePath: fileName,
+          line: callSite.getLineNumber(),
+          column: callSite.getColumnNumber(),
+          functionName: callSite.getFunctionName(),
+        };
+      }
+    }
+  } catch (error) {
+    // Silently fail and return nulls
+  } finally {
+    Error.prepareStackTrace = originalPrepareStackTrace;
+  }
+  return { filePath: null, line: null, column: null, functionName: null };
+}
+
 /**
  * Custom error class for element operation failures
  * Includes debugging information like screenshots and AI responses
@@ -1430,6 +1477,12 @@ class TestDriverSDK {
     this._lastPromiseSettled = true;
     this._lastCommandName = null;
 
+    // Auto-screenshots configuration
+    // When enabled, automatically captures screenshots before/after each command
+    // Screenshots are saved to .testdriver/screenshots/<test>/ with descriptive names
+    this.autoScreenshots = options.autoScreenshots !== false;
+    this._screenshotSequence = 0; // Counter for sequential screenshot naming
+
     // Set up command methods that lazy-await connection
     this._setupCommandMethods();
   }
@@ -2733,10 +2786,18 @@ CAPTCHA_SOLVER_EOF`,
 
       this._ensureConnected();
 
+      // Get caller info for auto-screenshot naming
+      const callerInfo = this.autoScreenshots ? getCallerInfo() : null;
+
       // Track this promise for unawaited detection
       this._lastCommandName = "find";
       this._lastPromiseSettled = false;
 
+      // Take "before" screenshot if enabled
+      if (this.autoScreenshots) {
+        await this._saveAutoScreenshot("find", "before", callerInfo, description);
+      }
+
       const element = new Element(
         description,
         this,
@@ -2744,6 +2805,12 @@ CAPTCHA_SOLVER_EOF`,
         this.commands,
       );
       const result = await element.find(null, options);
+
+      // Take "after" screenshot if enabled
+      if (this.autoScreenshots) {
+        await this._saveAutoScreenshot("find", "after", callerInfo, description);
+      }
+
       this._lastPromiseSettled = true;
       return result;
     })();
@@ -2792,10 +2859,18 @@ CAPTCHA_SOLVER_EOF`,
 
     this._ensureConnected();
 
+    // Get caller info for auto-screenshot naming
+    const callerInfo = this.autoScreenshots ? getCallerInfo() : null;
+
     // Track this promise for unawaited detection
     this._lastCommandName = "findAll";
     this._lastPromiseSettled = false;
 
+    // Take "before" screenshot if enabled
+    if (this.autoScreenshots) {
+      await this._saveAutoScreenshot("findAll", "before", callerInfo, description);
+    }
+
     // Capture absolute timestamp at the very start of the command
     // Frontend will calculate relative time using: timestamp - replay.clientStartDate
     const absoluteTimestamp = Date.now();
@@ -2951,6 +3026,11 @@ CAPTCHA_SOLVER_EOF`,
           this.emitter.emit(events.log.debug, `  Time: ${duration}ms`);
         }
 
+        // Take "after" screenshot if enabled
+        if (this.autoScreenshots) {
+          await this._saveAutoScreenshot("findAll", "after", callerInfo, description);
+        }
+
         this._lastPromiseSettled = true;
         return elements;
       } else {
@@ -2989,6 +3069,11 @@ CAPTCHA_SOLVER_EOF`,
             });
         }
 
+        // Take "after" screenshot if enabled (no elements found)
+        if (this.autoScreenshots) {
+          await this._saveAutoScreenshot("findAll", "after", callerInfo, description);
+        }
+
         // No elements found - return empty array
         this._lastPromiseSettled = true;
         return [];
@@ -3025,6 +3110,11 @@ CAPTCHA_SOLVER_EOF`,
           });
       }
 
+      // Take "error" screenshot if enabled
+      if (this.autoScreenshots) {
+        await this._saveAutoScreenshot("findAll", "error", callerInfo, description);
+      }
+
       this._lastPromiseSettled = true;
       return [];
     }
@@ -3072,6 +3162,7 @@ CAPTCHA_SOLVER_EOF`,
   /**
    * Dynamically set up command methods based on available commands
    * This creates camelCase methods that wrap the underlying command functions
+   * When autoScreenshots is enabled, captures before/after screenshots for each command
    * @private
    */
   _setupCommandMethods() {
@@ -3096,6 +3187,53 @@ CAPTCHA_SOLVER_EOF`,
       exec: "exec",
     };
 
+    // Helper to extract a description from command args for screenshot naming
+    const getDescriptionFromArgs = (methodName, args) => {
+      if (!args || args.length === 0) return "";
+      const firstArg = args[0];
+      
+      switch (methodName) {
+        case "type":
+          // For type, use the text being typed (truncated)
+          return typeof firstArg === "string" ? firstArg.substring(0, 20) : "";
+        case "pressKeys":
+          // For pressKeys, show the keys
+          return Array.isArray(firstArg) ? firstArg.join("+") : String(firstArg);
+        case "click":
+        case "hover":
+          // For click/hover, try to get coordinates or prompt
+          if (typeof firstArg === "object" && firstArg !== null) {
+            return firstArg.prompt || `${firstArg.x},${firstArg.y}`;
+          }
+          return typeof firstArg === "number" ? `${firstArg},${args[1]}` : "";
+        case "scroll":
+          // For scroll, show direction
+          return typeof firstArg === "string" ? firstArg : "down";
+        case "waitForText":
+        case "scrollUntilText":
+          // For text-based commands, use the text
+          if (typeof firstArg === "object" && firstArg !== null) {
+            return firstArg.text || "";
+          }
+          return typeof firstArg === "string" ? firstArg : "";
+        case "focusApplication":
+          // For focus, use the app name
+          return typeof firstArg === "string" ? firstArg : "";
+        case "assert":
+        case "extract":
+          // For assert/extract, use the assertion/description
+          return typeof firstArg === "string" ? firstArg.substring(0, 30) : "";
+        case "exec":
+          // For exec, show the language
+          if (typeof firstArg === "object" && firstArg !== null) {
+            return firstArg.language || "code";
+          }
+          return typeof firstArg === "string" ? firstArg : "code";
+        default:
+          return typeof firstArg === "string" ? firstArg.substring(0, 20) : "";
+      }
+    };
+
     // Create SDK methods that lazy-await connection then forward to this.commands
     for (const [commandName, methodName] of Object.entries(commandMapping)) {
       this[methodName] = async function (...args) {
@@ -3115,19 +3253,39 @@ CAPTCHA_SOLVER_EOF`,
 
         this._ensureConnected();
 
-        // Capture the call site for better error reporting
+        // Capture the call site for better error reporting AND for auto-screenshots
         const callSite = {};
         Error.captureStackTrace(callSite, this[methodName]);
 
+        // Get caller info for auto-screenshot naming
+        const callerInfo = this.autoScreenshots ? getCallerInfo() : null;
+        const description = this.autoScreenshots ? getDescriptionFromArgs(methodName, args) : "";
+
         // Track this promise for unawaited detection
         this._lastCommandName = methodName;
         this._lastPromiseSettled = false;
 
         try {
+          // Take "before" screenshot if enabled
+          if (this.autoScreenshots) {
+            await this._saveAutoScreenshot(methodName, "before", callerInfo, description);
+          }
+
           const result = await this.commands[commandName](...args);
+
+          // Take "after" screenshot if enabled
+          if (this.autoScreenshots) {
+            await this._saveAutoScreenshot(methodName, "after", callerInfo, description);
+          }
+
           this._lastPromiseSettled = true;
           return result;
         } catch (error) {
+          // Take "error" screenshot if enabled (instead of "after")
+          if (this.autoScreenshots) {
+            await this._saveAutoScreenshot(methodName, "error", callerInfo, description);
+          }
+
           this._lastPromiseSettled = true;
           // Ensure we have a proper Error object with a message
           let properError = error;
@@ -3212,6 +3370,80 @@ CAPTCHA_SOLVER_EOF`,
     return filePath;
   }
 
+  /**
+   * Save an automatic screenshot with descriptive naming
+   * Used internally when autoScreenshots is enabled
+   * @private
+   * @param {string} actionName - Name of the action (click, type, hover, etc.)
+   * @param {string} phase - 'before' or 'after'
+   * @param {Object} callerInfo - Caller information from getCallerInfo()
+   * @param {string} [description] - Optional description of the action target
+   * @returns {Promise<string|null>} The file path where the screenshot was saved, or null if failed
+   */
+  async _saveAutoScreenshot(actionName, phase, callerInfo, description = "") {
+    if (!this.autoScreenshots || !this.connected) {
+      return null;
+    }
+
+    try {
+      // Increment sequence for unique ordering
+      this._screenshotSequence++;
+      const seq = String(this._screenshotSequence).padStart(3, "0");
+
+      // Extract line number info
+      const lineInfo = callerInfo.line ? `L${callerInfo.line}` : "L???";
+
+      // Sanitize description for filename (remove special chars, limit length)
+      const sanitizedDesc = description
+        .replace(/[^a-zA-Z0-9\s-]/g, "")
+        .replace(/\s+/g, "-")
+        .substring(0, 30)
+        .toLowerCase();
+
+      // Build filename: 001-click-before-L42-submit-button.png
+      const descPart = sanitizedDesc ? `-${sanitizedDesc}` : "";
+      const filename = `${seq}-${actionName}-${phase}-${lineInfo}${descPart}.png`;
+
+      const base64Data = await this.system.captureScreenBase64(1, false, false);
+
+      // Save to .testdriver/screenshots/<test-file-name> directory
+      let screenshotsDir = path.join(process.cwd(), ".testdriver", "screenshots");
+      if (this.testFile) {
+        const testFileName = path.basename(
+          this.testFile,
+          path.extname(this.testFile),
+        );
+        screenshotsDir = path.join(screenshotsDir, testFileName);
+      }
+      if (!fs.existsSync(screenshotsDir)) {
+        fs.mkdirSync(screenshotsDir, { recursive: true });
+      }
+
+      const filePath = path.join(screenshotsDir, filename);
+
+      // Remove data:image/png;base64, prefix if present
+      const cleanBase64 = base64Data.replace(/^data:image\/\w+;base64,/, "");
+      const buffer = Buffer.from(cleanBase64, "base64");
+
+      fs.writeFileSync(filePath, buffer);
+
+      // Debug log in verbose mode
+      const debugMode = process.env.VERBOSE || process.env.DEBUG || process.env.TD_DEBUG;
+      if (debugMode) {
+        this.emitter.emit("log:debug", `📸 Auto-screenshot: ${filename}`);
+      }
+
+      return filePath;
+    } catch (error) {
+      // Don't fail the command if screenshot fails
+      const debugMode = process.env.VERBOSE || process.env.DEBUG || process.env.TD_DEBUG;
+      if (debugMode) {
+        this.emitter.emit("log:debug", `Failed to save auto-screenshot: ${error.message}`);
+      }
+      return null;
+    }
+  }
+
   /**
    * Ensure the SDK is connected before running commands
    * @private