@testdriverai/agent 7.8.0-canary.10

package/ai/skills/testdriver-waiting-for-elements/SKILL.md

@@ -0,0 +1,90 @@
+---
+name: testdriver:waiting-for-elements
+description: Handle async operations and prevent flaky tests
+---
+<!-- Generated from waiting-for-elements.mdx. DO NOT EDIT. -->
+
+## Waiting for Elements
+
+By default, `find()` automatically polls for up to 10 seconds, retrying every 5 seconds until the element is found. This means most elements that appear after short async operations will be found without any extra configuration.
+
+For longer operations, increase the `timeout`:
+
+```javascript
+// Default behavior - polls for up to 10 seconds automatically
+const element = await testdriver.find('Loading complete indicator');
+await element.click();
+
+// Wait up to 30 seconds for slower operations
+const element = await testdriver.find('Loading complete indicator', { timeout: 30000 });
+await element.click();
+
+// Useful after actions that trigger loading states
+await testdriver.find('submit button').click();
+await testdriver.find('success message', { timeout: 15000 });
+
+// Disable polling for instant checks
+const toast = await testdriver.find('notification toast', { timeout: 0 });
+```
+
+## Flake Prevention
+
+TestDriver automatically waits for the screen and network to stabilize after each action using **redraw detection**. This prevents flaky tests caused by animations, loading states, or dynamic content updates.
+
+<Note>
+  Redraw detection adds a small delay after each action but significantly reduces test flakiness.
+</Note>
+
+For example, when clicking a submit button that navigates to a new page:
+
+```javascript
+// Click submit - TestDriver automatically waits for the new page to load
+await testdriver.find('submit button').click();
+
+// By the time this runs, the page has fully loaded and stabilized
+await testdriver.assert('dashboard is displayed');
+await testdriver.find('welcome message');
+```
+
+Without redraw detection, you'd need manual waits or retries to handle the page transition. TestDriver handles this automatically by detecting when the screen stops changing and network requests complete.
+
+You can disable redraw detection or customize its behavior:
+
+```javascript
+// Disable redraw detection for faster tests (less reliable)
+const testdriver = TestDriver(context, { 
+  redraw: false 
+});
+```
+
+Here is an example of customizing redraw detection:
+
+```javascript
+// Fine-tune redraw detection
+const testdriver = TestDriver(context, { 
+  redraw: {
+    enabled: true,
+    diffThreshold: 0.1,      // Pixel difference threshold (0-1)
+    screenRedraw: true,      // Monitor screen changes
+    networkMonitor: true,    // Wait for network idle
+  }
+});
+```
+
+## Simple Delays with `wait()`
+
+For simple pauses — waiting for animations, transitions, or state changes after an action — use `wait()`:
+
+```javascript
+// Wait for an animation to complete
+await testdriver.find('menu toggle').click();
+await testdriver.wait(2000);
+
+// Wait for a page transition to settle
+await testdriver.find('next page button').click();
+await testdriver.wait(1000);
+```
+
+<Note>
+  For waiting for specific **elements** to appear, prefer `find()` with a `timeout` option. Use `wait()` only for simple time-based pauses.
+</Note>

package/ai/skills/testdriver-what-is-testdriver/SKILL.md

@@ -0,0 +1,54 @@
+---
+name: testdriver:what-is-testdriver
+description: Reliably test your most difficult user flows
+---
+<!-- Generated from what-is-testdriver.mdx. DO NOT EDIT. -->
+
+## The problem with modern testing tools
+
+Modern testing tools like Playwright are designed to test a single web application, running in a single browser tab using selectors.
+
+However, selectors are often either unreliable or unavailable in complex scenarios, leading to brittle and flaky tests:
+
+| Challenge | Problem | Examples |
+|-----------|---------|----------|
+| **Fast moving teams** | Frequently change UI structure, breaking CSS/XPath selectors | Agile teams, startups, vibe-coders |
+| **Dynamic content** | Cannot be targeted with selectors | AI chatbots, PDFs, images, videos |
+| **Software you don't own** | May lack proper accessibility attributes | Other websites, extensions, third-party applications |
+| **Multi-application workflows** | Cannot be tested with web-only tools | Desktop apps, browser extensions, IDEs |
+| **Visual states** | Impossible to verify with code-based selectors | Charts, graphs, videos, images, spelling errors, UI layout |
+
+## The TestDriver Solution
+
+TestDriver is a complete testing platform built specifically for handling these scenarios. It consists of a Javascript SDK, hosted infrastructure, and debugging tools that make it easy to write, run, and maintain tests for your most difficult user flows.
+
+### Javascript SDK
+
+Here is an example of a TestDriver test that installs a production Chrome extension from the Chrome Web Store and verifies that it appears in the extensions menu:
+
+```javascript Installing Loom from the Chrome Web Store
+import { describe, expect, it } from "vitest";
+import { TestDriver } from "testdriverai/vitest/hooks";
+
+describe("Chrome Extension Test", () => {
+  const testdriver = TestDriver(context);
+
+  // Launch Chrome with Loom loaded by its Chrome Web Store ID
+  await testdriver.provision.chromeExtension({
+    extensionId: 'liecbddmkiiihnedobmlmillhodjkdmb'
+  });
+
+  // Click on the extensions button (puzzle piece icon) in Chrome toolbar
+  const extensionsButton = await testdriver.find("The puzzle-shaped icon in the Chrome toolbar.");
+  await extensionsButton.click();
+
+  // Look for Loom in the extensions menu
+  const loomExtension = await testdriver.find("Loom extension in the extensions dropdown");
+  expect(loomExtension.found()).toBeTruthy();
+});
+```
+
+
+<Tip>[vitest](https://vitest.dev/) is the preferred test runner for TestDriver.</Tip>
+
+,

package/bin/testdriverai.js

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+
+// Initialize Sentry first, before any other modules
+const sentry = require("../lib/sentry");
+
+// Set process priority if possible
+const os = require("os");
+try {
+  const pid = process.pid;
+  os.setPriority(pid, -20);
+  // eslint-disable-next-line no-unused-vars
+} catch (error) {
+  // Ignore if not permitted
+}
+
+// Ensure Sentry flushes on exit
+process.on("beforeExit", async () => {
+  await sentry.flush();
+});
+
+// Run the CLI
+require("../interfaces/cli.js");

package/debugger/index.html

@@ -0,0 +1,469 @@
+<!doctype html>
+<html>
+  <head>
+    <meta charset="UTF-8">
+    <link rel="preconnect" href="https://fonts.googleapis.com" />
+    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
+    <link
+      href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:ital,wght@0,100;0,200;0,300;0,400;0,500;0,600;0,700;1,100;1,200;1,300;1,400;1,500;1,600;1,700&display=swap"
+      rel="stylesheet"
+    />
+    <link rel="icon" type="image/png" href="icon.png" />
+    <title>TestDriver - Debugger</title>
+    <style>
+      body {
+        background-color: #111;
+        background-image: url("bg.png");
+        background-repeat: repeat;
+        background-size: auto;
+      }
+
+      * {
+        margin: 0;
+        padding: 0;
+        box-sizing: border-box;
+        font-family: "IBM Plex Mono", monospace;
+        font-weight: 400;
+        font-style: normal;
+        font-size: 14px;
+        scrollbar-width: none; /* Hide scrollbars for Firefox */
+        pointer-events: none;
+      }
+
+      #overlay {
+        position: absolute;
+        top: 0;
+        left: 0;
+        transform-origin: top left;
+        overflow: hidden;
+      }
+
+      html,
+      body {
+        overflow: hidden;
+        width: 100%;
+        height: 100%;
+      }
+
+      *::-webkit-scrollbar {
+        display: none; /* Hide scrollbars for WebKit browsers */
+      }
+
+      @keyframes animate-glow {
+        0% {
+          opacity: 0;
+          filter: brightness(3) saturate(3);
+          transform: scale(0.8, 0.8);
+        }
+
+        30% {
+          opacity: 1;
+          filter: brightness(1) saturate(1);
+          transform: scale(1, 1);
+        }
+
+        100% {
+          /* opacity: 0; */
+          opacity: 1;
+          transform: scale(1, 1);
+        }
+      }
+
+      .overlay {
+        position: relative;
+        overflow: hidden;
+      }
+
+      .screenshot {
+        position: absolute;
+        inset: 0;
+        z-index: 1;
+        opacity: 0;
+      }
+
+      .box {
+        border: 1px solid #b0cf34;
+        position: absolute;
+        border-radius: 5px;
+        animation-duration: 5s;
+        animation-delay: 0s;
+        animation-timing-function: cubic-bezier(0.26, 0.53, 0.74, 1.48);
+        animation-fill-mode: backwards;
+        animation-name: animate-glow;
+        animation-timing-function: ease;
+        animation-fill-mode: forwards;
+        border-radius: 5px;
+      }
+
+      .effects {
+        inset: 0;
+        position: absolute;
+        z-index: 20;
+      }
+
+      #mouse {
+        margin-left: -100px;
+        margin-top: -100px;
+        width: 50px;
+        height: 50px;
+        opacity: 0;
+        position: absolute;
+        transform: translate(-50%, -50%);
+        border-radius: 70%;
+        background: #b0cf34;
+      }
+
+      #mouse #dot {
+        width: 7px;
+        height: 7px;
+        position: absolute;
+        top: 50%;
+        left: 50%;
+        transform: translate(-50%, -50%);
+        border-radius: 50%;
+        background-color: black;
+      }
+
+      .single-click {
+        animation: singleClick 0.7s ease-in-out forwards;
+      }
+
+      .double-click {
+        animation: doubleClick 0.7s ease-in-out forwards;
+      }
+
+      @keyframes singleClick {
+        0% {
+          opacity: 1;
+          transform: translate(-50%, -50%) scale(1);
+        }
+
+        100% {
+          opacity: 0;
+          transform: translate(-50%, -50%) scale(2);
+        }
+      }
+
+      @keyframes doubleClick {
+        0% {
+          opacity: 1;
+          transform: translate(-50%, -50%) scale(1);
+        }
+
+        45% {
+          opacity: 0;
+          transform: translate(-50%, -50%) scale(1.2);
+        }
+
+        55% {
+          opacity: 1;
+          transform: translate(-50%, -50%) scale(1);
+        }
+
+        100% {
+          opacity: 0;
+          transform: scale(2);
+        }
+      }
+
+      #boxes {
+        position: absolute;
+        top: 25px;
+        left: 0;
+        right: 0;
+        bottom: 0;
+        z-index: 2;
+      }
+
+      #vm-iframe {
+        display: none;
+        border: none;
+        pointer-events: none; /* view-only by default, enabled on user click */
+        position: absolute;
+        top: 0px;
+        left: 0px;
+        width: 100%;
+        height: 100%;
+        z-index: 1;
+        overflow: hidden;
+      }
+
+      #vm-iframe.interactive {
+        pointer-events: auto;
+      }
+
+      /* Loading screen styles */
+      .loading-screen {
+        position: absolute;
+        top: 0;
+        left: 0;
+        width: 100%;
+        height: 100%;
+        background-color: black;
+        display: flex;
+        flex-direction: column;
+        align-items: center;
+        justify-content: center;
+        z-index: 10;
+        transition: opacity 0.5s ease-out;
+      }
+
+      .loading-screen.hidden {
+        opacity: 0;
+        pointer-events: none;
+      }
+
+      .loading-text {
+        color: #b0cf34;
+        font-size: 18px;
+        font-weight: 500;
+        margin-top: 10px;
+        animation: fadeInOut 1.5s infinite;
+      }
+
+      @keyframes pulse {
+        0% {
+          transform: scale(1);
+          box-shadow: 0 8px 32px rgba(176, 207, 52, 0.3);
+        }
+        50% {
+          transform: scale(1.05);
+          box-shadow: 0 12px 48px rgba(176, 207, 52, 0.5);
+        }
+        100% {
+          transform: scale(1);
+          box-shadow: 0 8px 32px rgba(176, 207, 52, 0.3);
+        }
+      }
+
+      @keyframes fadeInOut {
+        0%,
+        100% {
+          opacity: 0.6;
+        }
+        50% {
+          opacity: 1;
+        }
+      }
+
+      .status {
+        position: absolute;
+        top: 20px;
+        left: 20px;
+        background: rgba(0, 0, 0, 0.8);
+        color: #b0cf34;
+        padding: 8px 12px;
+        border-radius: 4px;
+        font-size: 12px;
+        z-index: 11;
+        opacity: 0;
+        transition: opacity 0.3s ease;
+        pointer-events: none;
+      }
+
+      .status.visible {
+        opacity: 1;
+      }
+
+      .bounding-box {
+        position: absolute;
+        border: 2px solid #b0cf34;
+        background: rgba(176, 207, 52, 0.1);
+        pointer-events: none;
+        z-index: 9;
+        border-radius: 3px;
+        animation: animate-glow 0.6s ease-out;
+      }
+
+      .interaction-overlay {
+        position: absolute;
+        top: 0;
+        left: 0;
+        width: 100%;
+        height: 100%;
+        background: rgba(0, 0, 0, 0.5);
+        display: flex;
+        flex-direction: column;
+        align-items: center;
+        justify-content: center;
+        z-index: 12;
+        opacity: 0;
+        pointer-events: none;
+        transition: opacity 0.3s ease;
+        cursor: pointer;
+        color: white;
+        font-size: 16px;
+        font-weight: 500;
+      }
+
+      .interaction-overlay.visible {
+        opacity: 1;
+        pointer-events: auto;
+      }
+
+      .interaction-overlay .lock-icon {
+        width: 48px;
+        height: 48px;
+        margin-bottom: 12px;
+        fill: white;
+      }
+
+      .interaction-overlay .message {
+        text-align: center;
+        user-select: none;
+      }
+
+    </style>
+  </head>
+  <body>
+    <!-- Loading screen -->
+    <div class="loading-screen" id="loading-screen">
+      <div class="testdriver-logo">
+        <img
+          src="https://framerusercontent.com/images/eMQDjLmOzfeu91XBo1VPlIc1WmY.png"
+          height="120"
+          alt="TestDriver Logo"
+        />
+      </div>
+      <div class="loading-text">Loading...</div>
+    </div>
+
+    <div class="overlay" id="overlay">
+      <div class="effects" id="effects">
+        <div class="interaction-overlay" id="interaction-overlay">
+          <svg class="lock-icon" viewBox="0 0 24 24">
+            <path
+              d="M18,8h-1V6c0-2.76-2.24-5-5-5S7,3.24,7,6v2H6c-1.1,0-2,0.9-2,2v10c0,1.1,0.9,2,2,2h12c1.1,0,2-0.9,2-2V10C20,8.9,19.1,8,18,8z M12,17c-1.1,0-2-0.9-2-2s0.9-2,2-2s2,0.9,2,2S13.1,17,12,17z M15.1,8H8.9V6c0-1.71,1.39-3.1,3.1-3.1s3.1,1.39,3.1,3.1V8z"
+            />
+          </svg>
+          <div class="message">Click to interact with VM</div>
+        </div>
+      </div>
+      <iframe id="vm-iframe" src="" credentialless></iframe>
+    </div>
+
+    <script>
+      // Parse session data from URL
+      const urlParams = new URLSearchParams(window.location.search);
+      const data = urlParams.get("data");
+      let parsedData;
+      if (data) {
+        try {
+          parsedData = JSON.parse(atob(data));
+        } catch (e) {
+          console.error("Error parsing data:", e);
+        }
+      }
+      // Demo mode: use mock data if no data param provided (for local dev)
+      if (!parsedData || !parsedData.url) {
+        const isLocalDev = window.location.hostname === "localhost" || window.location.hostname === "127.0.0.1";
+        if (isLocalDev) {
+          console.log("[Debugger] No data param — using demo mode");
+          parsedData = {
+            url: "about:blank",
+            resolution: [1366, 768],
+            testFile: "demo-test.mjs",
+            os: "linux",
+            token: null,
+          };
+        } else {
+          alert("Missing or invalid data parameter.");
+        }
+      }
+
+      // Elements
+      const iframe = document.getElementById("vm-iframe");
+      const overlay = document.getElementById("overlay");
+      const loadingScreen = document.getElementById("loading-screen");
+      const interactionOverlay = document.getElementById("interaction-overlay");
+
+      let isInteractionEnabled = false;
+
+      const showInteractionOverlay = () => {
+        if (!isInteractionEnabled) {
+          interactionOverlay.classList.add("visible");
+        }
+      };
+
+      const hideInteractionOverlay = () => {
+        interactionOverlay.classList.remove("visible");
+        isInteractionEnabled = true;
+        iframe.classList.add("interactive");
+      };
+
+      const disableInteraction = () => {
+        isInteractionEnabled = false;
+        iframe.classList.remove("interactive");
+      };
+
+      overlay.addEventListener("mouseenter", showInteractionOverlay);
+      overlay.addEventListener("mouseleave", () => {
+        if (!isInteractionEnabled) {
+          interactionOverlay.classList.remove("visible");
+        }
+      });
+      interactionOverlay.addEventListener("click", hideInteractionOverlay);
+
+      window.addEventListener("blur", () => {
+        showInteractionOverlay();
+        disableInteraction();
+      });
+
+      // Set title from test file name
+      const testFileName = parsedData?.testFile
+        ? parsedData.testFile.split("/").pop().split("\\").pop()
+        : "TestDriver";
+      document.title = `${testFileName} - Debugger`;
+
+      // Embed the VNC URL in the iframe with token
+      iframe.style.display = "block";
+      const vncUrl = new URL(parsedData.url);
+      if (parsedData.token) {
+        vncUrl.searchParams.set("token", parsedData.token);
+      }
+      iframe.src = vncUrl.toString();
+
+      // Hide loading screen once iframe loads
+      iframe.addEventListener("load", () => {
+        loadingScreen.classList.add("hidden");
+      });
+      // Fallback: hide loading screen after 5s even if load event doesn't fire
+      setTimeout(() => loadingScreen.classList.add("hidden"), 5000);
+
+      // Resolution from session data
+      const overlayWidth = parsedData.resolution?.[0] || 1366;
+      const overlayHeight = parsedData.resolution?.[1] || 768;
+
+      iframe.style.width = overlayWidth + "px";
+      iframe.style.height = overlayHeight + "px";
+
+      // Throttle helper
+      function throttle(fn, wait) {
+        let last = 0;
+        return function (...args) {
+          const now = Date.now();
+          if (now - last >= wait) { last = now; fn.apply(this, args); }
+        };
+      }
+
+      // Responsive: scale and center the overlay to fit the window
+      const resizeOverlay = () => {
+        const ww = window.innerWidth;
+        const wh = window.innerHeight;
+        const scale = Math.min(ww / overlayWidth, wh / overlayHeight, 1);
+        const sw = overlayWidth * scale;
+        const sh = overlayHeight * scale;
+
+        overlay.style.width = overlayWidth + "px";
+        overlay.style.height = overlayHeight + "px";
+        overlay.style.position = "absolute";
+        overlay.style.transform = scale < 1 ? `scale(${scale})` : "none";
+        overlay.style.left = `${(ww - sw) / 2}px`;
+        overlay.style.top = `${(wh - sh) / 2}px`;
+      };
+
+      resizeOverlay();
+      window.addEventListener("resize", throttle(resizeOverlay, 100));
+    </script>
+  </body>
+</html>