testdriverai 7.8.0-test.7 → 7.8.0-test.71

package/docs/changelog.mdx

@@ -2,20 +2,160 @@
 title: "Changelog"
 description: "Product updates and announcements"
 rss: true
+icon: "code-compare"
 ---
 
-{/* New entries are prepended automatically by the release workflow. */}
-{/* Do not remove the CHANGELOG_MARKER comment below — the CI script inserts new <Update> blocks after it. */}
-{/* CHANGELOG_MARKER */}
-<Update label="v7.8.0-test.7" description="March 2026" tags={["test"]}>
-🔧 Maintenance
+<Update label="v7.8.0-test.22" description="March 2026" tags={["test"]}>
+
+Releases test.11 through test.22 include version bumps and internal CI improvements across all packages. No new user-facing changes — see [v7.8.0-test.10](#v780-test10) for the latest updates in this release stream.
+
+</Update>
+
+<Update label="v7.8.0-test.10" description="March 2026" tags={["test"]}>
+
+🔧 Improvements
+
+- **Parallel test concurrency** — Concurrency limits are now enforced with atomic slot tracking, eliminating a race condition where multiple tests launching at the same time could all bypass the limit. If you hit your concurrency cap, the SDK waits and retries automatically.
+- **Dashboard access for all plans** — Test recordings and metrics on the [dashboard](https://console.testdriver.ai) are now visible to all users, including those on the Free plan.
+- **Sandbox message throttling** — High-frequency interactions no longer risk hitting connection limits. The SDK automatically paces outgoing messages during fast command sequences.
+- **Windows Dashcam stability** — Web log tracking on Windows sandboxes now handles errors gracefully instead of failing the test.
+
+</Update>
+
+<Update label="v7.8.0-test.9" description="March 2026" tags={["test"]}>
+
+🔧 Improvements
+
+- **Stable release promotion** — Fixed an issue where promoting a test or canary release to stable could be incorrectly skipped. Stable releases now proceed reliably regardless of the prior pre-release channel.
+- **Example docs sync** — Example documentation now updates more reliably, continuing even when individual example tests fail so that passing examples still get refreshed.
+
+</Update>
+
+<Update label="v7.8.0-test.8" description="March 2026" tags={["test"]}>
+
+🔧 Improvements
 
-- Fix version bump syntax in release workflow [f3ac61dc]
+- **VS Code environment switching** — The TestDriver VS Code extension now includes a status bar indicator and a new **TestDriver: Switch Environment** command. You can switch between stable, canary, and test environments directly from VS Code, and your MCP server configuration updates automatically.
+- **Plan rename: Starter → Pro** — The "Starter" plan has been renamed to **Pro**. No changes to pricing or features — just a clearer name. The "Self-Hosted" tier is now labeled **Enterprise**.
+- **SDK network resilience** — API requests now automatically retry on transient server errors (500, 502, 503, 504) with exponential backoff, reducing flaky test failures caused by brief infrastructure hiccups.
+- **Pricing page refresh** — The [hosted plans page](/v7/hosted) now shows plan cards with included minutes, concurrency limits, and pricing at a glance.
+
+📚 Docs updates
+
+- **Quickstart redesign** — The [quickstart](/v7/quickstart) now has tabbed setup paths for CLI, GitHub Copilot, and manual installation.
+- **Deployment section** — "Cloud" is now [Hosted](/v7/hosted) and the previous Enterprise page has been consolidated into [Self-Hosted](/v7/self-hosted), which covers both standard and air-gapped deployments.
+
+</Update>
+
+<Update label="v7.8.0-test.7" description="March 2026" tags={["test"]}>
+This release includes all changes from v7.8.0-test.6 with version bumps across all packages. No additional user-facing changes.
 </Update>
 
 <Update label="v7.8.0-test.6" description="March 2026" tags={["test"]}>
-🔧 Maintenance
 
-- Fix changelog generation in release workflow (04a789ec)
+🔧 Improvements
+
+- **Sandbox spawning reliability** — Sandbox creation now uses a 3-minute timeout to handle slow infrastructure, then immediately switches to a 60-second orphan timeout for fast cleanup. This reduces sandbox creation failures during high-traffic periods.
+- **Command deadline timeouts** — Sandbox commands now enforce a hard deadline that terminates execution if the connection drops mid-command, preventing tests from hanging indefinitely.
+- **Real-time channel cleanup** — Fixed a compatibility issue with the real-time messaging layer that could cause channel cleanup to fail during shutdown.
+- **Dashboard timestamp tooltips** — Extended UTC date tooltips to the test history sidebar. Hovering over any relative timestamp (e.g., "5 minutes ago") across the [dashboard](https://console.testdriver.ai) now shows the full UTC date and time.
+
+</Update>
+
+<Update label="v7.8.0-test.4" description="March 2026" tags={["test"]}>
+
+✨ New features
+
+- **[GitHub Copilot integration](/v7/copilot/setup)** — Use TestDriver directly from GitHub Copilot in VS Code. The new MCP server lets Copilot launch sandboxes, interact with elements, and run assertions through natural language. Includes guides for [creating tests](/v7/copilot/creating-tests), [running tests](/v7/copilot/running-tests), [GitHub Actions integration](/v7/copilot/github), and [auto-healing](/v7/copilot/auto-healing).
+
+🔧 Improvements
+
+- **Sandbox reliability** — Linux sandboxes now use a longer creation timeout to handle slow infrastructure, then immediately switch to a short orphan timeout for fast cleanup of disconnected sessions.
+- **Command execution timeouts** — Sandbox commands now have deadline timeouts that prevent hangs if the connection drops mid-execution.
+- **Real-time connection stability** — Updated the real-time messaging layer for better compatibility and more reliable channel cleanup during shutdown.
+- **Dashboard date display** — Hovering over relative timestamps (e.g., "5 minutes ago") now shows the full UTC date and time.
+
+</Update>
+
+<Update label="v7.8.0-canary.5" description="March 2026" tags={["canary"]}>
+This release includes all changes from v7.8.0-canary.4 with version bumps.
+
+✨ New features
+
+- **[Cache API](/v7/cache)** — Speed up repeated test runs with screenshot-based caching. The system compares screenshots to cached results and reuses element positions when the screen hasn't changed, reducing AI calls.
+- **[Custom error classes](/v7/errors)** — New `ElementNotFoundError` and `AIError` classes with rich debugging info including screenshot paths, pixel diffs, and detailed messages.
+- **[Events system](/v7/events)** — Listen to SDK lifecycle events with wildcard support via `testdriver.emitter`. Uses colon-delimited namespaces (e.g., `command:start`, `log:*`).
+- **[Provision API](/v7/provision)** — Launch browsers, desktop apps, VS Code, and Chrome extensions in your sandbox before tests run. Access via `testdriver.provision.*`.
+- **[Redraw detection](/v7/redraw)** — Wait for screens to stabilize after interactions using two-phase detection with pixel comparison and z-score analysis.
+- **[Screenshots API](/v7/screenshots)** — Capture screenshots manually with `testdriver.screenshot()` or automatically before/after every command.
+
+📚 New examples
+
+- [Exec output](/v7/examples/exec-output) — Capture and use output from PowerShell commands
+- [Exec PowerShell](/v7/examples/exec-pwsh) — Generate dynamic data with PowerShell
+- [Focus window](/v7/examples/focus-window) — Switch focus between application windows
+
+🔧 Improvements
+
+- Improved console URL mapping for canary and test environments
+- Enhanced Ably channel cleanup for better resource management
+- Updated release workflow with improved changelog generation
+</Update>
+
+<Update label="v7.5.26" description="March 2026" tags={["stable"]}>
+🚀 Stable release
+
+This release promotes v7.5.25 to stable with deployment channel improvements.
+
+✨ Features
+
+- **Deployment channels** — SDK now supports stable and canary release channels. Stable releases use the `latest` npm tag, while canary releases use the `canary` tag. Install canary with `npm install testdriverai@canary`.
+
+🔧 Improvements
+
+- Improved cache hit debugging and logging
+- Enhanced element location reliability
+- Better error handling during test execution
+
+🐛 Bug fixes
+
+- Fixed restart behavior during auto-updates
+- Fixed console URL routing for various environments
+
+<Note>
+This release includes changes from v7.5.17 through v7.5.25.
+</Note>
+</Update>
+
+<Update label="v7.5.25" description="March 2026" tags={["stable"]}>
+🐛 Bug fixes
+
+- Fixed restart update behavior
+</Update>
+
+<Update label="v7.5.24" description="March 2026" tags={["stable"]}>
+🔧 Improvements
+
+- Disabled auto-update restart to improve CI stability
+</Update>
+
+<Update label="v7.5.22" description="March 2026" tags={["stable"]}>
+🔧 Improvements
+
+- Added markdown-to-HTML rendering for marketing content
+</Update>
+
+<Update label="v7.5.19" description="March 2026" tags={["stable"]}>
+🔧 Improvements
+
+- Enhanced debugging output for cache hits
+- Improved element location reliability
+</Update>
+
+<Update label="v7.5.17" description="March 2026" tags={["stable"]}>
+🔧 Improvements
+
+- Improved codespace scaling and development environment setup
+- Updated VS Code extension version
 </Update>
 

package/docs/docs.json

@@ -1,6 +1,6 @@
 {
   "$schema": "https://mintlify.com/docs.json",
-  "theme": "palm",
+  "theme": "almond",
   "name": "TestDriver",
   "colors": {
     "primary": "#b3d334",
@@ -48,19 +48,31 @@
                   "/v7/examples/windows-installer"
                 ]
               },
+
               {
-                "group": "Plans & Pricing",
+                "group": "Deployment",
                 "icon": "server",
                 "pages": [
-                  "/v7/cloud",
-                  "/v7/self-hosted",
-                  "/v7/enterprise"
+                  "/v7/hosted",
+                  "/v7/self-hosted"
                 ]
-              }
+              },
+              "/changelog"
+            ]
+          },
+
+          {
+            "group": "GitHub Copilot",
+            "pages": [
+              "/v7/copilot/setup",
+              "/v7/copilot/creating-tests",
+              "/v7/copilot/running-tests",
+              "/v7/copilot/github",
+              "/v7/copilot/auto-healing"
             ]
           },
           {
-            "group": "Creating Tests",
+            "group": "Guide",
             "pages": [
               "/v7/generating-tests",
               "/v7/device-config",
@@ -75,7 +87,8 @@
             "pages": [
               "/v7/running-tests",
               "/v7/caching",
-              "/v7/ci-cd"
+              "/v7/ci-cd",
+              "/v7/test-results-json"
             ]
           },
           {
@@ -94,40 +107,35 @@
             ]
           },
           {
-            "group": "Actions",
-            "pages": [
-              "/v7/ai",
-              "/v7/assert",
-              "/v7/captcha",
-              "/v7/click",
-              "/v7/double-click",
-              "/v7/exec",
-              "/v7/find",
-              "/v7/focus-application",
-              "/v7/hover",
-              "/v7/mouse-down",
-              "/v7/mouse-up",
-              "/v7/parse",
-              "/v7/press-keys",
-              "/v7/right-click",
-              "/v7/screenshot",
-              "/v7/type",
-              "/v7/scroll"
-            ]
-          },
-          {
-            "group": "SDK",
+            "group": "SDK Reference",
             "pages": [
-              "/v7/elements",
               "/v7/client",
+              "/v7/elements",
+              {
+                "group": "Actions",
+                "icon": "bolt",
+                "pages": [
+                  "/v7/ai",
+                  "/v7/assert",
+                  "/v7/captcha",
+                  "/v7/click",
+                  "/v7/double-click",
+                  "/v7/exec",
+                  "/v7/find",
+                  "/v7/focus-application",
+                  "/v7/hover",
+                  "/v7/mouse-down",
+                  "/v7/mouse-up",
+                  "/v7/parse",
+                  "/v7/press-keys",
+                  "/v7/right-click",
+                  "/v7/screenshot",
+                  "/v7/scroll",
+                  "/v7/type"
+                ]
+              },
               "/v7/dashcam"
             ]
-          },
-          {
-            "group": "",
-            "pages": [
-              "/changelog"
-            ]
           }
         ]
       },

package/docs/v7/cache.mdx

@@ -0,0 +1,223 @@
+---
+title: "Cache"
+sidebarTitle: "Cache"
+description: "Speed up tests with screenshot-based caching"
+icon: "bolt-lightning"
+mode: "wide"
+---
+
+## Overview
+
+The cache system speeds up repeated test runs by comparing screenshots to cached results. When the screen hasn't changed significantly, cached element positions are reused instead of making an AI call.
+
+Cache works at two levels:
+- **Screen cache** — pixel diff comparison between the current screenshot and the cached screenshot
+- **Element cache** — OpenCV template matching to verify the cached element position is still correct
+
+## How It Works
+
+1. On `find()`, the SDK sends the current screenshot and cache metadata to the API
+2. The API compares the screenshot against previously cached results for the same `cacheKey`
+3. If the screen pixel diff is within the `screen` threshold AND the element template match exceeds the `element` threshold, the cached position is returned
+4. Otherwise, a new AI call is made and the result is cached
+
+```mermaid
+flowchart LR
+    A[Screenshot + cacheKey] --> B{Screen Diff\n< threshold?}
+    B -- Yes --> C{Template\nMatch OK?}
+    C -- Yes --> D[Return cached position]
+    B -- No --> E[AI Call - fresh]
+    C -- No --> F[AI Call - fresh]
+```
+
+## Configuration
+
+### Constructor Options
+
+```javascript
+const testdriver = new TestDriver({
+  cache: {
+    enabled: true,
+    thresholds: {
+      find: {
+        screen: 0.05,     // 5% pixel diff allowed (default)
+        element: 0.8,     // 80% OpenCV correlation required (default)
+      },
+      assert: 0.05,       // 5% pixel diff for assertions (default)
+    },
+  },
+  cacheKey: 'my-custom-key',   // overrides auto-generated key
+});
+```
+
+<ParamField path="cache" type="CacheConfig | false">
+  Cache configuration object, or `false` to disable entirely.
+
+  <Expandable title="properties">
+    <ParamField path="enabled" type="boolean" default={true}>
+      Enable or disable the cache system. Requires a valid `cacheKey` to actually activate.
+    </ParamField>
+    
+    <ParamField path="thresholds" type="CacheThresholds">
+      Threshold configuration for different command types.
+
+      <Expandable title="properties">
+        <ParamField path="find" type="FindCacheThresholds">
+          Thresholds for `find()` and `findAll()`.
+
+          <Expandable title="properties">
+            <ParamField path="screen" type="number" default={0.05}>
+              Maximum pixel diff percentage allowed between the current screenshot and the cached screenshot. Lower values require a closer match. Range: `0` to `1`.
+            </ParamField>
+            
+            <ParamField path="element" type="number" default={0.8}>
+              Minimum OpenCV template matching correlation required for the cached element crop. Higher values require a closer match. Range: `0` to `1`. Only used for `find()`, not `findAll()`.
+            </ParamField>
+          </Expandable>
+        </ParamField>
+        
+        <ParamField path="assert" type="number" default={0.05}>
+          Maximum pixel diff allowed for assertion cache hits.
+        </ParamField>
+      </Expandable>
+    </ParamField>
+  </Expandable>
+</ParamField>
+
+<ParamField path="cacheKey" type="string">
+  Unique key for cache lookups. If not provided, an auto-generated key is created from a SHA-256 hash of the calling test file (first 16 hex characters). The cache key changes automatically when your test file changes, providing automatic cache invalidation.
+</ParamField>
+
+### Disabling Cache
+
+```javascript
+// Via constructor
+const testdriver = new TestDriver({ cache: false });
+
+// Via environment variable
+// TD_NO_CACHE=true npx vitest run
+```
+
+When cache is disabled, all thresholds are set to `-1` internally, causing the API to skip cache lookups.
+
+### Per-Command Overrides
+
+Override cache thresholds for individual commands:
+
+```javascript
+// Stricter screen matching for this specific find
+const el = await testdriver.find('submit button', {
+  cache: {
+    thresholds: { screen: 0.01, element: 0.95 },
+  },
+});
+
+// Custom cache key for a specific assertion
+await testdriver.assert('dashboard loaded', {
+  cache: { threshold: 0.01 },
+  cacheKey: 'dashboard-check',
+});
+```
+
+## Threshold Priority
+
+Thresholds are resolved in priority order (highest wins):
+
+| Priority | Source | Example |
+|----------|--------|---------|
+| 1 (highest) | Per-command option | `find(desc, { cache: { thresholds: { screen: 0.1 } } })` |
+| 2 | Legacy number argument | `find(desc, 0.1)` |
+| 3 | Global constructor config | `new TestDriver({ cache: { thresholds: { find: { screen: 0.1 } } } })` |
+| 4 (lowest) | Hard-coded defaults | `screen: 0.05`, `element: 0.8`, `assert: 0.05` |
+
+## Auto-Generated Cache Key
+
+When you don't specify a `cacheKey`, the SDK automatically generates one:
+
+1. Walks the call stack to find your test file
+2. Reads the file content
+3. Computes a **SHA-256 hash** of the content
+4. Uses the **first 16 hex characters** as the cache key
+
+This means:
+- **Same test file** → same cache key → cache hits
+- **Modified test file** → different hash → automatic cache invalidation
+- **Different test files** → different keys → isolated caches
+
+```javascript
+// Auto-generated cache key from test file hash
+const testdriver = new TestDriver();
+// cacheKey = "a3f2b1c4d5e6f7a8" (auto)
+
+// Manual override
+const testdriver = new TestDriver({ cacheKey: 'login-test-v2' });
+```
+
+## Template Matching (OpenCV)
+
+Element cache validation uses OpenCV's normalized cross-correlation coefficient (`TM_CCOEFF_NORMED`) to verify that the cached element is still visible at the expected position.
+
+**Algorithm:**
+1. Load the cached element crop (needle) and current screenshot (haystack)
+2. Run `cv.matchTemplate()` with `TM_CCOEFF_NORMED`
+3. Binary threshold at the configured element threshold
+4. Find contours to extract match positions
+5. Return matches with `{ x, y, width, height, centerX, centerY }`
+
+**Scale factors tried:** `[1, 0.5, 2, 0.75, 1.25, 1.5]`
+**Thresholds tried:** `[0.9, 0.8, 0.7]` (picks highest matching threshold)
+
+This accounts for minor scaling differences between screenshots taken at different times or resolutions.
+
+## Cache Partitioning
+
+Cache entries are partitioned by:
+- **`cacheKey`** — identifies the test file
+- **`os`** — operating system (linux, windows, darwin)
+- **`resolution`** — screen resolution
+
+This means cache from a Linux run won't be used for a Windows run, even with the same cache key.
+
+## Debugging Cache
+
+API responses include cache metadata:
+
+| Field | Description |
+|---|---|
+| `cacheHit` | `true` if cache was used |
+| `similarity` | Pixel diff percentage between screenshots |
+| `cacheSimilarity` | OpenCV template match score |
+
+Use `getDebugInfo()` on an element to inspect cache results:
+
+```javascript
+const el = await testdriver.find('submit button');
+const debug = el.getDebugInfo();
+console.log(debug);
+// { cacheHit: true, similarity: 0.02, cacheSimilarity: 0.92, ... }
+```
+
+## Types
+
+```typescript
+interface CacheConfig {
+  enabled?: boolean;            // Default: true
+  thresholds?: CacheThresholds;
+}
+
+interface CacheThresholds {
+  find?: FindCacheThresholds;
+  assert?: number;              // Default: 0.05
+}
+
+interface FindCacheThresholds {
+  screen?: number;              // Default: 0.05
+  element?: number;             // Default: 0.8
+}
+
+interface CacheDebugInfo {
+  cacheHit: boolean;
+  similarity: number;
+  cacheSimilarity: number;
+}
+```