@testdriverai/agent 7.11.240-test → 7.11.241-test

package/ai/skills/testdriver-cache/SKILL.md

@@ -1,81 +1,51 @@
 ---
 name: testdriver:cache
-description: Speed up tests with screenshot-based caching
+description: Every option that controls how find() and assert() reuse cached results.
 ---
 <!-- Generated from cache.mdx. DO NOT EDIT. -->
 
-## Overview
+This page lists every cache option, its default, and where you can set it. The cache is on by default, so you only need these to tune or turn it off.
 
-The cache system makes repeated test runs faster. It compares screenshots to cached results. When the screen did not change much, TestDriver uses the cached element positions again. It does not make an AI call.
-
-The cache works at two levels:
-- **Screen cache**: a pixel diff comparison between the present screenshot and the cached screenshot
-- **Element cache**: OpenCV template matching to make sure that the cached element position is still correct
-
-## How It Works
-
-1. On `find()`, the SDK sends the present screenshot and the cache metadata to the API.
-2. The API compares the screenshot against the cached results for the same `cacheKey`.
-3. If the screen pixel diff is in the `screen` threshold AND the element template match is more than the `element` threshold, TestDriver returns the cached position.
-4. If not, TestDriver makes a new AI call and caches the result.
-
-```mermaid
-flowchart LR
-    A[Screenshot + cacheKey] --> B{Screen Diff\n< threshold?}
-    B -- Yes --> C{Template\nMatch OK?}
-    C -- Yes --> D[Return cached position]
-    B -- No --> E[AI Call - fresh]
-    C -- No --> F[AI Call - fresh]
-```
-
-## Configuration
-
-### Constructor Options
+For how caching works and when it helps, see [Speed up reruns with caching](/caching).
 
 ```javascript
-const testdriver = new TestDriver({
+const testdriver = TestDriver(context, {
   cache: {
-    enabled: true,
     thresholds: {
-      find: {
-        screen: 0.05,     // 5% pixel diff allowed (default)
-        element: 0.8,     // 80% OpenCV correlation required (default)
-      },
-      assert: 0.05,       // 5% pixel diff for assertions (default)
+      find: { screen: 0.05, element: 0.8 },
+      assert: 0.05,
     },
   },
-  cacheKey: 'my-custom-key',   // overrides auto-generated key
+  cacheKey: 'checkout-flow',
 });
 ```
 
-<ParamField path="cache" type="CacheConfig | false">
-  Cache configuration object, or `false` to disable entirely.
+## Client options
 
-  <Expandable title="properties">
-    <ParamField path="enabled" type="boolean" default={true}>
-      Enable or disable the cache system. Requires a valid `cacheKey` to actually activate.
-    </ParamField>
-    
-    <ParamField path="thresholds" type="CacheThresholds">
-      Threshold configuration for different command types.
+Pass these to `TestDriver(context, options)` or `new TestDriver(options)`. They apply to every call the client makes.
 
+<ParamField path="cache" type="boolean | object" default="true">
+  Pass `false` to turn the cache off. Pass an object to change the match thresholds.
+
+  <Expandable title="properties">
+    <ParamField path="thresholds" type="object">
       <Expandable title="properties">
-        <ParamField path="find" type="FindCacheThresholds">
+        <ParamField path="find" type="object">
           Thresholds for `find()` and `findAll()`.
 
           <Expandable title="properties">
-            <ParamField path="screen" type="number" default={0.05}>
-              Maximum pixel diff percentage allowed between the current screenshot and the cached screenshot. Lower values require a closer match. Range: `0` to `1`.
+            <ParamField path="screen" type="number" default="0.05">
+              The largest share of pixels that may differ between the current screen and the cached screen, from `0` to `1`. `0.05` allows a 5% difference. Lower values require a closer match.
             </ParamField>
-            
-            <ParamField path="element" type="number" default={0.8}>
-              Minimum OpenCV template matching correlation required for the cached element crop. Higher values require a closer match. Range: `0` to `1`. Only used for `find()`, not `findAll()`.
+
+            <ParamField path="element" type="number" default="0.8">
+              How closely the cached image of the element must match what's on screen now, from `0` to `1`. Higher values require a closer match. Used by `find()` only, not `findAll()`.
             </ParamField>
           </Expandable>
         </ParamField>
-        
-        <ParamField path="assert" type="number" default={0.05}>
-          Maximum pixel diff allowed for assertion cache hits.
+
+        <ParamField path="assert" type="number" default="0.05">
+          The largest share of pixels that may differ for `assert()` to reuse a cached result, from `0` to `1`.
         </ParamField>
       </Expandable>
     </ParamField>
@@ -83,139 +53,162 @@ const testdriver = new TestDriver({
 </ParamField>
 
 <ParamField path="cacheKey" type="string">
-  Unique key for cache lookups. If not provided, an auto-generated key is created from a SHA-256 hash of the calling test file (first 16 hex characters). The cache key changes automatically when your test file changes, providing automatic cache invalidation.
+  The key cache entries are stored under. By default, TestDriver generates one from the test file. See [Cache keys](#cache-keys).
+</ParamField>
+
+<ParamField path="cacheThreshold" type="object">
+  **Deprecated.** Use `cache.thresholds` instead.
 </ParamField>
 
-### Disabling Cache
+## Per-call options
 
-```javascript
-// Via constructor
-const testdriver = new TestDriver({ cache: false });
+Override the client settings for a single call.
+
+### find()
 
-// Via environment variable
-// TD_NO_CACHE=true vitest run
+```javascript
+const priceLabel = await testdriver.find('Total price label', {
+  cacheKey: 'checkout-flow',
+  cache: { thresholds: { screen: 0.01, element: 0.95 } },
+});
 ```
 
-When cache is disabled, all thresholds are set to `-1` internally, causing the API to skip cache lookups.
+<ParamField path="cacheKey" type="string">
+  Use this key for this call instead of the client's key.
+</ParamField>
 
-### Per-Command Overrides
+<ParamField path="cache.thresholds.screen" type="number">
+  Overrides `cache.thresholds.find.screen` for this call.
+</ParamField>
 
-Override cache thresholds for individual commands:
+<ParamField path="cache.thresholds.element" type="number">
+  Overrides `cache.thresholds.find.element` for this call.
+</ParamField>
+
+### findAll()
+
+`findAll()` takes `cacheKey` and `cache.thresholds.screen`. It doesn't use the `element` threshold.
 
 ```javascript
-// Stricter screen matching for this specific find
-const el = await testdriver.find('submit button', {
-  cache: {
-    thresholds: { screen: 0.01, element: 0.95 },
-  },
+const productCards = await testdriver.findAll('Product card in the results grid', {
+  cache: { thresholds: { screen: 0.02 } },
 });
+```
 
-// Custom cache key for a specific assertion
-await testdriver.assert('dashboard loaded', {
-  cache: { threshold: 0.01 },
-  cacheKey: 'dashboard-check',
+### assert()
+
+```javascript
+await testdriver.assert('the order confirmation is visible', {
+  cacheKey: 'checkout-flow',
+  threshold: 0.01,
 });
 ```
 
-## Threshold Priority
+<ParamField path="cacheKey" type="string">
+  Use this key for this call instead of the client's key.
+</ParamField>
 
-Thresholds are resolved in priority order (highest wins):
+<ParamField path="threshold" type="number">
+  Overrides `cache.thresholds.assert` for this call.
+</ParamField>
 
-| Priority | Source | Example |
-|----------|--------|---------|
-| 1 (highest) | Per-command option | `find(desc, { cache: { thresholds: { screen: 0.1 } } })` |
-| 2 | Legacy number argument | `find(desc, 0.1)` |
-| 3 | Global constructor config | `new TestDriver({ cache: { thresholds: { find: { screen: 0.1 } } } })` |
-| 4 (lowest) | Hard-coded defaults | `screen: 0.05`, `element: 0.8`, `assert: 0.05` |
+## Threshold priority
 
-## Auto-Generated Cache Key
+When a threshold is set in more than one place, the most specific wins:
 
-When you don't specify a `cacheKey`, the SDK automatically generates one:
+1. The per-call option, such as `find(description, { cache: { thresholds: { screen: 0.1 } } })`
+2. A legacy number argument, such as `find(description, 0.1)`
+3. The client option, `cache.thresholds`
+4. The defaults: `screen` 0.05, `element` 0.8, `assert` 0.05
 
-1. Walks the call stack to find your test file
-2. Reads the file content
-3. Computes a **SHA-256 hash** of the content
-4. Uses the **first 16 hex characters** as the cache key
+## Cache keys
 
-This means:
-- **Same test file** → same cache key → cache hits
-- **Modified test file** → different hash → automatic cache invalidation
-- **Different test files** → different keys → isolated caches
+Cache entries are stored under a key. By default, TestDriver generates the key from the test file:
 
-```javascript
-// Auto-generated cache key from test file hash
-const testdriver = new TestDriver();
-// cacheKey = "a3f2b1c4d5e6f7a8" (auto)
+- It hashes the contents of the file that creates the client (SHA-256, first 16 hex characters).
+- Editing the file changes the hash, so the file starts with a fresh cache.
+- Each test file gets its own cache.
+
+Set `cacheKey` to keep a cache across edits, or to share one between files that use the same screens:
 
-// Manual override
-const testdriver = new TestDriver({ cacheKey: 'login-test-v2' });
+```javascript
+const testdriver = TestDriver(context, { cacheKey: 'checkout-flow' });
 ```
 
-## Template Matching (OpenCV)
+If TestDriver can't read the test file, it can't generate a key. `find()` and `findAll()` then skip the cache unless you set `cacheKey`.
 
-Element cache validation uses OpenCV's normalized cross-correlation coefficient (`TM_CCOEFF_NORMED`) to verify that the cached element is still visible at the expected position.
+## What must match
 
-**Algorithm:**
-1. Load the cached element crop (needle) and current screenshot (haystack)
-2. Run `cv.matchTemplate()` with `TM_CCOEFF_NORMED`
-3. Binary threshold at the configured element threshold
-4. Find contours to extract match positions
-5. Return matches with `{ x, y, width, height, centerX, centerY }`
+A cached entry is reused only when all of these match the run that created it:
 
-**Scale factors tried:** `[1, 0.5, 2, 0.75, 1.25, 1.5]`
-**Thresholds tried:** `[0.9, 0.8, 0.7]` (picks highest matching threshold)
+- The cache key.
+- The exact description passed to `find()`, or the exact assertion text.
+- The sandbox OS and screen resolution.
 
-This accounts for minor scaling differences between screenshots taken at different times or resolutions.
+Entries become usable only after the test that created them passes.
 
-## Cache Partitioning
+## Turn the cache off
 
-Cache entries are partitioned by:
-- **`cacheKey`** — identifies the test file
-- **`os`** — operating system (linux, windows, darwin)
-- **`resolution`** — screen resolution
+Turn the cache off for one client:
+
+```javascript
+const testdriver = TestDriver(context, { cache: false });
+```
 
-This means cache from a Linux run won't be used for a Windows run, even with the same cache key.
+Or for a whole run:
 
-## Debugging Cache
+```bash
+TD_NO_CACHE=true npx vitest run
+```
 
-API responses include cache metadata:
+<Warning>
+  `cache: { enabled: false }` doesn't turn the cache off. Use `cache: false` or `TD_NO_CACHE=true`.
+</Warning>
 
-| Field | Description |
-|---|---|
-| `cacheHit` | `true` if cache was used |
-| `similarity` | Pixel diff percentage between screenshots |
-| `cacheSimilarity` | OpenCV template match score |
+## Inspect a result
 
-Use `getDebugInfo()` on an element to inspect cache results:
+Read the cache details from the element that `find()` returns:
 
 ```javascript
-const el = await testdriver.find('submit button');
-const debug = el.getDebugInfo();
-console.log(debug);
-// { cacheHit: true, similarity: 0.02, cacheSimilarity: 0.92, ... }
+const submitButton = await testdriver.find('Submit button');
+const { cache } = submitButton.toJSON();
 ```
 
-## Types
-
-```typescript
-interface CacheConfig {
-  enabled?: boolean;            // Default: true
-  thresholds?: CacheThresholds;
-}
-
-interface CacheThresholds {
-  find?: FindCacheThresholds;
-  assert?: number;              // Default: 0.05
-}
-
-interface FindCacheThresholds {
-  screen?: number;              // Default: 0.05
-  element?: number;             // Default: 0.8
-}
-
-interface CacheDebugInfo {
-  cacheHit: boolean;
-  similarity: number;
-  cacheSimilarity: number;
-}
-```
+<ResponseField name="cache" type="object">
+  <Expandable title="properties">
+    <ResponseField name="hit" type="boolean">
+      `true` when the location came from the cache.
+    </ResponseField>
+    <ResponseField name="strategy" type="string">
+      How the cached entry was matched.
+    </ResponseField>
+    <ResponseField name="createdAt" type="string">
+      When the cached entry was created.
+    </ResponseField>
+    <ResponseField name="diffPercent" type="number">
+      The share of pixels that differed between the current screen and the cached screen, from `0` to `1`.
+    </ResponseField>
+    <ResponseField name="imageUrl" type="string">
+      A link to the cached screenshot.
+    </ResponseField>
+  </Expandable>
+</ResponseField>
+
+To browse or clear your team's cached entries, open the [Cache page in the TestDriver console](https://console.testdriver.ai/cache).
+
+## Related
+
+<CardGroup cols={2}>
+  <Card title="Caching" icon="brain" href="/caching">
+    How the cache works and when it helps.
+  </Card>
+  <Card title="Options" icon="sliders" href="/options">
+    Every client option, with its default.
+  </Card>
+  <Card title="find()" icon="magnifying-glass" href="/find">
+    Locate an element by describing it.
+  </Card>
+  <Card title="assert()" icon="check" href="/assert">
+    Check the screen with an AI assertion.
+  </Card>
+</CardGroup>

package/ai/skills/testdriver-caching/SKILL.md

@@ -1,132 +1,142 @@
 ---
 name: testdriver:caching
-description: How TestDriver learns your app and caches what it discovers for instant, deterministic replays
+description: Reuse element locations from passing runs to skip the AI when the screen is unchanged.
 ---
 <!-- Generated from caching.mdx. DO NOT EDIT. -->
 
-After the agent [explored your app](/generating-tests), TestDriver keeps what it found. TestDriver caches each element that the AI vision agent finds. It uses a vision fingerprint. This is a perceptual hash of the screen state at the element location. On the next run, TestDriver matches against that cache. It does not call the AI again. Tests that pass replay quickly, in the same way each time, and at a low cost.
+After a test passes, TestDriver remembers where it found each element.
 
-This learning makes TestDriver fast. The cache makes the test run a maximum of **1.7x faster**. It does not do the same AI vision analysis again. The agent thinks only when it sees something new.
+- When the screen looks the same on a later run, `find()` reuses that location instead of asking the AI.
+- The step is faster, cheaper, and gives the same result every time.
+- When the screen changes, TestDriver falls back to the AI and learns the new location.
 
+Caching is on by default. You don't need to configure anything.
 
-```javascript
-// First run: builds cache
-await testdriver.find('submit button');
+## How caching works
 
-// Second run: exact match
-await testdriver.find('submit button');
-``` 
+1. **The first run learns.** Each `find()` asks the AI to locate the element. TestDriver saves the result with an image of the screen and of the element.
+2. **A passing run confirms.** Saved entries become usable only once the test passes. A failing run never teaches TestDriver a wrong location.
+3. **Later runs reuse.** On the next run, `find()` compares the current screen with the saved one. TestDriver returns the saved location without calling the AI when both are true:
+    - The screen differs by no more than 5% of its pixels.
+    - The saved image of the element still matches what's at that spot.
 
-## Automatic Caching
+    Otherwise, it asks the AI and saves a new entry.
 
-TestDriver enables learning automatically. You do not need configuration. TestDriver computes the cache key from these. The cache key is the fingerprint that TestDriver uses to know what it learned:
+`assert()` is cached the same way. When the screen matches a confirmed earlier run, the saved result is reused.
 
-- **File hash**: The SHA-256 hash of the test file contents
-- **Selector prompt**: The exact text description that you give to `find()`
-- **Screenshot context**: The perceptual hash of the present screen state
-- **Platform**: The operating system and the browser version
+## What has to match
 
-When you change your test file, the hash changes automatically. This makes the old cache entries not valid. Then TestDriver does a new AI analysis with your new test logic.
+A saved entry is reused only when all of these match the run that created it:
 
-```javascript
-import { test } from 'vitest';
-import { chrome } from 'testdriverai/presets';
+- **The test file.** The cache is keyed on a hash of the test file's contents, so editing the file starts a fresh cache. A changed test can't reuse results that no longer apply.
+- **The description.** The text you pass to `find()` must match exactly.
+- **The OS and screen resolution** of the sandbox.
 
-test('auto-cached test', async (context) => {
-  const { testdriver } = await chrome(context, {
-    url: 'https://example.com'
-  });
+## Keep changing values out of descriptions
 
-  // First call: AI analyzes screen, saves to cache
-  await testdriver.find('More information link'); // 2.1s
+The description must match exactly. A description with a value that changes between runs misses the cache every time:
 
-  // Second call: cache hit, instant response
-  await testdriver.find('More information link'); // 12ms ⚡
-});
+```javascript
+// Misses the cache whenever the email changes
+const emailField = await testdriver.find(`input next to ${email}`);
 ```
 
-## Managing the Cache
+Describe the element itself, and pass the changing value to the action instead:
 
-You can clear the cache in the TestDriver console. There, you also find previews of cached elements, the input prompts, and analytics on the cache hit rates.
+```javascript
+// Same description every run, so it can be cached
+const emailField = await testdriver.find('Email input field');
+await emailField.click();
+await testdriver.type(email);
+```
 
-<Card href="https://console.testdriver.ai/cache" title="TestDriver Cache" icon="database">
-  Manage and clear your test cache from the TestDriver console.
-</Card>
+## Choose your own cache key
 
-## Debugging Cache Hits and Misses
+By default, each test file has its own cache, which resets when you edit the file.
 
-You can track what TestDriver has learned by inspecting cache performance in your tests:
+Set a `cacheKey` when you create the client to keep the cache across edits, or to share it between files that use the same screens:
 
 ```javascript
-test('monitor cache performance', async (context) => {
-  const { testdriver } = await chrome(context, { url });
-
-  const element = await testdriver.find('submit button');
-
-  if (element.cacheHit) {
-    console.log('✅ Cache hit - instant response');
-    console.log('Strategy:', element.cacheStrategy); // 'exact', 'pixeldiff', or 'template'
-    console.log('Similarity:', `${(element.similarity * 100).toFixed(1)}%`);
-    console.log('Cache age:', element.cacheCreatedAt);
-  } else {
-    console.log('⏱️  Cache miss - AI analysis performed');
-    console.log('New cache entry created');
-  }
-});
+const testdriver = TestDriver(context, { cacheKey: 'checkout-flow' });
 ```
 
-## Configuring the Cache
+You can also pass `cacheKey` to a single `find()` call.
+
+A stable key doesn't make TestDriver reuse a location for a changed page. Entries are still reused only when the description, OS, and resolution match and the screen looks the same.
+
+## Adjust how closely the screen must match
 
-You can configure how TestDriver learns globally when initializing TestDriver:
+Two thresholds control when a saved location is reused:
+
+| Threshold | Default | Meaning |
+| --- | --- | --- |
+| `screen` | `0.05` | The largest share of pixels that may differ from the saved screen (5%) |
+| `element` | `0.8` | How closely the saved element image must match what's on screen now (0 to 1) |
+
+Set them for every call when you create the client:
 
 ```javascript
-import { TestDriver } from 'testdriverai';
-
-const testdriver = new TestDriver({
-  apiKey: process.env.TD_API_KEY,
-  cacheKey: 'my-test-suite', // cache-key for this instance
-  cacheDefaults: {
-    threshold: 0.05,      // 95% similarity
-  }
+const testdriver = TestDriver(context, {
+  cache: {
+    thresholds: {
+      find: { screen: 0.02, element: 0.9 },
+      assert: 0.02,
+    },
+  },
 });
 ```
 
-It's also possible to override cache settings per `find()` call:
+Or override them for one call:
 
 ```javascript
-// Default: 95% similarity required
-await testdriver.find('submit button');
-
-// Explicit strict threshold
-await testdriver.find('submit button', {
-  cacheThreshold: 0.01 // 99% similarity
+const priceLabel = await testdriver.find('Total price label', {
+  cache: { thresholds: { screen: 0.01 } },
 });
 ```
 
-## Caching with Variables
+Lower `screen` and higher `element` values make reuse stricter. That helps on screens where small details matter, but costs more AI calls.
 
-Custom cache keys prevent cache pollution when using variables in prompts, dramatically improving cache hit rates—so TestDriver reuses what it learned even when your data changes.
+For every cache option, see [Configure the cache](/cache).
+
+## Turn caching off
+
+Disable caching for a client with `cache: false`, or for a whole run with `TD_NO_CACHE`:
 
 ```javascript
-// ❌ Without cache key - creates new cache for each variable value
-const email = 'user@example.com';
-await testdriver.find(`input for ${email}`); // Cache miss every time
-
-// ✅ With cache key - reuses cache regardless of variable
-const email = 'user@example.com';
-await testdriver.find(`input for ${email}`, {
-  cacheKey: 'email-input'
-});
+const testdriver = TestDriver(context, { cache: false });
+```
 
-// Also useful for dynamic IDs, names, or other changing data
-const orderId = generateOrderId();
-await testdriver.find(`order ${orderId} status`, {
-  cacheKey: 'order-status'  // Same cache for all orders
-});
+```bash
+TD_NO_CACHE=true npx vitest run
 ```
 
-## Next
+## See what's cached
+
+The [Cache page in the TestDriver console](https://console.testdriver.ai/cache) lists your team's cached elements and assertions. It shows a preview and the description for each, and lets you clear entries.
+
+To check a single `find()` in your test, read the cache details from the element:
+
+```javascript
+const submitButton = await testdriver.find('Submit button');
+const { cache } = submitButton.toJSON();
+
+console.log(cache.hit);      // true when the location came from the cache
+console.log(cache.strategy); // how the cached entry was matched
+```
 
-<Card href="/copilot/running-tests" title="Run" icon="play">
-  Now that TestDriver has learned your app, run your tests in CI and locally—replaying the cache for fast, deterministic results.
-</Card>
+## Next steps
+
+<CardGroup cols={2}>
+  <Card title="Configure the cache" icon="sliders" href="/cache">
+    Every cache option, threshold, and override.
+  </Card>
+  <Card title="Locate elements" icon="crosshairs" href="/locating-elements">
+    Write descriptions that find the right element and cache well.
+  </Card>
+  <Card title="Run tests" icon="play" href="/copilot/running-tests">
+    Run your tests locally and in parallel.
+  </Card>
+  <Card title="Run in CI/CD" icon="code-branch" href="/ci-cd">
+    Run cached tests on every pull request.
+  </Card>
+</CardGroup>