@testdriverai/agent 7.11.240-test → 7.11.241-test

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/ai/skills/testdriver-agent/SKILL.md +174 -91
  2. package/ai/skills/testdriver-assert/SKILL.md +73 -269
  3. package/ai/skills/testdriver-aws-setup/SKILL.md +194 -312
  4. package/ai/skills/testdriver-bring-your-own-machine/SKILL.md +137 -204
  5. package/ai/skills/testdriver-cache/SKILL.md +142 -149
  6. package/ai/skills/testdriver-caching/SKILL.md +97 -87
  7. package/ai/skills/testdriver-captcha/SKILL.md +126 -111
  8. package/ai/skills/testdriver-ci-cd/SKILL.md +153 -542
  9. package/ai/skills/testdriver-click/SKILL.md +74 -229
  10. package/ai/skills/testdriver-client/SKILL.md +175 -229
  11. package/ai/skills/testdriver-customizing-devices/SKILL.md +72 -250
  12. package/ai/skills/testdriver-dashcam/SKILL.md +71 -404
  13. package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +103 -369
  14. package/ai/skills/testdriver-double-click/SKILL.md +46 -67
  15. package/ai/skills/testdriver-errors/SKILL.md +102 -176
  16. package/ai/skills/testdriver-events/SKILL.md +170 -327
  17. package/ai/skills/testdriver-exec/SKILL.md +92 -256
  18. package/ai/skills/testdriver-extract/SKILL.md +69 -170
  19. package/ai/skills/testdriver-find/SKILL.md +140 -736
  20. package/ai/skills/testdriver-focus-application/SKILL.md +69 -246
  21. package/ai/skills/testdriver-generating-tests/SKILL.md +102 -136
  22. package/ai/skills/testdriver-hosted/SKILL.md +79 -143
  23. package/ai/skills/testdriver-hover/SKILL.md +62 -235
  24. package/ai/skills/testdriver-locating-elements/SKILL.md +71 -366
  25. package/ai/skills/testdriver-making-assertions/SKILL.md +114 -72
  26. package/ai/skills/testdriver-mcp-setup/SKILL.md +79 -62
  27. package/ai/skills/testdriver-mouse-down/SKILL.md +46 -124
  28. package/ai/skills/testdriver-mouse-up/SKILL.md +45 -128
  29. package/ai/skills/testdriver-options/SKILL.md +73 -46
  30. package/ai/skills/testdriver-parse/SKILL.md +95 -184
  31. package/ai/skills/testdriver-performing-actions/SKILL.md +101 -91
  32. package/ai/skills/testdriver-press-keys/SKILL.md +59 -299
  33. package/ai/skills/testdriver-provision/SKILL.md +135 -179
  34. package/ai/skills/testdriver-quickstart/SKILL.md +24 -8
  35. package/ai/skills/testdriver-quickstart-claude/SKILL.md +84 -0
  36. package/ai/skills/testdriver-quickstart-github/SKILL.md +62 -32
  37. package/ai/skills/testdriver-quickstart-manual/SKILL.md +59 -32
  38. package/ai/skills/testdriver-quickstart-vscode/SKILL.md +96 -0
  39. package/ai/skills/testdriver-redraw/SKILL.md +100 -164
  40. package/ai/skills/testdriver-reusable-code/SKILL.md +95 -199
  41. package/ai/skills/testdriver-right-click/SKILL.md +45 -88
  42. package/ai/skills/testdriver-screenshot/SKILL.md +47 -201
  43. package/ai/skills/testdriver-screenshots/SKILL.md +86 -136
  44. package/ai/skills/testdriver-scroll/SKILL.md +71 -154
  45. package/ai/skills/testdriver-secrets/SKILL.md +94 -90
  46. package/ai/skills/testdriver-self-hosted/SKILL.md +82 -140
  47. package/ai/skills/testdriver-test-results-json/SKILL.md +165 -202
  48. package/ai/skills/testdriver-type/SKILL.md +56 -304
  49. package/ai/skills/testdriver-variables/SKILL.md +119 -80
  50. package/ai/skills/testdriver-wait/SKILL.md +31 -29
  51. package/ai/skills/testdriver-what-is-testdriver/SKILL.md +80 -32
  52. package/package.json +1 -1
@@ -1,858 +1,262 @@
1
1
  ---
2
2
  name: testdriver:find
3
- description: Locate UI elements using natural language
3
+ description: Locate an element on screen by describing it in plain language.
4
4
  ---
5
5
  <!-- Generated from find.mdx. DO NOT EDIT. -->
6
6
 
7
- ## Overview
7
+ Describe an element the way a person would, and `find()` locates it on screen. Then act on it:
8
8
 
9
- Find UI elements on the screen with natural language descriptions and AI. This returns an `Element` object. You can interact with the object.
9
+ ```javascript
10
+ const signInButton = await testdriver.find('Sign in button below the password field');
11
+ await signInButton.click();
12
+ ```
10
13
 
11
- ## Syntax
14
+ For writing good descriptions, see [Locate elements](/locating-elements).
12
15
 
13
- ```javascript
14
- const element = await testdriver.find(description)
15
- const element = await testdriver.find(description, options)
16
+ ## Signature
17
+
18
+ ```typescript
19
+ testdriver.find(description: string, options?: FindOptions): ChainableElementPromise
16
20
  ```
17
21
 
18
22
  ## Parameters
19
23
 
20
24
  <ParamField path="description" type="string" required>
21
- Natural language description of the element to find
25
+ What the element looks like and where it is, such as `'Add to cart button under the TestDriver Hat'`.
22
26
  </ParamField>
23
27
 
24
- <ParamField path="options" type="object | number">
25
- Optional configuration for finding and caching
26
-
28
+ <ParamField path="options" type="object">
27
29
  <Expandable title="properties">
28
- <ParamField path="cacheKey" type="string">
29
- A custom cache key to store the element location. Use this to keep the cache clean when you use dynamic variables in prompts. You can also use it to share the cache between tests.
30
- </ParamField>
31
-
32
- <ParamField path="cacheThreshold" type="number" default={0.05}>
33
- The similarity threshold (0-1) for a cache match. A lower value needs more similarity. Set it to -1 to disable the cache.
30
+ <ParamField path="timeout" type="number" default="10000">
31
+ How long to keep looking, in milliseconds. `find()` checks the screen every 5 seconds until it finds the element or the time runs out. `0` checks once.
34
32
  </ParamField>
35
-
36
- <ParamField path="timeout" type="number" default={10000}>
37
- The maximum time in milliseconds to poll for the element. TestDriver tries again each 5 seconds until it finds the element or the timeout ends. The default is `10000` (10 seconds). Set it to `0` to disable the poll and try one time.
38
- </ParamField>
39
-
40
33
  <ParamField path="confidence" type="number">
41
- The minimum confidence threshold (0-1). If the confidence score of the found element is less than this value, the find is a failure (`element.found()` returns `false`). Use this to make sure of good matches in critical test steps.
34
+ The lowest AI confidence to accept, from 0 to 1. A match below it counts as not found.
42
35
  </ParamField>
43
-
44
- <ParamField path="type" type="string">
45
- Element type hint that wraps the description for better matching. Accepted values:
46
- - `"text"` — Wraps the prompt as `The text "..."`
47
- - `"image"` — Wraps the prompt as `The image "..."`
48
- - `"ui"` — Wraps the prompt as `The UI element "..."`
49
- - `"any"` — No wrapping, uses the description as-is (default behavior)
36
+ <ParamField path="verify" type="boolean" default="false">
37
+ Double-check the result with a second AI call. If the check fails, TestDriver locates the element again with a hint and checks once more. If that also fails, the element is returned as not found. Off by default, because it's slower. Cache hits aren't re-checked. Defaults to the client's [`verify` option](/options#param-verify).
50
38
  </ParamField>
51
-
52
- <ParamField path="zoom" type="boolean" default={false}>
53
- A two-phase zoom mode for more precision in full UIs that have many similar elements. It is disabled by default.
39
+ <ParamField path="zoom" type="boolean | number" default="false">
40
+ Locate in two passes for more precision. The AI finds the rough area, then locates the element again in a close-up crop of that area. `true` checks one area. A number from 1 to 4 checks up to that many candidate areas. More than one area is an Enterprise feature; on other plans it's treated as `1`.
54
41
  </ParamField>
55
-
56
- <ParamField path="verify" type="boolean" default={false}>
57
- This enables AI verification of the found element. When `true`, a second AI call makes sure that the coordinates agree with the correct element. This catches incorrect positions. It is disabled by default for less latency. When you do not set it for each call, it uses the global `verify` option from the [SDK options](/options#ai-and-element-location).
42
+ <ParamField path="type" type="string" default="any">
43
+ What kind of element you mean. TestDriver wraps your description before sending it to the AI:
44
+
45
+ - `'text'`: `The text "<description>"`
46
+ - `'image'`: `Image of "<description>"`
47
+ - `'ui'`: `The UI element "<description>"`
48
+ - `'any'`: the description as written
58
49
  </ParamField>
59
-
60
- <ParamField path="ai" type="object">
61
- AI sampling configuration for this find call (overrides global `ai` config from constructor).
62
-
50
+ <ParamField path="cacheKey" type="string">
51
+ Store and look up the cached position under this key instead of the client's. See [Caching](/caching).
52
+ </ParamField>
53
+ <ParamField path="cache" type="object">
54
+ Cache settings for this call.
63
55
  <Expandable title="properties">
64
- <ParamField path="temperature" type="number">
65
- Controls randomness. `0` = deterministic. Default: `0` for find verification.
66
- </ParamField>
67
-
68
- <ParamField path="top" type="object">
69
- Sampling parameters
70
-
56
+ <ParamField path="thresholds" type="object">
71
57
  <Expandable title="properties">
72
- <ParamField path="p" type="number">
73
- Top-P (nucleus sampling). Range: 0-1.
58
+ <ParamField path="screen" type="number" default="0.05">
59
+ How much the screen may differ from the cached run (0 to 1) and still reuse the position. `-1` turns the cache off for this call.
74
60
  </ParamField>
75
-
76
- <ParamField path="k" type="number">
77
- Top-K sampling. `1` = most deterministic.
61
+ <ParamField path="element" type="number" default="0.8">
62
+ How closely the element must match its cached image (0 to 1).
78
63
  </ParamField>
79
64
  </Expandable>
80
65
  </ParamField>
81
66
  </Expandable>
82
67
  </ParamField>
68
+ <ParamField path="ai" type="AIConfig">
69
+ Accepted, but `find()` doesn't apply it yet. Set the client's [`ai` option](/options#param-ai) instead.
70
+ </ParamField>
71
+ <ParamField path="cacheThreshold" type="number">
72
+ Legacy. Same as `cache.thresholds.screen`.
73
+ </ParamField>
83
74
  </Expandable>
84
75
  </ParamField>
85
76
 
86
77
  ## Returns
87
78
 
88
- `Promise<Element>` - The Element instance that TestDriver found automatically.
89
-
90
- ## Examples
91
-
92
- ### Basic Element Finding
79
+ `ChainableElementPromise`. It resolves to an `Element`, and you can also call an action on it directly (see [Chain an action](#chain-an-action)).
93
80
 
94
- ```javascript
95
- // Find by role
96
- const button = await testdriver.find('submit button');
97
- const input = await testdriver.find('email input field');
81
+ The element has these methods and properties:
98
82
 
99
- // Find by text content
100
- const link = await testdriver.find('Contact Us link');
101
- const heading = await testdriver.find('Welcome heading');
83
+ | Member | What it gives you |
84
+ | --- | --- |
85
+ | `found()` | `true` if the element was located |
86
+ | `click(action?)` | Clicks the element. See [click()](/click). |
87
+ | `hover()`, `doubleClick()`, `rightClick()` | The other mouse actions |
88
+ | `mouseDown()`, `mouseUp()` | Press and release, for dragging. See [mouseDown()](/mouse-down). |
89
+ | `x`, `y` | The point TestDriver clicks, in pixels |
90
+ | `coordinates` | The same point as `{ x, y }`, or `null` if not found |
91
+ | `confidence` | The AI's confidence, from 0 to 1 |
92
+ | `find(description?, options?)` | Locates the element again, optionally with a new description |
93
+ | `toJSON()` | Debug details, such as `reasoning` and whether the result came from the `cache` |
102
94
 
103
- // Find by visual appearance
104
- const icon = await testdriver.find('red warning icon');
105
- const image = await testdriver.find('company logo image');
106
- ```
95
+ ## Chain an action
107
96
 
108
- ### Finding with Context
97
+ You can call an action on `find()` without awaiting it first:
109
98
 
110
99
  ```javascript
111
- // Provide location context
112
- const field = await testdriver.find('username input in the login form');
113
- const button = await testdriver.find('delete button in the top right corner');
114
-
115
- // Describe nearby elements
116
- const input = await testdriver.find('input field below the email label');
117
- const checkbox = await testdriver.find('checkbox next to "Remember me"');
118
-
119
- // Describe visual position
120
- const menu = await testdriver.find('hamburger menu icon in the top left');
100
+ await testdriver.find('Sign in button').click();
121
101
  ```
122
102
 
123
- ### Interacting with Found Elements
124
-
125
- ```javascript
126
- // Find and click
127
- const submitBtn = await testdriver.find('submit button');
128
- await submitBtn.click();
129
-
130
- // Find and verify
131
- const message = await testdriver.find('success message');
132
- if (message.found()) {
133
- console.log('Success message appeared');
134
- }
135
-
136
- // Find and extract info
137
- const price = await testdriver.find('product price');
138
- console.log('Price location:', price.coordinates);
139
- console.log('Price text:', price.text);
140
- ```
141
-
142
- ## Element Object
143
-
144
- The `Element` object that TestDriver returns gives these:
145
-
146
- ### Methods
147
-
148
- - `found()` - Make a check if TestDriver found the element
149
- - `click(action)` - Click the element
150
- - `hover()` - Put the cursor on the element
151
- - `doubleClick()` - Double-click the element
152
- - `rightClick()` - Right-click the element
153
- - `find(newDescription)` - Find the element again with an optional new description
154
-
155
- ### Properties
103
+ These chain the same way: `hover()`, `doubleClick()`, `rightClick()`, `mouseDown()`, `mouseUp()`, and `found()`.
156
104
 
157
- - `coordinates` - Element position `{x, y, centerX, centerY}`
158
- - `x`, `y` - Top-left coordinates
159
- - `centerX`, `centerY` - Center coordinates
160
- - `text` - Text content (if available)
161
- - `screenshot` - Base64 screenshot (if available)
162
- - `confidence` - AI confidence score
163
- - `width`, `height` - Element dimensions
164
- - `boundingBox` - Complete bounding box
105
+ ## When the element isn't found
165
106
 
166
- See [Elements Reference](/elements) for complete details.
107
+ `find()` doesn't throw when nothing matches. It returns an element whose `found()` is `false` and whose `coordinates` is `null`.
167
108
 
168
- ### JSON Serialization
169
-
170
- You can serialize elements safely with `JSON.stringify()` for logs and for debug. TestDriver removes circular references automatically:
109
+ Acting on that element throws an `ElementNotFoundError`, so a missing element still fails the test at the action:
171
110
 
172
111
  ```javascript
173
- const element = await testdriver.find('login button');
174
-
175
- // Safe to stringify - no circular reference errors
176
- console.log(JSON.stringify(element, null, 2));
177
-
178
- // Output includes useful debugging info:
179
- // {
180
- // "description": "login button",
181
- // "coordinates": { "x": 100, "y": 200, "centerX": 150, "centerY": 225 },
182
- // "found": true,
183
- // "threshold": 0.01,
184
- // "x": 100,
185
- // "y": 200,
186
- // "cache": {
187
- // "hit": true,
188
- // "strategy": "pixel-diff",
189
- // "createdAt": "2025-12-09T10:30:00Z",
190
- // "diffPercent": 0.0023,
191
- // "imageUrl": "https://..."
192
- // },
193
- // "similarity": 0.98,
194
- // "confidence": 0.95,
195
- // "selector": "button#login",
196
- // "aiResponse": "Found the blue login button..."
197
- // }
112
+ // Throws ElementNotFoundError if the button wasn't found
113
+ await testdriver.find('Sign in button').click();
198
114
  ```
199
115
 
200
- Use this for these:
201
- - To debug problems with element detection
202
- - To log the details of the test
203
- - To share element data between processes
204
- - To examine the cache performance
205
-
206
- ## Best Practices
207
-
208
- <Check>
209
- **Be specific in descriptions**
210
-
211
- More specific descriptions make the accuracy better:
212
-
213
- ```javascript
214
- // ✅ Good
215
- await testdriver.find('blue submit button below the email field');
216
-
217
- // ❌ Too vague
218
- await testdriver.find('button');
219
- ```
220
- </Check>
221
-
222
- <Check>
223
- **Always check if found**
224
-
225
- Make sure that TestDriver found the elements before you interact with them:
226
-
227
- ```javascript
228
- const element = await testdriver.find('login button');
229
- if (!element.found()) {
230
- throw new Error('Login button not found');
231
- }
232
- await element.click();
233
- ```
234
- </Check>
235
-
236
- <Check>
237
- **Include visual or positional context**
238
-
239
- ```javascript
240
- // Include color
241
- await testdriver.find('red error icon');
242
-
243
- // Include position
244
- await testdriver.find('search button in the top navigation bar');
245
-
246
- // Include nearby text
247
- await testdriver.find('checkbox next to "I agree to terms"');
248
- ```
249
- </Check>
250
-
251
- ## Confidence Threshold
252
-
253
- Set a minimum confidence score for element matches. If the confidence is less than the threshold, `find()` makes the result "not found":
116
+ To handle an element that may not be there, check `found()`:
254
117
 
255
118
  ```javascript
256
- // Require at least 90% confidence
257
- const element = await testdriver.find('submit button', { confidence: 0.9 });
258
-
259
- if (!element.found()) {
260
- // AI found something but wasn't confident enough
261
- throw new Error('Could not confidently locate submit button');
119
+ const banner = await testdriver.find('cookie consent banner', { timeout: 0 });
120
+ if (banner.found()) {
121
+ await testdriver.find('Accept all cookies button').click();
262
122
  }
263
-
264
- await element.click();
265
123
  ```
266
124
 
267
- Use this for these:
268
- - Critical test steps. An incorrect click can cause more failures.
269
- - To tell the difference between similar elements (for example, many buttons)
270
- - To fail quickly when the UI changed
125
+ `ElementNotFoundError` is exported from `testdriverai`:
271
126
 
272
127
  ```javascript
273
- // Combine with timeout for robust polling with confidence gate
274
- const element = await testdriver.find('success notification', {
275
- confidence: 0.85,
276
- timeout: 15000,
277
- });
128
+ import { ElementNotFoundError } from 'testdriverai';
278
129
  ```
279
130
 
280
- <Tip>
281
- The `confidence` value is a float between 0 and 1 (for example, `0.9` = 90%). The AI returns its confidence with each find result. You can also read this from `element.confidence` after a good find.
282
- </Tip>
283
- ## Element Type
284
-
285
- Use the `type` option to show which kind of element you look for. This puts your description into a more specific prompt for the AI. It makes the match accuracy better, primarily when a description is short or not clear.
286
-
287
- ```javascript
288
- // Find text on the page
289
- const label = await testdriver.find('Sign In', { type: 'text' });
290
- // AI prompt becomes: The text "Sign In"
291
-
292
- // Find an image
293
- const logo = await testdriver.find('company logo', { type: 'image' });
294
- // AI prompt becomes: The image "company logo"
295
-
296
- // Find a UI element (button, input, checkbox, etc.)
297
- const btn = await testdriver.find('Submit', { type: 'ui' });
298
- // AI prompt becomes: The UI element "Submit"
299
-
300
- // No wrapping — same as omitting the option
301
- const el = await testdriver.find('the blue submit button', { type: 'any' });
302
- ```
303
-
304
- | Type | Prompt sent to AI |
305
- |------|----|
306
- | `"text"` | `The text "..."` |
307
- | `"image"` | `The image "..."` |
308
- | `"ui"` | `The UI element "..."` |
309
- | `"any"` | Original description (no wrapping) |
310
-
311
- <Tip>
312
- Use this primarily for short descriptions such as `"Submit"` or `"Login"`. In these, the AI can be not sure to look for a button, a link, or visible text. When you give the `type`, the description becomes clear.
313
- </Tip>
314
- ## Polling for Dynamic Elements
315
-
316
- By default, `find()` polls for a maximum of 10 seconds (it tries again each 5 seconds) until it finds the element. You can change this with the `timeout` option:
317
-
318
- ```javascript
319
- // Uses default 10s timeout - polls every 5 seconds
320
- const element = await testdriver.find('login button');
321
- await element.click();
322
-
323
- // Custom timeout - wait up to 30 seconds
324
- const element = await testdriver.find('login button', { timeout: 30000 });
325
- await element.click();
326
-
327
- // Disable polling - single attempt only
328
- const element = await testdriver.find('login button', { timeout: 0 });
329
- ```
330
-
331
- The `timeout` option:
332
- - Has a default of `10000` (10 seconds)
333
- - Tries to find the element again each 5 seconds
334
- - Stops when it finds the element or the timeout ends
335
- - Logs the progress during the poll
336
- - Returns the element (make a check with `element.found()` if it does not throw an error on a failure)
337
- - Set it to `0` to disable the poll and try one time
338
-
339
- ## Zoom Mode
340
-
341
- Zoom mode is **disabled by default**. It uses a two-phase method for more precision when it finds elements, primarily in full UIs that have many similar elements.
342
-
343
- To enable zoom for a specific find call, give `zoom: true`:
344
-
345
- ```javascript
346
- // Enable zoom for better precision in crowded UIs
347
- const extensionsBtn = await testdriver.find('extensions puzzle icon in Chrome toolbar', { zoom: true });
348
- await extensionsBtn.click();
349
-
350
- // Without zoom (default)
351
- const largeButton = await testdriver.find('big hero button');
352
- ```
353
-
354
- ### How Zoom Mode Works
355
-
356
- 1. **Phase 1**: The AI finds the approximate location of the element.
357
- 2. **Phase 2**: TestDriver makes a 30% crop of the screen around that location.
358
- 3. **Phase 3**: The AI does the precise location on the cropped image.
359
- 4. **Result**: TestDriver changes the coordinates back to the absolute screen position.
360
-
361
- This two-phase method gives the AI a higher-resolution view of the target area. It makes the accuracy better when many similar elements are near together.
362
-
363
- <Tip>
364
- Enable zoom with `zoom: true` when:
365
- - You target small elements in full UIs that have many similar elements
366
- - You need more precision for UI elements that are near together
367
- </Tip>
368
-
369
- ## Verify Mode
131
+ ## Examples
370
132
 
371
- Verify mode is **disabled by default**. When it is enabled, a second AI call makes sure that the coordinates from `find()` agree with the correct element. This catches incorrect positions.
133
+ ### Wait for something slow to appear
372
134
 
373
135
  ```javascript
374
- // Enable verification for critical interactions
375
- const deleteBtn = await testdriver.find('delete account button', { verify: true });
376
- await deleteBtn.click();
136
+ const confirmation = await testdriver.find('Order confirmation heading', { timeout: 30000 });
137
+ expect(confirmation.found()).toBe(true);
377
138
  ```
378
139
 
379
- ### How Verify Mode Works
140
+ ### Make the match stricter
380
141
 
381
- 1. **Phase 1**: The AI finds the element and returns coordinates.
382
- 2. **Phase 2**: A second AI call looks at the screenshot at those coordinates. It makes sure that the element agrees with the description.
383
- 3. **Result**: If the verification fails, TestDriver tries the find again or marks it "not found".
384
-
385
- ### Combining Zoom and Verify
386
-
387
- For the maximum accuracy, enable `zoom` and `verify` together. Use this for critical interactions. A click on the wrong element can cause more failures:
142
+ Use `confidence` and `verify` where clicking the wrong element is costly:
388
143
 
389
144
  ```javascript
390
- // Maximum accuracy: zoom for precision + verify to catch hallucinations
391
- const element = await testdriver.find('small cancel icon next to the subscription', {
392
- zoom: true,
145
+ await testdriver.find('Delete account button', {
146
+ confidence: 0.9,
393
147
  verify: true,
394
- });
395
- await element.click();
148
+ }).click();
396
149
  ```
397
150
 
398
- ## Cache Options
399
-
400
- When a test completes correctly, TestDriver caches the result of each `find()`. On later runs, TestDriver uses the cached match again. It does not make a new AI call. This finds the same element much more quickly. The cache is in your [dashboard](https://console.testdriver.ai/cache). TestDriver shares it between runs. Read the [Cache](/cache) page to see how the match, the thresholds, and the invalidation work.
401
-
402
- Control the cache to make the performance better, primarily when you use dynamic variables in prompts.
403
-
404
- ### Custom Cache Key
405
-
406
- Use `cacheKey` to keep the cache clean when prompts have variables:
151
+ ### Find a small target in a crowded screen
407
152
 
408
153
  ```javascript
409
- // ❌ Without cacheKey - creates new cache entry for each email value
410
- const email = 'user@example.com';
411
- await testdriver.find(`input for ${email}`); // Cache miss every time
412
-
413
- // ✅ With cacheKey - reuses cache regardless of variable
414
- const email = 'user@example.com';
415
- await testdriver.find(`input for ${email}`, {
416
- cacheKey: 'email-input'
417
- });
418
-
419
- // Also useful for dynamic IDs, names, or other changing data
420
- const orderId = generateOrderId();
421
- await testdriver.find(`order ${orderId} status`, {
422
- cacheKey: 'order-status' // Same cache for all orders
423
- });
154
+ await testdriver.find('extensions puzzle icon in the Chrome toolbar', { zoom: true }).click();
424
155
  ```
425
156
 
426
- ### Cache Threshold
427
-
428
- Control how similar a cached result must be before TestDriver uses it again:
157
+ ### Disambiguate a short description
429
158
 
430
159
  ```javascript
431
- // Default: 95% similarity required
432
- await testdriver.find('submit button');
433
-
434
- // Strict threshold - 99% similarity required
435
- await testdriver.find('submit button', {
436
- cacheThreshold: 0.01
437
- });
438
-
439
- // Disable cache entirely for this call
440
- await testdriver.find('submit button', {
441
- cacheThreshold: -1
442
- });
443
-
444
- // Combine cacheKey with threshold
445
- await testdriver.find('submit button', {
446
- cacheKey: 'submit-btn',
447
- cacheThreshold: 0.01
448
- });
160
+ // Matches the visible word "Total", not a button or icon
161
+ const totalLabel = await testdriver.find('Total', { type: 'text' });
449
162
  ```
450
163
 
451
- <Tip>
452
- By default, TestDriver makes a cache key automatically from the SHA-256 hash of your test file. When you change your test file, the hash changes automatically. This makes the old cache entries not valid.
453
- </Tip>
164
+ ### Keep the cache when the description changes
454
165
 
455
- ### Manual Polling (Alternative)
456
-
457
- If you need custom poll logic:
166
+ A description with a changing value never matches its cached entry. Give it a fixed `cacheKey`:
458
167
 
459
168
  ```javascript
460
- async function waitForElement(testdriver, description, timeout = 30000) {
461
- const startTime = Date.now();
462
-
463
- while (Date.now() - startTime < timeout) {
464
- const element = await testdriver.find(description);
465
- if (element.found()) return element;
466
- await new Promise(r => setTimeout(r, 1000));
467
- }
468
-
469
- throw new Error(`Element "${description}" not found after ${timeout}ms`);
470
- }
471
-
472
- // Usage
473
- const button = await waitForElement(testdriver, 'submit button', 10000);
474
- await button.click();
169
+ const orderId = '1042';
170
+ await testdriver.find(`Order #${orderId} row in the orders table`, {
171
+ cacheKey: 'order-row',
172
+ }).click();
475
173
  ```
476
174
 
477
- ## Use Cases
478
-
479
- <AccordionGroup>
480
- <Accordion title="Form Fields">
481
- ```javascript
482
- const emailField = await testdriver.find('email input field');
483
- await emailField.click();
484
- await testdriver.type('user@example.com');
485
-
486
- const passwordField = await testdriver.find('password input');
487
- await passwordField.click();
488
- await testdriver.type('MyP@ssw0rd');
489
- ```
490
- </Accordion>
491
-
492
- <Accordion title="Buttons and Links">
493
- ```javascript
494
- const submitBtn = await testdriver.find('submit button');
495
- await submitBtn.click();
496
-
497
- const cancelLink = await testdriver.find('cancel link');
498
- await cancelLink.click();
499
-
500
- const menuIcon = await testdriver.find('hamburger menu icon');
501
- await menuIcon.click();
502
- ```
503
- </Accordion>
504
-
505
- <Accordion title="Dynamic Content">
506
- ```javascript
507
- // Wait for loading to complete
508
- let content;
509
- for (let i = 0; i < 30; i++) {
510
- content = await testdriver.find('results table');
511
- if (content.found()) break;
512
- await new Promise(r => setTimeout(r, 1000));
513
- }
514
-
515
- // Interact with loaded content
516
- const firstRow = await testdriver.find('first row in the results table');
517
- await firstRow.click();
518
- ```
519
- </Accordion>
520
-
521
- <Accordion title="Complex UI Elements">
522
- ```javascript
523
- // Modals and dialogs
524
- const modal = await testdriver.find('confirmation dialog');
525
- if (modal.found()) {
526
- const confirmBtn = await testdriver.find('confirm button in the dialog');
527
- await confirmBtn.click();
528
- }
529
-
530
- // Dropdown menus
531
- const dropdown = await testdriver.find('country dropdown');
532
- await dropdown.click();
533
-
534
- const option = await testdriver.find('United States option');
535
- await option.click();
536
- ```
537
- </Accordion>
538
- </AccordionGroup>
539
-
540
- ## Complete Example
175
+ ### Tighten the cache for one call
541
176
 
542
177
  ```javascript
543
- import { beforeAll, afterAll, describe, it, expect } from 'vitest';
544
- import TestDriver from 'testdriverai';
545
-
546
- describe('Element Finding', () => {
547
- let testdriver;
548
-
549
- beforeAll(async () => {
550
- client = new TestDriver(process.env.TD_API_KEY);
551
- await testdriver.auth();
552
- await testdriver.connect();
553
- });
554
-
555
- afterAll(async () => {
556
- await testdriver.disconnect();
557
- });
558
-
559
- it('should find and interact with elements', async () => {
560
- await testdriver.focusApplication('Google Chrome');
561
-
562
- // Find login form elements
563
- const usernameField = await testdriver.find('username input field');
564
- expect(usernameField.found()).toBe(true);
565
-
566
- await usernameField.click();
567
- await testdriver.type('testuser');
568
-
569
- // Find with context
570
- const passwordField = await testdriver.find('password input below username');
571
- await passwordField.click();
572
- await testdriver.type('password123');
573
-
574
- // Find button
575
- const submitBtn = await testdriver.find('green submit button');
576
- expect(submitBtn.found()).toBe(true);
577
-
578
- console.log('Button location:', submitBtn.centerX, submitBtn.centerY);
579
-
580
- await submitBtn.click();
581
-
582
- // Wait for success message
583
- let successMsg;
584
- for (let i = 0; i < 10; i++) {
585
- successMsg = await testdriver.find('success notification');
586
- if (successMsg.found()) break;
587
- await new Promise(r => setTimeout(r, 1000));
588
- }
589
-
590
- expect(successMsg.found()).toBe(true);
591
- });
178
+ await testdriver.find('Submit button', {
179
+ cache: { thresholds: { screen: 0.01, element: 0.9 } },
592
180
  });
593
181
  ```
594
182
 
595
- ## Related Methods
596
-
597
- - [`click()`](/click) - Click on found elements
598
- - [`hover()`](/hover) - Hover over elements
599
- - [`assert()`](/assert) - Verify element states
600
- - [Elements Reference](/elements) - Complete Element API
601
-
602
- ---
603
-
604
183
  ## findAll()
605
184
 
606
- Locate **all elements** matching a description, rather than just one.
607
-
608
- ### Syntax
185
+ Locates every element that matches a description and returns them as an array.
609
186
 
610
- ```javascript
611
- const elements = await testdriver.findAll(description, options)
187
+ ```typescript
188
+ testdriver.findAll(description: string, options?: FindAllOptions): Promise<Element[]>
612
189
  ```
613
190
 
614
- ### Parameters
615
-
616
191
  <ParamField path="description" type="string" required>
617
- Natural language description of elements to find
192
+ What the elements look like, such as `'coffee cup icon'`.
618
193
  </ParamField>
619
194
 
620
- <ParamField path="options" type="object | number">
621
- Optional cache options (same as `find()`)
622
-
195
+ <ParamField path="options" type="object">
623
196
  <Expandable title="properties">
624
197
  <ParamField path="cacheKey" type="string">
625
- Cache key for storing element location
198
+ Store and look up the cached positions under this key instead of the client's.
626
199
  </ParamField>
627
-
628
- <ParamField path="cacheThreshold" type="number" default={-1}>
629
- Similarity threshold (0-1) for cache matching. Set to -1 to disable cache.
200
+ <ParamField path="cache" type="object">
201
+ `{ thresholds: { screen } }`. How much the screen may differ from the cached run (0 to 1) and still reuse the positions. Defaults to `0.05`. `-1` turns the cache off for this call.
202
+ </ParamField>
203
+ <ParamField path="cacheThreshold" type="number">
204
+ Legacy. Same as `cache.thresholds.screen`.
630
205
  </ParamField>
631
206
  </Expandable>
632
207
  </ParamField>
633
208
 
634
- ### Returns
635
-
636
- `Promise<Element[]>` - Array of Element instances
209
+ `findAll()` differs from `find()`:
637
210
 
638
- ### Examples
639
-
640
- #### Basic Usage
211
+ - It checks the screen once. There's no `timeout`.
212
+ - It returns `[]` when nothing matches. It doesn't throw.
213
+ - It doesn't support `confidence`, `verify`, `zoom`, or `type`.
214
+ - You can't chain an action onto it. Pick an element from the array first.
641
215
 
642
216
  ```javascript
643
- // Find all matching elements
644
- const buttons = await testdriver.findAll('button');
645
- console.log(`Found ${buttons.length} buttons`);
217
+ const coffeeIcons = await testdriver.findAll('coffee cup icon');
218
+ expect(coffeeIcons.length).toBeGreaterThan(0);
646
219
 
647
- // Interact with specific element
648
- if (buttons.length > 0) {
649
- await buttons[0].click(); // Click first button
650
- }
651
-
652
- // Iterate over all
653
- for (const button of buttons) {
654
- console.log(`Button at (${button.x}, ${button.y})`);
220
+ for (const icon of coffeeIcons) {
221
+ await icon.click();
655
222
  }
656
223
  ```
657
224
 
658
- #### Finding Multiple Items
225
+ See the [find all example](/examples/findall-coffee-icons) for a complete test.
659
226
 
660
- ```javascript
661
- // Find all list items
662
- const items = await testdriver.findAll('list item');
227
+ ## Complete example
663
228
 
664
- // Find specific item by index
665
- const thirdItem = items[2];
666
- await thirdItem.click();
229
+ ```javascript tests/login.test.mjs
230
+ import { describe, expect, it } from 'vitest';
231
+ import { TestDriver } from 'testdriverai/vitest/hooks';
667
232
 
668
- // Check all items
669
- for (let i = 0; i < items.length; i++) {
670
- console.log(`Item ${i + 1}: ${items[i].text || 'No text'}`);
671
- }
672
- ```
673
-
674
- #### With Caching
675
-
676
- ```javascript
677
- // Cache element locations for faster subsequent runs
678
- const menuItems = await testdriver.findAll('menu item', {
679
- cacheKey: 'main-menu-items'
680
- });
681
-
682
- // First run: ~2-3 seconds (AI call)
683
- // Subsequent runs: ~100ms (cache hit)
684
- ```
233
+ describe('Login', () => {
234
+ it('signs in with valid credentials', async (context) => {
235
+ const testdriver = TestDriver(context);
236
+ await testdriver.provision.chrome({ url: 'https://testdriver-sandbox.vercel.app/login' });
685
237
 
686
- #### Empty Results
238
+ // The sandbox login page shows its test password on screen
239
+ const password = await testdriver.extract('the password');
687
240
 
688
- ```javascript
689
- // Returns empty array if nothing found (doesn't throw error)
690
- const errors = await testdriver.findAll('error message');
241
+ await testdriver.find('Username input field').click();
242
+ await testdriver.type('standard_user');
691
243
 
692
- if (errors.length === 0) {
693
- console.log('No errors found - test passed!');
694
- } else {
695
- console.log(`Found ${errors.length} errors`);
696
- }
697
- ```
244
+ await testdriver.find('Password input field').click();
245
+ await testdriver.type(password, { secret: true });
698
246
 
699
- ### Differences from find()
700
-
701
- | Feature | find() | findAll() |
702
- |---------|--------|-----------|
703
- | Return type | Single `Element` | Array of `Element[]` |
704
- | If nothing found | Throws `ElementNotFoundError` | Returns empty array `[]` |
705
- | Chainable | ✅ Yes: `await find('button').click()` | ❌ No (returns array) |
706
- | Use case | One specific element | Multiple similar elements |
707
- | Cache support | ✅ Yes | ✅ Yes |
708
-
709
- ### Use Cases
710
-
711
- <AccordionGroup>
712
- <Accordion title="Table Rows">
713
- ```javascript
714
- // Find all rows in a table
715
- const rows = await testdriver.findAll('table row');
716
-
717
- // Click every row
718
- for (const row of rows) {
719
- await row.click();
720
- await new Promise(r => setTimeout(r, 500)); // Wait between clicks
721
- }
722
-
723
- // Or click specific row
724
- await rows[2].click(); // Click third row
725
- ```
726
- </Accordion>
727
-
728
- <Accordion title="Checkboxes/Radio Buttons">
729
- ```javascript
730
- // Find all checkboxes
731
- const checkboxes = await testdriver.findAll('checkbox');
732
-
733
- // Check all boxes
734
- for (const checkbox of checkboxes) {
735
- await checkbox.click();
736
- }
737
-
738
- // Or select first unchecked
739
- const unchecked = checkboxes[0];
740
- await unchecked.click();
741
- ```
742
- </Accordion>
743
-
744
- <Accordion title="Navigation Links">
745
- ```javascript
746
- // Find all navigation links
747
- const navLinks = await testdriver.findAll('navigation link');
748
-
749
- // Validate all are present
750
- expect(navLinks.length).toBeGreaterThan(0);
751
-
752
- // Click specific link by text
753
- const homeLink = navLinks.find(link =>
754
- link.text?.toLowerCase().includes('home')
755
- );
756
-
757
- if (homeLink) {
758
- await homeLink.click();
759
- }
760
- ```
761
- </Accordion>
762
-
763
- <Accordion title="Conditional Interactions">
764
- ```javascript
765
- // Check if any error messages exist
766
- const errors = await testdriver.findAll('error message');
767
-
768
- if (errors.length > 0) {
769
- console.log(`Found ${errors.length} validation errors`);
770
-
771
- // Log each error location
772
- errors.forEach((error, i) => {
773
- console.log(`Error ${i + 1} at (${error.x}, ${error.y})`);
774
- });
775
- } else {
776
- console.log('Form validation passed!');
777
- }
778
- ```
779
- </Accordion>
780
- </AccordionGroup>
781
-
782
- ### Complete Example
247
+ await testdriver.find('Sign in button below the password field').click();
783
248
 
784
- ```javascript
785
- import { test, expect } from 'vitest';
786
- import { chrome } from 'testdriverai/presets';
787
-
788
- test('select multiple items from list', async (context) => {
789
- const { testdriver } = await chrome(context, {
790
- url: 'https://example.com/products'
249
+ const addToCart = await testdriver.find('Add to Cart button under the TestDriver Hat', {
250
+ timeout: 30000,
251
+ });
252
+ expect(addToCart.found()).toBe(true);
791
253
  });
792
-
793
- // Find all product cards
794
- const products = await testdriver.findAll('product card');
795
-
796
- expect(products.length).toBeGreaterThan(0);
797
- console.log(`Found ${products.length} products`);
798
-
799
- // Click first 3 products
800
- const productsToSelect = Math.min(3, products.length);
801
-
802
- for (let i = 0; i < productsToSelect; i++) {
803
- await products[i].click();
804
- console.log(`Selected product ${i + 1}`);
805
- await new Promise(r => setTimeout(r, 500)); // Brief pause
806
- }
807
-
808
- // Verify selections
809
- const selectedBadges = await testdriver.findAll('selected badge');
810
- expect(selectedBadges.length).toBe(productsToSelect);
811
254
  });
812
255
  ```
813
256
 
814
- ### Best Practices
815
-
816
- <Check>
817
- **Handle empty arrays gracefully**
818
-
819
- ```javascript
820
- // ✅ Good - check length first
821
- const items = await testdriver.findAll('list item');
822
- if (items.length > 0) {
823
- await items[0].click();
824
- }
825
-
826
- // ❌ Bad - may throw error
827
- const items = await testdriver.findAll('list item');
828
- await items[0].click(); // Error if array is empty!
829
- ```
830
- </Check>
831
-
832
- <Check>
833
- **Use find() for single elements**
834
-
835
- ```javascript
836
- // ✅ Use find() when you need exactly one
837
- const submitBtn = await testdriver.find('submit button');
838
- await submitBtn.click();
839
-
840
- // ❌ Unnecessary - findAll() returns array
841
- const buttons = await testdriver.findAll('submit button');
842
- await buttons[0].click(); // Extra array handling
843
- ```
844
- </Check>
845
-
846
- <Check>
847
- **Cache for performance**
848
-
849
- ```javascript
850
- // First run - slow (AI call)
851
- const items = await testdriver.findAll('menu item', {
852
- cacheKey: 'menu-items'
853
- });
854
-
855
- // Subsequent runs - fast (cache hit)
856
- // ~10-20x faster than without cache
857
- ```
858
- </Check>
257
+ ## Related
258
+
259
+ - [Locate elements](/locating-elements): write descriptions that match reliably
260
+ - [click()](/click) and [hover()](/hover): act on what you found
261
+ - [Caching](/caching): how element positions are reused
262
+ - [Element not found example](/examples/element-not-found): handle a missing element