@testdriverai/agent 7.11.229-canary → 7.11.231-test

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (100) hide show
  1. package/docs/_data/examples-manifest.json +16 -16
  2. package/docs/_scripts/generate-examples.js +38 -45
  3. package/docs/agent.mdx +174 -93
  4. package/docs/assert.mdx +73 -269
  5. package/docs/aws-setup.mdx +196 -314
  6. package/docs/bring-your-own-machine.mdx +139 -206
  7. package/docs/cache.mdx +144 -151
  8. package/docs/caching.mdx +99 -89
  9. package/docs/captcha.mdx +126 -111
  10. package/docs/ci-cd.mdx +154 -543
  11. package/docs/click.mdx +74 -229
  12. package/docs/client.mdx +177 -231
  13. package/docs/copilot/auto-healing.mdx +86 -370
  14. package/docs/copilot/running-tests.mdx +121 -930
  15. package/docs/customizing-devices.mdx +74 -251
  16. package/docs/dashcam.mdx +72 -405
  17. package/docs/debugging-with-screenshots.mdx +105 -371
  18. package/docs/docs.json +75 -18
  19. package/docs/double-click.mdx +48 -68
  20. package/docs/errors.mdx +102 -176
  21. package/docs/events.mdx +170 -327
  22. package/docs/examples/{hover-text-with-description.mdx → add-to-cart.mdx} +30 -35
  23. package/docs/examples/assert.mdx +28 -41
  24. package/docs/examples/click-button.mdx +71 -0
  25. package/docs/examples/{hover-image.mdx → click-icon.mdx} +32 -28
  26. package/docs/examples/element-not-found.mdx +27 -26
  27. package/docs/examples/findall-coffee-icons.mdx +28 -32
  28. package/docs/examples/installer.mdx +61 -25
  29. package/docs/examples/launch-vscode-linux.mdx +28 -51
  30. package/docs/examples/navigate-and-scroll.mdx +82 -0
  31. package/docs/examples/parse.mdx +24 -28
  32. package/docs/examples/press-keys.mdx +28 -25
  33. package/docs/examples/scroll.mdx +25 -27
  34. package/docs/examples/type.mdx +26 -21
  35. package/docs/exec.mdx +92 -256
  36. package/docs/extract.mdx +69 -170
  37. package/docs/find.mdx +140 -736
  38. package/docs/focus-application.mdx +69 -246
  39. package/docs/generating-tests.mdx +103 -137
  40. package/docs/guide/best-practices-polling.mdx +65 -141
  41. package/docs/hosted.mdx +80 -144
  42. package/docs/hover.mdx +62 -235
  43. package/docs/locating-elements.mdx +74 -369
  44. package/docs/making-assertions.mdx +116 -74
  45. package/docs/mcp-setup.mdx +80 -63
  46. package/docs/mouse-down.mdx +48 -125
  47. package/docs/mouse-up.mdx +47 -129
  48. package/docs/options.mdx +75 -48
  49. package/docs/parse.mdx +95 -184
  50. package/docs/performing-actions.mdx +104 -94
  51. package/docs/press-keys.mdx +59 -299
  52. package/docs/provision.mdx +137 -182
  53. package/docs/quickstart-claude.mdx +85 -0
  54. package/docs/quickstart-github.mdx +62 -32
  55. package/docs/quickstart-manual.mdx +60 -33
  56. package/docs/quickstart-vscode.mdx +97 -0
  57. package/docs/quickstart.mdx +25 -9
  58. package/docs/redraw.mdx +101 -165
  59. package/docs/reusable-code.mdx +97 -200
  60. package/docs/right-click.mdx +47 -89
  61. package/docs/screenshot.mdx +47 -201
  62. package/docs/screenshots.mdx +87 -137
  63. package/docs/scroll.mdx +71 -154
  64. package/docs/secrets.mdx +96 -91
  65. package/docs/self-hosted.mdx +83 -142
  66. package/docs/snippets/enterprise.mdx +15 -0
  67. package/docs/styles.css +68 -0
  68. package/docs/test-results-json.mdx +167 -204
  69. package/docs/type.mdx +56 -304
  70. package/docs/variables.mdx +121 -81
  71. package/docs/wait.mdx +33 -31
  72. package/docs/what-is-testdriver.mdx +80 -32
  73. package/examples/{hover-text-with-description.test.mjs → add-to-cart.test.mjs} +4 -5
  74. package/examples/assert.test.mjs +3 -20
  75. package/examples/{hover-text.test.mjs → click-button.test.mjs} +3 -4
  76. package/examples/{hover-image.test.mjs → click-icon.test.mjs} +5 -6
  77. package/examples/element-not-found.test.mjs +3 -5
  78. package/examples/findall-coffee-icons.test.mjs +3 -3
  79. package/examples/installer.test.mjs +1 -1
  80. package/examples/launch-vscode-linux.test.mjs +6 -33
  81. package/examples/{scroll-keyboard.test.mjs → navigate-and-scroll.test.mjs} +4 -5
  82. package/examples/parse.test.mjs +1 -2
  83. package/examples/press-keys.test.mjs +4 -4
  84. package/examples/scroll.test.mjs +3 -6
  85. package/examples/type.test.mjs +5 -2
  86. package/lib/provision.js +6 -2
  87. package/package.json +1 -1
  88. package/sdk.d.ts +4 -2
  89. package/sdk.js +1 -1
  90. package/docs/claude-mcp-plugin.mdx +0 -244
  91. package/docs/elements.mdx +0 -606
  92. package/docs/examples/ai.mdx +0 -78
  93. package/docs/examples/captcha-api.mdx +0 -92
  94. package/docs/examples/chrome-extension.mdx +0 -156
  95. package/docs/examples/drag-and-drop.mdx +0 -100
  96. package/docs/examples/formatted-logging.mdx +0 -74
  97. package/docs/examples/hover-text.mdx +0 -70
  98. package/docs/examples/match-image.mdx +0 -97
  99. package/docs/examples/scroll-keyboard.mdx +0 -80
  100. package/docs/examples/scroll-until-text.mdx +0 -109
package/docs/extract.mdx CHANGED
@@ -1,225 +1,124 @@
1
1
  ---
2
2
  title: "extract()"
3
3
  sidebarTitle: "extract"
4
- description: "Read information from the screen using AI and return it as a string"
4
+ description: "Read a value off the screen with AI and get it back as a string."
5
5
  icon: "wand-magic-sparkles"
6
6
  ---
7
7
 
8
- ## Overview
8
+ Read a value off the screen, such as an order number or a price. Describe what you want, and the AI returns it as a string.
9
9
 
10
- Get information from the screen with AI. TestDriver returns it as a string. Describe what you want in natural language. The AI reads the screen and returns the correct value: text, numbers, labels, status messages, or other content on the screen.
10
+ ```javascript
11
+ const orderNumber = await testdriver.extract('the order confirmation number');
12
+ ```
11
13
 
12
- [`assert()`](/assert) returns a boolean result. But `extract()` returns the real value. Thus you can store it, compare it, or use it in later steps and in framework assertions.
14
+ [`assert()`](/assert) tells you whether something is true. `extract()` gives you the value itself, so you can compare it, log it, or type it somewhere else.
13
15
 
14
- ## Syntax
16
+ ## Signature
15
17
 
16
- ```javascript
17
- const value = await testdriver.extract(description)
18
- const value = await testdriver.extract({ description })
18
+ ```typescript
19
+ testdriver.extract(description: string): Promise<string>
20
+ testdriver.extract(options: { description: string }): Promise<string>
19
21
  ```
20
22
 
23
+ Both forms do the same thing.
24
+
21
25
  ## Parameters
22
26
 
23
27
  <ParamField path="description" type="string" required>
24
- A natural language description of the information to read from the screen.
28
+ What to read from the screen, such as `'the total price in the order summary'`. Say what format you want, too: `'the cart total as a number, without the dollar sign'`.
25
29
  </ParamField>
26
30
 
27
- <Info>
28
- `extract()` also accepts an options object, `extract({ description })`. This is the same as the positional form. The string form is the most common.
29
- </Info>
30
-
31
31
  ## Returns
32
32
 
33
- `Promise<string>` — The information from the screen. TestDriver returns the value as text. Parse it or cast it yourself if you need a number or another type.
33
+ `Promise<string>`. The value the AI read, with no extra text. Convert it yourself when you need a number or another type.
34
34
 
35
- ## Examples
36
-
37
- ### Basic Extraction
35
+ ## Errors
38
36
 
39
- ```javascript
40
- // Read text content
41
- const title = await testdriver.extract('the page title');
42
- const heading = await testdriver.extract('the main heading text');
37
+ `extract()` throws if the AI service fails, for example when it's temporarily unavailable.
43
38
 
44
- // Read numbers and prices
45
- const price = await testdriver.extract('the total price shown in the cart');
46
- const count = await testdriver.extract('the number of items in the list');
39
+ It doesn't throw when the value isn't on screen. Check the result before you rely on it.
47
40
 
48
- // Read status and confirmation values
49
- const status = await testdriver.extract('the order status');
50
- const orderNumber = await testdriver.extract('the order confirmation number');
51
- ```
41
+ ## Examples
52
42
 
53
- ### Using the Extracted Value
43
+ ### Compare a value with expect()
54
44
 
55
45
  ```javascript
56
- // Store and reuse across steps
57
- const orderNumber = await testdriver.extract('the order confirmation number');
58
- console.log('Order:', orderNumber);
59
-
60
- // Combine with framework assertions
61
- import { expect } from 'vitest';
62
-
63
46
  const message = await testdriver.extract('the success message text');
64
47
  expect(message).toContain('successfully');
65
-
66
- // Cast to a number when you need to compare
67
- const totalText = await testdriver.extract('the cart total as a number without currency symbol');
68
- expect(Number(totalText)).toBeGreaterThan(0);
69
48
  ```
70
49
 
71
- ## Best Practices
72
-
73
- <Check>
74
- **Be specific about what to read**
75
-
76
- Precise descriptions produce cleaner values:
77
-
78
- ```javascript
79
- // ❌ Too vague — may return extra surrounding text
80
- const price = await testdriver.extract('price');
81
-
82
- // ✅ Specific — targets a single value
83
- const price = await testdriver.extract('the total price in the order summary, digits only');
84
- ```
85
- </Check>
86
-
87
- <Check>
88
- **Ask for the format you want**
89
-
90
- Steer the output by describing the desired shape in the prompt:
50
+ ### Read a number
91
51
 
92
- ```javascript
93
- // Strip currency symbols
94
- const total = await testdriver.extract('the order total as a number without the dollar sign');
95
-
96
- // Isolate a single field
97
- const email = await testdriver.extract('the email address shown in the profile header');
98
- ```
99
- </Check>
100
-
101
- <Check>
102
- **Extract for detailed assertions**
103
-
104
- Use `extract()` when a boolean [`assert()`](/assert) isn't enough and you need the actual value to inspect:
105
-
106
- ```javascript
107
- const confirmation = await testdriver.extract('the confirmation number');
108
- expect(confirmation).toMatch(/^ORD-\d{6}$/);
109
- ```
110
- </Check>
111
-
112
- ## Use Cases
113
-
114
- <AccordionGroup>
115
- <Accordion title="Capturing Confirmation Details">
116
- ```javascript
117
- const submitBtn = await testdriver.find('place order button');
118
- await submitBtn.click();
119
-
120
- // Read the confirmation the app generated
121
- const orderNumber = await testdriver.extract('the order confirmation number');
122
- const eta = await testdriver.extract('the estimated delivery date');
52
+ ```javascript
53
+ const total = await testdriver.extract('the cart total as a number, without the dollar sign');
54
+ expect(Number(total)).toBeGreaterThan(0);
55
+ ```
123
56
 
124
- console.log(`Order ${orderNumber} arrives ${eta}`);
125
- ```
126
- </Accordion>
57
+ ### Check the format of a generated value
127
58
 
128
- <Accordion title="Reading Tooltip and Hover Content">
129
- ```javascript
130
- const icon = await testdriver.find('info icon next to the price');
131
- await icon.hover();
59
+ ```javascript
60
+ const confirmation = await testdriver.extract('the confirmation number');
61
+ expect(confirmation).toMatch(/^ORD-\d{6}$/);
62
+ ```
132
63
 
133
- const tooltipText = await testdriver.extract('the tooltip text');
134
- expect(tooltipText).toContain('tax included');
135
- ```
136
- </Accordion>
64
+ ### Pass a value to the next step
137
65
 
138
- <Accordion title="Verifying Dynamic Values">
139
- ```javascript
140
- // Read a value before an action
141
- const before = await testdriver.extract('the account balance');
66
+ ```javascript
67
+ const resetCode = await testdriver.extract('the password reset code');
142
68
 
143
- const addBtn = await testdriver.find('add funds button');
144
- await addBtn.click();
69
+ await testdriver.find('Reset code input field').click();
70
+ await testdriver.type(resetCode);
71
+ ```
145
72
 
146
- // Read it again after and compare
147
- const after = await testdriver.extract('the account balance');
148
- expect(Number(after.replace(/[^0-9.]/g, ''))).toBeGreaterThan(
149
- Number(before.replace(/[^0-9.]/g, ''))
150
- );
151
- ```
152
- </Accordion>
73
+ ### Read text that appears on hover
153
74
 
154
- <Accordion title="Passing Data Between Steps">
155
- ```javascript
156
- // Read a generated code on one screen...
157
- const resetCode = await testdriver.extract('the password reset code');
75
+ ```javascript
76
+ await testdriver.find('info icon next to the price').hover();
158
77
 
159
- // ...and type it into the next
160
- const codeField = await testdriver.find('reset code input');
161
- await codeField.click();
162
- await testdriver.type(resetCode);
163
- ```
164
- </Accordion>
165
- </AccordionGroup>
78
+ const tooltip = await testdriver.extract('the tooltip text');
79
+ expect(tooltip).toContain('tax included');
80
+ ```
166
81
 
167
- ## Complete Example
82
+ ## How it works
168
83
 
169
- ```javascript
170
- import { beforeAll, afterAll, describe, it, expect } from 'vitest';
171
- import TestDriver from 'testdriverai';
84
+ 1. TestDriver takes a screenshot of the sandbox.
85
+ 2. It sends the screenshot and your description to the AI.
86
+ 3. The AI returns only the value you asked for.
172
87
 
173
- describe('Extraction', () => {
174
- let testdriver;
88
+ Results aren't cached. Every call reads the current screen.
175
89
 
176
- beforeAll(async () => {
177
- testdriver = new TestDriver(process.env.TD_API_KEY);
178
- await testdriver.auth();
179
- await testdriver.connect();
180
- });
90
+ ## Complete example
181
91
 
182
- afterAll(async () => {
183
- await testdriver.disconnect();
184
- });
92
+ ```javascript tests/login.test.mjs
93
+ import { describe, expect, it } from 'vitest';
94
+ import { TestDriver } from 'testdriverai/vitest/hooks';
185
95
 
186
- it('should capture the order confirmation', async () => {
187
- await testdriver.focusApplication('Google Chrome');
96
+ describe('Login', () => {
97
+ it('signs in with the password shown on the page', async (context) => {
98
+ const testdriver = TestDriver(context);
99
+ await testdriver.provision.chrome({ url: 'https://testdriver-sandbox.vercel.app/login' });
188
100
 
189
- // Complete a checkout
190
- const checkoutBtn = await testdriver.find('checkout button');
191
- await checkoutBtn.click();
101
+ // The sandbox login page shows its test password on screen
102
+ const password = await testdriver.extract('the password');
103
+ expect(password).not.toBe('');
192
104
 
193
- const placeOrderBtn = await testdriver.find('place order button');
194
- await placeOrderBtn.click();
105
+ await testdriver.find('Username input field').click();
106
+ await testdriver.type('standard_user');
195
107
 
196
- // Verify we reached confirmation
197
- await testdriver.assert('the order confirmation page is displayed');
108
+ await testdriver.find('Password input field').click();
109
+ await testdriver.type(password, { secret: true });
198
110
 
199
- // Extract the details the app generated
200
- const orderNumber = await testdriver.extract('the order confirmation number');
201
- const total = await testdriver.extract('the order total as a number without currency symbol');
111
+ await testdriver.find('Sign in button below the password field').click();
202
112
 
203
- // Assert on the extracted values
204
- expect(orderNumber).toBeTruthy();
205
- expect(Number(total)).toBeGreaterThan(0);
113
+ const result = await testdriver.assert('the product list is shown');
114
+ expect(result).toBe(true);
206
115
  });
207
116
  });
208
117
  ```
209
118
 
210
- ## How It Works
211
-
212
- 1. TestDriver captures a screenshot of the current screen
213
- 2. The image and your description are sent to the TestDriver API
214
- 3. The AI reads the requested information from the screenshot
215
- 4. The extracted value is returned as a string
216
-
217
- <Note>
218
- Like [assertions](/making-assertions), `extract()` reads the screen fresh on every call — it is not cached — so it always reflects the current state of the app.
219
- </Note>
220
-
221
- ## Related Methods
119
+ ## Related
222
120
 
223
- - [`assert()`](/assert) - Verify screen state with a boolean AI judgment
224
- - [`find()`](/find) - Locate elements to interact with
225
- - [`parse()`](/parse) - Detect all UI elements on screen
121
+ - [assert()](/assert): check a statement about the screen
122
+ - [Make assertions](/making-assertions): choose between `assert()` and `extract()`
123
+ - [find()](/find): locate an element to act on
124
+ - [parse()](/parse): list every element on screen