testdriverai 7.2.9 → 7.2.10

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (124) hide show
  1. package/.github/workflows/testdriver.yml +127 -0
  2. package/.testdriver/last-sandbox +7 -0
  3. package/agent/events.js +1 -0
  4. package/agent/index.js +71 -54
  5. package/agent/lib/sandbox.js +11 -1
  6. package/agents.md +393 -0
  7. package/debug/01-table-initial.png +0 -0
  8. package/debug/02-after-ai-explore.png +0 -0
  9. package/debug/02-after-scroll.png +0 -0
  10. package/docs/docs.json +93 -125
  11. package/docs/v7/_drafts/caching.mdx +2 -2
  12. package/docs/v7/{getting-started → _drafts}/installation.mdx +0 -66
  13. package/docs/v7/{features/coverage.mdx → _drafts/powerful.mdx} +1 -90
  14. package/docs/v7/{features → _drafts}/scalable.mdx +126 -4
  15. package/docs/v7/_drafts/screenshot.mdx +155 -0
  16. package/docs/v7/_drafts/writing-tests.mdx +25 -0
  17. package/docs/v7/{api/act.mdx → ai.mdx} +27 -27
  18. package/docs/v7/{api/assert.mdx → assert.mdx} +3 -3
  19. package/docs/v7/aws-setup.mdx +338 -0
  20. package/docs/v7/caching.mdx +128 -0
  21. package/docs/v7/ci-cd.mdx +605 -0
  22. package/docs/v7/{api/click.mdx → click.mdx} +4 -4
  23. package/docs/v7/cloud.mdx +120 -0
  24. package/docs/v7/customizing-devices.mdx +129 -0
  25. package/docs/v7/{api/doubleClick.mdx → double-click.mdx} +5 -5
  26. package/docs/v7/enterprise.mdx +135 -0
  27. package/docs/v7/examples.mdx +5 -0
  28. package/docs/v7/{api/exec.mdx → exec.mdx} +3 -3
  29. package/docs/v7/{api/find.mdx → find.mdx} +17 -21
  30. package/docs/v7/{api/focusApplication.mdx → focus-application.mdx} +3 -3
  31. package/docs/v7/generating-tests.mdx +32 -0
  32. package/docs/v7/{api/hover.mdx → hover.mdx} +3 -3
  33. package/docs/v7/locating-elements.mdx +71 -0
  34. package/docs/v7/making-assertions.mdx +32 -0
  35. package/docs/v7/{api/mouseDown.mdx → mouse-down.mdx} +7 -7
  36. package/docs/v7/{api/mouseUp.mdx → mouse-up.mdx} +8 -8
  37. package/docs/v7/performing-actions.mdx +51 -0
  38. package/docs/v7/{api/pressKeys.mdx → press-keys.mdx} +3 -3
  39. package/docs/v7/quickstart.mdx +162 -0
  40. package/docs/v7/reusable-code.mdx +240 -0
  41. package/docs/v7/{api/rightClick.mdx → right-click.mdx} +5 -5
  42. package/docs/v7/running-tests.mdx +181 -0
  43. package/docs/v7/{api/scroll.mdx → scroll.mdx} +3 -3
  44. package/docs/v7/secrets.mdx +115 -0
  45. package/docs/v7/self-hosted.mdx +66 -0
  46. package/docs/v7/{api/type.mdx → type.mdx} +3 -3
  47. package/docs/v7/variables.mdx +111 -0
  48. package/docs/v7/waiting-for-elements.mdx +66 -0
  49. package/docs/v7/what-is-testdriver.mdx +54 -0
  50. package/lib/vitest/hooks.mjs +80 -68
  51. package/package.json +1 -1
  52. package/sdk.d.ts +22 -9
  53. package/sdk.js +177 -44
  54. package/test/manual/reconnect-provision.test.mjs +49 -0
  55. package/test/manual/reconnect-signin.test.mjs +41 -0
  56. package/test/testdriver/ai.test.mjs +30 -0
  57. package/test/testdriver/setup/testHelpers.mjs +0 -1
  58. package/test/testdriver/windows-installer.test.mjs +61 -0
  59. package/tests/table-sort-enrollments.test.mjs +72 -0
  60. package/tests/table-sort-experiment.test.mjs +42 -0
  61. package/tests/table-sort-setup.test.mjs +59 -0
  62. package/vitest.config.mjs +1 -0
  63. package/docs/v7/api/assertions.mdx +0 -403
  64. package/docs/v7/features/ai-native.mdx +0 -413
  65. package/docs/v7/features/application-logs.mdx +0 -353
  66. package/docs/v7/features/browser-logs.mdx +0 -414
  67. package/docs/v7/features/cache-management.mdx +0 -402
  68. package/docs/v7/features/continuous-testing.mdx +0 -346
  69. package/docs/v7/features/data-driven-testing.mdx +0 -441
  70. package/docs/v7/features/easy-to-write.mdx +0 -280
  71. package/docs/v7/features/enterprise.mdx +0 -656
  72. package/docs/v7/features/fast.mdx +0 -406
  73. package/docs/v7/features/managed-sandboxes.mdx +0 -384
  74. package/docs/v7/features/network-monitoring.mdx +0 -568
  75. package/docs/v7/features/parallel-execution.mdx +0 -381
  76. package/docs/v7/features/powerful.mdx +0 -531
  77. package/docs/v7/features/sandbox-customization.mdx +0 -229
  78. package/docs/v7/features/stable.mdx +0 -473
  79. package/docs/v7/features/system-performance.mdx +0 -616
  80. package/docs/v7/features/test-analytics.mdx +0 -373
  81. package/docs/v7/features/test-cases.mdx +0 -393
  82. package/docs/v7/features/test-replays.mdx +0 -408
  83. package/docs/v7/features/test-reports.mdx +0 -308
  84. package/docs/v7/getting-started/debugging-tests.mdx +0 -382
  85. package/docs/v7/getting-started/quickstart.mdx +0 -90
  86. package/docs/v7/getting-started/running-tests.mdx +0 -173
  87. package/docs/v7/getting-started/setting-up-in-ci.mdx +0 -612
  88. package/docs/v7/getting-started/writing-tests.mdx +0 -534
  89. package/docs/v7/overview/what-is-testdriver.mdx +0 -386
  90. package/docs/v7/presets/chrome-extension.mdx +0 -248
  91. package/docs/v7/presets/chrome.mdx +0 -300
  92. package/docs/v7/presets/electron.mdx +0 -460
  93. package/docs/v7/presets/vscode.mdx +0 -417
  94. package/docs/v7/presets/webapp.mdx +0 -393
  95. package/vitest.config.js +0 -18
  96. /package/docs/v7/{commands → _drafts/commands}/assert.mdx +0 -0
  97. /package/docs/v7/{commands → _drafts/commands}/exec.mdx +0 -0
  98. /package/docs/v7/{commands → _drafts/commands}/focus-application.mdx +0 -0
  99. /package/docs/v7/{commands → _drafts/commands}/hover-image.mdx +0 -0
  100. /package/docs/v7/{commands → _drafts/commands}/hover-text.mdx +0 -0
  101. /package/docs/v7/{commands → _drafts/commands}/if.mdx +0 -0
  102. /package/docs/v7/{commands → _drafts/commands}/match-image.mdx +0 -0
  103. /package/docs/v7/{commands → _drafts/commands}/press-keys.mdx +0 -0
  104. /package/docs/v7/{commands → _drafts/commands}/remember.mdx +0 -0
  105. /package/docs/v7/{commands → _drafts/commands}/run.mdx +0 -0
  106. /package/docs/v7/{commands → _drafts/commands}/scroll-until-image.mdx +0 -0
  107. /package/docs/v7/{commands → _drafts/commands}/scroll-until-text.mdx +0 -0
  108. /package/docs/v7/{commands → _drafts/commands}/scroll.mdx +0 -0
  109. /package/docs/v7/{commands → _drafts/commands}/type.mdx +0 -0
  110. /package/docs/v7/{commands → _drafts/commands}/wait-for-image.mdx +0 -0
  111. /package/docs/v7/{commands → _drafts/commands}/wait-for-text.mdx +0 -0
  112. /package/docs/v7/{commands → _drafts/commands}/wait.mdx +0 -0
  113. /package/docs/v7/{getting-started → _drafts}/configuration.mdx +0 -0
  114. /package/docs/v7/{features → _drafts}/observable.mdx +0 -0
  115. /package/docs/v7/{platforms → _drafts/platforms}/linux.mdx +0 -0
  116. /package/docs/v7/{platforms → _drafts/platforms}/macos.mdx +0 -0
  117. /package/docs/v7/{platforms → _drafts/platforms}/windows.mdx +0 -0
  118. /package/docs/v7/{playwright.mdx → _drafts/playwright.mdx} +0 -0
  119. /package/docs/v7/{overview → _drafts}/readme.mdx +0 -0
  120. /package/docs/v7/{features → _drafts}/reports.mdx +0 -0
  121. /package/docs/v7/{api/client.mdx → client.mdx} +0 -0
  122. /package/docs/v7/{api/dashcam.mdx → dashcam.mdx} +0 -0
  123. /package/docs/v7/{api/elements.mdx → elements.mdx} +0 -0
  124. /package/docs/v7/{api/sandbox.mdx → sandbox.mdx} +0 -0
@@ -1,402 +0,0 @@
1
- ---
2
- title: "Cache Management"
3
- description: "Browse cached selectors, view found targets, and manage cache entries"
4
- icon: "database"
5
- ---
6
-
7
- View and manage the intelligent cache that powers TestDriver's fast test execution. Browse all cached element locations, see visual previews, and clear cache when needed.
8
-
9
- ## Selector Cache Dashboard
10
-
11
- Access cache management at [app.testdriver.ai/cache](https://app.testdriver.ai/cache):
12
-
13
- <Card title="Selector Cache" icon="database" href="https://app.testdriver.ai/cache">
14
- Browse all cached selector entries with visual previews
15
- </Card>
16
-
17
- ### Cache Overview
18
-
19
- The cache stores AI vision analysis results for faster test execution:
20
-
21
- <CardGroup cols={2}>
22
- <Card title="Cache Hit Rate" icon="percent">
23
- Percentage of selectors found in cache
24
- - Higher is better
25
- - Typical: 80-95% after a few runs
26
- - Shows cache effectiveness
27
- </Card>
28
-
29
- <Card title="Total Entries" icon="hashtag">
30
- Number of cached selectors
31
- - Grows with test diversity
32
- - Automatically managed
33
- - Cleared on code changes
34
- </Card>
35
-
36
- <Card title="Cache Size" icon="hard-drive">
37
- Storage used by cache
38
- - Screenshots + metadata
39
- - Typical: 10-500 MB
40
- - Automatic cleanup
41
- </Card>
42
-
43
- <Card title="Minimum Hits" icon="filter">
44
- Filter by cache usage
45
- - 0 = Show all entries
46
- - 5+ = Show frequently used
47
- - Optimize cache value
48
- </Card>
49
- </CardGroup>
50
-
51
- ## Viewing Cache Entries
52
-
53
- Each cached entry shows:
54
-
55
- ```
56
- Cache Entry Example:
57
- ┌────────────────────────────────────────────────────┐
58
- │ Selector: "Sign in, black button below password" │
59
- │ │
60
- │ [Screenshot Preview] │
61
- │ ┌─────────────────┐ │
62
- │ │ │ │
63
- │ │ [Found Element]│ ← Visual target │
64
- │ │ │ │
65
- │ └─────────────────┘ │
66
- │ │
67
- │ File: test/testdriver.test.js │
68
- │ Context: by "standard_user" password field │
69
- │ Hash: a8f3c21b... │
70
- │ Status: ✓ Passed │
71
- │ │
72
- │ [TEST] [single] [windows] [s blac] │
73
- │ [fe68cfe63c784b2a47...] [hover:start-test] │
74
- └────────────────────────────────────────────────────┘
75
- ```
76
-
77
- ### Entry Information
78
-
79
- - **Selector Text** - Natural language description used
80
- - **Screenshot** - Visual preview showing found element
81
- - **File** - Which test file used this selector
82
- - **Context** - Additional selector context
83
- - **Hash** - Unique cache key identifier
84
- - **Status** - Whether element was found
85
- - **Tags** - Test metadata and labels
86
-
87
- ## Cache Key Generation
88
-
89
- Understanding how cache keys work:
90
-
91
- ```
92
- Cache Key Components:
93
- 1. Test File Hash: SHA-256 of test file contents
94
- 2. Selector Text: "submit button"
95
- 3. Screenshot Hash: Perceptual hash of screen
96
- 4. Platform: linux, windows, mac
97
-
98
- Combined Key:
99
- fe68cfe63c784b2a47bcd8a8f3c21b7e
100
- ```
101
-
102
- **Cache Invalidation:**
103
- - Test file changes → New hash → Fresh cache
104
- - UI changes → Different screenshot → New entry
105
- - Selector changes → Different text → New entry
106
- - Platform changes → Separate cache per OS
107
-
108
- ## Gallery View
109
-
110
- Visual browse mode for cache entries:
111
-
112
- ```
113
- [Selector Cache - Gallery View]
114
- ┌──────────┬──────────┬──────────┬──────────┐
115
- │ │ │ │ │
116
- │ [IMG] │ [IMG] │ [IMG] │ [IMG] │
117
- │ login │ submit │ menu │ cart │
118
- │ button │ button │ icon │ button │
119
- │ │ │ │ │
120
- ├──────────┼──────────┼──────────┼──────────┤
121
- │ │ │ │ │
122
- │ [IMG] │ [IMG] │ [IMG] │ [IMG] │
123
- │ search │ profile │ logout │ save │
124
- │ input │ avatar │ link │ button │
125
- │ │ │ │ │
126
- └──────────┴──────────┴──────────┴──────────┘
127
- ```
128
-
129
- **Features:**
130
- - Click to enlarge screenshot
131
- - Hover to see full selector text
132
- - Color-coded by status (green = found, red = not found)
133
- - Sort by usage frequency
134
-
135
- ## Table View
136
-
137
- Detailed list with sortable columns:
138
-
139
- ```
140
- [Selector Cache - Table View]
141
- ┌─────────────────────┬──────────┬─────────┬────────┬──────────┐
142
- │ Selector │ Preview │ File │ Status │ Cache Key│
143
- ├─────────────────────┼──────────┼─────────┼────────┼──────────┤
144
- │ Sign in, black btn │ [thumb] │ test.js │ PASSED │ fe68cfe..│
145
- │ password input │ [thumb] │ test.js │ PASSED │ a8f3c21..│
146
- │ username field │ [thumb] │ test.js │ PASSED │ 7b2d4e9..│
147
- │ submit button │ [thumb] │ test.js │ NOTFND │ c5a1f8e..│
148
- └─────────────────────┴──────────┴─────────┴────────┴──────────┘
149
-
150
- Sort by: ▼ Selector | File | Status | Usage
151
- ```
152
-
153
- ## Filtering Cache Entries
154
-
155
- Find specific cached items:
156
-
157
- <Tabs>
158
- <Tab title="By Cache Key">
159
- ```
160
- Filter by Cache Key:
161
- All cache keys
162
- fe68cfe6... (specific hash)
163
- a8f3c21b... (specific hash)
164
- ```
165
- </Tab>
166
-
167
- <Tab title="By Test File">
168
- ```
169
- Filter by File:
170
- All files
171
- test/login.test.js
172
- test/checkout.test.js
173
- test/search.test.js
174
- ```
175
- </Tab>
176
-
177
- <Tab title="By Status">
178
- ```
179
- Filter by Status:
180
- All statuses
181
- ✓ Passed only
182
- ✗ Not found only
183
- ```
184
- </Tab>
185
-
186
- <Tab title="By Selector Text">
187
- ```
188
- Search selectors:
189
- "button"
190
- "Sign in"
191
- "password"
192
- Partial text matching
193
- ```
194
- </Tab>
195
- </Tabs>
196
-
197
- ## Cache Statistics
198
-
199
- Per-file and global metrics:
200
-
201
- ```
202
- Cache Performance:
203
- ┌────────────────────────────────────────────────────┐
204
- │ test/login.test.js │
205
- │ • Entries: 12 │
206
- │ • Hit Rate: 95% │
207
- │ • Avg Match Time: 15ms │
208
- │ • Total Saves: ~28 seconds per run │
209
- ├────────────────────────────────────────────────────┤
210
- │ test/checkout.test.js │
211
- │ • Entries: 23 │
212
- │ • Hit Rate: 87% │
213
- │ • Avg Match Time: 18ms │
214
- │ • Total Saves: ~52 seconds per run │
215
- └────────────────────────────────────────────────────┘
216
-
217
- Global Stats:
218
- • Total Cached Selectors: 156
219
- • Overall Hit Rate: 91%
220
- • Total Time Saved: 8.5 minutes per full suite
221
- • Cache Size: 143 MB
222
- ```
223
-
224
- ## Clear Cache
225
-
226
- Manage cache storage:
227
-
228
- <CardGroup cols={2}>
229
- <Card title="Clear All" icon="trash">
230
- Remove all cache entries
231
- - Fresh start
232
- - Reclaim storage
233
- - Force re-analysis
234
- </Card>
235
-
236
- <Card title="Clear by File" icon="file-slash">
237
- Remove specific test file cache
238
- - After UI changes
239
- - Test refactoring
240
- - Targeted cleanup
241
- </Card>
242
-
243
- <Card title="Clear Failed" icon="xmark">
244
- Remove not-found entries
245
- - Clean up misses
246
- - Reduce cache size
247
- - Keep only successful matches
248
- </Card>
249
-
250
- <Card title="Clear Old" icon="clock-rotate-left">
251
- Remove entries older than X days
252
- - Automatic cleanup
253
- - Keep recent only
254
- - Configurable threshold
255
- </Card>
256
- </CardGroup>
257
-
258
- <Warning>
259
- Clearing cache will slow down the next test run as TestDriver rebuilds the cache with AI vision analysis.
260
- </Warning>
261
-
262
- ## Cache Configuration
263
-
264
- Customize caching behavior:
265
-
266
- ```javascript
267
- import { test } from 'vitest';
268
- import { chrome } from 'testdriverai/presets';
269
-
270
- test('cache configuration', async (context) => {
271
- const { testdriver } = await chrome(context, {
272
- url: 'https://example.com',
273
- cache: {
274
- enabled: true, // Enable/disable caching
275
- ttl: 30 * 24 * 60 * 60, // 30 days in seconds
276
- threshold: 0.85, // 85% similarity threshold
277
- maxSize: 500 * 1024 * 1024 // 500 MB max cache size
278
- }
279
- });
280
- });
281
- ```
282
-
283
- ### Cache Matching Strategies
284
-
285
- TestDriver uses three-tier matching:
286
-
287
- <Steps>
288
- <Step title="Exact Hash Match">
289
- **Fastest** - Perceptual hash comparison
290
-
291
- - Speed: < 1ms
292
- - Accuracy: 100%
293
- - Use case: Identical screenshots
294
- </Step>
295
-
296
- <Step title="Pixel Diff Match">
297
- **Fast** - Pixel-by-pixel comparison
298
-
299
- - Speed: < 50ms
300
- - Accuracy: 95%+ similarity
301
- - Use case: Minor UI changes (hover states)
302
- </Step>
303
-
304
- <Step title="Template Match">
305
- **Fallback** - Edge detection and structure
306
-
307
- - Speed: < 200ms
308
- - Accuracy: 75%+ confidence
309
- - Use case: Layout changes, responsive design
310
- </Step>
311
- </Steps>
312
-
313
- ## Found Targets Preview
314
-
315
- See exactly what elements were cached:
316
-
317
- ```
318
- Target Details:
319
- ┌────────────────────────────────────────────────────┐
320
- │ Element Found: ✓ │
321
- │ │
322
- │ Coordinates: (512, 387) │
323
- │ Bounding Box: │
324
- │ X: 450, Y: 370 │
325
- │ Width: 124, Height: 34 │
326
- │ │
327
- │ Visual Preview: │
328
- │ [Screenshot with highlight box around element] │
329
- │ │
330
- │ Confidence: 98.5% │
331
- │ Match Type: Exact Hash │
332
- │ Match Time: 12ms │
333
- └────────────────────────────────────────────────────┘
334
- ```
335
-
336
- ## Export Cache Data
337
-
338
- Download cache information:
339
-
340
- <Tabs>
341
- <Tab title="JSON">
342
- ```json
343
- {
344
- "cache": [
345
- {
346
- "selector": "Sign in, black button below password",
347
- "file": "test/testdriver.test.js",
348
- "cacheKey": "fe68cfe63c784b2a",
349
- "status": "PASSED",
350
- "coordinates": { "x": 512, "y": 387 },
351
- "timestamp": "2024-12-09T10:30:00Z",
352
- "hits": 45
353
- }
354
- ]
355
- }
356
- ```
357
- </Tab>
358
-
359
- <Tab title="CSV">
360
- ```csv
361
- Selector,File,Cache Key,Status,Hits,Last Used
362
- Sign in button,test.js,fe68cfe...,PASSED,45,2024-12-09
363
- password input,test.js,a8f3c21...,PASSED,38,2024-12-09
364
- ```
365
- </Tab>
366
- </Tabs>
367
-
368
- ## Learn More
369
-
370
- <CardGroup cols={2}>
371
- <Card
372
- title="Intelligent Caching"
373
- icon="bolt"
374
- href="/v7/features/fast"
375
- >
376
- How caching speeds up tests
377
- </Card>
378
-
379
- <Card
380
- title="Test Replays"
381
- icon="video"
382
- href="/v7/features/test-replays"
383
- >
384
- View cache hits in action
385
- </Card>
386
-
387
- <Card
388
- title="Performance"
389
- icon="gauge-high"
390
- href="/v7/guides/performance"
391
- >
392
- Optimize cache settings
393
- </Card>
394
-
395
- <Card
396
- title="Dashboard"
397
- icon="chart-simple"
398
- href="https://app.testdriver.ai"
399
- >
400
- Access cache management
401
- </Card>
402
- </CardGroup>