@testdriverai/mcp 7.9.150-test → 7.9.152-test

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (39) hide show
  1. package/ai/skills/testdriver-caching/SKILL.md +13 -6
  2. package/ai/skills/testdriver-ci-cd/SKILL.md +20 -34
  3. package/ai/skills/testdriver-client/SKILL.md +5 -1
  4. package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +14 -4
  5. package/ai/skills/testdriver-find/SKILL.md +2 -0
  6. package/ai/skills/testdriver-generating-tests/SKILL.md +159 -8
  7. package/ai/skills/testdriver-machine-setup/SKILL.md +75 -8
  8. package/ai/skills/testdriver-making-assertions/SKILL.md +78 -2
  9. package/ai/skills/testdriver-performing-actions/SKILL.md +97 -2
  10. package/ai/skills/testdriver-quickstart/SKILL.md +47 -28
  11. package/ai/skills/testdriver-wait/SKILL.md +1 -1
  12. package/ai/skills/testdriver:generating-tests/SKILL.md +0 -11
  13. package/docs/changelog.mdx +1 -1
  14. package/docs/docs.json +14 -39
  15. package/docs/v7/assert.mdx +0 -1
  16. package/docs/v7/caching.mdx +14 -10
  17. package/docs/v7/ci-cd.mdx +20 -34
  18. package/docs/v7/client.mdx +1 -1
  19. package/docs/v7/copilot/auto-healing.mdx +167 -18
  20. package/docs/v7/copilot/running-tests.mdx +915 -54
  21. package/docs/v7/debugging-with-screenshots.mdx +17 -7
  22. package/docs/v7/generating-tests.mdx +166 -10
  23. package/docs/v7/making-assertions.mdx +81 -4
  24. package/docs/v7/performing-actions.mdx +100 -4
  25. package/docs/v7/quickstart.mdx +299 -21
  26. package/docs/v7/wait.mdx +1 -1
  27. package/package.json +3 -3
  28. package/docs/v7/ai/agent.mdx +0 -72
  29. package/docs/v7/ai/mcp.mdx +0 -228
  30. package/docs/v7/ai/skills.mdx +0 -73
  31. package/docs/v7/ai.mdx +0 -205
  32. package/docs/v7/copilot/creating-tests.mdx +0 -156
  33. package/docs/v7/copilot/github.mdx +0 -143
  34. package/docs/v7/copilot/setup.mdx +0 -143
  35. package/docs/v7/device-config.mdx +0 -317
  36. package/docs/v7/locating-elements.mdx +0 -71
  37. package/docs/v7/machine-setup.mdx +0 -331
  38. package/docs/v7/running-tests.mdx +0 -185
  39. package/docs/v7/waiting-for-elements.mdx +0 -90
@@ -1,13 +1,13 @@
1
1
  ---
2
- title: "Debugging with Screenshots"
3
- sidebarTitle: "Debugging with Screenshots"
4
- description: "View and analyze saved screenshots using MCP commands for test debugging and development"
5
- icon: "magnifying-glass"
2
+ title: "Debug"
3
+ sidebarTitle: "Debug"
4
+ description: "Diagnose failing tests with screenshots, replays, and logs"
5
+ icon: "bug"
6
6
  ---
7
7
 
8
8
  ## Overview
9
9
 
10
- TestDriver MCP provides powerful commands to view and analyze screenshots saved during test execution. This enables rapid debugging, test development, and comparison workflows without manually opening image files.
10
+ When a test fails, debug it by seeing exactly what happened — don't guess. TestDriver captures screenshots, video replays, and logs as your test runs, so you can replay the moment of failure instead of squinting at a stack trace. TestDriver MCP provides powerful commands to view and analyze the screenshots saved during test execution, enabling rapid debugging, test development, and comparison workflows without manually opening image files.
11
11
 
12
12
  <Note>
13
13
  **Automatic Screenshots (Default: Enabled)**: TestDriver automatically captures screenshots before and after every command. Screenshots are named with the line number and action, making it easy to trace exactly which line of code produced each screenshot. For example: `001-click-before-L42-submit-button.png`
@@ -123,7 +123,7 @@ view_local_screenshot({ path: "/full/path/to/screenshot.png" })
123
123
 
124
124
  ### Test Debugging After Failures
125
125
 
126
- When a test fails, use powerful filtering to quickly find relevant screenshots:
126
+ When a test fails, you don't have to wonder what went wrong — use powerful filtering to quickly find the screenshots that show exactly what happened:
127
127
 
128
128
  **1. Find screenshots at the failing line:**
129
129
 
@@ -359,7 +359,7 @@ This helps verify your test logic before running the full test file.
359
359
 
360
360
  ### After Test Runs
361
361
 
362
- When tests fail or behave unexpectedly:
362
+ When tests fail or behave unexpectedly, replay what happened step by step:
363
363
 
364
364
  1. **Run the test** with `vitest run tests/my-test.test.mjs`
365
365
  2. **List screenshots** using `list_local_screenshots`
@@ -409,8 +409,18 @@ When tests fail or behave unexpectedly:
409
409
  </Accordion>
410
410
  </AccordionGroup>
411
411
 
412
+ ## Where this fits in the Guide
413
+
414
+ Debugging is what you reach for when a [Run](/v7/copilot/running-tests) goes sideways or a [Validate](/v7/making-assertions) assertion fails — the screenshots show you precisely what the AI saw before it acted. Once you've diagnosed the failure, the next step is to stop it from recurring.
415
+
412
416
  ## Related
413
417
 
414
418
  - [screenshot()](/v7/screenshot) - Capture screenshots during test execution
415
419
  - [Dashcam](/v7/dashcam) - Record full test sessions with video and logs
416
420
  - [assert()](/v7/assert) - Make AI-powered assertions that benefit from screenshot context
421
+
422
+ ## Next
423
+
424
+ <Card title="Prevent" icon="shield-check" href="/v7/copilot/auto-healing">
425
+ You've seen what went wrong — now keep it from happening again. Let auto-healing repair flaky tests automatically before they fail your suite.
426
+ </Card>
@@ -1,12 +1,21 @@
1
1
  ---
2
- title: "Generating Tests"
3
- description: "Use AI coding agents and exploration mode to generate TestDriver tests"
4
- icon: "wand-magic-sparkles"
2
+ title: "Explore"
3
+ sidebarTitle: "Explore"
4
+ description: "Generate tests by exploring your app with the AI vision agent"
5
+ icon: "compass"
5
6
  ---
6
7
 
8
+ Start by exploring your app. This is where tests begin: you describe a flow in plain English, and the TestDriver vision agent clicks, types, and reads the screen to figure it out — then writes the test for you. No selectors, no DOM, no brittle locators. Just describe what you want to test and let the agent discover the rest.
9
+
10
+ There are two ways to explore: chatting interactively with your AI assistant through the TestDriver MCP server, or handing a coding agent our instructions file and prompting it to generate a test.
11
+
12
+ <Info>
13
+ Both paths need an MCP-capable AI coding assistant. If you don't have one yet, start with **GitHub Copilot** — there's a [free tier](https://github.com/features/copilot/plans), no credit card required. See [Run → Setup](/v7/copilot/running-tests#setup) for the full install and sign-in walkthrough.
14
+ </Info>
15
+
7
16
  ## Instructions for Coding Agents
8
17
 
9
- We recommend starting with [our quickstart](./quickstart) then supplying your coding agent with our agent instructions file.
18
+ We recommend starting with [our quickstart](./quickstart), then supplying your coding agent with our agent instructions file.
10
19
 
11
20
  <Card title="TestDriver Agent Instructions" icon="link" arrow="true" horizontal href="https://github.com/testdriverai/testdriverai/blob/main/ai/agents/testdriver.md?plain=1">
12
21
  Copy the current version of our agent instructions to provide your coding agent with up-to-date instructions on how to generate TestDriver tests.
@@ -24,13 +33,160 @@ Push Submit button
24
33
  Verify new page contains expected text 'logged in'
25
34
  ```
26
35
 
27
- ## AI Exploration Mode
36
+ <Info>Explicit commands are preferred for production tests, as they are cheaper, faster, and more reliable.</Info>
37
+
38
+ ## Start a Conversation
39
+
40
+ With TestDriver's MCP server and your AI assistant (GitHub Copilot, Cursor, or Claude Desktop), you can create tests by chatting with an AI agent. The agent spawns a virtual machine, executes actions, and writes test code for you.
41
+
42
+ Open your AI assistant's chat. If your project has no other agents configured, the TestDriver agent is used by default. Otherwise, select **testdriver** from the agent dropdown in the chat panel.
28
43
 
29
- Within a test, the `ai()` method lets TestDriver autonomously figure out how to accomplish a task. It's useful for dynamic or unpredictable UIs where explicit actions may be difficult to define.
44
+ Describe what you want to test:
30
45
 
31
- ```javascript
32
- // Handle dynamic or unpredictable UI
33
- await testdriver.ai('dismiss any popups or modals that appear');
46
+ ```
47
+ Create a test that logs into my app at https://myapp.com
34
48
  ```
35
49
 
36
- <Info>Explicit commands are preferred for production tests, as they are cheaper, faster, and more reliable.</Info>
50
+ The agent will:
51
+ 1. Start a new session and spawn a Linux virtual machine
52
+ 2. Launch Chrome and navigate to your URL
53
+ 3. Execute actions based on your instructions
54
+ 4. Write the test code to a `.test.mjs` file
55
+
56
+ <Note>
57
+ The TestDriver agent appears in the agent selection dropdown if you have other agents configured (like `copilot-instructions.md` or other `.agent.md` files). Make sure to select **testdriver** to use TestDriver's MCP tools.
58
+ </Note>
59
+
60
+ ## Live Preview Panel
61
+
62
+ When the agent starts a session, a **live preview panel** opens in your editor. This lets you:
63
+
64
+ - **Watch tests execute** in real-time
65
+ - **Interact with the VM** — click, type, and navigate manually
66
+ - **Debug issues** — see exactly what the AI sees
67
+
68
+ <Note>
69
+ The live preview panel requires the TestDriver VS Code extension. Set `TD_PREVIEW=ide` in your MCP configuration to enable it. See the [Run page](/v7/copilot/running-tests) for all device and preview options.
70
+ </Note>
71
+
72
+ ## Interactive Workflow
73
+
74
+ The recommended workflow for exploring and creating tests:
75
+
76
+ <Steps>
77
+ <Step title="Describe Your Test">
78
+ Tell the agent what you want to test:
79
+
80
+ ```
81
+ Test the checkout flow on https://shop.example.com
82
+ ```
83
+ </Step>
84
+
85
+ <Step title="Watch and Guide">
86
+ The agent will start executing. Watch the preview panel and provide guidance if needed:
87
+
88
+ ```
89
+ The login modal appeared. Use test@example.com and password123
90
+ ```
91
+ </Step>
92
+
93
+ <Step title="Add Assertions">
94
+ Ask the agent to verify specific conditions:
95
+
96
+ ```
97
+ Assert that the order confirmation page is shown with the order number
98
+ ```
99
+ </Step>
100
+
101
+ <Step title="Review the Code">
102
+ The agent writes test code as it works. Review the generated `.test.mjs` file and make any adjustments.
103
+ </Step>
104
+ </Steps>
105
+
106
+ ## Example Conversation
107
+
108
+ Here's a complete example of exploring and creating a login test:
109
+
110
+ ```
111
+ You: Create a test for the login flow at https://console.testdriver.ai
112
+
113
+ Agent: I'll create a test for the login flow. Starting a session...
114
+ [Screenshot: Login page loaded]
115
+
116
+ I can see the login page. Let me find the email input.
117
+ [Screenshot: Email input highlighted]
118
+
119
+ Now I'll type the email and find the password field...
120
+
121
+ You: Use test@example.com for the email
122
+
123
+ Agent: Got it. Typing test@example.com...
124
+ [Screenshot: Email entered]
125
+
126
+ Now entering the password and clicking Sign In...
127
+ [Screenshot: Dashboard loaded]
128
+
129
+ ✅ Login successful! I've written the test to tests/login.test.mjs
130
+ ```
131
+
132
+ ## Generated Test Code
133
+
134
+ The agent generates standard Vitest test files:
135
+
136
+ ```javascript tests/login.test.mjs
137
+ import { describe, expect, it } from "vitest";
138
+ import { TestDriver } from "testdriverai/vitest/hooks";
139
+
140
+ describe("Login Flow", () => {
141
+ it("should log in successfully", async (context) => {
142
+ const testdriver = TestDriver(context);
143
+
144
+ await testdriver.provision.chrome({
145
+ url: "https://console.testdriver.ai"
146
+ });
147
+
148
+ const emailInput = await testdriver.find("email input field");
149
+ await emailInput.click();
150
+ await testdriver.type("test@example.com");
151
+
152
+ const passwordInput = await testdriver.find("password input field");
153
+ await passwordInput.click();
154
+ await testdriver.type("password123");
155
+
156
+ const signInButton = await testdriver.find("Sign In button");
157
+ await signInButton.click();
158
+
159
+ const result = await testdriver.assert("dashboard is visible");
160
+ expect(result).toBeTruthy();
161
+ });
162
+ });
163
+ ```
164
+
165
+ ## Tips for Better Tests
166
+
167
+ <AccordionGroup>
168
+ <Accordion title="Be specific with element descriptions">
169
+ Instead of "click the button", say "click the blue Sign In button in the header". More context helps the AI find the right element.
170
+ </Accordion>
171
+ <Accordion title="Add waits for dynamic content">
172
+ If your app has animations or loading states, tell the agent to wait:
173
+ ```
174
+ Wait for the loading spinner to disappear before continuing
175
+ ```
176
+ </Accordion>
177
+ <Accordion title="Use assertions liberally">
178
+ Add assertions after each major action to catch regressions early:
179
+ ```
180
+ Assert that the product was added to the cart
181
+ ```
182
+ </Accordion>
183
+ <Accordion title="Break complex flows into steps">
184
+ For long workflows, create the test incrementally and verify each step works before moving on.
185
+ </Accordion>
186
+ </AccordionGroup>
187
+
188
+ ## Next
189
+
190
+ <Card title="Learn" icon="brain" arrow="true" horizontal href="/v7/caching">
191
+ Once the agent has explored your app, TestDriver caches what it discovers so your tests replay instantly without re-reasoning over the screen every time.
192
+ </Card>
@@ -1,12 +1,83 @@
1
1
  ---
2
- title: "Making Assertions"
3
- description: "Verify application state with AI-powered assertions"
4
- icon: "check-double"
2
+ title: "Validate"
3
+ sidebarTitle: "Validate"
4
+ description: "Locate elements and verify app state with AI-powered assertions"
5
+ icon: "circle-check"
5
6
  ---
6
7
 
8
+ Once a test runs, validate that the app did what it should. Validation has two parts: locating the elements you want to check, and making assertions about the state of your app. TestDriver uses AI as a judge, returning a boolean plus reasoning about whether your app is in the expected state.
9
+
10
+ ## Locating Elements
11
+
12
+ ### Locating Single Elements
13
+
14
+ Use natural language to describe elements. Descriptions should be specific enough to locate the element, but not too-specific that they break with minor UI changes. For example:
15
+
16
+ ```javascript
17
+ await testdriver.find('email input field');
18
+ await testdriver.find('first product card in the grid');
19
+ await testdriver.find('dropdown menu labeled "Country"');
20
+ ```
21
+
22
+ <Info>TestDriver will cache found elements for improved performance on subsequent calls. Learn more about [element caching here](/v7/caching).</Info>
23
+
24
+ ### Debugging Found Elements
25
+
26
+ After finding an element, you can inspect its properties for debugging:
27
+
28
+ ```javascript
29
+ const button = await testdriver.find('submit button');
30
+ console.log(button);
31
+ ```
32
+
33
+ This outputs all element properties:
34
+
35
+ ```javascript
36
+ {
37
+ description: 'submit button',
38
+ found: true,
39
+ x: 150,
40
+ y: 300,
41
+ coordinates: { x: 150, y: 300, centerX: 200, centerY: 320 },
42
+ threshold: 0.8,
43
+ confidence: 0.95,
44
+ similarity: 0.92,
45
+ selector: 'button[type="submit"]',
46
+ cache: {
47
+ hit: true,
48
+ strategy: 'pixel-diff',
49
+ createdAt: '2025-01-15T10:30:00Z',
50
+ diffPercent: 0.02,
51
+ imageUrl: 'https://...'
52
+ }
53
+ }
54
+ ```
55
+
56
+ ### Working with Multiple Elements
57
+
58
+ Find and interact with multiple elements:
59
+
60
+ ```javascript
61
+ // Find all matching elements
62
+ const products = await testdriver.findAll('product card');
63
+ console.log(`Found ${products.length} products`);
64
+
65
+ // Interact with each
66
+ for (const product of products) {
67
+ const title = await product.find('title text');
68
+ console.log('Product:', title.text);
69
+
70
+ await product.find('add to cart button').click();
71
+ }
72
+
73
+ // Or find specific element
74
+ const firstProduct = products[0];
75
+ await firstProduct.click();
76
+ ```
77
+
7
78
  ## Making Assertions
8
79
 
9
- Use AI-powered assertions to verify application state:
80
+ Use AI-powered assertions to verify application state. TestDriver acts as a judge: it evaluates your natural-language statement against the current state of the app and returns a boolean plus reasoning explaining the verdict.
10
81
 
11
82
  ```javascript
12
83
  // Verify visibility
@@ -30,3 +101,9 @@ await testdriver.assert('form has red border');
30
101
  ```
31
102
 
32
103
  <Info>Assertions are not cached and always re-evaluated to ensure accuracy.</Info>
104
+
105
+ ## Next
106
+
107
+ <Card title="Adapt" icon="arrows-rotate" href="/v7/performing-actions">
108
+ Drive your app forward by performing actions on the elements you've located and validated.
109
+ </Card>
@@ -1,9 +1,14 @@
1
1
  ---
2
- title: "Performing Actions"
3
- description: "Click, type, hover, scroll and more with TestDriver"
4
- icon: "computer-mouse"
2
+ title: "Adapt"
3
+ sidebarTitle: "Adapt"
4
+ description: "Perform actions and handle dynamic, async UI so tests adapt to change"
5
+ icon: "arrows-rotate"
5
6
  ---
6
7
 
8
+ Real apps move, load, and change. Adapt your tests to handle it.
9
+
10
+ Once you've [generated](/v7/generating-tests) and [learned](/v7/caching) your tests and gotten them [running](/v7/copilot/running-tests), the next challenge is the real world: buttons appear after a spinner, pages navigate, animations play, and content streams in over the network. To keep tests reliable, you need to perform the right actions and handle timing so your tests adapt to how the UI actually behaves instead of breaking.
11
+
7
12
  ## Performing Actions
8
13
 
9
14
  TestDriver provides a variety of actions you can perform, like [clicking](/v7/click), [typing](/v7/type), [hovering](/v7/hover), and [scrolling](/v7/scroll). For a full list, see the [API Reference](/v7/click).
@@ -36,7 +41,7 @@ const price = await testdriver.extract('the total price');
36
41
  const orderNumber = await testdriver.extract('the order confirmation number');
37
42
  ```
38
43
 
39
- ## Chaining Actions
44
+ ### Chaining Actions
40
45
 
41
46
  TestDriver supports method chaining for cleaner code:
42
47
 
@@ -51,3 +56,94 @@ Or save element reference for later use:
51
56
  const button = await testdriver.find('submit button');
52
57
  await button.click();
53
58
  ```
59
+
60
+ ## Waiting for Dynamic Content
61
+
62
+ By default, `find()` automatically polls for up to 10 seconds, retrying every 5 seconds until the element is found. This means most elements that appear after short async operations will be found without any extra configuration.
63
+
64
+ For longer operations, increase the `timeout`:
65
+
66
+ ```javascript
67
+ // Default behavior - polls for up to 10 seconds automatically
68
+ const element = await testdriver.find('Loading complete indicator');
69
+ await element.click();
70
+
71
+ // Wait up to 30 seconds for slower operations
72
+ const element = await testdriver.find('Loading complete indicator', { timeout: 30000 });
73
+ await element.click();
74
+
75
+ // Useful after actions that trigger loading states
76
+ await testdriver.find('submit button').click();
77
+ await testdriver.find('success message', { timeout: 15000 });
78
+
79
+ // Disable polling for instant checks
80
+ const toast = await testdriver.find('notification toast', { timeout: 0 });
81
+ ```
82
+
83
+ ### Flake Prevention
84
+
85
+ TestDriver automatically waits for the screen and network to stabilize after each action using **redraw detection**. This prevents flaky tests caused by animations, loading states, or dynamic content updates.
86
+
87
+ <Note>
88
+ Redraw detection adds a small delay after each action but significantly reduces test flakiness.
89
+ </Note>
90
+
91
+ For example, when clicking a submit button that navigates to a new page:
92
+
93
+ ```javascript
94
+ // Click submit - TestDriver automatically waits for the new page to load
95
+ await testdriver.find('submit button').click();
96
+
97
+ // By the time this runs, the page has fully loaded and stabilized
98
+ await testdriver.assert('dashboard is displayed');
99
+ await testdriver.find('welcome message');
100
+ ```
101
+
102
+ Without redraw detection, you'd need manual waits or retries to handle the page transition. TestDriver handles this automatically by detecting when the screen stops changing and network requests complete.
103
+
104
+ You can disable redraw detection or customize its behavior:
105
+
106
+ ```javascript
107
+ // Disable redraw detection for faster tests (less reliable)
108
+ const testdriver = TestDriver(context, {
109
+ redraw: false
110
+ });
111
+ ```
112
+
113
+ Here is an example of customizing redraw detection:
114
+
115
+ ```javascript
116
+ // Fine-tune redraw detection
117
+ const testdriver = TestDriver(context, {
118
+ redraw: {
119
+ enabled: true,
120
+ diffThreshold: 0.1, // Pixel difference threshold (0-1)
121
+ screenRedraw: true, // Monitor screen changes
122
+ networkMonitor: true, // Wait for network idle
123
+ }
124
+ });
125
+ ```
126
+
127
+ ### Simple Delays with `wait()`
128
+
129
+ For simple pauses — waiting for animations, transitions, or state changes after an action — use `wait()`:
130
+
131
+ ```javascript
132
+ // Wait for an animation to complete
133
+ await testdriver.find('menu toggle').click();
134
+ await testdriver.wait(2000);
135
+
136
+ // Wait for a page transition to settle
137
+ await testdriver.find('next page button').click();
138
+ await testdriver.wait(1000);
139
+ ```
140
+
141
+ <Note>
142
+ For waiting for specific **elements** to appear, prefer `find()` with a `timeout` option. Use `wait()` only for simple time-based pauses.
143
+ </Note>
144
+
145
+ Once your tests can reliably act on a changing UI and [assert](/v7/making-assertions) the results, the next step is figuring out what happened when something does go wrong.
146
+
147
+ <Card title="Next: Debug" icon="bug" href="/v7/debugging-with-screenshots">
148
+ Use screenshots and run output to see exactly what your test saw and pinpoint failures.
149
+ </Card>