@testdriverai/agent 7.11.239-test → 7.11.241-test
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ai/skills/testdriver-agent/SKILL.md +174 -91
- package/ai/skills/testdriver-assert/SKILL.md +73 -269
- package/ai/skills/testdriver-aws-setup/SKILL.md +194 -312
- package/ai/skills/testdriver-bring-your-own-machine/SKILL.md +137 -204
- package/ai/skills/testdriver-cache/SKILL.md +142 -149
- package/ai/skills/testdriver-caching/SKILL.md +97 -87
- package/ai/skills/testdriver-captcha/SKILL.md +126 -111
- package/ai/skills/testdriver-ci-cd/SKILL.md +153 -542
- package/ai/skills/testdriver-click/SKILL.md +74 -229
- package/ai/skills/testdriver-client/SKILL.md +175 -229
- package/ai/skills/testdriver-customizing-devices/SKILL.md +72 -250
- package/ai/skills/testdriver-dashcam/SKILL.md +71 -404
- package/ai/skills/testdriver-debugging-with-screenshots/SKILL.md +103 -369
- package/ai/skills/testdriver-double-click/SKILL.md +46 -67
- package/ai/skills/testdriver-errors/SKILL.md +102 -176
- package/ai/skills/testdriver-events/SKILL.md +170 -327
- package/ai/skills/testdriver-exec/SKILL.md +92 -256
- package/ai/skills/testdriver-extract/SKILL.md +69 -170
- package/ai/skills/testdriver-find/SKILL.md +140 -736
- package/ai/skills/testdriver-focus-application/SKILL.md +69 -246
- package/ai/skills/testdriver-generating-tests/SKILL.md +102 -136
- package/ai/skills/testdriver-hosted/SKILL.md +79 -143
- package/ai/skills/testdriver-hover/SKILL.md +62 -235
- package/ai/skills/testdriver-locating-elements/SKILL.md +71 -366
- package/ai/skills/testdriver-making-assertions/SKILL.md +114 -72
- package/ai/skills/testdriver-mcp-setup/SKILL.md +79 -62
- package/ai/skills/testdriver-mouse-down/SKILL.md +46 -124
- package/ai/skills/testdriver-mouse-up/SKILL.md +45 -128
- package/ai/skills/testdriver-options/SKILL.md +73 -46
- package/ai/skills/testdriver-parse/SKILL.md +95 -184
- package/ai/skills/testdriver-performing-actions/SKILL.md +101 -91
- package/ai/skills/testdriver-press-keys/SKILL.md +59 -299
- package/ai/skills/testdriver-provision/SKILL.md +135 -179
- package/ai/skills/testdriver-quickstart/SKILL.md +24 -8
- package/ai/skills/testdriver-quickstart-claude/SKILL.md +84 -0
- package/ai/skills/testdriver-quickstart-github/SKILL.md +62 -32
- package/ai/skills/testdriver-quickstart-manual/SKILL.md +59 -32
- package/ai/skills/testdriver-quickstart-vscode/SKILL.md +96 -0
- package/ai/skills/testdriver-redraw/SKILL.md +100 -164
- package/ai/skills/testdriver-reusable-code/SKILL.md +95 -199
- package/ai/skills/testdriver-right-click/SKILL.md +45 -88
- package/ai/skills/testdriver-screenshot/SKILL.md +47 -201
- package/ai/skills/testdriver-screenshots/SKILL.md +86 -136
- package/ai/skills/testdriver-scroll/SKILL.md +71 -154
- package/ai/skills/testdriver-secrets/SKILL.md +94 -90
- package/ai/skills/testdriver-self-hosted/SKILL.md +82 -140
- package/ai/skills/testdriver-test-results-json/SKILL.md +165 -202
- package/ai/skills/testdriver-type/SKILL.md +56 -304
- package/ai/skills/testdriver-variables/SKILL.md +119 -80
- package/ai/skills/testdriver-wait/SKILL.md +31 -29
- package/ai/skills/testdriver-what-is-testdriver/SKILL.md +80 -32
- package/package.json +1 -1
|
@@ -1,53 +1,83 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: testdriver:quickstart-github
|
|
3
|
-
description:
|
|
3
|
+
description: Get an AI code review on every pull request, backed by a real run of your app.
|
|
4
4
|
---
|
|
5
5
|
<!-- Generated from quickstart-github.mdx. DO NOT EDIT. -->
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Install the TestDriver GitHub app, and the TestDriver agent reviews every pull request in your repository.
|
|
8
8
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
9
|
+
- It reads the diff for bugs.
|
|
10
|
+
- When a user would see the change, it runs your app in a real sandbox to prove what it finds.
|
|
11
|
+
- It writes and maintains end-to-end tests whenever you ask.
|
|
12
|
+
|
|
13
|
+
There's nothing to install locally.
|
|
14
|
+
|
|
15
|
+
## Before you begin
|
|
16
|
+
|
|
17
|
+
- **A GitHub repository** where you can install apps (you own it, or you're an admin of its organization).
|
|
18
|
+
- **A TestDriver account.** You create one or sign in during installation.
|
|
19
|
+
- **A paid plan or trial for automatic reviews.** On the free plan, the agent still reviews a pull request when you @mention it. See [pricing](https://testdriver.ai/pricing).
|
|
18
20
|
|
|
19
21
|
<Steps>
|
|
20
|
-
<Step title="Install the GitHub
|
|
21
|
-
|
|
22
|
+
<Step title="Install the GitHub app">
|
|
23
|
+
|
|
24
|
+
Open [TestDriver on GitHub](https://go.testdriver.ai/github). Install the app on the repositories you want reviewed, and sign in to TestDriver when prompted.
|
|
25
|
+
|
|
26
|
+
You can add more repositories later from your GitHub settings.
|
|
27
|
+
|
|
28
|
+
</Step>
|
|
29
|
+
<Step title="Get your first tests">
|
|
30
|
+
|
|
31
|
+
Choose a repository to set up. TestDriver opens a **Set up TestDriver** issue in it.
|
|
32
|
+
|
|
33
|
+
The agent picks up the issue, writes a few sample tests for your app, and runs them. It opens a pull request with the tests and their recordings. Review it and merge it like any other pull request.
|
|
34
|
+
|
|
22
35
|
</Step>
|
|
36
|
+
<Step title="Open a pull request">
|
|
23
37
|
|
|
24
|
-
|
|
25
|
-
|
|
38
|
+
From now on, every pull request you open gets a review without anyone asking. The agent reads the change and runs it when there's something to drive.
|
|
39
|
+
|
|
40
|
+
It posts a single review comment with:
|
|
41
|
+
|
|
42
|
+
- A summary
|
|
43
|
+
- A walkthrough of the changed files
|
|
44
|
+
- Any bugs it found, each labelled `observed` (seen during a run) or `suspected` (inferred from the code)
|
|
45
|
+
|
|
46
|
+
Pushing new commits triggers a fresh review.
|
|
47
|
+
|
|
48
|
+
The review is a comment, so it never approves or blocks a merge. To fail a pull request when a test breaks, [run your tests in CI](/ci-cd).
|
|
49
|
+
|
|
50
|
+
</Step>
|
|
51
|
+
<Step title="Ask for more with @testdriverai">
|
|
52
|
+
|
|
53
|
+
Mention the agent on any issue or pull request to ask a follow-up question or request a new test:
|
|
26
54
|
|
|
27
55
|
```text
|
|
28
|
-
@testdriverai Write a test that
|
|
56
|
+
@testdriverai Write a test that logs in, adds the first product to the cart, and confirms the cart badge shows 1.
|
|
29
57
|
```
|
|
30
58
|
|
|
31
|
-
|
|
59
|
+
The agent replies in the thread and posts screenshots as it works. When it's done, it opens a pull request with the test.
|
|
32
60
|
|
|
33
61
|
<Tip>
|
|
34
|
-
|
|
62
|
+
Describe what a user does and what they should see, not how the page is built. "Confirm the cart badge shows 1" works better than "check `#cart-count`".
|
|
35
63
|
</Tip>
|
|
36
|
-
</Step>
|
|
37
64
|
|
|
38
|
-
<Step title="Catch regressions automatically">
|
|
39
|
-
Once tests are committed, TestDriver runs them on every pull request that changes something a user would see and posts a review with what it observed, including a recording. Mention `@testdriverai` on any PR to ask follow-up questions or request more tests.
|
|
40
65
|
</Step>
|
|
41
66
|
</Steps>
|
|
42
67
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
<
|
|
46
|
-
title="
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
>
|
|
52
|
-
|
|
53
|
-
|
|
68
|
+
## Next steps
|
|
69
|
+
|
|
70
|
+
<CardGroup cols={2}>
|
|
71
|
+
<Card title="How the agent works" icon="hand-horns" href="/agent">
|
|
72
|
+
What the agent does on pull requests, issues, and mentions.
|
|
73
|
+
</Card>
|
|
74
|
+
<Card title="Prevent regressions" icon="shield-check" href="/copilot/auto-healing">
|
|
75
|
+
Run tests in CI and have the agent fix them when your app changes.
|
|
76
|
+
</Card>
|
|
77
|
+
<Card title="Run tests in CI/CD" icon="code-branch" href="/ci-cd">
|
|
78
|
+
Run your TestDriver tests in GitHub Actions.
|
|
79
|
+
</Card>
|
|
80
|
+
<Card title="Set up a project" icon="wrench" href="/quickstart-manual">
|
|
81
|
+
Write and run tests yourself in your own repository.
|
|
82
|
+
</Card>
|
|
83
|
+
</CardGroup>
|
|
@@ -1,27 +1,48 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: testdriver:quickstart-manual
|
|
3
|
-
description: Add TestDriver to
|
|
3
|
+
description: Add TestDriver to your own project and run your first Vitest test.
|
|
4
4
|
---
|
|
5
5
|
<!-- Generated from quickstart-manual.mdx. DO NOT EDIT. -->
|
|
6
6
|
|
|
7
|
-
Add TestDriver to
|
|
7
|
+
Add TestDriver to a new or existing project, then run your first test.
|
|
8
|
+
|
|
9
|
+
- **Fastest:** run `npx testdriverai init` and let it create every file for you.
|
|
10
|
+
- **By hand:** follow the steps below to see exactly what each file does.
|
|
8
11
|
|
|
9
12
|
<Tip>
|
|
10
|
-
|
|
13
|
+
Only want to drive TestDriver from chat? [MCP setup](/mcp-setup) needs one URL and nothing installed.
|
|
11
14
|
</Tip>
|
|
12
15
|
|
|
16
|
+
## Set up with init
|
|
17
|
+
|
|
18
|
+
Run `init` in your project folder:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
npx testdriverai init
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
It signs you in and saves your API key to `.env`. It then adds:
|
|
25
|
+
|
|
26
|
+
- A `package.json`, if you don't have one
|
|
27
|
+
- `vitest.config.js` with the TestDriver reporter
|
|
28
|
+
- An example test in `tests/`
|
|
29
|
+
- A GitHub Actions workflow
|
|
30
|
+
- The TestDriver agent and MCP server for your AI client
|
|
31
|
+
|
|
32
|
+
`init` keeps your existing `package.json`, Vitest config, tests, and workflow. It only adds to `.gitignore` and to MCP config files.
|
|
33
|
+
|
|
34
|
+
Then run the example:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
npx vitest run
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
## Set up by hand
|
|
41
|
+
|
|
13
42
|
<Steps>
|
|
14
|
-
<Step title="
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
title="Get an API Key"
|
|
18
|
-
icon="user-plus"
|
|
19
|
-
href="https://console.testdriver.ai/settings"
|
|
20
|
-
arrow
|
|
21
|
-
horizontal
|
|
22
|
-
>
|
|
23
|
-
Start with 60 free device minutes, no credit card required.
|
|
24
|
-
</Card>
|
|
43
|
+
<Step title="Add your API key">
|
|
44
|
+
|
|
45
|
+
Create an API key at [console.testdriver.ai/team/api-keys](https://console.testdriver.ai/team/api-keys). The free plan includes 30 testing minutes and needs no credit card.
|
|
25
46
|
|
|
26
47
|
Save the key in a `.env` file at your project root. The SDK loads it automatically:
|
|
27
48
|
|
|
@@ -29,7 +50,7 @@ If you only want to drive TestDriver from chat, [MCP Setup](/mcp-setup) is a sin
|
|
|
29
50
|
TD_API_KEY=your_api_key
|
|
30
51
|
```
|
|
31
52
|
|
|
32
|
-
Add `.env` to `.gitignore` so the key
|
|
53
|
+
Add `.env` to `.gitignore` so the key isn't committed.
|
|
33
54
|
|
|
34
55
|
</Step>
|
|
35
56
|
<Step title="Install dependencies">
|
|
@@ -40,14 +61,12 @@ If you only want to drive TestDriver from chat, [MCP Setup](/mcp-setup) is a sin
|
|
|
40
61
|
npm install --save-dev vitest testdriverai
|
|
41
62
|
```
|
|
42
63
|
|
|
43
|
-
|
|
44
|
-
TestDriver requires Node.js 20.19 or later and only runs on [Vitest](https://vitest.dev). Jest, Mocha, and other runners are not supported.
|
|
45
|
-
</Note>
|
|
64
|
+
TestDriver needs Node.js 20.19 or later. It only runs on [Vitest](https://vitest.dev), not Jest, Mocha, or other runners.
|
|
46
65
|
|
|
47
66
|
</Step>
|
|
48
67
|
<Step title="Configure Vitest">
|
|
49
68
|
|
|
50
|
-
Create `vitest.config.js` at your project root
|
|
69
|
+
Create `vitest.config.js` at your project root, or add these settings to your existing config:
|
|
51
70
|
|
|
52
71
|
```js vitest.config.js
|
|
53
72
|
import { defineConfig } from 'vitest/config';
|
|
@@ -64,9 +83,9 @@ If you only want to drive TestDriver from chat, [MCP Setup](/mcp-setup) is a sin
|
|
|
64
83
|
});
|
|
65
84
|
```
|
|
66
85
|
|
|
67
|
-
-
|
|
68
|
-
-
|
|
69
|
-
-
|
|
86
|
+
- **`TestDriver()`** is the reporter. It records each run in the TestDriver console, with its recording.
|
|
87
|
+
- **`setupFiles`** links your tests to the reporter, so each test's recording attaches to the run.
|
|
88
|
+
- **The timeouts** must be long. Vitest's defaults (5 seconds for tests, 10 for hooks) are too short for a real machine.
|
|
70
89
|
|
|
71
90
|
</Step>
|
|
72
91
|
<Step title="Write a test">
|
|
@@ -98,37 +117,45 @@ If you only want to drive TestDriver from chat, [MCP Setup](/mcp-setup) is a sin
|
|
|
98
117
|
});
|
|
99
118
|
```
|
|
100
119
|
|
|
101
|
-
If your
|
|
120
|
+
If your `package.json` doesn't set `"type": "module"`, name the file `search.test.mjs` instead.
|
|
102
121
|
|
|
103
122
|
</Step>
|
|
104
123
|
<Step title="Run the test">
|
|
105
124
|
|
|
106
125
|
```bash
|
|
107
|
-
vitest run
|
|
126
|
+
npx vitest run
|
|
108
127
|
```
|
|
109
128
|
|
|
110
|
-
A sandbox starts, Chrome opens, and a live preview appears in your browser.
|
|
129
|
+
A sandbox starts, Chrome opens, and a live preview appears in your browser.
|
|
130
|
+
|
|
131
|
+
When the run finishes, open the `TESTDRIVER_RUN_URL` link at the end of the output. It shows the recording and each step.
|
|
111
132
|
|
|
112
133
|
</Step>
|
|
113
134
|
</Steps>
|
|
114
135
|
|
|
115
|
-
##
|
|
136
|
+
## Connect an AI assistant
|
|
116
137
|
|
|
117
|
-
|
|
138
|
+
To write tests with an AI assistant, add the TestDriver agent and MCP server to your project. `--no-sample-test` skips the example test files:
|
|
118
139
|
|
|
119
140
|
```bash
|
|
120
141
|
npx testdriverai init --client cursor,claude-code,vscode --no-sample-test
|
|
121
142
|
```
|
|
122
143
|
|
|
123
|
-
|
|
144
|
+
Your existing config and tests stay as they are. For Claude Code or VS Code, you can use the [plugin](/quickstart-claude) or [extension](/quickstart-vscode) instead.
|
|
124
145
|
|
|
125
146
|
## Next steps
|
|
126
147
|
|
|
127
148
|
<CardGroup cols={2}>
|
|
128
|
-
<Card title="
|
|
129
|
-
|
|
149
|
+
<Card title="Launch apps" icon="rocket" href="/provision">
|
|
150
|
+
Start Chrome, extensions, VS Code, or desktop apps.
|
|
151
|
+
</Card>
|
|
152
|
+
<Card title="Perform actions" icon="arrow-pointer" href="/performing-actions">
|
|
153
|
+
Click, type, scroll, and wait for a changing UI.
|
|
154
|
+
</Card>
|
|
155
|
+
<Card title="Make assertions" icon="circle-check" href="/making-assertions">
|
|
156
|
+
Check what's on screen in plain language.
|
|
130
157
|
</Card>
|
|
131
|
-
<Card title="CI/CD" icon="
|
|
132
|
-
Run tests on every pull request
|
|
158
|
+
<Card title="Run in CI/CD" icon="code-branch" href="/ci-cd">
|
|
159
|
+
Run your tests on every pull request.
|
|
133
160
|
</Card>
|
|
134
161
|
</CardGroup>
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: testdriver:quickstart-vscode
|
|
3
|
+
description: Set up TestDriver in your editor and watch tests run in a live preview panel.
|
|
4
|
+
---
|
|
5
|
+
<!-- Generated from quickstart-vscode.mdx. DO NOT EDIT. -->
|
|
6
|
+
|
|
7
|
+
Install the TestDriver extension, and your editor's AI assistant can write and run end-to-end tests. You watch the sandbox in a panel beside your code.
|
|
8
|
+
|
|
9
|
+
The extension:
|
|
10
|
+
|
|
11
|
+
- Signs you in.
|
|
12
|
+
- Sets up your project and MCP server.
|
|
13
|
+
- Opens the live preview automatically whenever a test runs.
|
|
14
|
+
|
|
15
|
+
## Before you begin
|
|
16
|
+
|
|
17
|
+
- **VS Code 1.85 or later with GitHub Copilot**, or **Cursor**.
|
|
18
|
+
- **Node.js 20.19 or later**, which runs the MCP server and your tests.
|
|
19
|
+
- **A workspace folder open** in the editor, since the extension sets up TestDriver in it.
|
|
20
|
+
|
|
21
|
+
<Steps>
|
|
22
|
+
<Step title="Install the extension">
|
|
23
|
+
|
|
24
|
+
Install [TestDriver.ai from the VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=testdriver.testdriver), or search for **TestDriver.ai** in the Extensions view.
|
|
25
|
+
|
|
26
|
+
In Cursor, the extension may not appear in search. If so, download it with **Download Extension** on the Marketplace page. Then run **Extensions: Install from VSIX** from the Command Palette and choose the file.
|
|
27
|
+
|
|
28
|
+
</Step>
|
|
29
|
+
<Step title="Sign in">
|
|
30
|
+
|
|
31
|
+
Run **TestDriver: Sign In** from the Command Palette. It opens your browser, signs you in to TestDriver, and saves your API key in the editor.
|
|
32
|
+
|
|
33
|
+
</Step>
|
|
34
|
+
<Step title="Set up your project">
|
|
35
|
+
|
|
36
|
+
Run **TestDriver: Initialize Project**. It writes your API key to `.env` and runs `npx testdriverai init`, which installs the SDK and adds:
|
|
37
|
+
|
|
38
|
+
- An example test
|
|
39
|
+
- The TestDriver agent and skills
|
|
40
|
+
- A Vitest config
|
|
41
|
+
- A GitHub Actions workflow
|
|
42
|
+
|
|
43
|
+
</Step>
|
|
44
|
+
<Step title="Connect the MCP server">
|
|
45
|
+
|
|
46
|
+
Run **TestDriver: Install MCP Server** and choose where to save it: `.vscode/mcp.json` for VS Code or `.cursor/mcp.json` for Cursor.
|
|
47
|
+
|
|
48
|
+
If it asks whether to overwrite an existing TestDriver entry, choose **Yes**. This version of the config turns on the live preview panel.
|
|
49
|
+
|
|
50
|
+
</Step>
|
|
51
|
+
<Step title="Write your first test">
|
|
52
|
+
|
|
53
|
+
In VS Code, run **TestDriver: Chat with Agent**. It opens Copilot chat with the TestDriver agent and a sample prompt.
|
|
54
|
+
|
|
55
|
+
In Cursor, open the chat and ask:
|
|
56
|
+
|
|
57
|
+
```text
|
|
58
|
+
Use TestDriver to write a test that logs in to https://testdriver-sandbox.vercel.app/login and adds an item to the cart.
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
The live preview panel opens as the agent starts the sandbox. The agent writes the test under `tests/` as it goes.
|
|
62
|
+
|
|
63
|
+
</Step>
|
|
64
|
+
</Steps>
|
|
65
|
+
|
|
66
|
+
## Extension commands
|
|
67
|
+
|
|
68
|
+
| Command | What it does |
|
|
69
|
+
| --- | --- |
|
|
70
|
+
| **TestDriver: Sign In** | Signs in through your browser and saves your API key |
|
|
71
|
+
| **TestDriver: Set API Key** | Saves an API key you paste in instead |
|
|
72
|
+
| **TestDriver: Initialize Project** | Runs `npx testdriverai init` in the workspace |
|
|
73
|
+
| **TestDriver: Install MCP Server** | Adds the TestDriver MCP server to `.vscode/mcp.json` or `.cursor/mcp.json` |
|
|
74
|
+
| **TestDriver: Chat with Agent** | Opens the TestDriver agent in Copilot chat (VS Code) |
|
|
75
|
+
| **TestDriver: Turn into Repeatable Test** | Asks Copilot to turn your session into a test with a GitHub Actions workflow |
|
|
76
|
+
| **TestDriver: Open Live Preview** / **Close Live Preview** | Opens or closes the sandbox preview panel |
|
|
77
|
+
| **TestDriver: Open Getting Started** | Opens the extension's setup walkthrough |
|
|
78
|
+
|
|
79
|
+
The preview panel opens automatically when a test starts. To turn that off, set `testdriverai.autoOpenPreview` to `false` in your settings.
|
|
80
|
+
|
|
81
|
+
## Next steps
|
|
82
|
+
|
|
83
|
+
<CardGroup cols={2}>
|
|
84
|
+
<Card title="Generate tests with AI" icon="compass" href="/generating-tests">
|
|
85
|
+
Prompt the agent and steer it while it works.
|
|
86
|
+
</Card>
|
|
87
|
+
<Card title="Run tests" icon="play" href="/copilot/running-tests">
|
|
88
|
+
Run tests from the editor or the terminal.
|
|
89
|
+
</Card>
|
|
90
|
+
<Card title="Make assertions" icon="circle-check" href="/making-assertions">
|
|
91
|
+
Check what's on screen in plain language.
|
|
92
|
+
</Card>
|
|
93
|
+
<Card title="Run in CI/CD" icon="code-branch" href="/ci-cd">
|
|
94
|
+
Run your tests on every pull request.
|
|
95
|
+
</Card>
|
|
96
|
+
</CardGroup>
|