testdriverai 5.7.2 → 5.7.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/agent.js +2 -2
- package/docs/action/pre-post-scripts.mdx +4 -0
- package/docs/apps/chrome-extensions.mdx +11 -10
- package/docs/apps/desktop-apps.mdx +16 -10
- package/docs/apps/mobile-apps.mdx +1 -2
- package/docs/apps/static-websites.mdx +9 -6
- package/docs/commands/remember.mdx +40 -0
- package/docs/docs.json +17 -2
- package/docs/getting-started/vscode.mdx +25 -0
- package/docs/guide/lifecycle-prerun.mdx +15 -7
- package/docs/overview/performance.mdx +52 -0
- package/docs/overview/quickstart.mdx +2 -49
- package/docs/scenarios/ai-chatbot.mdx +1 -0
- package/docs/scenarios/cookie-banner.mdx +12 -13
- package/docs/scenarios/file-upload.mdx +12 -12
- package/docs/scenarios/form-filling.mdx +14 -13
- package/docs/scenarios/log-in.mdx +1 -0
- package/docs/scenarios/pdf-generation.mdx +1 -0
- package/docs/scenarios/spell-check.mdx +1 -0
- package/docs/snippets/lifecycle-warning.mdx +1 -0
- package/docs/snippets/test-prereqs.mdx +4 -5
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -115,7 +115,7 @@ After this, we will type the URL and press Enter.
|
|
|
115
115
|
|
|
116
116
|
TestDriver will execute the commands found in the yml codeblocks of the response.
|
|
117
117
|
|
|
118
|
-
See the yml TestDriver generated? That's our own schema. You can learn more about it in the [reference](https://docs.testdriver.ai/
|
|
118
|
+
See the yml TestDriver generated? That's our own schema. You can learn more about it in the [reference](https://docs.testdriver.ai/getting-started/editing).
|
|
119
119
|
|
|
120
120
|
> Take your hands off the mouse and keyboard while TestDriver executes! TestDriver is not a fan of backseat drivers.
|
|
121
121
|
|
|
@@ -152,6 +152,6 @@ git commit -am "Add TestDriver tests"
|
|
|
152
152
|
gh pr create --web
|
|
153
153
|
```
|
|
154
154
|
|
|
155
|
-
Your test will run on every commit and the results will be posted as a Dashcam.io video within your GitHub summary! Learn more about deploying on CI [here](https://docs.testdriver.ai/
|
|
155
|
+
Your test will run on every commit and the results will be posted as a Dashcam.io video within your GitHub summary! Learn more about deploying on CI [here](https://docs.testdriver.ai/action/setup).
|
|
156
156
|
|
|
157
157
|
# vscode
|
package/agent.js
CHANGED
|
@@ -1152,7 +1152,7 @@ const iffy = async (condition, then, otherwise, depth) => {
|
|
|
1152
1152
|
|
|
1153
1153
|
logger.info(generator.jsonToManual({ command: "if", condition }));
|
|
1154
1154
|
|
|
1155
|
-
let response = await commands.assert(condition);
|
|
1155
|
+
let response = await commands.assert(condition, false);
|
|
1156
1156
|
|
|
1157
1157
|
depth = depth + 1;
|
|
1158
1158
|
|
|
@@ -1265,7 +1265,7 @@ const start = async () => {
|
|
|
1265
1265
|
),
|
|
1266
1266
|
);
|
|
1267
1267
|
logger.info(
|
|
1268
|
-
chalk.dim("Read More: https://docs.testdriver.ai/security
|
|
1268
|
+
chalk.dim("Read More: https://docs.testdriver.ai/security/agent"),
|
|
1269
1269
|
);
|
|
1270
1270
|
logger.info("");
|
|
1271
1271
|
}
|
|
@@ -5,6 +5,8 @@ description: "How to handle pre and post test tasks."
|
|
|
5
5
|
icon: "hammer"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
+
import LifecycleWarning from '/snippets/lifecycle-warning.mdx'
|
|
9
|
+
|
|
8
10
|
## Overview
|
|
9
11
|
Prerun scripts are commands executed on a TestDriver virtual machine (VM) before each test in a CI/CD pipeline. They are used to prepare the environment by provisioning the VM, installing dependencies, configuring settings, or building the application. This ensures a consistent and reproducible environment for every test execution.
|
|
10
12
|
|
|
@@ -22,6 +24,8 @@ Prerun scripts are ideal for:
|
|
|
22
24
|
- Configuring the VM to match specific test requirements.
|
|
23
25
|
- Preparing staging environments or accessing private resources.
|
|
24
26
|
|
|
27
|
+
<LifecycleWarning/>
|
|
28
|
+
|
|
25
29
|
---
|
|
26
30
|
|
|
27
31
|
## Example: Installing Arc Browser
|
|
@@ -7,6 +7,16 @@ icon: "puzzle-piece"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
+
<iframe
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/UUSXqAnau0Q"
|
|
14
|
+
title="Testing Chrome Extensions with TestDriver"
|
|
15
|
+
frameborder="0"
|
|
16
|
+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
|
+
allowfullscreen
|
|
18
|
+
></iframe>
|
|
19
|
+
|
|
10
20
|
# Testing Chrome extensions with TestDriver
|
|
11
21
|
TestDriver can be used to test Chrome extensions. This guide provides an overview of how to set up and run tests for your Chrome extension using TestDriver.
|
|
12
22
|
|
|
@@ -26,16 +36,7 @@ You can test a preinstalled extension or test by installing an extension. If the
|
|
|
26
36
|
6. Click the 'puzzle piece' icon to pin the extension to the toolbar.
|
|
27
37
|
7. Click on the extension icon to open it.
|
|
28
38
|
|
|
29
|
-
|
|
30
|
-
<iframe
|
|
31
|
-
width="560"
|
|
32
|
-
height="315"
|
|
33
|
-
src="https://www.youtube.com/embed/czfq4It6-pQ"
|
|
34
|
-
title="Testing Chrome Extensions with TestDriver"
|
|
35
|
-
frameborder="0"
|
|
36
|
-
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
37
|
-
allowfullscreen
|
|
38
|
-
></iframe>
|
|
39
|
+
---
|
|
39
40
|
|
|
40
41
|
## Now what?
|
|
41
42
|
Once you have installed your extension, simply create a TestDriver test to test any functionality a user would use.
|
|
@@ -7,22 +7,28 @@ icon: "desktop"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
-
TestDriver is designed to work with any desktop application, including Electron, Java, and .NET apps. This guide provides an overview of how to use TestDriver with desktop applications, including installation, configuration, and running tests.
|
|
11
|
-
|
|
12
|
-
<TestPrereqs />
|
|
13
|
-
|
|
14
|
-
## Sample test steps
|
|
15
|
-
|
|
16
|
-
## TestDriver in action
|
|
17
10
|
<iframe
|
|
18
|
-
width="
|
|
19
|
-
height="
|
|
20
|
-
src="https://www.youtube.com/embed/
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/DzgZf2b5XJs"
|
|
21
14
|
title="Assert 2+2=4"
|
|
22
15
|
frameborder="0"
|
|
23
16
|
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
24
17
|
allowfullscreen
|
|
25
18
|
></iframe>
|
|
19
|
+
---
|
|
20
|
+
# Using TestDriver with Desktop Apps
|
|
21
|
+
|
|
22
|
+
TestDriver is designed to work with any desktop application, including Electron, Java, and .NET apps. This guide provides an overview of how to use TestDriver with desktop applications, including installation, configuration, and running tests.
|
|
23
|
+
|
|
24
|
+
<TestPrereqs />
|
|
25
|
+
|
|
26
|
+
## Sample test steps
|
|
27
|
+
1. **Launch the application**: Use the `exec` command to launch the application. You can specify the path to the executable file and any command-line arguments needed to start the application.
|
|
28
|
+
2. **Wait for the application to load**: Use the `wait-for-text` command to wait for a specific text or element to appear in the application window. This ensures that the application is fully loaded before proceeding with the test.
|
|
29
|
+
3. **Interact with the application**: Use the `focus-application` command to bring the application window to the foreground. Then, use the `hover-image` command to interact with specific elements in the application, such as buttons or text fields.
|
|
30
|
+
4. **Perform actions**: Use the `hover-image` command to click on buttons or enter text in input fields. You can also use the `wait-for-text` command to verify that the expected output appears after performing an action.
|
|
31
|
+
5. **Verify results**: Use the `assert` command to check if the expected result matches the actual output. This step is crucial for validating the functionality of the application.
|
|
26
32
|
|
|
27
33
|
## Supported desktop applications
|
|
28
34
|
TestDriver supports a wide range of desktop applications, including:
|
|
@@ -4,13 +4,12 @@ sidebarTitle: "Mobile Apps"
|
|
|
4
4
|
description: "Test mobile apps with TestDriver"
|
|
5
5
|
icon: "mobile"
|
|
6
6
|
---
|
|
7
|
+
|
|
7
8
|
# Testing mobile apps with TestDriver
|
|
8
9
|
|
|
9
10
|
## How do I test mobile apps?
|
|
10
11
|
TestDriver provides a powerful solution for testing mobile apps. With our platform, you can easily create and run tests to ensure your mobile app functions as expected across different devices and operating systems.
|
|
11
12
|
|
|
12
|
-
|
|
13
|
-
|
|
14
13
|
<Tip>You will need to install a mobile emulator for your deployed tests or use a device farm to test your mobile app.</Tip>
|
|
15
14
|
|
|
16
15
|
## Gettins started
|
|
@@ -4,19 +4,22 @@ sidebarTitle: "Static Websites"
|
|
|
4
4
|
description: "Test static websites with TestDriver"
|
|
5
5
|
icon: "browser"
|
|
6
6
|
---
|
|
7
|
-
TestDriver provides a powerful solution for testing static websites. With our platform, you can easily create and run tests to ensure your static website functions as expected across different browsers and devices.
|
|
8
|
-
|
|
9
|
-
## TestDriver in action
|
|
10
7
|
<iframe
|
|
11
|
-
width="
|
|
12
|
-
height="
|
|
13
|
-
src="https://www.youtube.com/embed/
|
|
8
|
+
width="600"
|
|
9
|
+
height="400"
|
|
10
|
+
src="https://www.youtube.com/embed/Jgj1sZRpDiU"
|
|
14
11
|
title="Form Filling with TestDriver"
|
|
15
12
|
frameborder="0"
|
|
16
13
|
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
14
|
allowfullscreen
|
|
18
15
|
></iframe>
|
|
19
16
|
|
|
17
|
+
---
|
|
18
|
+
# Testing Static Websites with TestDriver
|
|
19
|
+
|
|
20
|
+
TestDriver provides a powerful solution for testing static websites. With our platform, you can easily create and run tests to ensure your static website functions as expected across different browsers and devices.
|
|
21
|
+
|
|
22
|
+
|
|
20
23
|
## Getting started
|
|
21
24
|
To get started with testing static websites using TestDriver, follow these steps:
|
|
22
25
|
1. **Follow our [quick start guide](/getting-started/setup) to set up your TestDriver account.**
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "remember"
|
|
3
|
+
sidebarTitle: "remember"
|
|
4
|
+
description: "Save a variable to use later."
|
|
5
|
+
icon: "brain"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## Description
|
|
9
|
+
The `remember` command is used to save a variable for later use. This is useful for storing values that you want to reference in subsequent commands or steps, especially for things that are dynamic or change frequently from run to run.
|
|
10
|
+
|
|
11
|
+
**The `remember` command is only availble form `v5.7.0` and later.**
|
|
12
|
+
|
|
13
|
+
## Arguments
|
|
14
|
+
| Argument | Type | Description |
|
|
15
|
+
|----------|---------------------|-----------------------------------------------------------------------------|
|
|
16
|
+
| `output` | string | the variable name you will use to recall the value later. |
|
|
17
|
+
| `description` | string | A prompt describing the value you want to store. |
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Example usage
|
|
23
|
+
```yaml
|
|
24
|
+
- command: remember
|
|
25
|
+
output: my_variable
|
|
26
|
+
description: The date value shown to the right or 'Order Date'
|
|
27
|
+
```
|
|
28
|
+
## Protips
|
|
29
|
+
- Use the output later with this syntax `${OUTPUT.my_variable}`.
|
|
30
|
+
- The `remember` command does not persist variables across different test runs. If you need to store values for later use in different test runs, consider using external storage solutions or environment variables.
|
|
31
|
+
- For now, the `remember` command only supports string values. If you need to store complex data types, you may use `remember` multiple times for the values. This may change in a later version.
|
|
32
|
+
|
|
33
|
+
## Gotchas
|
|
34
|
+
- The `remeember` command is only available from `v5.7.0` and later.
|
|
35
|
+
- Ensure the variable name is unique to avoid overwriting existing variables.
|
|
36
|
+
- The variable name is case-sensitive, so `my_variable` and `My_Variable` would be treated as different variables.
|
|
37
|
+
|
|
38
|
+
## Notes
|
|
39
|
+
- The `remember` command is ideal for storing values that are generated during the test run, such as timestamps, IDs, the text value of a link you might click later, or any other dynamic data.
|
|
40
|
+
|
package/docs/docs.json
CHANGED
|
@@ -23,6 +23,7 @@
|
|
|
23
23
|
"/overview/what-is-testdriver",
|
|
24
24
|
"/overview/faq",
|
|
25
25
|
"/overview/comparison",
|
|
26
|
+
"/overview/performance",
|
|
26
27
|
"/account/enterprise"
|
|
27
28
|
]
|
|
28
29
|
},
|
|
@@ -63,8 +64,8 @@
|
|
|
63
64
|
{
|
|
64
65
|
"group": "Getting Started",
|
|
65
66
|
"pages": [
|
|
66
|
-
"/getting-started/vscode",
|
|
67
67
|
"/getting-started/setup",
|
|
68
|
+
"/getting-started/vscode",
|
|
68
69
|
"/getting-started/writing",
|
|
69
70
|
"/getting-started/generating",
|
|
70
71
|
"/getting-started/running",
|
|
@@ -174,6 +175,7 @@
|
|
|
174
175
|
"/commands/if",
|
|
175
176
|
"/commands/match-image",
|
|
176
177
|
"/commands/press-keys",
|
|
178
|
+
"/commands/remember",
|
|
177
179
|
"/commands/run",
|
|
178
180
|
"/commands/scroll",
|
|
179
181
|
"/commands/scroll-until-image",
|
|
@@ -255,7 +257,20 @@
|
|
|
255
257
|
"youtube": "https://www.youtube.com/@testdriverai"
|
|
256
258
|
}
|
|
257
259
|
},
|
|
258
|
-
"redirects": [
|
|
260
|
+
"redirects": [
|
|
261
|
+
{
|
|
262
|
+
"source": "/guides/github-actions",
|
|
263
|
+
"destination": "/action/setup"
|
|
264
|
+
},
|
|
265
|
+
{
|
|
266
|
+
"source": "/reference/test-steps",
|
|
267
|
+
"destination": "/features/selectorless"
|
|
268
|
+
},
|
|
269
|
+
{
|
|
270
|
+
"source": "/guides/debugging-test-runs",
|
|
271
|
+
"destination": "/getting-started/editing"
|
|
272
|
+
}
|
|
273
|
+
],
|
|
259
274
|
"integrations": {
|
|
260
275
|
"segment": {
|
|
261
276
|
"key": "oh4PK60u4Qvm2DKIjxfhxd0KCdRbQsQE"
|
|
@@ -16,7 +16,32 @@ import YmlWarning from '/snippets/yml-warning.mdx'
|
|
|
16
16
|
</Card>
|
|
17
17
|
|
|
18
18
|
<Steps>
|
|
19
|
+
<Accordion title="Install the Extension">
|
|
19
20
|
|
|
21
|
+
<Tabs>
|
|
22
|
+
<Tab title="VS Code">
|
|
23
|
+
|
|
24
|
+
VS Code is the recommended IDE for TestDriver. First, install the following recommended extensions:
|
|
25
|
+
|
|
26
|
+
- [GitHub Copilot Chat](vscode://github.copilot-chat) (The free plan is just fine!)
|
|
27
|
+
- [VS Code YAML](vscode://redhat.vscode-yaml) (Optional, but recommended for YAML validation)
|
|
28
|
+
|
|
29
|
+
Then, download the TestDriver extension:
|
|
30
|
+
|
|
31
|
+
<Card
|
|
32
|
+
title="Download the VS Code Extension"
|
|
33
|
+
icon="download"
|
|
34
|
+
href="https://github.com/testdriverai/vscode/releases/latest/download/testdriver.vsix"
|
|
35
|
+
>
|
|
36
|
+
Get the latest version from GitHub releases.
|
|
37
|
+
</Card>
|
|
38
|
+
|
|
39
|
+
Finally, install the extension in VS Code. You can do this by dragging and dropping the downloaded `.vsix` file into the VS Code window.
|
|
40
|
+
|
|
41
|
+
Use JetBrains IDEs, VIM, or other editors? No problem! You can use TestDriver with the CLI. This quickstart contains both GUI and terminal commands to help you get started.
|
|
42
|
+
</Tab>
|
|
43
|
+
</Tabs>
|
|
44
|
+
</Accordion>
|
|
20
45
|
<Step title="Install TestDriver">
|
|
21
46
|
Install the TestDriver CLI globally:
|
|
22
47
|
```bash
|
|
@@ -5,8 +5,12 @@ description: "Learn how to customize and set up your TestDriver environment to o
|
|
|
5
5
|
icon: "terminal"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
+
import LifecycleWarning from '/snippets/lifecycle-warning.mdx'
|
|
9
|
+
|
|
8
10
|
## Prerun configuration
|
|
9
11
|
|
|
12
|
+
<LifecycleWarning/>
|
|
13
|
+
|
|
10
14
|
## The default `prerun` script
|
|
11
15
|
When setting up your TestDriver environment, a default prerun script is automatically generated when you run:
|
|
12
16
|
```bash
|
|
@@ -20,7 +24,7 @@ testdriver
|
|
|
20
24
|
│ └── prerun.yaml
|
|
21
25
|
```
|
|
22
26
|
|
|
23
|
-
This script includes essential commands to ensure your VM is ready for testing. You can customize this script to suit your specific needs, but it's important to understand that it serves as a baseline for your test environment. It looks like this:
|
|
27
|
+
This script includes essential commands to ensure your **local agent or sandbox** VM is ready for testing. You can customize this script to suit your specific needs, but it's important to understand that it serves as a baseline for your test environment. It looks like this:
|
|
24
28
|
|
|
25
29
|
|
|
26
30
|
```yaml ./lifecycle/prerun.yaml
|
|
@@ -51,14 +55,18 @@ The `wait-For-text` command is crucial for ensuring that the browser is fully lo
|
|
|
51
55
|
|
|
52
56
|
If a file is found at `./lifecycle/prerun.yaml`, it's executed _before_ the test begins. _This will happen even if you have a pre or post test script defined in your GitHub Actions workflow._ This file works like any other TestDriver file and is commonly used to perform tasks such as opening a browser, navigating to a specific page, or resetting the application state.
|
|
53
57
|
|
|
54
|
-
|
|
58
|
+
Note that the `prerun` script is found in the GitHub Actions workflow file for deployed tests.
|
|
59
|
+
|
|
60
|
+
### Prerun flow
|
|
55
61
|
```mermaid
|
|
56
62
|
graph TD
|
|
57
|
-
A{Is
|
|
58
|
-
A -->|Yes| B[Run
|
|
59
|
-
B -->
|
|
60
|
-
A -->|No|
|
|
61
|
-
|
|
63
|
+
A{Is the test deployed to CI?}
|
|
64
|
+
A -->|Yes| B[Run .github/workflows/testdriver.yml]
|
|
65
|
+
B --> F
|
|
66
|
+
A -->|No| C{Is prerun available at ./lifecycle/prerun.yaml}
|
|
67
|
+
C -->|Yes| D[Run ./lifecycle/prerun.yaml]
|
|
68
|
+
C -->|No| F
|
|
69
|
+
D --> F[Run ./testdriver/testfile.yaml]
|
|
62
70
|
```
|
|
63
71
|
---
|
|
64
72
|
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Performance"
|
|
3
|
+
sidebarTitle: "Performance"
|
|
4
|
+
description: "Expected Command Performance"
|
|
5
|
+
icon: "rabbit-running"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
This guide outlines the typical performance characteristics of each TestDriver command based on observed average execution times. These measurements can help guide optimization, identify bottlenecks, and set realistic expectations when designing automated tests.
|
|
9
|
+
|
|
10
|
+
## Fastest Commands
|
|
11
|
+
|
|
12
|
+
These commands execute quickly and can be relied upon for high-frequency usage in complex test sequences.
|
|
13
|
+
|
|
14
|
+
| Command | Avg Duration | Notes |
|
|
15
|
+
|---------|--------------|-------|
|
|
16
|
+
| `exec` | **0.28s** | Fastest command — used for running system-level operations or internal scripting logic. |
|
|
17
|
+
| `wait-for-image` | 2.21s | Relatively fast if the image is readily present; can be slower if the UI takes time to render. |
|
|
18
|
+
| `remember` | 2.80s | Internal memory operation — used for tracking previous outputs or locations. |
|
|
19
|
+
| `hover-text` | 3.11s | Efficient for UI elements with immediate accessibility. |
|
|
20
|
+
| `scroll` | 3.34s | Smooth and fast in most scrollable containers. |
|
|
21
|
+
| `assert` | 3.47s | Used for validation — usually lightweight unless image or text detection is delayed. |
|
|
22
|
+
|
|
23
|
+
## Medium Performance Commands
|
|
24
|
+
|
|
25
|
+
These are reliable but may involve minor delays due to image processing, UI rendering, or input simulation.
|
|
26
|
+
|
|
27
|
+
| Command | Avg Duration | Notes |
|
|
28
|
+
|---------|--------------|-------|
|
|
29
|
+
| `focus-application` | 4.83s | Includes system-level context switching — may vary by OS and app state. |
|
|
30
|
+
| `scroll-until-text` | 5.94s | Slightly slower due to iterative scroll and search logic. |
|
|
31
|
+
| `click` | 6.15s | Includes visual target matching and precise cursor control. |
|
|
32
|
+
| `press-keys` | 6.18s | Slightly slower if key sequences involve modifier keys or application delays. |
|
|
33
|
+
| `type` | 7.32s | Simulates real typing — intentionally throttled for realism and stability. |
|
|
34
|
+
| `wait` | 7.50s | Direct sleep used for explicit pauses. Use sparingly for faster tests. |
|
|
35
|
+
|
|
36
|
+
## Slower Commands
|
|
37
|
+
|
|
38
|
+
These commands tend to be slower due to intensive image comparison, polling loops, or delays in dynamic content rendering.
|
|
39
|
+
|
|
40
|
+
| Command | Avg Duration | Notes |
|
|
41
|
+
|---------|--------------|-------|
|
|
42
|
+
| `hover-image` | 11.95s | Requires locating a target image — performance depends on image quality and rendering time. |
|
|
43
|
+
| `wait-for-text` | 12.08s | Polls repeatedly for expected text — delay depends on application speed and visibility. |
|
|
44
|
+
| `match-image` | **16.55s** | Most time-consuming operation — relies on pixel-level image detection which may be affected by resolution, anti-aliasing, and scaling. |
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
- **Avoid overusing `match-image`** unless strictly necessary. Prefer `wait-for-text` or `hover-text` when working with text-based UIs.
|
|
49
|
+
- **Use `remember` and `assert` early** in the test to catch failures before expensive commands.
|
|
50
|
+
- **Favor `exec` for background operations** like launching processes or setting up test conditions.
|
|
51
|
+
- **Use `wait` intentionally** and sparingly — prefer dynamic waits (`wait-for-image`, `wait-for-text`) when possible.
|
|
52
|
+
- **Monitor cumulative test time**. Replacing slower commands can dramatically improve test suite duration.
|
|
@@ -13,54 +13,7 @@ TestDriver isn't just a computer-use agent, it's an entire AI-native workflow fo
|
|
|
13
13
|
|
|
14
14
|
<AccordionGroup>
|
|
15
15
|
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
<Tabs>
|
|
19
|
-
<Tab title="VS Code">
|
|
20
|
-
|
|
21
|
-
VS Code is the recommended IDE for TestDriver. First, install the following recommended extensions:
|
|
22
|
-
|
|
23
|
-
- [GitHub Copilot Chat](vscode://github.copilot-chat) (The free plan is just fine!)
|
|
24
|
-
- [VS Code YAML](vscode://redhat.vscode-yaml) (Optional, but recommended for YAML validation)
|
|
25
|
-
|
|
26
|
-
Then, download the TestDriver extension:
|
|
27
|
-
|
|
28
|
-
<Card
|
|
29
|
-
title="Download the VS Code Extension"
|
|
30
|
-
icon="download"
|
|
31
|
-
href="https://github.com/testdriverai/vscode/releases/latest/download/testdriver.vsix"
|
|
32
|
-
>
|
|
33
|
-
Get the latest version from GitHub releases.
|
|
34
|
-
</Card>
|
|
35
|
-
|
|
36
|
-
Finally, install the extension in VS Code. You can do this by dragging and dropping the downloaded `.vsix` file into the VS Code window.
|
|
37
|
-
|
|
38
|
-
</Tab>
|
|
39
|
-
<Tab title="Cursor">
|
|
40
|
-
|
|
41
|
-
First, install the following recommended extension:
|
|
42
|
-
|
|
43
|
-
- [VS Code YAML](vscode://redhat.vscode-yaml) (Optional, but recommended for YAML validation)
|
|
44
|
-
|
|
45
|
-
Then download the TestDriver extension:
|
|
46
|
-
|
|
47
|
-
<Card
|
|
48
|
-
title="Download the VS Code Extension"
|
|
49
|
-
icon="download"
|
|
50
|
-
href="https://github.com/testdriverai/vscode/releases/latest/download/testdriver.vsix"
|
|
51
|
-
>
|
|
52
|
-
Install the TestDriver extension from the VS Code Marketplace.
|
|
53
|
-
</Card>
|
|
54
|
-
|
|
55
|
-
Finally, install the extension in Cursor. For more information on installing extensions in Cursor, check out the [Cursor documentation](https://forum.cursor.com/t/how-to-install-vsix-format-extension/1667).
|
|
56
|
-
|
|
57
|
-
</Tab>
|
|
58
|
-
<Tab title="Everything Else">
|
|
59
|
-
|
|
60
|
-
Use JetBrains IDEs, VIM, or other editors? No problem! You can use TestDriver with the CLI. This quickstart contains both GUI and terminal commands to help you get started.
|
|
61
|
-
</Tab>
|
|
62
|
-
</Tabs>
|
|
63
|
-
</Accordion>
|
|
16
|
+
|
|
64
17
|
<Accordion title="Install the CLI">
|
|
65
18
|
|
|
66
19
|
Ensure you have Node.js (v16 or higher) installed. Then, install the TestDriver CLI globally using `npm`:
|
|
@@ -217,4 +170,4 @@ TestDriver can also generate tests without any input! Instead, it will suggest t
|
|
|
217
170
|
|
|
218
171
|
</Step>
|
|
219
172
|
|
|
220
|
-
</Steps>
|
|
173
|
+
</Steps>
|
|
@@ -7,6 +7,18 @@ icon: "cookie"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
+
<iframe
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/VUmjDQ4rM9M"
|
|
14
|
+
title="Testing Cookie Banners with TestDriver"
|
|
15
|
+
frameborder="0"
|
|
16
|
+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
|
+
allowfullscreen
|
|
18
|
+
></iframe>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
# Testing Cookie Banners with TestDriver
|
|
10
22
|
With TestDriver, you can automate the testing of cookie banners on your web application. This scenario demonstrates how to check if a cookie banner appears when a user visits the site and how to interact with it.
|
|
11
23
|
|
|
12
24
|
<TestPrereqs />
|
|
@@ -15,16 +27,3 @@ With TestDriver, you can automate the testing of cookie banners on your web appl
|
|
|
15
27
|
1. **Visit the Site**: The test will navigate to the target URL.
|
|
16
28
|
2. **Check for Cookie Banner**: It will verify if the cookie banner is displayed.
|
|
17
29
|
3. **Decline Cookies**: The test will simulate a user clicking the "Decline All" button on the cookie banner.
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
## TestDriver in action
|
|
22
|
-
<iframe
|
|
23
|
-
width="560"
|
|
24
|
-
height="315"
|
|
25
|
-
src="https://www.youtube.com/embed/mhTffX0SG94"
|
|
26
|
-
title="Testing Cookie Banners with TestDriver"
|
|
27
|
-
frameborder="0"
|
|
28
|
-
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
29
|
-
allowfullscreen
|
|
30
|
-
></iframe>
|
|
@@ -7,6 +7,18 @@ icon: "upload"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
+
<iframe
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/9fMAJ0AC6Qw"
|
|
14
|
+
title="File Upload Testing with TestDriver"
|
|
15
|
+
frameborder="0"
|
|
16
|
+
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
17
|
+
allowfullscreen
|
|
18
|
+
></iframe>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
## Testing file upload functionality with TestDriver
|
|
10
22
|
This scenario demonstrates how to automate testing the file upload functionality of a web application using TestDriver's capabilities. The test will check if the file upload process works correctly and if the uploaded file is processed as expected.
|
|
11
23
|
|
|
12
24
|
<TestPrereqs />
|
|
@@ -16,15 +28,3 @@ This scenario demonstrates how to automate testing the file upload functionality
|
|
|
16
28
|
2. Generate a file that will be used for the upload.
|
|
17
29
|
3. Use TestDriver to perform the file upload process.
|
|
18
30
|
4. Check if the uploaded file is processed correctly by the application.
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
## TestDriver in action
|
|
22
|
-
<iframe
|
|
23
|
-
width="560"
|
|
24
|
-
height="315"
|
|
25
|
-
src="https://www.youtube.com/embed/50AN5hDnwAg"
|
|
26
|
-
title="File Upload Testing with TestDriver"
|
|
27
|
-
frameborder="0"
|
|
28
|
-
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
29
|
-
allowfullscreen
|
|
30
|
-
></iframe>
|
|
@@ -7,22 +7,23 @@ icon: "list"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
-
<TestPrereqs />
|
|
11
|
-
|
|
12
|
-
## Scenario overview
|
|
13
|
-
1. Open a webpage containing a form (Log in, Registration, etc.).
|
|
14
|
-
2. Use TestDriver to fill in the form fields with test data.
|
|
15
|
-
3. Submit the form and verify the expected outcome (for example, successful login, registration confirmation, redirect etc.).
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
## TestDriver in action
|
|
20
10
|
<iframe
|
|
21
|
-
width="
|
|
22
|
-
height="
|
|
23
|
-
src="https://www.youtube.com/embed/
|
|
11
|
+
width="600"
|
|
12
|
+
height="400"
|
|
13
|
+
src="https://www.youtube.com/embed/R0xHyuaBy3A"
|
|
24
14
|
title="Form Filling With TestDriver"
|
|
25
15
|
frameborder="0"
|
|
26
16
|
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
|
|
27
17
|
allowfullscreen
|
|
28
18
|
></iframe>
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
# Testing Form Filling with TestDriver
|
|
22
|
+
TestDriver provides a powerful solution for testing form filling functionality on web applications. This scenario demonstrates how to automate the process of filling out forms, such as login or registration forms, using TestDriver's capabilities.
|
|
23
|
+
|
|
24
|
+
<TestPrereqs />
|
|
25
|
+
|
|
26
|
+
## Scenario overview
|
|
27
|
+
1. Open a webpage containing a form (Log in, Registration, etc.).
|
|
28
|
+
2. Use TestDriver to fill in the form fields with test data.
|
|
29
|
+
3. Submit the form and verify the expected outcome (for example, successful login, registration confirmation, redirect etc.).
|
|
@@ -7,6 +7,7 @@ icon: "lock"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
+
# Testing Login Functionality with TestDriver
|
|
10
11
|
Test login functionality with TestDriver. This scenario demonstrates how to automate testing the login process for a web application using TestDriver.
|
|
11
12
|
|
|
12
13
|
<TestPrereqs />
|
|
@@ -7,6 +7,7 @@ icon: "file-pdf"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
+
# Testing PDF Generation with TestDriver
|
|
10
11
|
Test PDF generation functionality with TestDriver. This scenario demonstrates how to automate testing the PDF generation process for a web application using TestDriver's capabilities.
|
|
11
12
|
|
|
12
13
|
<TestPrereqs />
|
|
@@ -7,6 +7,7 @@ icon: "spell-check"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
+
# Testing Spell Check with TestDriver
|
|
10
11
|
This scenario demonstrates how to automate testing the spell check functionality of a web application using TestDriver's capabilities. The test will check if the spell checker correctly identifies and suggests corrections for misspelled words in a text input field.
|
|
11
12
|
|
|
12
13
|
<TestPrereqs />
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
<Warning>Prerun scripts for deployed tests happen in your GitHub Actions workflow setup file as per below. Prerun scripts that are part of the test lifecycle are run during local and sandbox runners only. This is likely to change in a future version.</Warning>
|
|
@@ -1,15 +1,14 @@
|
|
|
1
1
|
## Prerequisites
|
|
2
2
|
Before running the tests, ensure you have performed the following steps:
|
|
3
3
|
|
|
4
|
-
1.
|
|
5
|
-
2.
|
|
6
|
-
3. Install the [TestDriver CLI](/cli/overview) globally using npm or just follow the VS Code Extension Setup Walkthrough:
|
|
4
|
+
1. If you haven't already, signup for a Free Trial on the [TestDriver website](https://testdriver.ai/pricing)
|
|
5
|
+
2. Install the [TestDriver CLI](/cli/overview) globally using npm or just follow the VS Code Extension Setup Walkthrough:
|
|
7
6
|
```bash
|
|
8
7
|
npm install -g @testdriverai@beta
|
|
9
8
|
```
|
|
10
|
-
|
|
9
|
+
3. Run the `init` command to set up the TestDriver configuration using the API key you got when you signed up for the trial:
|
|
11
10
|
```bash
|
|
12
11
|
testdriverai init
|
|
13
12
|
```
|
|
14
13
|
|
|
15
|
-
<Check>Now you are ready to run the tests!</Check>
|
|
14
|
+
<Check>Now you are ready to run the tests!</Check>
|