testdriverai 5.5.5 → 5.5.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agent.js +29 -39
- package/docs/account/dashboard.mdx +16 -0
- package/docs/account/enterprise.mdx +97 -0
- package/docs/account/pricing.mdx +26 -0
- package/docs/account/projects.mdx +27 -0
- package/docs/account/team.mdx +31 -0
- package/docs/action/browser.mdx +1 -1
- package/docs/action/os.mdx +1 -2
- package/docs/action/output.mdx +2 -2
- package/docs/action/performance.mdx +4 -0
- package/docs/action/prerun.mdx +57 -1
- package/docs/action/secrets.mdx +3 -3
- package/docs/action/setup.mdx +6 -6
- package/docs/apps/chrome-extensions.mdx +41 -0
- package/docs/apps/desktop-apps.mdx +87 -0
- package/docs/apps/mobile-apps.mdx +22 -0
- package/docs/apps/static-websites.mdx +54 -0
- package/docs/bugs/jira.mdx +35 -28
- package/docs/cli/overview.mdx +19 -13
- package/docs/commands/assert.mdx +2 -0
- package/docs/commands/exec.mdx +96 -0
- package/docs/commands/focus-application.mdx +2 -0
- package/docs/commands/hover-image.mdx +2 -0
- package/docs/commands/hover-text.mdx +2 -0
- package/docs/commands/if.mdx +3 -2
- package/docs/commands/issues.mdx +27 -0
- package/docs/commands/match-image.mdx +2 -0
- package/docs/commands/press-keys.mdx +9 -5
- package/docs/commands/run.mdx +2 -1
- package/docs/commands/scroll-until-image.mdx +3 -1
- package/docs/commands/scroll-until-text.mdx +2 -0
- package/docs/commands/scroll.mdx +3 -1
- package/docs/commands/type.mdx +3 -1
- package/docs/commands/wait-for-image.mdx +3 -1
- package/docs/commands/wait-for-text.mdx +3 -1
- package/docs/commands/wait.mdx +3 -1
- package/docs/docs.json +260 -146
- package/docs/exporting/playwright.mdx +33 -26
- package/docs/features/auto-healing.mdx +25 -20
- package/docs/features/cross-platform.mdx +9 -8
- package/docs/features/generation.mdx +4 -3
- package/docs/features/github.mdx +22 -18
- package/docs/features/parallel-testing.mdx +15 -11
- package/docs/features/reusable-snippets.mdx +18 -14
- package/docs/features/selectorless.mdx +8 -3
- package/docs/features/visual-assertions.mdx +9 -10
- package/docs/getting-started/ci.mdx +7 -8
- package/docs/getting-started/editing.mdx +2 -0
- package/docs/getting-started/generating.mdx +1 -1
- package/docs/getting-started/running.mdx +2 -2
- package/docs/getting-started/setup.mdx +5 -12
- package/docs/getting-started/vscode.mdx +67 -79
- package/docs/getting-started/writing.mdx +7 -5
- package/docs/guide/assertions.mdx +6 -6
- package/docs/guide/authentication.mdx +18 -14
- package/docs/guide/code.mdx +3 -3
- package/docs/guide/environment-variables.mdx +64 -0
- package/docs/guide/lifecycle.mdx +154 -0
- package/docs/guide/locating.mdx +9 -9
- package/docs/guide/variables.mdx +9 -6
- package/docs/guide/waiting.mdx +9 -8
- package/docs/images/content/account/newprojectsettings.png +0 -0
- package/docs/images/content/account/projectpage.png +0 -0
- package/docs/images/content/account/projectreplays.png +0 -0
- package/docs/images/content/account/team-manage.png +0 -0
- package/docs/images/content/account/teampage.png +0 -0
- package/docs/images/content/side-by-side.png +0 -0
- package/docs/images/content/vscode/vscode-agent-preview.png +0 -0
- package/docs/images/content/vscode/vscode-copilot-ask.png +0 -0
- package/docs/images/content/vscode/vscode-file-creation.png +0 -0
- package/docs/images/content/vscode/vscode-install.png +0 -0
- package/docs/images/content/vscode/vscode-overview.png +0 -0
- package/docs/images/content/vscode/vscode-setup-walkthrough.png +0 -0
- package/docs/images/content/vscode/vscode-stopchat.png +0 -0
- package/docs/images/content/vscode/vscode-stoptest.png +0 -0
- package/docs/images/content/vscode/vscode-tdservice.png +0 -0
- package/docs/images/content/vscode/vscode-test-output.png +0 -0
- package/docs/images/content/vscode/vscode-testhistory.png +0 -0
- package/docs/images/content/vscode/vscode-testpane-runtests.png +0 -0
- package/docs/images/content/vscode/vscode-testpane.png +0 -0
- package/docs/importing/csv.mdx +9 -6
- package/docs/importing/gherkin.mdx +12 -10
- package/docs/importing/jira.mdx +11 -9
- package/docs/importing/testrail.mdx +10 -8
- package/docs/integrations/electron.mdx +26 -19
- package/docs/integrations/netlify.mdx +25 -24
- package/docs/integrations/vercel.mdx +22 -20
- package/docs/interactive/assert.mdx +4 -1
- package/docs/interactive/dry.mdx +51 -0
- package/docs/interactive/explore.mdx +92 -0
- package/docs/interactive/generate.mdx +6 -3
- package/docs/interactive/run.mdx +3 -0
- package/docs/interactive/save.mdx +3 -0
- package/docs/interactive/undo.mdx +3 -0
- package/docs/overview/comparison.mdx +6 -2
- package/docs/overview/faq.mdx +2 -1
- package/docs/overview/quickstart.mdx +123 -65
- package/docs/overview/upgrading.mdx +82 -0
- package/docs/overview/what-is-testdriver.mdx +13 -5
- package/docs/quickstart.mdx +1 -1
- package/docs/scenarios/ai-chatbot.mdx +26 -0
- package/docs/scenarios/cookie-banner.mdx +30 -0
- package/docs/scenarios/file-upload.mdx +30 -0
- package/docs/scenarios/form-filling.mdx +28 -0
- package/docs/scenarios/log-in.mdx +68 -0
- package/docs/scenarios/pdf-generation.mdx +21 -0
- package/docs/scenarios/spell-check.mdx +21 -0
- package/docs/security/action.mdx +7 -3
- package/docs/security/agent.mdx +1 -1
- package/docs/security/platform.mdx +9 -1
- package/docs/snippets/calendar-link.mdx +1 -0
- package/docs/snippets/gitignore-warning.mdx +4 -0
- package/docs/snippets/test-prereqs.mdx +15 -0
- package/docs/tutorials/advanced-test.mdx +2 -0
- package/docs/tutorials/basic-test.mdx +8 -7
- package/lib/commands.js +40 -21
- package/lib/config.js +0 -1
- package/lib/init.js +1 -1
- package/lib/logger.js +2 -2
- package/lib/parser.js +1 -0
- package/lib/sdk.js +14 -13
- package/package.json +2 -1
- package/schema.json +1 -4
- package/styles/config/vocabularies/Docs/accept.txt +15 -2
- package/styles/config/vocabularies/Docs/reject.txt +4 -0
- package/docs/30x30.mdx +0 -84
- package/docs/issues.mdx +0 -9
- package/docs/security/dashboard.mdx +0 -0
- package/testdriver/chrome.yaml +0 -102
|
@@ -1,89 +1,77 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Install the TestDriver
|
|
3
|
-
sidebarTitle: "
|
|
4
|
-
description: "Comprehensive guide to installing and setting up TestDriver
|
|
2
|
+
title: "Install the TestDriver VS Code Extension"
|
|
3
|
+
sidebarTitle: "VS Code Extension"
|
|
4
|
+
description: "Comprehensive guide to installing and setting up TestDriver for VS Code"
|
|
5
5
|
icon: "file-code"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
# Download the VSCode Extension
|
|
9
|
-
|
|
10
|
-
The TestDriver.ai VSCode extension is available for download on our GitHub releases page. This extension integrates seamlessly with Visual Studio Code, allowing you to leverage the power of TestDriver.ai directly within your development environment.
|
|
11
|
-
|
|
12
8
|
<Card
|
|
13
|
-
title="
|
|
14
|
-
icon="
|
|
15
|
-
href="
|
|
9
|
+
title="Get the VS Code Extension"
|
|
10
|
+
icon="link"
|
|
11
|
+
href="/getting-started/vscode"
|
|
16
12
|
>
|
|
17
|
-
|
|
13
|
+
Turn your IDE into a powerful computer-use automation tool with our VS Code extension.
|
|
18
14
|
</Card>
|
|
19
15
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
16
|
+
<Steps>
|
|
17
|
+
|
|
18
|
+
<Step title="Install TestDriver">
|
|
19
|
+
Install the TestDriver CLI globally:
|
|
20
|
+
```bash
|
|
21
|
+
npm install -g testdriverai@beta
|
|
22
|
+
```
|
|
23
|
+
</Step>
|
|
24
|
+
|
|
25
|
+
<Step title="Initialize Your Project">
|
|
26
|
+
Set up your project with:
|
|
27
|
+
```bash
|
|
28
|
+
testdriverai init
|
|
29
|
+
```
|
|
30
|
+
This creates a `.env` file and sample workflow files.
|
|
31
|
+
</Step>
|
|
32
|
+
|
|
33
|
+
<Step title="Run TestDriver in Interactive Mode">
|
|
34
|
+
Start TestDriver in interactive mode to create and refine tests dynamically:
|
|
35
|
+
```bash
|
|
36
|
+
testdriverai
|
|
37
|
+
```
|
|
38
|
+
Once started, you can use commands like `/try`, `/undo`, and `/save` to interactively build and test your flows.
|
|
39
|
+
</Step>
|
|
40
|
+
|
|
41
|
+
<Step title="Define Your Test Steps">
|
|
42
|
+
Use natural language to describe what you want to test. For example:
|
|
43
|
+
```
|
|
44
|
+
> Open Google Chrome and search for "testdriverai"
|
|
45
|
+
```
|
|
46
|
+
TestDriver will generate steps like:
|
|
47
|
+
```yaml
|
|
48
|
+
- command: focus-application
|
|
49
|
+
name: Google Chrome
|
|
50
|
+
- command: hover-text
|
|
51
|
+
text: Search Google or type a URL
|
|
52
|
+
description: main search bar
|
|
53
|
+
action: click
|
|
54
|
+
- command: type
|
|
55
|
+
text: testdriverai
|
|
56
|
+
- command: press-keys
|
|
57
|
+
keys: [enter]
|
|
58
|
+
```
|
|
59
|
+
</Step>
|
|
60
|
+
|
|
61
|
+
<Step title="Refine and Save Your Test">
|
|
62
|
+
- Use `/undo` to remove the last step if needed.
|
|
63
|
+
- Save your test with `/save` to generate a reusable YAML file.
|
|
64
|
+
</Step>
|
|
65
|
+
|
|
66
|
+
<Step title="Run Saved Tests">
|
|
67
|
+
Once saved, you can run your test file anytime:
|
|
68
|
+
```bash
|
|
69
|
+
testdriverai run path/to/test.yml
|
|
70
|
+
```
|
|
71
|
+
</Step>
|
|
72
|
+
|
|
73
|
+
</Steps>
|
|
66
74
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
You can still use the extension to run and manage tests. Use the CLI for test creation instead.
|
|
70
|
-
|
|
71
|
-
## The @testdriverai command doesn't show up
|
|
72
|
-
|
|
73
|
-
Make sure the GitHub Copilot Chat is set to "ask."
|
|
74
|
-
|
|
75
|
-

|
|
76
|
-
|
|
77
|
-
## How do I stop a running execution?
|
|
78
|
-
|
|
79
|
-
Click on the "Stop" button in the bottom of agent chat.
|
|
80
|
-
|
|
81
|
-

|
|
82
|
-
|
|
83
|
-
Click on the "Stop" button in the top right corner of the test execution. This only shows on hover.
|
|
84
|
-
|
|
85
|
-

|
|
86
|
-
|
|
87
|
-
A TestDriver icon will show in the tray bar. Click it, then click "quit."
|
|
75
|
+
---
|
|
88
76
|
|
|
89
|
-
|
|
77
|
+
**Pro Tip:** Always start with interactive mode (`testdriverai`) to explore and refine your tests before saving them. It's faster and more intuitive!
|
|
@@ -5,7 +5,7 @@ description: "Create tests with the help of TestDriver's interactive computer-us
|
|
|
5
5
|
icon: "glasses"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
The TestDriver
|
|
8
|
+
The TestDriver interactive CLI allows you to create, refine, and execute tests dynamically using natural language commands. Here's how to get started.
|
|
9
9
|
|
|
10
10
|
### Prepare Your Environment
|
|
11
11
|
|
|
@@ -21,7 +21,9 @@ When you enter commands into TestDriver, the current window will minimize and th
|
|
|
21
21
|
|
|
22
22
|
For now, set up your environment with a browser window and your terminal side by side like so:
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
<Frame caption="Left or right, it doesn't matter!">
|
|
25
|
+
<img height="200" src="/images/content/side-by-side.png" />
|
|
26
|
+
</Frame>
|
|
25
27
|
|
|
26
28
|
#### Application State
|
|
27
29
|
|
|
@@ -42,7 +44,7 @@ Run the following command to launch the TestDriver interactive mode:
|
|
|
42
44
|
```bash
|
|
43
45
|
testdriverai
|
|
44
46
|
```
|
|
45
|
-
You
|
|
47
|
+
You'll see a prompt (`>`) where you can enter commands.
|
|
46
48
|
|
|
47
49
|
### Write Tests Using Natural Language
|
|
48
50
|
|
|
@@ -80,7 +82,7 @@ command='hover-text' text='Sign Up' description='button in the header' action='c
|
|
|
80
82
|
|
|
81
83
|
### 4. Refine Your Test
|
|
82
84
|
|
|
83
|
-
If something doesn
|
|
85
|
+
If something doesn't work as intended:
|
|
84
86
|
|
|
85
87
|
- Use `/undo` to remove the last step:
|
|
86
88
|
```
|
|
@@ -91,7 +93,7 @@ Modify your prompt or add new steps to refine the test.
|
|
|
91
93
|
|
|
92
94
|
### 5. Save Your Test
|
|
93
95
|
|
|
94
|
-
Once you
|
|
96
|
+
Once you're satisfied with the test, save it to a YAML file:
|
|
95
97
|
|
|
96
98
|
```
|
|
97
99
|
> /save
|
|
@@ -1,13 +1,13 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Using Assertions in TestDriver
|
|
2
|
+
title: "Using Assertions in TestDriver"
|
|
3
3
|
sidebarTitle: "Assertions"
|
|
4
|
-
description: "Comprehensive guide to understanding and implementing assertions in TestDriver
|
|
4
|
+
description: "Comprehensive guide to understanding and implementing assertions in TestDriver for robust test validation."
|
|
5
5
|
icon: "square-check"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
# Guide: Using Assertions in TestDriver
|
|
8
|
+
# Guide: Using Assertions in TestDriver
|
|
9
9
|
|
|
10
|
-
Assertions in TestDriver
|
|
10
|
+
Assertions in TestDriver allow you to validate that your application behaves as expected during a test. By using the `assert` command and visual assertions, you can ensure that specific conditions are met, such as verifying the presence of text, images, or UI elements on the screen.
|
|
11
11
|
|
|
12
12
|
---
|
|
13
13
|
|
|
@@ -15,7 +15,7 @@ Assertions in TestDriver.ai allow you to validate that your application behaves
|
|
|
15
15
|
|
|
16
16
|
Assertions are checks that validate whether a specific condition is true. If the condition isn't met, the test will fail, providing feedback on what went wrong.
|
|
17
17
|
|
|
18
|
-
### Types of Assertions in TestDriver
|
|
18
|
+
### Types of Assertions in TestDriver:
|
|
19
19
|
1. **Text Assertions**: Verify that specific text is visible on the screen.
|
|
20
20
|
2. **Visual Assertions**: Validate the presence of images, icons, or UI elements.
|
|
21
21
|
3. **Custom Assertions**: Use descriptive conditions to check for specific outcomes.
|
|
@@ -164,7 +164,7 @@ steps:
|
|
|
164
164
|
## Debugging Assertions
|
|
165
165
|
|
|
166
166
|
1. **Review Error Messages**:
|
|
167
|
-
- If an assertion fails, TestDriver
|
|
167
|
+
- If an assertion fails, TestDriver provides detailed error messages to help identify the issue.
|
|
168
168
|
|
|
169
169
|
2. **Use Visual Feedback**:
|
|
170
170
|
- Leverage screenshots and visual feedback to verify the state of the application during the assertion.
|
|
@@ -1,15 +1,19 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Authentication in TestDriver
|
|
2
|
+
title: "Authentication in TestDriver"
|
|
3
3
|
sidebarTitle: "Authentication"
|
|
4
|
-
description: "Learn how to securely handle authentication workflows in TestDriver
|
|
5
|
-
icon: "
|
|
4
|
+
description: "Learn how to securely handle authentication workflows in TestDriver using GitHub Actions."
|
|
5
|
+
icon: "fingerprint"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
import GitignoreWarning from '/snippets/gitignore-warning.mdx';
|
|
9
|
+
|
|
10
|
+
This guide explains how to handle **authentication workflows** in **TestDriver** using GitHub Actions. It covers securely passing credentials (e.g., usernames and passwords) to the TestDriver action and using them in both the `prerun` script and test files. Save these locally in your `.env` file and use them in CI as Github secrets.
|
|
11
|
+
|
|
12
|
+
<GitignoreWarning/>
|
|
9
13
|
|
|
10
14
|
---
|
|
11
15
|
|
|
12
|
-
## How Authentication Works in TestDriver
|
|
16
|
+
## How Authentication Works in TestDriver
|
|
13
17
|
|
|
14
18
|
1. **Store Credentials Securely**:
|
|
15
19
|
- Use GitHub Secrets to store sensitive information like usernames, passwords, or API keys.
|
|
@@ -24,8 +28,8 @@ This guide explains how to handle **authentication workflows** in **TestDriver.a
|
|
|
24
28
|
2. Add the following secrets:
|
|
25
29
|
- **`TD_USERNAME`**: The username for login.
|
|
26
30
|
- **`TD_PASSWORD`**: The password for login.
|
|
27
|
-
- **`
|
|
28
|
-
- **`
|
|
31
|
+
- **`TD_API_KEY`**: Your TestDriver API key.
|
|
32
|
+
- **`TD_WEBSITE`**: The URL of the website to test.
|
|
29
33
|
|
|
30
34
|
## Step 2: Pass Credentials to the Workflow
|
|
31
35
|
|
|
@@ -36,7 +40,7 @@ Secrets are passed to the GitHub Action using the `secrets` context. They can be
|
|
|
36
40
|
### Example Workflow:
|
|
37
41
|
|
|
38
42
|
```yaml
|
|
39
|
-
name: TestDriver
|
|
43
|
+
name: TestDriver / Authentication
|
|
40
44
|
|
|
41
45
|
on:
|
|
42
46
|
push:
|
|
@@ -56,7 +60,7 @@ jobs:
|
|
|
56
60
|
- name: Run Authentication Test
|
|
57
61
|
uses: testdriverai/action@main
|
|
58
62
|
with:
|
|
59
|
-
key: ${{ secrets.
|
|
63
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
60
64
|
prompt: |
|
|
61
65
|
1. Open the login page
|
|
62
66
|
2. Enter the username: ${{ secrets.TD_USERNAME }}
|
|
@@ -67,7 +71,7 @@ jobs:
|
|
|
67
71
|
cd $env:TEMP
|
|
68
72
|
npm init -y
|
|
69
73
|
npm install dashcam-chrome
|
|
70
|
-
Start-Process "msedge" -ArgumentList "--start-maximized", "--load-extension=$(pwd)/node_modules/dashcam-chrome/build", "${{ secrets.
|
|
74
|
+
Start-Process "msedge" -ArgumentList "--start-maximized", "--load-extension=$(pwd)/node_modules/dashcam-chrome/build", "${{ secrets.TD_WEBSITE }}"
|
|
71
75
|
exit
|
|
72
76
|
env:
|
|
73
77
|
TD_USERNAME: ${{ secrets.TD_USERNAME }}
|
|
@@ -110,15 +114,15 @@ steps:
|
|
|
110
114
|
## How It Works Together
|
|
111
115
|
|
|
112
116
|
1. **Secrets in the Workflow**:
|
|
113
|
-
- Secrets like `TD_USERNAME` and `TD_PASSWORD` are passed as environment variables to the TestDriver
|
|
117
|
+
- Secrets like `TD_USERNAME` and `TD_PASSWORD` are passed as environment variables to the TestDriver action.
|
|
114
118
|
|
|
115
119
|
2. **Secrets in the `prerun` Script**:
|
|
116
|
-
- The `
|
|
120
|
+
- The `TD_WEBSITE` secret is used to launch the browser with the correct URL.
|
|
117
121
|
|
|
118
122
|
3. **Secrets in the Test File**:
|
|
119
123
|
- The test file dynamically references the secrets to fill in login credentials during the test.
|
|
120
124
|
|
|
121
|
-
## Benefits of Using Authentication in TestDriver
|
|
125
|
+
## Benefits of Using Authentication in TestDriver
|
|
122
126
|
|
|
123
127
|
1. **Secure Handling of Credentials**:
|
|
124
128
|
- Secrets are encrypted and not exposed in logs.
|
|
@@ -132,4 +136,4 @@ steps:
|
|
|
132
136
|
|
|
133
137
|
---
|
|
134
138
|
|
|
135
|
-
By securely passing credentials and using them in the `prerun` script and test files, you can automate authentication workflows in TestDriver
|
|
139
|
+
By securely passing credentials and using them in the `prerun` script and test files, you can automate authentication workflows in TestDriver while ensuring sensitive information remains protected.
|
package/docs/guide/code.mdx
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Custom Code in TestDriver
|
|
2
|
+
title: "Custom Code in TestDriver"
|
|
3
3
|
sidebarTitle: "Custom Code"
|
|
4
|
-
description: "Learn how to integrate custom Node.js scripts into your TestDriver
|
|
4
|
+
description: "Learn how to integrate custom Node.js scripts into your TestDriver workflows for dynamic testing."
|
|
5
5
|
icon: "code"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
TestDriver
|
|
8
|
+
TestDriver allows you to execute custom **Node.js** scripts and shell scripts within your test workflows using the `exec` command. This feature, introduced in version `5.1.0`, enables you to integrate custom logic, such as generating one-time passwords (OTPs), hitting APIs, or performing other dynamic operations, directly into your tests.
|
|
9
9
|
|
|
10
10
|
## Key Features
|
|
11
11
|
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: "Environment Variables"
|
|
3
|
+
sidebarTitle: "Environment Variables"
|
|
4
|
+
description: "Learn how which environment variables are supported by TestDriver."
|
|
5
|
+
icon: "key"
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
import GitignoreWarning from '/snippets/gitignore-warning.mdx'
|
|
9
|
+
|
|
10
|
+
|
|
11
|
+
The supported environment variables in TestDriver are:
|
|
12
|
+
|
|
13
|
+
| Variable | Type | Description |
|
|
14
|
+
|:--------:|:----:|:----------------------------|
|
|
15
|
+
| `TD_SPEAK` | `boolean` | Use text to speech to hear TestDriver speak as commands are run. Helpful when you don't use `TD_OVERLAY` and just plain fun. |
|
|
16
|
+
| `TD_ANALYTICS` | `boolean` | Send analytics to TestDriver servers. |
|
|
17
|
+
| `TD_NOTIFY` | `boolean` | Send desktop notifications when commands are run. |
|
|
18
|
+
| `TD_MINIMIZE` | `boolean` | When not using VMs, minimize the terminal window / calling app when running commands. Useful if you only have one monitor, you don't want the terminal to show up. |
|
|
19
|
+
| `TD_API_ROOT` | `string` | Developer only. Set API root to another URL. For on-prem. |
|
|
20
|
+
| `TD_API_KEY` | `string` | Set this to spawn VMs with TestDriver Pro. |
|
|
21
|
+
| `TD_OVERLAY` | `boolean` | Spawn overlay window that shows both the TestDriver console output, and draws the cursor and squares on the screen as TestDriver thinks. Defaults to true if not provided.|
|
|
22
|
+
| `TD_VM` | `boolean` | Run commands in a Linux VM instead of on own desktop. Requires API key. |
|
|
23
|
+
| `TD_VM_RESOLUTION` | `number[]` | Change resolution when TD_VM is true. |
|
|
24
|
+
| `TD_TYPE` | `string` | Set the type of test to run. Can be `website`, `desktop`, or `mobile`. Not required. |
|
|
25
|
+
|
|
26
|
+
<GitignoreWarning />
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
## Example
|
|
30
|
+
```
|
|
31
|
+
TD_VM=true
|
|
32
|
+
TD_API_KEY=your_api_key
|
|
33
|
+
TD_VM_RESOLUTION=[1024,768]
|
|
34
|
+
TD_MINIMIZE=true
|
|
35
|
+
TD_TYPE=website
|
|
36
|
+
TD_WEBSITE=www.jeep.com
|
|
37
|
+
TD_ANALYTICS=true
|
|
38
|
+
TD_OVERLAY=false
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
In this example `.env` file, we are running a website test in a local Linux VM with a resolution of 1024x768. The terminal will be minimized, and the overlay will not be shown. Analytics will be sent to TestDriver servers.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
## Using Environment Variables in Deployed tests
|
|
45
|
+
When running tests in a deployed environment, you can set environment variables in the `env` section of your GitHub Actions workflow file. This allows you to customize the behavior of TestDriver based on the environment in which it's running. Setup any environment variables you plan to use in the deployed test in your `.env` file and add them as Github secrets.
|
|
46
|
+
|
|
47
|
+
<Tip>For more info on how to set up environment variables with Github Actions, see [Managing Secrets in GitHub Actions](/actions/secrets)</Tip>
|
|
48
|
+
|
|
49
|
+
In this example, there is both a `TD_TEST_USERNAME` and `TD_TEST_PASSWORD` variables in the local `.env` file. This file is used to run tests locally and mask credentials used to log in to a website. When deployed, these variables are stored as secrets in GitHub and are passed to the TestDriver action when it runs.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
## Example 2
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
TD_VM=false
|
|
56
|
+
TD_MINIMIZE=true
|
|
57
|
+
TD_WEBSITE=www.rollsroyce.com
|
|
58
|
+
TD_ANALYTICS=false
|
|
59
|
+
TD_OVERLAY=true
|
|
60
|
+
TD_TEST_USERNAME=me@mysite.com
|
|
61
|
+
TD_TEST_PASSWORD=wokeupinanewbugatti
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
In this example `.env` file, we are running a website test with our desktop using the local agent. The terminal will be minimized, and the overlay will be shown. Analytics will not be sent to TestDriver servers. The `TD_TEST_USERNAME` and `TD_TEST_PASSWORD` variables are used to log in to the website. Be sure to expand these in your test with `${TD_TEST_USERNAME}` and `${TD_TEST_PASSWORD}` notation!
|
package/docs/guide/lifecycle.mdx
CHANGED
|
@@ -36,6 +36,160 @@ steps:
|
|
|
36
36
|
timeout: 30000
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
+
|
|
39
40
|
### Notes
|
|
40
41
|
|
|
41
42
|
- Currently, there is no teardown task implemented. However, when running on GitHub Actions, the virtual machine (VM) is ephemeral, meaning everything is destroyed after the test run.
|
|
43
|
+
|
|
44
|
+
|
|
45
|
+
## Advanced Configuration
|
|
46
|
+
If you need to perform more complex setup or teardown tasks, you can use the `lifecycle` folder in your repository. This folder can contain multiple files, each with a specific purpose.
|
|
47
|
+
|
|
48
|
+
## Workflow Overview
|
|
49
|
+
|
|
50
|
+
1. **Setup Tasks**:
|
|
51
|
+
- Use a dedicated action **before** the TestDriver action to prepare the environment (e.g., create a test user via an API).
|
|
52
|
+
- Pass the created user credentials to the TestDriver action using environment variables.
|
|
53
|
+
|
|
54
|
+
2. **Run Tests**:
|
|
55
|
+
- Execute the tests using TestDriver, leveraging the setup data (e.g., the test user).
|
|
56
|
+
|
|
57
|
+
3. **Teardown Tasks**:
|
|
58
|
+
- Use a dedicated action **after** the TestDriver action to clean up (e.g., delete the test user).
|
|
59
|
+
- Ensure the teardown step runs **no matter the result** of the TestDriver action.
|
|
60
|
+
|
|
61
|
+
---
|
|
62
|
+
|
|
63
|
+
## Example Workflow with Advanced Setup and Teardown
|
|
64
|
+
|
|
65
|
+
### Workflow File: `.github/workflows/testdriver-setup-teardown.yml`
|
|
66
|
+
|
|
67
|
+
```yaml
|
|
68
|
+
name: TestDriver with Setup and Teardown
|
|
69
|
+
|
|
70
|
+
on:
|
|
71
|
+
push:
|
|
72
|
+
branches:
|
|
73
|
+
- main
|
|
74
|
+
pull_request:
|
|
75
|
+
workflow_dispatch:
|
|
76
|
+
|
|
77
|
+
jobs:
|
|
78
|
+
test:
|
|
79
|
+
name: "TestDriver with Setup and Teardown"
|
|
80
|
+
runs-on: ubuntu-latest
|
|
81
|
+
steps:
|
|
82
|
+
# Step 1: Check out the repository
|
|
83
|
+
- name: Check out repository
|
|
84
|
+
uses: actions/checkout@v2
|
|
85
|
+
|
|
86
|
+
# Step 2: Setup - Create a test user via API
|
|
87
|
+
- name: Setup Test User
|
|
88
|
+
id: setup-user
|
|
89
|
+
run: |
|
|
90
|
+
echo "Creating test user via API..."
|
|
91
|
+
RESPONSE=$(curl -X POST -H "Content-Type: application/json" -d '{"name": "Test User", "email": "test@example.com", "password": "password123"}' https://api.example.com/users)
|
|
92
|
+
echo "USER_ID=$(echo $RESPONSE | jq -r '.id')" >> $GITHUB_ENV
|
|
93
|
+
echo "USER_EMAIL=$(echo $RESPONSE | jq -r '.email')" >> $GITHUB_ENV
|
|
94
|
+
echo "USER_PASSWORD=password123" >> $GITHUB_ENV
|
|
95
|
+
env:
|
|
96
|
+
API_KEY: ${{ secrets.API_KEY }}
|
|
97
|
+
|
|
98
|
+
# Step 3: Run Tests with TestDriver
|
|
99
|
+
- name: Run Tests with TestDriver
|
|
100
|
+
uses: testdriverai/action@main
|
|
101
|
+
with:
|
|
102
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
103
|
+
prompt: |
|
|
104
|
+
1. Log in with the test user
|
|
105
|
+
2. Perform actions on the dashboard
|
|
106
|
+
prerun: |
|
|
107
|
+
echo "Launching browser with test user credentials..."
|
|
108
|
+
echo "Email: $USER_EMAIL"
|
|
109
|
+
echo "Password: $USER_PASSWORD"
|
|
110
|
+
env:
|
|
111
|
+
USER_EMAIL: ${{ env.USER_EMAIL }}
|
|
112
|
+
USER_PASSWORD: ${{ env.USER_PASSWORD }}
|
|
113
|
+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
114
|
+
FORCE_COLOR: "3"
|
|
115
|
+
|
|
116
|
+
# Step 4: Teardown - Delete the test user
|
|
117
|
+
- name: Teardown Test User
|
|
118
|
+
if: always()
|
|
119
|
+
run: |
|
|
120
|
+
echo "Deleting test user via API..."
|
|
121
|
+
curl -X DELETE -H "Authorization: Bearer ${{ secrets.API_KEY }}" https://api.example.com/users/$USER_ID
|
|
122
|
+
echo "Test user deleted."
|
|
123
|
+
env:
|
|
124
|
+
USER_ID: ${{ env.USER_ID }}
|
|
125
|
+
API_KEY: ${{ secrets.API_KEY }}
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## Workflow Steps Explained
|
|
131
|
+
|
|
132
|
+
### 1. **Setup Test User**
|
|
133
|
+
- **Purpose**: Create a test user via an API before running the tests.
|
|
134
|
+
- **How It Works**:
|
|
135
|
+
- The `Setup Test User` step sends a `POST` request to the API to create a new user.
|
|
136
|
+
- The user ID, email, and password are extracted from the API response and stored as environment variables (`USER_ID`, `USER_EMAIL`, `USER_PASSWORD`).
|
|
137
|
+
- **Example Output**:
|
|
138
|
+
- `USER_ID`: `12345`
|
|
139
|
+
- `USER_EMAIL`: `test@example.com`
|
|
140
|
+
- `USER_PASSWORD`: `password123`
|
|
141
|
+
|
|
142
|
+
---
|
|
143
|
+
|
|
144
|
+
### 2. **Run Tests with TestDriver**
|
|
145
|
+
- **Purpose**: Execute tests using the TestDriver action.
|
|
146
|
+
- **How It Works**:
|
|
147
|
+
- The `USER_EMAIL` and `USER_PASSWORD` environment variables are passed to the `prerun` script.
|
|
148
|
+
- The test prompts use these credentials to log in and perform actions.
|
|
149
|
+
|
|
150
|
+
---
|
|
151
|
+
|
|
152
|
+
### 3. **Teardown Test User**
|
|
153
|
+
- **Purpose**: Delete the test user via an API after the tests are complete.
|
|
154
|
+
- **How It Works**:
|
|
155
|
+
- The `Teardown Test User` step sends a `DELETE` request to the API to remove the test user.
|
|
156
|
+
- The `if: always()` condition ensures this step runs even if the TestDriver action fails.
|
|
157
|
+
|
|
158
|
+
---
|
|
159
|
+
|
|
160
|
+
## Best Practices for Setup and Teardown
|
|
161
|
+
|
|
162
|
+
1. **Use APIs for Setup and Teardown**:
|
|
163
|
+
- Use APIs to create and delete test data dynamically, ensuring a clean environment for each test run.
|
|
164
|
+
|
|
165
|
+
2. **Pass Data via Environment Variables**:
|
|
166
|
+
- Store setup data (e.g., user credentials) in environment variables and pass them to the TestDriver action.
|
|
167
|
+
|
|
168
|
+
3. **Ensure Teardown Always Runs**:
|
|
169
|
+
- Use `if: always()` to ensure teardown tasks are executed regardless of the test results.
|
|
170
|
+
|
|
171
|
+
4. **Log Setup and Teardown Steps**:
|
|
172
|
+
- Add `echo` statements to log the progress of setup and teardown tasks for easier debugging.
|
|
173
|
+
|
|
174
|
+
5. **Test Locally**:
|
|
175
|
+
- Verify setup and teardown scripts locally before integrating them into the workflow.
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
## Example Use Cases
|
|
180
|
+
|
|
181
|
+
### 1. **User Management**
|
|
182
|
+
- Create a test user during setup.
|
|
183
|
+
- Delete the test user during teardown.
|
|
184
|
+
|
|
185
|
+
### 2. **Database Operations**
|
|
186
|
+
- Insert test data into a database during setup.
|
|
187
|
+
- Remove the test data during teardown.
|
|
188
|
+
|
|
189
|
+
### 3. **Mock Services**
|
|
190
|
+
- Start a mock API server during setup.
|
|
191
|
+
- Stop the mock server during teardown.
|
|
192
|
+
|
|
193
|
+
---
|
|
194
|
+
|
|
195
|
+
By structuring your workflow to handle **setup** before the TestDriver action and **teardown** after it, you can ensure a clean and reliable test environment for every run. This approach also ensures that teardown tasks are executed regardless of the test results, maintaining a consistent state for subsequent runs.
|
package/docs/guide/locating.mdx
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Locating Elements in TestDriver
|
|
2
|
+
title: "Locating Elements in TestDriver"
|
|
3
3
|
sidebarTitle: "Locating Elements"
|
|
4
|
-
description: "Learn how to effectively describe and locate elements using TestDriver
|
|
4
|
+
description: "Learn how to effectively describe and locate elements using TestDriver's visual understanding capabilities."
|
|
5
5
|
icon: "magnifying-glass"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
TestDriver
|
|
8
|
+
TestDriver uses **visual understanding** to locate elements on the screen. Unlike traditional testing frameworks that rely on selectors or element IDs, TestDriver identifies elements based on their **visual appearance**. This guide explains how to describe elements effectively to ensure TestDriver can locate them accurately.
|
|
9
9
|
|
|
10
10
|
---
|
|
11
11
|
|
|
@@ -24,7 +24,7 @@ TestDriver.ai uses **visual understanding** to locate elements on the screen. Un
|
|
|
24
24
|
- For icons or images, describe their shape, color, or associated label.
|
|
25
25
|
|
|
26
26
|
4. **Leverage Visual Feedback**:
|
|
27
|
-
- TestDriver
|
|
27
|
+
- TestDriver will draw **yellow boxes** around detected elements during test execution.
|
|
28
28
|
- Use this feedback to verify that the correct element is being targeted.
|
|
29
29
|
|
|
30
30
|
---
|
|
@@ -53,7 +53,7 @@ TestDriver.ai uses **visual understanding** to locate elements on the screen. Un
|
|
|
53
53
|
|
|
54
54
|
---
|
|
55
55
|
|
|
56
|
-
## How to Write TestDriver
|
|
56
|
+
## How to Write TestDriver Commands
|
|
57
57
|
|
|
58
58
|
### Example: Locating a Button
|
|
59
59
|
|
|
@@ -105,8 +105,8 @@ TestDriver.ai uses **visual understanding** to locate elements on the screen. Un
|
|
|
105
105
|
## Debugging Element Detection
|
|
106
106
|
|
|
107
107
|
1. **Run the Test**:
|
|
108
|
-
- Execute the test using TestDriver.
|
|
109
|
-
- TestDriver
|
|
108
|
+
- Execute the test using TestDriver.
|
|
109
|
+
- TestDriver will draw **yellow boxes** around detected elements.
|
|
110
110
|
|
|
111
111
|
2. **Verify the Correct Element**:
|
|
112
112
|
- Ensure the yellow box highlights the intended element.
|
|
@@ -127,11 +127,11 @@ TestDriver.ai uses **visual understanding** to locate elements on the screen. Un
|
|
|
127
127
|
- Run tests after adding or modifying commands to verify element detection.
|
|
128
128
|
|
|
129
129
|
3. **Use Visual Feedback**:
|
|
130
|
-
- Leverage the yellow boxes drawn by TestDriver
|
|
130
|
+
- Leverage the yellow boxes drawn by TestDriver to confirm the correct element is being targeted.
|
|
131
131
|
|
|
132
132
|
4. **Avoid Overloading Descriptions**:
|
|
133
133
|
- Keep descriptions concise but detailed enough to uniquely identify the element.
|
|
134
134
|
|
|
135
135
|
---
|
|
136
136
|
|
|
137
|
-
By focusing on **visual descriptions** and leveraging TestDriver
|
|
137
|
+
By focusing on **visual descriptions** and leveraging TestDriver's feedback, you can accurately locate elements and create robust, reliable tests.
|