testdriverai 5.5.5 → 5.5.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agent.js +29 -39
- package/docs/account/dashboard.mdx +16 -0
- package/docs/account/enterprise.mdx +97 -0
- package/docs/account/pricing.mdx +26 -0
- package/docs/account/projects.mdx +27 -0
- package/docs/account/team.mdx +31 -0
- package/docs/action/browser.mdx +1 -1
- package/docs/action/os.mdx +1 -2
- package/docs/action/output.mdx +2 -2
- package/docs/action/performance.mdx +4 -0
- package/docs/action/prerun.mdx +57 -1
- package/docs/action/secrets.mdx +3 -3
- package/docs/action/setup.mdx +6 -6
- package/docs/apps/chrome-extensions.mdx +41 -0
- package/docs/apps/desktop-apps.mdx +87 -0
- package/docs/apps/mobile-apps.mdx +22 -0
- package/docs/apps/static-websites.mdx +54 -0
- package/docs/bugs/jira.mdx +35 -28
- package/docs/cli/overview.mdx +19 -13
- package/docs/commands/assert.mdx +2 -0
- package/docs/commands/exec.mdx +96 -0
- package/docs/commands/focus-application.mdx +2 -0
- package/docs/commands/hover-image.mdx +2 -0
- package/docs/commands/hover-text.mdx +2 -0
- package/docs/commands/if.mdx +3 -2
- package/docs/commands/issues.mdx +27 -0
- package/docs/commands/match-image.mdx +2 -0
- package/docs/commands/press-keys.mdx +9 -5
- package/docs/commands/run.mdx +2 -1
- package/docs/commands/scroll-until-image.mdx +3 -1
- package/docs/commands/scroll-until-text.mdx +2 -0
- package/docs/commands/scroll.mdx +3 -1
- package/docs/commands/type.mdx +3 -1
- package/docs/commands/wait-for-image.mdx +3 -1
- package/docs/commands/wait-for-text.mdx +3 -1
- package/docs/commands/wait.mdx +3 -1
- package/docs/docs.json +260 -146
- package/docs/exporting/playwright.mdx +33 -26
- package/docs/features/auto-healing.mdx +25 -20
- package/docs/features/cross-platform.mdx +9 -8
- package/docs/features/generation.mdx +4 -3
- package/docs/features/github.mdx +22 -18
- package/docs/features/parallel-testing.mdx +15 -11
- package/docs/features/reusable-snippets.mdx +18 -14
- package/docs/features/selectorless.mdx +8 -3
- package/docs/features/visual-assertions.mdx +9 -10
- package/docs/getting-started/ci.mdx +7 -8
- package/docs/getting-started/editing.mdx +2 -0
- package/docs/getting-started/generating.mdx +1 -1
- package/docs/getting-started/running.mdx +2 -2
- package/docs/getting-started/setup.mdx +5 -12
- package/docs/getting-started/vscode.mdx +67 -79
- package/docs/getting-started/writing.mdx +7 -5
- package/docs/guide/assertions.mdx +6 -6
- package/docs/guide/authentication.mdx +18 -14
- package/docs/guide/code.mdx +3 -3
- package/docs/guide/environment-variables.mdx +64 -0
- package/docs/guide/lifecycle.mdx +154 -0
- package/docs/guide/locating.mdx +9 -9
- package/docs/guide/variables.mdx +9 -6
- package/docs/guide/waiting.mdx +9 -8
- package/docs/images/content/account/newprojectsettings.png +0 -0
- package/docs/images/content/account/projectpage.png +0 -0
- package/docs/images/content/account/projectreplays.png +0 -0
- package/docs/images/content/account/team-manage.png +0 -0
- package/docs/images/content/account/teampage.png +0 -0
- package/docs/images/content/side-by-side.png +0 -0
- package/docs/images/content/vscode/vscode-agent-preview.png +0 -0
- package/docs/images/content/vscode/vscode-copilot-ask.png +0 -0
- package/docs/images/content/vscode/vscode-file-creation.png +0 -0
- package/docs/images/content/vscode/vscode-install.png +0 -0
- package/docs/images/content/vscode/vscode-overview.png +0 -0
- package/docs/images/content/vscode/vscode-setup-walkthrough.png +0 -0
- package/docs/images/content/vscode/vscode-stopchat.png +0 -0
- package/docs/images/content/vscode/vscode-stoptest.png +0 -0
- package/docs/images/content/vscode/vscode-tdservice.png +0 -0
- package/docs/images/content/vscode/vscode-test-output.png +0 -0
- package/docs/images/content/vscode/vscode-testhistory.png +0 -0
- package/docs/images/content/vscode/vscode-testpane-runtests.png +0 -0
- package/docs/images/content/vscode/vscode-testpane.png +0 -0
- package/docs/importing/csv.mdx +9 -6
- package/docs/importing/gherkin.mdx +12 -10
- package/docs/importing/jira.mdx +11 -9
- package/docs/importing/testrail.mdx +10 -8
- package/docs/integrations/electron.mdx +26 -19
- package/docs/integrations/netlify.mdx +25 -24
- package/docs/integrations/vercel.mdx +22 -20
- package/docs/interactive/assert.mdx +4 -1
- package/docs/interactive/dry.mdx +51 -0
- package/docs/interactive/explore.mdx +92 -0
- package/docs/interactive/generate.mdx +6 -3
- package/docs/interactive/run.mdx +3 -0
- package/docs/interactive/save.mdx +3 -0
- package/docs/interactive/undo.mdx +3 -0
- package/docs/overview/comparison.mdx +6 -2
- package/docs/overview/faq.mdx +2 -1
- package/docs/overview/quickstart.mdx +123 -65
- package/docs/overview/upgrading.mdx +82 -0
- package/docs/overview/what-is-testdriver.mdx +13 -5
- package/docs/quickstart.mdx +1 -1
- package/docs/scenarios/ai-chatbot.mdx +26 -0
- package/docs/scenarios/cookie-banner.mdx +30 -0
- package/docs/scenarios/file-upload.mdx +30 -0
- package/docs/scenarios/form-filling.mdx +28 -0
- package/docs/scenarios/log-in.mdx +68 -0
- package/docs/scenarios/pdf-generation.mdx +21 -0
- package/docs/scenarios/spell-check.mdx +21 -0
- package/docs/security/action.mdx +7 -3
- package/docs/security/agent.mdx +1 -1
- package/docs/security/platform.mdx +9 -1
- package/docs/snippets/calendar-link.mdx +1 -0
- package/docs/snippets/gitignore-warning.mdx +4 -0
- package/docs/snippets/test-prereqs.mdx +15 -0
- package/docs/tutorials/advanced-test.mdx +2 -0
- package/docs/tutorials/basic-test.mdx +8 -7
- package/lib/commands.js +40 -21
- package/lib/config.js +0 -1
- package/lib/logger.js +2 -2
- package/lib/parser.js +1 -0
- package/lib/sdk.js +14 -13
- package/package.json +2 -1
- package/schema.json +1 -4
- package/styles/config/vocabularies/Docs/accept.txt +15 -2
- package/styles/config/vocabularies/Docs/reject.txt +4 -0
- package/docs/30x30.mdx +0 -84
- package/docs/issues.mdx +0 -9
- package/docs/security/dashboard.mdx +0 -0
- package/testdriver/chrome.yaml +0 -102
|
@@ -1,10 +1,11 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Cross-Platform Testing with TestDriver
|
|
2
|
+
title: "Cross-Platform Testing with TestDriver"
|
|
3
3
|
sidebarTitle: "Cross-Platform Testing"
|
|
4
|
-
description: "Discover how TestDriver
|
|
4
|
+
description: "Discover how TestDriver enables seamless cross-platform testing across Windows, Linux, and Mac operating systems."
|
|
5
|
+
icon: "computer"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
|
-
TestDriver
|
|
8
|
+
TestDriver provides robust cross-platform support, enabling you to run tests seamlessly on **Windows**, **Linux**, and **Mac** operating systems. This flexibility ensures that your applications are tested in environments that match your users' setups.
|
|
8
9
|
|
|
9
10
|
## Specifying the Operating System
|
|
10
11
|
|
|
@@ -24,7 +25,7 @@ jobs:
|
|
|
24
25
|
- uses: testdriverai/action@main
|
|
25
26
|
with:
|
|
26
27
|
os: linux
|
|
27
|
-
key: ${{ secrets.
|
|
28
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
28
29
|
prompt: |
|
|
29
30
|
1. Open the browser
|
|
30
31
|
2. Navigate to example.com
|
|
@@ -54,7 +55,7 @@ jobs:
|
|
|
54
55
|
- uses: testdriverai/action@main
|
|
55
56
|
with:
|
|
56
57
|
os: mac
|
|
57
|
-
key: ${{ secrets.
|
|
58
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
58
59
|
prompt: |
|
|
59
60
|
1. Open the browser
|
|
60
61
|
2. Navigate to example.com
|
|
@@ -75,7 +76,7 @@ jobs:
|
|
|
75
76
|
|
|
76
77
|
## Write Once, Run Anywhere
|
|
77
78
|
|
|
78
|
-
TestDriver
|
|
79
|
+
TestDriver uses **selectorless testing**, meaning tests are written in a generic format that works across all platforms. The AI dynamically adapts to the environment, ensuring that the same test can run on Windows, Linux, and Mac without modification.
|
|
79
80
|
|
|
80
81
|
### Benefits of Selectorless Testing
|
|
81
82
|
|
|
@@ -88,7 +89,7 @@ TestDriver.ai uses **selectorless testing**, meaning tests are written in a gene
|
|
|
88
89
|
```yaml
|
|
89
90
|
version: 4.2.18
|
|
90
91
|
steps:
|
|
91
|
-
- prompt: Open the browser and search for "TestDriver
|
|
92
|
+
- prompt: Open the browser and search for "TestDriver"
|
|
92
93
|
commands:
|
|
93
94
|
- command: focus-application
|
|
94
95
|
name: Google Chrome
|
|
@@ -97,7 +98,7 @@ steps:
|
|
|
97
98
|
description: main search bar
|
|
98
99
|
action: click
|
|
99
100
|
- command: type
|
|
100
|
-
text: TestDriver
|
|
101
|
+
text: TestDriver
|
|
101
102
|
- command: press-keys
|
|
102
103
|
keys: [enter]
|
|
103
104
|
|
|
@@ -1,10 +1,11 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Test Generation with TestDriver
|
|
2
|
+
title: "Test Generation with TestDriver"
|
|
3
3
|
sidebarTitle: "Test Generation"
|
|
4
4
|
description: "Simplify test creation with AI-driven exploratory testing."
|
|
5
|
+
icon: "wand-sparkles"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
|
-
Writing tests can be a tedious and time-consuming task. TestDriver
|
|
8
|
+
Writing tests can be a tedious and time-consuming task. TestDriver can generate tests just by exploring your app! This guide will show you how to generate tests using TestDriver.
|
|
8
9
|
|
|
9
10
|
# Generate Exploratory Tests
|
|
10
11
|
|
|
@@ -29,7 +30,7 @@ Then, run the `generate` command.
|
|
|
29
30
|
/generate web 10
|
|
30
31
|
```
|
|
31
32
|
|
|
32
|
-
This will generate 10 tests within the `testdriver/generate` directory. You can change the number of tests generated by changing the number after `10`. The generated tests will be in the form of a `.
|
|
33
|
+
This will generate 10 tests within the `testdriver/generate` directory. You can change the number of tests generated by changing the number after `10`. The generated tests will be in the form of a `.yaml` file, and you can run them using your test runner of choice (local or cloud hosted).
|
|
33
34
|
|
|
34
35
|
```bash
|
|
35
36
|
> /generate web 5
|
package/docs/features/github.mdx
CHANGED
|
@@ -1,26 +1,30 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "GitHub Integration with TestDriver
|
|
2
|
+
title: "GitHub Integration with TestDriver"
|
|
3
3
|
sidebarTitle: "GitHub Integration"
|
|
4
|
-
description: "Explore how TestDriver
|
|
4
|
+
description: "Explore how TestDriver enhances GitHub workflows with branch creation, manual dispatch, and detailed outputs for seamless test management."
|
|
5
|
+
icon: "github"
|
|
5
6
|
---
|
|
7
|
+
import GitignoreWarning from '/snippets/gitignore-warning.mdx';
|
|
6
8
|
|
|
7
|
-
TestDriver
|
|
9
|
+
TestDriver offers advanced GitHub integration features, including the ability to create its own branches, trigger workflows manually from the GitHub Actions UI, and provide detailed outputs for debugging and reporting. These features make it easier to manage test workflows, collaborate on test results, and maintain a clean and automated testing pipeline.
|
|
10
|
+
|
|
11
|
+
<GitignoreWarning/>
|
|
8
12
|
|
|
9
13
|
---
|
|
10
14
|
|
|
11
15
|
## 1. **Creating Branches Automatically**
|
|
12
16
|
|
|
13
|
-
TestDriver
|
|
17
|
+
TestDriver can create its own branches to store test results, updates, or auto-healed tests. This ensures that test-related changes are isolated from the main codebase and can be reviewed before merging.
|
|
14
18
|
|
|
15
19
|
### How It Works:
|
|
16
20
|
|
|
17
|
-
- **Branch Creation**: TestDriver
|
|
21
|
+
- **Branch Creation**: TestDriver creates a new branch (e.g., `test-results` or `auto-heal-updates`) during the workflow execution.
|
|
18
22
|
- **Pull Request**: A pull request is automatically opened from the new branch to the base branch (e.g., `main`), allowing developers to review the changes.
|
|
19
23
|
|
|
20
24
|
### Example Workflow with Branch Creation:
|
|
21
25
|
|
|
22
|
-
```yaml
|
|
23
|
-
name: TestDriver
|
|
26
|
+
```yaml {21-24}
|
|
27
|
+
name: TestDriver Auto-Healing
|
|
24
28
|
|
|
25
29
|
on:
|
|
26
30
|
pull_request:
|
|
@@ -34,14 +38,14 @@ jobs:
|
|
|
34
38
|
- name: Check out repository
|
|
35
39
|
uses: actions/checkout@v2
|
|
36
40
|
|
|
37
|
-
- name: Run TestDriver
|
|
41
|
+
- name: Run TestDriver
|
|
38
42
|
uses: testdriverai/action@main
|
|
39
43
|
with:
|
|
40
|
-
key: ${{ secrets.
|
|
44
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
41
45
|
prompt: |
|
|
42
46
|
1. Run all tests in the testdriver directory
|
|
43
47
|
create-pr: true
|
|
44
|
-
pr-title: "TestDriver
|
|
48
|
+
pr-title: "TestDriver Test Results"
|
|
45
49
|
pr-branch: test-results
|
|
46
50
|
pr-test-filename: testdriver-results.yml
|
|
47
51
|
env:
|
|
@@ -71,7 +75,7 @@ GitHub Actions supports manual triggering of workflows using the **workflow_disp
|
|
|
71
75
|
### Example Workflow with Manual Dispatch:
|
|
72
76
|
|
|
73
77
|
```yaml
|
|
74
|
-
name: TestDriver
|
|
78
|
+
name: TestDriver Manual Dispatch
|
|
75
79
|
|
|
76
80
|
on:
|
|
77
81
|
workflow_dispatch:
|
|
@@ -84,10 +88,10 @@ jobs:
|
|
|
84
88
|
- name: Check out repository
|
|
85
89
|
uses: actions/checkout@v2
|
|
86
90
|
|
|
87
|
-
- name: Run TestDriver
|
|
91
|
+
- name: Run TestDriver
|
|
88
92
|
uses: testdriverai/action@main
|
|
89
93
|
with:
|
|
90
|
-
key: ${{ secrets.
|
|
94
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
91
95
|
prompt: |
|
|
92
96
|
1. Run all tests in the testdriver directory
|
|
93
97
|
env:
|
|
@@ -108,11 +112,11 @@ Run Steps
|
|
|
108
112
|
|
|
109
113
|
## 3. **GitHub Actions Output**
|
|
110
114
|
|
|
111
|
-
TestDriver
|
|
115
|
+
TestDriver provides detailed outputs during workflow execution, which are visible in the **Actions** tab of your repository. These outputs include:
|
|
112
116
|
- **Test Summary**: A high-level overview of the test results.
|
|
113
117
|
- **Logs**: Step-by-step logs of the test execution.
|
|
114
118
|
- **Screenshots and GIFs**: Visual feedback for debugging.
|
|
115
|
-
- **Links to Dashcam.io**: Direct links to detailed test recordings and results.
|
|
119
|
+
- **Links to TestDriver Dashcam.io Replays**: Direct links to detailed test recordings and results.
|
|
116
120
|
|
|
117
121
|
### Example Outputs:
|
|
118
122
|
|
|
@@ -120,7 +124,7 @@ TestDriver.ai provides detailed outputs during workflow execution, which are vis
|
|
|
120
124
|
- ✅ All tests passed.
|
|
121
125
|
- ❌ 2 tests failed.
|
|
122
126
|
|
|
123
|
-
#### **TestDriver
|
|
127
|
+
#### **TestDriver Replay**:
|
|
124
128
|
|
|
125
129
|
```markdown
|
|
126
130
|
[View Full Test Results](https://app.testdriver.ai/...)
|
|
@@ -129,7 +133,7 @@ TestDriver.ai provides detailed outputs during workflow execution, which are vis
|
|
|
129
133
|
---
|
|
130
134
|
|
|
131
135
|
### Using Outputs in Workflows:
|
|
132
|
-
You can capture and use TestDriver
|
|
136
|
+
You can capture and use TestDriver outputs in subsequent steps of your workflow. For example, you can post the test summary as a comment on a pull request.
|
|
133
137
|
|
|
134
138
|
#### Example:
|
|
135
139
|
|
|
@@ -158,4 +162,4 @@ You can capture and use TestDriver.ai outputs in subsequent steps of your workfl
|
|
|
158
162
|
2. **Manual Dispatch**: Trigger workflows on demand from the GitHub Actions UI.
|
|
159
163
|
3. **Detailed Outputs**: Access logs, summaries, and visual feedback directly in the Actions tab.
|
|
160
164
|
|
|
161
|
-
These features make TestDriver
|
|
165
|
+
These features make TestDriver a powerful tool for managing automated testing workflows in GitHub, enabling better collaboration, debugging, and test management.
|
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Parallel Testing with TestDriver
|
|
2
|
+
title: "Parallel Testing with TestDriver"
|
|
3
3
|
sidebarTitle: "Parallel Testing"
|
|
4
|
-
description: "Learn how to leverage GitHub Actions' matrix strategy to run TestDriver
|
|
4
|
+
description: "Learn how to leverage GitHub Actions' matrix strategy to run TestDriver tests in parallel, reducing execution time and improving scalability."
|
|
5
|
+
icon: "server"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
8
|
Parallel testing allows you to run multiple tests simultaneously, significantly reducing the time required to execute your test suite. By leveraging GitHub Actions' **matrix strategy**, you can dynamically distribute your tests across multiple jobs, ensuring efficient and scalable test execution.
|
|
@@ -19,18 +20,19 @@ Parallel testing allows you to run multiple tests simultaneously, significantly
|
|
|
19
20
|
|
|
20
21
|
## Setting Up Parallel Testing with Matrix Strategy
|
|
21
22
|
|
|
22
|
-
|
|
23
|
+
<Steps>
|
|
24
|
+
<Step titel="Organize Your Test Files">
|
|
23
25
|
|
|
24
26
|
Ensure your test files are stored in a directory (e.g., `testdriver/`) and follow a consistent naming convention (e.g., `test1.yml`, `test2.yml`, etc.).
|
|
25
27
|
|
|
26
|
-
|
|
28
|
+
</Step>
|
|
27
29
|
|
|
28
|
-
|
|
30
|
+
<Step title="Define the GitHub Action Workflow">
|
|
29
31
|
|
|
30
|
-
Here
|
|
32
|
+
Here's an example of a GitHub Action workflow that uses the matrix strategy to run tests in parallel:
|
|
31
33
|
|
|
32
34
|
```yaml
|
|
33
|
-
name: Parallel Testing with TestDriver
|
|
35
|
+
name: Parallel Testing with TestDriver
|
|
34
36
|
|
|
35
37
|
on:
|
|
36
38
|
push:
|
|
@@ -68,10 +70,10 @@ jobs:
|
|
|
68
70
|
- name: Check out repository
|
|
69
71
|
uses: actions/checkout@v2
|
|
70
72
|
|
|
71
|
-
- name: Run TestDriver
|
|
73
|
+
- name: Run TestDriver
|
|
72
74
|
uses: testdriverai/action@main
|
|
73
75
|
with:
|
|
74
|
-
key: ${{ secrets.
|
|
76
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
75
77
|
prompt: |
|
|
76
78
|
1. /run ${{ matrix.test_file }}
|
|
77
79
|
env:
|
|
@@ -79,6 +81,8 @@ jobs:
|
|
|
79
81
|
FORCE_COLOR: "3"
|
|
80
82
|
```
|
|
81
83
|
|
|
84
|
+
</Step>
|
|
85
|
+
</Steps>
|
|
82
86
|
---
|
|
83
87
|
|
|
84
88
|
### Explanation of the Workflow
|
|
@@ -89,7 +93,7 @@ jobs:
|
|
|
89
93
|
|
|
90
94
|
2. **`run-tests` Job**:
|
|
91
95
|
- Uses the matrix strategy to dynamically create a job for each test file.
|
|
92
|
-
- Runs each test file in parallel using TestDriver.
|
|
96
|
+
- Runs each test file in parallel using TestDriver.
|
|
93
97
|
|
|
94
98
|
3. **Matrix Strategy**:
|
|
95
99
|
- The `matrix.test_file` variable represents each test file.
|
|
@@ -127,4 +131,4 @@ When this workflow runs:
|
|
|
127
131
|
|
|
128
132
|
## Conclusion
|
|
129
133
|
|
|
130
|
-
Parallel testing with the GitHub Action matrix strategy is a powerful way to speed up your TestDriver
|
|
134
|
+
Parallel testing with the GitHub Action matrix strategy is a powerful way to speed up your TestDriver test suite. By dynamically distributing tests across multiple jobs, you can ensure efficient execution and scalability, making it easier to maintain high-quality software.
|
|
@@ -1,9 +1,12 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Reusable Snippets in TestDriver
|
|
2
|
+
title: "Reusable Snippets in TestDriver"
|
|
3
3
|
sidebarTitle: "Reusable Snippets"
|
|
4
|
-
description: "Discover how to modularize your test workflows using reusable YAML snippets in TestDriver
|
|
4
|
+
description: "Discover how to modularize your test workflows using reusable YAML snippets in TestDriver for improved maintainability and scalability."
|
|
5
|
+
icon: "repeat"
|
|
5
6
|
---
|
|
6
7
|
|
|
8
|
+
import GitignoreWarning from '/snippets/gitignore-warning.mdx'
|
|
9
|
+
|
|
7
10
|
Reusable snippets in TestDriver allow you to modularize your test steps by creating smaller, reusable YAML files that can be embedded into larger test workflows. This approach improves test maintainability, reduces duplication, and makes your test suite more organized and scalable.
|
|
8
11
|
|
|
9
12
|
---
|
|
@@ -24,8 +27,8 @@ Reusable snippets are YAML files containing a set of test steps that perform a s
|
|
|
24
27
|
---
|
|
25
28
|
|
|
26
29
|
## How to Create and Use Reusable Snippets
|
|
27
|
-
|
|
28
|
-
|
|
30
|
+
<Steps>
|
|
31
|
+
<Step title="Create a Snippet">
|
|
29
32
|
|
|
30
33
|
Save a YAML file containing the reusable steps. For example, create a `snippets/login.yaml` file for logging into an application:
|
|
31
34
|
|
|
@@ -51,10 +54,9 @@ steps:
|
|
|
51
54
|
description: log in button
|
|
52
55
|
action: click
|
|
53
56
|
```
|
|
57
|
+
</Step>
|
|
54
58
|
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
### Step 2: Reference the Snippet in a Test
|
|
59
|
+
<Step title="Reference the Snippet in a Test">
|
|
58
60
|
|
|
59
61
|
Use the `run` command to include the snippet in your main test file. For example:
|
|
60
62
|
|
|
@@ -70,10 +72,9 @@ steps:
|
|
|
70
72
|
description: dashboard link in the navigation bar
|
|
71
73
|
action: click
|
|
72
74
|
```
|
|
75
|
+
</Step>
|
|
73
76
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
### Step 3: Parameterize Inputs
|
|
77
|
+
<Step title="Parameterize Inputs">
|
|
77
78
|
|
|
78
79
|
Use environment variables to pass dynamic data like usernames and passwords. Define these variables in your `.env` file or CI/CD pipeline:
|
|
79
80
|
|
|
@@ -81,7 +82,11 @@ Use environment variables to pass dynamic data like usernames and passwords. Def
|
|
|
81
82
|
TD_USERNAME=your_username
|
|
82
83
|
TD_PASSWORD=your_password
|
|
83
84
|
```
|
|
84
|
-
|
|
85
|
+
|
|
86
|
+
<GitignoreWarning/>
|
|
87
|
+
</Step>
|
|
88
|
+
</Steps>
|
|
89
|
+
|
|
85
90
|
|
|
86
91
|
## Example: Combining Multiple Snippets
|
|
87
92
|
|
|
@@ -99,8 +104,7 @@ steps:
|
|
|
99
104
|
- command: run
|
|
100
105
|
file: snippets/add_to_cart.yaml
|
|
101
106
|
```
|
|
102
|
-
|
|
103
|
-
---
|
|
107
|
+
---
|
|
104
108
|
|
|
105
109
|
## Best Practices for Reusable Snippets
|
|
106
110
|
|
|
@@ -121,4 +125,4 @@ steps:
|
|
|
121
125
|
|
|
122
126
|
---
|
|
123
127
|
|
|
124
|
-
Reusable snippets are a cornerstone of efficient test design in TestDriver.
|
|
128
|
+
Reusable snippets are a cornerstone of efficient test design in TestDriver. By modularizing your tests, you can create a scalable, maintainable, and reusable test suite that adapts to your application's evolving needs. Start by creating snippets for common actions like login or navigation, and expand your library as your test suite grows.
|
|
@@ -1,10 +1,13 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Selectorless Testing with TestDriver
|
|
2
|
+
title: "Selectorless Testing with TestDriver"
|
|
3
3
|
sidebarTitle: "Selectorless Testing"
|
|
4
4
|
description: "Selectorless testing approach simplifies end-to-end testing by using natural language and AI vision."
|
|
5
|
+
icon: "eye"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
|
-
Selectorless testing eliminates the need for brittle selectors like CSS classes, IDs, or XPath.
|
|
8
|
+
Selectorless testing eliminates the need for brittle selectors like CSS classes, IDs, or XPath.
|
|
9
|
+
|
|
10
|
+
Instead, TestDriver uses natural language prompts and AI-powered vision to interact with applications as a user would. This makes tests more resilient to UI changes and reduces maintenance overhead.
|
|
8
11
|
|
|
9
12
|
- Selectorless testing focuses on **what the user sees** rather than **how the UI is implemented**.
|
|
10
13
|
- Tests are resilient to changes like text updates, class renaming, or minor layout adjustments.
|
|
@@ -59,4 +62,6 @@ For example:
|
|
|
59
62
|
const button = await page.$('button[class="sign-up-btn"]');
|
|
60
63
|
```
|
|
61
64
|
|
|
62
|
-
|
|
65
|
+
<Warning>When using legacy frameworks, if the class name changes, the test will break, requiring updates to the test code!</Warning>
|
|
66
|
+
|
|
67
|
+
<Check>Selectorless testing avoids this by focusing on the intent of the interaction rather than the implementation details.</Check>
|
|
@@ -1,18 +1,17 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Visual Assertions with TestDriver
|
|
2
|
+
title: "Visual Assertions with TestDriver"
|
|
3
3
|
sidebarTitle: "Visual Assertions"
|
|
4
|
-
description: "Discover how TestDriver
|
|
4
|
+
description: "Discover how TestDriver leverages AI-powered visual assertions to validate UI behavior and enhance end-to-end testing."
|
|
5
|
+
icon: "binoculars"
|
|
5
6
|
---
|
|
6
7
|
|
|
7
|
-
Visual assertions in TestDriver
|
|
8
|
+
Visual assertions in TestDriver allow you to validate that your application behaves as expected by analyzing what's displayed on the screen. This approach ensures that your tests mimic real user interactions and verify outcomes visually, making it ideal for testing UI-heavy applications and dynamic content.
|
|
8
9
|
|
|
9
|
-
## What
|
|
10
|
+
## What Can TestDriver Test with Visual Assertions?
|
|
10
11
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
## What Can TestDriver.ai Test with Visual Assertions?
|
|
12
|
+
<Info>
|
|
13
|
+
See many of the following examples in the [TestDriver Scenarios](/scenarios/ai-chatbot)
|
|
14
|
+
</Info>
|
|
16
15
|
|
|
17
16
|
### AI Chatbots
|
|
18
17
|
- Validate that chatbot responses are displayed correctly.
|
|
@@ -120,4 +119,4 @@ Visual assertions check the state of the screen to confirm that specific element
|
|
|
120
119
|
|
|
121
120
|
---
|
|
122
121
|
|
|
123
|
-
Visual assertions in TestDriver
|
|
122
|
+
Visual assertions in TestDriver empower you to test a wide range of applications and scenarios with confidence, ensuring your software delivers a seamless user experience.
|
|
@@ -1,15 +1,15 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: "Running in GitHub Actions"
|
|
3
3
|
sidebarTitle: "Running in CI"
|
|
4
|
-
description: "Learn how to set up and run tests with TestDriver
|
|
4
|
+
description: "Learn how to set up and run tests with TestDriver using GitHub Actions for seamless end-to-end testing."
|
|
5
5
|
icon: "code-pull-request"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
This guide explains how to set up a basic GitHub Actions workflow to run tests using **TestDriver
|
|
8
|
+
This guide explains how to set up a basic GitHub Actions workflow to run tests using **TestDriver**.
|
|
9
9
|
|
|
10
10
|
## Prerequisites
|
|
11
11
|
|
|
12
|
-
1. **TestDriver
|
|
12
|
+
1. **TestDriver API Key**: Obtain your API key from TestDriver and store it as a GitHub secret (e.g., `TD_API_KEY`).
|
|
13
13
|
2. **Test Files**: Ensure your test files are saved in the `testdriver/` directory of your repository.
|
|
14
14
|
|
|
15
15
|
## Create a GitHub Actions Workflow
|
|
@@ -20,7 +20,7 @@ This guide explains how to set up a basic GitHub Actions workflow to run tests u
|
|
|
20
20
|
### Example Workflow: `.github/workflows/testdriver.yml`
|
|
21
21
|
|
|
22
22
|
```yaml
|
|
23
|
-
name: TestDriver
|
|
23
|
+
name: TestDriver / Run / Regressions
|
|
24
24
|
|
|
25
25
|
on:
|
|
26
26
|
push:
|
|
@@ -75,7 +75,7 @@ jobs:
|
|
|
75
75
|
|
|
76
76
|
- uses: testdriverai/action@main
|
|
77
77
|
with:
|
|
78
|
-
key: ${{ secrets.
|
|
78
|
+
key: ${{ secrets.TD_API_KEY }}
|
|
79
79
|
os: ${{ matrix.os }}
|
|
80
80
|
version: "5.2.3"
|
|
81
81
|
prompt: 1. /run testdriver/${{ matrix.test }}
|
|
@@ -101,8 +101,7 @@ jobs:
|
|
|
101
101
|
2. Select the workflow run to view the test results.
|
|
102
102
|
|
|
103
103
|
# Debugging Tests
|
|
104
|
-
|
|
105
|
-
TestDriver.ai provides a powerful debugging interface through its **app.testdriver.ai** platform (formerly Dashcam). This interface allows you to analyze test runs, identify failures, and optimize your test suite with detailed visual and textual feedback.
|
|
104
|
+
TestDriver provides a powerful debugging interface through its **app.TestDriver** platform (formerly Dashcam). This interface allows you to analyze test runs, identify failures, and optimize your test suite with detailed visual and textual feedback.
|
|
106
105
|
|
|
107
106
|
### Step-by-Step Execution Logs
|
|
108
107
|
- View each step of the test execution, including:
|
|
@@ -117,7 +116,7 @@ TestDriver.ai provides a powerful debugging interface through its **app.testdriv
|
|
|
117
116
|
- **Highlighting**: Elements interacted with during the test are highlighted in screenshots, helping you verify that the correct elements were targeted.
|
|
118
117
|
|
|
119
118
|
### 3. Error Details
|
|
120
|
-
- For failed steps, app.
|
|
119
|
+
- For failed steps, app.TestDriver provides:
|
|
121
120
|
- **Error messages**: Detailed descriptions of what went wrong (e.g., "Element not found").
|
|
122
121
|
- **Stack traces**: For advanced debugging of backend or script-related issues.
|
|
123
122
|
- **Suggestions**: Recommendations for fixing common issues, such as adjusting prompts or improving element descriptions.
|
|
@@ -45,6 +45,8 @@ steps:
|
|
|
45
45
|
keys: [enter]
|
|
46
46
|
```
|
|
47
47
|
|
|
48
|
+
<Tip>The `version` captured when you generate a test file is the version of TestDriver you used, and is not used when running the test. This does not affect your ability to use newer versions of TestDriver on existing test files nor do these need to be updated when migrating to a new version.</Tip>
|
|
49
|
+
|
|
48
50
|
### Modify the Test Steps
|
|
49
51
|
You can add, edit, or remove steps as needed. Each step contains:
|
|
50
52
|
|
|
@@ -5,7 +5,7 @@ description: "Generate test plans and integrate them into your testing workflow.
|
|
|
5
5
|
icon: "steering-wheel"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
Writing tests can be a tedious and time-consuming task. TestDriver
|
|
8
|
+
Writing tests can be a tedious and time-consuming task. TestDriver can generate tests just by exploring your app! This guide will show you how to generate tests using TestDriver.
|
|
9
9
|
|
|
10
10
|
# Launch TestDriver in Interactive Mode
|
|
11
11
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "Running Tests with TestDriver
|
|
2
|
+
title: "Running Tests with TestDriver"
|
|
3
3
|
sidebarTitle: "Running Tests"
|
|
4
4
|
description: "Execute regression test plans for debugging and validation."
|
|
5
5
|
icon: "play"
|
|
@@ -32,7 +32,7 @@ To run a test interactively:
|
|
|
32
32
|
|
|
33
33
|
## Notes
|
|
34
34
|
|
|
35
|
-
- Ensure the YAML file is valid and follows the TestDriver
|
|
35
|
+
- Ensure the YAML file is valid and follows the TestDriver specification.
|
|
36
36
|
- The default file path is `testdriver/testdriver.yaml` if no file is specified.
|
|
37
37
|
|
|
38
38
|
<Card
|
|
@@ -1,18 +1,11 @@
|
|
|
1
1
|
---
|
|
2
|
-
title: "
|
|
3
|
-
sidebarTitle: "
|
|
4
|
-
description: "
|
|
2
|
+
title: "Setting up TestDriver CLI"
|
|
3
|
+
sidebarTitle: "CLI Setup"
|
|
4
|
+
description: "Follow these steps to install and set up TestDriver CLI."
|
|
5
5
|
icon: "square-terminal"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
|
|
9
|
-
### **TestDriver.ai Installation Guide**
|
|
10
|
-
|
|
11
|
-
Follow these steps to install and set up TestDriver.ai:
|
|
12
|
-
|
|
13
|
-
---
|
|
14
|
-
|
|
15
|
-
|
|
16
9
|
<Steps>
|
|
17
10
|
|
|
18
11
|
<Step title="Install NodeJS">
|
|
@@ -40,7 +33,7 @@ Follow these steps to install and set up TestDriver.ai:
|
|
|
40
33
|
Install the TestDriver CLI globally using npm:
|
|
41
34
|
|
|
42
35
|
```bash
|
|
43
|
-
npm install -g testdriverai
|
|
36
|
+
npm install -g testdriverai@beta
|
|
44
37
|
```
|
|
45
38
|
|
|
46
39
|
Verify the installation:
|
|
@@ -53,7 +46,7 @@ Follow these steps to install and set up TestDriver.ai:
|
|
|
53
46
|
|
|
54
47
|
<Step title="Set Up Your Test Runner">
|
|
55
48
|
|
|
56
|
-
TestDriver
|
|
49
|
+
TestDriver requires a runner to execute tests. You can use either a hosted sandbox or your own computer.
|
|
57
50
|
|
|
58
51
|
- **TestDriver Sandbox** - Run tests in a ephemeral linux virtual machine. Recommended for ease of use, added privacy, and parallelization. Requires a TestDriver API key.
|
|
59
52
|
- **Local Runner** - Run tests on your own computer. Recommended for working with existing user sessions, files, and applications. Requires additional setup.
|