testdriverai 5.6.3 → 5.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/agent.js +17 -15
- package/docs/account/dashboard.mdx +3 -6
- package/docs/account/enterprise.mdx +16 -15
- package/docs/account/pricing.mdx +4 -4
- package/docs/account/projects.mdx +2 -2
- package/docs/account/team.mdx +3 -3
- package/docs/action/browser.mdx +9 -9
- package/docs/action/os.mdx +12 -12
- package/docs/action/output.mdx +10 -10
- package/docs/action/performance.mdx +11 -11
- package/docs/{guide/lifecycle.mdx → action/pre-post-scripts.mdx} +87 -69
- package/docs/action/secrets.mdx +11 -11
- package/docs/action/setup.mdx +8 -8
- package/docs/apps/chrome-extensions.mdx +9 -9
- package/docs/apps/desktop-apps.mdx +6 -5
- package/docs/apps/mobile-apps.mdx +3 -3
- package/docs/apps/static-websites.mdx +2 -3
- package/docs/bugs/jira.mdx +16 -16
- package/docs/cli/overview.mdx +8 -8
- package/docs/commands/exec.mdx +152 -10
- package/docs/commands/focus-application.mdx +1 -1
- package/docs/commands/hover-image.mdx +1 -1
- package/docs/commands/hover-text.mdx +2 -2
- package/docs/commands/if.mdx +1 -1
- package/docs/commands/issues.mdx +8 -6
- package/docs/commands/match-image.mdx +2 -2
- package/docs/commands/press-keys.mdx +19 -2
- package/docs/commands/run.mdx +1 -1
- package/docs/commands/scroll-until-image.mdx +1 -1
- package/docs/commands/scroll-until-text.mdx +2 -2
- package/docs/commands/scroll.mdx +2 -2
- package/docs/commands/type.mdx +1 -1
- package/docs/commands/wait-for-image.mdx +1 -1
- package/docs/commands/wait-for-text.mdx +1 -1
- package/docs/commands/wait.mdx +1 -1
- package/docs/docs.json +9 -9
- package/docs/exporting/playwright.mdx +16 -16
- package/docs/features/auto-healing.mdx +5 -5
- package/docs/features/cross-platform.mdx +4 -4
- package/docs/features/generation.mdx +7 -4
- package/docs/features/github.mdx +21 -21
- package/docs/features/parallel-testing.mdx +9 -9
- package/docs/features/reusable-snippets.mdx +7 -7
- package/docs/features/selectorless.mdx +4 -4
- package/docs/features/visual-assertions.mdx +26 -23
- package/docs/getting-started/ci.mdx +19 -19
- package/docs/getting-started/editing.mdx +12 -8
- package/docs/getting-started/generating.mdx +4 -5
- package/docs/getting-started/running.mdx +3 -3
- package/docs/getting-started/setup.mdx +9 -9
- package/docs/getting-started/vscode.mdx +63 -85
- package/docs/getting-started/writing.mdx +11 -7
- package/docs/guide/assertions.mdx +21 -21
- package/docs/guide/authentication.mdx +11 -11
- package/docs/guide/code.mdx +6 -6
- package/docs/guide/environment-variables.mdx +9 -9
- package/docs/guide/lifecycle-prerun.mdx +191 -0
- package/docs/guide/locating.mdx +15 -18
- package/docs/guide/variables.mdx +8 -8
- package/docs/guide/waiting.mdx +10 -10
- package/docs/importing/csv.mdx +23 -24
- package/docs/importing/gherkin.mdx +20 -20
- package/docs/importing/jira.mdx +22 -23
- package/docs/importing/testrail.mdx +17 -17
- package/docs/integrations/electron.mdx +7 -7
- package/docs/integrations/netlify.mdx +7 -7
- package/docs/integrations/vercel.mdx +13 -13
- package/docs/interactive/assert.mdx +3 -3
- package/docs/interactive/dry.mdx +7 -7
- package/docs/interactive/explore.mdx +9 -9
- package/docs/interactive/generate.mdx +2 -2
- package/docs/interactive/run.mdx +10 -10
- package/docs/interactive/save.mdx +3 -3
- package/docs/interactive/undo.mdx +1 -1
- package/docs/overview/comparison.mdx +6 -6
- package/docs/overview/faq.mdx +2 -2
- package/docs/overview/quickstart.mdx +1 -1
- package/docs/overview/upgrading.mdx +8 -4
- package/docs/overview/what-is-testdriver.mdx +6 -6
- package/docs/quickstart.mdx +1 -1
- package/docs/scenarios/ai-chatbot.mdx +3 -3
- package/docs/scenarios/cookie-banner.mdx +2 -2
- package/docs/scenarios/file-upload.mdx +2 -2
- package/docs/scenarios/form-filling.mdx +3 -3
- package/docs/scenarios/log-in.mdx +3 -3
- package/docs/scenarios/pdf-generation.mdx +3 -3
- package/docs/scenarios/spell-check.mdx +2 -2
- package/docs/security/action.mdx +6 -6
- package/docs/security/agent.mdx +10 -10
- package/docs/security/platform.mdx +10 -8
- package/docs/snippets/test-prereqs.mdx +1 -1
- package/docs/snippets/yml-warning.mdx +1 -0
- package/docs/tutorials/advanced-test.mdx +9 -9
- package/docs/tutorials/basic-test.mdx +3 -3
- package/package.json +1 -1
- package/styles/config/vocabularies/Docs/accept.txt +32 -4
- package/testdriver/spotify.yaml +5 -0
- package/docs/action/prerun.mdx +0 -137
|
@@ -5,66 +5,69 @@ description: "How to handle pre and post test tasks."
|
|
|
5
5
|
icon: "hammer"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
##
|
|
9
|
-
|
|
10
|
-
If a file is found at `testdriver/lifecycle/prerun.yaml`, it's executed before the test begins. This file works like any other TestDriver file and is commonly used to perform tasks such as opening a browser, navigating to a specific page, or resetting the application state.
|
|
11
|
-
|
|
12
|
-
For example, the `prerun.yaml` file can be used to open the Chrome browser and navigate to a page, similar to the example provided above. This ensures that the test environment is properly set up before the test starts.
|
|
13
|
-
|
|
14
|
-
### Example
|
|
15
|
-
|
|
16
|
-
Here is an example of a `prerun.yaml` file:
|
|
17
|
-
|
|
18
|
-
```yaml
|
|
19
|
-
version: 5.1.1
|
|
20
|
-
steps:
|
|
21
|
-
- prompt: launch chrome
|
|
22
|
-
commands:
|
|
23
|
-
- command: exec
|
|
24
|
-
lang: shell
|
|
25
|
-
linux: |
|
|
26
|
-
jumpapp google-chrome --disable-fre --no-default-browser-check --no-first-run "${TD_WEBSITE}" &
|
|
27
|
-
exit
|
|
28
|
-
mac: |
|
|
29
|
-
open -na "Google Chrome" --args --disable-fre --no-default-browser-check --no-first-run --disable-features=PasswordManagerEnabled "${TD_WEBSITE}" &
|
|
30
|
-
exit
|
|
31
|
-
windows:
|
|
32
|
-
Start-Process "C:/Program Files/Google/Chrome/Application/chrome.exe" -ArgumentList "--start-maximized", "${TD_WEBSITE}"
|
|
33
|
-
exit
|
|
34
|
-
- command: wait-for-text
|
|
35
|
-
text: "Google Chrome"
|
|
36
|
-
timeout: 30000
|
|
37
|
-
```
|
|
8
|
+
## Overview
|
|
9
|
+
Prerun scripts are commands executed on a TestDriver virtual machine (VM) before each test in a CI/CD pipeline. They are used to prepare the environment by provisioning the VM, installing dependencies, configuring settings, or building the application. This ensures a consistent and reproducible environment for every test execution.
|
|
38
10
|
|
|
11
|
+
By using prerun scripts, you can:
|
|
12
|
+
- Speed up test setup.
|
|
13
|
+
- Prevent test failures caused by inconsistent environments.
|
|
14
|
+
- Promote reproducible builds for reliable test results.
|
|
39
15
|
|
|
40
|
-
|
|
16
|
+
---
|
|
41
17
|
|
|
42
|
-
|
|
18
|
+
## Use cases
|
|
19
|
+
Prerun scripts are ideal for:
|
|
20
|
+
- Installing necessary dependencies (for example, browsers, libraries, or tools).
|
|
21
|
+
- Building your application or running setup scripts.
|
|
22
|
+
- Configuring the VM to match specific test requirements.
|
|
23
|
+
- Preparing staging environments or accessing private resources.
|
|
43
24
|
|
|
25
|
+
---
|
|
44
26
|
|
|
45
|
-
##
|
|
46
|
-
|
|
27
|
+
## Example: Installing Arc Browser
|
|
28
|
+
The following example demonstrates how to use a prerun script within the GitHub Actions workflow folder to download and install the Arc Browser on a Windows VM before running tests.
|
|
47
29
|
|
|
48
|
-
|
|
30
|
+
<Warning>Note this will be deprecated in favor of [lifecycle prerun scripts](/guide/lifecycle-prerun) from `v5` onward.</Warning>
|
|
49
31
|
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
- Pass the created user credentials to the TestDriver action using environment variables.
|
|
32
|
+
```yaml ./github/workflows/testdriver.yaml
|
|
33
|
+
# Permissions and other setup here
|
|
53
34
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
35
|
+
jobs:
|
|
36
|
+
test:
|
|
37
|
+
name: "TestDriver"
|
|
38
|
+
runs-on: ubuntu-latest
|
|
39
|
+
steps:
|
|
40
|
+
# Use the TestDriver GitHub Action
|
|
41
|
+
- uses: testdriverai/action@main
|
|
42
|
+
with:
|
|
43
|
+
prerun: |
|
|
44
|
+
# Get the IPv6 address of the VM
|
|
45
|
+
Get-NetIPAddress -AddressFamily IPv6
|
|
46
|
+
# URL for the Arc browser installer
|
|
47
|
+
$installerUrl = "https://releases.arc.net/windows/ArcInstaller.exe"
|
|
48
|
+
# Location to save the installer
|
|
49
|
+
$installerPath = "$env:USERPROFILE\Downloads\ArcInstaller.exe"
|
|
50
|
+
# Download the Arc browser installer
|
|
51
|
+
Write-Host "Downloading Arc browser installer..."
|
|
52
|
+
Invoke-WebRequest -Uri $installerUrl -OutFile $installerPath
|
|
53
|
+
# Check if the download was successful
|
|
54
|
+
if (Test-Path $installerPath) {
|
|
55
|
+
Write-Host "Download successful. Running the installer..."
|
|
56
|
+
Start-Process -FilePath $installerPath -ArgumentList '/silent' -Wait
|
|
57
|
+
Start-Sleep -Seconds 10
|
|
58
|
+
} else {
|
|
59
|
+
Write-Host "Failed to download the Arc browser installer."
|
|
60
|
+
}
|
|
61
|
+
```
|
|
60
62
|
|
|
61
63
|
---
|
|
62
64
|
|
|
63
|
-
## Example Workflow with Advanced Setup and Teardown
|
|
64
65
|
|
|
65
|
-
|
|
66
|
+
## Example workflow with advanced setup and teardown
|
|
67
|
+
|
|
68
|
+
### Workflow file
|
|
66
69
|
|
|
67
|
-
```yaml
|
|
70
|
+
```yaml .github/workflows/testdriver-setup-teardown.yaml [expandable]
|
|
68
71
|
name: TestDriver with Setup and Teardown
|
|
69
72
|
|
|
70
73
|
on:
|
|
@@ -83,7 +86,7 @@ jobs:
|
|
|
83
86
|
- name: Check out repository
|
|
84
87
|
uses: actions/checkout@v2
|
|
85
88
|
|
|
86
|
-
# Step 2: Setup -
|
|
89
|
+
# Step 2: Setup - create a test user via API
|
|
87
90
|
- name: Setup Test User
|
|
88
91
|
id: setup-user
|
|
89
92
|
run: |
|
|
@@ -95,7 +98,7 @@ jobs:
|
|
|
95
98
|
env:
|
|
96
99
|
API_KEY: ${{ secrets.API_KEY }}
|
|
97
100
|
|
|
98
|
-
# Step 3: Run
|
|
101
|
+
# Step 3: Run tests with TestDriver
|
|
99
102
|
- name: Run Tests with TestDriver
|
|
100
103
|
uses: testdriverai/action@main
|
|
101
104
|
with:
|
|
@@ -113,7 +116,7 @@ jobs:
|
|
|
113
116
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
114
117
|
FORCE_COLOR: "3"
|
|
115
118
|
|
|
116
|
-
# Step 4: Teardown -
|
|
119
|
+
# Step 4: Teardown - delete the test user
|
|
117
120
|
- name: Teardown Test User
|
|
118
121
|
if: always()
|
|
119
122
|
run: |
|
|
@@ -127,9 +130,23 @@ jobs:
|
|
|
127
130
|
|
|
128
131
|
---
|
|
129
132
|
|
|
130
|
-
## Workflow
|
|
133
|
+
## Workflow overview
|
|
131
134
|
|
|
132
|
-
|
|
135
|
+
1. **Setup Tasks**:
|
|
136
|
+
- Use a dedicated action **before** the TestDriver action to prepare the environment (for example, create a test user via an API).
|
|
137
|
+
- Pass the created user credentials to the TestDriver action using environment variables.
|
|
138
|
+
|
|
139
|
+
2. **Run Tests**:
|
|
140
|
+
- Execute the tests using TestDriver, leveraging the setup data (for example, the test user).
|
|
141
|
+
|
|
142
|
+
3. **Teardown Tasks**:
|
|
143
|
+
- Use a dedicated action **after** the TestDriver action to clean up (for example, delete the test user).
|
|
144
|
+
- Ensure the teardown step runs **no matter the result** of the TestDriver action.
|
|
145
|
+
|
|
146
|
+
### Workflow steps explained
|
|
147
|
+
|
|
148
|
+
<Steps>
|
|
149
|
+
<Step title="Setup test user">
|
|
133
150
|
- **Purpose**: Create a test user via an API before running the tests.
|
|
134
151
|
- **How It Works**:
|
|
135
152
|
- The `Setup Test User` step sends a `POST` request to the API to create a new user.
|
|
@@ -138,35 +155,32 @@ jobs:
|
|
|
138
155
|
- `USER_ID`: `12345`
|
|
139
156
|
- `USER_EMAIL`: `test@example.com`
|
|
140
157
|
- `USER_PASSWORD`: `password123`
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
### 2. **Run Tests with TestDriver**
|
|
158
|
+
</Step>
|
|
159
|
+
<Step title="Run tests with TestDriver">
|
|
145
160
|
- **Purpose**: Execute tests using the TestDriver action.
|
|
146
161
|
- **How It Works**:
|
|
147
162
|
- The `USER_EMAIL` and `USER_PASSWORD` environment variables are passed to the `prerun` script.
|
|
148
163
|
- The test prompts use these credentials to log in and perform actions.
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
### 3. **Teardown Test User**
|
|
164
|
+
</Step>
|
|
165
|
+
<Step title="Teardown test user">
|
|
153
166
|
- **Purpose**: Delete the test user via an API after the tests are complete.
|
|
154
167
|
- **How It Works**:
|
|
155
168
|
- The `Teardown Test User` step sends a `DELETE` request to the API to remove the test user.
|
|
156
169
|
- The `if: always()` condition ensures this step runs even if the TestDriver action fails.
|
|
157
|
-
|
|
170
|
+
</Step>
|
|
171
|
+
</Steps>
|
|
158
172
|
---
|
|
159
173
|
|
|
160
|
-
## Best
|
|
174
|
+
## Best practices for setup and teardown
|
|
161
175
|
|
|
162
176
|
1. **Use APIs for Setup and Teardown**:
|
|
163
177
|
- Use APIs to create and delete test data dynamically, ensuring a clean environment for each test run.
|
|
164
178
|
|
|
165
179
|
2. **Pass Data via Environment Variables**:
|
|
166
|
-
- Store setup data (
|
|
180
|
+
- Store setup data (for example, user credentials) in environment variables and pass them to the TestDriver action.
|
|
167
181
|
|
|
168
182
|
3. **Ensure Teardown Always Runs**:
|
|
169
|
-
- Use `if: always()` to ensure teardown tasks are executed regardless of the test results.
|
|
183
|
+
- Use [`if: always()`](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/evaluate-expressions-in-workflows-and-actions) to ensure teardown tasks are executed regardless of the test results.
|
|
170
184
|
|
|
171
185
|
4. **Log Setup and Teardown Steps**:
|
|
172
186
|
- Add `echo` statements to log the progress of setup and teardown tasks for easier debugging.
|
|
@@ -176,20 +190,24 @@ jobs:
|
|
|
176
190
|
|
|
177
191
|
---
|
|
178
192
|
|
|
179
|
-
##
|
|
193
|
+
## Advanced configuration
|
|
194
|
+
If you need to perform more complex setup or teardown tasks, you can use the [`lifecycle`](/guide/lifecycle-prerun) folder in your repository. This folder can contain multiple files, each with a specific purpose.
|
|
195
|
+
|
|
196
|
+
## Example use cases
|
|
180
197
|
|
|
181
|
-
### 1. **User
|
|
198
|
+
### 1. **User management**
|
|
182
199
|
- Create a test user during setup.
|
|
183
200
|
- Delete the test user during teardown.
|
|
184
201
|
|
|
185
|
-
### 2. **Database
|
|
202
|
+
### 2. **Database operations**
|
|
186
203
|
- Insert test data into a database during setup.
|
|
187
204
|
- Remove the test data during teardown.
|
|
188
205
|
|
|
189
|
-
### 3. **Mock
|
|
206
|
+
### 3. **Mock services**
|
|
190
207
|
- Start a mock API server during setup.
|
|
191
208
|
- Stop the mock server during teardown.
|
|
192
209
|
|
|
193
210
|
---
|
|
194
211
|
|
|
195
|
-
|
|
212
|
+
## Conclusion
|
|
213
|
+
By structuring your GitHub Actions workflow to handle **setup** before the TestDriver action and **teardown** after it, you can ensure a clean and reliable test environment for every run. This approach also ensures that teardown tasks are executed regardless of the test results, maintaining a consistent state for subsequent runs.
|
package/docs/action/secrets.mdx
CHANGED
|
@@ -10,17 +10,17 @@ When using TestDriver to test your application, you may need to securely store a
|
|
|
10
10
|
|
|
11
11
|
---
|
|
12
12
|
|
|
13
|
-
## Why
|
|
13
|
+
## Why use secrets?
|
|
14
14
|
- **Security**: Secrets are encrypted and stored securely in your GitHub repository.
|
|
15
15
|
- **Masking**: TestDriver automatically masks secrets in all test output, including debugging logs.
|
|
16
16
|
- **Reusability**: Secrets can be reused across multiple workflows and test files.
|
|
17
17
|
|
|
18
18
|
---
|
|
19
19
|
|
|
20
|
-
## Step 1: Replace
|
|
20
|
+
## Step 1: Replace hardcoded secrets in test files
|
|
21
21
|
To securely use secrets in your TestDriver test files, replace hardcoded values with placeholders in the format `${TD_YOUR_SECRET}`. TestDriver will parse and mask any secrets that begin with `TD_`.
|
|
22
22
|
|
|
23
|
-
### Example
|
|
23
|
+
### Example test file
|
|
24
24
|
```yaml
|
|
25
25
|
version: 4.1.35
|
|
26
26
|
steps:
|
|
@@ -48,21 +48,21 @@ steps:
|
|
|
48
48
|
|
|
49
49
|
---
|
|
50
50
|
|
|
51
|
-
## Step 2: Add
|
|
51
|
+
## Step 2: Add secrets to Your GitHub repository
|
|
52
52
|
1. Navigate to your GitHub repository.
|
|
53
53
|
2. Go to **Settings** > **Secrets and variables** > **Actions**.
|
|
54
54
|
3. Click **New repository secret**.
|
|
55
|
-
4. Add your secrets (
|
|
55
|
+
4. Add your secrets (for example, `TD_USERNAME`, `TD_PASSWORD`, `TD_API_KEY`).
|
|
56
56
|
|
|
57
57
|
For detailed instructions, refer to the [GitHub Docs on using secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets).
|
|
58
58
|
|
|
59
59
|
---
|
|
60
60
|
|
|
61
|
-
## Step 3: Supply
|
|
61
|
+
## Step 3: Supply secrets to GitHub Actions
|
|
62
62
|
When running TestDriver tests via GitHub Actions, supply your secrets in the `env` section of the workflow file.
|
|
63
63
|
|
|
64
|
-
### Example
|
|
65
|
-
```yaml
|
|
64
|
+
### Example workflow
|
|
65
|
+
```yaml .github/workflows/testdriver.yaml
|
|
66
66
|
name: TestDriver Test
|
|
67
67
|
|
|
68
68
|
permissions:
|
|
@@ -80,7 +80,7 @@ jobs:
|
|
|
80
80
|
with:
|
|
81
81
|
key: ${{ secrets.TD_API_KEY }}
|
|
82
82
|
prompt: |
|
|
83
|
-
1. /run tests/signin.
|
|
83
|
+
1. /run tests/signin.yaml
|
|
84
84
|
env:
|
|
85
85
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
86
86
|
TD_USERNAME: ${{ secrets.TD_USERNAME }}
|
|
@@ -89,8 +89,8 @@ jobs:
|
|
|
89
89
|
|
|
90
90
|
---
|
|
91
91
|
|
|
92
|
-
## Best
|
|
93
|
-
- **Use Descriptive Names**: Name your secrets clearly (
|
|
92
|
+
## Best practices
|
|
93
|
+
- **Use Descriptive Names**: Name your secrets clearly (for example, `TD_USERNAME`, `TD_PASSWORD`) to make them easy to identify.
|
|
94
94
|
- **Rotate Secrets Regularly**: Update your secrets periodically to enhance security.
|
|
95
95
|
- **Limit Access**: Only provide access to secrets for workflows and team members that require them.
|
|
96
96
|
- **Mask Secrets**: Ensure all secrets are masked in logs by using the `TD_` prefix.
|
package/docs/action/setup.mdx
CHANGED
|
@@ -17,7 +17,7 @@ TestDriver Cloud Testing is performed via the TestDriver GitHub Action. You can
|
|
|
17
17
|
|
|
18
18
|
---
|
|
19
19
|
|
|
20
|
-
## Step 2: Get Your API
|
|
20
|
+
## Step 2: Get Your API key
|
|
21
21
|
To execute TestDriver actions on our virtual machines, you'll need an API key. Follow these steps to retrieve and configure your API key:
|
|
22
22
|
|
|
23
23
|
1. **Upgrade Your Account**: Ensure you have a paid TestDriver account.
|
|
@@ -29,8 +29,8 @@ To execute TestDriver actions on our virtual machines, you'll need an API key. F
|
|
|
29
29
|
|
|
30
30
|
---
|
|
31
31
|
|
|
32
|
-
## Step 3: Create
|
|
33
|
-
Now it's time to create your first TestDriver workflow. Add the following configuration to `.github/workflows/testdriver.
|
|
32
|
+
## Step 3: Create your workflow
|
|
33
|
+
Now it's time to create your first TestDriver workflow. Add the following configuration to `.github/workflows/testdriver.yaml`:
|
|
34
34
|
|
|
35
35
|
```yaml
|
|
36
36
|
name: TestDriver
|
|
@@ -67,14 +67,14 @@ jobs:
|
|
|
67
67
|
FORCE_COLOR: "3"
|
|
68
68
|
```
|
|
69
69
|
|
|
70
|
-
### Key
|
|
71
|
-
- **Trigger Conditions**: The `on` section defines when the workflow runs (
|
|
70
|
+
### Key points:
|
|
71
|
+
- **Trigger Conditions**: The `on` section defines when the workflow runs (for example, pull requests, scheduled events, or manual triggers).
|
|
72
72
|
- **API Key**: The `key` field uses the `TD_API_KEY` secret for authentication.
|
|
73
73
|
- **Prompt**: The `prompt` field specifies the commands to execute. In this example, the `/run` command is used to execute a test file from the repository.
|
|
74
74
|
|
|
75
75
|
---
|
|
76
76
|
|
|
77
|
-
## Step 4: Deploy the
|
|
77
|
+
## Step 4: Deploy the workflow
|
|
78
78
|
Save the workflow file, create a new branch, and push it to your repository. Then, create a pull request to trigger the workflow.
|
|
79
79
|
|
|
80
80
|
```bash
|
|
@@ -86,7 +86,7 @@ gh pr create --web
|
|
|
86
86
|
|
|
87
87
|
---
|
|
88
88
|
|
|
89
|
-
## How
|
|
89
|
+
## How it works
|
|
90
90
|
1. **Trigger**: The GitHub Action is triggered based on the conditions defined in the `on` section.
|
|
91
91
|
2. **Authentication**: The `key` value authenticates your account.
|
|
92
92
|
3. **VM Setup**: An ephemeral virtual machine is spawned on TestDriver's infrastructure.
|
|
@@ -99,7 +99,7 @@ gh pr create --web
|
|
|
99
99
|
|
|
100
100
|
---
|
|
101
101
|
|
|
102
|
-
## Additional
|
|
102
|
+
## Additional features
|
|
103
103
|
- **Dynamic Prompts**: You can use commands like `/explore`, supply variables, or dynamically generate prompts from earlier steps.
|
|
104
104
|
- **Staging Workflows**: A common workflow involves waiting for staging to deploy before executing tests.
|
|
105
105
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
---
|
|
2
|
-
title:
|
|
2
|
+
title: Using TestDriver with Chrome Extensions
|
|
3
3
|
sidebarTitle: "Chrome Extensions"
|
|
4
4
|
description: "Test Chrome extensions with TestDriver"
|
|
5
5
|
icon: "puzzle-piece"
|
|
@@ -7,17 +7,17 @@ icon: "puzzle-piece"
|
|
|
7
7
|
|
|
8
8
|
import TestPrereqs from '/snippets/test-prereqs.mdx'
|
|
9
9
|
|
|
10
|
-
# Testing Chrome
|
|
10
|
+
# Testing Chrome extensions with TestDriver
|
|
11
11
|
TestDriver can be used to test Chrome extensions. This guide provides an overview of how to set up and run tests for your Chrome extension using TestDriver.
|
|
12
12
|
|
|
13
13
|
<TestPrereqs />
|
|
14
14
|
|
|
15
|
-
## How
|
|
15
|
+
## How do I test Chrome extensions?
|
|
16
16
|
TestDriver provides a powerful solution for testing Chrome extensions. With our platform, you can easily create and run tests to ensure your extension functions as expected across different browsers and operating systems.
|
|
17
|
-
## Getting
|
|
18
|
-
You can test a preinstalled extension or test by installing an extension. If the extension is in
|
|
17
|
+
## Getting started
|
|
18
|
+
You can test a preinstalled extension or test by installing an extension. If the extension is in development mode, turn this on by checking the "Developer mode" checkbox in the Chrome extensions page. From there you can load the unpacked extension by selecting the folder where your extension is located. To download the files onto the VM if you are performing a test on cloud hosted runners, see the [Lifecycle](/guide/lifecycle-prerun) and [Prerun](/action/pre-post-scripts) docs.
|
|
19
19
|
|
|
20
|
-
## Sample
|
|
20
|
+
## Sample test steps
|
|
21
21
|
1. Load a the Chrome Web Store.
|
|
22
22
|
2. Search for the extension you want to test.
|
|
23
23
|
3. Click on the extension to open its details page.
|
|
@@ -26,7 +26,7 @@ You can test a preinstalled extension or test by installing an extension. If the
|
|
|
26
26
|
6. Click the 'puzzle piece' icon to pin the extension to the toolbar.
|
|
27
27
|
7. Click on the extension icon to open it.
|
|
28
28
|
|
|
29
|
-
## TestDriver in
|
|
29
|
+
## TestDriver in action
|
|
30
30
|
<iframe
|
|
31
31
|
width="560"
|
|
32
32
|
height="315"
|
|
@@ -37,5 +37,5 @@ You can test a preinstalled extension or test by installing an extension. If the
|
|
|
37
37
|
allowfullscreen
|
|
38
38
|
></iframe>
|
|
39
39
|
|
|
40
|
-
## Now
|
|
41
|
-
Once you have
|
|
40
|
+
## Now what?
|
|
41
|
+
Once you have installed your extension, simply create a TestDriver test to test any functionality a user would use.
|
|
@@ -11,9 +11,9 @@ TestDriver is designed to work with any desktop application, including Electron,
|
|
|
11
11
|
|
|
12
12
|
<TestPrereqs />
|
|
13
13
|
|
|
14
|
-
## Sample
|
|
14
|
+
## Sample test steps
|
|
15
15
|
|
|
16
|
-
## TestDriver in
|
|
16
|
+
## TestDriver in action
|
|
17
17
|
<iframe
|
|
18
18
|
width="560"
|
|
19
19
|
height="315"
|
|
@@ -24,7 +24,7 @@ TestDriver is designed to work with any desktop application, including Electron,
|
|
|
24
24
|
allowfullscreen
|
|
25
25
|
></iframe>
|
|
26
26
|
|
|
27
|
-
## Supported
|
|
27
|
+
## Supported desktop applications
|
|
28
28
|
TestDriver supports a wide range of desktop applications, including:
|
|
29
29
|
- Electron apps
|
|
30
30
|
- Java apps
|
|
@@ -33,11 +33,13 @@ TestDriver supports a wide range of desktop applications, including:
|
|
|
33
33
|
- Native Mac applications
|
|
34
34
|
- Native Linux applications
|
|
35
35
|
- Any other desktop application that can be run on Windows, Mac, or Linux
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
36
39
|
## Installation
|
|
37
40
|
To use TestDriver with desktop applications, no special setup is needed. Simply run the local agent to interact with applications on your local machine, or add instructions to `prerun.yaml` to tell the runner to launch or install the application. Here is an example that performs a calculation on a calculator app:
|
|
38
41
|
```yaml
|
|
39
42
|
version: 5.5.5
|
|
40
|
-
session: 67f00511acbd9ccac373edf7
|
|
41
43
|
steps:
|
|
42
44
|
- prompt: launch a calculator
|
|
43
45
|
commands:
|
|
@@ -61,7 +63,6 @@ steps:
|
|
|
61
63
|
A `calculator_test.yaml` file might look like this:
|
|
62
64
|
```yaml
|
|
63
65
|
version: 5.5.5
|
|
64
|
-
session: 6810141e0e064b42739922f2
|
|
65
66
|
steps:
|
|
66
67
|
- prompt: /try performing the operation 2 + 2 = on the calculator that is opened
|
|
67
68
|
commands:
|
|
@@ -4,16 +4,16 @@ sidebarTitle: "Mobile Apps"
|
|
|
4
4
|
description: "Test mobile apps with TestDriver"
|
|
5
5
|
icon: "mobile"
|
|
6
6
|
---
|
|
7
|
-
# Testing
|
|
7
|
+
# Testing mobile apps with TestDriver
|
|
8
8
|
|
|
9
|
-
## How
|
|
9
|
+
## How do I test mobile apps?
|
|
10
10
|
TestDriver provides a powerful solution for testing mobile apps. With our platform, you can easily create and run tests to ensure your mobile app functions as expected across different devices and operating systems.
|
|
11
11
|
|
|
12
12
|
|
|
13
13
|
|
|
14
14
|
<Tip>You will need to install a mobile emulator for your deployed tests or use a device farm to test your mobile app.</Tip>
|
|
15
15
|
|
|
16
|
-
##
|
|
16
|
+
## Gettins started
|
|
17
17
|
To get started with testing mobile apps using TestDriver, follow these steps:
|
|
18
18
|
1. **Follow our [quick start guide](/overview/quickstart) to set up your TestDriver account.**
|
|
19
19
|
2. **Create a new test project** in your TestDriver dashboard.
|
|
@@ -6,7 +6,7 @@ icon: "browser"
|
|
|
6
6
|
---
|
|
7
7
|
TestDriver provides a powerful solution for testing static websites. With our platform, you can easily create and run tests to ensure your static website functions as expected across different browsers and devices.
|
|
8
8
|
|
|
9
|
-
## TestDriver in
|
|
9
|
+
## TestDriver in action
|
|
10
10
|
<iframe
|
|
11
11
|
width="560"
|
|
12
12
|
height="315"
|
|
@@ -17,7 +17,7 @@ TestDriver provides a powerful solution for testing static websites. With our pl
|
|
|
17
17
|
allowfullscreen
|
|
18
18
|
></iframe>
|
|
19
19
|
|
|
20
|
-
## Getting
|
|
20
|
+
## Getting started
|
|
21
21
|
To get started with testing static websites using TestDriver, follow these steps:
|
|
22
22
|
1. **Follow our [quick start guide](/getting-started/setup) to set up your TestDriver account.**
|
|
23
23
|
2. **Create a new test project** in your TestDriver dashboard.
|
|
@@ -26,7 +26,6 @@ It looks like this in the `prerun.yaml` file:
|
|
|
26
26
|
|
|
27
27
|
```yaml
|
|
28
28
|
version: 5.1.1
|
|
29
|
-
session: 67f00511acbd9ccac373edf7
|
|
30
29
|
steps:
|
|
31
30
|
- prompt: launch chrome
|
|
32
31
|
commands:
|
package/docs/bugs/jira.mdx
CHANGED
|
@@ -5,13 +5,13 @@ description: "Step-by-step instructions to integrate TestDriver with Jira for au
|
|
|
5
5
|
icon: "jira"
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
# Automating Jira
|
|
8
|
+
# Automating Jira ticket creation for test failures using TestDriver and Jira GitHub Actions
|
|
9
9
|
|
|
10
10
|
This guide explains how to integrate the **TestDriver GitHub Action** with the **Jira GitHub Action** to automatically create a Jira ticket whenever a TestDriver test fails. This workflow ensures that test failures are tracked in Jira, enabling teams to address issues promptly.
|
|
11
11
|
|
|
12
12
|
---
|
|
13
13
|
|
|
14
|
-
## Workflow
|
|
14
|
+
## Workflow overview
|
|
15
15
|
|
|
16
16
|
<Steps>
|
|
17
17
|
<Step title="Run Tests with TestDriver">Use the TestDriver GitHub Action to execute your test suite.</Step>
|
|
@@ -22,19 +22,19 @@ This guide explains how to integrate the **TestDriver GitHub Action** with the *
|
|
|
22
22
|
|
|
23
23
|
## Prerequisites
|
|
24
24
|
|
|
25
|
-
1. **TestDriver API Key**: Store your API key as a GitHub secret (
|
|
26
|
-
2. **Jira API Token**: Generate an API token from your Jira account and store it as a GitHub secret (
|
|
27
|
-
3. **Jira Base URL**: Your Jira instance URL (
|
|
28
|
-
4. **Jira Project Key**: The key of the Jira project where tickets will be created (
|
|
25
|
+
1. **TestDriver API Key**: Store your API key as a GitHub secret (for example, `TD_API_KEY`).
|
|
26
|
+
2. **Jira API Token**: Generate an API token from your Jira account and store it as a GitHub secret (for example, `JIRA_API_TOKEN`).
|
|
27
|
+
3. **Jira Base URL**: Your Jira instance URL (for example, `https://yourcompany.atlassian.net`).
|
|
28
|
+
4. **Jira Project Key**: The key of the Jira project where tickets will be created (for example, `TEST`).
|
|
29
29
|
|
|
30
30
|
---
|
|
31
31
|
|
|
32
|
-
## GitHub Actions
|
|
32
|
+
## GitHub Actions workflow
|
|
33
33
|
|
|
34
34
|
Here's a complete workflow that integrates TestDriver and Jira:
|
|
35
35
|
|
|
36
|
-
### Workflow File: `.github/workflows/testdriver-jira.
|
|
37
|
-
```yaml
|
|
36
|
+
### Workflow File: `.github/workflows/testdriver-jira.yaml`
|
|
37
|
+
```yaml [expandable]
|
|
38
38
|
name: TestDriver with Jira Integration
|
|
39
39
|
|
|
40
40
|
on:
|
|
@@ -104,7 +104,7 @@ jobs:
|
|
|
104
104
|
|
|
105
105
|
---
|
|
106
106
|
|
|
107
|
-
## Workflow
|
|
107
|
+
## Workflow steps explained
|
|
108
108
|
|
|
109
109
|
### 1. **Run Tests with TestDriver**
|
|
110
110
|
The `testdriverai/action@main` step runs your TestDriver tests and captures the results. The `outputs.success` variable indicates whether the tests passed or failed.
|
|
@@ -124,7 +124,7 @@ The `testdriverai/action@main` step runs your TestDriver tests and captures the
|
|
|
124
124
|
|
|
125
125
|
---
|
|
126
126
|
|
|
127
|
-
### 2. **Check for
|
|
127
|
+
### 2. **Check for test failures**
|
|
128
128
|
This step checks the `outputs.success` variable from the TestDriver action. If the tests failed, it sets an environment variable (`failure=true`) to trigger the Jira ticket creation step.
|
|
129
129
|
```yaml
|
|
130
130
|
- name: Check for Test Failures
|
|
@@ -142,7 +142,7 @@ This step checks the `outputs.success` variable from the TestDriver action. If t
|
|
|
142
142
|
|
|
143
143
|
---
|
|
144
144
|
|
|
145
|
-
### 3. **Create a Jira
|
|
145
|
+
### 3. **Create a Jira ticket**
|
|
146
146
|
If any tests failed, the `create-jira-ticket` job uses the `atlassian/gajira-create` action to create a new Jira ticket. The ticket includes:
|
|
147
147
|
- **Summary**: A brief description of the failure.
|
|
148
148
|
- **Description**: Detailed information about the failure, including the test summary and markdown output from TestDriver.
|
|
@@ -171,17 +171,17 @@ If any tests failed, the `create-jira-ticket` job uses the `atlassian/gajira-cre
|
|
|
171
171
|
|
|
172
172
|
---
|
|
173
173
|
|
|
174
|
-
## Secrets
|
|
174
|
+
## Secrets configuration
|
|
175
175
|
|
|
176
176
|
Add the following secrets to your GitHub repository:
|
|
177
177
|
1. **`TD_API_KEY`**: Your TestDriver API key.
|
|
178
178
|
2. **`JIRA_API_TOKEN`**: Your Jira API token.
|
|
179
|
-
3. **`JIRA_BASE_URL`**: Your Jira instance URL (
|
|
179
|
+
3. **`JIRA_BASE_URL`**: Your Jira instance URL (for example, `https://yourcompany.atlassian.net`).
|
|
180
180
|
4. **`JIRA_USERNAME`**: Your Jira account email.
|
|
181
181
|
|
|
182
182
|
---
|
|
183
183
|
|
|
184
|
-
## Example Jira
|
|
184
|
+
## Example Jira ticket
|
|
185
185
|
|
|
186
186
|
### Summary:
|
|
187
187
|
`Test Failure: Login Test Failed`
|
|
@@ -204,7 +204,7 @@ Please investigate the issue and resolve it promptly.
|
|
|
204
204
|
```
|
|
205
205
|
---
|
|
206
206
|
|
|
207
|
-
## Benefits of
|
|
207
|
+
## Benefits of this workflow
|
|
208
208
|
|
|
209
209
|
1. **Automated Issue Tracking**: Automatically creates Jira tickets for test failures, ensuring no issues are overlooked.
|
|
210
210
|
2. **Detailed Context**: Includes test summaries and failure details in the Jira ticket for easier debugging.
|
package/docs/cli/overview.mdx
CHANGED
|
@@ -12,7 +12,7 @@ The `testdriverai` CLI is the primary interface for running, managing, and debug
|
|
|
12
12
|
testdriverai <command> [options]
|
|
13
13
|
```
|
|
14
14
|
|
|
15
|
-
## Available
|
|
15
|
+
## Available commands
|
|
16
16
|
| Command | Description |
|
|
17
17
|
|---------------|-----------------------------------------------------------------------------|
|
|
18
18
|
| `run` | Executes a TestDriver script. |
|
|
@@ -21,33 +21,33 @@ testdriverai <command> [options]
|
|
|
21
21
|
| `version` | Displays the current version of the TestDriver CLI. |
|
|
22
22
|
| `help` | Displays help information for the CLI or a specific command. |
|
|
23
23
|
|
|
24
|
-
## Example
|
|
24
|
+
## Example usage
|
|
25
25
|
|
|
26
|
-
### Running a
|
|
26
|
+
### Running a test script
|
|
27
27
|
```bash
|
|
28
28
|
testdriverai run path/to/testdriver.yaml
|
|
29
29
|
```
|
|
30
30
|
This command runs the specified TestDriver script, executing all steps defined in the file.
|
|
31
31
|
|
|
32
|
-
### Validating a
|
|
32
|
+
### Validating a script
|
|
33
33
|
```bash
|
|
34
34
|
testdriverai validate path/to/testdriver.yaml
|
|
35
35
|
```
|
|
36
|
-
This command checks the syntax and structure of the script to ensure it
|
|
36
|
+
This command checks the syntax and structure of the script to ensure it's valid before execution.
|
|
37
37
|
|
|
38
|
-
### Initializing a
|
|
38
|
+
### Initializing a new project
|
|
39
39
|
```bash
|
|
40
40
|
testdriverai init
|
|
41
41
|
```
|
|
42
42
|
This command sets up a new TestDriver project in the current directory, creating the necessary folder structure and configuration files.
|
|
43
43
|
|
|
44
|
-
### Checking the
|
|
44
|
+
### Checking the version
|
|
45
45
|
```bash
|
|
46
46
|
testdriverai version
|
|
47
47
|
```
|
|
48
48
|
This command displays the installed version of the TestDriver CLI.
|
|
49
49
|
|
|
50
|
-
### Getting
|
|
50
|
+
### Getting help
|
|
51
51
|
```bash
|
|
52
52
|
testdriverai help
|
|
53
53
|
```
|