testdriverai 5.6.3 → 5.6.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (98) hide show
  1. package/agent.js +17 -15
  2. package/docs/account/dashboard.mdx +3 -6
  3. package/docs/account/enterprise.mdx +16 -15
  4. package/docs/account/pricing.mdx +4 -4
  5. package/docs/account/projects.mdx +2 -2
  6. package/docs/account/team.mdx +3 -3
  7. package/docs/action/browser.mdx +9 -9
  8. package/docs/action/os.mdx +12 -12
  9. package/docs/action/output.mdx +10 -10
  10. package/docs/action/performance.mdx +11 -11
  11. package/docs/{guide/lifecycle.mdx → action/pre-post-scripts.mdx} +87 -69
  12. package/docs/action/secrets.mdx +11 -11
  13. package/docs/action/setup.mdx +8 -8
  14. package/docs/apps/chrome-extensions.mdx +9 -9
  15. package/docs/apps/desktop-apps.mdx +6 -5
  16. package/docs/apps/mobile-apps.mdx +3 -3
  17. package/docs/apps/static-websites.mdx +2 -3
  18. package/docs/bugs/jira.mdx +16 -16
  19. package/docs/cli/overview.mdx +8 -8
  20. package/docs/commands/exec.mdx +152 -10
  21. package/docs/commands/focus-application.mdx +1 -1
  22. package/docs/commands/hover-image.mdx +1 -1
  23. package/docs/commands/hover-text.mdx +2 -2
  24. package/docs/commands/if.mdx +1 -1
  25. package/docs/commands/issues.mdx +8 -6
  26. package/docs/commands/match-image.mdx +2 -2
  27. package/docs/commands/press-keys.mdx +19 -2
  28. package/docs/commands/run.mdx +1 -1
  29. package/docs/commands/scroll-until-image.mdx +1 -1
  30. package/docs/commands/scroll-until-text.mdx +2 -2
  31. package/docs/commands/scroll.mdx +2 -2
  32. package/docs/commands/type.mdx +1 -1
  33. package/docs/commands/wait-for-image.mdx +1 -1
  34. package/docs/commands/wait-for-text.mdx +1 -1
  35. package/docs/commands/wait.mdx +1 -1
  36. package/docs/docs.json +9 -9
  37. package/docs/exporting/playwright.mdx +16 -16
  38. package/docs/features/auto-healing.mdx +5 -5
  39. package/docs/features/cross-platform.mdx +4 -4
  40. package/docs/features/generation.mdx +7 -4
  41. package/docs/features/github.mdx +21 -21
  42. package/docs/features/parallel-testing.mdx +9 -9
  43. package/docs/features/reusable-snippets.mdx +7 -7
  44. package/docs/features/selectorless.mdx +4 -4
  45. package/docs/features/visual-assertions.mdx +26 -23
  46. package/docs/getting-started/ci.mdx +19 -19
  47. package/docs/getting-started/editing.mdx +12 -8
  48. package/docs/getting-started/generating.mdx +4 -5
  49. package/docs/getting-started/running.mdx +3 -3
  50. package/docs/getting-started/setup.mdx +9 -9
  51. package/docs/getting-started/vscode.mdx +63 -85
  52. package/docs/getting-started/writing.mdx +11 -7
  53. package/docs/guide/assertions.mdx +21 -21
  54. package/docs/guide/authentication.mdx +11 -11
  55. package/docs/guide/code.mdx +6 -6
  56. package/docs/guide/environment-variables.mdx +9 -9
  57. package/docs/guide/lifecycle-prerun.mdx +191 -0
  58. package/docs/guide/locating.mdx +15 -18
  59. package/docs/guide/variables.mdx +8 -8
  60. package/docs/guide/waiting.mdx +10 -10
  61. package/docs/importing/csv.mdx +23 -24
  62. package/docs/importing/gherkin.mdx +20 -20
  63. package/docs/importing/jira.mdx +22 -23
  64. package/docs/importing/testrail.mdx +17 -17
  65. package/docs/integrations/electron.mdx +7 -7
  66. package/docs/integrations/netlify.mdx +7 -7
  67. package/docs/integrations/vercel.mdx +13 -13
  68. package/docs/interactive/assert.mdx +3 -3
  69. package/docs/interactive/dry.mdx +7 -7
  70. package/docs/interactive/explore.mdx +9 -9
  71. package/docs/interactive/generate.mdx +2 -2
  72. package/docs/interactive/run.mdx +10 -10
  73. package/docs/interactive/save.mdx +3 -3
  74. package/docs/interactive/undo.mdx +1 -1
  75. package/docs/overview/comparison.mdx +6 -6
  76. package/docs/overview/faq.mdx +2 -2
  77. package/docs/overview/quickstart.mdx +1 -1
  78. package/docs/overview/upgrading.mdx +8 -4
  79. package/docs/overview/what-is-testdriver.mdx +6 -6
  80. package/docs/quickstart.mdx +1 -1
  81. package/docs/scenarios/ai-chatbot.mdx +3 -3
  82. package/docs/scenarios/cookie-banner.mdx +2 -2
  83. package/docs/scenarios/file-upload.mdx +2 -2
  84. package/docs/scenarios/form-filling.mdx +3 -3
  85. package/docs/scenarios/log-in.mdx +3 -3
  86. package/docs/scenarios/pdf-generation.mdx +3 -3
  87. package/docs/scenarios/spell-check.mdx +2 -2
  88. package/docs/security/action.mdx +6 -6
  89. package/docs/security/agent.mdx +10 -10
  90. package/docs/security/platform.mdx +10 -8
  91. package/docs/snippets/test-prereqs.mdx +1 -1
  92. package/docs/snippets/yml-warning.mdx +1 -0
  93. package/docs/tutorials/advanced-test.mdx +9 -9
  94. package/docs/tutorials/basic-test.mdx +3 -3
  95. package/package.json +1 -1
  96. package/styles/config/vocabularies/Docs/accept.txt +32 -4
  97. package/testdriver/spotify.yaml +5 -0
  98. package/docs/action/prerun.mdx +0 -137
@@ -21,7 +21,7 @@ The `/generate` command is an experimental feature that instructs TestDriver to
21
21
  - TestDriver generates 10 Markdown files every time the `/generate` command is called.
22
22
  - The generated files are stored in the `./testdriver/generate/*.md` directory.
23
23
 
24
- ## Example Output
24
+ ## Example output
25
25
  Here's an example of a generated test file (`test-search-function.md`):
26
26
  ```markdown
27
27
  1. Click on the search icon.
@@ -31,7 +31,7 @@ Here's an example of a generated test file (`test-search-function.md`):
31
31
 
32
32
  ## Protips
33
33
  - Use `/generate` to quickly create exploratory tests for regression testing or feature validation.
34
- - Combine `/generate` with the GitHub action to generate and run regression tests in parallel.
34
+ - Combine `/generate` with the GitHub Action to generate and run regression tests in parallel.
35
35
  - Review the generated Markdown files and convert them into YAML test scripts for reuse.
36
36
 
37
37
  ## Gotchas
@@ -1,39 +1,39 @@
1
1
  ---
2
2
  title: "/run"
3
3
  sidebarTitle: "/run"
4
- description: "Run a test plan from a file."
4
+ description: "Run a test file from a file."
5
5
  icon: "play"
6
6
  ---
7
7
 
8
8
  ## Description
9
- The `/run` command is used to execute a test plan from a specified file. This command performs each step defined in the test plan and outputs the results.
9
+ The `/run` command is used to execute a test file from a specified file. This command performs each step defined in the test file and outputs the results.
10
10
 
11
11
  ## Usage
12
12
  ```bash
13
13
  /run <file>
14
14
  ```
15
15
 
16
- ## Example Usage
16
+ ## Example usage
17
17
  ```bash
18
18
  testdriverai
19
- > /run helloworld.yml
19
+ > /run helloworld.yaml
20
20
  ```
21
- This command runs the `helloworld.yml` test plan, executing each command in the file sequentially.
21
+ This command runs the `helloworld.yaml` test file, executing each command in the file sequentially.
22
22
 
23
23
  ## Behavior
24
- - TestDriver will execute the test plan, performing each command as defined in the file.
24
+ - TestDriver will execute the test file, performing each command as defined in the file.
25
25
  - If the test completes successfully, the program will exit with code `0`.
26
26
  - If any failures occur during the test, the program will output the errors and exit with code `1`.
27
27
 
28
28
  ## Protips
29
29
  - Ensure the test file path is correct and accessible before running the command.
30
- - Use descriptive filenames for your test plans to make them easier to identify.
30
+ - Use descriptive filenames for your test files to make them easier to identify.
31
31
  - Combine `/run` with debugging tools to troubleshoot failing tests.
32
32
 
33
33
  ## Gotchas
34
34
  - This command will exit the program upon execution, so ensure all necessary setup is complete before running it.
35
- - Any errors in the test plan (e.g., invalid commands or missing arguments) will cause the test to fail.
35
+ - Any errors in the test file (for example, invalid commands or missing arguments) will cause the test to fail.
36
36
 
37
37
  ## Notes
38
- - The `/run` command is ideal for executing pre-created test plans in an interactive session.
39
- - Use this command to validate and debug your test plans during development.
38
+ - The `/run` command is ideal for executing pre-created test files in an interactive session.
39
+ - Use this command to validate and debug your test files during development.
@@ -13,7 +13,7 @@ The `/save` command saves the current state of the test script to a file. This c
13
13
  /save
14
14
  ```
15
15
 
16
- ## Example Usage
16
+ ## Example usage
17
17
  ```bash
18
18
  testdriverai
19
19
  > /save
@@ -41,7 +41,7 @@ testdriverai
41
41
 
42
42
  ## Behavior
43
43
  - The `/save` command generates a YAML file with the current test script, including all executed steps and commands.
44
- - The file can be used as a reusable test plan for future executions.
44
+ - The file can be used as a reusable test file for future executions.
45
45
 
46
46
  ## Protips
47
47
  - Use `/save` frequently during interactive sessions to preserve your progress and avoid losing work.
@@ -49,7 +49,7 @@ testdriverai
49
49
 
50
50
  ## Gotchas
51
51
  - Ensure you have write permissions in the directory where the file will be saved.
52
- - The saved script reflects the current state of the session. Any unexecuted commands will not be included.
52
+ - The saved script reflects the current state of the session. Any unexecuted commands won't be included.
53
53
 
54
54
  ## Notes
55
55
  - The `/save` command is ideal for creating reusable test scripts from interactive sessions.
@@ -17,7 +17,7 @@ The `/undo` command removes the last generated command or step from the current
17
17
  - The `/undo` command removes the most recently added command or step from the test script.
18
18
  - You can call `/undo` multiple times to remove multiple commands or steps.
19
19
 
20
- ## Example Usage
20
+ ## Example usage
21
21
 
22
22
  ### Before `/undo`
23
23
  ```yaml
@@ -5,7 +5,7 @@ description: "TestDriver vs Playwright vs Selenium"
5
5
  icon: boxing-glove
6
6
  ---
7
7
 
8
- ### Application Support
8
+ ### Application support
9
9
 
10
10
  TestDriver operates a full desktop environment, so it can run any application.
11
11
 
@@ -16,7 +16,7 @@ TestDriver operates a full desktop environment, so it can run any application.
16
16
  | Chrome Extensions | ✅ | | |
17
17
 
18
18
 
19
- ### Testing Features
19
+ ### Testing features
20
20
 
21
21
  TestDriver is AI first.
22
22
 
@@ -31,7 +31,7 @@ TestDriver is AI first.
31
31
  | Team Dashboard | ✅ | | |
32
32
  | Team Collaboration | ✅ | | |
33
33
 
34
- ### Test Coverage
34
+ ### Test coverage
35
35
 
36
36
  TestDriver has more coverage than selector-based frameworks.
37
37
 
@@ -49,7 +49,7 @@ TestDriver has more coverage than selector-based frameworks.
49
49
  | `<canvas>` | ✅ | | |
50
50
  | `<video>` | ✅ | | |
51
51
 
52
- ### Debugging Features
52
+ ### Debugging features
53
53
 
54
54
  Debugging features are powered by [Dashcam.io](https://dashcam.io).
55
55
 
@@ -63,7 +63,7 @@ Debugging features are powered by [Dashcam.io](https://dashcam.io).
63
63
  | Team Dashboard | ✅ | | |
64
64
  | Team Collaboration | ✅ | | |
65
65
 
66
- ### Web Browser Support
66
+ ### Web browser support
67
67
 
68
68
  TestDriver is browser agnostic and supports any version of any browser.
69
69
 
@@ -77,7 +77,7 @@ TestDriver is browser agnostic and supports any version of any browser.
77
77
  | Opera | ✅ | | ✅ |
78
78
  | Safari | ✅ | | ✅ |
79
79
 
80
- ### Operating System Support
80
+ ### Operating system support
81
81
 
82
82
  TestDriver currently supports Mac and Windows!
83
83
 
@@ -6,7 +6,7 @@ icon: "block-question"
6
6
  ---
7
7
 
8
8
 
9
- ## What is TestDriver?
9
+ ## What's TestDriver?
10
10
  TestDriver is an AI-powered testing platform that simulates user interactions to automate end-to-end testing for web, desktop, and mobile applications.
11
11
 
12
12
  ## How does TestDriver work?
@@ -67,7 +67,7 @@ Yes. TestDriver supports parallel execution using multiple VMs or containers.
67
67
  Yes. It can log CPU, memory, load times, and frame rates to help catch performance regressions.
68
68
 
69
69
  ## Can it validate non-deterministic output?
70
- Yes. It uses AI assertions to verify outcomes even when outputs vary (e.g., generated text or dynamic UIs).
70
+ Yes. It uses AI assertions to verify outcomes even when outputs vary (for example, generated text or dynamic UIs).
71
71
 
72
72
  ## Can it test workflows with variable inputs?
73
73
  Yes. It supports data-driven tests using parameterized inputs.
@@ -217,4 +217,4 @@ TestDriver can also generate tests without any input! Instead, it will suggest t
217
217
 
218
218
  </Step>
219
219
 
220
- </Steps>
220
+ </Steps>
@@ -5,7 +5,9 @@ description: "Learn how to upgrade TestDriver for the latest features and improv
5
5
  icon: "circle-up"
6
6
  ---
7
7
 
8
- ## How to Upgrade TestDriver
8
+ import YmlWarning from '/snippets/yml-warning.mdx'
9
+
10
+ ## How to upgrade TestDriver
9
11
  To upgrade TestDriver, follow these steps:
10
12
  1. **Check for Updates**: Regularly check the [TestDriver NPM Package](https://www.npmjs.com/package/testdriverai)
11
13
  2. **Update Command**: Use the following command to update TestDriver to the latest version:
@@ -17,10 +19,12 @@ To upgrade TestDriver, follow these steps:
17
19
  npm install -g testdriverai@beta
18
20
  ```
19
21
 
20
- In your existing Github Action setup, you may need to update the `version` field in the `workflows/testdriver.yaml` file(s) to match the version of TestDriver you want to use. Leaving it blank will run `latest` by default. Here is an example of what to replace:
22
+ <YmlWarning/>
23
+
24
+ In your existing GitHub Actions setup, you may need to update the `version` field in the `workflows/testdriver.yaml` file(s) to match the version of TestDriver you want to use. Leaving it blank will run `latest` by default. Here is an example of what to replace:
21
25
 
22
26
  ```yaml {16}
23
- name: TestDriver.ai
27
+ name: TestDriver
24
28
 
25
29
  on:
26
30
  push:
@@ -56,7 +60,7 @@ jobs:
56
60
  Would become:
57
61
 
58
62
  ```yaml {16}
59
- name: TestDriver.ai
63
+ name: TestDriver
60
64
 
61
65
  on:
62
66
  push:
@@ -1,6 +1,6 @@
1
1
  ---
2
- title: "What is TestDriver?"
3
- sidebarTitle: "What is TestDriver?"
2
+ title: "What's TestDriver?"
3
+ sidebarTitle: "What's TestDriver?"
4
4
  description: "TestDriver is a computer-use agent for QA testing of user interfaces."
5
5
  icon: "circle-info"
6
6
  ---
@@ -21,9 +21,9 @@ TestDriver uses AI vision and keyboard and mouse control to automate end-to-end
21
21
 
22
22
  TestDriver is different from other computer-use agents in that it produces a `YAML` test script that increases the speed and repeatability of testing.
23
23
 
24
- ### Selectorless Testing
24
+ ### Selectorless testing
25
25
 
26
- Unlike traditional frameworks (e.g., Selenium, Playwright), TestDriver doesn't rely on CSS selectors or static analysis. Instead, tests are described in plain English, such as:
26
+ Unlike traditional frameworks (for example, Selenium, Playwright), TestDriver doesn't rely on CSS selectors or static analysis. Instead, tests are described in plain English, such as:
27
27
 
28
28
  ```
29
29
  > Open Google Chrome and search for "testdriver"
@@ -60,7 +60,7 @@ Old and Busted (Selectors) | New Hotness (TestDriver)
60
60
  |:----------------------|:-------------------|
61
61
  ```div[class="product-card"] >> text="Add to Cart" >> nth=2``` | buy the 2nd product
62
62
 
63
- These high level instructions are easier to create and maintain because they are loosely coupled from the codebase. We're describing a high level goal, not a low level interaction.
63
+ These high level instructions are easier to create and maintain because they're loosely coupled from the codebase. We're describing a high level goal, not a low level interaction.
64
64
 
65
65
  The tests will still continue to work even when the junior developer changes .product-card to .product.card or the designers change Add to Cart to Buy Now . The concepts remain the same so the AI will adapt.
66
66
 
@@ -68,7 +68,7 @@ The tests will still continue to work even when the junior developer changes .pr
68
68
 
69
69
  TestDriver uses a combination of reinforcement learning and computer vision. The context from successful text executions inform future executions. Here's an example of the context our model considers when locating a text match:
70
70
 
71
- Context| What is it? | Touchpoint
71
+ Context| What's it? | Touchpoint
72
72
  |:----------|:------------------|:----------------|
73
73
  Prompt | Desired outcome | User Input
74
74
  Screenshot | Image of computer desktop | Runtime
@@ -57,7 +57,7 @@ TestDriver will generate steps like:
57
57
  <Step title="Run Saved Tests">
58
58
  Once saved, you can run your test file anytime:
59
59
  ```bash
60
- testdriverai run path/to/test.yml
60
+ testdriverai run path/to/test.yaml
61
61
  ```
62
62
  </Step>
63
63
 
@@ -1,6 +1,6 @@
1
1
  ---
2
- title: "AI Chatbot"
3
- sidebarTitle: "AI Chatbot"
2
+ title: "AI Chatbots"
3
+ sidebarTitle: "AI Chatbots"
4
4
  description: "Integrate AI chatbots into your testing workflow."
5
5
  icon: "message-bot"
6
6
  ---
@@ -18,7 +18,7 @@ This scenario is an example of how to set up and run tests for an AI chatbot usi
18
18
 
19
19
  <TestPrereqs />
20
20
 
21
- ## Scenario Overview
21
+ ## Scenario overview
22
22
  1. Load a website with an AI chatbot.
23
23
  2. Type a chat message or ask a question.
24
24
  3. Wait for the chatbot to respond.
@@ -11,14 +11,14 @@ With TestDriver, you can automate the testing of cookie banners on your web appl
11
11
 
12
12
  <TestPrereqs />
13
13
 
14
- ## Scenario Overview
14
+ ## Scenario overview
15
15
  1. **Visit the Site**: The test will navigate to the target URL.
16
16
  2. **Check for Cookie Banner**: It will verify if the cookie banner is displayed.
17
17
  3. **Decline Cookies**: The test will simulate a user clicking the "Decline All" button on the cookie banner.
18
18
 
19
19
 
20
20
 
21
- ## TestDriver in Action
21
+ ## TestDriver in action
22
22
  <iframe
23
23
  width="560"
24
24
  height="315"
@@ -11,14 +11,14 @@ This scenario demonstrates how to automate testing the file upload functionality
11
11
 
12
12
  <TestPrereqs />
13
13
 
14
- ## Scenario Overview
14
+ ## Scenario overview
15
15
  1. Load a website with a file upload feature.
16
16
  2. Generate a file that will be used for the upload.
17
17
  3. Use TestDriver to perform the file upload process.
18
18
  4. Check if the uploaded file is processed correctly by the application.
19
19
 
20
20
 
21
- ## TestDriver in Action
21
+ ## TestDriver in action
22
22
  <iframe
23
23
  width="560"
24
24
  height="315"
@@ -9,14 +9,14 @@ import TestPrereqs from '/snippets/test-prereqs.mdx'
9
9
 
10
10
  <TestPrereqs />
11
11
 
12
- ## Scenario Overview
12
+ ## Scenario overview
13
13
  1. Open a webpage containing a form (Log in, Registration, etc.).
14
14
  2. Use TestDriver to fill in the form fields with test data.
15
- 3. Submit the form and verify the expected outcome (e.g., successful login, registration confirmation, redirect etc.).
15
+ 3. Submit the form and verify the expected outcome (for example, successful login, registration confirmation, redirect etc.).
16
16
 
17
17
 
18
18
 
19
- ## TestDriver in Action
19
+ ## TestDriver in action
20
20
  <iframe
21
21
  width="560"
22
22
  height="315"
@@ -1,5 +1,5 @@
1
1
  ---
2
- title: "Login Scenario"
2
+ title: "Login"
3
3
  sidebarTitle: "Login"
4
4
  description: "Test login functionality with TestDriver"
5
5
  icon: "lock"
@@ -11,7 +11,7 @@ Test login functionality with TestDriver. This scenario demonstrates how to auto
11
11
 
12
12
  <TestPrereqs />
13
13
 
14
- ## Scenario Overview
14
+ ## Scenario overview
15
15
  1. Visit the login page of the web application.
16
16
  2. Enter the username and password into the respective fields (see [Reusable Snippets](/features/reusable-snippets#How-to Create-and-Use-Reusable-Snippets)).
17
17
  3. Click the "Login" button.
@@ -20,7 +20,7 @@ Test login functionality with TestDriver. This scenario demonstrates how to auto
20
20
 
21
21
 
22
22
  ### Setup your test environment with predefined credentials.
23
- In this example, we will use GitHub secret management to store our credentials.
23
+ This example uses GitHub secret management to store credentials.
24
24
 
25
25
  <Tip>You can also use the [`exec`](commands/exec) command to fetch a test user dynamically from a database or API.</Tip>
26
26
 
@@ -11,11 +11,11 @@ Test PDF generation functionality with TestDriver. This scenario demonstrates ho
11
11
 
12
12
  <TestPrereqs />
13
13
 
14
- ## Scenario Overview
14
+ ## Scenario overview
15
15
  1. Open a webpage or desktop app that will allow you to generate a PDF.
16
- 2. Trigger the PDF generation process (e.g., by clicking a button).
16
+ 2. Trigger the PDF generation process (for example, by clicking a button).
17
17
  3. Verify that the PDF is generated successfully and contains the expected content.
18
18
  4. Optionally, check the file size and format of the generated PDF.
19
19
 
20
- ## What Next?
20
+ ## What's next
21
21
  From here you can automate and add coverage to your PDF generation process, removing manual testing from the equation.
@@ -11,10 +11,10 @@ This scenario demonstrates how to automate testing the spell check functionality
11
11
 
12
12
  <TestPrereqs />
13
13
 
14
- ## Scenario Overview
14
+ ## Scenario overview
15
15
  1. Open up a webpage that has text input and a spell check function.
16
16
  2. Enter a misspelled word into the text input field.
17
- 3. Trigger the spell check function (e.g., by clicking a button or losing focus on the input field).
17
+ 3. Trigger the spell check function (for example, by clicking a button or losing focus on the input field).
18
18
  4. Verify that the spell checker identifies the misspelled word and provides suggestions for correction.
19
19
  5. Optionally, select a suggestion and apply it to the text input field.
20
20
 
@@ -7,12 +7,12 @@ icon: shield-check
7
7
 
8
8
  import GitignoreWarning from '/snippets/gitignore-warning.mdx';
9
9
 
10
- ## Open Source
10
+ ## Open source
11
11
  The TestDriver Action is open source, and its source code is available for review. You can find the repository here:
12
12
 
13
13
  <Card horizontal="true" icon="github"> [GitHub - testdriverai/action](https://github.com/testdriverai/action) </Card>
14
14
 
15
- ## Ephemeral Virtual Machine Runners
15
+ ## Ephemeral virtual machine runners
16
16
  TestDriver tests are executed on private virtual machines (VMs) managed by Amazon EC2. These VMs are ephemeral, meaning they only exist for the duration of the test execution. Once the test is complete:
17
17
  - The VM is destroyed.
18
18
  - The hard disk is wiped to ensure no residual data remains.
@@ -20,15 +20,15 @@ TestDriver tests are executed on private virtual machines (VMs) managed by Amazo
20
20
  ## Secrets
21
21
  To securely manage private information, we recommend storing sensitive data as secrets in your GitHub repository. Learn more about [storing secrets in GitHub](https://docs.github.com/en/actions/security-guides/encrypted-secrets).
22
22
 
23
- ### Handling Secrets
23
+ ### Handling secrets
24
24
  - **Prerun Scripts**: Any secrets supplied within prerun scripts or prompts are transmitted over SSL to the TestDriver API. Prerun scripts are **not persisted**.
25
25
  - **Agent Prompts**: Secrets supplied to agent prompts are persisted (see [Agent Security](/security/agent)).
26
26
  - **Secure Workflows**: If your workflow requires secret sharing and you encounter issues, please contact us for assistance.
27
27
 
28
- ### Common Use Case
28
+ ### Common use case
29
29
  A common workflow involves using prerun scripts to securely access a private staging website via basic authentication. This allows you to log into staging environments without persisting sensitive data on TestDriver servers.
30
30
 
31
- ## Environment-Specific Security
31
+ ## Environment-specific security
32
32
 
33
33
  <GitignoreeWarning/>
34
34
 
@@ -46,7 +46,7 @@ Testing staging environments may require secure information, such as credentials
46
46
  ### Development
47
47
  TestDriver can clone feature branches and build code on its virtual machines using workflows similar to GitHub Actions.
48
48
 
49
- #### GitHub Token for Development
49
+ #### GitHub token for development
50
50
  To test development branches of private codebases, you must supply a GitHub personal access token within the GitHub Action. This token is used to:
51
51
  - Clone the codebase onto the VM.
52
52
  - Build and test the code in an isolated environment.
@@ -13,7 +13,7 @@ The TestDriver Agent doesn't contain any AI models within it. Instead, it upload
13
13
 
14
14
  The TestDriver API leverages OpenAI models behind the scenes. For more information about OpenAI and its privacy practices, visit their [privacy center](https://openai.com/privacy).
15
15
 
16
- ## Desktop Context Collected
16
+ ## Desktop context collected
17
17
  During execution, the TestDriver Agent uploads the following information to the API:
18
18
 
19
19
  - **User Input Prompts**: The commands and prompts you provide to TestDriver.
@@ -22,30 +22,30 @@ During execution, the TestDriver Agent uploads the following information to the
22
22
  - **Mouse Position**: The current position of the mouse pointer.
23
23
  - **Desktop Screenshots**: Screenshots of the primary display.
24
24
 
25
- ### Data Persistence
25
+ ### Data persistence
26
26
  - **Desktop Context**: All collected desktop context, except for screenshots, is persisted in the TestDriver database.
27
27
  - **Desktop Screenshots**: Screenshots are uploaded to the server for processing but aren't stored in the database.
28
28
 
29
- ### Desktop Screenshots
29
+ ### Desktop screenshots
30
30
  TestDriver frequently captures screenshots of the desktop to provide the AI with decision-making context. These screenshots are uploaded to the API but aren't persisted in the database.
31
31
 
32
32
  - **Primary Display Only**: Screenshots are limited to the primary display.
33
- - **Privacy Recommendation**: For complete privacy, it is recommended to run TestDriver within a virtual machine on your desktop.
33
+ - **Privacy Recommendation**: For complete privacy, it's recommended to run TestDriver within a virtual machine on your desktop.
34
34
 
35
35
  **Important**: TestDriver can't operate without visual context. don't install TestDriver if you are uncomfortable with capturing images of the desktop.
36
36
 
37
- ## Collected Data Details
37
+ ## Collected data details
38
38
 
39
- ### Active Window
39
+ ### Active window
40
40
  Information about the open windows on the desktop is reported using the `active-window` module. This includes details such as the application name and window titles.
41
41
 
42
- ### System Information
42
+ ### System information
43
43
  System details, such as operating system, CPU, and memory, are reported using the `systeminformation` module.
44
44
 
45
- ### User Prompts
45
+ ### User prompts
46
46
  The prompts you input into TestDriver are uploaded to the API and stored in a database. This data is used to provide the AI with a history of context for better decision-making.
47
47
 
48
- ## Additional Analytics
48
+ ## Additional analytics
49
49
  When running `testdriver init`, you will be asked if you would like to share additional analytics. Sharing usage analytics is **opt-in**, and no extra data will be collected unless explicitly enabled.
50
50
 
51
51
  To disable additional analytics, set the `TD_ANALYTICS` environment variable to `false`:
@@ -53,7 +53,7 @@ To disable additional analytics, set the `TD_ANALYTICS` environment variable to
53
53
  TD_ANALYTICS=false
54
54
  ```
55
55
 
56
- ## Rate Limiting and Restrictions
56
+ ## Rate limiting and restrictions
57
57
  While the TestDriver Agent is free to use, TestDriver reserves the right to rate limit or restrict usage by IP address for any reason.
58
58
 
59
59
  ## Notes
@@ -12,19 +12,19 @@ The TestDriver web dashboard provides a secure interface for managing and review
12
12
 
13
13
  Dashcam and TestDriver share the same API and web application back end, which includes robust privacy and security features.
14
14
 
15
- ## Security Features
15
+ ## Security features
16
16
 
17
17
  ### SSL
18
18
  - All data transmitted between your browser and the TestDriver web application is encrypted using HTTPS.
19
19
 
20
- ### OAuth Authentication
20
+ ### Authentication
21
21
  - Users can only authenticate via OAuth, provided by Auth0, ensuring secure and reliable user authentication.
22
22
 
23
- ### Team Management
23
+ ### Team management
24
24
  - Administrators can add or remove individual team members.
25
25
  - Only administrators have the ability to manage team settings.
26
26
 
27
- ### Role-Based Access Control (RBAC)
27
+ ### Role-based access control (RBAC)
28
28
  - The first user to create a team is designated as the administrator.
29
29
  - Administrators:
30
30
  - Are the only users who can view the API key.
@@ -32,9 +32,9 @@ Dashcam and TestDriver share the same API and web application back end, which in
32
32
  - can't be removed from the team.
33
33
  - All other users are normal members with limited access.
34
34
 
35
- ### API Key Rotation
35
+ ### API key rotation
36
36
  - Teams can rotate their API key for enhanced security.
37
- - It is recommended to rotate the API key every 90 days to minimize risk.
37
+ - It's recommended to rotate the API key every 90 days to minimize risk.
38
38
 
39
39
  <Check>
40
40
  For more details on **Team Management** see the [Team documentation](/account/team).
@@ -42,7 +42,7 @@ Dashcam and TestDriver share the same API and web application back end, which in
42
42
 
43
43
  <GitignoreWarning/>
44
44
 
45
- ### Secret Masking
45
+ ### Secret masking
46
46
  - Test replay logs and network requests are automatically scanned for sensitive information, such as:
47
47
  - Credit card numbers
48
48
  - Emails
@@ -50,7 +50,7 @@ Dashcam and TestDriver share the same API and web application back end, which in
50
50
  - API keys
51
51
  - Detected secrets are masked with asterisks (`****`) to prevent exposure.
52
52
 
53
- ### Encrypted At Rest
53
+ ### Encrypted at rest
54
54
  - Test replays and logs are securely stored on Amazon S3 and encrypted at rest.
55
55
  - Test results are only accessible via temporary signed URLs.
56
56
  - Signed URLs are generated exclusively for team users and expire after a set duration.
@@ -59,3 +59,5 @@ Dashcam and TestDriver share the same API and web application back end, which in
59
59
  - The TestDriver web dashboard is designed with privacy and security as top priorities.
60
60
  - For additional security, ensure your team rotates API keys regularly and reviews team member access permissions.
61
61
  - If you have specific security concerns or questions, please contact TestDriver support.
62
+
63
+ <Tip>See complete docs about the dashboard features [here](/account/dashboard)</Tip>
@@ -1,7 +1,7 @@
1
1
  ## Prerequisites
2
2
  Before running the tests, ensure you have performed the following steps:
3
3
 
4
- 1. Install the [VS Code Extension](/getting-started/vscode) for TestDriver
4
+ 1. Install the [VS Code extension](/getting-started/vscode) for TestDriver
5
5
  2. If you haven't already, signup for a Free Trial on the [TestDriver website](https://testdriver.ai/pricing)
6
6
  3. Install the [TestDriver CLI](/cli/overview) globally using npm or just follow the VS Code Extension Setup Walkthrough:
7
7
  ```bash
@@ -0,0 +1 @@
1
+ <Warning>TestDriver v5 uses the `.yaml` file extension for all test files. This is a breaking change from v4 which uses the `.yml` extension!</Warning>
@@ -1,11 +1,11 @@
1
1
  ---
2
- title: "Tutorial: Advanced Test"
2
+ title: "Tutorial: Advanced test"
3
3
  description: "Explore advanced testing techniques and features in TestDriver."
4
- sidebarTitle: "Advanced Test"
4
+ sidebarTitle: "Advanced test"
5
5
  icon: "flask-round"
6
6
  ---
7
7
 
8
- # Advanced Test Tutorial
8
+ # Advanced test tutorial
9
9
 
10
10
  Welcome to the Advanced Test Tutorial. In this guide, we will explore advanced testing techniques and features available in TestDriver. This tutorial assumes you have a basic understanding of TestDriver and its functionalities.
11
11
 
@@ -16,7 +16,7 @@ Before starting, ensure you have completed the basic TestDriver tutorial and hav
16
16
  - TestDriver installed and configured.
17
17
  - A sample project to test.
18
18
 
19
- ## Advanced Features Overview
19
+ ## Advanced features overview
20
20
 
21
21
  TestDriver provides several advanced features to enhance your testing capabilities:
22
22
 
@@ -25,9 +25,9 @@ TestDriver provides several advanced features to enhance your testing capabiliti
25
25
  3. **Test Hooks**: Use hooks to execute code before or after tests.
26
26
  4. **Parallel Execution**: Speed up testing by running tests in parallel.
27
27
 
28
- ## Step-by-Step Guide
28
+ ## Step-by-step guide
29
29
 
30
- ### Step 1: Setting Up Custom Assertions
30
+ ### Step 1: Setting up custom assertions
31
31
 
32
32
  Custom assertions allow you to define specific conditions for your tests. Here's an example:
33
33
 
@@ -36,7 +36,7 @@ def assert_custom_condition(value):
36
36
  assert value > 0, "Value must be greater than 0"
37
37
  ```
38
38
 
39
- ### Step 2: Using Parameterized Tests
39
+ ### Step 2: Using parameterized tests
40
40
 
41
41
  Parameterized tests enable you to test multiple scenarios with different inputs. Example:
42
42
 
@@ -48,7 +48,7 @@ def test_is_positive(input, expected):
48
48
  assert (input > 0) == expected
49
49
  ```
50
50
 
51
- ### Step 3: Implementing Test Hooks
51
+ ### Step 3: Implementing test hooks
52
52
 
53
53
  Test hooks let you execute code before or after tests. Example:
54
54
 
@@ -60,7 +60,7 @@ def teardown_function():
60
60
  print("Cleaning up test environment")
61
61
  ```
62
62
 
63
- ### Step 4: Running Tests in Parallel
63
+ ### Step 4: Running tests in parallel
64
64
 
65
65
  Parallel execution can be achieved using pytest-xdist. Install it with:
66
66