testdriverai 5.5.5 → 5.5.7

package/docs/integrations/vercel.mdx

@@ -1,17 +1,18 @@
 ---
 title: "Vercel"
+sidebarTitle: "Vercel"
+description: "Integrate TestDriver with Vercel deployments using GitHub Actions."
+icon: "triangle"
 ---
 
-# Testing Vercel Deployments with TestDriver.ai GitHub Action
-
-This guide explains how to integrate **TestDriver.ai** with **Vercel deployments** using the **GitHub Actions workflow**. By combining these tools, you can automatically test your Vercel preview deployments or production builds to ensure they meet your quality standards before merging or releasing.
+This guide explains how to integrate **TestDriver** with **Vercel deployments** using the **GitHub Actions workflow**. By combining these tools, you can automatically test your Vercel preview deployments or production builds to ensure they meet your quality standards before merging or releasing.
 
 ---
 
 ## Workflow Overview
 
 1. **Trigger Vercel Deployment**: Use Vercel's GitHub integration to deploy your application on every pull request or push to the main branch.
-2. **Run Tests on the Deployment URL**: Use the TestDriver.ai GitHub Action to test the deployed application using the Vercel deployment URL.
+2. **Run Tests on the Deployment URL**: Use the TestDriver GitHub Action to test the deployed application using the Vercel deployment URL.
 3. **Report Results**: View test results in the GitHub Actions dashboard or as comments on the pull request.
 
 ---
@@ -19,19 +20,18 @@ This guide explains how to integrate **TestDriver.ai** with **Vercel deployments
 ## Prerequisites
 
 1. **Vercel GitHub Integration**: Ensure your repository is connected to Vercel for automatic deployments.
-2. **TestDriver.ai API Key**: Store your API key as a GitHub secret (e.g., `TESTDRIVER_API_KEY`).
+2. **TestDriver API Key**: Store your API key as a GitHub secret (e.g., `TD_API_KEY`).
 3. **Vercel Deployment URL**: Use the `VERCEL_URL` environment variable provided by Vercel to access the deployment.
 
 ---
 
 ## GitHub Actions Workflow
 
-Here’s a complete workflow to test Vercel deployments with TestDriver.ai:
+Here's a complete workflow to test Vercel deployments with TestDriver:
 
 ### Workflow File: `.github/workflows/test-vercel.yml`
-
 ```yaml
-name: Test Vercel Deployment with TestDriver.ai
+name: Test Vercel Deployment with TestDriver
 
 on:
   pull_request:
@@ -59,10 +59,10 @@ jobs:
           VERCEL_ORG_ID: ${{ secrets.VERCEL_ORG_ID }}
           VERCEL_PROJECT_ID: ${{ secrets.VERCEL_PROJECT_ID }}
 
-      - name: Run Tests with TestDriver.ai
+      - name: Run Tests with TestDriver
         uses: testdriverai/action@main
         with:
-          key: ${{ secrets.TESTDRIVER_API_KEY }}
+          key: ${{ secrets.TD_API_KEY }}
           prompt: |
             1. Open the deployment URL: ${{ steps.vercel.outputs.url }}
             2. Verify the homepage loads correctly
@@ -101,14 +101,14 @@ The `amondnet/vercel-action` waits for the Vercel deployment to complete and ret
 
 ---
 
-### 2. **Run Tests with TestDriver.ai**
-The TestDriver.ai GitHub Action runs tests on the deployed application using the deployment URL. The `prompt` field specifies the test steps to execute.
+### 2. **Run Tests with TestDriver**
+The TestDriver GitHub Action runs tests on the deployed application using the deployment URL. The `prompt` field specifies the test steps to execute.
 
 ```yaml
-- name: Run Tests with TestDriver.ai
+- name: Run Tests with TestDriver
   uses: testdriverai/action@main
   with:
-    key: ${{ secrets.TESTDRIVER_API_KEY }}
+    key: ${{ secrets.TD_API_KEY }}
     prompt: |
       1. Open the deployment URL: ${{ steps.vercel.outputs.url }}
       2. Verify the homepage loads correctly
@@ -125,9 +125,11 @@ The TestDriver.ai GitHub Action runs tests on the deployed application using the
 
 ---
 
-## Example TestDriver.ai Prompt
+## Example TestDriver Prompt
 
-The `prompt` field in the TestDriver.ai action specifies the steps to test the Vercel deployment. For example:```yaml
+The `prompt` field in the TestDriver action specifies the steps to test the Vercel deployment. For example:
+
+```yaml
 prompt: |
   1. Open the deployment URL: ${{ steps.vercel.outputs.url }}
   2. Verify the homepage loads correctly
@@ -143,7 +145,7 @@ prompt: |
 ## Secrets Configuration
 
 Add the following secrets to your GitHub repository:
-1. **`TESTDRIVER_API_KEY`**: Your TestDriver.ai API key.
+1. **`TD_API_KEY`**: Your TestDriver API key.
 2. **`VERCEL_TOKEN`**: Your Vercel API token.
 3. **`VERCEL_ORG_ID`**: Your Vercel organization ID.
 4. **`VERCEL_PROJECT_ID`**: Your Vercel project ID.
@@ -155,7 +157,7 @@ Add the following secrets to your GitHub repository:
 1. **Automated Deployment Testing**: Automatically test every Vercel deployment, including preview and production builds.
 2. **Early Issue Detection**: Catch issues in pull requests before merging.
 3. **Detailed Feedback**: View test results directly in the GitHub Actions dashboard.
-4. **Seamless Integration**: Combine Vercel's deployment capabilities with TestDriver.ai's testing power.
+4. **Seamless Integration**: Combine Vercel's deployment capabilities with TestDriver's testing power.
 
 ---
 
@@ -165,7 +167,7 @@ Add the following secrets to your GitHub repository:
 - ✅ **Test Vercel Deployment**: All tests passed.
 - ❌ **Test Vercel Deployment**: 1 test failed. View logs for details.
 
-### TestDriver.ai Logs:
+### TestDriver Logs:
 - **Step 1**: Opened the deployment URL.
 - **Step 2**: Verified the homepage loaded correctly.
 - **Step 3**: Clicked the "Sign Up" button.
@@ -174,4 +176,4 @@ Add the following secrets to your GitHub repository:
 
 ---
 
-By integrating TestDriver.ai with Vercel deployments, you can ensure that every deployment is thoroughly tested, reducing the risk of bugs reaching production and improving the overall quality of your application.
+By integrating TestDriver with Vercel deployments, you can ensure that every deployment is thoroughly tested, reducing the risk of bugs reaching production and improving the overall quality of your application.

package/docs/interactive/assert.mdx

@@ -1,5 +1,8 @@
 ---
 title: "/assert"
+sidebarTitle: "/assert"
+description: "Use the /assert command to ensure specific conditions are met in your tests."
+icon: "clipboard-check"
 ---
 
 # Command: `/assert`
@@ -14,7 +17,7 @@ The `/assert` command ensures that a specific condition is true within your test
 
 ## Behavior
 - The `/assert` command generates an assertion based on the specified criteria.
-- TestDriver.ai takes a screenshot and uses it to verify that the condition described in the `expect` field is true.
+- TestDriver takes a screenshot and uses it to verify that the condition described in the `expect` field is true.
 - If the condition isn't met, the test will fail and exit immediately.
 
 ## Example Usage

package/docs/interactive/dry.mdx

@@ -0,0 +1,51 @@
+---
+title: "/dry"
+sidebarTitle: "/dry"
+description: "See and write a test plan without running the commands."
+icon: "forward-step"
+---
+
+## Description
+The `/dry` command is used to create a test plan and write to a specified file. This command won't run steps it creates. It's a **dry** run. 
+
+## Usage
+```bash
+/dry <file>
+```
+
+## Example Usage
+```bash
+testdriverai <filename.yaml>
+> /dry click the 'log in' button
+```
+This command generates the steps to `click the 'log in' button` with an explanation and adds them to the current test plan, without executing any command.
+
+## Behavior
+- TestDriver will create steps for the test plan, writing them in the file.
+- TestDriver will also show the reasoning and test steps in the CLI:
+
+```bash
+thinking...
+
+    To click the 'log in' button, we need to identify and
+    click on the text "Log in" visible in the screenshot.
+    Here's how to accomplish this:
+    
+    commands:
+      - command: hover-text
+        text: Log in
+        description: log in button in the code editor
+        action: click
+    
+    ### Explanation:
+    
+    1. Click the 'Log in' Button: We use hover-text to locate and click on the "Log in" text visible in the code editor. This ensures the button is activated.
+```
+## Protips
+- Ensure you run the `testdriverai` command with a valid file path before using `/dry`.
+- Use descriptive filenames for your test plans to make them easier to identify.
+- Combine `/dry` with `/undo` to write test plans without executing them.
+
+
+## Notes
+- The `/dry` command is ideal for writing out a series of steps before executing.

package/docs/interactive/explore.mdx

@@ -0,0 +1,92 @@
+---
+title: "/explore"
+sidebarTitle: "/explore"
+description: "Create and execute a test plan"
+icon: "play-pause"
+---
+
+## Description
+
+The `/explore` command is used to execute a single step in a test plan and add it to a specified file. This command generates and performs steps in an interactive way while creating a test plan.
+
+<Tip>`/explore` is the default command sent to TestDriver when no command is provided.</Tip>
+
+## Usage
+```bash
+testdriverai <filename.yaml>
+/run prompt
+```
+
+## Example Usage
+```bash
+> /explore add an item to the shopping cart
+```
+This command creates the command step(s) necessary to add an item to the shopping cart in the specified test plan, executing the generated command(s) after writing them into the test plan. If there are intermediate steps to be taken, they are added to the test plan as well.
+
+```bash
+thinking...
+
+    To add an item to the shopping cart, we need to identify
+    the "Add to Cart" button from the screenshot. However,
+    since the screenshot shows a code editor, we need to
+    assume the button is not visible here. Let's proceed with
+    a generic approach:
+    
+    commands:
+      - command: focus-application
+        name: Google Chrome
+      - command: hover-text
+        text: Add to Cart
+        description: add to cart button on the shopping page
+        action: click
+    
+    ### Explanation:
+    
+    1. Focus Google Chrome: Ensure the browser is active to interact with the shopping page.
+    2. Click 'Add to Cart': Use hover-text to find and click the "Add to Cart" button, assuming it's present on the shopping page.
+
+command='focus-application' name='Google Chrome'
+command='hover-text' text='Add to Cart' description='add to cart button on the shopping page' action='click'
+
+thinking...
+```
+
+In this case, the user wasn't logged in, so the AI will 'undo' the steps it thought it should take, and add steps to log in first.
+
+```bash
+undoing...
+thinking...
+
+    The screenshot shows a "Sign In" button on the webpage.
+    Let's try clicking that instead, as it might be necessary
+    before adding items to the cart.
+    
+    commands:
+      - command: hover-text
+        text: Sign In
+        description: sign in button on the webpage
+        action: click
+
+command='hover-text' text='Sign In' description='sign in button on the webpage' action='click'
+
+thinking...
+```
+
+
+## Behavior
+- TestDriver will generate an idea to complete the task, try to execute it, and add the steps to the test plan.
+- The steps will be executed in the order generated.
+- If the steps are successful, they are added to the test plan file.
+- If the steps fail, TestDriver will attempt to fall back to AI, undo the last step and provide an explanation of what went wrong. The AI will retry the step with new commands.
+- If the commands are successful, the program will output the results and exit with code `0`.
+- If any failures occur during the test, the program will output the errors and exit with code `1`.
+
+## Protips
+- Use descriptive prompts for higher success with generating test steps with `/explore`.
+- Use `/explore` multiple times in succession to build a comprehensive test plan.
+
+## Gotchas
+- The AI will retry 3 times if it can't complete the task.
+
+## Notes
+- The `/explore` command is ideal for creating test plans in an interactive session.

package/docs/interactive/generate.mdx

@@ -1,11 +1,14 @@
 ---
 title: "/generate"
+sidebarTitle: "/generate"
+description: "Use the /generate command to create exploratory test cases automatically."
+icon: "wand-sparkles"
 ---
 
 # Command: `/generate`
 
 ## Description
-The `/generate` command is an experimental feature that instructs TestDriver.ai to create its own exploratory prompts. This command is used in the "Generate a Test Suite" demo and is designed to help automate the creation of exploratory test cases.
+The `/generate` command is an experimental feature that instructs TestDriver to create its own exploratory prompts. This command is used in the "Generate a Test Suite" demo and is designed to help automate the creation of exploratory test cases.
 
 ## Usage
 ```bash
@@ -15,11 +18,11 @@ The `/generate` command is an experimental feature that instructs TestDriver.ai
 ## Behavior
 - The `/generate` command analyzes the display and generates exploratory prompts for testing.
 - Each exploratory test is saved as a simple Markdown file containing a list of steps.
-- TestDriver.ai generates 10 Markdown files every time the `/generate` command is called.
+- TestDriver generates 10 Markdown files every time the `/generate` command is called.
 - The generated files are stored in the `./testdriver/generate/*.md` directory.
 
 ## Example Output
-Here’s an example of a generated test file (`test-search-function.md`):
+Here's an example of a generated test file (`test-search-function.md`):
 ```markdown
 1. Click on the search icon.
 2. Type "real-time chat" into the search bar.

package/docs/interactive/run.mdx

@@ -1,5 +1,8 @@
 ---
 title: "/run"
+sidebarTitle: "/run"
+description: "Run a test plan from a file."
+icon: "play"
 ---
 
 ## Description

package/docs/interactive/save.mdx

@@ -1,5 +1,8 @@
 ---
 title: "/save"
+sidebarTitle: "/save"
+description: "Save the current test script to a file."
+icon: "floppy-disk"
 ---
 
 ## Description

package/docs/interactive/undo.mdx

@@ -1,5 +1,8 @@
 ---
 title: "/undo"
+sidebarTitle: "/undo"
+description: "Remove the last generated command or step from the current test script."
+icon: "arrow-rotate-left"
 ---
 
 ## Description

package/docs/overview/comparison.mdx

@@ -1,6 +1,8 @@
 ---
-title: "Comparison"
-icon: "boxing-glove"
+title: Comparison
+sidebarTitle: "Comparison"
+description: "TestDriver vs Playwright vs Selenium"
+icon: boxing-glove
 ---
 
 ### Application Support
@@ -84,3 +86,5 @@ TestDriver currently supports Mac and Windows!
 | Windows  | ✅            | ✅         | ✅       |
 | Mac      | ✅            | ✅         | ✅       |
 | Linux    |             | ✅         | ✅       |
+
+<Tip>You **can** use TestDriver on Linux with hosted runners, but not locally yet.</Tip>

package/docs/overview/faq.mdx

@@ -1,6 +1,7 @@
 ---
 title: "FAQ"
 sidebarTitle: "FAQ"
+description: "Frequently Asked Questions about TestDriver"
 icon: "block-question"
 ---
 
@@ -117,7 +118,7 @@ Yes. You can rewrite prompts, add constraints, or review and tweak auto-generate
 Yes. You can inspect the resolved steps, see element matches, and adjust test flows before execution.
 
 ## Can I use TestDriver to test new features?
-Yes. It’s great for validating changes, ensuring no regressions, and verifying rollout configurations.
+Yes. It's great for validating changes, ensuring no regressions, and verifying rollout configurations.
 
 ## Can it test seasonal or time-based behaviors?
 Yes. You can schedule tests or run them under specific date/time settings to verify time-sensitive logic.

package/docs/overview/quickstart.mdx

@@ -1,76 +1,134 @@
 ---
 title: "Quickstart"
-description: "Get started with TestDriver.ai in minutes."
-icon: "gauge-max"
+sidebarTitle: "Quickstart"
+description: "Get started with TestDriver in minutes."
+icon: "gauge-high"
 ---
 
+The fastest way to get started with TestDriver is to install the VS Code extension. This guide will walk you through the installation process and provide an overview of the extension's features.
+
 <Card
-  title="Get the VSCode Extension"
-  icon="link"
-  href="/getting-started/vscode"
+  title="Download the VS Code Extension"
+  icon="download"
+  href="https://github.com/testdriverai/vscode/releases/latest/download/testdriver.vsix"
 >
-  Turn your IDE into a powerful computer-use automation tool with our VSCode extension.
+  Get the latest version from GitHub releases. 
 </Card>
 
-<Steps>
-
-<Step title="Install TestDriver">
-Install the TestDriver CLI globally:  
-```bash
-npm install -g testdriverai@beta
-```
-</Step>
-
-<Step title="Initialize Your Project">
-Set up your project with:  
-```bash
-testdriverai init
-```  
-This creates a `.env` file and sample workflow files.
-</Step>
-
-<Step title="Run TestDriver in Interactive Mode">
-Start TestDriver in interactive mode to create and refine tests dynamically:  
-```bash
-testdriverai
-```  
-Once started, you can use commands like `/try`, `/undo`, and `/save` to interactively build and test your flows.
-</Step>
-
-<Step title="Define Your Test Steps">
-Use natural language to describe what you want to test. For example:  
-```
-> Open Google Chrome and search for "testdriverai"
-```
-TestDriver will generate steps like:  
-```yaml
-- command: focus-application
-  name: Google Chrome
-- command: hover-text
-  text: Search Google or type a URL
-  description: main search bar
-  action: click
-- command: type
-  text: testdriverai
-- command: press-keys
-  keys: [enter]
-```
-</Step>
-
-<Step title="Refine and Save Your Test">
-- Use `/undo` to remove the last step if needed.  
-- Save your test with `/save` to generate a reusable YAML file.
-</Step>
-
-<Step title="Run Saved Tests">
-Once saved, you can run your test file anytime:  
-```bash
-testdriverai run path/to/test.yml
-```
-</Step>
-
-</Steps>
+Don't use VS Code? <a href="/getting-started/setup">Get the TestDriver CLI →</a>
+
+---
+
+# Install the extension
+
+1. Open Visual Studio Code.
+2. Go to the Extensions view by clicking on the Extensions icon in the Activity Bar on the side of the window or by pressing `Ctrl+Shift+X`.
+3. In the Extensions view, click on the three-dot menu in the top-right corner and select "Install from VSIX...".
+4. Navigate to the location where you downloaded the `testdriverai.vsix` file and select it.
+
+<Frame>
+<img height="200" src="/images/content/vscode/vscode-install.png" />
+</Frame>
+
+---
+
+# Try the Walkthrough
+
+The extension should launch a walkthrough. If it doesn't, you can manually start it by running the command `TestDriver: Start Setup Walkthrough` from the Command Palette (`Ctrl+Shift+P`).
+
+<Frame caption="Walkthrough">
+<img height="200" src="/images/content/vscode/vscode-setup-walkthrough.png" />
+</Frame>
+
+---
+
+
+# Overview
+
+<Frame caption="VS Code Extension Overview">
+<img height="400" src="/images/content/vscode/vscode-overview.png" />
+</Frame>
+
+### 1. Test Pane
+  - Located in the **Activity Bar** (left-hand side of the editor).
+  - Clicking the <Icon icon="flask-conical" iconType="solid" /> **flask icon** opens the TestDriver test explorer.
+  - Lists all the available YAML-based test definitions.
+
+<Frame caption="The test pane is here">
+  <img height="100" src="/images/content/vscode/vscode-testpane.png" /> 
+</Frame>
+
+### 2. Run all Tests
+  - This button runs all tests shown in the Test Explorer.
+  - Found near the top of the Test Explorer view.
+  - Especially useful for quickly validating a batch of test cases.
+
+<Frame caption="Gotta run 'em all">
+    <img height="200" src="/images/content/vscode/vscode-testpane-runtests.png" /> 
+</Frame>
+
+### 3. AI Test Creation
+  - New files created by TestDriver will automatically be focused in the editor.
+  - Defines an automated test script created by TestDriver.
+
+<Frame caption="You will create a new file every time you start a chat">
+    <img height="200" src="/images/content/vscode/vscode-file-creation.png" />
+</Frame>
+
+### 4. AI Agent Test Preview
+  - Sidebar chat panel shows the TestDriver agent's response to the prompt.
+  - Offers feedback, interpretation, or suggested actions for the test steps.
+  - Example: It identifies the context (e.g., a terminal is in focus) and suggests the correct automation action.
+  - Additional files modified by the agent are listed for context.
+
+<img width="300" src="/images/content/vscode/vscode-agent-preview.png" />
+
+### 5. Test Output
+  - Terminal panel at the bottom of the screen.
+  - Displays live output from running the test file.
+  - Shows TestDriver version, current test script, and metadata like session ID.
+  - Helpful for debugging and understanding test execution flow.
+
+<Frame caption="You can move the terminal panel to the right as well!">
+    <img height="200" src="/images/content/vscode/vscode-test-output.png" />
+</Frame>
+
+### 6. Test History
+  - Shows a list of previously run test scripts.
+  - Allows developers to review or re-run older tests.
+  - Useful for regression testing or comparing AI-generated test iterations.
+
+<Frame caption="More history, less mystery">
+    <img src="/images/content/vscode/vscode-testhistory.png" />
+</Frame>
 
 ---
 
-**Pro Tip:** Always start with interactive mode (`testdriverai`) to explore and refine your tests before saving them. It’s faster and more intuitive!
+## Troubleshooting
+
+###ß I don't have GitHub Copilot Chat
+
+You can still use the extension to run and manage tests. Use the CLI for test creation instead.
+
+### The @testdriver command doesn't show up
+
+Make sure the GitHub Copilot Chat is set to "ask."
+
+<img src="/images/content/vscode/vscode-copilot-ask.png" />
+
+### How do I stop a running execution?
+
+Depending on the context, you can stop a running test execution in several ways:
+
+### 1. Click on the "Stop" button in the bottom of agent chat.
+
+<img height="200" src="/images/content/vscode/vscode-stopchat.png" />
+
+### 2. Click on the "Stop" button in the top right corner of the test execution. This only shows on hover.
+
+<img src="/images/content/vscode/vscode-stoptest.png" />
+
+### 3. A TestDriver icon will show in the tray bar. Click it, then click "quit."
+
+<img src="/images/content/vscode/vscode-tdservice.png" />
+

package/docs/overview/upgrading.mdx

@@ -0,0 +1,82 @@
+---
+title: "Upgrading Existing Tests"
+sidebarTitle: "Upgrading"
+description: "Learn how to upgrade TestDriver for the latest features and improvements."
+icon: "circle-up"
+---
+
+## How to Upgrade TestDriver
+To upgrade TestDriver, follow these steps:
+1. **Check for Updates**: Regularly check the [TestDriver NPM Package](https://www.npmjs.com/package/testdriverai)
+2. **Update Command**: Use the following command to update TestDriver to the latest version:
+   ```bash
+   npm install -g testdriverai
+   ```
+   If you want to use the beta version, use:
+   ```bash
+   npm install -g testdriverai@beta
+   ```
+
+In your existing Github Action setup, you may need to update the `version` field in the `workflows/testdriver.yaml` file(s) to match the version of TestDriver you want to use. Leaving it blank will run `latest` by default. Here is an example of what to replace:
+
+```yaml {16}
+name: TestDriver.ai
+
+on:
+  push:
+    branches: ["main"]
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  test:
+    name: "TestDriver"
+    runs-on: ubuntu-latest
+    steps:
+      - uses: testdriverai/action@main
+        with:
+          version: "4.2.2"
+          key: ${{secrets.TD_API_KEY}}
+          prompt: |
+            1. /run testdriver/test.yaml
+          prerun: |
+            cd $env:TEMP
+            npm init -y
+            npm install dashcam-chrome
+            Start-Process "C:/Program Files/Google/Chrome/Application/chrome.exe" -ArgumentList "--start-maximized", "--load-extension=$(pwd)/node_modules/dashcam-chrome/build", "${{ env.WEBSITE_URL }}"
+            exit
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          FORCE_COLOR: "3"
+          WEBSITE_URL: "https://orgfarm-abvxdff-dev-ed.develop.my.salesforce.com/" # Define the website URL here
+          TD_TEST_USERNAME: ${{secrets.TD_TEST_USERNAME}}
+          TD_TEST_PASSWORD: ${{secrets.TD_TEST_PASSWORD}}
+```
+
+Would become:
+
+```yaml {16}
+name: TestDriver.ai
+
+on:
+  push:
+    branches: ["main"]
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  test:
+    name: "TestDriver"
+    runs-on: ubuntu-latest
+    steps:
+      - uses: testdriverai/action@main
+        with:
+          version: "5.3.0"
+          key: ${{secrets.TESTDRIVER_API_KEY}}
+          prompt: |
+            1. /run testdriver/test.yaml
+            ...
+...
+```
+
+Now existing tests will run on v5.3.0