testdriverai 5.3.0 → 5.3.2

package/docs/getting-started/ci.mdx

@@ -1,21 +1,18 @@
 ---
-title: "Getting Started with TestDriver.ai"
-sidebarTitle: "Getting Started"
+title: "Running in GitHub Actions"
+sidebarTitle: "Running in CI"
 description: "Learn how to set up and run tests with TestDriver.ai using GitHub Actions for seamless end-to-end testing."
+icon: "code-pull-request"
 ---
 
 This guide explains how to set up a basic GitHub Actions workflow to run tests using **TestDriver.ai**.
 
----
-
-## Step 1: Prerequisites
+## Prerequisites
 
 1. **TestDriver.ai API Key**: Obtain your API key from TestDriver.ai and store it as a GitHub secret (e.g., `TESTDRIVER_API_KEY`).
 2. **Test Files**: Ensure your test files are saved in the `testdriver/` directory of your repository.
 
----
-
-## Step 2: Create a GitHub Actions Workflow
+## Create a GitHub Actions Workflow
 
 1. Create a new file in your repository: `.github/workflows/testdriver.yml`.
 2. Add the following workflow configuration:
@@ -23,174 +20,120 @@ This guide explains how to set up a basic GitHub Actions workflow to run tests u
 ### Example Workflow: `.github/workflows/testdriver.yml`
 
 ```yaml
-name: Run Tests with TestDriver.ai
+name: TestDriver.ai / Run / Regressions
 
 on:
   push:
-    branches:
-      - main
+    branches: ["main"]
   pull_request:
   workflow_dispatch:
+  schedule:
+    - cron: '0 13 * * *'  # Every day at 1 PM UTC (adjust if needed for timezone)
+
+permissions:
+  actions: read
+  contents: read
+  statuses: write
+  pull-requests: write
 
 jobs:
-  run-tests:
-    name: Run TestDriver.ai Tests
+  gather-test-files:
+    name: Setup Test Matrix (./testdriver/*.yml)
     runs-on: ubuntu-latest
+    outputs:
+      test_files: ${{ steps.test_list.outputs.files }}
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v2
+        with:
+          ref: ${{ github.event.ref }}
+      - name: Find all test files and extract filenames
+        id: test_list
+        run: |
+          FILES=$(ls ./testdriver/*.yml)
+          FILENAMES=$(basename -a $FILES)
+          FILES_JSON=$(echo "$FILENAMES" | jq -R -s -c 'split("\n")[:-1]')
+          echo "::set-output name=files::$FILES_JSON"
+
+  test:
+    needs: gather-test-files
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        test: ${{ fromJson(needs.gather-test-files.outputs.test_files) }}
+        os: ['linux', 'windows']
+      fail-fast: false
+    name: ${{ matrix.test }} (${{ matrix.os }})
     steps:
       - name: Check out repository
         uses: actions/checkout@v2
+        with:
+          ref: ${{ github.event.ref }}
+
+      - name: Display filename being tested
+        run: echo "Running job for file: ${{ matrix.test }} on ${{ matrix.os }}"
 
-      - name: Run TestDriver.ai
-        uses: testdriverai/action@main
+      - uses: testdriverai/action@main
         with:
           key: ${{ secrets.TESTDRIVER_API_KEY }}
-          prompt: |
-            1. Run all tests in the testdriver directory
+          os: ${{ matrix.os }}
+          version: "5.2.3"
+          prompt: 1. /run testdriver/${{ matrix.test }}
+          prerun: |
+            cd $env:TEMP
+            npm init -y
+            npm install dashcam-chrome
+            Start-Process "C:/Program Files/Google/Chrome/Application/chrome.exe" -ArgumentList "--start-maximized", "--load-extension=$(pwd)/node_modules/dashcam-chrome/build", "${{ env.TD_WEBSITE }}"
+            exit
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           FORCE_COLOR: "3"
-
-
 ```
 
-
----
-
-## Step 3: Add Your Test Files
-
-1. Save your test files in the `testdriver/` directory of your repository.
-2. Example test file: `testdriver/test.yml`:
-
-```yaml
-version: 4.2.18
-steps:
-  - prompt: Open the homepage
-  - prompt: Verify the login button is visible
-  - prompt: Click the login button
-  - prompt: Verify the login form is displayed
-
-
-```
-
-
----
-
-## Step 4: Trigger the Workflow
+## Trigger the Workflow
 
 1. Push changes to the `main` branch or open a pull request.
 2. Alternatively, manually trigger the workflow from the **Actions** tab in your GitHub repository.
 
----
-
-## Step 5: View Results
+## View Results
 
 1. Navigate to the **Actions** tab in your GitHub repository.
 2. Select the workflow run to view the test results.
 
----
-
-## Example Output
-
-- ✅ **Run TestDriver.ai Tests**: All tests passed.
-- ❌ **Run TestDriver.ai Tests**: 1 test failed. View logs for details.
-
----
-
-By following this guide, you can quickly set up and run tests using TestDriver.ai on GitHub Actions.
-
-# Debugging Tests in TestDriver.ai (app.testdriver.ai)
+# Debugging Tests
 
 TestDriver.ai provides a powerful debugging interface through its **app.testdriver.ai** platform (formerly Dashcam). This interface allows you to analyze test runs, identify failures, and optimize your test suite with detailed visual and textual feedback.
 
----
-
-## Key Features of app.testdriver.ai for Debugging
-
-### 1. **Step-by-Step Execution Logs**
+### Step-by-Step Execution Logs
 - View each step of the test execution, including:
   - The **action performed** (e.g., clicking a button, typing text).
   - The **expected outcome** (e.g., verifying a specific element is visible).
   - The **result** (pass, fail, or skipped).
 - Logs provide detailed context for each step, making it easier to pinpoint where and why a test failed.
 
----
-
-### 2. **Visual Feedback**
+### 2. Visual Feedback
 - **Screenshots**: See what the application looked like at each step of the test.
 - **GIF Previews**: Watch a replay of the entire test execution to understand the flow and identify UI issues.
 - **Highlighting**: Elements interacted with during the test are highlighted in screenshots, helping you verify that the correct elements were targeted.
 
----
-
-### 3. **Error Details**
+### 3. Error Details
 - For failed steps, app.testdriver.ai provides:
   - **Error messages**: Detailed descriptions of what went wrong (e.g., "Element not found").
   - **Stack traces**: For advanced debugging of backend or script-related issues.
   - **Suggestions**: Recommendations for fixing common issues, such as adjusting prompts or improving element descriptions.
 
----
-
-### 4. **Test History**
+### 4. Test History
 - Access the history of test runs to:
   - Compare results across different builds or environments.
   - Identify flaky tests by analyzing patterns in failures.
   - Track improvements or regressions over time.
 
----
-
-### 5. **Environment Context**
+### 5. Environment Context
 - View the environment details for each test run, including:
   - Operating system and browser version.
   - Screen resolution and viewport size.
   - Network conditions (if applicable).
 
----
-
-### 6. **Collaboration Tools**
+### 6. Collaboration Tools
 - Share test results with your team by generating a shareable link.
 - Add comments or annotations to specific steps to facilitate discussions and debugging.
-
----
-
-## How to Debug Tests in app.testdriver.ai
-
-1. **Access Test Results**:
-   - Navigate to [app.testdriver.ai](https://app.testdriver.ai).
-   - Log in with your TestDriver.ai account.
-   - Select the test run you want to debug from the dashboard.
-
-2. **Analyze the Test Run**:
-   - Review the **step-by-step logs** to identify where the test failed.
-   - Use the **screenshots** and **GIF previews** to visually verify the application's behavior.
-
-3. **Investigate Failures**:
-   - Click on the failed step to view detailed error messages and suggestions.
-   - Check the **highlighted elements** in the screenshots to ensure the correct elements were targeted.
-
-4. **Iterate and Fix**:
-   - Update your test prompts or application code based on the findings.
-   - Re-run the test to verify the fix.
-
----
-
-## Best Practices for Debugging in app.testdriver.ai
-
-1. **Use Descriptive Prompts**:
-   - Ensure your test prompts are clear and specific to reduce ambiguity.
-   - Example: Instead of "Click the button," use "Click the 'Sign In' button in the header."
-
-2. **Leverage Visual Feedback**:
-   - Use screenshots and GIFs to quickly identify UI issues, such as misaligned elements or missing buttons.
-
-3. **Track Flaky Tests**:
-   - Use the test history to identify tests that fail intermittently and investigate potential causes (e.g., timing issues or dynamic content).
-
-4. **Collaborate with Your Team**:
-   - Share test results with developers and QA engineers to resolve issues faster.
-
-5. **Optimize Test Steps**:
-   - Combine or simplify test steps to make debugging easier and reduce test execution time.
-
----
-
-By leveraging the debugging tools in **app.testdriver.ai**, you can efficiently analyze and resolve test failures, ensuring your application meets the highest quality standards.

package/docs/getting-started/editing.mdx

@@ -0,0 +1,77 @@
+---
+title: "Editing"
+sidebarTitle: "Editing"
+description: "Edit previously generated tests."
+icon: "file-pen"
+---
+
+Test scripts are written in YAML format, making them easy to read and modify manually if you need to.
+
+<Card
+  title="Commands Reference"
+  icon="link"
+  href="/commands/assert"
+>
+  Explore the available commands and their parameters.
+</Card>
+
+## Locate the Test File
+TestDriver test files are typically stored in the `testdriver/` directory of your project. The default file is `testdriver/testdriver.yaml`.
+
+### Open the Test File
+Use your preferred text editor or IDE (e.g., Visual Studio Code) to open the YAML file.
+
+### YAML Structure
+Each test file consists of:
+- **Version**: Specifies the TestDriver version.
+- **Session**: A unique identifier for the test session.
+- **Steps**: A list of prompts and commands to execute.
+
+```yaml
+version: 4.2.18
+session: abc123
+steps:
+  - prompt: Open Google Chrome and navigate to Airbnb
+    commands:
+      - command: focus-application
+        name: Google Chrome
+      - command: hover-text
+        text: Search Google or type a URL
+        description: main google search
+        action: click
+      - command: type
+        text: airbnb.com
+      - command: press-keys
+        keys: [enter]
+```
+
+### Modify the Test Steps
+You can add, edit, or remove steps as needed. Each step contains:
+
+- **Prompt**: A description of the action.
+- **Commands**: The specific actions to perform.
+
+#### Example Edits:
+- **Add a Wait Command**:
+  ```yaml
+  - command: wait
+    timeout: 3000
+  ```
+- **Change the URL**:
+  Update the `text` field in the `type` command:
+  ```yaml
+  - command: type
+    text: booking.com
+  ```
+
+### Validate the YAML Syntax
+Ensure the YAML file is properly formatted:
+- Use consistent indentation (2 spaces recommended).
+- Avoid trailing spaces or tabs.
+- Validate the file using a YAML linter if needed.
+
+### Test the Changes
+Run the updated test to verify it works as expected:
+```bash
+testdriverai run path/to/test.yaml
+```

package/docs/getting-started/generating.mdx

@@ -1,12 +1,11 @@
 ---
-title: "Generating Test Plans with TestDriver.ai"
-sidebarTitle: "Generating Tests"
-description: "Learn how to generate test plans locally with TestDriver.ai and integrate them into your testing workflow."
+title: "Self-Driving Tests"
+sidebarTitle: "Self-Driving Tests"
+description: "Generate test plans and integrate them into your testing workflow."
+icon: "steering-wheel"
 ---
 
-# Running Locally Generated Test Plans with TestDriver.ai and Managing Generated PRs
-
-This guide explains how to use **TestDriver.ai** to generate test plans locally, commit them to your repository, and run them in parallel using GitHub Actions. Additionally, it covers how **TestDriver.ai** can automatically create **pull requests (PRs)** for test results, enabling collaboration and review.
+This guide explains how to use **TestDriver.ai** to generate test plans locally and run them in parallel. You need
 
 ---
 
@@ -30,13 +29,10 @@ This guide explains how to use **TestDriver.ai** to generate test plans locally,
 ```bash
 testdriverai
 > /generate web 10
-
 ```
 
 This will generate 10 test plans for your website or application.
 
----
-
 ### Save the Generated Files
 
 TestDriver.ai will save the generated test plans as YAML files in the `testdriver/generate/` directory by default. For example:
@@ -47,164 +43,13 @@ testdriver/
 │   ├── test_1.yml
 │   ├── test_2.yml
 │   ├── test_3.yml
-
-```
-
----
-
-### Commit the Files to Your Repository
-
-1. Add the generated files to your Git repository:
-
-```bash
-git add testdriver/generate/
-git commit -m "Add locally generated test plans"
-git push origin main
-```
-
----
-
-## Step 2: Run Tests in Parallel Using GitHub Actions
-
-### Workflow: **Run Locally Generated Tests**
-
-This workflow dynamically discovers the committed test files and runs them in parallel.
-
-#### Example Workflow: `.github/workflows/run-tests.yml`
-
-```yaml
-name: Run Locally Generated Test Plans
-
-on:
-  push:
-    paths:
-      - 'testdriver/generate/**'
-  workflow_dispatch:
-
-jobs:
-  gather-test-files:
-    name: Gather Test Files
-    runs-on: ubuntu-latest
-    outputs:
-      test_files: ${{ steps.test_list.outputs.files }}
-    steps:
-      - name: Check out repository
-        uses: actions/checkout@v2
-
-      - name: Find all test files
-        id: test_list
-        run: |
-          FILES=$(ls ./testdriver/generate/*.yml)
-          FILES_JSON=$(echo "$FILES" | jq -R -s -c 'split("\n")[:-1]')
-          echo "::set-output name=files::$FILES_JSON"
-
-  run-tests:
-    name: Run Tests in Parallel
-    needs: gather-test-files
-    runs-on: ubuntu-latest
-    strategy:
-      matrix:
-        test_file: ${{ fromJson(needs.gather-test-files.outputs.test_files) }}
-      fail-fast: false
-    steps:
-      - name: Check out repository
-        uses: actions/checkout@v2
-
-      - name: Run TestDriver.ai
-        uses: testdriverai/action@main
-        with:
-          key: ${{ secrets.TESTDRIVER_API_KEY }}
-          prompt: |
-            1. /run ${{ matrix.test_file }}
-          create-pr: true
-          pr-title: "TestDriver.ai / Test Results for ${{ matrix.test_file }}"
-          pr-branch: testdriver/results-${{ matrix.test_file }}
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          FORCE_COLOR: "3"
-
-```
-
-
----
-
-### Workflow Steps Explained
-
-1. **Trigger on Push**:
-- The workflow triggers whenever files in the `testdriver/generate/` directory are pushed to the repository.
-
-2. **Gather Test Files**:
-- The `gather-test-files` job collects all YAML files in the `testdriver/generate/` directory and outputs them as a JSON array.
-
-3. **Run Tests in Parallel**:
-- The `run-tests` job uses the matrix strategy to run each test file in parallel.
-
-4. **Create PRs for Test Results**:
-- For each test file, a new branch is created with the test results, and a PR is opened for review.
-
----
-
-## Step 3: Generated PRs for Test Results
-
-### How PRs Are Created
-
-When the tests are executed, TestDriver.ai automatically:
-1. Creates a new branch for each test result (e.g., `testdriver/results-test_1.yml`).
-2. Commits the test results to the branch.
-3. Opens a pull request with the test results for review.
-
----
-
-### Example PR Details
-
-#### PR Title:
-`TestDriver.ai / Test Results for test_1.yml`
-
-#### PR Description:
-
-
 ```
-### Test Results for test_1.yml
-
-**Test Summary**:
-- ✅ Step 1: Opened the homepage.
-- ✅ Step 2: Verified the login button is visible.
-- ❌ Step 3: Failed to submit the login form.
-
-**Details**:
-- Error: The "Submit" button was not clickable.
-
-Please review the test results and address any issues.
-
-```
-
----
-
-### Benefits of Generated PRs
-
-1. **Collaboration**: Teams can review test results directly in GitHub and discuss issues in the PR comments.
-2. **Traceability**: Each test result is tied to a specific branch and PR, making it easy to track changes.
-3. **Continuous Improvement**: Failed tests can be addressed and merged back into the main branch after fixes.
-
----
-
-## Example Output
-
-### GitHub Actions Dashboard:
-- ✅ **Run Tests in Parallel**: All tests passed.
-- ❌ **Run Tests in Parallel**: 1 test failed. View logs for details.
-
-### Pull Requests:
-- **PR #123**: `TestDriver.ai / Test Results for test_1.yml`
-- **PR #124**: `TestDriver.ai / Test Results for test_2.yml`
-
----
-
-## Summary
-
-1. **Generate Tests Locally**: Use the `/generate` command to create test plans.
-2. **Commit and Run Tests**: Push the generated files to your repository and execute them in parallel.
-3. **Review Generated PRs**: Collaborate on test results using automatically created pull requests.
-4. **Merge Fixes**: Address issues and merge the PRs back into the main branch.
-
-By leveraging TestDriver.ai's ability to generate and manage PRs, you can streamline your testing workflow and improve collaboration across your team.
+### Run  the test files
+
+<Card
+  title="Running TestDriver Tests"
+  icon="link"
+  href="/getting-started/ci"
+>
+  Run your generated tests.
+</Card>

package/docs/getting-started/running.mdx

@@ -1,67 +1,44 @@
 ---
-title: "Running Tests Locally with TestDriver.ai"
+title: "Running Tests with TestDriver.ai"
 sidebarTitle: "Running Tests"
-description: "Learn how to execute test plans locally using TestDriver.ai's `/run` command for debugging and validation."
+description: "Execute regression test plans for debugging and validation."
+icon: "play"
 ---
 
-# Running Tests Locally with `/run` in TestDriver.ai
+Run tests using previously created test scripts. This command replays the steps defined in the YAML file, performing each action sequentially.
 
-The `/run` command in TestDriver.ai allows you to execute previously created test plans locally. This is particularly useful for debugging, validating test scripts, or running tests in a controlled environment before committing them to a CI/CD pipeline.
+You can run a test script by specifying the file path in the terminal or interactively within the TestDriver CLI.
 
----
-
-## How to Use `/run`
-
-1. **Navigate to Your Test Directory**:
-Ensure you are in the directory where your test files are stored (e.g., `testdriver/`).
-
-2. **Run a Specific Test File**:
-Use the `/run` command followed by the path to the test file you want to execute. For example:
+### Example in Terminal
 
 ```bash
-testdriverai
-> /run testdriver/test.yml
+testdriverai run path/to/test.yaml
 ```
-This will execute the test plan defined in `test.yml`.
 
----
-
-## Benefits of Running Tests Locally
+This will execute the test script located at `path/to/test.yaml`. If the test passes, the command will exit with a status code of `0`. If it fails, the command will exit with a status code of `1`.
 
-1. **Quick Feedback**:
-- Test changes immediately without waiting for CI/CD pipelines.
-- Validate test scripts before committing them to version control.
+### Interactive Mode
 
-2. **Debugging**:
-- Identify and fix issues in test scripts or application behavior in a controlled environment.
+To run a test interactively:
 
-3. **Offline Testing**:
-- Run tests locally without requiring access to cloud-based runners or external environments.
+1. Launch the TestDriver CLI:
+  ```bash
+  testdriverai
+  ```
+2. At the `>` prompt, type:
+  ```bash
+  /run path/to/test.yaml
+  ```
 
-4. **Iterative Development**:
-- Modify test scripts and re-run them quickly to refine test cases.
-
----
+## Notes
 
-## Example Workflow for Local Testing
-
-1. **Create a Test File**:
-Save the following YAML file as `testdriver/test.yml`:
-
-```yaml
-version: 4.2.18
-steps:
-  - prompt: Open the homepage
-  - prompt: Verify the login button is visible
-  - prompt: Click the login button
-  - prompt: Verify the login form is displayed
-```
-
-2. **Run the Test Locally**:
-Execute the test using the `/run` command:
-
-```bash
-testdriverai
-> /run testdriver/test.yml   
-```
+- Ensure the YAML file is valid and follows the TestDriver.ai specification.
+- The default file path is `testdriver/testdriver.yaml` if no file is specified.
 
+<Card
+  title="Running Tests in CI"
+  icon="link"
+  href="/getting-started/ci"
+>
+  Run tests with Hosted Runnings in GitHub Actions
+</Card>

package/docs/getting-started/setup.mdx

@@ -1,7 +1,8 @@
 ---
-title: "Setting Up TestDriver.ai"
-sidebarTitle: "Setup Guide"
+title: "Advanced Setup"
+sidebarTitle: "Advanced Setup"
 description: "Comprehensive guide to installing and setting up TestDriver.ai for your testing needs."
+icon: "square-terminal"
 ---
 
 
@@ -131,3 +132,23 @@ Follow these steps to install and set up TestDriver.ai:
 </Step>
 
 </Steps>
+
+<CardGroup cols={2}>
+
+  <Card
+    title="Self-Driving Tests"
+    icon="link"
+    href="/getting-started/generating"
+  >
+    TestDriver will come up with it's own ideas for testing.
+  </Card>
+
+  <Card
+    title="Test Copilot"
+    icon="link"
+    href="/getting-started/writing"
+  >
+    Write custom tests
+  </Card>
+
+</CardGroup>