testdriverai 5.2.2 → 5.3.0

package/docs/guide/authentication.mdx

@@ -0,0 +1,150 @@
+---
+title: "Authentication in TestDriver.ai"
+sidebarTitle: "Authentication"
+description: "Learn how to securely handle authentication workflows in TestDriver.ai using GitHub Actions."
+---
+
+This guide explains how to handle **authentication workflows** in **TestDriver.ai** using GitHub Actions. It covers securely passing credentials (e.g., usernames and passwords) to the TestDriver.ai action and using them in both the `prerun` script and test files.
+
+---
+
+## How Authentication Works in TestDriver.ai
+
+1. **Store Credentials Securely**:
+   - Use GitHub Secrets to store sensitive information like usernames, passwords, or API keys.
+2. **Pass Credentials to the Workflow**:
+   - Supply credentials as environment variables or directly in the workflow.
+3. **Use Credentials in Tests**:
+   - Dynamically reference credentials in the `prerun` script or test files to perform authentication steps.
+
+---
+
+## Step 1: Store Credentials in GitHub Secrets
+
+1. Navigate to your repository's **Settings** > **Secrets and variables** > **Actions**.
+2. Add the following secrets:
+   - **`TD_USERNAME`**: The username for login.
+   - **`TD_PASSWORD`**: The password for login.
+   - **`TESTDRIVER_API_KEY`**: Your TestDriver.ai API key.
+   - **`WEBSITE_URL`**: The URL of the website to test.
+
+---
+
+## Step 2: Pass Credentials to the Workflow
+
+Secrets are passed to the GitHub Action using the `secrets` context. They can be supplied as:
+- **Environment Variables**: Passed via the `env` block.
+- **Inline in the `prerun` Script**: Used directly in the script.
+
+### Example Workflow:
+
+```yaml
+name: TestDriver.ai / Authentication
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  test-authentication:
+    name: Test Authentication
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v2
+
+      - name: Run Authentication Test
+        uses: testdriverai/action@main
+        with:
+          key: ${{ secrets.TESTDRIVER_API_KEY }}
+          prompt: |
+            1. Open the login page
+            2. Enter the username: ${{ secrets.TD_USERNAME }}
+            3. Enter the password: ${{ secrets.TD_PASSWORD }}
+            4. Click the "Log In" button
+            5. Verify the dashboard is displayed
+          prerun: |
+            cd $env:TEMP
+            npm init -y
+            npm install dashcam-chrome
+            Start-Process "msedge" -ArgumentList "--start-maximized", "--load-extension=$(pwd)/node_modules/dashcam-chrome/build", "${{ secrets.WEBSITE_URL }}"
+            exit
+        env:
+          TD_USERNAME: ${{ secrets.TD_USERNAME }}
+          TD_PASSWORD: ${{ secrets.TD_PASSWORD }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          FORCE_COLOR: "3"
+
+
+```
+
+
+---
+
+## Step 3: Use Credentials in Test Files
+
+Secrets can be referenced in the test file using placeholders (e.g., `${TD_USERNAME}` and `${TD_PASSWORD}`).
+
+### Example Test File:
+
+```yaml
+version: 4.2.18
+steps:
+  - prompt: Log in to the application
+    commands:
+      - command: hover-text
+        text: Email
+        description: Email input field
+        action: click
+      - command: type
+        text: ${TD_USERNAME}
+      - command: hover-text
+        text: Password
+        description: Password input field
+        action: click
+      - command: type
+        text: ${TD_PASSWORD}
+      - command: hover-text
+        text: Log In
+        description: Log In button
+        action: click
+      - command: assert
+        expect: The dashboard is displayed
+
+
+```
+
+
+---
+
+## How It Works Together
+
+1. **Secrets in the Workflow**:
+   - Secrets like `TD_USERNAME` and `TD_PASSWORD` are passed as environment variables to the TestDriver.ai action.
+
+2. **Secrets in the `prerun` Script**:
+   - The `WEBSITE_URL` secret is used to launch the browser with the correct URL.
+
+3. **Secrets in the Test File**:
+   - The test file dynamically references the secrets to fill in login credentials during the test.
+
+---
+
+## Benefits of Using Authentication in TestDriver.ai
+
+1. **Secure Handling of Credentials**:
+   - Secrets are encrypted and not exposed in logs.
+   - Even if printed, they appear as `***`.
+
+2. **Dynamic Testing**:
+   - Easily switch between different environments (e.g., staging, production) by updating the secrets.
+
+3. **Reusability**:
+   - Use the same workflow and test files across multiple repositories or environments.
+
+---
+
+By securely passing credentials and using them in the `prerun` script and test files, you can automate authentication workflows in TestDriver.ai while ensuring sensitive information remains protected.

package/docs/guide/code.mdx

@@ -0,0 +1,169 @@
+---
+title: "Custom Code in TestDriver.ai"
+sidebarTitle: "Custom Code"
+description: "Learn how to integrate custom Node.js scripts into your TestDriver.ai workflows for dynamic testing."
+---
+
+TestDriver.ai allows you to execute custom **Node.js** scripts within your test workflows using the `exec` command. This feature, introduced in version `5.1.0`, enables you to integrate custom logic, such as generating one-time passwords (OTPs), hitting APIs, or performing other dynamic operations, directly into your tests.
+
+---
+
+## Key Features
+
+1. **Run Node.js Scripts**:
+   - Execute custom JavaScript code within your test steps.
+   - Use NPM modules to extend functionality.
+
+2. **Dynamic Outputs**:
+   - Store the result of your script in a variable for use in subsequent steps.
+
+3. **NPM Support**:
+   - Install and use NPM packages in your scripts.
+
+---
+
+## Example: One-Time Password (OTP) Validator
+
+This example demonstrates how to generate a one-time password (OTP) using the `totp-generator` NPM package and use it in a test.
+
+### Workflow File: `.github/workflows/testdriver-otp.yml`
+
+```yaml
+name: TestDriver.ai
+
+on:
+  push:
+    branches: ["main"]
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  test:
+    name: "TestDriver"
+    runs-on: ubuntu-latest
+    steps:
+      - uses: testdriverai/action@main
+        with:
+          version: "5.1.0"
+          key: ${{ secrets.TESTDRIVER_API_KEY }}
+          prompt: |
+            1. /run testdriver/testdriver.yaml
+          prerun: |
+            cd $env:TEMP
+            npm init -y
+            npm install totp-generator
+            exit
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          FORCE_COLOR: "3"
+
+
+```
+
+
+---
+
+### Test File: `testdriver/testdriver.yaml`
+
+```yaml
+version: 5.1.0
+session: 67e57d614dc25283aa0872a9
+steps:
+  - prompt: Log in with the totp
+    commands:
+      - command: exec
+        output: totp
+        js: |
+          const { TOTP } = require("totp-generator");
+          let otp = TOTP.generate("JBSWY3DPEB3W64TMMQQQ").otp;
+          result = otp;
+      - command: exec
+        js: |
+          console.log("${OUTPUT.totp}");
+      - command: type
+        text: ${OUTPUT.totp}
+
+
+```
+
+
+---
+
+## How It Works
+
+### Parameters for `exec`
+
+- **`js`**: The Node.js script to execute. The output must be assigned to the `result` variable.
+- **`output`**: The name of the variable to store the result. This variable can be referenced in subsequent steps as `${OUTPUT.<var>}`.
+
+---
+
+### Running the Demo Locally
+
+1. Clone the repository.
+2. Install dependencies:
+
+   ```bash
+   npm install
+
+```   
+
+3. Run the test:
+
+   ```bash
+   testdriverai run testdriver/testdriver.yaml
+
+```   
+
+---
+
+### Example Output
+
+- **Step 1**: Generates an OTP using the `totp-generator` package.
+- **Step 2**: Logs the OTP to the console.
+- **Step 3**: Types the OTP into the application.
+
+---
+
+## Pro Tips
+
+1. **Avoid Overwriting `result`**:
+   - Always assign the output of your script to the `result` variable.
+
+2. **Install Dependencies**:
+   - Ensure all required NPM packages are installed locally and in the `prerun` script when using GitHub Actions.
+
+3. **Node.js Context**:
+   - The script runs in the same context as the calling process and uses Node.js's [VM module](https://nodejs.org/api/vm.html) internally.
+
+---
+
+## Gotchas
+
+1. **Runner Support**:
+   - This feature is supported only on **hosted Windows runners**. Hosted Linux runners do not yet support `prerun`.
+
+2. **NPM Initialization**:
+   - Ensure you initialize NPM (`npm init`) and install required packages before running the test.
+
+---
+
+## Using NPM Modules from Scratch
+
+1. Initialize a new Node.js project:
+
+   ```bash
+   npm init
+
+```   
+
+2. Install the required package:
+
+   ```bash
+   npm install totp-generator --save
+
+```   
+
+---
+
+By leveraging the `exec` command, you can extend TestDriver.ai's capabilities with custom Node.js scripts, enabling dynamic and powerful test workflows.

package/docs/guide/locating.mdx

@@ -0,0 +1,136 @@
+---
+title: "Locating Elements in TestDriver.ai"
+sidebarTitle: "Locating Elements"
+description: "Learn how to effectively describe and locate elements using TestDriver.ai's visual understanding capabilities."
+---
+
+TestDriver.ai uses **visual understanding** to locate elements on the screen. Unlike traditional testing frameworks that rely on selectors or element IDs, TestDriver.ai identifies elements based on their **visual appearance**. This guide explains how to describe elements effectively to ensure TestDriver.ai can locate them accurately.
+
+---
+
+## Key Principles for Locating Elements
+
+1. **Describe the Element Visually**:
+   - Focus on the **appearance** of the element, not its behavior or function.
+   - Mention unique visual traits such as **text**, **color**, **size**, **position**, or **icon**.
+
+2. **Avoid Behavioral Descriptions**:
+   - Do not describe what the element does (e.g., "button that submits the form").
+   - Instead, describe how it looks (e.g., "blue button with the text 'Submit' in the bottom-right corner").
+
+3. **Use Unique Identifiers**:
+   - If the element has visible text, include it in the description.
+   - For icons or images, describe their shape, color, or associated label.
+
+4. **Leverage Visual Feedback**:
+   - TestDriver.ai will draw **yellow boxes** around detected elements during test execution.
+   - Use this feedback to verify that the correct element is being targeted.
+
+---
+
+## Examples of Effective Descriptions
+
+### 1. Buttons
+- **Good**: "Blue button with the text 'Sign In' in the top-right corner."
+- **Bad**: "Button that logs the user in."
+
+### 2. Links
+- **Good**: "Hyperlink with the text 'Learn More' in the footer."
+- **Bad**: "Link that navigates to the About page."
+
+### 3. Icons
+- **Good**: "Magnifying glass icon next to the search bar."
+- **Bad**: "Search icon that opens the search feature."
+
+### 4. Input Fields
+- **Good**: "White input box labeled 'Email Address' above the password field."
+- **Bad**: "Field where the user enters their email."
+
+### 5. Images
+- **Good**: "Company logo in the top-left corner, a blue circle with white text."
+- **Bad**: "Logo that redirects to the homepage."
+
+---
+
+## How to Write TestDriver.ai Commands
+
+### Example: Locating a Button
+
+#### YAML Command:
+
+```yaml
+- command: hover-text
+  text: Sign In
+  description: Blue button with the text 'Sign In' in the top-right corner
+  action: click
+
+
+```
+
+
+---
+
+### Example: Locating an Icon
+
+#### YAML Command:
+
+```yaml
+- command: hover-image
+  description: Magnifying glass icon next to the search bar
+  action: click
+
+
+```
+
+
+---
+
+### Example: Locating a Link
+
+#### YAML Command:
+
+```yaml
+- command: hover-text
+  text: Learn More
+  description: Hyperlink with the text 'Learn More' in the footer
+  action: click
+
+
+```
+
+
+---
+
+## Debugging Element Detection
+
+1. **Run the Test**:
+   - Execute the test using TestDriver.ai.
+   - TestDriver.ai will draw **yellow boxes** around detected elements.
+
+2. **Verify the Correct Element**:
+   - Ensure the yellow box highlights the intended element.
+   - If the wrong element is highlighted, refine your description.
+
+3. **Adjust the Description**:
+   - Add more specific visual details (e.g., color, position, associated text).
+   - Avoid generic terms like "button" or "icon" without additional context.
+
+---
+
+## Best Practices
+
+1. **Be Specific**:
+   - Include as many unique visual traits as possible to differentiate the element from others.
+
+2. **Test Incrementally**:
+   - Run tests after adding or modifying commands to verify element detection.
+
+3. **Use Visual Feedback**:
+   - Leverage the yellow boxes drawn by TestDriver.ai to confirm the correct element is being targeted.
+
+4. **Avoid Overloading Descriptions**:
+   - Keep descriptions concise but detailed enough to uniquely identify the element.
+
+---
+
+By focusing on **visual descriptions** and leveraging TestDriver.ai's feedback, you can accurately locate elements and create robust, reliable tests.

package/docs/guide/setup-teardown.mdx

@@ -0,0 +1,161 @@
+---
+title: "Setup & Teardown"
+sidebarTitle: "Setup & Teardown"
+description: "Learn how to effectively describe and locate elements using TestDriver.ai's visual understanding capabilities."
+---
+
+
+When running tests with **TestDriver.ai** in GitHub Actions, it's important to handle **setup** and **teardown** tasks properly. Setup tasks should be performed **before** the TestDriver.ai action (e.g., creating a new user via an API), and teardown tasks should be performed **after** the TestDriver.ai action, ensuring cleanup happens regardless of the test results.
+
+This guide explains how to structure your workflow to handle setup and teardown tasks effectively.
+
+---
+
+## Workflow Overview
+
+1. **Setup Tasks**:
+   - Use a dedicated action **before** the TestDriver.ai action to prepare the environment (e.g., create a test user via an API).
+   - Pass the created user credentials to the TestDriver.ai action using environment variables.
+
+2. **Run Tests**:
+   - Execute the tests using TestDriver.ai, leveraging the setup data (e.g., the test user).
+
+3. **Teardown Tasks**:
+   - Use a dedicated action **after** the TestDriver.ai action to clean up (e.g., delete the test user).
+   - Ensure the teardown step runs **no matter the result** of the TestDriver.ai action.
+
+---
+
+## Example Workflow with Setup and Teardown
+
+### Workflow File: `.github/workflows/testdriver-setup-teardown.yml`
+
+```yaml
+name: TestDriver.ai with Setup and Teardown
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  test:
+    name: "TestDriver with Setup and Teardown"
+    runs-on: ubuntu-latest
+    steps:
+      # Step 1: Check out the repository
+      - name: Check out repository
+        uses: actions/checkout@v2
+
+      # Step 2: Setup - Create a test user via API
+      - name: Setup Test User
+        id: setup-user
+        run: |
+          echo "Creating test user via API..."
+          RESPONSE=$(curl -X POST -H "Content-Type: application/json" -d '{"name": "Test User", "email": "test@example.com", "password": "password123"}' https://api.example.com/users)
+          echo "USER_ID=$(echo $RESPONSE | jq -r '.id')" >> $GITHUB_ENV
+          echo "USER_EMAIL=$(echo $RESPONSE | jq -r '.email')" >> $GITHUB_ENV
+          echo "USER_PASSWORD=password123" >> $GITHUB_ENV
+        env:
+          API_KEY: ${{ secrets.API_KEY }}
+
+      # Step 3: Run Tests with TestDriver.ai
+      - name: Run Tests with TestDriver.ai
+        uses: testdriverai/action@main
+        with:
+          key: ${{ secrets.TESTDRIVER_API_KEY }}
+          prompt: |
+            1. Log in with the test user
+            2. Perform actions on the dashboard
+          prerun: |
+            echo "Launching browser with test user credentials..."
+            echo "Email: $USER_EMAIL"
+            echo "Password: $USER_PASSWORD"
+        env:
+          USER_EMAIL: ${{ env.USER_EMAIL }}
+          USER_PASSWORD: ${{ env.USER_PASSWORD }}
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          FORCE_COLOR: "3"
+
+      # Step 4: Teardown - Delete the test user
+      - name: Teardown Test User
+        if: always()
+        run: |
+          echo "Deleting test user via API..."
+          curl -X DELETE -H "Authorization: Bearer ${{ secrets.API_KEY }}" https://api.example.com/users/$USER_ID
+          echo "Test user deleted."
+        env:
+          USER_ID: ${{ env.USER_ID }}
+          API_KEY: ${{ secrets.API_KEY }}```
+
+
+---
+
+## Workflow Steps Explained
+
+### 1. **Setup Test User**
+- **Purpose**: Create a test user via an API before running the tests.
+- **How It Works**:
+  - The `Setup Test User` step sends a `POST` request to the API to create a new user.
+  - The user ID, email, and password are extracted from the API response and stored as environment variables (`USER_ID`, `USER_EMAIL`, `USER_PASSWORD`).
+- **Example Output**:
+  - `USER_ID`: `12345`
+  - `USER_EMAIL`: `test@example.com`
+  - `USER_PASSWORD`: `password123`
+
+---
+
+### 2. **Run Tests with TestDriver.ai**
+- **Purpose**: Execute tests using the TestDriver.ai action.
+- **How It Works**:
+  - The `USER_EMAIL` and `USER_PASSWORD` environment variables are passed to the `prerun` script.
+  - The test prompts use these credentials to log in and perform actions.
+
+---
+
+### 3. **Teardown Test User**
+- **Purpose**: Delete the test user via an API after the tests are complete.
+- **How It Works**:
+  - The `Teardown Test User` step sends a `DELETE` request to the API to remove the test user.
+  - The `if: always()` condition ensures this step runs even if the TestDriver.ai action fails.
+
+---
+
+## Best Practices for Setup and Teardown
+
+1. **Use APIs for Setup and Teardown**:
+   - Use APIs to create and delete test data dynamically, ensuring a clean environment for each test run.
+
+2. **Pass Data via Environment Variables**:
+   - Store setup data (e.g., user credentials) in environment variables and pass them to the TestDriver.ai action.
+
+3. **Ensure Teardown Always Runs**:
+   - Use `if: always()` to ensure teardown tasks are executed regardless of the test results.
+
+4. **Log Setup and Teardown Steps**:
+   - Add `echo` statements to log the progress of setup and teardown tasks for easier debugging.
+
+5. **Test Locally**:
+   - Verify setup and teardown scripts locally before integrating them into the workflow.
+
+---
+
+## Example Use Cases
+
+### 1. **User Management**
+- Create a test user during setup.
+- Delete the test user during teardown.
+
+### 2. **Database Operations**
+- Insert test data into a database during setup.
+- Remove the test data during teardown.
+
+### 3. **Mock Services**
+- Start a mock API server during setup.
+- Stop the mock server during teardown.
+
+---
+
+By structuring your workflow to handle **setup** before the TestDriver.ai action and **teardown** after it, you can ensure a clean and reliable test environment for every run. This approach also ensures that teardown tasks are executed regardless of the test results, maintaining a consistent state for subsequent runs.