@testdriverai/agent 7.8.0-canary.10

package/docs/v6/guide/locating.mdx

@@ -0,0 +1,141 @@
+---
+title: "Locating Elements in TestDriver"
+sidebarTitle: "Locating Elements"
+description: "Learn how to effectively describe and locate elements using TestDriver's visual understanding capabilities."
+icon: "magnifying-glass"
+---
+
+TestDriver uses **visual understanding** to locate elements on the screen. Unlike traditional testing frameworks that rely on selectors or element IDs, TestDriver identifies elements based on their **visual appearance**. This guide explains how to describe elements effectively to ensure TestDriver can locate them accurately.
+
+---
+
+## KeypPrinciples for locating elements
+
+1. **Describe the Element Visually**:
+
+   - Focus on the **appearance** of the element, not its behavior or function.
+   - Mention unique visual traits such as **text**, **color**, **size**, **position**, or **icon**.
+
+2. **Avoid Behavioral Descriptions**:
+
+   - don't describe what the element does (for example, "button that submits the form").
+   - Instead, describe how it looks (for example, "blue button with the text 'Submit' in the bottom-right corner").
+
+3. **Use Unique Identifiers**:
+
+   - If the element has visible text, include it in the description.
+   - For icons or images, describe their shape, color, or associated label.
+
+4. **Leverage Visual Feedback**:
+   - TestDriver will draw **yellow boxes** around detected elements during test execution.
+   - Use this feedback to verify that the correct element is being targeted.
+
+---
+
+## Examples of effective descriptions
+
+### 1. Buttons
+
+- **Good**: "Blue button with the text 'Sign In' in the top-right corner."
+- **Bad**: "Button that logs the user in."
+
+### 2. Links
+
+- **Good**: "Hyperlink with the text 'Learn More' in the footer."
+- **Bad**: "Link that navigates to the About page."
+
+### 3. Icons
+
+- **Good**: "Magnifying glass icon next to the search bar."
+- **Bad**: "Search icon that opens the search feature."
+
+### 4. Input fields
+
+- **Good**: "White input box labeled 'Email Address' above the password field."
+- **Bad**: "Field where the user enters their email."
+
+### 5. Images
+
+- **Good**: "Company logo in the top-left corner, a blue circle with white text."
+- **Bad**: "Logo that redirects to the homepage."
+
+---
+
+## How to write TestDriver commands
+
+### Example: Locating a button
+
+#### TestDriver command:
+
+```yaml
+- command: hover-text
+  text: Sign In
+  description: Blue button with the text 'Sign In' in the top-right corner
+  action: click
+```
+
+---
+
+### Example: Locating an icon
+
+#### TestDriver command:
+
+```yaml
+- command: hover-image
+  description: Magnifying glass icon next to the search bar
+  action: click
+```
+
+---
+
+### Example: Locating a link
+
+#### TestDriver command:
+
+```yaml
+- command: hover-text
+  text: Learn More
+  description: Hyperlink with the text 'Learn More' in the footer
+  action: click
+```
+
+---
+
+## Debugging element detection
+
+1. **Run the Test**:
+
+   - Execute the test using TestDriver.
+   - TestDriver will draw **yellow boxes** around detected elements.
+
+2. **Verify the Correct Element**:
+
+   - Ensure the yellow box highlights the intended element.
+   - If the wrong element is highlighted, refine your description.
+
+3. **Adjust the Description**:
+   - Add more specific visual details (for example, color, position, associated text).
+   - Avoid generic terms like "button" or "icon" without additional context.
+
+---
+
+## Best practices
+
+1. **Be Specific**:
+
+   - Include as many unique visual traits as possible to differentiate the element from others.
+
+2. **Test Incrementally**:
+
+   - Run tests after adding or modifying commands to verify element detection.
+
+3. **Use Visual Feedback**:
+
+   - Leverage the yellow boxes drawn by TestDriver to confirm the correct element is being targeted.
+
+4. **Avoid Overloading Descriptions**:
+   - Keep descriptions concise but detailed enough to uniquely identify the element.
+
+---
+
+By focusing on **visual descriptions** and leveraging TestDriver's feedback, you can accurately locate elements and create robust, reliable tests.

package/docs/v6/guide/protips.mdx

@@ -0,0 +1,43 @@
+---
+title: "Pro Tips"
+sidebarTitle: "Pro Tips"
+description: "Navigate TestDriver like a pro with these tips and tricks."
+icon: "star"
+---
+
+TestDriver is a vision-based testing tool and is not aware of the underlying HTML structure of the page. Pay attention to the following tips to improve your testing experience and reduce test flake.
+
+### Use unique descriptions for [`hover-image`](/commands/hover-image) and [`hover-text`](/commands/hover-text)
+
+When executing the [`hover-image`](/commands/hover-image) and [`hover-text`](/commands/hover-text) command, TestDriver may use the description to identify the image, especially if multiple images are visible. If multiple images on the screen match the description, TestDriver may not be able to determine which one to interact with. To improve accuracy, use specific and unique descriptions for images.
+
+TestDriver may not accurately locate matches in certain cases. This can lead to unexpected behavior when using the [`hover-image`](/commands/hover-image) or [`hover-text`](/commands/hover-text) command. For example, if you specify a description that's too generic or similar to other images on the screen, TestDriver may not be able to identify the correct image to interact with. To improve accuracy, use specific and unique descriptions for images and text.
+When executing the [`hover-image`](/commands/hover-image) and [`hover-text`](/commands/hover-text) commands, TestDriver uses the provided description to identify the target image or text. If the description is too generic or matches multiple elements on the screen, TestDriver may not be able to determine which one to interact with, leading to unexpected behavior. To improve accuracy, always use specific and unique descriptions for both images and text.
+
+### Avoid using [`hover-text`](/commands/hover-text) for single characters (ex: `1`, `>`, `|`)
+
+To improve accuracy of our model, we do not include single characters or symbols in the matching data. This means that [`hover-text`](/commands/hover-text) may not work as expected for these cases. If you need to interact with single characters or symbols, consider using the [`match-image`](/commands/match-image) command with a screenshot of the target element instead.
+
+### Use `<label>`s to match empty text inputs.
+
+To focus empty text inputs, prefer to [`hover-text`](/commands/hover-text) on the label associated with the input. This [should](https://www.w3.org/WAI/tutorials/forms/labels/) focus the input field.
+
+This is more reliable than trying to match the empty input itself, which may not have any visible text. You can always use [`match-image`](/commands/match-image) to match the input field if needed.
+
+### Provide strict criteria for [`assert`](/commands/assert)
+
+Assertions use the current state of the page to determine if the step passes or fails. [`assert`](/commands/assert) is not aware of the past, so avoid writing "assert" steps that rely on previous state. Instead, use [`assert`](/commands/assert) to check the current state of the page against strict criteria.
+
+## Reporting issues
+
+If you encounter any issues while using TestDriver, please report them to us. You can do this by creating a new issue in the [discord](https://discord.com/invite/cWDFW8DzPm). We appreciate your feedback and will work to resolve any problems as quickly as possible.
+
+<Card
+  title="Report an Issue"
+  icon="arrow-turn-down-right"
+  iconType="duotone"
+  horizontal
+  href="https://discord.com/invite/cWDFW8DzPm"
+>
+  We'd love to hear from you!
+</Card>

package/docs/v6/guide/variables.mdx

@@ -0,0 +1,143 @@
+---
+title: "Variables"
+sidebarTitle: "Variables"
+description: "Learn how to use variables in TestDriver for dynamic data handling."
+icon: "square-root-variable"
+---
+
+# Using variables in TestDriver
+
+Variables in **TestDriver** allow you to dynamically store and reuse data during test execution. This feature is particularly useful for handling dynamic content, passing data between steps, and customizing test behavior based on runtime conditions.
+
+- Generate a random number or string and use it to fill out a form.
+- Capture API responses and validate their content.
+- Capture text or values from the screen and use them in assertions.
+- Pass different values to the test using environment variables for testing multiple scenarios.
+
+By leveraging variables in TestDriver, you can create dynamic, flexible, and reusable test scripts that adapt to changing conditions and data.
+
+#### Example:
+
+```yaml random.yaml highlight={6, 11, 19, 20}
+version: 6.0.0
+steps:
+  - prompt: Generate a random number
+    commands:
+      - command: exec
+        output: randomNumber
+        js: |
+          result = Math.floor(Math.random() * 1000);
+      - command: exec
+        js: |
+          console.log("Generated Random Number: ${OUTPUT.randomNumber}");
+
+  - prompt: Use the random number in a form
+    commands:
+      - command: hover-text
+        text: Enter Number
+        description: Input field for numbers
+        action: click
+      - command: type
+        text: ${OUTPUT.randomNumber}
+```
+
+## CI/CD pipeline example (platform-agnostic)
+
+You can run TestDriver in any CI/CD pipeline. Here is an example using the CLI in a generic workflow (adapt for your platform, e.g., GitHub Actions, GitLab CI, CircleCI, Jenkins, etc.):
+
+```yaml workflows/testdriver.yml focus={6-13}
+steps:
+  - name: Check out repository
+    # For GitHub Actions: uses: actions/checkout@v4
+    # For GitLab CI: git clone ...
+
+  - name: Run TestDriver CLI
+    env:
+      TD_API_KEY: ${{ secrets.TD_API_KEY }}
+      TD_USERNAME: ${{ secrets.TD_USERNAME }}
+      TD_PASSWORD: ${{ secrets.TD_PASSWORD }}
+      # Add any other required environment variables
+    run: |
+      npx testdriverai@latest run testdriver/login.yaml
+```
+
+Now inside the `login.yaml` test file you can refer the variables
+
+```yaml login.yaml highlight={21, 22, 27, 28}
+version: 6.0.0
+steps:
+  - prompt: Open the homepage
+    commands:
+      - command: hover-text
+        text: Login
+        description: Login button in the top-right corner
+        action: click
+
+  - prompt: Verify the login form is displayed
+    commands:
+      - command: assert
+        expect: The login form is displayed
+
+  - prompt: Enter credentials and submit
+    commands:
+      - command: hover-text
+        text: Email
+        description: Email input field
+        action: click
+      - command: type
+        text: ${TD_USERNAME}
+      - command: hover-text
+        text: Password
+        description: Password input field
+        action: click
+      - command: type
+        text: ${TD_PASSWORD}
+      - command: hover-text
+        text: Submit
+        description: Submit button
+        action: click
+```
+
+## **Capturing outputs as variables**
+
+#### Example:
+
+```yaml capture-text.yaml highlight={5, 6, 14, 15}
+version: 6.0.0
+steps:
+  - prompt: Capture text from the screen
+    commands:
+      - command: remember
+        output: welcomeMessage
+        description: Capture the welcome message
+
+  - prompt: Verify the dashboard is displayed
+    commands:
+      - command: exec
+        js: |
+          console.log("Captured Text: ${OUTPUT.welcomeMessage}");
+      - command: assert
+        expect: "${OUTPUT.welcomeMessage}" == "Welcome, Test User!"
+```
+
+## Best practices for using variables
+
+1. **Use Descriptive Names**:
+
+- Name variables clearly to indicate their purpose (for example, `capturedText`, `randomNumber`).
+
+2. **Secure Sensitive Data**:
+
+- Use environment variables for sensitive information like credentials or API keys.
+
+3. **Log Variable Values**:
+
+- Use `console.log` or similar commands to log variable values for debugging.
+
+4. **Test Variable Logic Locally**:
+
+- Verify the logic for custom variables locally before integrating them into workflows.
+
+5. **Combine Variables with Assertions**:
+
+- Use variables in assertions to validate dynamic content or conditions.

package/docs/v6/guide/waiting.mdx

@@ -0,0 +1,130 @@
+---
+title: Waiting in TestDriver
+sidebarTitle: Waiting
+description: "Waiting in TestDriver: Ensuring Stability and Reducing Flakiness"
+icon: "clock"
+---
+
+# Waiting in TestDriver: Ensuring stability and reducing flakiness
+
+Waiting is a critical feature in TestDriver that ensures tests are stable and reliable, even in dynamic or slow-loading environments.
+
+- Automatically adjusts to varying load times, reducing the need for hardcoded delays.
+- Ensures that tests wait for dynamic elements to appear, reducing false negatives caused by incomplete rendering.
+- By waiting for the screen and network to stabilize, `redraw` minimizes the risk of interacting with incomplete or incorrect elements.
+
+## Summary of waiting features
+
+| **Feature**                                      | **Description**                                                                |
+| ------------------------------------------------ | ------------------------------------------------------------------------------ |
+| **`redraw`(automatic)**                          | TestDriver automatically waits for UI changes and network activity to resolve. |
+| **[`wait-for-text`](/commands/wait-for-text)**   | Waits for specific text to appear on the screen.                               |
+| **[`wait-for-image`](/commands/wait-for-image)** | Waits for a specific image or visual element to appear on the screen.          |
+
+## Key waiting features in TestDriver
+
+1. **Automatic Waiting with `redraw`**:
+
+- TestDriver automatically waits for the machine before moving to the next step.
+- This includes waiting for:
+  - UI changes to complete.
+  - Network activity to stabilize (for example, API calls).
+- Reduces the need for manual waits, making tests faster and less prone to flakiness.
+
+2. **[`wait-for-text`](/commands/wait-for-text) Command**:
+
+- Waits for specific text to appear on the screen.
+- Useful for validating dynamic content or ensuring that a page has fully loaded before proceeding.
+
+3. **[`wait-for-image`](/commands/wait-for-image) Command**:
+
+- Waits for a specific image or visual element to appear on the screen.
+- Ideal for verifying the presence of icons, logos, or other graphical elements.
+
+## Commands for explicit waiting
+
+### 1. **[`wait-for-text`](/commands/wait-for-text)**
+
+The [`wait-for-text`](/commands/wait-for-text) command pauses the test until the specified text appears on the screen. This is particularly useful for dynamic content that takes time to load.
+
+#### Syntax
+
+```yaml
+- command: wait-for-text
+  text: <text to wait for>
+  timeout: <time in milliseconds> # Optional, defaults to 5000ms or 5 seconds
+```
+
+#### Example
+
+```yaml
+- command: wait-for-text
+  text: Welcome, Test User!
+  timeout: 10000
+```
+
+In this example, the test waits up to 10 seconds for the text "Welcome, Test User!" to appear.
+
+### **[`wait-for-image`](/commands/wait-for-image)**
+
+The [`wait-for-image`](/commands/wait-for-image) command pauses the test until the specified image or visual element appears on the screen.
+
+#### Syntax:
+
+```yaml
+- command: wait-for-image
+  description: <description of the image>
+  timeout: <time in milliseconds> # Optional, defaults to 10000ms or 10 seconds
+```
+
+#### Example:
+
+```yaml
+- command: wait-for-image
+  description: Company logo in the top-left corner
+  timeout: 8000
+```
+
+In this example, the test waits up to 8 seconds for the company logo to appear in the top-left corner.
+
+## Automatic waiting with `redraw`
+
+TestDriver's **`redraw` function** is a built-in mechanism that automatically waits for the screen to stabilize before proceeding to the next step. This includes:
+
+1. **UI Changes**:
+
+- Waits for animations, transitions, or DOM updates to complete.
+- Ensures that the screen is fully rendered before interacting with elements.
+
+2. **Network Stabilization**:
+
+- Waits for network activity (for example, API calls, AJAX requests) to finish.
+- Ensures that dynamic content is fully loaded before proceeding.
+
+3. **Screen Stabilization**:
+
+- Continuously monitors the screen for changes and only moves forward when the screen is stable.
+
+---
+
+## Best practices for waiting
+
+2. **Leverage Automatic Waiting**:
+
+- Rely on TestDriver's `redraw` function to handle most waiting scenarios automatically.
+
+1. **Use Explicit Waiting for Dynamic Elements**:
+
+- Use [`wait-for-text`](/commands/wait-for-text) or [`wait-for-image`](/commands/wait-for-image) for elements that take time to load.
+
+3. **Avoid Hardcoded Delays**:
+
+- Replace hardcoded `sleep` or [`wait`](/commands/wait) commands with dynamic waiting commands to improve test reliability.
+
+4. **Set Appropriate Timeouts**:
+
+- Use reasonable timeouts for explicit waiting commands to balance reliability and test execution time.
+
+5. **Test Incrementally**:
+
+- Add waiting commands step-by-step to ensure each part of the workflow is stable.

package/docs/v6/importing/csv.mdx

@@ -0,0 +1,196 @@
+---
+title: "CSV Import"
+sidebarTitle: "CSV Import"
+description: "Convert user stories from CSV files into individual TestDriver test files using Node.js and GitHub Actions."
+icon: "table"
+---
+
+This guide demonstrates how to convert user stories from **CSV files** into individual TestDriver test files using **Node.js**. Each user story will be saved as its own YAML test file, and the tests will be executed in parallel using a **GitHub Actions workflow**.
+
+---
+
+## Workflow overview
+
+1. **Export User Stories**: Extract user stories from TestRail, CSV, or Jira.
+2. **Convert to Test Files**: Use a Node.js script to generate individual YAML test files for each user story.
+3. **Run Tests**: Use a CI/CD workflow to execute the tests in parallel with the TestDriver CLI.
+
+---
+
+## Step 1: Export user stories
+
+Export user stories from your source (TestRail, CSV, or Jira) and save them in a structured format (for example, JSON or CSV). For example:
+
+### Example CSV file (`user_stories.csv`):
+
+| User Story ID | User Story Title          | Description                          |
+| ------------- | ------------------------- | ------------------------------------ |
+| 1             | Log in to the application | User logs in with valid credentials. |
+| 2             | Search for a product      | User searches for a product by name. |
+| 3             | Add product to cart       | User adds a product to the cart.     |
+
+---
+
+## Step 2: Node.js script to convert user stories into test files
+
+Create a Node.js script to read the exported data and generate individual YAML test files for each user story.
+
+### Install required dependencies
+
+```bash
+npm install yaml fs csv-parser
+
+```
+
+### Node.js script (`generate-tests.js`)
+
+```javascript
+const fs = require("fs");
+const path = require("path");
+const yaml = require("yaml");
+const csv = require("csv-parser");
+
+// Input and output directories
+const inputFile = "./user_stories.csv";
+const outputDir = "./testdriver";
+
+// Ensure the output directory exists
+if (!fs.existsSync(outputDir)) {
+  fs.mkdirSync(outputDir, { recursive: true });
+}
+
+// Function to generate a YAML test file for each user story
+function generateTestFile(userStory) {
+  const {
+    "User Story ID": id,
+    "User Story Title": title,
+    Description,
+  } = userStory;
+
+  const testContent = {
+    version: "6.0.0",
+    steps: [
+      {
+        prompt: title,
+      },
+    ],
+  };
+
+  const yamlContent = yaml.stringify(testContent);
+  const fileName = `test_${id}.yaml`;
+  const filePath = path.join(outputDir, fileName);
+
+  fs.writeFileSync(filePath, yamlContent, "utf8");
+  console.log(`Generated test file: ${filePath}`);
+}
+
+// Read the CSV file and generate test files
+fs.createReadStream(inputFile)
+  .pipe(csv())
+  .on("data", (row) => {
+    generateTestFile(row);
+  })
+  .on("end", () => {
+    console.log("All test files generated successfully!");
+  });
+```
+
+### Run the script
+
+```bash
+node generate-tests.js
+```
+
+This script will generate individual YAML test files (for example, `test_1.yaml`, `test_2.yaml`) in the `testdriver/` directory.
+
+---
+
+## Step 3: CI/CD workflow to run tests in parallel
+
+Create a CI/CD workflow to execute the generated test files in parallel using the TestDriver CLI. Below is a generalized example for any CI/CD system (adapt for your platform):
+
+```yaml .github/workflows/testdriver.yaml
+name: Run TestDriver Tests
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  gather-test-files:
+    name: Gather Test Files
+    runs-on: ubuntu-latest
+    outputs:
+      test_files: ${{ steps.test_list.outputs.files }}
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Find all test files
+        id: test_list
+        run: |
+          FILES=$(ls ./testdriver/*.yaml)
+          FILES_JSON=$(echo "$FILES" | jq -R -s -c 'split("\n")[:-1]')
+          echo "files=$FILES_JSON" >> $GITHUB_OUTPUT
+
+  run-tests:
+    name: Run Tests in Parallel
+    needs: gather-test-files
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        test_file: ${{ fromJson(needs.gather-test-files.outputs.test_files) }}
+      fail-fast: false
+    steps:
+      - name: Check out repository
+        uses: actions/checkout@v4
+
+      - name: Run TestDriver CLI
+        run: npx testdriverai@latest run ${{ matrix.test_file }}
+        env:
+          TD_API_KEY: ${{ secrets.TD_API_KEY }}
+```
+
+---
+
+## Step 4: Secure API key
+
+Store your TestDriver API key as a GitHub secret (for example, `TD_API_KEY`) to securely authenticate your tests.
+
+---
+
+## Step 5: Commit and push
+
+1. Commit the `generate-tests.js` script, `user_stories.csv`, and `.github/workflows/run-tests.yaml` to your repository.
+2. Push the changes to the `main` branch.
+
+---
+
+## Step 6: Run the workflow
+
+The GitHub Actions workflow will automatically:
+
+1. Gather all test files in the `testdriver/` directory.
+2. Execute each test file in parallel using the matrix strategy.
+
+---
+
+## Example output
+
+- **Generated Test Files**:
+
+  - `testdriver/test_1.yaml`
+  - `testdriver/test_2.yaml`
+  - `testdriver/test_3.yaml`
+
+- **GitHub Actions Dashboard**:
+  - Each test file is executed as a separate job, and the results are displayed in the Actions tab.
+
+---
+
+## Conclusion
+
+This setup automates the process of converting user stories into individual test files and running them in parallel using the TestDriver CLI in your CI/CD pipeline. It ensures comprehensive test coverage while optimizing execution time.