testdriverai 5.3.13 → 5.3.15

package/docs/guide/code.mdx

@@ -5,166 +5,56 @@ description: "Learn how to integrate custom Node.js scripts into your TestDriver
 icon: "code"
 ---
 
-TestDriver.ai allows you to execute custom **Node.js** scripts within your test workflows using the `exec` command. This feature, introduced in version `5.1.0`, enables you to integrate custom logic, such as generating one-time passwords (OTPs), hitting APIs, or performing other dynamic operations, directly into your tests.
-
----
+TestDriver.ai allows you to execute custom **Node.js** scripts and shell scripts within your test workflows using the `exec` command. This feature, introduced in version `5.1.0`, enables you to integrate custom logic, such as generating one-time passwords (OTPs), hitting APIs, or performing other dynamic operations, directly into your tests.
 
 ## Key Features
 
 1. **Run Node.js Scripts**:
-   - Execute custom JavaScript code within your test steps.
-   - Use NPM modules to extend functionality.
+  - Execute custom JavaScript code within your test steps.
+  - Use NPM modules to extend functionality.
 
 2. **Dynamic Outputs**:
-   - Store the result of your script in a variable for use in subsequent steps.
+  - Store the result of your script in a variable for use in subsequent steps.
 
 3. **NPM Support**:
-   - Install and use NPM packages in your scripts.
-
----
+  - Install and use NPM packages in your scripts.
 
-## Example: One-Time Password (OTP) Validator
+## Updated Example: One-Time Password (OTP) Validator
 
 This example demonstrates how to generate a one-time password (OTP) using the `totp-generator` NPM package and use it in a test.
 
-### Workflow File: `.github/workflows/testdriver-otp.yml`
-
-```yaml
-name: TestDriver.ai
-
-on:
-  push:
-    branches: ["main"]
-  pull_request:
-  workflow_dispatch:
-
-jobs:
-  test:
-    name: "TestDriver"
-    runs-on: ubuntu-latest
-    steps:
-      - uses: testdriverai/action@main
-        with:
-          version: "5.1.0"
-          key: ${{ secrets.TESTDRIVER_API_KEY }}
-          prompt: |
-            1. /run testdriver/testdriver.yaml
-          prerun: |
-            cd $env:TEMP
-            npm init -y
-            npm install totp-generator
-            exit
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-          FORCE_COLOR: "3"
-
-
-```
-
-
----
-
 ### Test File: `testdriver/testdriver.yaml`
 
 ```yaml
-version: 5.1.0
-session: 67e57d614dc25283aa0872a9
+version: 5.3.8
 steps:
-  - prompt: Log in with the totp
-    commands:
-      - command: exec
-        output: totp
-        js: |
-          const { TOTP } = require("totp-generator");
-          let otp = TOTP.generate("JBSWY3DPEB3W64TMMQQQ").otp;
-          result = otp;
-      - command: exec
-        js: |
-          console.log("${OUTPUT.totp}");
-      - command: type
-        text: ${OUTPUT.totp}
-
-
+  - commands:
+    - command: exec
+      lang: shell
+      linux: |
+        npm install totp-generator
+    - command: exec
+      lang: js
+      output: totp
+      linux: |
+        const { TOTP } = require("totp-generator");
+        let otp = TOTP.generate("JBSWY3DPEB3W64TMMQQQ").otp;
+        console.log(otp);
+        result = otp;
+    - command: type
+      text: ${OUTPUT.totp}
 ```
 
+## Additional Details
 
----
-
-## How It Works
-
-### Parameters for `exec`
-
-- **`js`**: The Node.js script to execute. The output must be assigned to the `result` variable.
-- **`output`**: The name of the variable to store the result. This variable can be referenced in subsequent steps as `${OUTPUT.<var>}`.
-
----
-
-### Running the Demo Locally
-
-1. Clone the repository.
-2. Install dependencies:
-
-   ```bash
-   npm install
-
-```   
-
-3. Run the test:
-
-   ```bash
-   testdriverai run testdriver/testdriver.yaml
-
-```   
-
----
-
-### Example Output
-
-- **Step 1**: Generates an OTP using the `totp-generator` package.
-- **Step 2**: Logs the OTP to the console.
-- **Step 3**: Types the OTP into the application.
-
----
+- The `exec` command now takes a `lang` argument and supports different operating systems (`linux`, `mac`, and `windows`).
+- Supported `lang` values are `js` or `shell`:
+  - `js` code is executed in a NodeJS VM on the host machine (your computer).
+  - `shell` code is executed in the shell on the runner.
+- Code specified in `linux`, `mac`, and `windows` is executed based on the platform of the runner machine.
 
 ## Pro Tips
 
-1. **Avoid Overwriting `result`**:
-   - Always assign the output of your script to the `result` variable.
-
-2. **Install Dependencies**:
-   - Ensure all required NPM packages are installed locally and in the `prerun` script when using GitHub Actions.
-
-3. **Node.js Context**:
-   - The script runs in the same context as the calling process and uses Node.js's [VM module](https://nodejs.org/api/vm.html) internally.
-
----
-
-## Gotchas
-
-1. **Runner Support**:
-   - This feature is supported only on **hosted Windows runners**. Hosted Linux runners do not yet support `prerun`.
-
-2. **NPM Initialization**:
-   - Ensure you initialize NPM (`npm init`) and install required packages before running the test.
-
----
-
-## Using NPM Modules from Scratch
-
-1. Initialize a new Node.js project:
-
-   ```bash
-   npm init
-
-```   
-
-2. Install the required package:
-
-   ```bash
-   npm install totp-generator --save
-
-```   
-
----
-
-By leveraging the `exec` command, you can extend TestDriver.ai's capabilities with custom Node.js scripts, enabling dynamic and powerful test workflows.
+- Always assign the output of your script to the `result` variable.
+- Ensure all required NPM packages are installed locally and in the `prerun` script when using GitHub Actions.
+- The script runs in the same context as the calling process and uses Node.js's [VM module](https://nodejs.org/api/vm.html) internally.

package/docs/guide/lifecycle.mdx

@@ -0,0 +1,41 @@
+---
+title: "Setup & Teardown"
+sidebarTitle: "Setup & Teardown"
+description: "How to handle pre and post test tasks."
+icon: "hammer"
+---
+
+## Pre-run Configuration
+
+If a file is found at `testdriver/lifecycle/prerun.yaml`, it's executed before the test begins. This file works like any other TestDriver file and is commonly used to perform tasks such as opening a browser, navigating to a specific page, or resetting the application state.
+
+For example, the `prerun.yaml` file can be used to open the Chrome browser and navigate to a page, similar to the example provided above. This ensures that the test environment is properly set up before the test starts.
+
+### Example
+
+Here is an example of a `prerun.yaml` file:
+
+```yaml
+version: 5.1.1
+steps:
+  - prompt: launch chrome
+    commands:
+      - command: exec
+        lang: shell
+        linux: |
+          jumpapp google-chrome --disable-fre --no-default-browser-check --no-first-run "${TD_WEBSITE}" &
+          exit
+        mac: |
+          open -na "Google Chrome" --args --disable-fre --no-default-browser-check --no-first-run --disable-features=PasswordManagerEnabled "${TD_WEBSITE}" &
+          exit
+        windows:
+          Start-Process "C:/Program Files/Google/Chrome/Application/chrome.exe" -ArgumentList "--start-maximized", "${TD_WEBSITE}"
+          exit
+      - command: wait-for-text
+        text: "Google Chrome"
+        timeout: 30000
+```
+
+### Notes
+
+- Currently, there is no teardown task implemented. However, when running on GitHub Actions, the virtual machine (VM) is ephemeral, meaning everything is destroyed after the test run.

package/docs/guide/locating.mdx

@@ -16,7 +16,7 @@ TestDriver.ai uses **visual understanding** to locate elements on the screen. Un
    - Mention unique visual traits such as **text**, **color**, **size**, **position**, or **icon**.
 
 2. **Avoid Behavioral Descriptions**:
-   - Do not describe what the element does (e.g., "button that submits the form").
+   - don't describe what the element does (e.g., "button that submits the form").
    - Instead, describe how it looks (e.g., "blue button with the text 'Submit' in the bottom-right corner").
 
 3. **Use Unique Identifiers**:

package/docs/guide/variables.mdx

@@ -6,59 +6,39 @@ icon: "square-root-variable"
 
 Variables in **TestDriver.ai** allow you to dynamically store and reuse data during test execution. This feature is particularly useful for handling dynamic content, passing data between steps, and customizing test behavior based on runtime conditions.
 
----
-
-## Key Features of Variables in TestDriver.ai
-
-1. **Dynamic Data Handling**:
-   - Store data (e.g., text, numbers, or results) during test execution.
-   - Reuse stored data in subsequent steps.
-
-2. **Environment Variables**:
-   - Use environment variables to pass sensitive information (e.g., usernames, passwords, API keys) securely.
-
-3. **Custom Variables**:
-   - Define and manipulate custom variables within the test file.
-
-4. **Output Variables**:
-   - Capture outputs from commands (e.g., text from the screen, API responses) and store them in variables.
-
----
-
-## Types of Variables
-
-### 1. **Environment Variables**
-Environment variables are passed to the test from the workflow or the local environment. They are typically used for sensitive data like credentials or configuration values.
-
-#### Example:
-- **Workflow Environment Variable**: `${TD_USERNAME}`
-- **Test File Reference**: `${TD_USERNAME}`
-
----
-
-### 2. **Custom Variables**
-Custom variables are defined and manipulated within the test file. They can store dynamic data generated during the test.
-
-#### Example:
-- Define a variable: `result = "Test Passed"`
-- Reference the variable: `${OUTPUT.result}`
-
----
-
-### 3. **Output Variables**
-Output variables capture the result of a command and store it for later use.
+- Generate a random number or string and use it to fill out a form.
+- Capture API responses and validate their content.
+- Capture text or values from the screen and use them in assertions.
+- Pass different values to the test using environment variables for testing multiple scenarios.
 
-#### Example:
-- Capture text from the screen: `output: capturedText`
-- Reference the variable: `${OUTPUT.capturedText}`
+By leveraging variables in TestDriver.ai, you can create dynamic, flexible, and reusable test scripts that adapt to changing conditions and data.
 
----
+#### Test File Example:
 
-## How to Use Variables in TestDriver.ai
+```yaml
+version: 5.1.0
+steps:
+  - prompt: Generate a random number
+    commands:
+      - command: exec
+        output: randomNumber
+        js: |
+          result = Math.floor(Math.random() * 1000);
+      - command: exec
+        js: |
+          console.log("Generated Random Number: ${OUTPUT.randomNumber}");
 
-### 1. **Using Environment Variables**
+  - prompt: Use the random number in a form
+    commands:
+      - command: hover-text
+        text: Enter Number
+        description: Input field for numbers
+        action: click
+      - command: type
+        text: ${OUTPUT.randomNumber}
+```
 
-#### Workflow Example:
+## GitHub Actions Example
 
 ```yaml
 name: TestDriver.ai with Variables
@@ -90,12 +70,8 @@ jobs:
           TD_PASSWORD: ${{ secrets.TD_PASSWORD }}
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           FORCE_COLOR: "3"
-
-
 ```
-
-
-#### Test File Example:
+#### Test File
 
 ```yaml
 version: 4.2.18
@@ -118,46 +94,9 @@ steps:
         text: Log In
         description: Log In button
         action: click
-
-
 ```
 
-
----
-
-### 2. **Defining and Using Custom Variables**
-
-#### Test File Example:
-
-```yaml
-version: 5.1.0
-steps:
-  - prompt: Generate a random number
-    commands:
-      - command: exec
-        output: randomNumber
-        js: |
-          result = Math.floor(Math.random() * 1000);
-      - command: exec
-        js: |
-          console.log("Generated Random Number: ${OUTPUT.randomNumber}");
-
-  - prompt: Use the random number in a form
-    commands:
-      - command: hover-text
-        text: Enter Number
-        description: Input field for numbers
-        action: click
-      - command: type
-        text: ${OUTPUT.randomNumber}
-
-
-```
-
-
----
-
-### 3. **Capturing Outputs as Variables**
+### **Capturing Outputs as Variables**
 
 #### Test File Example:
 
@@ -177,46 +116,21 @@ steps:
           console.log("Captured Text: ${OUTPUT.welcomeMessage}");
       - command: assert
         expect: "${OUTPUT.welcomeMessage}" == "Welcome, Test User!"
-
-
 ```
 
-
----
-
 ## Best Practices for Using Variables
 
 1. **Use Descriptive Names**:
-   - Name variables clearly to indicate their purpose (e.g., `capturedText`, `randomNumber`).
+  - Name variables clearly to indicate their purpose (e.g., `capturedText`, `randomNumber`).
 
 2. **Secure Sensitive Data**:
-   - Use environment variables for sensitive information like credentials or API keys.
+  - Use environment variables for sensitive information like credentials or API keys.
 
 3. **Log Variable Values**:
-   - Use `console.log` or similar commands to log variable values for debugging.
+  - Use `console.log` or similar commands to log variable values for debugging.
 
 4. **Test Variable Logic Locally**:
-   - Verify the logic for custom variables locally before integrating them into workflows.
+  - Verify the logic for custom variables locally before integrating them into workflows.
 
 5. **Combine Variables with Assertions**:
-   - Use variables in assertions to validate dynamic content or conditions.
-
----
-
-## Example Use Cases
-
-### 1. **Dynamic Form Filling**
-- Generate a random number or string and use it to fill out a form.
-
-### 2. **API Testing**
-- Capture API responses and validate their content.
-
-### 3. **Dynamic Assertions**
-- Capture text or values from the screen and use them in assertions.
-
-### 4. **Parameterized Tests**
-- Pass different values to the test using environment variables for testing multiple scenarios.
-
----
-
-By leveraging variables in TestDriver.ai, you can create dynamic, flexible, and reusable test scripts that adapt to changing conditions and data.
+  - Use variables in assertions to validate dynamic content or conditions.

package/docs/guide/waiting.mdx

@@ -8,6 +8,10 @@ icon: "hourglass-half"
 
 Waiting is a critical feature in TestDriver.ai that ensures tests are stable and reliable, even in dynamic or slow-loading environments. 
 
+- Automatically adjusts to varying load times, reducing the need for hardcoded delays.
+- Ensures that tests wait for dynamic elements to appear, reducing false negatives caused by incomplete rendering.
+- By waiting for the screen and network to stabilize, `redraw` minimizes the risk of interacting with incomplete or incorrect elements.
+
 
 ## Summary of Waiting Features
 
@@ -20,22 +24,20 @@ Waiting is a critical feature in TestDriver.ai that ensures tests are stable and
 ## Key Waiting Features in TestDriver.ai
 
 1. **Automatic Waiting with `redraw`**:
-   - TestDriver.ai automatically waits for the machine before moving to the next step.
-   - This includes waiting for:
-     - UI changes to complete.
-     - Network activity to stabilize (e.g., API calls).
-   - Reduces the need for manual waits, making tests faster and less prone to flakiness.
+  - TestDriver.ai automatically waits for the machine before moving to the next step.
+  - This includes waiting for:
+    - UI changes to complete.
+    - Network activity to stabilize (e.g., API calls).
+  - Reduces the need for manual waits, making tests faster and less prone to flakiness.
 
 2. **`wait-for-text` Command**:
-   - Waits for specific text to appear on the screen.
-   - Useful for validating dynamic content or ensuring that a page has fully loaded before proceeding.
+  - Waits for specific text to appear on the screen.
+  - Useful for validating dynamic content or ensuring that a page has fully loaded before proceeding.
 
 3. **`wait-for-image` Command**:
-   - Waits for a specific image or visual element to appear on the screen.
-   - Ideal for verifying the presence of icons, logos, or other graphical elements.
-
+  - Waits for a specific image or visual element to appear on the screen.
+  - Ideal for verifying the presence of icons, logos, or other graphical elements.
 
----
 
 ## Commands for Explicit Waiting
 
@@ -61,9 +63,7 @@ The `wait-for-text` command pauses the test until the specified text appears on
 
 In this example, the test waits up to 10 seconds for the text "Welcome, Test User!" to appear.
 
----
-
-### 2. **`wait-for-image`**
+### **`wait-for-image`**
 
 The `wait-for-image` command pauses the test until the specified image or visual element appears on the screen.
 
@@ -73,123 +73,48 @@ The `wait-for-image` command pauses the test until the specified image or visual
 - command: wait-for-image
   description: <description of the image>
   timeout: <time in milliseconds> # Optional, defaults to 5000ms
-
-
 ```
 
-
 #### Example:
 
 ```yaml
 - command: wait-for-image
   description: Company logo in the top-left corner
   timeout: 8000
-
-
 ```
 
 In this example, the test waits up to 8 seconds for the company logo to appear in the top-left corner.
 
----
-
 ## Automatic Waiting with `redraw`
 
 TestDriver.ai's **`redraw` function** is a built-in mechanism that automatically waits for the screen to stabilize before proceeding to the next step. This includes:
 
 1. **UI Changes**:
-   - Waits for animations, transitions, or DOM updates to complete.
-   - Ensures that the screen is fully rendered before interacting with elements.
+  - Waits for animations, transitions, or DOM updates to complete.
+  - Ensures that the screen is fully rendered before interacting with elements.
 
 2. **Network Stabilization**:
-   - Waits for network activity (e.g., API calls, AJAX requests) to finish.
-   - Ensures that dynamic content is fully loaded before proceeding.
+  - Waits for network activity (e.g., API calls, AJAX requests) to finish.
+  - Ensures that dynamic content is fully loaded before proceeding.
 
 3. **Screen Stabilization**:
-   - Continuously monitors the screen for changes and only moves forward when the screen is stable.
-
----
-
-### How `redraw` Reduces Flakiness
-
-1. **Eliminates Timing Issues**:
-   - Automatically adjusts to varying load times, reducing the need for hardcoded delays.
-
-2. **Handles Dynamic Content**:
-   - Ensures that tests wait for dynamic elements to appear, reducing false negatives caused by incomplete rendering.
-
-3. **Improves Test Reliability**:
-   - By waiting for the screen and network to stabilize, `redraw` minimizes the risk of interacting with incomplete or incorrect elements.
-
----
-
-## Combining Waiting Features
-
-You can combine explicit waiting commands (`wait-for-text`, `wait-for-image`) with TestDriver.ai's automatic waiting to create robust and reliable tests.
-
-### Example: Login Workflow with Waiting
-
-```yaml
-version: 4.2.18
-steps:
-  - prompt: Open the login page
-    commands:
-      - command: hover-text
-        text: Login
-        description: Login button in the top-right corner
-        action: click
-
-  - prompt: Wait for the login form to appear
-    commands:
-      - command: wait-for-text
-        text: Email
-        timeout: 10000
-
-  - prompt: Enter credentials and submit
-    commands:
-      - command: hover-text
-        text: Email
-        description: Email input field
-        action: click
-      - command: type
-        text: user@example.com
-      - command: hover-text
-        text: Password
-        description: Password input field
-        action: click
-      - command: type
-        text: password123
-      - command: hover-text
-        text: Submit
-        description: Submit button
-        action: click
-
-  - prompt: Wait for the dashboard to load
-    commands:
-      - command: wait-for-text
-        text: Welcome, Test User!
-        timeout: 15000
-
-  - prompt: Verify the dashboard is displayed
-    commands:
-      - command: assert
-        expect: The dashboard is displayed
-```
+  - Continuously monitors the screen for changes and only moves forward when the screen is stable.
 
 ---
 
 ## Best Practices for Waiting
 
-1. **Use Explicit Waiting for Dynamic Elements**:
-   - Use `wait-for-text` or `wait-for-image` for elements that take time to load.
-
 2. **Leverage Automatic Waiting**:
-   - Rely on TestDriver.ai's `redraw` function to handle most waiting scenarios automatically.
+  - Rely on TestDriver.ai's `redraw` function to handle most waiting scenarios automatically.
+
+1. **Use Explicit Waiting for Dynamic Elements**:
+  - Use `wait-for-text` or `wait-for-image` for elements that take time to load.
 
 3. **Avoid Hardcoded Delays**:
-   - Replace hardcoded `sleep` or `wait` commands with dynamic waiting commands to improve test reliability.
+  - Replace hardcoded `sleep` or `wait` commands with dynamic waiting commands to improve test reliability.
 
 4. **Set Appropriate Timeouts**:
-   - Use reasonable timeouts for explicit waiting commands to balance reliability and test execution time.
+  - Use reasonable timeouts for explicit waiting commands to balance reliability and test execution time.
 
 5. **Test Incrementally**:
-   - Add waiting commands step-by-step to ensure each part of the workflow is stable.
+  - Add waiting commands step-by-step to ensure each part of the workflow is stable.

package/docs/interactive/assert.mdx

@@ -15,7 +15,7 @@ The `/assert` command ensures that a specific condition is true within your test
 ## Behavior
 - The `/assert` command generates an assertion based on the specified criteria.
 - TestDriver.ai takes a screenshot and uses it to verify that the condition described in the `expect` field is true.
-- If the condition is not met, the test will fail and exit immediately.
+- If the condition isn't met, the test will fail and exit immediately.
 
 ## Example Usage
 
@@ -43,7 +43,7 @@ To speed up tests, use `async: true` to allow the test to continue without waiti
 - Ensure the `expect` field clearly describes the condition to avoid ambiguity.
 
 ## Gotchas
-- If the condition in `expect` is not met, the test will fail and exit immediately.
+- If the condition in `expect` isn't met, the test will fail and exit immediately.
 - Overusing assertions can make tests slower and harder to maintain.
 
 ## Notes

package/docs/interactive/undo.mdx

@@ -40,7 +40,7 @@ The `/undo` command removes the last generated command or step from the current
 
 ## Gotchas
 - The `/undo` command only removes the last generated command or step. If you need to undo multiple actions, call `/undo` repeatedly.
-- Once a session is closed, you cannot undo previously saved commands.
+- Once a session is closed, you can't undo previously saved commands.
 
 ## Notes
 - The `/undo` command is ideal for interactive sessions where you are experimenting with test generation.

package/docs/overview/what-is-testdriver.mdx

@@ -5,7 +5,7 @@ icon: "circle-info"
 mode: "wide"
 ---
 
-TestDriver uses AI vision and keyboard and mouse control to automate end-to-end testing. TestDriver is `selectorless` meaning it is not aware of the underlying code structure.
+TestDriver uses AI vision and keyboard and mouse control to automate end-to-end testing. TestDriver is `selectorless` meaning it isn't aware of the underlying code structure.
 
 - **Easier set up:** No need to craft complex selectors
 - **Less Maintenance:** Tests don't break when code changes