testdriverai 5.3.12 → 5.3.14

package/docs/features/generation.mdx

@@ -4,177 +4,111 @@ sidebarTitle: "Test Generation"
 description: "Simplify test creation with AI-driven exploratory testing."
 ---
 
-TestDriver.ai provides a powerful and flexible framework for generating tests from various sources, including:
-- **TestRail test cases**
-- **CSV files**
-- **Jira tickets**
-- **Gherkin scenarios**
+Writing tests can be a tedious and time-consuming task. TestDriver.ai can generate tests just by exploring your app! This guide will show you how to generate tests using TestDriver.ai.
 
-Additionally, TestDriver.ai can autonomously generate its own tests by recursively mapping the UI and exploring workflows.
+# Generate Exploratory Tests
 
-## **Test Generation from CSV Files**
+Launch `testdriverai` in interactive mode with the following command:
 
-### Workflow Overview
-CSV files are a simple and structured way to define test cases. Each row in the CSV file can represent a test case, with columns specifying the test steps, expected outcomes, or other metadata.
+```bash
+testdriverai
+```
 
-### Example CSV Structure:
+TestDriver will boot up. 
 
-| Test Name         | Step 1                  | Step 2                  | Step 3                  | Expected Outcome         |
-|--------------------|-------------------------|-------------------------|-------------------------|--------------------------|
-| Login Test         | Open login page         | Enter valid credentials | Click "Log In" button   | Dashboard is displayed   |
-| Invalid Login Test | Open login page         | Enter invalid credentials | Click "Log In" button | Error message is shown   |
+```bash
+❯ testdriverai                                                        
+Howdy! I'm TestDriver v5.3.11
+```
 
-### Conversion to TestDriver.ai YAML
-A script can parse the CSV file and convert each row into a YAML test file.
+Ensure your website or app is visible on your test runner's display.
 
-#### Example YAML for "Login Test":```yaml
-version: 4.2.18
-steps:
-  - prompt: Open login page
-  - prompt: Enter valid credentials
-  - prompt: Click "Log In" button
-  - prompt: Verify that the dashboard is displayed
+Then, run the `generate` command.
 
+```bash
+/generate web 10
 ```
 
-#### Automation Script
-Use a Node.js or Python script to automate the conversion process. The script reads the CSV file, extracts the test cases, and generates YAML files for each test.
+This will generate 10 tests within the `testdriver/generate` directory. You can change the number of tests generated by changing the number after `10`. The generated tests will be in the form of a `.js` file, and you can run them using your test runner of choice.
 
----
+```bash
+> /generate web 5
 
-## 2. **Test Generation from Jira Tickets**
+thinking...
+```
 
-### Workflow Overview
-Jira tickets often contain **user stories** or **acceptance criteria** that can be converted into test cases. By using the Jira API, you can fetch tickets and extract relevant fields like the title, description, and acceptance criteria.
+Now your directory structure will look like this:
 
-### Example Jira Ticket:
-- **Title**: Login functionality
-- **Acceptance Criteria**:
-  - The user can log in with valid credentials.
-  - An error message is displayed for invalid credentials.
-  - The login page is responsive on mobile devices.
+```bash
+testdriver
+├── generate
+│   ├── test-error-user-login.yaml
+│   ├── test-locked-out-user-login.yaml
+│   ├── test-navigation-to-shop-page.yaml
+│   ├── test-problem-user-login.yaml
+│   └── test-standard-user-login.yaml
+```
 
-### Conversion to TestDriver.ai YAML
-Each acceptance criterion can be converted into a `prompt` in a YAML test file.
+Notice that each test file has no commands! It's only a list of prompts. This is similar to an exploratory test.
 
-#### Example YAML:```yaml
-version: 4.2.18
+```yaml
+version: 5.3.11
 steps:
-  - prompt: The user can log in with valid credentials.
-  - prompt: An error message is displayed for invalid credentials.
-  - prompt: The login page is responsive on mobile devices.
-
+  - prompt: Fill in the "Username" field with "error_user".
+  - prompt: Fill in the "Password" field with "secret_password".
+  - prompt: Click the "Sign in" button.
+  - prompt: Assert that there are issues during checkout and an error is displayed.
 ```
 
-#### Automation Script
-Use a script to fetch Jira tickets via the Jira API, extract the acceptance criteria, and generate YAML files.
-
----
-
-## 3. **Test Generation from Gherkin Scenarios**
+# Generate Regression Tests from Exploratory Tests
 
-### Workflow Overview
-Gherkin scenarios, commonly used with tools like Cucumber, describe test cases in a human-readable format using keywords like `Given`, `When`, and `Then`.
+Now it's time to generate the regression test.
 
-### Example Gherkin Scenario:```gherkin
-Scenario: Successful login
-  Given the user is on the login page
-  When the user enters valid credentials
-  And clicks the "Log In" button
-  Then the user should see the dashboard
-
-```### Conversion to TestDriver.ai YAML
-Each Gherkin step can be mapped to a `prompt` in a YAML test file.
-
-#### Example YAML:```yaml
-version: 4.2.18
-steps:
-  - prompt: The user is on the login page
-  - prompt: The user enters valid credentials
-  - prompt: Clicks the "Log In" button
-  - prompt: The user should see the dashboard
+Run the tests with the `run` command and use the `--save` parameter:
 
+```bash
+testdriverai run testdriver/generate/test-error-user-login.yaml --save
 ```
 
-#### Automation Script
-A script can parse Gherkin files, map the steps to prompts, and generate YAML files.
-
----
+When a test has no `commands`, TestDriver falls back to the `prompt`. 
 
-## 4. **Test Generation from TestRail**
-
-### Workflow Overview
-TestRail is a popular test management tool. You can use the TestRail API to fetch test cases and convert them into TestDriver.ai YAML files.
+```
+> Fill in the "Username" field with "error_user".
+No commands found, running exploratory
+```
 
-### Example TestRail Test Case:
-- **Title**: Login Test
-- **Steps**:
-  1. Open the login page.
-  2. Enter valid credentials.
-  3. Click the "Log In" button.
-- **Expected Result**: The dashboard is displayed.
+When using the `--save` command, TestDriver appends any generated commands to the test file.
 
-### Conversion to TestDriver.ai YAML
-Each step becomes a `prompt`, and the expected result can be added as an assertion.
+So as your test run, TestDriver will append successful commands to the yaml file and save it.
 
-#### Example YAML:```yaml
-version: 4.2.18
+```yaml
+version: 5.3.11
+session: 6801430b5f2c252a8f4bed60
 steps:
-  - prompt: Open the login page
-  - prompt: Enter valid credentials
-  - prompt: Click the "Log In" button
-  - prompt: Verify that the dashboard is displayed
-
+  - prompt: Focus the browser window.
+    commands:
+      - command: focus-application
+        name: Google Chrome
+      - command: hover-text
+        text: Username
+        description: username input label in the login form
+        action: click
+  - prompt: Fill in the "Password" field with "secret_password".
+    commands:
+      - command: hover-text
+        text: Password
+        description: password input label in the login form
+        action: click
+      - command: type
+        text: secret_password
+      - command: hover-text
+        text: Password
+        description: password input field in the login form
+        action: click
 ```
 
-#### Automation Script
-A script can fetch test cases from TestRail, extract the steps and expected results, and generate YAML files.
+Now the next time you run the test, TestDriver will use the commands instead of the prompt!
 
----
-
-## 5. **AI-Driven Exploratory Testing**
-
-### Workflow Overview
-TestDriver.ai can autonomously generate tests by exploring the UI and mapping workflows. This is particularly useful for:
-- Discovering untested paths.
-- Validating dynamic or complex UIs.
-- Generating tests for new features without requiring manual input.
-
-### How It Works:
-1. **Recursive UI Mapping**: The AI navigates through the application, identifying clickable elements, input fields, and other interactive components.
-2. **Test Generation**: The AI generates test cases based on the discovered workflows, including actions like clicking buttons, filling forms, and verifying outcomes.
-3. **Self-Healing**: If the UI changes, the AI adapts the tests to match the new structure.
-
-### Example AI-Generated Test:
-For a login page, the AI might generate the following test:
-
-
-### Benefits:
-- **No Manual Input Required**: The AI generates tests automatically.
-- **Comprehensive Coverage**: The AI explores all possible paths, ensuring thorough testing.
-- **Dynamic Adaptation**: Tests are updated automatically when the UI changes.
-
----
-
-## Summary of Test Generation Methods
-
-| **Source**         | **Description**                                                                 | **Automation**                                                                 |
-|---------------------|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------|
-| **CSV Files**       | Convert rows into YAML test files.                                              | Use a script to parse the CSV and generate YAML.                              |
-| **Jira Tickets**    | Extract user stories or acceptance criteria.                                    | Use the Jira API to fetch tickets and generate YAML.                          |
-| **Gherkin Scenarios** | Map `Given`, `When`, `Then` steps to YAML prompts.                            | Use a script to parse Gherkin files and generate YAML.                        |
-| **TestRail**        | Fetch test cases and steps from TestRail.                                       | Use the TestRail API to extract test cases and generate YAML.                 |
-| **AI-Driven Testing** | Automatically generate tests by exploring the UI and mapping workflows.       | Let TestDriver.ai autonomously generate and adapt tests.                      |
-
----
-
-## Best Practices
-
-1. **Automate Test Generation**: Use scripts to streamline the conversion of test cases from various sources into YAML files.
-2. **Leverage AI for Coverage**: Use TestDriver.ai's AI-driven exploratory testing to discover untested paths and validate dynamic UIs.
-3. **Organize Test Files**: Store generated YAML files in a structured directory (e.g., `testdriver/`) for easy management.
-4. **Review Generated Tests**: Manually review the generated tests to ensure they align with your testing requirements.
-
----
-
-By combining structured test case imports (from CSV, Jira, Gherkin, and TestRail) with AI-driven exploratory testing, TestDriver.ai provides a comprehensive solution for generating and maintaining high-quality test suites.
+```bash
+testdriverai run testdriver/generate/test-error-user-login.yaml
+```

package/docs/features/parallel-testing.mdx

@@ -120,7 +120,7 @@ When this workflow runs:
 
 1. **Organize Test Files**: Use a consistent naming convention for test files to simplify management.
 2. **Monitor Test Results**: Review the GitHub Actions dashboard to identify and debug failing tests.
-3. **Optimize Test Files**: Ensure each test file is self-contained and does not depend on the execution of other tests.
+3. **Optimize Test Files**: Ensure each test file is self-contained and doesn't depend on the execution of other tests.
 4. **Use Fail-Fast Judiciously**: Enable `fail-fast: true` only if you want to stop all tests when one fails.
 
 ---

package/docs/features/selectorless.mdx

@@ -32,7 +32,7 @@ steps:
 This allows TestDriver locates the target for `hover-text` based on its context and description. The agent will search for elements: in the following order.
 
 - `text` - exact element to match
-- `description` - a description of the element given the exact text is not found, or there are multiple matches
+- `description` - a description of the element given the exact text isn't found, or there are multiple matches
 - `prompt` - a high level prompt used to regenerate the test if no match is found
 
 ### What Happens When "Sign Up" Changes to "Register"?

package/docs/getting-started/generating.mdx

@@ -5,51 +5,111 @@ description: "Generate test plans and integrate them into your testing workflow.
 icon: "steering-wheel"
 ---
 
-This guide explains how to use **TestDriver.ai** to generate test plans locally and run them in parallel. You need
+Writing tests can be a tedious and time-consuming task. TestDriver.ai can generate tests just by exploring your app! This guide will show you how to generate tests using TestDriver.ai.
 
----
+# Launch TestDriver in Interactive Mode
 
-## Workflow Overview
+Launch `testdriverai` in interactive mode with the following command:
 
-1. **Generate Test Plans Locally**: Use the `/generate` command to create test plans.
-2. **Commit Test Plans to the Repository**: Add the generated files to your repository for version control.
-3. **Run Tests in Parallel**: Use GitHub Actions to execute the committed test plans in parallel.
-4. **Create PRs for Test Results**: Automatically generate PRs for test results, allowing teams to review and merge updates.
+```bash
+testdriverai
+```
 
----
+TestDriver will boot up. 
 
-## Step 1: Generate Test Plans Locally
+```bash
+❯ testdriverai                                                        
+Howdy! I'm TestDriver v5.3.11
+```
 
-### Run `/generate` Locally
+Ensure your website or app is visible on your test runner's display.
 
-1. Open your terminal and navigate to your project directory.
-2. Run the following command to start TestDriver.ai locally:
+Then, run the `generate` command.
 
+```bash
+/generate web 10
+```
+
+This will generate 10 tests within the `testdriver/generate` directory. You can change the number of tests generated by changing the number after `10`. The generated tests will be in the form of a `.js` file, and you can run them using your test runner of choice.
 
 ```bash
-testdriverai
-> /generate web 10
+> /generate web 5
+
+thinking...
+```
+
+Now your directory structure will look like this:
+
+```bash
+testdriver
+├── generate
+│   ├── test-error-user-login.yaml
+│   ├── test-locked-out-user-login.yaml
+│   ├── test-navigation-to-shop-page.yaml
+│   ├── test-problem-user-login.yaml
+│   └── test-standard-user-login.yaml
+```
+
+Notice that each test file has no commands! It's only a list of prompts. This is similar to an exploratory test.
+
+```yaml
+version: 5.3.11
+steps:
+  - prompt: Fill in the "Username" field with "error_user".
+  - prompt: Fill in the "Password" field with "secret_password".
+  - prompt: Click the "Sign in" button.
+  - prompt: Assert that there are issues during checkout and an error is displayed.
 ```
 
-This will generate 10 test plans for your website or application.
+# Generate Regression Tests from Exploratory Tests
 
-### Save the Generated Files
+Now it's time to generate the regression test.
+
+Run the tests with the `run` command and use the `--save` parameter:
+
+```bash
+testdriverai run testdriver/generate/test-error-user-login.yaml --save
+```
 
-TestDriver.ai will save the generated test plans as YAML files in the `testdriver/generate/` directory by default. For example:
+When a test has no `commands`, TestDriver falls back to the `prompt`. 
 
 ```
-testdriver/
-├── generate/
-│   ├── test_1.yml
-│   ├── test_2.yml
-│   ├── test_3.yml
+> Fill in the "Username" field with "error_user".
+No commands found, running exploratory
 ```
-### Run  the test files
 
-<Card
-  title="Running TestDriver Tests"
-  icon="link"
-  href="/getting-started/ci"
->
-  Run your generated tests.
-</Card>
+When using the `--save` command, TestDriver appends any generated commands to the test file.
+
+So as your test run, TestDriver will append successful commands to the yaml file and save it.
+
+```yaml
+version: 5.3.11
+session: 6801430b5f2c252a8f4bed60
+steps:
+  - prompt: Focus the browser window.
+    commands:
+      - command: focus-application
+        name: Google Chrome
+      - command: hover-text
+        text: Username
+        description: username input label in the login form
+        action: click
+  - prompt: Fill in the "Password" field with "secret_password".
+    commands:
+      - command: hover-text
+        text: Password
+        description: password input label in the login form
+        action: click
+      - command: type
+        text: secret_password
+      - command: hover-text
+        text: Password
+        description: password input field in the login form
+        action: click
+```
+
+Now the next time you run the test, TestDriver will use the commands instead of the prompt!
+
+```bash
+testdriverai run testdriver/generate/test-error-user-login.yaml
+```

package/docs/getting-started/vscode.mdx

@@ -28,7 +28,7 @@ The TestDriver.ai VSCode extension is available for download on our GitHub relea
 
 # Try the Walkthrough
 
-The extension should launch a walkthrough. If it does not, you can manually start it by running the command `TestDriver: Start Setup Walkthrough` from the Command Palette (`Ctrl+Shift+P`).
+The extension should launch a walkthrough. If it doesn't, you can manually start it by running the command `TestDriver: Start Setup Walkthrough` from the Command Palette (`Ctrl+Shift+P`).
 
 ![](/images/content/vscode-walkthrough.png)
 

package/docs/guide/assertions.mdx

@@ -13,7 +13,7 @@ Assertions in TestDriver.ai allow you to validate that your application behaves
 
 ## What Are Assertions?
 
-Assertions are checks that validate whether a specific condition is true. If the condition is not met, the test will fail, providing feedback on what went wrong.
+Assertions are checks that validate whether a specific condition is true. If the condition isn't met, the test will fail, providing feedback on what went wrong.
 
 ### Types of Assertions in TestDriver.ai:
 1. **Text Assertions**: Verify that specific text is visible on the screen.

package/docs/guide/authentication.mdx

@@ -12,13 +12,11 @@ This guide explains how to handle **authentication workflows** in **TestDriver.a
 ## How Authentication Works in TestDriver.ai
 
 1. **Store Credentials Securely**:
-   - Use GitHub Secrets to store sensitive information like usernames, passwords, or API keys.
+  - Use GitHub Secrets to store sensitive information like usernames, passwords, or API keys.
 2. **Pass Credentials to the Workflow**:
-   - Supply credentials as environment variables or directly in the workflow.
+  - Supply credentials as environment variables or directly in the workflow.
 3. **Use Credentials in Tests**:
-   - Dynamically reference credentials in the `prerun` script or test files to perform authentication steps.
-
----
+  - Dynamically reference credentials in the `prerun` script or test files to perform authentication steps.
 
 ## Step 1: Store Credentials in GitHub Secrets
 
@@ -29,8 +27,6 @@ This guide explains how to handle **authentication workflows** in **TestDriver.a
    - **`TESTDRIVER_API_KEY`**: Your TestDriver.ai API key.
    - **`WEBSITE_URL`**: The URL of the website to test.
 
----
-
 ## Step 2: Pass Credentials to the Workflow
 
 Secrets are passed to the GitHub Action using the `secrets` context. They can be supplied as:
@@ -78,13 +74,8 @@ jobs:
           TD_PASSWORD: ${{ secrets.TD_PASSWORD }}
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
           FORCE_COLOR: "3"
-
-
 ```
 
-
----
-
 ## Step 3: Use Credentials in Test Files
 
 Secrets can be referenced in the test file using placeholders (e.g., `${TD_USERNAME}` and `${TD_PASSWORD}`).
@@ -114,37 +105,30 @@ steps:
         action: click
       - command: assert
         expect: The dashboard is displayed
-
-
 ```
 
-
----
-
 ## How It Works Together
 
 1. **Secrets in the Workflow**:
-   - Secrets like `TD_USERNAME` and `TD_PASSWORD` are passed as environment variables to the TestDriver.ai action.
+  - Secrets like `TD_USERNAME` and `TD_PASSWORD` are passed as environment variables to the TestDriver.ai action.
 
 2. **Secrets in the `prerun` Script**:
-   - The `WEBSITE_URL` secret is used to launch the browser with the correct URL.
+  - The `WEBSITE_URL` secret is used to launch the browser with the correct URL.
 
 3. **Secrets in the Test File**:
-   - The test file dynamically references the secrets to fill in login credentials during the test.
-
----
+  - The test file dynamically references the secrets to fill in login credentials during the test.
 
 ## Benefits of Using Authentication in TestDriver.ai
 
 1. **Secure Handling of Credentials**:
-   - Secrets are encrypted and not exposed in logs.
-   - Even if printed, they appear as `***`.
+  - Secrets are encrypted and not exposed in logs.
+  - Even if printed, they appear as `***`.
 
 2. **Dynamic Testing**:
-   - Easily switch between different environments (e.g., staging, production) by updating the secrets.
+  - Easily switch between different environments (e.g., staging, production) by updating the secrets.
 
 3. **Reusability**:
-   - Use the same workflow and test files across multiple repositories or environments.
+  - Use the same workflow and test files across multiple repositories or environments.
 
 ---