testdriverai 6.2.0 → 6.2.1

package/docs/docs.json

@@ -38,7 +38,6 @@
             "group": "CLI",
             "icon": "terminal",
             "pages": [
-              "/getting-started/cli",
               "/cli/overview",
               {
                 "group": "Commands",
@@ -50,7 +49,10 @@
                 ]
               }
             ]
-          }
+          },
+          "/getting-started/self-hosting",
+          "/getting-started/playwright",
+          "/getting-started/vscode"
         ]
       },
       {
@@ -63,7 +65,8 @@
               "apps/static-websites",
               "apps/desktop-apps",
               "apps/chrome-extensions",
-              "apps/mobile-apps"
+              "apps/mobile-apps",
+              "apps/tauri-apps"
             ]
           },
           {
@@ -100,6 +103,7 @@
               "/guide/authentication",
               "/guide/variables",
               "/guide/lifecycle",
+              "/guide/dashcam",
               "/guide/environment-variables",
               "/action/ami"
             ]
@@ -153,39 +157,32 @@
       {
         "group": "Reference",
         "pages": [
-          "/commands/assert",
-          "/commands/exec",
-          "/commands/focus-application",
-          "/commands/hover-image",
-          "/commands/hover-text",
-          "/commands/if",
-          "/commands/match-image",
-          "/commands/press-keys",
-          "/commands/remember",
-          "/commands/run",
-          "/commands/scroll",
-          "/commands/scroll-until-image",
-          "/commands/scroll-until-text",
-          "/commands/type",
-          "/commands/wait",
-          "/commands/wait-for-image",
-          "/commands/wait-for-text"
+          {
+            "group": "YAML",
+            "icon": "file-code",
+            "pages": [
+              "/commands/assert",
+              "/commands/exec",
+              "/commands/focus-application",
+              "/commands/hover-image",
+              "/commands/hover-text",
+              "/commands/if",
+              "/commands/match-image",
+              "/commands/press-keys",
+              "/commands/remember",
+              "/commands/run",
+              "/commands/scroll",
+              "/commands/scroll-until-image",
+              "/commands/scroll-until-text",
+              "/commands/type",
+              "/commands/wait",
+              "/commands/wait-for-image",
+              "/commands/wait-for-text"
+            ]
+          }
         ]
       }
-    ],
-    "global": {
-      "anchors": [
-        {
-          "anchor": "Book a Demo",
-          "href": "https://testdriver.ai/demo",
-          "icon": "calendar"
-        }
-      ]
-    }
-  },
-  "modeToggle": {
-    "enabled": false,
-    "default": "dark"
+    ]
   },
   "logo": {
     "light": "/images/template/light.png",
@@ -198,13 +195,18 @@
         "label": "Discord",
         "icon": "discord",
         "href": "https://discord.com/invite/cWDFW8DzPm"
+      },
+      {
+        "label": "Book a Demo",
+        "href": "https://form.typeform.com/to/UECf9rDx?typeform-source=docs.testdriver.ai",
+        "icon": "calendar"
+      },
+      {
+        "icon": "gauge-high",
+        "label": "Dashboard",
+        "href": "https://app.testdriver.ai"
       }
-    ],
-    "primary": {
-      "type": "button",
-      "label": "Dashboard",
-      "href": "https://app.testdriver.ai"
-    }
+    ]
   },
   "footer": {
     "socials": {

package/docs/getting-started/playwright.mdx

@@ -0,0 +1,342 @@
+---
+title: "Get Started with TestDriver & Playwright"
+sidebarTitle: "Playwright"
+tag: "NEW"
+description: "In this guide, you'll setup your TestDriver account, create a new Playwright project, and leverage TestDriver's AI to convert tests to natural language."
+icon: "masks-theater"
+---
+
+## Overview
+
+`@testdriver.ai/playwright` is a backwards-compatible wrapper around `@playwright/test` that uses TestDriver's Vision AI to:
+
+- Make [natural language assertions](#assertions-with-expect-tomatchprompt)
+- [Replace brittle selectors](#locating-elements-with-testdriver-locate) with natural language
+- [Perform actions](#performing-actions-with-testdriver-act) with a prompt
+- Test with an [automated agent](#agentic-tests-with-test-agent)
+
+We'll be incrementally converting Playwright's example test from the sample code:
+
+```typescript tests/example.spec.ts icon=square-js
+// Before
+test("get started link", async ({ page }) => {
+  await page.goto("https://playwright.dev/");
+  await page.getByRole("link", { name: "Get started" }).click();
+  await expect(
+    page.getByRole("heading", { name: "Installation" }),
+  ).toBeVisible();
+});
+```
+
+To using natural language with TestDriver:
+
+```typescript tests/example.spec.ts icon=square-js
+// After
+import { test } from "@testdriver.ai/playwright";
+
+test.describe("get started link", () => {
+  test.beforeEach(async ({ page }) => page.goto("https://playwright.dev/"));
+  test.agent(`
+    - Click the 'Get started' link
+    - Verify the 'Installation' heading is visible
+  `);
+});
+```
+
+## Prerequisites
+
+<Steps>
+  <Step title="Create a TestDriver Account">
+    You will need a [Free TestDriver Account](https://app.testdriver.ai/team) to get an API key.
+
+    <Card title="Sign Up for TestDriver" icon="user-plus" horizontal href="https://app.testdriver.ai/team">
+
+    </Card>
+
+  </Step>
+  <Step title="Set up your environment">
+    Copy your API key from [the TestDriver dashboard](https://app.testdriver.ai/team), and set it as an environment variable.
+
+    <Tabs>
+      <Tab title="macOS / Linux">
+        ```bash Export an environment variable on macOS or Linux systems
+        export TD_API_KEY="your_api_key_here"
+        ```
+      </Tab>
+      <Tab title="Windows">
+        ```powershell Export an environment variable in PowerShell
+        setx TD_API_KEY "your_api_key_here"
+        ```
+      </Tab>
+    </Tabs>
+
+  </Step>
+</Steps>
+
+## Setup Playwright
+
+<Steps>
+  <Step title="Initialize Playwright">
+    <Info>
+      This is a condensed version of [Playwright's Installation Instructions](https://playwright.dev/docs/intro).
+
+      **If you're new to Playwright, you should follow their guide first.**
+    </Info>
+    In a new folder or an existing, run:
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npm init playwright@latest
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn create playwright
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm create playwright
+        ```
+      </Tab>
+    </Tabs>
+    Select the following options when prompted:
+
+    ```console
+    ✔ Do you want to use TypeScript or JavaScript?
+    > TypeScript
+    ✔ Where to put your end-to-end tests?
+    > tests
+    ✔ Add a GitHub Actions workflow? (y/N)
+    > N
+    ✔ Install Playwright browsers (can be done manually via 'npx playwright install')? (Y/n)
+    > Y
+    ```
+
+    Then, confirm Playwright works by running:
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npx playwright test
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn playwright test
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm exec playwright test
+        ```
+      </Tab>
+    </Tabs>
+
+  </Step>
+</Steps>
+
+## Setup `@testdriver.ai/playwright`
+
+<Steps>
+  <Step title="Install TestDriver">
+    `@testdriver.ai/playwright` as a backwards-compatible wrapper around `@playwright/test`:
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npm install @testdriver.ai/playwright
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn add @testdriver.ai/playwright
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm add @testdriver.ai/playwright
+        ```
+      </Tab>
+    </Tabs>
+
+  </Step>
+  <Step title="Run Playwright">
+    Before we start using TestDriver in our tests, run Playwright in [UI Mode](https://playwright.dev/docs/test-ui-mode):
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npx playwright test --ui
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn playwright test --ui
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm exec playwright test --ui
+        ```
+      </Tab>
+    </Tabs>
+    ![Playwright UI Mode](https://playwright.dev/assets/ideal-img/ui-mode.4e54d6b.3598.png)
+
+    Clicking the ▶️ button should successfully run the tests in the UI,
+    just as they did before with `playwright test` in the CLI.
+
+  </Step>
+  <Step title="Import TestDriver">
+    For the sake of simplicity, we'll be working with one test file for now.
+
+    Open `tests/example.spec.ts` in your editor & rename the `@playwright/test`
+    import to `@testdriver.ai/playwright`:
+
+    ```typescript tests/example.spec.ts icon=square-js
+    import { test, expect } from '@playwright/test'; // [!code --]
+    import { test, expect } from '@testdriver.ai/playwright'; // [!code ++]
+    ```
+
+    Click the <Icon icon="play" /> button to run the test and verify everything still works.
+
+    <Tip>
+      Click the <Icon icon="eye" /> button to automatically re-run tests on save.
+    </Tip>
+
+  </Step>
+</Steps>
+
+## Usage
+
+Because TestDriver uses AI vision instead of selectors, we can use natural language for
+[assertions](#assertions-with-expect-tomatchprompt),
+[locating](#locating-elements-with-testdriver-locate),
+performing [actions](#performing-actions-with-testdriver-act),
+or even having an [agent](#agentic-tests-with-test-agent) test for you!
+
+### Assertions with `expect.toMatchPrompt`
+
+Replace `toBeVisible` with `toMatchPrompt` to assert that the element is visible on the screen:
+
+```typescript tests/example.spec.ts icon=square-js
+// [!code --:2]
+// Expects page to have a heading with the name of Installation.
+await expect(page.getByRole("heading", { name: "Installation" })).toBeVisible();
+// [!code ++:2]
+await expect(page).toMatchPrompt("'Installation' heading is visible");
+```
+
+Before, the test needed code comments to describe what the assertion is _actually checking_.
+
+With `toMatchPrompt`, natural language acts as a description, selector, and assertion in one
+
+<Tip>
+  TestDriver can reduce the amount of complexity in your tests, but you can
+  still "opt-in" to Playwright assertions and selectors if you need to (e.g.
+  validating accessibility with `page.getByRole`).
+</Tip>
+
+### Locating elements with `testdriver.locate`
+
+TestDriver can replace `data-testid`s, `getByRole`, and CSS selectors with natural language.
+
+First, update your test to get access to the `testdriver` fixture:
+
+```typescript tests/example.spec.ts icon=square-js
+// [!code --]
+test('get started link', async ({ page, }) => {
+// [!code ++]
+test('get started link', async ({ page, testdriver }) => {
+```
+
+Then, replace `getByRole` with `testdriver.locate`:
+
+```typescript tests/example.spec.ts icon=square-js
+test('get started link', async ({ page, testdriver }) => {
+  await page.goto('https://playwright.dev/');
+
+  // [!code --:2]
+  // Click the get started link.
+  await page.getByRole("link", { name: "Get started" }).click();
+  // [!code ++:2]
+  const link = await testdriver(page).locate("Get started link");
+  await link.click();
+
+  await expect(page).toMatchPrompt("'Installation' heading is visible");
+```
+
+Now, our test uses natural language to both describe & locate the element.
+
+<Tip>
+  In the example above, you can still use Playwright to assert that the element is indeed a link for accessibility:
+
+```typescript tests/example.spec.ts icon=square-js
+const link = await testdriver(page).locate("Get started link");
+// [!code ++]
+expect(link).toHaveRole("link");
+await link.click();
+```
+
+This way you can write user-centric tests _and_ validate the implementation.
+
+</Tip>
+
+### Performing actions with `testdriver.act`
+
+We can combine `locate` and `click` from the previous example into one line with `testdriver.act`:
+
+```typescript tests/example.spec.ts icon=square-js
+test("get started link", async ({ page, testdriver }) => {
+  await page.goto("https://playwright.dev/");
+
+  // [!code --:2]
+  const link = await testdriver(page).locate("Get started link");
+  await link.click();
+  // [!code ++]
+  await testdriver(page).act("Click the 'Get started' link");
+
+  await expect(page).toMatchPrompt("'Installation' heading is visible");
+});
+```
+
+Now the test uses the page the way a user would!
+
+### Agentic tests with `test.agent`
+
+TestDriver can automatically perform the entire test for you with an AI agent:
+
+```typescript tests/example.spec.ts icon=square-js
+// [!code --:6]
+test("get started link", async ({ page, testdriver }) => {
+  await page.goto("https://playwright.dev/");
+  await testdriver(page).act("Click the 'Get started' link");
+  await expect(page).toMatchPrompt("'Installation' heading is visible");
+});
+
+// [!code ++:7]
+test.describe("get started link", () => {
+  test.beforeEach(async ({ page }) => page.goto("https://playwright.dev/"));
+  test.agent(`
+    - Click the 'Get started' link
+    - Verify the 'Installation' heading is visible
+  `);
+});
+```
+
+Instead of writing the test implementation, we've used [`test.describe`](https://playwright.dev/docs/api/class-test#test-describe) to describe the test still,
+but replaced the `test` itself with `test.agent`.
+
+<Tip>
+  Use `test.beforeEach` to prepare the page for the agent (e.g.
+  [`page.goto`](https://playwright.dev/docs/api/class-page#page-goto), calling
+  an API to create a user). Use
+  [`test.afterEach`](https://playwright.dev/docs/api/class-test#test-after-each)
+  to clean up after the agent (e.g. `page.close`) or perform additional logic
+  (e.g. clearing the session).
+</Tip>
+
+## Conclusion
+
+With `@testdriver.ai/playwright`, you can use as much or as little of Playwright's _or_ TestDriver's API as you need to validate correctness. It's up to you!