@testdriverai/agent 7.8.0-test.38

package/docs/v7/_drafts/playwright.mdx

@@ -0,0 +1,342 @@
+---
+title: "Get Started with TestDriver & Playwright"
+sidebarTitle: "Playwright"
+tag: "NEW"
+description: "In this guide, you'll setup your TestDriver account, create a new Playwright project, and leverage TestDriver's AI to convert tests to natural language."
+icon: "masks-theater"
+---
+
+## Overview
+
+`@testdriver.ai/playwright` is a backwards-compatible wrapper around `@playwright/test` that uses TestDriver's Vision AI to:
+
+- Make [natural language assertions](#assertions-with-expect-tomatchprompt)
+- [Replace brittle selectors](#locating-elements-with-testdriver-locate) with natural language
+- [Perform actions](#performing-actions-with-testdriver-act) with a prompt
+- Test with an [automated agent](#agentic-tests-with-test-agent)
+
+We'll be incrementally converting Playwright's example test from the sample code:
+
+```typescript tests/example.spec.ts icon=square-js
+// Before
+test("get started link", async ({ page }) => {
+  await page.goto("https://playwright.dev/");
+  await page.getByRole("link", { name: "Get started" }).click();
+  await expect(
+    page.getByRole("heading", { name: "Installation" }),
+  ).toBeVisible();
+});
+```
+
+To using natural language with TestDriver:
+
+```typescript tests/example.spec.ts icon=square-js
+// After
+import { test } from "@testdriver.ai/playwright";
+
+test.describe("get started link", () => {
+  test.beforeEach(async ({ page }) => page.goto("https://playwright.dev/"));
+  test.agent(`
+    - Click the 'Get started' link
+    - Verify the 'Installation' heading is visible
+  `);
+});
+```
+
+## Prerequisites
+
+<Steps>
+  <Step title="Create a TestDriver Account">
+    You will need a [Free TestDriver Account](https://console.testdriver.ai/team) to get an API key.
+
+    <Card title="Sign Up for TestDriver" icon="user-plus" horizontal href="https://console.testdriver.ai/team">
+
+    </Card>
+
+  </Step>
+  <Step title="Set up your environment">
+    Copy your API key from [the TestDriver dashboard](https://console.testdriver.ai/team), and set it as an environment variable.
+
+    <Tabs>
+      <Tab title="macOS / Linux">
+        ```bash Export an environment variable on macOS or Linux systems
+        export TD_API_KEY="your_api_key_here"
+        ```
+      </Tab>
+      <Tab title="Windows">
+        ```powershell Export an environment variable in PowerShell
+        setx TD_API_KEY "your_api_key_here"
+        ```
+      </Tab>
+    </Tabs>
+
+  </Step>
+</Steps>
+
+## Setup Playwright
+
+<Steps>
+  <Step title="Initialize Playwright">
+    <Info>
+      This is a condensed version of [Playwright's Installation Instructions](https://playwright.dev/docs/intro).
+
+      **If you're new to Playwright, you should follow their guide first.**
+    </Info>
+    In a new folder or an existing, run:
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npm init playwright@latest
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn create playwright
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm create playwright
+        ```
+      </Tab>
+    </Tabs>
+    Select the following options when prompted:
+
+    ```console
+    ✔ Do you want to use TypeScript or JavaScript?
+    > TypeScript
+    ✔ Where to put your end-to-end tests?
+    > tests
+    ✔ Add a GitHub Actions workflow? (y/N)
+    > N
+    ✔ Install Playwright browsers (can be done manually via 'npx playwright install')? (Y/n)
+    > Y
+    ```
+
+    Then, confirm Playwright works by running:
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npx playwright test
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn playwright test
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm exec playwright test
+        ```
+      </Tab>
+    </Tabs>
+
+  </Step>
+</Steps>
+
+## Setup `@testdriver.ai/playwright`
+
+<Steps>
+  <Step title="Install TestDriver">
+    `@testdriver.ai/playwright` as a backwards-compatible wrapper around `@playwright/test`:
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npm install @testdriver.ai/playwright
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn add @testdriver.ai/playwright
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm add @testdriver.ai/playwright
+        ```
+      </Tab>
+    </Tabs>
+
+  </Step>
+  <Step title="Run Playwright">
+    Before we start using TestDriver in our tests, run Playwright in [UI Mode](https://playwright.dev/docs/test-ui-mode):
+
+    <Tabs>
+      <Tab title="npm">
+        ```bash
+        npx playwright test --ui
+        ```
+      </Tab>
+      <Tab title="yarn">
+        ```bash
+        yarn playwright test --ui
+        ```
+      </Tab>
+      <Tab title="pnpm">
+        ```bash
+        pnpm exec playwright test --ui
+        ```
+      </Tab>
+    </Tabs>
+    ![Playwright UI Mode](https://playwright.dev/assets/ideal-img/ui-mode.4e54d6b.3598.png)
+
+    Clicking the ▶️ button should successfully run the tests in the UI,
+    just as they did before with `playwright test` in the CLI.
+
+  </Step>
+  <Step title="Import TestDriver">
+    For the sake of simplicity, we'll be working with one test file for now.
+
+    Open `tests/example.spec.ts` in your editor & rename the `@playwright/test`
+    import to `@testdriver.ai/playwright`:
+
+    ```typescript tests/example.spec.ts icon=square-js
+    import { test, expect } from '@playwright/test'; // [!code --]
+    import { test, expect } from '@testdriver.ai/playwright'; // [!code ++]
+    ```
+
+    Click the <Icon icon="play" /> button to run the test and verify everything still works.
+
+    <Tip>
+      Click the <Icon icon="eye" /> button to automatically re-run tests on save.
+    </Tip>
+
+  </Step>
+</Steps>
+
+## Usage
+
+Because TestDriver uses AI vision instead of selectors, we can use natural language for
+[assertions](#assertions-with-expect-tomatchprompt),
+[locating](#locating-elements-with-testdriver-locate),
+performing [actions](#performing-actions-with-testdriver-act),
+or even having an [agent](#agentic-tests-with-test-agent) test for you!
+
+### Assertions with `expect.toMatchPrompt`
+
+Replace `toBeVisible` with `toMatchPrompt` to assert that the element is visible on the screen:
+
+```typescript tests/example.spec.ts icon=square-js
+// [!code --:2]
+// Expects page to have a heading with the name of Installation.
+await expect(page.getByRole("heading", { name: "Installation" })).toBeVisible();
+// [!code ++:2]
+await expect(page).toMatchPrompt("'Installation' heading is visible");
+```
+
+Before, the test needed code comments to describe what the assertion is _actually checking_.
+
+With `toMatchPrompt`, natural language acts as a description, selector, and assertion in one
+
+<Tip>
+  TestDriver can reduce the amount of complexity in your tests, but you can
+  still "opt-in" to Playwright assertions and selectors if you need to (e.g.
+  validating accessibility with `page.getByRole`).
+</Tip>
+
+### Locating elements with `testdriver.locate`
+
+TestDriver can replace `data-testid`s, `getByRole`, and CSS selectors with natural language.
+
+First, update your test to get access to the `testdriver` fixture:
+
+```typescript tests/example.spec.ts icon=square-js
+// [!code --]
+test('get started link', async ({ page, }) => {
+// [!code ++]
+test('get started link', async ({ page, testdriver }) => {
+```
+
+Then, replace `getByRole` with `testdriver.locate`:
+
+```typescript tests/example.spec.ts icon=square-js
+test('get started link', async ({ page, testdriver }) => {
+  await page.goto('https://playwright.dev/');
+
+  // [!code --:2]
+  // Click the get started link.
+  await page.getByRole("link", { name: "Get started" }).click();
+  // [!code ++:2]
+  const link = await testdriver(page).locate("Get started link");
+  await link.click();
+
+  await expect(page).toMatchPrompt("'Installation' heading is visible");
+```
+
+Now, our test uses natural language to both describe & locate the element.
+
+<Tip>
+  In the example above, you can still use Playwright to assert that the element is indeed a link for accessibility:
+
+```typescript tests/example.spec.ts icon=square-js
+const link = await testdriver(page).locate("Get started link");
+// [!code ++]
+expect(link).toHaveRole("link");
+await link.click();
+```
+
+This way you can write user-centric tests _and_ validate the implementation.
+
+</Tip>
+
+### Performing actions with `testdriver.act`
+
+We can combine `locate` and `click` from the previous example into one line with `testdriver.act`:
+
+```typescript tests/example.spec.ts icon=square-js
+test("get started link", async ({ page, testdriver }) => {
+  await page.goto("https://playwright.dev/");
+
+  // [!code --:2]
+  const link = await testdriver(page).locate("Get started link");
+  await link.click();
+  // [!code ++]
+  await testdriver(page).act("Click the 'Get started' link");
+
+  await expect(page).toMatchPrompt("'Installation' heading is visible");
+});
+```
+
+Now the test uses the page the way a user would!
+
+### Agentic tests with `test.agent`
+
+TestDriver can automatically perform the entire test for you with an AI agent:
+
+```typescript tests/example.spec.ts icon=square-js
+// [!code --:6]
+test("get started link", async ({ page, testdriver }) => {
+  await page.goto("https://playwright.dev/");
+  await testdriver(page).act("Click the 'Get started' link");
+  await expect(page).toMatchPrompt("'Installation' heading is visible");
+});
+
+// [!code ++:7]
+test.describe("get started link", () => {
+  test.beforeEach(async ({ page }) => page.goto("https://playwright.dev/"));
+  test.agent(`
+    - Click the 'Get started' link
+    - Verify the 'Installation' heading is visible
+  `);
+});
+```
+
+Instead of writing the test implementation, we've used [`test.describe`](https://playwright.dev/docs/api/class-test#test-describe) to describe the test still,
+but replaced the `test` itself with `test.agent`.
+
+<Tip>
+  Use `test.beforeEach` to prepare the page for the agent (e.g.
+  [`page.goto`](https://playwright.dev/docs/api/class-page#page-goto), calling
+  an API to create a user). Use
+  [`test.afterEach`](https://playwright.dev/docs/api/class-test#test-after-each)
+  to clean up after the agent (e.g. `page.close`) or perform additional logic
+  (e.g. clearing the session).
+</Tip>
+
+## Conclusion
+
+With `@testdriver.ai/playwright`, you can use as much or as little of Playwright's _or_ TestDriver's API as you need to validate correctness. It's up to you!

package/docs/v7/_drafts/plugin-migration.mdx

@@ -0,0 +1,220 @@
+# Vitest Reporter → Plugin Migration
+
+## Summary
+
+Converted the Vitest reporter to a plugin architecture with much simpler dashcam URL tracking.
+
+## Key Changes
+
+### 1. Plugin Architecture ✅
+
+**Before (Reporter):**
+
+```javascript
+class TestDriverReporter {
+  constructor(options = {}) {
+    this.options = options;
+    this.testRun = null;
+    // ... many instance properties
+  }
+}
+```
+
+**After (Plugin):**
+
+```javascript
+const pluginState = {
+  testRun: null,
+  testRunId: null,
+  dashcamUrls: new Map(), // In-memory tracking!
+  // ... all state in one place
+};
+
+export default function testDriverPlugin(options = {}) {
+  return {
+    name: "testdriver-plugin",
+    // ... hooks
+  };
+}
+```
+
+### 2. Dashcam URL Tracking (MUCH SIMPLER) ✅
+
+**Before:**
+
+- ❌ Write JSON to temp files in `/tmp/`
+- ❌ Read temp files on test completion
+- ❌ Complex matching logic (sessionId, taskId, filename patterns)
+- ❌ Manual cleanup of temp files
+- ❌ Race conditions with parallel tests
+- ❌ Multiple fallback mechanisms (globalThis, registry, meta)
+
+**After:**
+
+- ✅ Simple in-memory Map (testId → dashcamUrl)
+- ✅ Direct API: `registerDashcamUrl(testId, url, platform)`
+- ✅ No file system operations
+- ✅ No complex matching
+- ✅ No cleanup needed
+- ✅ Thread-safe for parallel tests
+
+### 3. Usage in Tests
+
+**Before:**
+
+```javascript
+// Tests had to write to temp files
+const tempFile = path.join(
+  os.tmpdir(),
+  `testdriver-dashcam-${sessionId}-${taskId}-${Date.now()}.json`,
+);
+fs.writeFileSync(
+  tempFile,
+  JSON.stringify({
+    dashcamUrl,
+    replayObjectId,
+    platform: client.os,
+    timestamp: Date.now(),
+    pid: process.pid,
+    sessionId,
+  }),
+);
+
+// AND set global variables
+globalThis.__testdriverMeta.__lastDashcamUrl__ = dashcamUrl;
+
+// AND register in registry
+globalThis.__testdriverRegistry.setClient(taskId, client);
+
+// AND store in task.meta
+task.meta.testdriverDashcamUrl = dashcamUrl;
+```
+
+**After:**
+
+```javascript
+// Just one simple call
+globalThis.__testdriverPlugin.registerDashcamUrl(taskId, dashcamUrl, client.os);
+```
+
+### 4. Configuration
+
+**Before:**
+
+```javascript
+export default defineConfig({
+  test: {
+    reporters: [
+      "default",
+      [
+        "./interfaces/vitest-reporter.js",
+        {
+          apiKey: process.env.TD_API_KEY,
+          apiRoot: process.env.TD_API_ROOT,
+        },
+      ],
+    ],
+  },
+});
+```
+
+**After:**
+
+```javascript
+export default defineConfig({
+  plugins: [
+    testDriverPlugin({
+      apiKey: process.env.TD_API_KEY,
+      apiRoot: process.env.TD_API_ROOT,
+    }),
+  ],
+});
+```
+
+## Files Modified
+
+### Plugin
+
+- ✅ **Created**: `interfaces/vitest-plugin.mjs` (new plugin)
+- ✅ **Deleted**: `interfaces/vitest-reporter.js` (old reporter)
+- ✅ **Updated**: `vitest.config.mjs` (uses plugin)
+- ✅ **Updated**: `vitest.config.example.js` (uses plugin)
+
+### Test Helpers
+
+- ✅ **Removed**: `examples/setup/` folder (legacy helpers)
+  - Code moved to `lib/vitest/hooks.mjs` framework
+  - Tests now use `TestDriver(context, options)` pattern directly
+
+### Documentation
+
+- ✅ **Updated**: `docs/QUICK_START_TEST_RECORDING.md`
+- ✅ **Updated**: `docs/TEST_RECORDING.md`
+- ✅ **Updated**: `docs/ARCHITECTURE.md`
+
+## Code Removed
+
+### From Plugin (vs old reporter)
+
+- ❌ ~200 lines of file system operations
+- ❌ `indexDashcamFiles()` - complex temp file reading
+- ❌ `findReplayForTest()` - complex sessionId matching
+- ❌ `findReplayForTestInline()` - complex fallback logic
+- ❌ Imports: `fs`, `os` (no longer needed!)
+
+### From Test Helpers
+
+- ❌ ~60 lines of temp file writing
+- ❌ Global registry setup
+- ❌ Global meta initialization
+- ❌ Multiple storage mechanisms
+
+## Benefits
+
+1. **Simpler**: ~300 fewer lines of complex code
+2. **Faster**: No file I/O overhead
+3. **More Reliable**: No file system race conditions
+4. **Better DX**: Single API call to register dashcam URLs
+5. **Easier to Debug**: All state in one place
+6. **Thread-Safe**: Map-based storage handles parallel tests
+7. **Cleaner**: No temp file cleanup needed
+
+## Testing
+
+```bash
+# Verify plugin loads
+node -e "import('./interfaces/vitest-plugin.mjs').then(m => console.log('Exports:', Object.keys(m)))"
+
+# Run tests
+TD_API_KEY=xxx vitest run
+```
+
+## Migration for Users
+
+If you're using the TestDriver SDK in your tests:
+
+**Old way (still works but deprecated):**
+
+```javascript
+// Multiple places to set dashcam URL
+globalThis.__testdriverMeta.__lastDashcamUrl__ = dashcamUrl;
+```
+
+**New way:**
+
+```javascript
+// Single plugin API
+if (globalThis.__testdriverPlugin) {
+  globalThis.__testdriverPlugin.registerDashcamUrl(
+    testId,
+    dashcamUrl,
+    platform,
+  );
+}
+```
+
+## Notes
+
+- The plugin automatically makes itself available globally as `globalThis.__testdriverPlugin`
+- Test helpers now use this API automatically
+- No breaking changes for end users - just better internals!