testdriverai 7.11.264-test → 7.11.265-test

package/ai/skills/testdriver-claude-mcp-plugin/SKILL.md

@@ -1,245 +0,0 @@
----
-name: testdriver:claude-mcp-plugin
-description: claude-mcp-plugin
----
-<!-- Generated from claude-mcp-plugin.mdx. DO NOT EDIT. -->
-
-TestDriver ships as a native [Claude Code plugin](https://docs.claude.com/en/docs/claude-code/plugins) that bundles the **TestDriver MCP server**, the **`testdriver` expert sub-agent**, and all **TestDriver skills**. You get everything you need to drive TestDriver from Claude Code with a single install.
-
-> The plugin lives inside the `testdriverai` npm package at `ai/.claude-plugin/plugin.json`, and the marketplace entry lives at `.claude-plugin/marketplace.json` in this repo.
-
-## 1. Get a TestDriver API key
-
-- Visit your team page (e.g. `https://console.testdriver.ai/settings`)
-- Create or copy a **Team API Key** (or User API Key)
-- Export it in your shell so Claude Code can pass it to the MCP server:
-
-  ```bash
-  export TD_API_KEY="your_api_key_here"
-  ```
-
-## 2. Install the plugin in Claude Code
-
-From inside Claude Code, add this repo as a plugin marketplace and install the `testdriver` plugin:
-
-```text
-/plugin marketplace add testdriverai/testdriverai
-/plugin install testdriver@testdriver
-```
-
-That registers three things:
-
-- **`testdriver` MCP server** — spawned via `npx -p testdriverai testdriverai-mcp`, with `TD_API_KEY` forwarded from your environment.
-- **`testdriver` sub-agent** — the TestDriver expert agent from `ai/agents/testdriver.md`. Invoke it with `@testdriver ...`.
-- **TestDriver skills** — every `ai/skills/testdriver-*` skill, auto-loaded by Claude Code.
-
-## 3. Write tests with the agent
-
-In a Claude Code session, delegate to the agent:
-
-```text
-@testdriver Write a test that signs into https://example.com and adds an item to the cart.
-```
-
-The agent will use the TestDriver MCP tools (`session_start`, `find`, `click`, `type`, `assert`, …) to interactively build a Vitest test, append generated code to your test file after every action, and run it with `vitest run` until it passes.
-
-For the full agent guide, see the [`testdriver` agent definition](https://github.com/testdriverai/testdriverai/blob/main/ai/agents/testdriver.md) and the [MCP workflow skill](https://github.com/testdriverai/testdriverai/blob/main/ai/skills/testdriver-mcp-workflow/SKILL.md).
-
-## Manual MCP configuration (no plugin)
-
-If you prefer not to use the plugin, you can register the MCP server manually in any MCP-compatible client (Claude Desktop, Cursor, VS Code, …):
-
-```json
-{
-  "mcpServers": {
-    "testdriver": {
-      "command": "npx",
-      "args": ["-p", "testdriverai", "testdriverai-mcp"],
-      "env": {
-        "TD_API_KEY": "${TD_API_KEY}"
-      }
-    }
-  }
-}
-```
-
-This is the same config the plugin wires up for you — the plugin just bundles it alongside the agent and skills.
-
----
-
-## Remote (hosted URL) install — no CLI, browser login
-
-TestDriver also runs a **hosted MCP server** you can connect to with just a URL. There is no `npx` command and no API key to paste: the server speaks OAuth 2.1, so your client opens a browser, you sign in with TestDriver, and the tools appear. It exposes the **full live tool set** (`session_start`, `find`, `click`, `type`, `assert`, …) plus the read-only data tools, all scoped to your team.
-
-Hosted endpoint:
-
-```text
-https://mcp.testdriver.ai/mcp
-```
-
-The server advertises its authorization server (Auth0) via [RFC 9728](https://www.rfc-editor.org/rfc/rfc9728) protected-resource metadata at:
-
-```text
-https://mcp.testdriver.ai/.well-known/oauth-protected-resource
-```
-
-Spec-compliant clients discover and complete the OAuth flow automatically:
-
-- **Claude** — add a custom connector pointing at the URL above; Claude runs the browser login for you.
-- **VS Code / Copilot** — add a remote MCP server entry with the URL; VS Code handles the OAuth handshake.
-- **ChatGPT** — add it as a connector by URL.
-- **Any spec-compliant client** — point it at the URL; it will read the metadata, sign you in, and connect.
-
-```json
-{
-  "mcpServers": {
-    "testdriver": {
-      "url": "https://mcp.testdriver.ai/mcp"
-    }
-  }
-}
-```
-
-Each connection gets its own isolated sandbox, so multiple people (or multiple chats) can run tests at the same time without interfering. The API-key paths above still work for automation and CI.
-
----
-
-## Observing test runs via HTTP MCP
-
-TestDriver also exposes **test results and analytics** over an HTTP MCP endpoint, so Claude Code (or any MCP-compatible client) can inspect your test runs, failures, and filters without provisioning a sandbox.
-
-### HTTP MCP endpoint contract
-
-The HTTP endpoint lives at:
-
-```text
-POST /api/v1/mcp
-```
-
-It expects the TestDriver API key in the `X-Api-Key` header (or `Authorization: Bearer <key>`).
-
-Common request shapes:
-
-- **List tools**
-
-```json
-{
-  "kind": "list_tools"
-}
-```
-
-- **Call a tool**
-
-```json
-{
-  "kind": "call_tool",
-  "tool": "list_test_runs",
-  "arguments": {
-    "status": "failed",
-    "page": 1,
-    "limit": 20
-  }
-}
-```
-
-Responses from tool calls follow the MCP content convention:
-
-```json
-{
-  "content": [
-    {
-      "type": "json",
-      "json": {
-        "testRuns": [],
-        "totalCount": 0,
-        "hasMore": false
-      }
-    }
-  ]
-}
-```
-
-### Available tools
-
-The MCP server advertises at least these tools in `list_tools`:
-
-- `list_test_runs`  
-  List recent TestDriver test runs for the current team, with filters and pagination.
-
-- `get_test_run_detail`  
-  Get a single test run and its test cases (including replay IDs / share keys when available).
-
-- `list_test_cases`  
-  List individual test cases for the team with status, duration, error messages, and replay info.
-
-- `get_filter_options`  
-  Get branch, suite, repo, filename, commit, status, platform, and test name options for building queries.
-
-### Pointing Claude Code at the HTTP MCP endpoint
-
-You can point Claude Code at the HTTP MCP endpoint using a JSON configuration similar to:
-
-```json
-{
-  "$schema": "https://schema.anthropic.com/mcp/servers.json",
-  "mcpServers": {
-    "testdriver-cloud": {
-      "type": "sse",
-      "url": "https://your-api-host.example.com/api/v1/mcp",
-      "requestHeaders": {
-        "X-Api-Key": "${TD_API_KEY}"
-      },
-      "description": "Query TestDriver test runs, test cases, and filters for your team using an API key."
-    }
-  }
-}
-```
-
-You can find this exact snippet in the repo at:
-
-- `claude-mcp-config.example.json`
-
-Replace `https://your-api-host.example.com` with your actual API origin (e.g. `https://api.testdriver.ai` or `http://localhost:1337` in development).
-
-### Local development
-
-For local development:
-
-- Run your API server on `http://localhost:1337`
-- Point `baseUrl` at `http://localhost:1337/api/v1/mcp`
-- Use a local team or user API key in `TD_API_KEY`
-
-```json
-{
-  "mcpServers": {
-    "testdriver-cloud-local": {
-      "type": "sse",
-      "url": "http://localhost:1337/api/v1/mcp",
-      "requestHeaders": {
-        "X-Api-Key": "${TD_API_KEY}"
-      }
-    }
-  }
-}
-```
-
-### Skills documentation for Claude
-
-Claude Code loads the agent and skills automatically when you install the plugin (see step 2). The underlying sources are:
-
-- `ai/agents/testdriver.md` contains the **full TestDriver Agent Guide**
-- `ai/skills/testdriver-*/SKILL.md` provide task-specific skills (MCP workflow, assertions, provisioning, etc.)
-
-Use these as the primary reference for:
-
-- How to initialize the `TestDriver` SDK in Vitest
-- The MCP workflow for building tests interactively with visual feedback
-- How to find elements, click, type, scroll, assert, and capture screenshots
-
-The MCP tools described above are **read-only** helpers for:
-
-- Inspecting recent test runs and failures
-- Discovering branches, files, and suites to focus on
-- Pulling detailed test case and replay information into Claude for analysis
-
-Use the SDK (`testdriverai`) for **driving tests**, and the HTTP MCP server (`/api/v1/mcp`) for **observing and debugging** them from Claude Code.

package/ai/skills/testdriver-cloud/SKILL.md

@@ -1,119 +0,0 @@
----
-name: testdriver:cloud
-description: The fastest way to get started with TestDriver. Just set your API key and start testing.
----
-<!-- Generated from cloud.mdx. DO NOT EDIT. -->
-
-Cloud pricing is based on **device-seconds**: the amount of time your tests run on **our infrastructure**.
-
-- **Zero Setup** — Start testing immediately. No DevOps required.
-- **Free Tier** — Get started with a limited preview at no cost.
-- **Pay As You Go** — Only pay for the device-seconds you use.
-
-## Get Started
-Cloud is the default when you follow the Quickstart guide. 
-<Card
-  title="Try the Quickstart"  
-  icon="play"
-  href="/v7/quickstart"
->
-  Set your API key and start testing in minutes.
-</Card>
-
-## Parallel Testing Limits
-
-Your account has a set number of **license slots** that determine how many devices can run simultaneously. You can view your available slots in the [TestDriver Dashboard](https://console.testdriver.ai).
-
-<Info>
-  **When is a slot in use?** A license slot is occupied when a test client is connected. As soon as your device is destroyed the slot becomes available immediately.
-</Info>
-
-## Avoiding Slot Conflicts
-
-To prevent tests from failing due to exceeding your license slot limit, we recommend two key configurations:
-
-<AccordionGroup>
-  <Accordion title="Set Maximum Concurrency in Vitest">
-    Limit concurrent tests to match your available license slots:
-
-    ```javascript vitest.config.mjs
-    import { defineConfig } from 'vitest/config';
-    import { TestDriver } from 'testdriverai/vitest';
-
-    export default defineConfig({
-      test: {
-        testTimeout: 900000,
-        hookTimeout: 900000,
-        maxConcurrency: 5, // Set to your license slot limit
-        reporters: ['default', TestDriver()],
-        setupFiles: ['testdriverai/vitest/setup'],
-      },
-    });
-    ```
-
-    <Tip>
-      Check your slot count at [console.testdriver.ai](https://console.testdriver.ai) and set `maxConcurrency` to that number or lower.
-    </Tip>
-  </Accordion>
-
-  <Accordion title="Use GitHub Concurrency Keys">
-    Prevent multiple workflow runs from competing for the same slots by using [GitHub's concurrency controls](https://docs.github.com/actions/writing-workflows/choosing-what-your-workflow-does/control-the-concurrency-of-workflows-and-jobs):
-
-    ```yaml .github/workflows/test.yml
-    name: Tests
-
-    on:
-      push:
-        branches: [main]
-      pull_request:
-
-    # Prevent concurrent runs from competing for license slots
-    concurrency:
-      group: ${{ github.workflow }}-${{ github.ref }}
-      cancel-in-progress: true
-
-    jobs:
-      test:
-        runs-on: ubuntu-latest
-        steps:
-          - uses: actions/checkout@v4
-          
-          - name: Setup Node.js
-            uses: actions/setup-node@v4
-            with:
-              node-version: '20'
-          
-          - name: Install dependencies
-            run: npm install
-          
-          - name: Run tests
-            run: vitest run
-            env:
-              TD_API_KEY: ${{ secrets.TD_API_KEY }}
-    ```
-
-    The `concurrency` block ensures:
-    - Only one workflow run per branch runs at a time
-    - New pushes cancel in-progress runs on the same branch
-    - Different branches/PRs can run in parallel (up to your slot limit)
-  </Accordion>
-</AccordionGroup>
-
-## When to Consider Self-Hosted
-
-Cloud is perfect for getting started and for teams that want zero infrastructure management. However, you might consider [Self-Hosted](/v7/self-hosted) if you:
-
-- Want to escape per-second billing with a flat license fee
-- Require greater concurrency than offered in Cloud plans
-- Need full control over your infrastructure and privacy
-- Want to use your own AI API keys
-- Require custom hardware configurations
-- Have high test volumes that make self-hosting more economical
-
-<Card
-  title="Explore Self-Hosted"
-  icon="server"
-  href="/v7/self-hosted"
->
-  Learn about self-hosting for unlimited test execution at a flat rate.
-</Card>

package/ai/skills/testdriver-device-config/SKILL.md

@@ -1,317 +0,0 @@
----
-name: testdriver:device-config
-description: Launch browsers, desktop apps, and extensions in your TestDriver sandbox
----
-<!-- Generated from device-config.mdx. DO NOT EDIT. -->
-
-Provision methods are the starting point for most tests. They launch applications in your sandbox and prepare the environment for testing.
-
-## Chrome Browser
-
-The most common starting point for web testing. Launches Chrome browser and navigates to a URL.
-
-```javascript
-await testdriver.provision.chrome({
-  url: 'https://example.com',
-});
-```
-
-### Options
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `url` | string | `'http://testdriver-sandbox.vercel.app/'` | URL to navigate to |
-| `maximized` | boolean | `true` | Start browser maximized |
-| `guest` | boolean | `false` | Use guest mode (no profile) |
-
-### Example: Basic Web Test
-
-```javascript
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "testdriverai/vitest/hooks";
-
-describe("Login Flow", () => {
-  it("should log in successfully", async (context) => {
-    const testdriver = TestDriver(context);
-    
-    await testdriver.provision.chrome({
-      url: 'https://myapp.com/login',
-    });
-
-    await testdriver.find("Email input").click();
-    await testdriver.type("user@example.com");
-    
-    await testdriver.find("Password input").click();
-    await testdriver.type("password123");
-    
-    await testdriver.find("Sign In button").click();
-    
-    const result = await testdriver.assert("the dashboard is visible");
-    expect(result).toBeTruthy();
-  });
-});
-```
-
-<Info>
-  `provision.chrome()` automatically starts Dashcam recording and waits for Chrome to be ready before returning.
-</Info>
-
----
-
-## Chrome Extensions
-
-Launch Chrome with a custom extension loaded. Supports both local extensions and Chrome Web Store extensions.
-
-### Load from Local Path
-
-Clone or create an extension locally, then load it:
-
-```javascript
-// First, get the extension onto the sandbox
-await testdriver.exec(
-  'sh',
-  'git clone https://github.com/user/my-extension.git /tmp/my-extension',
-  60000
-);
-
-// Launch Chrome with the extension
-await testdriver.provision.chromeExtension({
-  extensionPath: '/tmp/my-extension',
-  url: 'https://example.com'
-});
-```
-
-### Load from Chrome Web Store
-
-Load any published extension by its Chrome Web Store ID:
-
-```javascript
-await testdriver.provision.chromeExtension({
-  extensionId: 'cjpalhdlnbpafiamejdnhcphjbkeiagm', // uBlock Origin
-  url: 'https://example.com'
-});
-```
-
-<Tip>
-  Find the extension ID in the Chrome Web Store URL. For example, `https://chrome.google.com/webstore/detail/ublock-origin/cjpalhdlnbpafiamejdnhcphjbkeiagm` → ID is `cjpalhdlnbpafiamejdnhcphjbkeiagm`
-</Tip>
-
-### Options
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `extensionPath` | string | - | Local path to unpacked extension directory |
-| `extensionId` | string | - | Chrome Web Store extension ID |
-| `url` | string | - | URL to navigate to after launch |
-| `maximized` | boolean | `true` | Start browser maximized |
-
-<Warning>
-  You must provide either `extensionPath` or `extensionId`, but not both.
-</Warning>
-
-### Example: Testing a Chrome Extension
-
-```javascript
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "testdriverai/vitest/hooks";
-
-describe("Chrome Extension Test", () => {
-  it("should load and interact with extension", async (context) => {
-    const testdriver = TestDriver(context);
-
-    // Clone extension from GitHub
-    await testdriver.exec(
-      'sh',
-      'git clone https://github.com/user/my-extension.git /tmp/my-extension',
-      60000,
-      true
-    );
-
-    // Launch Chrome with extension loaded
-    await testdriver.provision.chromeExtension({
-      extensionPath: '/tmp/my-extension',
-      url: 'https://testdriver.ai'
-    });
-
-    // Click extensions puzzle icon
-    const extensionsButton = await testdriver.find("puzzle-shaped icon in Chrome toolbar");
-    await extensionsButton.click();
-
-    // Interact with your extension
-    const myExtension = await testdriver.find("My Extension in the dropdown");
-    await myExtension.click();
-
-    const result = await testdriver.assert("extension popup is visible");
-    expect(result).toBeTruthy();
-  });
-});
-```
-
----
-
-## Desktop Apps
-
-Download and install desktop applications. Supports `.deb`, `.rpm`, `.msi`, `.exe`, `.AppImage`, `.dmg`, `.pkg`, and shell scripts.
-
-```javascript
-const filePath = await testdriver.provision.installer({
-  url: 'https://example.com/app.deb',
-  appName: 'MyApp',  // Focus this app after install
-  launch: true,      // Auto-launch after install
-});
-```
-
-### Options
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `url` | string | **required** | URL to download the installer from |
-| `filename` | string | auto-detected | Filename to save as |
-| `appName` | string | - | Application name to focus after install |
-| `launch` | boolean | `true` | Launch the app after installation |
-
-### Supported File Types
-
-| Extension | OS | Install Method |
-|-----------|-----|----------------|
-| `.deb` | Linux | `dpkg -i` + `apt-get install -f` |
-| `.rpm` | Linux | `rpm -i` |
-| `.AppImage` | Linux | `chmod +x` |
-| `.sh` | Linux | `chmod +x` + execute |
-| `.msi` | Windows | `msiexec /i /quiet` |
-| `.exe` | Windows | Silent install (`/S`) |
-| `.dmg` | macOS | Mount + copy to Applications |
-| `.pkg` | macOS | `installer -pkg` |
-
-### Example: Install and Test a Desktop App
-
-```javascript
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "testdriverai/vitest/hooks";
-
-describe("Desktop App Test", () => {
-  it("should install and launch app", async (context) => {
-    const testdriver = TestDriver(context);
-
-    // Download and install
-    const installerPath = await testdriver.provision.installer({
-      url: 'https://github.com/sharkdp/bat/releases/download/v0.24.0/bat_0.24.0_amd64.deb',
-    });
-
-    // Verify installation
-    const output = await testdriver.exec('sh', 'bat --version', 5000);
-    expect(output).toContain('bat');
-  });
-});
-```
-
-### Example: Windows Installer
-
-```javascript
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "testdriverai/vitest/hooks";
-
-describe("Windows App Test", () => {
-  it("should install on Windows", async (context) => {
-    const testdriver = TestDriver(context, { 
-      os: 'windows'
-    });
-
-    // Download MSI installer
-    const installerPath = await testdriver.provision.installer({
-      url: 'https://example.com/app.msi',
-      launch: false,  // Don't auto-launch
-    });
-
-    // Custom installation if needed
-    await testdriver.exec(
-      'pwsh',
-      `Start-Process msiexec.exe -ArgumentList "/i", "${installerPath}", "/qn" -Wait`,
-      120000
-    );
-
-    // Verify installation
-    const result = await testdriver.assert("application is installed");
-    expect(result).toBeTruthy();
-  });
-});
-```
-
-### Manual Installation
-
-Set `launch: false` to download without auto-installing:
-
-```javascript
-const filePath = await testdriver.provision.installer({
-  url: 'https://example.com/custom-script.sh',
-  launch: false,
-});
-
-// Run custom install commands
-await testdriver.exec('sh', `chmod +x "${filePath}"`, 5000);
-await testdriver.exec('sh', `"${filePath}" --custom-flag`, 60000);
-```
-
----
-
-## VS Code
-
-Launch Visual Studio Code with optional workspace and extensions.
-
-```javascript
-await testdriver.provision.vscode({
-  workspace: '/home/testdriver/my-project',
-  extensions: ['ms-python.python', 'esbenp.prettier-vscode'],
-});
-```
-
-### Options
-
-| Option | Type | Default | Description |
-|--------|------|---------|-------------|
-| `workspace` | string | - | Workspace folder to open |
-| `extensions` | string[] | `[]` | Extensions to install (by ID) |
-
-### Example: VS Code Extension Test
-
-```javascript
-import { describe, expect, it } from "vitest";
-import { TestDriver } from "testdriverai/vitest/hooks";
-
-describe("VS Code Test", () => {
-  it("should open workspace with extensions", async (context) => {
-    const testdriver = TestDriver(context);
-
-    // Create a test project
-    await testdriver.exec('sh', 'mkdir -p /tmp/test-project && echo "print(1)" > /tmp/test-project/test.py', 10000);
-
-    // Launch VS Code
-    await testdriver.provision.vscode({
-      workspace: '/tmp/test-project',
-      extensions: ['ms-python.python'],
-    });
-
-    // Verify VS Code is ready
-    const result = await testdriver.assert("VS Code is open with the project");
-    expect(result).toBeTruthy();
-
-    // Open the Python file
-    await testdriver.find("test.py in the explorer").click();
-  });
-});
-```
-
----
-
-## Choosing the Right Provision Method
-
-| Use Case | Method |
-|----------|--------|
-| Testing a website | `provision.chrome` |
-| Testing a Chrome extension | `provision.chromeExtension` |
-| Testing a desktop app (needs installation) | `provision.installer` |
-| Testing VS Code or VS Code extensions | `provision.vscode` |
-
-<Tip>
-  All provision methods automatically start Dashcam recording and wait for the application to be ready before returning. You don't need to call `dashcam.start()` manually.
-</Tip>