@testdriverai/mcp 7.10.0-canary.0 → 7.10.1-test

package/ai/skills/testdriver:secrets/SKILL.md

@@ -0,0 +1,115 @@
+---
+name: testdriver:secrets
+description: Securely manage passwords and sensitive data in your tests
+---
+<!-- Generated from secrets.mdx. DO NOT EDIT. -->
+
+Protect sensitive information like passwords, API keys, and tokens in your TestDriver tests.
+
+## Typing Secrets Securely
+
+When typing sensitive information like passwords, use the `secret: true` option to prevent the value from being logged or stored:
+
+```javascript
+import { test } from 'vitest';
+import { chrome } from 'testdriverai/presets';
+
+test('login with secure password', async (context) => {
+  const { testdriver } = await chrome(context, { 
+    url: 'https://myapp.com/login' 
+  });
+
+  await testdriver.find('email input').click();
+  await testdriver.type(process.env.TD_USERNAME);
+  
+  await testdriver.find('password input').click();
+  // Password is masked in logs and recordings
+  await testdriver.type(process.env.TD_PASSWORD, { secret: true });
+  
+  await testdriver.find('login button').click();
+  await testdriver.assert('dashboard is visible');
+});
+```
+
+<Note>
+When `secret: true` is set, the typed text appears as `****` in all logs, recordings, and dashcam output.
+</Note>
+
+## Storing Secrets in GitHub
+
+Store sensitive credentials as GitHub repository secrets so they're never exposed in your code:
+
+<Steps>
+  <Step title="Navigate to Repository Settings">
+    Go to your GitHub repository → **Settings** → **Secrets and variables** → **Actions**
+  </Step>
+  <Step title="Add Repository Secrets">
+    Click **New repository secret** and add your secrets:
+    - `TD_API_KEY` - Your TestDriver API key
+    - `TD_USERNAME` - Test account username
+    - `TD_PASSWORD` - Test account password
+  </Step>
+  <Step title="Use in GitHub Actions">
+    Reference secrets in your workflow file:
+    ```yaml .github/workflows/test.yml
+    - name: Run TestDriver tests
+      env:
+        TD_API_KEY: ${{ secrets.TD_API_KEY }}
+        TD_USERNAME: ${{ secrets.TD_USERNAME }}
+        TD_PASSWORD: ${{ secrets.TD_PASSWORD }}
+      run: vitest run
+    ```
+  </Step>
+</Steps>
+
+## Local Development
+
+For local development, store secrets in a `.env` file:
+
+```bash .env
+TD_API_KEY=your_api_key_here
+TD_USERNAME=testuser@example.com
+TD_PASSWORD=your_secure_password
+```
+
+<Warning>
+Never commit `.env` files to version control. Add `.env` to your `.gitignore` file.
+</Warning>
+
+## Complete Example
+
+Here's a full login test with proper secrets handling:
+
+```javascript tests/login.test.js
+import { test, expect } from 'vitest';
+import { chrome } from 'testdriverai/presets';
+
+test('secure login flow', async (context) => {
+  const { testdriver } = await chrome(context, { 
+    url: process.env.TD_WEBSITE || 'https://staging.myapp.com'
+  });
+
+  // Enter username (not sensitive)
+  await testdriver.find('email input').click();
+  await testdriver.type(process.env.TD_USERNAME);
+  
+  // Enter password securely
+  await testdriver.find('password input').click();
+  await testdriver.type(process.env.TD_PASSWORD, { secret: true });
+  
+  // Submit login
+  await testdriver.find('login button').click();
+  
+  // Verify successful login
+  const loggedIn = await testdriver.assert('user is logged in');
+  expect(loggedIn).toBeTruthy();
+});
+```
+
+<Card title="Secrets Best Practices" icon="shield-check">
+  - **Always use `secret: true`** when typing passwords, tokens, or sensitive data
+  - **Use environment variables** to keep secrets out of code
+  - **Store secrets in your CI provider** (GitHub Actions, GitLab CI, etc.)
+  - **Never commit secrets** to version control
+  - **Rotate secrets regularly** to maintain security
+</Card>

package/ai/skills/testdriver:self-hosted/SKILL.md

@@ -0,0 +1,147 @@
+---
+name: testdriver:self-hosted
+description: Our enterprise solution with unlimited test execution, assisted setup, and dedicated support.
+---
+<!-- Generated from self-hosted.mdx. DO NOT EDIT. -->
+
+Self-hosted is our enterprise solution for teams that need unlimited test execution, infrastructure control, and dedicated support. Pricing is based on **parallel test capacity** with a flat license fee — no per-second billing.
+
+<CardGroup cols={2}>
+  <Card title="Unlimited Execution" icon="infinity">
+    Run as many tests as you want with no device-second metering. Predictable monthly costs.
+  </Card>
+  <Card title="Assisted Setup & Support" icon="headset">
+    Our team helps you deploy, configure, and optimize your infrastructure. Dedicated engineering support included.
+  </Card>
+  <Card title="Full Control" icon="gear">
+    Use your own AI keys, custom hardware, specific software, and network configurations. RDP into test machines for debugging.
+  </Card>
+  <Card title="Security & Compliance" icon="shield-check">
+    Keep data in your environment. Air-gapped deployment available for regulated industries.
+  </Card>
+</CardGroup>
+
+## Deployment Options
+
+Choose the level of control you need:
+
+| Component | Standard | Air-Gapped |
+|-----------|----------|------------|
+| **Test Sandboxes** | Your AWS | Your infrastructure (any cloud or on-prem) |
+| **Dashboard** | TestDriver hosted | Your infrastructure |
+| **API** | TestDriver hosted | Your infrastructure |
+| **AI Processing** | Your API keys | Your infrastructure |
+| **Data Storage** | Your AWS account | 100% your infrastructure |
+| **Network** | Internet access required | Fully air-gapped |
+| **Cloud Providers** | AWS | AWS, Azure, GCP, on-prem |
+
+### Standard Deployment
+
+Run test sandboxes on your AWS infrastructure while using TestDriver's hosted dashboard and API:
+
+- **Quick setup** via CloudFormation — deploy in hours
+- **Dashboard access** at [console.testdriver.ai](https://console.testdriver.ai)
+- **Your AI keys** — control costs with your own OpenAI, Anthropic, or other provider
+- **Custom AMIs** — install specific software, configure networking
+
+<Card
+  title="AWS Setup Guide"
+  icon="aws"
+  href="/v7/aws-setup"
+>
+  Step-by-step instructions for deploying TestDriver on your AWS infrastructure using CloudFormation.
+</Card>
+
+### Air-Gapped Deployment
+
+Deploy the entire TestDriver stack in your environment for complete isolation:
+
+- **Full stack** — dashboard, API, and test infrastructure all in your environment
+- **No external dependencies** — data never leaves your network perimeter
+- **Any infrastructure** — AWS, Azure, GCP, or on-premises
+- **Regulated industries** — government, defense, healthcare, finance
+
+## Custom VM Images
+
+Build test environments with your applications, dependencies, and user data pre-installed. You get full access to:
+
+- **Golden VM** — our pre-configured base image with TestDriver agent, drivers, and optimizations
+- **Packer scripts** — build custom AMIs with your applications, user data, and configurations
+- **Faster test startup** — skip installation steps by baking dependencies into your image
+- **Consistent environments** — every test runs on an identical, reproducible machine
+
+<AccordionGroup>
+  <Accordion title="What can you customize?">
+    - Install applications (browsers, desktop apps, dev tools)
+    - Configure user accounts and credentials
+    - Set up network proxies and certificates
+    - Install fonts, language packs, and locales
+    - Pre-seed databases or test fixtures
+    - Configure Windows/Linux settings
+  </Accordion>
+  
+  <Accordion title="How it works">
+    1. We provide our golden VM base image and Packer scripts
+    2. You customize the scripts to install your software and configuration
+    3. Run Packer to build your custom AMI
+    4. Configure TestDriver to use your custom AMI for test sandboxes
+    5. Tests spin up with everything pre-installed — no setup time wasted
+  </Accordion>
+</AccordionGroup>
+
+## Implementation Process
+
+<Steps>
+  <Step title="Discovery Call">
+    Discuss your requirements, security constraints, and integration needs with our team.
+  </Step>
+  
+  <Step title="Architecture Review">
+    Our engineers design a deployment architecture that meets your security and compliance requirements.
+  </Step>
+  
+  <Step title="Deployment">
+    We work with your team to deploy TestDriver, including assisted setup and configuration.
+  </Step>
+  
+  <Step title="Integration">
+    Connect TestDriver to your CI/CD pipelines, internal tools, and workflows.
+  </Step>
+  
+  <Step title="Training & Handoff">
+    Comprehensive training for your team on operating and maintaining the deployment.
+  </Step>
+</Steps>
+
+## What's Included
+
+- **Flat license fee** per parallel test slot
+- **Unlimited test execution** — no device-second charges
+- **Assisted setup** — our team helps you deploy and configure
+- **Dedicated support** — direct access to our engineering team
+- **Custom contract terms** — volume-based pricing, custom SLAs
+- **Professional services** — implementation assistance and training
+
+## Comparison: Hosted vs Self-Hosted
+
+| Feature | Hosted | Self-Hosted |
+|---------|--------|-------------|
+| **Setup Time** | Minutes | Hours (assisted) |
+| **Pricing Model** | Device-seconds | Flat license fee |
+| **Infrastructure** | TestDriver | Your AWS or any cloud |
+| **AI API Keys** | TestDriver's | Your own |
+| **Custom Software** | Limited | Full control |
+| **Hardware Selection** | Standard | Your choice |
+| **Debugging Access** | Replays only | Full RDP access |
+| **Support** | Community/Standard | Dedicated engineering |
+| **Air-Gapped Option** | No | Yes |
+
+## Get Started
+
+<Card
+  title="Schedule a Consultation"
+  icon="calendar"
+  href="https://testdriver.ai/demo"
+>
+  Discuss your requirements with our team and get a custom proposal for your self-hosted deployment.
+</Card>

package/ai/skills/testdriver:test-results-json/SKILL.md

@@ -0,0 +1,257 @@
+---
+name: testdriver:test-results-json
+description: Per-test JSON result files with metadata, versions, and infrastructure details
+---
+<!-- Generated from test-results-json.mdx. DO NOT EDIT. -->
+
+## Overview
+
+TestDriver automatically writes a JSON result file for each test case after it finishes. These files contain comprehensive metadata about the test run, including SDK and runner versions, infrastructure details, interaction statistics, and links to recordings.
+
+Result files are written to:
+
+```
+.testdriver/results/<testFile>/<testName>.json
+```
+
+For example, a test file `tests/login.test.mjs` with a test named `"should log in"` produces:
+
+```
+.testdriver/results/tests/login.test.mjs/should_log_in.json
+```
+
+<Note>
+  Test names are sanitized for filesystem use — special characters are replaced with underscores and names are truncated to 200 characters.
+</Note>
+
+## Enabling
+
+No configuration is required. The JSON files are written automatically by the TestDriver Vitest reporter plugin whenever tests run.
+
+## JSON Schema
+
+Each result file is organized into logical groups:
+
+### `versions`
+
+| Field | Type | Description |
+|---|---|---|
+| `versions.sdk` | `string \| null` | TestDriver SDK version (e.g. `"7.8.0"`) |
+| `versions.vitest` | `string \| null` | Vitest version used to run the test |
+| `versions.api` | `string \| null` | TestDriver API server version |
+| `versions.runnerBefore` | `string \| null` | Runner version at sandbox start |
+| `versions.runnerAfter` | `string \| null` | Runner version after auto-update |
+| `versions.runnerWasUpdated` | `boolean` | Whether the runner was auto-updated during provisioning |
+
+### `test`
+
+| Field | Type | Description |
+|---|---|---|
+| `test.file` | `string \| null` | Relative path to the test file |
+| `test.name` | `string \| null` | Name of the test case |
+| `test.suite` | `string \| null` | Name of the parent `describe` block |
+| `test.passed` | `boolean` | Whether the test passed |
+| `test.caseId` | `string \| null` | Database ID for this test case |
+| `test.runId` | `string \| null` | Database ID for the overall test run |
+| `test.error` | `string \| null` | Error message if the test failed |
+| `test.errorStack` | `string \| null` | Error stack trace if the test failed |
+
+### `urls`
+
+| Field | Type | Description |
+|---|---|---|
+| `urls.api` | `string \| null` | API root URL used for this test |
+| `urls.console` | `string \| null` | TestDriver console base URL |
+| `urls.vnc` | `string \| null` | VNC URL for the sandbox |
+| `urls.testRun` | `string \| null` | Direct link to this test case in the console |
+
+### `replay`
+
+The `replay` object contains the recording replay URL and derived embed links. The `gifUrl` and `embedUrl` are generated automatically from the replay URL.
+
+| Field | Type | Description |
+|---|---|---|
+| `replay.url` | `string \| null` | Recording replay URL |
+| `replay.gifUrl` | `string \| null` | Animated GIF thumbnail of the recording |
+| `replay.embedUrl` | `string \| null` | Embeddable replay URL (appends `&embed=true`) |
+| `replay.markdown` | `string \| null` | Ready-to-use Markdown embed with GIF linking to the replay |
+
+The `replay.markdown` field produces a clickable GIF badge you can paste directly into PR comments, README files, or issue descriptions:
+
+```markdown
+[![Test Recording](https://api.testdriver.ai/replay/abc123/gif?shareKey=xyz)](https://console.testdriver.ai/replay/abc123?share=xyz)
+```
+
+### `date`
+
+| Field | Type | Description |
+|---|---|---|
+| `date` | `string` | ISO 8601 timestamp when the test finished |
+
+### `team`
+
+| Field | Type | Description |
+|---|---|---|
+| `team.id` | `string \| null` | Team ID from the sandbox |
+| `team.sessionId` | `string \| null` | SDK session ID |
+
+### `infrastructure`
+
+| Field | Type | Description |
+|---|---|---|
+| `infrastructure.sandboxId` | `string \| null` | Sandbox instance ID |
+| `infrastructure.instanceId` | `string \| null` | Instance ID |
+| `infrastructure.os` | `string \| null` | Operating system of the sandbox (`"linux"` or `"windows"`) |
+| `infrastructure.amiId` | `string \| null` | AWS AMI ID used for provisioning |
+| `infrastructure.e2bTemplateId` | `string \| null` | E2B template ID used for provisioning |
+| `infrastructure.imageVersion` | `string \| null` | Sandbox image version |
+
+### `realtime`
+
+| Field | Type | Description |
+|---|---|---|
+| `realtime.channel` | `string \| null` | Ably channel name used for communication |
+| `realtime.messageCount` | `number` | Number of messages published to the realtime channel |
+
+### `interactions`
+
+| Field | Type | Description |
+|---|---|---|
+| `interactions.total` | `number` | Total number of interactions recorded |
+| `interactions.cached` | `number` | Number of interactions served from cache |
+| `interactions.byType` | `object` | Breakdown of interactions by type (e.g. `find`, `click`, `assert`) |
+
+## Example Output
+
+```json
+{
+  "sdkVersion": "7.8.0",
+  "vitestVersion": "4.0.0",
+  "apiVersion": "1.45.0",
+  "runnerVersionBefore": "2.1.0",
+  "runnerVersionAfter": "2.1.1",
+  "wasUpdated": true,
+  "apiUrl": "https://api.testdriver.ai",
+  "consoleUrl": "https://console.testdriver.ai",
+  "testRunLink": "https://console.testdriver.ai/runs/abc123/def456",
+  "dashcamUrl": "https://app.dashcam.io/replay/abc123",
+  "vncUrl": "wss://sandbox-123.testdriver.ai/vnc",
+  "date": "2025-01-15T14:30:00.000Z",
+  "team": {
+    "id": "team_abc123",
+    "sessionId": "sess_xyz789"
+  },
+  "infrastructure": {
+    "sandboxId": "sandbox-123",
+    "instanceId": "i-abc123",
+    "os": "linux",
+    "amiId": "ami-0abc123",
+    "e2bTemplateId": null,
+    "imageVersion": "v2.1.0"
+  },
+  "realtime": {
+    "channel": "sandbox:sandbox-123",
+    "messageCount": 42
+  },
+  "interactions": {
+    "total": 15,
+    "cached": 3,
+    "byType": {
+      "find": 8,
+      "click": 5,
+      "assert": 2
+    }
+  }
+}
+```
+
+## Using Result Files in CI
+
+Result files are useful for extracting test metadata in CI pipelines without parsing log output.
+
+### GitHub Actions Example
+
+Use `fromJSON` to parse a result file into a GitHub Actions expression you can reference in subsequent steps:
+
+```yaml
+- name: Run tests
+  run: npx vitest run tests/login.test.mjs
+
+- name: Parse result
+  id: result
+  run: |
+    # Read the first JSON result file
+    FILE=$(find .testdriver/results -name '*.json' | head -n 1)
+    echo "json=$(cat "$FILE")" >> "$GITHUB_OUTPUT"
+
+- name: Comment on PR
+  if: fromJSON(steps.result.outputs.json).test.passed == false
+  uses: actions/github-script@v7
+  with:
+    script: |
+      const result = ${{ steps.result.outputs.json }};
+      await github.rest.issues.createComment({
+        owner: context.repo.owner,
+        repo: context.repo.repo,
+        issue_number: context.issue.number,
+        body: [
+          `❌ **${result.test.name}** failed`,
+          ``,
+          `Error: ${result.test.error}`,
+          ``,
+          result.replay.markdown,
+          ``,
+          `[View full recording](${result.urls.testRun})`
+        ].join('\n')
+      });
+```
+
+You can also load all results into a matrix or iterate over them:
+
+```yaml
+- name: Run tests
+  run: npx vitest run tests/*.test.mjs
+
+- name: Collect results
+  id: results
+  run: |
+    # Merge all result files into a JSON array
+    echo "json=$(find .testdriver/results -name '*.json' -exec cat {} + | jq -s '.')" >> "$GITHUB_OUTPUT"
+
+- name: Summary
+  run: |
+    echo '## Test Results' >> $GITHUB_STEP_SUMMARY
+    RESULTS='${{ steps.results.outputs.json }}'
+    echo "$RESULTS" | jq -r '.[] | "| \(.test.name) | \(if .test.passed then "✅" else "❌" end) | \(.urls.testRun) |"' >> $GITHUB_STEP_SUMMARY
+```
+
+### Reading Results Programmatically
+
+```javascript
+import fs from "fs";
+import path from "path";
+
+const resultsDir = ".testdriver/results";
+
+function readResults(dir) {
+  const results = [];
+  for (const testDir of fs.readdirSync(dir, { recursive: true })) {
+    const fullPath = path.join(dir, testDir);
+    if (fullPath.endsWith(".json") && fs.statSync(fullPath).isFile()) {
+      results.push(JSON.parse(fs.readFileSync(fullPath, "utf-8")));
+    }
+  }
+  return results;
+}
+
+const results = readResults(resultsDir);
+const passed = results.filter(r => r.test.passed);
+const failed = results.filter(r => !r.test.passed);
+
+console.log(`${passed.length} passed, ${failed.length} failed`);
+for (const r of failed) {
+  console.log(`  FAIL: ${r.test.name} — ${r.test.error}`);
+  console.log(`  Recording: ${r.urls.testRun}`);
+  console.log(`  Embed: ${r.replay.markdown}`);
+}
+```