@testdriverai/agent 7.8.0-canary.10

package/docs/v7/_drafts/architecture.mdx

@@ -0,0 +1,399 @@
+# TestDriver Test Recording Architecture
+
+## Overview
+
+This system provides comprehensive test execution tracking, linking test runs with dashcam screen recordings and CI/CD pipelines in the TestDriver dashboard.
+
+## Architecture Diagram
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    Developer's Machine / CI                  │
+├─────────────────────────────────────────────────────────────┤
+│                                                               │
+│  ┌──────────────┐         ┌──────────────┐                  │
+│  │   Vitest     │────────▶│  TD Vitest   │                  │
+│  │  Test Runner │         │   Plugin     │                  │
+│  └──────────────┘         └──────┬───────┘                  │
+│                                   │                           │
+│  ┌──────────────┐                 │                          │
+│  │   Dashcam    │                 │                          │
+│  │  Recording   │                 │                          │
+│  └──────┬───────┘                 │                          │
+│         │                         │                           │
+│         │ (records screen)        │ (reports results)        │
+│         │                         │                           │
+└─────────┼─────────────────────────┼───────────────────────────┘
+          │                         │
+          │                         │
+          ▼                         ▼
+┌─────────────────────────────────────────────────────────────┐
+│                   TestDriver API Server                      │
+├─────────────────────────────────────────────────────────────┤
+│                                                               │
+│  ┌──────────────┐         ┌──────────────┐                  │
+│  │ Replay API   │         │ Test Run API │                  │
+│  │ (dashcam)    │         │ (new)        │                  │
+│  └──────┬───────┘         └──────┬───────┘                  │
+│         │                        │                           │
+│         └────────┬───────────────┘                           │
+│                  │                                            │
+│                  ▼                                            │
+│         ┌────────────────┐                                   │
+│         │   MongoDB      │                                   │
+│         │                │                                   │
+│         │ • TdTestRun    │                                   │
+│         │ • TdTestCase   │                                   │
+│         │ • Replay       │                                   │
+│         └────────────────┘                                   │
+│                                                               │
+└───────────────────────────┬───────────────────────────────────┘
+                            │
+                            │
+                            ▼
+┌─────────────────────────────────────────────────────────────┐
+│                  TestDriver Web Dashboard                    │
+├─────────────────────────────────────────────────────────────┤
+│                                                               │
+│  ┌─────────────────────────────────────────┐                │
+│  │        Test Runs View (NEW)             │                │
+│  │                                          │                │
+│  │  • List all test runs                   │                │
+│  │  • Filter by status, date, CI           │                │
+│  │  • Show pass/fail statistics            │                │
+│  │  • Link to CI/CD runs                   │                │
+│  └─────────────────────────────────────────┘                │
+│                                                               │
+│  ┌─────────────────────────────────────────┐                │
+│  │      Test Run Detail View (NEW)         │                │
+│  │                                          │                │
+│  │  • Test case list with status           │                │
+│  │  • Dashcam replay player (embedded)     │                │
+│  │  • Error messages and stack traces      │                │
+│  │  • Sandbox details                      │                │
+│  │  • Git commit info                      │                │
+│  └─────────────────────────────────────────┘                │
+│                                                               │
+└───────────────────────────────────────────────────────────────┘
+```
+
+## Data Models
+
+### TdTestRun
+Represents a complete test suite execution (e.g., `vitest run`).
+
+**Key Fields:**
+- `runId`: Unique identifier
+- `suiteName`: Name of the test suite
+- `status`: running | passed | failed | cancelled
+- `totalTests`, `passedTests`, `failedTests`: Statistics
+- `platform`: windows | mac | linux
+- CI/CD metadata (provider, runId, url)
+- Git metadata (repo, branch, commit)
+- `dashcamSessionId`: Links to dashcam recordings
+
+**Relationships:**
+- `team`: Owner team
+- `testCases`: Collection of TdTestCase
+- `replays`: Associated Replay records
+
+### TdTestCase
+Represents an individual test within a test run.
+
+**Key Fields:**
+- `testName`: Test name from `test('...')`
+- `testFile`: Path to test file
+- `suiteName`: Describe block name
+- `status`: passed | failed | skipped | pending
+- `duration`: Test duration in ms
+- `errorMessage`, `errorStack`: Failure details
+- `replayUrl`: Direct link to dashcam replay
+- `replayStartTime`, `replayEndTime`: Timestamps within replay
+
+**Relationships:**
+- `testRun`: Parent TdTestRun
+- `replay`: Associated Replay recor
+### Replay (Extended)
+Existing model extended with test run associations.
+
+**New Fields:**
+- `tdTestRun`: Associated test run
+- `tdTestCase`: Associated test case
+
+## API Endpoints
+
+### POST /api/v1/testdriver/test-run-create
+Create a new test run.
+
+**Auth:** Required (Bearer token)
+
+**Request:**
+```json
+{
+  "runId": "vitest-1234567890-abc123",
+  "suiteName": "Integration Tests",
+  "platform": "windows",
+  "sandboxId": "sandbox-xyz",
+  "ciProvider": "GitHub Actions",
+  "ciRunId": "12345",
+  "repo": "myorg/myrepo",
+  "branch": "main",
+  "commit": "abc123def456"
+}
+```
+
+**Response:**
+```json
+{
+  "data": {
+    "id": "...",
+    "runId": "vitest-1234567890-abc123",
+    "status": "running",
+    "startTime": 1700000000000
+  }
+}
+```
+
+### POST /api/v1/testdriver/test-run-complete
+Mark a test run as complete.
+
+**Auth:** Required
+
+**Request:**
+```json
+{
+  "runId": "vitest-1234567890-abc123",
+  "status": "passed",
+  "totalTests": 25,
+  "passedTests": 24,
+  "failedTests": 1,
+  "skippedTests": 0
+}
+```
+
+### POST /api/v1/testdriver/test-case-create
+Record a test case result (create or update).
+
+**Auth:** Required
+
+**Request:**
+```json
+{
+  "runId": "vitest-1234567890-abc123",
+  "testName": "should login successfully",
+  "testFile": "tests/auth/login.test.js",
+  "suiteName": "Authentication Tests",
+  "status": "passed",
+  "startTime": 1700000001000,
+  "endTime": 1700000002500,
+  "duration": 1500,
+  "replayUrl": "https://app.dashcam.io/replay/abc123"
+}
+```
+
+## Components
+
+### Vitest Plugin (`interfaces/vitest-plugin.mjs`)
+Automatically integrates with Vitest test runs.
+
+**Features:**
+- Auto-detects CI/CD environment (GitHub Actions, GitLab, etc.)
+- Extracts Git metadata from environment or git commands
+- Creates test run at start
+- Records each test case result
+- Associates with dashcam session if `DASHCAM_SESSION_ID` is set
+- Completes test run with statistics
+- Uses plugin architecture for better global state management
+
+**Usage:**
+```javascript
+// vitest.config.mjs
+import testDriverPlugin from './interfaces/vitest-plugin.mjs';
+
+export default {
+  plugins: [
+    testDriverPlugin({
+      apiKey: process.env.TD_API_KEY,
+      apiRoot: process.env.TD_API_ROOT || 'https://api.testdriver.ai',
+    }),
+  ],
+}
+```
+
+### SDK Methods (`sdk.js`)
+
+#### `client.createTestRun(options)`
+Create a test run programmatically.
+
+#### `client.recordTestCase(options)`
+Record a test case result.
+
+#### `client.completeTestRun(options)`
+Mark test run as complete.
+
+## Integration Flows
+
+### Flow 1: Automated with Vitest Reporter
+
+```
+1. Developer runs: vitest run
+2. Vitest starts, reporter initializes
+3. Reporter creates TdTestRun
+4. For each test:
+   - Vitest runs test
+   - Reporter records TdTestCase (passed/failed)
+5. All tests complete
+6. Reporter calls completeTestRun()
+7. Results visible in dashboard
+```
+
+### Flow 2: With Dashcam Recording
+
+```
+1. Start dashcam: dashcam start
+2. Set session ID: export DASHCAM_SESSION_ID=$(dashcam session-id)
+3. Run tests: vitest run
+4. Reporter creates test run with dashcamSessionId
+5. Tests execute, dashcam records
+6. Stop dashcam: dashcam stop
+7. Publish: dashcam publish -p PROJECT_ID
+8. Replay URL returned
+9. Dashboard shows test results + replay link
+```
+
+### Flow 3: CI/CD Pipeline (GitHub Actions)
+
+```yaml
+jobs:
+  test:
+    runs-on: windows-latest
+    steps:
+      - uses: actions/checkout@v3
+      - run: npm install
+      - name: Start Dashcam
+        run: |
+          dashcam start
+          echo "DASHCAM_SESSION_ID=$(dashcam session-id)" >> $GITHUB_ENV
+      - name: Run Tests
+        env:
+          TD_API_KEY: ${{ secrets.TD_API_KEY }}
+        run: vitest run
+      - name: Publish Recording
+        if: always()
+        run: |
+          dashcam stop
+          dashcam publish -p ${{ secrets.DASHCAM_PROJECT_ID }}
+```
+
+Reporter auto-detects:
+- GitHub repo, branch, commit
+- Workflow run ID and URL
+- Job ID
+- Actor (who triggered)
+
+## Dashcam Association Strategies
+
+### Strategy 1: Session ID (Implemented)
+- Set `DASHCAM_SESSION_ID` environment variable
+- Reporter includes in test run creation
+- Dashboard queries replays by session ID
+- Shows all replays from that session
+
+### Strategy 2: Explicit URL (Implemented)
+- Dashcam publishes, returns URL
+- Pass URL to `recordTestCase()`
+- Direct 1:1 link between test and replay
+
+### Strategy 3: Timestamp Matching (Future)
+- Parse dashcam logs for replay timestamps
+- Match test start/end times with replay markers
+- Automatically associate without manual linking
+- Allows seeking to exact test within long replay
+
+### Strategy 4: Log Parsing (Future)
+- Dashcam logs test names/files during recording
+- Parse logs to extract test-to-timestamp mapping
+- Generate replay URLs with timestamp seek parameters
+- Example: `https://app.dashcam.io/replay/abc123?t=45000` (seek to 45s)
+
+## Dashboard Views (To Be Built)
+
+### Test Runs List
+- Table of all test runs
+- Columns: Suite Name, Status, Tests (passed/failed), Duration, Date, CI Link
+- Filters: Status, Date range, CI provider, Platform
+- Search: By suite name, repo, branch
+
+### Test Run Detail
+- Header: Suite name, status, duration, platform
+- Statistics card: Total/passed/failed/skipped
+- Test cases table: Name, Status, Duration, Replay link
+- Sidebar: Git info, CI info, Sandbox details
+- Dashcam replay player (embedded iframe)
+- Click test case → seek replay to that test's time range
+
+### Sandbox Management
+- List of active/terminated sandboxes
+- Lifecycle timeline visualization
+- Cost tracking (duration × instance type)
+- Associated test runs
+
+## Environment Variables
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `TD_API_KEY` | Yes | TestDriver API key for authentication |
+| `DASHCAM_SESSION_ID` | No | Links test run to dashcam session |
+| `TD_SANDBOX_ID` | No | Sandbox ID if running in TestDriver sandbox |
+| `GITHUB_ACTIONS` | Auto | Detected for GitHub Actions integration |
+| `GITLAB_CI` | Auto | Detected for GitLab CI integration |
+| `CIRCLECI` | Auto | Detected for CircleCI integration |
+
+## Future Enhancements
+
+1. **Real-time Test Streaming**
+   - WebSocket connection from reporter
+   - Live test progress in dashboard
+   - See tests pass/fail as they run
+
+2. **Flaky Test Detection**
+   - Track test history across runs
+   - Identify tests that intermittently fail
+   - Suggest fixes based on error patterns
+
+3. **Performance Regression Detection**
+   - Compare test durations across runs
+   - Alert on significant slowdowns
+   - Visualize performance trends
+
+4. **Advanced Dashcam Integration**
+   - Automatic timestamp extraction from logs
+   - AI-powered test failure analysis from replays
+   - Highlight exact moment of failure in replay
+
+5. **Multi-Framework Support**
+   - Jest reporter
+   - Mocha reporter
+   - Playwright reporter
+   - Cypress plugin
+
+6. **Cost Analytics**
+   - Track sandbox costs per test run
+   - Optimize instance types
+   - Budget alerts
+
+## Security Considerations
+
+- API keys stored securely (environment variables)
+- Bearer token authentication for all API calls
+- Team-based access control (tests only visible to team members)
+- Replay access control (dashcam's existing permissions)
+- No sensitive data in test metadata (sanitize error messages)
+
+## Performance Considerations
+
+- Async test case recording (doesn't slow tests)
+- Batch updates for large test suites
+- Efficient database indexing (runId, testFile, status)
+- Replay association is lazy (doesn't block test recording)
+- Optional reporter (disable in local development)

package/docs/v7/_drafts/auto-cache-key.mdx

@@ -0,0 +1,167 @@
+# Auto-Generated Cache Keys from File Hash
+
+## Overview
+
+When you create a TestDriver instance without providing an explicit `cacheKey`, the SDK will automatically generate one based on the SHA-256 hash of the calling file. This provides automatic cache invalidation when your test file changes, while enabling cache hits for identical test runs.
+
+## How It Works
+
+1. **Stack Trace Analysis**: When `TestDriver()` is called, the SDK analyzes the call stack to find the caller file
+2. **File Hashing**: The content of the caller file is hashed using SHA-256
+3. **Cache Key Generation**: The first 16 characters of the hash are used as the cache key
+4. **Automatic Updates**: When the test file is modified, the hash changes, automatically invalidating the cache
+
+## Benefits
+
+- ✅ **No Manual Cache Management**: Cache keys are automatically generated and updated
+- ✅ **File-Scoped Caching**: All tests in the same file share the same cache
+- ✅ **Automatic Invalidation**: Cache is invalidated when the test file changes
+- ✅ **Explicit Override**: You can still provide a manual `cacheKey` if needed
+
+## Usage Examples
+
+### Automatic Cache Key (Recommended)
+
+```javascript
+import { TestDriver } from 'testdriverai/vitest/hooks';
+
+test('login test', async (context) => {
+  // No cacheKey provided - will be auto-generated from this file's hash
+  const testdriver = TestDriver(context, { 
+    headless: true 
+  });
+  
+  // Cache key is automatically set based on this file
+  console.log(testdriver.options.cacheKey); // e.g., "4cae7be040f293b9"
+  
+  const button = await testdriver.find('login button');
+  // Subsequent calls in this test file will use the same cache
+});
+```
+
+### Explicit Cache Key (Override)
+
+```javascript
+import { TestDriver } from 'testdriverai/vitest/hooks';
+
+test('login test', async (context) => {
+  // Explicit cacheKey provided - auto-generation is skipped
+  const testdriver = TestDriver(context, { 
+    headless: true,
+    cacheKey: 'my-custom-key-v1'
+  });
+  
+  console.log(testdriver.options.cacheKey); // "my-custom-key-v1"
+  
+  const button = await testdriver.find('login button');
+});
+```
+
+## Cache Behavior
+
+### With Auto-Generated Key
+
+1. **First Run**: Creates cache entries with key = hash of test file
+2. **Subsequent Runs** (file unchanged): Cache hits
+3. **After File Modification**: New hash = new cache key = cache miss
+
+### Cache Hit Example
+
+```javascript
+// File: login.test.mjs (hash: 4cae7be040f293b9)
+
+const testdriver = TestDriver(context);
+// Auto-generated cacheKey: "4cae7be040f293b9"
+
+const button1 = await testdriver.find('login button'); // Cache MISS (first time)
+const button2 = await testdriver.find('login button'); // Cache HIT (same file, same test run)
+```
+
+After modifying the file (adding a comment, changing test logic, etc.):
+
+```javascript
+// File: login.test.mjs (hash: 7f3d9a2b1c5e8f6a) <- changed!
+
+const testdriver = TestDriver(context);
+// Auto-generated cacheKey: "7f3d9a2b1c5e8f6a" <- different from before
+
+const button1 = await testdriver.find('login button'); // Cache MISS (new hash)
+```
+
+## Debug Mode
+
+To see the auto-generated cache key in debug logs:
+
+```bash
+# Set environment variable
+export TD_DEBUG=1
+
+# Or in your test
+process.env.TD_DEBUG = '1';
+```
+
+With debug mode enabled, you'll see:
+```
+🔍 find() threshold: 0.05 (cache ENABLED, cacheKey: 4cae7be040f293b9 (auto-generated from file hash))
+```
+
+## Implementation Details
+
+### Stack Trace Filtering
+
+The auto-generation skips the following in the stack trace to find the actual test file:
+- `sdk.js` (TestDriver SDK)
+- `hooks.mjs` / `hooks.js` (Vitest hooks)
+- `node_modules` (dependencies)
+- `node:internal` (Node.js internals)
+
+### File Path Handling
+
+The implementation handles both:
+- Regular file paths: `/Users/you/project/test.mjs`
+- File URL format: `file:///Users/you/project/test.mjs`
+
+### Hash Format
+
+- Algorithm: SHA-256
+- Output: First 16 hexadecimal characters (e.g., `4cae7be040f293b9`)
+- Collision probability: Effectively zero for practical purposes
+
+## When to Use Manual Cache Keys
+
+Consider using manual `cacheKey` values when:
+
+1. **Cross-File Caching**: You want to share cache across multiple test files
+2. **Version-Based Caching**: You want explicit control over cache invalidation
+3. **CI/CD Integration**: You want to tie caching to build numbers or git commits
+
+Example:
+
+```javascript
+const cacheKey = \`test-suite-\${process.env.GIT_COMMIT}\`;
+const testdriver = TestDriver(context, { cacheKey });
+```
+
+## Migration from Manual Keys
+
+If you currently use manual cache keys:
+
+### Before
+```javascript
+const testdriver = TestDriver(context, { 
+  cacheKey: 'login-test-v1' 
+});
+```
+
+### After (automatic)
+```javascript
+// Just remove the cacheKey - it will be auto-generated!
+const testdriver = TestDriver(context);
+```
+
+The cache will automatically invalidate when the test file changes, which is usually the desired behavior.
+
+## See Also
+
+- [SDK_README.md](./SDK_README.md) - Cache configuration options
+- [CACHE_ARCHITECTURE.md](../api/CACHE_ARCHITECTURE.md) - Cache system architecture

package/docs/v7/_drafts/awesome-logs-quick-ref.mdx

@@ -0,0 +1,100 @@
+# AWESOME Logs - Quick Reference 🎨
+
+## At a Glance
+
+Your TestDriver SDK now has **beautiful, emoji-rich logs** with great DX!
+
+### What You'll See
+
+```
+[2.34s] 🔍 Found "submit button" · 📍 (682, 189) · ⏱️ 167ms · ⚡ cached
+[2.51s] 👆 Click "submit button"
+[2.67s] 👉 Hover "menu item"
+[2.89s] ⌨️ Type → hello world
+[3.12s] ✅ Assert "page title correct" · ✓ PASSED · ⏱️ 45ms
+```
+
+## Quick Emoji Guide
+
+| Emoji | Meaning |
+|-------|---------|
+| 🔍 | Finding element |
+| 👆 | Click |
+| 👉 | Hover |
+| ⌨️ | Type |
+| 📜 | Scroll |
+| ✅ | Success/Passed |
+| ❌ | Error/Failed |
+| ⚡ | Cache hit |
+| 💤 | Cache miss |
+| 📍 | Coordinates |
+| ⏱️ | Duration |
+| 🔌 | Connect/Disconnect |
+| 📸 | Screenshot |
+
+## Performance Colors
+
+Duration times are color-coded:
+
+- **🟢 < 100ms** - Fast
+- **🟡 100-500ms** - Acceptable  
+- **🔴 > 500ms** - Slow
+
+## Enable/Disable
+
+```javascript
+// Enabled by default
+const client = new TestDriver(apiKey, {
+  logging: true
+});
+
+// Disable if needed
+client.setLogging(false);
+```
+
+## Debug Mode
+
+For detailed cache information:
+
+```bash
+VERBOSE=true node your-test.js
+```
+
+You'll see:
+- Cache strategy (image/text)
+- Similarity scores
+- Pixel diff percentages
+- Cache age
+
+## Examples
+
+Try these:
+
+```bash
+# Simple example
+TD_API_KEY=your_key node examples/sdk-simple-example.js
+
+# Full demo
+TD_API_KEY=your_key node examples/sdk-awesome-logs-demo.js
+
+# Cache demo with debug info
+TD_API_KEY=your_key VERBOSE=true node examples/sdk-cache-thresholds.js
+```
+
+## Custom Formatting
+
+```javascript
+const { formatter } = require('testdriverai/sdk-log-formatter');
+
+console.log(formatter.formatHeader('My Test', '🧪'));
+console.log(formatter.formatAction('click', 'button'));
+console.log(formatter.formatProgress(3, 10, 'Processing'));
+```
+
+## Full Documentation
+
+See [SDK_AWESOME_LOGS.md](./SDK_AWESOME_LOGS.md) for complete documentation.
+
+---
+
+**Enjoy your AWESOME logs! 🎨✨**