@validate.qa/runner 1.0.0

package/README.md

@@ -0,0 +1,555 @@
+# @validate.qa/runner
+
+Local test runner for the **[validate.qa](https://validate.qa)** AI testing platform.
+
+This package enables you to execute Playwright and Appium tests generated from your voice narrations directly on your local machine, securely connecting to your validate.qa cloud project.
+
+**Features:**
+- 🎭 **Web Testing** — Execute Playwright tests in headless Chromium
+- 📱 **Mobile Testing** — Execute Appium tests on iOS Simulator, Android Emulator, AND real devices via USB
+- 🔧 **Doctor Command** — Check system prerequisites for web + mobile testing
+- 🔐 **Secure Authentication** — Project-scoped runner tokens
+- 🌐 **Tunnel Support** — Connect HTTPS dashboards to local devices with token-based auth
+
+---
+
+## Quick Start
+
+### 1. Install (Global or NPX)
+
+You don't need to install it globally if you use `npx`, but it's completely supported:
+
+```bash
+npm install -g @validate.qa/runner
+```
+
+### 2. Login
+
+Generate a runner token from your validate.qa **Project Settings → Local Runner**. Your credentials will be saved securely to `~/.validate.qa/credentials.json`.
+
+```bash
+npx @validate.qa/runner login --token urt_xxxxxxxxxxxx --server https://your-server.com
+```
+
+### 3. Start the Runner
+
+Starts the runner daemon, polling for pending test runs from the cloud and executing them locally:
+
+```bash
+npx @validate.qa/runner start
+```
+
+---
+
+## Commands
+
+| Command | Description |
+|---------|-------------|
+| `login` | Saves credentials for future test executions |
+| `start` | Starts the local runner (uses saved credentials or CLI flags) |
+| `status` | Pings the server to verify your runner token and connection |
+| `logout` | Clears all saved runner credentials from your machine |
+| `doctor` | Checks system prerequisites for web and mobile testing |
+
+### Command Options
+
+#### `login`
+```bash
+npx @validate.qa/runner login \
+  --token <token> \        # Runner token (urt_...) from dashboard
+  --server <url> \         # validate.qa server URL
+  --project <id>           # Optional: Project ID (embedded in urt_ tokens)
+```
+
+#### `start`
+```bash
+npx @validate.qa/runner start \
+  --token <token> \        # Override saved token
+  --server <url> \         # Override saved server URL
+  --project <id> \         # Override saved project ID
+  --mobile \               # Enable mobile testing (Appium + bridge)
+  --tunnel \               # Enable ngrok tunnel for HTTPS dashboard access (with token auth)
+  --mode <mode> \          # Execution mode: playwright-native (default)
+  --poll-interval <ms>     # Poll interval: 2000 (default)
+```
+
+#### `status`
+```bash
+npx @validate.qa/runner status
+# Shows: Server URL, Project ID, Token (masked), Connection status
+```
+
+#### `doctor`
+```bash
+npx @validate.qa/runner doctor
+# Checks: Node.js, Playwright, Xcode, Android SDK, Appium
+```
+
+---
+
+## 📱 Mobile Testing
+
+### Overview
+
+The runner supports native mobile app testing on:
+- **iOS:** Simulators (macOS) + Real devices via USB
+- **Android:** Emulators + Real devices via USB
+
+Tests are executed using **Appium 2** with XCUITest (iOS) and UiAutomator2 (Android) drivers.
+
+### How It Works
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│ 1. START RUNNER WITH MOBILE                                 │
+│    npx @validate.qa/runner start --mobile                 │
+│    → Starts Appium server (port 4723)                       │
+│    → Starts mobile bridge (port 9515)                       │
+│    → Discovers available devices                            │
+└─────────────────────────────────────────────────────────────┘
+                           ↓
+┌─────────────────────────────────────────────────────────────┐
+│ 2. RECORD FROM DASHBOARD                                    │
+│    - Open validate.qa dashboard                             │
+│    - Start Recording → Select iOS/Android Native            │
+│    - Interact with mirrored device screen                   │
+│    - Actions captured with screenshots + UI trees           │
+└─────────────────────────────────────────────────────────────┘
+                           ↓
+┌─────────────────────────────────────────────────────────────┐
+│ 3. GENERATE TESTS                                           │
+│    - Stop recording                                         │
+│    - Click "Generate from Recording"                        │
+│    - AI creates MobileTestStep[] array                      │
+│    - Test saved with framework: 'APPIUM'                    │
+└─────────────────────────────────────────────────────────────┘
+                           ↓
+┌─────────────────────────────────────────────────────────────┐
+│ 4. EXECUTE TESTS                                            │
+│    - Click "Run" in dashboard (or queue from API)           │
+│    - Runner claims mobile test run                          │
+│    → Boots simulator/emulator                               │
+│    → Launches app via Appium                                │
+│    → Executes each step (tap, type, swipe, assert)          │
+│    → Captures screenshots                                   │
+│    → Reports results back to server                         │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Prerequisites
+
+#### macOS (Required for iOS)
+- **Xcode** — Install from Mac App Store
+- **Xcode Command Line Tools**: `xcode-select --install`
+- **iOS Simulator** — Included with Xcode
+
+#### Android (Optional)
+- **Android Studio** — https://developer.android.com/studio
+- **Android SDK** — Installed via Android Studio
+- **Environment variables**: `ANDROID_HOME`, `ANDROID_SDK_ROOT`
+- **Emulator** — Create via Android Studio → Device Manager
+
+#### Node.js
+- **Version**: Node 18 or higher
+- **npm**: Latest stable version
+
+### Setup Steps
+
+#### 1. Check Prerequisites
+```bash
+npx @validate.qa/runner doctor
+```
+
+Expected output:
+```
+✓  Node.js v20.x.x
+✓  Playwright installed
+✓  Xcode 15.x (iOS SDK)
+✓  Android SDK (API 34)
+✓  Appium 2.x available
+```
+
+#### 2. Boot a Simulator/Emulator
+
+**iOS:**
+```bash
+# List available simulators
+xcrun simctl list devices
+
+# Boot a specific simulator
+xcrun simctl boot "iPhone 15"
+
+# Or open Simulator.app and boot from menu
+open -a Simulator
+```
+
+**Android:**
+```bash
+# List available emulators
+emulator -list-avds
+
+# Start an emulator
+emulator -avd Pixel_7_API_34
+
+# Or use Android Studio → Device Manager → Start
+```
+
+#### 3. Install Your App
+
+**iOS:**
+```bash
+# Install .app or .ipa on simulator
+xcrun simctl install booted /path/to/YourApp.app
+```
+
+**Android:**
+```bash
+# Install APK on emulator
+adb install /path/to/YourApp.apk
+
+# Or drag-and-drop APK onto emulator window
+```
+
+#### 4. Start Runner with Mobile
+
+**For local dashboard (http://localhost):**
+```bash
+npx @validate.qa/runner start --mobile
+```
+
+**For production dashboard (HTTPS):**
+```bash
+npx @validate.qa/runner start --mobile --tunnel
+```
+
+Expected output (with tunnel):
+```
+🎙️  validate.qa Local Runner
+─────────────────────────────────────
+ℹ  Server:   https://your-server.com
+ℹ  Project:  your-project-id
+ℹ  Mode:     🎭 Playwright Native + 📱 Mobile 🌐 Tunnel
+ℹ  Interval: 2000ms
+✓  Browser:  Chrome with user profile
+ℹ  📱 Mobile: iOS ✓ | Android ✓ | 2 device(s)
+✓  📱 Appium server started
+🌐 Public tunnel started: https://abc123.ngrok.io
+🔐 Tunnel token: a7f3b2c8d9e1f4a5b6c7d8e9f0a1b2c3d4e5f6a7
+   Add this token to your dashboard Project Settings
+
+✓  Runner started — waiting for test runs... (web + mobile)
+```
+
+> **Note:** The tunnel token is automatically sent to your dashboard via the server API. You don't need to manually configure it.
+
+#### 5. Record Your First Mobile Test
+
+1. **Open Dashboard** → Your Project
+2. **Click "Start Recording"** → Select **"iOS Native"** or **"Android Native"**
+3. **Enter App ID**:
+   - iOS: Bundle ID (e.g., `com.example.myapp`)
+   - Android: Package name (e.g., `com.example.myapp`)
+4. **Select Device** from the dropdown (detected automatically)
+5. **Click "Start"** — Device screen mirrors in your browser
+6. **Interact**:
+   - Click on screen → Taps recorded
+   - Use swipe buttons → Swipes recorded
+   - Click "Type Text" → Keyboard input
+   - Back/Home buttons → Navigation
+7. **Click "Stop Recording"** when done
+8. **Generate Tests** → Click "Generate from Recording"
+
+#### 6. Run Mobile Tests
+
+**Note:** The "Run Test" button in the UI is currently disabled for mobile tests (shows "Run Pending"). Tests execute automatically when:
+
+1. You queue a run from the dashboard
+2. The runner is online with `--mobile` flag
+3. A matching device is available
+
+The runner will:
+- Claim the mobile test run from the queue
+- Launch the app on the target device
+- Execute each step via Appium
+- Capture screenshots after each step
+- Report pass/fail results
+
+---
+
+## Runner Modes
+
+### Web Testing (Default)
+```bash
+npx @validate.qa/runner start
+# or explicitly:
+npx @validate.qa/runner start --mode playwright-native
+```
+
+Executes Playwright tests in headless Chromium with:
+- Persistent browser profiles (cookies/logins persist)
+- Test variable injection
+- AI healing support (optional)
+
+### Mobile Testing
+```bash
+npx @validate.qa/runner start --mobile
+```
+
+Enables:
+- Appium 2 server (port 4723)
+- Mobile bridge server (port 9515)
+- Device discovery and capabilities reporting
+- iOS Simulator + Android Emulator support
+
+### Execution Flow Comparison
+
+| Feature | Web Tests | Mobile Tests |
+|---------|-----------|--------------|
+| Framework | Playwright | Appium 2 |
+| Browser/Device | Headless Chromium | iOS Simulator / Android Emulator |
+| Locator Strategy | CSS selectors, XPath | Accessibility ID, XPath, Resource ID |
+| Actions | click, fill, navigate | tap, type, swipe, back, home |
+| Screenshots | On failure | After each step |
+| AI Healing | ✅ Supported | ❌ Not supported |
+| Cloud Runner | ✅ Supported | ❌ Local only |
+
+---
+
+## Configuration
+
+### Saved Credentials
+
+Credentials are stored in `~/.validate.qa/credentials.json`:
+
+```json
+{
+  "token": "urt_xxxxxxxxxxxx",
+  "serverUrl": "https://your-server.com",
+  "projectId": "your-project-id"
+}
+```
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `SERVER_URL` | validate.qa server URL | — |
+| `AUTH_TOKEN` | Runner token | — |
+| `PROJECT_ID` | Project ID | — |
+| `POLL_INTERVAL_MS` | Poll interval | `2000` |
+| `RUNS_DIR` | Directory for test run artifacts | `~/.validate.qa/runs` |
+| `CHROME_USER_DATA_DIR` | Chrome profile path | Auto-detected |
+
+### Example: Custom Configuration
+
+```bash
+# Run with environment variables
+SERVER_URL=https://your-server.com \
+AUTH_TOKEN=urt_xxxx \
+POLL_INTERVAL_MS=5000 \
+npx @validate.qa/runner start --mobile
+```
+
+---
+
+## Troubleshooting
+
+### "No devices found"
+```bash
+# iOS: Boot a simulator
+xcrun simctl boot "iPhone 15"
+
+# Android: Start an emulator
+emulator -avd Pixel_7_API_34
+
+# Then restart runner with --mobile
+```
+
+### "App not installed"
+```bash
+# iOS
+xcrun simctl install booted /path/to/App.app
+
+# Android
+adb install /path/to/App.apk
+```
+
+### "Bridge not responding"
+```bash
+# Check if bridge is running
+curl http://127.0.0.1:9515/bridge/health
+
+# Restart runner with --mobile flag
+```
+
+### "Appium failed to start"
+```bash
+# Check Appium installation
+appium --version
+
+# Reinstall Appium drivers
+appium driver install xcuitest
+appium driver install uiautomator2
+
+# Run doctor to diagnose
+npx @validate.qa/runner doctor
+```
+
+### "Tests stay PENDING"
+```bash
+# 1. Verify runner is online
+npx @validate.qa/runner status
+
+# 2. Check runner is started with --mobile
+npx @validate.qa/runner start --mobile
+
+# 3. Verify device is booted
+xcrun simctl list devices  # iOS
+adb devices                # Android
+```
+
+---
+
+## Security
+
+### Runner Token Security
+
+`urt_` runner tokens are:
+- **Project-scoped:** They cannot access sessions, user accounts, or cross-project data.
+- **Revocable:** Instantly cancel tokens from the dashboard if compromised.
+- **Client-only:** Your local browser/device instances run locally and post test results back.
+- **No code upload:** Tests execute on your machine — source code never leaves your infrastructure.
+
+### Mobile Tunnel Security
+
+When using `--tunnel` for mobile testing from an HTTPS dashboard:
+
+**Security Features:**
+- **Token-based authentication:** Each tunnel session generates a random 64-character hex token
+- **Per-session tokens:** Tokens are regenerated every time you restart the runner
+- **Header validation:** All bridge requests require `X-Bridge-Token` header or `tunnelToken` query param
+- **Health endpoint public:** Only `/bridge/health` is accessible without auth (for monitoring)
+- **All other endpoints protected:** Device control, screenshots, and actions require valid token
+
+**How It Works:**
+```
+1. Runner starts with --tunnel
+   → Generates random token (e.g., a7f3b2c8d9e1f4a5...)
+   → Starts ngrok tunnel to your local bridge
+   → Displays token in console
+
+2. Dashboard fetches tunnel URL + token from server API
+   → Server receives tunnel info via runner heartbeat
+   → Dashboard stores token in memory (never persisted)
+
+3. All bridge requests include token:
+   → HTTP header: X-Bridge-Token: a7f3b2c8d9e1f4a5...
+   → Or query param: ?tunnelToken=a7f3b2c8d9e1f4a5...
+
+4. Bridge validates token on every request:
+   → Invalid/missing token → 401 Unauthorized
+   → Valid token → Request processed
+```
+
+**Best Practices:**
+- ✅ **Safe for production use:** Token is required for all device control operations
+- ✅ **Token never exposed to client:** Dashboard fetches from server API, not console
+- ✅ **Session-limited:** Token changes each time runner restarts
+- ⚠️ **Don't share console output:** The full token is shown in runner console
+- ⚠️ **Use HTTPS dashboard:** Prevents man-in-the-middle attacks on token transmission
+
+**For Enterprise Deployments:**
+Consider using a paid ngrok account for:
+- Fixed domain URLs (no changing URLs)
+- IP whitelisting
+- Request inspection
+- Higher bandwidth limits
+
+---
+
+## Architecture
+
+```
+┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
+│   Dashboard     │────▶│   Server API     │◀────│   Local Runner  │
+│  (validate.qa)  │     │  (your-infra)    │     │  (your-machine) │
+└─────────────────┘     └──────────────────┘     └─────────────────┘
+                                │                        │
+                                │                        ├──→ Playwright (web)
+                                │                        └──→ Appium (mobile)
+                                │                               │
+                                │                               ├──→ iOS Simulator
+                                │                               └──→ Android Emulator
+                                ▼
+                        ┌──────────────────┐
+                        │   PostgreSQL     │
+                        │   (test metadata)│
+                        └──────────────────┘
+```
+
+---
+
+## Deploying
+
+### GitHub Actions (Automated)
+
+Publishing is automated via GitHub Actions. Push a tag to trigger the workflow:
+
+```bash
+# 1. Bump version in packages/runner/package.json
+# 2. Create and push a tag (format: runner-v<version>)
+git tag runner-v0.1.11
+git push origin runner-v0.1.11
+```
+
+The workflow (`.github/workflows/publish-runner.yml`) will:
+1. Install dependencies
+2. Build shared package
+3. Build runner package
+4. Publish to npmjs as `@validate.qa/runner`
+
+### Manual (Local)
+
+If GitHub Actions is unavailable:
+
+```bash
+# 1. Login to npm
+npm login
+
+# 2. Navigate to runner package
+cd packages/runner
+
+# 3. Bump version (patch, minor, or major)
+npm version patch
+
+# 4. Build and publish
+npm run build
+npm publish --access public
+```
+
+### Version
+
+**Current:** 0.1.10
+
+**Changelog:** See [git history](https://github.com/Validate-QA/validate.qa/commits/main/packages/runner) for the full change log. Recent highlights:
+
+- **0.1.10** — Rename package to `@validate.qa/runner`; API test capabilities; discovery pipeline refactor
+- **0.1.9** — Mobile execution improvements; mobile bridge security hardening
+- **0.1.8** — Mobile implementation (Appium 2, device discovery, bridge server)
+- **0.1.7** — Security hardening; audio race condition fix; Voice Insights filter fix
+- **0.1.6** — Pre-deploy bug fixes
+- **0.1.5** — Redesigned local runner UI; publish workflow fixes
+- **0.1.2** — Build workflow fix
+- **0.1.0** — Initial standalone runner package with secure runner tokens
+
+---
+
+## Support
+
+- **Documentation:** [validate.qa docs](https://github.com/Validate-QA/validate.qa/tree/main/docs)
+- **npm Package:** https://www.npmjs.com/package/@validate.qa/runner
+- **GitHub:** https://github.com/Validate-QA/validate.qa
+
+---
+
+*Generated by validate.qa*

package/dist/cli.d.mts

@@ -0,0 +1 @@
+#!/usr/bin/env node

package/dist/cli.d.ts

@@ -0,0 +1 @@
+#!/usr/bin/env node