@windagency/valora 2.0.1

package/data/prompts/06_test/execute-tests.md

@@ -0,0 +1,534 @@
+---
+id: test.execute-tests
+version: 1.0.0
+category: test
+experimental: true
+name: Execute Tests
+description: Run test suites based on scope and type, collect results and coverage metrics
+tags:
+  - testing
+  - test-execution
+  - coverage-analysis
+model_requirements:
+  min_context: 128000
+  recommended:
+    - claude-haiku-4.5
+    - claude-sonnet-4.5
+agents:
+  - qa
+dependencies:
+  requires:
+    - context.use-modern-cli-tools
+    - test.analyze-test-infrastructure
+inputs:
+  - name: test_scope
+    description: Scope of tests to execute
+    type: string
+    required: false
+    default: all
+  - name: test_type
+    description: Type of tests to run
+    type: string
+    required: false
+    default: all
+    validation:
+      enum: ['unit', 'integration', 'e2e', 'all']
+  - name: coverage_threshold
+    description: Minimum coverage percentage required
+    type: number
+    required: false
+    default: 80
+    validation:
+      min: 0
+      max: 100
+  - name: test_framework
+    description: Test framework info from analyze-test-infrastructure
+    type: object
+    required: true
+  - name: test_structure
+    description: Test structure from analyze-test-infrastructure
+    type: object
+    required: true
+outputs:
+  - test_results
+  - coverage_metrics
+  - failed_tests
+tokens:
+  avg: 5000
+  max: 12000
+  min: 2500
+---
+
+# Execute Tests
+
+## Objective
+
+Run test suites in the appropriate order, capture comprehensive results including failures and stack traces, generate coverage reports, and determine pass/fail status against thresholds.
+
+## CLI Tools for Output Processing
+
+When processing terminal output from test commands, use modern CLI tools:
+
+- **`jq`** to parse JSON test results and coverage reports (e.g., `--reporter=json` output). Extract specific fields instead of dumping entire JSON.
+- **`rg`** to search test output for failure patterns, stack traces, or specific error messages.
+- **`fd`** to locate test files, coverage reports, or configuration files by extension or name.
+
+```bash
+# Parse test results JSON for failures only
+jq '[.testResults[] | select(.numFailedTests > 0) | {file: .testFilePath, failed: .numFailedTests}]' test-results.json
+
+# Parse coverage summary for threshold check
+jq '.total | {lines: .lines.pct, branches: .branches.pct, functions: .functions.pct}' coverage/coverage-summary.json
+
+# Find all test result files
+fd -g 'test-results*.json' --type f
+
+# Search test output for specific error patterns
+rg "FAIL|Error:|TypeError:" test-output.log
+```
+
+## Instructions
+
+### Step 1: Prepare Test Environment
+
+**Setup environment variables**:
+
+```bash
+export NODE_ENV=test
+export TEST_DATABASE_URL=postgresql://localhost:5432/test_db
+# Additional env vars from test_framework.environment_requirements
+```
+
+**Setup test infrastructure** (if required):
+
+```bash
+# Start test database
+npm run db:test:setup
+
+# Start Docker containers (if needed)
+docker-compose -f docker-compose.test.yml up -d
+
+# Run migrations
+npm run migrate:test
+```
+
+### Step 2: Determine Test Execution Order
+
+**Based on test_type parameter**:
+
+- **test_type = "unit"**: Run only unit tests
+- **test_type = "integration"**: Run only integration tests
+- **test_type = "e2e"**: Run only e2e tests
+- **test_type = "all"**: Run in order: unit → integration → e2e
+
+**Execution strategy**:
+
+```json
+{
+	"execution_plan": [
+		{
+			"type": "unit",
+			"command": "npm run test:unit -- --coverage",
+			"timeout": "5m",
+			"parallel": true
+		},
+		{
+			"type": "integration",
+			"command": "npm run test:integration",
+			"timeout": "10m",
+			"parallel": false,
+			"depends_on": ["unit"]
+		},
+		{
+			"type": "e2e",
+			"command": "npm run test:e2e",
+			"timeout": "15m",
+			"parallel": false,
+			"depends_on": ["integration"]
+		}
+	]
+}
+```
+
+### Step 3: Execute Unit Tests
+
+**Run unit test suite**:
+
+```bash
+# With coverage
+npm run test:unit -- --coverage --reporter=json --outputFile=test-results-unit.json
+
+# Apply scope if provided
+npm run test:unit -- src/auth --coverage
+```
+
+**Capture output**:
+
+- Test results (pass/fail counts)
+- Execution time
+- Coverage data (statements, branches, functions, lines)
+- Failed test details
+- Stack traces
+
+**Example output**:
+
+```json
+{
+	"unit_tests": {
+		"total": 45,
+		"passed": 43,
+		"failed": 2,
+		"skipped": 0,
+		"duration": "2.34s",
+		"suites": [
+			{
+				"name": "AuthService",
+				"file": "src/auth/service.test.ts",
+				"tests": 12,
+				"passed": 12,
+				"failed": 0,
+				"duration": "0.45s"
+			},
+			{
+				"name": "EmailService",
+				"file": "src/email/service.test.ts",
+				"tests": 8,
+				"passed": 6,
+				"failed": 2,
+				"duration": "0.67s"
+			}
+		],
+		"failures": [
+			{
+				"test": "should send verification email",
+				"file": "src/email/service.test.ts",
+				"line": 45,
+				"error": "Expected email to be sent but emailService.send was not called",
+				"stack": "Error: Expected email...\n    at Object.<anonymous> (src/email/service.test.ts:45:5)"
+			}
+		],
+		"coverage": {
+			"statements": { "covered": 342, "total": 380, "pct": 90.0 },
+			"branches": { "covered": 78, "total": 95, "pct": 82.1 },
+			"functions": { "covered": 54, "total": 58, "pct": 93.1 },
+			"lines": { "covered": 334, "total": 372, "pct": 89.8 }
+		}
+	}
+}
+```
+
+### Step 4: Execute Integration Tests
+
+**Run integration test suite**:
+
+```bash
+npm run test:integration -- --reporter=json --outputFile=test-results-integration.json
+```
+
+**Capture output** (similar structure to unit tests):
+
+```json
+{
+	"integration_tests": {
+		"total": 12,
+		"passed": 12,
+		"failed": 0,
+		"skipped": 0,
+		"duration": "5.67s",
+		"suites": [
+			{
+				"name": "Auth Flow",
+				"file": "tests/integration/auth-flow.test.ts",
+				"tests": 8,
+				"passed": 8,
+				"failed": 0,
+				"duration": "3.21s"
+			}
+		],
+		"failures": []
+	}
+}
+```
+
+### Step 5: Execute E2E Tests
+
+**Run e2e test suite** (if test_type includes e2e):
+
+```bash
+npm run test:e2e -- --reporter=json --outputFile=test-results-e2e.json
+```
+
+**Capture screenshots on failure** (if supported):
+
+```json
+{
+	"e2e_tests": {
+		"total": 8,
+		"passed": 7,
+		"failed": 1,
+		"skipped": 0,
+		"duration": "45.23s",
+		"suites": [
+			{
+				"name": "User Registration",
+				"file": "tests/e2e/registration.test.ts",
+				"tests": 4,
+				"passed": 3,
+				"failed": 1,
+				"duration": "22.15s"
+			}
+		],
+		"failures": [
+			{
+				"test": "should complete registration flow",
+				"file": "tests/e2e/registration.test.ts",
+				"line": 23,
+				"error": "Timeout: Element 'Submit' not found",
+				"screenshot": "tests/e2e/screenshots/registration-failure.png"
+			}
+		]
+	}
+}
+```
+
+### Step 6: Aggregate Coverage Metrics
+
+**Use `jq` to merge coverage from JSON reports:**
+
+```bash
+# Extract overall coverage from the report
+jq '.total | {statements: .statements.pct, branches: .branches.pct, functions: .functions.pct, lines: .lines.pct}' coverage/coverage-summary.json
+
+# Find files below threshold
+jq --arg threshold 80 '[to_entries[] | select(.key != "total") | select(.value.lines.pct < ($threshold | tonumber)) | {file: .key, coverage: .value.lines.pct}]' coverage/coverage-summary.json
+```
+
+**Merge coverage from all test types**:
+
+```json
+{
+	"coverage": {
+		"overall": {
+			"statements": { "pct": 92.5 },
+			"branches": { "pct": 88.3 },
+			"functions": { "pct": 95.2 },
+			"lines": { "pct": 91.8 }
+		},
+		"by_file": [
+			{
+				"file": "src/auth/service.ts",
+				"statements": 95.2,
+				"branches": 91.5,
+				"functions": 100.0,
+				"lines": 94.8,
+				"uncovered_lines": [47, 48]
+			},
+			{
+				"file": "src/email/service.ts",
+				"statements": 78.5,
+				"branches": 72.1,
+				"functions": 87.5,
+				"lines": 77.9,
+				"uncovered_lines": [23, 24, 89, 90, 91]
+			}
+		],
+		"threshold": 80,
+		"threshold_met": true
+	}
+}
+```
+
+### Step 7: Identify Coverage Gaps
+
+**Find files below threshold**:
+
+```json
+{
+	"files_below_threshold": [
+		{
+			"file": "src/email/service.ts",
+			"coverage": 78.5,
+			"threshold": 80,
+			"gap": -1.5,
+			"uncovered_critical_paths": [
+				{
+					"function": "handleEmailError",
+					"lines": [89, 90, 91],
+					"reason": "Error recovery path not tested"
+				}
+			]
+		}
+	]
+}
+```
+
+### Step 8: Detect Flaky Tests
+
+**Identify intermittent failures** (if re-running):
+
+- Tests that pass on retry
+- Tests with timing dependencies
+- Tests with race conditions
+
+```json
+{
+	"flaky_tests": [
+		{
+			"test": "should update user profile",
+			"file": "tests/integration/profile.test.ts",
+			"pass_rate": "3/5",
+			"reason": "Timing-dependent assertion"
+		}
+	]
+}
+```
+
+### Step 9: Handle Test Failures
+
+**Continue execution strategy**:
+
+- For unit test failures: Continue to integration tests
+- For integration test failures: Skip e2e tests
+- Preserve all failure details
+
+**Collect error context**:
+
+- Full stack traces
+- Environment state
+- Test data used
+- Related tests
+
+### Step 10: Generate Test Summary
+
+**Aggregate all results**:
+
+```json
+{
+	"summary": {
+		"total_tests": 65,
+		"passed": 62,
+		"failed": 3,
+		"skipped": 0,
+		"duration": "53.24s",
+		"pass_rate": "95.4%",
+		"coverage": {
+			"overall": 92.5,
+			"threshold": 80,
+			"threshold_met": true
+		},
+		"status": "FAIL",
+		"reason": "3 test failures"
+	}
+}
+```
+
+## Output Format
+
+```json
+{
+	"test_results": {
+		"summary": {
+			"total_tests": 65,
+			"passed": 62,
+			"failed": 3,
+			"skipped": 0,
+			"duration": "53.24s",
+			"pass_rate": "95.4%"
+		},
+		"unit_tests": {
+			"total": 45,
+			"passed": 43,
+			"failed": 2,
+			"duration": "2.34s",
+			"failures": [
+				{
+					"test": "should send verification email",
+					"file": "src/email/service.test.ts",
+					"line": 45,
+					"error": "Expected email to be sent but emailService.send was not called",
+					"stack": "Error: Expected email...\n    at Object.<anonymous> (src/email/service.test.ts:45:5)"
+				}
+			]
+		},
+		"integration_tests": {
+			"total": 12,
+			"passed": 12,
+			"failed": 0,
+			"duration": "5.67s"
+		},
+		"e2e_tests": {
+			"total": 8,
+			"passed": 7,
+			"failed": 1,
+			"duration": "45.23s",
+			"failures": [
+				{
+					"test": "should complete registration flow",
+					"file": "tests/e2e/registration.test.ts",
+					"screenshot": "tests/e2e/screenshots/registration-failure.png"
+				}
+			]
+		}
+	},
+	"coverage_metrics": {
+		"overall": {
+			"statements": 92.5,
+			"branches": 88.3,
+			"functions": 95.2,
+			"lines": 91.8
+		},
+		"threshold": 80,
+		"threshold_met": true,
+		"files_below_threshold": [
+			{
+				"file": "src/email/service.ts",
+				"coverage": 78.5,
+				"uncovered_lines": [23, 24, 89, 90, 91]
+			}
+		]
+	},
+	"failed_tests": [
+		{
+			"type": "unit",
+			"test": "should send verification email",
+			"file": "src/email/service.test.ts",
+			"error": "Expected email to be sent but emailService.send was not called"
+		},
+		{
+			"type": "e2e",
+			"test": "should complete registration flow",
+			"file": "tests/e2e/registration.test.ts",
+			"error": "Timeout: Element 'Submit' not found"
+		}
+	]
+}
+```
+
+## Success Criteria
+
+- ✅ All test suites executed in correct order
+- ✅ Test results captured (pass/fail/skip counts)
+- ✅ Execution time recorded
+- ✅ Failure details with stack traces collected
+- ✅ Coverage metrics generated
+- ✅ Coverage threshold evaluated
+- ✅ Flaky tests detected (if applicable)
+
+## Rules
+
+**DO**:
+
+- ✅ Run tests in appropriate order (unit → integration → e2e)
+- ✅ Capture full stack traces for failures
+- ✅ Generate coverage reports
+- ✅ Continue execution even if tests fail (to gather all results)
+- ✅ Apply scope filters correctly
+- ✅ Record timing information
+
+**DON'T**:
+
+- ❌ Don't stop at first failure
+- ❌ Don't skip coverage generation
+- ❌ Don't ignore flaky tests
+- ❌ Don't run tests in parallel if they have dependencies
+- ❌ Don't proceed if test framework fails to start