agentic-qe 3.8.1 → 3.8.3

package/.claude/skills/test-design-techniques/SKILL.md

@@ -16,29 +16,16 @@ validation:
   schema_path: schemas/output.json
   validator_path: scripts/validate-config.json
   eval_path: evals/test-design-techniques.yaml
-
 ---
 
 # Test Design Techniques
 
 <default_to_action>
-When designing test cases systematically:
-1. APPLY Boundary Value Analysis (test at min, max, edges)
-2. USE Equivalence Partitioning (one test per partition)
-3. CREATE Decision Tables (for complex business rules)
-4. MODEL State Transitions (for stateful behavior)
-5. REDUCE with Pairwise Testing (for combinations)
-
-**Quick Design Selection:**
+When designing test cases, select technique by input type:
 - Numeric ranges → BVA + EP
 - Multiple conditions → Decision Tables
 - Workflows → State Transition
-- Many parameters → Pairwise Testing
-
-**Critical Success Factors:**
-- Systematic design finds more bugs with fewer tests
-- Random testing is inefficient
-- 40+ years of research backs these techniques
+- Many parameter combinations → Pairwise Testing
 </default_to_action>
 
 ## Quick Reference Card
@@ -49,139 +36,6 @@ When designing test cases systematically:
 - Complex business rules
 - Reducing test redundancy
 
-### Technique Selection Guide
-| Scenario | Technique |
-|----------|-----------|
-| **Numeric input ranges** | BVA + EP |
-| **Multiple conditions** | Decision Tables |
-| **Stateful workflows** | State Transition |
-| **Many parameter combinations** | Pairwise |
-| **All combinations critical** | Full Factorial |
-
----
-
-## Boundary Value Analysis (BVA)
-
-**Principle:** Bugs cluster at boundaries.
-
-**Test at boundaries:**
-- Minimum valid value
-- Just below minimum (invalid)
-- Just above minimum (valid)
-- Maximum valid value
-- Just above maximum (invalid)
-
-```javascript
-// Age field: 18-120 valid
-const boundaryTests = [
-  { input: 17, expected: 'invalid' },  // Below min
-  { input: 18, expected: 'valid' },    // Min boundary
-  { input: 19, expected: 'valid' },    // Above min
-  { input: 119, expected: 'valid' },   // Below max
-  { input: 120, expected: 'valid' },   // Max boundary
-  { input: 121, expected: 'invalid' }  // Above max
-];
-```
-
----
-
-## Equivalence Partitioning (EP)
-
-**Principle:** One test per equivalent class.
-
-```javascript
-// Discount rules:
-// 1-10:  No discount
-// 11-100: 10% discount
-// 101+:   20% discount
-
-const partitionTests = [
-  { quantity: -1, expected: 'invalid' },  // Invalid partition
-  { quantity: 5, expected: 0 },           // Partition 1: 1-10
-  { quantity: 50, expected: 0.10 },       // Partition 2: 11-100
-  { quantity: 200, expected: 0.20 }       // Partition 3: 101+
-];
-
-// 4 tests cover all behavior (vs 200+ if testing every value)
-```
-
----
-
-## Decision Tables
-
-**Use for:** Complex business rules with multiple conditions.
-
-```
-Loan Approval Rules:
-┌──────────────┬───────┬───────┬───────┬───────┬───────┐
-│ Conditions   │ R1    │ R2    │ R3    │ R4    │ R5    │
-├──────────────┼───────┼───────┼───────┼───────┼───────┤
-│ Age ≥ 18     │ Yes   │ Yes   │ Yes   │ No    │ Yes   │
-│ Credit ≥ 700 │ Yes   │ Yes   │ No    │ Yes   │ No    │
-│ Income ≥ 50k │ Yes   │ No    │ Yes   │ Yes   │ Yes   │
-├──────────────┼───────┼───────┼───────┼───────┼───────┤
-│ Result       │Approve│Approve│Reject │Reject │Reject │
-└──────────────┴───────┴───────┴───────┴───────┴───────┘
-
-// 5 tests cover all decision combinations
-```
-
----
-
-## State Transition Testing
-
-**Model state changes:**
-
-```
-States: Logged Out → Logged In → Premium → Suspended
-
-Valid Transitions:
-- Login: Logged Out → Logged In
-- Upgrade: Logged In → Premium
-- Payment Fail: Premium → Suspended
-- Logout: Any → Logged Out
-
-Invalid Transitions to Test:
-- Logged Out → Premium (should reject)
-- Suspended → Premium (should reject)
-```
-
-```javascript
-test('cannot upgrade without login', async () => {
-  const result = await user.upgrade(); // While logged out
-  expect(result.error).toBe('Login required');
-});
-```
-
----
-
-## Pairwise (Combinatorial) Testing
-
-**Problem:** All combinations explode exponentially.
-
-```javascript
-// Parameters:
-// Browser: Chrome, Firefox, Safari (3)
-// OS: Windows, Mac, Linux (3)
-// Screen: Desktop, Tablet, Mobile (3)
-
-// All combinations: 3 × 3 × 3 = 27 tests
-// Pairwise: 9 tests cover all pairs
-
-const pairwiseTests = [
-  { browser: 'Chrome', os: 'Windows', screen: 'Desktop' },
-  { browser: 'Chrome', os: 'Mac', screen: 'Tablet' },
-  { browser: 'Chrome', os: 'Linux', screen: 'Mobile' },
-  { browser: 'Firefox', os: 'Windows', screen: 'Tablet' },
-  { browser: 'Firefox', os: 'Mac', screen: 'Mobile' },
-  { browser: 'Firefox', os: 'Linux', screen: 'Desktop' },
-  { browser: 'Safari', os: 'Windows', screen: 'Mobile' },
-  { browser: 'Safari', os: 'Mac', screen: 'Desktop' },
-  { browser: 'Safari', os: 'Linux', screen: 'Tablet' }
-];
-// Each pair appears at least once
-```
-
 ---
 
 ## Agent-Driven Test Design
@@ -243,8 +97,4 @@ const designFleet = await FleetManager.coordinate({
 
 ## Remember
 
-**Systematic design > Random testing.** 40+ years of research shows these techniques find more bugs with fewer tests than ad-hoc approaches.
-
-**Combine techniques for comprehensive coverage.** BVA for boundaries, EP for partitions, decision tables for rules, pairwise for combinations.
-
 **With Agents:** `qe-test-generator` applies these techniques automatically, generating optimal test suites with maximum coverage and minimum redundancy. Agents identify boundaries, partitions, and combinations from code analysis.

package/.claude/skills/test-environment-management/SKILL.md

@@ -14,7 +14,6 @@ tags: [environment, docker, kubernetes, infrastructure, parity, cost-optimizatio
 trust_tier: 1
 validation:
   schema_path: schemas/output.json
-
 ---
 
 # Test Environment Management
@@ -48,23 +47,6 @@ When managing test environments:
 - Reducing test infrastructure costs
 - Ensuring dev/prod parity
 
-### Environment Types
-| Type | Purpose | Lifetime |
-|------|---------|----------|
-| **Local** | Fast feedback | Developer session |
-| **CI** | Automated tests | Per build (ephemeral) |
-| **Staging** | Pre-prod validation | Persistent |
-| **Production** | Canary/synthetic | Continuous |
-
-### Dev/Prod Parity Checklist
-| Item | Must Match |
-|------|------------|
-| OS | Same version |
-| Database | Same type + version |
-| Dependencies | Same versions |
-| Config | Same structure |
-| Env vars | Same names |
-
 ---
 
 ## Docker for Test Environments
@@ -104,32 +86,6 @@ docker-compose -f docker-compose.test.yml down
 
 ---
 
-## Infrastructure as Code
-
-```hcl
-# test-environment.tf
-resource "aws_instance" "test_server" {
-  ami           = "ami-0c55b159cbfafe1f0"
-  instance_type = "t3.medium"
-
-  tags = {
-    Name         = "test-environment"
-    Environment  = "test"
-    AutoShutdown = "20:00" # Cost optimization
-  }
-}
-
-resource "aws_rds_instance" "test_db" {
-  engine                  = "postgres"
-  engine_version          = "15"
-  instance_class          = "db.t3.micro"
-  backup_retention_period = 0 # No backups needed for test
-  skip_final_snapshot     = true
-}
-```
-
----
-
 ## Service Virtualization
 
 ```javascript
@@ -240,8 +196,4 @@ const envFleet = await FleetManager.coordinate({
 
 ## Remember
 
-**Environment inconsistency = flaky tests.** "Works on my machine" problems come from: different OS/versions, missing dependencies, configuration differences, data differences.
-
-**Infrastructure as Code ensures repeatability.** Version control your environment configurations. Spin up identical environments on demand.
-
 **With Agents:** Agents automatically provision test environments matching production, ensure parity, mock external services, and optimize costs with auto-scaling and auto-shutdown.

package/.claude/skills/test-failure-investigator/SKILL.md

@@ -0,0 +1,99 @@
+---
+name: test-failure-investigator
+description: "Use when a test is failing and you need to determine root cause: is it flaky, an environment issue, or a real regression? Traces failure from symptom to fix."
+user-invocable: true
+---
+
+# Test Failure Investigator
+
+Runbook-style skill for systematic test failure investigation. Given a failing test, determines root cause and recommends action.
+
+## Activation
+
+```
+/test-failure-investigator [test-name-or-file]
+```
+
+## Investigation Flow
+
+### Step 1: Classify the Failure
+
+Run the test 3 times and classify:
+
+| Result Pattern | Classification | Action |
+|---------------|---------------|--------|
+| Fails consistently | **Regression** or **Environment** | Continue to Step 2 |
+| Fails intermittently | **Flaky** | Skip to Step 4 |
+| Passes now | **Transient** | Check CI logs, environment diff |
+
+```bash
+# Run test 3 times
+for i in 1 2 3; do npx jest {{test_file}} 2>&1 | tail -5; echo "--- Run $i ---"; done
+```
+
+### Step 2: Narrow the Scope
+
+```bash
+# When did it start failing?
+git log --oneline -20 -- {{related_source_files}}
+
+# What changed recently?
+git diff HEAD~5 -- {{related_source_files}}
+
+# Does it fail in isolation?
+npx jest {{test_file}} --testNamePattern="{{test_name}}"
+
+# Does it fail with other tests?
+npx jest --runInBand  # sequential execution
+```
+
+### Step 3: Root Cause Analysis
+
+| Symptom | Likely Cause | Investigation |
+|---------|-------------|--------------|
+| Timeout | Network/DB dependency | Check external service availability |
+| Assertion mismatch | Logic change | Compare expected vs actual, check git blame |
+| Import error | Dependency change | Check package.json changes, run `npm ci` |
+| Permission denied | Environment | Check file permissions, Docker volumes |
+| Out of memory | Resource leak | Profile with `--detectOpenHandles` |
+
+### Step 4: Flaky Test Investigation
+
+```bash
+# Run 10 times to confirm flakiness
+for i in $(seq 1 10); do npx jest {{test_file}} --forceExit 2>&1 | grep -E 'PASS|FAIL'; done
+
+# Common flaky causes:
+# - Shared state between tests (missing cleanup)
+# - Time-dependent assertions (use fake timers)
+# - Race conditions (missing await)
+# - Port conflicts (use random ports)
+# - Order dependency (run with --randomize)
+```
+
+### Step 5: Report
+
+```markdown
+## Test Failure Report
+- **Test**: {{test_name}}
+- **File**: {{test_file}}
+- **Classification**: Regression / Flaky / Environment / Transient
+- **Root Cause**: {{description}}
+- **First Failed**: {{commit_hash}} ({{date}})
+- **Fix**: {{recommended_action}}
+- **Verified**: [ ] Fix applied and test passes 3x consecutively
+```
+
+## Composition
+
+After investigation, compose with:
+- **`/bug-reporting-excellence`** — if regression found, file a bug report
+- **`/regression-testing`** — if regression, add to regression suite
+- **`/qe-test-execution`** — for re-running tests after fix
+
+## Gotchas
+
+- Agent may guess at root cause without running the test — always reproduce first
+- "Works on my machine" is not a diagnosis — compare environments (node version, OS, deps)
+- Flaky tests that pass 9/10 times will still be reported as "passing" by CI — run 10+ times
+- Test isolation failures are the #1 cause of flaky tests — check for shared state in beforeAll/afterAll

package/.claude/skills/test-metrics-dashboard/SKILL.md

@@ -0,0 +1,97 @@
+---
+name: test-metrics-dashboard
+description: "Use when querying test history, analyzing flakiness rates, tracking MTTR, or building quality trend dashboards from test execution data."
+user-invocable: true
+---
+
+# Test Metrics Dashboard
+
+Data & Analysis skill for querying test execution history, identifying trends, and surfacing actionable quality metrics.
+
+## Activation
+
+```
+/test-metrics-dashboard
+```
+
+## Key Metrics
+
+### Test Health Metrics
+
+| Metric | Formula | Target | Alert |
+|--------|---------|--------|-------|
+| **Pass Rate** | Passed / Total | > 95% | < 90% |
+| **Flakiness Rate** | Flaky / Total | < 5% | > 10% |
+| **MTTR** | Avg time from failure to fix | < 4 hours | > 24 hours |
+| **Execution Time** | Total suite duration | < 10 min | > 20 min |
+| **Coverage Delta** | Current - Previous | >= 0% | < -2% |
+
+### Data Collection
+
+```bash
+# Export Jest results to JSON
+npx jest --json --outputFile=test-results/$(date +%Y-%m-%d).json
+
+# Parse results for dashboard
+jq '{
+  date: .startTime,
+  total: .numTotalTests,
+  passed: .numPassedTests,
+  failed: .numFailedTests,
+  duration_ms: (.testResults | map(.endTime - .startTime) | add),
+  pass_rate: ((.numPassedTests / .numTotalTests) * 100),
+  flaky: [.testResults[] | select(.numPendingTests > 0)] | length
+}' test-results/$(date +%Y-%m-%d).json
+```
+
+### Trend Analysis
+
+```bash
+# Compare last 5 runs
+for f in $(ls -t test-results/*.json | head -5); do
+  jq --arg file "$f" '{
+    file: $file,
+    pass_rate: ((.numPassedTests / .numTotalTests) * 100 | floor),
+    duration_s: ((.testResults | map(.endTime - .startTime) | add) / 1000 | floor)
+  }' "$f"
+done
+```
+
+### Top Failing Tests
+
+```bash
+# Find most frequently failing tests across runs
+for f in test-results/*.json; do
+  jq -r '.testResults[] | select(.numFailingTests > 0) | .testFilePath' "$f"
+done | sort | uniq -c | sort -rn | head -10
+```
+
+## Run History
+
+Store dashboard data in `${CLAUDE_PLUGIN_DATA}/test-metrics.log`:
+
+```
+2026-03-18|95.2|4.1|312|82.5|3
+```
+
+Format: `date|pass_rate|flakiness_rate|duration_s|coverage_pct|failed_count`
+
+Read history for trend detection:
+```bash
+# Coverage trending down?
+tail -5 "${CLAUDE_PLUGIN_DATA}/test-metrics.log" | awk -F'|' '{print $5}' | sort -n | head -1
+```
+
+## Composition
+
+Feeds into:
+- **`/qe-quality-assessment`** — quality gate decisions based on metrics
+- **`/test-failure-investigator`** — investigate top failing tests
+- **`/coverage-drop-investigator`** — when coverage trends down
+
+## Gotchas
+
+- Metrics without baselines are meaningless — establish baselines before tracking trends
+- Flakiness rate is underreported — a test that fails 1/100 times still breaks CI weekly
+- Duration trends upward over time as test count grows — set alerts on rate of increase, not absolute value
+- Agent may report metrics from a single run as "trends" — need 5+ data points for meaningful trends

package/.claude/skills/test-reporting-analytics/SKILL.md

@@ -16,7 +16,6 @@ validation:
   schema_path: schemas/output.json
   validator_path: scripts/validate-config.json
   eval_path: evals/test-reporting-analytics.yaml
-
 ---
 
 # Test Reporting & Analytics