@cregis-dev/cckit 0.2.0 → 0.4.0

package/templates/bmad/tea/workflows/testarch/test-review/steps-c/step-03-quality-evaluation.md

@@ -1,14 +1,14 @@
 ---
 name: 'step-03-quality-evaluation'
-description: 'Orchestrate parallel quality dimension checks (4 subprocesses)'
+description: 'Orchestrate adaptive quality dimension checks (agent-team, subagent, or sequential)'
 nextStepFile: './step-03f-aggregate-scores.md'
 ---
 
-# Step 3: Orchestrate Parallel Quality Evaluation
+# Step 3: Orchestrate Adaptive Quality Evaluation
 
 ## STEP GOAL
 
-Launch 4 parallel subprocesses to evaluate test quality dimensions:
+Select execution mode deterministically, then evaluate quality dimensions using agent-team, subagent, or sequential execution while preserving output contracts:
 
 - Determinism
 - Isolation
@@ -21,30 +21,31 @@ Coverage is intentionally excluded from this workflow and handled by `trace`.
 
 - 📖 Read the entire step file before acting
 - ✅ Speak in `{communication_language}`
-- ✅ Launch four subprocesses in PARALLEL
-- ✅ Wait for all subprocesses to complete
-- ❌ Do NOT evaluate quality sequentially (use subprocesses)
-- ❌ Do NOT proceed until all subprocesses finish
+- ✅ Resolve execution mode from config (`tea_execution_mode`, `tea_capability_probe`)
+- ✅ Apply fallback rules deterministically when requested mode is unsupported
+- ✅ Wait for required worker steps to complete
+- ❌ Do NOT skip capability checks when probing is enabled
+- ❌ Do NOT proceed until required worker steps finish
 
 ---
 
 ## EXECUTION PROTOCOLS:
 
 - 🎯 Follow the MANDATORY SEQUENCE exactly
-- 💾 Wait for subprocess outputs
+- 💾 Wait for subagent outputs
 - 📖 Load the next step only when instructed
 
 ## CONTEXT BOUNDARIES:
 
 - Available context: test files from Step 2, knowledge fragments
-- Focus: subprocess orchestration only
-- Limits: do not evaluate quality directly (delegate to subprocesses)
+- Focus: orchestration only (mode selection + worker dispatch)
+- Limits: do not evaluate quality directly (delegate to worker steps)
 
 ---
 
 ## MANDATORY SEQUENCE
 
-### 1. Prepare Subprocess Inputs
+### 1. Prepare Execution Context
 
 **Generate unique timestamp:**
 
@@ -52,56 +53,156 @@ Coverage is intentionally excluded from this workflow and handled by `trace`.
 const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
 ```
 
-**Prepare context for all subprocesses:**
+**Prepare context for all subagents:**
 
 ```javascript
-const subprocessContext = {
+const parseBooleanFlag = (value, defaultValue = true) => {
+  if (typeof value === 'string') {
+    const normalized = value.trim().toLowerCase();
+    if (['false', '0', 'off', 'no'].includes(normalized)) return false;
+    if (['true', '1', 'on', 'yes'].includes(normalized)) return true;
+  }
+  if (value === undefined || value === null) return defaultValue;
+  return Boolean(value);
+};
+
+const subagentContext = {
   test_files: /* from Step 2 */,
   knowledge_fragments_loaded: ['test-quality'],
+  config: {
+    execution_mode: config.tea_execution_mode || 'auto',  // "auto" | "subagent" | "agent-team" | "sequential"
+    capability_probe: parseBooleanFlag(config.tea_capability_probe, true),  // supports booleans and "false"/"true" strings
+  },
   timestamp: timestamp
 };
 ```
 
 ---
 
-### 2. Launch 4 Parallel Quality Subprocesses
+### 2. Resolve Execution Mode with Capability Probe
+
+```javascript
+const normalizeUserExecutionMode = (mode) => {
+  if (typeof mode !== 'string') return null;
+  const normalized = mode.trim().toLowerCase().replace(/[-_]/g, ' ').replace(/\s+/g, ' ');
+
+  if (normalized === 'auto') return 'auto';
+  if (normalized === 'sequential') return 'sequential';
+  if (normalized === 'subagent' || normalized === 'sub agent' || normalized === 'subagents' || normalized === 'sub agents') {
+    return 'subagent';
+  }
+  if (normalized === 'agent team' || normalized === 'agent teams' || normalized === 'agentteam') {
+    return 'agent-team';
+  }
+
+  return null;
+};
+
+const normalizeConfigExecutionMode = (mode) => {
+  if (mode === 'subagent') return 'subagent';
+  if (mode === 'auto' || mode === 'sequential' || mode === 'subagent' || mode === 'agent-team') {
+    return mode;
+  }
+  return null;
+};
 
-**Subprocess A: Determinism**
+// Explicit user instruction in the active run takes priority over config.
+const explicitModeFromUser = normalizeUserExecutionMode(runtime.getExplicitExecutionModeHint?.() || null);
 
-- File: `./step-03a-subprocess-determinism.md`
+const requestedMode = explicitModeFromUser || normalizeConfigExecutionMode(subagentContext.config.execution_mode) || 'auto';
+const probeEnabled = subagentContext.config.capability_probe;
+
+const supports = {
+  subagent: false,
+  agentTeam: false,
+};
+
+if (probeEnabled) {
+  supports.subagent = runtime.canLaunchSubagents?.() === true;
+  supports.agentTeam = runtime.canLaunchAgentTeams?.() === true;
+}
+
+let resolvedMode = requestedMode;
+
+if (requestedMode === 'auto') {
+  if (supports.agentTeam) resolvedMode = 'agent-team';
+  else if (supports.subagent) resolvedMode = 'subagent';
+  else resolvedMode = 'sequential';
+} else if (probeEnabled && requestedMode === 'agent-team' && !supports.agentTeam) {
+  resolvedMode = supports.subagent ? 'subagent' : 'sequential';
+} else if (probeEnabled && requestedMode === 'subagent' && !supports.subagent) {
+  resolvedMode = 'sequential';
+}
+
+subagentContext.execution = {
+  requestedMode,
+  resolvedMode,
+  probeEnabled,
+  supports,
+};
+```
+
+Resolution precedence:
+
+1. Explicit user request in this run (`agent team` => `agent-team`; `subagent` => `subagent`; `sequential`; `auto`)
+2. `tea_execution_mode` from config
+3. Runtime capability fallback (when probing enabled)
+
+If probing is disabled, honor the requested mode strictly. If that mode cannot be executed at runtime, fail with explicit error instead of silent fallback.
+
+---
+
+### 3. Dispatch 4 Quality Workers
+
+**Subagent A: Determinism**
+
+- File: `./step-03a-subagent-determinism.md`
 - Output: `/tmp/tea-test-review-determinism-${timestamp}.json`
-- Status: Running in parallel... ⟳
+- Execution:
+  - `agent-team` or `subagent`: launch non-blocking
+  - `sequential`: run blocking and wait
+- Status: Running... ⟳
 
-**Subprocess B: Isolation**
+**Subagent B: Isolation**
 
-- File: `./step-03b-subprocess-isolation.md`
+- File: `./step-03b-subagent-isolation.md`
 - Output: `/tmp/tea-test-review-isolation-${timestamp}.json`
-- Status: Running in parallel... ⟳
+- Status: Running... ⟳
 
-**Subprocess C: Maintainability**
+**Subagent C: Maintainability**
 
-- File: `./step-03c-subprocess-maintainability.md`
+- File: `./step-03c-subagent-maintainability.md`
 - Output: `/tmp/tea-test-review-maintainability-${timestamp}.json`
-- Status: Running in parallel... ⟳
+- Status: Running... ⟳
 
-**Subprocess D: Performance**
+**Subagent D: Performance**
 
-- File: `./step-03e-subprocess-performance.md`
+- File: `./step-03e-subagent-performance.md`
 - Output: `/tmp/tea-test-review-performance-${timestamp}.json`
-- Status: Running in parallel... ⟳
+- Status: Running... ⟳
+
+In `agent-team` and `subagent` modes, runtime decides worker scheduling and concurrency.
 
 ---
 
-### 3. Wait for All Subprocesses
+### 4. Wait for Expected Worker Completion
+
+**If `resolvedMode` is `agent-team` or `subagent`:**
+
+```
+⏳ Waiting for 4 quality subagents to complete...
+✅ All 4 quality subagents completed successfully!
+```
+
+**If `resolvedMode` is `sequential`:**
 
 ```
-⏳ Waiting for 4 quality subprocesses to complete...
-✅ All 4 quality subprocesses completed successfully!
+✅ Sequential mode: each worker already completed during dispatch.
 ```
 
 ---
 
-### 4. Verify All Outputs Exist
+### 5. Verify All Outputs Exist
 
 ```javascript
 const outputs = ['determinism', 'isolation', 'maintainability', 'performance'].map(
@@ -110,26 +211,25 @@ const outputs = ['determinism', 'isolation', 'maintainability', 'performance'].m
 
 outputs.forEach((output) => {
   if (!fs.existsSync(output)) {
-    throw new Error(`Subprocess output missing: ${output}`);
+    throw new Error(`Subagent output missing: ${output}`);
   }
 });
 ```
 
 ---
 
-### 5. Performance Report
+### 6. Execution Report
 
 ```
 🚀 Performance Report:
-- Execution Mode: PARALLEL (4 subprocesses)
-- Total Elapsed: ~max(all subprocesses) minutes
-- Sequential Would Take: ~sum(all subprocesses) minutes
-- Performance Gain: ~60-70% faster!
+- Execution Mode: {resolvedMode}
+- Total Elapsed: ~mode-dependent
+- Parallel Gain: ~60-70% faster when mode is subagent/agent-team
 ```
 
 ---
 
-### 6. Proceed to Aggregation
+### 7. Proceed to Aggregation
 
 Pass the same `timestamp` value to Step 3F (do not regenerate it). Step 3F must read the exact temp files written in this step.
 
@@ -137,7 +237,7 @@ Load next step: `{nextStepFile}`
 
 The aggregation step (3F) will:
 
-- Read all 4 subprocess outputs
+- Read all 4 subagent outputs
 - Calculate weighted overall score (0-100)
 - Aggregate violations by severity
 - Generate review report with top suggestions
@@ -148,11 +248,11 @@ The aggregation step (3F) will:
 
 Proceed to Step 3F when:
 
-- ✅ All 4 subprocesses completed successfully
+- ✅ All 4 subagents completed successfully
 - ✅ All output files exist and are valid JSON
-- ✅ Performance metrics displayed
+- ✅ Execution metrics displayed
 
-**Do NOT proceed if any subprocess failed.**
+**Do NOT proceed if any subagent failed.**
 
 ---
 
@@ -160,14 +260,15 @@ Proceed to Step 3F when:
 
 ### ✅ SUCCESS:
 
-- All 4 subprocesses launched and completed
+- All 4 subagents launched and completed
+- All required worker steps completed
 - Output files generated and valid
-- Parallel execution achieved ~60% performance gain
+- Fallback behavior respected configuration and capability probe rules
 
 ### ❌ FAILURE:
 
-- One or more subprocesses failed
+- One or more subagents failed
 - Output files missing or invalid
-- Sequential evaluation instead of parallel
+- Unsupported requested mode with probing disabled
 
-**Master Rule:** Parallel subprocess execution is MANDATORY for performance.
+**Master Rule:** Deterministic mode selection + stable output contract. Use the best supported mode, then aggregate normally.

package/templates/bmad/tea/workflows/testarch/test-review/steps-c/step-03a-subagent-determinism.md

@@ -0,0 +1,214 @@
+---
+name: 'step-03a-subagent-determinism'
+description: 'Subagent: Check test determinism (no random/time dependencies)'
+subagent: true
+outputFile: '/tmp/tea-test-review-determinism-{{timestamp}}.json'
+---
+
+# Subagent 3A: Determinism Quality Check
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other quality dimension checks.
+
+**What you have from parent workflow:**
+
+- Test files discovered in Step 2
+- Knowledge fragment: test-quality (determinism criteria)
+- Config: test framework
+
+**Your task:** Analyze test files for DETERMINISM violations only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- 📖 Read this entire subagent file before acting
+- ✅ Check DETERMINISM only (not other quality dimensions)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT check isolation, maintainability, coverage, or performance (other subagents)
+- ❌ Do NOT modify test files (read-only analysis)
+- ❌ Do NOT run tests (just analyze code)
+
+---
+
+## SUBAGENT TASK
+
+### 1. Identify Determinism Violations
+
+**Scan test files for non-deterministic patterns:**
+
+**HIGH SEVERITY Violations**:
+
+- `Math.random()` - Random number generation
+- `Date.now()` or `new Date()` without mocking
+- `setTimeout` / `setInterval` without proper waits
+- External API calls without mocking
+- File system operations on random paths
+- Database queries with non-deterministic ordering
+
+**MEDIUM SEVERITY Violations**:
+
+- `page.waitForTimeout(N)` - Hard waits instead of conditions
+- Flaky selectors (CSS classes that may change)
+- Race conditions (missing proper synchronization)
+- Test order dependencies (test A must run before test B)
+
+**LOW SEVERITY Violations**:
+
+- Missing test isolation (shared state between tests)
+- Console timestamps without fixed timezone
+
+### 2. Analyze Each Test File
+
+For each test file from Step 2:
+
+```javascript
+const violations = [];
+
+// Check for Math.random()
+if (testFileContent.includes('Math.random()')) {
+  violations.push({
+    file: testFile,
+    line: findLineNumber('Math.random()'),
+    severity: 'HIGH',
+    category: 'random-generation',
+    description: 'Test uses Math.random() - non-deterministic',
+    suggestion: 'Use faker.seed(12345) for deterministic random data',
+  });
+}
+
+// Check for Date.now()
+if (testFileContent.includes('Date.now()') || testFileContent.includes('new Date()')) {
+  violations.push({
+    file: testFile,
+    line: findLineNumber('Date.now()'),
+    severity: 'HIGH',
+    category: 'time-dependency',
+    description: 'Test uses Date.now() or new Date() without mocking',
+    suggestion: 'Mock system time with test.useFakeTimers() or use fixed timestamps',
+  });
+}
+
+// Check for hard waits
+if (testFileContent.includes('waitForTimeout')) {
+  violations.push({
+    file: testFile,
+    line: findLineNumber('waitForTimeout'),
+    severity: 'MEDIUM',
+    category: 'hard-wait',
+    description: 'Test uses waitForTimeout - creates flakiness',
+    suggestion: 'Replace with expect(locator).toBeVisible() or waitForResponse',
+  });
+}
+
+// ... check other patterns
+```
+
+### 3. Calculate Determinism Score
+
+**Scoring Logic**:
+
+```javascript
+const totalChecks = testFiles.length * checksPerFile;
+const failedChecks = violations.length;
+const passedChecks = totalChecks - failedChecks;
+
+// Weight violations by severity
+const severityWeights = { HIGH: 10, MEDIUM: 5, LOW: 2 };
+const totalPenalty = violations.reduce((sum, v) => sum + severityWeights[v.severity], 0);
+
+// Score: 100 - (penalty points)
+const score = Math.max(0, 100 - totalPenalty);
+```
+
+---
+
+## OUTPUT FORMAT
+
+Write JSON to temp file: `/tmp/tea-test-review-determinism-{{timestamp}}.json`
+
+```json
+{
+  "dimension": "determinism",
+  "score": 85,
+  "max_score": 100,
+  "grade": "B",
+  "violations": [
+    {
+      "file": "tests/api/user.spec.ts",
+      "line": 42,
+      "severity": "HIGH",
+      "category": "random-generation",
+      "description": "Test uses Math.random() - non-deterministic",
+      "suggestion": "Use faker.seed(12345) for deterministic random data",
+      "code_snippet": "const userId = Math.random() * 1000;"
+    },
+    {
+      "file": "tests/e2e/checkout.spec.ts",
+      "line": 78,
+      "severity": "MEDIUM",
+      "category": "hard-wait",
+      "description": "Test uses waitForTimeout - creates flakiness",
+      "suggestion": "Replace with expect(locator).toBeVisible()",
+      "code_snippet": "await page.waitForTimeout(5000);"
+    }
+  ],
+  "passed_checks": 12,
+  "failed_checks": 3,
+  "total_checks": 15,
+  "violation_summary": {
+    "HIGH": 1,
+    "MEDIUM": 1,
+    "LOW": 1
+  },
+  "recommendations": [
+    "Use faker with fixed seed for all random data",
+    "Replace all waitForTimeout with conditional waits",
+    "Mock Date.now() in tests that use current time"
+  ],
+  "summary": "Tests are mostly deterministic with 3 violations (1 HIGH, 1 MEDIUM, 1 LOW)"
+}
+```
+
+**On Error:**
+
+```json
+{
+  "dimension": "determinism",
+  "success": false,
+  "error": "Error message describing what went wrong"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when:
+
+- ✅ All test files analyzed for determinism violations
+- ✅ Score calculated (0-100)
+- ✅ Violations categorized by severity
+- ✅ Recommendations generated
+- ✅ JSON output written to temp file
+
+**Subagent terminates here.** Parent workflow will read output and aggregate with other quality dimensions.
+
+---
+
+## 🚨 SUBAGENT SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- All test files scanned for determinism violations
+- Score calculated with proper severity weighting
+- JSON output valid and complete
+- Only determinism checked (not other dimensions)
+
+### ❌ FAILURE:
+
+- Checked quality dimensions other than determinism
+- Invalid or missing JSON output
+- Score calculation incorrect
+- Modified test files (should be read-only)

package/templates/bmad/tea/workflows/testarch/test-review/steps-c/step-03b-subagent-isolation.md

@@ -0,0 +1,125 @@
+---
+name: 'step-03b-subagent-isolation'
+description: 'Subagent: Check test isolation (no shared state/dependencies)'
+subagent: true
+outputFile: '/tmp/tea-test-review-isolation-{{timestamp}}.json'
+---
+
+# Subagent 3B: Isolation Quality Check
+
+## SUBAGENT CONTEXT
+
+This is an **isolated subagent** running in parallel with other quality dimension checks.
+
+**Your task:** Analyze test files for ISOLATION violations only.
+
+---
+
+## MANDATORY EXECUTION RULES
+
+- ✅ Check ISOLATION only (not other quality dimensions)
+- ✅ Output structured JSON to temp file
+- ❌ Do NOT check determinism, maintainability, coverage, or performance
+- ❌ Do NOT modify test files (read-only analysis)
+
+---
+
+## SUBAGENT TASK
+
+### 1. Identify Isolation Violations
+
+**Scan test files for isolation issues:**
+
+**HIGH SEVERITY Violations**:
+
+- Global state mutations (global variables modified)
+- Test order dependencies (test B depends on test A running first)
+- Shared database records without cleanup
+- beforeAll/afterAll with side effects leaking to other tests
+
+**MEDIUM SEVERITY Violations**:
+
+- Missing test cleanup (created data not deleted)
+- Shared fixtures that mutate state
+- Tests that assume specific execution order
+- Environment variables modified without restoration
+
+**LOW SEVERITY Violations**:
+
+- Tests sharing test data (but not mutating)
+- Missing test.describe grouping
+- Tests that could be more isolated
+
+### 2. Calculate Isolation Score
+
+```javascript
+const totalChecks = testFiles.length * checksPerFile;
+const failedChecks = violations.length;
+const severityWeights = { HIGH: 10, MEDIUM: 5, LOW: 2 };
+const totalPenalty = violations.reduce((sum, v) => sum + severityWeights[v.severity], 0);
+const score = Math.max(0, 100 - totalPenalty);
+```
+
+---
+
+## OUTPUT FORMAT
+
+```json
+{
+  "dimension": "isolation",
+  "score": 90,
+  "max_score": 100,
+  "grade": "A-",
+  "violations": [
+    {
+      "file": "tests/api/integration.spec.ts",
+      "line": 15,
+      "severity": "HIGH",
+      "category": "test-order-dependency",
+      "description": "Test depends on previous test creating user record",
+      "suggestion": "Each test should create its own test data in beforeEach",
+      "code_snippet": "test('should update user', async () => { /* assumes user exists */ });"
+    }
+  ],
+  "passed_checks": 14,
+  "failed_checks": 1,
+  "total_checks": 15,
+  "violation_summary": {
+    "HIGH": 1,
+    "MEDIUM": 0,
+    "LOW": 0
+  },
+  "recommendations": [
+    "Add beforeEach hooks to create test data",
+    "Add afterEach hooks to cleanup created records",
+    "Use test.describe.configure({ mode: 'parallel' }) to enforce isolation"
+  ],
+  "summary": "Tests are well isolated with 1 HIGH severity violation"
+}
+```
+
+---
+
+## EXIT CONDITION
+
+Subagent completes when:
+
+- ✅ All test files analyzed for isolation violations
+- ✅ Score calculated
+- ✅ JSON output written to temp file
+
+**Subagent terminates here.**
+
+---
+
+## 🚨 SUBAGENT SUCCESS METRICS
+
+### ✅ SUCCESS:
+
+- Only isolation checked (not other dimensions)
+- JSON output valid and complete
+
+### ❌ FAILURE:
+
+- Checked quality dimensions other than isolation
+- Invalid or missing JSON output