maestro-flow 0.3.10 → 0.3.12

package/workflows/integration-test.md

@@ -1,343 +1,355 @@
-# Integration Test Workflow
-
-Self-iterating integration test cycle that combines exploration, test design, execution, reflection, and adaptive strategy adjustment. Runs automated tests in a closed loop that self-corrects until convergence.
-
-6-phase cycle: Explore -> Design -> Develop -> Test -> Reflect -> Adjust
-Adaptive strategy: Conservative -> Aggressive -> Surgical -> Reflective
-L0-L3 progressive layers: Static Analysis -> Unit -> Integration -> E2E
-
----
-
-### Step 1: Parse Input and Initialize
-
-**Parse arguments:**
-
-| Input | Result |
-|-------|--------|
-| No arguments | Error E001 |
-| Phase number | Resolve `.workflow/phases/{NN}-{slug}/` |
-| `--max-iter N` | Set MAX_ITER = N (default 5) |
-| `--layer L2` | Start from L2 layer |
-
-Check for existing integration test session:
-```bash
-ls .workflow/phases/{NN}-*/.tests/integration/state.json 2>/dev/null
-```
-
-If session exists: offer resume or restart.
-
-**Initialize session:**
-
-Create `.tests/integration/` directory.
-
-Write initial `state.json`:
-```json
-{
-  "phase": "{phase}",
-  "started_at": "{ISO timestamp}",
-  "current_iteration": 0,
-  "max_iterations": 5,
-  "strategy": "conservative",
-  "current_layer": "L0",
-  "pass_rates": [],
-  "convergence_threshold": 95,
-  "status": "running"
-}
-```
-
-Initialize `reflection-log.md`:
-```markdown
-# Integration Test Reflection Log
-Phase: {phase_name}
-Started: {timestamp}
-
-## Iterations
-```
-
----
-
-### Step 1.3: Load Project Specs
-
-```
-specs_content = maestro spec load --category test
-arch_content = maestro spec load --category planning
-```
-
-`specs_content` for test conventions; `arch_content` for module boundaries in Step 2-3.
-
----
-
-### Step 1.5: Load Previously Generated Tests
-
-Check for tests already generated by `quality-test-gen`:
-
-```
-generated_tests = []
-IF file exists "${PHASE_DIR}/.tests/test-gen-report.json":
-  report = read_json("${PHASE_DIR}/.tests/test-gen-report.json")
-  FOR test_entry IN report.generated:
-    IF test_entry.layer is "integration" or "e2e":
-      generated_tests.push(test_entry)
-  IF generated_tests.length > 0:
-    display "Found {generated_tests.length} pre-generated integration/e2e tests from test-gen. Will incorporate into test plan."
-```
-
-These tests are merged into the test plan in Step 3 (Design) — they are executed but not re-generated.
-
----
-
-### Step 2: Explore
-
-**Phase 1: Explore codebase for testable integration points.**
-
-Discover:
-- Module boundaries and cross-module calls
-- API endpoints and their handlers
-- Database interactions and queries
-- External service integrations
-- Event flows and message passing
-
-```bash
-# Find cross-module imports
-grep -r "import.*from.*\.\./" src/ --include="*.ts" --include="*.tsx" -l 2>/dev/null | head -20
-
-# Find API route definitions
-grep -r "router\.\|app\.\(get\|post\|put\|delete\)" src/ --include="*.ts" -l 2>/dev/null | head -20
-
-# Find database calls
-grep -r "prisma\.\|query\|findMany\|findOne\|create\|update\|delete" src/ --include="*.ts" -l 2>/dev/null | head -20
-```
-
-Map integration points: which modules talk to which, through what interfaces.
-
----
-
-### Step 3: Design
-
-**Phase 2: Design integration test plan.**
-
-**Merge pre-generated tests** (from Step 1.5):
-If `generated_tests` is not empty, incorporate them into the test plan directly — mark as "pre-existing" so Step 4 (Develop) skips writing them and Step 5 (Test Execute) includes them in the run.
-
-Based on exploration (and pre-generated tests if available), design tests per layer:
-
-**L0 - Static Analysis:**
-- Type check command
-- Lint command
-- Dead code detection
-
-**L1 - Unit Tests (isolation):**
-- Critical functions identified in explore
-- Edge cases for core logic
-
-**L2 - Integration Tests:**
-- API endpoint tests (request -> response)
-- Cross-module interaction tests
-- Database query tests (with fixtures)
-
-**L3 - E2E Tests:**
-- Critical user flows
-- Happy path + error path
-
-Write test plan to `.tests/integration/test-plan.json`:
-```json
-{
-  "layers": {
-    "L0": { "commands": ["tsc --noEmit", "eslint src/"], "expected_pass": true },
-    "L1": { "test_files": [...], "test_count": N },
-    "L2": { "test_files": [...], "test_count": N },
-    "L3": { "test_files": [...], "test_count": N }
-  },
-  "total_tests": M
-}
-```
-
----
-
-### Step 4: Develop
-
-**Phase 3: Develop/write test code.**
-
-For each layer (current and below), write tests following existing patterns.
-Use quality-test-gen's RED-GREEN methodology for test writing.
-
-For integration tests (L2) specifically:
-- Set up test fixtures (DB seeds, mock services)
-- Write request/response assertions
-- Test error handling paths
-- Verify cross-module data flow
-
----
-
-### Step 5: Test Execute
-
-**Phase 4: Execute tests for current layer.**
-
-Run tests progressively (L0 must pass before L1, etc.):
-
-```bash
-# L0: Static analysis
-tsc --noEmit 2>&1 | tail -30
-eslint src/ 2>&1 | tail -30
-
-# L1: Unit tests
-{test_command} --testPathPattern="unit|__tests__" 2>&1 | tail -50
-
-# L2: Integration tests
-{test_command} --testPathPattern="integration" 2>&1 | tail -50
-
-# L3: E2E tests
-{test_command} --testPathPattern="e2e" 2>&1 | tail -50
-```
-
-Record results in `test-results-iter-{N}.json`:
-```json
-{
-  "iteration": N,
-  "timestamp": "{ISO timestamp}",
-  "strategy": "{current_strategy}",
-  "layers": {
-    "L0": { "status": "pass|fail", "errors": [...] },
-    "L1": { "total": N, "passed": P, "failed": F, "pass_rate": 95.0 },
-    "L2": { "total": N, "passed": P, "failed": F, "pass_rate": 87.5 },
-    "L3": { "total": N, "passed": P, "failed": F, "pass_rate": 100.0 }
-  },
-  "overall_pass_rate": 92.3
-}
-```
-
-Calculate overall pass rate.
-
----
-
-### Step 6: Reflect
-
-**Phase 5: Reflect on iteration results.**
-
-Analyze what happened:
-- Which tests failed and why?
-- Is pass rate improving, plateauing, or regressing?
-- Are failures clustered in one area or spread out?
-- Is the current strategy working?
-
-Append to `reflection-log.md`:
-```markdown
-## Iteration {N}
-Strategy: {strategy_name}
-Pass rate: {rate}% (previous: {prev_rate}%)
-Delta: {+/-}%
-
-### What worked
-- {observation}
-
-### What failed
-- {test}: {reason}
-
-### Pattern detected
-- {pattern, e.g., "all failures in auth module"}
-
-### Strategy assessment
-- Current strategy: {effective|ineffective|partially_effective}
-- Recommendation: {keep|switch_to_X}
-```
-
----
-
-### Step 7: Adjust
-
-**Phase 6: Adjust strategy based on reflection.**
-
-**Adaptive Strategy Engine:**
-
-| Condition | Strategy | Behavior |
-|-----------|----------|----------|
-| Iteration 1-2 | Conservative | Fix obvious failures, don't refactor |
-| Pass rate >80% AND failures similar to previous | Aggressive | Batch-fix related failures together |
-| New regressions appeared | Surgical | Revert last changes, fix regression only |
-| Stuck 3+ iterations (rate not improving) | Reflective | Step back, re-analyze root cause pattern |
-
-**Strategy transitions:**
-```
-Conservative -> (pass rate >80%) -> Aggressive
-Aggressive -> (regression) -> Surgical
-Surgical -> (regression fixed) -> Aggressive
-Any -> (stuck 3+ iters) -> Reflective
-Reflective -> (new insight) -> Conservative (restart approach)
-```
-
-Update state.json with new strategy and iteration count.
-
-**Convergence check:**
-- If overall_pass_rate >= convergence_threshold (95%): CONVERGED -> go to Step 8
-- If iteration >= max_iterations: MAX_ITER_REACHED -> go to Step 8
-- Otherwise: go back to Step 4 (next iteration)
-
----
-
-### Step 8: Complete
-
-**Complete integration test cycle.**
-
-Update state.json: status -> "complete" or "max_iter_reached".
-
-Write `.tests/integration/summary.json`:
-```json
-{
-  "phase": "{phase}",
-  "completed_at": "{ISO timestamp}",
-  "iterations": N,
-  "final_pass_rate": 97.5,
-  "converged": true,
-  "strategy_history": ["conservative", "conservative", "aggressive"],
-  "layers": {
-    "L0": { "status": "pass" },
-    "L1": { "pass_rate": 100.0 },
-    "L2": { "pass_rate": 95.0 },
-    "L3": { "pass_rate": 100.0 }
-  },
-  "bugs_discovered": [...],
-  "regressions_fixed": [...]
-}
-```
-
-Update index.json with integration test results.
-
----
-
-### Step 9: Report
-
-```
-=== INTEGRATION TEST CYCLE ===
-Phase:       {phase_name}
-Iterations:  {N}/{max_iter}
-Converged:   {yes/no} (threshold: {threshold}%)
-
-Layer Results:
-  L0 Static:      {pass/fail}
-  L1 Unit:        {pass_rate}% ({passed}/{total})
-  L2 Integration: {pass_rate}% ({passed}/{total})
-  L3 E2E:         {pass_rate}% ({passed}/{total})
-
-Overall:     {final_pass_rate}%
-Strategy:    {final_strategy} (transitioned {N} times)
-
-Bugs Found:  {bug_count}
-Regressions: {regression_count} found, {fixed_count} fixed
-
-Files:
-  {target_dir}/.tests/integration/state.json
-  {target_dir}/.tests/integration/reflection-log.md
-  {target_dir}/.tests/integration/summary.json
-
-Next steps:
-  {suggested_next_command}
-```
-
-**Next step routing:**
-
-| Result | Suggestion |
-|--------|------------|
-| Converged (>=95%) | Skill({ skill: "maestro-verify", args: "{phase}" }) to update validation |
-| Max iter, >80% | Skill({ skill: "quality-test", args: "{phase}" }) for manual UAT on remaining gaps |
-| Max iter, <80% | Skill({ skill: "quality-debug" }) for deep investigation |
-| Bugs discovered | Skill({ skill: "maestro-plan", args: "{phase} --gaps" }) to plan fixes |
+# Integration Test Workflow
+
+Self-iterating integration test cycle that combines exploration, test design, execution, reflection, and adaptive strategy adjustment. Runs automated tests in a closed loop that self-corrects until convergence.
+
+6-phase cycle: Explore -> Design -> Develop -> Test -> Reflect -> Adjust
+Adaptive strategy: Conservative -> Aggressive -> Surgical -> Reflective
+L0-L3 progressive layers: Static Analysis -> Unit -> Integration -> E2E
+
+---
+
+### Step 1: Parse Input and Initialize
+
+**Parse arguments:**
+
+| Input | Result |
+|-------|--------|
+| No arguments | Error E001 |
+| Phase number | Resolve phase dir (artifact registry or legacy `.workflow/phases/{NN}-{slug}/`) |
+| `--max-iter N` | Set MAX_ITER = N (default 5) |
+| `--layer L2` | Start from L2 layer |
+
+**Resolve phase dir:**
+```
+Read .workflow/state.json → state
+artifacts = state.artifacts ?? []
+IF artifacts.length > 0:
+  art = artifacts.find(a => a.type === 'execute' && a.phase === phaseNum)
+  PHASE_DIR = ".workflow/" + art.path
+ELSE:
+  Glob: .workflow/phases/{NN}-*/
+  PHASE_DIR = resolved path
+```
+
+Check for existing integration test session:
+```bash
+ls ${PHASE_DIR}/.tests/integration/state.json 2>/dev/null
+```
+
+If session exists: offer resume or restart.
+
+**Initialize session:**
+
+Create `.tests/integration/` directory.
+
+Write initial `state.json`:
+```json
+{
+  "phase": "{phase}",
+  "started_at": "{ISO timestamp}",
+  "current_iteration": 0,
+  "max_iterations": 5,
+  "strategy": "conservative",
+  "current_layer": "L0",
+  "pass_rates": [],
+  "convergence_threshold": 95,
+  "status": "running"
+}
+```
+
+Initialize `reflection-log.md`:
+```markdown
+# Integration Test Reflection Log
+Phase: {phase_name}
+Started: {timestamp}
+
+## Iterations
+```
+
+---
+
+### Step 1.3: Load Project Specs
+
+```
+specs_content = maestro spec load --category test
+arch_content = maestro spec load --category arch
+```
+
+`specs_content` for test conventions; `arch_content` for module boundaries in Step 2-3.
+
+---
+
+### Step 1.5: Load Previously Generated Tests
+
+Check for tests already generated by `quality-test-gen`:
+
+```
+generated_tests = []
+IF file exists "${PHASE_DIR}/.tests/test-gen-report.json":
+  report = read_json("${PHASE_DIR}/.tests/test-gen-report.json")
+  FOR test_entry IN report.generated:
+    IF test_entry.layer is "integration" or "e2e":
+      generated_tests.push(test_entry)
+  IF generated_tests.length > 0:
+    display "Found {generated_tests.length} pre-generated integration/e2e tests from test-gen. Will incorporate into test plan."
+```
+
+These tests are merged into the test plan in Step 3 (Design) — they are executed but not re-generated.
+
+---
+
+### Step 2: Explore
+
+**Phase 1: Explore codebase for testable integration points.**
+
+Discover:
+- Module boundaries and cross-module calls
+- API endpoints and their handlers
+- Database interactions and queries
+- External service integrations
+- Event flows and message passing
+
+```bash
+# Find cross-module imports
+grep -r "import.*from.*\.\./" src/ --include="*.ts" --include="*.tsx" -l 2>/dev/null | head -20
+
+# Find API route definitions
+grep -r "router\.\|app\.\(get\|post\|put\|delete\)" src/ --include="*.ts" -l 2>/dev/null | head -20
+
+# Find database calls
+grep -r "prisma\.\|query\|findMany\|findOne\|create\|update\|delete" src/ --include="*.ts" -l 2>/dev/null | head -20
+```
+
+Map integration points: which modules talk to which, through what interfaces.
+
+---
+
+### Step 3: Design
+
+**Phase 2: Design integration test plan.**
+
+**Merge pre-generated tests** (from Step 1.5):
+If `generated_tests` is not empty, incorporate them into the test plan directly — mark as "pre-existing" so Step 4 (Develop) skips writing them and Step 5 (Test Execute) includes them in the run.
+
+Based on exploration (and pre-generated tests if available), design tests per layer:
+
+**L0 - Static Analysis:**
+- Type check command
+- Lint command
+- Dead code detection
+
+**L1 - Unit Tests (isolation):**
+- Critical functions identified in explore
+- Edge cases for core logic
+
+**L2 - Integration Tests:**
+- API endpoint tests (request -> response)
+- Cross-module interaction tests
+- Database query tests (with fixtures)
+
+**L3 - E2E Tests:**
+- Critical user flows
+- Happy path + error path
+
+Write test plan to `.tests/integration/test-plan.json`:
+```json
+{
+  "layers": {
+    "L0": { "commands": ["tsc --noEmit", "eslint src/"], "expected_pass": true },
+    "L1": { "test_files": [...], "test_count": N },
+    "L2": { "test_files": [...], "test_count": N },
+    "L3": { "test_files": [...], "test_count": N }
+  },
+  "total_tests": M
+}
+```
+
+---
+
+### Step 4: Develop
+
+**Phase 3: Develop/write test code.**
+
+For each layer (current and below), write tests following existing patterns.
+Use quality-test-gen's RED-GREEN methodology for test writing.
+
+For integration tests (L2) specifically:
+- Set up test fixtures (DB seeds, mock services)
+- Write request/response assertions
+- Test error handling paths
+- Verify cross-module data flow
+
+---
+
+### Step 5: Test Execute
+
+**Phase 4: Execute tests for current layer.**
+
+Run tests progressively (L0 must pass before L1, etc.):
+
+```bash
+# L0: Static analysis
+tsc --noEmit 2>&1 | tail -30
+eslint src/ 2>&1 | tail -30
+
+# L1: Unit tests
+{test_command} --testPathPattern="unit|__tests__" 2>&1 | tail -50
+
+# L2: Integration tests
+{test_command} --testPathPattern="integration" 2>&1 | tail -50
+
+# L3: E2E tests
+{test_command} --testPathPattern="e2e" 2>&1 | tail -50
+```
+
+Record results in `test-results-iter-{N}.json`:
+```json
+{
+  "iteration": N,
+  "timestamp": "{ISO timestamp}",
+  "strategy": "{current_strategy}",
+  "layers": {
+    "L0": { "status": "pass|fail", "errors": [...] },
+    "L1": { "total": N, "passed": P, "failed": F, "pass_rate": 95.0 },
+    "L2": { "total": N, "passed": P, "failed": F, "pass_rate": 87.5 },
+    "L3": { "total": N, "passed": P, "failed": F, "pass_rate": 100.0 }
+  },
+  "overall_pass_rate": 92.3
+}
+```
+
+Calculate overall pass rate.
+
+---
+
+### Step 6: Reflect
+
+**Phase 5: Reflect on iteration results.**
+
+Analyze what happened:
+- Which tests failed and why?
+- Is pass rate improving, plateauing, or regressing?
+- Are failures clustered in one area or spread out?
+- Is the current strategy working?
+
+Append to `reflection-log.md`:
+```markdown
+## Iteration {N}
+Strategy: {strategy_name}
+Pass rate: {rate}% (previous: {prev_rate}%)
+Delta: {+/-}%
+
+### What worked
+- {observation}
+
+### What failed
+- {test}: {reason}
+
+### Pattern detected
+- {pattern, e.g., "all failures in auth module"}
+
+### Strategy assessment
+- Current strategy: {effective|ineffective|partially_effective}
+- Recommendation: {keep|switch_to_X}
+```
+
+---
+
+### Step 7: Adjust
+
+**Phase 6: Adjust strategy based on reflection.**
+
+**Adaptive Strategy Engine:**
+
+| Condition | Strategy | Behavior |
+|-----------|----------|----------|
+| Iteration 1-2 | Conservative | Fix obvious failures, don't refactor |
+| Pass rate >80% AND failures similar to previous | Aggressive | Batch-fix related failures together |
+| New regressions appeared | Surgical | Revert last changes, fix regression only |
+| Stuck 3+ iterations (rate not improving) | Reflective | Step back, re-analyze root cause pattern |
+
+**Strategy transitions:**
+```
+Conservative -> (pass rate >80%) -> Aggressive
+Aggressive -> (regression) -> Surgical
+Surgical -> (regression fixed) -> Aggressive
+Any -> (stuck 3+ iters) -> Reflective
+Reflective -> (new insight) -> Conservative (restart approach)
+```
+
+Update state.json with new strategy and iteration count.
+
+**Convergence check:**
+- If overall_pass_rate >= convergence_threshold (95%): CONVERGED -> go to Step 8
+- If iteration >= max_iterations: MAX_ITER_REACHED -> go to Step 8
+- Otherwise: go back to Step 4 (next iteration)
+
+---
+
+### Step 8: Complete
+
+**Complete integration test cycle.**
+
+Update state.json: status -> "complete" or "max_iter_reached".
+
+Write `.tests/integration/summary.json`:
+```json
+{
+  "phase": "{phase}",
+  "completed_at": "{ISO timestamp}",
+  "iterations": N,
+  "final_pass_rate": 97.5,
+  "converged": true,
+  "strategy_history": ["conservative", "conservative", "aggressive"],
+  "layers": {
+    "L0": { "status": "pass" },
+    "L1": { "pass_rate": 100.0 },
+    "L2": { "pass_rate": 95.0 },
+    "L3": { "pass_rate": 100.0 }
+  },
+  "bugs_discovered": [...],
+  "regressions_fixed": [...]
+}
+```
+
+Update index.json with integration test results.
+
+---
+
+### Step 9: Report
+
+```
+=== INTEGRATION TEST CYCLE ===
+Phase:       {phase_name}
+Iterations:  {N}/{max_iter}
+Converged:   {yes/no} (threshold: {threshold}%)
+
+Layer Results:
+  L0 Static:      {pass/fail}
+  L1 Unit:        {pass_rate}% ({passed}/{total})
+  L2 Integration: {pass_rate}% ({passed}/{total})
+  L3 E2E:         {pass_rate}% ({passed}/{total})
+
+Overall:     {final_pass_rate}%
+Strategy:    {final_strategy} (transitioned {N} times)
+
+Bugs Found:  {bug_count}
+Regressions: {regression_count} found, {fixed_count} fixed
+
+Files:
+  {target_dir}/.tests/integration/state.json
+  {target_dir}/.tests/integration/reflection-log.md
+  {target_dir}/.tests/integration/summary.json
+
+Next steps:
+  {suggested_next_command}
+```
+
+**Next step routing:**
+
+| Result | Suggestion |
+|--------|------------|
+| Converged (>=95%) | Skill({ skill: "maestro-verify", args: "{phase}" }) to update validation |
+| Max iter, >80% | Skill({ skill: "quality-test", args: "{phase}" }) for manual UAT on remaining gaps |
+| Max iter, <80% | Skill({ skill: "quality-debug" }) for deep investigation |
+| Bugs discovered | Skill({ skill: "maestro-plan", args: "{phase} --gaps" }) to plan fixes |