maestro-flow 0.5.3 → 0.5.31
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/learn-follow/SKILL.md +114 -114
- package/.agents/skills/learn-investigate/SKILL.md +138 -139
- package/.agents/skills/learn-second-opinion/SKILL.md +105 -109
- package/.agents/skills/maestro/SKILL.md +2 -10
- package/.agents/skills/maestro-amend/SKILL.md +152 -152
- package/.agents/skills/maestro-analyze/SKILL.md +201 -252
- package/.agents/skills/maestro-blueprint/SKILL.md +175 -190
- package/.agents/skills/maestro-brainstorm/SKILL.md +196 -200
- package/.agents/skills/maestro-collab/SKILL.md +159 -159
- package/.agents/skills/maestro-companion/SKILL.md +517 -517
- package/.agents/skills/maestro-composer/SKILL.md +173 -164
- package/.agents/skills/maestro-execute/SKILL.md +169 -170
- package/.agents/skills/maestro-fork/SKILL.md +97 -96
- package/.agents/skills/maestro-grill/SKILL.md +161 -162
- package/.agents/skills/maestro-guard/SKILL.md +93 -92
- package/.agents/skills/maestro-impeccable/SKILL.md +296 -253
- package/.agents/skills/maestro-init/SKILL.md +117 -118
- package/.agents/skills/maestro-merge/SKILL.md +73 -66
- package/.agents/skills/maestro-milestone-audit/SKILL.md +4 -10
- package/.agents/skills/maestro-milestone-complete/SKILL.md +6 -7
- package/.agents/skills/maestro-milestone-release/SKILL.md +122 -131
- package/.agents/skills/maestro-next/SKILL.md +241 -245
- package/.agents/skills/maestro-overlay/SKILL.md +176 -166
- package/.agents/skills/maestro-plan/SKILL.md +211 -197
- package/.agents/skills/maestro-player/SKILL.md +167 -167
- package/.agents/skills/maestro-quick/SKILL.md +69 -63
- package/.agents/skills/maestro-ralph/SKILL.md +2 -36
- package/.agents/skills/maestro-ralph-beta/SKILL.md +861 -872
- package/.agents/skills/maestro-ralph-execute/SKILL.md +234 -234
- package/.agents/skills/maestro-roadmap/SKILL.md +159 -172
- package/.agents/skills/maestro-swarm-workflow/SKILL.md +229 -250
- package/.agents/skills/maestro-tools-execute/SKILL.md +108 -103
- package/.agents/skills/maestro-tools-register/SKILL.md +148 -143
- package/.agents/skills/maestro-ui-codify/SKILL.md +103 -86
- package/.agents/skills/maestro-universal-workflow/SKILL.md +534 -547
- package/.agents/skills/maestro-update/SKILL.md +109 -106
- package/.agents/skills/manage-codebase-rebuild/SKILL.md +73 -71
- package/.agents/skills/manage-harvest/SKILL.md +83 -81
- package/.agents/skills/manage-issue/SKILL.md +59 -60
- package/.agents/skills/manage-issue-discover/SKILL.md +70 -68
- package/.agents/skills/manage-kg-extractors/SKILL.md +130 -0
- package/.agents/skills/manage-knowhow/SKILL.md +70 -66
- package/.agents/skills/manage-knowhow-capture/SKILL.md +79 -69
- package/.agents/skills/manage-knowledge-audit/SKILL.md +91 -74
- package/.agents/skills/manage-status/SKILL.md +52 -42
- package/.agents/skills/manage-wiki/SKILL.md +69 -58
- package/.agents/skills/odyssey-debug/SKILL.md +445 -459
- package/.agents/skills/odyssey-improve/SKILL.md +477 -491
- package/.agents/skills/odyssey-planex/SKILL.md +576 -587
- package/.agents/skills/odyssey-review-test-fix/SKILL.md +400 -413
- package/.agents/skills/odyssey-ui/SKILL.md +431 -448
- package/.agents/skills/quality-auto-test/SKILL.md +140 -123
- package/.agents/skills/quality-debug/SKILL.md +145 -106
- package/.agents/skills/quality-refactor/SKILL.md +91 -53
- package/.agents/skills/quality-retrospective/SKILL.md +109 -63
- package/.agents/skills/quality-review/SKILL.md +141 -114
- package/.agents/skills/quality-sync/SKILL.md +74 -38
- package/.agents/skills/quality-test/SKILL.md +133 -103
- package/.agents/skills/security-audit/SKILL.md +217 -166
- package/.agents/skills/spec-add/SKILL.md +66 -59
- package/.agents/skills/spec-load/SKILL.md +68 -68
- package/.agents/skills/spec-remove/SKILL.md +42 -42
- package/.agents/skills/spec-setup/SKILL.md +38 -41
- package/.agy/skills/learn-follow/SKILL.md +114 -114
- package/.agy/skills/learn-investigate/SKILL.md +138 -139
- package/.agy/skills/learn-second-opinion/SKILL.md +105 -109
- package/.agy/skills/maestro/SKILL.md +2 -10
- package/.agy/skills/maestro-amend/SKILL.md +152 -152
- package/.agy/skills/maestro-analyze/SKILL.md +201 -252
- package/.agy/skills/maestro-blueprint/SKILL.md +175 -190
- package/.agy/skills/maestro-brainstorm/SKILL.md +196 -200
- package/.agy/skills/maestro-collab/SKILL.md +159 -159
- package/.agy/skills/maestro-companion/SKILL.md +517 -517
- package/.agy/skills/maestro-composer/SKILL.md +173 -164
- package/.agy/skills/maestro-execute/SKILL.md +169 -170
- package/.agy/skills/maestro-fork/SKILL.md +97 -96
- package/.agy/skills/maestro-grill/SKILL.md +161 -162
- package/.agy/skills/maestro-guard/SKILL.md +93 -92
- package/.agy/skills/maestro-impeccable/SKILL.md +296 -253
- package/.agy/skills/maestro-init/SKILL.md +117 -118
- package/.agy/skills/maestro-merge/SKILL.md +73 -66
- package/.agy/skills/maestro-milestone-audit/SKILL.md +4 -10
- package/.agy/skills/maestro-milestone-complete/SKILL.md +6 -7
- package/.agy/skills/maestro-milestone-release/SKILL.md +122 -131
- package/.agy/skills/maestro-next/SKILL.md +241 -245
- package/.agy/skills/maestro-overlay/SKILL.md +176 -166
- package/.agy/skills/maestro-plan/SKILL.md +211 -197
- package/.agy/skills/maestro-player/SKILL.md +167 -167
- package/.agy/skills/maestro-quick/SKILL.md +69 -63
- package/.agy/skills/maestro-ralph/SKILL.md +2 -36
- package/.agy/skills/maestro-ralph-beta/SKILL.md +861 -872
- package/.agy/skills/maestro-ralph-execute/SKILL.md +234 -234
- package/.agy/skills/maestro-roadmap/SKILL.md +159 -172
- package/.agy/skills/maestro-swarm-workflow/SKILL.md +229 -250
- package/.agy/skills/maestro-tools-execute/SKILL.md +108 -103
- package/.agy/skills/maestro-tools-register/SKILL.md +148 -143
- package/.agy/skills/maestro-ui-codify/SKILL.md +103 -86
- package/.agy/skills/maestro-universal-workflow/SKILL.md +534 -547
- package/.agy/skills/maestro-update/SKILL.md +109 -106
- package/.agy/skills/manage-codebase-rebuild/SKILL.md +73 -71
- package/.agy/skills/manage-harvest/SKILL.md +83 -81
- package/.agy/skills/manage-issue/SKILL.md +59 -60
- package/.agy/skills/manage-issue-discover/SKILL.md +70 -68
- package/.agy/skills/manage-kg-extractors/SKILL.md +130 -0
- package/.agy/skills/manage-knowhow/SKILL.md +70 -66
- package/.agy/skills/manage-knowhow-capture/SKILL.md +79 -69
- package/.agy/skills/manage-knowledge-audit/SKILL.md +91 -74
- package/.agy/skills/manage-status/SKILL.md +52 -42
- package/.agy/skills/manage-wiki/SKILL.md +69 -58
- package/.agy/skills/odyssey-debug/SKILL.md +445 -459
- package/.agy/skills/odyssey-improve/SKILL.md +477 -491
- package/.agy/skills/odyssey-planex/SKILL.md +576 -587
- package/.agy/skills/odyssey-review-test-fix/SKILL.md +400 -413
- package/.agy/skills/odyssey-ui/SKILL.md +431 -448
- package/.agy/skills/quality-auto-test/SKILL.md +140 -123
- package/.agy/skills/quality-debug/SKILL.md +145 -106
- package/.agy/skills/quality-refactor/SKILL.md +91 -53
- package/.agy/skills/quality-retrospective/SKILL.md +109 -63
- package/.agy/skills/quality-review/SKILL.md +141 -114
- package/.agy/skills/quality-sync/SKILL.md +74 -38
- package/.agy/skills/quality-test/SKILL.md +133 -103
- package/.agy/skills/security-audit/SKILL.md +217 -166
- package/.agy/skills/spec-add/SKILL.md +66 -59
- package/.agy/skills/spec-load/SKILL.md +68 -68
- package/.agy/skills/spec-remove/SKILL.md +42 -42
- package/.agy/skills/spec-setup/SKILL.md +38 -41
- package/.claude/commands/learn-follow.md +127 -127
- package/.claude/commands/learn-investigate.md +151 -152
- package/.claude/commands/learn-second-opinion.md +118 -122
- package/.claude/commands/maestro-amend.md +164 -164
- package/.claude/commands/maestro-analyze.md +215 -266
- package/.claude/commands/maestro-blueprint.md +189 -204
- package/.claude/commands/maestro-brainstorm.md +209 -213
- package/.claude/commands/maestro-collab.md +172 -172
- package/.claude/commands/maestro-companion.md +531 -531
- package/.claude/commands/maestro-composer.md +188 -179
- package/.claude/commands/maestro-execute.md +183 -184
- package/.claude/commands/maestro-fork.md +111 -110
- package/.claude/commands/maestro-grill.md +175 -176
- package/.claude/commands/maestro-guard.md +103 -102
- package/.claude/commands/maestro-impeccable.md +311 -268
- package/.claude/commands/maestro-init.md +130 -131
- package/.claude/commands/maestro-merge.md +87 -80
- package/.claude/commands/maestro-milestone-audit.md +4 -10
- package/.claude/commands/maestro-milestone-complete.md +6 -7
- package/.claude/commands/maestro-milestone-release.md +136 -145
- package/.claude/commands/maestro-next.md +253 -257
- package/.claude/commands/maestro-overlay.md +188 -178
- package/.claude/commands/maestro-plan.md +225 -211
- package/.claude/commands/maestro-player.md +182 -182
- package/.claude/commands/maestro-quick.md +83 -77
- package/.claude/commands/maestro-ralph-beta.md +875 -886
- package/.claude/commands/maestro-ralph-execute.md +247 -247
- package/.claude/commands/maestro-ralph.md +2 -36
- package/.claude/commands/maestro-roadmap.md +173 -186
- package/.claude/commands/maestro-swarm-workflow.md +243 -264
- package/.claude/commands/maestro-tools-execute.md +122 -117
- package/.claude/commands/maestro-tools-register.md +162 -157
- package/.claude/commands/maestro-ui-codify.md +117 -100
- package/.claude/commands/maestro-universal-workflow.md +548 -561
- package/.claude/commands/maestro-update.md +122 -119
- package/.claude/commands/maestro.md +2 -10
- package/.claude/commands/manage-codebase-rebuild.md +87 -85
- package/.claude/commands/manage-harvest.md +97 -95
- package/.claude/commands/manage-issue-discover.md +83 -81
- package/.claude/commands/manage-issue.md +72 -73
- package/.claude/commands/manage-kg-extractors.md +128 -0
- package/.claude/commands/manage-knowhow-capture.md +92 -82
- package/.claude/commands/manage-knowhow.md +83 -79
- package/.claude/commands/manage-knowledge-audit.md +105 -88
- package/.claude/commands/manage-status.md +62 -52
- package/.claude/commands/manage-wiki.md +82 -71
- package/.claude/commands/odyssey-debug.md +459 -473
- package/.claude/commands/odyssey-improve.md +491 -505
- package/.claude/commands/odyssey-planex.md +590 -601
- package/.claude/commands/odyssey-review-test-fix.md +414 -427
- package/.claude/commands/odyssey-ui.md +445 -462
- package/.claude/commands/quality-auto-test.md +153 -136
- package/.claude/commands/quality-debug.md +159 -120
- package/.claude/commands/quality-refactor.md +105 -67
- package/.claude/commands/quality-retrospective.md +123 -77
- package/.claude/commands/quality-review.md +155 -128
- package/.claude/commands/quality-sync.md +88 -52
- package/.claude/commands/quality-test.md +147 -117
- package/.claude/commands/security-audit.md +230 -179
- package/.claude/commands/spec-add.md +77 -70
- package/.claude/commands/spec-load.md +78 -78
- package/.claude/commands/spec-remove.md +55 -55
- package/.claude/commands/spec-setup.md +49 -52
- package/dist/src/cli.js +1 -1
- package/dist/src/cli.js.map +1 -1
- package/dist/src/commands/kg.d.ts.map +1 -1
- package/dist/src/commands/kg.js +11 -5
- package/dist/src/commands/kg.js.map +1 -1
- package/dist/src/graph/kg/extraction/code/code-extractor.d.ts +2 -0
- package/dist/src/graph/kg/extraction/code/code-extractor.d.ts.map +1 -1
- package/dist/src/graph/kg/extraction/code/code-extractor.js +32 -3
- package/dist/src/graph/kg/extraction/code/code-extractor.js.map +1 -1
- package/dist/src/graph/kg/extraction/code/plugin-engine.d.ts +35 -0
- package/dist/src/graph/kg/extraction/code/plugin-engine.d.ts.map +1 -0
- package/dist/src/graph/kg/extraction/code/plugin-engine.js +573 -0
- package/dist/src/graph/kg/extraction/code/plugin-engine.js.map +1 -0
- package/dist/src/graph/kg/extraction/code/plugin-types.d.ts +95 -0
- package/dist/src/graph/kg/extraction/code/plugin-types.d.ts.map +1 -0
- package/dist/src/graph/kg/extraction/code/plugin-types.js +5 -0
- package/dist/src/graph/kg/extraction/code/plugin-types.js.map +1 -0
- package/dist/src/graph/kg/extraction/orchestrator.d.ts.map +1 -1
- package/dist/src/graph/kg/extraction/orchestrator.js +17 -5
- package/dist/src/graph/kg/extraction/orchestrator.js.map +1 -1
- package/dist/src/graph/kg/schema.sql +16 -11
- package/dist/src/graph/kg/surface/cli.d.ts.map +1 -1
- package/dist/src/graph/kg/surface/cli.js +153 -56
- package/dist/src/graph/kg/surface/cli.js.map +1 -1
- package/dist/src/hooks/workspace.d.ts +4 -2
- package/dist/src/hooks/workspace.d.ts.map +1 -1
- package/dist/src/hooks/workspace.js +6 -2
- package/dist/src/hooks/workspace.js.map +1 -1
- package/package.json +91 -91
- package/workflows/analyze.md +25 -49
- package/workflows/auto-test.md +699 -699
- package/workflows/blueprint.md +403 -431
- package/workflows/brainstorm.md +54 -195
- package/workflows/business-test.md +570 -570
- package/workflows/claude-instructions.md +23 -51
- package/workflows/codex-instructions.md +27 -77
- package/workflows/coding-philosophy.md +69 -69
- package/workflows/command-authoring.md +823 -823
- package/workflows/debug.md +43 -98
- package/workflows/delegate-usage.md +39 -241
- package/workflows/execute.md +4 -53
- package/workflows/grill.md +12 -56
- package/workflows/harvest.md +22 -68
- package/workflows/init.md +148 -148
- package/workflows/instruction-authoring-guide.md +97 -0
- package/workflows/issue-execute.md +110 -110
- package/workflows/issue-gaps-analyze.codex.md +260 -260
- package/workflows/issue-gaps-analyze.md +216 -216
- package/workflows/issue-plan.md +110 -110
- package/workflows/issue.md +338 -346
- package/workflows/knowhow.md +0 -32
- package/workflows/learn.md +277 -277
- package/workflows/maestro-chain-execute.md +20 -20
- package/workflows/refactor.md +22 -44
- package/workflows/retrospective.md +16 -65
- package/workflows/review.md +446 -486
- package/workflows/roadmap.md +35 -132
- package/workflows/skill-authoring.md +265 -265
- package/workflows/spec-generate.md +470 -470
- package/workflows/specs-remove.md +104 -104
- package/workflows/sync.md +11 -41
- package/workflows/test-gen.md +226 -226
- package/workflows/test.md +385 -475
- package/workflows/ui-design.md +391 -391
- package/workflows/ui-style.md +199 -199
- package/workflows/wiki-connect.md +151 -151
- package/workflows/wiki-digest.md +178 -178
- package/workflows/wiki-manage.md +109 -109
- package/workflows/cli-tools-usage.md +0 -252
- package/workflows/delegate-protocol.codex.md +0 -65
|
@@ -1,570 +1,570 @@
|
|
|
1
|
-
# Business Test Workflow (PRD-Forward)
|
|
2
|
-
|
|
3
|
-
Validate built features against PRD acceptance criteria through automated multi-layer business testing with requirement traceability, fixture generation, and feedback loop.
|
|
4
|
-
|
|
5
|
-
PRD-forward: starts from REQ-*.md acceptance criteria, not from code coverage gaps.
|
|
6
|
-
Progressive layers: L1 Interface Contract -> L2 Business Rule -> L3 Business Scenario (E2E).
|
|
7
|
-
Generator-Critic loop: max 3 iterations per layer to distinguish test defects from code defects.
|
|
8
|
-
|
|
9
|
-
**Philosophy: Business rules are the source of truth. Code must satisfy them, not the other way around.**
|
|
10
|
-
|
|
11
|
-
---
|
|
12
|
-
|
|
13
|
-
### Step 1: Resolve Target & Load Spec Package
|
|
14
|
-
|
|
15
|
-
**Parse arguments:**
|
|
16
|
-
|
|
17
|
-
| Input | Result |
|
|
18
|
-
|-------|--------|
|
|
19
|
-
| No arguments | Error E001 |
|
|
20
|
-
| Phase number | Resolve `.workflow/phases/{NN}-{slug}/` |
|
|
21
|
-
| `--spec SPEC-xxx` | Explicit spec reference |
|
|
22
|
-
| `--layer L1\|L2\|L3` | Run only specific layer |
|
|
23
|
-
| `--gen-code` | Generate framework-specific test classes |
|
|
24
|
-
| `--dry-run` | Extract scenarios only, don't execute |
|
|
25
|
-
| `--re-run` | Re-run only previously failed scenarios |
|
|
26
|
-
| `--auto` | Skip interactive confirmations |
|
|
27
|
-
|
|
28
|
-
**Load spec package:**
|
|
29
|
-
|
|
30
|
-
```
|
|
31
|
-
1. Read ${PHASE_DIR}/index.json -> extract blueprint_ref (if present)
|
|
32
|
-
2. IF --spec provided: SPEC_DIR = .workflow/blueprint/{blueprint_ref}/
|
|
33
|
-
ELSE IF index.json.blueprint_ref: SPEC_DIR = .workflow/blueprint/{blueprint_ref}/
|
|
34
|
-
ELSE: try .workflow/blueprint/SPEC-*/ (most recent)
|
|
35
|
-
|
|
36
|
-
3. IF SPEC_DIR found:
|
|
37
|
-
- Read requirements/_index.md (requirement summary + traceability matrix)
|
|
38
|
-
- Read all requirements/REQ-*.md (functional requirements with acceptance criteria)
|
|
39
|
-
- Read all requirements/NFR-*.md (non-functional requirements)
|
|
40
|
-
- Read architecture/_index.md (API endpoints, data model, state machines)
|
|
41
|
-
- Read epics/EPIC-*.md (user stories -> E2E scenario source)
|
|
42
|
-
SPEC_MODE = "full"
|
|
43
|
-
|
|
44
|
-
4. IF no spec package found (DEGRADED MODE):
|
|
45
|
-
- Read index.json.success_criteria
|
|
46
|
-
- Read plan.json tasks with convergence.criteria
|
|
47
|
-
- Read .summaries/TASK-*-summary.md for implemented behavior
|
|
48
|
-
SPEC_MODE = "degraded"
|
|
49
|
-
Display: "No spec package found. Using success_criteria + plan.json for scenario extraction (degraded mode)."
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Check for existing business test session:
|
|
53
|
-
```bash
|
|
54
|
-
ls ${PHASE_DIR}/.tests/business/business-test-report.json 2>/dev/null
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
If session exists AND `--re-run`: load previous report, filter to failed/blocked scenarios only.
|
|
58
|
-
If session exists AND no `--re-run`: offer resume or restart.
|
|
59
|
-
|
|
60
|
-
---
|
|
61
|
-
|
|
62
|
-
### Step 2: Extract Business Test Scenarios from PRD
|
|
63
|
-
|
|
64
|
-
**Full mode (SPEC_MODE = "full"):**
|
|
65
|
-
|
|
66
|
-
For each `REQ-NNN-{slug}.md`:
|
|
67
|
-
|
|
68
|
-
1. Parse `## Acceptance Criteria` section
|
|
69
|
-
2. Extract each numbered criterion as a test scenario seed
|
|
70
|
-
3. Map RFC 2119 keywords to test priority:
|
|
71
|
-
|
|
72
|
-
| Keyword | Priority | Failure Severity |
|
|
73
|
-
|---------|----------|-----------------|
|
|
74
|
-
| MUST / MUST NOT / SHALL / SHALL NOT | critical | blocker |
|
|
75
|
-
| SHOULD / SHOULD NOT / RECOMMENDED | high | major |
|
|
76
|
-
| MAY / OPTIONAL | medium | minor |
|
|
77
|
-
|
|
78
|
-
4. Classify scenario into layer:
|
|
79
|
-
|
|
80
|
-
| Source | Layer | Category |
|
|
81
|
-
|--------|-------|----------|
|
|
82
|
-
| Architecture API endpoints + REQ AC about request/response | L1 | api_contract |
|
|
83
|
-
| REQ AC about business logic, validation, state changes | L2 | business_rule |
|
|
84
|
-
| REQ AC about state transitions (from architecture state machines) | L2 | state_transition |
|
|
85
|
-
| Epic user stories (multi-step flows) | L3 | user_flow |
|
|
86
|
-
| NFR performance/security constraints | L2 | non_functional |
|
|
87
|
-
|
|
88
|
-
5. Generate scenario:
|
|
89
|
-
```json
|
|
90
|
-
{
|
|
91
|
-
"id": "BT-{NNN}",
|
|
92
|
-
"req_ref": "REQ-{NNN}:AC-{N}",
|
|
93
|
-
"layer": "L1|L2|L3",
|
|
94
|
-
"priority": "critical|high|medium",
|
|
95
|
-
"name": "descriptive name derived from AC text",
|
|
96
|
-
"category": "api_contract|business_rule|state_transition|user_flow|non_functional",
|
|
97
|
-
"endpoint": "METHOD /path (if L1)",
|
|
98
|
-
"input": { "$fixture_ref": "REQ-{NNN}/valid|invalid|boundary/name" },
|
|
99
|
-
"expected": {
|
|
100
|
-
"status": 200,
|
|
101
|
-
"body_contains": [],
|
|
102
|
-
"behavior": "description of expected outcome"
|
|
103
|
-
},
|
|
104
|
-
"preconditions": ["list of required state"],
|
|
105
|
-
"postconditions": ["list of expected state changes"],
|
|
106
|
-
"mock_services": ["list of services to mock for isolation"]
|
|
107
|
-
}
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
**Degraded mode (SPEC_MODE = "degraded"):**
|
|
111
|
-
|
|
112
|
-
- Extract scenarios from `index.json.success_criteria` (each criterion -> one L2 scenario)
|
|
113
|
-
- Extract from `plan.json` task convergence criteria (each criterion -> one L1 or L2 scenario)
|
|
114
|
-
- Extract from summaries (each implemented behavior -> one L1 scenario)
|
|
115
|
-
- All scenarios default to priority: "high"
|
|
116
|
-
- No L3 scenarios in degraded mode (no Epic stories available)
|
|
117
|
-
|
|
118
|
-
---
|
|
119
|
-
|
|
120
|
-
### Step 3: Generate Test Data (Fixtures)
|
|
121
|
-
|
|
122
|
-
For each REQ with data model definitions:
|
|
123
|
-
|
|
124
|
-
**Tier 1: Schema-derived fixtures**
|
|
125
|
-
```
|
|
126
|
-
FOR each entity in REQ data model:
|
|
127
|
-
valid: object satisfying all field constraints
|
|
128
|
-
invalid: one variant per constraint violation:
|
|
129
|
-
- null_required: set required field to null
|
|
130
|
-
- empty_string: set string field to ""
|
|
131
|
-
- overflow: exceed max length or max value
|
|
132
|
-
- wrong_type: provide string where number expected, etc.
|
|
133
|
-
boundary: edge value variants:
|
|
134
|
-
- min_value: exact minimum
|
|
135
|
-
- max_value: exact maximum
|
|
136
|
-
- min_minus_one: minimum - 1
|
|
137
|
-
- max_plus_one: maximum + 1
|
|
138
|
-
```
|
|
139
|
-
|
|
140
|
-
**Tier 2: Acceptance-criteria-derived expectations**
|
|
141
|
-
```
|
|
142
|
-
FOR each "MUST return X when Y" in acceptance criteria:
|
|
143
|
-
fixture: { input: Y, expected: X }
|
|
144
|
-
FOR each "MUST validate Z":
|
|
145
|
-
fixture: { input: invalid_Z, expected: validation_error }
|
|
146
|
-
FOR each "SHOULD support W":
|
|
147
|
-
fixture: { input: W_params, expected: W_result }
|
|
148
|
-
```
|
|
149
|
-
|
|
150
|
-
**Tier 3: Business-scenario-derived data sets (L3 only)**
|
|
151
|
-
```
|
|
152
|
-
FOR each Epic user story:
|
|
153
|
-
scenario_pack: coordinated data set across story steps
|
|
154
|
-
Example: { user: registered_user, order: valid_order, payment: valid_card }
|
|
155
|
-
Entity IDs and relationships pre-coordinated across steps
|
|
156
|
-
```
|
|
157
|
-
|
|
158
|
-
**Microservice mock data:**
|
|
159
|
-
```
|
|
160
|
-
FOR each API endpoint in architecture/_index.md:
|
|
161
|
-
mock_contract: { request_pattern, response_fixture }
|
|
162
|
-
Used for: L2 WireMock stubs, L3 service isolation
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
Organize fixtures by REQ:
|
|
166
|
-
```json
|
|
167
|
-
{
|
|
168
|
-
"REQ-001": {
|
|
169
|
-
"valid": [{ "name": "standard_user", "data": {...} }],
|
|
170
|
-
"invalid": [{ "name": "missing_email", "data": {...}, "expected_error": "email is required" }],
|
|
171
|
-
"boundary": [{ "name": "max_length_name", "data": {...} }]
|
|
172
|
-
}
|
|
173
|
-
}
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
---
|
|
177
|
-
|
|
178
|
-
### Step 4: Write Test Plan & Confirm
|
|
179
|
-
|
|
180
|
-
**Archive previous business test artifacts** before writing:
|
|
181
|
-
```
|
|
182
|
-
IF file exists "${PHASE_DIR}/.tests/business/business-test-plan.json":
|
|
183
|
-
mkdir -p "${PHASE_DIR}/.history"
|
|
184
|
-
TIMESTAMP = format(now(), "YYYY-MM-DDTHH-mm-ss")
|
|
185
|
-
mv "${PHASE_DIR}/.tests/business/business-test-plan.json" "${PHASE_DIR}/.history/business-test-plan-${TIMESTAMP}.json"
|
|
186
|
-
```
|
|
187
|
-
|
|
188
|
-
Write `business-test-plan.json` to `.tests/business/`:
|
|
189
|
-
```json
|
|
190
|
-
{
|
|
191
|
-
"phase": "{phase}",
|
|
192
|
-
"blueprint_ref": "{SPEC_DIR name or 'degraded'}",
|
|
193
|
-
"spec_mode": "full|degraded",
|
|
194
|
-
"generated_at": "{ISO timestamp}",
|
|
195
|
-
"layers": {
|
|
196
|
-
"L1": { "scenario_count": N, "priority_distribution": { "critical": X, "high": Y, "medium": Z } },
|
|
197
|
-
"L2": { "scenario_count": N, "priority_distribution": {...} },
|
|
198
|
-
"L3": { "scenario_count": N, "priority_distribution": {...} }
|
|
199
|
-
},
|
|
200
|
-
"scenarios": [ ... ],
|
|
201
|
-
"fixtures": { ... },
|
|
202
|
-
"mock_contracts": [ ... ],
|
|
203
|
-
"requirement_coverage_plan": {
|
|
204
|
-
"requirements_targeted": ["REQ-001", "REQ-002"],
|
|
205
|
-
"requirements_skipped": [],
|
|
206
|
-
"skip_reasons": {}
|
|
207
|
-
}
|
|
208
|
-
}
|
|
209
|
-
```
|
|
210
|
-
|
|
211
|
-
**If not `--auto`:**
|
|
212
|
-
|
|
213
|
-
Display plan summary:
|
|
214
|
-
```
|
|
215
|
-
=== BUSINESS TEST PLAN ===
|
|
216
|
-
Spec mode: {full|degraded}
|
|
217
|
-
Requirements: {N} targeted, {M} skipped
|
|
218
|
-
|
|
219
|
-
Layer Distribution:
|
|
220
|
-
L1 Interface: {N} scenarios ({X} critical, {Y} high)
|
|
221
|
-
L2 Business: {N} scenarios ({X} critical, {Y} high)
|
|
222
|
-
L3 E2E: {N} scenarios ({X} critical, {Y} high)
|
|
223
|
-
|
|
224
|
-
Fixtures: {N} REQs × {avg} variants = {total} data sets
|
|
225
|
-
|
|
226
|
-
Proceed? (yes/edit/cancel)
|
|
227
|
-
```
|
|
228
|
-
|
|
229
|
-
Wait for user confirmation. If "edit": let user modify plan interactively.
|
|
230
|
-
|
|
231
|
-
**If `--dry-run`:** Stop here, report plan.
|
|
232
|
-
|
|
233
|
-
---
|
|
234
|
-
|
|
235
|
-
### Step 5: Generate Test Code (if --gen-code)
|
|
236
|
-
|
|
237
|
-
**Detect project tech stack:**
|
|
238
|
-
```
|
|
239
|
-
IF file exists ".workflow/project.md":
|
|
240
|
-
tech = parse Tech Stack section from project.md
|
|
241
|
-
ELSE:
|
|
242
|
-
Scan project for indicators (pom.xml -> Java, package.json -> Node, etc.)
|
|
243
|
-
```
|
|
244
|
-
|
|
245
|
-
**Generate framework-specific test classes:**
|
|
246
|
-
|
|
247
|
-
| Stack | L1 | L2 | L3 |
|
|
248
|
-
|-------|----|----|-----|
|
|
249
|
-
| Java/Spring Boot | RestAssured + MockMvc | JUnit 5 Parameterized + WireMock | TestContainers |
|
|
250
|
-
| TypeScript/Node | supertest + vitest | vitest + nock | playwright/cypress |
|
|
251
|
-
| Python | httpx + pytest | pytest + responses | pytest + selenium |
|
|
252
|
-
|
|
253
|
-
**Generation rules:**
|
|
254
|
-
- Follow existing test patterns (discover from codebase, same as test-gen Step 1)
|
|
255
|
-
- Each test method has `@DisplayName` (or equivalent) with REQ-NNN:AC-N reference
|
|
256
|
-
- Each test class has layer marker (`@Tag("business-L1")` or equivalent)
|
|
257
|
-
- Test files placed in `.tests/business/{layer}/` directory
|
|
258
|
-
|
|
259
|
-
**If no `--gen-code`:** Scenarios remain as structured JSON for AI agent execution mode (Step 6).
|
|
260
|
-
|
|
261
|
-
---
|
|
262
|
-
|
|
263
|
-
### Step 6: Execute Tests (Progressive L1 -> L2 -> L3)
|
|
264
|
-
|
|
265
|
-
**Fail-fast rule:**
|
|
266
|
-
- If L1 has ANY "critical" priority failure -> STOP. Do not proceed to L2.
|
|
267
|
-
- If L2 has ANY "critical" priority failure -> STOP. Do not proceed to L3.
|
|
268
|
-
- "high" and "medium" failures do NOT block next layer.
|
|
269
|
-
|
|
270
|
-
**For each layer (starting from `--layer` if specified, else L1):**
|
|
271
|
-
|
|
272
|
-
Run Generator-Critic loop (max 3 iterations):
|
|
273
|
-
|
|
274
|
-
**Iteration 1: Execute all scenarios**
|
|
275
|
-
|
|
276
|
-
`--gen-code` mode:
|
|
277
|
-
```bash
|
|
278
|
-
# Run generated test classes for current layer
|
|
279
|
-
{test_command} --testPathPattern="business-{layer}" 2>&1 | tail -50
|
|
280
|
-
```
|
|
281
|
-
|
|
282
|
-
Agent execution mode:
|
|
283
|
-
```
|
|
284
|
-
FOR each scenario in current layer:
|
|
285
|
-
Execute scenario against running application
|
|
286
|
-
Record: { status, actual_response, duration_ms, error_detail }
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
**Critic phase: Classify failures**
|
|
290
|
-
|
|
291
|
-
For each failed scenario:
|
|
292
|
-
```
|
|
293
|
-
Analyze failure evidence -> classify as:
|
|
294
|
-
|
|
295
|
-
| Classification | Meaning | Action |
|
|
296
|
-
|---------------|---------|--------|
|
|
297
|
-
| test_defect | Test itself is wrong (wrong endpoint, bad fixture) | Auto-fix test in next iteration |
|
|
298
|
-
| code_defect | Business rule violated (actual != expected per REQ) | Record as failure |
|
|
299
|
-
| env_issue | Service down, config missing, timeout | Record as blocked |
|
|
300
|
-
```
|
|
301
|
-
|
|
302
|
-
**Iteration 2: Re-run with fixed tests**
|
|
303
|
-
- Fix test_defects from iteration 1
|
|
304
|
-
- Re-run ALL scenarios (not just failed ones — catch regressions)
|
|
305
|
-
|
|
306
|
-
**Iteration 3: Final confirmation**
|
|
307
|
-
- Remaining failures = confirmed code_defects
|
|
308
|
-
- All test_defects should be resolved
|
|
309
|
-
- Blocked scenarios marked with env_issue reason
|
|
310
|
-
|
|
311
|
-
Record results per iteration in `.tests/business/test-results-iter-{N}.json`.
|
|
312
|
-
|
|
313
|
-
---
|
|
314
|
-
|
|
315
|
-
### Step 7: Build Traceability Matrix
|
|
316
|
-
|
|
317
|
-
Map each scenario result back to requirement:
|
|
318
|
-
|
|
319
|
-
```
|
|
320
|
-
FOR each REQ in requirements targeted:
|
|
321
|
-
req_results = {
|
|
322
|
-
req_id: "REQ-NNN",
|
|
323
|
-
title: REQ.title,
|
|
324
|
-
total_criteria: count of AC in REQ,
|
|
325
|
-
criteria_results: []
|
|
326
|
-
}
|
|
327
|
-
|
|
328
|
-
FOR each AC in REQ:
|
|
329
|
-
scenarios_for_ac = filter scenarios where req_ref == "REQ-NNN:AC-{N}"
|
|
330
|
-
ac_status = "passed" if ALL scenarios passed
|
|
331
|
-
"failed" if ANY scenario failed
|
|
332
|
-
"blocked" if ANY blocked and none failed
|
|
333
|
-
"untested" if no scenarios mapped
|
|
334
|
-
|
|
335
|
-
req_results.criteria_results.push({
|
|
336
|
-
ac_id: "AC-{N}",
|
|
337
|
-
description: AC text,
|
|
338
|
-
priority: mapped from RFC 2119 keyword,
|
|
339
|
-
status: ac_status,
|
|
340
|
-
tests: [scenario IDs],
|
|
341
|
-
failure: { expected, actual, evidence } if failed
|
|
342
|
-
})
|
|
343
|
-
|
|
344
|
-
req_results.coverage_pct = passed_criteria / total_criteria * 100
|
|
345
|
-
req_results.verdict = "verified" if all MUST+SHOULD passed
|
|
346
|
-
"partial" if some failed
|
|
347
|
-
"unverified" if all failed or untested
|
|
348
|
-
```
|
|
349
|
-
|
|
350
|
-
---
|
|
351
|
-
|
|
352
|
-
### Step 8: Generate Reports
|
|
353
|
-
|
|
354
|
-
**Archive previous reports:**
|
|
355
|
-
```
|
|
356
|
-
IF file exists "${PHASE_DIR}/.tests/business/business-test-report.json":
|
|
357
|
-
mkdir -p "${PHASE_DIR}/.history"
|
|
358
|
-
TIMESTAMP = format(now(), "YYYY-MM-DDTHH-mm-ss")
|
|
359
|
-
mv report and summary to .history/
|
|
360
|
-
```
|
|
361
|
-
|
|
362
|
-
Write `.tests/business/business-test-report.json`:
|
|
363
|
-
```json
|
|
364
|
-
{
|
|
365
|
-
"phase": "{phase}",
|
|
366
|
-
"blueprint_ref": "{spec reference}",
|
|
367
|
-
"spec_mode": "full|degraded",
|
|
368
|
-
"completed_at": "{ISO timestamp}",
|
|
369
|
-
"execution_mode": "gen-code|agent",
|
|
370
|
-
"iterations": {
|
|
371
|
-
"L1": { "count": N, "converged": true },
|
|
372
|
-
"L2": { "count": N, "converged": true },
|
|
373
|
-
"L3": { "count": N, "converged": false }
|
|
374
|
-
},
|
|
375
|
-
"layers": {
|
|
376
|
-
"L1": { "total": N, "passed": P, "failed": F, "blocked": B, "pass_rate": 95.0 },
|
|
377
|
-
"L2": { "total": N, "passed": P, "failed": F, "blocked": B, "pass_rate": 87.5 },
|
|
378
|
-
"L3": { "total": N, "passed": P, "failed": F, "blocked": B, "pass_rate": 100.0 }
|
|
379
|
-
},
|
|
380
|
-
"requirement_coverage": [
|
|
381
|
-
{
|
|
382
|
-
"req_id": "REQ-001",
|
|
383
|
-
"title": "...",
|
|
384
|
-
"total_criteria": 5,
|
|
385
|
-
"criteria_results": [
|
|
386
|
-
{
|
|
387
|
-
"ac_id": "AC-1",
|
|
388
|
-
"description": "...",
|
|
389
|
-
"priority": "critical",
|
|
390
|
-
"status": "passed|failed|blocked|untested",
|
|
391
|
-
"tests": ["BT-001", "BT-002"],
|
|
392
|
-
"failure": null
|
|
393
|
-
}
|
|
394
|
-
],
|
|
395
|
-
"coverage_pct": 80.0,
|
|
396
|
-
"verdict": "verified|partial|unverified"
|
|
397
|
-
}
|
|
398
|
-
],
|
|
399
|
-
"failures": [
|
|
400
|
-
{
|
|
401
|
-
"id": "BF-001",
|
|
402
|
-
"test_id": "BT-005",
|
|
403
|
-
"req_ref": "REQ-001:AC-3",
|
|
404
|
-
"layer": "L1",
|
|
405
|
-
"severity": "critical",
|
|
406
|
-
"classification": "code_defect",
|
|
407
|
-
"description": "...",
|
|
408
|
-
"expected": "...",
|
|
409
|
-
"actual": "...",
|
|
410
|
-
"fix_suggestion": {
|
|
411
|
-
"file": "src/...",
|
|
412
|
-
"line": 42,
|
|
413
|
-
"direction": "Add null guard for..."
|
|
414
|
-
}
|
|
415
|
-
}
|
|
416
|
-
],
|
|
417
|
-
"summary": {
|
|
418
|
-
"total_requirements": 10,
|
|
419
|
-
"fully_verified": 8,
|
|
420
|
-
"partially_verified": 1,
|
|
421
|
-
"unverified": 1,
|
|
422
|
-
"coverage_pct": 85.0,
|
|
423
|
-
"total_scenarios": 30,
|
|
424
|
-
"total_passed": 26,
|
|
425
|
-
"total_failed": 3,
|
|
426
|
-
"total_blocked": 1
|
|
427
|
-
}
|
|
428
|
-
}
|
|
429
|
-
```
|
|
430
|
-
|
|
431
|
-
Write `.tests/business/business-test-summary.md`:
|
|
432
|
-
```markdown
|
|
433
|
-
---
|
|
434
|
-
phase: {phase}
|
|
435
|
-
blueprint_ref: {spec reference}
|
|
436
|
-
completed_at: {ISO timestamp}
|
|
437
|
-
verdict: passed|gaps_found
|
|
438
|
-
---
|
|
439
|
-
|
|
440
|
-
# Business Test Results
|
|
441
|
-
|
|
442
|
-
## Requirement Coverage
|
|
443
|
-
|
|
444
|
-
| REQ | Title | AC Total | Passed | Failed | Coverage | Verdict |
|
|
445
|
-
|-----|-------|----------|--------|--------|----------|---------|
|
|
446
|
-
| REQ-001 | ... | 5 | 4 | 1 | 80% | partial |
|
|
447
|
-
| REQ-002 | ... | 3 | 3 | 0 | 100% | verified |
|
|
448
|
-
|
|
449
|
-
## Layer Results
|
|
450
|
-
|
|
451
|
-
| Layer | Total | Passed | Failed | Blocked | Pass Rate |
|
|
452
|
-
|-------|-------|--------|--------|---------|-----------|
|
|
453
|
-
| L1 Interface | 10 | 9 | 1 | 0 | 90.0% |
|
|
454
|
-
| L2 Business | 15 | 13 | 1 | 1 | 86.7% |
|
|
455
|
-
| L3 E2E | 5 | 5 | 0 | 0 | 100.0% |
|
|
456
|
-
|
|
457
|
-
## Failures
|
|
458
|
-
|
|
459
|
-
### BF-001: REQ-001:AC-3 (critical)
|
|
460
|
-
- Layer: L1
|
|
461
|
-
- Expected: 201 Created with user object
|
|
462
|
-
- Actual: 400 Bad Request
|
|
463
|
-
- Fix: Add email validation bypass for internal accounts (src/auth.ts:42)
|
|
464
|
-
|
|
465
|
-
## Next Steps
|
|
466
|
-
{routing suggestion}
|
|
467
|
-
```
|
|
468
|
-
|
|
469
|
-
Update `index.json` with business_test section:
|
|
470
|
-
```json
|
|
471
|
-
{
|
|
472
|
-
"business_test": {
|
|
473
|
-
"status": "passed|gaps_found",
|
|
474
|
-
"spec_mode": "full|degraded",
|
|
475
|
-
"req_coverage_pct": 85.0,
|
|
476
|
-
"layers": {
|
|
477
|
-
"L1": { "pass_rate": 90.0 },
|
|
478
|
-
"L2": { "pass_rate": 86.7 },
|
|
479
|
-
"L3": { "pass_rate": 100.0 }
|
|
480
|
-
},
|
|
481
|
-
"failures": [
|
|
482
|
-
{ "id": "BF-001", "req_ref": "REQ-001:AC-3", "severity": "critical" }
|
|
483
|
-
]
|
|
484
|
-
}
|
|
485
|
-
}
|
|
486
|
-
```
|
|
487
|
-
|
|
488
|
-
---
|
|
489
|
-
|
|
490
|
-
### Step 9: Feedback Loop
|
|
491
|
-
|
|
492
|
-
**Auto-create issues from failures:**
|
|
493
|
-
```
|
|
494
|
-
FOR each failure in report.failures:
|
|
495
|
-
mkdir -p ".workflow/issues"
|
|
496
|
-
|
|
497
|
-
today = format(now(), "YYYYMMDD")
|
|
498
|
-
counter = next available sequence for today
|
|
499
|
-
|
|
500
|
-
issue = {
|
|
501
|
-
id: "ISS-{today}-{counter:03d}",
|
|
502
|
-
title: "Business Test: " + failure.req_ref + " - " + failure.description (truncated 100 chars),
|
|
503
|
-
status: "registered",
|
|
504
|
-
priority: severity_to_priority(failure.severity),
|
|
505
|
-
severity: failure.severity,
|
|
506
|
-
source: "business-test",
|
|
507
|
-
phase_ref: PHASE_NUM,
|
|
508
|
-
gap_ref: failure.id,
|
|
509
|
-
description: "Business test failed for " + failure.req_ref + ". Expected: " + failure.expected + ". Actual: " + failure.actual,
|
|
510
|
-
fix_direction: failure.fix_suggestion.direction,
|
|
511
|
-
context: {
|
|
512
|
-
location: failure.fix_suggestion.file + ":" + failure.fix_suggestion.line,
|
|
513
|
-
suggested_fix: failure.fix_suggestion.direction,
|
|
514
|
-
notes: "req_ref: " + failure.req_ref + ", layer: " + failure.layer
|
|
515
|
-
},
|
|
516
|
-
tags: ["business-test", failure.layer],
|
|
517
|
-
affected_components: [failure.fix_suggestion.file],
|
|
518
|
-
feedback: [],
|
|
519
|
-
issue_history: [],
|
|
520
|
-
created_at: now(),
|
|
521
|
-
updated_at: now(),
|
|
522
|
-
resolved_at: null,
|
|
523
|
-
resolution: null
|
|
524
|
-
}
|
|
525
|
-
Append JSON line to .workflow/issues/issues.jsonl
|
|
526
|
-
```
|
|
527
|
-
|
|
528
|
-
**Report:**
|
|
529
|
-
```
|
|
530
|
-
=== BUSINESS TEST RESULTS ===
|
|
531
|
-
Phase: {phase_name}
|
|
532
|
-
Spec mode: {full|degraded}
|
|
533
|
-
|
|
534
|
-
Requirement Coverage: {coverage_pct}%
|
|
535
|
-
Verified: {fully_verified}/{total_requirements}
|
|
536
|
-
Partial: {partially_verified}
|
|
537
|
-
Unverified: {unverified}
|
|
538
|
-
|
|
539
|
-
Layer Results:
|
|
540
|
-
L1 Interface: {pass_rate}% ({passed}/{total})
|
|
541
|
-
L2 Business: {pass_rate}% ({passed}/{total})
|
|
542
|
-
L3 E2E: {pass_rate}% ({passed}/{total})
|
|
543
|
-
|
|
544
|
-
Failures: {failure_count} ({blocker_count} blockers)
|
|
545
|
-
Issues: {issue_count} auto-created
|
|
546
|
-
|
|
547
|
-
Files:
|
|
548
|
-
{PHASE_DIR}/.tests/business/business-test-plan.json
|
|
549
|
-
{PHASE_DIR}/.tests/business/business-test-report.json
|
|
550
|
-
{PHASE_DIR}/.tests/business/business-test-summary.md
|
|
551
|
-
|
|
552
|
-
Next steps:
|
|
553
|
-
{suggested_next_command}
|
|
554
|
-
```
|
|
555
|
-
|
|
556
|
-
**Next step routing:**
|
|
557
|
-
|
|
558
|
-
| Result | Suggestion |
|
|
559
|
-
|--------|------------|
|
|
560
|
-
| All requirements verified | Skill({ skill: "maestro-phase-transition", args: "{phase}" }) |
|
|
561
|
-
| Failures found | Skill({ skill: "quality-debug", args: "--from-business-test {phase}" }) |
|
|
562
|
-
| `--re-run` all pass after fix | Skill({ skill: "maestro-execute", args: "{phase}" }) |
|
|
563
|
-
| Low coverage (< 60%) | Skill({ skill: "quality-auto-test", args: "{phase}" }) |
|
|
564
|
-
| Need integration tests | Skill({ skill: "quality-auto-test", args: "{phase}" }) |
|
|
565
|
-
|
|
566
|
-
**Closure criteria:**
|
|
567
|
-
A requirement is marked "verified" ONLY when:
|
|
568
|
-
- ALL acceptance criteria with MUST/SHALL keywords: passed
|
|
569
|
-
- ALL acceptance criteria with SHOULD keywords: passed
|
|
570
|
-
- No blocker-severity failures remain for this requirement
|
|
1
|
+
# Business Test Workflow (PRD-Forward)
|
|
2
|
+
|
|
3
|
+
Validate built features against PRD acceptance criteria through automated multi-layer business testing with requirement traceability, fixture generation, and feedback loop.
|
|
4
|
+
|
|
5
|
+
PRD-forward: starts from REQ-*.md acceptance criteria, not from code coverage gaps.
|
|
6
|
+
Progressive layers: L1 Interface Contract -> L2 Business Rule -> L3 Business Scenario (E2E).
|
|
7
|
+
Generator-Critic loop: max 3 iterations per layer to distinguish test defects from code defects.
|
|
8
|
+
|
|
9
|
+
**Philosophy: Business rules are the source of truth. Code must satisfy them, not the other way around.**
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
### Step 1: Resolve Target & Load Spec Package
|
|
14
|
+
|
|
15
|
+
**Parse arguments:**
|
|
16
|
+
|
|
17
|
+
| Input | Result |
|
|
18
|
+
|-------|--------|
|
|
19
|
+
| No arguments | Error E001 |
|
|
20
|
+
| Phase number | Resolve `.workflow/phases/{NN}-{slug}/` |
|
|
21
|
+
| `--spec SPEC-xxx` | Explicit spec reference |
|
|
22
|
+
| `--layer L1\|L2\|L3` | Run only specific layer |
|
|
23
|
+
| `--gen-code` | Generate framework-specific test classes |
|
|
24
|
+
| `--dry-run` | Extract scenarios only, don't execute |
|
|
25
|
+
| `--re-run` | Re-run only previously failed scenarios |
|
|
26
|
+
| `--auto` | Skip interactive confirmations |
|
|
27
|
+
|
|
28
|
+
**Load spec package:**
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
1. Read ${PHASE_DIR}/index.json -> extract blueprint_ref (if present)
|
|
32
|
+
2. IF --spec provided: SPEC_DIR = .workflow/blueprint/{blueprint_ref}/
|
|
33
|
+
ELSE IF index.json.blueprint_ref: SPEC_DIR = .workflow/blueprint/{blueprint_ref}/
|
|
34
|
+
ELSE: try .workflow/blueprint/SPEC-*/ (most recent)
|
|
35
|
+
|
|
36
|
+
3. IF SPEC_DIR found:
|
|
37
|
+
- Read requirements/_index.md (requirement summary + traceability matrix)
|
|
38
|
+
- Read all requirements/REQ-*.md (functional requirements with acceptance criteria)
|
|
39
|
+
- Read all requirements/NFR-*.md (non-functional requirements)
|
|
40
|
+
- Read architecture/_index.md (API endpoints, data model, state machines)
|
|
41
|
+
- Read epics/EPIC-*.md (user stories -> E2E scenario source)
|
|
42
|
+
SPEC_MODE = "full"
|
|
43
|
+
|
|
44
|
+
4. IF no spec package found (DEGRADED MODE):
|
|
45
|
+
- Read index.json.success_criteria
|
|
46
|
+
- Read plan.json tasks with convergence.criteria
|
|
47
|
+
- Read .summaries/TASK-*-summary.md for implemented behavior
|
|
48
|
+
SPEC_MODE = "degraded"
|
|
49
|
+
Display: "No spec package found. Using success_criteria + plan.json for scenario extraction (degraded mode)."
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
Check for existing business test session:
|
|
53
|
+
```bash
|
|
54
|
+
ls ${PHASE_DIR}/.tests/business/business-test-report.json 2>/dev/null
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
If session exists AND `--re-run`: load previous report, filter to failed/blocked scenarios only.
|
|
58
|
+
If session exists AND no `--re-run`: offer resume or restart.
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
### Step 2: Extract Business Test Scenarios from PRD
|
|
63
|
+
|
|
64
|
+
**Full mode (SPEC_MODE = "full"):**
|
|
65
|
+
|
|
66
|
+
For each `REQ-NNN-{slug}.md`:
|
|
67
|
+
|
|
68
|
+
1. Parse `## Acceptance Criteria` section
|
|
69
|
+
2. Extract each numbered criterion as a test scenario seed
|
|
70
|
+
3. Map RFC 2119 keywords to test priority:
|
|
71
|
+
|
|
72
|
+
| Keyword | Priority | Failure Severity |
|
|
73
|
+
|---------|----------|-----------------|
|
|
74
|
+
| MUST / MUST NOT / SHALL / SHALL NOT | critical | blocker |
|
|
75
|
+
| SHOULD / SHOULD NOT / RECOMMENDED | high | major |
|
|
76
|
+
| MAY / OPTIONAL | medium | minor |
|
|
77
|
+
|
|
78
|
+
4. Classify scenario into layer:
|
|
79
|
+
|
|
80
|
+
| Source | Layer | Category |
|
|
81
|
+
|--------|-------|----------|
|
|
82
|
+
| Architecture API endpoints + REQ AC about request/response | L1 | api_contract |
|
|
83
|
+
| REQ AC about business logic, validation, state changes | L2 | business_rule |
|
|
84
|
+
| REQ AC about state transitions (from architecture state machines) | L2 | state_transition |
|
|
85
|
+
| Epic user stories (multi-step flows) | L3 | user_flow |
|
|
86
|
+
| NFR performance/security constraints | L2 | non_functional |
|
|
87
|
+
|
|
88
|
+
5. Generate scenario:
|
|
89
|
+
```json
|
|
90
|
+
{
|
|
91
|
+
"id": "BT-{NNN}",
|
|
92
|
+
"req_ref": "REQ-{NNN}:AC-{N}",
|
|
93
|
+
"layer": "L1|L2|L3",
|
|
94
|
+
"priority": "critical|high|medium",
|
|
95
|
+
"name": "descriptive name derived from AC text",
|
|
96
|
+
"category": "api_contract|business_rule|state_transition|user_flow|non_functional",
|
|
97
|
+
"endpoint": "METHOD /path (if L1)",
|
|
98
|
+
"input": { "$fixture_ref": "REQ-{NNN}/valid|invalid|boundary/name" },
|
|
99
|
+
"expected": {
|
|
100
|
+
"status": 200,
|
|
101
|
+
"body_contains": [],
|
|
102
|
+
"behavior": "description of expected outcome"
|
|
103
|
+
},
|
|
104
|
+
"preconditions": ["list of required state"],
|
|
105
|
+
"postconditions": ["list of expected state changes"],
|
|
106
|
+
"mock_services": ["list of services to mock for isolation"]
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
**Degraded mode (SPEC_MODE = "degraded"):**
|
|
111
|
+
|
|
112
|
+
- Extract scenarios from `index.json.success_criteria` (each criterion -> one L2 scenario)
|
|
113
|
+
- Extract from `plan.json` task convergence criteria (each criterion -> one L1 or L2 scenario)
|
|
114
|
+
- Extract from summaries (each implemented behavior -> one L1 scenario)
|
|
115
|
+
- All scenarios default to priority: "high"
|
|
116
|
+
- No L3 scenarios in degraded mode (no Epic stories available)
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
### Step 3: Generate Test Data (Fixtures)
|
|
121
|
+
|
|
122
|
+
For each REQ with data model definitions:
|
|
123
|
+
|
|
124
|
+
**Tier 1: Schema-derived fixtures**
|
|
125
|
+
```
|
|
126
|
+
FOR each entity in REQ data model:
|
|
127
|
+
valid: object satisfying all field constraints
|
|
128
|
+
invalid: one variant per constraint violation:
|
|
129
|
+
- null_required: set required field to null
|
|
130
|
+
- empty_string: set string field to ""
|
|
131
|
+
- overflow: exceed max length or max value
|
|
132
|
+
- wrong_type: provide string where number expected, etc.
|
|
133
|
+
boundary: edge value variants:
|
|
134
|
+
- min_value: exact minimum
|
|
135
|
+
- max_value: exact maximum
|
|
136
|
+
- min_minus_one: minimum - 1
|
|
137
|
+
- max_plus_one: maximum + 1
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
**Tier 2: Acceptance-criteria-derived expectations**
|
|
141
|
+
```
|
|
142
|
+
FOR each "MUST return X when Y" in acceptance criteria:
|
|
143
|
+
fixture: { input: Y, expected: X }
|
|
144
|
+
FOR each "MUST validate Z":
|
|
145
|
+
fixture: { input: invalid_Z, expected: validation_error }
|
|
146
|
+
FOR each "SHOULD support W":
|
|
147
|
+
fixture: { input: W_params, expected: W_result }
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
**Tier 3: Business-scenario-derived data sets (L3 only)**
|
|
151
|
+
```
|
|
152
|
+
FOR each Epic user story:
|
|
153
|
+
scenario_pack: coordinated data set across story steps
|
|
154
|
+
Example: { user: registered_user, order: valid_order, payment: valid_card }
|
|
155
|
+
Entity IDs and relationships pre-coordinated across steps
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
**Microservice mock data:**
|
|
159
|
+
```
|
|
160
|
+
FOR each API endpoint in architecture/_index.md:
|
|
161
|
+
mock_contract: { request_pattern, response_fixture }
|
|
162
|
+
Used for: L2 WireMock stubs, L3 service isolation
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
Organize fixtures by REQ:
|
|
166
|
+
```json
|
|
167
|
+
{
|
|
168
|
+
"REQ-001": {
|
|
169
|
+
"valid": [{ "name": "standard_user", "data": {...} }],
|
|
170
|
+
"invalid": [{ "name": "missing_email", "data": {...}, "expected_error": "email is required" }],
|
|
171
|
+
"boundary": [{ "name": "max_length_name", "data": {...} }]
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
### Step 4: Write Test Plan & Confirm
|
|
179
|
+
|
|
180
|
+
**Archive previous business test artifacts** before writing:
|
|
181
|
+
```
|
|
182
|
+
IF file exists "${PHASE_DIR}/.tests/business/business-test-plan.json":
|
|
183
|
+
mkdir -p "${PHASE_DIR}/.history"
|
|
184
|
+
TIMESTAMP = format(now(), "YYYY-MM-DDTHH-mm-ss")
|
|
185
|
+
mv "${PHASE_DIR}/.tests/business/business-test-plan.json" "${PHASE_DIR}/.history/business-test-plan-${TIMESTAMP}.json"
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Write `business-test-plan.json` to `.tests/business/`:
|
|
189
|
+
```json
|
|
190
|
+
{
|
|
191
|
+
"phase": "{phase}",
|
|
192
|
+
"blueprint_ref": "{SPEC_DIR name or 'degraded'}",
|
|
193
|
+
"spec_mode": "full|degraded",
|
|
194
|
+
"generated_at": "{ISO timestamp}",
|
|
195
|
+
"layers": {
|
|
196
|
+
"L1": { "scenario_count": N, "priority_distribution": { "critical": X, "high": Y, "medium": Z } },
|
|
197
|
+
"L2": { "scenario_count": N, "priority_distribution": {...} },
|
|
198
|
+
"L3": { "scenario_count": N, "priority_distribution": {...} }
|
|
199
|
+
},
|
|
200
|
+
"scenarios": [ ... ],
|
|
201
|
+
"fixtures": { ... },
|
|
202
|
+
"mock_contracts": [ ... ],
|
|
203
|
+
"requirement_coverage_plan": {
|
|
204
|
+
"requirements_targeted": ["REQ-001", "REQ-002"],
|
|
205
|
+
"requirements_skipped": [],
|
|
206
|
+
"skip_reasons": {}
|
|
207
|
+
}
|
|
208
|
+
}
|
|
209
|
+
```
|
|
210
|
+
|
|
211
|
+
**If not `--auto`:**
|
|
212
|
+
|
|
213
|
+
Display plan summary:
|
|
214
|
+
```
|
|
215
|
+
=== BUSINESS TEST PLAN ===
|
|
216
|
+
Spec mode: {full|degraded}
|
|
217
|
+
Requirements: {N} targeted, {M} skipped
|
|
218
|
+
|
|
219
|
+
Layer Distribution:
|
|
220
|
+
L1 Interface: {N} scenarios ({X} critical, {Y} high)
|
|
221
|
+
L2 Business: {N} scenarios ({X} critical, {Y} high)
|
|
222
|
+
L3 E2E: {N} scenarios ({X} critical, {Y} high)
|
|
223
|
+
|
|
224
|
+
Fixtures: {N} REQs × {avg} variants = {total} data sets
|
|
225
|
+
|
|
226
|
+
Proceed? (yes/edit/cancel)
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Wait for user confirmation. If "edit": let user modify plan interactively.
|
|
230
|
+
|
|
231
|
+
**If `--dry-run`:** Stop here, report plan.
|
|
232
|
+
|
|
233
|
+
---
|
|
234
|
+
|
|
235
|
+
### Step 5: Generate Test Code (if --gen-code)
|
|
236
|
+
|
|
237
|
+
**Detect project tech stack:**
|
|
238
|
+
```
|
|
239
|
+
IF file exists ".workflow/project.md":
|
|
240
|
+
tech = parse Tech Stack section from project.md
|
|
241
|
+
ELSE:
|
|
242
|
+
Scan project for indicators (pom.xml -> Java, package.json -> Node, etc.)
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
**Generate framework-specific test classes:**
|
|
246
|
+
|
|
247
|
+
| Stack | L1 | L2 | L3 |
|
|
248
|
+
|-------|----|----|-----|
|
|
249
|
+
| Java/Spring Boot | RestAssured + MockMvc | JUnit 5 Parameterized + WireMock | TestContainers |
|
|
250
|
+
| TypeScript/Node | supertest + vitest | vitest + nock | playwright/cypress |
|
|
251
|
+
| Python | httpx + pytest | pytest + responses | pytest + selenium |
|
|
252
|
+
|
|
253
|
+
**Generation rules:**
|
|
254
|
+
- Follow existing test patterns (discover from codebase, same as test-gen Step 1)
|
|
255
|
+
- Each test method has `@DisplayName` (or equivalent) with REQ-NNN:AC-N reference
|
|
256
|
+
- Each test class has layer marker (`@Tag("business-L1")` or equivalent)
|
|
257
|
+
- Test files placed in `.tests/business/{layer}/` directory
|
|
258
|
+
|
|
259
|
+
**If no `--gen-code`:** Scenarios remain as structured JSON for AI agent execution mode (Step 6).
|
|
260
|
+
|
|
261
|
+
---
|
|
262
|
+
|
|
263
|
+
### Step 6: Execute Tests (Progressive L1 -> L2 -> L3)
|
|
264
|
+
|
|
265
|
+
**Fail-fast rule:**
|
|
266
|
+
- If L1 has ANY "critical" priority failure -> STOP. Do not proceed to L2.
|
|
267
|
+
- If L2 has ANY "critical" priority failure -> STOP. Do not proceed to L3.
|
|
268
|
+
- "high" and "medium" failures do NOT block next layer.
|
|
269
|
+
|
|
270
|
+
**For each layer (starting from `--layer` if specified, else L1):**
|
|
271
|
+
|
|
272
|
+
Run Generator-Critic loop (max 3 iterations):
|
|
273
|
+
|
|
274
|
+
**Iteration 1: Execute all scenarios**
|
|
275
|
+
|
|
276
|
+
`--gen-code` mode:
|
|
277
|
+
```bash
|
|
278
|
+
# Run generated test classes for current layer
|
|
279
|
+
{test_command} --testPathPattern="business-{layer}" 2>&1 | tail -50
|
|
280
|
+
```
|
|
281
|
+
|
|
282
|
+
Agent execution mode:
|
|
283
|
+
```
|
|
284
|
+
FOR each scenario in current layer:
|
|
285
|
+
Execute scenario against running application
|
|
286
|
+
Record: { status, actual_response, duration_ms, error_detail }
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
**Critic phase: Classify failures**
|
|
290
|
+
|
|
291
|
+
For each failed scenario:
|
|
292
|
+
```
|
|
293
|
+
Analyze failure evidence -> classify as:
|
|
294
|
+
|
|
295
|
+
| Classification | Meaning | Action |
|
|
296
|
+
|---------------|---------|--------|
|
|
297
|
+
| test_defect | Test itself is wrong (wrong endpoint, bad fixture) | Auto-fix test in next iteration |
|
|
298
|
+
| code_defect | Business rule violated (actual != expected per REQ) | Record as failure |
|
|
299
|
+
| env_issue | Service down, config missing, timeout | Record as blocked |
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
**Iteration 2: Re-run with fixed tests**
|
|
303
|
+
- Fix test_defects from iteration 1
|
|
304
|
+
- Re-run ALL scenarios (not just failed ones — catch regressions)
|
|
305
|
+
|
|
306
|
+
**Iteration 3: Final confirmation**
|
|
307
|
+
- Remaining failures = confirmed code_defects
|
|
308
|
+
- All test_defects should be resolved
|
|
309
|
+
- Blocked scenarios marked with env_issue reason
|
|
310
|
+
|
|
311
|
+
Record results per iteration in `.tests/business/test-results-iter-{N}.json`.
|
|
312
|
+
|
|
313
|
+
---
|
|
314
|
+
|
|
315
|
+
### Step 7: Build Traceability Matrix
|
|
316
|
+
|
|
317
|
+
Map each scenario result back to requirement:
|
|
318
|
+
|
|
319
|
+
```
|
|
320
|
+
FOR each REQ in requirements targeted:
|
|
321
|
+
req_results = {
|
|
322
|
+
req_id: "REQ-NNN",
|
|
323
|
+
title: REQ.title,
|
|
324
|
+
total_criteria: count of AC in REQ,
|
|
325
|
+
criteria_results: []
|
|
326
|
+
}
|
|
327
|
+
|
|
328
|
+
FOR each AC in REQ:
|
|
329
|
+
scenarios_for_ac = filter scenarios where req_ref == "REQ-NNN:AC-{N}"
|
|
330
|
+
ac_status = "passed" if ALL scenarios passed
|
|
331
|
+
"failed" if ANY scenario failed
|
|
332
|
+
"blocked" if ANY blocked and none failed
|
|
333
|
+
"untested" if no scenarios mapped
|
|
334
|
+
|
|
335
|
+
req_results.criteria_results.push({
|
|
336
|
+
ac_id: "AC-{N}",
|
|
337
|
+
description: AC text,
|
|
338
|
+
priority: mapped from RFC 2119 keyword,
|
|
339
|
+
status: ac_status,
|
|
340
|
+
tests: [scenario IDs],
|
|
341
|
+
failure: { expected, actual, evidence } if failed
|
|
342
|
+
})
|
|
343
|
+
|
|
344
|
+
req_results.coverage_pct = passed_criteria / total_criteria * 100
|
|
345
|
+
req_results.verdict = "verified" if all MUST+SHOULD passed
|
|
346
|
+
"partial" if some failed
|
|
347
|
+
"unverified" if all failed or untested
|
|
348
|
+
```
|
|
349
|
+
|
|
350
|
+
---
|
|
351
|
+
|
|
352
|
+
### Step 8: Generate Reports
|
|
353
|
+
|
|
354
|
+
**Archive previous reports:**
|
|
355
|
+
```
|
|
356
|
+
IF file exists "${PHASE_DIR}/.tests/business/business-test-report.json":
|
|
357
|
+
mkdir -p "${PHASE_DIR}/.history"
|
|
358
|
+
TIMESTAMP = format(now(), "YYYY-MM-DDTHH-mm-ss")
|
|
359
|
+
mv report and summary to .history/
|
|
360
|
+
```
|
|
361
|
+
|
|
362
|
+
Write `.tests/business/business-test-report.json`:
|
|
363
|
+
```json
|
|
364
|
+
{
|
|
365
|
+
"phase": "{phase}",
|
|
366
|
+
"blueprint_ref": "{spec reference}",
|
|
367
|
+
"spec_mode": "full|degraded",
|
|
368
|
+
"completed_at": "{ISO timestamp}",
|
|
369
|
+
"execution_mode": "gen-code|agent",
|
|
370
|
+
"iterations": {
|
|
371
|
+
"L1": { "count": N, "converged": true },
|
|
372
|
+
"L2": { "count": N, "converged": true },
|
|
373
|
+
"L3": { "count": N, "converged": false }
|
|
374
|
+
},
|
|
375
|
+
"layers": {
|
|
376
|
+
"L1": { "total": N, "passed": P, "failed": F, "blocked": B, "pass_rate": 95.0 },
|
|
377
|
+
"L2": { "total": N, "passed": P, "failed": F, "blocked": B, "pass_rate": 87.5 },
|
|
378
|
+
"L3": { "total": N, "passed": P, "failed": F, "blocked": B, "pass_rate": 100.0 }
|
|
379
|
+
},
|
|
380
|
+
"requirement_coverage": [
|
|
381
|
+
{
|
|
382
|
+
"req_id": "REQ-001",
|
|
383
|
+
"title": "...",
|
|
384
|
+
"total_criteria": 5,
|
|
385
|
+
"criteria_results": [
|
|
386
|
+
{
|
|
387
|
+
"ac_id": "AC-1",
|
|
388
|
+
"description": "...",
|
|
389
|
+
"priority": "critical",
|
|
390
|
+
"status": "passed|failed|blocked|untested",
|
|
391
|
+
"tests": ["BT-001", "BT-002"],
|
|
392
|
+
"failure": null
|
|
393
|
+
}
|
|
394
|
+
],
|
|
395
|
+
"coverage_pct": 80.0,
|
|
396
|
+
"verdict": "verified|partial|unverified"
|
|
397
|
+
}
|
|
398
|
+
],
|
|
399
|
+
"failures": [
|
|
400
|
+
{
|
|
401
|
+
"id": "BF-001",
|
|
402
|
+
"test_id": "BT-005",
|
|
403
|
+
"req_ref": "REQ-001:AC-3",
|
|
404
|
+
"layer": "L1",
|
|
405
|
+
"severity": "critical",
|
|
406
|
+
"classification": "code_defect",
|
|
407
|
+
"description": "...",
|
|
408
|
+
"expected": "...",
|
|
409
|
+
"actual": "...",
|
|
410
|
+
"fix_suggestion": {
|
|
411
|
+
"file": "src/...",
|
|
412
|
+
"line": 42,
|
|
413
|
+
"direction": "Add null guard for..."
|
|
414
|
+
}
|
|
415
|
+
}
|
|
416
|
+
],
|
|
417
|
+
"summary": {
|
|
418
|
+
"total_requirements": 10,
|
|
419
|
+
"fully_verified": 8,
|
|
420
|
+
"partially_verified": 1,
|
|
421
|
+
"unverified": 1,
|
|
422
|
+
"coverage_pct": 85.0,
|
|
423
|
+
"total_scenarios": 30,
|
|
424
|
+
"total_passed": 26,
|
|
425
|
+
"total_failed": 3,
|
|
426
|
+
"total_blocked": 1
|
|
427
|
+
}
|
|
428
|
+
}
|
|
429
|
+
```
|
|
430
|
+
|
|
431
|
+
Write `.tests/business/business-test-summary.md`:
|
|
432
|
+
```markdown
|
|
433
|
+
---
|
|
434
|
+
phase: {phase}
|
|
435
|
+
blueprint_ref: {spec reference}
|
|
436
|
+
completed_at: {ISO timestamp}
|
|
437
|
+
verdict: passed|gaps_found
|
|
438
|
+
---
|
|
439
|
+
|
|
440
|
+
# Business Test Results
|
|
441
|
+
|
|
442
|
+
## Requirement Coverage
|
|
443
|
+
|
|
444
|
+
| REQ | Title | AC Total | Passed | Failed | Coverage | Verdict |
|
|
445
|
+
|-----|-------|----------|--------|--------|----------|---------|
|
|
446
|
+
| REQ-001 | ... | 5 | 4 | 1 | 80% | partial |
|
|
447
|
+
| REQ-002 | ... | 3 | 3 | 0 | 100% | verified |
|
|
448
|
+
|
|
449
|
+
## Layer Results
|
|
450
|
+
|
|
451
|
+
| Layer | Total | Passed | Failed | Blocked | Pass Rate |
|
|
452
|
+
|-------|-------|--------|--------|---------|-----------|
|
|
453
|
+
| L1 Interface | 10 | 9 | 1 | 0 | 90.0% |
|
|
454
|
+
| L2 Business | 15 | 13 | 1 | 1 | 86.7% |
|
|
455
|
+
| L3 E2E | 5 | 5 | 0 | 0 | 100.0% |
|
|
456
|
+
|
|
457
|
+
## Failures
|
|
458
|
+
|
|
459
|
+
### BF-001: REQ-001:AC-3 (critical)
|
|
460
|
+
- Layer: L1
|
|
461
|
+
- Expected: 201 Created with user object
|
|
462
|
+
- Actual: 400 Bad Request
|
|
463
|
+
- Fix: Add email validation bypass for internal accounts (src/auth.ts:42)
|
|
464
|
+
|
|
465
|
+
## Next Steps
|
|
466
|
+
{routing suggestion}
|
|
467
|
+
```
|
|
468
|
+
|
|
469
|
+
Update `index.json` with business_test section:
|
|
470
|
+
```json
|
|
471
|
+
{
|
|
472
|
+
"business_test": {
|
|
473
|
+
"status": "passed|gaps_found",
|
|
474
|
+
"spec_mode": "full|degraded",
|
|
475
|
+
"req_coverage_pct": 85.0,
|
|
476
|
+
"layers": {
|
|
477
|
+
"L1": { "pass_rate": 90.0 },
|
|
478
|
+
"L2": { "pass_rate": 86.7 },
|
|
479
|
+
"L3": { "pass_rate": 100.0 }
|
|
480
|
+
},
|
|
481
|
+
"failures": [
|
|
482
|
+
{ "id": "BF-001", "req_ref": "REQ-001:AC-3", "severity": "critical" }
|
|
483
|
+
]
|
|
484
|
+
}
|
|
485
|
+
}
|
|
486
|
+
```
|
|
487
|
+
|
|
488
|
+
---
|
|
489
|
+
|
|
490
|
+
### Step 9: Feedback Loop
|
|
491
|
+
|
|
492
|
+
**Auto-create issues from failures:**
|
|
493
|
+
```
|
|
494
|
+
FOR each failure in report.failures:
|
|
495
|
+
mkdir -p ".workflow/issues"
|
|
496
|
+
|
|
497
|
+
today = format(now(), "YYYYMMDD")
|
|
498
|
+
counter = next available sequence for today
|
|
499
|
+
|
|
500
|
+
issue = {
|
|
501
|
+
id: "ISS-{today}-{counter:03d}",
|
|
502
|
+
title: "Business Test: " + failure.req_ref + " - " + failure.description (truncated 100 chars),
|
|
503
|
+
status: "registered",
|
|
504
|
+
priority: severity_to_priority(failure.severity),
|
|
505
|
+
severity: failure.severity,
|
|
506
|
+
source: "business-test",
|
|
507
|
+
phase_ref: PHASE_NUM,
|
|
508
|
+
gap_ref: failure.id,
|
|
509
|
+
description: "Business test failed for " + failure.req_ref + ". Expected: " + failure.expected + ". Actual: " + failure.actual,
|
|
510
|
+
fix_direction: failure.fix_suggestion.direction,
|
|
511
|
+
context: {
|
|
512
|
+
location: failure.fix_suggestion.file + ":" + failure.fix_suggestion.line,
|
|
513
|
+
suggested_fix: failure.fix_suggestion.direction,
|
|
514
|
+
notes: "req_ref: " + failure.req_ref + ", layer: " + failure.layer
|
|
515
|
+
},
|
|
516
|
+
tags: ["business-test", failure.layer],
|
|
517
|
+
affected_components: [failure.fix_suggestion.file],
|
|
518
|
+
feedback: [],
|
|
519
|
+
issue_history: [],
|
|
520
|
+
created_at: now(),
|
|
521
|
+
updated_at: now(),
|
|
522
|
+
resolved_at: null,
|
|
523
|
+
resolution: null
|
|
524
|
+
}
|
|
525
|
+
Append JSON line to .workflow/issues/issues.jsonl
|
|
526
|
+
```
|
|
527
|
+
|
|
528
|
+
**Report:**
|
|
529
|
+
```
|
|
530
|
+
=== BUSINESS TEST RESULTS ===
|
|
531
|
+
Phase: {phase_name}
|
|
532
|
+
Spec mode: {full|degraded}
|
|
533
|
+
|
|
534
|
+
Requirement Coverage: {coverage_pct}%
|
|
535
|
+
Verified: {fully_verified}/{total_requirements}
|
|
536
|
+
Partial: {partially_verified}
|
|
537
|
+
Unverified: {unverified}
|
|
538
|
+
|
|
539
|
+
Layer Results:
|
|
540
|
+
L1 Interface: {pass_rate}% ({passed}/{total})
|
|
541
|
+
L2 Business: {pass_rate}% ({passed}/{total})
|
|
542
|
+
L3 E2E: {pass_rate}% ({passed}/{total})
|
|
543
|
+
|
|
544
|
+
Failures: {failure_count} ({blocker_count} blockers)
|
|
545
|
+
Issues: {issue_count} auto-created
|
|
546
|
+
|
|
547
|
+
Files:
|
|
548
|
+
{PHASE_DIR}/.tests/business/business-test-plan.json
|
|
549
|
+
{PHASE_DIR}/.tests/business/business-test-report.json
|
|
550
|
+
{PHASE_DIR}/.tests/business/business-test-summary.md
|
|
551
|
+
|
|
552
|
+
Next steps:
|
|
553
|
+
{suggested_next_command}
|
|
554
|
+
```
|
|
555
|
+
|
|
556
|
+
**Next step routing:**
|
|
557
|
+
|
|
558
|
+
| Result | Suggestion |
|
|
559
|
+
|--------|------------|
|
|
560
|
+
| All requirements verified | Skill({ skill: "maestro-phase-transition", args: "{phase}" }) |
|
|
561
|
+
| Failures found | Skill({ skill: "quality-debug", args: "--from-business-test {phase}" }) |
|
|
562
|
+
| `--re-run` all pass after fix | Skill({ skill: "maestro-execute", args: "{phase}" }) |
|
|
563
|
+
| Low coverage (< 60%) | Skill({ skill: "quality-auto-test", args: "{phase}" }) |
|
|
564
|
+
| Need integration tests | Skill({ skill: "quality-auto-test", args: "{phase}" }) |
|
|
565
|
+
|
|
566
|
+
**Closure criteria:**
|
|
567
|
+
A requirement is marked "verified" ONLY when:
|
|
568
|
+
- ALL acceptance criteria with MUST/SHALL keywords: passed
|
|
569
|
+
- ALL acceptance criteria with SHOULD keywords: passed
|
|
570
|
+
- No blocker-severity failures remain for this requirement
|