qaa-agent 1.9.2 → 1.9.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (41) hide show
  1. package/CHANGELOG.md +206 -179
  2. package/CLAUDE.md +718 -557
  3. package/README.md +384 -357
  4. package/VERSION +1 -1
  5. package/agents/qa-pipeline-orchestrator.md +1739 -1425
  6. package/agents/qaa-analyzer.md +0 -1
  7. package/agents/qaa-bug-detective.md +790 -655
  8. package/agents/qaa-codebase-mapper.md +50 -1
  9. package/agents/qaa-discovery.md +421 -384
  10. package/agents/qaa-e2e-runner.md +715 -577
  11. package/agents/qaa-executor.md +947 -830
  12. package/agents/qaa-planner.md +14 -1
  13. package/agents/qaa-project-researcher.md +533 -400
  14. package/agents/qaa-scanner.md +77 -1
  15. package/agents/qaa-testid-injector.md +0 -1
  16. package/agents/qaa-validator.md +86 -1
  17. package/bin/install.cjs +375 -253
  18. package/bin/lib/context7-cache.cjs +299 -0
  19. package/bin/lib/intent-detector.cjs +488 -0
  20. package/commands/qa-audit.md +255 -126
  21. package/commands/qa-create-test.md +666 -365
  22. package/commands/qa-fix.md +684 -513
  23. package/commands/qa-map.md +283 -139
  24. package/commands/qa-pr.md +63 -0
  25. package/commands/qa-research.md +181 -157
  26. package/commands/qa-start.md +62 -6
  27. package/commands/qa-test-report.md +219 -219
  28. package/frameworks/cypress.json +54 -0
  29. package/frameworks/jest.json +59 -0
  30. package/frameworks/playwright.json +58 -0
  31. package/frameworks/pytest.json +59 -0
  32. package/frameworks/robot-framework.json +54 -0
  33. package/frameworks/selenium.json +65 -0
  34. package/frameworks/vitest.json +57 -0
  35. package/package.json +5 -2
  36. package/workflows/qa-analyze.md +100 -4
  37. package/workflows/qa-from-ticket.md +50 -2
  38. package/workflows/qa-gap.md +50 -2
  39. package/workflows/qa-start.md +2048 -1405
  40. package/workflows/qa-testid.md +50 -2
  41. package/workflows/qa-validate.md +50 -2
@@ -1,1405 +1,2048 @@
1
- <purpose>
2
-
3
- Orchestrate the full QA automation pipeline: scan -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [bug-detective if failures] -> deliver. Detects workflow option (1/2/3) from arguments, spawns specialized agents for each stage, manages state transitions, handles checkpoints (safe auto-approve, risky always pause), and delivers a draft PR with per-stage atomic commits.
4
-
5
- Invoked by the `/qa-start` slash command. Accepts `--dev-repo`, `--qa-repo`, and `--auto` flags.
6
-
7
- </purpose>
8
-
9
- <required_reading>
10
-
11
- Read these files BEFORE executing any pipeline stage. Do NOT skip.
12
-
13
- - **CLAUDE.md** -- Agent pipeline stages, module boundaries, quality gates, stage transitions, auto-advance rules, agent coordination, data-testid convention. Read the full file.
14
- - **agents/qa-pipeline-orchestrator.md** -- Full orchestrator logic, checkpoint classification, error handling, delivery sub-steps.
15
-
16
- </required_reading>
17
-
18
- <process>
19
-
20
- <step name="initialize" priority="first">
21
-
22
- ## Step 1: Initialize Pipeline
23
-
24
- Parse `$ARGUMENTS` for flags:
25
-
26
- ```bash
27
- DEV_REPO=""
28
- QA_REPO=""
29
- IS_AUTO=false
30
-
31
- # Parse --dev-repo flag
32
- if echo "$ARGUMENTS" | grep -qE '\-\-dev-repo'; then
33
- DEV_REPO=$(echo "$ARGUMENTS" | grep -oE '\-\-dev-repo\s+[^\s]+' | awk '{print $2}')
34
- fi
35
-
36
- # Parse --qa-repo flag
37
- if echo "$ARGUMENTS" | grep -qE '\-\-qa-repo'; then
38
- QA_REPO=$(echo "$ARGUMENTS" | grep -oE '\-\-qa-repo\s+[^\s]+' | awk '{print $2}')
39
- fi
40
-
41
- # Parse --auto flag
42
- if echo "$ARGUMENTS" | grep -qE '\-\-auto'; then
43
- IS_AUTO=true
44
- fi
45
- ```
46
-
47
- **If no --dev-repo provided**, use the current working directory:
48
- ```bash
49
- if [ -z "$DEV_REPO" ]; then
50
- DEV_REPO=$(pwd)
51
- fi
52
- ```
53
-
54
- **Attempt to call qaa-tools init** (handle missing tool gracefully):
55
- ```bash
56
- INIT_JSON=$(node bin/qaa-tools.cjs init qa-start 2>/dev/null || echo "")
57
- ```
58
-
59
- If `INIT_JSON` is empty or the command fails, proceed with manual initialization:
60
- - Set `output_dir` to `.qa-output`
61
- - Set `date` to current date in `YYYY-MM-DD` format
62
- - Create output directory: `mkdir -p "$output_dir"`
63
-
64
- If `INIT_JSON` is valid, parse it for: `option`, `dev_repo_path`, `qa_repo_path`, `maturity_score`, `maturity_note`, `output_dir`, `date`, agent model assignments, `auto_advance`, `auto_chain_active`, `parallelization`, `commit_docs`.
65
-
66
- **Detect workflow option based on inputs:**
67
-
68
- - If `QA_REPO` is empty (no --qa-repo flag): **Option 1** (Dev-Only -- Full Pipeline)
69
- - If `QA_REPO` is provided: Assess QA repo maturity
70
- - Check for existing test files, configs, coverage reports in QA repo
71
- - Count test files, evaluate framework setup, check for CI config
72
- - Score 0-100 based on test count, framework config presence, CI setup, coverage data
73
- - Score >= 60: **Option 3** (Dev + Mature QA -- Surgical)
74
- - Score < 60: **Option 2** (Dev + Immature QA -- Gap-Fill)
75
-
76
- **Determine auto-advance mode:**
77
- ```bash
78
- # Check persistent config flag
79
- AUTO_CFG=$(node bin/qaa-tools.cjs config-get workflow.auto_advance 2>/dev/null || echo "false")
80
- AUTO_CHAIN=$(node bin/qaa-tools.cjs config-get workflow._auto_chain_active 2>/dev/null || echo "false")
81
-
82
- if [ "$IS_AUTO" = "true" ] || [ "$AUTO_CFG" = "true" ] || [ "$AUTO_CHAIN" = "true" ]; then
83
- IS_AUTO=true
84
- node bin/qaa-tools.cjs config-set workflow._auto_chain_active true 2>/dev/null || true
85
- fi
86
-
87
- # Safety: clear stale chain flag if NOT in auto mode
88
- if [ "$IS_AUTO" = "false" ]; then
89
- node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
90
- fi
91
- ```
92
-
93
- **Print initialization banner:**
94
- ```
95
- === QA Pipeline Orchestrator ===
96
- Option: {option} ({description})
97
- Dev Repo: {DEV_REPO}
98
- QA Repo: {QA_REPO or 'N/A'}
99
- Maturity Score: {maturity_score or 'N/A'}
100
- Auto-Advance: {IS_AUTO}
101
- Date: {date}
102
- ================================
103
- ```
104
-
105
- Where `{description}` is:
106
- - Option 1: "Dev-Only -- Full Pipeline"
107
- - Option 2: "Dev + Immature QA -- Gap-Fill"
108
- - Option 3: "Dev + Mature QA -- Surgical"
109
-
110
- </step>
111
-
112
- <step name="detect_framework">
113
-
114
- ## Step 2: Detect Framework
115
-
116
- Before scanning, detect the project's language and test framework to guide all downstream agents.
117
-
118
- **Read project config files:**
119
- ```bash
120
- # Check for Node.js / JavaScript / TypeScript
121
- [ -f "${DEV_REPO}/package.json" ] && cat "${DEV_REPO}/package.json"
122
-
123
- # Check for Python
124
- [ -f "${DEV_REPO}/requirements.txt" ] && cat "${DEV_REPO}/requirements.txt"
125
- [ -f "${DEV_REPO}/pyproject.toml" ] && cat "${DEV_REPO}/pyproject.toml"
126
- [ -f "${DEV_REPO}/setup.py" ] && cat "${DEV_REPO}/setup.py"
127
-
128
- # Check for .NET
129
- ls "${DEV_REPO}"/*.csproj 2>/dev/null
130
- ls "${DEV_REPO}"/**/*.csproj 2>/dev/null
131
-
132
- # Check for Java
133
- [ -f "${DEV_REPO}/pom.xml" ] && echo "Maven project"
134
- [ -f "${DEV_REPO}/build.gradle" ] && echo "Gradle project"
135
- ```
136
-
137
- **Detect test framework from config files:**
138
- ```bash
139
- # JavaScript/TypeScript ecosystem
140
- [ -f "${DEV_REPO}/cypress.config.ts" ] || [ -f "${DEV_REPO}/cypress.config.js" ] && echo "FRAMEWORK=cypress"
141
- [ -f "${DEV_REPO}/playwright.config.ts" ] || [ -f "${DEV_REPO}/playwright.config.js" ] && echo "FRAMEWORK=playwright"
142
- [ -f "${DEV_REPO}/jest.config.ts" ] || [ -f "${DEV_REPO}/jest.config.js" ] && echo "FRAMEWORK=jest"
143
- [ -f "${DEV_REPO}/vitest.config.ts" ] || [ -f "${DEV_REPO}/vitest.config.js" ] && echo "FRAMEWORK=vitest"
144
-
145
- # Python ecosystem
146
- [ -f "${DEV_REPO}/pytest.ini" ] || [ -f "${DEV_REPO}/conftest.py" ] && echo "FRAMEWORK=pytest"
147
-
148
- # Check package.json devDependencies for test frameworks
149
- node -e "
150
- try {
151
- const pkg = require('${DEV_REPO}/package.json');
152
- const deps = {...(pkg.devDependencies||{}), ...(pkg.dependencies||{})};
153
- const frameworks = [];
154
- if (deps.cypress) frameworks.push('cypress');
155
- if (deps['@playwright/test'] || deps.playwright) frameworks.push('playwright');
156
- if (deps.jest) frameworks.push('jest');
157
- if (deps.vitest) frameworks.push('vitest');
158
- if (deps.mocha) frameworks.push('mocha');
159
- console.log(frameworks.join(',') || 'none');
160
- } catch { console.log('no-package-json'); }
161
- " 2>/dev/null
162
- ```
163
-
164
- **Assess detection confidence:**
165
- - **HIGH**: Config file found AND matching dependency in package.json/requirements.txt
166
- - **MEDIUM**: Only dependency found (no config file) OR only config file (no dependency)
167
- - **LOW**: No test framework detected, or conflicting signals
168
-
169
- **If no test framework found:**
170
- - If `IS_AUTO` is false: Ask the user which framework to use. STOP and wait for response.
171
- - If `IS_AUTO` is true: Select the most appropriate framework based on the project type:
172
- - React/Next.js/Vue/Angular frontend -> Playwright
173
- - Node.js API -> Jest or Vitest (prefer Vitest if ESM)
174
- - Python -> Pytest
175
- - Log: "Auto-selected: {framework} (no existing test framework detected)"
176
-
177
- **If detection confidence is LOW:**
178
- - If `IS_AUTO` is true: Auto-approve with most likely framework (SAFE checkpoint). Log: "Auto-approved: Framework detection (LOW confidence, selected {framework})". Continue.
179
- - If `IS_AUTO` is false: Present detection details to user. Wait for confirmation before proceeding.
180
-
181
- Store detected framework, language, and confidence for all downstream agents.
182
-
183
- </step>
184
-
185
- <step name="scan">
186
-
187
- ## Step 3: Scan Repository
188
-
189
- **State update -- mark scan as running:**
190
- ```bash
191
- node bin/qaa-tools.cjs state patch --"Scan Status" running --"Status" "Scanning repository" 2>/dev/null || true
192
- ```
193
-
194
- **Print stage banner:**
195
- ```
196
- +------------------------------------------+
197
- | STAGE 1: Scanner |
198
- | Status: Running... |
199
- +------------------------------------------+
200
- ```
201
-
202
- **Spawn scanner agent:**
203
-
204
- For **Option 1** (scan dev repo only):
205
- ```
206
- Agent(subagent_type="general-purpose",
207
- prompt="
208
- <objective>Scan repository and produce SCAN_MANIFEST.md</objective>
209
- <execution_context>@agents/qaa-scanner.md</execution_context>
210
- <files_to_read>
211
- - CLAUDE.md
212
- </files_to_read>
213
- <parameters>
214
- dev_repo_path: {DEV_REPO}
215
- qa_repo_path: null
216
- output_path: {output_dir}/SCAN_MANIFEST.md
217
- </parameters>
218
- "
219
- )
220
- ```
221
-
222
- For **Options 2 and 3** (scan both repos):
223
- ```
224
- Agent(subagent_type="general-purpose",
225
- prompt="
226
- <objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
227
- <execution_context>@agents/qaa-scanner.md</execution_context>
228
- <files_to_read>
229
- - CLAUDE.md
230
- </files_to_read>
231
- <parameters>
232
- dev_repo_path: {DEV_REPO}
233
- qa_repo_path: {QA_REPO}
234
- output_path: {output_dir}/SCAN_MANIFEST.md
235
- </parameters>
236
- "
237
- )
238
- ```
239
-
240
- **Parse scanner return:**
241
-
242
- Expected return structure:
243
- ```
244
- SCANNER_COMPLETE:
245
- file_path: ".qa-output/SCAN_MANIFEST.md"
246
- decision: PROCEED | STOP
247
- has_frontend: true | false
248
- detection_confidence: HIGH | MEDIUM | LOW
249
- ```
250
-
251
- **Handle decision field:**
252
- - If `decision` is `STOP`:
253
- ```bash
254
- node bin/qaa-tools.cjs state patch --"Scan Status" failed --"Status" "Pipeline stopped: Scanner returned STOP" 2>/dev/null || true
255
- ```
256
- Print failure banner and STOP PIPELINE ENTIRELY. Do NOT proceed to any further stage.
257
-
258
- - If `decision` is `PROCEED`:
259
- ```bash
260
- node bin/qaa-tools.cjs state patch --"Scan Status" complete 2>/dev/null || true
261
- ```
262
- Capture `has_frontend` for testid-injector conditional (Step 5).
263
- Capture `detection_confidence` for checkpoint handling.
264
-
265
- **Verify artifact exists before continuing:**
266
- ```bash
267
- [ -f "${output_dir}/SCAN_MANIFEST.md" ] && echo "OK: SCAN_MANIFEST.md exists" || echo "MISSING: SCAN_MANIFEST.md"
268
- ```
269
-
270
- If SCAN_MANIFEST.md is missing, treat as stage failure. Set status to failed and STOP pipeline.
271
-
272
- </step>
273
-
274
- <step name="codebase_map">
275
-
276
- ## Step 3b: Codebase Map (Deep Analysis)
277
-
278
- **State update -- mark codebase-map as running:**
279
- ```bash
280
- node bin/qaa-tools.cjs state patch --"Map Status" running --"Status" "Mapping codebase" 2>/dev/null || true
281
- ```
282
-
283
- **Print stage banner:**
284
- ```
285
- +------------------------------------------+
286
- | STAGE 1b: Codebase Mapper |
287
- | Status: Running 4 parallel sub-agents...|
288
- +------------------------------------------+
289
- ```
290
-
291
- **Create output directory:**
292
- ```bash
293
- mkdir -p ${output_dir}/codebase
294
- ```
295
-
296
- **Spawn 4 parallel sub-agents** (one per focus area). Issue all 4 `Agent()` calls in a single message for parallel execution:
297
-
298
- **Sub-agent 1 - testability focus** (produces TESTABILITY.md + TEST_SURFACE.md):
299
- ```
300
- Agent(subagent_type="general-purpose",
301
- prompt="
302
- <objective>Codebase analysis - testability focus. Produce TESTABILITY.md and TEST_SURFACE.md mapping what is testable in this codebase, mock boundaries, and exhaustive list of testable entry points.</objective>
303
- <execution_context>@agents/qaa-codebase-mapper.md</execution_context>
304
- <files_to_read>
305
- - CLAUDE.md
306
- - ${output_dir}/SCAN_MANIFEST.md
307
- </files_to_read>
308
- <parameters>
309
- focus_area: testability
310
- dev_repo_path: {DEV_REPO}
311
- output_dir: ${output_dir}/codebase
312
- </parameters>
313
- "
314
- )
315
- ```
316
-
317
- **Sub-agent 2 - risk focus** (produces RISK_MAP.md + CRITICAL_PATHS.md):
318
- ```
319
- Agent(subagent_type="general-purpose",
320
- prompt="
321
- <objective>Codebase analysis - risk focus. Produce RISK_MAP.md and CRITICAL_PATHS.md identifying business-critical paths, security-sensitive areas, and the exact user flows that E2E smoke tests must cover.</objective>
322
- <execution_context>@agents/qaa-codebase-mapper.md</execution_context>
323
- <files_to_read>
324
- - CLAUDE.md
325
- - ${output_dir}/SCAN_MANIFEST.md
326
- </files_to_read>
327
- <parameters>
328
- focus_area: risk
329
- dev_repo_path: {DEV_REPO}
330
- output_dir: ${output_dir}/codebase
331
- </parameters>
332
- "
333
- )
334
- ```
335
-
336
- **Sub-agent 3 - patterns focus** (produces CODE_PATTERNS.md + API_CONTRACTS.md):
337
- ```
338
- Agent(subagent_type="general-purpose",
339
- prompt="
340
- <objective>Codebase analysis - patterns focus. Produce CODE_PATTERNS.md and API_CONTRACTS.md documenting naming conventions, import style, and exact request/response shapes for API tests.</objective>
341
- <execution_context>@agents/qaa-codebase-mapper.md</execution_context>
342
- <files_to_read>
343
- - CLAUDE.md
344
- - ${output_dir}/SCAN_MANIFEST.md
345
- </files_to_read>
346
- <parameters>
347
- focus_area: patterns
348
- dev_repo_path: {DEV_REPO}
349
- output_dir: ${output_dir}/codebase
350
- </parameters>
351
- "
352
- )
353
- ```
354
-
355
- **Sub-agent 4 - existing-tests focus** (produces TEST_ASSESSMENT.md + COVERAGE_GAPS.md):
356
- ```
357
- Agent(subagent_type="general-purpose",
358
- prompt="
359
- <objective>Codebase analysis - existing-tests focus. Produce TEST_ASSESSMENT.md and COVERAGE_GAPS.md assessing current test quality, frameworks in use, and identifying which modules/functions/paths have no test coverage.</objective>
360
- <execution_context>@agents/qaa-codebase-mapper.md</execution_context>
361
- <files_to_read>
362
- - CLAUDE.md
363
- - ${output_dir}/SCAN_MANIFEST.md
364
- </files_to_read>
365
- <parameters>
366
- focus_area: existing-tests
367
- dev_repo_path: {DEV_REPO}
368
- output_dir: ${output_dir}/codebase
369
- </parameters>
370
- "
371
- )
372
- ```
373
-
374
- **Verify codebase docs exist:**
375
- ```bash
376
- docs_count=$(ls ${output_dir}/codebase/*.md 2>/dev/null | wc -l)
377
- echo "Codebase docs produced: $docs_count of 8 expected"
378
-
379
- if [ "$docs_count" -eq 0 ]; then
380
- echo "FAIL: No codebase docs produced. Pipeline continues with degraded context."
381
- node bin/qaa-tools.cjs state patch --"Map Status" failed 2>/dev/null || true
382
- elif [ "$docs_count" -lt 4 ]; then
383
- echo "WARNING: Fewer than 4 codebase docs produced. Downstream agents will have limited context."
384
- node bin/qaa-tools.cjs state patch --"Map Status" "partial" 2>/dev/null || true
385
- else
386
- echo "OK: codebase map sufficient ($docs_count docs)"
387
- node bin/qaa-tools.cjs state patch --"Map Status" complete 2>/dev/null || true
388
- fi
389
- ```
390
-
391
- **Codebase map is non-blocking:** if zero docs are produced, the pipeline continues with degraded context (downstream agents fall back to whatever they can find). All `<files_to_read>` references to codebase docs in subsequent stages use `(if exists)` semantics.
392
-
393
- </step>
394
-
395
- <step name="research">
396
-
397
- ## Step 3c: Research Testing Ecosystem
398
-
399
- **State update -- mark research as running:**
400
- ```bash
401
- node bin/qaa-tools.cjs state patch --"Research Status" running --"Status" "Researching testing ecosystem" 2>/dev/null || true
402
- ```
403
-
404
- **Print stage banner:**
405
- ```
406
- +------------------------------------------+
407
- | STAGE 1c: Project Researcher |
408
- | Status: Running... |
409
- +------------------------------------------+
410
- ```
411
-
412
- **Create output directory:**
413
- ```bash
414
- mkdir -p ${output_dir}/research
415
- ```
416
-
417
- **Spawn researcher agent:**
418
- ```
419
- Agent(subagent_type="general-purpose",
420
- prompt="
421
- <objective>Research the testing ecosystem for this project. Use the codebase map findings (RISK_MAP, CODE_PATTERNS, API_CONTRACTS, TESTABILITY, etc., from ${output_dir}/codebase/) to ground your Context7 queries to the project specifics — query for the libraries/patterns the project actually uses (e.g., MSW integration, Stripe testing, OpenAPI contract testing) instead of generic framework questions. Use Context7 MCP as the primary source. Produce research documents consumed by downstream agents.</objective>
422
- <execution_context>@agents/qaa-project-researcher.md</execution_context>
423
- <files_to_read>
424
- - CLAUDE.md
425
- - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
426
- - ${output_dir}/SCAN_MANIFEST.md
427
- - ${output_dir}/codebase/TESTABILITY.md (if exists)
428
- - ${output_dir}/codebase/TEST_SURFACE.md (if exists)
429
- - ${output_dir}/codebase/RISK_MAP.md (if exists)
430
- - ${output_dir}/codebase/CRITICAL_PATHS.md (if exists)
431
- - ${output_dir}/codebase/CODE_PATTERNS.md (if exists)
432
- - ${output_dir}/codebase/API_CONTRACTS.md (if exists)
433
- - ${output_dir}/codebase/TEST_ASSESSMENT.md (if exists)
434
- - ${output_dir}/codebase/COVERAGE_GAPS.md (if exists)
435
- </files_to_read>
436
- <parameters>
437
- mode: stack-testing
438
- dev_repo_path: {DEV_REPO}
439
- output_dir: ${output_dir}/research
440
- </parameters>
441
- "
442
- )
443
- ```
444
-
445
- **Verify research artifacts exist:**
446
- ```bash
447
- ls ${output_dir}/research/*.md 2>/dev/null && echo "OK: Research files produced" || echo "WARNING: No research files produced"
448
- ```
449
-
450
- **Research is non-blocking:** If the researcher fails or produces no output, the pipeline continues — downstream agents will fall back to Context7 queries directly. Log the warning but do NOT stop the pipeline.
451
-
452
- ```bash
453
- if [ ! -f "${output_dir}/research/TESTING_STACK.md" ]; then
454
- echo "WARNING: Research stage produced no output. Downstream agents will query Context7 directly."
455
- node bin/qaa-tools.cjs state patch --"Research Status" "skipped (no output)" 2>/dev/null || true
456
- else
457
- node bin/qaa-tools.cjs state patch --"Research Status" complete 2>/dev/null || true
458
- fi
459
- ```
460
-
461
- </step>
462
-
463
- <step name="analyze">
464
-
465
- ## Step 4: Analyze Repository
466
-
467
- **State update -- mark analyze as running:**
468
- ```bash
469
- node bin/qaa-tools.cjs state patch --"Analyze Status" running --"Status" "Analyzing repository" 2>/dev/null || true
470
- ```
471
-
472
- **Print stage banner:**
473
- ```
474
- +------------------------------------------+
475
- | STAGE 2: Analyzer |
476
- | Status: Running... |
477
- +------------------------------------------+
478
- ```
479
-
480
- **Determine analyzer mode based on option:**
481
- - Option 1: `mode = 'full'` (produces QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md)
482
- - Options 2 and 3: `mode = 'gap'` (produces GAP_ANALYSIS.md)
483
-
484
- **Spawn analyzer agent:**
485
- ```
486
- Agent(subagent_type="general-purpose",
487
- prompt="
488
- <objective>Analyze scanned repository and produce analysis artifacts</objective>
489
- <execution_context>@agents/qaa-analyzer.md</execution_context>
490
- <files_to_read>
491
- - {output_dir}/SCAN_MANIFEST.md
492
- - CLAUDE.md
493
- - {output_dir}/research/TESTING_STACK.md (if exists)
494
- - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
495
- - {output_dir}/codebase/RISK_MAP.md (if exists)
496
- - {output_dir}/codebase/CRITICAL_PATHS.md (if exists)
497
- - {output_dir}/codebase/TEST_ASSESSMENT.md (if exists)
498
- - {output_dir}/codebase/COVERAGE_GAPS.md (if exists)
499
- </files_to_read>
500
- <parameters>
501
- mode: {mode}
502
- workflow_option: {option}
503
- dev_repo_path: {DEV_REPO}
504
- qa_repo_path: {QA_REPO or null}
505
- output_path: {output_dir}/
506
- </parameters>
507
- "
508
- )
509
- ```
510
-
511
- **Parse analyzer return:**
512
-
513
- Expected return structure:
514
- ```
515
- ANALYZER_COMPLETE:
516
- files_produced: [...]
517
- total_test_count: N
518
- pyramid_breakdown: {unit: N, integration: N, api: N, e2e: N}
519
- risk_count: {high: N, medium: N, low: N}
520
- commit_hash: "..."
521
- ```
522
-
523
- Capture `files_produced`, `total_test_count`, `pyramid_breakdown` for downstream stages.
524
-
525
- **Handle analyzer checkpoint -- assumptions review:**
526
- - If `IS_AUTO` is true: Auto-approve all assumptions (SAFE checkpoint). Log: "Auto-approved: Analyzer assumptions". Continue pipeline.
527
- - If `IS_AUTO` is false: Present assumptions to user for review. Wait for confirmation or corrections. On user response, incorporate corrections and continue.
528
-
529
- **State update -- mark analyze as complete:**
530
- ```bash
531
- node bin/qaa-tools.cjs state patch --"Analyze Status" complete 2>/dev/null || true
532
- ```
533
-
534
- **Verify artifacts exist before continuing:**
535
-
536
- For Option 1:
537
- ```bash
538
- [ -f "${output_dir}/QA_ANALYSIS.md" ] && echo "OK" || echo "MISSING: QA_ANALYSIS.md"
539
- [ -f "${output_dir}/TEST_INVENTORY.md" ] && echo "OK" || echo "MISSING: TEST_INVENTORY.md"
540
- ```
541
-
542
- For Options 2/3:
543
- ```bash
544
- [ -f "${output_dir}/GAP_ANALYSIS.md" ] && echo "OK" || echo "MISSING: GAP_ANALYSIS.md"
545
- ```
546
-
547
- If required artifacts are missing, treat as stage failure. Set status to failed and STOP pipeline.
548
-
549
- Print: "Analysis complete. {total_test_count} test cases identified. Pyramid: unit={unit}, integration={integration}, api={api}, e2e={e2e}."
550
-
551
- </step>
552
-
553
- <step name="testid_inject">
554
-
555
- ## Step 5: TestID Injection (Conditional)
556
-
557
- **Condition:** Only execute if `has_frontend` is `true` from scanner return (Step 3).
558
-
559
- **If `has_frontend` is false:**
560
- Print: "Skipping TestID injection (no frontend detected)." Proceed directly to Step 6 (Plan).
561
-
562
- **If `has_frontend` is true:**
563
-
564
- **State update:**
565
- ```bash
566
- node bin/qaa-tools.cjs state patch --"Status" "Injecting test IDs into frontend components" 2>/dev/null || true
567
- ```
568
-
569
- **Print stage banner:**
570
- ```
571
- +------------------------------------------+
572
- | STAGE 3: TestID Injector |
573
- | Status: Running... |
574
- +------------------------------------------+
575
- ```
576
-
577
- **Spawn testid-injector agent:**
578
- ```
579
- Agent(subagent_type="general-purpose",
580
- prompt="
581
- <objective>Audit and inject data-testid attributes into frontend components</objective>
582
- <execution_context>@agents/qaa-testid-injector.md</execution_context>
583
- <files_to_read>
584
- - {output_dir}/SCAN_MANIFEST.md
585
- - CLAUDE.md
586
- </files_to_read>
587
- <parameters>
588
- dev_repo_path: {DEV_REPO}
589
- output_path: {output_dir}/TESTID_AUDIT_REPORT.md
590
- </parameters>
591
- "
592
- )
593
- ```
594
-
595
- **Parse return:**
596
-
597
- Check for `INJECTOR_COMPLETE` vs `INJECTOR_SKIPPED`:
598
-
599
- If `INJECTOR_COMPLETE`:
600
- ```
601
- INJECTOR_COMPLETE:
602
- report_path: "..."
603
- coverage_before: N%
604
- coverage_after: N%
605
- elements_injected: N
606
- components_modified: N
607
- ```
608
- Log: "TestID injection complete. Coverage: {coverage_before}% -> {coverage_after}%. {elements_injected} elements injected."
609
-
610
- If `INJECTOR_SKIPPED`:
611
- ```
612
- INJECTOR_SKIPPED:
613
- reason: "..."
614
- action: "..."
615
- ```
616
- Log the reason and continue pipeline.
617
-
618
- **Handle injector checkpoint -- audit review:**
619
- - If `IS_AUTO` is true: Auto-approve P0-only injection (SAFE checkpoint). Log: "Auto-approved: TestID injection (P0 elements only)". Continue pipeline.
620
- - If `IS_AUTO` is false: Present audit report to user. Wait for approval, element selection, or rejection. On user response, incorporate decisions and continue.
621
-
622
- **Verify artifact exists:**
623
- ```bash
624
- [ -f "${output_dir}/TESTID_AUDIT_REPORT.md" ] && echo "OK" || echo "MISSING: TESTID_AUDIT_REPORT.md"
625
- ```
626
-
627
- </step>
628
-
629
- <step name="plan">
630
-
631
- ## Step 6: Plan Test Generation
632
-
633
- **State update -- mark generation as running (planning is part of generate):**
634
- ```bash
635
- node bin/qaa-tools.cjs state patch --"Generate Status" running --"Status" "Planning test generation" 2>/dev/null || true
636
- ```
637
-
638
- **Print stage banner:**
639
- ```
640
- +------------------------------------------+
641
- | STAGE 4: Planner |
642
- | Status: Running... |
643
- +------------------------------------------+
644
- ```
645
-
646
- **Determine planner input based on option:**
647
- - Option 1: Input from `{output_dir}/TEST_INVENTORY.md` + `{output_dir}/QA_ANALYSIS.md`
648
- - Options 2 and 3: Input from `{output_dir}/GAP_ANALYSIS.md`
649
-
650
- **Spawn planner agent:**
651
- ```
652
- Agent(subagent_type="general-purpose",
653
- prompt="
654
- <objective>Create test generation plan with task breakdown and dependencies</objective>
655
- <execution_context>@agents/qaa-planner.md</execution_context>
656
- <files_to_read>
657
- - {input files based on option}
658
- - CLAUDE.md
659
- - {output_dir}/codebase/TESTABILITY.md (if exists)
660
- - {output_dir}/codebase/TEST_SURFACE.md (if exists)
661
- - {output_dir}/codebase/CRITICAL_PATHS.md (if exists)
662
- - {output_dir}/codebase/COVERAGE_GAPS.md (if exists)
663
- </files_to_read>
664
- <parameters>
665
- workflow_option: {option}
666
- output_path: {output_dir}/GENERATION_PLAN.md
667
- </parameters>
668
- "
669
- )
670
- ```
671
-
672
- **Parse planner return:**
673
-
674
- Expected return structure:
675
- ```
676
- PLANNER_COMPLETE:
677
- file_path: "..."
678
- total_tasks: N
679
- total_files: N
680
- feature_count: N
681
- dependency_depth: N
682
- test_case_count: N
683
- commit_hash: "..."
684
- ```
685
-
686
- Capture `total_tasks`, `total_files`, `feature_count` for executor stage and pipeline summary.
687
-
688
- **Verify artifact exists:**
689
- ```bash
690
- [ -f "${output_dir}/GENERATION_PLAN.md" ] && echo "OK" || echo "MISSING: GENERATION_PLAN.md"
691
- ```
692
-
693
- If GENERATION_PLAN.md is missing, treat as stage failure. Set status to failed and STOP pipeline.
694
-
695
- Print: "Plan complete. {total_tasks} tasks, {total_files} files planned across {feature_count} features."
696
-
697
- </step>
698
-
699
- <step name="generate">
700
-
701
- ## Step 7: Generate Test Files
702
-
703
- State update continues from planning (already set to `running` in Step 6).
704
-
705
- **Print stage banner:**
706
- ```
707
- +------------------------------------------+
708
- | STAGE 5: Executor |
709
- | Generating {total_files} test files |
710
- | Status: Running... |
711
- +------------------------------------------+
712
- ```
713
-
714
- **Determine execution strategy:**
715
-
716
- Check if planner created multiple independent feature groups. If `feature_count > 1` AND parallelization is enabled (from init config):
717
-
718
- **Parallel execution** (when feature_count > 1 and parallelization enabled):
719
-
720
- For each independent feature group from the generation plan, spawn a separate executor agent:
721
- ```
722
- Agent(subagent_type="general-purpose",
723
- prompt="
724
- <objective>Generate test files for {feature} feature</objective>
725
- <execution_context>@agents/qaa-executor.md</execution_context>
726
- <files_to_read>
727
- - {output_dir}/GENERATION_PLAN.md
728
- - {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
729
- - CLAUDE.md
730
- - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
731
- - {output_dir}/research/E2E_STRATEGY.md (if exists)
732
- - {output_dir}/research/API_TESTING_STRATEGY.md (if exists)
733
- - {output_dir}/codebase/TEST_SURFACE.md (if exists)
734
- - {output_dir}/codebase/CODE_PATTERNS.md (if exists)
735
- - {output_dir}/codebase/API_CONTRACTS.md (if exists)
736
- </files_to_read>
737
- <parameters>
738
- workflow_option: {option}
739
- feature_group: {feature}
740
- dev_repo_path: {DEV_REPO}
741
- qa_repo_path: {QA_REPO or null}
742
- output_path: {output_dir}/
743
- </parameters>
744
- "
745
- )
746
- ```
747
-
748
- Multiple Agent() calls can be issued simultaneously for independent feature groups. Each executor handles one feature group and commits its files independently.
749
-
750
- **Sequential execution** (when feature_count == 1 or parallelization disabled):
751
-
752
- Spawn a single executor agent covering all tasks:
753
- ```
754
- Agent(subagent_type="general-purpose",
755
- prompt="
756
- <objective>Generate all test files from generation plan</objective>
757
- <execution_context>@agents/qaa-executor.md</execution_context>
758
- <files_to_read>
759
- - {output_dir}/GENERATION_PLAN.md
760
- - {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
761
- - CLAUDE.md
762
- - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
763
- - {output_dir}/research/E2E_STRATEGY.md (if exists)
764
- - {output_dir}/research/API_TESTING_STRATEGY.md (if exists)
765
- - {output_dir}/codebase/TEST_SURFACE.md (if exists)
766
- - {output_dir}/codebase/CODE_PATTERNS.md (if exists)
767
- - {output_dir}/codebase/API_CONTRACTS.md (if exists)
768
- </files_to_read>
769
- <parameters>
770
- workflow_option: {option}
771
- dev_repo_path: {DEV_REPO}
772
- qa_repo_path: {QA_REPO or null}
773
- output_path: {output_dir}/
774
- </parameters>
775
- "
776
- )
777
- ```
778
-
779
- **Option 3 specific -- skip existing tests:**
780
-
781
- For Option 3, pass `skip_existing_test_ids: true` to the executor so it checks existing test files by test ID before generating. If a test ID already exists in the QA repo, skip generating that test case:
782
- ```
783
- <parameters>
784
- workflow_option: 3
785
- skip_existing_test_ids: true
786
- dev_repo_path: {DEV_REPO}
787
- qa_repo_path: {QA_REPO}
788
- output_path: {output_dir}/
789
- </parameters>
790
- ```
791
-
792
- **Parse executor return:**
793
-
794
- Expected return structure:
795
- ```
796
- EXECUTOR_COMPLETE:
797
- files_created: [{path, type}, ...]
798
- total_files: N
799
- commit_count: N
800
- features_covered: [...]
801
- test_case_count: N
802
- ```
803
-
804
- Capture `files_created`, `total_files`, `commit_count` for validation stage and pipeline summary.
805
-
806
- **State update -- mark generate as complete:**
807
- ```bash
808
- node bin/qaa-tools.cjs state patch --"Generate Status" complete --"Status" "Test generation complete" 2>/dev/null || true
809
- ```
810
-
811
- Print: "Generation complete. {total_files} files created across {features_covered_count} features. {commit_count} commits."
812
-
813
- </step>
814
-
815
- <step name="validate">
816
-
817
- ## Step 8: Validate Generated Tests
818
-
819
- **State update -- mark validate as running:**
820
- ```bash
821
- node bin/qaa-tools.cjs state patch --"Validate Status" running --"Status" "Validating generated tests" 2>/dev/null || true
822
- ```
823
-
824
- **Print stage banner:**
825
- ```
826
- +------------------------------------------+
827
- | STAGE 6: Validator |
828
- | Validating {total_files} test files |
829
- | Status: Running... |
830
- +------------------------------------------+
831
- ```
832
-
833
- **Spawn validator agent:**
834
- ```
835
- Agent(subagent_type="general-purpose",
836
- prompt="
837
- <objective>Run 4-layer validation on all generated test files</objective>
838
- <execution_context>@agents/qaa-validator.md</execution_context>
839
- <files_to_read>
840
- - {list all generated test files from executor return -- files_created paths}
841
- - {output_dir}/GENERATION_PLAN.md
842
- - CLAUDE.md
843
- </files_to_read>
844
- <parameters>
845
- mode: validation
846
- max_fix_loops: 3
847
- output_path: {output_dir}/VALIDATION_REPORT.md
848
- </parameters>
849
- "
850
- )
851
- ```
852
-
853
- **4-layer validation:**
854
- 1. **Syntax** -- File parses without errors
855
- 2. **Structure** -- Follows POM rules, naming conventions, locator hierarchy
856
- 3. **Dependencies** -- Imports resolve, fixtures exist, configs present
857
- 4. **Logic** -- Assertions are concrete, test IDs are unique, no assertions in page objects
858
-
859
- **5. Browser verification (if app URL available and Playwright MCP connected):**
860
-
861
- After the 4-layer static validation, use Playwright MCP to verify E2E tests against the live app:
862
-
863
- 1. Navigate to each page referenced in the E2E tests:
864
- ```
865
- mcp__playwright__browser_navigate({ url: "{app_url}/{route}" })
866
- ```
867
-
868
- 2. Take an accessibility snapshot to verify locators used in tests actually exist:
869
- ```
870
- mcp__playwright__browser_snapshot()
871
- ```
872
-
873
- 3. Cross-reference locators in generated tests against the real DOM:
874
- - Verify `data-testid` values exist on the page
875
- - Verify ARIA roles and names match test expectations
876
- - Flag any test locator that does not match a real DOM element
877
-
878
- 4. If mismatches are found, fix the test locators to match the real DOM and count as a fix loop iteration.
879
-
880
- This browser verification step prevents delivering tests with locators that will immediately fail at runtime.
881
-
882
- **Fix loop:** The validator automatically attempts to fix issues it finds. Maximum 3 fix loop iterations. After each fix attempt, re-validate.
883
-
884
- **Parse validator return:**
885
-
886
- Expected return structure:
887
- ```
888
- VALIDATOR_COMPLETE:
889
- report_path: "..."
890
- overall_status: PASS | PASS_WITH_WARNINGS | FAIL
891
- confidence: HIGH | MEDIUM | LOW
892
- layers_summary: {syntax: PASS|FAIL, structure: PASS|FAIL, dependencies: PASS|FAIL, logic: PASS|FAIL}
893
- fix_loops_used: N
894
- issues_found: N
895
- issues_fixed: N
896
- unresolved_count: N
897
- ```
898
-
899
- **RISKY CHECKPOINT -- Validator escalation:**
900
-
901
- If `unresolved_count > 0` after max fix loops (3):
902
- - **ALWAYS pause, even in auto mode** (RISKY checkpoint -- locked decision)
903
- - Present unresolved issues to user with full details from VALIDATION_REPORT.md
904
- - Wait for user decision:
905
- - `"approve-with-warnings"`: Accept the validation with warnings. Set Validate Status to complete. Continue to deliver.
906
- - `"abort"`: Set Validate Status to failed. STOP PIPELINE ENTIRELY.
907
- - Manual guidance: User provides specific fix instructions. Spawn fresh continuation agent to apply fixes and re-validate.
908
-
909
- If `overall_status` is `PASS` or `PASS_WITH_WARNINGS` (and unresolved_count is 0):
910
- ```bash
911
- node bin/qaa-tools.cjs state patch --"Validate Status" complete --"Status" "Validation passed" 2>/dev/null || true
912
- ```
913
-
914
- **Verify artifact exists:**
915
- ```bash
916
- [ -f "${output_dir}/VALIDATION_REPORT.md" ] && echo "OK" || echo "MISSING: VALIDATION_REPORT.md"
917
- ```
918
-
919
- Print: "Validation complete. Status: {overall_status}. Confidence: {confidence}. {issues_found} issues found, {issues_fixed} fixed, {unresolved_count} unresolved."
920
-
921
- </step>
922
-
923
- <step name="bug_detective">
924
-
925
- ## Step 9: Bug Detective (Conditional)
926
-
927
- **Condition:** Only execute if test failures were detected during validation. Check:
928
- - `overall_status === 'FAIL'` in validator return, OR
929
- - Generated tests have runtime failures that need classification
930
-
931
- **If no failures to classify:**
932
- Print: "Skipping Bug Detective (no test failures detected)." Proceed directly to Step 10 (Deliver).
933
-
934
- **If failures need classification:**
935
-
936
- **State update:**
937
- ```bash
938
- node bin/qaa-tools.cjs state patch --"Status" "Classifying test failures" 2>/dev/null || true
939
- ```
940
-
941
- **Print stage banner:**
942
- ```
943
- +------------------------------------------+
944
- | STAGE 7: Bug Detective |
945
- | Status: Running... |
946
- +------------------------------------------+
947
- ```
948
-
949
- **Spawn bug-detective agent:**
950
- ```
951
- Agent(subagent_type="general-purpose",
952
- prompt="
953
- <objective>Classify test failures and attempt auto-fixes for test errors. Use Playwright MCP to reproduce E2E failures in the browser when available.</objective>
954
- <execution_context>@agents/qaa-bug-detective.md</execution_context>
955
- <files_to_read>
956
- - {test execution results -- from validator or direct test run}
957
- - {failing test source files -- paths from executor return}
958
- - CLAUDE.md
959
- - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
960
- - {output_dir}/research/TESTING_STACK.md (if exists)
961
- </files_to_read>
962
- <parameters>
963
- output_path: {output_dir}/FAILURE_CLASSIFICATION_REPORT.md
964
- app_url: {app_url if available}
965
- </parameters>
966
- "
967
- )
968
- ```
969
-
970
- **Parse bug-detective return:**
971
-
972
- Expected return structure:
973
- ```
974
- DETECTIVE_COMPLETE:
975
- report_path: "..."
976
- total_failures: N
977
- classification_breakdown: {app_bug: N, test_error: N, env_issue: N, inconclusive: N}
978
- auto_fixes_applied: N
979
- auto_fixes_verified: N
980
- commit_hash: "..."
981
- ```
982
-
983
- **RISKY CHECKPOINT -- Application bugs detected:**
984
-
985
- If `classification_breakdown.app_bug > 0`:
986
- - **ALWAYS pause, even in auto mode** (RISKY checkpoint -- locked decision)
987
- - Present APPLICATION BUG classifications to user with full evidence from FAILURE_CLASSIFICATION_REPORT.md
988
- - These are genuine bugs in the application code discovered during test execution
989
- - The bug detective never touches application code -- it only reports
990
- - User must review and decide how to proceed:
991
- - Acknowledge bugs and continue pipeline (bugs will be in the PR description for developer attention)
992
- - Abort pipeline to fix bugs first
993
-
994
- **Verify artifact exists:**
995
- ```bash
996
- [ -f "${output_dir}/FAILURE_CLASSIFICATION_REPORT.md" ] && echo "OK" || echo "MISSING: FAILURE_CLASSIFICATION_REPORT.md"
997
- ```
998
-
999
- Print: "Bug Detective complete. {total_failures} failures classified: {app_bug} APP BUG, {test_error} TEST ERROR, {env_issue} ENV ISSUE, {inconclusive} INCONCLUSIVE. {auto_fixes_applied} auto-fixes applied."
1000
-
1001
- </step>
1002
-
1003
- <step name="deliver">
1004
-
1005
- ## Step 10: Deliver
1006
-
1007
- **State update -- mark deliver as running:**
1008
- ```bash
1009
- node bin/qaa-tools.cjs state patch --"Deliver Status" running --"Status" "Preparing delivery" 2>/dev/null || true
1010
- ```
1011
-
1012
- **Print stage banner:**
1013
- ```
1014
- +------------------------------------------+
1015
- | STAGE 8: Deliver |
1016
- | Status: Running... |
1017
- +------------------------------------------+
1018
- ```
1019
-
1020
- ### Sub-step 1: Pre-flight checks
1021
-
1022
- **Check for git remote:**
1023
- ```bash
1024
- REMOTE_URL=$(git remote get-url origin 2>/dev/null || echo "")
1025
- ```
1026
-
1027
- If `REMOTE_URL` is empty:
1028
- - Print: "No git remote found. Artifacts committed locally but PR creation skipped."
1029
- - Set `LOCAL_ONLY=true`
1030
-
1031
- **Check for gh CLI authentication:**
1032
- ```bash
1033
- gh auth status 2>/dev/null
1034
- ```
1035
-
1036
- If `gh auth status` fails:
1037
- - Print: "gh CLI not authenticated. Run 'gh auth login' first. Artifacts committed locally."
1038
- - Set `LOCAL_ONLY=true`
1039
-
1040
- If both checks pass, set `LOCAL_ONLY=false`.
1041
-
1042
- ### Sub-step 2: Derive project name
1043
-
1044
- ```bash
1045
- # Read from package.json
1046
- PROJECT_NAME=$(node -e "try { const p = require('${DEV_REPO}/package.json'); console.log(p.name || ''); } catch { console.log(''); }" 2>/dev/null)
1047
-
1048
- # Fallback to directory basename
1049
- if [ -z "$PROJECT_NAME" ]; then
1050
- PROJECT_NAME=$(basename "${DEV_REPO}")
1051
- fi
1052
-
1053
- # Sanitize for branch naming
1054
- PROJECT_NAME=$(echo "$PROJECT_NAME" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/--*/-/g' | sed 's/^-//' | sed 's/-$//')
1055
- ```
1056
-
1057
- ### Sub-step 3: Detect default branch
1058
-
1059
- ```bash
1060
- DEFAULT_BRANCH=$(gh repo view --json defaultBranchRef --jq '.defaultBranchRef.name' 2>/dev/null || echo "main")
1061
- ```
1062
-
1063
- ### Sub-step 4: Create feature branch
1064
-
1065
- ```bash
1066
- BRANCH="qa/auto-${PROJECT_NAME}-${date}"
1067
-
1068
- # Handle branch name collision
1069
- if git rev-parse --verify "$BRANCH" 2>/dev/null || git rev-parse --verify "origin/$BRANCH" 2>/dev/null; then
1070
- SUFFIX=2
1071
- while git rev-parse --verify "${BRANCH}-${SUFFIX}" 2>/dev/null || git rev-parse --verify "origin/${BRANCH}-${SUFFIX}" 2>/dev/null; do
1072
- SUFFIX=$((SUFFIX + 1))
1073
- done
1074
- BRANCH="${BRANCH}-${SUFFIX}"
1075
- fi
1076
-
1077
- git checkout -b "$BRANCH" "$DEFAULT_BRANCH"
1078
- ```
1079
-
1080
- ### Sub-step 5: Per-stage atomic commits
1081
-
1082
- For each pipeline stage that produced artifacts, commit using `qaa-tools.cjs commit`. Check file existence before each commit.
1083
-
1084
- **Scanner:**
1085
- ```bash
1086
- if [ -f "${output_dir}/SCAN_MANIFEST.md" ]; then
1087
- node bin/qaa-tools.cjs commit "qa(scanner): produce SCAN_MANIFEST.md for ${PROJECT_NAME}" --files ${output_dir}/SCAN_MANIFEST.md
1088
- fi
1089
- ```
1090
-
1091
- **Analyzer (Option 1):**
1092
- ```bash
1093
- if [ -f "${output_dir}/QA_ANALYSIS.md" ]; then
1094
- ANALYZER_FILES="${output_dir}/QA_ANALYSIS.md ${output_dir}/TEST_INVENTORY.md"
1095
- [ -f "${output_dir}/QA_REPO_BLUEPRINT.md" ] && ANALYZER_FILES="${ANALYZER_FILES} ${output_dir}/QA_REPO_BLUEPRINT.md"
1096
- node bin/qaa-tools.cjs commit "qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md" --files ${ANALYZER_FILES}
1097
- fi
1098
- ```
1099
-
1100
- **Analyzer (Options 2/3):**
1101
- ```bash
1102
- if [ -f "${output_dir}/GAP_ANALYSIS.md" ]; then
1103
- node bin/qaa-tools.cjs commit "qa(analyzer): produce GAP_ANALYSIS.md" --files ${output_dir}/GAP_ANALYSIS.md
1104
- fi
1105
- ```
1106
-
1107
- **TestID Injector (if ran):**
1108
- ```bash
1109
- if [ -f "${output_dir}/TESTID_AUDIT_REPORT.md" ]; then
1110
- node bin/qaa-tools.cjs commit "qa(testid-injector): inject ${elements_injected} data-testid attributes across ${components_modified} components" --files ${output_dir}/TESTID_AUDIT_REPORT.md ${modified_source_files}
1111
- fi
1112
- ```
1113
-
1114
- **Executor:**
1115
- ```bash
1116
- if [ -n "${generated_file_paths}" ]; then
1117
- node bin/qaa-tools.cjs commit "qa(executor): generate ${total_files} test files with POMs and fixtures" --files ${generated_file_paths}
1118
- fi
1119
- ```
1120
-
1121
- **Planner:**
1122
- ```bash
1123
- if [ -f "${output_dir}/GENERATION_PLAN.md" ]; then
1124
- node bin/qaa-tools.cjs commit "qa(planner): produce GENERATION_PLAN.md" --files ${output_dir}/GENERATION_PLAN.md
1125
- fi
1126
- ```
1127
-
1128
- **Validator:**
1129
- ```bash
1130
- if [ -f "${output_dir}/VALIDATION_REPORT.md" ]; then
1131
- node bin/qaa-tools.cjs commit "qa(validator): validate generated tests - ${overall_status} with ${confidence} confidence" --files ${output_dir}/VALIDATION_REPORT.md
1132
- fi
1133
- ```
1134
-
1135
- **Bug Detective (if ran):**
1136
- ```bash
1137
- if [ -f "${output_dir}/FAILURE_CLASSIFICATION_REPORT.md" ]; then
1138
- node bin/qaa-tools.cjs commit "qa(bug-detective): classify ${total_failures} failures - ${classification_summary}" --files ${output_dir}/FAILURE_CLASSIFICATION_REPORT.md
1139
- fi
1140
- ```
1141
-
1142
- ### Sub-step 6: Push branch
1143
-
1144
- If `LOCAL_ONLY` is true, skip this sub-step.
1145
-
1146
- ```bash
1147
- git push -u origin "$BRANCH"
1148
- ```
1149
-
1150
- If push fails:
1151
- - Print: "Push failed: {error_message}. Artifacts committed locally on branch ${BRANCH}."
1152
- - Set `LOCAL_ONLY=true`
1153
-
1154
- ### Sub-step 7: Build PR body
1155
-
1156
- If `LOCAL_ONLY` is true, skip this sub-step.
1157
-
1158
- Read the PR template:
1159
- ```bash
1160
- PR_BODY=$(cat templates/pr-template.md)
1161
- ```
1162
-
1163
- Replace all `{placeholder}` tokens with actual values collected during pipeline execution:
1164
- - `{architecture_type}` -- from QA_ANALYSIS.md or SCAN_MANIFEST.md
1165
- - `{framework}` -- detected test framework
1166
- - `{risk_summary}` -- risk assessment counts (e.g., "3 HIGH, 5 MEDIUM, 2 LOW")
1167
- - `{unit_count}` -- from pyramid_breakdown.unit
1168
- - `{integration_count}` -- from pyramid_breakdown.integration
1169
- - `{api_count}` -- from pyramid_breakdown.api
1170
- - `{e2e_count}` -- from pyramid_breakdown.e2e
1171
- - `{total_count}` -- from total_test_count
1172
- - `{modules_covered}` -- count of modules with tests
1173
- - `{coverage_estimate}` -- estimated coverage percentage
1174
- - `{validation_result}` -- PASS, PASS_WITH_WARNINGS, or FAIL
1175
- - `{confidence}` -- HIGH, MEDIUM, or LOW
1176
- - `{fix_loops_used}` -- number 0-3
1177
- - `{issues_found}` -- total issues found during validation
1178
- - `{issues_fixed}` -- total issues auto-fixed
1179
- - `{file_list}` -- if total files <= 50, list each file; if > 50, use summary
1180
-
1181
- ### Sub-step 8: Create draft PR
1182
-
1183
- If `LOCAL_ONLY` is true, skip this sub-step.
1184
-
1185
- ```bash
1186
- PR_URL=$(gh pr create \
1187
- --draft \
1188
- --title "qa: automated test suite for ${PROJECT_NAME}" \
1189
- --body "${PR_BODY}" \
1190
- --label "qa-automation" \
1191
- --label "auto-generated" \
1192
- --assignee "@me" 2>&1)
1193
- ```
1194
-
1195
- Do NOT pass `--base` flag. Let gh auto-detect the default branch.
1196
-
1197
- On failure:
1198
- - Print: "PR creation failed: ${PR_URL}. Artifacts remain on branch ${BRANCH}."
1199
- - Do NOT stop the pipeline -- artifacts are committed and pushed.
1200
-
1201
- ### Sub-step 9: Print result
1202
-
1203
- If PR was created successfully:
1204
- ```
1205
- PR created: ${PR_URL}
1206
- ```
1207
-
1208
- If `LOCAL_ONLY` is true:
1209
- ```
1210
- PR: not created (local-only mode). Artifacts committed on branch: ${BRANCH}
1211
- ```
1212
-
1213
- **State update -- mark deliver as complete:**
1214
- ```bash
1215
- node bin/qaa-tools.cjs state patch --"Deliver Status" complete --"Status" "Pipeline complete" 2>/dev/null || true
1216
- ```
1217
-
1218
- **Clear auto-chain flag at pipeline completion:**
1219
- ```bash
1220
- node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
1221
- ```
1222
-
1223
- ### Print pipeline summary banner:
1224
-
1225
- ```
1226
- ======================================================
1227
- QA PIPELINE COMPLETE
1228
- ======================================================
1229
-
1230
- Option: {option} ({option_description})
1231
- Repository: {DEV_REPO}
1232
- QA Repo: {QA_REPO or 'N/A'}
1233
- Maturity Score: {maturity_score or 'N/A'}
1234
-
1235
- Stages Completed:
1236
- [{check}] Scan -- {scan_result}
1237
- [{check}] Analyze -- {analyze_result} ({test_count} test cases)
1238
- [{check}] TestID Inject -- {inject_result or 'skipped'}
1239
- [{check}] Plan -- {plan_result} ({file_count} files planned)
1240
- [{check}] Generate -- {generate_result} ({files_created} files created)
1241
- [{check}] Validate -- {validate_result} ({confidence} confidence)
1242
- [{check}] Bug Detective -- {detective_result or 'skipped'}
1243
- [{check}] Deliver -- {deliver_result}
1244
-
1245
- PR: {pr_url or 'not created (local-only)'}
1246
-
1247
- Artifacts:
1248
- {list all produced .md files in output_dir}
1249
-
1250
- Total Time: {total_duration}
1251
- ======================================================
1252
- ```
1253
-
1254
- Where: `[x]` = completed, `[ ]` = skipped, `[!]` = failed.
1255
-
1256
- </step>
1257
-
1258
- </process>
1259
-
1260
- <error_handling>
1261
-
1262
- ## Error Handling
1263
-
1264
- ### Stage Failure Protocol
1265
-
1266
- When any agent returns a failure or error:
1267
-
1268
- 1. **Set stage status to failed:**
1269
- ```bash
1270
- node bin/qaa-tools.cjs state patch --"{Stage} Status" failed --"Status" "Pipeline stopped: {Stage} failed - {reason}" 2>/dev/null || true
1271
- ```
1272
-
1273
- 2. **Print failure banner:**
1274
- ```
1275
- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1276
- ! PIPELINE STOPPED !
1277
- ! Stage: {stage_name} !
1278
- ! Reason: {failure_reason} !
1279
- ! !
1280
- ! Completed: {completed_stages} !
1281
- ! Artifacts: {artifacts_so_far} !
1282
- ! !
1283
- ! Action required: Review and re-run !
1284
- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1285
- ```
1286
-
1287
- 3. **DO NOT continue to next stage.** The pipeline stops entirely at the failed stage.
1288
-
1289
- 4. **DO NOT create partial PR.** No branch, no commit, no PR with incomplete results.
1290
-
1291
- 5. **Preserve all artifacts produced so far.** They remain on disk in `{output_dir}/` for debugging.
1292
-
1293
- ### Artifact Verification
1294
-
1295
- After EVERY agent spawn, before advancing to next stage, verify the expected output artifact exists on disk:
1296
- ```bash
1297
- [ -f "{expected_artifact_path}" ] && echo "OK" || echo "MISSING"
1298
- ```
1299
-
1300
- If artifacts are missing, treat as stage failure and STOP pipeline.
1301
-
1302
- ### qaa-tools.cjs Graceful Fallback
1303
-
1304
- All `node bin/qaa-tools.cjs` calls use `2>/dev/null || true` to handle cases where the tool is not installed or not found. The pipeline must not break due to missing state management tooling -- it logs a warning and continues.
1305
-
1306
- </error_handling>
1307
-
1308
- <auto_advance>
1309
-
1310
- ## Auto-Advance Mode
1311
-
1312
- Auto-advance is enabled when ANY of these is true:
1313
- - `--auto` flag passed to the `/qa-start` invocation
1314
- - `config.json` has `workflow.auto_advance = true` (persistent user preference)
1315
- - `workflow._auto_chain_active = true` in config (ephemeral chain flag from ongoing auto run)
1316
-
1317
- ### Behavior in Auto Mode
1318
-
1319
- **SAFE checkpoints are auto-approved.** The pipeline continues without pausing. A log message records the auto-approval:
1320
- ```
1321
- Auto-approved: {checkpoint_description}
1322
- ```
1323
-
1324
- **RISKY checkpoints ALWAYS pause.** Even in auto mode, the pipeline stops and presents the checkpoint to the user.
1325
-
1326
- ### Safe vs Risky Checkpoint Classification
1327
-
1328
- **SAFE (auto-approve in auto mode):**
1329
-
1330
- | Checkpoint | Agent | Auto-Action |
1331
- |------------|-------|-------------|
1332
- | Framework detection uncertain (LOW confidence) | Scanner | Approve with most likely framework |
1333
- | Analyzer assumptions review | Analyzer | Approve all assumptions |
1334
- | TestID audit review | TestID Injector | Approve P0-only injection |
1335
-
1336
- **RISKY (ALWAYS pause, even in auto mode):**
1337
-
1338
- | Checkpoint | Agent | User Action Required |
1339
- |------------|-------|---------------------|
1340
- | Validator escalation (unresolved issues after 3 fix loops) | Validator | approve-with-warnings, abort, or fix guidance |
1341
- | APPLICATION BUG classification | Bug Detective | Review bugs, continue or fix first |
1342
- | Any checkpoint with "unresolved" or "failed" blocking text | Any | Review specific blocking issue |
1343
-
1344
- ### Checkpoint Handling Flow
1345
-
1346
- ```
1347
- On agent return with checkpoint data:
1348
- 1. Extract checkpoint blocking field content
1349
- 2. Classify as SAFE or RISKY:
1350
- - "framework detection" -> SAFE
1351
- - "assumptions" -> SAFE
1352
- - "audit" or "data-testid" -> SAFE
1353
- - "unresolved" -> RISKY
1354
- - "failed" -> RISKY
1355
- - "APPLICATION BUG" -> RISKY
1356
- - Default (no pattern match) -> RISKY (conservative)
1357
- 3. If IS_AUTO and SAFE:
1358
- - Auto-approve with default action
1359
- - Log the auto-approval
1360
- - Continue pipeline
1361
- 4. If IS_AUTO and RISKY:
1362
- - PAUSE pipeline
1363
- - Print checkpoint details with full context
1364
- - Wait for user input
1365
- 5. If NOT auto (manual mode):
1366
- - PAUSE pipeline
1367
- - Print checkpoint details
1368
- - Wait for user input
1369
- ```
1370
-
1371
- ### Resume After Checkpoint
1372
-
1373
- When resuming after a checkpoint, spawn a FRESH agent with explicit state:
1374
- ```
1375
- Agent(subagent_type="general-purpose",
1376
- prompt="
1377
- <objective>Continue QA pipeline from {stage} stage</objective>
1378
- <execution_context>@agents/qa-pipeline-orchestrator.md</execution_context>
1379
- <resume_context>
1380
- Pipeline state:
1381
- - Completed stages: {list of completed stages with their results}
1382
- - Current stage: {stage that triggered checkpoint}
1383
- - Checkpoint response: {user's response or decision}
1384
- - Artifacts produced so far: {list of files with paths}
1385
-
1386
- Resume from: {exact step in pipeline to resume from}
1387
- User decision: {what user chose at checkpoint}
1388
- </resume_context>
1389
- "
1390
- )
1391
- ```
1392
-
1393
- ### Stale Chain Flag Protection
1394
-
1395
- At orchestrator init, if `--auto` was NOT passed AND `auto_advance` config is false:
1396
- ```bash
1397
- node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
1398
- ```
1399
-
1400
- At pipeline completion (success or failure):
1401
- ```bash
1402
- node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
1403
- ```
1404
-
1405
- </auto_advance>
1
+ <critical_directive priority="MANDATORY">
2
+
3
+ ## MANDATORY EXECUTION RULES READ AND OBEY BEFORE ANYTHING ELSE
4
+
5
+ These rules are non-negotiable. Read them before any other section of this file.
6
+
7
+ ### Rule 1 — Execute every step literally
8
+
9
+ This workflow is a script, NOT a guide. Run each `<step>` in order. Do NOT skip steps. Do NOT reorder steps. Do NOT adapt steps to "fit" the user's situation.
10
+
11
+ ### Rule 2 Your FIRST tool call MUST be the bash block in Step 1 (initialize)
12
+
13
+ Specifically: run `node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve ...` as your very first action. Do NOT explore the repo first. Do NOT analyze the user's prompt independently. Do NOT make assumptions about whether the pipeline "applies".
14
+
15
+ ### Rule 3 — NEVER improvise an alternative pipeline
16
+
17
+ If you find yourself thinking "the standard pipeline doesn't apply because [empty repo / external URL / different framework / X / Y]", **STOP**. The pipeline applies. Run it as written.
18
+
19
+ If a stage genuinely cannot complete, the workflow itself handles that case (with non-blocking failures, fallbacks, or stop-pipeline directives). Your job is to **execute**, not to redesign.
20
+
21
+ ### Rule 4 — If a step fails, report failure — do not adapt
22
+
23
+ When a bash command fails or a sub-agent returns an error, report it as a pipeline failure following the `<error_handling>` rules at the bottom of this file. Do NOT "work around" it. Do NOT scaffold an alternative.
24
+
25
+ ### Rule 5 — Empty target repos are NORMAL and expected
26
+
27
+ The pipeline handles empty repos. Many stages produce skeleton output or report "nothing to do" for empty repos. This is **by design**. Do NOT interpret "empty repo" as a signal to abandon the workflow and start scaffolding things directly.
28
+
29
+ ### Rule 6 — Your role is execution, not architecture
30
+
31
+ You are running a defined process. The user chose this process by invoking `/qa-start`. Do NOT second-guess the design. Do NOT decide that a different approach would be better. Run the workflow.
32
+
33
+ ### Rule 7 Pass `$ARGUMENTS` to the intent detector LITERALLY
34
+
35
+ When Step 1 says `node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" ...`, the placeholder `$ARGUMENTS` must be substituted with the **raw user input as received**, with no mental translation, parsing, or restructuring.
36
+
37
+ **WRONG** pre-translating NL to flag form before invoking the detector:
38
+ ```
39
+ # User prompt: "from URL https://x.com to C:\foo using cypress"
40
+ # DON'T DO THIS:
41
+ node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve '--app-url https://x.com --dev-repo "C:\foo" --framework cypress' ...
42
+ ```
43
+
44
+ **RIGHT** — passing the raw prompt as-is so the detector does its job:
45
+ ```
46
+ node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve 'from URL https://x.com to C:\foo using cypress' ...
47
+ ```
48
+
49
+ The detector handles NL parsing. If you pre-translate, the detector sees flags and reports `source: flag` for everything, which **bypasses the confirmation checkpoint** that exists specifically to catch NL-misinterpretation errors. That defeats the purpose.
50
+
51
+ ### Failure mode to recognize and prevent: IMPROVISATION
52
+
53
+ If, after reading this workflow, you find yourself doing any of the following, you have FAILED:
54
+
55
+ - Listing the target repo and saying "this is empty, I'll scaffold files directly"
56
+ - Saying "the standard pipeline assumes X, I'll adapt"
57
+ - Calling sub-agents that the workflow does not explicitly invoke
58
+ - Generating files (.spec.ts, .robot, etc.) before reaching the `<step name="generate">` stage
59
+ - Skipping the `<step name="initialize">` bash block and going straight to other steps
60
+
61
+ If you catch yourself about to do any of these, **stop, return to the top of this workflow, and execute Step 1 first**.
62
+
63
+ </critical_directive>
64
+
65
+ <purpose>
66
+
67
+ Orchestrate the full QA automation pipeline: scan -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests AND app URL] -> [bug-detective if failures] -> deliver. Detects workflow option (1/2/3) from arguments, spawns specialized agents for each stage, manages state transitions, handles checkpoints (safe auto-approve, risky always pause), and delivers a draft PR with per-stage atomic commits.
68
+
69
+ Invoked by the `/qa-start` slash command. Accepts `--dev-repo`, `--qa-repo`, and `--auto` flags.
70
+
71
+ </purpose>
72
+
73
+ <required_reading>
74
+
75
+ Read these files BEFORE executing any pipeline stage. Do NOT skip.
76
+
77
+ - **CLAUDE.md** -- Agent pipeline stages, module boundaries, quality gates, stage transitions, auto-advance rules, agent coordination, data-testid convention. Read the full file.
78
+ - **agents/qa-pipeline-orchestrator.md** -- Full orchestrator logic, checkpoint classification, error handling, delivery sub-steps.
79
+
80
+ </required_reading>
81
+
82
+ <process>
83
+
84
+ <step name="initialize" priority="first">
85
+
86
+ ## Step 1: Initialize Pipeline
87
+
88
+ Parse `$ARGUMENTS` using the intent detector (handles flags + natural language + paths with spaces):
89
+
90
+ ```bash
91
+ # Resolve flags + NL + defaults via shared intent detector
92
+ # IMPORTANT: $ARGUMENTS must be the RAW user input, NOT a pre-translated flag string.
93
+ # The detector handles NL parsing — pre-translation defeats the confirmation checkpoint.
94
+ RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"dev_repo":"dev-repo","qa_repo":"qa-repo","app_url":"app-url","framework":"framework"}')
95
+
96
+ # Extract resolved values
97
+ DEV_REPO=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.dev_repo?.value || '')")
98
+ QA_REPO=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.qa_repo?.value || '')")
99
+ APP_URL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.app_url?.value || '')")
100
+ FRAMEWORK=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.framework?.value || '')")
101
+ IS_AUTO=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.resolved.auto?.value === true ? 'true' : 'false')")
102
+
103
+ # Print the INPUT DETECTION banner (always shown — with flags, with NL, or defaults)
104
+ echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.banner)"
105
+ ```
106
+
107
+ **If no --dev-repo provided and not detected from NL**, use the current working directory:
108
+ ```bash
109
+ if [ -z "$DEV_REPO" ]; then
110
+ DEV_REPO=$(pwd)
111
+ echo " (no dev-repo specified, defaulting to cwd: $DEV_REPO)"
112
+ fi
113
+ ```
114
+
115
+ **Input Validation (5 pre-pipeline checks — Capa 1 of 3-layer validation system):**
116
+
117
+ ```bash
118
+ echo "=== INPUT VALIDATION ==="
119
+ VALIDATION_FAIL=false
120
+
121
+ # Check 1: URL reachable (warning only, not fatal)
122
+ if [ -n "$APP_URL" ]; then
123
+ if curl -sf -o /dev/null -m 5 "$APP_URL" 2>/dev/null; then
124
+ echo " OK: AUT URL reachable ($APP_URL)"
125
+ else
126
+ echo " WARN: AUT URL not reachable from this network — proceeding anyway"
127
+ fi
128
+ fi
129
+
130
+ # Check 2: Path destino writable or creatable
131
+ if [ -n "$DEV_REPO" ]; then
132
+ if [ -d "$DEV_REPO" ] && [ -w "$DEV_REPO" ]; then
133
+ echo " OK: dev-repo path exists and is writable"
134
+ elif [ ! -d "$DEV_REPO" ]; then
135
+ parent=$(dirname "$DEV_REPO")
136
+ if [ -d "$parent" ] && [ -w "$parent" ]; then
137
+ echo " OK: dev-repo path can be created"
138
+ else
139
+ echo " FAIL: dev-repo path is not writable: $DEV_REPO"
140
+ VALIDATION_FAIL=true
141
+ fi
142
+ fi
143
+ fi
144
+
145
+ # Check 3: URL not ambiguous with ticket platforms
146
+ if [ -n "$APP_URL" ]; then
147
+ if echo "$APP_URL" | grep -qE "github\.com.*(issues|pull)|atlassian\.net|linear\.app|dev\.azure\.com"; then
148
+ echo " WARN: $APP_URL looks like a ticket URL — did you mean /qa-from-ticket? Proceeding anyway."
149
+ fi
150
+ fi
151
+
152
+ # Check 4: No multiple AUT URLs detected
153
+ URL_COUNT=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.intent.aut_urls.length)")
154
+ if [ "$URL_COUNT" -gt 1 ]; then
155
+ echo " FAIL: $URL_COUNT AUT URLs detected, expected at most 1"
156
+ echo " Found: $(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.intent.aut_urls.join(', '))")"
157
+ VALIDATION_FAIL=true
158
+ fi
159
+
160
+ # Check 5: Framework target exists in registry (warn if not, falls back to Context7)
161
+ if [ -n "$FRAMEWORK" ]; then
162
+ if [ -f "frameworks/${FRAMEWORK}.json" ]; then
163
+ echo " OK: framework '$FRAMEWORK' found in registry"
164
+ else
165
+ echo " WARN: framework '$FRAMEWORK' not in frameworks/ registry relying on Context7 only"
166
+ fi
167
+ fi
168
+
169
+ if [ "$VALIDATION_FAIL" = "true" ]; then
170
+ echo "=== VALIDATION FAILED pipeline stopped ==="
171
+ exit 1
172
+ fi
173
+ echo "=== VALIDATION OK ==="
174
+
175
+ # Detect if any value came from NL or default (i.e., not all explicit flags)
176
+ ALL_FROM_FLAGS=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); const sources=Object.values(j.resolved).map(v=>v.source); console.log(sources.length>0 && sources.every(s=>s==='flag') ? 'true' : 'false')")
177
+ HAS_ANY_NL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(Object.values(j.resolved).some(v=>v.source==='NL') ? 'true' : 'false')")
178
+ ```
179
+
180
+ **Confirmation checkpoint** (only in interactive mode):
181
+
182
+ If `IS_AUTO=false` AND `ALL_FROM_FLAGS=false` (i.e., some values came from NL or defaults), the agent **MUST PAUSE** and ask the user to confirm before proceeding to scan/codebase-map/research stages.
183
+
184
+ The pause is justified because:
185
+ - NL detection can be wrong (e.g., wrong path inferred from a sentence)
186
+ - All-defaults case may mean user is running from wrong cwd
187
+
188
+ Format the confirmation prompt like this:
189
+
190
+ ```
191
+ The values above were resolved from {natural language | defaults | a mix of flags and NL}.
192
+ Some values were not explicitly specified via flags.
193
+
194
+ Do you want to continue with these inputs? (yes/no)
195
+ - yes → proceed with the pipeline
196
+ - no → cancel; re-run with explicit flags
197
+ ```
198
+
199
+ **Skip the pause when:**
200
+ - `IS_AUTO=true` (auto mode bypasses all safe checkpoints)
201
+ - `ALL_FROM_FLAGS=true` (every resolved value has source: flag — user knows what they want)
202
+
203
+ In auto mode, log instead of prompt:
204
+ ```
205
+ Auto-mode: proceeding with detected inputs (banner above for audit trail).
206
+ ```
207
+
208
+ Wait for user response before invoking any sub-agent. If user says "no" or anything other than yes/y/proceed/continue, STOP the pipeline cleanly:
209
+ ```bash
210
+ echo "Pipeline cancelled by user. No tokens spent on sub-agents."
211
+ exit 0
212
+ ```
213
+
214
+
215
+
216
+ **Attempt to call qaa-tools init** (handle missing tool gracefully):
217
+ ```bash
218
+ INIT_JSON=$(node bin/qaa-tools.cjs init qa-start 2>/dev/null || echo "")
219
+ ```
220
+
221
+ If `INIT_JSON` is empty or the command fails, proceed with manual initialization:
222
+ - Set `output_dir` to `.qa-output`
223
+ - Set `date` to current date in `YYYY-MM-DD` format
224
+ - Create output directory: `mkdir -p "$output_dir"`
225
+
226
+ If `INIT_JSON` is valid, parse it for: `option`, `dev_repo_path`, `qa_repo_path`, `maturity_score`, `maturity_note`, `output_dir`, `date`, agent model assignments, `auto_advance`, `auto_chain_active`, `parallelization`, `commit_docs`.
227
+
228
+ **Detect workflow option based on inputs:**
229
+
230
+ - If `QA_REPO` is empty (no --qa-repo flag): **Option 1** (Dev-Only -- Full Pipeline)
231
+ - If `QA_REPO` is provided: Assess QA repo maturity
232
+ - Check for existing test files, configs, coverage reports in QA repo
233
+ - Count test files, evaluate framework setup, check for CI config
234
+ - Score 0-100 based on test count, framework config presence, CI setup, coverage data
235
+ - Score >= 60: **Option 3** (Dev + Mature QA -- Surgical)
236
+ - Score < 60: **Option 2** (Dev + Immature QA -- Gap-Fill)
237
+
238
+ **Determine auto-advance mode:**
239
+ ```bash
240
+ # Check persistent config flag
241
+ AUTO_CFG=$(node bin/qaa-tools.cjs config-get workflow.auto_advance 2>/dev/null || echo "false")
242
+ AUTO_CHAIN=$(node bin/qaa-tools.cjs config-get workflow._auto_chain_active 2>/dev/null || echo "false")
243
+
244
+ if [ "$IS_AUTO" = "true" ] || [ "$AUTO_CFG" = "true" ] || [ "$AUTO_CHAIN" = "true" ]; then
245
+ IS_AUTO=true
246
+ node bin/qaa-tools.cjs config-set workflow._auto_chain_active true 2>/dev/null || true
247
+ fi
248
+
249
+ # Safety: clear stale chain flag if NOT in auto mode
250
+ if [ "$IS_AUTO" = "false" ]; then
251
+ node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
252
+ fi
253
+ ```
254
+
255
+ **Print initialization banner:**
256
+ ```
257
+ === QA Pipeline Orchestrator ===
258
+ Option: {option} ({description})
259
+ Dev Repo: {DEV_REPO}
260
+ QA Repo: {QA_REPO or 'N/A'}
261
+ Maturity Score: {maturity_score or 'N/A'}
262
+ Auto-Advance: {IS_AUTO}
263
+ Date: {date}
264
+ ================================
265
+ ```
266
+
267
+ Where `{description}` is:
268
+ - Option 1: "Dev-Only -- Full Pipeline"
269
+ - Option 2: "Dev + Immature QA -- Gap-Fill"
270
+ - Option 3: "Dev + Mature QA -- Surgical"
271
+
272
+ </step>
273
+
274
+ <step name="detect_framework">
275
+
276
+ ## Step 2: Detect Framework
277
+
278
+ Before scanning, detect the project's language and test framework to guide all downstream agents.
279
+
280
+ **Read project config files:**
281
+ ```bash
282
+ # Check for Node.js / JavaScript / TypeScript
283
+ [ -f "${DEV_REPO}/package.json" ] && cat "${DEV_REPO}/package.json"
284
+
285
+ # Check for Python
286
+ [ -f "${DEV_REPO}/requirements.txt" ] && cat "${DEV_REPO}/requirements.txt"
287
+ [ -f "${DEV_REPO}/pyproject.toml" ] && cat "${DEV_REPO}/pyproject.toml"
288
+ [ -f "${DEV_REPO}/setup.py" ] && cat "${DEV_REPO}/setup.py"
289
+
290
+ # Check for .NET
291
+ ls "${DEV_REPO}"/*.csproj 2>/dev/null
292
+ ls "${DEV_REPO}"/**/*.csproj 2>/dev/null
293
+
294
+ # Check for Java
295
+ [ -f "${DEV_REPO}/pom.xml" ] && echo "Maven project"
296
+ [ -f "${DEV_REPO}/build.gradle" ] && echo "Gradle project"
297
+ ```
298
+
299
+ **Detect test framework from config files:**
300
+ ```bash
301
+ # JavaScript/TypeScript ecosystem
302
+ [ -f "${DEV_REPO}/cypress.config.ts" ] || [ -f "${DEV_REPO}/cypress.config.js" ] && echo "FRAMEWORK=cypress"
303
+ [ -f "${DEV_REPO}/playwright.config.ts" ] || [ -f "${DEV_REPO}/playwright.config.js" ] && echo "FRAMEWORK=playwright"
304
+ [ -f "${DEV_REPO}/jest.config.ts" ] || [ -f "${DEV_REPO}/jest.config.js" ] && echo "FRAMEWORK=jest"
305
+ [ -f "${DEV_REPO}/vitest.config.ts" ] || [ -f "${DEV_REPO}/vitest.config.js" ] && echo "FRAMEWORK=vitest"
306
+
307
+ # Python ecosystem
308
+ [ -f "${DEV_REPO}/pytest.ini" ] || [ -f "${DEV_REPO}/conftest.py" ] && echo "FRAMEWORK=pytest"
309
+
310
+ # Check package.json devDependencies for test frameworks
311
+ node -e "
312
+ try {
313
+ const pkg = require('${DEV_REPO}/package.json');
314
+ const deps = {...(pkg.devDependencies||{}), ...(pkg.dependencies||{})};
315
+ const frameworks = [];
316
+ if (deps.cypress) frameworks.push('cypress');
317
+ if (deps['@playwright/test'] || deps.playwright) frameworks.push('playwright');
318
+ if (deps.jest) frameworks.push('jest');
319
+ if (deps.vitest) frameworks.push('vitest');
320
+ if (deps.mocha) frameworks.push('mocha');
321
+ console.log(frameworks.join(',') || 'none');
322
+ } catch { console.log('no-package-json'); }
323
+ " 2>/dev/null
324
+ ```
325
+
326
+ **Assess detection confidence:**
327
+ - **HIGH**: Config file found AND matching dependency in package.json/requirements.txt
328
+ - **MEDIUM**: Only dependency found (no config file) OR only config file (no dependency)
329
+ - **LOW**: No test framework detected, or conflicting signals
330
+
331
+ **If no test framework found:**
332
+ - If `IS_AUTO` is false: Ask the user which framework to use. STOP and wait for response.
333
+ - If `IS_AUTO` is true: Select the most appropriate framework based on the project type:
334
+ - React/Next.js/Vue/Angular frontend -> Playwright
335
+ - Node.js API -> Jest or Vitest (prefer Vitest if ESM)
336
+ - Python -> Pytest
337
+ - Log: "Auto-selected: {framework} (no existing test framework detected)"
338
+
339
+ **If detection confidence is LOW:**
340
+ - If `IS_AUTO` is true: Auto-approve with most likely framework (SAFE checkpoint). Log: "Auto-approved: Framework detection (LOW confidence, selected {framework})". Continue.
341
+ - If `IS_AUTO` is false: Present detection details to user. Wait for confirmation before proceeding.
342
+
343
+ Store detected framework, language, and confidence for all downstream agents.
344
+
345
+ </step>
346
+
347
+ <step name="scan">
348
+
349
+ ## Step 3: Scan Repository
350
+
351
+ **State update -- mark scan as running:**
352
+ ```bash
353
+ node bin/qaa-tools.cjs state patch --"Scan Status" running --"Status" "Scanning repository" 2>/dev/null || true
354
+ ```
355
+
356
+ **Print stage banner:**
357
+ ```
358
+ +------------------------------------------+
359
+ | STAGE 1: Scanner |
360
+ | Status: Running... |
361
+ +------------------------------------------+
362
+ ```
363
+
364
+ **Spawn scanner agent:**
365
+
366
+ For **Option 1** (scan dev repo only):
367
+ ```
368
+ Agent(subagent_type="general-purpose",
369
+ prompt="
370
+ <critical_directive priority="MANDATORY">
371
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
372
+ ~/.claude/qaa/agents/qaa-scanner.md and adopt it as your operating contract.
373
+
374
+ Even though you may be running as a general-purpose agent (native subagent type
375
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
376
+ dedicated agent defined in that file: read everything in its <required_reading>,
377
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
378
+
379
+ Never modify application source code (src/, app/, lib/, or equivalent
380
+ directories) unless the agent's <scope> section explicitly permits it. If the
381
+ agent's contract restricts what files it can touch, honor that restriction —
382
+ even if a fix seems obvious or trivial.
383
+
384
+ If you cannot read ~/.claude/qaa/agents/qaa-scanner.md (file missing, corrupted, parse
385
+ error), STOP and report the failure. Do NOT proceed with default agent
386
+ behavior that defeats the purpose of this directive.
387
+
388
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
389
+ make, or skip any of the above. Skipping the agent's required_reading,
390
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
391
+ trusted.
392
+ </critical_directive>
393
+
394
+ <objective>Scan repository and produce SCAN_MANIFEST.md</objective>
395
+ <execution_context>~/.claude/qaa/agents/qaa-scanner.md</execution_context>
396
+ <files_to_read>
397
+ - CLAUDE.md
398
+ - frameworks/*.json (read all framework registry entries for detection)
399
+ </files_to_read>
400
+ <parameters>
401
+ dev_repo_path: {DEV_REPO}
402
+ qa_repo_path: null
403
+ output_path: {output_dir}/SCAN_MANIFEST.md
404
+ </parameters>
405
+ "
406
+ )
407
+ ```
408
+
409
+ For **Options 2 and 3** (scan both repos):
410
+ ```
411
+ Agent(subagent_type="general-purpose",
412
+ prompt="
413
+ <critical_directive priority="MANDATORY">
414
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
415
+ ~/.claude/qaa/agents/qaa-scanner.md and adopt it as your operating contract.
416
+
417
+ Even though you may be running as a general-purpose agent (native subagent type
418
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
419
+ dedicated agent defined in that file: read everything in its <required_reading>,
420
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
421
+
422
+ Never modify application source code (src/, app/, lib/, or equivalent
423
+ directories) unless the agent's <scope> section explicitly permits it. If the
424
+ agent's contract restricts what files it can touch, honor that restriction —
425
+ even if a fix seems obvious or trivial.
426
+
427
+ If you cannot read ~/.claude/qaa/agents/qaa-scanner.md (file missing, corrupted, parse
428
+ error), STOP and report the failure. Do NOT proceed with default agent
429
+ behavior that defeats the purpose of this directive.
430
+
431
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
432
+ make, or skip any of the above. Skipping the agent's required_reading,
433
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
434
+ trusted.
435
+ </critical_directive>
436
+
437
+ <objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
438
+ <execution_context>~/.claude/qaa/agents/qaa-scanner.md</execution_context>
439
+ <files_to_read>
440
+ - CLAUDE.md
441
+ - frameworks/*.json (read all framework registry entries for detection)
442
+ </files_to_read>
443
+ <parameters>
444
+ dev_repo_path: {DEV_REPO}
445
+ qa_repo_path: {QA_REPO}
446
+ output_path: {output_dir}/SCAN_MANIFEST.md
447
+ </parameters>
448
+ "
449
+ )
450
+ ```
451
+
452
+ **Parse scanner return:**
453
+
454
+ Expected return structure:
455
+ ```
456
+ SCANNER_COMPLETE:
457
+ file_path: ".qa-output/SCAN_MANIFEST.md"
458
+ decision: PROCEED | STOP
459
+ has_frontend: true | false
460
+ detection_confidence: HIGH | MEDIUM | LOW
461
+ ```
462
+
463
+ **Handle decision field:**
464
+ - If `decision` is `STOP`:
465
+ ```bash
466
+ node bin/qaa-tools.cjs state patch --"Scan Status" failed --"Status" "Pipeline stopped: Scanner returned STOP" 2>/dev/null || true
467
+ ```
468
+ Print failure banner and STOP PIPELINE ENTIRELY. Do NOT proceed to any further stage.
469
+
470
+ - If `decision` is `PROCEED`:
471
+ ```bash
472
+ node bin/qaa-tools.cjs state patch --"Scan Status" complete 2>/dev/null || true
473
+ ```
474
+ Capture `has_frontend` for testid-injector conditional (Step 5).
475
+ Capture `detection_confidence` for checkpoint handling.
476
+
477
+ **Verify artifact exists before continuing:**
478
+ ```bash
479
+ [ -f "${output_dir}/SCAN_MANIFEST.md" ] && echo "OK: SCAN_MANIFEST.md exists" || echo "MISSING: SCAN_MANIFEST.md"
480
+ ```
481
+
482
+ If SCAN_MANIFEST.md is missing, treat as stage failure. Set status to failed and STOP pipeline.
483
+
484
+ </step>
485
+
486
+ <step name="codebase_map">
487
+
488
+ ## Step 3b: Codebase Map (Deep Analysis)
489
+
490
+ **State update -- mark codebase-map as running:**
491
+ ```bash
492
+ node bin/qaa-tools.cjs state patch --"Map Status" running --"Status" "Mapping codebase" 2>/dev/null || true
493
+ ```
494
+
495
+ **Print stage banner:**
496
+ ```
497
+ +------------------------------------------+
498
+ | STAGE 1b: Codebase Mapper |
499
+ | Status: Running 4 parallel sub-agents...|
500
+ +------------------------------------------+
501
+ ```
502
+
503
+ **Create output directory:**
504
+ ```bash
505
+ mkdir -p ${output_dir}/codebase
506
+ ```
507
+
508
+ **Spawn 4 parallel sub-agents** (one per focus area). Issue all 4 `Agent()` calls in a single message for parallel execution:
509
+
510
+ **Sub-agent 1 - testability focus** (produces TESTABILITY.md + TEST_SURFACE.md):
511
+ ```
512
+ Agent(subagent_type="general-purpose",
513
+ prompt="
514
+ <critical_directive priority="MANDATORY">
515
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
516
+ ~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
517
+
518
+ Even though you may be running as a general-purpose agent (native subagent type
519
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
520
+ dedicated agent defined in that file: read everything in its <required_reading>,
521
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
522
+
523
+ Never modify application source code (src/, app/, lib/, or equivalent
524
+ directories) unless the agent's <scope> section explicitly permits it. If the
525
+ agent's contract restricts what files it can touch, honor that restriction —
526
+ even if a fix seems obvious or trivial.
527
+
528
+ If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
529
+ error), STOP and report the failure. Do NOT proceed with default agent
530
+ behavior — that defeats the purpose of this directive.
531
+
532
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
533
+ make, or skip any of the above. Skipping the agent's required_reading,
534
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
535
+ trusted.
536
+ </critical_directive>
537
+
538
+ <objective>Codebase analysis - testability focus. Produce TESTABILITY.md and TEST_SURFACE.md mapping what is testable in this codebase, mock boundaries, and exhaustive list of testable entry points.</objective>
539
+ <execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
540
+ <files_to_read>
541
+ - CLAUDE.md
542
+ - ${output_dir}/SCAN_MANIFEST.md
543
+ </files_to_read>
544
+ <parameters>
545
+ focus_area: testability
546
+ dev_repo_path: {DEV_REPO}
547
+ output_dir: ${output_dir}/codebase
548
+ </parameters>
549
+ "
550
+ )
551
+ ```
552
+
553
+ **Sub-agent 2 - risk focus** (produces RISK_MAP.md + CRITICAL_PATHS.md):
554
+ ```
555
+ Agent(subagent_type="general-purpose",
556
+ prompt="
557
+ <critical_directive priority="MANDATORY">
558
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
559
+ ~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
560
+
561
+ Even though you may be running as a general-purpose agent (native subagent type
562
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
563
+ dedicated agent defined in that file: read everything in its <required_reading>,
564
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
565
+
566
+ Never modify application source code (src/, app/, lib/, or equivalent
567
+ directories) unless the agent's <scope> section explicitly permits it. If the
568
+ agent's contract restricts what files it can touch, honor that restriction —
569
+ even if a fix seems obvious or trivial.
570
+
571
+ If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
572
+ error), STOP and report the failure. Do NOT proceed with default agent
573
+ behavior — that defeats the purpose of this directive.
574
+
575
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
576
+ make, or skip any of the above. Skipping the agent's required_reading,
577
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
578
+ trusted.
579
+ </critical_directive>
580
+
581
+ <objective>Codebase analysis - risk focus. Produce RISK_MAP.md and CRITICAL_PATHS.md identifying business-critical paths, security-sensitive areas, and the exact user flows that E2E smoke tests must cover.</objective>
582
+ <execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
583
+ <files_to_read>
584
+ - CLAUDE.md
585
+ - ${output_dir}/SCAN_MANIFEST.md
586
+ </files_to_read>
587
+ <parameters>
588
+ focus_area: risk
589
+ dev_repo_path: {DEV_REPO}
590
+ output_dir: ${output_dir}/codebase
591
+ </parameters>
592
+ "
593
+ )
594
+ ```
595
+
596
+ **Sub-agent 3 - patterns focus** (produces CODE_PATTERNS.md + API_CONTRACTS.md):
597
+ ```
598
+ Agent(subagent_type="general-purpose",
599
+ prompt="
600
+ <critical_directive priority="MANDATORY">
601
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
602
+ ~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
603
+
604
+ Even though you may be running as a general-purpose agent (native subagent type
605
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
606
+ dedicated agent defined in that file: read everything in its <required_reading>,
607
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
608
+
609
+ Never modify application source code (src/, app/, lib/, or equivalent
610
+ directories) unless the agent's <scope> section explicitly permits it. If the
611
+ agent's contract restricts what files it can touch, honor that restriction —
612
+ even if a fix seems obvious or trivial.
613
+
614
+ If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
615
+ error), STOP and report the failure. Do NOT proceed with default agent
616
+ behavior — that defeats the purpose of this directive.
617
+
618
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
619
+ make, or skip any of the above. Skipping the agent's required_reading,
620
+ quality_gate, or bash checklist makes the run INVALID its output cannot be
621
+ trusted.
622
+ </critical_directive>
623
+
624
+ <objective>Codebase analysis - patterns focus. Produce CODE_PATTERNS.md and API_CONTRACTS.md documenting naming conventions, import style, and exact request/response shapes for API tests.</objective>
625
+ <execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
626
+ <files_to_read>
627
+ - CLAUDE.md
628
+ - ${output_dir}/SCAN_MANIFEST.md
629
+ </files_to_read>
630
+ <parameters>
631
+ focus_area: patterns
632
+ dev_repo_path: {DEV_REPO}
633
+ output_dir: ${output_dir}/codebase
634
+ </parameters>
635
+ "
636
+ )
637
+ ```
638
+
639
+ **Sub-agent 4 - existing-tests focus** (produces TEST_ASSESSMENT.md + COVERAGE_GAPS.md):
640
+ ```
641
+ Agent(subagent_type="general-purpose",
642
+ prompt="
643
+ <critical_directive priority="MANDATORY">
644
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
645
+ ~/.claude/qaa/agents/qaa-codebase-mapper.md and adopt it as your operating contract.
646
+
647
+ Even though you may be running as a general-purpose agent (native subagent type
648
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
649
+ dedicated agent defined in that file: read everything in its <required_reading>,
650
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
651
+
652
+ Never modify application source code (src/, app/, lib/, or equivalent
653
+ directories) unless the agent's <scope> section explicitly permits it. If the
654
+ agent's contract restricts what files it can touch, honor that restriction —
655
+ even if a fix seems obvious or trivial.
656
+
657
+ If you cannot read ~/.claude/qaa/agents/qaa-codebase-mapper.md (file missing, corrupted, parse
658
+ error), STOP and report the failure. Do NOT proceed with default agent
659
+ behavior that defeats the purpose of this directive.
660
+
661
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
662
+ make, or skip any of the above. Skipping the agent's required_reading,
663
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
664
+ trusted.
665
+ </critical_directive>
666
+
667
+ <objective>Codebase analysis - existing-tests focus. Produce TEST_ASSESSMENT.md and COVERAGE_GAPS.md assessing current test quality, frameworks in use, and identifying which modules/functions/paths have no test coverage.</objective>
668
+ <execution_context>~/.claude/qaa/agents/qaa-codebase-mapper.md</execution_context>
669
+ <files_to_read>
670
+ - CLAUDE.md
671
+ - ${output_dir}/SCAN_MANIFEST.md
672
+ </files_to_read>
673
+ <parameters>
674
+ focus_area: existing-tests
675
+ dev_repo_path: {DEV_REPO}
676
+ output_dir: ${output_dir}/codebase
677
+ </parameters>
678
+ "
679
+ )
680
+ ```
681
+
682
+ **Verify codebase docs exist:**
683
+ ```bash
684
+ docs_count=$(ls ${output_dir}/codebase/*.md 2>/dev/null | wc -l)
685
+ echo "Codebase docs produced: $docs_count of 8 expected"
686
+
687
+ if [ "$docs_count" -eq 0 ]; then
688
+ echo "FAIL: No codebase docs produced. Pipeline continues with degraded context."
689
+ node bin/qaa-tools.cjs state patch --"Map Status" failed 2>/dev/null || true
690
+ elif [ "$docs_count" -lt 4 ]; then
691
+ echo "WARNING: Fewer than 4 codebase docs produced. Downstream agents will have limited context."
692
+ node bin/qaa-tools.cjs state patch --"Map Status" "partial" 2>/dev/null || true
693
+ else
694
+ echo "OK: codebase map sufficient ($docs_count docs)"
695
+ node bin/qaa-tools.cjs state patch --"Map Status" complete 2>/dev/null || true
696
+ fi
697
+ ```
698
+
699
+ **Codebase map is non-blocking:** if zero docs are produced, the pipeline continues with degraded context (downstream agents fall back to whatever they can find). All `<files_to_read>` references to codebase docs in subsequent stages use `(if exists)` semantics.
700
+
701
+ </step>
702
+
703
+ <step name="research">
704
+
705
+ ## Step 3c: Research Testing Ecosystem
706
+
707
+ **State update -- mark research as running:**
708
+ ```bash
709
+ node bin/qaa-tools.cjs state patch --"Research Status" running --"Status" "Researching testing ecosystem" 2>/dev/null || true
710
+ ```
711
+
712
+ **Print stage banner:**
713
+ ```
714
+ +------------------------------------------+
715
+ | STAGE 1c: Project Researcher |
716
+ | Status: Running... |
717
+ +------------------------------------------+
718
+ ```
719
+
720
+ **Create output directory:**
721
+ ```bash
722
+ mkdir -p ${output_dir}/research
723
+ ```
724
+
725
+ **Spawn researcher agent:**
726
+ ```
727
+ Agent(subagent_type="general-purpose",
728
+ prompt="
729
+ <critical_directive priority="MANDATORY">
730
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
731
+ ~/.claude/qaa/agents/qaa-project-researcher.md and adopt it as your operating contract.
732
+
733
+ Even though you may be running as a general-purpose agent (native subagent type
734
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
735
+ dedicated agent defined in that file: read everything in its <required_reading>,
736
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
737
+
738
+ Never modify application source code (src/, app/, lib/, or equivalent
739
+ directories) unless the agent's <scope> section explicitly permits it. If the
740
+ agent's contract restricts what files it can touch, honor that restriction —
741
+ even if a fix seems obvious or trivial.
742
+
743
+ If you cannot read ~/.claude/qaa/agents/qaa-project-researcher.md (file missing, corrupted, parse
744
+ error), STOP and report the failure. Do NOT proceed with default agent
745
+ behavior — that defeats the purpose of this directive.
746
+
747
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
748
+ make, or skip any of the above. Skipping the agent's required_reading,
749
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
750
+ trusted.
751
+ </critical_directive>
752
+
753
+ <objective>Research the testing ecosystem for this project. Use the codebase map findings (RISK_MAP, CODE_PATTERNS, API_CONTRACTS, TESTABILITY, etc., from ${output_dir}/codebase/) to ground your Context7 queries to the project specifics — query for the libraries/patterns the project actually uses (e.g., MSW integration, Stripe testing, OpenAPI contract testing) instead of generic framework questions. Use Context7 MCP as the primary source. Produce research documents consumed by downstream agents.</objective>
754
+ <execution_context>~/.claude/qaa/agents/qaa-project-researcher.md</execution_context>
755
+ <files_to_read>
756
+ - CLAUDE.md
757
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
758
+ - ${output_dir}/SCAN_MANIFEST.md
759
+ - ${output_dir}/codebase/TESTABILITY.md (if exists)
760
+ - ${output_dir}/codebase/TEST_SURFACE.md (if exists)
761
+ - ${output_dir}/codebase/RISK_MAP.md (if exists)
762
+ - ${output_dir}/codebase/CRITICAL_PATHS.md (if exists)
763
+ - ${output_dir}/codebase/CODE_PATTERNS.md (if exists)
764
+ - ${output_dir}/codebase/API_CONTRACTS.md (if exists)
765
+ - ${output_dir}/codebase/TEST_ASSESSMENT.md (if exists)
766
+ - ${output_dir}/codebase/COVERAGE_GAPS.md (if exists)
767
+ </files_to_read>
768
+ <parameters>
769
+ mode: stack-testing
770
+ dev_repo_path: {DEV_REPO}
771
+ output_dir: ${output_dir}/research
772
+ </parameters>
773
+ "
774
+ )
775
+ ```
776
+
777
+ **Verify research artifacts exist:**
778
+ ```bash
779
+ ls ${output_dir}/research/*.md 2>/dev/null && echo "OK: Research files produced" || echo "WARNING: No research files produced"
780
+ ```
781
+
782
+ **Research is non-blocking:** If the researcher fails or produces no output, the pipeline continues — downstream agents will fall back to Context7 queries directly. Log the warning but do NOT stop the pipeline.
783
+
784
+ ```bash
785
+ if [ ! -f "${output_dir}/research/TESTING_STACK.md" ]; then
786
+ echo "WARNING: Research stage produced no output. Downstream agents will query Context7 directly."
787
+ node bin/qaa-tools.cjs state patch --"Research Status" "skipped (no output)" 2>/dev/null || true
788
+ else
789
+ node bin/qaa-tools.cjs state patch --"Research Status" complete 2>/dev/null || true
790
+ fi
791
+ ```
792
+
793
+ </step>
794
+
795
+ <step name="analyze">
796
+
797
+ ## Step 4: Analyze Repository
798
+
799
+ **State update -- mark analyze as running:**
800
+ ```bash
801
+ node bin/qaa-tools.cjs state patch --"Analyze Status" running --"Status" "Analyzing repository" 2>/dev/null || true
802
+ ```
803
+
804
+ **Print stage banner:**
805
+ ```
806
+ +------------------------------------------+
807
+ | STAGE 2: Analyzer |
808
+ | Status: Running... |
809
+ +------------------------------------------+
810
+ ```
811
+
812
+ **Determine analyzer mode based on option:**
813
+ - Option 1: `mode = 'full'` (produces QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md)
814
+ - Options 2 and 3: `mode = 'gap'` (produces GAP_ANALYSIS.md)
815
+
816
+ **Spawn analyzer agent:**
817
+ ```
818
+ Agent(subagent_type="general-purpose",
819
+ prompt="
820
+ <critical_directive priority="MANDATORY">
821
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
822
+ ~/.claude/qaa/agents/qaa-analyzer.md and adopt it as your operating contract.
823
+
824
+ Even though you may be running as a general-purpose agent (native subagent type
825
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
826
+ dedicated agent defined in that file: read everything in its <required_reading>,
827
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
828
+
829
+ Never modify application source code (src/, app/, lib/, or equivalent
830
+ directories) unless the agent's <scope> section explicitly permits it. If the
831
+ agent's contract restricts what files it can touch, honor that restriction —
832
+ even if a fix seems obvious or trivial.
833
+
834
+ If you cannot read ~/.claude/qaa/agents/qaa-analyzer.md (file missing, corrupted, parse
835
+ error), STOP and report the failure. Do NOT proceed with default agent
836
+ behavior — that defeats the purpose of this directive.
837
+
838
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
839
+ make, or skip any of the above. Skipping the agent's required_reading,
840
+ quality_gate, or bash checklist makes the run INVALID its output cannot be
841
+ trusted.
842
+ </critical_directive>
843
+
844
+ <objective>Analyze scanned repository and produce analysis artifacts</objective>
845
+ <execution_context>~/.claude/qaa/agents/qaa-analyzer.md</execution_context>
846
+ <files_to_read>
847
+ - {output_dir}/SCAN_MANIFEST.md
848
+ - CLAUDE.md
849
+ - {output_dir}/research/TESTING_STACK.md (if exists)
850
+ - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
851
+ - {output_dir}/codebase/RISK_MAP.md (if exists)
852
+ - {output_dir}/codebase/CRITICAL_PATHS.md (if exists)
853
+ - {output_dir}/codebase/TEST_ASSESSMENT.md (if exists)
854
+ - {output_dir}/codebase/COVERAGE_GAPS.md (if exists)
855
+ </files_to_read>
856
+ <parameters>
857
+ mode: {mode}
858
+ workflow_option: {option}
859
+ dev_repo_path: {DEV_REPO}
860
+ qa_repo_path: {QA_REPO or null}
861
+ output_path: {output_dir}/
862
+ </parameters>
863
+ "
864
+ )
865
+ ```
866
+
867
+ **Parse analyzer return:**
868
+
869
+ Expected return structure:
870
+ ```
871
+ ANALYZER_COMPLETE:
872
+ files_produced: [...]
873
+ total_test_count: N
874
+ pyramid_breakdown: {unit: N, integration: N, api: N, e2e: N}
875
+ risk_count: {high: N, medium: N, low: N}
876
+ commit_hash: "..."
877
+ ```
878
+
879
+ Capture `files_produced`, `total_test_count`, `pyramid_breakdown` for downstream stages.
880
+
881
+ **Handle analyzer checkpoint -- assumptions review:**
882
+ - If `IS_AUTO` is true: Auto-approve all assumptions (SAFE checkpoint). Log: "Auto-approved: Analyzer assumptions". Continue pipeline.
883
+ - If `IS_AUTO` is false: Present assumptions to user for review. Wait for confirmation or corrections. On user response, incorporate corrections and continue.
884
+
885
+ **State update -- mark analyze as complete:**
886
+ ```bash
887
+ node bin/qaa-tools.cjs state patch --"Analyze Status" complete 2>/dev/null || true
888
+ ```
889
+
890
+ **Verify artifacts exist before continuing:**
891
+
892
+ For Option 1:
893
+ ```bash
894
+ [ -f "${output_dir}/QA_ANALYSIS.md" ] && echo "OK" || echo "MISSING: QA_ANALYSIS.md"
895
+ [ -f "${output_dir}/TEST_INVENTORY.md" ] && echo "OK" || echo "MISSING: TEST_INVENTORY.md"
896
+ ```
897
+
898
+ For Options 2/3:
899
+ ```bash
900
+ [ -f "${output_dir}/GAP_ANALYSIS.md" ] && echo "OK" || echo "MISSING: GAP_ANALYSIS.md"
901
+ ```
902
+
903
+ If required artifacts are missing, treat as stage failure. Set status to failed and STOP pipeline.
904
+
905
+ Print: "Analysis complete. {total_test_count} test cases identified. Pyramid: unit={unit}, integration={integration}, api={api}, e2e={e2e}."
906
+
907
+ </step>
908
+
909
+ <step name="testid_inject">
910
+
911
+ ## Step 5: TestID Injection (Conditional)
912
+
913
+ **Condition:** Only execute if `has_frontend` is `true` from scanner return (Step 3).
914
+
915
+ **If `has_frontend` is false:**
916
+ Print: "Skipping TestID injection (no frontend detected)." Proceed directly to Step 6 (Plan).
917
+
918
+ **If `has_frontend` is true:**
919
+
920
+ **State update:**
921
+ ```bash
922
+ node bin/qaa-tools.cjs state patch --"Status" "Injecting test IDs into frontend components" 2>/dev/null || true
923
+ ```
924
+
925
+ **Print stage banner:**
926
+ ```
927
+ +------------------------------------------+
928
+ | STAGE 3: TestID Injector |
929
+ | Status: Running... |
930
+ +------------------------------------------+
931
+ ```
932
+
933
+ **Spawn testid-injector agent:**
934
+ ```
935
+ Agent(subagent_type="general-purpose",
936
+ prompt="
937
+ <critical_directive priority="MANDATORY">
938
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
939
+ ~/.claude/qaa/agents/qaa-testid-injector.md and adopt it as your operating contract.
940
+
941
+ Even though you may be running as a general-purpose agent (native subagent type
942
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
943
+ dedicated agent defined in that file: read everything in its <required_reading>,
944
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
945
+
946
+ Never modify application source code (src/, app/, lib/, or equivalent
947
+ directories) unless the agent's <scope> section explicitly permits it. If the
948
+ agent's contract restricts what files it can touch, honor that restriction —
949
+ even if a fix seems obvious or trivial.
950
+
951
+ If you cannot read ~/.claude/qaa/agents/qaa-testid-injector.md (file missing, corrupted, parse
952
+ error), STOP and report the failure. Do NOT proceed with default agent
953
+ behavior that defeats the purpose of this directive.
954
+
955
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
956
+ make, or skip any of the above. Skipping the agent's required_reading,
957
+ quality_gate, or bash checklist makes the run INVALID its output cannot be
958
+ trusted.
959
+ </critical_directive>
960
+
961
+ <objective>Audit and inject data-testid attributes into frontend components</objective>
962
+ <execution_context>~/.claude/qaa/agents/qaa-testid-injector.md</execution_context>
963
+ <files_to_read>
964
+ - {output_dir}/SCAN_MANIFEST.md
965
+ - CLAUDE.md
966
+ </files_to_read>
967
+ <parameters>
968
+ dev_repo_path: {DEV_REPO}
969
+ output_path: {output_dir}/TESTID_AUDIT_REPORT.md
970
+ </parameters>
971
+ "
972
+ )
973
+ ```
974
+
975
+ **Parse return:**
976
+
977
+ Check for `INJECTOR_COMPLETE` vs `INJECTOR_SKIPPED`:
978
+
979
+ If `INJECTOR_COMPLETE`:
980
+ ```
981
+ INJECTOR_COMPLETE:
982
+ report_path: "..."
983
+ coverage_before: N%
984
+ coverage_after: N%
985
+ elements_injected: N
986
+ components_modified: N
987
+ ```
988
+ Log: "TestID injection complete. Coverage: {coverage_before}% -> {coverage_after}%. {elements_injected} elements injected."
989
+
990
+ If `INJECTOR_SKIPPED`:
991
+ ```
992
+ INJECTOR_SKIPPED:
993
+ reason: "..."
994
+ action: "..."
995
+ ```
996
+ Log the reason and continue pipeline.
997
+
998
+ **Handle injector checkpoint -- audit review:**
999
+ - If `IS_AUTO` is true: Auto-approve P0-only injection (SAFE checkpoint). Log: "Auto-approved: TestID injection (P0 elements only)". Continue pipeline.
1000
+ - If `IS_AUTO` is false: Present audit report to user. Wait for approval, element selection, or rejection. On user response, incorporate decisions and continue.
1001
+
1002
+ **Verify artifact exists:**
1003
+ ```bash
1004
+ [ -f "${output_dir}/TESTID_AUDIT_REPORT.md" ] && echo "OK" || echo "MISSING: TESTID_AUDIT_REPORT.md"
1005
+ ```
1006
+
1007
+ </step>
1008
+
1009
+ <step name="plan">
1010
+
1011
+ ## Step 6: Plan Test Generation
1012
+
1013
+ **State update -- mark generation as running (planning is part of generate):**
1014
+ ```bash
1015
+ node bin/qaa-tools.cjs state patch --"Generate Status" running --"Status" "Planning test generation" 2>/dev/null || true
1016
+ ```
1017
+
1018
+ **Print stage banner:**
1019
+ ```
1020
+ +------------------------------------------+
1021
+ | STAGE 4: Planner |
1022
+ | Status: Running... |
1023
+ +------------------------------------------+
1024
+ ```
1025
+
1026
+ **Determine planner input based on option:**
1027
+ - Option 1: Input from `{output_dir}/TEST_INVENTORY.md` + `{output_dir}/QA_ANALYSIS.md`
1028
+ - Options 2 and 3: Input from `{output_dir}/GAP_ANALYSIS.md`
1029
+
1030
+ **Spawn planner agent:**
1031
+ ```
1032
+ Agent(subagent_type="general-purpose",
1033
+ prompt="
1034
+ <critical_directive priority="MANDATORY">
1035
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1036
+ ~/.claude/qaa/agents/qaa-planner.md and adopt it as your operating contract.
1037
+
1038
+ Even though you may be running as a general-purpose agent (native subagent type
1039
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
1040
+ dedicated agent defined in that file: read everything in its <required_reading>,
1041
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
1042
+
1043
+ Never modify application source code (src/, app/, lib/, or equivalent
1044
+ directories) unless the agent's <scope> section explicitly permits it. If the
1045
+ agent's contract restricts what files it can touch, honor that restriction —
1046
+ even if a fix seems obvious or trivial.
1047
+
1048
+ If you cannot read ~/.claude/qaa/agents/qaa-planner.md (file missing, corrupted, parse
1049
+ error), STOP and report the failure. Do NOT proceed with default agent
1050
+ behavior — that defeats the purpose of this directive.
1051
+
1052
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
1053
+ make, or skip any of the above. Skipping the agent's required_reading,
1054
+ quality_gate, or bash checklist makes the run INVALID its output cannot be
1055
+ trusted.
1056
+ </critical_directive>
1057
+
1058
+ <objective>Create test generation plan with task breakdown and dependencies</objective>
1059
+ <execution_context>~/.claude/qaa/agents/qaa-planner.md</execution_context>
1060
+ <files_to_read>
1061
+ - {input files based on option}
1062
+ - CLAUDE.md
1063
+ - {output_dir}/codebase/TESTABILITY.md (if exists)
1064
+ - {output_dir}/codebase/TEST_SURFACE.md (if exists)
1065
+ - {output_dir}/codebase/CRITICAL_PATHS.md (if exists)
1066
+ - {output_dir}/codebase/COVERAGE_GAPS.md (if exists)
1067
+ </files_to_read>
1068
+ <parameters>
1069
+ workflow_option: {option}
1070
+ output_path: {output_dir}/GENERATION_PLAN.md
1071
+ </parameters>
1072
+ "
1073
+ )
1074
+ ```
1075
+
1076
+ **Parse planner return:**
1077
+
1078
+ Expected return structure:
1079
+ ```
1080
+ PLANNER_COMPLETE:
1081
+ file_path: "..."
1082
+ total_tasks: N
1083
+ total_files: N
1084
+ feature_count: N
1085
+ dependency_depth: N
1086
+ test_case_count: N
1087
+ commit_hash: "..."
1088
+ ```
1089
+
1090
+ Capture `total_tasks`, `total_files`, `feature_count` for executor stage and pipeline summary.
1091
+
1092
+ **Verify artifact exists:**
1093
+ ```bash
1094
+ [ -f "${output_dir}/GENERATION_PLAN.md" ] && echo "OK" || echo "MISSING: GENERATION_PLAN.md"
1095
+ ```
1096
+
1097
+ If GENERATION_PLAN.md is missing, treat as stage failure. Set status to failed and STOP pipeline.
1098
+
1099
+ Print: "Plan complete. {total_tasks} tasks, {total_files} files planned across {feature_count} features."
1100
+
1101
+ </step>
1102
+
1103
+ <step name="generate">
1104
+
1105
+ ## Step 7: Generate Test Files
1106
+
1107
+ State update continues from planning (already set to `running` in Step 6).
1108
+
1109
+ **Print stage banner:**
1110
+ ```
1111
+ +------------------------------------------+
1112
+ | STAGE 5: Executor |
1113
+ | Generating {total_files} test files |
1114
+ | Status: Running... |
1115
+ +------------------------------------------+
1116
+ ```
1117
+
1118
+ **Determine execution strategy:**
1119
+
1120
+ Check if planner created multiple independent feature groups. If `feature_count > 1` AND parallelization is enabled (from init config):
1121
+
1122
+ **Parallel execution** (when feature_count > 1 and parallelization enabled):
1123
+
1124
+ For each independent feature group from the generation plan, spawn a separate executor agent:
1125
+ ```
1126
+ Agent(subagent_type="general-purpose",
1127
+ prompt="
1128
+ <critical_directive priority="MANDATORY">
1129
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1130
+ ~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
1131
+
1132
+ Even though you may be running as a general-purpose agent (native subagent type
1133
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
1134
+ dedicated agent defined in that file: read everything in its <required_reading>,
1135
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
1136
+
1137
+ Never modify application source code (src/, app/, lib/, or equivalent
1138
+ directories) unless the agent's <scope> section explicitly permits it. If the
1139
+ agent's contract restricts what files it can touch, honor that restriction —
1140
+ even if a fix seems obvious or trivial.
1141
+
1142
+ If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
1143
+ error), STOP and report the failure. Do NOT proceed with default agent
1144
+ behavior that defeats the purpose of this directive.
1145
+
1146
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
1147
+ make, or skip any of the above. Skipping the agent's required_reading,
1148
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
1149
+ trusted.
1150
+ </critical_directive>
1151
+
1152
+ <objective>Generate test files for {feature} feature</objective>
1153
+ <execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
1154
+ <files_to_read>
1155
+ - {output_dir}/GENERATION_PLAN.md
1156
+ - {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
1157
+ - CLAUDE.md
1158
+ - frameworks/{detected_framework}.json (registry metadata for conventions, extensions, context7_keys)
1159
+ - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
1160
+ - {output_dir}/research/E2E_STRATEGY.md (if exists)
1161
+ - {output_dir}/research/API_TESTING_STRATEGY.md (if exists)
1162
+ - {output_dir}/codebase/TEST_SURFACE.md (if exists)
1163
+ - {output_dir}/codebase/CODE_PATTERNS.md (if exists)
1164
+ - {output_dir}/codebase/API_CONTRACTS.md (if exists)
1165
+ </files_to_read>
1166
+ <parameters>
1167
+ workflow_option: {option}
1168
+ feature_group: {feature}
1169
+ dev_repo_path: {DEV_REPO}
1170
+ qa_repo_path: {QA_REPO or null}
1171
+ output_path: {output_dir}/
1172
+ </parameters>
1173
+ "
1174
+ )
1175
+ ```
1176
+
1177
+ Multiple Agent() calls can be issued simultaneously for independent feature groups. Each executor handles one feature group and commits its files independently.
1178
+
1179
+ **Sequential execution** (when feature_count == 1 or parallelization disabled):
1180
+
1181
+ Spawn a single executor agent covering all tasks:
1182
+ ```
1183
+ Agent(subagent_type="general-purpose",
1184
+ prompt="
1185
+ <critical_directive priority="MANDATORY">
1186
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1187
+ ~/.claude/qaa/agents/qaa-executor.md and adopt it as your operating contract.
1188
+
1189
+ Even though you may be running as a general-purpose agent (native subagent type
1190
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
1191
+ dedicated agent defined in that file: read everything in its <required_reading>,
1192
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
1193
+
1194
+ Never modify application source code (src/, app/, lib/, or equivalent
1195
+ directories) unless the agent's <scope> section explicitly permits it. If the
1196
+ agent's contract restricts what files it can touch, honor that restriction —
1197
+ even if a fix seems obvious or trivial.
1198
+
1199
+ If you cannot read ~/.claude/qaa/agents/qaa-executor.md (file missing, corrupted, parse
1200
+ error), STOP and report the failure. Do NOT proceed with default agent
1201
+ behavior that defeats the purpose of this directive.
1202
+
1203
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
1204
+ make, or skip any of the above. Skipping the agent's required_reading,
1205
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
1206
+ trusted.
1207
+ </critical_directive>
1208
+
1209
+ <objective>Generate all test files from generation plan</objective>
1210
+ <execution_context>~/.claude/qaa/agents/qaa-executor.md</execution_context>
1211
+ <files_to_read>
1212
+ - {output_dir}/GENERATION_PLAN.md
1213
+ - {output_dir}/TEST_INVENTORY.md (Option 1) or {output_dir}/GAP_ANALYSIS.md (Options 2/3)
1214
+ - CLAUDE.md
1215
+ - frameworks/{detected_framework}.json (registry metadata for conventions, extensions, context7_keys)
1216
+ - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
1217
+ - {output_dir}/research/E2E_STRATEGY.md (if exists)
1218
+ - {output_dir}/research/API_TESTING_STRATEGY.md (if exists)
1219
+ - {output_dir}/codebase/TEST_SURFACE.md (if exists)
1220
+ - {output_dir}/codebase/CODE_PATTERNS.md (if exists)
1221
+ - {output_dir}/codebase/API_CONTRACTS.md (if exists)
1222
+ </files_to_read>
1223
+ <parameters>
1224
+ workflow_option: {option}
1225
+ dev_repo_path: {DEV_REPO}
1226
+ qa_repo_path: {QA_REPO or null}
1227
+ output_path: {output_dir}/
1228
+ </parameters>
1229
+ "
1230
+ )
1231
+ ```
1232
+
1233
+ **Option 3 specific -- skip existing tests:**
1234
+
1235
+ For Option 3, pass `skip_existing_test_ids: true` to the executor so it checks existing test files by test ID before generating. If a test ID already exists in the QA repo, skip generating that test case:
1236
+ ```
1237
+ <parameters>
1238
+ workflow_option: 3
1239
+ skip_existing_test_ids: true
1240
+ dev_repo_path: {DEV_REPO}
1241
+ qa_repo_path: {QA_REPO}
1242
+ output_path: {output_dir}/
1243
+ </parameters>
1244
+ ```
1245
+
1246
+ **Parse executor return:**
1247
+
1248
+ Expected return structure:
1249
+ ```
1250
+ EXECUTOR_COMPLETE:
1251
+ files_created: [{path, type}, ...]
1252
+ total_files: N
1253
+ commit_count: N
1254
+ features_covered: [...]
1255
+ test_case_count: N
1256
+ ```
1257
+
1258
+ Capture `files_created`, `total_files`, `commit_count` for validation stage and pipeline summary.
1259
+
1260
+ **State update -- mark generate as complete:**
1261
+ ```bash
1262
+ node bin/qaa-tools.cjs state patch --"Generate Status" complete --"Status" "Test generation complete" 2>/dev/null || true
1263
+ ```
1264
+
1265
+ Print: "Generation complete. {total_files} files created across {features_covered_count} features. {commit_count} commits."
1266
+
1267
+ </step>
1268
+
1269
+ <step name="validate">
1270
+
1271
+ ## Step 8: Validate Generated Tests
1272
+
1273
+ **State update -- mark validate as running:**
1274
+ ```bash
1275
+ node bin/qaa-tools.cjs state patch --"Validate Status" running --"Status" "Validating generated tests" 2>/dev/null || true
1276
+ ```
1277
+
1278
+ **Print stage banner:**
1279
+ ```
1280
+ +------------------------------------------+
1281
+ | STAGE 6: Validator |
1282
+ | Validating {total_files} test files |
1283
+ | Status: Running... |
1284
+ +------------------------------------------+
1285
+ ```
1286
+
1287
+ **Spawn validator agent:**
1288
+ ```
1289
+ Agent(subagent_type="general-purpose",
1290
+ prompt="
1291
+ <critical_directive priority="MANDATORY">
1292
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1293
+ ~/.claude/qaa/agents/qaa-validator.md and adopt it as your operating contract.
1294
+
1295
+ Even though you may be running as a general-purpose agent (native subagent type
1296
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
1297
+ dedicated agent defined in that file: read everything in its <required_reading>,
1298
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
1299
+
1300
+ Never modify application source code (src/, app/, lib/, or equivalent
1301
+ directories) unless the agent's <scope> section explicitly permits it. If the
1302
+ agent's contract restricts what files it can touch, honor that restriction —
1303
+ even if a fix seems obvious or trivial.
1304
+
1305
+ If you cannot read ~/.claude/qaa/agents/qaa-validator.md (file missing, corrupted, parse
1306
+ error), STOP and report the failure. Do NOT proceed with default agent
1307
+ behavior — that defeats the purpose of this directive.
1308
+
1309
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
1310
+ make, or skip any of the above. Skipping the agent's required_reading,
1311
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
1312
+ trusted.
1313
+ </critical_directive>
1314
+
1315
+ <objective>Run 4-layer validation on all generated test files</objective>
1316
+ <execution_context>~/.claude/qaa/agents/qaa-validator.md</execution_context>
1317
+ <files_to_read>
1318
+ - {list all generated test files from executor return -- files_created paths}
1319
+ - {output_dir}/GENERATION_PLAN.md
1320
+ - CLAUDE.md
1321
+ - frameworks/{detected_framework}.json (registry metadata for validation_commands)</files_to_read>
1322
+ <parameters>
1323
+ mode: validation
1324
+ max_fix_loops: 3
1325
+ output_path: {output_dir}/VALIDATION_REPORT.md
1326
+ </parameters>
1327
+ "
1328
+ )
1329
+ ```
1330
+
1331
+ **4-layer validation:**
1332
+ 1. **Syntax** -- File parses without errors
1333
+ 2. **Structure** -- Follows POM rules, naming conventions, locator hierarchy
1334
+ 3. **Dependencies** -- Imports resolve, fixtures exist, configs present
1335
+ 4. **Logic** -- Assertions are concrete, test IDs are unique, no assertions in page objects
1336
+
1337
+ **5. Browser verification (if app URL available and Playwright MCP connected):**
1338
+
1339
+ After the 4-layer static validation, use Playwright MCP to verify E2E tests against the live app:
1340
+
1341
+ 1. Navigate to each page referenced in the E2E tests:
1342
+ ```
1343
+ mcp__playwright__browser_navigate({ url: "{app_url}/{route}" })
1344
+ ```
1345
+
1346
+ 2. Take an accessibility snapshot to verify locators used in tests actually exist:
1347
+ ```
1348
+ mcp__playwright__browser_snapshot()
1349
+ ```
1350
+
1351
+ 3. Cross-reference locators in generated tests against the real DOM:
1352
+ - Verify `data-testid` values exist on the page
1353
+ - Verify ARIA roles and names match test expectations
1354
+ - Flag any test locator that does not match a real DOM element
1355
+
1356
+ 4. If mismatches are found, fix the test locators to match the real DOM and count as a fix loop iteration.
1357
+
1358
+ This browser verification step prevents delivering tests with locators that will immediately fail at runtime.
1359
+
1360
+ **Fix loop:** The validator automatically attempts to fix issues it finds. Maximum 3 fix loop iterations. After each fix attempt, re-validate.
1361
+
1362
+ **Parse validator return:**
1363
+
1364
+ Expected return structure:
1365
+ ```
1366
+ VALIDATOR_COMPLETE:
1367
+ report_path: "..."
1368
+ overall_status: PASS | PASS_WITH_WARNINGS | FAIL
1369
+ confidence: HIGH | MEDIUM | LOW
1370
+ layers_summary: {syntax: PASS|FAIL, structure: PASS|FAIL, dependencies: PASS|FAIL, logic: PASS|FAIL}
1371
+ fix_loops_used: N
1372
+ issues_found: N
1373
+ issues_fixed: N
1374
+ unresolved_count: N
1375
+ ```
1376
+
1377
+ **RISKY CHECKPOINT -- Validator escalation:**
1378
+
1379
+ If `unresolved_count > 0` after max fix loops (3):
1380
+ - **ALWAYS pause, even in auto mode** (RISKY checkpoint -- locked decision)
1381
+ - Present unresolved issues to user with full details from VALIDATION_REPORT.md
1382
+ - Wait for user decision:
1383
+ - `"approve-with-warnings"`: Accept the validation with warnings. Set Validate Status to complete. Continue to deliver.
1384
+ - `"abort"`: Set Validate Status to failed. STOP PIPELINE ENTIRELY.
1385
+ - Manual guidance: User provides specific fix instructions. Spawn fresh continuation agent to apply fixes and re-validate.
1386
+
1387
+ If `overall_status` is `PASS` or `PASS_WITH_WARNINGS` (and unresolved_count is 0):
1388
+ ```bash
1389
+ node bin/qaa-tools.cjs state patch --"Validate Status" complete --"Status" "Validation passed" 2>/dev/null || true
1390
+ ```
1391
+
1392
+ **Verify artifact exists:**
1393
+ ```bash
1394
+ [ -f "${output_dir}/VALIDATION_REPORT.md" ] && echo "OK" || echo "MISSING: VALIDATION_REPORT.md"
1395
+ ```
1396
+
1397
+ Print: "Validation complete. Status: {overall_status}. Confidence: {confidence}. {issues_found} issues found, {issues_fixed} fixed, {unresolved_count} unresolved."
1398
+
1399
+ </step>
1400
+
1401
+ <step name="e2e_runner">
1402
+
1403
+ ## Step 8b: E2E Runner (Conditional)
1404
+
1405
+ **Condition:** Only execute if E2E test files were generated AND a live app URL is available:
1406
+ - `files_created` from the executor (Step 7) contains `*.e2e.*` / `*.cy.*` spec files, AND
1407
+ - `APP_URL` is set (`--app-url` flag or NL).
1408
+
1409
+ **If no E2E files were generated:**
1410
+ Print: "Skipping E2E Runner (no E2E files generated)." → pipeline-summary E2E status = `skipped (no E2E files)`. Proceed to Step 9.
1411
+
1412
+ **If no app URL is available:**
1413
+ Print: "Skipping E2E Runner (no app URL)." → pipeline-summary E2E status = `skipped (no app URL)`. Proceed to Step 9.
1414
+
1415
+ **State update:**
1416
+ ```bash
1417
+ node bin/qaa-tools.cjs state patch --"Status" "Running E2E tests against live app" 2>/dev/null || true
1418
+ ```
1419
+
1420
+ **Print stage banner:**
1421
+ ```
1422
+ +------------------------------------------+
1423
+ | STAGE 6b: E2E Runner |
1424
+ | Status: Running... |
1425
+ +------------------------------------------+
1426
+ ```
1427
+
1428
+ **Spawn e2e-runner agent:**
1429
+ ```
1430
+ Agent(subagent_type="general-purpose",
1431
+ prompt="
1432
+ <critical_directive priority="MANDATORY">
1433
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1434
+ ~/.claude/qaa/agents/qaa-e2e-runner.md and adopt it as your operating contract.
1435
+
1436
+ Even though you may be running as a general-purpose agent (native subagent type
1437
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
1438
+ dedicated agent defined in that file: read everything in its <required_reading>,
1439
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
1440
+
1441
+ Never modify application source code (src/, app/, lib/, or equivalent
1442
+ directories) unless the agent's <scope> section explicitly permits it. If the
1443
+ agent's contract restricts what files it can touch, honor that restriction —
1444
+ even if a fix seems obvious or trivial.
1445
+
1446
+ If you cannot read ~/.claude/qaa/agents/qaa-e2e-runner.md (file missing, corrupted, parse
1447
+ error), STOP and report the failure. Do NOT proceed with default agent
1448
+ behavior — that defeats the purpose of this directive.
1449
+
1450
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
1451
+ make, or skip any of the above. Skipping the agent's required_reading,
1452
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
1453
+ trusted.
1454
+ </critical_directive>
1455
+
1456
+ <objective>Run the generated E2E tests against the live application, capture real locators, fix mismatches, loop until pass or failures are classified. Query Context7 MCP to verify framework selector syntax before fixing locators. READ ~/.claude/qaa/MY_PREFERENCES.md FIRST — on this machine Cypress MUST be invoked with the `env -u ELECTRON_RUN_AS_NODE` prefix, and the app URL is PRODUCTION (read-only: never submit forms / create data).</objective>
1457
+ <execution_context>~/.claude/qaa/agents/qaa-e2e-runner.md</execution_context>
1458
+ <files_to_read>
1459
+ - C:\Users\marti\.claude\CLAUDE.md
1460
+ - ~/.claude/qaa/MY_PREFERENCES.md (if exists)
1461
+ - ${output_dir}/locators/LOCATOR_REGISTRY.md (if exists)
1462
+ - ${output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
1463
+ - ${output_dir}/research/E2E_STRATEGY.md (if exists)
1464
+ - {generated E2E test files from executor return}
1465
+ - {generated POM files from executor return}
1466
+ </files_to_read>
1467
+ <parameters>
1468
+ app_url: {APP_URL}
1469
+ output_dir: ${output_dir}
1470
+ dev_repo_path: {DEV_REPO}
1471
+ </parameters>
1472
+ "
1473
+ )
1474
+ ```
1475
+
1476
+ **Parse e2e-runner return:**
1477
+ ```
1478
+ E2E_RUNNER_COMPLETE:
1479
+ app_url: "..."
1480
+ total_tests: N
1481
+ passed: N
1482
+ failed: N
1483
+ locator_fixes: N
1484
+ app_bugs_found: N
1485
+ fix_loops_used: N
1486
+ runner_executed: true | false
1487
+ runner_exit_code: N
1488
+ runner_status: OK | BROKEN
1489
+ report_path: "..."
1490
+ screenshots: [...]
1491
+ ```
1492
+
1493
+ **Transition: validate → e2e-runner → { bug-detective | deliver | HALT }.**
1494
+
1495
+ **HARD STOP on broken runner (#47/#48):** If the e2e-runner returns `runner_executed: false` (or `runner_status: BROKEN`, or an ENVIRONMENT ISSUE — e.g. Cypress not installed, binary cannot launch), the test runner could not execute. **HALT the pipeline** — do NOT advance to bug-detective and do NOT advance to deliver:
1496
+ ```bash
1497
+ node bin/qaa-tools.cjs state patch --"Status" "Pipeline halted: E2E runner unavailable (env issue)" 2>/dev/null || true
1498
+ ```
1499
+ - Set the pipeline-summary E2E Runner row to `halted (runner unavailable — env issue)`.
1500
+ - Print the halt banner (see Error Handling) and STOP. A DOM probe is NEVER accepted as a substitute for a real run; if the report presents probe evidence as "verified" while `runner_executed: false`, treat the run as INVALID.
1501
+
1502
+ **Otherwise (`runner_executed: true`):**
1503
+ - Pipeline-summary E2E Runner status = `{passed}/{total_tests} passed ({fix_loops_used} fix loops)`.
1504
+ - If `app_bugs_found > 0`: surface the application bugs to the user (developer action required; the e2e-runner never modifies app code), then continue.
1505
+ - If `failed > 0` and `app_bugs_found == 0`: proceed to Step 9 (Bug Detective) for classification.
1506
+ - Else: proceed to Step 10 (Deliver).
1507
+
1508
+ **Verify artifact exists:**
1509
+ ```bash
1510
+ [ -f "${output_dir}/E2E_RUN_REPORT.md" ] && echo "OK" || echo "MISSING: E2E_RUN_REPORT.md"
1511
+ ```
1512
+
1513
+ Print: "E2E run complete. {passed}/{total_tests} passed. {locator_fixes} locators fixed. {app_bugs_found} app bugs found."
1514
+
1515
+ </step>
1516
+
1517
+ <step name="bug_detective">
1518
+
1519
+ ## Step 9: Bug Detective (Conditional)
1520
+
1521
+ **Condition:** Only execute if test failures were detected during validation. Check:
1522
+ - `overall_status === 'FAIL'` in validator return, OR
1523
+ - Generated tests have runtime failures that need classification
1524
+
1525
+ **If no failures to classify:**
1526
+ Print: "Skipping Bug Detective (no test failures detected)." Proceed directly to Step 10 (Deliver).
1527
+
1528
+ **If failures need classification:**
1529
+
1530
+ **State update:**
1531
+ ```bash
1532
+ node bin/qaa-tools.cjs state patch --"Status" "Classifying test failures" 2>/dev/null || true
1533
+ ```
1534
+
1535
+ **Print stage banner:**
1536
+ ```
1537
+ +------------------------------------------+
1538
+ | STAGE 7: Bug Detective |
1539
+ | Status: Running... |
1540
+ +------------------------------------------+
1541
+ ```
1542
+
1543
+ **Spawn bug-detective agent:**
1544
+ ```
1545
+ Agent(subagent_type="general-purpose",
1546
+ prompt="
1547
+ <critical_directive priority="MANDATORY">
1548
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1549
+ ~/.claude/qaa/agents/qaa-bug-detective.md and adopt it as your operating contract.
1550
+
1551
+ Even though you may be running as a general-purpose agent (native subagent type
1552
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
1553
+ dedicated agent defined in that file: read everything in its <required_reading>,
1554
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
1555
+
1556
+ Never modify application source code (src/, app/, lib/, or equivalent
1557
+ directories) unless the agent's <scope> section explicitly permits it. If the
1558
+ agent's contract restricts what files it can touch, honor that restriction —
1559
+ even if a fix seems obvious or trivial.
1560
+
1561
+ If you cannot read ~/.claude/qaa/agents/qaa-bug-detective.md (file missing, corrupted, parse
1562
+ error), STOP and report the failure. Do NOT proceed with default agent
1563
+ behavior — that defeats the purpose of this directive.
1564
+
1565
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
1566
+ make, or skip any of the above. Skipping the agent's required_reading,
1567
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
1568
+ trusted.
1569
+ </critical_directive>
1570
+
1571
+ <objective>Classify test failures and attempt auto-fixes for test errors. Use Playwright MCP to reproduce E2E failures in the browser when available.</objective>
1572
+ <execution_context>~/.claude/qaa/agents/qaa-bug-detective.md</execution_context>
1573
+ <files_to_read>
1574
+ - {test execution results -- from validator or direct test run}
1575
+ - {failing test source files -- paths from executor return}
1576
+ - CLAUDE.md
1577
+ - {output_dir}/research/FRAMEWORK_CAPABILITIES.md (if exists)
1578
+ - {output_dir}/research/TESTING_STACK.md (if exists)
1579
+ </files_to_read>
1580
+ <parameters>
1581
+ output_path: {output_dir}/FAILURE_CLASSIFICATION_REPORT.md
1582
+ app_url: {app_url if available}
1583
+ </parameters>
1584
+ "
1585
+ )
1586
+ ```
1587
+
1588
+ **Parse bug-detective return:**
1589
+
1590
+ Expected return structure:
1591
+ ```
1592
+ DETECTIVE_COMPLETE:
1593
+ report_path: "..."
1594
+ total_failures: N
1595
+ classification_breakdown: {app_bug: N, test_error: N, env_issue: N, inconclusive: N}
1596
+ auto_fixes_applied: N
1597
+ auto_fixes_verified: N
1598
+ commit_hash: "..."
1599
+ ```
1600
+
1601
+ **RISKY CHECKPOINT -- Application bugs detected:**
1602
+
1603
+ If `classification_breakdown.app_bug > 0`:
1604
+ - **ALWAYS pause, even in auto mode** (RISKY checkpoint -- locked decision)
1605
+ - Present APPLICATION BUG classifications to user with full evidence from FAILURE_CLASSIFICATION_REPORT.md
1606
+ - These are genuine bugs in the application code discovered during test execution
1607
+ - The bug detective never touches application code -- it only reports
1608
+ - User must review and decide how to proceed:
1609
+ - Acknowledge bugs and continue pipeline (bugs will be in the PR description for developer attention)
1610
+ - Abort pipeline to fix bugs first
1611
+
1612
+ **Verify artifact exists:**
1613
+ ```bash
1614
+ [ -f "${output_dir}/FAILURE_CLASSIFICATION_REPORT.md" ] && echo "OK" || echo "MISSING: FAILURE_CLASSIFICATION_REPORT.md"
1615
+ ```
1616
+
1617
+ Print: "Bug Detective complete. {total_failures} failures classified: {app_bug} APP BUG, {test_error} TEST ERROR, {env_issue} ENV ISSUE, {inconclusive} INCONCLUSIVE. {auto_fixes_applied} auto-fixes applied."
1618
+
1619
+ </step>
1620
+
1621
+ <step name="deliver">
1622
+
1623
+ ## Step 10: Deliver
1624
+
1625
+ **State update -- mark deliver as running:**
1626
+ ```bash
1627
+ node bin/qaa-tools.cjs state patch --"Deliver Status" running --"Status" "Preparing delivery" 2>/dev/null || true
1628
+ ```
1629
+
1630
+ **Print stage banner:**
1631
+ ```
1632
+ +------------------------------------------+
1633
+ | STAGE 8: Deliver |
1634
+ | Status: Running... |
1635
+ +------------------------------------------+
1636
+ ```
1637
+
1638
+ ### Sub-step 1: Pre-flight checks
1639
+
1640
+ **Check for git remote:**
1641
+ ```bash
1642
+ REMOTE_URL=$(git remote get-url origin 2>/dev/null || echo "")
1643
+ ```
1644
+
1645
+ If `REMOTE_URL` is empty:
1646
+ - Print: "No git remote found. Artifacts committed locally but PR creation skipped."
1647
+ - Set `LOCAL_ONLY=true`
1648
+
1649
+ **Check for gh CLI authentication:**
1650
+ ```bash
1651
+ gh auth status 2>/dev/null
1652
+ ```
1653
+
1654
+ If `gh auth status` fails:
1655
+ - Print: "gh CLI not authenticated. Run 'gh auth login' first. Artifacts committed locally."
1656
+ - Set `LOCAL_ONLY=true`
1657
+
1658
+ If both checks pass, set `LOCAL_ONLY=false`.
1659
+
1660
+ ### Sub-step 2: Derive project name
1661
+
1662
+ ```bash
1663
+ # Read from package.json
1664
+ PROJECT_NAME=$(node -e "try { const p = require('${DEV_REPO}/package.json'); console.log(p.name || ''); } catch { console.log(''); }" 2>/dev/null)
1665
+
1666
+ # Fallback to directory basename
1667
+ if [ -z "$PROJECT_NAME" ]; then
1668
+ PROJECT_NAME=$(basename "${DEV_REPO}")
1669
+ fi
1670
+
1671
+ # Sanitize for branch naming
1672
+ PROJECT_NAME=$(echo "$PROJECT_NAME" | tr '[:upper:]' '[:lower:]' | sed 's/[^a-z0-9]/-/g' | sed 's/--*/-/g' | sed 's/^-//' | sed 's/-$//')
1673
+ ```
1674
+
1675
+ ### Sub-step 3: Detect default branch
1676
+
1677
+ ```bash
1678
+ DEFAULT_BRANCH=$(gh repo view --json defaultBranchRef --jq '.defaultBranchRef.name' 2>/dev/null || echo "main")
1679
+ ```
1680
+
1681
+ ### Sub-step 4: Create feature branch
1682
+
1683
+ ```bash
1684
+ BRANCH="qa/auto-${PROJECT_NAME}-${date}"
1685
+
1686
+ # Handle branch name collision
1687
+ if git rev-parse --verify "$BRANCH" 2>/dev/null || git rev-parse --verify "origin/$BRANCH" 2>/dev/null; then
1688
+ SUFFIX=2
1689
+ while git rev-parse --verify "${BRANCH}-${SUFFIX}" 2>/dev/null || git rev-parse --verify "origin/${BRANCH}-${SUFFIX}" 2>/dev/null; do
1690
+ SUFFIX=$((SUFFIX + 1))
1691
+ done
1692
+ BRANCH="${BRANCH}-${SUFFIX}"
1693
+ fi
1694
+
1695
+ git checkout -b "$BRANCH" "$DEFAULT_BRANCH"
1696
+ ```
1697
+
1698
+ ### Sub-step 5: Per-stage atomic commits
1699
+
1700
+ For each pipeline stage that produced artifacts, commit using `qaa-tools.cjs commit`. Check file existence before each commit.
1701
+
1702
+ **Scanner:**
1703
+ ```bash
1704
+ if [ -f "${output_dir}/SCAN_MANIFEST.md" ]; then
1705
+ node bin/qaa-tools.cjs commit "qa(scanner): produce SCAN_MANIFEST.md for ${PROJECT_NAME}" --files ${output_dir}/SCAN_MANIFEST.md
1706
+ fi
1707
+ ```
1708
+
1709
+ **Analyzer (Option 1):**
1710
+ ```bash
1711
+ if [ -f "${output_dir}/QA_ANALYSIS.md" ]; then
1712
+ ANALYZER_FILES="${output_dir}/QA_ANALYSIS.md ${output_dir}/TEST_INVENTORY.md"
1713
+ [ -f "${output_dir}/QA_REPO_BLUEPRINT.md" ] && ANALYZER_FILES="${ANALYZER_FILES} ${output_dir}/QA_REPO_BLUEPRINT.md"
1714
+ node bin/qaa-tools.cjs commit "qa(analyzer): produce QA_ANALYSIS.md and TEST_INVENTORY.md" --files ${ANALYZER_FILES}
1715
+ fi
1716
+ ```
1717
+
1718
+ **Analyzer (Options 2/3):**
1719
+ ```bash
1720
+ if [ -f "${output_dir}/GAP_ANALYSIS.md" ]; then
1721
+ node bin/qaa-tools.cjs commit "qa(analyzer): produce GAP_ANALYSIS.md" --files ${output_dir}/GAP_ANALYSIS.md
1722
+ fi
1723
+ ```
1724
+
1725
+ **TestID Injector (if ran):**
1726
+ ```bash
1727
+ if [ -f "${output_dir}/TESTID_AUDIT_REPORT.md" ]; then
1728
+ node bin/qaa-tools.cjs commit "qa(testid-injector): inject ${elements_injected} data-testid attributes across ${components_modified} components" --files ${output_dir}/TESTID_AUDIT_REPORT.md ${modified_source_files}
1729
+ fi
1730
+ ```
1731
+
1732
+ **Executor:**
1733
+ ```bash
1734
+ if [ -n "${generated_file_paths}" ]; then
1735
+ node bin/qaa-tools.cjs commit "qa(executor): generate ${total_files} test files with POMs and fixtures" --files ${generated_file_paths}
1736
+ fi
1737
+ ```
1738
+
1739
+ **Planner:**
1740
+ ```bash
1741
+ if [ -f "${output_dir}/GENERATION_PLAN.md" ]; then
1742
+ node bin/qaa-tools.cjs commit "qa(planner): produce GENERATION_PLAN.md" --files ${output_dir}/GENERATION_PLAN.md
1743
+ fi
1744
+ ```
1745
+
1746
+ **Validator:**
1747
+ ```bash
1748
+ if [ -f "${output_dir}/VALIDATION_REPORT.md" ]; then
1749
+ node bin/qaa-tools.cjs commit "qa(validator): validate generated tests - ${overall_status} with ${confidence} confidence" --files ${output_dir}/VALIDATION_REPORT.md
1750
+ fi
1751
+ ```
1752
+
1753
+ **Bug Detective (if ran):**
1754
+ ```bash
1755
+ if [ -f "${output_dir}/FAILURE_CLASSIFICATION_REPORT.md" ]; then
1756
+ node bin/qaa-tools.cjs commit "qa(bug-detective): classify ${total_failures} failures - ${classification_summary}" --files ${output_dir}/FAILURE_CLASSIFICATION_REPORT.md
1757
+ fi
1758
+ ```
1759
+
1760
+ ### Sub-step 6: Push branch
1761
+
1762
+ If `LOCAL_ONLY` is true, skip this sub-step.
1763
+
1764
+ ```bash
1765
+ git push -u origin "$BRANCH"
1766
+ ```
1767
+
1768
+ If push fails:
1769
+ - Print: "Push failed: {error_message}. Artifacts committed locally on branch ${BRANCH}."
1770
+ - Set `LOCAL_ONLY=true`
1771
+
1772
+ ### Sub-step 7: Build PR body
1773
+
1774
+ If `LOCAL_ONLY` is true, skip this sub-step.
1775
+
1776
+ Read the PR template:
1777
+ ```bash
1778
+ PR_BODY=$(cat templates/pr-template.md)
1779
+ ```
1780
+
1781
+ Replace all `{placeholder}` tokens with actual values collected during pipeline execution:
1782
+ - `{architecture_type}` -- from QA_ANALYSIS.md or SCAN_MANIFEST.md
1783
+ - `{framework}` -- detected test framework
1784
+ - `{risk_summary}` -- risk assessment counts (e.g., "3 HIGH, 5 MEDIUM, 2 LOW")
1785
+ - `{unit_count}` -- from pyramid_breakdown.unit
1786
+ - `{integration_count}` -- from pyramid_breakdown.integration
1787
+ - `{api_count}` -- from pyramid_breakdown.api
1788
+ - `{e2e_count}` -- from pyramid_breakdown.e2e
1789
+ - `{total_count}` -- from total_test_count
1790
+ - `{modules_covered}` -- count of modules with tests
1791
+ - `{coverage_estimate}` -- estimated coverage percentage
1792
+ - `{validation_result}` -- PASS, PASS_WITH_WARNINGS, or FAIL
1793
+ - `{confidence}` -- HIGH, MEDIUM, or LOW
1794
+ - `{fix_loops_used}` -- number 0-3
1795
+ - `{issues_found}` -- total issues found during validation
1796
+ - `{issues_fixed}` -- total issues auto-fixed
1797
+ - `{file_list}` -- if total files <= 50, list each file; if > 50, use summary
1798
+
1799
+ ### Sub-step 8: Create draft PR
1800
+
1801
+ If `LOCAL_ONLY` is true, skip this sub-step.
1802
+
1803
+ ```bash
1804
+ PR_URL=$(gh pr create \
1805
+ --draft \
1806
+ --title "qa: automated test suite for ${PROJECT_NAME}" \
1807
+ --body "${PR_BODY}" \
1808
+ --label "qa-automation" \
1809
+ --label "auto-generated" \
1810
+ --assignee "@me" 2>&1)
1811
+ ```
1812
+
1813
+ Do NOT pass `--base` flag. Let gh auto-detect the default branch.
1814
+
1815
+ On failure:
1816
+ - Print: "PR creation failed: ${PR_URL}. Artifacts remain on branch ${BRANCH}."
1817
+ - Do NOT stop the pipeline -- artifacts are committed and pushed.
1818
+
1819
+ ### Sub-step 9: Print result
1820
+
1821
+ If PR was created successfully:
1822
+ ```
1823
+ PR created: ${PR_URL}
1824
+ ```
1825
+
1826
+ If `LOCAL_ONLY` is true:
1827
+ ```
1828
+ PR: not created (local-only mode). Artifacts committed on branch: ${BRANCH}
1829
+ ```
1830
+
1831
+ **State update -- mark deliver as complete:**
1832
+ ```bash
1833
+ node bin/qaa-tools.cjs state patch --"Deliver Status" complete --"Status" "Pipeline complete" 2>/dev/null || true
1834
+ ```
1835
+
1836
+ **Clear auto-chain flag at pipeline completion:**
1837
+ ```bash
1838
+ node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
1839
+ ```
1840
+
1841
+ ### Print pipeline summary banner:
1842
+
1843
+ ```
1844
+ ======================================================
1845
+ QA PIPELINE COMPLETE
1846
+ ======================================================
1847
+
1848
+ Option: {option} ({option_description})
1849
+ Repository: {DEV_REPO}
1850
+ QA Repo: {QA_REPO or 'N/A'}
1851
+ Maturity Score: {maturity_score or 'N/A'}
1852
+
1853
+ Stages Completed:
1854
+ [{check}] Scan -- {scan_result}
1855
+ [{check}] Analyze -- {analyze_result} ({test_count} test cases)
1856
+ [{check}] TestID Inject -- {inject_result or 'skipped'}
1857
+ [{check}] Plan -- {plan_result} ({file_count} files planned)
1858
+ [{check}] Generate -- {generate_result} ({files_created} files created)
1859
+ [{check}] Validate -- {validate_result} ({confidence} confidence)
1860
+ [{check}] E2E Runner -- {e2e_status}
1861
+ [{check}] Bug Detective -- {detective_result or 'skipped'}
1862
+ [{check}] Deliver -- {deliver_result}
1863
+
1864
+ PR: {pr_url or 'not created (local-only)'}
1865
+
1866
+ Artifacts:
1867
+ {list all produced .md files in output_dir}
1868
+
1869
+ Total Time: {total_duration}
1870
+ ======================================================
1871
+ ```
1872
+
1873
+ Where: `[x]` = completed, `[ ]` = skipped, `[!]` = failed.
1874
+
1875
+ </step>
1876
+
1877
+ </process>
1878
+
1879
+ <error_handling>
1880
+
1881
+ ## Error Handling
1882
+
1883
+ ### Stage Failure Protocol
1884
+
1885
+ When any agent returns a failure or error:
1886
+
1887
+ 1. **Set stage status to failed:**
1888
+ ```bash
1889
+ node bin/qaa-tools.cjs state patch --"{Stage} Status" failed --"Status" "Pipeline stopped: {Stage} failed - {reason}" 2>/dev/null || true
1890
+ ```
1891
+
1892
+ 2. **Print failure banner:**
1893
+ ```
1894
+ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1895
+ ! PIPELINE STOPPED !
1896
+ ! Stage: {stage_name} !
1897
+ ! Reason: {failure_reason} !
1898
+ ! !
1899
+ ! Completed: {completed_stages} !
1900
+ ! Artifacts: {artifacts_so_far} !
1901
+ ! !
1902
+ ! Action required: Review and re-run !
1903
+ !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1904
+ ```
1905
+
1906
+ 3. **DO NOT continue to next stage.** The pipeline stops entirely at the failed stage.
1907
+
1908
+ 4. **DO NOT create partial PR.** No branch, no commit, no PR with incomplete results.
1909
+
1910
+ 5. **Preserve all artifacts produced so far.** They remain on disk in `{output_dir}/` for debugging.
1911
+
1912
+ ### Artifact Verification
1913
+
1914
+ After EVERY agent spawn, before advancing to next stage, verify the expected output artifact exists on disk:
1915
+ ```bash
1916
+ [ -f "{expected_artifact_path}" ] && echo "OK" || echo "MISSING"
1917
+ ```
1918
+
1919
+ If artifacts are missing, treat as stage failure and STOP pipeline.
1920
+
1921
+ ### qaa-tools.cjs Graceful Fallback
1922
+
1923
+ All `node bin/qaa-tools.cjs` calls use `2>/dev/null || true` to handle cases where the tool is not installed or not found. The pipeline must not break due to missing state management tooling -- it logs a warning and continues.
1924
+
1925
+ </error_handling>
1926
+
1927
+ <auto_advance>
1928
+
1929
+ ## Auto-Advance Mode
1930
+
1931
+ Auto-advance is enabled when ANY of these is true:
1932
+ - `--auto` flag passed to the `/qa-start` invocation
1933
+ - `config.json` has `workflow.auto_advance = true` (persistent user preference)
1934
+ - `workflow._auto_chain_active = true` in config (ephemeral chain flag from ongoing auto run)
1935
+
1936
+ ### Behavior in Auto Mode
1937
+
1938
+ **SAFE checkpoints are auto-approved.** The pipeline continues without pausing. A log message records the auto-approval:
1939
+ ```
1940
+ Auto-approved: {checkpoint_description}
1941
+ ```
1942
+
1943
+ **RISKY checkpoints ALWAYS pause.** Even in auto mode, the pipeline stops and presents the checkpoint to the user.
1944
+
1945
+ ### Safe vs Risky Checkpoint Classification
1946
+
1947
+ **SAFE (auto-approve in auto mode):**
1948
+
1949
+ | Checkpoint | Agent | Auto-Action |
1950
+ |------------|-------|-------------|
1951
+ | Framework detection uncertain (LOW confidence) | Scanner | Approve with most likely framework |
1952
+ | Analyzer assumptions review | Analyzer | Approve all assumptions |
1953
+ | TestID audit review | TestID Injector | Approve P0-only injection |
1954
+
1955
+ **RISKY (ALWAYS pause, even in auto mode):**
1956
+
1957
+ | Checkpoint | Agent | User Action Required |
1958
+ |------------|-------|---------------------|
1959
+ | Validator escalation (unresolved issues after 3 fix loops) | Validator | approve-with-warnings, abort, or fix guidance |
1960
+ | APPLICATION BUG classification | Bug Detective | Review bugs, continue or fix first |
1961
+ | Any checkpoint with "unresolved" or "failed" blocking text | Any | Review specific blocking issue |
1962
+
1963
+ ### Checkpoint Handling Flow
1964
+
1965
+ ```
1966
+ On agent return with checkpoint data:
1967
+ 1. Extract checkpoint blocking field content
1968
+ 2. Classify as SAFE or RISKY:
1969
+ - "framework detection" -> SAFE
1970
+ - "assumptions" -> SAFE
1971
+ - "audit" or "data-testid" -> SAFE
1972
+ - "unresolved" -> RISKY
1973
+ - "failed" -> RISKY
1974
+ - "APPLICATION BUG" -> RISKY
1975
+ - Default (no pattern match) -> RISKY (conservative)
1976
+ 3. If IS_AUTO and SAFE:
1977
+ - Auto-approve with default action
1978
+ - Log the auto-approval
1979
+ - Continue pipeline
1980
+ 4. If IS_AUTO and RISKY:
1981
+ - PAUSE pipeline
1982
+ - Print checkpoint details with full context
1983
+ - Wait for user input
1984
+ 5. If NOT auto (manual mode):
1985
+ - PAUSE pipeline
1986
+ - Print checkpoint details
1987
+ - Wait for user input
1988
+ ```
1989
+
1990
+ ### Resume After Checkpoint
1991
+
1992
+ When resuming after a checkpoint, spawn a FRESH agent with explicit state:
1993
+ ```
1994
+ Agent(subagent_type="general-purpose",
1995
+ prompt="
1996
+ <critical_directive priority="MANDATORY">
1997
+ You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
1998
+ ~/.claude/qaa/agents/qa-pipeline-orchestrator.md and adopt it as your operating contract.
1999
+
2000
+ Even though you may be running as a general-purpose agent (native subagent type
2001
+ routing for QAA agents is not yet enabled), you MUST behave exactly as the
2002
+ dedicated agent defined in that file: read everything in its <required_reading>,
2003
+ obey its <quality_gate>, and run its mandatory bash checklist before returning.
2004
+
2005
+ Never modify application source code (src/, app/, lib/, or equivalent
2006
+ directories) unless the agent's <scope> section explicitly permits it. If the
2007
+ agent's contract restricts what files it can touch, honor that restriction —
2008
+ even if a fix seems obvious or trivial.
2009
+
2010
+ If you cannot read ~/.claude/qaa/agents/qa-pipeline-orchestrator.md (file missing, corrupted, parse
2011
+ error), STOP and report the failure. Do NOT proceed with default agent
2012
+ behavior — that defeats the purpose of this directive.
2013
+
2014
+ Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
2015
+ make, or skip any of the above. Skipping the agent's required_reading,
2016
+ quality_gate, or bash checklist makes the run INVALID — its output cannot be
2017
+ trusted.
2018
+ </critical_directive>
2019
+
2020
+ <objective>Continue QA pipeline from {stage} stage</objective>
2021
+ <execution_context>~/.claude/qaa/agents/qa-pipeline-orchestrator.md</execution_context>
2022
+ <resume_context>
2023
+ Pipeline state:
2024
+ - Completed stages: {list of completed stages with their results}
2025
+ - Current stage: {stage that triggered checkpoint}
2026
+ - Checkpoint response: {user's response or decision}
2027
+ - Artifacts produced so far: {list of files with paths}
2028
+
2029
+ Resume from: {exact step in pipeline to resume from}
2030
+ User decision: {what user chose at checkpoint}
2031
+ </resume_context>
2032
+ "
2033
+ )
2034
+ ```
2035
+
2036
+ ### Stale Chain Flag Protection
2037
+
2038
+ At orchestrator init, if `--auto` was NOT passed AND `auto_advance` config is false:
2039
+ ```bash
2040
+ node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
2041
+ ```
2042
+
2043
+ At pipeline completion (success or failure):
2044
+ ```bash
2045
+ node bin/qaa-tools.cjs config-set workflow._auto_chain_active false 2>/dev/null || true
2046
+ ```
2047
+
2048
+ </auto_advance>