qaa-agent 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/qa-analyze.md +4 -43
- package/.claude/commands/qa-from-ticket.md +4 -69
- package/.claude/commands/qa-gap.md +4 -38
- package/.claude/commands/qa-start.md +4 -15
- package/.claude/commands/qa-testid.md +7 -42
- package/.claude/commands/qa-validate.md +7 -41
- package/bin/install.cjs +6 -0
- package/package.json +2 -1
- package/workflows/qa-analyze.md +296 -0
- package/workflows/qa-from-ticket.md +474 -0
- package/workflows/qa-gap.md +303 -0
- package/workflows/qa-start.md +1168 -0
- package/workflows/qa-testid.md +356 -0
- package/workflows/qa-validate.md +295 -0
|
@@ -10,51 +10,12 @@ Analysis-only mode. Scans a repository, detects framework and stack, and produce
|
|
|
10
10
|
- --dev-repo: explicit path to developer repository
|
|
11
11
|
- --qa-repo: path to existing QA repository (produces gap analysis instead of blueprint)
|
|
12
12
|
|
|
13
|
-
## What It Produces
|
|
14
|
-
|
|
15
|
-
- SCAN_MANIFEST.md -- file tree, framework detection, testable surfaces
|
|
16
|
-
- QA_ANALYSIS.md -- architecture overview, risk assessment, testing pyramid
|
|
17
|
-
- TEST_INVENTORY.md -- prioritized test cases with IDs and explicit outcomes
|
|
18
|
-
- QA_REPO_BLUEPRINT.md (if no QA repo) or GAP_ANALYSIS.md (if QA repo provided)
|
|
19
|
-
|
|
20
13
|
## Instructions
|
|
21
14
|
|
|
22
15
|
1. Read `CLAUDE.md` -- all QA standards.
|
|
23
|
-
2.
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
3. Invoke scanner agent:
|
|
28
|
-
|
|
29
|
-
Task(
|
|
30
|
-
prompt="
|
|
31
|
-
<objective>Scan repository and produce SCAN_MANIFEST.md</objective>
|
|
32
|
-
<execution_context>@agents/qaa-scanner.md</execution_context>
|
|
33
|
-
<files_to_read>
|
|
34
|
-
- CLAUDE.md
|
|
35
|
-
</files_to_read>
|
|
36
|
-
<parameters>
|
|
37
|
-
user_input: $ARGUMENTS
|
|
38
|
-
</parameters>
|
|
39
|
-
"
|
|
40
|
-
)
|
|
41
|
-
|
|
42
|
-
4. Invoke analyzer agent:
|
|
43
|
-
|
|
44
|
-
Task(
|
|
45
|
-
prompt="
|
|
46
|
-
<objective>Analyze repository and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and blueprint or gap analysis</objective>
|
|
47
|
-
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
48
|
-
<files_to_read>
|
|
49
|
-
- CLAUDE.md
|
|
50
|
-
- .qa-output/SCAN_MANIFEST.md
|
|
51
|
-
</files_to_read>
|
|
52
|
-
<parameters>
|
|
53
|
-
user_input: $ARGUMENTS
|
|
54
|
-
</parameters>
|
|
55
|
-
"
|
|
56
|
-
)
|
|
57
|
-
|
|
58
|
-
5. Present results to user. No git operations. No test generation.
|
|
16
|
+
2. Execute the workflow:
|
|
17
|
+
|
|
18
|
+
Follow the workflow defined in `@workflows/qa-analyze.md` end-to-end.
|
|
19
|
+
Preserve all workflow gates (scan verification, artifact checks).
|
|
59
20
|
|
|
60
21
|
$ARGUMENTS
|
|
@@ -7,82 +7,17 @@ Generate test cases and executable test files from a ticket (Jira, Linear, GitHu
|
|
|
7
7
|
/qa-from-ticket <ticket-source> [--dev-repo <path>]
|
|
8
8
|
|
|
9
9
|
- ticket-source: one of:
|
|
10
|
-
- URL to Jira/Linear
|
|
10
|
+
- URL to GitHub/Jira/Linear issue
|
|
11
11
|
- Plain text user story or acceptance criteria
|
|
12
12
|
- File path to a .md or .txt with ticket details
|
|
13
13
|
- --dev-repo: path to developer repository (default: current directory)
|
|
14
14
|
|
|
15
|
-
## What It Produces
|
|
16
|
-
|
|
17
|
-
- TEST_CASES_FROM_TICKET.md -- test cases derived from acceptance criteria
|
|
18
|
-
- Test spec files (unit, API, E2E as appropriate)
|
|
19
|
-
- Page Object Model files (for E2E tests if needed)
|
|
20
|
-
- Fixture files (test data)
|
|
21
|
-
- Traceability matrix: which test covers which acceptance criterion
|
|
22
|
-
|
|
23
15
|
## Instructions
|
|
24
16
|
|
|
25
17
|
1. Read `CLAUDE.md` -- all QA standards.
|
|
18
|
+
2. Execute the workflow:
|
|
26
19
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
**If URL:** Fetch the ticket content using WebFetch or gh CLI (for GitHub issues).
|
|
30
|
-
```bash
|
|
31
|
-
# GitHub Issues
|
|
32
|
-
gh issue view <number> --json title,body,labels,assignees
|
|
33
|
-
|
|
34
|
-
# For other URLs, use WebFetch to read the page
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
**If plain text:** Use the text directly as the ticket content.
|
|
38
|
-
|
|
39
|
-
**If file path:** Read the file content.
|
|
40
|
-
|
|
41
|
-
3. Extract from the ticket:
|
|
42
|
-
- **Title/Summary**: what the feature or bug is about
|
|
43
|
-
- **Acceptance Criteria**: the specific conditions that must be met (look for "AC:", "Given/When/Then", checkboxes, numbered criteria)
|
|
44
|
-
- **User Story**: "As a [role] I want [action] so that [benefit]"
|
|
45
|
-
- **Edge Cases**: any mentioned edge cases, error states, or constraints
|
|
46
|
-
- **Priority**: ticket priority if available (maps to test priority P0/P1/P2)
|
|
47
|
-
|
|
48
|
-
4. Read the dev repo source code related to the ticket:
|
|
49
|
-
- Search for files matching keywords from the ticket title
|
|
50
|
-
- Read routes, controllers, services, models related to the feature
|
|
51
|
-
- Identify the actual implementation (or lack of it if the feature is not built yet)
|
|
52
|
-
|
|
53
|
-
5. Generate test cases that map 1:1 to acceptance criteria:
|
|
54
|
-
- Each acceptance criterion becomes at least one test case
|
|
55
|
-
- Add negative tests for each criterion (what should NOT happen)
|
|
56
|
-
- Add edge case tests if mentioned in the ticket
|
|
57
|
-
- Every test case follows CLAUDE.md rules: unique ID, concrete inputs, explicit expected outcome
|
|
58
|
-
|
|
59
|
-
6. Write TEST_CASES_FROM_TICKET.md with:
|
|
60
|
-
```markdown
|
|
61
|
-
# Test Cases from Ticket: [TICKET-ID] [Title]
|
|
62
|
-
|
|
63
|
-
## Source
|
|
64
|
-
- Ticket: [URL or "manual input"]
|
|
65
|
-
- Date: [YYYY-MM-DD]
|
|
66
|
-
|
|
67
|
-
## Acceptance Criteria Mapping
|
|
68
|
-
|
|
69
|
-
| AC # | Criterion | Test ID(s) | Status |
|
|
70
|
-
|------|-----------|------------|--------|
|
|
71
|
-
| AC-1 | [criterion text] | UT-XXX-001, E2E-XXX-001 | Covered |
|
|
72
|
-
| AC-2 | [criterion text] | API-XXX-001 | Covered |
|
|
73
|
-
|
|
74
|
-
## Test Cases
|
|
75
|
-
[test cases following CLAUDE.md format]
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
7. Generate executable test files using the qa-template-engine skill.
|
|
79
|
-
|
|
80
|
-
8. Validate generated tests using the qa-self-validator skill.
|
|
81
|
-
|
|
82
|
-
9. Present summary:
|
|
83
|
-
- How many acceptance criteria found
|
|
84
|
-
- How many test cases generated (by pyramid level)
|
|
85
|
-
- Traceability: every AC is covered
|
|
86
|
-
- Any ACs that could not be tested (and why)
|
|
20
|
+
Follow the workflow defined in `@workflows/qa-from-ticket.md` end-to-end.
|
|
21
|
+
Preserve all workflow gates (ticket parsing, traceability, validation).
|
|
87
22
|
|
|
88
23
|
$ARGUMENTS
|
|
@@ -9,46 +9,12 @@ Compare a developer repository against its QA repository to identify coverage ga
|
|
|
9
9
|
- --dev-repo: path to the developer repository (required)
|
|
10
10
|
- --qa-repo: path to the existing QA repository (required)
|
|
11
11
|
|
|
12
|
-
## What It Produces
|
|
13
|
-
|
|
14
|
-
- SCAN_MANIFEST.md -- scan of both repositories
|
|
15
|
-
- GAP_ANALYSIS.md -- coverage map, missing tests with IDs, broken tests, quality assessment
|
|
16
|
-
|
|
17
12
|
## Instructions
|
|
18
13
|
|
|
19
14
|
1. Read `CLAUDE.md` -- testing pyramid, test spec rules, quality gates.
|
|
20
|
-
2.
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
<objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
|
|
25
|
-
<execution_context>@agents/qaa-scanner.md</execution_context>
|
|
26
|
-
<files_to_read>
|
|
27
|
-
- CLAUDE.md
|
|
28
|
-
</files_to_read>
|
|
29
|
-
<parameters>
|
|
30
|
-
user_input: $ARGUMENTS
|
|
31
|
-
</parameters>
|
|
32
|
-
"
|
|
33
|
-
)
|
|
34
|
-
|
|
35
|
-
3. Invoke analyzer agent in gap mode:
|
|
36
|
-
|
|
37
|
-
Task(
|
|
38
|
-
prompt="
|
|
39
|
-
<objective>Produce GAP_ANALYSIS.md comparing dev repo against QA repo</objective>
|
|
40
|
-
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
41
|
-
<files_to_read>
|
|
42
|
-
- CLAUDE.md
|
|
43
|
-
- .qa-output/SCAN_MANIFEST.md
|
|
44
|
-
</files_to_read>
|
|
45
|
-
<parameters>
|
|
46
|
-
user_input: $ARGUMENTS
|
|
47
|
-
mode: gap
|
|
48
|
-
</parameters>
|
|
49
|
-
"
|
|
50
|
-
)
|
|
51
|
-
|
|
52
|
-
4. Present results. No test generation. No git operations.
|
|
15
|
+
2. Execute the workflow:
|
|
16
|
+
|
|
17
|
+
Follow the workflow defined in `@workflows/qa-gap.md` end-to-end.
|
|
18
|
+
Preserve all workflow gates (both-repo scanning, gap metrics).
|
|
53
19
|
|
|
54
20
|
$ARGUMENTS
|
|
@@ -14,20 +14,9 @@ Run the complete QA automation pipeline. Analyzes a repository, generates a stan
|
|
|
14
14
|
## Instructions
|
|
15
15
|
|
|
16
16
|
1. Read `CLAUDE.md` -- all QA standards that govern the pipeline.
|
|
17
|
-
2.
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
prompt="
|
|
22
|
-
<objective>Run complete QA automation pipeline</objective>
|
|
23
|
-
<execution_context>@agents/qa-pipeline-orchestrator.md</execution_context>
|
|
24
|
-
<files_to_read>
|
|
25
|
-
- CLAUDE.md
|
|
26
|
-
</files_to_read>
|
|
27
|
-
<parameters>
|
|
28
|
-
user_input: $ARGUMENTS
|
|
29
|
-
</parameters>
|
|
30
|
-
"
|
|
31
|
-
)
|
|
17
|
+
2. Execute the workflow:
|
|
18
|
+
|
|
19
|
+
Follow the workflow defined in `@workflows/qa-start.md` end-to-end.
|
|
20
|
+
Preserve all workflow gates (stage transitions, checkpoints, error handling, state updates, commits).
|
|
32
21
|
|
|
33
22
|
$ARGUMENTS
|
|
@@ -4,51 +4,16 @@ Scan frontend source code, audit missing data-testid attributes, and inject them
|
|
|
4
4
|
|
|
5
5
|
## Usage
|
|
6
6
|
|
|
7
|
-
/qa-testid <
|
|
7
|
+
/qa-testid [<source-directory>]
|
|
8
8
|
|
|
9
|
-
- path
|
|
10
|
-
|
|
11
|
-
## What It Produces
|
|
12
|
-
|
|
13
|
-
- TESTID_AUDIT_REPORT.md -- coverage score, missing elements, proposed values by priority
|
|
14
|
-
- Modified source files with data-testid attributes injected
|
|
9
|
+
- source-directory: path to frontend source (auto-detects if omitted)
|
|
15
10
|
|
|
16
11
|
## Instructions
|
|
17
12
|
|
|
18
|
-
1. Read `CLAUDE.md` -- data-testid Convention section
|
|
19
|
-
2.
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
3. Invoke scanner to identify component files:
|
|
24
|
-
|
|
25
|
-
Task(
|
|
26
|
-
prompt="
|
|
27
|
-
<objective>Scan repository to identify frontend component files</objective>
|
|
28
|
-
<execution_context>@agents/qaa-scanner.md</execution_context>
|
|
29
|
-
<files_to_read>
|
|
30
|
-
- CLAUDE.md
|
|
31
|
-
</files_to_read>
|
|
32
|
-
<parameters>
|
|
33
|
-
user_input: $ARGUMENTS
|
|
34
|
-
</parameters>
|
|
35
|
-
"
|
|
36
|
-
)
|
|
37
|
-
|
|
38
|
-
4. Invoke testid-injector agent:
|
|
39
|
-
|
|
40
|
-
Task(
|
|
41
|
-
prompt="
|
|
42
|
-
<objective>Audit missing data-testid attributes and inject following naming convention</objective>
|
|
43
|
-
<execution_context>@agents/qaa-testid-injector.md</execution_context>
|
|
44
|
-
<files_to_read>
|
|
45
|
-
- CLAUDE.md
|
|
46
|
-
- .qa-output/SCAN_MANIFEST.md
|
|
47
|
-
</files_to_read>
|
|
48
|
-
<parameters>
|
|
49
|
-
user_input: $ARGUMENTS
|
|
50
|
-
</parameters>
|
|
51
|
-
"
|
|
52
|
-
)
|
|
13
|
+
1. Read `CLAUDE.md` -- data-testid Convention section.
|
|
14
|
+
2. Execute the workflow:
|
|
15
|
+
|
|
16
|
+
Follow the workflow defined in `@workflows/qa-testid.md` end-to-end.
|
|
17
|
+
Preserve all workflow gates (framework detection, user approval, branch creation).
|
|
53
18
|
|
|
54
19
|
$ARGUMENTS
|
|
@@ -4,51 +4,17 @@ Validate existing test files against CLAUDE.md standards. Runs 4-layer validatio
|
|
|
4
4
|
|
|
5
5
|
## Usage
|
|
6
6
|
|
|
7
|
-
/qa-validate <
|
|
7
|
+
/qa-validate [<test-directory>] [--classify]
|
|
8
8
|
|
|
9
|
-
-
|
|
10
|
-
- --
|
|
11
|
-
|
|
12
|
-
## What It Produces
|
|
13
|
-
|
|
14
|
-
- VALIDATION_REPORT.md -- pass/fail per file per validation layer, confidence level
|
|
15
|
-
- FAILURE_CLASSIFICATION_REPORT.md -- if failures found, classifies as APP BUG / TEST ERROR / ENV ISSUE / INCONCLUSIVE
|
|
9
|
+
- test-directory: path to test files (auto-detects if omitted)
|
|
10
|
+
- --classify: also run bug-detective to classify failures
|
|
16
11
|
|
|
17
12
|
## Instructions
|
|
18
13
|
|
|
19
14
|
1. Read `CLAUDE.md` -- quality gates, locator tiers, assertion rules.
|
|
20
|
-
2.
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
<objective>Validate test files with 4-layer validation and produce VALIDATION_REPORT.md</objective>
|
|
25
|
-
<execution_context>@agents/qaa-validator.md</execution_context>
|
|
26
|
-
<files_to_read>
|
|
27
|
-
- CLAUDE.md
|
|
28
|
-
</files_to_read>
|
|
29
|
-
<parameters>
|
|
30
|
-
user_input: $ARGUMENTS
|
|
31
|
-
mode: validation
|
|
32
|
-
</parameters>
|
|
33
|
-
"
|
|
34
|
-
)
|
|
35
|
-
|
|
36
|
-
3. If failures detected, invoke bug-detective agent:
|
|
37
|
-
|
|
38
|
-
Task(
|
|
39
|
-
prompt="
|
|
40
|
-
<objective>Classify test failures and auto-fix TEST CODE ERRORS</objective>
|
|
41
|
-
<execution_context>@agents/qaa-bug-detective.md</execution_context>
|
|
42
|
-
<files_to_read>
|
|
43
|
-
- CLAUDE.md
|
|
44
|
-
- .qa-output/VALIDATION_REPORT.md
|
|
45
|
-
</files_to_read>
|
|
46
|
-
<parameters>
|
|
47
|
-
user_input: $ARGUMENTS
|
|
48
|
-
</parameters>
|
|
49
|
-
"
|
|
50
|
-
)
|
|
51
|
-
|
|
52
|
-
4. Present results. No git operations.
|
|
15
|
+
2. Execute the workflow:
|
|
16
|
+
|
|
17
|
+
Follow the workflow defined in `@workflows/qa-validate.md` end-to-end.
|
|
18
|
+
Preserve all workflow gates (fix loops, classification).
|
|
53
19
|
|
|
54
20
|
$ARGUMENTS
|
package/bin/install.cjs
CHANGED
|
@@ -106,6 +106,12 @@ async function main() {
|
|
|
106
106
|
const skillCount = copyDir(skillsSrc, skillsDest);
|
|
107
107
|
ok(`Installed ${skillCount} skill files (6 skills)`);
|
|
108
108
|
|
|
109
|
+
// Install workflows
|
|
110
|
+
const workflowsSrc = path.join(ROOT, 'workflows');
|
|
111
|
+
const workflowsDest = path.join(qaaDir, 'workflows');
|
|
112
|
+
const wfCount = copyDir(workflowsSrc, workflowsDest);
|
|
113
|
+
ok(`Installed ${wfCount} workflows`);
|
|
114
|
+
|
|
109
115
|
// Install agents
|
|
110
116
|
const agentsSrc = path.join(ROOT, 'agents');
|
|
111
117
|
const agentsDest = path.join(qaaDir, 'agents');
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "qaa-agent",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "QA Automation Agent for Claude Code — multi-agent pipeline that analyzes repos, generates tests, validates, and creates PRs",
|
|
5
5
|
"bin": {
|
|
6
6
|
"qaa-agent": "./bin/install.cjs"
|
|
@@ -24,6 +24,7 @@
|
|
|
24
24
|
"files": [
|
|
25
25
|
"bin/",
|
|
26
26
|
"agents/",
|
|
27
|
+
"workflows/",
|
|
27
28
|
"templates/",
|
|
28
29
|
".claude/commands/",
|
|
29
30
|
".claude/skills/",
|
|
@@ -0,0 +1,296 @@
|
|
|
1
|
+
<purpose>
|
|
2
|
+
Analysis-only workflow. Scans a developer repository (and optionally a QA repository) to produce comprehensive analysis artifacts -- SCAN_MANIFEST.md, QA_ANALYSIS.md, TEST_INVENTORY.md, and optionally QA_REPO_BLUEPRINT.md or GAP_ANALYSIS.md. No test generation, no git operations, no PR. Use this workflow to understand a codebase's testability before committing to full test generation.
|
|
3
|
+
</purpose>
|
|
4
|
+
|
|
5
|
+
<required_reading>
|
|
6
|
+
- `CLAUDE.md` -- QA automation standards, pipeline stages, module boundaries, verification commands
|
|
7
|
+
- `agents/qaa-scanner.md` -- Scanner agent definition (produces SCAN_MANIFEST.md)
|
|
8
|
+
- `agents/qaa-analyzer.md` -- Analyzer agent definition (produces QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md or GAP_ANALYSIS.md)
|
|
9
|
+
- `templates/scan-manifest.md` -- SCAN_MANIFEST.md format contract
|
|
10
|
+
- `templates/qa-analysis.md` -- QA_ANALYSIS.md format contract
|
|
11
|
+
- `templates/test-inventory.md` -- TEST_INVENTORY.md format contract
|
|
12
|
+
- `templates/qa-repo-blueprint.md` -- QA_REPO_BLUEPRINT.md format contract (Option 1 only)
|
|
13
|
+
- `templates/gap-analysis.md` -- GAP_ANALYSIS.md format contract (Option 2/3 only)
|
|
14
|
+
</required_reading>
|
|
15
|
+
|
|
16
|
+
<process>
|
|
17
|
+
|
|
18
|
+
<step name="parse_arguments">
|
|
19
|
+
## Step 1: Parse Arguments
|
|
20
|
+
|
|
21
|
+
Parse `$ARGUMENTS` for repository paths.
|
|
22
|
+
|
|
23
|
+
**Supported arguments:**
|
|
24
|
+
- `--dev-repo <path>` -- Path to the developer repository to analyze
|
|
25
|
+
- `--qa-repo <path>` -- Path to an existing QA repository (optional)
|
|
26
|
+
- No arguments -- defaults to the current working directory as the dev repo
|
|
27
|
+
|
|
28
|
+
**Parsing logic:**
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
DEV_REPO=""
|
|
32
|
+
QA_REPO=""
|
|
33
|
+
|
|
34
|
+
# Parse --dev-repo and --qa-repo from $ARGUMENTS
|
|
35
|
+
# If --dev-repo is provided, use it
|
|
36
|
+
# If --qa-repo is provided, use it
|
|
37
|
+
# If neither is provided, DEV_REPO defaults to current working directory
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
**Validation:**
|
|
41
|
+
- If `--dev-repo` is provided, verify the path exists and is a directory. If not, print error: `"Error: Dev repo path does not exist: {path}"` and STOP.
|
|
42
|
+
- If `--qa-repo` is provided, verify the path exists and is a directory. If not, print error: `"Error: QA repo path does not exist: {path}"` and STOP.
|
|
43
|
+
- If no arguments provided, set `DEV_REPO` to the current working directory.
|
|
44
|
+
</step>
|
|
45
|
+
|
|
46
|
+
<step name="determine_mode">
|
|
47
|
+
## Step 2: Determine Analysis Mode
|
|
48
|
+
|
|
49
|
+
Set the workflow option based on whether a QA repo was provided.
|
|
50
|
+
|
|
51
|
+
**Mode selection:**
|
|
52
|
+
|
|
53
|
+
| QA Repo Provided | Mode | Description |
|
|
54
|
+
|-------------------|------|-------------|
|
|
55
|
+
| No | `full` (Option 1) | Dev-only analysis. Produces QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md |
|
|
56
|
+
| Yes | `gap` (Option 2/3) | Gap analysis. Produces QA_ANALYSIS.md + TEST_INVENTORY.md + GAP_ANALYSIS.md |
|
|
57
|
+
|
|
58
|
+
```
|
|
59
|
+
MODE="full"
|
|
60
|
+
if QA_REPO is not empty:
|
|
61
|
+
MODE="gap"
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
**Set output directory:**
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
OUTPUT_DIR=".qa-output"
|
|
68
|
+
mkdir -p "${OUTPUT_DIR}"
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
**Print analysis banner:**
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
=== QA Analysis Workflow ===
|
|
75
|
+
Mode: {MODE} ({description})
|
|
76
|
+
Dev Repo: {DEV_REPO}
|
|
77
|
+
QA Repo: {QA_REPO or 'N/A'}
|
|
78
|
+
Output: {OUTPUT_DIR}
|
|
79
|
+
============================
|
|
80
|
+
```
|
|
81
|
+
</step>
|
|
82
|
+
|
|
83
|
+
<step name="run_scanner">
|
|
84
|
+
## Step 3: Spawn Scanner Agent
|
|
85
|
+
|
|
86
|
+
Spawn the scanner agent to produce SCAN_MANIFEST.md.
|
|
87
|
+
|
|
88
|
+
**For full mode (Option 1):**
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
Task(
|
|
92
|
+
prompt="
|
|
93
|
+
<objective>Scan repository and produce SCAN_MANIFEST.md</objective>
|
|
94
|
+
<execution_context>@agents/qaa-scanner.md</execution_context>
|
|
95
|
+
<files_to_read>
|
|
96
|
+
- CLAUDE.md
|
|
97
|
+
</files_to_read>
|
|
98
|
+
<parameters>
|
|
99
|
+
dev_repo_path: {DEV_REPO}
|
|
100
|
+
qa_repo_path: null
|
|
101
|
+
output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
|
|
102
|
+
</parameters>
|
|
103
|
+
"
|
|
104
|
+
)
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**For gap mode (Option 2/3):**
|
|
108
|
+
|
|
109
|
+
```
|
|
110
|
+
Task(
|
|
111
|
+
prompt="
|
|
112
|
+
<objective>Scan both developer and QA repositories and produce SCAN_MANIFEST.md</objective>
|
|
113
|
+
<execution_context>@agents/qaa-scanner.md</execution_context>
|
|
114
|
+
<files_to_read>
|
|
115
|
+
- CLAUDE.md
|
|
116
|
+
</files_to_read>
|
|
117
|
+
<parameters>
|
|
118
|
+
dev_repo_path: {DEV_REPO}
|
|
119
|
+
qa_repo_path: {QA_REPO}
|
|
120
|
+
output_path: {OUTPUT_DIR}/SCAN_MANIFEST.md
|
|
121
|
+
</parameters>
|
|
122
|
+
"
|
|
123
|
+
)
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
**Handle scanner return:**
|
|
127
|
+
|
|
128
|
+
- If scanner returns `decision: STOP` -- print the stop reason and STOP the workflow. The repository has no testable surfaces or an uncertain framework.
|
|
129
|
+
- If scanner returns `decision: PROCEED` -- continue to next step.
|
|
130
|
+
- Extract `has_frontend` and `detection_confidence` from scanner return for use in the summary.
|
|
131
|
+
|
|
132
|
+
**Verify SCAN_MANIFEST.md exists:**
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
[ -f "${OUTPUT_DIR}/SCAN_MANIFEST.md" ] && echo "FOUND" || echo "MISSING"
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
If missing, print error: `"Error: Scanner did not produce SCAN_MANIFEST.md. Check scanner output for details."` and STOP.
|
|
139
|
+
</step>
|
|
140
|
+
|
|
141
|
+
<step name="run_analyzer">
|
|
142
|
+
## Step 4: Spawn Analyzer Agent
|
|
143
|
+
|
|
144
|
+
Spawn the analyzer agent to produce analysis artifacts.
|
|
145
|
+
|
|
146
|
+
**For full mode (Option 1):**
|
|
147
|
+
|
|
148
|
+
```
|
|
149
|
+
Task(
|
|
150
|
+
prompt="
|
|
151
|
+
<objective>Analyze scanned repository and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and QA_REPO_BLUEPRINT.md</objective>
|
|
152
|
+
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
153
|
+
<files_to_read>
|
|
154
|
+
- {OUTPUT_DIR}/SCAN_MANIFEST.md
|
|
155
|
+
- CLAUDE.md
|
|
156
|
+
</files_to_read>
|
|
157
|
+
<parameters>
|
|
158
|
+
workflow_option: 1
|
|
159
|
+
qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
|
|
160
|
+
test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
|
|
161
|
+
blueprint_path: {OUTPUT_DIR}/QA_REPO_BLUEPRINT.md
|
|
162
|
+
</parameters>
|
|
163
|
+
"
|
|
164
|
+
)
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
**For gap mode (Option 2/3):**
|
|
168
|
+
|
|
169
|
+
```
|
|
170
|
+
Task(
|
|
171
|
+
prompt="
|
|
172
|
+
<objective>Analyze scanned repositories and produce QA_ANALYSIS.md, TEST_INVENTORY.md, and GAP_ANALYSIS.md</objective>
|
|
173
|
+
<execution_context>@agents/qaa-analyzer.md</execution_context>
|
|
174
|
+
<files_to_read>
|
|
175
|
+
- {OUTPUT_DIR}/SCAN_MANIFEST.md
|
|
176
|
+
- CLAUDE.md
|
|
177
|
+
</files_to_read>
|
|
178
|
+
<parameters>
|
|
179
|
+
workflow_option: 2
|
|
180
|
+
qa_analysis_path: {OUTPUT_DIR}/QA_ANALYSIS.md
|
|
181
|
+
test_inventory_path: {OUTPUT_DIR}/TEST_INVENTORY.md
|
|
182
|
+
gap_analysis_path: {OUTPUT_DIR}/GAP_ANALYSIS.md
|
|
183
|
+
</parameters>
|
|
184
|
+
"
|
|
185
|
+
)
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
**Handle analyzer return:**
|
|
189
|
+
|
|
190
|
+
- Extract `total_test_count`, `pyramid_breakdown`, and `risk_count` from the analyzer's return values.
|
|
191
|
+
- Verify all expected artifacts exist on disk.
|
|
192
|
+
|
|
193
|
+
**Verify artifacts exist:**
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
[ -f "${OUTPUT_DIR}/QA_ANALYSIS.md" ] && echo "FOUND: QA_ANALYSIS.md" || echo "MISSING: QA_ANALYSIS.md"
|
|
197
|
+
[ -f "${OUTPUT_DIR}/TEST_INVENTORY.md" ] && echo "FOUND: TEST_INVENTORY.md" || echo "MISSING: TEST_INVENTORY.md"
|
|
198
|
+
|
|
199
|
+
# For Option 1 only:
|
|
200
|
+
[ -f "${OUTPUT_DIR}/QA_REPO_BLUEPRINT.md" ] && echo "FOUND: QA_REPO_BLUEPRINT.md" || echo "MISSING: QA_REPO_BLUEPRINT.md"
|
|
201
|
+
|
|
202
|
+
# For Option 2/3 only:
|
|
203
|
+
[ -f "${OUTPUT_DIR}/GAP_ANALYSIS.md" ] && echo "FOUND: GAP_ANALYSIS.md" || echo "MISSING: GAP_ANALYSIS.md"
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
If any required artifact is missing, print error with the specific missing file and STOP.
|
|
207
|
+
</step>
|
|
208
|
+
|
|
209
|
+
<step name="print_summary">
|
|
210
|
+
## Step 5: Print Analysis Summary
|
|
211
|
+
|
|
212
|
+
Print a human-readable summary of the analysis results. No git operations are performed.
|
|
213
|
+
|
|
214
|
+
**Read QA_ANALYSIS.md to extract summary data:**
|
|
215
|
+
|
|
216
|
+
- Count risks by severity: HIGH, MEDIUM, LOW
|
|
217
|
+
- Count test cases by pyramid tier from TEST_INVENTORY.md
|
|
218
|
+
- Extract Top 10 unit test targets
|
|
219
|
+
- Extract architecture overview (framework, language, runtime)
|
|
220
|
+
|
|
221
|
+
**Print summary:**
|
|
222
|
+
|
|
223
|
+
```
|
|
224
|
+
=== Analysis Complete ===
|
|
225
|
+
|
|
226
|
+
Architecture:
|
|
227
|
+
Framework: {framework}
|
|
228
|
+
Language: {language}
|
|
229
|
+
Runtime: {runtime}
|
|
230
|
+
Frontend: {has_frontend}
|
|
231
|
+
Detection Confidence: {detection_confidence}
|
|
232
|
+
|
|
233
|
+
Risk Assessment:
|
|
234
|
+
HIGH: {high_count}
|
|
235
|
+
MEDIUM: {medium_count}
|
|
236
|
+
LOW: {low_count}
|
|
237
|
+
|
|
238
|
+
Test Case Inventory:
|
|
239
|
+
Unit Tests: {unit_count} ({unit_percent}%)
|
|
240
|
+
Integration Tests: {integration_count} ({integration_percent}%)
|
|
241
|
+
API Tests: {api_count} ({api_percent}%)
|
|
242
|
+
E2E Tests: {e2e_count} ({e2e_percent}%)
|
|
243
|
+
--------------------------
|
|
244
|
+
Total: {total_count}
|
|
245
|
+
|
|
246
|
+
Priority Distribution:
|
|
247
|
+
P0 (blocks release): {p0_count}
|
|
248
|
+
P1 (should fix): {p1_count}
|
|
249
|
+
P2 (nice to have): {p2_count}
|
|
250
|
+
|
|
251
|
+
Artifacts Produced:
|
|
252
|
+
- {OUTPUT_DIR}/SCAN_MANIFEST.md
|
|
253
|
+
- {OUTPUT_DIR}/QA_ANALYSIS.md
|
|
254
|
+
- {OUTPUT_DIR}/TEST_INVENTORY.md
|
|
255
|
+
- {OUTPUT_DIR}/QA_REPO_BLUEPRINT.md (Option 1 only)
|
|
256
|
+
- {OUTPUT_DIR}/GAP_ANALYSIS.md (Option 2/3 only)
|
|
257
|
+
|
|
258
|
+
No git operations performed. No PR created.
|
|
259
|
+
Run the full pipeline with /qa-start to generate tests.
|
|
260
|
+
===========================
|
|
261
|
+
```
|
|
262
|
+
</step>
|
|
263
|
+
|
|
264
|
+
</process>
|
|
265
|
+
|
|
266
|
+
<output>
|
|
267
|
+
This workflow produces analysis artifacts only. No test files are generated. No git operations are performed.
|
|
268
|
+
|
|
269
|
+
**Artifacts produced:**
|
|
270
|
+
|
|
271
|
+
| Artifact | When Produced | Description |
|
|
272
|
+
|----------|---------------|-------------|
|
|
273
|
+
| SCAN_MANIFEST.md | Always | Repository scan with file tree, framework detection, testable surfaces |
|
|
274
|
+
| QA_ANALYSIS.md | Always | Architecture overview, risks, top 10 targets, API targets, testing pyramid |
|
|
275
|
+
| TEST_INVENTORY.md | Always | Complete test case inventory with IDs, inputs, expected outcomes |
|
|
276
|
+
| QA_REPO_BLUEPRINT.md | Option 1 (no QA repo) | Repository structure, configs, CI/CD strategy for new QA repo |
|
|
277
|
+
| GAP_ANALYSIS.md | Option 2/3 (QA repo exists) | Coverage gaps, missing tests, broken tests, quality assessment |
|
|
278
|
+
|
|
279
|
+
**No side effects:**
|
|
280
|
+
- No git branches created
|
|
281
|
+
- No git commits made
|
|
282
|
+
- No PRs created
|
|
283
|
+
- No test files generated
|
|
284
|
+
- No source files modified
|
|
285
|
+
</output>
|
|
286
|
+
|
|
287
|
+
<error_handling>
|
|
288
|
+
| Error | Cause | Action |
|
|
289
|
+
|-------|-------|--------|
|
|
290
|
+
| Dev repo path does not exist | Invalid --dev-repo argument | Print error with path, STOP |
|
|
291
|
+
| QA repo path does not exist | Invalid --qa-repo argument | Print error with path, STOP |
|
|
292
|
+
| Scanner returns STOP | No testable surfaces or uncertain framework | Print scanner's reason, STOP |
|
|
293
|
+
| SCAN_MANIFEST.md missing after scanner | Scanner failed silently | Print error, STOP |
|
|
294
|
+
| Analyzer artifacts missing | Analyzer failed silently | Print error with specific missing file, STOP |
|
|
295
|
+
| Framework detection LOW confidence | Ambiguous tech stack | Scanner checkpoints for user confirmation |
|
|
296
|
+
</error_handling>
|