qaa-agent 1.6.1 → 1.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/.claude/commands/create-test.md +164 -164
  2. package/.claude/commands/qa-audit.md +37 -37
  3. package/.claude/commands/qa-blueprint.md +54 -54
  4. package/.claude/commands/qa-fix.md +36 -36
  5. package/.claude/commands/qa-from-ticket.md +24 -24
  6. package/.claude/commands/qa-gap.md +20 -20
  7. package/.claude/commands/qa-map.md +47 -47
  8. package/.claude/commands/qa-pom.md +36 -36
  9. package/.claude/commands/qa-pr.md +23 -23
  10. package/.claude/commands/qa-pyramid.md +37 -37
  11. package/.claude/commands/qa-report.md +38 -38
  12. package/.claude/commands/qa-research.md +33 -33
  13. package/.claude/commands/qa-start.md +22 -22
  14. package/.claude/commands/qa-testid.md +19 -19
  15. package/.claude/commands/qa-validate.md +42 -42
  16. package/.claude/commands/update-test.md +58 -58
  17. package/.claude/settings.json +20 -20
  18. package/.claude/skills/qa-bug-detective/SKILL.md +122 -122
  19. package/.claude/skills/qa-learner/SKILL.md +150 -150
  20. package/.claude/skills/qa-repo-analyzer/SKILL.md +88 -88
  21. package/.claude/skills/qa-self-validator/SKILL.md +109 -109
  22. package/.claude/skills/qa-template-engine/SKILL.md +113 -113
  23. package/.claude/skills/qa-testid-injector/SKILL.md +93 -93
  24. package/.claude/skills/qa-workflow-documenter/SKILL.md +87 -87
  25. package/.mcp.json +8 -8
  26. package/CHANGELOG.md +71 -71
  27. package/CLAUDE.md +553 -553
  28. package/agents/qa-pipeline-orchestrator.md +1378 -1378
  29. package/agents/qaa-analyzer.md +524 -524
  30. package/agents/qaa-bug-detective.md +446 -446
  31. package/agents/qaa-codebase-mapper.md +935 -935
  32. package/agents/qaa-e2e-runner.md +415 -415
  33. package/agents/qaa-executor.md +651 -651
  34. package/agents/qaa-planner.md +390 -390
  35. package/agents/qaa-project-researcher.md +319 -319
  36. package/agents/qaa-scanner.md +424 -424
  37. package/agents/qaa-testid-injector.md +585 -585
  38. package/agents/qaa-validator.md +452 -452
  39. package/bin/install.cjs +198 -198
  40. package/bin/lib/commands.cjs +709 -709
  41. package/bin/lib/config.cjs +307 -307
  42. package/bin/lib/core.cjs +497 -497
  43. package/bin/lib/frontmatter.cjs +299 -299
  44. package/bin/lib/init.cjs +989 -989
  45. package/bin/lib/milestone.cjs +241 -241
  46. package/bin/lib/model-profiles.cjs +60 -60
  47. package/bin/lib/phase.cjs +911 -911
  48. package/bin/lib/roadmap.cjs +306 -306
  49. package/bin/lib/state.cjs +748 -748
  50. package/bin/lib/template.cjs +222 -222
  51. package/bin/lib/verify.cjs +842 -842
  52. package/bin/qaa-tools.cjs +607 -607
  53. package/docs/COMMANDS.md +341 -341
  54. package/docs/DEMO.md +182 -182
  55. package/docs/TESTING.md +156 -156
  56. package/package.json +41 -41
  57. package/templates/failure-classification.md +391 -391
  58. package/templates/gap-analysis.md +409 -409
  59. package/templates/pr-template.md +48 -48
  60. package/templates/qa-analysis.md +381 -381
  61. package/templates/qa-audit-report.md +465 -465
  62. package/templates/qa-repo-blueprint.md +636 -636
  63. package/templates/scan-manifest.md +312 -312
  64. package/templates/test-inventory.md +582 -582
  65. package/templates/testid-audit-report.md +354 -354
  66. package/templates/validation-report.md +243 -243
  67. package/workflows/qa-analyze.md +296 -296
  68. package/workflows/qa-from-ticket.md +536 -536
  69. package/workflows/qa-gap.md +303 -303
  70. package/workflows/qa-pr.md +389 -389
  71. package/workflows/qa-start.md +1168 -1168
  72. package/workflows/qa-testid.md +356 -356
  73. package/workflows/qa-validate.md +295 -295
@@ -1,58 +1,58 @@
1
- # Update and Improve Existing Tests
2
-
3
- Audit existing test files and apply targeted improvements. NEVER deletes or rewrites working tests without user approval. Surgical: add, fix, improve -- never replace.
4
-
5
- ## Usage
6
-
7
- /update-test <path-to-tests> [--scope fix|improve|add|full]
8
-
9
- - path-to-tests: directory or specific test files to improve
10
- - --scope: what to do (default: full)
11
- - fix: repair broken tests only
12
- - improve: upgrade locators, assertions, POM structure
13
- - add: add missing test cases without modifying existing
14
- - full: audit everything, then improve with approval
15
-
16
- ## What It Produces
17
-
18
- - QA_AUDIT_REPORT.md -- current quality assessment
19
- - Improved test files (after user approval of audit findings)
20
-
21
- ## Instructions
22
-
23
- 1. Read `CLAUDE.md` -- quality gates, locator tiers, assertion rules, POM rules.
24
- 2. Invoke validator agent in audit mode first:
25
-
26
- Task(
27
- prompt="
28
- <objective>Audit existing test quality and produce QA_AUDIT_REPORT.md</objective>
29
- <execution_context>@agents/qaa-validator.md</execution_context>
30
- <files_to_read>
31
- - CLAUDE.md
32
- </files_to_read>
33
- <parameters>
34
- user_input: $ARGUMENTS
35
- mode: audit
36
- </parameters>
37
- "
38
- )
39
-
40
- 3. Present audit results and wait for user approval.
41
- 4. Invoke executor agent to apply approved improvements:
42
-
43
- Task(
44
- prompt="
45
- <objective>Apply approved improvements to existing tests without deleting working tests</objective>
46
- <execution_context>@agents/qaa-executor.md</execution_context>
47
- <files_to_read>
48
- - CLAUDE.md
49
- - .qa-output/QA_AUDIT_REPORT.md
50
- </files_to_read>
51
- <parameters>
52
- user_input: $ARGUMENTS
53
- mode: update
54
- </parameters>
55
- "
56
- )
57
-
58
- $ARGUMENTS
1
+ # Update and Improve Existing Tests
2
+
3
+ Audit existing test files and apply targeted improvements. NEVER deletes or rewrites working tests without user approval. Surgical: add, fix, improve -- never replace.
4
+
5
+ ## Usage
6
+
7
+ /update-test <path-to-tests> [--scope fix|improve|add|full]
8
+
9
+ - path-to-tests: directory or specific test files to improve
10
+ - --scope: what to do (default: full)
11
+ - fix: repair broken tests only
12
+ - improve: upgrade locators, assertions, POM structure
13
+ - add: add missing test cases without modifying existing
14
+ - full: audit everything, then improve with approval
15
+
16
+ ## What It Produces
17
+
18
+ - QA_AUDIT_REPORT.md -- current quality assessment
19
+ - Improved test files (after user approval of audit findings)
20
+
21
+ ## Instructions
22
+
23
+ 1. Read `CLAUDE.md` -- quality gates, locator tiers, assertion rules, POM rules.
24
+ 2. Invoke validator agent in audit mode first:
25
+
26
+ Task(
27
+ prompt="
28
+ <objective>Audit existing test quality and produce QA_AUDIT_REPORT.md</objective>
29
+ <execution_context>@agents/qaa-validator.md</execution_context>
30
+ <files_to_read>
31
+ - CLAUDE.md
32
+ </files_to_read>
33
+ <parameters>
34
+ user_input: $ARGUMENTS
35
+ mode: audit
36
+ </parameters>
37
+ "
38
+ )
39
+
40
+ 3. Present audit results and wait for user approval.
41
+ 4. Invoke executor agent to apply approved improvements:
42
+
43
+ Task(
44
+ prompt="
45
+ <objective>Apply approved improvements to existing tests without deleting working tests</objective>
46
+ <execution_context>@agents/qaa-executor.md</execution_context>
47
+ <files_to_read>
48
+ - CLAUDE.md
49
+ - .qa-output/QA_AUDIT_REPORT.md
50
+ </files_to_read>
51
+ <parameters>
52
+ user_input: $ARGUMENTS
53
+ mode: update
54
+ </parameters>
55
+ "
56
+ )
57
+
58
+ $ARGUMENTS
@@ -1,20 +1,20 @@
1
- {
2
- "permissions": {
3
- "allow": [
4
- "Bash(*)",
5
- "Read",
6
- "Write",
7
- "Edit",
8
- "Glob",
9
- "Grep",
10
- "Agent",
11
- "WebFetch",
12
- "WebSearch",
13
- "NotebookEdit",
14
- "mcp__playwright__*"
15
- ]
16
- },
17
- "env": {
18
- "CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1"
19
- }
20
- }
1
+ {
2
+ "permissions": {
3
+ "allow": [
4
+ "Bash(*)",
5
+ "Read",
6
+ "Write",
7
+ "Edit",
8
+ "Glob",
9
+ "Grep",
10
+ "Agent",
11
+ "WebFetch",
12
+ "WebSearch",
13
+ "NotebookEdit",
14
+ "mcp__playwright__*"
15
+ ]
16
+ },
17
+ "env": {
18
+ "CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS": "1"
19
+ }
20
+ }
@@ -1,122 +1,122 @@
1
- ---
2
- name: qa-bug-detective
3
- description: QA Bug Detective. Runs generated tests and classifies failures as APPLICATION BUG, TEST CODE ERROR, ENVIRONMENT ISSUE, or INCONCLUSIVE with evidence and confidence levels. Use when user wants to run tests and classify results, investigate test failures, determine if failures are bugs or test issues, debug failing tests, triage test results, or understand why tests are failing. Triggers on "run tests", "classify failures", "why is this failing", "test failures", "debug tests", "triage results", "is this a bug or test error", "investigate failures".
4
- ---
5
-
6
- # QA Bug Detective
7
-
8
- ## Purpose
9
-
10
- Run generated tests and classify every failure into one of four categories with evidence and confidence levels. Auto-fix TEST CODE ERRORS when confidence is HIGH.
11
-
12
- ## Classification Decision Tree
13
-
14
- ```
15
- Test fails
16
- ├── Syntax/import error in TEST file?
17
- │ └── YES → TEST CODE ERROR
18
- ├── Error occurs in PRODUCTION code path?
19
- │ ├── Known bug / unexpected behavior? → APPLICATION BUG
20
- │ └── Code works as designed but test expectation wrong? → TEST CODE ERROR
21
- ├── Connection refused / timeout / missing env var?
22
- │ └── YES → ENVIRONMENT ISSUE
23
- └── Can't determine?
24
- └── INCONCLUSIVE
25
- ```
26
-
27
- ## Classification Categories
28
-
29
- ### APPLICATION BUG
30
- - Error manifests in production code (not test code)
31
- - Stack trace points to src/ or app/ code
32
- - Behavior contradicts documented requirements or API contracts
33
- - **Action**: Report only. NEVER auto-fix application code.
34
-
35
- ### TEST CODE ERROR
36
- - Import/require fails (wrong path, missing module)
37
- - Selector doesn't match current DOM
38
- - Assertion expects wrong value (test written incorrectly)
39
- - Missing await, wrong API usage, stale fixture reference
40
- - **Action**: Auto-fix if HIGH confidence. Report if MEDIUM or lower.
41
-
42
- ### ENVIRONMENT ISSUE
43
- - Connection refused (database, API, external service)
44
- - Timeout waiting for resource
45
- - Missing environment variable
46
- - File/directory not found (test infrastructure)
47
- - **Action**: Report with suggested resolution steps.
48
-
49
- ### INCONCLUSIVE
50
- - Error is ambiguous
51
- - Could be multiple root causes
52
- - Insufficient data to classify
53
- - **Action**: Report with what's known, request more info.
54
-
55
- ## Evidence Requirements
56
-
57
- Every classification MUST include:
58
- 1. **File path**: Exact file where error occurs
59
- 2. **Line number**: Specific line of failure
60
- 3. **Error message**: Complete error text
61
- 4. **Code snippet**: The specific code proving the classification
62
- 5. **Confidence level**: HIGH / MEDIUM-HIGH / MEDIUM / LOW
63
- 6. **Reasoning**: Why this classification, not another
64
-
65
- ## Confidence Levels
66
-
67
- | Level | Definition |
68
- |-------|------------|
69
- | HIGH | Clear evidence in one direction, no ambiguity |
70
- | MEDIUM-HIGH | Strong evidence but minor ambiguity |
71
- | MEDIUM | Evidence points one way but alternatives exist |
72
- | LOW | Insufficient data, multiple possible causes |
73
-
74
- ## Auto-Fix Rules
75
-
76
- Only auto-fix when:
77
- - Classification = TEST CODE ERROR
78
- - Confidence = HIGH
79
- - Fix is mechanical (import path, selector, assertion value, config)
80
-
81
- Fix types:
82
- - Import path corrections
83
- - Selector updates (match current DOM/data-testid)
84
- - Assertion value updates (match current actual behavior)
85
- - Config fixes (baseURL, timeout values)
86
- - Missing await keywords
87
- - Fixture path corrections
88
-
89
- **NEVER auto-fix**: Application bugs, environment issues, anything with confidence < HIGH.
90
-
91
- ## Output: FAILURE_CLASSIFICATION_REPORT.md
92
-
93
- ```markdown
94
- # Failure Classification Report
95
-
96
- ## Summary
97
- | Classification | Count | Auto-Fixed | Needs Attention |
98
- |---------------|-------|-----------|----------------|
99
- | APPLICATION BUG | N | 0 | N |
100
- | TEST CODE ERROR | N | N | N |
101
- | ENVIRONMENT ISSUE | N | 0 | N |
102
- | INCONCLUSIVE | N | 0 | N |
103
-
104
- ## Detailed Analysis
105
-
106
- ### Failure 1: [test name]
107
- - **Classification**: [category]
108
- - **Confidence**: [level]
109
- - **File**: [path]:[line]
110
- - **Error**: [message]
111
- - **Evidence**: [code snippet + reasoning]
112
- - **Action Taken**: [auto-fixed / reported]
113
- - **Resolution**: [what was fixed / what needs human attention]
114
- ```
115
-
116
- ## Quality Gate
117
-
118
- - [ ] Every failure classified with evidence
119
- - [ ] Confidence level assigned to each
120
- - [ ] No application bugs auto-fixed
121
- - [ ] Auto-fixes only applied at HIGH confidence
122
- - [ ] FAILURE_CLASSIFICATION_REPORT.md produced
1
+ ---
2
+ name: qa-bug-detective
3
+ description: QA Bug Detective. Runs generated tests and classifies failures as APPLICATION BUG, TEST CODE ERROR, ENVIRONMENT ISSUE, or INCONCLUSIVE with evidence and confidence levels. Use when user wants to run tests and classify results, investigate test failures, determine if failures are bugs or test issues, debug failing tests, triage test results, or understand why tests are failing. Triggers on "run tests", "classify failures", "why is this failing", "test failures", "debug tests", "triage results", "is this a bug or test error", "investigate failures".
4
+ ---
5
+
6
+ # QA Bug Detective
7
+
8
+ ## Purpose
9
+
10
+ Run generated tests and classify every failure into one of four categories with evidence and confidence levels. Auto-fix TEST CODE ERRORS when confidence is HIGH.
11
+
12
+ ## Classification Decision Tree
13
+
14
+ ```
15
+ Test fails
16
+ ├── Syntax/import error in TEST file?
17
+ │ └── YES → TEST CODE ERROR
18
+ ├── Error occurs in PRODUCTION code path?
19
+ │ ├── Known bug / unexpected behavior? → APPLICATION BUG
20
+ │ └── Code works as designed but test expectation wrong? → TEST CODE ERROR
21
+ ├── Connection refused / timeout / missing env var?
22
+ │ └── YES → ENVIRONMENT ISSUE
23
+ └── Can't determine?
24
+ └── INCONCLUSIVE
25
+ ```
26
+
27
+ ## Classification Categories
28
+
29
+ ### APPLICATION BUG
30
+ - Error manifests in production code (not test code)
31
+ - Stack trace points to src/ or app/ code
32
+ - Behavior contradicts documented requirements or API contracts
33
+ - **Action**: Report only. NEVER auto-fix application code.
34
+
35
+ ### TEST CODE ERROR
36
+ - Import/require fails (wrong path, missing module)
37
+ - Selector doesn't match current DOM
38
+ - Assertion expects wrong value (test written incorrectly)
39
+ - Missing await, wrong API usage, stale fixture reference
40
+ - **Action**: Auto-fix if HIGH confidence. Report if MEDIUM or lower.
41
+
42
+ ### ENVIRONMENT ISSUE
43
+ - Connection refused (database, API, external service)
44
+ - Timeout waiting for resource
45
+ - Missing environment variable
46
+ - File/directory not found (test infrastructure)
47
+ - **Action**: Report with suggested resolution steps.
48
+
49
+ ### INCONCLUSIVE
50
+ - Error is ambiguous
51
+ - Could be multiple root causes
52
+ - Insufficient data to classify
53
+ - **Action**: Report with what's known, request more info.
54
+
55
+ ## Evidence Requirements
56
+
57
+ Every classification MUST include:
58
+ 1. **File path**: Exact file where error occurs
59
+ 2. **Line number**: Specific line of failure
60
+ 3. **Error message**: Complete error text
61
+ 4. **Code snippet**: The specific code proving the classification
62
+ 5. **Confidence level**: HIGH / MEDIUM-HIGH / MEDIUM / LOW
63
+ 6. **Reasoning**: Why this classification, not another
64
+
65
+ ## Confidence Levels
66
+
67
+ | Level | Definition |
68
+ |-------|------------|
69
+ | HIGH | Clear evidence in one direction, no ambiguity |
70
+ | MEDIUM-HIGH | Strong evidence but minor ambiguity |
71
+ | MEDIUM | Evidence points one way but alternatives exist |
72
+ | LOW | Insufficient data, multiple possible causes |
73
+
74
+ ## Auto-Fix Rules
75
+
76
+ Only auto-fix when:
77
+ - Classification = TEST CODE ERROR
78
+ - Confidence = HIGH
79
+ - Fix is mechanical (import path, selector, assertion value, config)
80
+
81
+ Fix types:
82
+ - Import path corrections
83
+ - Selector updates (match current DOM/data-testid)
84
+ - Assertion value updates (match current actual behavior)
85
+ - Config fixes (baseURL, timeout values)
86
+ - Missing await keywords
87
+ - Fixture path corrections
88
+
89
+ **NEVER auto-fix**: Application bugs, environment issues, anything with confidence < HIGH.
90
+
91
+ ## Output: FAILURE_CLASSIFICATION_REPORT.md
92
+
93
+ ```markdown
94
+ # Failure Classification Report
95
+
96
+ ## Summary
97
+ | Classification | Count | Auto-Fixed | Needs Attention |
98
+ |---------------|-------|-----------|----------------|
99
+ | APPLICATION BUG | N | 0 | N |
100
+ | TEST CODE ERROR | N | N | N |
101
+ | ENVIRONMENT ISSUE | N | 0 | N |
102
+ | INCONCLUSIVE | N | 0 | N |
103
+
104
+ ## Detailed Analysis
105
+
106
+ ### Failure 1: [test name]
107
+ - **Classification**: [category]
108
+ - **Confidence**: [level]
109
+ - **File**: [path]:[line]
110
+ - **Error**: [message]
111
+ - **Evidence**: [code snippet + reasoning]
112
+ - **Action Taken**: [auto-fixed / reported]
113
+ - **Resolution**: [what was fixed / what needs human attention]
114
+ ```
115
+
116
+ ## Quality Gate
117
+
118
+ - [ ] Every failure classified with evidence
119
+ - [ ] Confidence level assigned to each
120
+ - [ ] No application bugs auto-fixed
121
+ - [ ] Auto-fixes only applied at HIGH confidence
122
+ - [ ] FAILURE_CLASSIFICATION_REPORT.md produced