qaa-agent 1.9.5 → 1.9.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +38 -0
- package/CLAUDE.md +5 -3
- package/VERSION +1 -1
- package/agents/qaa-bug-detective.md +2 -0
- package/agents/qaa-codebase-mapper.md +4 -0
- package/agents/qaa-discovery.md +10 -0
- package/agents/qaa-e2e-runner.md +2 -0
- package/agents/qaa-executor.md +19 -0
- package/agents/qaa-project-researcher.md +6 -0
- package/agents/qaa-scanner.md +9 -0
- package/bin/install.cjs +13 -3
- package/bin/lib/intent-detector.cjs +6 -4
- package/commands/qa-audit.md +77 -26
- package/commands/qa-create-test-ado.md +486 -404
- package/commands/qa-create-test.md +58 -0
- package/commands/qa-fix.md +73 -3
- package/commands/qa-map.md +76 -4
- package/commands/qa-pr.md +36 -0
- package/commands/qa-research.md +67 -0
- package/commands/qa-start.md +37 -0
- package/commands/qa-test-report.md +82 -0
- package/commands/qa-testid.md +85 -0
- package/package.json +41 -41
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,44 @@
|
|
|
3
3
|
|
|
4
4
|
All notable changes to QAA (QA Automation Agent) are documented here.
|
|
5
5
|
|
|
6
|
+
## [1.9.7] - 2026-07-29
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
|
|
10
|
+
- **Pre-flight checks with UX-friendly error messages across all 10 commands** — every slash command now runs guard checks before its main flow and, on failure, prints a clear, actionable error: a one-line title, a short explanation, two or three concrete options, and a `Try:` command. Coverage includes invalid `--dev-repo`/source paths, missing required MCP servers (Playwright for `/qa-audit`, `/qa-testid`, `/qa-create-test` from-aut and `/qa-fix --run`; Context7 for `/qa-research`; Azure DevOps for `/qa-create-test-ado` and `/qa-test-report`), undetectable frameworks, non-git repositories, and a missing/unauthenticated GitHub CLI. Required-dependency failures STOP; optional-enhancement failures (e.g. live DOM verification) WARN and continue in degraded mode. Every suggested flag is validated against what the command actually accepts.
|
|
11
|
+
|
|
12
|
+
### Fixed
|
|
13
|
+
|
|
14
|
+
- **Installer now syncs `workflows/` to the live location** — `install.cjs` Step 4 mirrors `workflows/qa-*.md` into `<claudeDir>/workflows/` alongside commands and agents, using the same scope-aware sync. Commands reference workflows via `@workflows/qa-*.md`; syncing them makes that resolution direct instead of a fallback runtime search, on every install (global or local).
|
|
15
|
+
|
|
16
|
+
### Documentation
|
|
17
|
+
|
|
18
|
+
- **QAA_SYSTEM_OVERVIEW.md refreshed for the current system state** — the Installation section now documents the four installer steps, including the live sync of commands, agents, workflows, and CLAUDE.md to the locations Claude Code loads from.
|
|
19
|
+
|
|
20
|
+
## [1.9.6] - 2026-07-14
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
|
|
24
|
+
- **Native subagent routing across the 5 active commands** — all 12 QAA agents are now registered as native Claude Code subagents (added YAML frontmatter to `qaa-scanner` and `qaa-discovery`, the last two that lacked it). `/qa-fix`, `/qa-audit`, `/qa-map`, `/qa-research`, and `/qa-create-test` now spawn their sub-agents via `subagent_type="qaa-X"` for direct routing. The change is additive: the embedded Task() directive is kept in every spawn as a fallback for environments where native routing isn't available yet (e.g. a fresh install before restart).
|
|
25
|
+
- **Context7 cache cascade wired into the executor's generation process** — the executor now runs the cache cascade (cache → research docs → query Context7 → write back) as an explicit step before generating files, instead of only declaring it in a side section. Reduces redundant Context7 calls during generation.
|
|
26
|
+
- **`--dev-repo` natural-language alias in `/qa-audit`** — the intent detector now resolves a dev-repo path from natural language for the audit command (for coverage cross-reference), not only from the explicit flag.
|
|
27
|
+
- **Non-numeric project-key ticket detection** — the intent detector now recognizes standalone ticket references like `#PROJ-abc-fix` (uppercase project prefix + hyphen), in addition to numeric refs, without matching plain words.
|
|
28
|
+
|
|
29
|
+
### Changed
|
|
30
|
+
|
|
31
|
+
- **MANDATORY execution headers extended to the remaining natural-language commands** — `/qa-audit`, `/qa-map`, `/qa-research`, and `/qa-testid` now open with a `## ⚠ MANDATORY: How to Execute This Command` block defining their full ordered flow plus DO / DO NOT, matching the commands that already had it. (`/qa-audit`'s partial "First Tool Call" header was upgraded to the full form.)
|
|
32
|
+
- **MANDATORY verification headers added to the remaining agents** — `qaa-scanner`, `qaa-codebase-mapper`, `qaa-discovery`, and `qaa-project-researcher` now carry the `## MANDATORY verification` header before their bash checklist, matching the other agents.
|
|
33
|
+
- **`CLAUDE.md` updated to reflect native routing is enabled** — the "Standard Task() Directive" section no longer claims native `subagent_type` routing is disabled; it documents native routing as the primary mechanism where wired, with the directive as the fallback.
|
|
34
|
+
|
|
35
|
+
### Fixed
|
|
36
|
+
|
|
37
|
+
- **`/qa-fix` FIX MODE no longer allows a silent Phase 1 → Phase 2 transition** — the confirmation checkpoint before applying fixes is now an explicit HARD STOP gate (no approval = no fixes), and Phase 2 restates that it must not be reached without explicit user approval.
|
|
38
|
+
- **DOM-probe wording tightened to prevent substitution for real test runs** — the `/qa-fix` locator cross-check layer is relabeled from "Optional" to "Conditional" (it still runs only when Playwright MCP and an app URL are available) and now states the probe MUST NEVER be substituted for a real test run. A command-level runner HARD STOP was added: if the test runner cannot execute, the command reports an ENVIRONMENT ISSUE and stops instead of falling back to a DOM probe.
|
|
39
|
+
|
|
40
|
+
### Documentation
|
|
41
|
+
|
|
42
|
+
- **Context7 `resolve-library-id` call shape documented** — `qaa-project-researcher`, `qaa-executor`, `qaa-e2e-runner`, and `qaa-bug-detective` now note that the library name is passed as a string via `libraryName` (`{ libraryName: "<string>" }`) and what an "expected: string" validation error indicates.
|
|
43
|
+
|
|
6
44
|
## [1.9.5] - 2026-05-05
|
|
7
45
|
|
|
8
46
|
### Added
|
package/CLAUDE.md
CHANGED
|
@@ -679,7 +679,9 @@ These are **Capa 1** of a 3-layer validation system. Capa 2 is each agent's `<qu
|
|
|
679
679
|
|
|
680
680
|
## Standard Task() Directive
|
|
681
681
|
|
|
682
|
-
Every `Task()` / `Agent()` spawn in this codebase (orchestrator, commands, workflows) MUST embed the following directive **verbatim** at the very top of the spawned prompt, substituting ONLY `<AGENT_FILE>` with the target agent's exact filename (e.g. `qaa-scanner`, `qa-pipeline-orchestrator`). Do NOT paraphrase it
|
|
682
|
+
Every `Task()` / `Agent()` spawn in this codebase (orchestrator, commands, workflows) MUST embed the following directive **verbatim** at the very top of the spawned prompt, substituting ONLY `<AGENT_FILE>` with the target agent's exact filename (e.g. `qaa-scanner`, `qa-pipeline-orchestrator`). Do NOT paraphrase it.
|
|
683
|
+
|
|
684
|
+
Native subagent routing **is enabled** for the 12 QAA agents — they are registered in `~/.claude/agents/` with YAML frontmatter (verified via live routing smoke tests). Spawns SHOULD therefore set `subagent_type: "qaa-X"` to route directly to the dedicated agent. The directive below is retained in every spawn prompt as a **fallback**: it preserves the operating contract even where native routing isn't available yet (e.g. a fresh install before Claude Code is restarted). As of v1.9.6 the 5 active commands (qa-fix, qa-audit, qa-map, qa-research, qa-create-test) use native `subagent_type`; `workflows/qa-start.md` and the orchestrator still rely on the directive alone (see architectural note below).
|
|
683
685
|
|
|
684
686
|
```
|
|
685
687
|
<critical_directive priority="MANDATORY">
|
|
@@ -712,7 +714,7 @@ trusted.
|
|
|
712
714
|
QAA hardens execution at two layers, and new commands/agents MUST follow both:
|
|
713
715
|
|
|
714
716
|
1. **Command headers** — every NL command opens with a `## ⚠ MANDATORY: How to Execute This Command` block defining the full ordered flow + DO / DO NOT (not just the first tool call).
|
|
715
|
-
2. **Task() prompts** — every spawn embeds the Standard Task() Directive above.
|
|
717
|
+
2. **Task() prompts** — every spawn sets `subagent_type: "qaa-X"` for native routing (where wired) AND embeds the Standard Task() Directive above as a fallback.
|
|
716
718
|
|
|
717
|
-
This is the standard QAA enforcement pattern. Native `subagent_type` routing is
|
|
719
|
+
This is the standard QAA enforcement pattern. Native `subagent_type` routing **is now enabled** (all 12 agents are registered in `~/.claude/agents/`), so it is the primary routing mechanism where wired; the embedded directive remains as the **fallback** that guarantees the operating contract even when native routing is unavailable. Removing the directive is deferred to a future cleanup once native routing has been proven in production.
|
|
718
720
|
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
1.9.
|
|
1
|
+
1.9.7
|
|
@@ -93,6 +93,8 @@ When the project's framework version is known (detected from `package.json`, `re
|
|
|
93
93
|
```
|
|
94
94
|
# 1. Resolve base libraryId
|
|
95
95
|
RESOLVED_ID = mcp__context7__resolve-library-id({ libraryName: "{framework-name}" })
|
|
96
|
+
# SHAPE: pass the name as a STRING via `libraryName` → { libraryName: "<string>" }.
|
|
97
|
+
# An "expected: string" validation error means the argument shape is wrong (don't pass a bare string).
|
|
96
98
|
# example: "/microsoft/playwright"
|
|
97
99
|
|
|
98
100
|
# 2. If project version is detected (e.g., "1.40.0"):
|
|
@@ -945,6 +945,10 @@ Before considering codebase mapping complete, verify ALL of the following:
|
|
|
945
945
|
If any check fails, fix the issue before returning. Do not deliver codebase docs that fail their own quality gate.
|
|
946
946
|
</quality_gate>
|
|
947
947
|
|
|
948
|
+
## MANDATORY verification — run ALL commands below, no exceptions, no skipping
|
|
949
|
+
|
|
950
|
+
Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
|
|
951
|
+
|
|
948
952
|
<bash_checklist>
|
|
949
953
|
Run this checklist verbatim before returning control:
|
|
950
954
|
|
package/agents/qaa-discovery.md
CHANGED
|
@@ -1,3 +1,9 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qaa-discovery
|
|
3
|
+
description: Interactive QA discovery agent — gathers project context, risk priorities, and scope at three checkpoints (pre-scan, mid-pipeline, post-validate).
|
|
4
|
+
tools: Read, Write, Bash, Grep, Glob, AskUserQuestion
|
|
5
|
+
---
|
|
6
|
+
|
|
1
7
|
<purpose>
|
|
2
8
|
Extract the context and decisions needed to run a high-quality QA pipeline. This agent runs at three points:
|
|
3
9
|
|
|
@@ -378,6 +384,10 @@ Before considering the discovery checkpoint complete, verify ALL of the followin
|
|
|
378
384
|
If --auto and no input expected, return immediately with skipped status. Otherwise wait for user input.
|
|
379
385
|
</quality_gate>
|
|
380
386
|
|
|
387
|
+
## MANDATORY verification — run ALL commands below, no exceptions, no skipping
|
|
388
|
+
|
|
389
|
+
Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
|
|
390
|
+
|
|
381
391
|
<bash_checklist>
|
|
382
392
|
Run this checklist verbatim before returning:
|
|
383
393
|
|
package/agents/qaa-e2e-runner.md
CHANGED
|
@@ -91,6 +91,8 @@ When the project's framework version is known (detected from `package.json`, `re
|
|
|
91
91
|
```
|
|
92
92
|
# 1. Resolve base libraryId
|
|
93
93
|
RESOLVED_ID = mcp__context7__resolve-library-id({ libraryName: "{framework-name}" })
|
|
94
|
+
# SHAPE: pass the name as a STRING via `libraryName` → { libraryName: "<string>" }.
|
|
95
|
+
# An "expected: string" validation error means the argument shape is wrong (don't pass a bare string).
|
|
94
96
|
# example: "/microsoft/playwright"
|
|
95
97
|
|
|
96
98
|
# 2. If project version is detected (e.g., "1.40.0"):
|
package/agents/qaa-executor.md
CHANGED
|
@@ -157,6 +157,8 @@ When the project's framework version is known (detected from `package.json`, `re
|
|
|
157
157
|
```
|
|
158
158
|
# 1. Resolve base libraryId
|
|
159
159
|
RESOLVED_ID = mcp__context7__resolve-library-id({ libraryName: "{framework-name}" })
|
|
160
|
+
# SHAPE: pass the name as a STRING via `libraryName` → { libraryName: "<string>" }.
|
|
161
|
+
# An "expected: string" validation error means the argument shape is wrong (don't pass a bare string).
|
|
160
162
|
# example: "/microsoft/playwright"
|
|
161
163
|
|
|
162
164
|
# 2. If project version is detected (e.g., "1.40.0"):
|
|
@@ -404,6 +406,23 @@ Only commit if files were actually created. If all infrastructure already exists
|
|
|
404
406
|
<step name="generate_per_task">
|
|
405
407
|
For each task in the generation plan (in dependency order from topological sort), generate the assigned files.
|
|
406
408
|
|
|
409
|
+
**Before generating — resolve framework docs via the Context7 cache cascade (run this here so it is actually exercised, not just declared in the section above):**
|
|
410
|
+
|
|
411
|
+
For the detected `{framework}` + `{version}` (from SCAN_MANIFEST.md / the framework registry), resolve the selector/locator + assertion API BEFORE writing any file:
|
|
412
|
+
|
|
413
|
+
1. **Read cache first:**
|
|
414
|
+
```bash
|
|
415
|
+
CACHED=$(node ~/.claude/qaa/bin/lib/context7-cache.cjs read {framework} {version} "generation-api" 2>/dev/null)
|
|
416
|
+
```
|
|
417
|
+
2. If `CACHED` is a hit → use it; skip the Context7 call.
|
|
418
|
+
3. If cache miss → check research docs (`.qa-output/research/FRAMEWORK_CAPABILITIES.md`, `.qa-output/research/TESTING_STACK.md`). If they cover the API → use them.
|
|
419
|
+
4. If both miss → query Context7 (`resolve-library-id` → `query-docs`), then write the response back to cache for next time:
|
|
420
|
+
```bash
|
|
421
|
+
node ~/.claude/qaa/bin/lib/context7-cache.cjs write {framework} {version} "generation-api" "$RESPONSE"
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
Do NOT generate framework-specific syntax from memory when the cascade can supply the verified API. See `<context7_verification>` for the full cascade contract.
|
|
425
|
+
|
|
407
426
|
**Execution loop:**
|
|
408
427
|
|
|
409
428
|
For each task (ordered by dependencies):
|
|
@@ -170,6 +170,8 @@ When the project's framework version is known (detected from `package.json`, `re
|
|
|
170
170
|
```
|
|
171
171
|
# 1. Resolve base libraryId
|
|
172
172
|
RESOLVED_ID = mcp__context7__resolve-library-id({ libraryName: "{framework-name}" })
|
|
173
|
+
# SHAPE: pass the name as a STRING via `libraryName` → { libraryName: "<string>" }.
|
|
174
|
+
# An "expected: string" validation error means the argument shape is wrong (don't pass a bare string).
|
|
173
175
|
# example: "/microsoft/playwright"
|
|
174
176
|
|
|
175
177
|
# 2. If project version is detected (e.g., "1.40.0"):
|
|
@@ -480,6 +482,10 @@ Before considering research complete, verify ALL of the following:
|
|
|
480
482
|
If any check fails, fix before delivering. The researcher must NOT commit; the orchestrator commits.
|
|
481
483
|
</quality_gate>
|
|
482
484
|
|
|
485
|
+
## MANDATORY verification — run ALL commands below, no exceptions, no skipping
|
|
486
|
+
|
|
487
|
+
Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
|
|
488
|
+
|
|
483
489
|
<bash_checklist>
|
|
484
490
|
Run this checklist verbatim before returning:
|
|
485
491
|
|
package/agents/qaa-scanner.md
CHANGED
|
@@ -1,3 +1,9 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qaa-scanner
|
|
3
|
+
description: Scans a developer repo to produce SCAN_MANIFEST.md — detects framework, language, runtime, and testable surfaces. First stage of the QA pipeline.
|
|
4
|
+
tools: Read, Write, Bash, Grep, Glob
|
|
5
|
+
---
|
|
6
|
+
|
|
1
7
|
<purpose>
|
|
2
8
|
Scan a developer repository to produce a comprehensive SCAN_MANIFEST.md. Reads the repo's file tree, package manifests, configuration files, and source code to detect framework, language, runtime, and all testable surfaces. This agent is spawned by the orchestrator as the first stage of the QA automation pipeline. It accepts a DEV repo path and an output path from the orchestrator prompt, scans the target repository, and writes a structured manifest that downstream agents (qa-analyzer, qa-testid-injector) consume without needing to re-read the source repository.
|
|
3
9
|
</purpose>
|
|
@@ -456,6 +462,9 @@ Before considering the scan complete, verify ALL of the following.
|
|
|
456
462
|
If any check fails, fix the issue before writing the final output. Do not proceed with a failing quality gate.
|
|
457
463
|
</quality_gate>
|
|
458
464
|
|
|
465
|
+
## MANDATORY verification — run ALL commands below, no exceptions, no skipping
|
|
466
|
+
|
|
467
|
+
Before returning control, copy-paste and run this ENTIRE block. Do NOT decide which commands "apply" — run all of them every time. The output confirms what happened; you do not get to assume the answer.
|
|
459
468
|
|
|
460
469
|
<bash_checklist>
|
|
461
470
|
Run this checklist verbatim before returning:
|
package/bin/install.cjs
CHANGED
|
@@ -320,8 +320,10 @@ async function main() {
|
|
|
320
320
|
//
|
|
321
321
|
// Claude Code loads slash commands from <claudeDir>/commands, sub-agents from
|
|
322
322
|
// <claudeDir>/agents, and global standards from <claudeDir>/CLAUDE.md — NOT
|
|
323
|
-
// from the qaa/ install dir.
|
|
324
|
-
//
|
|
323
|
+
// from the qaa/ install dir. Commands also reference workflows via
|
|
324
|
+
// @workflows/qa-*.md; mirroring workflows to <claudeDir>/workflows makes that
|
|
325
|
+
// resolution direct instead of a fragile runtime search. Mirror QAA files
|
|
326
|
+
// there so new versions activate. Only QAA-owned files (qa*.md) are touched.
|
|
325
327
|
|
|
326
328
|
console.log('');
|
|
327
329
|
|
|
@@ -344,6 +346,14 @@ async function main() {
|
|
|
344
346
|
success(`${tag}Live agents → ${path.join(claudeDir, 'agents')} (${agentSync.copied.length} synced${agentSync.pruned.length ? `, ${agentSync.pruned.length} pruned` : ''})`);
|
|
345
347
|
if (agentSync.pruned.length) log(`${tag}Pruned: ${agentSync.pruned.join(', ')}`);
|
|
346
348
|
|
|
349
|
+
const workflowSync = syncLive(
|
|
350
|
+
path.join(installDir, 'workflows'),
|
|
351
|
+
path.join(claudeDir, 'workflows'),
|
|
352
|
+
qaMatch, dryRun
|
|
353
|
+
);
|
|
354
|
+
success(`${tag}Live workflows → ${path.join(claudeDir, 'workflows')} (${workflowSync.copied.length} synced${workflowSync.pruned.length ? `, ${workflowSync.pruned.length} pruned` : ''})`);
|
|
355
|
+
if (workflowSync.pruned.length) log(`${tag}Pruned: ${workflowSync.pruned.join(', ')}`);
|
|
356
|
+
|
|
347
357
|
const cmd = syncClaudeMd(installDir, claudeDir, dryRun, pkgVersion);
|
|
348
358
|
if (cmd.action === 'created') success(`${tag}CLAUDE.md created (QAA markers) → ${path.join(claudeDir, 'CLAUDE.md')}`);
|
|
349
359
|
else if (cmd.action === 'updated-section') success(`${tag}CLAUDE.md QAA section updated → ${path.join(claudeDir, 'CLAUDE.md')}`);
|
|
@@ -359,7 +369,7 @@ async function main() {
|
|
|
359
369
|
console.log('');
|
|
360
370
|
if (dryRun) log('Dry-run complete — nothing was written.');
|
|
361
371
|
log(`${totalFiles} files installed to ${installDir}`);
|
|
362
|
-
log(`Live commands &
|
|
372
|
+
log(`Live commands, agents & workflows synced to ${claudeDir}`);
|
|
363
373
|
log('MCP servers: playwright, context7');
|
|
364
374
|
log('');
|
|
365
375
|
log('Restart Claude Code, then run any QAA command:');
|
|
@@ -175,15 +175,17 @@ function detectFramework(text) {
|
|
|
175
175
|
}
|
|
176
176
|
|
|
177
177
|
// Ticket references in FREE TEXT (not URLs): "ticket #PROJ-123", "issue 42",
|
|
178
|
-
// "#v195-fixes". Keyword-anchored matches are most reliable; a
|
|
179
|
-
// "#id" is accepted
|
|
180
|
-
//
|
|
178
|
+
// "#v195-fixes", "#PROJ-abc-fix". Keyword-anchored matches are most reliable; a
|
|
179
|
+
// standalone "#id" is accepted when it contains a digit OR looks like a
|
|
180
|
+
// project key (UPPERCASE prefix + hyphen, e.g. PROJ-abc-fix) — this avoids
|
|
181
|
+
// matching plain words like "#fixes". Returns the bare id (leading "#"
|
|
182
|
+
// stripped) for branch naming.
|
|
181
183
|
function detectTicketRefs(text) {
|
|
182
184
|
const refs = [];
|
|
183
185
|
const kw = text.match(/\b(?:ticket|issue|story|bug)\s+#?([A-Za-z0-9][\w.\-]*)/i);
|
|
184
186
|
if (kw) refs.push(kw[1]);
|
|
185
187
|
for (const m of text.matchAll(/(?:^|\s)#([A-Za-z0-9][\w.\-]*)/g)) {
|
|
186
|
-
if (/\d/.test(m[1]) && !refs.includes(m[1])) refs.push(m[1]);
|
|
188
|
+
if ((/\d/.test(m[1]) || /^[A-Z][A-Za-z0-9]*-/.test(m[1])) && !refs.includes(m[1])) refs.push(m[1]);
|
|
187
189
|
}
|
|
188
190
|
return refs;
|
|
189
191
|
}
|
package/commands/qa-audit.md
CHANGED
|
@@ -3,43 +3,54 @@
|
|
|
3
3
|
Comprehensive quality audit of a test suite with 6-dimension scoring, testing pyramid analysis, and status reporting. Supports three output modes: full audit, pyramid analysis only, or status report adapted to audience.
|
|
4
4
|
|
|
5
5
|
|
|
6
|
-
## ⚠ MANDATORY:
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
1. Determine `ALL_FROM_FLAGS`:
|
|
6
|
+
## ⚠ MANDATORY: How to Execute This Command
|
|
7
|
+
|
|
8
|
+
Execute these steps IN ORDER. Every step is mandatory. Do NOT improvise,
|
|
9
|
+
reorder, skip, or invent steps.
|
|
10
|
+
|
|
11
|
+
1. FIRST tool call: run the intent detector with `$ARGUMENTS` passed LITERALLY
|
|
12
|
+
(the actual command lives in the "## Intent Detection" section below).
|
|
13
|
+
2. Print the INPUT DETECTION banner (exact output from intent-detector.cjs). Do
|
|
14
|
+
NOT show a custom-format banner.
|
|
15
|
+
3. Determine `ALL_FROM_FLAGS`. If interactive (`IS_AUTO=false`) AND any value
|
|
16
|
+
came from NL/default (`ALL_FROM_FLAGS=false`), PAUSE for yes/no confirmation.
|
|
17
|
+
Do NOT proceed without it.
|
|
20
18
|
```bash
|
|
21
19
|
ALL_FROM_FLAGS=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); const sources=Object.values(j.resolved).map(v=>v.source); console.log(sources.length>0 && sources.every(s=>s==='flag') ? 'true' : 'false')")
|
|
22
20
|
```
|
|
23
|
-
|
|
24
|
-
2. If **`IS_AUTO=false` AND `ALL_FROM_FLAGS=false`** (some values came from NL or defaults), **PAUSE and ask the user**:
|
|
21
|
+
Pause prompt (when applicable):
|
|
25
22
|
```
|
|
26
23
|
The values above were resolved from natural language and/or defaults.
|
|
27
24
|
Do you want to continue with these inputs? (yes/no)
|
|
28
25
|
```
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
26
|
+
4. Determine MODE from the resolved values (not from your own NL parsing):
|
|
27
|
+
`--pyramid` → PYRAMID MODE, `--report` → REPORT MODE, otherwise AUDIT MODE.
|
|
28
|
+
5. Read `CLAUDE.md` and `~/.claude/qaa/MY_PREFERENCES.md` (if it exists) before
|
|
29
|
+
invoking any agent.
|
|
30
|
+
6. Execute the mode's flow IN ORDER (see "## Instructions"). Every mode invokes
|
|
31
|
+
its sub-agent via Task() with the `<critical_directive>` block — AUDIT MODE
|
|
32
|
+
spawns the validator; PYRAMID and REPORT MODE spawn the analyzer. Do NOT run
|
|
33
|
+
the audit/analysis inline; always go through the sub-agent. Present results
|
|
34
|
+
only after the agent returns.
|
|
32
35
|
|
|
33
36
|
### DO NOT
|
|
37
|
+
- Skip the INPUT DETECTION banner, or show a custom-format banner
|
|
38
|
+
- Run AskUserQuestion, Bash, Read, or Glob before the intent detector
|
|
39
|
+
- Decide the mode before the detector runs, or from your own NL parsing
|
|
40
|
+
- Run the audit/pyramid/report analysis inline instead of spawning the sub-agent
|
|
41
|
+
- Invent steps not in this command, or reorder the ones that are
|
|
42
|
+
- Assume path, app URL, or any value from a previous command invocation — each
|
|
43
|
+
/qa-audit call is self-contained
|
|
34
44
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
- Decide mode (from-aut, from-ticket, etc.) before the detector runs
|
|
38
|
-
- Make tool calls (Bash, Read, Glob) before the detector runs
|
|
45
|
+
If you find yourself improvising, presenting results before the agent returns,
|
|
46
|
+
or asking questions out of order — STOP and restart from step 1.
|
|
39
47
|
|
|
40
|
-
###
|
|
48
|
+
### Pass user input literally to the intent detector
|
|
41
49
|
|
|
42
|
-
|
|
50
|
+
When you run the intent detector with `$ARGUMENTS`, substitute that placeholder
|
|
51
|
+
with the RAW user input as received, NOT a pre-translated flag string. If you
|
|
52
|
+
pre-translate, the detector reports every value as `source: flag`, sets
|
|
53
|
+
`ALL_FROM_FLAGS=true`, and bypasses the confirmation checkpoint.
|
|
43
54
|
|
|
44
55
|
|
|
45
56
|
## Usage
|
|
@@ -81,7 +92,7 @@ else:
|
|
|
81
92
|
Before processing arguments, run the intent detector to support natural-language input alongside flags. Flags always win over NL.
|
|
82
93
|
|
|
83
94
|
```bash
|
|
84
|
-
RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"app_url":"app-url"}')
|
|
95
|
+
RESOLVED=$(node ~/.claude/qaa/bin/lib/intent-detector.cjs --resolve "$ARGUMENTS" --aliases '{"app_url":"app-url","dev_repo":"dev-repo"}')
|
|
85
96
|
|
|
86
97
|
# Print INPUT DETECTION banner (always shown, even if defaults are used)
|
|
87
98
|
echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileSync(0,'utf8')); console.log(j.banner)"
|
|
@@ -92,6 +103,43 @@ APP_URL=$(echo "$RESOLVED" | node -e "const j=JSON.parse(require('fs').readFileS
|
|
|
92
103
|
|
|
93
104
|
If you see values you didn't intend in the banner above (interactive mode), you can press Ctrl-C and re-run with explicit flags.
|
|
94
105
|
|
|
106
|
+
## Pre-flight checks
|
|
107
|
+
|
|
108
|
+
Run these BEFORE the main flow. A **[STOP]** check that fails: print the error EXACTLY as shown (fill in the `{...}` placeholders) and STOP. A **[WARN]** check that fails: print the notice and continue in degraded mode.
|
|
109
|
+
|
|
110
|
+
### Dev-repo path does not exist — --dev-repo given [STOP]
|
|
111
|
+
If `--dev-repo` was passed (coverage cross-reference) but the path does not exist:
|
|
112
|
+
```
|
|
113
|
+
❌ Dev-repo path not found: {path}
|
|
114
|
+
|
|
115
|
+
You passed --dev-repo for coverage cross-reference, but that path does not exist
|
|
116
|
+
(or is not a directory).
|
|
117
|
+
|
|
118
|
+
Options:
|
|
119
|
+
1. Re-run with a correct path: /qa-audit <tests> --dev-repo <path>
|
|
120
|
+
2. Drop --dev-repo to audit the test suite without coverage cross-reference
|
|
121
|
+
3. Check for a typo or an unexpanded ~ in the path
|
|
122
|
+
|
|
123
|
+
Try: ls {path}
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### Playwright MCP not connected — --app-url given [WARN]
|
|
127
|
+
If `--app-url` was passed (live locator verification) but Playwright MCP is unavailable:
|
|
128
|
+
```
|
|
129
|
+
❌ Playwright MCP is not connected — live locator verification will be skipped.
|
|
130
|
+
|
|
131
|
+
You passed --app-url so /qa-audit can check test locators against the real DOM
|
|
132
|
+
via Playwright MCP. It isn't available, so that check can't run — the rest of
|
|
133
|
+
the audit still proceeds.
|
|
134
|
+
|
|
135
|
+
Options:
|
|
136
|
+
1. Connect Playwright MCP (ships with QAA — check ~/.claude.json), then re-run for full verification
|
|
137
|
+
2. Reinstall to re-register the MCP servers: npx qaa-agent
|
|
138
|
+
3. Continue now with static scoring only (no live DOM verification)
|
|
139
|
+
|
|
140
|
+
Try: /mcp
|
|
141
|
+
```
|
|
142
|
+
|
|
95
143
|
## Instructions
|
|
96
144
|
|
|
97
145
|
### AUDIT MODE (default)
|
|
@@ -103,6 +151,7 @@ Scores across 6 dimensions: Locator Quality (20%), Assertion Specificity (20%),
|
|
|
103
151
|
2. Invoke validator agent in audit mode:
|
|
104
152
|
|
|
105
153
|
Task(
|
|
154
|
+
subagent_type="qaa-validator",
|
|
106
155
|
prompt="
|
|
107
156
|
<critical_directive priority="MANDATORY">
|
|
108
157
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -156,6 +205,7 @@ Analyze test distribution against the ideal testing pyramid from CLAUDE.md (Unit
|
|
|
156
205
|
2. Invoke analyzer agent for pyramid analysis:
|
|
157
206
|
|
|
158
207
|
Task(
|
|
208
|
+
subagent_type="qaa-analyzer",
|
|
159
209
|
prompt="
|
|
160
210
|
<critical_directive priority="MANDATORY">
|
|
161
211
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
@@ -212,6 +262,7 @@ Generate a summary report of current QA status. Adapts detail level to audience.
|
|
|
212
262
|
2. Invoke analyzer agent for status reporting:
|
|
213
263
|
|
|
214
264
|
Task(
|
|
265
|
+
subagent_type="qaa-analyzer",
|
|
215
266
|
prompt="
|
|
216
267
|
<critical_directive priority="MANDATORY">
|
|
217
268
|
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|