qaa-agent 1.9.1 → 1.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +43 -22
- package/CLAUDE.md +170 -9
- package/README.md +384 -357
- package/VERSION +1 -0
- package/agents/qa-pipeline-orchestrator.md +336 -22
- package/agents/qaa-analyzer.md +0 -1
- package/agents/qaa-bug-detective.md +163 -4
- package/agents/qaa-codebase-mapper.md +50 -1
- package/agents/qaa-discovery.md +421 -384
- package/agents/qaa-e2e-runner.md +163 -1
- package/agents/qaa-executor.md +142 -1
- package/agents/qaa-planner.md +14 -1
- package/agents/qaa-project-researcher.md +194 -0
- package/agents/qaa-scanner.md +77 -1
- package/agents/qaa-testid-injector.md +0 -1
- package/agents/qaa-validator.md +86 -1
- package/bin/install.cjs +375 -253
- package/bin/lib/context7-cache.cjs +299 -0
- package/bin/lib/intent-detector.cjs +488 -0
- package/commands/qa-audit.md +255 -126
- package/commands/qa-create-test.md +666 -365
- package/commands/qa-fix.md +684 -513
- package/commands/qa-map.md +283 -139
- package/commands/qa-pr.md +63 -0
- package/commands/qa-research.md +181 -157
- package/commands/qa-start.md +62 -6
- package/commands/qa-test-report.md +219 -219
- package/frameworks/cypress.json +54 -0
- package/frameworks/jest.json +59 -0
- package/frameworks/playwright.json +58 -0
- package/frameworks/pytest.json +59 -0
- package/frameworks/robot-framework.json +54 -0
- package/frameworks/selenium.json +65 -0
- package/frameworks/vitest.json +57 -0
- package/package.json +7 -3
- package/workflows/qa-analyze.md +100 -4
- package/workflows/qa-from-ticket.md +50 -2
- package/workflows/qa-gap.md +50 -2
- package/workflows/qa-start.md +819 -33
- package/workflows/qa-testid.md +50 -2
- package/workflows/qa-validate.md +50 -2
package/CHANGELOG.md
CHANGED
|
@@ -3,6 +3,48 @@
|
|
|
3
3
|
|
|
4
4
|
All notable changes to QAA (QA Automation Agent) are documented here.
|
|
5
5
|
|
|
6
|
+
## [1.9.5] - 2026-05-05
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
|
|
10
|
+
- **Framework Registry** (`frameworks/` directory) — folder-based metadata system with one JSON file per supported framework (Playwright, Cypress, Jest, Vitest, pytest, Selenium, Robot Framework). Each entry declares stable metadata: extensions, detection patterns, validation commands, conventions, Context7 keys. Scanner uses the registry for detection; executor and validator use the detected framework's entry for conventions and validation commands. Code idioms still come from Context7 at runtime — the registry tells the system "what to query in Context7", Context7 returns "how to write the code". Adding a new framework = adding a JSON file (no agent prompt changes).
|
|
11
|
+
- **Intent Detector** (`bin/lib/intent-detector.cjs`) — natural-language argument parser shared by 5 commands (qa-start, qa-create-test, qa-fix, qa-audit, qa-pr). Extracts URLs, paths (with spaces, quoted, Windows/Unix), framework hints, and ticket URLs from any input. Hierarchy: flags > NL > defaults. Robust path detection handles English-word trimming for greedy-matched paths.
|
|
12
|
+
- **INPUT DETECTION banner** — shown before every pipeline run on every command. Displays each resolved value with its source (flag, NL, or default). Always visible — with flags, with NL, or with defaults only. Prevents "wrong directory disasters" by showing assumptions before tokens are spent.
|
|
13
|
+
- **Confirmation checkpoint after intent detection** (interactive mode) — when any value comes from NL or defaults (not all explicit flags), the pipeline pauses and asks the user to confirm the resolved inputs before invoking sub-agents. Skipped when `--auto` is set or when every value has `source: flag`. Prevents pipeline runs based on misinferred NL.
|
|
14
|
+
- **Input validations (5 pre-pipeline checks)** — Capa 1 of 3-layer validation system. Validates: URL reachable, path writable, URL ambiguity with ticket platforms, no multiple AUT URLs, framework in registry. Capa 2 is each agent's `<quality_gate>`; Capa 3 is the bash checklist.
|
|
15
|
+
- **Flag `--app-url` in `/qa-start`** — communicates "app running at this URL" to the pipeline. Propagated to e2e-runner, validator Layer 5, testid-injector verification, bug-detective reproduction. Aligns `/qa-start` with `/qa-create-test`.
|
|
16
|
+
- **Mode `from-aut` in `/qa-create-test`** — single-shot URL → tests. Detector extracts non-ticket URLs and routes them to a lightweight flow that navigates the URL with Playwright MCP and generates tests directly. Bonus: fixes existing bug where non-ticket URLs incorrectly fell to `from-code` mode.
|
|
17
|
+
- **Context7 response cache** (`bin/lib/context7-cache.cjs`) — version-specific cache at `.qa-output/frameworks/FRAMEWORK-VERSION.json` storing raw Context7 responses. Default TTL 30 days. Auto-invalidates when project version changes (filename includes version). CLI: `read`, `write`, `status`, `clear`, `init` operations.
|
|
18
|
+
- **Verification Cascade in Context7-using agents** — researcher, executor, e2e-runner, bug-detective now follow `cache → research docs → query Context7 → WebFetch fallback`. Verification remains mandatory; the cascade just prioritizes cheaper local sources first. Reduces Context7 free-tier pressure (60 req/h limit).
|
|
19
|
+
|
|
20
|
+
### Fixed
|
|
21
|
+
|
|
22
|
+
- **Deployment gap — installer never synced to Claude Code's live locations (finding #52)** — `install.cjs` copied everything to `~/.claude/qaa/` (or `./.claude/qaa/`), but Claude Code loads slash commands from `<claudeDir>/commands/`, sub-agents from `<claudeDir>/agents/`, and global standards from `<claudeDir>/CLAUDE.md`. New versions never activated — even after reinstalling — because the live locations kept the old files. Added **Step 4: Sync to live locations** — scope-aware (global → `~/.claude`, local → `./.claude`), mirrors only QAA-owned files (`qa*.md` / `qaa-*.md`) into `commands/` and `agents/`, prunes QAA orphans that no longer ship (never touching foreign files like `gsd-*`, `sc:*`), and merges `CLAUDE.md` via `<!-- QAA-BEGIN --> / <!-- QAA-END -->` markers. **First install with this fix requires reinstall + restart of Claude Code.** If a global `CLAUDE.md` existed without QAA markers, it is backed up to `CLAUDE.md.bak-<timestamp>` before being replaced with a marked version. Also added a `--dry-run` flag (prints planned copies/prunes without writing) and a guard for missing `HOME`/`USERPROFILE`.
|
|
23
|
+
- **`qaa-codebase-mapper`, `qaa-discovery`, `qaa-project-researcher` were missing both quality_gate and bash checklist** — added both to all 3 agents. `qaa-e2e-runner` was missing only quality_gate (added). `qaa-scanner` was missing only bash checklist (added). All 12 agents now have both verification mechanisms in place.
|
|
24
|
+
- **Framework hardcode dispersed across 20+ files** — agents now consult the Framework Registry as the source of truth for detection, validation commands, and conventions. Existing hardcoded examples remain as fallback/illustration. New `<framework_registry>` section in qaa-scanner, qaa-executor, qaa-validator explains the lookup order.
|
|
25
|
+
- **`required_reading` of agents and `<files_to_read>` of workflow updated** — scanner reads all `frameworks/*.json` for detection; executor and validator read only the framework JSON for the detected framework. Module Boundaries and "MUST Read Before Producing Output" tables in CLAUDE.md updated accordingly.
|
|
26
|
+
|
|
27
|
+
### Documentation
|
|
28
|
+
|
|
29
|
+
- **CLAUDE.md** — added "Framework Registry" section (~70 lines) documenting registry structure, consumers, fallback behavior, and Context7 cache. Added "Intent Detection" section (~30 lines) explaining hierarchy, banner, and validations.
|
|
30
|
+
- **README.md** — Robot Framework added to supported frameworks list. Architecture diagram updated with `frameworks/` and `bin/lib/`. New examples in Quick Start: `--app-url`, `from-aut` mode, natural language input.
|
|
31
|
+
- **QAA_SYSTEM_OVERVIEW.md** — new sections: Framework Registry, Intent Detection, Context7 Cache. Version updated to 1.9.5.
|
|
32
|
+
|
|
33
|
+
## [1.9.2] - 2026-05-05
|
|
34
|
+
|
|
35
|
+
### Fixed
|
|
36
|
+
|
|
37
|
+
- **`qaa-codebase-mapper` was missing from `/qa-start` pipeline since v1.6.3** — the codebase-mapper agent existed and was documented as part of the pipeline (CLAUDE.md, README.md, qa-pipeline-orchestrator.md), but the `workflows/qa-start.md` file never invoked it as a step. Downstream agents (analyzer, planner, executor) had `(if exists)` references to the 8 codebase docs that always failed because no agent produced them. The fix adds a new `<step name="codebase_map">` between scan and research that spawns 4 parallel sub-agents (testability, risk, patterns, existing-tests) producing the 8 documents in `.qa-output/codebase/`. Pipeline order now matches documentation: `scan → codebase-map → research → analyze → ...`.
|
|
38
|
+
- **Pipeline order corrected: `research` now runs AFTER `codebase-map`** — research stage was previously placed before codebase-map (added in v1.9.0 as STAGE 1b). The correct logical order is `scan → codebase-map → research` so the researcher can use codebase findings (RISK_MAP, CODE_PATTERNS, API_CONTRACTS) to make Context7 queries specific to the project's actual stack instead of generic framework questions. Research stage renamed to STAGE 1c.
|
|
39
|
+
- **Downstream agents now receive the appropriate codebase docs in their `<files_to_read>`** — analyzer reads RISK_MAP, CRITICAL_PATHS, TEST_ASSESSMENT, COVERAGE_GAPS; planner reads TESTABILITY, TEST_SURFACE, CRITICAL_PATHS, COVERAGE_GAPS; executor reads TEST_SURFACE, CODE_PATTERNS, API_CONTRACTS. Mapping follows the codebase-mapper's own `why_this_matters` contract.
|
|
40
|
+
- **`qaa-executor`, `qaa-e2e-runner`, and `qaa-bug-detective` now declare Context7 tools explicitly in frontmatter** — these agents had prompts saying "MUST query Context7" but did not declare the `mcp__context7__resolve-library-id` and `mcp__context7__query-docs` tools in their YAML frontmatter, relying on tool inheritance from the parent. Now declared explicitly along with Playwright MCP tools where applicable. Improves portability and respects the principle of least privilege.
|
|
41
|
+
- **`VERSION` file now distributed in npm package** — added `"VERSION"` to the `files` array in `package.json` so the `VERSION` file is included in published packages. Previously, end users who ran `npx qaa-agent` never received an updated `VERSION` file because it was excluded from the npm package whitelist, leaving stale version numbers that did not match `package.json`.
|
|
42
|
+
|
|
43
|
+
### Added
|
|
44
|
+
|
|
45
|
+
- **Version-aware `libraryId` in Context7 queries** — when the project's framework version is detected from `package.json`/`requirements.txt`/lock files, agents now use a versioned library ID format (`/org/project/v1.40.0`) in `query-docs` calls. Context7 returns documentation specific to that version, not the latest. Generated code matches the framework version the project actually uses, avoiding APIs that don't exist or have changed. Applied to `qaa-project-researcher`, `qaa-executor`, `qaa-e2e-runner`, and `qaa-bug-detective`. Falls back to base library ID if version cannot be detected.
|
|
46
|
+
- **`<codebase_grounding>` section in `qaa-project-researcher`** — explicit instructions for the researcher to use codebase map findings (RISK_MAP, CODE_PATTERNS, API_CONTRACTS, etc.) to ground Context7 queries to project specifics (e.g., "MSW integration with Playwright" instead of "Playwright mocking capabilities"). Includes a comparison table of generic vs grounded queries.
|
|
47
|
+
|
|
6
48
|
## [1.9.1] - 2026-04-27
|
|
7
49
|
|
|
8
50
|
### Added
|
|
@@ -162,25 +204,4 @@ All notable changes to QAA (QA Automation Agent) are documented here.
|
|
|
162
204
|
## [1.2.0]
|
|
163
205
|
|
|
164
206
|
### Added
|
|
165
|
-
- `qaa-codebase-mapper` agent -- 4 parallel
|
|
166
|
-
- `qaa-project-researcher` agent -- researches best testing stack and practices
|
|
167
|
-
- 8 codebase map documents produced by mapper
|
|
168
|
-
|
|
169
|
-
## [1.1.0]
|
|
170
|
-
|
|
171
|
-
### Added
|
|
172
|
-
- Workflow definitions for all pipeline stages
|
|
173
|
-
- Interactive installer (`npx qaa-agent`)
|
|
174
|
-
- `qaa init` command for per-project initialization
|
|
175
|
-
- npm package distribution
|
|
176
|
-
|
|
177
|
-
## [1.0.0]
|
|
178
|
-
|
|
179
|
-
### Added
|
|
180
|
-
- Full QA automation pipeline -- 11 agents, 17 commands, 10 templates, 7 workflows
|
|
181
|
-
- 3 workflow options (dev-only, immature QA, mature QA)
|
|
182
|
-
- 4-layer test validation (syntax, structure, dependencies, logic)
|
|
183
|
-
- Page Object Model generation with CLAUDE.md standards
|
|
184
|
-
- Test ID injection for frontend components
|
|
185
|
-
- Bug detective failure classification
|
|
186
|
-
- Draft PR delivery with branch naming convention
|
|
207
|
+
- `qaa-codebase-mapper` agent -- 4 parallel fo
|
package/CLAUDE.md
CHANGED
|
@@ -197,7 +197,7 @@ The QA automation system runs agents in a defined pipeline. Each stage produces
|
|
|
197
197
|
### Pipeline Stages
|
|
198
198
|
|
|
199
199
|
```
|
|
200
|
-
scan ->
|
|
200
|
+
scan -> codebase-map -> research -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
|
|
201
201
|
```
|
|
202
202
|
|
|
203
203
|
### Workflow Options
|
|
@@ -205,7 +205,7 @@ scan -> research -> codebase-map -> analyze -> [testid-inject if frontend] -> pl
|
|
|
205
205
|
**Option 1: Dev-Only Repo (no existing QA repo)**
|
|
206
206
|
Full pipeline from scratch:
|
|
207
207
|
```
|
|
208
|
-
scan ->
|
|
208
|
+
scan -> codebase-map -> research -> analyze -> [testid-inject if frontend] -> plan -> generate -> validate -> [e2e-runner if E2E tests] -> [bug-detective if failures] -> deliver
|
|
209
209
|
```
|
|
210
210
|
Produces: SCAN_MANIFEST.md -> 8 codebase map documents -> QA_ANALYSIS.md + TEST_INVENTORY.md + QA_REPO_BLUEPRINT.md -> [TESTID_AUDIT_REPORT.md] -> generation plan -> test files + POMs + fixtures + configs -> VALIDATION_REPORT.md -> [E2E_RUN_REPORT.md] -> branch + PR
|
|
211
211
|
|
|
@@ -249,12 +249,12 @@ Each agent owns specific artifacts. No agent may produce artifacts assigned to a
|
|
|
249
249
|
|
|
250
250
|
| Agent | Reads | Produces | Template |
|
|
251
251
|
|-------|-------|----------|----------|
|
|
252
|
-
| qa-scanner | repo source files, package.json, file tree | SCAN_MANIFEST.md | templates/scan-manifest.md |
|
|
252
|
+
| qa-scanner | repo source files, package.json, file tree, frameworks/*.json (registry) | SCAN_MANIFEST.md | templates/scan-manifest.md |
|
|
253
253
|
| qa-project-researcher | SCAN_MANIFEST.md, repo source files, Context7 MCP, WebSearch, WebFetch | TESTING_STACK.md, FRAMEWORK_CAPABILITIES.md, API_TESTING_STRATEGY.md, E2E_STRATEGY.md | -- |
|
|
254
254
|
| qa-codebase-mapper | SCAN_MANIFEST.md, repo source files, CLAUDE.md | TESTABILITY.md, TEST_SURFACE.md, RISK_MAP.md, CRITICAL_PATHS.md, CODE_PATTERNS.md, API_CONTRACTS.md, TEST_ASSESSMENT.md, COVERAGE_GAPS.md | -- |
|
|
255
255
|
| qa-analyzer | SCAN_MANIFEST.md, codebase map documents, CLAUDE.md | QA_ANALYSIS.md, TEST_INVENTORY.md, QA_REPO_BLUEPRINT.md (Option 1) or GAP_ANALYSIS.md (Option 2/3) | templates/qa-analysis.md, templates/test-inventory.md, templates/qa-repo-blueprint.md, templates/gap-analysis.md |
|
|
256
256
|
| qa-planner | TEST_INVENTORY.md, QA_ANALYSIS.md, codebase map documents | Generation plan (internal) | -- |
|
|
257
|
-
| qa-executor | TEST_INVENTORY.md, codebase map documents, CLAUDE.md | test files, POMs, fixtures, configs | qa-template-engine patterns |
|
|
257
|
+
| qa-executor | TEST_INVENTORY.md, codebase map documents, CLAUDE.md, frameworks/{detected_framework}.json (registry) | test files, POMs, fixtures, configs | qa-template-engine patterns |
|
|
258
258
|
| qa-validator | generated test files, CLAUDE.md | VALIDATION_REPORT.md (validation mode) or QA_AUDIT_REPORT.md (audit mode) | templates/validation-report.md, templates/qa-audit-report.md |
|
|
259
259
|
| qa-e2e-runner | generated E2E test files, POM files, CLAUDE.md, live application | E2E_RUN_REPORT.md, fixed test/POM files with real locators | -- |
|
|
260
260
|
| qa-testid-injector | repo source files, SCAN_MANIFEST.md, CLAUDE.md | TESTID_AUDIT_REPORT.md, modified source files with data-testid attributes | templates/scan-manifest.md, templates/testid-audit-report.md |
|
|
@@ -399,8 +399,8 @@ Each agent operates on the same branch. No worktree splits are needed for this s
|
|
|
399
399
|
|
|
400
400
|
Respect stage transitions from the Agent Pipeline section:
|
|
401
401
|
1. qa-scanner runs first (no dependencies)
|
|
402
|
-
2. qa-project-researcher runs after
|
|
403
|
-
3. qa-codebase-mapper runs after
|
|
402
|
+
2. qa-project-researcher runs after codebase-map (depends on SCAN_MANIFEST.md, uses Context7 MCP — non-blocking, pipeline continues if it fails)
|
|
403
|
+
3. qa-codebase-mapper runs after scanner (depends on SCAN_MANIFEST.md, spawns 4 parallel focus agents)
|
|
404
404
|
4. qa-analyzer and qa-testid-injector run after codebase-map (both depend on SCAN_MANIFEST.md + codebase map documents + research documents if available)
|
|
405
405
|
5. qa-planner runs after analyzer (depends on QA_ANALYSIS.md + TEST_INVENTORY.md + codebase map documents)
|
|
406
406
|
6. qa-executor runs after planner (depends on generation plan + codebase map documents + research documents if available + Context7 MCP)
|
|
@@ -429,12 +429,12 @@ Every agent MUST read its required inputs before producing any output. Failure t
|
|
|
429
429
|
|
|
430
430
|
| Agent | MUST Read Before Producing Output |
|
|
431
431
|
|-------|-----------------------------------|
|
|
432
|
-
| qa-scanner | package.json (or equivalent), folder tree structure, all source file extensions
|
|
432
|
+
| qa-scanner | package.json (or equivalent), folder tree structure, all source file extensions, frameworks/*.json (registry) for detection patterns |
|
|
433
433
|
| qa-project-researcher | SCAN_MANIFEST.md, repo source files, MY_PREFERENCES.md. Uses Context7 MCP as primary source. |
|
|
434
434
|
| qa-analyzer | SCAN_MANIFEST.md (complete, verified), CLAUDE.md (all QA standards sections), research documents (if available) |
|
|
435
435
|
| qa-planner | TEST_INVENTORY.md (all test cases), QA_ANALYSIS.md (architecture and risk context) |
|
|
436
|
-
| qa-executor | TEST_INVENTORY.md (test cases to implement), CLAUDE.md (POM rules, locator hierarchy, assertion rules, naming conventions, quality gates), research documents (if available). Must query Context7 MCP before generating code. |
|
|
437
|
-
| qa-validator | CLAUDE.md (quality gates, locator tiers, assertion rules), all generated test files to validate |
|
|
436
|
+
| qa-executor | TEST_INVENTORY.md (test cases to implement), CLAUDE.md (POM rules, locator hierarchy, assertion rules, naming conventions, quality gates), research documents (if available), frameworks/{detected_framework}.json (registry: conventions, extensions, context7_keys). Must query Context7 MCP before generating code. |
|
|
437
|
+
| qa-validator | CLAUDE.md (quality gates, locator tiers, assertion rules), all generated test files to validate, frameworks/{detected_framework}.json (registry: validation_commands) |
|
|
438
438
|
| qa-testid-injector | SCAN_MANIFEST.md (component file list), CLAUDE.md (data-testid Convention section for naming rules) |
|
|
439
439
|
| qa-bug-detective | Test execution output (stdout/stderr, exit codes), test source files (to read the failing code), CLAUDE.md (classification rules), research documents (if available). Must query Context7 MCP before auto-fixing. |
|
|
440
440
|
| qa-e2e-runner | Generated E2E test files, POM files, CLAUDE.md, live application, research documents (if available). Must query Context7 MCP before fixing locators. |
|
|
@@ -555,3 +555,164 @@ When adding `data-testid` to third-party UI library components, use this priorit
|
|
|
555
555
|
<TextField inputProps={{ 'data-testid': 'login-email-input' }} />
|
|
556
556
|
<Autocomplete slotProps={{ input: { 'data-testid': 'search-query-input' } }} />
|
|
557
557
|
```
|
|
558
|
+
---
|
|
559
|
+
|
|
560
|
+
## Framework Registry
|
|
561
|
+
|
|
562
|
+
The Framework Registry is a folder-based metadata system at `frameworks/` containing one JSON file per supported framework. Each file declares stable metadata that the system needs to recognize and operate the framework — extensions, detection patterns, validation commands, conventions, Context7 keys.
|
|
563
|
+
|
|
564
|
+
### What lives in `frameworks/<name>.json`
|
|
565
|
+
|
|
566
|
+
```json
|
|
567
|
+
{
|
|
568
|
+
"name": "playwright",
|
|
569
|
+
"display_name": "Playwright",
|
|
570
|
+
"languages": ["typescript", "javascript"],
|
|
571
|
+
"extensions": [".spec.ts", ".spec.js"],
|
|
572
|
+
"detection": {
|
|
573
|
+
"manifest_files": ["package.json"],
|
|
574
|
+
"manifest_grep": "@playwright/test",
|
|
575
|
+
"config_files": ["playwright.config.ts"],
|
|
576
|
+
"file_patterns": ["tests/**/*.spec.ts"]
|
|
577
|
+
},
|
|
578
|
+
"validation_commands": {
|
|
579
|
+
"syntax_check_ts": "tsc --noEmit",
|
|
580
|
+
"dry_run": "npx playwright test --list",
|
|
581
|
+
"lint": "eslint {file}"
|
|
582
|
+
},
|
|
583
|
+
"conventions": {
|
|
584
|
+
"test_dirs": ["tests/", "e2e/"],
|
|
585
|
+
"pom_dirs": ["pages/"],
|
|
586
|
+
"fixture_dirs": ["fixtures/"],
|
|
587
|
+
"config_filename": "playwright.config.ts"
|
|
588
|
+
},
|
|
589
|
+
"context7_keys": ["playwright", "@playwright/test"]
|
|
590
|
+
}
|
|
591
|
+
```
|
|
592
|
+
|
|
593
|
+
### What does NOT live in the registry
|
|
594
|
+
|
|
595
|
+
The registry stores **stable metadata**, NOT **code idioms**. Code idioms (imports, syntax, API usage, pattern examples) come from **Context7** at runtime — they change with framework releases and Context7 always has the latest. The registry tells the system "what to query in Context7" via `context7_keys`; Context7 tells the system "how to write the code".
|
|
596
|
+
|
|
597
|
+
### Who consumes the registry
|
|
598
|
+
|
|
599
|
+
| Agent | What it reads | How |
|
|
600
|
+
|-------|---------------|-----|
|
|
601
|
+
| qa-scanner | All `frameworks/*.json` | Iterate for detection (manifest_grep, config_files, file_patterns) |
|
|
602
|
+
| qa-executor | `frameworks/<detected>.json` | Use `extensions`, `conventions`, `context7_keys` |
|
|
603
|
+
| qa-validator | `frameworks/<detected>.json` | Use `validation_commands` for Layer 1 (Syntax) and Layer 2 (Structure) |
|
|
604
|
+
|
|
605
|
+
Other agents (analyzer, planner, testid-injector, bug-detective, e2e-runner) get the framework name from the scanner's output (SCAN_MANIFEST.md) and don't need to read the registry directly.
|
|
606
|
+
|
|
607
|
+
### Adding a new framework
|
|
608
|
+
|
|
609
|
+
1. Create `frameworks/<new-framework>.json` with the schema above
|
|
610
|
+
2. Add `"VERSION"` and `"frameworks/"` are already in `package.json` `files` array (no change needed)
|
|
611
|
+
3. Test detection by running scanner against a project that uses the framework
|
|
612
|
+
4. Test code generation: executor will read `context7_keys` and query Context7 for syntax
|
|
613
|
+
|
|
614
|
+
### Fallback when framework not in registry
|
|
615
|
+
|
|
616
|
+
If a project uses a framework not in `frameworks/`, the system:
|
|
617
|
+
1. Logs a WARNING: "Framework `<name>` not in registry — relying on Context7 only"
|
|
618
|
+
2. Asks Context7 for everything (slower, more queries, but still works)
|
|
619
|
+
3. The pipeline does NOT halt unless the user requests `--strict-registry`
|
|
620
|
+
|
|
621
|
+
The registry is an **optimization**, not a gate. Common frameworks are pre-registered for speed; rare frameworks fall back to Context7.
|
|
622
|
+
|
|
623
|
+
### Context7 cache (project-local)
|
|
624
|
+
|
|
625
|
+
Each project, after the first pipeline run, has a `.qa-output/frameworks/<framework>-<version>.json` file that caches Context7 query responses. This:
|
|
626
|
+
- Reduces Context7 API calls (free tier: 60/hour)
|
|
627
|
+
- Auto-invalidates when project framework version changes (filename includes version)
|
|
628
|
+
- Default TTL: 30 days
|
|
629
|
+
|
|
630
|
+
The cache is consumed by 4 agents: project-researcher (writes), executor / e2e-runner / bug-detective (read first, fall back to Context7 if cache miss). See the `<context7_verification>` → "Verification Cascade" section in those agents.
|
|
631
|
+
|
|
632
|
+
---
|
|
633
|
+
|
|
634
|
+
## Intent Detection
|
|
635
|
+
|
|
636
|
+
The Intent Detector at `bin/lib/intent-detector.cjs` parses command arguments to extract structured intent (URLs, paths, framework hints, tickets, flags) and supports both explicit flags and natural language.
|
|
637
|
+
|
|
638
|
+
### Hierarchy
|
|
639
|
+
|
|
640
|
+
```
|
|
641
|
+
Flags > Natural Language > Defaults
|
|
642
|
+
```
|
|
643
|
+
|
|
644
|
+
- **Flags always win**: `--app-url https://example.com` is authoritative
|
|
645
|
+
- **NL fills gaps**: if `--app-url` is not passed but a URL is mentioned, the detector extracts it
|
|
646
|
+
- **Defaults**: if neither flag nor NL provides a value, fall back to defaults (cwd for dev-repo, etc.)
|
|
647
|
+
|
|
648
|
+
### INPUT DETECTION banner
|
|
649
|
+
|
|
650
|
+
Every command (qa-start, qa-create-test, qa-fix, qa-audit, qa-pr) shows an **INPUT DETECTION** banner before starting work. This banner is shown ALWAYS — with flags, with NL, or with defaults — and lists every resolved value with its source:
|
|
651
|
+
|
|
652
|
+
```
|
|
653
|
+
=== INPUT DETECTION ===
|
|
654
|
+
From explicit flags:
|
|
655
|
+
--dev-repo /home/user/proj (source: flag)
|
|
656
|
+
Detected from natural language:
|
|
657
|
+
--app-url https://staging.example.com (source: NL)
|
|
658
|
+
--framework playwright (source: NL)
|
|
659
|
+
Defaults applied:
|
|
660
|
+
--auto false (source: default)
|
|
661
|
+
=======================
|
|
662
|
+
```
|
|
663
|
+
|
|
664
|
+
In interactive mode, the user can press Ctrl-C if the detection looks wrong. In `--auto` mode, the banner serves as a structured log.
|
|
665
|
+
|
|
666
|
+
### Input validations (5 pre-pipeline checks)
|
|
667
|
+
|
|
668
|
+
After intent detection, qa-start runs 5 validation checks before invoking any sub-agent:
|
|
669
|
+
|
|
670
|
+
1. **URL reachable** — `curl` ping with 5s timeout (warning, not fatal)
|
|
671
|
+
2. **Path writable** — verify dev-repo path exists and is writable, or parent is writable
|
|
672
|
+
3. **URL not ambiguous with ticket platforms** — warn if AUT URL looks like a ticket URL
|
|
673
|
+
4. **No multiple AUT URLs** — fail if more than 1 AUT URL detected (excluding tickets)
|
|
674
|
+
5. **Framework in registry** — warn if `--framework <name>` not in `frameworks/`
|
|
675
|
+
|
|
676
|
+
These are **Capa 1** of a 3-layer validation system. Capa 2 is each agent's `<quality_gate>`. Capa 3 is each agent's mandatory bash checklist.
|
|
677
|
+
|
|
678
|
+
---
|
|
679
|
+
|
|
680
|
+
## Standard Task() Directive
|
|
681
|
+
|
|
682
|
+
Every `Task()` / `Agent()` spawn in this codebase (orchestrator, commands, workflows) MUST embed the following directive **verbatim** at the very top of the spawned prompt, substituting ONLY `<AGENT_FILE>` with the target agent's exact filename (e.g. `qaa-scanner`, `qa-pipeline-orchestrator`). Do NOT paraphrase it, and do NOT add `subagent_type: "qaa-X"` — native subagent routing for QAA agents is not enabled and would hard-error (see the architectural note below).
|
|
683
|
+
|
|
684
|
+
```
|
|
685
|
+
<critical_directive priority="MANDATORY">
|
|
686
|
+
You are executing as a QAA pipeline agent. Your FIRST action MUST be to read
|
|
687
|
+
~/.claude/qaa/agents/<AGENT_FILE>.md and adopt it as your operating contract.
|
|
688
|
+
|
|
689
|
+
Even though you may be running as a general-purpose agent (native subagent type
|
|
690
|
+
routing for QAA agents is not yet enabled), you MUST behave exactly as the
|
|
691
|
+
dedicated agent defined in that file: read everything in its <required_reading>,
|
|
692
|
+
obey its <quality_gate>, and run its mandatory bash checklist before returning.
|
|
693
|
+
|
|
694
|
+
Never modify application source code (src/, app/, lib/, or equivalent
|
|
695
|
+
directories) unless the agent's <scope> section explicitly permits it. If the
|
|
696
|
+
agent's contract restricts what files it can touch, honor that restriction —
|
|
697
|
+
even if a fix seems obvious or trivial.
|
|
698
|
+
|
|
699
|
+
If you cannot read ~/.claude/qaa/agents/<AGENT_FILE>.md (file missing, corrupted, parse
|
|
700
|
+
error), STOP and report the failure. Do NOT proceed with default agent
|
|
701
|
+
behavior — that defeats the purpose of this directive.
|
|
702
|
+
|
|
703
|
+
Do NOT improvise, substitute steps, apply changes the agent isn't allowed to
|
|
704
|
+
make, or skip any of the above. Skipping the agent's required_reading,
|
|
705
|
+
quality_gate, or bash checklist makes the run INVALID — its output cannot be
|
|
706
|
+
trusted.
|
|
707
|
+
</critical_directive>
|
|
708
|
+
```
|
|
709
|
+
|
|
710
|
+
### Architectural convention: wrap the flow in MANDATORY
|
|
711
|
+
|
|
712
|
+
QAA hardens execution at two layers, and new commands/agents MUST follow both:
|
|
713
|
+
|
|
714
|
+
1. **Command headers** — every NL command opens with a `## ⚠ MANDATORY: How to Execute This Command` block defining the full ordered flow + DO / DO NOT (not just the first tool call).
|
|
715
|
+
2. **Task() prompts** — every spawn embeds the Standard Task() Directive above.
|
|
716
|
+
|
|
717
|
+
This is the standard QAA enforcement pattern. Native `subagent_type` routing is deferred (the agents are not registered in `~/.claude/agents/`); until then the embedded directive is the enforcement mechanism.
|
|
718
|
+
|