@roopesh.yadava/qa-pack 1.0.3 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +71 -25
- package/bin/postinstall.js +95 -14
- package/claude/commands/impacted-tests.md +11 -0
- package/claude/commands/qa-agent.md +5 -5
- package/claude/skills/SKILLS_CONTEXT.md +66 -32
- package/claude/skills/accessibility-testing/SKILL.md +9 -9
- package/claude/skills/automation/LOCATOR_PATTERNS.md +1 -1
- package/claude/skills/automation/SKILL.md +200 -78
- package/claude/skills/bug-reporting/SKILL.md +2 -1
- package/claude/skills/impacted-tests/SKILL.md +111 -0
- package/claude/skills/manual-testing/SKILL.md +24 -40
- package/claude/skills/qa-agent/SKILL.md +197 -240
- package/claude/skills/qa-agent/product_context/CONTEXT_SCHEMA.md +10 -13
- package/claude/skills/qa-agent/product_context/README.md +1 -1
- package/claude/skills/ui-test-figma/SKILL.md +8 -4
- package/package.json +1 -1
- package/templates/.env.example +22 -0
- package/templates/CLAUDE.md +20 -12
package/README.md
CHANGED
|
@@ -10,7 +10,7 @@ echo "@roopesh.yadava:registry=https://registry.npmjs.org" >> .npmrc
|
|
|
10
10
|
npm install --save-dev @roopesh.yadava/qa-pack
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
On install, the postinstall script copies all skills and commands into `.claude/` and creates one-time config files (`CLAUDE.md`, `.mcp.json`, `cucumber.cjs`) if they don't already exist. It also writes the `.npmrc` scope entry above automatically, so `npm update` works with no extra steps after the first install.
|
|
13
|
+
On install, the postinstall script copies all skills and commands into `.claude/` (git-ignored — see Git hygiene below) and creates one-time config files (`CLAUDE.md`, `.mcp.json`, `cucumber.cjs`, `.env`) if they don't already exist. It also writes the `.npmrc` scope entry above automatically, so `npm update` works with no extra steps after the first install.
|
|
14
14
|
|
|
15
15
|
## Update
|
|
16
16
|
|
|
@@ -18,19 +18,18 @@ On install, the postinstall script copies all skills and commands into `.claude/
|
|
|
18
18
|
npm update @roopesh.yadava/qa-pack
|
|
19
19
|
```
|
|
20
20
|
|
|
21
|
-
Skill files are overwritten with the latest version. Your `CLAUDE.md`, `.mcp.json`, and — critically — `product_context/` (your accumulated run history and known bugs) are **never touched**.
|
|
21
|
+
Skill files are overwritten with the latest version. Your `CLAUDE.md`, `.mcp.json`, `.env`, and — critically — `product_context/` (your accumulated run history and known bugs) are **never touched**. The installed `SKILLS_CONTEXT.md` is stamped with the pack version, so you can always check which version a repo is running.
|
|
22
|
+
|
|
23
|
+
**Upgrading from 1.0.x:** credentials used to live in `product_context/{PRODUCT}/context.md` — skills now read them from `.env` instead. Move your username/password/OTP into `.env` and delete them from any existing `context.md`.
|
|
22
24
|
|
|
23
25
|
## First-time setup
|
|
24
26
|
|
|
25
|
-
After installing
|
|
27
|
+
After installing:
|
|
26
28
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
OTP Required: yes (always 999999) | no
|
|
32
|
-
Username / Password: your test credentials
|
|
33
|
-
```
|
|
29
|
+
1. Open `.env` (gitignored) and fill in the secrets:
|
|
30
|
+
`QA_APP_URL`, `QA_LOGIN_URL`, `QA_USERNAME`, `QA_PASSWORD`, `QA_OTP_CODE`
|
|
31
|
+
(plus `JIRA_BASE_URL` / `JIRA_EMAIL` / `JIRA_API_TOKEN` if you want bug screenshots attached).
|
|
32
|
+
2. Open `CLAUDE.md` and fill in the non-secret project facts (Jira key, environment, auth method).
|
|
34
33
|
|
|
35
34
|
Then open the repo in Claude Code and run:
|
|
36
35
|
|
|
@@ -38,38 +37,83 @@ Then open the repo in Claude Code and run:
|
|
|
38
37
|
run qa PROJ-123
|
|
39
38
|
```
|
|
40
39
|
|
|
41
|
-
##
|
|
40
|
+
## How the QA agent works
|
|
41
|
+
|
|
42
|
+
Every run follows the same structured flow:
|
|
43
|
+
|
|
44
|
+
1. **Card input** — give a Jira card ID (`run qa PROJ-123`)
|
|
45
|
+
2. **Product context** — saved context + the Jira card are loaded automatically
|
|
46
|
+
3. **Gap questions** — you're asked only what isn't already known (one message)
|
|
47
|
+
4. **Phase selection**:
|
|
48
|
+
- **Phase 1 — Manual testing** (UI/Figma diff → browser tests → bugs → charter)
|
|
49
|
+
- **Phase 2 — Write automation** (reuse audit → Gherkin → steps → POM → real run + self-heal)
|
|
50
|
+
- **Phase 3 — Both** (manual first — its findings feed the automation)
|
|
51
|
+
|
|
52
|
+
Automation is **reuse-first**: existing `.feature` files and step definitions are catalogued
|
|
53
|
+
before any Gherkin is written, drafted steps must match existing ones (exact → parameterized
|
|
54
|
+
→ reworded) before a new step definition is allowed, and each run reports its reuse %.
|
|
55
|
+
|
|
56
|
+
## Commands to run skills
|
|
57
|
+
|
|
58
|
+
Every skill is plain-English triggered — no slash command required — except `/qa-agent`
|
|
59
|
+
and `/write-acceptance-criteria`, which are also available as explicit slash commands.
|
|
42
60
|
|
|
43
|
-
|
|
61
|
+
### QA Agent (orchestrator)
|
|
62
|
+
|
|
63
|
+
| Command | What happens |
|
|
44
64
|
|---|---|
|
|
45
|
-
| `run qa PROJ-123` |
|
|
46
|
-
| `manual test PROJ-123` |
|
|
47
|
-
| `automate PROJ-123` |
|
|
48
|
-
| `
|
|
49
|
-
| `
|
|
50
|
-
|
|
51
|
-
|
|
65
|
+
| `/qa-agent PROJ-123` or `run qa PROJ-123` | Loads context, asks only the missing gaps, then shows the phase menu |
|
|
66
|
+
| `manual test PROJ-123` / `test PROJ-123` / `do QA on PROJ-123` | Phase 1 — manual testing + bug filing + charter |
|
|
67
|
+
| `automate PROJ-123` / `write automation for PROJ-123` | Phase 2 — reuse-first BDD automation |
|
|
68
|
+
| `full QA PROJ-123` / `run everything for PROJ-123` / `end to end PROJ-123` | Phase 3 — manual first, then automation |
|
|
69
|
+
| `run qa PROJ-123 --reset-context` | Same as above but ignores saved product context and starts fresh (overwrites it on completion) |
|
|
70
|
+
|
|
71
|
+
### Standalone skills (no card required, or card optional)
|
|
72
|
+
|
|
73
|
+
| Command | Skill | What happens |
|
|
74
|
+
|---|---|---|
|
|
75
|
+
| `ui test` / `check design` / `compare with figma` | ui-test-figma | Live app vs Figma design diff |
|
|
76
|
+
| `accessibility test PROJ-123` / `a11y test [URL]` | accessibility-testing | WCAG 2.1 A/AA audit, files bugs on the card if given |
|
|
77
|
+
| `file a bug` / `log a bug` | bug-reporting | Interactive bug filing to Jira (new or existing card) |
|
|
78
|
+
| `generate a test charter` / `create a charter` | test-charter | Publishes a Test Charter from a saved execution report |
|
|
79
|
+
| `delete files` / `clean up outputs` | delete-files | Prompts to delete/keep files in `outputs/` |
|
|
80
|
+
| `/write-acceptance-criteria PROJ-123` | write-acceptance-criteria | Generates AC, appends to the Jira card description |
|
|
81
|
+
| `/impacted-tests` / `which tests are impacted by this pull` | impacted-tests | After pulling dev changes into a test branch, reports which Cucumber feature files are at risk — report-only, no card needed |
|
|
52
82
|
|
|
53
83
|
## What postinstall does
|
|
54
84
|
|
|
55
85
|
| File | Behaviour |
|
|
56
86
|
|---|---|
|
|
57
87
|
| `.claude/skills/*/SKILL.md` + companion `.md` files | Always overwritten (versioned logic) |
|
|
88
|
+
| `.claude/skills/SKILLS_CONTEXT.md` | Always overwritten + stamped with the installed pack version |
|
|
58
89
|
| `.claude/commands/*.md` | Always overwritten |
|
|
59
90
|
| `.claude/skills/qa-agent/product_context/**` | **Never touched** after first seed |
|
|
60
91
|
| `.claude/settings.json` | Created once, never overwritten |
|
|
61
|
-
| `CLAUDE.md`, `.mcp.json`, `cucumber.cjs` | Created once, never overwritten |
|
|
92
|
+
| `CLAUDE.md`, `.mcp.json`, `cucumber.cjs`, `.env` | Created once, never overwritten |
|
|
93
|
+
| `.env.example` | Always refreshed (shows latest env keys) |
|
|
62
94
|
| `.claude/settings.local.json` | Created once from example |
|
|
63
95
|
| `.claude/settings.local.json.example` | Always refreshed (shows latest options) |
|
|
64
|
-
| `.gitignore` | `outputs
|
|
96
|
+
| `.gitignore` | Managed `# >>> qa-pack` block regenerated on every install — ignores all pack-installed skills/commands plus `outputs/`, session files, and local settings |
|
|
97
|
+
|
|
98
|
+
## Git hygiene
|
|
99
|
+
|
|
100
|
+
Everything qa-pack copies into `.claude/` is git-ignored via a managed block in your `.gitignore` (between `# >>> qa-pack` and `# <<< qa-pack` markers). Your product repo's **own** skills in `.claude/skills/` are not affected — only the pack's skill folders are listed by name. The block is regenerated on every install, so skills added in future versions are ignored automatically; anything you write outside the markers is never touched.
|
|
101
|
+
|
|
102
|
+
Secrets live only in `.env` (gitignored). `product_context/` contains no credentials but lives inside an ignored folder, so it also stays out of git — each machine builds its own context from its runs.
|
|
103
|
+
|
|
104
|
+
If you installed an older qa-pack version and already committed the skill files, postinstall prints the one-time `git rm -r --cached` command to untrack them (files stay on disk).
|
|
65
105
|
|
|
66
106
|
## Publishing (maintainers)
|
|
67
107
|
|
|
68
108
|
```bash
|
|
69
|
-
# In qa-pack/ repo
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
#
|
|
109
|
+
# In qa-pack/ repo — commit your changes first (npm version needs a clean tree)
|
|
110
|
+
git add . && git commit -m "your message"
|
|
111
|
+
|
|
112
|
+
# Bump: patch = fixes, minor = new/changed skill behavior, major = breaking
|
|
113
|
+
npm version minor -m "release: v%s — short description"
|
|
114
|
+
|
|
115
|
+
git push origin main --follow-tags
|
|
116
|
+
# GitHub Actions publishes to npmjs automatically (triggers on package.json change on main)
|
|
73
117
|
```
|
|
74
118
|
|
|
75
119
|
## Requirements
|
|
@@ -77,3 +121,5 @@ git add . && git commit -m "your message" && git push origin main
|
|
|
77
121
|
- Claude Code CLI
|
|
78
122
|
- Node.js 18+
|
|
79
123
|
- Atlassian MCP connected in Claude Code (for Jira)
|
|
124
|
+
- Playwright MCP — configured automatically via the seeded `.mcp.json`
|
|
125
|
+
- Python 3 — only if you enable the optional token tracking (`QA_TRACKING_DIR` in `.env`)
|
package/bin/postinstall.js
CHANGED
|
@@ -71,6 +71,16 @@ const commandsSrc = path.join(PACK_DIR, 'claude', 'commands');
|
|
|
71
71
|
const commandsDest = path.join(PROJECT_ROOT, '.claude', 'commands');
|
|
72
72
|
copyDir(commandsSrc, commandsDest, () => true);
|
|
73
73
|
|
|
74
|
+
// ── 2b. Stamp pack version into the copied SKILLS_CONTEXT.md ──────────────────
|
|
75
|
+
const packVersion = require(path.join(PACK_DIR, 'package.json')).version;
|
|
76
|
+
const contextMapDest = path.join(skillsDest, 'SKILLS_CONTEXT.md');
|
|
77
|
+
if (fs.existsSync(contextMapDest)) {
|
|
78
|
+
const stamped = fs
|
|
79
|
+
.readFileSync(contextMapDest, 'utf8')
|
|
80
|
+
.replace('{{QA_PACK_VERSION}}', packVersion);
|
|
81
|
+
fs.writeFileSync(contextMapDest, stamped);
|
|
82
|
+
}
|
|
83
|
+
|
|
74
84
|
// ── 3. settings.json — create only if missing ─────────────────────────────────
|
|
75
85
|
copyFile(
|
|
76
86
|
path.join(PACK_DIR, 'claude', 'settings.json'),
|
|
@@ -99,7 +109,14 @@ const templates = [
|
|
|
99
109
|
['templates/CLAUDE.md', 'CLAUDE.md'],
|
|
100
110
|
['templates/mcp.json', '.mcp.json'],
|
|
101
111
|
['templates/cucumber.cjs', 'cucumber.cjs'],
|
|
112
|
+
['templates/.env.example', '.env'],
|
|
102
113
|
];
|
|
114
|
+
// Always refresh the .env.example reference copy (like settings.local.json.example)
|
|
115
|
+
copyFile(
|
|
116
|
+
path.join(PACK_DIR, 'templates', '.env.example'),
|
|
117
|
+
path.join(PROJECT_ROOT, '.env.example'),
|
|
118
|
+
{ overwrite: true }
|
|
119
|
+
);
|
|
103
120
|
for (const [src, dest] of templates) {
|
|
104
121
|
copyFile(
|
|
105
122
|
path.join(PACK_DIR, src),
|
|
@@ -122,20 +139,65 @@ if (fs.existsSync(npmrcPath)) {
|
|
|
122
139
|
log.added.push('.npmrc');
|
|
123
140
|
}
|
|
124
141
|
|
|
125
|
-
// ── 7. .gitignore —
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
142
|
+
// ── 7. .gitignore — managed block covering everything qa-pack installs ────────
|
|
143
|
+
// Regenerated on every install from the pack contents, so newly added skills
|
|
144
|
+
// are ignored automatically. Delimited by markers; user entries outside the
|
|
145
|
+
// block are never touched.
|
|
146
|
+
const GITIGNORE_START = '# >>> qa-pack (managed by @roopesh.yadava/qa-pack — do not edit this block)';
|
|
147
|
+
const GITIGNORE_END = '# <<< qa-pack';
|
|
148
|
+
|
|
149
|
+
function packIgnoreEntries() {
|
|
150
|
+
const entries = [
|
|
151
|
+
'outputs/',
|
|
152
|
+
'.env',
|
|
153
|
+
'.playwright-session.json',
|
|
154
|
+
'.claude/settings.local.json',
|
|
155
|
+
'.claude/settings.local.json.example',
|
|
156
|
+
];
|
|
157
|
+
for (const entry of fs.readdirSync(skillsSrc, { withFileTypes: true })) {
|
|
158
|
+
entries.push(entry.isDirectory()
|
|
159
|
+
? `.claude/skills/${entry.name}/`
|
|
160
|
+
: `.claude/skills/${entry.name}`);
|
|
135
161
|
}
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
162
|
+
for (const entry of fs.readdirSync(commandsSrc, { withFileTypes: true })) {
|
|
163
|
+
if (entry.isFile()) entries.push(`.claude/commands/${entry.name}`);
|
|
164
|
+
}
|
|
165
|
+
return entries;
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
const gitignorePath = path.join(PROJECT_ROOT, '.gitignore');
|
|
169
|
+
const ignoreEntries = packIgnoreEntries();
|
|
170
|
+
const managedBlock = `${GITIGNORE_START}\n${ignoreEntries.join('\n')}\n${GITIGNORE_END}\n`;
|
|
171
|
+
|
|
172
|
+
let gitignoreContent = fs.existsSync(gitignorePath)
|
|
173
|
+
? fs.readFileSync(gitignorePath, 'utf8')
|
|
174
|
+
: '';
|
|
175
|
+
const gitignoreExisted = gitignoreContent !== '' || fs.existsSync(gitignorePath);
|
|
176
|
+
const before = gitignoreContent;
|
|
177
|
+
|
|
178
|
+
// Drop any previous managed block
|
|
179
|
+
const blockRe = new RegExp(
|
|
180
|
+
`${GITIGNORE_START.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}[\\s\\S]*?${GITIGNORE_END.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\n?`,
|
|
181
|
+
'g'
|
|
182
|
+
);
|
|
183
|
+
gitignoreContent = gitignoreContent.replace(blockRe, '');
|
|
184
|
+
|
|
185
|
+
// Drop the legacy "# qa-pack" header and loose duplicates of managed entries
|
|
186
|
+
gitignoreContent = gitignoreContent
|
|
187
|
+
.split('\n')
|
|
188
|
+
.filter(line => {
|
|
189
|
+
const t = line.trim();
|
|
190
|
+
return t !== '# qa-pack' && !ignoreEntries.includes(t);
|
|
191
|
+
})
|
|
192
|
+
.join('\n');
|
|
193
|
+
|
|
194
|
+
if (gitignoreContent && !gitignoreContent.endsWith('\n')) gitignoreContent += '\n';
|
|
195
|
+
if (gitignoreContent && !gitignoreContent.endsWith('\n\n')) gitignoreContent += '\n';
|
|
196
|
+
gitignoreContent += managedBlock;
|
|
197
|
+
|
|
198
|
+
if (gitignoreContent !== before) {
|
|
199
|
+
fs.writeFileSync(gitignorePath, gitignoreContent);
|
|
200
|
+
(gitignoreExisted ? log.updated : log.added).push('.gitignore');
|
|
139
201
|
}
|
|
140
202
|
|
|
141
203
|
// ── Summary ───────────────────────────────────────────────────────────────────
|
|
@@ -146,6 +208,25 @@ if (log.skipped.length) console.log(' Skipped :', log.skipped.join(', '));
|
|
|
146
208
|
if (log.protected.length) console.log(' Protected:', log.protected.join(', '), '(product context — never overwritten)');
|
|
147
209
|
|
|
148
210
|
if (log.added.includes('CLAUDE.md')) {
|
|
149
|
-
console.log('\n Next:
|
|
211
|
+
console.log('\n Next: fill in secrets in .env (QA_APP_URL, QA_USERNAME, QA_PASSWORD, ...)');
|
|
212
|
+
console.log(' and project facts in CLAUDE.md (Jira key, environment, auth method).');
|
|
150
213
|
}
|
|
214
|
+
|
|
215
|
+
// If pack files were committed before this version, .gitignore alone won't hide
|
|
216
|
+
// them — they must be removed from the git index once.
|
|
217
|
+
try {
|
|
218
|
+
const { execSync } = require('child_process');
|
|
219
|
+
const packPaths = ignoreEntries.filter(e => e.startsWith('.claude/'));
|
|
220
|
+
const tracked = execSync(
|
|
221
|
+
`git ls-files -- ${packPaths.map(p => JSON.stringify(p)).join(' ')}`,
|
|
222
|
+
{ cwd: PROJECT_ROOT, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }
|
|
223
|
+
).trim();
|
|
224
|
+
if (tracked) {
|
|
225
|
+
const trackedRoots = [...new Set(
|
|
226
|
+
tracked.split('\n').map(f => packPaths.find(p => f === p || f.startsWith(p)))
|
|
227
|
+
)].filter(Boolean);
|
|
228
|
+
console.log('\n Note: qa-pack files are currently tracked by git. To untrack them (keeps files on disk):');
|
|
229
|
+
console.log(` git rm -r --cached ${trackedRoots.join(' ')} && git commit -m "chore: untrack qa-pack files"`);
|
|
230
|
+
}
|
|
231
|
+
} catch (_) { /* not a git repo or git unavailable — nothing to do */ }
|
|
151
232
|
console.log('');
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: impacted-tests
|
|
3
|
+
description: Map the source changes a developer-branch pull brought in to the BDD/Cucumber feature files they can break, and print a report.
|
|
4
|
+
argument-hint: "[base-branch-or-commit] [changed file paths...]"
|
|
5
|
+
user-invocable: true
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
Invoke the `impacted-tests` skill.
|
|
9
|
+
|
|
10
|
+
If the user provided a base branch/commit or explicit file paths as arguments, pass them
|
|
11
|
+
directly into the skill so it skips diff auto-detection and uses those instead.
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: qa-agent
|
|
3
|
-
description: Launch the QA Agent —
|
|
4
|
-
argument-hint: "[Jira card ID | 'full QA' | skill name | plain description]"
|
|
3
|
+
description: Launch the QA Agent — card → product context → gap questions → phase selection (1 Manual / 2 Automation / 3 Both).
|
|
4
|
+
argument-hint: "[Jira card ID | 'full QA PROJ-123' | skill name | plain description]"
|
|
5
5
|
user-invocable: true
|
|
6
6
|
---
|
|
7
7
|
|
|
8
8
|
Invoke the `qa-agent` skill.
|
|
9
9
|
|
|
10
|
-
If the user provided an argument (e.g. a Jira card ID
|
|
11
|
-
pass it directly into the skill as the initial intent so the agent does not
|
|
12
|
-
it has already received.
|
|
10
|
+
If the user provided an argument (e.g. a Jira card ID, "full QA for PROJ-123", or a phase
|
|
11
|
+
choice), pass it directly into the skill as the initial intent so the agent does not
|
|
12
|
+
re-ask for input it has already received.
|
|
@@ -1,7 +1,21 @@
|
|
|
1
1
|
# QA Skills — Context Map
|
|
2
|
+
|
|
3
|
+
**qa-pack version: {{QA_PACK_VERSION}}**
|
|
4
|
+
|
|
2
5
|
<!-- Generated by graphify from .claude/skills. Load this instead of individual SKILL.md files
|
|
3
6
|
when you only need pipeline structure. Load a specific SKILL.md only when executing that skill. -->
|
|
4
7
|
|
|
8
|
+
## Secrets Rule — All Skills Must Follow
|
|
9
|
+
|
|
10
|
+
App URL, credentials, OTP code, and Jira REST tokens live in `.env` at the repo root
|
|
11
|
+
(`QA_APP_URL`, `QA_LOGIN_URL`, `QA_USERNAME`, `QA_PASSWORD`, `QA_OTP_CODE`,
|
|
12
|
+
`JIRA_BASE_URL`, `JIRA_EMAIL`, `JIRA_API_TOKEN`).
|
|
13
|
+
|
|
14
|
+
- Read `.env` ONCE at skill start (Read tool). Precedence: `.env` > product context > ask the user.
|
|
15
|
+
- **Never print `QA_PASSWORD` or `JIRA_API_TOKEN`** to chat, reports, hints files, or product context.
|
|
16
|
+
- Never write credentials into `context.md`, `CLAUDE.md`, or any `outputs/` file.
|
|
17
|
+
- If a required value is missing from `.env`, ask for it once and suggest the user add it to `.env`.
|
|
18
|
+
|
|
5
19
|
## Global Token Rules — All Skills Must Follow
|
|
6
20
|
|
|
7
21
|
These rules apply to every skill, every run. Violating them is the single biggest source of token waste.
|
|
@@ -65,26 +79,28 @@ Jira Card ID (Atlassian MCP)
|
|
|
65
79
|
│
|
|
66
80
|
▼
|
|
67
81
|
┌──────────────┐
|
|
68
|
-
│ qa-agent │ ← entry point
|
|
82
|
+
│ qa-agent │ ← entry point: card → context → gap questions → phase menu
|
|
69
83
|
└──────┬───────┘
|
|
70
84
|
│
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
[
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
+
┌─────┼──────────────┐
|
|
86
|
+
▼ ▼ ▼
|
|
87
|
+
[PHASE 1] [PHASE 2] [PHASE 3 = Phase 1 THEN Phase 2]
|
|
88
|
+
MANUAL AUTOMATION (manual first — its automation-hints
|
|
89
|
+
│ │ file feeds Phase 2's selectors)
|
|
90
|
+
▼ ▼
|
|
91
|
+
ui-test-figma reuse audit (step catalog + repo layout discovery)
|
|
92
|
+
│ │
|
|
93
|
+
manual-testing Gherkin → Step Defs → POM
|
|
94
|
+
│ │
|
|
95
|
+
bug-reporting dry run → REAL RUN → self-heal (≤3)
|
|
96
|
+
│ │
|
|
97
|
+
test-charter │
|
|
98
|
+
└─────┬──────┘
|
|
99
|
+
│
|
|
100
|
+
Regression suggestion (Covered Flows overlap)
|
|
85
101
|
│
|
|
86
102
|
Save Product Context
|
|
87
|
-
(product_context/{PRODUCT}/context.md)
|
|
103
|
+
(product_context/{PRODUCT}/context.md — no secrets)
|
|
88
104
|
│
|
|
89
105
|
End-to-End Testing complete
|
|
90
106
|
```
|
|
@@ -94,7 +110,7 @@ automation ui-test-figma (Figma MCP + Playwright CLI)
|
|
|
94
110
|
| Skill | Input | Output | MCP Needed |
|
|
95
111
|
|-------|-------|--------|------------|
|
|
96
112
|
| `qa-agent` | Jira card ID or menu choice | Dispatches to correct skill | Atlassian |
|
|
97
|
-
| `automation` | Jira card ID | Gherkin + Step Defs + POM +
|
|
113
|
+
| `automation` | Jira card ID | Reuse audit + Gherkin + Step Defs + POM + real run (reuse % reported) | Atlassian, Playwright |
|
|
98
114
|
| `manual-testing` | Jira card ID + app URL | Execution report + bugs + charter | Atlassian, Playwright |
|
|
99
115
|
| `ui-test-figma` | Figma URL + app URL | UI mismatch report, Jira comment | Playwright (CLI+MCP), Figma (optional) |
|
|
100
116
|
| `accessibility-testing` | Full page URL + Jira card (optional) | WCAG 2.1 A/AA report + Jira bugs | Playwright (CLI+MCP), Atlassian |
|
|
@@ -105,22 +121,26 @@ automation ui-test-figma (Figma MCP + Playwright CLI)
|
|
|
105
121
|
|
|
106
122
|
1. `automation` — 13 edges (Gherkin→StepDefs→POM chain, token tracking, Playwright MCP)
|
|
107
123
|
2. `manual-testing` — 12 edges (orchestrates ui-test-figma, bug-reporting, test-charter)
|
|
108
|
-
3. `qa-agent` — 7 edges (dispatches all paths, owns
|
|
124
|
+
3. `qa-agent` — 7 edges (dispatches all paths, owns the Phase 3 pipeline)
|
|
109
125
|
4. `test-charter` — 6 edges (reads execution report, publishes to Decision Record API)
|
|
110
126
|
5. `ui-test-figma` — 6 edges (Figma MCP preferred, Playwright CLI fallback)
|
|
111
127
|
|
|
112
128
|
## Dispatch Map (qa-agent routes)
|
|
113
129
|
|
|
114
130
|
```
|
|
115
|
-
"
|
|
116
|
-
"
|
|
117
|
-
"
|
|
118
|
-
"
|
|
119
|
-
"
|
|
120
|
-
"
|
|
121
|
-
"
|
|
131
|
+
Phase 1 / "manual test" → manual-testing (calls ui-test-figma → bug-reporting → test-charter internally)
|
|
132
|
+
Phase 2 / "automate CARD" → automation (reuse audit → Gherkin → steps → POM → run + self-heal)
|
|
133
|
+
Phase 3 / "full QA" → manual-testing THEN automation (hints file reused) → End-to-End complete
|
|
134
|
+
"ui test" → ui-test-figma (standalone)
|
|
135
|
+
"accessibility test" → accessibility-testing (standalone, URL + optional Jira card)
|
|
136
|
+
"file bug" → bug-reporting (standalone)
|
|
137
|
+
"charter" → test-charter (standalone)
|
|
122
138
|
```
|
|
123
139
|
|
|
140
|
+
**Input collection happens ONCE in qa-agent.** Sub-skills receive their parameters in the
|
|
141
|
+
invocation message (card ID, URLs, flags) and read secrets from `.env` — they must not
|
|
142
|
+
re-ask for anything qa-agent already passed.
|
|
143
|
+
|
|
124
144
|
## Playwright Usage Guide (token budget)
|
|
125
145
|
|
|
126
146
|
### Expensive (avoid unless needed)
|
|
@@ -140,17 +160,28 @@ browser_evaluate("() => JSON.stringify({ls: {...localStorage}, ss: {...sessionSt
|
|
|
140
160
|
# Later CLI screenshots use: --storage-state .playwright-session.json
|
|
141
161
|
```
|
|
142
162
|
|
|
143
|
-
## Token Tracking — canonical pattern (all skills follow this)
|
|
163
|
+
## Token Tracking — OPTIONAL, canonical pattern (all skills follow this)
|
|
164
|
+
|
|
165
|
+
Token tracking is **disabled by default** and must never break a run.
|
|
166
|
+
|
|
167
|
+
**Enablement check — run ONCE at skill start, silently:**
|
|
168
|
+
```bash
|
|
169
|
+
[ -n "$QA_TRACKING_DIR" ] && [ -f "$QA_TRACKING_DIR/track_tokens.py" ] \
|
|
170
|
+
&& echo TRACKING_ENABLED || echo TRACKING_DISABLED
|
|
171
|
+
```
|
|
172
|
+
(`QA_TRACKING_DIR` comes from `.env`. If the variable is unset, the scripts are missing,
|
|
173
|
+
or any tracking command errors — **skip ALL tracking commands for the rest of the run
|
|
174
|
+
silently**. Never mention tracking to the user, never retry a failed tracking call.)
|
|
144
175
|
|
|
145
|
-
**
|
|
146
|
-
Scripts live at `/home/user/projects/Smart-testing-with-AI-Agents/`.
|
|
176
|
+
**When enabled** — silent background task. Replace `CARD_ID` with the actual card ID:
|
|
147
177
|
|
|
148
178
|
```bash
|
|
149
|
-
PROJECT
|
|
179
|
+
PROJECT=$QA_TRACKING_DIR
|
|
180
|
+
MODEL=${QA_TRACKING_MODEL:-claude}
|
|
150
181
|
|
|
151
182
|
# Entry (before any work)
|
|
152
183
|
SNAP=$(python3 $PROJECT/context_snapshot.py --phase start) && \
|
|
153
|
-
python3 $PROJECT/track_tokens.py start --card CARD_ID $SNAP --model
|
|
184
|
+
python3 $PROJECT/track_tokens.py start --card CARD_ID $SNAP --model $MODEL
|
|
154
185
|
|
|
155
186
|
# After each phase (repeat per checkpoint, replacing PHASE_NAME)
|
|
156
187
|
SNAP=$(python3 $PROJECT/context_snapshot.py --phase PHASE_NAME) && \
|
|
@@ -158,13 +189,13 @@ python3 $PROJECT/track_tokens.py phase --card CARD_ID --name PHASE_NAME $SNAP
|
|
|
158
189
|
|
|
159
190
|
# Exit (after skill completes)
|
|
160
191
|
SNAP=$(python3 $PROJECT/context_snapshot.py --phase end) && \
|
|
161
|
-
python3 $PROJECT/track_tokens.py end --card CARD_ID $SNAP --model
|
|
192
|
+
python3 $PROJECT/track_tokens.py end --card CARD_ID $SNAP --model $MODEL && \
|
|
162
193
|
python3 $PROJECT/track_tokens.py report && \
|
|
163
194
|
python3 $PROJECT/track_tokens.py session
|
|
164
195
|
```
|
|
165
196
|
|
|
166
197
|
**Phase names by skill:**
|
|
167
|
-
- automation: `start` → `jira_fetch` → `gherkin_generation` → `step_definitions` → `pom_generation` → `end`
|
|
198
|
+
- automation: `start` → `jira_fetch` → `reuse_audit` → `gherkin_generation` → `step_definitions` → `pom_generation` → `test_run` → `end`
|
|
168
199
|
- manual-testing: `start` → `jira_fetch` → `ui_testing` → `test_execution` → `end`
|
|
169
200
|
- ui-test-figma: `start` → `login` → `comparison` → `end`
|
|
170
201
|
- accessibility-testing: `start` → `login` → `a11y_checks` → `end`
|
|
@@ -189,6 +220,9 @@ First run will always be 0% (cold cache). Second and subsequent runs should cach
|
|
|
189
220
|
| Accessibility report | `outputs/a11y-report-[slug]-[date].md` |
|
|
190
221
|
| Accessibility baseline screenshot | `outputs/screenshots/a11y-baseline-[slug].png` |
|
|
191
222
|
| Auth session | `.playwright-session.json` (gitignored) |
|
|
223
|
+
| Secrets / environment | `.env` at repo root (gitignored) |
|
|
224
|
+
| Step catalog (reuse audit) | `.claude/skills/qa-agent/product_context/[PRODUCT]/step-catalog.md` |
|
|
225
|
+
| Automation hints | `outputs/automation-hints-[CARD]-[date].md` |
|
|
192
226
|
| Token analytics | `~/.claude/token_analytics.png` |
|
|
193
227
|
| Knowledge graph | `graphify-out/graph.html` (open in browser) |
|
|
194
228
|
| Product QA context | `.claude/skills/qa-agent/product_context/[PRODUCT]/context.md` |
|
|
@@ -4,7 +4,7 @@ description: >
|
|
|
4
4
|
Tests a live web page against WCAG 2.1 Level A and AA criteria. Injects axe-core via
|
|
5
5
|
browser_evaluate for automated rule checks, then runs manual keyboard, focus, skip-link,
|
|
6
6
|
heading, and alt-text checks via Playwright MCP. Accepts a full page URL, logs in with
|
|
7
|
-
Playwright MCP (OTP
|
|
7
|
+
Playwright MCP (credentials/OTP from .env), navigates to the target page, runs all checks, saves
|
|
8
8
|
a structured report to outputs/, and files violations as Jira bugs on the provided card.
|
|
9
9
|
Uses Playwright CLI for zero-token screenshots. Token tracking enabled.
|
|
10
10
|
Standalone or callable from qa-agent.
|
|
@@ -13,7 +13,7 @@ description: >
|
|
|
13
13
|
or "check accessibility".
|
|
14
14
|
compatibility: >
|
|
15
15
|
Playwright MCP must be running. axe-core injected at runtime from CDN — no install needed.
|
|
16
|
-
OTP
|
|
16
|
+
OTP code from .env QA_OTP_CODE. Playwright CLI (npx playwright) must be available.
|
|
17
17
|
user-invocable: true
|
|
18
18
|
---
|
|
19
19
|
|
|
@@ -30,7 +30,8 @@ user-invocable: true
|
|
|
30
30
|
|
|
31
31
|
## Token Tracking
|
|
32
32
|
|
|
33
|
-
|
|
33
|
+
Optional and silent — follow the **Token Tracking** pattern in `SKILLS_CONTEXT.md`
|
|
34
|
+
(run its enablement check once at skill start; if disabled, skip every checkpoint silently).
|
|
34
35
|
Checkpoints: `start` → `login` → `a11y_checks` → `end + report + session`.
|
|
35
36
|
Use `CARD_ID` if Jira card provided, else `"a11y-test"`. Never show tracking output to user.
|
|
36
37
|
|
|
@@ -87,11 +88,10 @@ Check the current URL:
|
|
|
87
88
|
- Redirected to `/login`, `/sign-in`, `/auth` → session not active, ask for credentials.
|
|
88
89
|
- Dashboard / main nav visible → session already active, skip to Step 2.
|
|
89
90
|
|
|
90
|
-
If login needed:
|
|
91
|
-
|
|
92
|
-
> (after answer) "And the **password**?"
|
|
91
|
+
If login needed: read `.env` for `QA_USERNAME` / `QA_PASSWORD` / `QA_OTP_CODE` first.
|
|
92
|
+
Only if still missing, ask for email + password in one message (never print the password).
|
|
93
93
|
|
|
94
|
-
Store: `EMAIL`, `PASSWORD`.
|
|
94
|
+
Store: `EMAIL`, `PASSWORD`, `OTP_CODE`.
|
|
95
95
|
|
|
96
96
|
Once all inputs collected:
|
|
97
97
|
> "Got it — logging in and starting the accessibility checks..."
|
|
@@ -111,7 +111,7 @@ Once all inputs collected:
|
|
|
111
111
|
|
|
112
112
|
**If OTP screen appears** (6-digit input / PIN boxes):
|
|
113
113
|
7. `browser_click` first OTP input box
|
|
114
|
-
8. `browser_type(text:
|
|
114
|
+
8. `browser_type(text: OTP_CODE)` — the `QA_OTP_CODE` value from `.env`, auto-advances through boxes
|
|
115
115
|
9. `browser_click` Verify / Confirm button
|
|
116
116
|
10. `browser_wait_for(state: "networkidle")`
|
|
117
117
|
|
|
@@ -313,5 +313,5 @@ Accessibility Testing Complete
|
|
|
313
313
|
| `browser_evaluate` returns null | Log "script returned null" for that check, continue |
|
|
314
314
|
| Jira card not found | Ask to verify key; offer to save report only |
|
|
315
315
|
| Jira bug creation fails | Note failure, continue remaining bugs, report failures at end |
|
|
316
|
-
| OTP screen appears unexpectedly | Type `
|
|
316
|
+
| OTP screen appears unexpectedly | Type the `QA_OTP_CODE` value from `.env`, proceed |
|
|
317
317
|
| Page requires further navigation after login | Follow redirect, confirm TARGET_URL loads |
|
|
@@ -113,7 +113,7 @@ first box only populates one digit and the component does not auto-advance.
|
|
|
113
113
|
|
|
114
114
|
```javascript
|
|
115
115
|
// ❌ WRONG — only fills box 1, leaves 5 empty
|
|
116
|
-
await this.otpInput.fill(
|
|
116
|
+
await this.otpInput.fill(process.env.QA_OTP_CODE);
|
|
117
117
|
|
|
118
118
|
// ✅ CORRECT — click first box to focus, then type digit-by-digit
|
|
119
119
|
await this.otpInput.waitFor({ state: 'visible', timeout: 15000 });
|