@roopesh.yadava/qa-pack 1.3.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +23 -7
- package/bin/postinstall.js +31 -0
- package/claude/skills/SKILLS_CONTEXT.md +61 -5
- package/claude/skills/automation/SKILL.md +53 -6
- package/claude/skills/bug-reporting/SKILL.md +38 -0
- package/claude/skills/k6-framework-scaffold/SKILL.md +58 -0
- package/claude/skills/k6-framework-scaffold/scaffold.sh +388 -0
- package/claude/skills/manual-testing/SKILL.md +67 -2
- package/claude/skills/qa-agent/SKILL.md +30 -0
- package/claude/skills/qa-agent/product_context/README.md +16 -0
- package/claude/skills/qa-agent/toolkit/qa-toolkit.cjs +705 -0
- package/claude/skills/qa-insights/SKILL.md +55 -0
- package/claude/skills/roam-testing/SKILL.md +187 -0
- package/package.json +5 -2
package/README.md
CHANGED
|
@@ -31,11 +31,12 @@ After installing:
|
|
|
31
31
|
(plus `JIRA_BASE_URL` / `JIRA_EMAIL` / `JIRA_API_TOKEN` if you want bug screenshots attached).
|
|
32
32
|
2. Open `CLAUDE.md` and fill in the non-secret project facts (Jira key, environment, auth method).
|
|
33
33
|
|
|
34
|
-
`.claude/settings.local.json` (seeded
|
|
35
|
-
Playwright MCP tool calls, since those run constantly
|
|
36
|
-
the app under test. Every Atlassian/Jira MCP call —
|
|
37
|
-
transitioning — and any action needing your input
|
|
38
|
-
overwriting product context) still prompts for
|
|
34
|
+
`.claude/settings.local.json` (seeded from `settings.local.json.example`, and re-checked on
|
|
35
|
+
every install/update) auto-approves Playwright MCP tool calls, since those run constantly
|
|
36
|
+
during test execution and are scoped to the app under test. Every Atlassian/Jira MCP call —
|
|
37
|
+
reading a card, filing a bug, commenting, transitioning — and any action needing your input
|
|
38
|
+
(filing a bug, publishing a charter, overwriting product context) still prompts for
|
|
39
|
+
confirmation.
|
|
39
40
|
|
|
40
41
|
Then open the repo in Claude Code and run:
|
|
41
42
|
|
|
@@ -90,19 +91,34 @@ and `/write-acceptance-criteria`, which are also available as explicit slash com
|
|
|
90
91
|
| `delete files` / `clean up outputs` | delete-files | Prompts to delete/keep files in `outputs/` |
|
|
91
92
|
| `/write-acceptance-criteria PROJ-123` | write-acceptance-criteria | Generates AC, appends to the Jira card description |
|
|
92
93
|
| `/impacted-tests` / `which tests are impacted by this pull` | impacted-tests | After pulling dev changes into a test branch, reports which Cucumber feature files are at risk — report-only, no card needed |
|
|
94
|
+
| `set up k6` / `scaffold performance tests` | k6-framework-scaffold | Scaffolds a `k6-performance-tests/` framework (Grafana Cloud, protocol + optional browser layers) with commented templates to fill in — no card needed |
|
|
95
|
+
| `roam mode` / `explore the app` | roam-testing | Card-free exploratory testing — capped breadth-first crawl, report + optional bug filing + charter |
|
|
96
|
+
| `qa dashboard` / `weekly digest` / `token roi` | qa-insights | Cross-product health dashboard, weekly digest, or token-spend/ROI view — 100% script-generated, no card needed |
|
|
97
|
+
|
|
98
|
+
## The toolkit — why these features stay cheap
|
|
99
|
+
|
|
100
|
+
Several features above (self-improving locator memory, DOM fingerprint caching, duplicate-bug
|
|
101
|
+
detection, PII/secrets scanning, risk-based test ordering, the trust ratchet, and every
|
|
102
|
+
`qa-insights` report) are powered by one dependency-free script:
|
|
103
|
+
`.claude/skills/qa-agent/toolkit/qa-toolkit.cjs`. Skills shell out to it and read back a
|
|
104
|
+
single line — parsing markdown tables, hashing DOM snapshots, scoring bug-title similarity,
|
|
105
|
+
and aggregating every product's history are pure computation, so none of it costs a model
|
|
106
|
+
token beyond the one line of output. This is also why `qa-insights` (dashboard/digest/ROI
|
|
107
|
+
across every product you've ever tested) costs about the same whether you have 2 products or
|
|
108
|
+
200 — the script does the aggregation, not the model.
|
|
93
109
|
|
|
94
110
|
## What postinstall does
|
|
95
111
|
|
|
96
112
|
| File | Behaviour |
|
|
97
113
|
|---|---|
|
|
98
|
-
| `.claude/skills/*/SKILL.md` + companion `.md` files | Always overwritten (versioned logic) |
|
|
114
|
+
| `.claude/skills/*/SKILL.md` + companion `.md`/`.cjs`/`.sh` files (e.g. the toolkit script) | Always overwritten (versioned logic) |
|
|
99
115
|
| `.claude/skills/SKILLS_CONTEXT.md` | Always overwritten + stamped with the installed pack version |
|
|
100
116
|
| `.claude/commands/*.md` | Always overwritten |
|
|
101
117
|
| `.claude/skills/qa-agent/product_context/**` | **Never touched** after first seed |
|
|
102
118
|
| `.claude/settings.json` | Created once, never overwritten |
|
|
103
119
|
| `CLAUDE.md`, `.mcp.json`, `cucumber.cjs`, `.env` | Created once, never overwritten |
|
|
104
120
|
| `.env.example` | Always refreshed (shows latest env keys) |
|
|
105
|
-
| `.claude/settings.local.json` | Created once from example |
|
|
121
|
+
| `.claude/settings.local.json` | Created once from example. Every subsequent install/update also checks for the Playwright MCP auto-approve rule (`mcp__playwright`) and adds it if missing — merged in without touching any other key you've set in the file. If the file isn't valid JSON, this merge is skipped with a warning and the file is left untouched. |
|
|
106
122
|
| `.claude/settings.local.json.example` | Always refreshed (shows latest options) |
|
|
107
123
|
| `.gitignore` | Managed `# >>> qa-pack` block regenerated on every install — ignores all pack-installed skills/commands plus `outputs/`, session files, and local settings |
|
|
108
124
|
|
package/bin/postinstall.js
CHANGED
|
@@ -104,6 +104,37 @@ copyFile(
|
|
|
104
104
|
{ overwrite: true }
|
|
105
105
|
);
|
|
106
106
|
|
|
107
|
+
// ── 4b. Force-ensure the Playwright MCP auto-approve rule ─────────────────────
|
|
108
|
+
// Runs on every install/update, whether settings.local.json was just created above,
|
|
109
|
+
// already existed from an older pack version that predates this rule, or was hand-
|
|
110
|
+
// edited. Merges in only this one permissions.allow entry — every other key/value in
|
|
111
|
+
// the file (including any other allow/deny/ask rules) is left exactly as it was.
|
|
112
|
+
const PLAYWRIGHT_MCP_RULE = 'mcp__playwright';
|
|
113
|
+
if (fs.existsSync(localSettingsDest)) {
|
|
114
|
+
try {
|
|
115
|
+
const settings = JSON.parse(fs.readFileSync(localSettingsDest, 'utf8'));
|
|
116
|
+
if (typeof settings.permissions !== 'object' || settings.permissions === null) {
|
|
117
|
+
settings.permissions = {};
|
|
118
|
+
}
|
|
119
|
+
if (!Array.isArray(settings.permissions.allow)) {
|
|
120
|
+
settings.permissions.allow = [];
|
|
121
|
+
}
|
|
122
|
+
const hasRule = settings.permissions.allow.some(
|
|
123
|
+
(rule) => rule === PLAYWRIGHT_MCP_RULE || rule === `${PLAYWRIGHT_MCP_RULE}__*`
|
|
124
|
+
);
|
|
125
|
+
if (!hasRule) {
|
|
126
|
+
settings.permissions.allow.push(PLAYWRIGHT_MCP_RULE);
|
|
127
|
+
fs.writeFileSync(localSettingsDest, `${JSON.stringify(settings, null, 2)}\n`);
|
|
128
|
+
log.updated.push(`${path.relative(PROJECT_ROOT, localSettingsDest)} (added Playwright MCP auto-approve rule)`);
|
|
129
|
+
}
|
|
130
|
+
} catch (e) {
|
|
131
|
+
console.warn(
|
|
132
|
+
` Warning: could not parse ${path.relative(PROJECT_ROOT, localSettingsDest)} as JSON — left untouched.\n` +
|
|
133
|
+
` Add "${PLAYWRIGHT_MCP_RULE}" to its permissions.allow array manually to auto-approve Playwright MCP calls.`
|
|
134
|
+
);
|
|
135
|
+
}
|
|
136
|
+
}
|
|
137
|
+
|
|
107
138
|
// ── 5. One-time templates (never overwrite — user fills these in) ─────────────
|
|
108
139
|
const templates = [
|
|
109
140
|
['templates/CLAUDE.md', 'CLAUDE.md'],
|
|
@@ -70,6 +70,45 @@ Structure every skill invocation in this order so the stable prefix can be cache
|
|
|
70
70
|
The stable block qualifies for Anthropic prompt caching when it exceeds 1024 tokens.
|
|
71
71
|
Cache TTL is 5 minutes. Keep the stable block identical across runs for the same product.
|
|
72
72
|
|
|
73
|
+
### The Toolkit — the low-token path for anything product_context-shaped
|
|
74
|
+
|
|
75
|
+
`.claude/skills/qa-agent/toolkit/qa-toolkit.cjs` is a dependency-free Node script, not a
|
|
76
|
+
skill. It exists because parsing a markdown table, hashing a DOM snapshot, scoring string
|
|
77
|
+
similarity, or aggregating dozens of products' history is pure computation — routing it
|
|
78
|
+
through the model (Read the file → reason over it → maybe Edit it back) burns tokens on work
|
|
79
|
+
that doesn't need a model at all. **Any skill needing one of the things below calls the
|
|
80
|
+
toolkit and reads only its one-line result — never Read a whole context.md for this.**
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs <command> [--flags]
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
| Command | Used by | Purpose |
|
|
87
|
+
|---------|---------|---------|
|
|
88
|
+
| `get-bugs` / `get-runs` / `get-selectors` --product P | any skill | Cheap reads of one context.md table, pipe-delimited rows |
|
|
89
|
+
| `fingerprint` --product P --url U --testids "a,b" | manual-testing, automation, roam-testing | DOM fingerprint cache — `UNCHANGED` means skip re-discovery |
|
|
90
|
+
| `pii-scan` (stdin or `--file`) | bug-reporting, manual-testing, roam-testing | Flags emails/keys/tokens before anything is posted to Jira |
|
|
91
|
+
| `dup-bug` --product P --summary S | bug-reporting, manual-testing, roam-testing | Fuzzy-matches a new bug against Known Bugs, no LLM comparison needed |
|
|
92
|
+
| `cost-estimate` --product P [--phase N] | qa-agent | One-line run-cost estimate from Runs Log history |
|
|
93
|
+
| `risk-score` --product P --modules "a,b" | manual-testing | Orders tests by git churn + past bug density, not AC order |
|
|
94
|
+
| `trust-record` / `trust-status` --product P | automation (record), qa-agent (status) | Trust ratchet on Gate 1/Gate 2 approvals |
|
|
95
|
+
| `locator-record` / `locator-query` --product P --page U | automation | Self-improving locator memory across self-heal fixes |
|
|
96
|
+
| `dashboard` / `digest [--days N]` / `roi` | qa-insights | Cross-product reports — 100% script-generated, zero synthesis |
|
|
97
|
+
|
|
98
|
+
This file is versioned logic (always overwritten on `npm update`, like every other skill
|
|
99
|
+
file) — never store product data inside it. Its outputs live under
|
|
100
|
+
`product_context/{PRODUCT}/` (`dom-fingerprints.json`, `trust.json`,
|
|
101
|
+
`locator-learnings.md` — all new, all gitignored the same way `context.md` already is) or
|
|
102
|
+
`outputs/` (`dashboard.html`, `qa-weekly-digest-*.md`, `roi-report.md`).
|
|
103
|
+
|
|
104
|
+
**Never inline arbitrary text into a toolkit shell call.** Bug descriptions, locator
|
|
105
|
+
strings, and anything else that isn't a short agent-controlled token (a card ID, a product
|
|
106
|
+
folder name, a URL) can contain quotes, `` ` ``, `$(...)`, or `|` — inlined into a bash
|
|
107
|
+
argument, that is a command-injection bug, not just an escaping nuisance. Write the text to
|
|
108
|
+
a file with the Write tool first and pass the path: `pii-scan --file`, `dup-bug
|
|
109
|
+
--summary-file`, `locator-record --file <json>`. Only short, structurally-constrained values
|
|
110
|
+
(URLs, card IDs, product folder names) are safe to inline directly.
|
|
111
|
+
|
|
73
112
|
---
|
|
74
113
|
|
|
75
114
|
## Pipeline Overview
|
|
@@ -105,6 +144,10 @@ test-charter │
|
|
|
105
144
|
End-to-End Testing complete
|
|
106
145
|
```
|
|
107
146
|
|
|
147
|
+
`roam-testing` (card-free exploratory) and `qa-insights` (dashboard/digest/roi) sit outside
|
|
148
|
+
this diagram — they don't take a Jira card, and qa-insights doesn't touch Playwright at all.
|
|
149
|
+
Both are driven by the toolkit rather than by each other.
|
|
150
|
+
|
|
108
151
|
## Skills — One-Line Summary
|
|
109
152
|
|
|
110
153
|
| Skill | Input | Output | MCP Needed |
|
|
@@ -116,14 +159,17 @@ test-charter │
|
|
|
116
159
|
| `accessibility-testing` | Full page URL + Jira card (optional) | WCAG 2.1 A/AA report + Jira bugs | Playwright (CLI+MCP), Atlassian |
|
|
117
160
|
| `bug-reporting` | Bug description | Bug filed on Jira card | Atlassian |
|
|
118
161
|
| `test-charter` | Execution report MD file | Charter MD + published to API | Playwright (login) |
|
|
162
|
+
| `roam-testing` | App URL, no card required | Roam report + optional bugs + charter | Playwright, Atlassian (optional) |
|
|
163
|
+
| `qa-insights` | Nothing (reads all products) | Dashboard HTML / digest MD / ROI MD | None — pure toolkit |
|
|
119
164
|
|
|
120
165
|
## God Nodes (highest connectivity — touch these carefully)
|
|
121
166
|
|
|
122
|
-
1. `automation` —
|
|
123
|
-
2. `manual-testing` —
|
|
124
|
-
3. `qa-
|
|
125
|
-
4. `
|
|
126
|
-
5. `
|
|
167
|
+
1. `automation` — 15 edges (Gherkin→StepDefs→POM chain, token tracking, Playwright MCP, locator memory, trust ratchet)
|
|
168
|
+
2. `manual-testing` — 15 edges (orchestrates ui-test-figma, bug-reporting, test-charter, risk-score, fingerprint, dup-bug)
|
|
169
|
+
3. `qa-toolkit.cjs` — 6 skills call into it (qa-agent, automation, manual-testing, bug-reporting, roam-testing, qa-insights) — not a skill itself, but the single highest-fan-in file in the pack
|
|
170
|
+
4. `qa-agent` — 9 edges (dispatches all paths, owns the Phase 3 pipeline, cost estimate + trust status)
|
|
171
|
+
5. `test-charter` — 7 edges (reads execution report, publishes to Decision Record API; also used by roam-testing)
|
|
172
|
+
6. `ui-test-figma` — 6 edges (Figma MCP preferred, Playwright CLI fallback)
|
|
127
173
|
|
|
128
174
|
## Dispatch Map (qa-agent routes)
|
|
129
175
|
|
|
@@ -135,6 +181,8 @@ Phase 3 / "full QA" → manual-testing THEN automation (hints file reused
|
|
|
135
181
|
"accessibility test" → accessibility-testing (standalone, URL + optional Jira card)
|
|
136
182
|
"file bug" → bug-reporting (standalone)
|
|
137
183
|
"charter" → test-charter (standalone)
|
|
184
|
+
"roam mode" / "explore" → roam-testing (standalone, no card — routed outside qa-agent)
|
|
185
|
+
"dashboard"/"digest"/"roi" → qa-insights (standalone, no card — routed outside qa-agent)
|
|
138
186
|
```
|
|
139
187
|
|
|
140
188
|
**Input collection happens ONCE in qa-agent.** Sub-skills receive their parameters in the
|
|
@@ -236,3 +284,11 @@ First run will always be 0% (cold cache). Second and subsequent runs should cach
|
|
|
236
284
|
| Token analytics | `~/.claude/token_analytics.png` |
|
|
237
285
|
| Knowledge graph | `graphify-out/graph.html` (open in browser) |
|
|
238
286
|
| Product QA context | `.claude/skills/qa-agent/product_context/[PRODUCT]/context.md` |
|
|
287
|
+
| DOM fingerprint cache | `.claude/skills/qa-agent/product_context/[PRODUCT]/dom-fingerprints.json` |
|
|
288
|
+
| Trust ratchet state | `.claude/skills/qa-agent/product_context/[PRODUCT]/trust.json` |
|
|
289
|
+
| Locator learnings | `.claude/skills/qa-agent/product_context/[PRODUCT]/locator-learnings.md` |
|
|
290
|
+
| Roam session report | `outputs/roam-report-[timestamp].md` |
|
|
291
|
+
| Product health dashboard | `outputs/dashboard.html` |
|
|
292
|
+
| QA weekly digest | `outputs/qa-weekly-digest-[date].md` |
|
|
293
|
+
| Token-spend / ROI view | `outputs/roi-report.md` |
|
|
294
|
+
| Shared toolkit CLI | `.claude/skills/qa-agent/toolkit/qa-toolkit.cjs` |
|
|
@@ -269,14 +269,21 @@ Present Gherkin + reuse report and ask:
|
|
|
269
269
|
>
|
|
270
270
|
> **Type "looks good" or "confirmed" to proceed to step definitions.**
|
|
271
271
|
|
|
272
|
-
Do not write any `.cjs` files until confirmed. Iterate until approved.
|
|
272
|
+
Do not write any `.cjs` files until confirmed. Iterate until approved. Once approved, record
|
|
273
|
+
the trust ratchet — `clean` if the user approved on first presentation with no revision
|
|
274
|
+
requests this gate, `edited` if they asked for any change before approving:
|
|
275
|
+
|
|
276
|
+
```bash
|
|
277
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs trust-record --product {PRODUCT_FOLDER} --gate gate1 --result clean|edited
|
|
278
|
+
```
|
|
273
279
|
|
|
274
280
|
**If AUTO_APPROVE = true (called from qa-agent full pipeline):**
|
|
275
281
|
|
|
276
282
|
Display the reuse report + a compact summary — do NOT wait:
|
|
277
283
|
> "Gherkin generated for [CARD-ID]: {N} Rules, {N} scenarios, {X}% step reuse. Auto-approved — proceeding to step definitions."
|
|
278
284
|
|
|
279
|
-
Immediately move to Phase 2 without waiting for any input.
|
|
285
|
+
Immediately move to Phase 2 without waiting for any input. Do not call `trust-record` here —
|
|
286
|
+
there was no human review to score.
|
|
280
287
|
|
|
281
288
|
After either path: run `gherkin_generation` token checkpoint, then move to Phase 2.
|
|
282
289
|
|
|
@@ -329,14 +336,19 @@ Present step definitions and ask:
|
|
|
329
336
|
>
|
|
330
337
|
> **Type "looks good" or "confirmed" to proceed to the POM.**
|
|
331
338
|
|
|
332
|
-
Do not write the POM until confirmed.
|
|
339
|
+
Do not write the POM until confirmed. Once approved, record the trust ratchet the same way
|
|
340
|
+
as Gate 1:
|
|
341
|
+
|
|
342
|
+
```bash
|
|
343
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs trust-record --product {PRODUCT_FOLDER} --gate gate2 --result clean|edited
|
|
344
|
+
```
|
|
333
345
|
|
|
334
346
|
**If AUTO_APPROVE = true (called from qa-agent full pipeline):**
|
|
335
347
|
|
|
336
348
|
Display a compact summary only — do NOT wait:
|
|
337
349
|
> "Step definitions generated for [CARD-ID]: {N} steps across {N} files. Auto-approved — proceeding to POM."
|
|
338
350
|
|
|
339
|
-
Immediately move to Phase 3 without waiting for any input.
|
|
351
|
+
Immediately move to Phase 3 without waiting for any input. Do not call `trust-record` here.
|
|
340
352
|
|
|
341
353
|
After either path: run `step_definitions` token checkpoint, then move to Phase 3.
|
|
342
354
|
|
|
@@ -356,7 +368,31 @@ class. Only create a new POM class for a page that has none.
|
|
|
356
368
|
Use the POM class template from BDD_TEMPLATES.md for new classes; match the existing
|
|
357
369
|
class's style when extending.
|
|
358
370
|
|
|
359
|
-
|
|
371
|
+
**Locator memory check — once per page, before opening Playwright MCP for it:**
|
|
372
|
+
```bash
|
|
373
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs locator-query --product {PRODUCT_FOLDER} --page "PAGE_URL"
|
|
374
|
+
```
|
|
375
|
+
`NONE` → nothing learned yet for this page, proceed as usual. Rows returned → for each
|
|
376
|
+
listed element, avoid its recorded **Failed Locator** even if it looks like the obvious
|
|
377
|
+
choice; that pattern already broke a real test on this page. Prefer its recorded **Working
|
|
378
|
+
Locator** if the element matches.
|
|
379
|
+
|
|
380
|
+
**DOM fingerprint check — once per distinct page URL this run:**
|
|
381
|
+
```javascript
|
|
382
|
+
browser_evaluate({ expression: `
|
|
383
|
+
Array.from(document.querySelectorAll('[data-testid]')).map(el => el.getAttribute('data-testid')).join(',')
|
|
384
|
+
` })
|
|
385
|
+
```
|
|
386
|
+
```bash
|
|
387
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs fingerprint --product {PRODUCT_FOLDER} --url "PAGE_URL" --testids "RESULT_FROM_ABOVE"
|
|
388
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs get-selectors --product {PRODUCT_FOLDER} --url "PAGE_URL"
|
|
389
|
+
```
|
|
390
|
+
If fingerprint says `UNCHANGED` and `get-selectors` returns rows covering the elements this
|
|
391
|
+
Rule needs, write the POM locators straight from those rows — skip the per-element DOM
|
|
392
|
+
inspection below entirely for this page. Otherwise (`NEW`/`CHANGED`, or a needed element is
|
|
393
|
+
missing from `get-selectors`) fall through to the full inspection:
|
|
394
|
+
|
|
395
|
+
For every locator not already resolved above:
|
|
360
396
|
1. Navigate to the real page in the live app via Playwright MCP
|
|
361
397
|
2. Inspect the target element in the DOM
|
|
362
398
|
3. Check if `data-testid` already exists
|
|
@@ -420,7 +456,18 @@ On failure, diagnose and fix — **never hand off a silently failing test**:
|
|
|
420
456
|
|
|
421
457
|
1. Read the failure: locator timeout? assertion mismatch? navigation/auth issue?
|
|
422
458
|
2. Locator failures → re-inspect that element via Playwright MCP, fix the POM locator
|
|
423
|
-
(respect the locator priority table).
|
|
459
|
+
(respect the locator priority table). Once the fix passes, record it so no future run on
|
|
460
|
+
this product repeats the same wrong guess. Locator strings routinely contain quotes and
|
|
461
|
+
`|` (xpath unions) — never inline them into a shell command; write a small JSON file
|
|
462
|
+
instead (Write tool) and pass its path:
|
|
463
|
+
```
|
|
464
|
+
Write outputs/.locator-tmp.json containing:
|
|
465
|
+
{"product": "{PRODUCT_FOLDER}", "page": "PAGE_URL", "element": "ELEMENT_LABEL",
|
|
466
|
+
"failed": "OLD_LOCATOR", "fixed": "NEW_LOCATOR", "reason": "ONE_LINE_WHY", "card": "{CARD_ID}"}
|
|
467
|
+
```
|
|
468
|
+
```bash
|
|
469
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs locator-record --file outputs/.locator-tmp.json
|
|
470
|
+
```
|
|
424
471
|
3. Assertion failures → check whether expected text/behavior on the card matches the app.
|
|
425
472
|
If the app appears genuinely wrong, this is a **bug, not a test fix** — stop healing
|
|
426
473
|
that scenario, mark it failing, and note it as a bug candidate in the hand-off.
|
|
@@ -27,6 +27,12 @@ Display the list and ask:
|
|
|
27
27
|
|
|
28
28
|
Wait for the user to select a project by number or board key. Save the selected project key for the rest of the flow.
|
|
29
29
|
|
|
30
|
+
**Resolve `PRODUCT_FOLDER` (best-effort — only used by the duplicate check in Step 2a):**
|
|
31
|
+
check whether `.claude/skills/qa-agent/product_context/{KEY}/context.md` exists, or any
|
|
32
|
+
folder under `product_context/` starts with `{KEY}`. Found → store as `PRODUCT_FOLDER`.
|
|
33
|
+
Not found → leave `PRODUCT_FOLDER` unset and skip the duplicate half of Step 2a silently;
|
|
34
|
+
this is not an error, most exploratory-bug filing has no prior product context yet.
|
|
35
|
+
|
|
30
36
|
---
|
|
31
37
|
|
|
32
38
|
## Step 1A/1B — Ask card type
|
|
@@ -110,6 +116,38 @@ Do NOT ask the user for any missing fields. Use what was given and proceed.
|
|
|
110
116
|
|
|
111
117
|
---
|
|
112
118
|
|
|
119
|
+
## Step 2a — Duplicate + PII/secrets check (script calls, no extra questions unless flagged)
|
|
120
|
+
|
|
121
|
+
The user typed this text freely — it can contain quotes, `$`, backticks, anything. **Never**
|
|
122
|
+
interpolate it directly into a shell command; write it to a file with the Write tool first
|
|
123
|
+
and pass the file path instead.
|
|
124
|
+
|
|
125
|
+
**Duplicate check — only if `PRODUCT_FOLDER` was resolved in Step 1:**
|
|
126
|
+
```
|
|
127
|
+
Write outputs/.dupcheck-tmp.txt containing: {Bug Title}
|
|
128
|
+
```
|
|
129
|
+
```bash
|
|
130
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs dup-bug --product {PRODUCT_FOLDER} --summary-file outputs/.dupcheck-tmp.txt
|
|
131
|
+
```
|
|
132
|
+
`NO_DUPLICATE_FOUND` → proceed silently. `POSSIBLE_DUPLICATE: {BUG-ID} (...) — "{title}"` →
|
|
133
|
+
mention it once above the formatted report in Step 3: `"Heads up — this looks similar to
|
|
134
|
+
{BUG-ID}: \"{title}\" ({status}). File as a new bug anyway, or add to that one instead?"`
|
|
135
|
+
Follow whichever the user picks.
|
|
136
|
+
|
|
137
|
+
**PII/secrets scan — always, regardless of PRODUCT_FOLDER:**
|
|
138
|
+
```
|
|
139
|
+
Write outputs/.piicheck-tmp.txt containing: {Bug Title}\n{Expected Outcome}\n{Actual Outcome}\n{Steps to Reproduce}
|
|
140
|
+
```
|
|
141
|
+
```bash
|
|
142
|
+
node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs pii-scan --file outputs/.piicheck-tmp.txt
|
|
143
|
+
```
|
|
144
|
+
Delete both temp files once this bug is posted (Step 6). `CLEAN` → proceed silently.
|
|
145
|
+
`FLAGGED: ...` → show the flagged pattern types (never the raw
|
|
146
|
+
match) and ask once: `"This bug report may contain real {types}. Redact before posting, or
|
|
147
|
+
file as-is? (redact / as-is)"`. Apply the choice, then continue to Step 3.
|
|
148
|
+
|
|
149
|
+
---
|
|
150
|
+
|
|
113
151
|
## Step 3 — Display the formatted bug report
|
|
114
152
|
|
|
115
153
|
Show the bug report in this exact format before posting:
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: k6-framework-scaffold
|
|
3
|
+
description: Scaffold a ready-to-use k6 load-testing framework (Grafana Cloud) in a repo. Creates the k6-performance-tests/ tree — shared config/ + lib/, a protocol/ (API load) layer with api/flows/tests, and an OPTIONAL browser/ (UI) layer — with commented template files that testers then fill in. Use when someone asks to "set up k6", "create the k6 load testing structure/framework", "scaffold performance tests", or start load testing a new app.
|
|
4
|
+
compatibility: >
|
|
5
|
+
Standalone skill — not part of the qa-agent pipeline. The bundled scaffold.sh only writes
|
|
6
|
+
files; the k6 CLI (and a Grafana Cloud account for `k6 cloud run`) is only needed later,
|
|
7
|
+
when the tester actually runs the generated tests.
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# k6 Framework Scaffold
|
|
11
|
+
|
|
12
|
+
Generates a standard, reusable k6 performance-testing framework so a tester can start from a
|
|
13
|
+
ready structure and just fill in their app's endpoints, weights, and tokens.
|
|
14
|
+
|
|
15
|
+
## What it creates
|
|
16
|
+
|
|
17
|
+
```
|
|
18
|
+
k6-performance-tests/
|
|
19
|
+
├── config/ env.js · secrets.js · profiles.js (shared: URLs, tokens, load shapes)
|
|
20
|
+
├── lib/ http.js · rand.js · browser.js (shared helpers)
|
|
21
|
+
├── protocol/ api/{userTypeA,userTypeB}/ · flows/ · tests/{smoke,load,spike,breakpoint,endurance}/
|
|
22
|
+
│ ── API / HTTP load — the CORE, needed for every app
|
|
23
|
+
├── browser/ pages/ · flows/ · tests/
|
|
24
|
+
│ ── real-browser UX — OPTIONAL, only for UI-heavy apps (delete if unneeded)
|
|
25
|
+
└── README.md
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Design principle: **separate WHAT we test (api → flows) from HOW MUCH load (profiles → tests)**,
|
|
29
|
+
so the same endpoints/journeys are reused across every test type. All files are commented
|
|
30
|
+
templates with `TODO`s.
|
|
31
|
+
|
|
32
|
+
## How to run it
|
|
33
|
+
|
|
34
|
+
From the target repo root, run the bundled script:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
bash .claude/skills/k6-framework-scaffold/scaffold.sh
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
- It creates `k6-performance-tests/` in the current directory.
|
|
41
|
+
- It is **safe**: if `k6-performance-tests/` already exists it aborts (won't overwrite).
|
|
42
|
+
Pass `--force` to scaffold into an existing folder (only adds missing files; still never
|
|
43
|
+
overwrites an existing file).
|
|
44
|
+
|
|
45
|
+
After it runs, tell the tester the next steps:
|
|
46
|
+
1. `config/env.js` — set `BASE` (API URL), `UI_BASE`s, `PROJECT_ID`, scale numbers, test data.
|
|
47
|
+
2. `config/secrets.js` — paste auth tokens/credentials (git-ignored pattern).
|
|
48
|
+
3. `protocol/api/**` — replace the sample endpoints with the real ones (one function per call).
|
|
49
|
+
4. `protocol/flows/**` — set the weighted action table (feature-usage %) + thresholds (SLA).
|
|
50
|
+
5. Browser layer is **optional** — keep it only for UI-heavy apps; otherwise delete `browser/`.
|
|
51
|
+
6. Install k6, `k6 cloud login --token <token>`, then run from inside `k6-performance-tests/`:
|
|
52
|
+
`k6 run protocol/tests/smoke/example.smoke.js` (local) or `k6 cloud run ...`.
|
|
53
|
+
|
|
54
|
+
## Notes
|
|
55
|
+
- The script only writes files; it does not install k6 or run tests.
|
|
56
|
+
- Keep real tokens out of version control (the scaffold's `.gitignore` covers `secrets.local.*`).
|
|
57
|
+
- If the user wants the structure tailored (their user types, real endpoints from a Postman
|
|
58
|
+
collection), scaffold first, then edit the generated templates to match.
|