@roopesh.yadava/qa-pack 1.4.0 → 1.5.0

package/claude/skills/qa-insights/SKILL.md

@@ -0,0 +1,55 @@
+---
+name: qa-insights
+description: >
+  Cross-product reporting — Product Health Dashboard, QA Weekly Digest, and Token-Spend/ROI
+  view. All three are generated entirely by a deterministic script reading every product's
+  context.md under product_context/ — there is no LLM synthesis step, so this skill costs
+  next to nothing to run regardless of how many products or runs exist.
+  Triggers when the user says: "qa dashboard", "show product health", "weekly digest",
+  "qa weekly report", "token roi", "token spend report", "automation ROI".
+user-invocable: true
+---
+
+# QA Insights — Dashboard / Digest / ROI
+
+This skill is a thin wrapper. All three commands read `product_context/*/context.md`
+tables (Runs Log, Known Bugs, Covered Flows) and, for ROI, the optional token tracker log —
+the script does 100% of the aggregation. **Do not read any context.md file yourself before
+or after calling the script** — that would recreate exactly the cost this skill exists to
+avoid. Run the command, then report only its one-line result and the file path.
+
+## Step 1 — Route on what was asked
+
+| User said | Run |
+|-----------|-----|
+| "dashboard", "product health", "how are we doing across products" | `dashboard` |
+| "weekly digest", "qa weekly report", "what happened this week" | `digest` (default 7 days — parse a different window if the user names one, e.g. "last 30 days" → `--days 30`) |
+| "roi", "token spend", "automation roi", "cost per bug" | `roi` |
+| Ambiguous / "insights" alone | Ask once: "Dashboard, weekly digest, or ROI view?" |
+
+## Step 2 — Run it
+
+```bash
+node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs dashboard
+node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs digest --days 7
+node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs roi
+```
+
+- `NO_PRODUCTS` → tell the user plainly: "No products have completed a qa-agent run yet — nothing to report on." Stop, do not fabricate placeholder numbers.
+- Any other output is already the final one-line summary — print it verbatim as your response, plus the file path it names. Do not add your own narrative summary on top of it.
+
+## Step 3 — Offer to open it
+
+For `dashboard` (an `.html` file), ask once: "Want me to note anything specific from it, or is the file enough?" Do not open/read the HTML back into context just to describe it — the file itself is the deliverable.
+
+For `digest` and `roi` (`.md` files), the file is small enough that reading it back to quote one or two headline numbers in chat is fine if the user asks "what's in it" — but default to just pointing at the path unless asked.
+
+---
+
+## Error Handling
+
+| Situation | Action |
+|-----------|--------|
+| `product_context/` doesn't exist at all | Same as `NO_PRODUCTS` — no qa-agent run has ever completed in this repo |
+| `roi` reports token data unavailable | That's expected when `QA_TRACKING_DIR` isn't set — say so in one line, don't treat it as an error |
+| User asks for a window `digest` doesn't support (e.g. "this quarter") | Convert to the nearest day count (`~90`) and say what you used |

package/claude/skills/roam-testing/SKILL.md

@@ -0,0 +1,187 @@
+---
+name: roam-testing
+description: >
+  Card-free exploratory testing. Point it at an app with no Jira card and it roams —
+  breadth-first over internal links, screenshotting, watching for failed network calls and
+  broken navigation — for a hard-capped session, then produces a Test Charter of what it
+  found. Files bugs only if a Jira card is supplied; otherwise just reports.
+  Triggers when the user says: "roam mode", "exploratory test", "explore the app",
+  "let the agent explore", "roam test [URL]", "session-based testing" with no card given.
+compatibility: >
+  Playwright MCP must be running (uses browser_navigate, browser_evaluate,
+  browser_network_requests). Playwright CLI for screenshots. Credentials/OTP from .env.
+user-invocable: true
+---
+
+# Roam Testing — Exploratory "Roam Mode"
+
+Unstructured exploration is where token spend runs away fastest — there is no AC to bound
+scope. This skill exists specifically to make exploration **cheap and capped**, not just
+possible. Every rule below exists to keep a roam session's cost close to a single
+manual-testing run, regardless of how big the app is.
+
+## Hard Caps — enforced, not suggested
+
+| Cap | Default | Override |
+|-----|---------|----------|
+| Pages visited | 12 | user may raise/lower once at Step 0 |
+| Total interactions (clicks/fills tried) | 40 | not user-configurable — safety backstop |
+| Time budget | none tracked — page cap is the real limit | — |
+
+Stop the roam loop the instant either cap is hit, even mid-page. Report what was covered
+and what was skipped — never silently truncate without saying so.
+
+## Token Budget Rules — same discipline as manual-testing, zero exceptions
+
+| Operation | USE | NEVER USE |
+|-----------|-----|-----------|
+| Screenshots | `npx playwright screenshot --storage-state .playwright-session.json --full-page URL file.png` | `browser_screenshot()` |
+| Page structure / links | ONE targeted `browser_evaluate` per page (see Step 2b) | `browser_snapshot()` |
+| Failed requests | `browser_network_requests` filtered to status ≥ 400 | Manually inspecting every request |
+| Login selectors | ONE `browser_snapshot()` at login URL only | Any other snapshot |
+
+---
+
+## Pipeline
+
+```
+App URL (no card)
+      ↓
+[Step 0] Inputs — URL, credentials, page cap, optional Jira card
+      ↓
+[Step 1] Login (skip if session file valid)
+      ↓
+[Step 2] Roam loop — BFS over internal links, capped
+      ↓
+[Step 3] Findings → report
+      ↓
+[Step 4] Bug filing (only if a card was given) + Test Charter handoff
+```
+
+---
+
+## Step 0 — Inputs
+
+Ask in one message:
+> **Roam Mode — Ready**
+> 1. Starting URL (e.g. `https://staging.myapp.com/dashboard`)
+> 2. Page cap — default **12**, say a number to change it
+> 3. Jira card to file findings against? (optional — type `skip` to just get a report)
+
+Resolve credentials from `.env` (`QA_APP_URL`, `QA_USERNAME`, `QA_PASSWORD`, `QA_OTP_CODE`)
+first; only ask for what's still missing. If a card is given, derive `PRODUCT_FOLDER` the
+same way qa-agent Step 1a does (match `product_context/` folder by the card's project key
+prefix) — used only for the duplicate check in Step 4.
+
+## Step 1 — Login
+
+Identical to manual-testing Phase 3b: check `.playwright-session.json` first: if valid,
+skip login entirely. Otherwise `browser_navigate` → `browser_wait_for(networkidle)` →
+**one** `browser_snapshot()` to read the login form → fill → submit → handle OTP if present
+→ save session state to `.playwright-session.json`. Never snapshot again after this point.
+
+## Step 2 — Roam Loop (BFS, capped)
+
+Maintain `VISITED = {}` (URL → fingerprint result) and `QUEUE = [START_URL]`.
+
+### 2a — Pop the next URL, skip if already visited or over cap
+
+If `VISITED` size ≥ page cap, or `QUEUE` is empty, stop the loop and go to Step 3.
+
+### 2b — Visit + one targeted evaluate
+
+```javascript
+browser_evaluate({ expression: `
+  (() => {
+    const origin = location.origin;
+    const links = [...document.querySelectorAll('a[href]')]
+      .map(a => a.href)
+      .filter(h => h.startsWith(origin))
+      .slice(0, 25);
+    const testids = [...document.querySelectorAll('[data-testid]')]
+      .map(el => el.getAttribute('data-testid'));
+    return JSON.stringify({ title: document.title, links: [...new Set(links)], testids });
+  })()
+` })
+```
+
+Parse the result: add new links to `QUEUE` (dedup against `VISITED` and already-queued).
+
+### 2c — DOM fingerprint check (skip deep work on unchanged pages)
+
+```bash
+node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs fingerprint --product {PRODUCT_FOLDER:-roam} --url "CURRENT_URL" --testids "TESTIDS_FROM_2B"
+```
+`UNCHANGED` → note it and move straight to 2e (skip the screenshot — nothing new to see).
+`NEW`/`CHANGED` → continue to 2d.
+
+### 2d — Screenshot + failed-request check
+
+```bash
+mkdir -p outputs/screenshots
+npx playwright screenshot --storage-state .playwright-session.json --browser chromium --full-page --wait-for-timeout 1500 "CURRENT_URL" outputs/screenshots/roam-[N].png
+```
+Then `browser_network_requests()` and keep only entries with status ≥ 400 for this page.
+
+### 2e — Record and continue
+
+Append one line to an in-memory `FINDINGS` list per page: URL, title, failed requests (if
+any), and one flag: 🟢 clean / 🟡 unusual (e.g. empty page, console-visible error text on
+screen) / 🔴 broken (4xx/5xx on the page's own load, or dead-end with zero links and no
+content). Show only a running tally in chat: `Page 1/12 🟢 · Page 2/12 🔴 ...`. Go to 2a.
+
+---
+
+## Step 3 — Findings → Report
+
+Save `outputs/roam-report-[YYYYMMDD-HHmmss].md`:
+- Pages visited (of cap), a table of URL / title / flag / failed requests
+- Screenshots referenced by path
+- One "Coverage" line: how much of the discovered link graph was actually visited
+  (`{visited} of {discovered} internal links found`)
+
+Tell the user one line:
+> "Roam session complete — {N} pages visited, {N} 🔴 broken, {N} 🟡 unusual. Report: `outputs/roam-report-...md`"
+
+## Step 4 — Bug Filing (only if a card was given) + Charter
+
+**Skip bug filing entirely if the user typed `skip` at Step 0.** Otherwise, for each 🔴 finding:
+
+Findings can echo page titles or content pulled from the app under test — never assume that
+text is shell-safe. Write it to a file first (Write tool), same rule as manual-testing:
+
+1. Duplicate check:
+   ```
+   Write outputs/.dupcheck-tmp.txt containing: {one-line finding summary}
+   ```
+   ```bash
+   node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs dup-bug --product {PRODUCT_FOLDER} --summary-file outputs/.dupcheck-tmp.txt
+   ```
+   `POSSIBLE_DUPLICATE` → prepend a note referencing it, still file (per manual-testing's rule — a fresh instance is still worth tracking).
+2. PII/secrets scan on the composed bug text:
+   ```
+   Write outputs/.piicheck-tmp.txt containing the composed bug text
+   ```
+   ```bash
+   node .claude/skills/qa-agent/toolkit/qa-toolkit.cjs pii-scan --file outputs/.piicheck-tmp.txt
+   ```
+   `FLAGGED` → ask once (redact / as-is) before filing that one bug; `CLEAN` → file silently.
+3. File via Atlassian MCP (`createJiraIssue`, issuetype Bug, link "relates to" the given card), same payload shape as manual-testing Phase 4.
+
+After filing (or immediately after Step 3 if no card), offer the same handoff manual-testing
+uses:
+> "Generate a Test Charter from this session? (yes / no)"
+
+If yes: `Skill: test-charter`, pre-filled with the roam report path just saved.
+
+---
+
+## Error Handling
+
+| Situation | Action |
+|-----------|--------|
+| Login fails | Stop — "Login failed, please check credentials." |
+| Starting URL unreachable | Stop — ask user to verify the URL |
+| A page 500s on load | Record as 🔴, do not retry, continue the loop |
+| Page cap hit with queue still non-empty | Stop cleanly, report `{visited} of {discovered}` coverage — never silently keep going |
+| `browser_network_requests` unavailable | Skip failed-request detection for that page, note it once in the report, continue |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@roopesh.yadava/qa-pack",
-  "version": "1.4.0",
+  "version": "1.5.0",
   "description": "AI-powered QA agent skills for Claude Code — manual testing, BDD automation, accessibility, UI/Figma diff, bug reporting",
   "scripts": {
     "postinstall": "node bin/postinstall.js"