@roopesh.yadava/qa-pack 1.1.0 → 1.3.0

package/README.md

@@ -18,7 +18,9 @@ On install, the postinstall script copies all skills and commands into `.claude/
 npm update @roopesh.yadava/qa-pack
 ```
 
-Skill files are overwritten with the latest version. Your `CLAUDE.md`, `.mcp.json`, `.env`, and — critically — `product_context/` (your accumulated run history and known bugs) are **never touched**.
+Skill files are overwritten with the latest version. Your `CLAUDE.md`, `.mcp.json`, `.env`, and — critically — `product_context/` (your accumulated run history and known bugs) are **never touched**. The installed `SKILLS_CONTEXT.md` is stamped with the pack version, so you can always check which version a repo is running.
+
+**Upgrading from 1.0.x:** credentials used to live in `product_context/{PRODUCT}/context.md` — skills now read them from `.env` instead. Move your username/password/OTP into `.env` and delete them from any existing `context.md`.
 
 ## First-time setup
 
@@ -29,6 +31,12 @@ After installing:
    (plus `JIRA_BASE_URL` / `JIRA_EMAIL` / `JIRA_API_TOKEN` if you want bug screenshots attached).
 2. Open `CLAUDE.md` and fill in the non-secret project facts (Jira key, environment, auth method).
 
+`.claude/settings.local.json` (seeded once from `settings.local.json.example`) auto-approves
+Playwright MCP tool calls, since those run constantly during test execution and are scoped to
+the app under test. Every Atlassian/Jira MCP call — reading a card, filing a bug, commenting,
+transitioning — and any action needing your input (filing a bug, publishing a charter,
+overwriting product context) still prompts for confirmation.
+
 Then open the repo in Claude Code and run:
 
 ```
@@ -41,8 +49,13 @@ Every run follows the same structured flow:
 
 1. **Card input** — give a Jira card ID (`run qa PROJ-123`)
 2. **Product context** — saved context + the Jira card are loaded automatically
-3. **Gap questions** — you're asked only what isn't already known (one message)
-4. **Phase selection**:
+3. **Model recommendation** — a one-line tier suggestion based on the card's Jira issue type
+   (Story/Feature/Epic → default high-capability model, Task/Bug/Sub-task → a lighter model
+   like Haiku). Informational only — switch with `/model` if you want, or ignore it; set
+   `QA_MODEL_ROUTING=off` in `.env` to skip it. See `SKILLS_CONTEXT.md` for the opt-in
+   subagent-override pattern.
+4. **Gap questions** — you're asked only what isn't already known (one message)
+5. **Phase selection**:
    - **Phase 1 — Manual testing** (UI/Figma diff → browser tests → bugs → charter)
    - **Phase 2 — Write automation** (reuse audit → Gherkin → steps → POM → real run + self-heal)
    - **Phase 3 — Both** (manual first — its findings feed the automation)
@@ -51,24 +64,39 @@ Automation is **reuse-first**: existing `.feature` files and step definitions ar
 before any Gherkin is written, drafted steps must match existing ones (exact → parameterized
 → reworded) before a new step definition is allowed, and each run reports its reuse %.
 
-## Triggers
+## Commands to run skills
+
+Every skill is plain-English triggered — no slash command required — except `/qa-agent`
+and `/write-acceptance-criteria`, which are also available as explicit slash commands.
 
-| Say this | What happens |
+### QA Agent (orchestrator)
+
+| Command | What happens |
 |---|---|
-| `run qa PROJ-123` | Structured flow — context, questions, then phase menu |
-| `manual test PROJ-123` | Phase 1 — manual testing + bug filing |
-| `automate PROJ-123` | Phase 2 — reuse-first BDD automation |
-| `full QA PROJ-123` | Phase 3 — manual first, then automation |
-| `accessibility test PROJ-123` | WCAG 2.1 A/AA audit |
-| `ui test PROJ-123` | Live app vs Figma design diff |
-| `file a bug` | Files bug directly to Jira |
-| `/write-acceptance-criteria PROJ-123` | Generates AC, appends to Jira card |
+| `/qa-agent PROJ-123` or `run qa PROJ-123` | Loads context, asks only the missing gaps, then shows the phase menu |
+| `manual test PROJ-123` / `test PROJ-123` / `do QA on PROJ-123` | Phase 1 — manual testing + bug filing + charter |
+| `automate PROJ-123` / `write automation for PROJ-123` | Phase 2 — reuse-first BDD automation |
+| `full QA PROJ-123` / `run everything for PROJ-123` / `end to end PROJ-123` | Phase 3 — manual first, then automation |
+| `run qa PROJ-123 --reset-context` | Same as above but ignores saved product context and starts fresh (overwrites it on completion) |
+
+### Standalone skills (no card required, or card optional)
+
+| Command | Skill | What happens |
+|---|---|---|
+| `ui test` / `check design` / `compare with figma` | ui-test-figma | Live app vs Figma design diff |
+| `accessibility test PROJ-123` / `a11y test [URL]` | accessibility-testing | WCAG 2.1 A/AA audit, files bugs on the card if given |
+| `file a bug` / `log a bug` | bug-reporting | Interactive bug filing to Jira (new or existing card) |
+| `generate a test charter` / `create a charter` | test-charter | Publishes a Test Charter from a saved execution report |
+| `delete files` / `clean up outputs` | delete-files | Prompts to delete/keep files in `outputs/` |
+| `/write-acceptance-criteria PROJ-123` | write-acceptance-criteria | Generates AC, appends to the Jira card description |
+| `/impacted-tests` / `which tests are impacted by this pull` | impacted-tests | After pulling dev changes into a test branch, reports which Cucumber feature files are at risk — report-only, no card needed |
 
 ## What postinstall does
 
 | File | Behaviour |
 |---|---|
 | `.claude/skills/*/SKILL.md` + companion `.md` files | Always overwritten (versioned logic) |
+| `.claude/skills/SKILLS_CONTEXT.md` | Always overwritten + stamped with the installed pack version |
 | `.claude/commands/*.md` | Always overwritten |
 | `.claude/skills/qa-agent/product_context/**` | **Never touched** after first seed |
 | `.claude/settings.json` | Created once, never overwritten |
@@ -89,10 +117,14 @@ If you installed an older qa-pack version and already committed the skill files,
 ## Publishing (maintainers)
 
 ```bash
-# In qa-pack/ repo, bump version and push
-npm version patch   # or minor / major
-git add . && git commit -m "your message" && git push origin main
-# GitHub Actions publishes to npmjs automatically
+# In qa-pack/ repo — commit your changes first (npm version needs a clean tree)
+git add . && git commit -m "your message"
+
+# Bump: patch = fixes, minor = new/changed skill behavior, major = breaking
+npm version minor -m "release: v%s — short description"
+
+git push origin main --follow-tags
+# GitHub Actions publishes to npmjs automatically (triggers on package.json change on main)
 ```
 
 ## Requirements
@@ -100,3 +132,5 @@ git add . && git commit -m "your message" && git push origin main
 - Claude Code CLI
 - Node.js 18+
 - Atlassian MCP connected in Claude Code (for Jira)
+- Playwright MCP — configured automatically via the seeded `.mcp.json`
+- Python 3 — only if you enable the optional token tracking (`QA_TRACKING_DIR` in `.env`)

package/claude/commands/impacted-tests.md

@@ -0,0 +1,11 @@
+---
+name: impacted-tests
+description: Map the source changes a developer-branch pull brought in to the BDD/Cucumber feature files they can break, and print a report.
+argument-hint: "[base-branch-or-commit] [changed file paths...]"
+user-invocable: true
+---
+
+Invoke the `impacted-tests` skill.
+
+If the user provided a base branch/commit or explicit file paths as arguments, pass them
+directly into the skill so it skips diff auto-detection and uses those instead.

package/claude/skills/SKILLS_CONTEXT.md

@@ -210,6 +210,16 @@ Also check `token_usage_log.json` for `cache_read_input_tokens` per phase to see
 Target: `cache_efficiency_pct` ≥ 70 after the first run on a given product.
 First run will always be 0% (cold cache). Second and subsequent runs should cache the stable block (product context + skill instructions).
 
+## Model Routing (optional)
+
+`QA_MODEL_ROUTING` in `.env` controls how qa-agent responds to Jira issue type at Step 1c:
+
+| Value | Behavior |
+|-------|----------|
+| unset or `recommend` (default) | qa-agent prints a one-line model-tier recommendation (Story/Feature/Epic → high-capability, Task/Bug/Sub-task → lighter model) after fetching the card, then continues on whatever model is already active. Portable — depends on no special tooling. |
+| `off` | Skip the recommendation line entirely. |
+| `subagent` (advanced, opt-in — not wired up by the shipped skills) | For Claude Code setups where phase dispatch runs through a subagent/Task-style tool that accepts a `model` parameter: adapt qa-agent's Step 4 dispatch to pass `model: haiku` for Task/Bug and the default model for Story/Feature when invoking `manual-testing` / `automation`. This is a pattern to wire into your own environment's actual subagent tool — qa-pack itself doesn't assume one exists, since not every Claude Code setup qa-pack installs into exposes the same dispatch mechanism. |
+
 ## Artifact Locations
 
 | Artifact | Path |

package/claude/skills/impacted-tests/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: impacted-tests
+description: After pulling a developer branch into a test branch, analyze ONLY the src changes that pull brought in and map them to the BDD/Cucumber test files (feature files, step definitions, page objects) they can break, then print a report. Report-only - it never runs tests; the QA opens and runs the listed files manually. Repo-agnostic - it auto-discovers the test folder, framework, and diff base at runtime. Use when the user asks "which tests are impacted by this pull" or "map dev changes to tests". Usage - /impacted-tests [base-branch-or-commit] [changed file paths...]
+---
+
+# Impacted Tests Analyzer
+
+Map the source-code changes a developer-branch pull brought in to the BDD test files they can break, so QA can review/run those tests locally before CI. This skill is **repo-agnostic**: it discovers everything about the project at runtime. Do NOT assume any specific folder names, framework version, or paths — always run Step 0 first.
+
+## Token-efficiency rules (mandatory)
+
+- Work from the **diff**, never by reading whole source files. `git diff --name-only` / `-U0` gives everything needed.
+- Map with `grep -rl` / `grep -rn` (filenames + line numbers only); never `cat` a feature/step/page file in full. Read at most ±10 lines around a hit, and only when a mapping is ambiguous.
+- Batch all greps for one identifier list into a single command with `-E "id1|id2|..."`.
+- Never search `node_modules`, `dist`, `build`, `coverage*`, or other generated/vendor dirs.
+- If the diff spans more than ~30 src files, delegate the mapping (Step 3) to one `Explore` subagent (breadth: medium) with the identifier list; ask it to return only file paths + one-line reasons.
+
+## Step 0 — Discover the project (do this first, every run)
+
+Detect these facts by inspection; keep the results in mind for later steps. Do NOT hardcode.
+
+1. **Test root**: the folder holding BDD tests. Find it, don't assume:
+   ```bash
+   ls -d test tests e2e cypress 2>/dev/null
+   git ls-files '*.feature' | sed 's#/[^/]*$##' | sort -u | head
+   ```
+   The common ancestor of the `*.feature` files is the test root. Feature files, step definitions, and page objects all live under it (subfolder names vary per repo — discover them, e.g. `git ls-files "$TEST_ROOT/**" | sed 's#[^/]*$##' | sort -u`).
+2. **Framework + run command**: read `package.json` scripts and any cucumber config (`cucumber.js`, `cucumber.cjs`, `cucumber.json`, `.cucumberrc*`). Derive the single-feature run command from the test script or CI script rather than inventing one. If none is discoverable, fall back to `npx cucumber-js <feature-path>`. (This is only for the manual-run footer in the report — the skill never runs anything itself.)
+3. **Source root**: the app code the tests exercise — usually `src/`, else the top-level dir(s) that are neither the test root nor config/vendor. Confirm with `git ls-files | grep -vE "^($TEST_ROOT|node_modules|dist)" | ...`.
+
+If there are no `*.feature` files anywhere, tell the user this skill targets BDD/Cucumber repos and stop.
+
+## Step 1 — Collect the diff from the most recent pull INTO the current branch
+
+The input is exactly what the last pull brought into **this** branch — no need to know or name the developer branch. When you `git pull` (or merge) another branch into your current branch, git records a merge commit whose second parent is the pulled tip; `$MERGE^1..$MERGE` is precisely "what that pull added on top of my branch". That is the primary and preferred signal:
+
+```bash
+# Primary: the most recent merge commit on the CURRENT branch = the most recent pull into it
+MERGE=$(git log --merges -1 --format=%H)
+git log -1 --format='%s  (merged in: %p)' $MERGE   # sanity check — this is the pull you just did
+git diff --name-status $MERGE^1 $MERGE -- <SRC_ROOT>
+DIFF_RANGE="$MERGE^1 $MERGE"
+```
+Fallbacks, in order (only if there is no merge commit — e.g. a fast-forward pull, or you pulled before creating any commit):
+- User passed a base ref/commit as the first argument: `git diff --name-status <arg>...HEAD -- <SRC_ROOT>` — use this whenever the auto-detected pull is wrong.
+- Otherwise diff against the current branch's upstream/tracking ref if set (`@{u}`), else against whichever of `origin/develop-temp`, `origin/develop`, `origin/main`, `origin/master` exists: `git diff --name-status HEAD...<ref> -- <SRC_ROOT>`.
+- User passed explicit file paths or a pasted diff: use those directly.
+
+If the src diff is empty everywhere, tell the user no pulled source changes were found (ask when they last pulled, or to pass the base ref explicitly) and stop — do NOT fall back to analyzing their own test edits.
+
+Scope: only files under `<SRC_ROOT>` are analysis input. Ignore unit-test files in the diff (`*.test.*`, `*.spec.*`, `__tests__/`) — they are not BDD-flow relevant. The QA's own changed test files are NOT impact findings (mention them, if at all, in one closing line).
+
+## Step 2 — Extract impact identifiers (from the diff only)
+
+Build a small identifier list without opening files (use `$DIFF_RANGE` from Step 1):
+
+```bash
+git diff -U0 $DIFF_RANGE -- <SRC_ROOT> | grep -oE 'data-testid="[^"]+"' | sort -u                      # changed testids
+git diff -U0 $DIFF_RANGE -- <SRC_ROOT> | grep -E '^[+-]' | grep -oE '"[A-Z][A-Za-z0-9 /&-]{3,40}"' | sort -u  # added/removed UI strings
+```
+Identifiers = (a) changed `data-testid`/locator attribute values, (b) component/page basenames from the changed file paths, (c) route/path strings in the diff, (d) user-facing label/heading/button/message strings added or removed. `data-testid` and UI strings are the strongest signals; include **removed** ones too — a deleted locator is exactly what breaks a test.
+
+## Step 3 — Map identifiers to test files, then down to scenarios
+
+Chain cheapest-first, filenames only, all under the discovered `<TEST_ROOT>`:
+
+1. **Filename match**: identifier keywords vs feature/step filenames (`git ls-files "$TEST_ROOT/**/*.feature" "$TEST_ROOT/**/*.[jt]s" | grep -iE "kw1|kw2"`).
+2. **Content match**: `grep -rilE "id1|id2|LabelText" "$TEST_ROOT" --include="*.feature" --include="*.js" --include="*.ts"`.
+3. **Close the chain**: for a hit in a step-definition or page-object that has no same-named feature, grep its distinctive step text into the feature files to find the owning feature(s).
+4. **Resolve to scenarios**: for each impacted feature file, `grep -nE "Scenario:|Example:|Rule:|<step text>" <feature>` to get the affected scenario title(s) and line number(s). The report cites the **feature file + scenario + line**, because testers navigate by scenario, not by step-definition code.
+
+Risk level (two only): 🔴 **Will fail** = a locator/step the test uses was removed or renamed; 🟡 **At risk** = shares a changed screen/component/steps but locators likely survive.
+
+## Step 4 — Output the report
+
+The report IS the deliverable. In this exact order:
+
+1. FIRST save it to `outputs/impacted-tests-report.md` with the Write tool (create `outputs/` if needed) so it survives the session.
+2. THEN print it as the **final text message of the turn** — no tool calls after it, printed exactly ONCE, beginning with `## Impacted tests report`. Never print it before a tool call and repeat it after.
+
+Format (per-line, not a table — tables wrap badly in terminals):
+
+```
+## Impacted tests report
+
+Analyzed <N> source files pulled from <ref> (merge <shorthash>).
+
+**Impacted test files (<N>) — highest risk first:**
+
+1. 🔴 <TEST_ROOT>/path/to/Some.feature
+   **Will fail** — Example: "<scenario title>" (<TEST_ROOT>/path/to/Some.feature:34) — <one sentence, in tester/UI terms, of what the developer changed>.
+
+2. 🟡 <TEST_ROOT>/path/to/Another.feature
+   At risk — Rule: "<rule title>" (<TEST_ROOT>/path/to/Another.feature:41) — <one sentence>.
+
+**Coverage gaps (no test exercises these yet):**
+- <new dev functionality with no scenario> — needs a new scenario.
+
+**To run any of these manually** (start your app first):
+<discovered single-feature run command> <feature-path>
+```
+
+Rules for each entry:
+- Risk marker first, then ONE sentence citing the affected scenario by its `Scenario:`/`Example:`/`Rule:` title with a clickable `feature-path:line`, describing the change in **UI/tester terms** ("the dropdown was replaced with an input + Verify button").
+- Do NOT reference step-definition filenames, page objects, or source component filenames in the reasons — testers read feature files, not code. Source filenames may appear only in the coverage-gaps section when there is no feature to point to.
+- **Hyperlink rule**: write feature paths as bare repo-relative paths in plain text (with `:line`) — NOT as `[label](path)` markdown links. Terminals auto-detect plain paths and make them ctrl+clickable; a markdown label hides the path and breaks that.
+- No confidence column, no per-source-file table, no separate flat path list — the numbered paths ARE the list.
+
+## Step 5 — Stop
+
+The report is the deliverable. Do NOT run any test, do NOT ask whether to run them — the QA reviews the report and runs files manually using the footer command. End the turn after printing the report.

package/claude/skills/manual-testing/SKILL.md

@@ -27,6 +27,14 @@ user-invocable: true
 **After the ONE allowed login snapshot — never call `browser_snapshot()` again in this run.**
 Use `browser_evaluate` with a specific CSS selector or JS expression for all DOM inspection.
 
+## Test Data & Entity Selection Rules
+
+| Situation | Forbidden | Do instead |
+|-----------|-----------|------------|
+| A test step needs a value you don't have — a user record, a transaction PIN, a one-time user code | Calling a user-list endpoint or any other app API to fetch/guess it | Ask the user once, in the normal gap-question flow ("This test needs a valid user code — please provide one, or a test user's ID"), or pull it from `.env` / product context if already recorded there |
+| A test requires two specifically related entities (e.g. a parent user + a specific child user, an account + a specific linked card) and the AC/COS doesn't name the exact pair | Picking one at random and proceeding as if it passed | Ask the user to specify the pair once. If they can't provide it inline, mark that test `🔒 BLOCKED` with the note "entity relationship not specified — needs manual verification" and continue with the rest of the plan — don't stop the whole run over one test |
+| The AC/COS doesn't say how to reach the feature under test (menu path, URL, entry point) | Reading or grepping the `src/` tree to reverse-engineer the route | Ask once: "AC doesn't say how to reach {feature} — what's the navigation path (menu/page) to get there?" |
+
 ## Token Tracking
 
 Optional and silent — follow the **Token Tracking** pattern in `SKILLS_CONTEXT.md`
@@ -152,13 +160,19 @@ HINTS = {
 
 ### 3a — Generate Test Plan
 
+Before drafting test ideas, check each AC item for a stated entry point (menu path, URL,
+button/link name). Per the **Test Data & Entity Selection Rules** above, do not scan `src/`
+to infer a missing one — collect a single navigation question per missing entry point and
+fold it into the confirmation prompt below instead of guessing.
+
 From the AC, generate numbered test ideas:
 - 1–3 tests per AC item
 - 2+ negative/edge case tests
 - 1+ error state test
 
 Show as a compact table (T-01, T-02 ... with name and expected outcome).
-Ask: `"Ready to run these tests? (yes / no or edit)"`
+Ask: `"Ready to run these tests? (yes / no or edit)"` — include any missing navigation
+questions in this same message.
 Proceed on confirmation.
 
 ### 3b — Setup Browser + Capture Login Selectors

package/claude/skills/qa-agent/SKILL.md

@@ -95,6 +95,23 @@ Criteria / COS**, **Figma link**, **issue type**, **project name** (for Step 6 f
 If the description exceeds ~800 words, keep only Title + AC/Given-When-Then blocks +
 Figma links; discard prose, comment threads, embedded images. Do not mention the truncation.
 
+### 1c — Model Recommendation
+
+Using the issue type fetched in 1b, print one line — informational only, never blocking:
+
+| Issue type | Recommendation |
+|---|---|
+| Story / Feature / Epic | high-capability model (current default) |
+| Task / Bug / Sub-task | lighter model (e.g. Haiku) |
+
+> "Issue type: {type} → {tier} recommended for this run. Switch now with `/model` if you'd
+> like, or continue on the current model."
+
+Skip this line entirely if `.env` sets `QA_MODEL_ROUTING=off`. This is a recommendation
+only — qa-pack has no way to force a mid-run model switch, so proceed on whatever model is
+active regardless of the user's choice. See `SKILLS_CONTEXT.md` → "Model Routing" for the
+opt-in subagent-override pattern some environments can wire up instead.
+
 ---
 
 ## Step 2 — Ask Relevant Questions (gaps only)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@roopesh.yadava/qa-pack",
-  "version": "1.1.0",
+  "version": "1.3.0",
   "description": "AI-powered QA agent skills for Claude Code — manual testing, BDD automation, accessibility, UI/Figma diff, bug reporting",
   "scripts": {
     "postinstall": "node bin/postinstall.js"

package/templates/.env.example

@@ -20,3 +20,7 @@ JIRA_API_TOKEN=
 # When unset or the scripts are missing, skills skip all tracking silently.
 # QA_TRACKING_DIR=
 # QA_TRACKING_MODEL=
+
+# ── Optional: model-tier recommendation for qa-agent (see SKILLS_CONTEXT.md) ──
+# recommend (default) | off | subagent (advanced — see "Model Routing" in SKILLS_CONTEXT.md)
+# QA_MODEL_ROUTING=recommend

package/templates/settings.local.json.example

@@ -5,11 +5,7 @@
       "Bash(npx cucumber-js *)",
       "Bash(echo \"Exit: $?\")",
       "Bash(playwright --version)",
-      "mcp__claude_ai_Atlassian__getAccessibleAtlassianResources",
-      "mcp__claude_ai_Atlassian__getJiraIssue",
-      "mcp__claude_ai_Atlassian__createJiraIssue",
-      "mcp__claude_ai_Atlassian__createIssueLink",
-      "mcp__claude_ai_Atlassian__addCommentToJiraIssue"
+      "mcp__playwright"
     ]
   },
   "enableAllProjectMcpServers": true,