@kensaurus/skills 0.0.0-stage → 2.0.0

package/skills/test-qa/SKILL.md

@@ -0,0 +1,364 @@
+---
+name: test-qa
+description: >
+  Web-app CRUD and story QA when no project-specific skill applies. Use when
+  "QA the app", "test CRUD", or "smoke test". Native → mobile-emulator-test.
+  Guest vs logged in → test-exploratory. Pixel diffs → test-visual-regression.
+license: MIT
+---
+
+# test-qa — Story-driven CRUD, not a monkey test
+
+**Degree of freedom: MIXED** — story generation and severity `[HIGH freedom]`;
+session, anti-stall, and CRUD verify-after-refresh `[LOW freedom — run exactly]`.
+Driver is **playwright-cli**, never Playwright MCP. Guest-vs-authed wander is
+`test-exploratory` — do not reclaim it.
+
+Full QA of a live webapp as a senior QA engineer preparing a production
+release: user-story-driven coverage, CRUD, data-pipeline integrity, UX, and
+edge cases. This is **not** a page-navigation monkey test.
+Your turn ends after Phase 5 with every discovered route crawled and every CRUD entity proven after refresh — not after the crawl with CRUD still owed, and not with an auth-required page silently skipped instead of marked BLOCKED.
+
+Read `protocol-browser-anti-stall` before any browser step (Rule 0: manual &
+headed). Also `references/playwright-session-coordination.md` in that folder.
+
+## This skill vs neighbors
+
+| Skill | Owns |
+|:------|:-----|
+| **test-qa** (this) | Story-driven CRUD + smoke; persist after refresh (+ DB) |
+| `test-exploratory` | Unscripted wander + **guest vs authed diff** |
+| `test-playwright` | This-diff PDCA + fix-as-you-go |
+| `test-red-team` | Hostile feature×dimension matrix |
+| `test-visual-regression` | Pixel baselines |
+| `plan-stub-checker` | Dead buttons / fake components (plan) |
+
+## How to reason (every finding)
+
+1. **Observe** — screenshot + console + network (+ DB if available)
+2. **Interpret** — persist fail, dead control, UX defect, or expected empty?
+3. **Classify** — PASS / FAIL / BLOCKED (auth) — never "sort of works"
+4. **Severity** — Critical (blocks ship) / Major (bad UX) / Minor (polish)
+
+## Worked example
+
+> **Observe:** POST /api/items 201; list shows "QA-TEST-Widget"; after hard
+> `goto` the row is gone. Console clean.
+> **Interpret:** optimistic UI; write never persisted.
+> **Classify:** FAIL — data pipeline (ghost create).
+> **Severity:** Critical — CRUD not verified E2E.
+
+## Critical rules
+
+- **Manual & headed, never scripted.** One `playwright-cli` action at a time.
+  `eval` / `run-code` inspection-only. No `*.spec.ts`, no `npx playwright test`.
+- **User in the chair, engineer in the head.** First-open navigation; inspect
+  like you know the stack.
+- **Every mutation verified E2E.** Create/update/delete is untested until the
+  change survives a refresh **and** (if DB access exists) the row matches.
+- **Evidence for every finding.** Screenshot + console + network + repro.
+- **Clean up.** Delete `QA-TEST-` data. Leave auth session intact.
+- **No invented routes.** Discover pages/entities from source first.
+
+---
+
+## Phase 0: Codebase Discovery  [HIGH freedom]
+
+### 0a. Detect Tech Stack
+
+Read the dependency manifest (`package.json`, `requirements.txt`,
+`pyproject.toml`, `Cargo.toml`, `go.mod`). Record framework, UI lib, auth,
+DB/ORM, state, CSS, and `scripts.dev` port (3000 / 3001 / 5173 / 5174).
+
+### 0b. Discover Routes
+
+| Framework | Scan Pattern |
+|-----------|-------------|
+| Next.js App Router | `app/**/page.tsx` or `app/**/page.js` |
+| Next.js Pages | `pages/**/*.tsx` (excluding `_app`, `_document`) |
+| Remix | `app/routes/**/*.tsx` |
+| SvelteKit | `src/routes/**/+page.svelte` |
+| Nuxt | `pages/**/*.vue` |
+| React Router (SPA) | Grep for `<Route` or `createBrowserRouter` |
+| Django | `urls.py` files |
+| Rails | `config/routes.rb` |
+
+For each route: path, dynamic segments, layout nesting, auth guard.
+
+### 0c. Discover Data Model
+
+| Source | Where |
+|--------|-------|
+| Supabase migrations | `supabase/migrations/*.sql` — tables, RLS |
+| Prisma schema | `prisma/schema.prisma` |
+| Drizzle schema | `drizzle/schema.ts` or `src/db/schema.ts` |
+| TypeScript types | `types/*.ts`, `**/types.ts` |
+| API routes | `app/api/**` — CRUD endpoints |
+| Feature files | `features/*/` — services, hooks, components |
+
+For each entity: name, key fields, CRUD capabilities, relationships.
+
+### 0d. Discover Auth Pattern
+
+Provider config (`createClient`, `NextAuth`, `ClerkProvider`, `Auth0Provider`);
+login (`signIn` / `login` / `authenticate`); test credentials (`.env.local`,
+`.env.test`, `.env.example`, README); protected routes (middleware, guards).
+
+### 0e. Read Feature Documentation
+
+Feature-directory READMEs (`@_*-README.md`, `docs/`) for expected behavior.
+
+### 0f. Record Discovery Results
+
+```
+APP DISCOVERY:
+- Framework: [name + version]
+- Dev server: http://localhost:[port]
+- Auth: [provider + method]
+- Test account: [email / password, or "none found — ask user"]
+- Routes discovered: [count]
+ - Public: [list]
+ - Auth-required: [list]
+ - Dynamic: [list with param patterns]
+- Data entities: [list with CRUD capabilities]
+- API endpoints: [count]
+```
+
+---
+
+## Phase 1: Environment Verification  [LOW freedom — session exact]
+
+### 1a. Verify Dev Server Running
+
+Check the terminals folder for `npm run dev` / `pnpm dev` / `next dev`. If
+none, tell the user and stop.
+
+### 1b. Load the app (own your session)
+
+1. Session name `-s=qa-<app>`; never reuse another agent's name.
+2. Isolated browser — add `--persistent --profile` if login is required:
+
+```bash
+PW="npx --yes @playwright/cli@latest"
+$PW -s=qa-<app> open --headed http://localhost:3000
+sleep 2
+$PW -s=qa-<app> snapshot
+$PW -s=qa-<app> screenshot --filename .playwright-mcp/qa-baseline.png
+$PW -s=qa-<app> console
+$PW -s=qa-<app> requests
+```
+
+Blank after 3 incremental waits (6s) → blocker.
+
+### 1c. Authenticate (reuse session — do not re-login every run)
+
+Follow `protocol-browser-anti-stall/references/playwright-session-coordination.md`.
+
+1. Hit a **protected route** — if already signed in, skip to 1d.
+2. `state-load .playwright-mcp/auth/<host>.json` if it exists (or reuse the
+   `--persistent --profile` directory).
+3. If still logged out:
+   - Email/password: credentials from `.env.test` / README (never paste secrets).
+   - **OAuth / SSO**: sign in in the visible window; incremental snapshots;
+     then `state-save .playwright-mcp/auth/<host>.json`.
+   - **Google** cannot sign in from a Playwright-launched browser — use the
+     one-time real-Chrome profile in the coordination reference.
+4. Verify auth (avatar, dashboard, protected pages).
+5. **Do not log out** at cleanup unless testing logout.
+
+If auth is impossible, mark auth-required pages BLOCKED and test public only.
+
+### 1d. Capture Baseline
+
+```
+BASELINE:
+- URL: [current URL]
+- Console errors: [count — list if any]
+- Network failures: [count — list if any]
+- Visible content: [brief description]
+- Screenshot: [reference]
+```
+
+---
+
+## Phase 2: Intelligent Page Crawl  [HIGH freedom]
+
+For EVERY route from Phase 0b:
+
+### 2a. Navigate and Capture
+
+`goto` → anti-stall (2s → snapshot → verify) → `screenshot` → `console` →
+`requests`.
+
+### 2b. Classify the Page
+
+| Classification | Signals |
+|---------------|---------|
+| **CRUD page** | Forms, edit/delete, data tables |
+| **Display page** | Data, no mutation (dashboards, profiles) |
+| **Settings page** | Toggles, selects, save for preferences |
+| **Auth page** | Login, register, forgot password |
+| **Static page** | About, terms, privacy |
+| **Navigation hub** | Links to children (home, index) |
+
+### 2c. Detect findings during crawl
+
+| Check | What to Look For |
+|-------|-----------------|
+| **Dead page** | 404, error boundary, blank |
+| **Console errors** | JS / React errors, failed assertions |
+| **Network failures** | 4xx/5xx, CORS, timeouts |
+| **Missing content** | "undefined", "null", "NaN", "[object Object]" |
+| **Mock data** | "Lorem ipsum", "TODO", "test", "example@" |
+| **Dead buttons** | No onClick / navigate nowhere (snapshot) |
+| **Missing metadata** | Empty or generic `document.title` |
+| **Loading stuck** | Spinner/skeleton after 6s |
+| **Empty state** | No data AND no helpful empty message |
+| **Broken images** | Missing/broken `src` or error fallback |
+| **Overflow** | Text/elements past their containers |
+
+### 2d. Build Feature Map
+
+```
+FEATURE MAP:
+- CRUD pages: [entity + operations]
+- Forms found: [page + purpose]
+- Data displays: [tables, lists, cards, charts]
+- Interactive elements: [buttons, toggles, dropdowns per page]
+- Search/filter: [pages]
+- Settings: [configurable preferences]
+- Dead buttons found: [page + element]
+- Pages with errors: [list]
+```
+
+---
+
+## Phase 3: Dynamic User Story Generation  [HIGH freedom]
+
+Stories come from the feature map, not a canned list.
+
+| Cat | Story | Drive |
+|-----|-------|-------|
+| A | First impression | Home: value prop in 3s, CTA, guest-reachable nav, trust |
+| B | Core journey | Primary purpose from Phase 0, start to finish |
+| C | CRUD per entity | **Phase 4** — verify persist after refresh (+ DB) |
+| D | Navigation | Every nav link, back/forward, breadcrumbs, no dead ends |
+| E | Search/filter | Known hit, empty miss + helpful empty, filters |
+| F | Errors | Empty required, invalid input, `/nonexistent` 404, network fail |
+| G | Settings | Change, navigate away, come back — preference persisted |
+
+---
+
+## Phase 4: CRUD Testing  [LOW freedom — verify after refresh]
+
+For each CRUD-capable entity, run the lifecycle.
+
+### 4a. Create
+
+1. Open the creation form; identify fields via `snapshot`
+2. Fill realistic data: text `QA-TEST-[field]-[timestamp]`; domain-sane
+   numbers/dates; valid select; non-default toggle
+3. `screenshot` before submit
+4. Submit; `requests` must be 2xx; `snapshot` + `screenshot` for success
+   feedback (toast, redirect, confirmation)
+
+**Record** the item's ID, name, and URL.
+
+### 4b. Read
+
+List view → find the item → displayed fields match → detail view (if any)
+matches → `screenshot`.
+
+### 4c. Update
+
+Edit 1–2 fields → `screenshot` before save → save → `requests` 2xx → UI
+updates → hard-refresh (`goto` same URL) → changes still there.
+
+### 4d. Delete
+
+`screenshot` before → delete (+ confirm if any) → `requests` 2xx → gone from
+list → detail URL is 404 or redirect.
+
+### 4e. Validation Testing
+
+1. **Empty submission** → inline validation
+2. **Invalid data** — wrong format, text in number field → specific errors
+3. **Boundary** — 500+ chars, zero, negative, past dates
+4. **Special characters** — `<script>alert('xss')</script>`, `'; DROP TABLE`,
+   emoji, Unicode
+5. **Duplicate** — create the same item twice → duplicate handling
+
+### 4f. Data Pipeline Verification
+
+After each mutation:
+
+1. **Network** — `requests`: call made? status?
+2. **Response** — body has expected data?
+3. **UI** — reflects mutation without a manual refresh?
+4. **Refresh** — after `goto` same page, still visible?
+5. **DB** (if Supabase MCP or DB access):
+
+```json
+supabase:execute_sql
+{
+ "query": "SELECT * FROM <table> WHERE <identifying_column> LIKE 'QA-TEST-%' ORDER BY created_at DESC LIMIT 5"
+}
+```
+
+**Pipeline failures:** optimistic UI but API failed; stale cache; missing
+invalidation (item appears only after refresh); ghost data (deleted item
+returns); silent 4xx/5xx.
+
+---
+
+## Phase 5: UX Quality Audit  [HIGH freedom]
+
+Assess each page. Layout breakage → `audit-responsive`. Empty/error chrome →
+`audit-ui-states`. Do not re-own those audits.
+
+### 5a. Visual Quality
+
+| Check | How to Verify |
+|-------|--------------|
+| Consistent spacing | Screenshot — no irregular gaps or cramped areas |
+| Typography | No mixed sizes where they should match; no orphaned headings |
+| Truncation | Ellipsis where appropriate, not clipped |
+| Image loading | All images render; no broken-image icons |
+| Icons | All icons render (no missing squares) |
+| Dark mode | If supported: toggle; no white flashes |
+| Responsive | 1280 / 768 / 375 — layout adapts without breaking |
+
+### 5b. Interaction Quality
+
+| Check | How to Verify |
+|-------|--------------|
+| Dead buttons | Click every button. Does it do something? |
+| Form labels | Every input has a visible label or accessible name |
+| Loading states | Data fetch shows a loading indicator |
+| Success feedback | After mutations — toast, confirmation, or visual change |
+| Error feedback | Failures — helpful, visible |
+| Disabled states | Visually distinct; reason clear |
+| Focus management | After modal close or submit, focus moves |
+
+### 5c. Information Architecture
+
+| Check | How to Verify |
+|-------|--------------|
+| Page titles | `document.title` descriptive and unique |
+| Active nav state | Current page highlighted |
+| Dead ends | Any page with no way forward or back? |
+| Empty states | No data — helpful message? |
+| 404 page | `/nonexistent-page` — helpful 404? |
+
+## Self-critique before reporting  [LOW freedom — do not skip]
+
+1. **CRUD E2E** — create/update/delete proven after refresh (+ DB if available)
+2. **Evidence** — every FAIL has screenshot + console + network + repro
+3. **Cleanup** — `QA-TEST-` rows gone; auth session left intact
+4. **Right owner** — guest/authed wander → `test-exploratory`; pixel →
+   `test-visual-regression`; hostile → `test-red-team`
+5. **Binary** — no "sort of works"
+
+## Further reading
+
+- [5d. Data Display Quality and more](references/details.md)

package/skills/test-qa/references/details.md

@@ -0,0 +1,268 @@
+### 5d. Data Display Quality
+
+| Check | What to Look For |
+|-------|-----------------|
+| Formatting | Currency, dates, percentages displayed correctly for locale |
+| Raw data leaks | "undefined", "null", "NaN", "[object Object]", "Invalid Date" visible |
+| Mock/placeholder | "Lorem ipsum", "TODO", "test@example.com", "John Doe" in production data |
+| Number accuracy | Totals match sum of line items. Percentages sum to ~100%. |
+| Table headers | Every column has a header. Sort indicators if sortable. |
+| Chart labels | Axes labeled, legend present, values readable |
+
+### 5e. Accessibility Basics
+
+| Check | How to Verify |
+|-------|--------------|
+| Accessible names | Interactive elements have `aria-label`, `aria-labelledby`, or visible text |
+| Color contrast | Text readable against background (use screenshot inspection) |
+| Focus visible | Tab through the page — can you see where focus is? |
+| Form labels | `<label>` elements associated with `<input>` via `for`/`htmlFor` |
+| Alt text | Images have `alt` attributes (check via snapshot) |
+
+---
+
+## Phase 6: Edge Case and Stress Testing
+
+### 6a. Input Boundary Testing
+
+For each form field:
+
+| Input Type | Test Values |
+|-----------|------------|
+| Text | Empty, 1 char, 500 chars, 5000 chars |
+| Number | 0, -1, 999999999, 0.001, NaN (type "abc") |
+| Email | Missing @, double @, special chars, very long |
+| Date | Past (1900-01-01), future (2099-12-31), today |
+| URL | No protocol, invalid TLD, very long |
+| Select | First option, last option, no selection |
+
+### 6b. Rapid Interaction
+
+1. Double-click submit buttons — does it submit twice? (Check network requests)
+2. Click a navigation link 3 times rapidly — does it crash or navigate correctly?
+3. Toggle a setting on/off/on/off quickly — does it end in the correct state?
+
+### 6c. Navigation Edge Cases
+
+1. Browser back button from every page — does it go to the right place?
+2. Browser forward button after going back — works correctly?
+3. Direct URL access to every page (not via navigation) — loads correctly?
+4. Bookmark a page, close tab, reopen — same state?
+
+### 6d. Error State Testing
+
+1. Navigate to `/this-page-does-not-exist` → expect proper 404
+2. If possible, trigger API errors → expect user-facing error messages
+3. Submit invalid data in every form → expect inline validation
+
+---
+
+## Phase 7: Cleanup
+
+Remove all test data created during testing:
+
+1. Delete any items prefixed with `QA-TEST-`
+2. Reset any settings changed during testing
+3. If DB access available, verify cleanup:
+
+```sql
+SELECT * FROM <table> WHERE <name_column> LIKE 'QA-TEST-%';
+-- Should return 0 rows
+```
+
+4. **Keep auth session** — do not log out; leave the session's `--profile` directory (or
+   `state-save` file) intact for the next
+   agent/skill unless you were testing logout explicitly.
+
+---
+
+## Phase 8: Report
+
+Produce the final QA report:
+
+```markdown
+## QA Test Report — [Project Name]
+
+### Environment
+- URL: [url tested]
+- Framework: [detected framework + version]
+- Auth: [method used or "guest"]
+- Date: [date]
+
+### Discovery Summary
+- Routes in codebase: [N]
+- Pages tested: [N]
+- Data entities found: [N]
+- CRUD-capable entities tested: [N]
+
+### Test Summary
+
+| Metric | Value |
+|--------|-------|
+| User stories executed | [N] |
+| Individual test steps | [N] |
+| CRUD operations performed | [N] |
+| Passed | [N] |
+| Failed | [N] |
+| Blocked | [N] |
+| Pass rate | [%] |
+
+### Critical Issues (blocks production release)
+
+| # | Page | Issue | Evidence | Steps to Reproduce |
+|---|------|-------|----------|-------------------|
+| 1 | [url] | [description] | [screenshot/console ref] | [steps] |
+
+### Major Issues (degrades user experience)
+
+| # | Page | Issue | Evidence | Recommendation |
+|---|------|-------|----------|---------------|
+| 1 | [url] | [description] | [ref] | [how to fix] |
+
+### Minor Issues (polish for design award)
+
+| # | Page | Issue | Quick Fix |
+|---|------|-------|-----------|
+| 1 | [url] | [description] | [suggestion] |
+
+### Data Pipeline Issues
+
+| # | Entity | Operation | Issue | Evidence |
+|---|--------|-----------|-------|----------|
+| 1 | [entity] | [create/update/delete] | [what went wrong] | [network/DB evidence] |
+
+### Dead / Mock Code Detected
+
+| # | Page | Finding |
+|---|------|---------|
+| 1 | [url] | [dead button / mock data / placeholder text] |
+
+### UX Quality Highlights
+
+#### What Works Well
+1. [genuine positive with evidence]
+2. [genuine positive with evidence]
+
+#### Design-Award Readiness
+- Visual consistency: [score 1-5]
+- Interaction feedback: [score 1-5]
+- Empty/error states: [score 1-5]
+- Data display quality: [score 1-5]
+- Accessibility basics: [score 1-5]
+
+### Pages Not Tested
+
+| Page | Reason |
+|------|--------|
+| [url] | [BLOCKED: auth / SKIPPED: reason] |
+
+### Cleanup Verification
+- Test data created: [N items]
+- Test data deleted: [N items]
+- Orphaned test data: [N — details if any]
+
+### Production Readiness Score
+
+**[N] / 10** — [1-2 sentence justification]
+
+| Score | Meaning |
+|-------|---------|
+| 9-10 | Ship it. Design-award ready. |
+| 7-8 | Ship after fixing critical/major issues. Solid. |
+| 5-6 | Needs work. Multiple major issues. |
+| 3-4 | Not ready. Fundamental problems. |
+| 1-2 | Broken. Major rework needed. |
+```
+
+---
+
+## Sentry Integration (optional)
+
+If the project has Sentry configured (detected in Phase 0), check for production errors
+related to tested features:
+
+```json
+sentry:search_issues
+{
+  "organizationSlug": "<ORG_SLUG>",
+  "query": "unresolved issues from the last 7 days",
+  "projectSlugOrId": "<PROJECT_SLUG>",
+  "regionUrl": "<REGION_URL>",
+  "limit": 30
+}
+```
+
+Cross-reference Sentry issues with pages tested. If a page you tested has known production
+errors, note them in the report under a "Known Production Issues" section.
+
+---
+
+## Adapting to Different App Types
+
+The skill auto-adapts based on what Phase 0 discovers. Here are common patterns:
+
+| App Type | Focus Areas |
+|----------|------------|
+| **CRUD / SaaS** | Heavy CRUD lifecycle testing, data pipeline verification, form validation |
+| **Content / CMS** | Content rendering, search, filtering, pagination, media loading |
+| **E-commerce** | Cart operations, checkout flow, pricing accuracy, inventory states |
+| **Social** | User interactions, feed rendering, notifications, privacy settings |
+| **Language learning** | Lesson flow completion, progress tracking, content accuracy |
+| **Dashboard / Analytics** | Chart rendering, data accuracy, filter interactions, export |
+| **Mobile-first PWA** | Touch targets, offline states, install prompt, viewport testing |
+
+---
+
+## Test Data Guidelines
+
+When generating test data for CRUD operations:
+
+- **Prefix all test data** with `QA-TEST-` for easy identification and cleanup
+- **Use realistic but obviously fake values**: `QA-TEST-Widget-1712345678`, `qa-test@example.com`
+- **Never use real personal data** or production credentials
+- **Respect domain constraints**: if the app is financial, use realistic but fake numbers
+- **Cover multiple data types**: strings, numbers, dates, booleans, selections
+- **Test with the app's locale**: if the app is in Japanese, test with Japanese input too
+
+---
+
+## Execution Protocol
+
+For each test step, follow this cycle:
+
+```bash
+PW="npx --yes @playwright/cli@latest"; S="-s=qa-<app>"
+$PW $S goto "<url>"                                   # 1. if moving pages
+sleep 2 && $PW $S snapshot                            # 2. anti-stall: verify loaded + get refs
+$PW $S click <ref>                                    # 3. interact (click / fill / type / press)
+$PW $S snapshot                                       # 4. fresh refs after interaction
+$PW $S console                                        # 5. check for errors
+$PW $S requests                                       # 6. check for failures
+$PW $S screenshot --filename ".playwright-mcp/<step>.png"   # 7. evidence
+```
+
+8. Evaluate: PASS or FAIL with evidence.
+9. `$PW $S close` when the run is done.
+
+**Timeout budget:**
+- Single interaction: 15 seconds max
+- Page navigation + verification: 30 seconds max
+- Full CRUD lifecycle per entity: 5 minutes max
+- Full test suite: 30 minutes max
+
+If any step exceeds its budget, log `[TIMEOUT]` and move on.
+
+---
+
+## Important Rules
+
+1. **Read the codebase first (Phase 0).** Routes, entities, and auth come from the code, not from guessing at the UI.
+2. **Use playwright-cli** for all browser interaction: `goto`, `snapshot`, `screenshot`, `click`, `fill`, `type`, `press`, `console`, `requests`, `close`, `find`.
+3. **Apply anti-stall protocol** to every interaction. Read `protocol-browser-anti-stall` skill first.
+4. **Screenshot every test step.** Every pass and fail needs visual evidence.
+5. **Check console and network** after every page load and every mutation.
+6. **Clean up test data.** Delete everything prefixed with `QA-TEST-` at the end.
+7. **Be binary on findings.** Each check either PASSES or FAILS. No "sort of works."
+8. **Report severity honestly.** Critical = blocks release. Major = bad UX. Minor = polish.
+9. **Note what works well.** Good QA reports include positives, not just bugs.
+10. **Respect time budgets.** Skip stuck steps rather than freezing the entire session.