qa-ai-repo 0.2.0 → 0.3.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "qa-ai-repo",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Install reusable QA skills, agents, and MCP servers into Claude Code, Cursor, Windsurf, and other AI coding tools with one command.",
   "type": "module",
   "bin": {

package/test-pyramid/agents/test-architect.md

@@ -0,0 +1,44 @@
+---
+name: test-architect
+description: Use to analyze a full-stack application (frontend, backend, middleware) and produce a complete test pyramid strategy — what to test in the FE, what in the BE, what at the middleware/seams, and at which level. It inspects the codebase, maps the layers, and writes a per-layer test plan with tooling and CI wiring.
+tools: Read, Grep, Glob, Bash, Write
+---
+
+You are a test architect. You design testing for an application as a whole
+system of layers, so every behavior is verified at the lowest effective level
+and seams are covered by contracts rather than heavy end-to-end tests.
+
+## Process
+
+1. **Discover the architecture.** Inspect the repo to identify each layer and its
+   stack: frontend framework/state/routing; backend services, API style, data
+   stores; middleware (gateway/BFF, auth, queues/event bus, cache, workers,
+   third-party integrations). Read package manifests, framework configs, `src/`
+   layout, infra/compose files, and CI. Confirm findings; don't assume.
+2. **Inventory current tests** and their pyramid shape (unit vs integration vs
+   E2E). Flag if it's inverted (mostly slow E2E).
+3. **List behaviors per layer**, then assign each to the **lowest** level that
+   can prove it:
+   - pure logic/rendering/validation → unit
+   - needs a real collaborator (DB, rendered tree + network, queue) → integration/component
+   - agreement across a boundary → contract (Pact / OpenAPI / AsyncAPI)
+   - only a full critical journey → E2E (keep to a handful)
+4. **Cover every seam with a contract** instead of re-testing both sides via E2E.
+5. **Write `TEST-PYRAMID.md`** with: architecture map; FE / BE / middleware test
+   plans (concrete tests + tools); a seams→contracts table; the few E2E journeys;
+   target proportions vs. current gap; tooling summary; and a Now/Next/Later
+   build order with owners.
+
+## Principles
+
+- Push tests down; contracts replace integration E2E.
+- Test behavior, not implementation — especially in the FE (assert what the user
+  sees, not internal state), and never drive backend rules through the UI.
+- Concentrate depth on revenue/safety-critical flows.
+- Be specific: name the actual modules/endpoints/queues to cover and the tool for
+  each, not generic advice.
+
+## Report
+
+The path to `TEST-PYRAMID.md`, the layers found, the biggest coverage gaps, and
+the top 3 tests to add first.

package/test-pyramid/objective.json

@@ -0,0 +1,4 @@
+{
+  "title": "Full-Stack Test Pyramid Strategy",
+  "description": "Analyze an application end to end — frontend, backend, and middleware — and produce a complete test pyramid: exactly what to test at each layer and each level (unit, integration/component, contract, E2E), where each behavior belongs, and which tools to use."
+}

package/test-pyramid/skills/test-pyramid/SKILL.md

@@ -0,0 +1,72 @@
+---
+name: test-pyramid
+description: Analyze a full-stack application (frontend, backend, middleware) and design a complete test pyramid — deciding exactly which tests belong in the FE, which in the BE, which at middleware/seams, and at what level (unit, integration/component, contract, E2E). Use when asked to "design a testing strategy for the whole app", "what should we test where", "build a test pyramid", or to rebalance an inverted (E2E-heavy) suite.
+---
+
+# Full-Stack Test Pyramid
+
+Design testing for an application **as a whole system of layers**, not one suite.
+The goal: every behavior is tested at the **lowest level that can meaningfully
+verify it**, seams are covered by **contract tests**, and only a handful of
+journeys reach **end-to-end**. See `layer-test-matrix.md` for the full
+layer × level grid of what to test and which tools to use.
+
+## Method
+
+1. **Map the architecture.** Identify each layer and its technology:
+   - **Frontend** — SPA/SSR framework, state management, design system, routes.
+   - **Backend** — services, APIs (REST/GraphQL/gRPC), domain logic, data stores.
+   - **Middleware** — API gateway/BFF, auth, message queues/event bus, caching,
+     service mesh, background workers, third-party integrations.
+   Inspect the repo (package manifests, framework configs, `src/` layout, CI) and
+   confirm rather than assume.
+
+2. **List behaviors per layer**, then **assign each to the lowest suitable
+   level** using this decision order:
+   - Pure logic / rendering / validation → **unit**.
+   - Behavior that needs a real collaborator (DB, rendered tree + network, queue)
+     → **integration / component**.
+   - Agreement across a boundary (FE↔API, service↔service, producer↔consumer)
+     → **contract** (Pact / OpenAPI / AsyncAPI) — this is what lets you keep E2E small.
+   - Only a full critical user journey that no lower level can prove → **E2E**.
+
+3. **Cover the seams, not the internals twice.** Where two layers meet, use a
+   contract test once instead of re-testing both sides through E2E.
+
+4. **Set the shape.** Aim for a true pyramid, roughly:
+   - ~70% unit · ~20% integration/component · <10% contract+E2E (E2E a small
+     handful). If the current suite is an inverted "ice-cream cone" (mostly E2E),
+     call it out and give the rebalancing plan.
+
+5. **Output the plan** using `plan-template.md`: per-layer test lists, the seam
+   contracts, the few E2E journeys, target proportions, tooling, CI wiring, and
+   what to build first.
+
+## What goes where (summary — detail in `layer-test-matrix.md`)
+
+- **Frontend:** unit test component logic/hooks/reducers/utils; component-test
+  rendered UI with mocked network (Testing Library + MSW), accessibility (axe),
+  and visual regression; **consumer contract** tests against the API; a few E2E
+  journeys. Do NOT drive backend business rules through the UI.
+- **Backend:** unit test domain/business logic and validators; integration-test
+  repositories and route handlers against a real (containerized) DB and real
+  adapters; **provider contract** verification + OpenAPI/schema conformance;
+  service/component tests with downstreams mocked.
+- **Middleware:** unit test routing/transformation/auth/rate-limit logic;
+  integration-test queue producers/consumers, gateway routing, and cache
+  behavior with real infra (Testcontainers); **message/event contracts**
+  (AsyncAPI, Pact message pacts); resilience tests for retries, timeouts,
+  circuit breakers, and idempotency.
+- **Cross-cutting (top):** a small set of full E2E journeys, plus performance/
+  load (k6) and security (SAST/DAST) as their own tracks.
+
+## Principles
+
+- **Push tests down.** A bug catchable by a unit test should not need an E2E.
+- **Contracts replace integration E2E.** Seams verified by contracts let you
+  delete most cross-service E2E.
+- **Test behavior, not implementation.** Especially in the FE — assert what the
+  user sees, not internal state.
+- **Isolation + speed at the base**, realism concentrated at the seams, breadth
+  only at the tip.
+- **Right-size to risk:** put the extra depth on revenue/safety-critical flows.

package/test-pyramid/skills/test-pyramid/layer-test-matrix.md

@@ -0,0 +1,50 @@
+# Layer × Level Test Matrix
+
+For each layer, what to test at each pyramid level and typical tools. Assign each
+behavior to the **lowest** level that can prove it.
+
+## Frontend (UI)
+
+| Level | What to test in the FE | Tools |
+|-------|------------------------|-------|
+| Unit | Pure logic: hooks, reducers/stores, selectors, formatters, validation, utility fns | Jest / Vitest |
+| Component / integration | Rendered components & flows with **network mocked**; forms, conditional UI, routing; accessibility; visual regression | Testing Library, MSW, jest-axe, Storybook + Playwright/Chromatic snapshots |
+| Contract (consumer) | The shape/behavior the FE expects from each API it calls | Pact (consumer), or types generated from OpenAPI/GraphQL schema |
+| E2E | A few critical user journeys through the real app | Playwright / Cypress |
+
+**Don't** test backend business rules or data validation *through* the UI — mock
+the API and test those rules in the BE.
+
+## Backend (services / APIs)
+
+| Level | What to test in the BE | Tools |
+|-------|------------------------|-------|
+| Unit | Domain/business logic, calculations, state machines, validators, mappers — no I/O | Jest/Vitest, Pytest, JUnit, Go test, RSpec |
+| Integration | Repositories/ORM against a **real DB**, route/controller handlers, external adapters, migrations | Testcontainers, Supertest, test DB, WireMock for third parties |
+| Contract (provider) | Verify the provider satisfies every consumer contract; conform to the published OpenAPI/GraphQL schema | Pact (provider verification), Schemathesis, Dredd, oasdiff |
+| Component / service | The whole service in isolation with downstreams stubbed | in-process HTTP + mocked deps |
+
+## Middleware (gateway / queues / auth / cache / workers)
+
+| Level | What to test in middleware | Tools |
+|-------|----------------------------|-------|
+| Unit | Routing/transformation rules, auth/authorization middleware, rate limiting, serialization | framework test runner |
+| Integration | Queue producers/consumers, event handlers, gateway routing/BFF aggregation, cache read/write/invalidation | Testcontainers (Kafka/RabbitMQ/Redis), LocalStack |
+| Contract (message/event) | Event & message schemas between producers and consumers | AsyncAPI validation, Pact message pacts |
+| Resilience | Retries, timeouts, circuit breakers, idempotency, dead-letter handling, backpressure | Toxiproxy, fault-injection, chaos tests |
+
+## Cross-cutting (top of the pyramid — keep small)
+
+| Concern | What | Tools |
+|---------|------|-------|
+| E2E journeys | A handful of full-stack critical paths only | Playwright / Cypress |
+| Performance / load | Throughput, latency, soak, spike | k6, Gatling, Locust |
+| Security | SAST, dependency scan, DAST | Semgrep/CodeQL, Snyk/Dependabot, OWASP ZAP |
+| Accessibility | End-to-end a11y on key flows | axe, Lighthouse CI |
+
+## Target shape
+
+- ~70% unit · ~20% integration/component · ~7% contract · ~3% E2E (a small,
+  fixed set). Contracts do the heavy lifting at seams so E2E stays tiny.
+- Inverted suite (mostly slow E2E)? Push each E2E down: replace with a component
+  test (FE), an integration test (BE), or a contract test (seam) wherever possible.

package/test-pyramid/skills/test-pyramid/plan-template.md

@@ -0,0 +1,58 @@
+# Test Pyramid Strategy — <Application Name>
+
+> Generated <date> · Scope: frontend + backend + middleware
+
+## 0. Architecture map
+Layers detected and their tech:
+- **Frontend:** framework, state, routing, design system.
+- **Backend:** services, API style (REST/GraphQL/gRPC), data stores.
+- **Middleware:** gateway/BFF, auth, queues/event bus, cache, workers, 3rd parties.
+Diagram or bullet list of how requests/events flow across layers.
+
+## 1. Frontend test plan
+- **Unit:** <hooks/reducers/utils/validators to cover> — tool.
+- **Component/integration:** <rendered flows, forms, a11y, visual> — tool.
+- **Consumer contracts:** <each API the FE consumes> — tool.
+- Explicitly **not** in the FE: <backend rules to push down>.
+
+## 2. Backend test plan
+- **Unit:** <domain logic, calculations, validators>.
+- **Integration:** <repositories, handlers, adapters, migrations> — real DB via Testcontainers.
+- **Provider contracts / schema conformance:** <APIs to verify>.
+- **Service/component:** <services to test in isolation>.
+
+## 3. Middleware test plan
+- **Unit:** <routing/auth/rate-limit/transform logic>.
+- **Integration:** <queues, gateway, cache> with real infra.
+- **Message/event contracts:** <topics/queues and their schemas>.
+- **Resilience:** <retries, timeouts, circuit breakers, idempotency>.
+
+## 4. Seams & contracts (the glue)
+Table of every boundary → contract that covers it, so E2E can stay small.
+| Seam | Consumer | Provider | Contract |
+|------|----------|----------|----------|
+| FE ↔ Orders API | web app | orders-svc | Pact / OpenAPI |
+| orders-svc ↔ payments | orders-svc | payments-svc | Pact |
+| orders-svc → events | producer | notif-worker | AsyncAPI / message pact |
+
+## 5. End-to-end journeys (keep to a handful)
+List the few critical full-stack paths that justify E2E, and why each can't be
+covered lower down.
+
+## 6. Cross-cutting
+Performance/load, security (SAST/DAST/deps), accessibility — owners and cadence.
+
+## 7. Target proportions & current gap
+| Level | Target | Now | Action |
+|-------|--------|-----|--------|
+| Unit | ~70% | ? | |
+| Integration/component | ~20% | ? | |
+| Contract | ~7% | ? | |
+| E2E | ~3% | ? | |
+Note if the current suite is inverted and the rebalancing moves.
+
+## 8. Tooling summary
+Per layer: chosen runners, mocking, contract tooling, CI reporting.
+
+## 9. Build order (Now / Next / Later)
+Concrete first tests to add, then build-out, then hardening — with owners.