create-byan-agent 2.0.1 → 2.1.0

package/_bmad/tea/workflows/testarch/test-design/test-design-architecture-template.md

@@ -0,0 +1,222 @@
+# Test Design for Architecture: {Feature Name}
+
+**Purpose:** Architectural concerns, testability gaps, and NFR requirements for review by Architecture/Dev teams. Serves as a contract between QA and Engineering on what must be addressed before test development begins.
+
+**Date:** {date}
+**Author:** {author}
+**Status:** Architecture Review Pending
+**Project:** {project_name}
+**PRD Reference:** {prd_link}
+**ADR Reference:** {adr_link}
+
+---
+
+## Executive Summary
+
+**Scope:** {Brief description of feature scope}
+
+**Business Context** (from PRD):
+
+- **Revenue/Impact:** {Business metrics if applicable}
+- **Problem:** {Problem being solved}
+- **GA Launch:** {Target date or timeline}
+
+**Architecture** (from ADR {adr_number}):
+
+- **Key Decision 1:** {e.g., OAuth 2.1 authentication}
+- **Key Decision 2:** {e.g., Centralized MCP Server pattern}
+- **Key Decision 3:** {e.g., Stack: TypeScript, SDK v1.x}
+
+**Expected Scale** (from ADR):
+
+- {RPS, volume, users, etc.}
+
+**Risk Summary:**
+
+- **Total risks**: {N}
+- **High-priority (≥6)**: {N} risks requiring immediate mitigation
+- **Test effort**: ~{N} tests (~{X} weeks for 1 QA, ~{Y} weeks for 2 QAs)
+
+---
+
+## Quick Guide
+
+### 🚨 BLOCKERS - Team Must Decide (Can't Proceed Without)
+
+**Sprint 0 Critical Path** - These MUST be completed before QA can write integration tests:
+
+1. **{Blocker ID}: {Blocker Title}** - {What architecture must provide} (recommended owner: {Team/Role})
+2. **{Blocker ID}: {Blocker Title}** - {What architecture must provide} (recommended owner: {Team/Role})
+3. **{Blocker ID}: {Blocker Title}** - {What architecture must provide} (recommended owner: {Team/Role})
+
+**What we need from team:** Complete these {N} items in Sprint 0 or test development is blocked.
+
+---
+
+### ⚠️ HIGH PRIORITY - Team Should Validate (We Provide Recommendation, You Approve)
+
+1. **{Risk ID}: {Title}** - {Recommendation + who should approve} (Sprint {N})
+2. **{Risk ID}: {Title}** - {Recommendation + who should approve} (Sprint {N})
+3. **{Risk ID}: {Title}** - {Recommendation + who should approve} (Sprint {N})
+
+**What we need from team:** Review recommendations and approve (or suggest changes).
+
+---
+
+### 📋 INFO ONLY - Solutions Provided (Review, No Decisions Needed)
+
+1. **Test strategy**: {Test level split} ({Rationale})
+2. **Tooling**: {Test frameworks and utilities}
+3. **Tiered CI/CD**: {Execution tiers with timing}
+4. **Coverage**: ~{N} test scenarios prioritized P0-P3 with risk-based classification
+5. **Quality gates**: {Pass criteria}
+
+**What we need from team:** Just review and acknowledge (we already have the solution).
+
+---
+
+## For Architects and Devs - Open Topics 👷
+
+### Risk Assessment
+
+**Total risks identified**: {N} ({X} high-priority score ≥6, {Y} medium, {Z} low)
+
+#### High-Priority Risks (Score ≥6) - IMMEDIATE ATTENTION
+
+| Risk ID    | Category  | Description   | Probability | Impact | Score       | Mitigation            | Owner   | Timeline |
+| ---------- | --------- | ------------- | ----------- | ------ | ----------- | --------------------- | ------- | -------- |
+| **{R-ID}** | **{CAT}** | {Description} | {1-3}       | {1-3}  | **{Score}** | {Mitigation strategy} | {Owner} | {Date}   |
+
+#### Medium-Priority Risks (Score 3-5)
+
+| Risk ID | Category | Description   | Probability | Impact | Score   | Mitigation   | Owner   |
+| ------- | -------- | ------------- | ----------- | ------ | ------- | ------------ | ------- |
+| {R-ID}  | {CAT}    | {Description} | {1-3}       | {1-3}  | {Score} | {Mitigation} | {Owner} |
+
+#### Low-Priority Risks (Score 1-2)
+
+| Risk ID | Category | Description   | Probability | Impact | Score   | Action  |
+| ------- | -------- | ------------- | ----------- | ------ | ------- | ------- |
+| {R-ID}  | {CAT}    | {Description} | {1-3}       | {1-3}  | {Score} | Monitor |
+
+#### Risk Category Legend
+
+- **TECH**: Technical/Architecture (flaws, integration, scalability)
+- **SEC**: Security (access controls, auth, data exposure)
+- **PERF**: Performance (SLA violations, degradation, resource limits)
+- **DATA**: Data Integrity (loss, corruption, inconsistency)
+- **BUS**: Business Impact (UX harm, logic errors, revenue)
+- **OPS**: Operations (deployment, config, monitoring)
+
+---
+
+### Testability Concerns and Architectural Gaps
+
+**🚨 ACTIONABLE CONCERNS - Architecture Team Must Address**
+
+{If system has critical testability concerns, list them here. If architecture supports testing well, state "No critical testability concerns identified" and skip to Testability Assessment Summary}
+
+#### 1. Blockers to Fast Feedback (WHAT WE NEED FROM ARCHITECTURE)
+
+| Concern            | Impact              | What Architecture Must Provide         | Owner  | Timeline |
+| ------------------ | ------------------- | -------------------------------------- | ------ | -------- |
+| **{Concern name}** | {Impact on testing} | {Specific architectural change needed} | {Team} | {Sprint} |
+
+**Example:**
+
+- **No API for test data seeding** → Cannot parallelize tests → Provide POST /test/seed endpoint (Backend, Sprint 0)
+
+#### 2. Architectural Improvements Needed (WHAT SHOULD BE CHANGED)
+
+{List specific improvements that would make the system more testable}
+
+1. **{Improvement name}**
+   - **Current problem**: {What's wrong}
+   - **Required change**: {What architecture must do}
+   - **Impact if not fixed**: {Consequences}
+   - **Owner**: {Team}
+   - **Timeline**: {Sprint}
+
+---
+
+### Testability Assessment Summary
+
+**📊 CURRENT STATE - FYI**
+
+{Only include this section if there are passing items worth mentioning. Otherwise omit.}
+
+#### What Works Well
+
+- ✅ {Passing item 1} (e.g., "API-first design supports parallel test execution")
+- ✅ {Passing item 2} (e.g., "Feature flags enable test isolation")
+- ✅ {Passing item 3}
+
+#### Accepted Trade-offs (No Action Required)
+
+For {Feature} Phase 1, the following trade-offs are acceptable:
+
+- **{Trade-off 1}** - {Why acceptable for now}
+- **{Trade-off 2}** - {Why acceptable for now}
+
+{This is technical debt OR acceptable for Phase 1} that {should be revisited post-GA OR maintained as-is}
+
+---
+
+### Risk Mitigation Plans (High-Priority Risks ≥6)
+
+**Purpose**: Detailed mitigation strategies for all {N} high-priority risks (score ≥6). These risks MUST be addressed before {GA launch date or milestone}.
+
+#### {R-ID}: {Risk Description} (Score: {Score}) - {CRITICALITY LEVEL}
+
+**Mitigation Strategy:**
+
+1. {Step 1}
+2. {Step 2}
+3. {Step 3}
+
+**Owner:** {Owner}
+**Timeline:** {Sprint or date}
+**Status:** Planned / In Progress / Complete
+**Verification:** {How to verify mitigation is effective}
+
+---
+
+{Repeat for all high-priority risks}
+
+---
+
+### Assumptions and Dependencies
+
+#### Assumptions
+
+1. {Assumption about architecture or requirements}
+2. {Assumption about team or timeline}
+3. {Assumption about scope or constraints}
+
+#### Dependencies
+
+1. {Dependency} - Required by {date/sprint}
+2. {Dependency} - Required by {date/sprint}
+
+#### Risks to Plan
+
+- **Risk**: {Risk to the test plan itself}
+  - **Impact**: {How it affects testing}
+  - **Contingency**: {Backup plan}
+
+---
+
+**End of Architecture Document**
+
+**Next Steps for Architecture Team:**
+
+1. Review Quick Guide (🚨/⚠️/📋) and prioritize blockers
+2. Assign owners and timelines for high-priority risks (≥6)
+3. Validate assumptions and dependencies
+4. Provide feedback to QA on testability gaps
+
+**Next Steps for QA Team:**
+
+1. Wait for Sprint 0 blockers to be resolved
+2. Refer to companion QA doc (test-design-qa.md) for test scenarios
+3. Begin test infrastructure setup (factories, fixtures, environments)

package/_bmad/tea/workflows/testarch/test-design/test-design-qa-template.md

@@ -0,0 +1,296 @@
+# Test Design for QA: {Feature Name}
+
+**Purpose:** Test execution recipe for QA team. Defines what to test, how to test it, and what QA needs from other teams.
+
+**Date:** {date}
+**Author:** {author}
+**Status:** Draft
+**Project:** {project_name}
+
+**Related:** See Architecture doc (test-design-architecture.md) for testability concerns and architectural blockers.
+
+---
+
+## Executive Summary
+
+**Scope:** {Brief description of testing scope}
+
+**Risk Summary:**
+
+- Total Risks: {N} ({X} high-priority score ≥6, {Y} medium, {Z} low)
+- Critical Categories: {Categories with most high-priority risks}
+
+**Coverage Summary:**
+
+- P0 tests: ~{N} (critical paths, security)
+- P1 tests: ~{N} (important features, integration)
+- P2 tests: ~{N} (edge cases, regression)
+- P3 tests: ~{N} (exploratory, benchmarks)
+- **Total**: ~{N} tests (~{X}-{Y} weeks with 1 QA)
+
+---
+
+## Dependencies & Test Blockers
+
+**CRITICAL:** QA cannot proceed without these items from other teams.
+
+### Backend/Architecture Dependencies (Sprint 0)
+
+**Source:** See Architecture doc "Quick Guide" for detailed mitigation plans
+
+1. **{Dependency 1}** - {Team} - {Timeline}
+   - {What QA needs}
+   - {Why it blocks testing}
+
+2. **{Dependency 2}** - {Team} - {Timeline}
+   - {What QA needs}
+   - {Why it blocks testing}
+
+### QA Infrastructure Setup (Sprint 0)
+
+1. **Test Data Factories** - QA
+   - {Entity} factory with faker-based randomization
+   - Auto-cleanup fixtures for parallel safety
+
+2. **Test Environments** - QA
+   - Local: {Setup details}
+   - CI/CD: {Setup details}
+   - Staging: {Setup details}
+
+**Example factory pattern:**
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
+import { expect } from '@playwright/test';
+import { faker } from '@faker-js/faker';
+
+test('example test @p0', async ({ apiRequest }) => {
+  const testData = {
+    id: `test-${faker.string.uuid()}`,
+    email: faker.internet.email(),
+  };
+
+  const { status } = await apiRequest({
+    method: 'POST',
+    path: '/api/resource',
+    body: testData,
+  });
+
+  expect(status).toBe(201);
+});
+```
+
+---
+
+## Risk Assessment
+
+**Note:** Full risk details in Architecture doc. This section summarizes risks relevant to QA test planning.
+
+### High-Priority Risks (Score ≥6)
+
+| Risk ID    | Category | Description         | Score       | QA Test Coverage             |
+| ---------- | -------- | ------------------- | ----------- | ---------------------------- |
+| **{R-ID}** | {CAT}    | {Brief description} | **{Score}** | {How QA validates this risk} |
+
+### Medium/Low-Priority Risks
+
+| Risk ID | Category | Description         | Score   | QA Test Coverage             |
+| ------- | -------- | ------------------- | ------- | ---------------------------- |
+| {R-ID}  | {CAT}    | {Brief description} | {Score} | {How QA validates this risk} |
+
+---
+
+## Test Coverage Plan
+
+**IMPORTANT:** P0/P1/P2/P3 = **priority and risk level** (what to focus on if time-constrained), NOT execution timing. See "Execution Strategy" for when tests run.
+
+### P0 (Critical)
+
+**Criteria:** Blocks core functionality + High risk (≥6) + No workaround + Affects majority of users
+
+| Test ID    | Requirement   | Test Level | Risk Link | Notes   |
+| ---------- | ------------- | ---------- | --------- | ------- |
+| **P0-001** | {Requirement} | {Level}    | {R-ID}    | {Notes} |
+| **P0-002** | {Requirement} | {Level}    | {R-ID}    | {Notes} |
+
+**Total P0:** ~{N} tests
+
+---
+
+### P1 (High)
+
+**Criteria:** Important features + Medium risk (3-4) + Common workflows + Workaround exists but difficult
+
+| Test ID    | Requirement   | Test Level | Risk Link | Notes   |
+| ---------- | ------------- | ---------- | --------- | ------- |
+| **P1-001** | {Requirement} | {Level}    | {R-ID}    | {Notes} |
+| **P1-002** | {Requirement} | {Level}    | {R-ID}    | {Notes} |
+
+**Total P1:** ~{N} tests
+
+---
+
+### P2 (Medium)
+
+**Criteria:** Secondary features + Low risk (1-2) + Edge cases + Regression prevention
+
+| Test ID    | Requirement   | Test Level | Risk Link | Notes   |
+| ---------- | ------------- | ---------- | --------- | ------- |
+| **P2-001** | {Requirement} | {Level}    | {R-ID}    | {Notes} |
+
+**Total P2:** ~{N} tests
+
+---
+
+### P3 (Low)
+
+**Criteria:** Nice-to-have + Exploratory + Performance benchmarks + Documentation validation
+
+| Test ID    | Requirement   | Test Level | Notes   |
+| ---------- | ------------- | ---------- | ------- |
+| **P3-001** | {Requirement} | {Level}    | {Notes} |
+
+**Total P3:** ~{N} tests
+
+---
+
+## Execution Strategy
+
+**Philosophy:** Run everything in PRs unless there's significant infrastructure overhead. Playwright with parallelization is extremely fast (100s of tests in ~10-15 min).
+
+**Organized by TOOL TYPE:**
+
+### Every PR: Playwright Tests (~10-15 min)
+
+**All functional tests** (from any priority level):
+
+- All E2E, API, integration, unit tests using Playwright
+- Parallelized across {N} shards
+- Total: ~{N} Playwright tests (includes P0, P1, P2, P3)
+
+**Why run in PRs:** Fast feedback, no expensive infrastructure
+
+### Nightly: k6 Performance Tests (~30-60 min)
+
+**All performance tests** (from any priority level):
+
+- Load, stress, spike, endurance tests
+- Total: ~{N} k6 tests (may include P0, P1, P2)
+
+**Why defer to nightly:** Expensive infrastructure (k6 Cloud), long-running (10-40 min per test)
+
+### Weekly: Chaos & Long-Running (~hours)
+
+**Special infrastructure tests** (from any priority level):
+
+- Multi-region failover (requires AWS Fault Injection Simulator)
+- Disaster recovery (backup restore, 4+ hours)
+- Endurance tests (4+ hours runtime)
+
+**Why defer to weekly:** Very expensive infrastructure, very long-running, infrequent validation sufficient
+
+**Manual tests** (excluded from automation):
+
+- DevOps validation (deployment, monitoring)
+- Finance validation (cost alerts)
+- Documentation validation
+
+---
+
+## QA Effort Estimate
+
+**QA test development effort only** (excludes DevOps, Backend, Data Eng, Finance work):
+
+| Priority  | Count | Effort Range       | Notes                                             |
+| --------- | ----- | ------------------ | ------------------------------------------------- |
+| P0        | ~{N}  | ~{X}-{Y} weeks     | Complex setup (security, performance, multi-step) |
+| P1        | ~{N}  | ~{X}-{Y} weeks     | Standard coverage (integration, API tests)        |
+| P2        | ~{N}  | ~{X}-{Y} days      | Edge cases, simple validation                     |
+| P3        | ~{N}  | ~{X}-{Y} days      | Exploratory, benchmarks                           |
+| **Total** | ~{N}  | **~{X}-{Y} weeks** | **1 QA engineer, full-time**                      |
+
+**Assumptions:**
+
+- Includes test design, implementation, debugging, CI integration
+- Excludes ongoing maintenance (~10% effort)
+- Assumes test infrastructure (factories, fixtures) ready
+
+**Dependencies from other teams:**
+
+- See "Dependencies & Test Blockers" section for what QA needs from Backend, DevOps, Data Eng
+
+---
+
+## Appendix A: Code Examples & Tagging
+
+**Playwright Tags for Selective Execution:**
+
+```typescript
+import { test } from '@seontechnologies/playwright-utils/api-request/fixtures';
+import { expect } from '@playwright/test';
+
+// P0 critical test
+test('@P0 @API @Security unauthenticated request returns 401', async ({ apiRequest }) => {
+  const { status, body } = await apiRequest({
+    method: 'POST',
+    path: '/api/endpoint',
+    body: { data: 'test' },
+    skipAuth: true,
+  });
+
+  expect(status).toBe(401);
+  expect(body.error).toContain('unauthorized');
+});
+
+// P1 integration test
+test('@P1 @Integration data syncs correctly', async ({ apiRequest }) => {
+  // Seed data
+  await apiRequest({
+    method: 'POST',
+    path: '/api/seed',
+    body: {
+      /* test data */
+    },
+  });
+
+  // Validate
+  const { status, body } = await apiRequest({
+    method: 'GET',
+    path: '/api/resource',
+  });
+
+  expect(status).toBe(200);
+  expect(body).toHaveProperty('data');
+});
+```
+
+**Run specific tags:**
+
+```bash
+# Run only P0 tests
+npx playwright test --grep @P0
+
+# Run P0 + P1 tests
+npx playwright test --grep "@P0|@P1"
+
+# Run only security tests
+npx playwright test --grep @Security
+
+# Run all Playwright tests in PR (default)
+npx playwright test
+```
+
+---
+
+## Appendix B: Knowledge Base References
+
+- **Risk Governance**: `risk-governance.md` - Risk scoring methodology
+- **Test Priorities Matrix**: `test-priorities-matrix.md` - P0-P3 criteria
+- **Test Levels Framework**: `test-levels-framework.md` - E2E vs API vs Unit selection
+- **Test Quality**: `test-quality.md` - Definition of Done (no hard waits, <300 lines, <1.5 min)
+
+---
+
+**Generated by:** BMad TEA Agent
+**Workflow:** `_bmad/tea/testarch/test-design`
+**Version:** 4.0 (BMad v6)