bmad-method 6.0.0-alpha.4 → 6.0.0-alpha.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/agents/bmad-analysis/api-documenter.md +102 -0
- package/.claude/agents/bmad-analysis/codebase-analyzer.md +82 -0
- package/.claude/agents/bmad-analysis/data-analyst.md +101 -0
- package/.claude/agents/bmad-analysis/pattern-detector.md +84 -0
- package/.claude/agents/bmad-planning/dependency-mapper.md +83 -0
- package/.claude/agents/bmad-planning/epic-optimizer.md +81 -0
- package/.claude/agents/bmad-planning/requirements-analyst.md +61 -0
- package/.claude/agents/bmad-planning/technical-decisions-curator.md +168 -0
- package/.claude/agents/bmad-planning/trend-spotter.md +115 -0
- package/.claude/agents/bmad-planning/user-journey-mapper.md +123 -0
- package/.claude/agents/bmad-planning/user-researcher.md +72 -0
- package/.claude/agents/bmad-research/market-researcher.md +51 -0
- package/.claude/agents/bmad-research/tech-debt-auditor.md +106 -0
- package/.claude/agents/bmad-review/document-reviewer.md +102 -0
- package/.claude/agents/bmad-review/technical-evaluator.md +68 -0
- package/.claude/agents/bmad-review/test-coverage-analyzer.md +108 -0
- package/.claude/commands/bmad/bmm/agents/architect.md +0 -1
- package/.claude/commands/bmad/bmm/agents/sm.md +1 -1
- package/.claude/commands/bmad/bmm/agents/{paige.md → tech-writer.md} +3 -3
- package/.claude/commands/bmad/bmm/workflows/README.md +3 -3
- package/.claude/commands/bmad/bmm/workflows/epic-tech-context.md +15 -0
- package/.claude/commands/bmad/bmm/workflows/prd.md +1 -1
- package/.claude/commands/bmad/bmm/workflows/tech-spec.md +3 -3
- package/.claude/settings.local.json +3 -8
- package/CHANGELOG.md +305 -0
- package/CONTRIBUTING.md +1 -13
- package/README.md +88 -39
- package/bmad/_cfg/agent-manifest.csv +1 -6
- package/bmad/_cfg/files-manifest.csv +40 -114
- package/bmad/_cfg/ides/claude-code.yaml +4 -3
- package/bmad/_cfg/manifest.yaml +3 -4
- package/bmad/_cfg/task-manifest.csv +0 -1
- package/bmad/_cfg/workflow-manifest.csv +3 -7
- package/bmad/bmb/config.yaml +2 -2
- package/bmad/bmb/workflows/audit-workflow/instructions.md +1 -1
- package/bmad/bmm/README.md +79 -120
- package/bmad/bmm/agents/architect.md +0 -1
- package/bmad/bmm/agents/sm.md +1 -1
- package/bmad/bmm/agents/{paige.md → tech-writer.md} +3 -3
- package/bmad/bmm/config.yaml +4 -3
- package/bmad/bmm/tasks/daily-standup.xml +1 -1
- package/bmad/bmm/workflows/2-plan-workflows/create-ux-design/instructions.md +1 -19
- package/bmad/bmm/workflows/2-plan-workflows/prd/checklist.md +10 -9
- package/bmad/bmm/workflows/2-plan-workflows/prd/create-epics-and-stories/epics-template.md +23 -34
- package/bmad/bmm/workflows/2-plan-workflows/prd/create-epics-and-stories/instructions.md +105 -331
- package/bmad/bmm/workflows/2-plan-workflows/prd/create-epics-and-stories/workflow.yaml +23 -11
- package/bmad/bmm/workflows/2-plan-workflows/prd/instructions.md +23 -38
- package/bmad/bmm/workflows/2-plan-workflows/prd/workflow.yaml +1 -1
- package/bmad/bmm/workflows/2-plan-workflows/tech-spec/epics-template.md +38 -16
- package/bmad/bmm/workflows/2-plan-workflows/tech-spec/instructions.md +1 -19
- package/bmad/bmm/workflows/2-plan-workflows/tech-spec/user-story-template.md +35 -32
- package/bmad/bmm/workflows/2-plan-workflows/tech-spec/workflow.yaml +2 -2
- package/bmad/bmm/workflows/3-solutioning/architecture/instructions.md +7 -18
- package/bmad/bmm/workflows/3-solutioning/solutioning-gate-check/instructions.md +1 -18
- package/bmad/bmm/workflows/3-solutioning/solutioning-gate-check/workflow.yaml +6 -6
- package/bmad/bmm/workflows/4-implementation/epic-tech-context/workflow.yaml +1 -1
- package/bmad/bmm/workflows/techdoc/documentation-standards.md +26 -2
- package/bmad/bmm/workflows/testarch/atdd/atdd-checklist-template.md +2 -2
- package/bmad/bmm/workflows/workflow-status/init/instructions.md +623 -242
- package/bmad/bmm/workflows/workflow-status/paths/enterprise-brownfield.yaml +120 -0
- package/bmad/bmm/workflows/workflow-status/paths/enterprise-greenfield.yaml +108 -0
- package/bmad/bmm/workflows/workflow-status/paths/{brownfield-level-3.yaml → method-brownfield.yaml} +33 -31
- package/{src/modules/bmm/workflows/workflow-status/paths/greenfield-level-2.yaml → bmad/bmm/workflows/workflow-status/paths/method-greenfield.yaml} +31 -21
- package/bmad/bmm/workflows/workflow-status/paths/{brownfield-level-1.yaml → quick-flow-brownfield.yaml} +18 -18
- package/bmad/bmm/workflows/workflow-status/paths/{greenfield-level-1.yaml → quick-flow-greenfield.yaml} +16 -18
- package/bmad/bmm/workflows/workflow-status/workflow-status-template.yaml +4 -4
- package/bmad/core/config.yaml +2 -2
- package/bmad/core/tasks/workflow.xml +1 -11
- package/package.json +1 -1
- package/src/core/tasks/workflow.xml +1 -11
- package/src/modules/bmb/workflows/audit-workflow/instructions.md +1 -1
- package/src/modules/bmm/README.md +1 -1
- package/src/modules/bmm/_module-installer/install-config.yaml +5 -0
- package/src/modules/bmm/agents/architect.agent.yaml +0 -4
- package/src/modules/bmm/agents/game-dev.agent.yaml +8 -12
- package/src/modules/bmm/agents/sm.agent.yaml +1 -1
- package/src/modules/bmm/agents/{paige.agent.yaml → tech-writer.agent.yaml} +4 -4
- package/src/modules/bmm/docs/README.md +19 -18
- package/src/modules/bmm/docs/agents-guide.md +47 -99
- package/src/modules/bmm/docs/brownfield-guide.md +216 -96
- package/src/modules/bmm/docs/enterprise-agentic-development.md +380 -740
- package/src/modules/bmm/docs/faq.md +16 -16
- package/src/modules/bmm/docs/glossary.md +44 -50
- package/src/modules/bmm/docs/party-mode.md +110 -1122
- package/src/modules/bmm/docs/quick-spec-flow.md +36 -36
- package/src/modules/bmm/docs/quick-start.md +34 -34
- package/src/modules/bmm/docs/scale-adaptive-system.md +304 -454
- package/{bmad/bmm/testarch/README.md → src/modules/bmm/docs/test-architecture.md} +56 -38
- package/src/modules/bmm/docs/troubleshooting.md +7 -7
- package/src/modules/bmm/docs/workflows-analysis.md +64 -28
- package/src/modules/bmm/docs/workflows-implementation.md +196 -1671
- package/src/modules/bmm/docs/workflows-planning.md +74 -35
- package/src/modules/bmm/docs/workflows-solutioning.md +47 -17
- package/src/modules/bmm/tasks/daily-standup.xml +1 -1
- package/src/modules/bmm/workflows/2-plan-workflows/create-ux-design/instructions.md +1 -19
- package/src/modules/bmm/workflows/2-plan-workflows/prd/checklist.md +10 -9
- package/src/modules/bmm/workflows/2-plan-workflows/prd/create-epics-and-stories/epics-template.md +23 -34
- package/src/modules/bmm/workflows/2-plan-workflows/prd/create-epics-and-stories/instructions.md +105 -331
- package/src/modules/bmm/workflows/2-plan-workflows/prd/create-epics-and-stories/workflow.yaml +23 -11
- package/src/modules/bmm/workflows/2-plan-workflows/prd/instructions.md +23 -38
- package/src/modules/bmm/workflows/2-plan-workflows/prd/workflow.yaml +2 -2
- package/src/modules/bmm/workflows/2-plan-workflows/tech-spec/epics-template.md +38 -16
- package/src/modules/bmm/workflows/2-plan-workflows/tech-spec/instructions.md +1 -19
- package/src/modules/bmm/workflows/2-plan-workflows/tech-spec/user-story-template.md +35 -32
- package/src/modules/bmm/workflows/2-plan-workflows/tech-spec/workflow.yaml +2 -2
- package/src/modules/bmm/workflows/3-solutioning/architecture/instructions.md +7 -18
- package/src/modules/bmm/workflows/3-solutioning/solutioning-gate-check/instructions.md +1 -18
- package/src/modules/bmm/workflows/3-solutioning/solutioning-gate-check/workflow.yaml +6 -6
- package/src/modules/bmm/workflows/4-implementation/epic-tech-context/workflow.yaml +1 -1
- package/src/modules/bmm/workflows/techdoc/documentation-standards.md +25 -2
- package/src/modules/bmm/workflows/testarch/atdd/atdd-checklist-template.md +2 -2
- package/src/modules/bmm/workflows/workflow-status/init/instructions.md +623 -242
- package/src/modules/bmm/workflows/workflow-status/paths/enterprise-brownfield.yaml +120 -0
- package/src/modules/bmm/workflows/workflow-status/paths/enterprise-greenfield.yaml +108 -0
- package/src/modules/bmm/workflows/workflow-status/paths/{brownfield-level-3.yaml → method-brownfield.yaml} +33 -31
- package/{bmad/bmm/workflows/workflow-status/paths/greenfield-level-2.yaml → src/modules/bmm/workflows/workflow-status/paths/method-greenfield.yaml} +31 -21
- package/src/modules/bmm/workflows/workflow-status/paths/{brownfield-level-1.yaml → quick-flow-brownfield.yaml} +18 -18
- package/src/modules/bmm/workflows/workflow-status/paths/{greenfield-level-1.yaml → quick-flow-greenfield.yaml} +16 -18
- package/src/modules/bmm/workflows/workflow-status/workflow-status-template.yaml +4 -4
- package/tools/cli/installers/lib/core/config-collector.js +39 -11
- package/tools/cli/installers/lib/core/ide-config-manager.js +3 -1
- package/tools/cli/installers/lib/core/installer.js +20 -9
- package/tools/cli/installers/lib/core/manifest-generator.js +3 -1
- package/tools/cli/installers/lib/core/manifest.js +6 -2
- package/tools/cli/installers/lib/modules/manager.js +6 -0
- package/tools/cli/lib/config.js +3 -1
- package/tools/cli/lib/ui.js +5 -5
- package/tools/cli/lib/yaml-format.js +2 -1
- package/tools/schema/agent.js +2 -0
- package/.claude/commands/bmad/bmm/workflows/tech-spec-sm.md +0 -15
- package/bmad/_cfg/agents/cis-brainstorming-coach.customize.yaml +0 -42
- package/bmad/_cfg/agents/cis-creative-problem-solver.customize.yaml +0 -42
- package/bmad/_cfg/agents/cis-design-thinking-coach.customize.yaml +0 -42
- package/bmad/_cfg/agents/cis-innovation-strategist.customize.yaml +0 -42
- package/bmad/_cfg/agents/cis-storyteller.customize.yaml +0 -42
- package/bmad/bmb/agents/bmad-builder.md.bak +0 -70
- package/bmad/bmb/workflows/audit-workflow/workflow.yaml.bak +0 -23
- package/bmad/bmb/workflows/create-module/workflow.yaml.bak +0 -42
- package/bmad/bmb/workflows/create-workflow/workflow-template/workflow.yaml.bak +0 -39
- package/bmad/bmb/workflows/create-workflow/workflow.yaml.bak +0 -40
- package/bmad/bmb/workflows/edit-agent/workflow.yaml.bak +0 -33
- package/bmad/bmb/workflows/edit-module/workflow.yaml.bak +0 -34
- package/bmad/bmb/workflows/edit-workflow/workflow.yaml.bak +0 -27
- package/bmad/bmb/workflows/module-brief/workflow.yaml.bak +0 -29
- package/bmad/bmb/workflows/redoc/workflow.yaml.bak +0 -32
- package/bmad/bmm/docs/brownfield-guide.md +0 -1260
- package/bmad/bmm/docs/quick-spec-flow.md +0 -645
- package/bmad/bmm/docs/quick-start.md +0 -341
- package/bmad/bmm/docs/scale-adaptive-system.md +0 -1045
- package/bmad/bmm/tasks/retrospective.xml +0 -104
- package/bmad/bmm/workflows/1-analysis/brainstorm-project/README.md +0 -113
- package/bmad/bmm/workflows/1-analysis/product-brief/README.md +0 -180
- package/bmad/bmm/workflows/1-analysis/research/README.md +0 -454
- package/bmad/bmm/workflows/2-plan-workflows/README.md +0 -258
- package/bmad/bmm/workflows/3-solutioning/README.md +0 -1
- package/bmad/bmm/workflows/3-solutioning/architecture/README.md +0 -318
- package/bmad/bmm/workflows/3-solutioning/solutioning-gate-check/README.md +0 -177
- package/bmad/bmm/workflows/4-implementation/README.md +0 -221
- package/bmad/bmm/workflows/4-implementation/code-review/README.md +0 -69
- package/bmad/bmm/workflows/4-implementation/correct-course/README.md +0 -73
- package/bmad/bmm/workflows/4-implementation/create-story/README.md +0 -146
- package/bmad/bmm/workflows/4-implementation/dev-story/README.md +0 -206
- package/bmad/bmm/workflows/4-implementation/epic-tech-context/README.md +0 -195
- package/bmad/bmm/workflows/4-implementation/retrospective/README.md +0 -77
- package/bmad/bmm/workflows/4-implementation/sprint-planning/README.md +0 -156
- package/bmad/bmm/workflows/4-implementation/story-context/README.md +0 -234
- package/bmad/bmm/workflows/README.md +0 -256
- package/bmad/bmm/workflows/document-project/README.md +0 -444
- package/bmad/bmm/workflows/document-project/templates/README.md +0 -38
- package/bmad/bmm/workflows/testarch/README.md +0 -26
- package/bmad/bmm/workflows/testarch/atdd/README.md +0 -672
- package/bmad/bmm/workflows/testarch/automate/README.md +0 -869
- package/bmad/bmm/workflows/testarch/ci/README.md +0 -493
- package/bmad/bmm/workflows/testarch/framework/README.md +0 -340
- package/bmad/bmm/workflows/testarch/nfr-assess/README.md +0 -469
- package/bmad/bmm/workflows/testarch/test-design/README.md +0 -493
- package/bmad/bmm/workflows/testarch/test-review/README.md +0 -775
- package/bmad/bmm/workflows/testarch/trace/README.md +0 -802
- package/bmad/bmm/workflows/workflow-status/README.md +0 -260
- package/bmad/bmm/workflows/workflow-status/paths/brownfield-level-0.yaml +0 -54
- package/bmad/bmm/workflows/workflow-status/paths/brownfield-level-2.yaml +0 -76
- package/bmad/bmm/workflows/workflow-status/paths/brownfield-level-4.yaml +0 -88
- package/bmad/bmm/workflows/workflow-status/paths/greenfield-level-0.yaml +0 -45
- package/bmad/bmm/workflows/workflow-status/paths/greenfield-level-3.yaml +0 -73
- package/bmad/bmm/workflows/workflow-status/paths/greenfield-level-4.yaml +0 -75
- package/bmad/cis/README.md +0 -153
- package/bmad/cis/agents/README.md +0 -104
- package/bmad/cis/agents/brainstorming-coach.md +0 -62
- package/bmad/cis/agents/creative-problem-solver.md +0 -62
- package/bmad/cis/agents/design-thinking-coach.md +0 -62
- package/bmad/cis/agents/innovation-strategist.md +0 -62
- package/bmad/cis/agents/storyteller.md +0 -59
- package/bmad/cis/config.yaml +0 -10
- package/bmad/cis/teams/creative-squad.yaml +0 -6
- package/bmad/cis/workflows/README.md +0 -139
- package/bmad/cis/workflows/design-thinking/README.md +0 -56
- package/bmad/cis/workflows/design-thinking/design-methods.csv +0 -31
- package/bmad/cis/workflows/design-thinking/instructions.md +0 -200
- package/bmad/cis/workflows/design-thinking/template.md +0 -111
- package/bmad/cis/workflows/design-thinking/workflow.yaml +0 -32
- package/bmad/cis/workflows/innovation-strategy/README.md +0 -56
- package/bmad/cis/workflows/innovation-strategy/innovation-frameworks.csv +0 -31
- package/bmad/cis/workflows/innovation-strategy/instructions.md +0 -274
- package/bmad/cis/workflows/innovation-strategy/template.md +0 -189
- package/bmad/cis/workflows/innovation-strategy/workflow.yaml +0 -32
- package/bmad/cis/workflows/problem-solving/README.md +0 -56
- package/bmad/cis/workflows/problem-solving/instructions.md +0 -250
- package/bmad/cis/workflows/problem-solving/solving-methods.csv +0 -31
- package/bmad/cis/workflows/problem-solving/template.md +0 -165
- package/bmad/cis/workflows/problem-solving/workflow.yaml +0 -32
- package/bmad/cis/workflows/storytelling/README.md +0 -58
- package/bmad/cis/workflows/storytelling/instructions.md +0 -291
- package/bmad/cis/workflows/storytelling/story-types.csv +0 -26
- package/bmad/cis/workflows/storytelling/template.md +0 -113
- package/bmad/cis/workflows/storytelling/workflow.yaml +0 -32
- package/bmad/core/agents/bmad-master.md.bak +0 -69
- package/src/modules/bmm/docs/brownfield-guide.md.backup +0 -1324
- package/src/modules/bmm/docs/workflows-testing.md +0 -1572
- package/src/modules/bmm/workflows/workflow-status/paths/brownfield-level-0.yaml +0 -54
- package/src/modules/bmm/workflows/workflow-status/paths/brownfield-level-2.yaml +0 -76
- package/src/modules/bmm/workflows/workflow-status/paths/brownfield-level-4.yaml +0 -88
- package/src/modules/bmm/workflows/workflow-status/paths/greenfield-level-0.yaml +0 -45
- package/src/modules/bmm/workflows/workflow-status/paths/greenfield-level-3.yaml +0 -73
- package/src/modules/bmm/workflows/workflow-status/paths/greenfield-level-4.yaml +0 -75
- /package/bmad/_cfg/agents/{bmm-paige.customize.yaml → bmm-tech-writer.customize.yaml} +0 -0
|
@@ -1,869 +0,0 @@
|
|
|
1
|
-
# Automate Workflow
|
|
2
|
-
|
|
3
|
-
Expands test automation coverage by generating comprehensive test suites at appropriate levels (E2E, API, Component, Unit) with supporting infrastructure. This workflow operates in **dual mode** - works seamlessly WITH or WITHOUT BMad artifacts.
|
|
4
|
-
|
|
5
|
-
**Core Principle**: Generate prioritized, deterministic tests that avoid duplicate coverage and follow testing best practices.
|
|
6
|
-
|
|
7
|
-
## Usage
|
|
8
|
-
|
|
9
|
-
```bash
|
|
10
|
-
bmad tea *automate
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
The TEA agent runs this workflow when:
|
|
14
|
-
|
|
15
|
-
- **BMad-Integrated**: After story implementation to expand coverage beyond ATDD tests
|
|
16
|
-
- **Standalone**: Point at any codebase/feature and generate tests independently ("work out of thin air")
|
|
17
|
-
- **Auto-discover**: No targets specified - scans codebase for features needing tests
|
|
18
|
-
|
|
19
|
-
## Inputs
|
|
20
|
-
|
|
21
|
-
**Execution Modes:**
|
|
22
|
-
|
|
23
|
-
1. **BMad-Integrated Mode** (story available) - OPTIONAL
|
|
24
|
-
2. **Standalone Mode** (no BMad artifacts) - Direct code analysis
|
|
25
|
-
3. **Auto-discover Mode** (no targets) - Scan for coverage gaps
|
|
26
|
-
|
|
27
|
-
**Required Context Files:**
|
|
28
|
-
|
|
29
|
-
- **Framework configuration**: Test framework config (playwright.config.ts or cypress.config.ts) - REQUIRED
|
|
30
|
-
|
|
31
|
-
**Optional Context (BMad-Integrated Mode):**
|
|
32
|
-
|
|
33
|
-
- **Story markdown** (`{story_file}`): User story with acceptance criteria (enhances coverage targeting but NOT required)
|
|
34
|
-
- **Tech spec**: Technical specification (provides architectural context)
|
|
35
|
-
- **Test design**: Risk/priority context (P0-P3 alignment)
|
|
36
|
-
- **PRD**: Product requirements (business context)
|
|
37
|
-
|
|
38
|
-
**Optional Context (Standalone Mode):**
|
|
39
|
-
|
|
40
|
-
- **Source code**: Feature implementation to analyze
|
|
41
|
-
- **Existing tests**: Current test suite for gap analysis
|
|
42
|
-
|
|
43
|
-
**Workflow Variables:**
|
|
44
|
-
|
|
45
|
-
- `standalone_mode`: Can work without BMad artifacts (default: true)
|
|
46
|
-
- `story_file`: Path to story markdown (optional)
|
|
47
|
-
- `target_feature`: Feature name or directory to analyze (e.g., "user-authentication" or "src/auth/")
|
|
48
|
-
- `target_files`: Specific files to analyze (comma-separated paths)
|
|
49
|
-
- `test_dir`: Directory for test files (default: `{project-root}/tests`)
|
|
50
|
-
- `source_dir`: Source code directory (default: `{project-root}/src`)
|
|
51
|
-
- `auto_discover_features`: Automatically find features needing tests (default: true)
|
|
52
|
-
- `analyze_coverage`: Check existing test coverage gaps (default: true)
|
|
53
|
-
- `coverage_target`: Coverage strategy - "critical-paths", "comprehensive", "selective" (default: "critical-paths")
|
|
54
|
-
- `test_levels`: Which levels to generate - "e2e,api,component,unit" (default: all)
|
|
55
|
-
- `avoid_duplicate_coverage`: Don't test same behavior at multiple levels (default: true)
|
|
56
|
-
- `include_p0`: Include P0 critical path tests (default: true)
|
|
57
|
-
- `include_p1`: Include P1 high priority tests (default: true)
|
|
58
|
-
- `include_p2`: Include P2 medium priority tests (default: true)
|
|
59
|
-
- `include_p3`: Include P3 low priority tests (default: false)
|
|
60
|
-
- `use_given_when_then`: BDD-style test structure (default: true)
|
|
61
|
-
- `one_assertion_per_test`: Atomic test design (default: true)
|
|
62
|
-
- `network_first`: Route interception before navigation (default: true)
|
|
63
|
-
- `deterministic_waits`: No hard waits or sleeps (default: true)
|
|
64
|
-
- `generate_fixtures`: Create/enhance fixture architecture (default: true)
|
|
65
|
-
- `generate_factories`: Create/enhance data factories (default: true)
|
|
66
|
-
- `update_helpers`: Add utility functions (default: true)
|
|
67
|
-
- `use_test_design`: Load test-design.md if exists (default: true)
|
|
68
|
-
- `use_tech_spec`: Load tech-spec.md if exists (default: true)
|
|
69
|
-
- `use_prd`: Load PRD.md if exists (default: true)
|
|
70
|
-
- `update_readme`: Update test README with new specs (default: true)
|
|
71
|
-
- `update_package_scripts`: Add test execution scripts (default: true)
|
|
72
|
-
- `output_summary`: Path for automation summary (default: `{output_folder}/automation-summary.md`)
|
|
73
|
-
- `max_test_duration`: Maximum seconds per test (default: 90)
|
|
74
|
-
- `max_file_lines`: Maximum lines per test file (default: 300)
|
|
75
|
-
- `require_self_cleaning`: All tests must clean up data (default: true)
|
|
76
|
-
- `auto_load_knowledge`: Load relevant knowledge fragments (default: true)
|
|
77
|
-
- `run_tests_after_generation`: Verify tests pass/fail as expected (default: true)
|
|
78
|
-
- `auto_validate`: Run generated tests after creation (default: true) **NEW**
|
|
79
|
-
- `auto_heal_failures`: Enable automatic healing (default: false, opt-in) **NEW**
|
|
80
|
-
- `max_healing_iterations`: Maximum healing attempts per test (default: 3) **NEW**
|
|
81
|
-
- `fail_on_unhealable`: Fail workflow if tests can't be healed (default: false) **NEW**
|
|
82
|
-
- `mark_unhealable_as_fixme`: Mark unfixable tests with test.fixme() (default: true) **NEW**
|
|
83
|
-
- `use_mcp_healing`: Use Playwright MCP if available (default: true) **NEW**
|
|
84
|
-
- `healing_knowledge_fragments`: Healing patterns to load (default: "test-healing-patterns,selector-resilience,timing-debugging") **NEW**
|
|
85
|
-
|
|
86
|
-
## Outputs
|
|
87
|
-
|
|
88
|
-
**Primary Deliverable:**
|
|
89
|
-
|
|
90
|
-
- **Automation Summary** (`automation-summary.md`): Comprehensive report containing:
|
|
91
|
-
- Execution mode (BMad-Integrated, Standalone, Auto-discover)
|
|
92
|
-
- Feature analysis (source files analyzed, coverage gaps)
|
|
93
|
-
- Tests created (E2E, API, Component, Unit) with counts and paths
|
|
94
|
-
- Infrastructure created (fixtures, factories, helpers)
|
|
95
|
-
- Test execution instructions
|
|
96
|
-
- Coverage analysis (P0-P3 breakdown, coverage percentage)
|
|
97
|
-
- Definition of Done checklist
|
|
98
|
-
- Next steps and recommendations
|
|
99
|
-
|
|
100
|
-
**Test Files Created:**
|
|
101
|
-
|
|
102
|
-
- **E2E tests** (`tests/e2e/{feature-name}.spec.ts`): Critical user journeys (P0-P1)
|
|
103
|
-
- **API tests** (`tests/api/{feature-name}.api.spec.ts`): Business logic and contracts (P1-P2)
|
|
104
|
-
- **Component tests** (`tests/component/{ComponentName}.test.tsx`): UI behavior (P1-P2)
|
|
105
|
-
- **Unit tests** (`tests/unit/{module-name}.test.ts`): Pure logic (P2-P3)
|
|
106
|
-
|
|
107
|
-
**Supporting Infrastructure:**
|
|
108
|
-
|
|
109
|
-
- **Fixtures** (`tests/support/fixtures/{feature}.fixture.ts`): Setup/teardown with auto-cleanup
|
|
110
|
-
- **Data factories** (`tests/support/factories/{entity}.factory.ts`): Random test data using faker
|
|
111
|
-
- **Helpers** (`tests/support/helpers/{utility}.ts`): Utility functions (waitFor, retry, etc.)
|
|
112
|
-
|
|
113
|
-
**Documentation Updates:**
|
|
114
|
-
|
|
115
|
-
- **Test README** (`tests/README.md`): Test suite overview, execution instructions, priority tagging, patterns
|
|
116
|
-
- **package.json scripts**: Test execution commands (test:e2e, test:e2e:p0, test:api, etc.)
|
|
117
|
-
|
|
118
|
-
**Validation Safeguards:**
|
|
119
|
-
|
|
120
|
-
- All tests follow Given-When-Then format
|
|
121
|
-
- All tests have priority tags ([P0], [P1], [P2], [P3])
|
|
122
|
-
- All tests use data-testid selectors (stable, not CSS classes)
|
|
123
|
-
- All tests are self-cleaning (fixtures with auto-cleanup)
|
|
124
|
-
- No hard waits or flaky patterns (deterministic)
|
|
125
|
-
- Test files under 300 lines (lean and focused)
|
|
126
|
-
- Tests run under 1.5 minutes each (fast feedback)
|
|
127
|
-
|
|
128
|
-
## Key Features
|
|
129
|
-
|
|
130
|
-
### Dual-Mode Operation
|
|
131
|
-
|
|
132
|
-
**BMad-Integrated Mode** (story available):
|
|
133
|
-
|
|
134
|
-
- Uses story acceptance criteria for coverage targeting
|
|
135
|
-
- Aligns with test-design risk/priority assessment
|
|
136
|
-
- Expands ATDD tests with edge cases and negative paths
|
|
137
|
-
- Optional - story enhances coverage but not required
|
|
138
|
-
|
|
139
|
-
**Standalone Mode** (no story):
|
|
140
|
-
|
|
141
|
-
- Analyzes source code independently
|
|
142
|
-
- Identifies coverage gaps automatically
|
|
143
|
-
- Generates tests based on code analysis
|
|
144
|
-
- Works with any project (BMad or non-BMad)
|
|
145
|
-
|
|
146
|
-
**Auto-discover Mode** (no targets):
|
|
147
|
-
|
|
148
|
-
- Scans codebase for features needing tests
|
|
149
|
-
- Prioritizes features with no coverage
|
|
150
|
-
- Generates comprehensive test plan
|
|
151
|
-
|
|
152
|
-
### Avoid Duplicate Coverage
|
|
153
|
-
|
|
154
|
-
**Critical principle**: Don't test same behavior at multiple levels
|
|
155
|
-
|
|
156
|
-
**Good coverage strategy:**
|
|
157
|
-
|
|
158
|
-
- **E2E**: User can login → Dashboard loads (critical happy path only)
|
|
159
|
-
- **API**: POST /auth/login returns correct status codes (variations: 200, 401, 400)
|
|
160
|
-
- **Component**: LoginForm validates input (UI edge cases: empty fields, invalid format)
|
|
161
|
-
- **Unit**: validateEmail() logic (pure function edge cases)
|
|
162
|
-
|
|
163
|
-
**Bad coverage (duplicate):**
|
|
164
|
-
|
|
165
|
-
- E2E: User can login → Dashboard loads
|
|
166
|
-
- E2E: User can login with different emails → Dashboard loads (unnecessary duplication)
|
|
167
|
-
- API: POST /auth/login returns 200 (already covered in E2E)
|
|
168
|
-
|
|
169
|
-
Use E2E sparingly for critical paths. Use API/Component/Unit for variations and edge cases.
|
|
170
|
-
|
|
171
|
-
### Healing Capabilities (NEW - Phase 2.5)
|
|
172
|
-
|
|
173
|
-
**automate** automatically validates and heals test failures after generation.
|
|
174
|
-
|
|
175
|
-
**Configuration**: Controlled by `config.tea_use_mcp_enhancements` (default: true)
|
|
176
|
-
|
|
177
|
-
- If true + MCP available → MCP-assisted healing
|
|
178
|
-
- If true + MCP unavailable → Pattern-based healing
|
|
179
|
-
- If false → No healing, document failures for manual review
|
|
180
|
-
|
|
181
|
-
**Constants**: Max 3 healing attempts, unfixable tests marked as `test.fixme()`
|
|
182
|
-
|
|
183
|
-
**How Healing Works (Default - Pattern-Based):**
|
|
184
|
-
|
|
185
|
-
TEA heals tests using pattern-based analysis by:
|
|
186
|
-
|
|
187
|
-
1. **Parsing error messages** from test output logs
|
|
188
|
-
2. **Matching patterns** against known failure signatures
|
|
189
|
-
3. **Applying fixes** from healing knowledge fragments:
|
|
190
|
-
- `test-healing-patterns.md` - Common failure patterns (selectors, timing, data, network)
|
|
191
|
-
- `selector-resilience.md` - Selector refactoring (CSS → data-testid, nth() → filter())
|
|
192
|
-
- `timing-debugging.md` - Race condition fixes (hard waits → event-based waits)
|
|
193
|
-
4. **Re-running tests** to verify fix (max 3 iterations)
|
|
194
|
-
5. **Marking unfixable tests** as `test.fixme()` with detailed comments
|
|
195
|
-
|
|
196
|
-
**This works well for:**
|
|
197
|
-
|
|
198
|
-
- ✅ Common failure patterns (stale selectors, timing issues, dynamic data)
|
|
199
|
-
- ✅ Text-based errors with clear signatures
|
|
200
|
-
- ✅ Issues documented in knowledge base
|
|
201
|
-
- ✅ Automated CI environments without browser access
|
|
202
|
-
|
|
203
|
-
**What MCP Adds (Interactive Debugging Enhancement):**
|
|
204
|
-
|
|
205
|
-
When Playwright MCP is available, TEA **additionally**:
|
|
206
|
-
|
|
207
|
-
1. **Debugs failures interactively** before applying pattern-based fixes:
|
|
208
|
-
- **Pause test execution** with `playwright_test_debug_test` (step through, inspect state)
|
|
209
|
-
- **See visual failure context** with `browser_snapshot` (screenshot of failure state)
|
|
210
|
-
- **Inspect live DOM** with browser tools (find why selector doesn't match)
|
|
211
|
-
- **Analyze console logs** with `browser_console_messages` (JS errors, warnings, debug output)
|
|
212
|
-
- **Inspect network activity** with `browser_network_requests` (failed API calls, CORS errors, timeouts)
|
|
213
|
-
|
|
214
|
-
2. **Enhances pattern-based fixes** with real-world data:
|
|
215
|
-
- **Pattern match identifies issue** (e.g., "stale selector")
|
|
216
|
-
- **MCP discovers actual selector** with `browser_generate_locator` from live page
|
|
217
|
-
- **TEA applies refined fix** using real DOM structure (not just pattern guess)
|
|
218
|
-
- **Verification happens in browser** (see if fix works visually)
|
|
219
|
-
|
|
220
|
-
3. **Catches root causes** pattern matching might miss:
|
|
221
|
-
- **Network failures**: MCP shows 500 error on API call (not just timeout)
|
|
222
|
-
- **JS errors**: MCP shows `TypeError: undefined` in console (not just "element not found")
|
|
223
|
-
- **Timing issues**: MCP shows loading spinner still visible (not just "selector timeout")
|
|
224
|
-
- **State problems**: MCP shows modal blocking button (not just "not clickable")
|
|
225
|
-
|
|
226
|
-
**Key Benefits of MCP Enhancement:**
|
|
227
|
-
|
|
228
|
-
- ✅ **Pattern-based fixes** (fast, automated) **+** **MCP verification** (accurate, context-aware)
|
|
229
|
-
- ✅ **Visual debugging**: See exactly what user sees when test fails
|
|
230
|
-
- ✅ **DOM inspection**: Discover why selectors don't match (element missing, wrong attributes, dynamic IDs)
|
|
231
|
-
- ✅ **Network visibility**: Identify API failures, slow requests, CORS issues
|
|
232
|
-
- ✅ **Console analysis**: Catch JS errors that break page functionality
|
|
233
|
-
- ✅ **Robust selectors**: Generate locators from actual DOM (role, text, testid hierarchy)
|
|
234
|
-
- ✅ **Faster iteration**: Debug and fix in same browser session (no restart needed)
|
|
235
|
-
- ✅ **Higher success rate**: MCP helps diagnose failures pattern matching can't solve
|
|
236
|
-
|
|
237
|
-
**Example Enhancement Flow:**
|
|
238
|
-
|
|
239
|
-
```
|
|
240
|
-
1. Pattern-based healing identifies issue
|
|
241
|
-
→ Error: "Locator '.submit-btn' resolved to 0 elements"
|
|
242
|
-
→ Pattern match: Stale selector (CSS class)
|
|
243
|
-
→ Suggested fix: Replace with data-testid
|
|
244
|
-
|
|
245
|
-
2. MCP enhances diagnosis (if available)
|
|
246
|
-
→ browser_snapshot shows button exists but has class ".submit-button" (not ".submit-btn")
|
|
247
|
-
→ browser_generate_locator finds: button[type="submit"].submit-button
|
|
248
|
-
→ browser_console_messages shows no errors
|
|
249
|
-
|
|
250
|
-
3. TEA applies refined fix
|
|
251
|
-
→ await page.locator('button[type="submit"]').click()
|
|
252
|
-
→ (More accurate than pattern-based guess)
|
|
253
|
-
```
|
|
254
|
-
|
|
255
|
-
**Healing Modes:**
|
|
256
|
-
|
|
257
|
-
1. **MCP-Enhanced Healing** (when Playwright MCP available):
|
|
258
|
-
- Pattern-based analysis **+** Interactive debugging
|
|
259
|
-
- Visual context with `browser_snapshot`
|
|
260
|
-
- Console log analysis with `browser_console_messages`
|
|
261
|
-
- Network inspection with `browser_network_requests`
|
|
262
|
-
- Live DOM inspection with `browser_generate_locator`
|
|
263
|
-
- Step-by-step debugging with `playwright_test_debug_test`
|
|
264
|
-
|
|
265
|
-
2. **Pattern-Based Healing** (always available):
|
|
266
|
-
- Error message parsing and pattern matching
|
|
267
|
-
- Automated fixes from healing knowledge fragments
|
|
268
|
-
- Text-based analysis (no visual/DOM inspection)
|
|
269
|
-
- Works in CI without browser access
|
|
270
|
-
|
|
271
|
-
**Healing Workflow:**
|
|
272
|
-
|
|
273
|
-
```
|
|
274
|
-
1. Generate tests → Run tests
|
|
275
|
-
2. IF pass → Success ✅
|
|
276
|
-
3. IF fail AND auto_heal_failures=false → Report failures ⚠️
|
|
277
|
-
4. IF fail AND auto_heal_failures=true → Enter healing loop:
|
|
278
|
-
a. Identify failure pattern (selector, timing, data, network)
|
|
279
|
-
b. Apply automated fix from knowledge base
|
|
280
|
-
c. Re-run test (max 3 iterations)
|
|
281
|
-
d. IF healed → Success ✅
|
|
282
|
-
e. IF unhealable → Mark test.fixme() with detailed comment
|
|
283
|
-
```
|
|
284
|
-
|
|
285
|
-
**Example Healing Outcomes:**
|
|
286
|
-
|
|
287
|
-
```typescript
|
|
288
|
-
// ❌ Original (failing): CSS class selector
|
|
289
|
-
await page.locator('.btn-primary').click();
|
|
290
|
-
|
|
291
|
-
// ✅ Healed: data-testid selector
|
|
292
|
-
await page.getByTestId('submit-button').click();
|
|
293
|
-
|
|
294
|
-
// ❌ Original (failing): Hard wait
|
|
295
|
-
await page.waitForTimeout(3000);
|
|
296
|
-
|
|
297
|
-
// ✅ Healed: Network-first pattern
|
|
298
|
-
await page.waitForResponse('**/api/data');
|
|
299
|
-
|
|
300
|
-
// ❌ Original (failing): Hardcoded ID
|
|
301
|
-
await expect(page.getByText('User 123')).toBeVisible();
|
|
302
|
-
|
|
303
|
-
// ✅ Healed: Regex pattern
|
|
304
|
-
await expect(page.getByText(/User \d+/)).toBeVisible();
|
|
305
|
-
```
|
|
306
|
-
|
|
307
|
-
**Unfixable Tests (Marked as test.fixme()):**
|
|
308
|
-
|
|
309
|
-
```typescript
|
|
310
|
-
test.fixme('[P1] should handle complex interaction', async ({ page }) => {
|
|
311
|
-
// FIXME: Test healing failed after 3 attempts
|
|
312
|
-
// Failure: "Locator 'button[data-action="submit"]' resolved to 0 elements"
|
|
313
|
-
// Attempted fixes:
|
|
314
|
-
// 1. Replaced with page.getByTestId('submit-button') - still failing
|
|
315
|
-
// 2. Replaced with page.getByRole('button', { name: 'Submit' }) - still failing
|
|
316
|
-
// 3. Added waitForLoadState('networkidle') - still failing
|
|
317
|
-
// Manual investigation needed: Selector may require application code changes
|
|
318
|
-
// TODO: Review with team, may need data-testid added to button component
|
|
319
|
-
// Original test code...
|
|
320
|
-
});
|
|
321
|
-
```
|
|
322
|
-
|
|
323
|
-
**When to Enable Healing:**
|
|
324
|
-
|
|
325
|
-
- ✅ Enable for greenfield projects (catch generated test issues early)
|
|
326
|
-
- ✅ Enable for brownfield projects (auto-fix legacy selector patterns)
|
|
327
|
-
- ❌ Disable if environment not ready (application not deployed/seeded)
|
|
328
|
-
- ❌ Disable if preferring manual review of all generated tests
|
|
329
|
-
|
|
330
|
-
**Healing Report Example:**
|
|
331
|
-
|
|
332
|
-
```markdown
|
|
333
|
-
## Test Healing Report
|
|
334
|
-
|
|
335
|
-
**Auto-Heal Enabled**: true
|
|
336
|
-
**Healing Mode**: Pattern-based
|
|
337
|
-
**Iterations Allowed**: 3
|
|
338
|
-
|
|
339
|
-
### Validation Results
|
|
340
|
-
|
|
341
|
-
- **Total tests**: 10
|
|
342
|
-
- **Passing**: 7
|
|
343
|
-
- **Failing**: 3
|
|
344
|
-
|
|
345
|
-
### Healing Outcomes
|
|
346
|
-
|
|
347
|
-
**Successfully Healed (2 tests):**
|
|
348
|
-
|
|
349
|
-
- `tests/e2e/login.spec.ts:15` - Stale selector (CSS class → data-testid)
|
|
350
|
-
- `tests/e2e/checkout.spec.ts:42` - Race condition (added network-first interception)
|
|
351
|
-
|
|
352
|
-
**Unable to Heal (1 test):**
|
|
353
|
-
|
|
354
|
-
- `tests/e2e/complex-flow.spec.ts:67` - Marked as test.fixme()
|
|
355
|
-
- Requires application code changes (add data-testid to component)
|
|
356
|
-
|
|
357
|
-
### Healing Patterns Applied
|
|
358
|
-
|
|
359
|
-
- **Selector fixes**: 1
|
|
360
|
-
- **Timing fixes**: 1
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
**Graceful Degradation:**
|
|
364
|
-
|
|
365
|
-
- Healing is OPTIONAL (default: disabled)
|
|
366
|
-
- Works without Playwright MCP (pattern-based fallback)
|
|
367
|
-
- Unfixable tests marked clearly (not silently broken)
|
|
368
|
-
- Manual investigation path documented
|
|
369
|
-
|
|
370
|
-
### Recording Mode (NEW - Phase 2.5)
|
|
371
|
-
|
|
372
|
-
**automate** can record complex UI interactions instead of AI generation.
|
|
373
|
-
|
|
374
|
-
**Activation**: Automatic for complex UI scenarios when config.tea_use_mcp_enhancements is true and MCP available
|
|
375
|
-
|
|
376
|
-
- Complex scenarios: drag-drop, wizards, multi-page flows
|
|
377
|
-
- Fallback: AI generation (silent, automatic)
|
|
378
|
-
|
|
379
|
-
**When to Use Recording Mode:**
|
|
380
|
-
|
|
381
|
-
- ✅ Complex UI interactions (drag-drop, multi-step forms, wizards)
|
|
382
|
-
- ✅ Visual workflows (modals, dialogs, animations, transitions)
|
|
383
|
-
- ✅ Unclear requirements (exploratory, discovering behavior)
|
|
384
|
-
- ✅ Multi-page flows (checkout, registration, onboarding)
|
|
385
|
-
- ❌ NOT for simple CRUD (AI generation faster)
|
|
386
|
-
- ❌ NOT for API-only tests (no UI to record)
|
|
387
|
-
|
|
388
|
-
**When to Use AI Generation (Default):**
|
|
389
|
-
|
|
390
|
-
- ✅ Clear requirements available
|
|
391
|
-
- ✅ Standard patterns (login, CRUD, navigation)
|
|
392
|
-
- ✅ Need many tests quickly
|
|
393
|
-
- ✅ API/backend tests (no UI interaction)
|
|
394
|
-
|
|
395
|
-
**Recording Workflow (Same as atdd):**
|
|
396
|
-
|
|
397
|
-
```
|
|
398
|
-
1. Set generation_mode: "recording"
|
|
399
|
-
2. Use generator_setup_page to init recording
|
|
400
|
-
3. For each test scenario:
|
|
401
|
-
- Execute with browser_* tools (navigate, click, type, select)
|
|
402
|
-
- Add verifications with browser_verify_* tools
|
|
403
|
-
- Capture log and generate test file
|
|
404
|
-
4. Enhance with knowledge base patterns:
|
|
405
|
-
- Given-When-Then structure
|
|
406
|
-
- data-testid selectors
|
|
407
|
-
- Network-first interception
|
|
408
|
-
- Fixtures/factories
|
|
409
|
-
5. Validate (run tests if auto_validate enabled)
|
|
410
|
-
6. Heal if needed (if auto_heal_failures enabled)
|
|
411
|
-
```
|
|
412
|
-
|
|
413
|
-
**Combination: Recording + Healing:**
|
|
414
|
-
|
|
415
|
-
automate can use BOTH recording and healing together:
|
|
416
|
-
|
|
417
|
-
- Generate tests via recording (complex flows captured interactively)
|
|
418
|
-
- Run tests to validate (auto_validate)
|
|
419
|
-
- Heal failures automatically (auto_heal_failures)
|
|
420
|
-
|
|
421
|
-
This is particularly powerful for brownfield projects where:
|
|
422
|
-
|
|
423
|
-
- Requirements unclear → Use recording to capture existing behavior
|
|
424
|
-
- Application complex → Recording captures nuances AI might miss
|
|
425
|
-
- Tests may fail → Healing fixes common issues automatically
|
|
426
|
-
|
|
427
|
-
**Graceful Degradation:**
|
|
428
|
-
|
|
429
|
-
- Recording mode is OPTIONAL (default: AI generation)
|
|
430
|
-
- Requires Playwright MCP (falls back to AI if unavailable)
|
|
431
|
-
- Works with or without healing enabled
|
|
432
|
-
- Same quality output regardless of generation method
|
|
433
|
-
|
|
434
|
-
### Test Level Selection Framework
|
|
435
|
-
|
|
436
|
-
**E2E (End-to-End)**:
|
|
437
|
-
|
|
438
|
-
- Critical user journeys (login, checkout, core workflows)
|
|
439
|
-
- Multi-system integration
|
|
440
|
-
- User-facing acceptance criteria
|
|
441
|
-
- Characteristics: High confidence, slow execution, brittle
|
|
442
|
-
|
|
443
|
-
**API (Integration)**:
|
|
444
|
-
|
|
445
|
-
- Business logic validation
|
|
446
|
-
- Service contracts and data transformations
|
|
447
|
-
- Backend integration without UI
|
|
448
|
-
- Characteristics: Fast feedback, good balance, stable
|
|
449
|
-
|
|
450
|
-
**Component**:
|
|
451
|
-
|
|
452
|
-
- UI component behavior (buttons, forms, modals)
|
|
453
|
-
- Interaction testing (click, hover, keyboard navigation)
|
|
454
|
-
- State management within component
|
|
455
|
-
- Characteristics: Fast, isolated, granular
|
|
456
|
-
|
|
457
|
-
**Unit**:
|
|
458
|
-
|
|
459
|
-
- Pure business logic and algorithms
|
|
460
|
-
- Edge cases and error handling
|
|
461
|
-
- Minimal dependencies
|
|
462
|
-
- Characteristics: Fastest, most granular
|
|
463
|
-
|
|
464
|
-
### Priority Classification (P0-P3)
|
|
465
|
-
|
|
466
|
-
**P0 (Critical - Every commit)**:
|
|
467
|
-
|
|
468
|
-
- Critical user paths that must always work
|
|
469
|
-
- Security-critical functionality (auth, permissions)
|
|
470
|
-
- Data integrity scenarios
|
|
471
|
-
- Run in pre-commit hooks or PR checks
|
|
472
|
-
|
|
473
|
-
**P1 (High - PR to main)**:
|
|
474
|
-
|
|
475
|
-
- Important features with high user impact
|
|
476
|
-
- Integration points between systems
|
|
477
|
-
- Error handling for common failures
|
|
478
|
-
- Run before merging to main branch
|
|
479
|
-
|
|
480
|
-
**P2 (Medium - Nightly)**:
|
|
481
|
-
|
|
482
|
-
- Edge cases with moderate impact
|
|
483
|
-
- Less-critical feature variations
|
|
484
|
-
- Performance/load testing
|
|
485
|
-
- Run in nightly CI builds
|
|
486
|
-
|
|
487
|
-
**P3 (Low - On-demand)**:
|
|
488
|
-
|
|
489
|
-
- Nice-to-have validations
|
|
490
|
-
- Rarely-used features
|
|
491
|
-
- Exploratory testing scenarios
|
|
492
|
-
- Run manually or weekly
|
|
493
|
-
|
|
494
|
-
**Priority tagging enables selective execution:**
|
|
495
|
-
|
|
496
|
-
```bash
|
|
497
|
-
npm run test:e2e:p0 # Run only P0 tests (critical paths)
|
|
498
|
-
npm run test:e2e:p1 # Run P0 + P1 tests (pre-merge)
|
|
499
|
-
```
|
|
500
|
-
|
|
501
|
-
### Given-When-Then Test Structure
|
|
502
|
-
|
|
503
|
-
All tests follow BDD format for clarity:
|
|
504
|
-
|
|
505
|
-
```typescript
|
|
506
|
-
test('[P0] should login with valid credentials and load dashboard', async ({ page }) => {
|
|
507
|
-
// GIVEN: User is on login page
|
|
508
|
-
await page.goto('/login');
|
|
509
|
-
|
|
510
|
-
// WHEN: User submits valid credentials
|
|
511
|
-
await page.fill('[data-testid="email-input"]', 'user@example.com');
|
|
512
|
-
await page.fill('[data-testid="password-input"]', 'Password123!');
|
|
513
|
-
await page.click('[data-testid="login-button"]');
|
|
514
|
-
|
|
515
|
-
// THEN: User is redirected to dashboard
|
|
516
|
-
await expect(page).toHaveURL('/dashboard');
|
|
517
|
-
await expect(page.locator('[data-testid="user-name"]')).toBeVisible();
|
|
518
|
-
});
|
|
519
|
-
```
|
|
520
|
-
|
|
521
|
-
### One Assertion Per Test (Atomic Design)
|
|
522
|
-
|
|
523
|
-
Each test verifies exactly one behavior:
|
|
524
|
-
|
|
525
|
-
```typescript
|
|
526
|
-
// ✅ CORRECT: One assertion
|
|
527
|
-
test('[P0] should display user name', async ({ page }) => {
|
|
528
|
-
await expect(page.locator('[data-testid="user-name"]')).toHaveText('John');
|
|
529
|
-
});
|
|
530
|
-
|
|
531
|
-
// ❌ WRONG: Multiple assertions (not atomic)
|
|
532
|
-
test('[P0] should display user info', async ({ page }) => {
|
|
533
|
-
await expect(page.locator('[data-testid="user-name"]')).toHaveText('John');
|
|
534
|
-
await expect(page.locator('[data-testid="user-email"]')).toHaveText('john@example.com');
|
|
535
|
-
});
|
|
536
|
-
```
|
|
537
|
-
|
|
538
|
-
**Why?** If second assertion fails, you don't know if first is still valid. Split into separate tests for clear failure diagnosis.
|
|
539
|
-
|
|
540
|
-
### Network-First Testing Pattern
|
|
541
|
-
|
|
542
|
-
**Critical pattern to prevent race conditions**:
|
|
543
|
-
|
|
544
|
-
```typescript
|
|
545
|
-
test('should load user dashboard after login', async ({ page }) => {
|
|
546
|
-
// CRITICAL: Intercept routes BEFORE navigation
|
|
547
|
-
await page.route('**/api/user', (route) =>
|
|
548
|
-
route.fulfill({
|
|
549
|
-
status: 200,
|
|
550
|
-
body: JSON.stringify({ id: 1, name: 'Test User' }),
|
|
551
|
-
}),
|
|
552
|
-
);
|
|
553
|
-
|
|
554
|
-
// NOW navigate
|
|
555
|
-
await page.goto('/dashboard');
|
|
556
|
-
|
|
557
|
-
await expect(page.locator('[data-testid="user-name"]')).toHaveText('Test User');
|
|
558
|
-
});
|
|
559
|
-
```
|
|
560
|
-
|
|
561
|
-
Always set up route interception before navigating to pages that make network requests.
|
|
562
|
-
|
|
563
|
-
### Fixture Architecture with Auto-Cleanup
|
|
564
|
-
|
|
565
|
-
Playwright fixtures with automatic data cleanup:
|
|
566
|
-
|
|
567
|
-
```typescript
|
|
568
|
-
// tests/support/fixtures/auth.fixture.ts
|
|
569
|
-
import { test as base } from '@playwright/test';
|
|
570
|
-
import { createUser, deleteUser } from '../factories/user.factory';
|
|
571
|
-
|
|
572
|
-
export const test = base.extend({
|
|
573
|
-
authenticatedUser: async ({ page }, use) => {
|
|
574
|
-
// Setup: Create and authenticate user
|
|
575
|
-
const user = await createUser();
|
|
576
|
-
await page.goto('/login');
|
|
577
|
-
await page.fill('[data-testid="email"]', user.email);
|
|
578
|
-
await page.fill('[data-testid="password"]', user.password);
|
|
579
|
-
await page.click('[data-testid="login-button"]');
|
|
580
|
-
await page.waitForURL('/dashboard');
|
|
581
|
-
|
|
582
|
-
// Provide to test
|
|
583
|
-
await use(user);
|
|
584
|
-
|
|
585
|
-
// Cleanup: Delete user automatically
|
|
586
|
-
await deleteUser(user.id);
|
|
587
|
-
},
|
|
588
|
-
});
|
|
589
|
-
```
|
|
590
|
-
|
|
591
|
-
**Fixture principles:**
|
|
592
|
-
|
|
593
|
-
- Auto-cleanup (always delete created data in teardown)
|
|
594
|
-
- Composable (fixtures can use other fixtures)
|
|
595
|
-
- Isolated (each test gets fresh data)
|
|
596
|
-
- Type-safe with TypeScript
|
|
597
|
-
|
|
598
|
-
### Data Factory Architecture
|
|
599
|
-
|
|
600
|
-
Use faker for all test data generation:
|
|
601
|
-
|
|
602
|
-
```typescript
|
|
603
|
-
// tests/support/factories/user.factory.ts
|
|
604
|
-
import { faker } from '@faker-js/faker';
|
|
605
|
-
|
|
606
|
-
export const createUser = (overrides = {}) => ({
|
|
607
|
-
id: faker.number.int(),
|
|
608
|
-
email: faker.internet.email(),
|
|
609
|
-
password: faker.internet.password(),
|
|
610
|
-
name: faker.person.fullName(),
|
|
611
|
-
role: 'user',
|
|
612
|
-
createdAt: faker.date.recent().toISOString(),
|
|
613
|
-
...overrides,
|
|
614
|
-
});
|
|
615
|
-
|
|
616
|
-
export const createUsers = (count: number) => Array.from({ length: count }, () => createUser());
|
|
617
|
-
|
|
618
|
-
// API helper for cleanup
|
|
619
|
-
export const deleteUser = async (userId: number) => {
|
|
620
|
-
await fetch(`/api/users/${userId}`, { method: 'DELETE' });
|
|
621
|
-
};
|
|
622
|
-
```
|
|
623
|
-
|
|
624
|
-
**Factory principles:**
|
|
625
|
-
|
|
626
|
-
- Use faker for random data (no hardcoded values to prevent collisions)
|
|
627
|
-
- Support overrides for specific test scenarios
|
|
628
|
-
- Generate complete valid objects matching API contracts
|
|
629
|
-
- Include helper functions for bulk creation and cleanup
|
|
630
|
-
|
|
631
|
-
### No Page Objects
|
|
632
|
-
|
|
633
|
-
**Do NOT create page object classes.** Keep tests simple and direct:
|
|
634
|
-
|
|
635
|
-
```typescript
|
|
636
|
-
// ✅ CORRECT: Direct test
|
|
637
|
-
test('should login', async ({ page }) => {
|
|
638
|
-
await page.goto('/login');
|
|
639
|
-
await page.fill('[data-testid="email"]', 'user@example.com');
|
|
640
|
-
await page.click('[data-testid="login-button"]');
|
|
641
|
-
await expect(page).toHaveURL('/dashboard');
|
|
642
|
-
});
|
|
643
|
-
|
|
644
|
-
// ❌ WRONG: Page object abstraction
|
|
645
|
-
class LoginPage {
|
|
646
|
-
async login(email, password) { ... }
|
|
647
|
-
}
|
|
648
|
-
```
|
|
649
|
-
|
|
650
|
-
Use fixtures for setup/teardown, not page objects for actions.
|
|
651
|
-
|
|
652
|
-
### Deterministic Tests Only
|
|
653
|
-
|
|
654
|
-
**No flaky patterns allowed:**
|
|
655
|
-
|
|
656
|
-
```typescript
|
|
657
|
-
// ❌ WRONG: Hard wait
|
|
658
|
-
await page.waitForTimeout(2000);
|
|
659
|
-
|
|
660
|
-
// ✅ CORRECT: Explicit wait
|
|
661
|
-
await page.waitForSelector('[data-testid="user-name"]');
|
|
662
|
-
await expect(page.locator('[data-testid="user-name"]')).toBeVisible();
|
|
663
|
-
|
|
664
|
-
// ❌ WRONG: Conditional flow
|
|
665
|
-
if (await element.isVisible()) {
|
|
666
|
-
await element.click();
|
|
667
|
-
}
|
|
668
|
-
|
|
669
|
-
// ✅ CORRECT: Deterministic assertion
|
|
670
|
-
await expect(element).toBeVisible();
|
|
671
|
-
await element.click();
|
|
672
|
-
|
|
673
|
-
// ❌ WRONG: Try-catch for test logic
|
|
674
|
-
try {
|
|
675
|
-
await element.click();
|
|
676
|
-
} catch (e) {
|
|
677
|
-
// Test shouldn't catch errors
|
|
678
|
-
}
|
|
679
|
-
|
|
680
|
-
// ✅ CORRECT: Let test fail if element not found
|
|
681
|
-
await element.click();
|
|
682
|
-
```
|
|
683
|
-
|
|
684
|
-
## Integration with Other Workflows
|
|
685
|
-
|
|
686
|
-
**Before this workflow:**
|
|
687
|
-
|
|
688
|
-
- **framework** workflow: Establish test framework architecture (Playwright/Cypress config, directory structure) - REQUIRED
|
|
689
|
-
- **test-design** workflow: Optional for P0-P3 priority alignment and risk assessment context (BMad-Integrated mode only)
|
|
690
|
-
- **atdd** workflow: Optional - automate expands beyond ATDD tests with edge cases (BMad-Integrated mode only)
|
|
691
|
-
|
|
692
|
-
**After this workflow:**
|
|
693
|
-
|
|
694
|
-
- **trace** workflow: Update traceability matrix with new test coverage (Phase 1) and make quality gate decision (Phase 2)
|
|
695
|
-
- **CI pipeline**: Run tests in burn-in loop to detect flaky patterns
|
|
696
|
-
|
|
697
|
-
**Coordinates with:**
|
|
698
|
-
|
|
699
|
-
- **DEV agent**: Tests validate implementation correctness
|
|
700
|
-
- **Story workflow**: Tests cover acceptance criteria (BMad-Integrated mode only)
|
|
701
|
-
|
|
702
|
-
## Important Notes
|
|
703
|
-
|
|
704
|
-
### Works Out of Thin Air
|
|
705
|
-
|
|
706
|
-
**automate does NOT require BMad artifacts:**
|
|
707
|
-
|
|
708
|
-
- Can analyze any codebase independently
|
|
709
|
-
- User can point TEA at a feature: "automate tests for src/auth/"
|
|
710
|
-
- Works on non-BMad projects
|
|
711
|
-
- BMad artifacts (story, tech-spec, PRD) are OPTIONAL enhancements, not requirements
|
|
712
|
-
|
|
713
|
-
**Similar to:**
|
|
714
|
-
|
|
715
|
-
- **framework**: Can scaffold tests on any project
|
|
716
|
-
- **ci**: Can generate CI config without BMad context
|
|
717
|
-
|
|
718
|
-
**Different from:**
|
|
719
|
-
|
|
720
|
-
- **atdd**: REQUIRES story with acceptance criteria (halt if missing)
|
|
721
|
-
- **test-design**: REQUIRES PRD/epic context (halt if missing)
|
|
722
|
-
- **trace (Phase 2)**: REQUIRES test results for gate decision (halt if missing)
|
|
723
|
-
|
|
724
|
-
### File Size Limits
|
|
725
|
-
|
|
726
|
-
**Keep test files lean (under 300 lines):**
|
|
727
|
-
|
|
728
|
-
- If file exceeds limit, split into multiple files by feature area
|
|
729
|
-
- Group related tests in describe blocks
|
|
730
|
-
- Extract common setup to fixtures
|
|
731
|
-
|
|
732
|
-
### Quality Standards Enforced
|
|
733
|
-
|
|
734
|
-
**Every test must:**
|
|
735
|
-
|
|
736
|
-
- ✅ Use Given-When-Then format
|
|
737
|
-
- ✅ Have clear, descriptive name with priority tag
|
|
738
|
-
- ✅ One assertion per test (atomic)
|
|
739
|
-
- ✅ No hard waits or sleeps
|
|
740
|
-
- ✅ Use data-testid selectors (not CSS classes)
|
|
741
|
-
- ✅ Self-cleaning (fixtures with auto-cleanup)
|
|
742
|
-
- ✅ Deterministic (no flaky patterns)
|
|
743
|
-
- ✅ Fast (under 90 seconds)
|
|
744
|
-
|
|
745
|
-
**Forbidden patterns:**
|
|
746
|
-
|
|
747
|
-
- ❌ Hard waits: `await page.waitForTimeout(2000)`
|
|
748
|
-
- ❌ Conditional flow: `if (await element.isVisible()) { ... }`
|
|
749
|
-
- ❌ Try-catch for test logic
|
|
750
|
-
- ❌ Hardcoded test data (use factories with faker)
|
|
751
|
-
- ❌ Page objects
|
|
752
|
-
- ❌ Shared state between tests
|
|
753
|
-
|
|
754
|
-
## Knowledge Base References
|
|
755
|
-
|
|
756
|
-
This workflow automatically consults:
|
|
757
|
-
|
|
758
|
-
- **test-levels-framework.md** - Test level selection (E2E vs API vs Component vs Unit) with characteristics and use cases
|
|
759
|
-
- **test-priorities.md** - Priority classification (P0-P3) with execution timing and risk alignment
|
|
760
|
-
- **fixture-architecture.md** - Test fixture patterns with setup/teardown and auto-cleanup using Playwright's test.extend()
|
|
761
|
-
- **data-factories.md** - Factory patterns using @faker-js/faker for random test data generation with overrides
|
|
762
|
-
- **selective-testing.md** - Targeted test execution strategies for CI optimization
|
|
763
|
-
- **ci-burn-in.md** - Flaky test detection patterns (10 iterations to catch intermittent failures)
|
|
764
|
-
- **test-quality.md** - Test design principles (Given-When-Then, determinism, isolation, atomic assertions)
|
|
765
|
-
|
|
766
|
-
**Healing Knowledge (If `auto_heal_failures` enabled):**
|
|
767
|
-
|
|
768
|
-
- **test-healing-patterns.md** - Common failure patterns and automated fixes (selectors, timing, data, network, hard waits)
|
|
769
|
-
- **selector-resilience.md** - Robust selector strategies and debugging (data-testid hierarchy, filter vs nth, anti-patterns)
|
|
770
|
-
- **timing-debugging.md** - Race condition identification and deterministic wait fixes (network-first, event-based waits)
|
|
771
|
-
|
|
772
|
-
See `tea-index.csv` for complete knowledge fragment mapping (22 fragments total).
|
|
773
|
-
|
|
774
|
-
## Example Output
|
|
775
|
-
|
|
776
|
-
### BMad-Integrated Mode
|
|
777
|
-
|
|
778
|
-
````markdown
|
|
779
|
-
# Automation Summary - User Authentication
|
|
780
|
-
|
|
781
|
-
**Date:** 2025-10-14
|
|
782
|
-
**Story:** Epic 3, Story 5
|
|
783
|
-
**Coverage Target:** critical-paths
|
|
784
|
-
|
|
785
|
-
## Tests Created
|
|
786
|
-
|
|
787
|
-
### E2E Tests (2 tests, P0-P1)
|
|
788
|
-
|
|
789
|
-
- `tests/e2e/user-authentication.spec.ts` (87 lines)
|
|
790
|
-
- [P0] Login with valid credentials → Dashboard loads
|
|
791
|
-
- [P1] Display error for invalid credentials
|
|
792
|
-
|
|
793
|
-
### API Tests (3 tests, P1-P2)
|
|
794
|
-
|
|
795
|
-
- `tests/api/auth.api.spec.ts` (102 lines)
|
|
796
|
-
- [P1] POST /auth/login - valid credentials → 200 + token
|
|
797
|
-
- [P1] POST /auth/login - invalid credentials → 401 + error
|
|
798
|
-
- [P2] POST /auth/login - missing fields → 400 + validation
|
|
799
|
-
|
|
800
|
-
### Component Tests (2 tests, P1)
|
|
801
|
-
|
|
802
|
-
- `tests/component/LoginForm.test.tsx` (45 lines)
|
|
803
|
-
- [P1] Empty fields → submit button disabled
|
|
804
|
-
- [P1] Valid input → submit button enabled
|
|
805
|
-
|
|
806
|
-
## Infrastructure Created
|
|
807
|
-
|
|
808
|
-
- Fixtures: `tests/support/fixtures/auth.fixture.ts`
|
|
809
|
-
- Factories: `tests/support/factories/user.factory.ts`
|
|
810
|
-
|
|
811
|
-
## Test Execution
|
|
812
|
-
|
|
813
|
-
```bash
|
|
814
|
-
npm run test:e2e # Run all tests
|
|
815
|
-
npm run test:e2e:p0 # Critical paths only
|
|
816
|
-
npm run test:e2e:p1 # P0 + P1 tests
|
|
817
|
-
```
|
|
818
|
-
````
|
|
819
|
-
|
|
820
|
-
## Coverage Analysis
|
|
821
|
-
|
|
822
|
-
**Total:** 7 tests (P0: 1, P1: 5, P2: 1)
|
|
823
|
-
**Levels:** E2E: 2, API: 3, Component: 2
|
|
824
|
-
|
|
825
|
-
✅ All acceptance criteria covered
|
|
826
|
-
✅ Happy path (E2E + API)
|
|
827
|
-
✅ Error cases (API)
|
|
828
|
-
✅ UI validation (Component)
|
|
829
|
-
|
|
830
|
-
````
|
|
831
|
-
|
|
832
|
-
### Standalone Mode
|
|
833
|
-
|
|
834
|
-
```markdown
|
|
835
|
-
# Automation Summary - src/auth/
|
|
836
|
-
|
|
837
|
-
**Date:** 2025-10-14
|
|
838
|
-
**Target:** src/auth/ (standalone analysis)
|
|
839
|
-
**Coverage Target:** critical-paths
|
|
840
|
-
|
|
841
|
-
## Feature Analysis
|
|
842
|
-
|
|
843
|
-
**Source Files Analyzed:**
|
|
844
|
-
- `src/auth/login.ts`
|
|
845
|
-
- `src/auth/session.ts`
|
|
846
|
-
- `src/auth/validation.ts`
|
|
847
|
-
|
|
848
|
-
**Existing Coverage:** 0 tests found
|
|
849
|
-
|
|
850
|
-
**Coverage Gaps:**
|
|
851
|
-
- ❌ No E2E tests for login flow
|
|
852
|
-
- ❌ No API tests for /auth/login endpoint
|
|
853
|
-
- ❌ No unit tests for validateEmail()
|
|
854
|
-
|
|
855
|
-
## Tests Created
|
|
856
|
-
|
|
857
|
-
{Same structure as BMad-Integrated mode}
|
|
858
|
-
|
|
859
|
-
## Recommendations
|
|
860
|
-
|
|
861
|
-
1. **High Priority (P0-P1):**
|
|
862
|
-
- Add E2E test for password reset flow
|
|
863
|
-
- Add API tests for token refresh endpoint
|
|
864
|
-
|
|
865
|
-
2. **Medium Priority (P2):**
|
|
866
|
-
- Add unit tests for session timeout logic
|
|
867
|
-
````
|
|
868
|
-
|
|
869
|
-
Ready to continue?
|