bmad-method-test-architecture-enterprise 1.2.2 → 1.2.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/README.md +14 -12
  2. package/docs/how-to/workflows/setup-ci.md +3 -1
  3. package/docs/how-to/workflows/setup-test-framework.md +29 -6
  4. package/docs/reference/configuration.md +97 -0
  5. package/docs/reference/knowledge-base.md +15 -6
  6. package/package.json +1 -1
  7. package/release_notes.md +6 -4
  8. package/src/agents/tea.agent.yaml +2 -2
  9. package/src/module.yaml +78 -5
  10. package/src/testarch/knowledge/adr-quality-readiness-checklist.md +9 -9
  11. package/src/testarch/tea-index.csv +36 -36
  12. package/src/workflows/testarch/atdd/atdd-checklist-template.md +2 -0
  13. package/src/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md +65 -12
  14. package/src/workflows/testarch/atdd/steps-c/step-02-generation-mode.md +5 -0
  15. package/src/workflows/testarch/atdd/steps-c/step-03-test-strategy.md +10 -1
  16. package/src/workflows/testarch/atdd/steps-c/step-05-validate-and-complete.md +13 -2
  17. package/src/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md +46 -2
  18. package/src/workflows/testarch/automate/steps-c/step-02-identify-targets.md +12 -0
  19. package/src/workflows/testarch/automate/steps-c/step-03-generate-tests.md +110 -31
  20. package/src/workflows/testarch/automate/steps-c/step-03b-subprocess-backend.md +246 -0
  21. package/src/workflows/testarch/automate/steps-c/step-03c-aggregate.md +90 -38
  22. package/src/workflows/testarch/automate/steps-c/step-04-validate-and-summarize.md +13 -2
  23. package/src/workflows/testarch/ci/azure-pipelines-template.yaml +155 -0
  24. package/src/workflows/testarch/ci/checklist.md +48 -7
  25. package/src/workflows/testarch/ci/github-actions-template.yaml +22 -10
  26. package/src/workflows/testarch/ci/gitlab-ci-template.yaml +21 -12
  27. package/src/workflows/testarch/ci/harness-pipeline-template.yaml +159 -0
  28. package/src/workflows/testarch/ci/jenkins-pipeline-template.groovy +129 -0
  29. package/src/workflows/testarch/ci/steps-c/step-01-preflight.md +58 -17
  30. package/src/workflows/testarch/ci/steps-c/step-02-generate-pipeline.md +21 -10
  31. package/src/workflows/testarch/ci/steps-c/step-03-configure-quality-gates.md +5 -0
  32. package/src/workflows/testarch/ci/workflow.yaml +5 -3
  33. package/src/workflows/testarch/framework/checklist.md +11 -10
  34. package/src/workflows/testarch/framework/steps-c/step-01-preflight.md +34 -2
  35. package/src/workflows/testarch/framework/steps-c/step-02-select-framework.md +20 -1
  36. package/src/workflows/testarch/framework/steps-c/step-03-scaffold-framework.md +56 -5
  37. package/src/workflows/testarch/framework/steps-c/step-04-docs-and-scripts.md +16 -4
  38. package/src/workflows/testarch/nfr-assess/nfr-report-template.md +3 -1
  39. package/src/workflows/testarch/nfr-assess/steps-c/step-01-load-context.md +12 -0
  40. package/src/workflows/testarch/nfr-assess/steps-c/step-05-generate-report.md +14 -3
  41. package/src/workflows/testarch/test-design/checklist.md +20 -9
  42. package/src/workflows/testarch/test-design/instructions.md +3 -3
  43. package/src/workflows/testarch/test-design/steps-c/step-02-load-context.md +34 -0
  44. package/src/workflows/testarch/test-design/steps-c/step-05-generate-output.md +29 -2
  45. package/src/workflows/testarch/test-design/test-design-architecture-template.md +16 -14
  46. package/src/workflows/testarch/test-design/test-design-handoff-template.md +70 -0
  47. package/src/workflows/testarch/test-design/test-design-qa-template.md +11 -9
  48. package/src/workflows/testarch/test-design/workflow.yaml +8 -1
  49. package/src/workflows/testarch/test-review/steps-c/step-01-load-context.md +34 -1
  50. package/src/workflows/testarch/test-review/steps-c/step-04-generate-report.md +14 -3
  51. package/src/workflows/testarch/test-review/test-review-template.md +4 -2
  52. package/src/workflows/testarch/test-review/workflow.yaml +1 -0
  53. package/src/workflows/testarch/trace/trace-template.md +7 -5
  54. package/test/test-installation-components.js +1 -1
  55. package/test/test-knowledge-base.js +10 -1
@@ -2,7 +2,6 @@
2
2
  name: 'step-02-generate-pipeline'
3
3
  description: 'Generate CI pipeline configuration'
4
4
  nextStepFile: './step-03-configure-quality-gates.md'
5
- pipelineOutputFile: '{project-root}/.github/workflows/test.yml'
6
5
  outputFile: '{test_artifacts}/ci-pipeline-progress.md'
7
6
  ---
8
7
 
@@ -36,16 +35,20 @@ Create platform-specific CI configuration with test execution, sharding, burn-in
36
35
 
37
36
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
38
37
 
39
- ## 1. Select Template
38
+ ## 1. Resolve Output Path and Select Template
40
39
 
41
- Choose template based on platform:
40
+ Determine the pipeline output file path based on the detected `ci_platform`:
42
41
 
43
- - GitHub Actions `.github/workflows/test.yml`
44
- - GitLab CI `.gitlab-ci.yml`
45
- - Circle CI `.circleci/config.yml`
46
- - Jenkins `Jenkinsfile`
42
+ | CI Platform | Output Path | Template File |
43
+ | ---------------- | ------------------------------------------- | --------------------------------------------------- |
44
+ | `github-actions` | `{project-root}/.github/workflows/test.yml` | `{installed_path}/github-actions-template.yaml` |
45
+ | `gitlab-ci` | `{project-root}/.gitlab-ci.yml` | `{installed_path}/gitlab-ci-template.yaml` |
46
+ | `jenkins` | `{project-root}/Jenkinsfile` | `{installed_path}/jenkins-pipeline-template.groovy` |
47
+ | `azure-devops` | `{project-root}/azure-pipelines.yml` | `{installed_path}/azure-pipelines-template.yaml` |
48
+ | `harness` | `{project-root}/.harness/pipeline.yaml` | `{installed_path}/harness-pipeline-template.yaml` |
49
+ | `circle-ci` | `{project-root}/.circleci/config.yml` | _(no template; generate from first principles)_ |
47
50
 
48
- Use templates from `{installed_path}` when available (e.g., `github-actions-template.yaml`, `gitlab-ci-template.yaml`).
51
+ Use templates from `{installed_path}` when available. Adapt the template to the project's `test_stack_type` and `test_framework`.
49
52
 
50
53
  ---
51
54
 
@@ -65,9 +68,17 @@ Include stages:
65
68
  - Parallel sharding enabled
66
69
  - CI retries configured
67
70
  - Capture artifacts (HTML report, JUnit XML, traces/videos on failure)
68
- - Cache dependencies (node_modules / pnpm / npm cache)
71
+ - Cache dependencies (language-appropriate: node_modules, .venv, .m2, go module cache, NuGet, bundler)
69
72
 
70
- Write the selected pipeline configuration to `{pipelineOutputFile}` (or adjust the path if a non-GitHub platform was chosen).
73
+ Write the selected pipeline configuration to the resolved output path from step 1. Adjust test commands based on `test_stack_type` and `test_framework`:
74
+
75
+ - **Frontend/Fullstack**: Include browser install, E2E/component test commands, Playwright/Cypress artifacts
76
+ - **Backend (Node.js)**: Use `npm test` or framework-specific commands (`vitest`, `jest`), skip browser install
77
+ - **Backend (Python)**: Use `pytest` with coverage (`pytest --cov`), install via `pip install -r requirements.txt` or `poetry install`
78
+ - **Backend (Java/Kotlin)**: Use `mvn test` or `gradle test`, cache `.m2/repository` or `.gradle/caches`
79
+ - **Backend (Go)**: Use `go test ./...` with coverage (`-coverprofile`), cache Go modules
80
+ - **Backend (C#/.NET)**: Use `dotnet test` with coverage, restore NuGet packages
81
+ - **Backend (Ruby)**: Use `bundle exec rspec` with coverage, cache `vendor/bundle`
71
82
 
72
83
  ---
73
84
 
@@ -43,6 +43,11 @@ Use `{knowledgeIndex}` to load `ci-burn-in.md` guidance:
43
43
  - Run N-iteration burn-in for flaky detection
44
44
  - Gate promotion based on burn-in stability
45
45
 
46
+ **Stack-conditional burn-in:**
47
+
48
+ - **Frontend or Fullstack** (`test_stack_type` is `frontend` or `fullstack`): Enable burn-in by default. Burn-in targets UI flakiness (race conditions, selector instability, timing issues).
49
+ - **Backend only** (`test_stack_type` is `backend`): Skip burn-in by default. Backend tests (unit, integration, API) are deterministic and rarely exhibit UI-related flakiness. If the user explicitly requests burn-in for backend, honor that override.
50
+
46
51
  ---
47
52
 
48
53
  ## 2. Quality Gates
@@ -19,11 +19,13 @@ validation: "{installed_path}/checklist.md"
19
19
 
20
20
  # Variables and inputs
21
21
  variables:
22
- ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins - user can override
22
+ ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins, azure-devops, harness - user can override
23
23
  test_dir: "{project-root}/tests" # Root test directory
24
+ test_stack_type: "auto" # auto, frontend, backend, fullstack - detected or user override
25
+ test_framework: "auto" # auto, playwright, cypress, jest, vitest - detected or user override
24
26
 
25
- # Output configuration
26
- default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default
27
+ # Output configuration (resolved dynamically based on ci_platform detection)
28
+ default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default; overridden per platform
27
29
 
28
30
  # Required tools
29
31
  required_tools:
@@ -8,10 +8,10 @@ This checklist ensures the framework workflow completes successfully and all del
8
8
 
9
9
  Before starting the workflow:
10
10
 
11
- - [ ] Project root contains valid `package.json`
12
- - [ ] No existing modern E2E framework detected (`playwright.config.*`, `cypress.config.*`)
13
- - [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, etc.)
14
- - [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable
11
+ - [ ] Project root contains a valid project manifest (`package.json`, `pyproject.toml`, `pom.xml`, `build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, or `Cargo.toml`)
12
+ - [ ] No existing test framework detected that conflicts with the target setup
13
+ - [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, Python, Java, Go, .NET, Ruby, Rust, etc.)
14
+ - [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable (backend projects)
15
15
  - [ ] User has write permissions to create directories and files
16
16
 
17
17
  ---
@@ -20,7 +20,8 @@ Before starting the workflow:
20
20
 
21
21
  ### Step 1: Preflight Checks
22
22
 
23
- - [ ] package.json successfully read and parsed
23
+ - [ ] Stack type detected (`frontend`, `backend`, or `fullstack`)
24
+ - [ ] Project manifest successfully read and parsed (`package.json`, `pyproject.toml`, `pom.xml`, `go.mod`, etc.)
24
25
  - [ ] Project type extracted correctly
25
26
  - [ ] Bundler identified (or marked as N/A for backend projects)
26
27
  - [ ] No framework conflicts detected
@@ -29,8 +30,8 @@ Before starting the workflow:
29
30
  ### Step 2: Framework Selection
30
31
 
31
32
  - [ ] Framework auto-detection logic executed
32
- - [ ] Framework choice justified (Playwright vs Cypress)
33
- - [ ] Framework preference respected (if explicitly set)
33
+ - [ ] Framework choice justified (Playwright vs Cypress for frontend; pytest/JUnit/Go test/xUnit/RSpec for backend)
34
+ - [ ] Framework preference respected (if explicitly set via `config.test_framework`)
34
35
  - [ ] User notified of framework selection and rationale
35
36
 
36
37
  ### Step 3: Directory Structure
@@ -115,9 +116,9 @@ Before starting the workflow:
115
116
  - [ ] Knowledge base references included
116
117
  - [ ] Troubleshooting section included
117
118
 
118
- ### Step 11: Package.json Updates
119
+ ### Step 11: Build & Test Script Updates
119
120
 
120
- - [ ] Minimal test script added to package.json: `test:e2e`
121
+ - [ ] Minimal test script added to appropriate config (`package.json` for frontend, `Makefile`/`pyproject.toml`/`build.gradle` for backend)
121
122
  - [ ] Test framework dependency added (if not already present)
122
123
  - [ ] Type definitions added (if TypeScript)
123
124
  - [ ] Users can extend with additional scripts as needed
@@ -231,7 +232,7 @@ Before starting the workflow:
231
232
  - [ ] All quality checks passed
232
233
  - [ ] All integration points verified
233
234
  - [ ] Sample test executes successfully
234
- - [ ] User can run `npm run test:e2e` without errors
235
+ - [ ] User can run the appropriate test command without errors (`npm run test:e2e`, `pytest`, `go test ./...`, etc.)
235
236
  - [ ] Documentation is complete and accurate
236
237
  - [ ] No critical issues or blockers identified
237
238
 
@@ -36,19 +36,51 @@ Verify the project is ready for framework scaffolding and gather key context.
36
36
 
37
37
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
38
38
 
39
- ## 1. Validate Prerequisites
39
+ ## 1. Stack Detection
40
+
41
+ **Read `config.test_stack_type`** from `{config_source}`.
42
+
43
+ **Auto-Detection Algorithm** (when `test_stack_type` is `"auto"` or not configured):
44
+
45
+ - Scan `{project-root}` for project manifests:
46
+ - **Frontend indicators**: `package.json` with react/vue/angular/next dependencies, `playwright.config.*`, `vite.config.*`, `webpack.config.*`
47
+ - **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`
48
+ - **Both present** = `fullstack`; only frontend = `frontend`; only backend = `backend`
49
+ - Explicit `test_stack_type` config value overrides auto-detection
50
+ - **Backward compatibility**: if `test_stack_type` is not in config, treat as `"auto"` (preserves current frontend behavior for existing installs)
51
+
52
+ Store result as `{detected_stack}` = `frontend` | `backend` | `fullstack`
53
+
54
+ ---
55
+
56
+ ## 2. Validate Prerequisites
57
+
58
+ **If {detected_stack} is `frontend` or `fullstack`:**
40
59
 
41
60
  - `package.json` exists in project root
42
61
  - No existing E2E framework (`playwright.config.*`, `cypress.config.*`, `cypress.json`)
62
+
63
+ **If {detected_stack} is `backend` or `fullstack`:**
64
+
65
+ - At least one backend project manifest exists (`pyproject.toml`, `pom.xml`, `build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`)
66
+ - No existing test framework config that conflicts (e.g., `conftest.py` with full pytest suite, `src/test/` with JUnit suite)
67
+
43
68
  - Architecture/stack context available (project type, bundler, dependencies)
44
69
 
45
70
  If any fail, **HALT** and report the missing requirement.
46
71
 
47
72
  ---
48
73
 
49
- ## 2. Gather Project Context
74
+ ## 3. Gather Project Context
75
+
76
+ **If {detected_stack} is `frontend` or `fullstack`:**
50
77
 
51
78
  - Read `package.json` to identify framework, bundler, dependencies
79
+
80
+ **If {detected_stack} is `backend` or `fullstack`:**
81
+
82
+ - Read the relevant project manifest (`pyproject.toml`, `pom.xml`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`) to identify language, framework, and dependencies
83
+
52
84
  - Check for architecture docs (`architecture.md`, `tech-spec*.md`) if available
53
85
  - Note auth requirements and APIs (if documented)
54
86
 
@@ -37,6 +37,10 @@ Choose the most appropriate framework and document the rationale.
37
37
 
38
38
  ## 1. Selection Logic
39
39
 
40
+ Use `{detected_stack}` from Step 1 to guide framework selection.
41
+
42
+ **If {detected_stack} is `frontend` or `fullstack` (browser-based testing):**
43
+
40
44
  Default to **Playwright** unless strong reasons suggest Cypress.
41
45
 
42
46
  **Playwright recommended when:**
@@ -52,7 +56,22 @@ Default to **Playwright** unless strong reasons suggest Cypress.
52
56
  - Component testing focus
53
57
  - Simpler setup needed
54
58
 
55
- Respect `framework_preference` if explicitly set.
59
+ **If {detected_stack} is `backend` (no browser-based testing):**
60
+
61
+ Select the framework matching the project language:
62
+
63
+ - **Python**: pytest (default), unittest
64
+ - **Java/Kotlin**: JUnit 5 (default), TestNG
65
+ - **Go**: Go test (built-in)
66
+ - **C#/.NET**: xUnit (default), NUnit, MSTest
67
+ - **Ruby**: RSpec (default), Minitest
68
+ - **Rust**: cargo test (built-in)
69
+
70
+ **If {detected_stack} is `fullstack`:**
71
+
72
+ Select both a browser-based framework (Playwright/Cypress) AND the appropriate backend framework for the detected language.
73
+
74
+ Respect `config.test_framework` if explicitly set (not `"auto"`).
56
75
 
57
76
  ---
58
77
 
@@ -39,17 +39,32 @@ Generate the test directory structure, configuration files, fixtures, factories,
39
39
 
40
40
  ## 1. Create Directory Structure
41
41
 
42
- Create:
42
+ Use `{detected_stack}` from Step 1 to determine directory layout.
43
+
44
+ **If {detected_stack} is `frontend` or `fullstack`:**
43
45
 
44
46
  - `{test_dir}/e2e/`
45
47
  - `{test_dir}/support/fixtures/`
46
48
  - `{test_dir}/support/helpers/`
47
49
  - `{test_dir}/support/page-objects/` (optional)
48
50
 
51
+ **If {detected_stack} is `backend` or `fullstack`:**
52
+
53
+ Create the idiomatic test directory for the detected language:
54
+
55
+ - **Python (pytest)**: `tests/` with `conftest.py`, `tests/unit/`, `tests/integration/`, `tests/api/`
56
+ - **Java/Kotlin (JUnit)**: `src/test/java/` mirroring `src/main/java/` package structure, with `unit/`, `integration/`, `api/` sub-packages
57
+ - **Go**: `*_test.go` files alongside source files (Go convention), plus `testdata/` for fixtures
58
+ - **C#/.NET (xUnit)**: `tests/` project with `Unit/`, `Integration/`, `Api/` directories
59
+ - **Ruby (RSpec)**: `spec/` with `spec/unit/`, `spec/integration/`, `spec/api/`, `spec/support/`
60
+ - **Rust**: `tests/` for integration tests, inline `#[cfg(test)]` modules for unit tests
61
+
49
62
  ---
50
63
 
51
64
  ## 2. Generate Framework Config
52
65
 
66
+ **If {detected_stack} is `frontend` or `fullstack`:**
67
+
53
68
  Create `playwright.config.ts` or `cypress.config.ts` with:
54
69
 
55
70
  - **Timeouts**: action 15s, navigation 30s, test 60s
@@ -60,15 +75,38 @@ Create `playwright.config.ts` or `cypress.config.ts` with:
60
75
 
61
76
  Use TypeScript if `use_typescript: true`.
62
77
 
78
+ **If {detected_stack} is `backend` or `fullstack`:**
79
+
80
+ Create the idiomatic test config for the detected framework:
81
+
82
+ - **pytest**: `pyproject.toml` `[tool.pytest.ini_options]` or `pytest.ini` with markers, test paths, coverage settings
83
+ - **JUnit**: `build.gradle`/`pom.xml` test configuration with JUnit 5 dependencies, Surefire/Failsafe plugins
84
+ - **Go test**: no config file needed (Go convention); optionally create `Makefile` test targets
85
+ - **xUnit**: `.csproj` test project with xUnit and coverlet dependencies
86
+ - **RSpec**: `.rspec` config file with `spec_helper.rb` and `rails_helper.rb` (if Rails)
87
+
63
88
  ---
64
89
 
65
- ## 3. Environment & Node
90
+ ## 3. Environment Setup
91
+
92
+ Create `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`.
66
93
 
67
- Create:
94
+ **Stack-conditional environment files:**
95
+
96
+ **If {detected_stack} is `frontend` or `fullstack` (Node.js):**
68
97
 
69
- - `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`
70
98
  - `.nvmrc` using current LTS Node (prefer Node 24+)
71
99
 
100
+ **If {detected_stack} is `backend`:**
101
+
102
+ Create the idiomatic version file for the detected language:
103
+
104
+ - **Python**: `.python-version` with current stable Python (prefer 3.12+)
105
+ - **Java**: `.java-version` or `JAVA_HOME` documentation in `.env.example`
106
+ - **Go**: Go version is already in `go.mod` (no additional file needed)
107
+ - **C#/.NET**: `global.json` with SDK version if not already present
108
+ - **Ruby**: `.ruby-version` with current stable Ruby
109
+
72
110
  ---
73
111
 
74
112
  ## 4. Fixtures & Factories
@@ -94,6 +132,8 @@ Implement:
94
132
 
95
133
  ## 5. Sample Tests & Helpers
96
134
 
135
+ **If {detected_stack} is `frontend` or `fullstack`:**
136
+
97
137
  Create example tests in `{test_dir}/e2e/` demonstrating:
98
138
 
99
139
  - Given/When/Then format
@@ -101,11 +141,22 @@ Create example tests in `{test_dir}/e2e/` demonstrating:
101
141
  - Factory usage
102
142
  - Network interception pattern (if applicable)
103
143
 
144
+ **If {detected_stack} is `backend` or `fullstack`:**
145
+
146
+ Create example tests in the idiomatic location for the detected language:
147
+
148
+ - **Python**: `tests/test_example.py` with pytest fixtures, parametrize, and factory usage
149
+ - **Java**: `src/test/java/.../ExampleTest.java` with JUnit 5 annotations, `@BeforeEach` setup
150
+ - **Go**: `example_test.go` alongside source with table-driven tests and `testify` assertions
151
+ - **C#/.NET**: `tests/ExampleTests.cs` with xUnit `[Fact]`/`[Theory]` and fixture injection
152
+ - **Ruby**: `spec/example_spec.rb` with RSpec `describe`/`context`/`it` and factory_bot
153
+
104
154
  Create helpers for:
105
155
 
106
156
  - API clients (if needed)
107
- - Network utilities
157
+ - Network utilities (frontend/fullstack only)
108
158
  - Auth helpers
159
+ - Test data factories (language-idiomatic patterns)
109
160
 
110
161
  ---
111
162
 
@@ -10,7 +10,7 @@ progressFile: '{test_artifacts}/framework-setup-progress.md'
10
10
 
11
11
  ## STEP GOAL
12
12
 
13
- Create `tests/README.md` and update `package.json` scripts.
13
+ Create test documentation and add build/test scripts appropriate for `{detected_stack}`.
14
14
 
15
15
  ## MANDATORY EXECUTION RULES
16
16
 
@@ -49,11 +49,23 @@ Create `{outputFile}` and include:
49
49
 
50
50
  ---
51
51
 
52
- ## 2. package.json Scripts
52
+ ## 2. Build & Test Scripts
53
53
 
54
- Add at minimum:
54
+ **If {detected_stack} is `frontend` or `fullstack`:**
55
55
 
56
- - `test:e2e`: framework execution command
56
+ Add to `package.json` at minimum:
57
+
58
+ - `test:e2e`: framework execution command (e.g., `npx playwright test`)
59
+
60
+ **If {detected_stack} is `backend` or `fullstack`:**
61
+
62
+ Add the idiomatic test commands for the detected framework:
63
+
64
+ - **Python (pytest)**: Add to `pyproject.toml` scripts or `Makefile`: `pytest`, `pytest --cov`, `pytest -m integration`
65
+ - **Java (JUnit)**: Add to `build.gradle`/`pom.xml`: `./gradlew test`, `mvn test`, `mvn verify` (integration)
66
+ - **Go**: Add to `Makefile`: `go test ./...`, `go test -race ./...`, `go test -cover ./...`
67
+ - **C#/.NET**: Add to CI scripts or `Makefile`: `dotnet test`, `dotnet test --collect:"XPlat Code Coverage"`
68
+ - **Ruby (RSpec)**: Add to `Gemfile` binstubs or `Makefile`: `bundle exec rspec`, `bundle exec rspec spec/integration`
57
69
 
58
70
  ---
59
71
 
@@ -2,6 +2,8 @@
2
2
  stepsCompleted: []
3
3
  lastStep: ''
4
4
  lastSaved: ''
5
+ workflowType: 'testarch-nfr-assess'
6
+ inputDocuments: []
5
7
  ---
6
8
 
7
9
  # NFR Assessment - {FEATURE_NAME}
@@ -262,7 +264,7 @@ Note: This assessment summarizes existing evidence; it does not run tests or CI
262
264
  - {SPECIFIC_STEPS}
263
265
  - {VALIDATION_CRITERIA}
264
266
 
265
- ### Short-term (Next Sprint) - MEDIUM Priority
267
+ ### Short-term (Next Milestone) - MEDIUM Priority
266
268
 
267
269
  1. **{ACTION_TITLE_3}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
268
270
  - {ACTION_DESCRIPTION}
@@ -54,6 +54,16 @@ From `{config_source}`:
54
54
 
55
55
  ---
56
56
 
57
+ ### Tiered Knowledge Loading
58
+
59
+ Load fragments based on their `tier` classification in `tea-index.csv`:
60
+
61
+ 1. **Core tier** (always load): Foundational fragments required for this workflow
62
+ 2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
63
+ 3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
64
+
65
+ > **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
66
+
57
67
  ## 3. Load Knowledge Base Fragments
58
68
 
59
69
  From `{knowledgeIndex}` load:
@@ -112,6 +122,8 @@ Summarize loaded NFR sources and evidence availability.
112
122
  - Set `lastSaved: '{date}'`
113
123
  - Append this step's output to the appropriate section of the document.
114
124
 
125
+ **Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
126
+
115
127
  Load next step: `{nextStepFile}`
116
128
 
117
129
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
@@ -45,7 +45,18 @@ Use `nfr-report-template.md` to produce `{outputFile}` containing:
45
45
 
46
46
  ---
47
47
 
48
- ## 2. Validation
48
+ ## 2. Polish Output
49
+
50
+ Before finalizing, review the complete output document for quality:
51
+
52
+ 1. **Remove duplication**: Progressive-append workflow may have created repeated sections — consolidate
53
+ 2. **Verify consistency**: Ensure terminology, risk scores, and references are consistent throughout
54
+ 3. **Check completeness**: All template sections should be populated or explicitly marked N/A
55
+ 4. **Format cleanup**: Ensure markdown formatting is clean (tables aligned, headers consistent, no orphaned references)
56
+
57
+ ---
58
+
59
+ ## 3. Validation
49
60
 
50
61
  Validate against `checklist.md` and fix gaps.
51
62
 
@@ -53,7 +64,7 @@ Validate against `checklist.md` and fix gaps.
53
64
 
54
65
  ---
55
66
 
56
- ## 3. Save Progress
67
+ ## 4. Save Progress
57
68
 
58
69
  **Save this step's accumulated work to `{outputFile}`.**
59
70
 
@@ -77,7 +88,7 @@ Validate against `checklist.md` and fix gaps.
77
88
 
78
89
  ---
79
90
 
80
- ## 4. Completion Summary
91
+ ## 5. Completion Summary
81
92
 
82
93
  Report:
83
94
 
@@ -190,7 +190,7 @@
190
190
  - [ ] Prerequisites for testing start are clearly defined
191
191
  - [ ] Environment readiness included
192
192
  - [ ] Test data readiness included
193
- - [ ] Sprint 0 blocker resolution referenced
193
+ - [ ] Pre-implementation blocker resolution referenced
194
194
 
195
195
  ### Exit Criteria
196
196
 
@@ -225,7 +225,7 @@
225
225
  - [ ] **Purpose statement** at top (serves as contract with Architecture team)
226
226
  - [ ] **Executive Summary** with scope, business context, architecture decisions, risk summary
227
227
  - [ ] **Quick Guide** section with three tiers:
228
- - [ ] 🚨 BLOCKERS - Team Must Decide (Sprint 0 critical path items)
228
+ - [ ] 🚨 BLOCKERS - Team Must Decide (pre-implementation critical path items)
229
229
  - [ ] ⚠️ HIGH PRIORITY - Team Should Validate (recommendations for approval)
230
230
  - [ ] 📋 INFO ONLY - Solutions Provided (no decisions needed)
231
231
  - [ ] **Risk Assessment** section - **ACTIONABLE**
@@ -262,7 +262,7 @@
262
262
  - [ ] NO "Test Levels Strategy" section (unit/integration/E2E split belongs in QA doc only)
263
263
  - [ ] NO "NFR Testing Approach" section with detailed test procedures (belongs in QA doc only)
264
264
  - [ ] NO "Test Environment Requirements" section (belongs in QA doc only)
265
- - [ ] NO "Recommendations for Sprint 0" section with test framework setup (belongs in QA doc only)
265
+ - [ ] NO "Recommendations for pre-implementation" section with test framework setup (belongs in QA doc only)
266
266
  - [ ] NO "Quality Gate Criteria" section (pass rates, coverage targets belong in QA doc only)
267
267
  - [ ] NO "Tool Selection" section (Playwright, k6, etc. belongs in QA doc only)
268
268
 
@@ -295,11 +295,11 @@
295
295
  - [ ] **QA Effort Estimate** section (QA effort ONLY)
296
296
  - [ ] Interval-based estimates (e.g., "~1-2 weeks" NOT "36 hours")
297
297
  - [ ] NO DevOps, Backend, Data Eng, Finance effort
298
- - [ ] No per-sprint effort breakdowns in this section
299
- - [ ] **Sprint Planning Handoff** section (optional)
298
+ - [ ] No per-milestone effort breakdowns in this section
299
+ - [ ] **Implementation Planning Handoff** section (optional)
300
300
  - [ ] Only include if implementation tasks must be scheduled
301
301
  - [ ] Owners assigned (QA/Dev/Platform/etc)
302
- - [ ] Target sprint may be noted, but avoid detailed per-sprint breakdowns
302
+ - [ ] Target milestone may be noted, but avoid detailed per-milestone breakdowns
303
303
  - [ ] **Appendix A: Code Examples & Tagging**
304
304
  - [ ] **Appendix B: Knowledge Base References**
305
305
 
@@ -315,14 +315,14 @@
315
315
  - [ ] ❌ NO Follow-on Workflows section (BMAD commands self-explanatory)
316
316
  - [ ] ❌ NO Approval section
317
317
  - [ ] ❌ NO Infrastructure/DevOps/Finance effort tables (out of scope)
318
- - [ ] ❌ NO detailed sprint-by-sprint breakdown tables (use Sprint Planning Handoff if needed)
319
- - [ ] ❌ NO generic Next Steps section (use Sprint Planning Handoff if needed)
318
+ - [ ] ❌ NO detailed milestone-by-milestone breakdown tables (use Implementation Planning Handoff if needed)
319
+ - [ ] ❌ NO generic Next Steps section (use Implementation Planning Handoff if needed)
320
320
 
321
321
  ### Cross-Document Consistency
322
322
 
323
323
  - [ ] Both documents reference same risks by ID (R-001, R-002, etc.)
324
324
  - [ ] Both documents use consistent priority levels (P0, P1, P2, P3)
325
- - [ ] Both documents reference same Sprint 0 blockers
325
+ - [ ] Both documents reference same pre-implementation blockers
326
326
  - [ ] No duplicate content (cross-reference instead)
327
327
  - [ ] Dates and authors match across documents
328
328
  - [ ] ADR and PRD references consistent
@@ -361,6 +361,16 @@
361
361
  - [ ] Actionable ASRs included in 🚨 or ⚠️ sections
362
362
  - [ ] FYI ASRs included in 📋 section or omitted if obvious
363
363
 
364
+ ## BMAD Handoff Validation (System-Level Mode Only)
365
+
366
+ - [ ] Handoff document generated at `{test_artifacts}/test-design/{project_name}-handoff.md`
367
+ - [ ] TEA Artifacts Inventory table populated with actual paths
368
+ - [ ] Epic-Level Integration Guidance populated with P0/P1 risks
369
+ - [ ] Story-Level Integration Guidance populated with critical test scenarios
370
+ - [ ] Risk-to-Story Mapping table populated from risk register
371
+ - [ ] Recommended workflow sequence is accurate
372
+ - [ ] Phase transition quality gates are defined
373
+
364
374
  ## Completion Criteria
365
375
 
366
376
  **All must be true:**
@@ -372,6 +382,7 @@
372
382
  - [ ] All integration points verified
373
383
  - [ ] Output file(s) complete and well-formatted
374
384
  - [ ] **System-level mode:** Both documents validated (if applicable)
385
+ - [ ] **System-level mode:** Handoff document validated (if applicable)
375
386
  - [ ] **Epic-level mode:** Single document validated (if applicable)
376
387
  - [ ] Team review scheduled (if required)
377
388
 
@@ -74,7 +74,7 @@ When populating templates in step 5, apply the following guidance for these sect
74
74
 
75
75
  - **Entry criteria**: Derive from Dependencies and Test Blockers -- what must be resolved before QA can start testing
76
76
  - **Exit criteria**: Derive from Quality Gate Criteria -- what constitutes "done" for the testing phase
77
- - Include project-specific criteria based on context (e.g., "feature flag enabled in staging", "seed data loaded", "Sprint 0 blockers resolved")
77
+ - Include project-specific criteria based on context (e.g., "feature flag enabled in staging", "seed data loaded", "pre-implementation blockers resolved")
78
78
 
79
79
  ### Project Team (Optional)
80
80
 
@@ -91,12 +91,12 @@ When populating templates in step 5, apply the following guidance for these sect
91
91
  - Mark each item's status as Ready or Pending based on available information
92
92
  - This section applies only to `test-design-qa-template.md` output
93
93
 
94
- ### Sprint Planning Handoff (Optional)
94
+ ### Implementation Planning Handoff (Optional)
95
95
 
96
96
  - Include only if test design produces implementation tasks that must be scheduled
97
97
  - Derive items from Dependencies & Test Blockers, tooling/access needs, and QA infra setup
98
98
  - If no dedicated QA, assign ownership to Dev/Platform as appropriate
99
- - Keep the list short; avoid per-sprint breakdown tables
99
+ - Keep the list short; avoid per-milestone breakdown tables
100
100
 
101
101
  ### Interworking & Regression
102
102
 
@@ -43,8 +43,18 @@ From `{config_source}`:
43
43
 
44
44
  - Read `tea_use_playwright_utils`
45
45
  - Read `tea_browser_automation`
46
+ - Read `test_stack_type` (if not set, default to `"auto"`)
46
47
  - Note `test_artifacts`
47
48
 
49
+ **Stack Detection** (for context-aware loading):
50
+
51
+ If `test_stack_type` is `"auto"` or not configured, infer `{detected_stack}` by scanning `{project-root}`:
52
+
53
+ - **Frontend indicators**: `playwright.config.*`, `cypress.config.*`, `package.json` with react/vue/angular
54
+ - **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`
55
+ - **Both present** → `fullstack`; only frontend → `frontend`; only backend → `backend`
56
+ - Explicit `test_stack_type` overrides auto-detection
57
+
48
58
  ---
49
59
 
50
60
  ## 2. Load Project Artifacts (Mode-Specific)
@@ -108,6 +118,28 @@ Store artifacts under `{test_artifacts}/exploration/`
108
118
 
109
119
  ---
110
120
 
121
+ ### Tiered Knowledge Loading
122
+
123
+ Load fragments based on their `tier` classification in `tea-index.csv`:
124
+
125
+ 1. **Core tier** (always load): Foundational fragments required for this workflow
126
+ 2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
127
+ 3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
128
+
129
+ > **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
130
+
131
+ ### Playwright Utils Loading Profiles
132
+
133
+ **If `tea_use_playwright_utils` is enabled**, select the appropriate loading profile:
134
+
135
+ - **API-only profile** (when `{detected_stack}` is `backend` or no `page.goto`/`page.locator` found in test files):
136
+ Load: `overview`, `api-request`, `auth-session`, `recurse` (~1,800 lines)
137
+
138
+ - **Full UI+API profile** (when `{detected_stack}` is `frontend`/`fullstack` or browser tests detected):
139
+ Load: all Playwright Utils core fragments (~4,500 lines)
140
+
141
+ **Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.
142
+
111
143
  ## 4. Load Knowledge Base Fragments
112
144
 
113
145
  Use `{knowledgeIndex}` to select and load only relevant fragments.
@@ -164,6 +196,8 @@ Summarize what was loaded and confirm with the user if anything is missing.
164
196
  - Set `lastSaved: '{date}'`
165
197
  - Append this step's output to the appropriate section of the document.
166
198
 
199
+ **Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
200
+
167
201
  Load next step: `{nextStepFile}`
168
202
 
169
203
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS: