bmad-method-test-architecture-enterprise 1.2.2 → 1.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -12
- package/docs/how-to/workflows/setup-ci.md +3 -1
- package/docs/how-to/workflows/setup-test-framework.md +29 -6
- package/docs/reference/configuration.md +97 -0
- package/docs/reference/knowledge-base.md +15 -6
- package/package.json +1 -1
- package/release_notes.md +6 -4
- package/src/agents/tea.agent.yaml +2 -2
- package/src/module.yaml +78 -5
- package/src/testarch/knowledge/adr-quality-readiness-checklist.md +9 -9
- package/src/testarch/tea-index.csv +36 -36
- package/src/workflows/testarch/atdd/atdd-checklist-template.md +2 -0
- package/src/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md +65 -12
- package/src/workflows/testarch/atdd/steps-c/step-02-generation-mode.md +5 -0
- package/src/workflows/testarch/atdd/steps-c/step-03-test-strategy.md +10 -1
- package/src/workflows/testarch/atdd/steps-c/step-05-validate-and-complete.md +13 -2
- package/src/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md +46 -2
- package/src/workflows/testarch/automate/steps-c/step-02-identify-targets.md +12 -0
- package/src/workflows/testarch/automate/steps-c/step-03-generate-tests.md +110 -31
- package/src/workflows/testarch/automate/steps-c/step-03b-subprocess-backend.md +246 -0
- package/src/workflows/testarch/automate/steps-c/step-03c-aggregate.md +90 -38
- package/src/workflows/testarch/automate/steps-c/step-04-validate-and-summarize.md +13 -2
- package/src/workflows/testarch/ci/azure-pipelines-template.yaml +155 -0
- package/src/workflows/testarch/ci/checklist.md +48 -7
- package/src/workflows/testarch/ci/github-actions-template.yaml +22 -10
- package/src/workflows/testarch/ci/gitlab-ci-template.yaml +21 -12
- package/src/workflows/testarch/ci/harness-pipeline-template.yaml +159 -0
- package/src/workflows/testarch/ci/jenkins-pipeline-template.groovy +129 -0
- package/src/workflows/testarch/ci/steps-c/step-01-preflight.md +58 -17
- package/src/workflows/testarch/ci/steps-c/step-02-generate-pipeline.md +21 -10
- package/src/workflows/testarch/ci/steps-c/step-03-configure-quality-gates.md +5 -0
- package/src/workflows/testarch/ci/workflow.yaml +5 -3
- package/src/workflows/testarch/framework/checklist.md +11 -10
- package/src/workflows/testarch/framework/steps-c/step-01-preflight.md +34 -2
- package/src/workflows/testarch/framework/steps-c/step-02-select-framework.md +20 -1
- package/src/workflows/testarch/framework/steps-c/step-03-scaffold-framework.md +56 -5
- package/src/workflows/testarch/framework/steps-c/step-04-docs-and-scripts.md +16 -4
- package/src/workflows/testarch/nfr-assess/nfr-report-template.md +3 -1
- package/src/workflows/testarch/nfr-assess/steps-c/step-01-load-context.md +12 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-05-generate-report.md +14 -3
- package/src/workflows/testarch/test-design/checklist.md +20 -9
- package/src/workflows/testarch/test-design/instructions.md +3 -3
- package/src/workflows/testarch/test-design/steps-c/step-02-load-context.md +34 -0
- package/src/workflows/testarch/test-design/steps-c/step-05-generate-output.md +29 -2
- package/src/workflows/testarch/test-design/test-design-architecture-template.md +16 -14
- package/src/workflows/testarch/test-design/test-design-handoff-template.md +70 -0
- package/src/workflows/testarch/test-design/test-design-qa-template.md +11 -9
- package/src/workflows/testarch/test-design/workflow.yaml +8 -1
- package/src/workflows/testarch/test-review/steps-c/step-01-load-context.md +34 -1
- package/src/workflows/testarch/test-review/steps-c/step-04-generate-report.md +14 -3
- package/src/workflows/testarch/test-review/test-review-template.md +4 -2
- package/src/workflows/testarch/test-review/workflow.yaml +1 -0
- package/src/workflows/testarch/trace/trace-template.md +7 -5
- package/test/test-installation-components.js +1 -1
- package/test/test-knowledge-base.js +10 -1
|
@@ -2,7 +2,6 @@
|
|
|
2
2
|
name: 'step-02-generate-pipeline'
|
|
3
3
|
description: 'Generate CI pipeline configuration'
|
|
4
4
|
nextStepFile: './step-03-configure-quality-gates.md'
|
|
5
|
-
pipelineOutputFile: '{project-root}/.github/workflows/test.yml'
|
|
6
5
|
outputFile: '{test_artifacts}/ci-pipeline-progress.md'
|
|
7
6
|
---
|
|
8
7
|
|
|
@@ -36,16 +35,20 @@ Create platform-specific CI configuration with test execution, sharding, burn-in
|
|
|
36
35
|
|
|
37
36
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
|
38
37
|
|
|
39
|
-
## 1. Select Template
|
|
38
|
+
## 1. Resolve Output Path and Select Template
|
|
40
39
|
|
|
41
|
-
|
|
40
|
+
Determine the pipeline output file path based on the detected `ci_platform`:
|
|
42
41
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
-
|
|
46
|
-
-
|
|
42
|
+
| CI Platform | Output Path | Template File |
|
|
43
|
+
| ---------------- | ------------------------------------------- | --------------------------------------------------- |
|
|
44
|
+
| `github-actions` | `{project-root}/.github/workflows/test.yml` | `{installed_path}/github-actions-template.yaml` |
|
|
45
|
+
| `gitlab-ci` | `{project-root}/.gitlab-ci.yml` | `{installed_path}/gitlab-ci-template.yaml` |
|
|
46
|
+
| `jenkins` | `{project-root}/Jenkinsfile` | `{installed_path}/jenkins-pipeline-template.groovy` |
|
|
47
|
+
| `azure-devops` | `{project-root}/azure-pipelines.yml` | `{installed_path}/azure-pipelines-template.yaml` |
|
|
48
|
+
| `harness` | `{project-root}/.harness/pipeline.yaml` | `{installed_path}/harness-pipeline-template.yaml` |
|
|
49
|
+
| `circle-ci` | `{project-root}/.circleci/config.yml` | _(no template; generate from first principles)_ |
|
|
47
50
|
|
|
48
|
-
Use templates from `{installed_path}` when available
|
|
51
|
+
Use templates from `{installed_path}` when available. Adapt the template to the project's `test_stack_type` and `test_framework`.
|
|
49
52
|
|
|
50
53
|
---
|
|
51
54
|
|
|
@@ -65,9 +68,17 @@ Include stages:
|
|
|
65
68
|
- Parallel sharding enabled
|
|
66
69
|
- CI retries configured
|
|
67
70
|
- Capture artifacts (HTML report, JUnit XML, traces/videos on failure)
|
|
68
|
-
- Cache dependencies (node_modules
|
|
71
|
+
- Cache dependencies (language-appropriate: node_modules, .venv, .m2, go module cache, NuGet, bundler)
|
|
69
72
|
|
|
70
|
-
Write the selected pipeline configuration to
|
|
73
|
+
Write the selected pipeline configuration to the resolved output path from step 1. Adjust test commands based on `test_stack_type` and `test_framework`:
|
|
74
|
+
|
|
75
|
+
- **Frontend/Fullstack**: Include browser install, E2E/component test commands, Playwright/Cypress artifacts
|
|
76
|
+
- **Backend (Node.js)**: Use `npm test` or framework-specific commands (`vitest`, `jest`), skip browser install
|
|
77
|
+
- **Backend (Python)**: Use `pytest` with coverage (`pytest --cov`), install via `pip install -r requirements.txt` or `poetry install`
|
|
78
|
+
- **Backend (Java/Kotlin)**: Use `mvn test` or `gradle test`, cache `.m2/repository` or `.gradle/caches`
|
|
79
|
+
- **Backend (Go)**: Use `go test ./...` with coverage (`-coverprofile`), cache Go modules
|
|
80
|
+
- **Backend (C#/.NET)**: Use `dotnet test` with coverage, restore NuGet packages
|
|
81
|
+
- **Backend (Ruby)**: Use `bundle exec rspec` with coverage, cache `vendor/bundle`
|
|
71
82
|
|
|
72
83
|
---
|
|
73
84
|
|
|
@@ -43,6 +43,11 @@ Use `{knowledgeIndex}` to load `ci-burn-in.md` guidance:
|
|
|
43
43
|
- Run N-iteration burn-in for flaky detection
|
|
44
44
|
- Gate promotion based on burn-in stability
|
|
45
45
|
|
|
46
|
+
**Stack-conditional burn-in:**
|
|
47
|
+
|
|
48
|
+
- **Frontend or Fullstack** (`test_stack_type` is `frontend` or `fullstack`): Enable burn-in by default. Burn-in targets UI flakiness (race conditions, selector instability, timing issues).
|
|
49
|
+
- **Backend only** (`test_stack_type` is `backend`): Skip burn-in by default. Backend tests (unit, integration, API) are deterministic and rarely exhibit UI-related flakiness. If the user explicitly requests burn-in for backend, honor that override.
|
|
50
|
+
|
|
46
51
|
---
|
|
47
52
|
|
|
48
53
|
## 2. Quality Gates
|
|
@@ -19,11 +19,13 @@ validation: "{installed_path}/checklist.md"
|
|
|
19
19
|
|
|
20
20
|
# Variables and inputs
|
|
21
21
|
variables:
|
|
22
|
-
ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins - user can override
|
|
22
|
+
ci_platform: "auto" # auto, github-actions, gitlab-ci, circle-ci, jenkins, azure-devops, harness - user can override
|
|
23
23
|
test_dir: "{project-root}/tests" # Root test directory
|
|
24
|
+
test_stack_type: "auto" # auto, frontend, backend, fullstack - detected or user override
|
|
25
|
+
test_framework: "auto" # auto, playwright, cypress, jest, vitest - detected or user override
|
|
24
26
|
|
|
25
|
-
# Output configuration
|
|
26
|
-
default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default
|
|
27
|
+
# Output configuration (resolved dynamically based on ci_platform detection)
|
|
28
|
+
default_output_file: "{project-root}/.github/workflows/test.yml" # GitHub Actions default; overridden per platform
|
|
27
29
|
|
|
28
30
|
# Required tools
|
|
29
31
|
required_tools:
|
|
@@ -8,10 +8,10 @@ This checklist ensures the framework workflow completes successfully and all del
|
|
|
8
8
|
|
|
9
9
|
Before starting the workflow:
|
|
10
10
|
|
|
11
|
-
- [ ] Project root contains valid `package.json`
|
|
12
|
-
- [ ] No existing
|
|
13
|
-
- [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, etc.)
|
|
14
|
-
- [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable
|
|
11
|
+
- [ ] Project root contains a valid project manifest (`package.json`, `pyproject.toml`, `pom.xml`, `build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, or `Cargo.toml`)
|
|
12
|
+
- [ ] No existing test framework detected that conflicts with the target setup
|
|
13
|
+
- [ ] Project type identifiable (React, Vue, Angular, Next.js, Node, Python, Java, Go, .NET, Ruby, Rust, etc.)
|
|
14
|
+
- [ ] Bundler identifiable (Vite, Webpack, Rollup, esbuild) or not applicable (backend projects)
|
|
15
15
|
- [ ] User has write permissions to create directories and files
|
|
16
16
|
|
|
17
17
|
---
|
|
@@ -20,7 +20,8 @@ Before starting the workflow:
|
|
|
20
20
|
|
|
21
21
|
### Step 1: Preflight Checks
|
|
22
22
|
|
|
23
|
-
- [ ]
|
|
23
|
+
- [ ] Stack type detected (`frontend`, `backend`, or `fullstack`)
|
|
24
|
+
- [ ] Project manifest successfully read and parsed (`package.json`, `pyproject.toml`, `pom.xml`, `go.mod`, etc.)
|
|
24
25
|
- [ ] Project type extracted correctly
|
|
25
26
|
- [ ] Bundler identified (or marked as N/A for backend projects)
|
|
26
27
|
- [ ] No framework conflicts detected
|
|
@@ -29,8 +30,8 @@ Before starting the workflow:
|
|
|
29
30
|
### Step 2: Framework Selection
|
|
30
31
|
|
|
31
32
|
- [ ] Framework auto-detection logic executed
|
|
32
|
-
- [ ] Framework choice justified (Playwright vs Cypress)
|
|
33
|
-
- [ ] Framework preference respected (if explicitly set)
|
|
33
|
+
- [ ] Framework choice justified (Playwright vs Cypress for frontend; pytest/JUnit/Go test/xUnit/RSpec for backend)
|
|
34
|
+
- [ ] Framework preference respected (if explicitly set via `config.test_framework`)
|
|
34
35
|
- [ ] User notified of framework selection and rationale
|
|
35
36
|
|
|
36
37
|
### Step 3: Directory Structure
|
|
@@ -115,9 +116,9 @@ Before starting the workflow:
|
|
|
115
116
|
- [ ] Knowledge base references included
|
|
116
117
|
- [ ] Troubleshooting section included
|
|
117
118
|
|
|
118
|
-
### Step 11:
|
|
119
|
+
### Step 11: Build & Test Script Updates
|
|
119
120
|
|
|
120
|
-
- [ ] Minimal test script added to package.json
|
|
121
|
+
- [ ] Minimal test script added to appropriate config (`package.json` for frontend, `Makefile`/`pyproject.toml`/`build.gradle` for backend)
|
|
121
122
|
- [ ] Test framework dependency added (if not already present)
|
|
122
123
|
- [ ] Type definitions added (if TypeScript)
|
|
123
124
|
- [ ] Users can extend with additional scripts as needed
|
|
@@ -231,7 +232,7 @@ Before starting the workflow:
|
|
|
231
232
|
- [ ] All quality checks passed
|
|
232
233
|
- [ ] All integration points verified
|
|
233
234
|
- [ ] Sample test executes successfully
|
|
234
|
-
- [ ] User can run `npm run test:e2e`
|
|
235
|
+
- [ ] User can run the appropriate test command without errors (`npm run test:e2e`, `pytest`, `go test ./...`, etc.)
|
|
235
236
|
- [ ] Documentation is complete and accurate
|
|
236
237
|
- [ ] No critical issues or blockers identified
|
|
237
238
|
|
|
@@ -36,19 +36,51 @@ Verify the project is ready for framework scaffolding and gather key context.
|
|
|
36
36
|
|
|
37
37
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
|
38
38
|
|
|
39
|
-
## 1.
|
|
39
|
+
## 1. Stack Detection
|
|
40
|
+
|
|
41
|
+
**Read `config.test_stack_type`** from `{config_source}`.
|
|
42
|
+
|
|
43
|
+
**Auto-Detection Algorithm** (when `test_stack_type` is `"auto"` or not configured):
|
|
44
|
+
|
|
45
|
+
- Scan `{project-root}` for project manifests:
|
|
46
|
+
- **Frontend indicators**: `package.json` with react/vue/angular/next dependencies, `playwright.config.*`, `vite.config.*`, `webpack.config.*`
|
|
47
|
+
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`
|
|
48
|
+
- **Both present** = `fullstack`; only frontend = `frontend`; only backend = `backend`
|
|
49
|
+
- Explicit `test_stack_type` config value overrides auto-detection
|
|
50
|
+
- **Backward compatibility**: if `test_stack_type` is not in config, treat as `"auto"` (preserves current frontend behavior for existing installs)
|
|
51
|
+
|
|
52
|
+
Store result as `{detected_stack}` = `frontend` | `backend` | `fullstack`
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 2. Validate Prerequisites
|
|
57
|
+
|
|
58
|
+
**If {detected_stack} is `frontend` or `fullstack`:**
|
|
40
59
|
|
|
41
60
|
- `package.json` exists in project root
|
|
42
61
|
- No existing E2E framework (`playwright.config.*`, `cypress.config.*`, `cypress.json`)
|
|
62
|
+
|
|
63
|
+
**If {detected_stack} is `backend` or `fullstack`:**
|
|
64
|
+
|
|
65
|
+
- At least one backend project manifest exists (`pyproject.toml`, `pom.xml`, `build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`)
|
|
66
|
+
- No existing test framework config that conflicts (e.g., `conftest.py` with full pytest suite, `src/test/` with JUnit suite)
|
|
67
|
+
|
|
43
68
|
- Architecture/stack context available (project type, bundler, dependencies)
|
|
44
69
|
|
|
45
70
|
If any fail, **HALT** and report the missing requirement.
|
|
46
71
|
|
|
47
72
|
---
|
|
48
73
|
|
|
49
|
-
##
|
|
74
|
+
## 3. Gather Project Context
|
|
75
|
+
|
|
76
|
+
**If {detected_stack} is `frontend` or `fullstack`:**
|
|
50
77
|
|
|
51
78
|
- Read `package.json` to identify framework, bundler, dependencies
|
|
79
|
+
|
|
80
|
+
**If {detected_stack} is `backend` or `fullstack`:**
|
|
81
|
+
|
|
82
|
+
- Read the relevant project manifest (`pyproject.toml`, `pom.xml`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`) to identify language, framework, and dependencies
|
|
83
|
+
|
|
52
84
|
- Check for architecture docs (`architecture.md`, `tech-spec*.md`) if available
|
|
53
85
|
- Note auth requirements and APIs (if documented)
|
|
54
86
|
|
|
@@ -37,6 +37,10 @@ Choose the most appropriate framework and document the rationale.
|
|
|
37
37
|
|
|
38
38
|
## 1. Selection Logic
|
|
39
39
|
|
|
40
|
+
Use `{detected_stack}` from Step 1 to guide framework selection.
|
|
41
|
+
|
|
42
|
+
**If {detected_stack} is `frontend` or `fullstack` (browser-based testing):**
|
|
43
|
+
|
|
40
44
|
Default to **Playwright** unless strong reasons suggest Cypress.
|
|
41
45
|
|
|
42
46
|
**Playwright recommended when:**
|
|
@@ -52,7 +56,22 @@ Default to **Playwright** unless strong reasons suggest Cypress.
|
|
|
52
56
|
- Component testing focus
|
|
53
57
|
- Simpler setup needed
|
|
54
58
|
|
|
55
|
-
|
|
59
|
+
**If {detected_stack} is `backend` (no browser-based testing):**
|
|
60
|
+
|
|
61
|
+
Select the framework matching the project language:
|
|
62
|
+
|
|
63
|
+
- **Python**: pytest (default), unittest
|
|
64
|
+
- **Java/Kotlin**: JUnit 5 (default), TestNG
|
|
65
|
+
- **Go**: Go test (built-in)
|
|
66
|
+
- **C#/.NET**: xUnit (default), NUnit, MSTest
|
|
67
|
+
- **Ruby**: RSpec (default), Minitest
|
|
68
|
+
- **Rust**: cargo test (built-in)
|
|
69
|
+
|
|
70
|
+
**If {detected_stack} is `fullstack`:**
|
|
71
|
+
|
|
72
|
+
Select both a browser-based framework (Playwright/Cypress) AND the appropriate backend framework for the detected language.
|
|
73
|
+
|
|
74
|
+
Respect `config.test_framework` if explicitly set (not `"auto"`).
|
|
56
75
|
|
|
57
76
|
---
|
|
58
77
|
|
|
@@ -39,17 +39,32 @@ Generate the test directory structure, configuration files, fixtures, factories,
|
|
|
39
39
|
|
|
40
40
|
## 1. Create Directory Structure
|
|
41
41
|
|
|
42
|
-
|
|
42
|
+
Use `{detected_stack}` from Step 1 to determine directory layout.
|
|
43
|
+
|
|
44
|
+
**If {detected_stack} is `frontend` or `fullstack`:**
|
|
43
45
|
|
|
44
46
|
- `{test_dir}/e2e/`
|
|
45
47
|
- `{test_dir}/support/fixtures/`
|
|
46
48
|
- `{test_dir}/support/helpers/`
|
|
47
49
|
- `{test_dir}/support/page-objects/` (optional)
|
|
48
50
|
|
|
51
|
+
**If {detected_stack} is `backend` or `fullstack`:**
|
|
52
|
+
|
|
53
|
+
Create the idiomatic test directory for the detected language:
|
|
54
|
+
|
|
55
|
+
- **Python (pytest)**: `tests/` with `conftest.py`, `tests/unit/`, `tests/integration/`, `tests/api/`
|
|
56
|
+
- **Java/Kotlin (JUnit)**: `src/test/java/` mirroring `src/main/java/` package structure, with `unit/`, `integration/`, `api/` sub-packages
|
|
57
|
+
- **Go**: `*_test.go` files alongside source files (Go convention), plus `testdata/` for fixtures
|
|
58
|
+
- **C#/.NET (xUnit)**: `tests/` project with `Unit/`, `Integration/`, `Api/` directories
|
|
59
|
+
- **Ruby (RSpec)**: `spec/` with `spec/unit/`, `spec/integration/`, `spec/api/`, `spec/support/`
|
|
60
|
+
- **Rust**: `tests/` for integration tests, inline `#[cfg(test)]` modules for unit tests
|
|
61
|
+
|
|
49
62
|
---
|
|
50
63
|
|
|
51
64
|
## 2. Generate Framework Config
|
|
52
65
|
|
|
66
|
+
**If {detected_stack} is `frontend` or `fullstack`:**
|
|
67
|
+
|
|
53
68
|
Create `playwright.config.ts` or `cypress.config.ts` with:
|
|
54
69
|
|
|
55
70
|
- **Timeouts**: action 15s, navigation 30s, test 60s
|
|
@@ -60,15 +75,38 @@ Create `playwright.config.ts` or `cypress.config.ts` with:
|
|
|
60
75
|
|
|
61
76
|
Use TypeScript if `use_typescript: true`.
|
|
62
77
|
|
|
78
|
+
**If {detected_stack} is `backend` or `fullstack`:**
|
|
79
|
+
|
|
80
|
+
Create the idiomatic test config for the detected framework:
|
|
81
|
+
|
|
82
|
+
- **pytest**: `pyproject.toml` `[tool.pytest.ini_options]` or `pytest.ini` with markers, test paths, coverage settings
|
|
83
|
+
- **JUnit**: `build.gradle`/`pom.xml` test configuration with JUnit 5 dependencies, Surefire/Failsafe plugins
|
|
84
|
+
- **Go test**: no config file needed (Go convention); optionally create `Makefile` test targets
|
|
85
|
+
- **xUnit**: `.csproj` test project with xUnit and coverlet dependencies
|
|
86
|
+
- **RSpec**: `.rspec` config file with `spec_helper.rb` and `rails_helper.rb` (if Rails)
|
|
87
|
+
|
|
63
88
|
---
|
|
64
89
|
|
|
65
|
-
## 3. Environment
|
|
90
|
+
## 3. Environment Setup
|
|
91
|
+
|
|
92
|
+
Create `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`.
|
|
66
93
|
|
|
67
|
-
|
|
94
|
+
**Stack-conditional environment files:**
|
|
95
|
+
|
|
96
|
+
**If {detected_stack} is `frontend` or `fullstack` (Node.js):**
|
|
68
97
|
|
|
69
|
-
- `.env.example` with `TEST_ENV`, `BASE_URL`, `API_URL`
|
|
70
98
|
- `.nvmrc` using current LTS Node (prefer Node 24+)
|
|
71
99
|
|
|
100
|
+
**If {detected_stack} is `backend`:**
|
|
101
|
+
|
|
102
|
+
Create the idiomatic version file for the detected language:
|
|
103
|
+
|
|
104
|
+
- **Python**: `.python-version` with current stable Python (prefer 3.12+)
|
|
105
|
+
- **Java**: `.java-version` or `JAVA_HOME` documentation in `.env.example`
|
|
106
|
+
- **Go**: Go version is already in `go.mod` (no additional file needed)
|
|
107
|
+
- **C#/.NET**: `global.json` with SDK version if not already present
|
|
108
|
+
- **Ruby**: `.ruby-version` with current stable Ruby
|
|
109
|
+
|
|
72
110
|
---
|
|
73
111
|
|
|
74
112
|
## 4. Fixtures & Factories
|
|
@@ -94,6 +132,8 @@ Implement:
|
|
|
94
132
|
|
|
95
133
|
## 5. Sample Tests & Helpers
|
|
96
134
|
|
|
135
|
+
**If {detected_stack} is `frontend` or `fullstack`:**
|
|
136
|
+
|
|
97
137
|
Create example tests in `{test_dir}/e2e/` demonstrating:
|
|
98
138
|
|
|
99
139
|
- Given/When/Then format
|
|
@@ -101,11 +141,22 @@ Create example tests in `{test_dir}/e2e/` demonstrating:
|
|
|
101
141
|
- Factory usage
|
|
102
142
|
- Network interception pattern (if applicable)
|
|
103
143
|
|
|
144
|
+
**If {detected_stack} is `backend` or `fullstack`:**
|
|
145
|
+
|
|
146
|
+
Create example tests in the idiomatic location for the detected language:
|
|
147
|
+
|
|
148
|
+
- **Python**: `tests/test_example.py` with pytest fixtures, parametrize, and factory usage
|
|
149
|
+
- **Java**: `src/test/java/.../ExampleTest.java` with JUnit 5 annotations, `@BeforeEach` setup
|
|
150
|
+
- **Go**: `example_test.go` alongside source with table-driven tests and `testify` assertions
|
|
151
|
+
- **C#/.NET**: `tests/ExampleTests.cs` with xUnit `[Fact]`/`[Theory]` and fixture injection
|
|
152
|
+
- **Ruby**: `spec/example_spec.rb` with RSpec `describe`/`context`/`it` and factory_bot
|
|
153
|
+
|
|
104
154
|
Create helpers for:
|
|
105
155
|
|
|
106
156
|
- API clients (if needed)
|
|
107
|
-
- Network utilities
|
|
157
|
+
- Network utilities (frontend/fullstack only)
|
|
108
158
|
- Auth helpers
|
|
159
|
+
- Test data factories (language-idiomatic patterns)
|
|
109
160
|
|
|
110
161
|
---
|
|
111
162
|
|
|
@@ -10,7 +10,7 @@ progressFile: '{test_artifacts}/framework-setup-progress.md'
|
|
|
10
10
|
|
|
11
11
|
## STEP GOAL
|
|
12
12
|
|
|
13
|
-
Create
|
|
13
|
+
Create test documentation and add build/test scripts appropriate for `{detected_stack}`.
|
|
14
14
|
|
|
15
15
|
## MANDATORY EXECUTION RULES
|
|
16
16
|
|
|
@@ -49,11 +49,23 @@ Create `{outputFile}` and include:
|
|
|
49
49
|
|
|
50
50
|
---
|
|
51
51
|
|
|
52
|
-
## 2.
|
|
52
|
+
## 2. Build & Test Scripts
|
|
53
53
|
|
|
54
|
-
|
|
54
|
+
**If {detected_stack} is `frontend` or `fullstack`:**
|
|
55
55
|
|
|
56
|
-
|
|
56
|
+
Add to `package.json` at minimum:
|
|
57
|
+
|
|
58
|
+
- `test:e2e`: framework execution command (e.g., `npx playwright test`)
|
|
59
|
+
|
|
60
|
+
**If {detected_stack} is `backend` or `fullstack`:**
|
|
61
|
+
|
|
62
|
+
Add the idiomatic test commands for the detected framework:
|
|
63
|
+
|
|
64
|
+
- **Python (pytest)**: Add to `pyproject.toml` scripts or `Makefile`: `pytest`, `pytest --cov`, `pytest -m integration`
|
|
65
|
+
- **Java (JUnit)**: Add to `build.gradle`/`pom.xml`: `./gradlew test`, `mvn test`, `mvn verify` (integration)
|
|
66
|
+
- **Go**: Add to `Makefile`: `go test ./...`, `go test -race ./...`, `go test -cover ./...`
|
|
67
|
+
- **C#/.NET**: Add to CI scripts or `Makefile`: `dotnet test`, `dotnet test --collect:"XPlat Code Coverage"`
|
|
68
|
+
- **Ruby (RSpec)**: Add to `Gemfile` binstubs or `Makefile`: `bundle exec rspec`, `bundle exec rspec spec/integration`
|
|
57
69
|
|
|
58
70
|
---
|
|
59
71
|
|
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
stepsCompleted: []
|
|
3
3
|
lastStep: ''
|
|
4
4
|
lastSaved: ''
|
|
5
|
+
workflowType: 'testarch-nfr-assess'
|
|
6
|
+
inputDocuments: []
|
|
5
7
|
---
|
|
6
8
|
|
|
7
9
|
# NFR Assessment - {FEATURE_NAME}
|
|
@@ -262,7 +264,7 @@ Note: This assessment summarizes existing evidence; it does not run tests or CI
|
|
|
262
264
|
- {SPECIFIC_STEPS}
|
|
263
265
|
- {VALIDATION_CRITERIA}
|
|
264
266
|
|
|
265
|
-
### Short-term (Next
|
|
267
|
+
### Short-term (Next Milestone) - MEDIUM Priority
|
|
266
268
|
|
|
267
269
|
1. **{ACTION_TITLE_3}** - {PRIORITY} - {ESTIMATED_EFFORT} - {OWNER}
|
|
268
270
|
- {ACTION_DESCRIPTION}
|
|
@@ -54,6 +54,16 @@ From `{config_source}`:
|
|
|
54
54
|
|
|
55
55
|
---
|
|
56
56
|
|
|
57
|
+
### Tiered Knowledge Loading
|
|
58
|
+
|
|
59
|
+
Load fragments based on their `tier` classification in `tea-index.csv`:
|
|
60
|
+
|
|
61
|
+
1. **Core tier** (always load): Foundational fragments required for this workflow
|
|
62
|
+
2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
|
|
63
|
+
3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
|
|
64
|
+
|
|
65
|
+
> **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
|
|
66
|
+
|
|
57
67
|
## 3. Load Knowledge Base Fragments
|
|
58
68
|
|
|
59
69
|
From `{knowledgeIndex}` load:
|
|
@@ -112,6 +122,8 @@ Summarize loaded NFR sources and evidence availability.
|
|
|
112
122
|
- Set `lastSaved: '{date}'`
|
|
113
123
|
- Append this step's output to the appropriate section of the document.
|
|
114
124
|
|
|
125
|
+
**Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
|
|
126
|
+
|
|
115
127
|
Load next step: `{nextStepFile}`
|
|
116
128
|
|
|
117
129
|
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|
|
@@ -45,7 +45,18 @@ Use `nfr-report-template.md` to produce `{outputFile}` containing:
|
|
|
45
45
|
|
|
46
46
|
---
|
|
47
47
|
|
|
48
|
-
## 2.
|
|
48
|
+
## 2. Polish Output
|
|
49
|
+
|
|
50
|
+
Before finalizing, review the complete output document for quality:
|
|
51
|
+
|
|
52
|
+
1. **Remove duplication**: Progressive-append workflow may have created repeated sections — consolidate
|
|
53
|
+
2. **Verify consistency**: Ensure terminology, risk scores, and references are consistent throughout
|
|
54
|
+
3. **Check completeness**: All template sections should be populated or explicitly marked N/A
|
|
55
|
+
4. **Format cleanup**: Ensure markdown formatting is clean (tables aligned, headers consistent, no orphaned references)
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## 3. Validation
|
|
49
60
|
|
|
50
61
|
Validate against `checklist.md` and fix gaps.
|
|
51
62
|
|
|
@@ -53,7 +64,7 @@ Validate against `checklist.md` and fix gaps.
|
|
|
53
64
|
|
|
54
65
|
---
|
|
55
66
|
|
|
56
|
-
##
|
|
67
|
+
## 4. Save Progress
|
|
57
68
|
|
|
58
69
|
**Save this step's accumulated work to `{outputFile}`.**
|
|
59
70
|
|
|
@@ -77,7 +88,7 @@ Validate against `checklist.md` and fix gaps.
|
|
|
77
88
|
|
|
78
89
|
---
|
|
79
90
|
|
|
80
|
-
##
|
|
91
|
+
## 5. Completion Summary
|
|
81
92
|
|
|
82
93
|
Report:
|
|
83
94
|
|
|
@@ -190,7 +190,7 @@
|
|
|
190
190
|
- [ ] Prerequisites for testing start are clearly defined
|
|
191
191
|
- [ ] Environment readiness included
|
|
192
192
|
- [ ] Test data readiness included
|
|
193
|
-
- [ ]
|
|
193
|
+
- [ ] Pre-implementation blocker resolution referenced
|
|
194
194
|
|
|
195
195
|
### Exit Criteria
|
|
196
196
|
|
|
@@ -225,7 +225,7 @@
|
|
|
225
225
|
- [ ] **Purpose statement** at top (serves as contract with Architecture team)
|
|
226
226
|
- [ ] **Executive Summary** with scope, business context, architecture decisions, risk summary
|
|
227
227
|
- [ ] **Quick Guide** section with three tiers:
|
|
228
|
-
- [ ] 🚨 BLOCKERS - Team Must Decide (
|
|
228
|
+
- [ ] 🚨 BLOCKERS - Team Must Decide (pre-implementation critical path items)
|
|
229
229
|
- [ ] ⚠️ HIGH PRIORITY - Team Should Validate (recommendations for approval)
|
|
230
230
|
- [ ] 📋 INFO ONLY - Solutions Provided (no decisions needed)
|
|
231
231
|
- [ ] **Risk Assessment** section - **ACTIONABLE**
|
|
@@ -262,7 +262,7 @@
|
|
|
262
262
|
- [ ] NO "Test Levels Strategy" section (unit/integration/E2E split belongs in QA doc only)
|
|
263
263
|
- [ ] NO "NFR Testing Approach" section with detailed test procedures (belongs in QA doc only)
|
|
264
264
|
- [ ] NO "Test Environment Requirements" section (belongs in QA doc only)
|
|
265
|
-
- [ ] NO "Recommendations for
|
|
265
|
+
- [ ] NO "Recommendations for pre-implementation" section with test framework setup (belongs in QA doc only)
|
|
266
266
|
- [ ] NO "Quality Gate Criteria" section (pass rates, coverage targets belong in QA doc only)
|
|
267
267
|
- [ ] NO "Tool Selection" section (Playwright, k6, etc. belongs in QA doc only)
|
|
268
268
|
|
|
@@ -295,11 +295,11 @@
|
|
|
295
295
|
- [ ] **QA Effort Estimate** section (QA effort ONLY)
|
|
296
296
|
- [ ] Interval-based estimates (e.g., "~1-2 weeks" NOT "36 hours")
|
|
297
297
|
- [ ] NO DevOps, Backend, Data Eng, Finance effort
|
|
298
|
-
- [ ] No per-
|
|
299
|
-
- [ ] **
|
|
298
|
+
- [ ] No per-milestone effort breakdowns in this section
|
|
299
|
+
- [ ] **Implementation Planning Handoff** section (optional)
|
|
300
300
|
- [ ] Only include if implementation tasks must be scheduled
|
|
301
301
|
- [ ] Owners assigned (QA/Dev/Platform/etc)
|
|
302
|
-
- [ ] Target
|
|
302
|
+
- [ ] Target milestone may be noted, but avoid detailed per-milestone breakdowns
|
|
303
303
|
- [ ] **Appendix A: Code Examples & Tagging**
|
|
304
304
|
- [ ] **Appendix B: Knowledge Base References**
|
|
305
305
|
|
|
@@ -315,14 +315,14 @@
|
|
|
315
315
|
- [ ] ❌ NO Follow-on Workflows section (BMAD commands self-explanatory)
|
|
316
316
|
- [ ] ❌ NO Approval section
|
|
317
317
|
- [ ] ❌ NO Infrastructure/DevOps/Finance effort tables (out of scope)
|
|
318
|
-
- [ ] ❌ NO detailed
|
|
319
|
-
- [ ] ❌ NO generic Next Steps section (use
|
|
318
|
+
- [ ] ❌ NO detailed milestone-by-milestone breakdown tables (use Implementation Planning Handoff if needed)
|
|
319
|
+
- [ ] ❌ NO generic Next Steps section (use Implementation Planning Handoff if needed)
|
|
320
320
|
|
|
321
321
|
### Cross-Document Consistency
|
|
322
322
|
|
|
323
323
|
- [ ] Both documents reference same risks by ID (R-001, R-002, etc.)
|
|
324
324
|
- [ ] Both documents use consistent priority levels (P0, P1, P2, P3)
|
|
325
|
-
- [ ] Both documents reference same
|
|
325
|
+
- [ ] Both documents reference same pre-implementation blockers
|
|
326
326
|
- [ ] No duplicate content (cross-reference instead)
|
|
327
327
|
- [ ] Dates and authors match across documents
|
|
328
328
|
- [ ] ADR and PRD references consistent
|
|
@@ -361,6 +361,16 @@
|
|
|
361
361
|
- [ ] Actionable ASRs included in 🚨 or ⚠️ sections
|
|
362
362
|
- [ ] FYI ASRs included in 📋 section or omitted if obvious
|
|
363
363
|
|
|
364
|
+
## BMAD Handoff Validation (System-Level Mode Only)
|
|
365
|
+
|
|
366
|
+
- [ ] Handoff document generated at `{test_artifacts}/test-design/{project_name}-handoff.md`
|
|
367
|
+
- [ ] TEA Artifacts Inventory table populated with actual paths
|
|
368
|
+
- [ ] Epic-Level Integration Guidance populated with P0/P1 risks
|
|
369
|
+
- [ ] Story-Level Integration Guidance populated with critical test scenarios
|
|
370
|
+
- [ ] Risk-to-Story Mapping table populated from risk register
|
|
371
|
+
- [ ] Recommended workflow sequence is accurate
|
|
372
|
+
- [ ] Phase transition quality gates are defined
|
|
373
|
+
|
|
364
374
|
## Completion Criteria
|
|
365
375
|
|
|
366
376
|
**All must be true:**
|
|
@@ -372,6 +382,7 @@
|
|
|
372
382
|
- [ ] All integration points verified
|
|
373
383
|
- [ ] Output file(s) complete and well-formatted
|
|
374
384
|
- [ ] **System-level mode:** Both documents validated (if applicable)
|
|
385
|
+
- [ ] **System-level mode:** Handoff document validated (if applicable)
|
|
375
386
|
- [ ] **Epic-level mode:** Single document validated (if applicable)
|
|
376
387
|
- [ ] Team review scheduled (if required)
|
|
377
388
|
|
|
@@ -74,7 +74,7 @@ When populating templates in step 5, apply the following guidance for these sect
|
|
|
74
74
|
|
|
75
75
|
- **Entry criteria**: Derive from Dependencies and Test Blockers -- what must be resolved before QA can start testing
|
|
76
76
|
- **Exit criteria**: Derive from Quality Gate Criteria -- what constitutes "done" for the testing phase
|
|
77
|
-
- Include project-specific criteria based on context (e.g., "feature flag enabled in staging", "seed data loaded", "
|
|
77
|
+
- Include project-specific criteria based on context (e.g., "feature flag enabled in staging", "seed data loaded", "pre-implementation blockers resolved")
|
|
78
78
|
|
|
79
79
|
### Project Team (Optional)
|
|
80
80
|
|
|
@@ -91,12 +91,12 @@ When populating templates in step 5, apply the following guidance for these sect
|
|
|
91
91
|
- Mark each item's status as Ready or Pending based on available information
|
|
92
92
|
- This section applies only to `test-design-qa-template.md` output
|
|
93
93
|
|
|
94
|
-
###
|
|
94
|
+
### Implementation Planning Handoff (Optional)
|
|
95
95
|
|
|
96
96
|
- Include only if test design produces implementation tasks that must be scheduled
|
|
97
97
|
- Derive items from Dependencies & Test Blockers, tooling/access needs, and QA infra setup
|
|
98
98
|
- If no dedicated QA, assign ownership to Dev/Platform as appropriate
|
|
99
|
-
- Keep the list short; avoid per-
|
|
99
|
+
- Keep the list short; avoid per-milestone breakdown tables
|
|
100
100
|
|
|
101
101
|
### Interworking & Regression
|
|
102
102
|
|
|
@@ -43,8 +43,18 @@ From `{config_source}`:
|
|
|
43
43
|
|
|
44
44
|
- Read `tea_use_playwright_utils`
|
|
45
45
|
- Read `tea_browser_automation`
|
|
46
|
+
- Read `test_stack_type` (if not set, default to `"auto"`)
|
|
46
47
|
- Note `test_artifacts`
|
|
47
48
|
|
|
49
|
+
**Stack Detection** (for context-aware loading):
|
|
50
|
+
|
|
51
|
+
If `test_stack_type` is `"auto"` or not configured, infer `{detected_stack}` by scanning `{project-root}`:
|
|
52
|
+
|
|
53
|
+
- **Frontend indicators**: `playwright.config.*`, `cypress.config.*`, `package.json` with react/vue/angular
|
|
54
|
+
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`, `Gemfile`, `Cargo.toml`
|
|
55
|
+
- **Both present** → `fullstack`; only frontend → `frontend`; only backend → `backend`
|
|
56
|
+
- Explicit `test_stack_type` overrides auto-detection
|
|
57
|
+
|
|
48
58
|
---
|
|
49
59
|
|
|
50
60
|
## 2. Load Project Artifacts (Mode-Specific)
|
|
@@ -108,6 +118,28 @@ Store artifacts under `{test_artifacts}/exploration/`
|
|
|
108
118
|
|
|
109
119
|
---
|
|
110
120
|
|
|
121
|
+
### Tiered Knowledge Loading
|
|
122
|
+
|
|
123
|
+
Load fragments based on their `tier` classification in `tea-index.csv`:
|
|
124
|
+
|
|
125
|
+
1. **Core tier** (always load): Foundational fragments required for this workflow
|
|
126
|
+
2. **Extended tier** (load on-demand): Load when deeper analysis is needed or when the user's context requires it
|
|
127
|
+
3. **Specialized tier** (load only when relevant): Load only when the specific use case matches (e.g., contract-testing only for microservices, email-auth only for email flows)
|
|
128
|
+
|
|
129
|
+
> **Context Efficiency**: Loading only core fragments reduces context usage by 40-50% compared to loading all fragments.
|
|
130
|
+
|
|
131
|
+
### Playwright Utils Loading Profiles
|
|
132
|
+
|
|
133
|
+
**If `tea_use_playwright_utils` is enabled**, select the appropriate loading profile:
|
|
134
|
+
|
|
135
|
+
- **API-only profile** (when `{detected_stack}` is `backend` or no `page.goto`/`page.locator` found in test files):
|
|
136
|
+
Load: `overview`, `api-request`, `auth-session`, `recurse` (~1,800 lines)
|
|
137
|
+
|
|
138
|
+
- **Full UI+API profile** (when `{detected_stack}` is `frontend`/`fullstack` or browser tests detected):
|
|
139
|
+
Load: all Playwright Utils core fragments (~4,500 lines)
|
|
140
|
+
|
|
141
|
+
**Detection**: Scan `{test_dir}` for files containing `page.goto` or `page.locator`. If none found, use API-only profile.
|
|
142
|
+
|
|
111
143
|
## 4. Load Knowledge Base Fragments
|
|
112
144
|
|
|
113
145
|
Use `{knowledgeIndex}` to select and load only relevant fragments.
|
|
@@ -164,6 +196,8 @@ Summarize what was loaded and confirm with the user if anything is missing.
|
|
|
164
196
|
- Set `lastSaved: '{date}'`
|
|
165
197
|
- Append this step's output to the appropriate section of the document.
|
|
166
198
|
|
|
199
|
+
**Update `inputDocuments`**: Set `inputDocuments` in the output template frontmatter to the list of artifact paths loaded in this step (e.g., knowledge fragments, test design documents, configuration files).
|
|
200
|
+
|
|
167
201
|
Load next step: `{nextStepFile}`
|
|
168
202
|
|
|
169
203
|
## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
|