bmad-method-test-architecture-enterprise 1.2.2 → 1.2.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -12
- package/docs/how-to/workflows/setup-ci.md +3 -1
- package/docs/how-to/workflows/setup-test-framework.md +29 -6
- package/docs/reference/configuration.md +97 -0
- package/docs/reference/knowledge-base.md +15 -6
- package/package.json +1 -1
- package/release_notes.md +6 -4
- package/src/agents/tea.agent.yaml +2 -2
- package/src/module.yaml +78 -5
- package/src/testarch/knowledge/adr-quality-readiness-checklist.md +9 -9
- package/src/testarch/tea-index.csv +36 -36
- package/src/workflows/testarch/atdd/atdd-checklist-template.md +2 -0
- package/src/workflows/testarch/atdd/steps-c/step-01-preflight-and-context.md +65 -12
- package/src/workflows/testarch/atdd/steps-c/step-02-generation-mode.md +5 -0
- package/src/workflows/testarch/atdd/steps-c/step-03-test-strategy.md +10 -1
- package/src/workflows/testarch/atdd/steps-c/step-05-validate-and-complete.md +13 -2
- package/src/workflows/testarch/automate/steps-c/step-01-preflight-and-context.md +46 -2
- package/src/workflows/testarch/automate/steps-c/step-02-identify-targets.md +12 -0
- package/src/workflows/testarch/automate/steps-c/step-03-generate-tests.md +110 -31
- package/src/workflows/testarch/automate/steps-c/step-03b-subprocess-backend.md +246 -0
- package/src/workflows/testarch/automate/steps-c/step-03c-aggregate.md +90 -38
- package/src/workflows/testarch/automate/steps-c/step-04-validate-and-summarize.md +13 -2
- package/src/workflows/testarch/ci/azure-pipelines-template.yaml +155 -0
- package/src/workflows/testarch/ci/checklist.md +48 -7
- package/src/workflows/testarch/ci/github-actions-template.yaml +22 -10
- package/src/workflows/testarch/ci/gitlab-ci-template.yaml +21 -12
- package/src/workflows/testarch/ci/harness-pipeline-template.yaml +159 -0
- package/src/workflows/testarch/ci/jenkins-pipeline-template.groovy +129 -0
- package/src/workflows/testarch/ci/steps-c/step-01-preflight.md +58 -17
- package/src/workflows/testarch/ci/steps-c/step-02-generate-pipeline.md +21 -10
- package/src/workflows/testarch/ci/steps-c/step-03-configure-quality-gates.md +5 -0
- package/src/workflows/testarch/ci/workflow.yaml +5 -3
- package/src/workflows/testarch/framework/checklist.md +11 -10
- package/src/workflows/testarch/framework/steps-c/step-01-preflight.md +34 -2
- package/src/workflows/testarch/framework/steps-c/step-02-select-framework.md +20 -1
- package/src/workflows/testarch/framework/steps-c/step-03-scaffold-framework.md +56 -5
- package/src/workflows/testarch/framework/steps-c/step-04-docs-and-scripts.md +16 -4
- package/src/workflows/testarch/nfr-assess/nfr-report-template.md +3 -1
- package/src/workflows/testarch/nfr-assess/steps-c/step-01-load-context.md +12 -0
- package/src/workflows/testarch/nfr-assess/steps-c/step-05-generate-report.md +14 -3
- package/src/workflows/testarch/test-design/checklist.md +20 -9
- package/src/workflows/testarch/test-design/instructions.md +3 -3
- package/src/workflows/testarch/test-design/steps-c/step-02-load-context.md +34 -0
- package/src/workflows/testarch/test-design/steps-c/step-05-generate-output.md +29 -2
- package/src/workflows/testarch/test-design/test-design-architecture-template.md +16 -14
- package/src/workflows/testarch/test-design/test-design-handoff-template.md +70 -0
- package/src/workflows/testarch/test-design/test-design-qa-template.md +11 -9
- package/src/workflows/testarch/test-design/workflow.yaml +8 -1
- package/src/workflows/testarch/test-review/steps-c/step-01-load-context.md +34 -1
- package/src/workflows/testarch/test-review/steps-c/step-04-generate-report.md +14 -3
- package/src/workflows/testarch/test-review/test-review-template.md +4 -2
- package/src/workflows/testarch/test-review/workflow.yaml +1 -0
- package/src/workflows/testarch/trace/trace-template.md +7 -5
- package/test/test-installation-components.js +1 -1
- package/test/test-knowledge-base.js +10 -1
|
@@ -4,11 +4,23 @@
|
|
|
4
4
|
|
|
5
5
|
- [ ] Git repository initialized (`.git/` exists)
|
|
6
6
|
- [ ] Git remote configured (`git remote -v` shows origin)
|
|
7
|
-
- [ ] Test framework configured (
|
|
8
|
-
- [ ] Local tests pass (
|
|
7
|
+
- [ ] Test framework configured (appropriate config for detected stack type)
|
|
8
|
+
- [ ] Local tests pass (test command succeeds)
|
|
9
9
|
- [ ] Team agrees on CI platform
|
|
10
10
|
- [ ] Access to CI platform settings (if updating)
|
|
11
11
|
|
|
12
|
+
### Multi-Stack Detection
|
|
13
|
+
|
|
14
|
+
- [ ] Test stack type detected or configured (`frontend`, `backend`, `fullstack`)
|
|
15
|
+
- [ ] Test framework detected or configured (Playwright, Cypress, Jest, Vitest, etc.)
|
|
16
|
+
- [ ] Stack-appropriate test commands identified
|
|
17
|
+
|
|
18
|
+
### Multi-Platform Detection
|
|
19
|
+
|
|
20
|
+
- [ ] CI platform detected or configured
|
|
21
|
+
- [ ] Supported platform: GitHub Actions, GitLab CI, Jenkins, Azure DevOps, Harness, or Circle CI
|
|
22
|
+
- [ ] Platform-specific template selected
|
|
23
|
+
|
|
12
24
|
Note: CI setup is typically a one-time task per repo and can be run any time after the test framework is configured.
|
|
13
25
|
|
|
14
26
|
## Process Steps
|
|
@@ -24,11 +36,21 @@ Note: CI setup is typically a one-time task per repo and can be run any time aft
|
|
|
24
36
|
|
|
25
37
|
### Step 2: CI Pipeline Configuration
|
|
26
38
|
|
|
27
|
-
- [ ] CI configuration file created
|
|
28
|
-
-
|
|
29
|
-
-
|
|
39
|
+
- [ ] CI configuration file created at platform-correct path
|
|
40
|
+
- GitHub Actions: `.github/workflows/test.yml`
|
|
41
|
+
- GitLab CI: `.gitlab-ci.yml`
|
|
42
|
+
- Jenkins: `Jenkinsfile`
|
|
43
|
+
- Azure DevOps: `azure-pipelines.yml`
|
|
44
|
+
- Harness: `.harness/pipeline.yaml`
|
|
45
|
+
- Circle CI: `.circleci/config.yml`
|
|
46
|
+
- [ ] File is syntactically valid (no YAML/Groovy errors)
|
|
47
|
+
- [ ] Correct framework commands configured for detected stack type
|
|
30
48
|
- [ ] Node version matches project
|
|
31
49
|
- [ ] Test directory paths correct
|
|
50
|
+
- [ ] Stack-conditional steps applied:
|
|
51
|
+
- [ ] Browser install included for frontend/fullstack stacks
|
|
52
|
+
- [ ] Browser install omitted for backend-only stacks
|
|
53
|
+
- [ ] Test commands match detected framework
|
|
32
54
|
|
|
33
55
|
### Step 3: Parallel Sharding
|
|
34
56
|
|
|
@@ -39,11 +61,12 @@ Note: CI setup is typically a one-time task per repo and can be run any time aft
|
|
|
39
61
|
|
|
40
62
|
### Step 4: Burn-In Loop
|
|
41
63
|
|
|
42
|
-
- [ ] Burn-in job created
|
|
43
|
-
- [ ] 10 iterations configured
|
|
64
|
+
- [ ] Burn-in job created (frontend/fullstack stacks) or intentionally skipped (backend-only)
|
|
65
|
+
- [ ] 10 iterations configured (when enabled)
|
|
44
66
|
- [ ] Proper exit on failure (`|| exit 1`)
|
|
45
67
|
- [ ] Runs on appropriate triggers (PR, cron)
|
|
46
68
|
- [ ] Failure artifacts uploaded
|
|
69
|
+
- [ ] Backend-only stacks: burn-in skipped by default (documented reason: targets UI flakiness)
|
|
47
70
|
|
|
48
71
|
### Step 5: Caching Configuration
|
|
49
72
|
|
|
@@ -237,6 +260,24 @@ If workflow fails:
|
|
|
237
260
|
- Runners: Shared or project-specific
|
|
238
261
|
- Pipeline quota: 400 minutes/month free tier
|
|
239
262
|
|
|
263
|
+
**Jenkins:**
|
|
264
|
+
|
|
265
|
+
- Credentials: Manage Jenkins → Manage Credentials
|
|
266
|
+
- Agents: Configure build agents with Node.js
|
|
267
|
+
- Plugins: Pipeline, JUnit, HTML Publisher recommended
|
|
268
|
+
|
|
269
|
+
**Azure DevOps:**
|
|
270
|
+
|
|
271
|
+
- Variables: Pipelines → Library → Variable groups
|
|
272
|
+
- Agent pools: Azure-hosted or self-hosted
|
|
273
|
+
- Parallel jobs: 1 free (Microsoft-hosted)
|
|
274
|
+
|
|
275
|
+
**Harness:**
|
|
276
|
+
|
|
277
|
+
- Connectors: Configure container registry and code repo connectors
|
|
278
|
+
- Delegates: Install Harness delegate in target infrastructure
|
|
279
|
+
- Steps: Use Run steps with appropriate container images
|
|
280
|
+
|
|
240
281
|
---
|
|
241
282
|
|
|
242
283
|
**Checklist Complete**: Sign off when all items validated.
|
|
@@ -1,6 +1,14 @@
|
|
|
1
1
|
# GitHub Actions CI/CD Pipeline for Test Execution
|
|
2
2
|
# Generated by BMad TEA Agent - Test Architect Module
|
|
3
|
-
# Optimized for:
|
|
3
|
+
# Optimized for: Parallel Sharding, Burn-In Loop
|
|
4
|
+
# Stack: {test_stack_type} | Framework: {test_framework}
|
|
5
|
+
#
|
|
6
|
+
# Variables to customize per project:
|
|
7
|
+
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile, yarn --frozen-lockfile)
|
|
8
|
+
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
|
|
9
|
+
# LINT_CMD - lint command (e.g., npm run lint)
|
|
10
|
+
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
|
|
11
|
+
# BROWSER_CACHE_PATH - browser cache path (frontend/fullstack only; omit for backend)
|
|
4
12
|
|
|
5
13
|
name: Test Pipeline
|
|
6
14
|
|
|
@@ -45,10 +53,10 @@ jobs:
|
|
|
45
53
|
cache: "npm"
|
|
46
54
|
|
|
47
55
|
- name: Install dependencies
|
|
48
|
-
run: npm ci
|
|
56
|
+
run: npm ci # Replace with INSTALL_CMD
|
|
49
57
|
|
|
50
58
|
- name: Run linter
|
|
51
|
-
run: npm run lint
|
|
59
|
+
run: npm run lint # Replace with LINT_CMD
|
|
52
60
|
|
|
53
61
|
# Test stage - Parallel execution with sharding
|
|
54
62
|
test:
|
|
@@ -91,13 +99,14 @@ jobs:
|
|
|
91
99
|
${{ runner.os }}-playwright-
|
|
92
100
|
|
|
93
101
|
- name: Install dependencies
|
|
94
|
-
run: npm ci
|
|
102
|
+
run: npm ci # Replace with INSTALL_CMD
|
|
95
103
|
|
|
104
|
+
# Frontend/Fullstack only — remove this step for backend-only stacks
|
|
96
105
|
- name: Install Playwright browsers
|
|
97
|
-
run: npx playwright install --with-deps chromium
|
|
106
|
+
run: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
|
|
98
107
|
|
|
99
108
|
- name: Run tests (shard ${{ matrix.shard }}/4)
|
|
100
|
-
run: npm run test:e2e -- --shard=${{ matrix.shard }}/4
|
|
109
|
+
run: npm run test:e2e -- --shard=${{ matrix.shard }}/4 # Replace with TEST_CMD + shard args
|
|
101
110
|
|
|
102
111
|
- name: Upload test results
|
|
103
112
|
if: failure()
|
|
@@ -138,18 +147,21 @@ jobs:
|
|
|
138
147
|
node-version: ${{ steps.node-version.outputs.value }}
|
|
139
148
|
cache: "npm"
|
|
140
149
|
|
|
150
|
+
# Frontend/Fullstack only — remove this step for backend-only stacks
|
|
141
151
|
- name: Cache Playwright browsers
|
|
142
152
|
uses: actions/cache@v4
|
|
143
153
|
with:
|
|
144
|
-
path: ~/.cache/ms-playwright
|
|
154
|
+
path: ~/.cache/ms-playwright # Replace with BROWSER_CACHE_PATH
|
|
145
155
|
key: ${{ runner.os }}-playwright-${{ hashFiles('**/package-lock.json') }}
|
|
146
156
|
|
|
147
157
|
- name: Install dependencies
|
|
148
|
-
run: npm ci
|
|
158
|
+
run: npm ci # Replace with INSTALL_CMD
|
|
149
159
|
|
|
160
|
+
# Frontend/Fullstack only — remove this step for backend-only stacks
|
|
150
161
|
- name: Install Playwright browsers
|
|
151
|
-
run: npx playwright install --with-deps chromium
|
|
162
|
+
run: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
|
|
152
163
|
|
|
164
|
+
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this job entirely.
|
|
153
165
|
- name: Run burn-in loop (10 iterations)
|
|
154
166
|
run: |
|
|
155
167
|
echo "🔥 Starting burn-in loop - detecting flaky tests"
|
|
@@ -157,7 +169,7 @@ jobs:
|
|
|
157
169
|
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
|
158
170
|
echo "🔥 Burn-in iteration $i/10"
|
|
159
171
|
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
|
160
|
-
npm run test:e2e || exit 1
|
|
172
|
+
npm run test:e2e || exit 1 # Replace with TEST_CMD
|
|
161
173
|
done
|
|
162
174
|
echo "✅ Burn-in complete - no flaky tests detected"
|
|
163
175
|
|
|
@@ -1,6 +1,14 @@
|
|
|
1
1
|
# GitLab CI/CD Pipeline for Test Execution
|
|
2
2
|
# Generated by BMad TEA Agent - Test Architect Module
|
|
3
|
-
# Optimized for:
|
|
3
|
+
# Optimized for: Parallel Sharding, Burn-In Loop
|
|
4
|
+
# Stack: {test_stack_type} | Framework: {test_framework}
|
|
5
|
+
#
|
|
6
|
+
# Variables to customize per project:
|
|
7
|
+
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
|
|
8
|
+
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
|
|
9
|
+
# LINT_CMD - lint command (e.g., npm run lint)
|
|
10
|
+
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
|
|
11
|
+
# BROWSER_CACHE_PATH - browser cache path (frontend/fullstack only; omit for backend)
|
|
4
12
|
|
|
5
13
|
stages:
|
|
6
14
|
- lint
|
|
@@ -39,9 +47,9 @@ lint:
|
|
|
39
47
|
npm install -g n
|
|
40
48
|
n "$NODE_VERSION"
|
|
41
49
|
node -v
|
|
42
|
-
- npm ci
|
|
50
|
+
- npm ci # Replace with INSTALL_CMD
|
|
43
51
|
script:
|
|
44
|
-
- npm run lint
|
|
52
|
+
- npm run lint # Replace with LINT_CMD
|
|
45
53
|
timeout: 5 minutes
|
|
46
54
|
|
|
47
55
|
# Test stage - Parallel execution with sharding
|
|
@@ -57,8 +65,8 @@ lint:
|
|
|
57
65
|
npm install -g n
|
|
58
66
|
n "$NODE_VERSION"
|
|
59
67
|
node -v
|
|
60
|
-
- npm ci
|
|
61
|
-
- npx playwright install --with-deps chromium
|
|
68
|
+
- npm ci # Replace with INSTALL_CMD
|
|
69
|
+
- npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL; remove for backend-only
|
|
62
70
|
artifacts:
|
|
63
71
|
when: on_failure
|
|
64
72
|
paths:
|
|
@@ -70,22 +78,22 @@ lint:
|
|
|
70
78
|
test:shard-1:
|
|
71
79
|
<<: *test-template
|
|
72
80
|
script:
|
|
73
|
-
- npm run test:e2e -- --shard=1/4
|
|
81
|
+
- npm run test:e2e -- --shard=1/4 # Replace with TEST_CMD + shard args
|
|
74
82
|
|
|
75
83
|
test:shard-2:
|
|
76
84
|
<<: *test-template
|
|
77
85
|
script:
|
|
78
|
-
- npm run test:e2e -- --shard=2/4
|
|
86
|
+
- npm run test:e2e -- --shard=2/4 # Replace with TEST_CMD + shard args
|
|
79
87
|
|
|
80
88
|
test:shard-3:
|
|
81
89
|
<<: *test-template
|
|
82
90
|
script:
|
|
83
|
-
- npm run test:e2e -- --shard=3/4
|
|
91
|
+
- npm run test:e2e -- --shard=3/4 # Replace with TEST_CMD + shard args
|
|
84
92
|
|
|
85
93
|
test:shard-4:
|
|
86
94
|
<<: *test-template
|
|
87
95
|
script:
|
|
88
|
-
- npm run test:e2e -- --shard=4/4
|
|
96
|
+
- npm run test:e2e -- --shard=4/4 # Replace with TEST_CMD + shard args
|
|
89
97
|
|
|
90
98
|
# Burn-in stage - Flaky test detection
|
|
91
99
|
burn-in:
|
|
@@ -107,8 +115,9 @@ burn-in:
|
|
|
107
115
|
npm install -g n
|
|
108
116
|
n "$NODE_VERSION"
|
|
109
117
|
node -v
|
|
110
|
-
- npm ci
|
|
111
|
-
- npx playwright install --with-deps chromium
|
|
118
|
+
- npm ci # Replace with INSTALL_CMD
|
|
119
|
+
- npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL; remove for backend-only
|
|
120
|
+
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this job entirely.
|
|
112
121
|
script:
|
|
113
122
|
- |
|
|
114
123
|
echo "🔥 Starting burn-in loop - detecting flaky tests"
|
|
@@ -116,7 +125,7 @@ burn-in:
|
|
|
116
125
|
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
|
117
126
|
echo "🔥 Burn-in iteration $i/10"
|
|
118
127
|
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
|
|
119
|
-
npm run test:e2e || exit 1
|
|
128
|
+
npm run test:e2e || exit 1 # Replace with TEST_CMD
|
|
120
129
|
done
|
|
121
130
|
echo "✅ Burn-in complete - no flaky tests detected"
|
|
122
131
|
artifacts:
|
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# Harness CI Pipeline for Test Execution
|
|
2
|
+
# Generated by BMad TEA Agent - Test Architect Module
|
|
3
|
+
# Optimized for: Parallel Sharding, Burn-In Loop
|
|
4
|
+
# Stack: {test_stack_type} | Framework: {test_framework}
|
|
5
|
+
#
|
|
6
|
+
# Variables to customize per project:
|
|
7
|
+
# INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
|
|
8
|
+
# TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
|
|
9
|
+
# LINT_CMD - lint command (e.g., npm run lint)
|
|
10
|
+
# BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
|
|
11
|
+
|
|
12
|
+
pipeline:
|
|
13
|
+
name: Test Pipeline
|
|
14
|
+
identifier: test_pipeline
|
|
15
|
+
projectIdentifier: default
|
|
16
|
+
orgIdentifier: default
|
|
17
|
+
stages:
|
|
18
|
+
# Lint stage - Code quality checks
|
|
19
|
+
- stage:
|
|
20
|
+
name: Lint
|
|
21
|
+
identifier: lint
|
|
22
|
+
type: CI
|
|
23
|
+
spec:
|
|
24
|
+
cloneCodebase: true
|
|
25
|
+
infrastructure:
|
|
26
|
+
type: KubernetesDirect
|
|
27
|
+
spec:
|
|
28
|
+
connectorRef: account.harnessImage
|
|
29
|
+
namespace: default
|
|
30
|
+
execution:
|
|
31
|
+
steps:
|
|
32
|
+
- step:
|
|
33
|
+
type: Run
|
|
34
|
+
name: Install dependencies
|
|
35
|
+
identifier: install
|
|
36
|
+
spec:
|
|
37
|
+
connectorRef: account.harnessImage
|
|
38
|
+
image: node:24
|
|
39
|
+
shell: Sh
|
|
40
|
+
command: npm ci # Replace with INSTALL_CMD
|
|
41
|
+
|
|
42
|
+
- step:
|
|
43
|
+
type: Run
|
|
44
|
+
name: Run linter
|
|
45
|
+
identifier: lint
|
|
46
|
+
spec:
|
|
47
|
+
connectorRef: account.harnessImage
|
|
48
|
+
image: node:24
|
|
49
|
+
shell: Sh
|
|
50
|
+
command: npm run lint # Replace with LINT_CMD
|
|
51
|
+
|
|
52
|
+
# Test stage - Parallel execution with sharding
|
|
53
|
+
- stage:
|
|
54
|
+
name: Test
|
|
55
|
+
identifier: test
|
|
56
|
+
type: CI
|
|
57
|
+
spec:
|
|
58
|
+
cloneCodebase: true
|
|
59
|
+
infrastructure:
|
|
60
|
+
type: KubernetesDirect
|
|
61
|
+
spec:
|
|
62
|
+
connectorRef: account.harnessImage
|
|
63
|
+
namespace: default
|
|
64
|
+
execution:
|
|
65
|
+
steps:
|
|
66
|
+
- step:
|
|
67
|
+
type: Run
|
|
68
|
+
name: Install dependencies
|
|
69
|
+
identifier: install
|
|
70
|
+
spec:
|
|
71
|
+
connectorRef: account.harnessImage
|
|
72
|
+
image: node:24
|
|
73
|
+
shell: Sh
|
|
74
|
+
command: npm ci # Replace with INSTALL_CMD
|
|
75
|
+
|
|
76
|
+
# Frontend/Fullstack only — remove this step for backend-only stacks
|
|
77
|
+
- step:
|
|
78
|
+
type: Run
|
|
79
|
+
name: Install browsers
|
|
80
|
+
identifier: browsers
|
|
81
|
+
spec:
|
|
82
|
+
connectorRef: account.harnessImage
|
|
83
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
84
|
+
shell: Sh
|
|
85
|
+
command: npx playwright install --with-deps chromium # Replace with BROWSER_INSTALL
|
|
86
|
+
|
|
87
|
+
- parallel:
|
|
88
|
+
- step:
|
|
89
|
+
type: Run
|
|
90
|
+
name: Test Shard 1
|
|
91
|
+
identifier: shard_1
|
|
92
|
+
spec:
|
|
93
|
+
connectorRef: account.harnessImage
|
|
94
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
95
|
+
shell: Sh
|
|
96
|
+
command: npm run test:e2e -- --shard=1/4 # Replace with TEST_CMD + shard args
|
|
97
|
+
- step:
|
|
98
|
+
type: Run
|
|
99
|
+
name: Test Shard 2
|
|
100
|
+
identifier: shard_2
|
|
101
|
+
spec:
|
|
102
|
+
connectorRef: account.harnessImage
|
|
103
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
104
|
+
shell: Sh
|
|
105
|
+
command: npm run test:e2e -- --shard=2/4 # Replace with TEST_CMD + shard args
|
|
106
|
+
- step:
|
|
107
|
+
type: Run
|
|
108
|
+
name: Test Shard 3
|
|
109
|
+
identifier: shard_3
|
|
110
|
+
spec:
|
|
111
|
+
connectorRef: account.harnessImage
|
|
112
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
113
|
+
shell: Sh
|
|
114
|
+
command: npm run test:e2e -- --shard=3/4 # Replace with TEST_CMD + shard args
|
|
115
|
+
- step:
|
|
116
|
+
type: Run
|
|
117
|
+
name: Test Shard 4
|
|
118
|
+
identifier: shard_4
|
|
119
|
+
spec:
|
|
120
|
+
connectorRef: account.harnessImage
|
|
121
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
122
|
+
shell: Sh
|
|
123
|
+
command: npm run test:e2e -- --shard=4/4 # Replace with TEST_CMD + shard args
|
|
124
|
+
|
|
125
|
+
# Burn-in stage - Flaky test detection
|
|
126
|
+
# Note: Burn-in targets UI flakiness. For backend-only stacks, remove this stage entirely.
|
|
127
|
+
- stage:
|
|
128
|
+
name: Burn-In
|
|
129
|
+
identifier: burn_in
|
|
130
|
+
type: CI
|
|
131
|
+
when:
|
|
132
|
+
condition: <+pipeline.triggerType> == "WEBHOOK" || <+pipeline.triggerType> == "SCHEDULER"
|
|
133
|
+
spec:
|
|
134
|
+
cloneCodebase: true
|
|
135
|
+
infrastructure:
|
|
136
|
+
type: KubernetesDirect
|
|
137
|
+
spec:
|
|
138
|
+
connectorRef: account.harnessImage
|
|
139
|
+
namespace: default
|
|
140
|
+
execution:
|
|
141
|
+
steps:
|
|
142
|
+
- step:
|
|
143
|
+
type: Run
|
|
144
|
+
name: Install and burn-in
|
|
145
|
+
identifier: burn_in_loop
|
|
146
|
+
spec:
|
|
147
|
+
connectorRef: account.harnessImage
|
|
148
|
+
image: mcr.microsoft.com/playwright:v1.50.0-noble
|
|
149
|
+
shell: Sh
|
|
150
|
+
command: |
|
|
151
|
+
npm ci
|
|
152
|
+
npx playwright install --with-deps chromium
|
|
153
|
+
echo "Starting burn-in loop - detecting flaky tests"
|
|
154
|
+
for i in $(seq 1 10); do
|
|
155
|
+
echo "Burn-in iteration $i/10"
|
|
156
|
+
npm run test:e2e || exit 1
|
|
157
|
+
done
|
|
158
|
+
echo "Burn-in complete - no flaky tests detected"
|
|
159
|
+
# Replace npm ci with INSTALL_CMD, npm run test:e2e with TEST_CMD
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
// Jenkinsfile CI/CD Pipeline for Test Execution
|
|
2
|
+
// Generated by BMad TEA Agent - Test Architect Module
|
|
3
|
+
// Optimized for: Parallel Sharding, Burn-In Loop
|
|
4
|
+
// Stack: {test_stack_type} | Framework: {test_framework}
|
|
5
|
+
//
|
|
6
|
+
// Variables to customize per project:
|
|
7
|
+
// INSTALL_CMD - dependency install command (e.g., npm ci, pnpm install --frozen-lockfile)
|
|
8
|
+
// TEST_CMD - main test command (e.g., npm run test:e2e, npm test, npx vitest)
|
|
9
|
+
// LINT_CMD - lint command (e.g., npm run lint)
|
|
10
|
+
// BROWSER_INSTALL - browser install command (frontend/fullstack only; omit for backend)
|
|
11
|
+
//
|
|
12
|
+
// Node.js version management — choose one:
|
|
13
|
+
// Option A (recommended): Configure NodeJS Plugin in Jenkins Global Tool Configuration,
|
|
14
|
+
// then add to pipeline: tools { nodejs 'NodeJS-24' }
|
|
15
|
+
// Option B: Use nvm (pre-installed on agent) — this template uses nvm as the default
|
|
16
|
+
// Option C: Use a Docker agent — agent { docker { image 'node:24' } }
|
|
17
|
+
|
|
18
|
+
pipeline {
|
|
19
|
+
agent any
|
|
20
|
+
|
|
21
|
+
environment {
|
|
22
|
+
CI = 'true'
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
options {
|
|
26
|
+
timeout(time: 45, unit: 'MINUTES')
|
|
27
|
+
disableConcurrentBuilds()
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
stages {
|
|
31
|
+
stage('Checkout') {
|
|
32
|
+
steps {
|
|
33
|
+
checkout scm
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
stage('Install') {
|
|
38
|
+
steps {
|
|
39
|
+
// Detect and apply Node.js version from .nvmrc (falls back to v24)
|
|
40
|
+
// If using NodeJS Plugin instead, remove this block and add: tools { nodejs 'NodeJS-24' }
|
|
41
|
+
sh '''
|
|
42
|
+
export NVM_DIR="$HOME/.nvm"
|
|
43
|
+
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
|
|
44
|
+
NODE_VERSION=$(cat .nvmrc 2>/dev/null || echo "24")
|
|
45
|
+
nvm install "$NODE_VERSION" 2>/dev/null || true
|
|
46
|
+
nvm use "$NODE_VERSION" 2>/dev/null || true
|
|
47
|
+
node --version
|
|
48
|
+
npm ci
|
|
49
|
+
''' // Replace npm ci with INSTALL_CMD
|
|
50
|
+
// Stash installed dependencies so parallel shards can restore them
|
|
51
|
+
stash includes: 'node_modules/**', name: 'deps'
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
stage('Lint') {
|
|
56
|
+
steps {
|
|
57
|
+
sh 'npm run lint' // Replace with LINT_CMD
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
|
|
61
|
+
// Test stage - Parallel execution with sharding
|
|
62
|
+
// Each shard restores dependencies via unstash for workspace safety
|
|
63
|
+
stage('Test') {
|
|
64
|
+
parallel {
|
|
65
|
+
stage('Shard 1') {
|
|
66
|
+
steps {
|
|
67
|
+
unstash 'deps'
|
|
68
|
+
// Frontend/Fullstack only — remove browser install for backend-only stacks
|
|
69
|
+
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
|
|
70
|
+
sh 'npm run test:e2e -- --shard=1/4' // Replace with TEST_CMD + shard args
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
stage('Shard 2') {
|
|
74
|
+
steps {
|
|
75
|
+
unstash 'deps'
|
|
76
|
+
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
|
|
77
|
+
sh 'npm run test:e2e -- --shard=2/4' // Replace with TEST_CMD + shard args
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
stage('Shard 3') {
|
|
81
|
+
steps {
|
|
82
|
+
unstash 'deps'
|
|
83
|
+
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
|
|
84
|
+
sh 'npm run test:e2e -- --shard=3/4' // Replace with TEST_CMD + shard args
|
|
85
|
+
}
|
|
86
|
+
}
|
|
87
|
+
stage('Shard 4') {
|
|
88
|
+
steps {
|
|
89
|
+
unstash 'deps'
|
|
90
|
+
sh 'npx playwright install --with-deps chromium' // Replace with BROWSER_INSTALL
|
|
91
|
+
sh 'npm run test:e2e -- --shard=4/4' // Replace with TEST_CMD + shard args
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
// Burn-in stage - Flaky test detection
|
|
98
|
+
// Note: Burn-in targets UI flakiness. For backend-only stacks, remove this stage entirely.
|
|
99
|
+
stage('Burn-In') {
|
|
100
|
+
when {
|
|
101
|
+
anyOf {
|
|
102
|
+
changeRequest()
|
|
103
|
+
triggeredBy 'TimerTrigger'
|
|
104
|
+
}
|
|
105
|
+
}
|
|
106
|
+
steps {
|
|
107
|
+
sh '''
|
|
108
|
+
echo "Starting burn-in loop - detecting flaky tests"
|
|
109
|
+
for i in $(seq 1 10); do
|
|
110
|
+
echo "Burn-in iteration $i/10"
|
|
111
|
+
npm run test:e2e || exit 1
|
|
112
|
+
done
|
|
113
|
+
echo "Burn-in complete - no flaky tests detected"
|
|
114
|
+
''' // Replace npm run test:e2e with TEST_CMD
|
|
115
|
+
}
|
|
116
|
+
}
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
post {
|
|
120
|
+
always {
|
|
121
|
+
// Archive test results and reports
|
|
122
|
+
archiveArtifacts artifacts: 'test-results/**,playwright-report/**', allowEmptyArchive: true
|
|
123
|
+
junit testResults: 'test-results/**/*.xml', allowEmptyResults: true
|
|
124
|
+
}
|
|
125
|
+
failure {
|
|
126
|
+
echo 'Pipeline failed - check test results and artifacts'
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
}
|
|
@@ -45,43 +45,84 @@ If missing: **HALT** with "Git repository required for CI/CD setup."
|
|
|
45
45
|
|
|
46
46
|
---
|
|
47
47
|
|
|
48
|
-
## 2.
|
|
48
|
+
## 2. Detect Test Stack Type
|
|
49
49
|
|
|
50
|
-
|
|
51
|
-
|
|
50
|
+
Determine the project's test stack type (`test_stack_type`) using the following algorithm:
|
|
51
|
+
|
|
52
|
+
1. If `test_stack_type` is explicitly set in config (not `"auto"`), use that value.
|
|
53
|
+
2. Otherwise, auto-detect by scanning project manifests:
|
|
54
|
+
- **Frontend indicators**: `playwright.config.*`, `cypress.config.*`, `vite.config.*`, `next.config.*`, `src/components/`, `src/pages/`, `src/app/`
|
|
55
|
+
- **Backend indicators**: `pyproject.toml`, `pom.xml`/`build.gradle`, `go.mod`, `*.csproj`/`*.sln`, `Gemfile`, `Cargo.toml`, `jest.config.*`, `vitest.config.*`, `src/routes/`, `src/controllers/`, `src/api/`, `Dockerfile`, `serverless.yml`
|
|
56
|
+
- **Both present** → `fullstack`
|
|
57
|
+
- **Only frontend** → `frontend`
|
|
58
|
+
- **Only backend** → `backend`
|
|
59
|
+
- **Cannot determine** → default to `fullstack` and note assumption
|
|
60
|
+
|
|
61
|
+
Record detected `test_stack_type` in step output.
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## 3. Verify Test Framework
|
|
66
|
+
|
|
67
|
+
- Check for framework configuration based on detected stack:
|
|
68
|
+
- **Frontend/Fullstack**: `playwright.config.*` or `cypress.config.*` exists
|
|
69
|
+
- **Backend (Node.js)**: `jest.config.*` or `vitest.config.*` or test scripts in `package.json`
|
|
70
|
+
- **Backend (Python)**: `pyproject.toml` with `[tool.pytest]` or `pytest.ini` or `setup.cfg` with pytest config
|
|
71
|
+
- **Backend (Java/Kotlin)**: `pom.xml` with surefire/failsafe plugins or `build.gradle` with test task
|
|
72
|
+
- **Backend (Go)**: `*_test.go` files present (Go convention — no config file needed)
|
|
73
|
+
- **Backend (C#/.NET)**: `*.csproj` with xUnit/NUnit/MSTest references
|
|
74
|
+
- **Backend (Ruby)**: `Gemfile` with rspec or `.rspec` config file
|
|
75
|
+
- If `test_framework` is `"auto"`, detect from config files and project manifests found
|
|
76
|
+
- Verify test dependencies are installed (language-appropriate package manager)
|
|
52
77
|
|
|
53
78
|
If missing: **HALT** with "Run `framework` workflow first."
|
|
54
79
|
|
|
55
80
|
---
|
|
56
81
|
|
|
57
|
-
##
|
|
82
|
+
## 4. Ensure Tests Pass Locally
|
|
58
83
|
|
|
59
|
-
- Run the main test command
|
|
84
|
+
- Run the main test command based on detected stack and framework:
|
|
85
|
+
- **Node.js**: `npm test` or `npm run test:e2e`
|
|
86
|
+
- **Python**: `pytest` or `python -m pytest`
|
|
87
|
+
- **Java**: `mvn test` or `gradle test`
|
|
88
|
+
- **Go**: `go test ./...`
|
|
89
|
+
- **C#/.NET**: `dotnet test`
|
|
90
|
+
- **Ruby**: `bundle exec rspec`
|
|
60
91
|
- If failing: **HALT** and request fixes before CI setup
|
|
61
92
|
|
|
62
93
|
---
|
|
63
94
|
|
|
64
|
-
##
|
|
95
|
+
## 5. Detect CI Platform
|
|
65
96
|
|
|
66
|
-
-
|
|
67
|
-
|
|
68
|
-
- `.
|
|
69
|
-
- `.
|
|
70
|
-
- `Jenkinsfile`
|
|
97
|
+
- If `ci_platform` is explicitly set in config (not `"auto"`), use that value.
|
|
98
|
+
- Otherwise, scan for existing CI configuration files:
|
|
99
|
+
- `.github/workflows/*.yml` → `github-actions`
|
|
100
|
+
- `.gitlab-ci.yml` → `gitlab-ci`
|
|
101
|
+
- `Jenkinsfile` → `jenkins`
|
|
102
|
+
- `azure-pipelines.yml` → `azure-devops`
|
|
103
|
+
- `.harness/*.yaml` → `harness`
|
|
104
|
+
- `.circleci/config.yml` → `circle-ci`
|
|
71
105
|
- If found, ask whether to update or replace
|
|
72
|
-
- If not found, infer from git remote (github.com →
|
|
73
|
-
-
|
|
106
|
+
- If not found, infer from git remote (github.com → `github-actions`, gitlab.com → `gitlab-ci`)
|
|
107
|
+
- If still unresolved, default to `github-actions`
|
|
108
|
+
|
|
109
|
+
Record detected `ci_platform` in step output.
|
|
74
110
|
|
|
75
111
|
---
|
|
76
112
|
|
|
77
|
-
##
|
|
113
|
+
## 6. Read Environment Context
|
|
78
114
|
|
|
79
|
-
- Read
|
|
80
|
-
- Read `package.json` for dependency caching strategy
|
|
115
|
+
- Read environment context based on detected stack:
|
|
116
|
+
- **Node.js**: Read `.nvmrc` if present (default to Node 24+ LTS if missing); read `package.json` for dependency caching strategy
|
|
117
|
+
- **Python**: Read `.python-version` or `pyproject.toml` for Python version; note `pip`/`poetry`/`pipenv` for caching
|
|
118
|
+
- **Java**: Read `pom.xml`/`build.gradle` for Java version; note Maven/Gradle for caching
|
|
119
|
+
- **Go**: Read `go.mod` for Go version; note Go module cache path
|
|
120
|
+
- **C#/.NET**: Read `*.csproj`/`global.json` for .NET SDK version; note NuGet cache
|
|
121
|
+
- **Ruby**: Read `.ruby-version` or `Gemfile` for Ruby version; note Bundler cache
|
|
81
122
|
|
|
82
123
|
---
|
|
83
124
|
|
|
84
|
-
###
|
|
125
|
+
### 7. Save Progress
|
|
85
126
|
|
|
86
127
|
**Save this step's accumulated work to `{outputFile}`.**
|
|
87
128
|
|