bmad-method-test-architecture-enterprise 1.23.2 → 1.23.3-next.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/CHANGELOG.md +19 -0
  3. package/docs/explanation/step-file-architecture.md +1 -3
  4. package/docs/reference/configuration.md +8 -0
  5. package/eslint.config.mjs +0 -8
  6. package/package.json +1 -1
  7. package/src/workflows/testarch/bmad-teach-me-testing/steps-v/step-v-01-validate.md +18 -12
  8. package/src/workflows/testarch/bmad-testarch-atdd/steps-v/step-01-validate.md +16 -6
  9. package/src/workflows/testarch/bmad-testarch-automate/steps-v/step-01-validate.md +16 -6
  10. package/src/workflows/testarch/bmad-testarch-ci/steps-v/step-01-validate.md +17 -7
  11. package/src/workflows/testarch/bmad-testarch-framework/steps-v/step-01-validate.md +16 -6
  12. package/src/workflows/testarch/bmad-testarch-nfr/steps-v/step-01-validate.md +16 -6
  13. package/src/workflows/testarch/bmad-testarch-test-design/steps-v/step-01-validate.md +16 -6
  14. package/src/workflows/testarch/bmad-testarch-test-review/steps-v/step-01-validate.md +16 -6
  15. package/src/workflows/testarch/bmad-testarch-trace/steps-v/step-01-validate.md +16 -6
  16. package/test/README.md +1 -1
  17. package/test/test-installation-components.js +38 -0
  18. package/website/astro.config.mjs +1 -0
  19. package/src/workflows/testarch/bmad-teach-me-testing/workflow-plan-teach-me-testing.md +0 -950
  20. package/src/workflows/testarch/bmad-testarch-atdd/validation-report-20260127-095021.md +0 -73
  21. package/src/workflows/testarch/bmad-testarch-atdd/validation-report-20260127-102401.md +0 -116
  22. package/src/workflows/testarch/bmad-testarch-atdd/workflow-plan.md +0 -21
  23. package/src/workflows/testarch/bmad-testarch-automate/validation-report-20260127-095021.md +0 -72
  24. package/src/workflows/testarch/bmad-testarch-automate/validation-report-20260127-102401.md +0 -114
  25. package/src/workflows/testarch/bmad-testarch-automate/workflow-plan.md +0 -20
  26. package/src/workflows/testarch/bmad-testarch-ci/validation-report-20260127-095021.md +0 -72
  27. package/src/workflows/testarch/bmad-testarch-ci/validation-report-20260127-102401.md +0 -114
  28. package/src/workflows/testarch/bmad-testarch-ci/workflow-plan.md +0 -20
  29. package/src/workflows/testarch/bmad-testarch-framework/validation-report-20260127-095021.md +0 -73
  30. package/src/workflows/testarch/bmad-testarch-framework/validation-report-20260127-102401.md +0 -116
  31. package/src/workflows/testarch/bmad-testarch-framework/workflow-plan.md +0 -22
  32. package/src/workflows/testarch/bmad-testarch-nfr/validation-report-20260127-095021.md +0 -73
  33. package/src/workflows/testarch/bmad-testarch-nfr/validation-report-20260127-102401.md +0 -116
  34. package/src/workflows/testarch/bmad-testarch-nfr/workflow-plan.md +0 -19
  35. package/src/workflows/testarch/bmad-testarch-test-design/validation-report-20260127-095021.md +0 -73
  36. package/src/workflows/testarch/bmad-testarch-test-design/validation-report-20260127-102401.md +0 -116
  37. package/src/workflows/testarch/bmad-testarch-test-design/workflow-plan.md +0 -23
  38. package/src/workflows/testarch/bmad-testarch-test-review/validation-report-20260127-095021.md +0 -72
  39. package/src/workflows/testarch/bmad-testarch-test-review/validation-report-20260127-102401.md +0 -114
  40. package/src/workflows/testarch/bmad-testarch-test-review/workflow-plan.md +0 -18
  41. package/src/workflows/testarch/bmad-testarch-trace/validation-report-20260127-095021.md +0 -73
  42. package/src/workflows/testarch/bmad-testarch-trace/validation-report-20260127-102401.md +0 -116
  43. package/src/workflows/testarch/bmad-testarch-trace/workflow-plan.md +0 -28
  44. package/test/unit-test-schema.js +0 -133
  45. package/test/validate-agent-schema.js +0 -110
  46. package/tools/verify-paths.js +0 -100
@@ -31,7 +31,7 @@
31
31
  "name": "bmad-method-test-architecture-enterprise",
32
32
  "source": "./",
33
33
  "description": "Master Test Architect module for quality strategy, test automation, CI/CD quality gates, and structured testing education. Part of the BMad Method ecosystem.",
34
- "version": "1.23.2",
34
+ "version": "1.23.3-next.1",
35
35
  "author": {
36
36
  "name": "Murat K Ozcan (TEA Creator) & Brian (BMad) Madison"
37
37
  },
package/CHANGELOG.md CHANGED
@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ### Fixed
11
+
12
+ - Validate mode now writes a scope-specific, timestamped report for each run instead of reusing one fixed filename. Artifact validation reports record the exact files checked, and every validate workflow atomically reserves its report through exclusive creation with collision retry, so sequential and parallel QA runs retain independent history (#140).
13
+
14
+ ## [1.23.3] - 2026-08-19
15
+
16
+ ### Removed
17
+
18
+ - Stale BMad Builder run artifacts from the skills migration. `e295715` carried 16 `validation-report-20260127-*.md` files, two per workflow across the eight `bmad-testarch-*` workflows, recording a 2026-01-27 validation run. Every one of them asserted `workflow.md present: YES` for a file that no longer exists in any workflow (it became `SKILL.md` in that same migration), and every step-file count and line count in them disagreed with the tree they shipped alongside. Each workflow directory is a published skill root in `.claude-plugin/marketplace.json`, so these were shipping to consumers as skill payload, where a wrong file listing is worse than no file listing.
19
+ - The nine authoring-time workflow plans: `workflow-plan.md` in each of the eight `bmad-testarch-*` workflows, plus `workflow-plan-teach-me-testing.md`. Nothing in `src/`, `test/`, `tools/`, `cli/`, or the docs read them; the only inbound references were from the validation reports removed above. Seven of the eight listed a step set that no longer matched their own directory, and the `teach-me-testing` one was a 950-line build-session log carrying `status: FOUNDATION_COMPLETE` frontmatter and a `cp -r` deployment instruction pointing at an external project root. Like the reports, they shipped inside a skill root.
20
+ - Three orphaned scripts from the TEA migration, each with zero inbound references from `package.json`, CI, or the pre-commit hook. `test/validate-agent-schema.js` was a stale copy of `tools/validate-agent-schema.js` whose own usage line named the `tools/` path; it globbed the abandoned `src/{core,modules/*}/agents/` layout and printed `No agent files found` when run. `test/unit-test-schema.js` failed one of its own three assertions and exited 0 regardless, so wiring it up would have gated nothing. `tools/verify-paths.js` was a one-off checker for leftover BMM references, and the migration it guarded is finished.
21
+ - Dead ignore entries that never matched anything in this repository's history: `test/template-test-generator/**` and its two sibling globs, `tools/template-test-generator/test-scenarios/**`, `test-project-install/**`, `sample-project/**`, `src/modules/*/sub-modules/**`, and `.bundler-temp/**` from `eslint.config.mjs`; `test-output/*` from `.gitignore`, along with its redundant `build/*.txt` entry, which `build/` already covers.
22
+
23
+ ### Fixed
24
+
25
+ - `docs/explanation/step-file-architecture.md` no longer claims the workflows carry committed validation reports "each scoring 100%". No score appears anywhere in the 16 reports the sentence described, so the claim was unverifiable against its own evidence before those reports were removed. The section now states what validation checks and that its output is a point-in-time reading of the working tree rather than a committed artifact.
26
+ - `docs/explanation/eval-quality-roadmap.md` is reachable from the documentation site. It was the only page in `docs/explanation/` missing from the Starlight sidebar in `website/astro.config.mjs`, so it built and published but could only be found through the direct link in `README.md`.
27
+ - `test/README.md` names the validator the suite actually loads. It cited `tools/schema/agent.js`, while `test/test-agent-schema.js` requires the byte-identical copy at `test/schema/agent.js`.
28
+
10
29
  ## [1.23.1] - 2026-08-16
11
30
 
12
31
  ### Changed
@@ -194,9 +194,7 @@ Users do not need to know any of this to run a workflow. What they see is consis
194
194
 
195
195
  ## Validation
196
196
 
197
- Eight of the nine TEA workflows carry BMad Builder validation reports at `src/workflows/testarch/<workflow>/validation-report-*.md`, each scoring 100%. `bmad-teach-me-testing` has no validation report.
198
-
199
- Validation checks for granular instructions, explicit exit conditions, context injection in every step, strict action boundaries, and subagent support where the workflow supports it.
197
+ Every workflow is validated with BMad Builder, which checks for granular instructions, explicit exit conditions, context injection in every step, strict action boundaries, and subagent support where the workflow supports it. Validation runs against the working tree at the time it is invoked, so its output is a point-in-time reading rather than a durable artifact; the reports are not committed. Re-run BMad Builder validation after editing a step file, and read the result from that run.
200
198
 
201
199
  All nine workflows have been exercised against real projects: `teach-me-testing` across a multi-session flow with persisted progress, `test-design` against a real story and epic, `automate` against real codebases, `atdd` for the red phase with failing tests confirmed, `test-review` against known good and bad suites, `nfr-assess` against a complex system, `trace` for both the coverage matrix and the gate decision, `framework` for Playwright and Cypress scaffolds, and `ci` for GitHub Actions and GitLab CI generation.
202
200
 
@@ -474,6 +474,14 @@ Paths are relative to `{test_artifacts}` unless noted. Deliverables are declared
474
474
 
475
475
  `trace` also reads an optional input it never writes: `live-verification-results.json`. Any producer may write it (an agent, a shell script, a CI job, or a person recording an outcome by hand). See [Live Verification Results](/docs/reference/live-verification-results.md) for the contract.
476
476
 
477
+ ### Validation Report History
478
+
479
+ Validate mode preserves every report as a separate artifact. The eight artifact-producing workflows write `{workflow}-validation-report-{validation_scope}-{run_timestamp}.md` under `{test_artifacts}`. The workflow identifier is `atdd`, `automate`, `ci`, `framework`, `nfr-assess`, `test-design`, `test-review`, or `trace`.
480
+
481
+ `validation_scope` identifies what was checked, such as `story-1-2`, `epic-9`, `system`, or `pull-request-123`. `run_timestamp` is the UTC start time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format. Each report also records the exact project-relative paths of its validated artifacts. Validate mode atomically reserves the resolved path with exclusive creation. A collision produces a fresh timestamp and retry, so two concurrent runs cannot claim the same report.
482
+
483
+ The `teach-me-testing` workflow validates its own workflow definition rather than a selected output scope. Its reports use `workflow-validation/teach-me-testing-validation-{run_timestamp}.md` under `{test_artifacts}` and follow the same no-overwrite rule.
484
+
477
485
  ---
478
486
 
479
487
  ## Environment Variables
package/eslint.config.mjs CHANGED
@@ -11,9 +11,6 @@ export default [
11
11
  'dist/**',
12
12
  'coverage/**',
13
13
  '**/*.min.js',
14
- 'test/template-test-generator/**',
15
- 'test/template-test-generator/**/*.js',
16
- 'test/template-test-generator/**/*.md',
17
14
  'test/fixtures/**',
18
15
  'test/fixtures/**/*.yaml',
19
16
  '_bmad/**',
@@ -31,11 +28,6 @@ export default [
31
28
  '.agentvibes/**',
32
29
  '.kiro/**',
33
30
  '.roo/**',
34
- 'test-project-install/**',
35
- 'sample-project/**',
36
- 'tools/template-test-generator/test-scenarios/**',
37
- 'src/modules/*/sub-modules/**',
38
- '.bundler-temp/**',
39
31
  ],
40
32
  },
41
33
 
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://json.schemastore.org/package.json",
3
3
  "name": "bmad-method-test-architecture-enterprise",
4
- "version": "1.23.2",
4
+ "version": "1.23.3-next.1",
5
5
  "description": "Master Test Architect for quality strategy, test automation, and release gates",
6
6
  "keywords": [
7
7
  "bmad",
@@ -4,7 +4,7 @@ description: 'Validate teach-me-testing workflow quality against BMAD standards'
4
4
 
5
5
  workflowPath: '{skill-root}'
6
6
  checklistFile: '{skill-root}/checklist.md'
7
- validationReport: '{test_artifacts}/workflow-validation/teach-me-testing-validation-{date}.md'
7
+ validationReport: '{test_artifacts}/workflow-validation/teach-me-testing-validation-{run_timestamp}.md'
8
8
  ---
9
9
 
10
10
  # Validate Step 1: Quality Validation
@@ -30,6 +30,7 @@ To systematically validate the teach-me-testing workflow against BMAD quality st
30
30
 
31
31
  - 🎯 Focus on comprehensive validation
32
32
  - 🚫 FORBIDDEN to skip any checks
33
+ - 🚫 Never overwrite an existing validation report
33
34
  - 💬 Report findings clearly
34
35
 
35
36
  ## EXECUTION PROTOCOLS:
@@ -40,7 +41,11 @@ To systematically validate the teach-me-testing workflow against BMAD quality st
40
41
 
41
42
  ## MANDATORY SEQUENCE
42
43
 
43
- ### 1. Validation Start
44
+ ### 1. Resolve Unique Report Path
45
+
46
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{validationReport}`. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `workflow: teach-me-testing`, `run_timestamp`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
47
+
48
+ ### 2. Validation Start
44
49
 
45
50
  "**Validating Workflow: teach-me-testing**
46
51
 
@@ -59,7 +64,7 @@ This will validate:
59
64
 
60
65
  **Starting validation...**"
61
66
 
62
- ### 2. Foundation Structure Validation
67
+ ### 3. Foundation Structure Validation
63
68
 
64
69
  **Check:**
65
70
 
@@ -72,7 +77,7 @@ This will validate:
72
77
 
73
78
  Report findings: Pass/Fail for each check.
74
79
 
75
- ### 3. Template Validation
80
+ ### 4. Template Validation
76
81
 
77
82
  **Check templates/:**
78
83
 
@@ -85,7 +90,7 @@ Report findings: Pass/Fail for each check.
85
90
 
86
91
  Report findings.
87
92
 
88
- ### 4. Step File Validation (CREATE Mode)
93
+ ### 5. Step File Validation (CREATE Mode)
89
94
 
90
95
  **For each of 12 steps in steps-c/:**
91
96
 
@@ -100,7 +105,7 @@ Report findings.
100
105
 
101
106
  Report findings per step.
102
107
 
103
- ### 5. Data File Validation
108
+ ### 6. Data File Validation
104
109
 
105
110
  **Check data/:**
106
111
 
@@ -112,7 +117,7 @@ Report findings per step.
112
117
 
113
118
  Report findings.
114
119
 
115
- ### 6. Content Quality Validation
120
+ ### 7. Content Quality Validation
116
121
 
117
122
  **Check session steps:**
118
123
 
@@ -125,7 +130,7 @@ Report findings.
125
130
 
126
131
  Report findings.
127
132
 
128
- ### 7. State Management Validation
133
+ ### 8. State Management Validation
129
134
 
130
135
  **Check continuable workflow features:**
131
136
 
@@ -138,7 +143,7 @@ Report findings.
138
143
 
139
144
  Report findings.
140
145
 
141
- ### 8. User Experience Validation
146
+ ### 9. User Experience Validation
142
147
 
143
148
  **Check UX:**
144
149
 
@@ -151,14 +156,15 @@ Report findings.
151
156
 
152
157
  Report findings.
153
158
 
154
- ### 9. Generate Validation Report
159
+ ### 10. Generate Validation Report
155
160
 
156
- Create {validationReport}:
161
+ Replace the `IN_PROGRESS` body in this run's reserved {validationReport} with:
157
162
 
158
163
  ```markdown
159
164
  ---
160
165
  workflow: teach-me-testing
161
166
  validation_date: { current_date }
167
+ run_timestamp: { run_timestamp }
162
168
  validator: TEA Validation Workflow
163
169
  overall_status: PASS / FAIL / PASS_WITH_WARNINGS
164
170
  ---
@@ -231,7 +237,7 @@ overall_status: PASS / FAIL / PASS_WITH_WARNINGS
231
237
  **Status:** {READY_FOR_PRODUCTION / NEEDS_FIXES / PASS_WITH_MINOR_ISSUES}
232
238
  ```
233
239
 
234
- ### 10. Display Results
240
+ ### 11. Display Results
235
241
 
236
242
  "**Validation Complete!**
237
243
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: 'step-01-validate'
3
3
  description: 'Validate workflow outputs against checklist'
4
- outputFile: '{test_artifacts}/atdd-validation-report.md'
4
+ outputFile: '{test_artifacts}/atdd-validation-report-{validation_scope}-{run_timestamp}.md'
5
5
  validationChecklist: '{skill-root}/checklist.md'
6
6
  ---
7
7
 
@@ -26,6 +26,7 @@ Validate outputs using the workflow checklist and record findings.
26
26
 
27
27
  - 🎯 Validate against `{validationChecklist}`
28
28
  - 🚫 Do not skip checks
29
+ - 🚫 Never overwrite an existing validation report
29
30
 
30
31
  ## EXECUTION PROTOCOLS:
31
32
 
@@ -34,7 +35,7 @@ Validate outputs using the workflow checklist and record findings.
34
35
 
35
36
  ## CONTEXT BOUNDARIES:
36
37
 
37
- - Available context: workflow outputs and checklist
38
+ - Available context: user-selected workflow outputs and checklist
38
39
  - Focus: validation only
39
40
  - Limits: do not modify outputs in this step
40
41
 
@@ -42,17 +43,25 @@ Validate outputs using the workflow checklist and record findings.
42
43
 
43
44
  **CRITICAL:** Follow this sequence exactly.
44
45
 
45
- ### 1. Load Checklist
46
+ ### 1. Select Scope and Resolve Report Path
47
+
48
+ Use the artifact paths the user supplied with the Validate request. If none were supplied, list the likely outputs for this workflow and ask which exact file or files to validate. When several candidates exist, do not guess.
49
+
50
+ Read the selected artifacts. Derive `validation_scope` from their shared story, epic, system, pull request, or other meaningful scope. Use an artifact basename without its extension when no broader scope is available. Normalize the value to lowercase ASCII with only letters, numbers, and single hyphens. Remove leading and trailing hyphens. Ask for a short scope label if normalization leaves an empty value.
51
+
52
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{outputFile}` with both values. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `validation_scope`, `run_timestamp`, `validated_artifacts`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
53
+
54
+ ### 2. Load Checklist
46
55
 
47
56
  Read `{validationChecklist}` and list all criteria.
48
57
 
49
- ### 2. Validate Outputs
58
+ ### 3. Validate Outputs
50
59
 
51
60
  Evaluate outputs against each checklist item.
52
61
 
53
- ### 3. Write Report
62
+ ### 4. Write Report
54
63
 
55
- Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
64
+ Replace the `IN_PROGRESS` body in this run's reserved `{outputFile}` with the final validation report. Include PASS/WARN/FAIL per section plus the original `validation_scope`, `run_timestamp`, and `validated_artifacts` metadata. Record every selected artifact using its exact project-relative path.
56
65
 
57
66
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
58
67
 
@@ -60,6 +69,7 @@ Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
60
69
 
61
70
  - Validation report written
62
71
  - All checklist items evaluated
72
+ - All selected artifacts recorded in the report
63
73
 
64
74
  ### ❌ SYSTEM FAILURE:
65
75
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: 'step-01-validate'
3
3
  description: 'Validate workflow outputs against checklist'
4
- outputFile: '{test_artifacts}/automate-validation-report.md'
4
+ outputFile: '{test_artifacts}/automate-validation-report-{validation_scope}-{run_timestamp}.md'
5
5
  validationChecklist: '{skill-root}/checklist.md'
6
6
  ---
7
7
 
@@ -26,6 +26,7 @@ Validate outputs using the workflow checklist and record findings.
26
26
 
27
27
  - 🎯 Validate against `{validationChecklist}`
28
28
  - 🚫 Do not skip checks
29
+ - 🚫 Never overwrite an existing validation report
29
30
 
30
31
  ## EXECUTION PROTOCOLS:
31
32
 
@@ -34,7 +35,7 @@ Validate outputs using the workflow checklist and record findings.
34
35
 
35
36
  ## CONTEXT BOUNDARIES:
36
37
 
37
- - Available context: workflow outputs and checklist
38
+ - Available context: user-selected workflow outputs and checklist
38
39
  - Focus: validation only
39
40
  - Limits: do not modify outputs in this step
40
41
 
@@ -42,17 +43,25 @@ Validate outputs using the workflow checklist and record findings.
42
43
 
43
44
  **CRITICAL:** Follow this sequence exactly.
44
45
 
45
- ### 1. Load Checklist
46
+ ### 1. Select Scope and Resolve Report Path
47
+
48
+ Use the artifact paths the user supplied with the Validate request. If none were supplied, list the likely outputs for this workflow and ask which exact file or files to validate. When several candidates exist, do not guess.
49
+
50
+ Read the selected artifacts. Derive `validation_scope` from their shared story, epic, system, pull request, or other meaningful scope. Use an artifact basename without its extension when no broader scope is available. Normalize the value to lowercase ASCII with only letters, numbers, and single hyphens. Remove leading and trailing hyphens. Ask for a short scope label if normalization leaves an empty value.
51
+
52
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{outputFile}` with both values. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `validation_scope`, `run_timestamp`, `validated_artifacts`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
53
+
54
+ ### 2. Load Checklist
46
55
 
47
56
  Read `{validationChecklist}` and list all criteria.
48
57
 
49
- ### 2. Validate Outputs
58
+ ### 3. Validate Outputs
50
59
 
51
60
  Evaluate outputs against each checklist item.
52
61
 
53
- ### 3. Write Report
62
+ ### 4. Write Report
54
63
 
55
- Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
64
+ Replace the `IN_PROGRESS` body in this run's reserved `{outputFile}` with the final validation report. Include PASS/WARN/FAIL per section plus the original `validation_scope`, `run_timestamp`, and `validated_artifacts` metadata. Record every selected artifact using its exact project-relative path.
56
65
 
57
66
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
58
67
 
@@ -60,6 +69,7 @@ Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
60
69
 
61
70
  - Validation report written
62
71
  - All checklist items evaluated
72
+ - All selected artifacts recorded in the report
63
73
 
64
74
  ### ❌ SYSTEM FAILURE:
65
75
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: 'step-01-validate'
3
3
  description: 'Validate workflow outputs against checklist'
4
- outputFile: '{test_artifacts}/ci-validation-report.md'
4
+ outputFile: '{test_artifacts}/ci-validation-report-{validation_scope}-{run_timestamp}.md'
5
5
  validationChecklist: '{skill-root}/checklist.md'
6
6
  ---
7
7
 
@@ -26,6 +26,7 @@ Validate outputs using the workflow checklist and record findings.
26
26
 
27
27
  - 🎯 Validate against `{validationChecklist}`
28
28
  - 🚫 Do not skip checks
29
+ - 🚫 Never overwrite an existing validation report
29
30
 
30
31
  ## EXECUTION PROTOCOLS:
31
32
 
@@ -34,7 +35,7 @@ Validate outputs using the workflow checklist and record findings.
34
35
 
35
36
  ## CONTEXT BOUNDARIES:
36
37
 
37
- - Available context: workflow outputs and checklist
38
+ - Available context: user-selected workflow outputs and checklist
38
39
  - Focus: validation only
39
40
  - Limits: do not modify outputs in this step
40
41
 
@@ -42,15 +43,23 @@ Validate outputs using the workflow checklist and record findings.
42
43
 
43
44
  **CRITICAL:** Follow this sequence exactly.
44
45
 
45
- ### 1. Load Checklist
46
+ ### 1. Select Scope and Resolve Report Path
47
+
48
+ Use the artifact paths the user supplied with the Validate request. If none were supplied, list the likely outputs for this workflow and ask which exact file or files to validate. When several candidates exist, do not guess.
49
+
50
+ Read the selected artifacts. Derive `validation_scope` from their shared story, epic, system, pull request, or other meaningful scope. Use an artifact basename without its extension when no broader scope is available. Normalize the value to lowercase ASCII with only letters, numbers, and single hyphens. Remove leading and trailing hyphens. Ask for a short scope label if normalization leaves an empty value.
51
+
52
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{outputFile}` with both values. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `validation_scope`, `run_timestamp`, `validated_artifacts`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
53
+
54
+ ### 2. Load Checklist
46
55
 
47
56
  Read `{validationChecklist}` and list all criteria.
48
57
 
49
- ### 2. Validate Outputs
58
+ ### 3. Validate Outputs
50
59
 
51
60
  Evaluate outputs against each checklist item.
52
61
 
53
- ### 2a. Script Injection Scan
62
+ ### 3a. Script Injection Scan
54
63
 
55
64
  Scan all generated YAML workflow files for unsafe interpolation patterns inside `run:` blocks.
56
65
 
@@ -64,9 +73,9 @@ Scan all generated YAML workflow files for unsafe interpolation patterns inside
64
73
 
65
74
  **Safe patterns to ignore** (exempt from flagging): `${{ steps.*.outputs.* }}`, `${{ matrix.* }}`, `${{ runner.os }}`, `${{ github.sha }}`, `${{ github.ref }}`, `${{ secrets.* }}`, `${{ env.* }}` — these are safe from GitHub expression injection when used in `run:` blocks.
66
75
 
67
- ### 3. Write Report
76
+ ### 4. Write Report
68
77
 
69
- Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
78
+ Replace the `IN_PROGRESS` body in this run's reserved `{outputFile}` with the final validation report. Include PASS/WARN/FAIL per section plus the original `validation_scope`, `run_timestamp`, and `validated_artifacts` metadata. Record every selected artifact using its exact project-relative path.
70
79
 
71
80
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
72
81
 
@@ -74,6 +83,7 @@ Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
74
83
 
75
84
  - Validation report written
76
85
  - All checklist items evaluated
86
+ - All selected artifacts recorded in the report
77
87
 
78
88
  ### ❌ SYSTEM FAILURE:
79
89
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: 'step-01-validate'
3
3
  description: 'Validate workflow outputs against checklist'
4
- outputFile: '{test_artifacts}/framework-validation-report.md'
4
+ outputFile: '{test_artifacts}/framework-validation-report-{validation_scope}-{run_timestamp}.md'
5
5
  validationChecklist: '{skill-root}/checklist.md'
6
6
  ---
7
7
 
@@ -26,6 +26,7 @@ Validate outputs using the workflow checklist and record findings.
26
26
 
27
27
  - 🎯 Validate against `{validationChecklist}`
28
28
  - 🚫 Do not skip checks
29
+ - 🚫 Never overwrite an existing validation report
29
30
 
30
31
  ## EXECUTION PROTOCOLS:
31
32
 
@@ -34,7 +35,7 @@ Validate outputs using the workflow checklist and record findings.
34
35
 
35
36
  ## CONTEXT BOUNDARIES:
36
37
 
37
- - Available context: workflow outputs and checklist
38
+ - Available context: user-selected workflow outputs and checklist
38
39
  - Focus: validation only
39
40
  - Limits: do not modify outputs in this step
40
41
 
@@ -42,17 +43,25 @@ Validate outputs using the workflow checklist and record findings.
42
43
 
43
44
  **CRITICAL:** Follow this sequence exactly.
44
45
 
45
- ### 1. Load Checklist
46
+ ### 1. Select Scope and Resolve Report Path
47
+
48
+ Use the artifact paths the user supplied with the Validate request. If none were supplied, list the likely outputs for this workflow and ask which exact file or files to validate. When several candidates exist, do not guess.
49
+
50
+ Read the selected artifacts. Derive `validation_scope` from their shared story, epic, system, pull request, or other meaningful scope. Use an artifact basename without its extension when no broader scope is available. Normalize the value to lowercase ASCII with only letters, numbers, and single hyphens. Remove leading and trailing hyphens. Ask for a short scope label if normalization leaves an empty value.
51
+
52
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{outputFile}` with both values. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `validation_scope`, `run_timestamp`, `validated_artifacts`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
53
+
54
+ ### 2. Load Checklist
46
55
 
47
56
  Read `{validationChecklist}` and list all criteria.
48
57
 
49
- ### 2. Validate Outputs
58
+ ### 3. Validate Outputs
50
59
 
51
60
  Evaluate outputs against each checklist item.
52
61
 
53
- ### 3. Write Report
62
+ ### 4. Write Report
54
63
 
55
- Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
64
+ Replace the `IN_PROGRESS` body in this run's reserved `{outputFile}` with the final validation report. Include PASS/WARN/FAIL per section plus the original `validation_scope`, `run_timestamp`, and `validated_artifacts` metadata. Record every selected artifact using its exact project-relative path.
56
65
 
57
66
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
58
67
 
@@ -60,6 +69,7 @@ Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
60
69
 
61
70
  - Validation report written
62
71
  - All checklist items evaluated
72
+ - All selected artifacts recorded in the report
63
73
 
64
74
  ### ❌ SYSTEM FAILURE:
65
75
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: 'step-01-validate'
3
3
  description: 'Validate workflow outputs against checklist'
4
- outputFile: '{test_artifacts}/nfr-assess-validation-report.md'
4
+ outputFile: '{test_artifacts}/nfr-assess-validation-report-{validation_scope}-{run_timestamp}.md'
5
5
  validationChecklist: '{skill-root}/checklist.md'
6
6
  ---
7
7
 
@@ -26,6 +26,7 @@ Validate outputs using the workflow checklist and record findings.
26
26
 
27
27
  - 🎯 Validate against `{validationChecklist}`
28
28
  - 🚫 Do not skip checks
29
+ - 🚫 Never overwrite an existing validation report
29
30
 
30
31
  ## EXECUTION PROTOCOLS:
31
32
 
@@ -34,7 +35,7 @@ Validate outputs using the workflow checklist and record findings.
34
35
 
35
36
  ## CONTEXT BOUNDARIES:
36
37
 
37
- - Available context: workflow outputs and checklist
38
+ - Available context: user-selected workflow outputs and checklist
38
39
  - Focus: validation only
39
40
  - Limits: do not modify outputs in this step
40
41
 
@@ -42,17 +43,25 @@ Validate outputs using the workflow checklist and record findings.
42
43
 
43
44
  **CRITICAL:** Follow this sequence exactly.
44
45
 
45
- ### 1. Load Checklist
46
+ ### 1. Select Scope and Resolve Report Path
47
+
48
+ Use the artifact paths the user supplied with the Validate request. If none were supplied, list the likely outputs for this workflow and ask which exact file or files to validate. When several candidates exist, do not guess.
49
+
50
+ Read the selected artifacts. Derive `validation_scope` from their shared story, epic, system, pull request, or other meaningful scope. Use an artifact basename without its extension when no broader scope is available. Normalize the value to lowercase ASCII with only letters, numbers, and single hyphens. Remove leading and trailing hyphens. Ask for a short scope label if normalization leaves an empty value.
51
+
52
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{outputFile}` with both values. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `validation_scope`, `run_timestamp`, `validated_artifacts`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
53
+
54
+ ### 2. Load Checklist
46
55
 
47
56
  Read `{validationChecklist}` and list all criteria.
48
57
 
49
- ### 2. Validate Outputs
58
+ ### 3. Validate Outputs
50
59
 
51
60
  Evaluate outputs against each checklist item.
52
61
 
53
- ### 3. Write Report
62
+ ### 4. Write Report
54
63
 
55
- Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
64
+ Replace the `IN_PROGRESS` body in this run's reserved `{outputFile}` with the final validation report. Include PASS/WARN/FAIL per section plus the original `validation_scope`, `run_timestamp`, and `validated_artifacts` metadata. Record every selected artifact using its exact project-relative path.
56
65
 
57
66
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
58
67
 
@@ -60,6 +69,7 @@ Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
60
69
 
61
70
  - Validation report written
62
71
  - All checklist items evaluated
72
+ - All selected artifacts recorded in the report
63
73
 
64
74
  ### ❌ SYSTEM FAILURE:
65
75
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  name: 'step-01-validate'
3
3
  description: 'Validate workflow outputs against checklist'
4
- outputFile: '{test_artifacts}/test-design-validation-report.md'
4
+ outputFile: '{test_artifacts}/test-design-validation-report-{validation_scope}-{run_timestamp}.md'
5
5
  validationChecklist: '{skill-root}/checklist.md'
6
6
  ---
7
7
 
@@ -26,6 +26,7 @@ Validate outputs using the workflow checklist and record findings.
26
26
 
27
27
  - 🎯 Validate against `{validationChecklist}`
28
28
  - 🚫 Do not skip checks
29
+ - 🚫 Never overwrite an existing validation report
29
30
 
30
31
  ## EXECUTION PROTOCOLS:
31
32
 
@@ -34,7 +35,7 @@ Validate outputs using the workflow checklist and record findings.
34
35
 
35
36
  ## CONTEXT BOUNDARIES:
36
37
 
37
- - Available context: workflow outputs and checklist
38
+ - Available context: user-selected workflow outputs and checklist
38
39
  - Focus: validation only
39
40
  - Limits: do not modify outputs in this step
40
41
 
@@ -42,17 +43,25 @@ Validate outputs using the workflow checklist and record findings.
42
43
 
43
44
  **CRITICAL:** Follow this sequence exactly.
44
45
 
45
- ### 1. Load Checklist
46
+ ### 1. Select Scope and Resolve Report Path
47
+
48
+ Use the artifact paths the user supplied with the Validate request. If none were supplied, list the likely outputs for this workflow and ask which exact file or files to validate. When several candidates exist, do not guess.
49
+
50
+ Read the selected artifacts. Derive `validation_scope` from their shared story, epic, system, pull request, or other meaningful scope. Use an artifact basename without its extension when no broader scope is available. Normalize the value to lowercase ASCII with only letters, numbers, and single hyphens. Remove leading and trailing hyphens. Ask for a short scope label if normalization leaves an empty value.
51
+
52
+ Set `run_timestamp` to the current UTC time with milliseconds in `YYYYMMDDTHHmmssSSSZ` format and resolve `{outputFile}` with both values. Atomically reserve that path using an exclusive-create operation that fails if the file already exists. A separate existence check followed by a normal write is forbidden. On collision, generate a fresh timestamp, resolve a new path, and retry exclusive creation until it succeeds. Initialize the reserved file with `validation_scope`, `run_timestamp`, `validated_artifacts`, and `status: IN_PROGRESS`. This run may update only the file it reserved. If the workflow stops, leave that reservation in place. Never delete, truncate, or reuse a report from another run. Always refuse to overwrite prior validation history.
53
+
54
+ ### 2. Load Checklist
46
55
 
47
56
  Read `{validationChecklist}` and list all criteria.
48
57
 
49
- ### 2. Validate Outputs
58
+ ### 3. Validate Outputs
50
59
 
51
60
  Evaluate outputs against each checklist item.
52
61
 
53
- ### 3. Write Report
62
+ ### 4. Write Report
54
63
 
55
- Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
64
+ Replace the `IN_PROGRESS` body in this run's reserved `{outputFile}` with the final validation report. Include PASS/WARN/FAIL per section plus the original `validation_scope`, `run_timestamp`, and `validated_artifacts` metadata. Record every selected artifact using its exact project-relative path.
56
65
 
57
66
  ## 🚨 SYSTEM SUCCESS/FAILURE METRICS:
58
67
 
@@ -60,6 +69,7 @@ Write a validation report to `{outputFile}` with PASS/WARN/FAIL per section.
60
69
 
61
70
  - Validation report written
62
71
  - All checklist items evaluated
72
+ - All selected artifacts recorded in the report
63
73
 
64
74
  ### ❌ SYSTEM FAILURE:
65
75