bmad-module-skill-forge 0.6.0 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (144) hide show
  1. package/README.md +2 -2
  2. package/docs/agents.md +9 -5
  3. package/docs/concepts.md +17 -13
  4. package/docs/examples.md +19 -4
  5. package/docs/getting-started.md +34 -19
  6. package/docs/how-it-works.md +87 -84
  7. package/docs/index.md +1 -1
  8. package/docs/workflows.md +88 -40
  9. package/package.json +1 -1
  10. package/src/agents/forger.agent.yaml +17 -5
  11. package/src/forger/README.md +1 -1
  12. package/src/forger/forge-tier.yaml +2 -1
  13. package/src/forger/preferences.yaml +5 -7
  14. package/src/knowledge/agentskills-spec.md +1 -1
  15. package/src/knowledge/architecture-verification.md +102 -0
  16. package/src/knowledge/ccc-bridge.md +3 -2
  17. package/src/knowledge/confidence-tiers.md +12 -1
  18. package/src/knowledge/doc-fetcher.md +1 -1
  19. package/src/knowledge/manual-section-integrity.md +2 -2
  20. package/src/knowledge/overview.md +11 -7
  21. package/src/knowledge/progressive-capability.md +5 -2
  22. package/src/knowledge/qmd-registry.md +5 -2
  23. package/src/knowledge/skf-knowledge-index.csv +2 -0
  24. package/src/knowledge/skill-lifecycle.md +26 -2
  25. package/src/knowledge/split-body-strategy.md +16 -2
  26. package/src/knowledge/tool-resolution.md +55 -0
  27. package/src/knowledge/zero-hallucination.md +11 -8
  28. package/src/module-help.csv +9 -6
  29. package/src/module.yaml +3 -0
  30. package/src/workflows/README.md +35 -17
  31. package/src/workflows/analyze-source/data/skill-brief-schema.md +18 -5
  32. package/src/workflows/analyze-source/steps-c/step-01-init.md +2 -2
  33. package/src/workflows/analyze-source/steps-c/step-01b-continue.md +1 -1
  34. package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +1 -1
  35. package/src/workflows/analyze-source/steps-c/step-03-identify-units.md +1 -1
  36. package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +3 -3
  37. package/src/workflows/analyze-source/steps-c/step-05-recommend.md +1 -1
  38. package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +2 -2
  39. package/src/workflows/analyze-source/workflow.md +2 -2
  40. package/src/workflows/audit-skill/data/severity-rules.md +1 -1
  41. package/src/workflows/audit-skill/steps-c/step-01-init.md +2 -1
  42. package/src/workflows/audit-skill/steps-c/step-02-re-index.md +8 -1
  43. package/src/workflows/audit-skill/steps-c/step-03-structural-diff.md +2 -2
  44. package/src/workflows/audit-skill/steps-c/step-04-semantic-diff.md +2 -2
  45. package/src/workflows/audit-skill/steps-c/step-05-severity-classify.md +1 -1
  46. package/src/workflows/audit-skill/workflow.md +2 -1
  47. package/src/workflows/brief-skill/data/scope-templates.md +10 -0
  48. package/src/workflows/brief-skill/data/skill-brief-schema.md +25 -23
  49. package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +1 -1
  50. package/src/workflows/brief-skill/steps-c/step-02-analyze-target.md +1 -1
  51. package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +3 -0
  52. package/src/workflows/create-skill/data/extraction-patterns.md +7 -1
  53. package/src/workflows/create-skill/data/skill-sections.md +12 -11
  54. package/src/workflows/create-skill/data/source-resolution-protocols.md +2 -2
  55. package/src/workflows/create-skill/data/tier-degradation-rules.md +1 -1
  56. package/src/workflows/create-skill/steps-c/step-01-load-brief.md +14 -10
  57. package/src/workflows/create-skill/steps-c/step-02b-ccc-discover.md +4 -0
  58. package/src/workflows/create-skill/steps-c/step-03-extract.md +19 -19
  59. package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +3 -3
  60. package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +3 -1
  61. package/src/workflows/create-skill/steps-c/step-04-enrich.md +3 -1
  62. package/src/workflows/create-skill/steps-c/step-05-compile.md +4 -3
  63. package/src/workflows/create-skill/steps-c/step-06-validate.md +2 -2
  64. package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +33 -44
  65. package/src/workflows/create-skill/steps-c/step-08-report.md +25 -11
  66. package/src/workflows/create-skill/workflow.md +2 -2
  67. package/src/workflows/create-stack-skill/data/compose-mode-rules.md +65 -0
  68. package/src/workflows/create-stack-skill/data/manifest-patterns.md +23 -1
  69. package/src/workflows/create-stack-skill/data/stack-skill-template.md +28 -6
  70. package/src/workflows/create-stack-skill/steps-c/step-01-init.md +25 -7
  71. package/src/workflows/create-stack-skill/steps-c/step-02-detect-manifests.md +37 -3
  72. package/src/workflows/create-stack-skill/steps-c/step-03-rank-and-confirm.md +29 -3
  73. package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +50 -7
  74. package/src/workflows/create-stack-skill/steps-c/step-05-detect-integrations.md +35 -3
  75. package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +5 -1
  76. package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +53 -5
  77. package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +3 -3
  78. package/src/workflows/create-stack-skill/steps-c/step-09-report.md +8 -2
  79. package/src/workflows/create-stack-skill/workflow.md +2 -2
  80. package/src/workflows/export-skill/data/managed-section-format.md +8 -1
  81. package/src/workflows/export-skill/data/snippet-format.md +1 -1
  82. package/src/workflows/export-skill/steps-c/step-04-update-context.md +7 -2
  83. package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
  84. package/src/workflows/export-skill/workflow.md +3 -1
  85. package/src/workflows/quick-skill/data/skill-template.md +18 -6
  86. package/src/workflows/quick-skill/steps-c/step-04-compile.md +26 -6
  87. package/src/workflows/quick-skill/steps-c/step-05-validate.md +1 -1
  88. package/src/workflows/quick-skill/steps-c/step-06-write.md +3 -0
  89. package/src/workflows/quick-skill/workflow.md +1 -1
  90. package/src/workflows/refine-architecture/data/refinement-rules.md +102 -0
  91. package/src/workflows/refine-architecture/steps-c/step-01-init.md +188 -0
  92. package/src/workflows/refine-architecture/steps-c/step-02-gap-analysis.md +179 -0
  93. package/src/workflows/refine-architecture/steps-c/step-03-issue-detection.md +182 -0
  94. package/src/workflows/refine-architecture/steps-c/step-04-improvements.md +169 -0
  95. package/src/workflows/refine-architecture/steps-c/step-05-compile.md +200 -0
  96. package/src/workflows/refine-architecture/steps-c/step-06-report.md +147 -0
  97. package/src/workflows/refine-architecture/workflow.md +61 -0
  98. package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +2 -1
  99. package/src/workflows/setup-forge/steps-c/step-01b-ccc-index.md +5 -3
  100. package/src/workflows/setup-forge/steps-c/step-02-write-config.md +15 -16
  101. package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +1 -1
  102. package/src/workflows/setup-forge/steps-c/step-04-report.md +2 -2
  103. package/src/workflows/setup-forge/workflow.md +3 -2
  104. package/src/workflows/test-skill/data/scoring-rules.md +3 -0
  105. package/src/workflows/test-skill/data/source-access-protocol.md +1 -1
  106. package/src/workflows/test-skill/steps-c/step-01-init.md +1 -0
  107. package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -0
  108. package/src/workflows/test-skill/workflow.md +2 -2
  109. package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +3 -3
  110. package/src/workflows/update-skill/steps-c/step-03-re-extract.md +8 -3
  111. package/src/workflows/update-skill/steps-c/step-05-validate.md +8 -12
  112. package/src/workflows/update-skill/steps-c/step-06-write.md +5 -2
  113. package/src/workflows/update-skill/workflow.md +1 -1
  114. package/src/workflows/verify-stack/data/coverage-patterns.md +45 -0
  115. package/src/workflows/verify-stack/data/feasibility-report-template.md +63 -0
  116. package/src/workflows/verify-stack/data/integration-verification-rules.md +73 -0
  117. package/src/workflows/verify-stack/steps-c/step-01-init.md +174 -0
  118. package/src/workflows/verify-stack/steps-c/step-02-coverage.md +181 -0
  119. package/src/workflows/verify-stack/steps-c/step-03-integrations.md +195 -0
  120. package/src/workflows/verify-stack/steps-c/step-04-requirements.md +168 -0
  121. package/src/workflows/verify-stack/steps-c/step-05-synthesize.md +191 -0
  122. package/src/workflows/verify-stack/steps-c/step-06-report.md +197 -0
  123. package/src/workflows/verify-stack/workflow.md +61 -0
  124. package/src/workflows/analyze-source/validation-report.md +0 -657
  125. package/src/workflows/analyze-source/workflow-plan-analyze-source.md +0 -385
  126. package/src/workflows/audit-skill/validation-report.md +0 -545
  127. package/src/workflows/audit-skill/workflow-plan-audit-skill.md +0 -318
  128. package/src/workflows/brief-skill/validation-report.md +0 -630
  129. package/src/workflows/brief-skill/workflow-plan-brief-skill.md +0 -360
  130. package/src/workflows/create-skill/validation-report.md +0 -189
  131. package/src/workflows/create-skill/workflow-plan-create-skill.md +0 -525
  132. package/src/workflows/create-stack-skill/validation-report.md +0 -401
  133. package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +0 -481
  134. package/src/workflows/export-skill/validation-report.md +0 -263
  135. package/src/workflows/export-skill/workflow-plan-export-skill.md +0 -369
  136. package/src/workflows/quick-skill/validation-report.md +0 -543
  137. package/src/workflows/quick-skill/workflow-plan-quick-skill.md +0 -405
  138. package/src/workflows/setup-forge/validation-report.md +0 -504
  139. package/src/workflows/setup-forge/workflow-plan-setup-forge.md +0 -430
  140. package/src/workflows/test-skill/validation-report.md +0 -390
  141. package/src/workflows/test-skill/workflow-plan-test-skill.md +0 -292
  142. package/src/workflows/update-skill/data/tier-degradation-rules.md +0 -46
  143. package/src/workflows/update-skill/validation-report.md +0 -559
  144. package/src/workflows/update-skill/workflow-plan-update-skill.md +0 -441
@@ -1,390 +0,0 @@
1
- ---
2
- validationDate: 2026-02-26
3
- completionDate: 2026-02-26
4
- workflowName: test-skill
5
- workflowPath: _bmad-output/bmb-creations/workflows/test-skill
6
- validationStatus: COMPLETE
7
- ---
8
-
9
- # Validation Report: test-skill
10
-
11
- **Validation Started:** 2026-02-26
12
- **Validation Completed:** 2026-02-26
13
- **Validator:** BMAD Workflow Validation System
14
- **Standards Version:** BMAD Workflow Standards
15
-
16
- ---
17
-
18
- ## File Structure & Size
19
-
20
- ### Folder Structure: PASS
21
-
22
- ```
23
- test-skill/
24
- ├── workflow.md (57 lines)
25
- ├── data/
26
- │ ├── scoring-rules.md (72 lines)
27
- │ └── output-section-formats.md (97 lines)
28
- ├── steps-c/
29
- │ ├── step-01-init.md (185 lines) ✅
30
- │ ├── step-02-detect-mode.md (162 lines) ✅
31
- │ ├── step-03-coverage-check.md (222 lines) ⚠️ Approaching limit
32
- │ ├── step-04-coherence-check.md (249 lines) ⚠️ Approaching limit
33
- │ ├── step-05-score.md (222 lines) ⚠️ Approaching limit
34
- │ └── step-06-report.md (175 lines) ✅
35
- └── templates/
36
- └── test-report-template.md (24 lines)
37
- ```
38
-
39
- ### Required Files: PASS
40
-
41
- - ✅ workflow.md exists
42
- - ✅ All 6 design steps have corresponding files
43
- - ✅ Sequential numbering (01-06) with no gaps
44
- - ✅ Final step (step-06) exists
45
- - ✅ Data folder with scoring rules and output formats
46
- - ✅ Templates folder with output template
47
-
48
- ### File Size Analysis: PASS (after remediation)
49
-
50
- | File | Lines | Status |
51
- |------|-------|--------|
52
- | step-01-init.md | 185 | ✅ Good |
53
- | step-02-detect-mode.md | 162 | ✅ Good |
54
- | step-03-coverage-check.md | 222 | ⚠️ Approaching limit |
55
- | step-04-coherence-check.md | 249 | ⚠️ Approaching limit (fixed from 294) |
56
- | step-05-score.md | 222 | ⚠️ Approaching limit |
57
- | step-06-report.md | 175 | ✅ Good (fixed from 254) |
58
-
59
- **Remediation applied:**
60
- - Created `data/output-section-formats.md` — extracted output templates from step-04 and step-06
61
- - step-04: 294 → 249 lines (output format templates moved to data file)
62
- - step-06: 254 → 175 lines (gap format templates and remediation examples moved to data file)
63
-
64
- ---
65
-
66
- ## Frontmatter Validation
67
-
68
- ### Frontmatter Variables: PASS
69
-
70
- All 21 frontmatter variables across 7 files are consumed in their step bodies.
71
-
72
- | Step | Variables | All Used | Status |
73
- |------|-----------|----------|--------|
74
- | workflow.md | name, description, web_bundle, installed_path | Yes (metadata fields) | PASS |
75
- | step-01 | nextStepFile, outputFile, templateFile, sidecarFile, skillsOutputFolder | Yes | PASS |
76
- | step-02 | nextStepFile, outputFile | Yes | PASS |
77
- | step-03 | nextStepFile, outputFile, scoringRulesFile | Yes | PASS |
78
- | step-04 | nextStepFile, outputFile, outputFormatsFile | Yes | PASS |
79
- | step-05 | nextStepFile, outputFile, scoringRulesFile | Yes | PASS |
80
- | step-06 | outputFile, scoringRulesFile, outputFormatsFile | Yes | PASS |
81
-
82
- ### Path Format: PASS
83
-
84
- - Step-to-step paths use `./filename.md` format (5/5)
85
- - Parent-folder paths use `../filename.md` format (6/6)
86
- - External paths use `{project-root}` prefix (2/2)
87
- - No `{workflow_path}` variable present (7/7)
88
- - All files have `name` + `description` fields (7/7)
89
-
90
- ---
91
-
92
- ## Critical Path Violations
93
-
94
- ### Hardcoded Paths in Step Bodies: PASS
95
-
96
- All 6 step files scanned — zero violations. All path references use frontmatter variables.
97
-
98
- ### Workflow.md Bootstrap Paths: PASS (with 2 standard warnings)
99
-
100
- | Line | Path | Context | Verdict |
101
- |------|------|---------|---------|
102
- | 50 | `{project-root}/_bmad/skf/config.yaml` | Config bootstrap | WARNING — standard entrypoint pattern |
103
- | 57 | `./steps-c/step-01-init.md` | First step bootstrap | WARNING — standard entrypoint pattern |
104
-
105
- Both are expected bootstrap patterns for workflow entrypoints.
106
-
107
- ### File Reference Chain: PASS
108
-
109
- | Source | Reference | Target Exists |
110
- |--------|-----------|---------------|
111
- | step-01 → step-02 | `./step-02-detect-mode.md` | ✅ |
112
- | step-02 → step-03 | `./step-03-coverage-check.md` | ✅ |
113
- | step-03 → step-04 | `./step-04-coherence-check.md` | ✅ |
114
- | step-04 → step-05 | `./step-05-score.md` | ✅ |
115
- | step-05 → step-06 | `./step-06-report.md` | ✅ |
116
- | step-06 | (final step — no nextStepFile) | ✅ |
117
- | step-01 → template | `../templates/test-report-template.md` | ✅ |
118
- | step-03, 05, 06 → scoring | `../data/scoring-rules.md` | ✅ |
119
- | step-04, 06 → formats | `../data/output-section-formats.md` | ✅ |
120
-
121
- ---
122
-
123
- ## Menu Handling Validation
124
-
125
- ### Menu Compliance: PASS (71/71 checks)
126
-
127
- | Step | Expected Pattern | Actual | Status |
128
- |------|-----------------|--------|--------|
129
- | step-01 | Auto-proceed (init) | Auto-proceed, no user menu | PASS |
130
- | step-02 | Auto-proceed | Auto-proceed, no user menu | PASS |
131
- | step-03 | Auto-proceed | Auto-proceed, no user menu | PASS |
132
- | step-04 | Auto-proceed | Auto-proceed, no user menu | PASS |
133
- | step-05 | Auto-proceed | Auto-proceed, no user menu | PASS |
134
- | step-06 | C-only (final) | `[C] Finish` with halt-and-wait | PASS |
135
-
136
- - ✅ No A/P options in any auto-proceed step
137
- - ✅ Step-06 has "halt and wait" execution rule
138
- - ✅ Step-06 redisplays menu for non-C input
139
- - ✅ Step-06 correctly omits "loading next step" universal rule
140
-
141
- ---
142
-
143
- ## Step Type Validation
144
-
145
- ### Step Type Compliance: PASS (71/71 checks)
146
-
147
- | Step | Expected Type | Match | All Sections Present | Status |
148
- |------|--------------|-------|---------------------|--------|
149
- | step-01 | Init + Input Discovery | ✅ | 6/6 | PASS |
150
- | step-02 | Middle (Simple) | ✅ | 6/6 | PASS |
151
- | step-03 | Validation Sequence | ✅ | 6/6 | PASS |
152
- | step-04 | Validation Sequence (conditional) | ✅ | 6/6 | PASS |
153
- | step-05 | Middle (Simple) | ✅ | 6/6 | PASS |
154
- | step-06 | Final Step (C-only) | ✅ | 6/6 | PASS |
155
-
156
- All steps have: STEP GOAL, MANDATORY EXECUTION RULES, EXECUTION PROTOCOLS, CONTEXT BOUNDARIES, MANDATORY SEQUENCE, SUCCESS/FAILURE METRICS.
157
-
158
- **Remediation applied:** Fixed step-06 section numbering gap (was 1,2,3,4,6,7,8 → now 1,2,3,4,5,6,7).
159
-
160
- ---
161
-
162
- ## Output Format Validation
163
-
164
- ### Template Type: PASS
165
-
166
- Template matches "structured" type with:
167
- - YAML frontmatter with defined metadata fields
168
- - Clear section headers: Test Summary, Coverage Analysis, Coherence Analysis, Completeness Score, Gap Report
169
- - Skeletal structure (headers only) — each step fills its section
170
-
171
- ### Step-to-Output Mapping: PASS
172
-
173
- | Step | Action | Output Section | outputFile in FM | Appends Before Next |
174
- |------|--------|---------------|------------------|-------------------|
175
- | 01 | Creates doc | (from template) | ✅ | ✅ |
176
- | 02 | Appends | Test Summary | ✅ | ✅ |
177
- | 03 | Appends | Coverage Analysis | ✅ | ✅ |
178
- | 04 | Appends | Coherence Analysis | ✅ | ✅ |
179
- | 05 | Appends | Completeness Score | ✅ | ✅ |
180
- | 06 | Appends | Gap Report | ✅ | ✅ (final step) |
181
-
182
- Every step has a CRITICAL STEP COMPLETION NOTE enforcing write-before-proceed.
183
-
184
- ### Final Polish Step: PASS (N/A)
185
-
186
- Not required — structured template does not need editorial polish.
187
-
188
- ---
189
-
190
- ## Validation Design Check
191
-
192
- ### Validation Steps Required? N/A — PASS
193
-
194
- test-skill IS itself a testing/validation workflow. It is the quality gate that tests skills. Adding validation steps (`steps-v/`) to a validation workflow would be circular.
195
-
196
- - ✅ No `steps-v/` folder — correct for workflow purpose
197
- - ✅ Workflow plan confirms: "Lifecycle Support: create-only" (no steps-e/ or steps-v/)
198
- - ✅ The workflow's entire purpose is testing — every step IS a validation activity
199
-
200
- ---
201
-
202
- ## Instruction Style Check
203
-
204
- ### Domain Assessment: Prescriptive (correct)
205
-
206
- test-skill is an audit/testing workflow (Ferris Audit mode) with deterministic analysis procedures, defined formulas, and exact comparison logic. Prescriptive style is appropriate.
207
-
208
- ### Per-Step Classification: PASS
209
-
210
- | Step | Classification | Appropriate | Status |
211
- |------|---------------|-------------|--------|
212
- | step-01 | Prescriptive | ✅ Yes | PASS |
213
- | step-02 | Prescriptive | ✅ Yes | PASS (1 warning) |
214
- | step-03 | Prescriptive | ✅ Yes | PASS |
215
- | step-04 | Prescriptive | ✅ Yes | PASS (1 warning) |
216
- | step-05 | Prescriptive | ✅ Yes | PASS |
217
- | step-06 | Prescriptive | ✅ Yes | PASS |
218
-
219
- **Warnings:**
220
- - ⚠️ step-02 lines 62-66: Skill type indicators mix definitive metadata checks with interpretive signals ("Self-contained API surface"). These are supplementary confirmation signals, not primary decision criteria (metadata field is checked first).
221
- - ⚠️ step-04 lines 160-165: Integration pattern completeness criteria use qualitative language ("complete flow, not fragments"). Could benefit from more concrete criteria.
222
-
223
- **Remediation applied:** Added coherence score aggregation formula to `scoring-rules.md` (was missing — agents would have had to guess how to combine reference validity and integration completeness).
224
-
225
- ---
226
-
227
- ## Collaborative Experience Check
228
-
229
- ### Interaction Pattern: Mostly Autonomous — PASS
230
-
231
- | Step | Interaction | Quality |
232
- |------|------------|---------|
233
- | step-01 | Input discovery | Natural, progressive, single question — not a laundry list |
234
- | step-02 | Autonomous | Clean auto-proceed, informative mode report |
235
- | step-03 | Autonomous | Good progress reporting with coverage metrics |
236
- | step-04 | Autonomous | Clean naive/contextual branching with proper fallbacks |
237
- | step-05 | Autonomous | Excellent mathematical transparency ("show the math") |
238
- | step-06 | Terminal menu | Clear results presentation, appropriate C-only menu |
239
-
240
- ### Workflow Progression: PASS
241
-
242
- ```
243
- Init → Classify → Analyze Coverage → Analyze Coherence → Score → Report
244
- ```
245
-
246
- Each step has explicit CONTEXT BOUNDARIES with FORBIDDEN markers preventing scope creep. Steps build logically on each other. Append-only document serves as state mechanism.
247
-
248
- ### Error Handling: Good
249
-
250
- - step-01: Four-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml). Checks `preferences.yaml` for `tier_override` after loading forge tier.
251
- - step-03/04: Subprocess fallback rules at multiple locations
252
- - step-06: Zero-gaps clean pass path
253
-
254
- **Warning:** step-03 has no explicit error handling for missing/inaccessible source files at `source_path`. Minor edge case.
255
-
256
- ### Overall Experience Quality: **Good**
257
-
258
- ---
259
-
260
- ## Subprocess Optimization Opportunities
261
-
262
- ### Usage Assessment: Good
263
-
264
- | Step | Patterns Used | Fallback | Status |
265
- |------|--------------|----------|--------|
266
- | 01 | None (not needed) | Universal rule | PASS |
267
- | 02 | None (not needed) | Universal rule | PASS |
268
- | 03 | Pattern 2 (per-file AST analysis) | 3 locations | PASS |
269
- | 04 | Pattern 1 (grep refs) + Pattern 2 (per-ref validation) | 3 locations | PASS |
270
- | 05 | None (not needed) | Universal rule | PASS |
271
- | 06 | None (not needed) | Universal rule | PASS |
272
-
273
- Subprocess optimization correctly concentrated in steps 03 and 04 (heavy analysis). Lighter steps correctly avoid unnecessary overhead. All steps have subprocess fallback rules.
274
-
275
- ### Minor Opportunities:
276
-
277
- - ⚠️ step-03 Quick tier: Could use Pattern 1 (single grep) for pre-scan before per-file analysis
278
- - ⚠️ step-04: Per-reference validations are independent — could note Pattern 4 (parallel) opportunity
279
-
280
- ### Data Files: Well-Structured
281
-
282
- - `scoring-rules.md` (72 lines): Clear tables, formulas, severity rules
283
- - `output-section-formats.md` (97 lines): Self-contained format templates per section
284
-
285
- ---
286
-
287
- ## Cohesive Review
288
-
289
- ### Overall Assessment: **Good**
290
-
291
- **Cohesiveness:** Excellent — all 6 steps build logically. Clean separation of concerns with each step having a single responsibility.
292
-
293
- **Data Flow:** Good — append-only document building with proper frontmatter state tracking. One minor gap: step 02 references `{threshold}%` in Test Summary before step 05 loads it from scoring rules.
294
-
295
- **Consistency:** Excellent — identical structure pattern across all steps (frontmatter → goal → rules → protocols → boundaries → sequence → metrics). Same role reinforcement ("Ferris Audit mode"), same emoji conventions, same subprocess fallback wording.
296
-
297
- **Goal Achievement:** Good — would produce a useful, structured test report with pass/fail determination and actionable gap remediation. Tier-aware depth scaling ensures graceful degradation.
298
-
299
- ### Strengths:
300
-
301
- 1. Rigorous traceability (zero hallucination with file:line citations)
302
- 2. Four-tier depth scaling (Quick/Forge/Forge+/Deep)
303
- 3. Clean separation of concerns with explicit FORBIDDEN markers
304
- 4. Well-designed subprocess optimization with structured return formats
305
- 5. Actionable remediation quality rules with good/bad examples
306
- 6. Data files properly externalized for reuse
307
-
308
- ### Weaknesses:
309
-
310
- 1. Weight tables duplicated between `scoring-rules.md` and step-05 inline — maintenance risk
311
- 2. Batch mode referenced in workflow plan but not implemented in step files
312
- 3. Quick tier pass may give false confidence (name-matching only)
313
-
314
- ---
315
-
316
- ## Plan Quality Validation
317
-
318
- ### Plan Completeness: PASS
319
-
320
- All 7 required sections present and substantive. Status: BUILD_COMPLETE with all 9 pipeline steps completed.
321
-
322
- ### Design-to-Implementation: PASS (1 warning fixed)
323
-
324
- - ✅ All 6 planned steps have corresponding files with matching names
325
- - ✅ Step descriptions align with plan goals
326
- - ✅ File structure matches plan (fixed: added `output-section-formats.md` to plan)
327
-
328
- ### Consistency: PASS (1 warning)
329
-
330
- - ✅ Step types match plan
331
- - ✅ Menu patterns match plan
332
- - ✅ Data flow matches plan
333
- - ⚠️ Subprocess Pattern 4 (batch parallel) attributed to step 03 in plan table but is actually a workflow-level orchestration concern
334
-
335
- ### Plan Quality: PASS
336
-
337
- - Conversion notes are insightful with genuine analysis
338
- - Classification decisions well-reasoned with structure implications
339
- - Requirements clear and complete with testable success criteria
340
- - Design specification detailed enough to rebuild the workflow
341
-
342
- ---
343
-
344
- ## Summary
345
-
346
- ### Validation Complete: 2026-02-26
347
-
348
- **Overall Status: PASS**
349
-
350
- ### Results Overview
351
-
352
- | Validation Step | Result |
353
- |----------------|--------|
354
- | File Structure & Size | PASS (after remediation) |
355
- | Frontmatter Validation | PASS |
356
- | Critical Path Violations | PASS (2 standard warnings) |
357
- | Menu Handling Validation | PASS (71/71) |
358
- | Step Type Validation | PASS (71/71) |
359
- | Output Format Validation | PASS |
360
- | Validation Design Check | PASS (N/A) |
361
- | Instruction Style Check | PASS (2 warnings) |
362
- | Collaborative Experience Check | PASS |
363
- | Subprocess Optimization | Good |
364
- | Cohesive Review | Good |
365
- | Plan Quality Validation | PASS (1 warning fixed) |
366
-
367
- ### Issues Summary
368
-
369
- **Critical Issues:** 0
370
- **Blocking Issues:** 0
371
- **Warnings (non-blocking):** 7
372
-
373
- 1. step-02: Interpretive skill type indicators alongside definitive metadata check
374
- 2. step-04: Qualitative integration completeness criteria
375
- 3. step-03: No error handling for missing source files at source_path
376
- 4. step-05: Weight tables duplicated from scoring-rules.md
377
- 5. step-02: Threshold referenced before loaded by step-05
378
- 6. step-04: Could note Pattern 4 (parallel) for independent reference validations
379
- 7. Batch mode in plan but not in step files (unimplemented feature)
380
-
381
- ### Remediation Applied During Validation
382
-
383
- 1. Created `data/output-section-formats.md` — extracted templates to reduce step-04 and step-06 sizes
384
- 2. Fixed step-06 section numbering gap (4 → 6 became 4 → 5)
385
- 3. Added coherence score aggregation formula to `scoring-rules.md`
386
- 4. Updated plan file structure to include `output-section-formats.md`
387
-
388
- ### Recommendation
389
-
390
- **Ready to use.** The test-skill workflow is well-architected with strong traceability, clean separation of concerns, and proper subprocess optimization. The 7 non-blocking warnings are minor quality improvements that can be addressed in a future iteration. The workflow will function correctly for single-skill testing in all three forge tiers.
@@ -1,292 +0,0 @@
1
- ---
2
- conversionFrom: 'src/workflows/test-skill/test-skill.spec.md'
3
- originalFormat: 'BMAD Workflow Specification (placeholder)'
4
- stepsCompleted: ['step-00-conversion', 'step-02-classification', 'step-03-requirements', 'step-04-tools', 'step-05-plan-review', 'step-06-design', 'step-07-foundation', 'step-08-build-step-01', 'step-09-build-all-steps']
5
- created: 2026-02-26
6
- status: VALIDATED
7
- approvedDate: 2026-02-26
8
- validationDate: 2026-02-26
9
- validationStatus: PASS
10
- validationReport: './validation-report.md'
11
- ---
12
-
13
- # Workflow Creation Plan
14
-
15
- ## Conversion Source
16
-
17
- **Original Path:** src/workflows/test-skill/test-skill.spec.md
18
- **Original Format:** BMAD Workflow Specification (placeholder — not yet implemented)
19
- **Detected Structure:** Single specification document defining a 6-step create-only workflow
20
-
21
- ---
22
-
23
- ## Original Workflow Analysis
24
-
25
- ### Goal (from source)
26
-
27
- Cognitive completeness verification — Verifies that a skill is complete enough to be useful to an AI agent. Supports two modes: naive (individual skill API surface coverage) and contextual (stack skill coherence validation). Produces a completeness score and gap report. Acts as a quality gate before export.
28
-
29
- ### Original Steps (Complete List)
30
-
31
- **Step 1:** load-skill - Load skill and metadata
32
- **Step 2:** detect-mode - Determine naive (individual) vs contextual (stack) testing mode
33
- **Step 3:** coverage-check - Compare documented exports against source API surface
34
- **Step 4:** coherence-check - Validate internal consistency (stack: references match SKILL.md)
35
- **Step 5:** score - Calculate completeness score
36
- **Step 6:** gap-report - Generate gap report with specific remediation suggestions
37
-
38
- ### Output / Deliverable
39
-
40
- - Completeness score (pass/fail with percentage)
41
- - Gap report with specific missing items and remediation suggestions
42
-
43
- ### Input Requirements
44
-
45
- **Required:**
46
- - Skill path (`{skills_output_folder}/{name}/`)
47
-
48
- **Optional:**
49
- - `--batch` flag with list of skills or directory
50
- - Source path override
51
-
52
- ### Key Instructions to LLM
53
-
54
- - Spec assigns **Ferris (Audit mode)** as the primary agent
55
- - Create-only mode (steps-c/ only) — no edit or validate modal variants
56
- - Document-producing workflow
57
- - Must support `--batch` for testing multiple skills
58
- - Quality gate role: determines if a skill is ready for export
59
- - Two distinct analysis modes (naive vs contextual) require branching logic
60
-
61
- ---
62
-
63
- ## Conversion Notes
64
-
65
- **What works well in original:**
66
- - Clear step decomposition with well-defined goals
67
- - Dual-mode architecture (naive/contextual) is well-scoped
68
- - Input/output requirements are explicit
69
- - Quality gate purpose is clear
70
- - Correct lifecycle positioning: test-skill sits between create-skill and export-skill (Setup → Analyze → Brief → Create → **Test** → Export)
71
- - Ferris Audit mode assignment is architecturally correct
72
-
73
- **What needs improvement:**
74
- - No step-level detail — only names and one-line goals exist
75
- - No menu handling, user interaction points, or continuation flow defined
76
- - No frontmatter configuration for steps
77
- - No data files, templates, or output format specifications
78
- - Batch mode mechanics are unspecified
79
- - No forge tier awareness — coverage depth must scale with Quick/Forge/Forge+/Deep tiers
80
- - No reference to forge-tier.yaml sidecar state for tier detection
81
- - Zero hallucination principle not encoded — scoring must trace to AST citations
82
-
83
- **Compliance gaps identified:**
84
- - Missing: workflow.md entry point with proper frontmatter
85
- - Missing: Step files with BMAD step-file architecture
86
- - Missing: Frontmatter variables for configurable paths
87
- - Missing: Menu handling and user interaction patterns
88
- - Missing: State tracking via stepsCompleted
89
- - Missing: Output document template/format specification
90
- - Missing: Step processing rules and execution protocols
91
-
92
- **Memory-informed design additions:**
93
- - Tier-aware coverage: Quick=file/structure check, Forge=AST API surface, Deep=cross-repo coherence
94
- - Batch mode: --batch with --continue for sidecar-checkpointed resume (matches create-skill pattern)
95
- - Batch produces per-skill reports with optional consolidated summary
96
- - Configurable pass/fail threshold (default overridable)
97
- - Must read forge-tier.yaml from forger-sidecar to determine available analysis depth
98
-
99
- ---
100
-
101
- ## Classification Decisions
102
-
103
- **Workflow Name:** test-skill
104
- **Target Path:** {project-root}/_bmad/skf/workflows/test-skill
105
-
106
- **4 Key Decisions:**
107
- 1. **Document Output:** true (completeness score + gap report)
108
- 2. **Module Affiliation:** SKF (Skill Forge)
109
- 3. **Session Type:** single-session
110
- 4. **Lifecycle Support:** create-only
111
-
112
- **Structure Implications:**
113
- - Only `steps-c/` folder needed (no steps-e/ or steps-v/)
114
- - Standard init (no step-01b-continue.md needed)
115
- - Free-form output template for test report document
116
- - Access to SKF module variables (forge tier config, skills paths)
117
- - Batch mode is a loop wrapper, not a multi-session concern
118
-
119
- ---
120
-
121
- ## Requirements
122
-
123
- **Flow Structure:**
124
- - Pattern: branching (by naive/contextual mode at step 02)
125
- - Phases: Loading → Analysis → Scoring → Reporting
126
- - Estimated steps: 6
127
- - Branch point: Step 02 detect-mode determines naive vs contextual path
128
- - Step 04 (coherence-check) is conditional — full execution in contextual mode, simplified/skipped in naive mode
129
-
130
- **User Interaction:**
131
- - Style: mostly autonomous (Ferris Audit mode — deterministic analysis)
132
- - Decision points: none mid-flow; mode detection is automatic based on skill type
133
- - Checkpoint frequency: entry (input validation) and exit (report delivery) only
134
-
135
- **Inputs Required:**
136
- - Required: skill path (`{skills_output_folder}/{name}/`), forge tier state (`forger-sidecar/forge-tier.yaml`)
137
- - Optional: `--batch` flag with skill list or directory, `--continue` for interrupted batch resume, source path override, custom pass/fail threshold
138
- - Prerequisites: setup-forge completed (tier detected), skill created via create-skill (SKILL.md exists)
139
-
140
- **Output Specifications:**
141
- - Type: document (structured test report)
142
- - Format: structured (defined sections for machine-readability and batch comparison)
143
- - Sections: Test Summary, Coverage Analysis, Coherence Analysis (contextual only), Completeness Score, Gap Report, Metadata
144
- - Frequency: per-skill report; batch produces per-skill reports + optional consolidated summary
145
-
146
- **Success Criteria:**
147
- - Completeness score is accurate and traceable to code via AST citations (zero hallucination)
148
- - Every gap has specific, actionable remediation suggestion
149
- - Pass/fail threshold respected and clearly reported
150
- - Naive mode correctly limits scope to API surface coverage
151
- - Contextual mode validates cross-reference coherence for stack skills
152
- - Batch mode produces consistent per-skill reports
153
- - Report format parseable for downstream automation (export-skill can read it)
154
-
155
- **Instruction Style:**
156
- - Overall: prescriptive (audit/testing — exact procedures per check)
157
- - Notes: Ferris Audit mode operates methodically; coverage check has specific comparison logic, coherence check has specific validation rules, scoring has defined formula
158
-
159
- ---
160
-
161
- ## Tools Configuration
162
-
163
- **Core BMAD Tools:**
164
- - **Party Mode:** excluded — deterministic audit, no creative exploration
165
- - **Advanced Elicitation:** excluded — no facilitative questioning needed
166
- - **Brainstorming:** excluded — no idea generation in testing/scoring workflow
167
-
168
- **LLM Features:**
169
- - **Web-Browsing:** excluded — all analysis is local (source code + SKILL.md)
170
- - **File I/O:** included — read SKILL.md, metadata.json, forge-tier.yaml, source files; write test report
171
- - **Sub-Agents:** included — coverage-check and coherence-check as parallel sub-agents (batch efficiency)
172
- - **Sub-Processes:** included — batch mode parallel skill testing
173
-
174
- **Memory:**
175
- - Type: single-session (no continuation tracking)
176
- - Sidecar: read-only access to forge-tier.yaml for tier detection; batch checkpointing for --continue
177
-
178
- **External Integrations:**
179
- - **ast-grep:** required for Forge/Deep tier — AST-backed API surface comparison in coverage-check
180
- - **QMD:** optional for Deep tier — cross-reference knowledge search for coherence enrichment
181
- - **gh CLI:** optional for Deep tier — cross-repo reference verification
182
-
183
- **Installation Requirements:**
184
- - All external tools managed by forge tier system (setup-forge detects availability)
185
- - No additional installation — test-skill gracefully degrades based on detected tier
186
-
187
- **Structure Preview:**
188
- - Phase 1: Initialization — validate skill path, read forge tier, load metadata
189
- - Phase 2: Mode Detection — determine naive vs contextual based on skill type
190
- - Phase 3: Analysis — coverage-check (all tiers) + coherence-check (contextual only), depth scales with tier
191
- - Phase 4: Scoring & Reporting — calculate score, apply threshold, generate gap report, write test report
192
-
193
- ---
194
-
195
- ## Workflow Design
196
-
197
- ### Step Sequence
198
-
199
- | # | Name | Type | Menu | Goal |
200
- |---|------|------|------|------|
201
- | 01 | init | Init with Input Discovery | Auto-proceed | Discover skill, load forge tier, validate inputs |
202
- | 02 | detect-mode | Middle (Simple) | Auto-proceed | Determine naive vs contextual from skill metadata |
203
- | 03 | coverage-check | Validation Sequence | Auto-proceed | Compare documented exports against source API surface |
204
- | 04 | coherence-check | Validation Sequence (conditional) | Auto-proceed | Validate references/coherence (contextual=full, naive=simplified) |
205
- | 04b | external-validators | Middle (Simple) | Auto-proceed | Run external validation tools (skill-check, tessl) and capture scores |
206
- | 05 | score | Middle (Simple) | Auto-proceed | Calculate completeness score from findings |
207
- | 06 | report | Final Step | C only | Generate gap report, finalize document, recommend next workflow |
208
-
209
- ### Interaction Flow
210
-
211
- ```
212
- User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
213
- [03] coverage-check (auto) → [04] coherence-check (auto) →
214
- [04b] external-validators (auto) → [05] score (auto) →
215
- [06] report → [C] finalize
216
- ```
217
-
218
- ### Data Flow
219
-
220
- - Step 01: loads SKILL.md, metadata.json, forge-tier.yaml → creates output from template
221
- - Step 02: reads metadata type → sets {testMode} naive|contextual → appends Test Summary
222
- - Step 03: reads SKILL.md exports + source files → subprocess AST analysis → appends Coverage Analysis
223
- - Step 04: reads SKILL.md references → conditional depth by mode → appends Coherence Analysis
224
- - Step 04b: runs skill-check and tessl against skill directory → captures external scores → appends External Validation
225
- - Step 05: reads coverage + coherence + external validation findings → calculates score → appends Completeness Score
226
- - Step 06: reads all findings → generates remediation per gap → appends Gap Report → finalizes
227
-
228
- ### Subprocess Optimization
229
-
230
- | Step | Pattern | What | Returns |
231
- |------|---------|------|---------|
232
- | 03 | Pattern 2 (deep analysis) | Per-source-file AST export comparison | {file, exports_found, exports_documented, missing, extra} |
233
- | 03 | Pattern 4 (parallel) | Batch mode: parallel per-skill testing | Per-skill coverage findings |
234
- | 04 | Pattern 1 (grep/regex) | Grep skill files for references | {references_found, files_checked, broken_refs} |
235
- | 04 | Pattern 2 (deep analysis) | Per-reference coherence validation | {ref, target_exists, type_matches, signature_matches} |
236
-
237
- Fallback: All subprocesses have main-thread fallback.
238
-
239
- ### File Structure
240
-
241
- ```
242
- test-skill/
243
- ├── workflow.md
244
- ├── data/
245
- │ ├── scoring-rules.md
246
- │ └── output-section-formats.md
247
- ├── steps-c/
248
- │ ├── step-01-init.md
249
- │ ├── step-02-detect-mode.md
250
- │ ├── step-03-coverage-check.md
251
- │ ├── step-04-coherence-check.md
252
- │ ├── step-04b-external-validators.md
253
- │ ├── step-05-score.md
254
- │ └── step-06-report.md
255
- └── templates/
256
- └── test-report-template.md
257
- ```
258
-
259
- ### Role & Persona
260
-
261
- Ferris (Audit mode): methodical, precise, evidence-based. Zero hallucination — every finding cites AST evidence with file:line references. Tier-aware — states analysis depth in report. Non-judgmental — reports facts, suggests remediation.
262
-
263
- ### Workflow Chaining
264
-
265
- - **Previous:** create-skill (required — SKILL.md must exist)
266
- - **Next:** export-skill (if pass) or update-skill (if fail)
267
- - **Input contract:** `{skills_output_folder}/{name}/SKILL.md` + `metadata.json` + `{sidecar_path}/forge-tier.yaml`
268
- - **Output contract:** `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md` with frontmatter: workflowType, testResult, score, nextWorkflow
269
-
270
- ### Output Template (Structured)
271
-
272
- ```markdown
273
- ---
274
- workflowType: 'test-skill'
275
- skillName: '{skill_name}'
276
- testMode: '{naive|contextual}'
277
- forgeTier: '{Quick|Forge|Forge+|Deep}'
278
- testResult: '{pass|fail}'
279
- score: '{N%}'
280
- threshold: '{N%}'
281
- testDate: '{date}'
282
- nextWorkflow: '{export-skill|update-skill}'
283
- ---
284
-
285
- # Test Report: {skill_name}
286
-
287
- ## Test Summary
288
- ## Coverage Analysis
289
- ## Coherence Analysis
290
- ## Completeness Score
291
- ## Gap Report
292
- ```