bmad-module-skill-forge 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +73 -159
- package/docs/agents.md +2 -2
- package/docs/architecture.md +155 -25
- package/docs/concepts.md +96 -0
- package/docs/examples.md +52 -2
- package/docs/getting-started.md +30 -14
- package/docs/index.md +16 -41
- package/docs/workflows.md +3 -3
- package/package.json +6 -4
- package/src/forger/forge-tier.yaml +1 -1
- package/src/forger/preferences.yaml +8 -1
- package/src/knowledge/doc-fetcher.md +55 -0
- package/src/knowledge/overview.md +17 -16
- package/src/knowledge/progressive-capability.md +9 -1
- package/src/knowledge/provenance-tracking.md +31 -23
- package/src/knowledge/qmd-registry.md +132 -0
- package/src/knowledge/skf-knowledge-index.csv +2 -0
- package/src/module.yaml +2 -2
- package/src/workflows/analyze-source/data/skill-brief-schema.md +24 -4
- package/src/workflows/analyze-source/steps-c/step-01-init.md +19 -9
- package/src/workflows/analyze-source/steps-c/step-01b-continue.md +2 -2
- package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +10 -2
- package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +17 -2
- package/src/workflows/analyze-source/steps-c/step-05-recommend.md +2 -1
- package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +3 -3
- package/src/workflows/analyze-source/templates/analysis-report-template.md +1 -1
- package/src/workflows/analyze-source/validation-report.md +1 -1
- package/src/workflows/analyze-source/workflow-plan-analyze-source.md +1 -1
- package/src/workflows/analyze-source/workflow.md +1 -1
- package/src/workflows/audit-skill/steps-c/step-01-init.md +8 -5
- package/src/workflows/audit-skill/steps-c/step-02-re-index.md +12 -2
- package/src/workflows/audit-skill/validation-report.md +2 -2
- package/src/workflows/audit-skill/workflow-plan-audit-skill.md +1 -1
- package/src/workflows/brief-skill/data/scope-templates.md +60 -0
- package/src/workflows/brief-skill/data/skill-brief-schema.md +38 -3
- package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +20 -4
- package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +20 -64
- package/src/workflows/brief-skill/steps-c/step-04-confirm-brief.md +1 -1
- package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +60 -5
- package/src/workflows/brief-skill/workflow.md +2 -1
- package/src/workflows/create-skill/data/compile-assembly-rules.md +110 -0
- package/src/workflows/create-skill/data/extraction-patterns.md +216 -3
- package/src/workflows/create-skill/data/skill-sections.md +81 -27
- package/src/workflows/create-skill/data/source-resolution-protocols.md +138 -0
- package/src/workflows/create-skill/data/tier-degradation-rules.md +46 -0
- package/src/workflows/create-skill/steps-c/step-01-load-brief.md +7 -4
- package/src/workflows/create-skill/steps-c/step-02-ecosystem-check.md +10 -7
- package/src/workflows/create-skill/steps-c/step-03-extract.md +51 -10
- package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +238 -0
- package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +220 -0
- package/src/workflows/create-skill/steps-c/step-04-enrich.md +40 -10
- package/src/workflows/create-skill/steps-c/step-05-compile.md +49 -82
- package/src/workflows/create-skill/steps-c/step-06-validate.md +152 -63
- package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +53 -3
- package/src/workflows/create-skill/steps-c/step-08-report.md +1 -1
- package/src/workflows/create-skill/workflow-plan-create-skill.md +42 -10
- package/src/workflows/create-skill/workflow.md +3 -2
- package/src/workflows/create-stack-skill/data/stack-skill-template.md +20 -7
- package/src/workflows/create-stack-skill/steps-c/step-01-init.md +2 -2
- package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +14 -0
- package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +28 -7
- package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +10 -7
- package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +66 -68
- package/src/workflows/create-stack-skill/validation-report.md +1 -1
- package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +4 -4
- package/src/workflows/export-skill/data/managed-section-format.md +1 -0
- package/src/workflows/export-skill/data/snippet-format.md +33 -14
- package/src/workflows/export-skill/steps-c/step-01-load-skill.md +16 -2
- package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +29 -22
- package/src/workflows/export-skill/steps-c/step-04-update-context.md +1 -1
- package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
- package/src/workflows/export-skill/validation-report.md +1 -1
- package/src/workflows/export-skill/workflow-plan-export-skill.md +8 -8
- package/src/workflows/export-skill/workflow.md +1 -1
- package/src/workflows/quick-skill/data/skill-template.md +27 -7
- package/src/workflows/quick-skill/steps-c/step-04-compile.md +44 -13
- package/src/workflows/quick-skill/steps-c/step-05-validate.md +79 -16
- package/src/workflows/quick-skill/validation-report.md +4 -4
- package/src/workflows/quick-skill/workflow-plan-quick-skill.md +4 -4
- package/src/workflows/quick-skill/workflow.md +2 -0
- package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +12 -3
- package/src/workflows/setup-forge/steps-c/step-02-write-config.md +10 -2
- package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +111 -40
- package/src/workflows/setup-forge/steps-c/step-04-report.md +13 -0
- package/src/workflows/setup-forge/workflow.md +1 -0
- package/src/workflows/test-skill/data/scoring-rules.md +14 -6
- package/src/workflows/test-skill/data/source-access-protocol.md +45 -0
- package/src/workflows/test-skill/steps-c/step-01-init.md +38 -13
- package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -3
- package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +25 -5
- package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +22 -47
- package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +219 -0
- package/src/workflows/test-skill/steps-c/step-05-score.md +25 -27
- package/src/workflows/test-skill/steps-c/step-06-report.md +9 -4
- package/src/workflows/test-skill/templates/test-report-template.md +2 -1
- package/src/workflows/test-skill/validation-report.md +1 -1
- package/src/workflows/test-skill/workflow-plan-test-skill.md +7 -3
- package/src/workflows/test-skill/workflow.md +1 -1
- package/src/workflows/update-skill/data/remote-source-resolution.md +43 -0
- package/src/workflows/update-skill/data/tier-degradation-rules.md +46 -0
- package/src/workflows/update-skill/steps-c/step-01-init.md +13 -7
- package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +26 -0
- package/src/workflows/update-skill/steps-c/step-03-re-extract.md +38 -9
- package/src/workflows/update-skill/steps-c/step-04-merge.md +3 -3
- package/src/workflows/update-skill/steps-c/step-05-validate.md +63 -67
- package/src/workflows/update-skill/steps-c/step-06-write.md +37 -14
- package/src/workflows/update-skill/steps-c/step-07-report.md +1 -1
- package/src/workflows/update-skill/validation-report.md +4 -4
- package/src/workflows/update-skill/workflow-plan-update-skill.md +4 -4
- package/src/workflows/update-skill/workflow.md +1 -1
- package/tools/cli/commands/status.js +4 -4
- package/tools/cli/commands/uninstall.js +1 -1
- package/tools/cli/commands/update.js +2 -2
- package/tools/cli/lib/ui.js +60 -16
- package/tools/cli/lib/version-check.js +2 -2
|
@@ -3,15 +3,16 @@ name: 'step-05-score'
|
|
|
3
3
|
description: 'Calculate completeness score from coverage and coherence findings'
|
|
4
4
|
|
|
5
5
|
nextStepFile: './step-06-report.md'
|
|
6
|
-
outputFile: '{
|
|
6
|
+
outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
|
|
7
7
|
scoringRulesFile: '../data/scoring-rules.md'
|
|
8
|
+
sourceAccessProtocol: '../data/{sourceAccessProtocol}'
|
|
8
9
|
---
|
|
9
10
|
|
|
10
11
|
# Step 5: Score
|
|
11
12
|
|
|
12
13
|
## STEP GOAL:
|
|
13
14
|
|
|
14
|
-
Calculate the overall completeness score by aggregating coverage and
|
|
15
|
+
Calculate the overall completeness score by aggregating coverage, coherence, and external validation category scores with the appropriate weight distribution (naive or contextual), apply the pass/fail threshold, and determine the test result.
|
|
15
16
|
|
|
16
17
|
## MANDATORY EXECUTION RULES (READ FIRST):
|
|
17
18
|
|
|
@@ -46,10 +47,10 @@ Calculate the overall completeness score by aggregating coverage and coherence c
|
|
|
46
47
|
|
|
47
48
|
## CONTEXT BOUNDARIES:
|
|
48
49
|
|
|
49
|
-
- Available: Coverage Analysis (step 03)
|
|
50
|
+
- Available: Coverage Analysis (step 03), Coherence Analysis (step 04), and External Validation (step 04b) in {outputFile}
|
|
50
51
|
- Focus: Score calculation and pass/fail determination only
|
|
51
52
|
- Limits: Do NOT generate gap remediation — that's step 06
|
|
52
|
-
- Dependencies: steps 03 and
|
|
53
|
+
- Dependencies: steps 03, 04, and 04b must have appended their analysis sections
|
|
53
54
|
|
|
54
55
|
## MANDATORY SEQUENCE
|
|
55
56
|
|
|
@@ -76,33 +77,15 @@ Read `{outputFile}` and extract the category scores calculated in previous steps
|
|
|
76
77
|
- Combined Coherence: {percentage}% (contextual mode only)
|
|
77
78
|
- Or: not scored (naive mode — weight redistributed)
|
|
78
79
|
|
|
80
|
+
**From External Validation (step 04b):**
|
|
81
|
+
- External Validation Score: {percentage}% (combined skill-check + tessl average)
|
|
82
|
+
- Or: N/A (if neither tool was available — weight redistributed to other categories)
|
|
83
|
+
|
|
79
84
|
### 3. Apply Weight Distribution
|
|
80
85
|
|
|
81
86
|
**Read testMode from {outputFile} frontmatter.**
|
|
82
87
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
| Category | Score | Weight | Weighted |
|
|
86
|
-
|----------|-------|--------|----------|
|
|
87
|
-
| Export Coverage | {N}% | 50% | {N * 0.50}% |
|
|
88
|
-
| Signature Accuracy | {N}% | 30% | {N * 0.30}% |
|
|
89
|
-
| Type Coverage | {N}% | 20% | {N * 0.20}% |
|
|
90
|
-
| **Total** | | **100%** | **{sum}%** |
|
|
91
|
-
|
|
92
|
-
**IF contextual mode — use full weights:**
|
|
93
|
-
|
|
94
|
-
| Category | Score | Weight | Weighted |
|
|
95
|
-
|----------|-------|--------|----------|
|
|
96
|
-
| Export Coverage | {N}% | 40% | {N * 0.40}% |
|
|
97
|
-
| Signature Accuracy | {N}% | 25% | {N * 0.25}% |
|
|
98
|
-
| Type Coverage | {N}% | 15% | {N * 0.15}% |
|
|
99
|
-
| Coherence | {N}% | 20% | {N * 0.20}% |
|
|
100
|
-
| **Total** | | **100%** | **{sum}%** |
|
|
101
|
-
|
|
102
|
-
**Quick tier adjustment:**
|
|
103
|
-
If Signature Accuracy and Type Coverage are N/A (Quick tier, no AST):
|
|
104
|
-
- Naive: Export Coverage gets 100% weight
|
|
105
|
-
- Contextual: Export Coverage 60%, Coherence 40%
|
|
88
|
+
Apply the weight distribution from `{scoringRulesFile}` for the detected mode (naive or contextual). The scoring rules define category weights, external validation redistribution when unavailable, and Quick tier adjustments. Calculate the weighted score for each category and sum for the total.
|
|
106
89
|
|
|
107
90
|
### 4. Determine Pass/Fail
|
|
108
91
|
|
|
@@ -139,6 +122,7 @@ Append the **Completeness Score** section to `{outputFile}`:
|
|
|
139
122
|
| Signature Accuracy | {N}% | {W}% | {WS}% |
|
|
140
123
|
| Type Coverage | {N}% | {W}% | {WS}% |
|
|
141
124
|
| Coherence | {N}% | {W}% | {WS}% |
|
|
125
|
+
| External Validation | {N}% | {W}% | {WS}% |
|
|
142
126
|
| **Total** | | **100%** | **{total}%** |
|
|
143
127
|
|
|
144
128
|
### Result
|
|
@@ -149,6 +133,18 @@ Append the **Completeness Score** section to `{outputFile}`:
|
|
|
149
133
|
|
|
150
134
|
**Weight Distribution:** {naive (redistributed) | contextual (full)}
|
|
151
135
|
**Tier Adjustment:** {none | Quick tier — signature and type coverage not scored}
|
|
136
|
+
**External Validators:** {both available | skill-check only | tessl only | none — weight redistributed}
|
|
137
|
+
**Analysis Confidence:** {full | provenance-map | metadata-only | remote-only | docs-only}
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
If `analysis_confidence` is not `full`, append a degradation notice. **The notice must be confidence-aware** — see the degradation notice rules in `{sourceAccessProtocol}`:
|
|
141
|
+
|
|
142
|
+
```markdown
|
|
143
|
+
### Access Degradation Notice
|
|
144
|
+
|
|
145
|
+
**Resolved via:** {analysis_confidence} {confidence breakdown if provenance-map, e.g., "(T1 AST-verified at compilation time)" or "(12 T1, 3 T1-low)"}
|
|
146
|
+
**Impact:** {describe limitation — e.g., "Signature checks limited to name-matching. Source file:line citations from provenance-map, not live AST." — or "Provenance data is at highest confidence; no limitation." for all-T1 provenance-map}
|
|
147
|
+
**Recommendation:** {confidence-dependent — see {sourceAccessProtocol} degradation notice rules. Do NOT recommend local clone when provenance-map entries are already T1.}
|
|
152
148
|
```
|
|
153
149
|
|
|
154
150
|
### 7. Update Output Frontmatter
|
|
@@ -157,6 +153,7 @@ Update `{outputFile}` frontmatter:
|
|
|
157
153
|
- `testResult: '{pass|fail}'`
|
|
158
154
|
- `score: '{total}%'`
|
|
159
155
|
- `threshold: '{threshold}%'`
|
|
156
|
+
- `analysisConfidence: '{analysis_confidence}'`
|
|
160
157
|
- `nextWorkflow: '{export-skill|update-skill}'`
|
|
161
158
|
- Append `'step-05-score'` to `stepsCompleted`
|
|
162
159
|
|
|
@@ -172,6 +169,7 @@ Update `{outputFile}` frontmatter:
|
|
|
172
169
|
| Signature Accuracy | {N}% | {WS}% |
|
|
173
170
|
| Type Coverage | {N}% | {WS}% |
|
|
174
171
|
| Coherence | {N}% | {WS}% |
|
|
172
|
+
| External Validation | {N}% | {WS}% |
|
|
175
173
|
|
|
176
174
|
**Threshold:** {threshold}%
|
|
177
175
|
**Recommendation:** {export-skill if pass | update-skill if fail}
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: 'step-06-report'
|
|
3
3
|
description: 'Generate gap report with remediation suggestions and finalize test report'
|
|
4
4
|
|
|
5
|
-
outputFile: '{
|
|
5
|
+
outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
|
|
6
6
|
scoringRulesFile: '../data/scoring-rules.md'
|
|
7
7
|
outputFormatsFile: '../data/output-section-formats.md'
|
|
8
8
|
---
|
|
@@ -17,7 +17,7 @@ Generate a detailed gap report listing every issue found during coverage and coh
|
|
|
17
17
|
|
|
18
18
|
### Universal Rules:
|
|
19
19
|
|
|
20
|
-
- 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03 and
|
|
20
|
+
- 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03, 04, and 04b
|
|
21
21
|
- 📖 CRITICAL: Read the complete step file before taking any action
|
|
22
22
|
- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
|
|
23
23
|
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
|
@@ -69,6 +69,10 @@ Read `{outputFile}` and extract every issue found across all analysis sections:
|
|
|
69
69
|
- Incomplete integration patterns (contextual mode)
|
|
70
70
|
- Structural issues (naive mode — missing sections, broken examples)
|
|
71
71
|
|
|
72
|
+
**From External Validation (step 04b):**
|
|
73
|
+
- skill-check diagnostics (unresolved errors and warnings)
|
|
74
|
+
- tessl judge suggestions (content quality and actionability improvements)
|
|
75
|
+
|
|
72
76
|
### 2. Load Severity Rules
|
|
73
77
|
|
|
74
78
|
Load `{scoringRulesFile}` for gap severity classification:
|
|
@@ -78,6 +82,7 @@ Load `{scoringRulesFile}` for gap severity classification:
|
|
|
78
82
|
| **Critical** | Missing exported function/class documentation |
|
|
79
83
|
| **High** | Signature mismatch between source and SKILL.md |
|
|
80
84
|
| **Medium** | Missing type or interface documentation |
|
|
85
|
+
| **Medium** | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
|
|
81
86
|
| **Low** | Missing optional metadata or examples |
|
|
82
87
|
| **Info** | Style suggestions, non-blocking observations |
|
|
83
88
|
|
|
@@ -96,7 +101,7 @@ If no gaps found, append a clean pass message recommending **export-skill** work
|
|
|
96
101
|
### 5. Finalize Output Document
|
|
97
102
|
|
|
98
103
|
Update `{outputFile}` frontmatter:
|
|
99
|
-
- Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-05-score', 'step-06-report']`
|
|
104
|
+
- Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-04b-external-validators', 'step-05-score', 'step-06-report']`
|
|
100
105
|
|
|
101
106
|
### 6. Present Final Report
|
|
102
107
|
|
|
@@ -154,7 +159,7 @@ This is the final step of the test-skill workflow. When the user selects C, the
|
|
|
154
159
|
|
|
155
160
|
### ✅ SUCCESS:
|
|
156
161
|
|
|
157
|
-
- Every gap traces to a finding from steps 03 or
|
|
162
|
+
- Every gap traces to a finding from steps 03, 04, or 04b (zero fabrication)
|
|
158
163
|
- Gaps classified by severity using scoring rules
|
|
159
164
|
- Gaps ordered by severity (Critical first)
|
|
160
165
|
- Every gap has a specific, actionable remediation suggestion
|
|
@@ -247,7 +247,7 @@ Each step has explicit CONTEXT BOUNDARIES with FORBIDDEN markers preventing scop
|
|
|
247
247
|
|
|
248
248
|
### Error Handling: Good
|
|
249
249
|
|
|
250
|
-
- step-01: Three-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml)
|
|
250
|
+
- step-01: Three-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml). Checks `preferences.yaml` for `tier_override` after loading forge tier.
|
|
251
251
|
- step-03/04: Subprocess fallback rules at multiple locations
|
|
252
252
|
- step-06: Zero-gaps clean pass path
|
|
253
253
|
|
|
@@ -202,6 +202,7 @@ Cognitive completeness verification — Verifies that a skill is complete enough
|
|
|
202
202
|
| 02 | detect-mode | Middle (Simple) | Auto-proceed | Determine naive vs contextual from skill metadata |
|
|
203
203
|
| 03 | coverage-check | Validation Sequence | Auto-proceed | Compare documented exports against source API surface |
|
|
204
204
|
| 04 | coherence-check | Validation Sequence (conditional) | Auto-proceed | Validate references/coherence (contextual=full, naive=simplified) |
|
|
205
|
+
| 04b | external-validators | Middle (Simple) | Auto-proceed | Run external validation tools (skill-check, tessl) and capture scores |
|
|
205
206
|
| 05 | score | Middle (Simple) | Auto-proceed | Calculate completeness score from findings |
|
|
206
207
|
| 06 | report | Final Step | C only | Generate gap report, finalize document, recommend next workflow |
|
|
207
208
|
|
|
@@ -210,7 +211,8 @@ Cognitive completeness verification — Verifies that a skill is complete enough
|
|
|
210
211
|
```
|
|
211
212
|
User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
|
|
212
213
|
[03] coverage-check (auto) → [04] coherence-check (auto) →
|
|
213
|
-
[
|
|
214
|
+
[04b] external-validators (auto) → [05] score (auto) →
|
|
215
|
+
[06] report → [C] finalize
|
|
214
216
|
```
|
|
215
217
|
|
|
216
218
|
### Data Flow
|
|
@@ -219,7 +221,8 @@ User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
|
|
|
219
221
|
- Step 02: reads metadata type → sets {testMode} naive|contextual → appends Test Summary
|
|
220
222
|
- Step 03: reads SKILL.md exports + source files → subprocess AST analysis → appends Coverage Analysis
|
|
221
223
|
- Step 04: reads SKILL.md references → conditional depth by mode → appends Coherence Analysis
|
|
222
|
-
- Step
|
|
224
|
+
- Step 04b: runs skill-check and tessl against skill directory → captures external scores → appends External Validation
|
|
225
|
+
- Step 05: reads coverage + coherence + external validation findings → calculates score → appends Completeness Score
|
|
223
226
|
- Step 06: reads all findings → generates remediation per gap → appends Gap Report → finalizes
|
|
224
227
|
|
|
225
228
|
### Subprocess Optimization
|
|
@@ -246,6 +249,7 @@ test-skill/
|
|
|
246
249
|
│ ├── step-02-detect-mode.md
|
|
247
250
|
│ ├── step-03-coverage-check.md
|
|
248
251
|
│ ├── step-04-coherence-check.md
|
|
252
|
+
│ ├── step-04b-external-validators.md
|
|
249
253
|
│ ├── step-05-score.md
|
|
250
254
|
│ └── step-06-report.md
|
|
251
255
|
└── templates/
|
|
@@ -261,7 +265,7 @@ Ferris (Audit mode): methodical, precise, evidence-based. Zero hallucination —
|
|
|
261
265
|
- **Previous:** create-skill (required — SKILL.md must exist)
|
|
262
266
|
- **Next:** export-skill (if pass) or update-skill (if fail)
|
|
263
267
|
- **Input contract:** `{skills_output_folder}/{name}/SKILL.md` + `metadata.json` + `{sidecar_path}/forge-tier.yaml`
|
|
264
|
-
- **Output contract:** `{
|
|
268
|
+
- **Output contract:** `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md` with frontmatter: workflowType, testResult, score, nextWorkflow
|
|
265
269
|
|
|
266
270
|
### Output Template (Structured)
|
|
267
271
|
|
|
@@ -49,7 +49,7 @@ installed_path: '{project-root}/_bmad/skf/workflows/test-skill'
|
|
|
49
49
|
|
|
50
50
|
Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
|
|
51
51
|
|
|
52
|
-
- `project_name`, `
|
|
52
|
+
- `project_name`, `user_name`, `communication_language`, `document_output_language`
|
|
53
53
|
- `skills_output_folder`, `forge_data_folder`, `sidecar_path`
|
|
54
54
|
|
|
55
55
|
### 2. First Step Execution
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Remote Source Resolution (Forge/Deep Tier)
|
|
2
|
+
|
|
3
|
+
If `source_root` is a local path: proceed with the tier-appropriate strategy as normal.
|
|
4
|
+
|
|
5
|
+
If `source_root` (from metadata.json) is a remote URL (GitHub URL or owner/repo format) AND tier is Forge or Deep:
|
|
6
|
+
|
|
7
|
+
1. **Check `git` availability:** Verify `git` is functional (`git --version`). If `git` is not available, skip to the fallback warning below.
|
|
8
|
+
|
|
9
|
+
2. **Ephemeral sparse clone:** Clone only the changed files from the change manifest to a system temp path. Note: at this point in the flow, `{source_root}` is known to be a remote URL (the local-path case was already handled above).
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
temp_path = {system_temp}/skf-ephemeral-{skill-name}-{timestamp}/
|
|
13
|
+
git clone --depth 1 --single-branch --filter=blob:none --sparse {source_root} {temp_path}
|
|
14
|
+
git -C {temp_path} sparse-checkout set --skip-checks {changed_files_from_manifest}
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
**Note:** `--skip-checks` is required because `changed_files_from_manifest` contains individual file paths (e.g., `src/core/parser.py`), not directories. Without this flag, `git sparse-checkout set` rejects non-directory entries.
|
|
18
|
+
|
|
19
|
+
No `--branch` flag is used — the clone targets the remote's default branch, which must match the branch used during the original [CS] Create Skill run. This scopes the clone to only the files identified in step-02's change manifest, avoiding a full repository download.
|
|
20
|
+
|
|
21
|
+
3. **If clone succeeds:** Update the working source path to `{temp_path}` for all subsequent AST operations in this step. Proceed with the **Forge tier** extraction strategy below. Mark `ephemeral_clone_active = true` for cleanup.
|
|
22
|
+
|
|
23
|
+
4. **If clone fails (network error, auth failure, timeout):**
|
|
24
|
+
|
|
25
|
+
Warning message: "Ephemeral clone of `{source_root}` failed: {error}. Degrading to source reading (T1-low) for this run. For T1 (AST-verified) confidence, clone the repository locally and re-run [CS] Create Skill with the local path, then re-run this update."
|
|
26
|
+
|
|
27
|
+
Override the extraction strategy to Quick tier for this run. Note the degradation reason in context for the evidence report.
|
|
28
|
+
|
|
29
|
+
## Ephemeral Clone Cleanup
|
|
30
|
+
|
|
31
|
+
After extraction is complete for all files in scope (whether successful or partially failed), before presenting the extraction summary, if `ephemeral_clone_active`, delete the `{temp_path}` directory. Log: "Ephemeral source clone cleaned up." This ensures cleanup runs even if some extractions failed, as long as the step itself is still executing.
|
|
32
|
+
|
|
33
|
+
## Version Reconciliation
|
|
34
|
+
|
|
35
|
+
After the source path is accessible, check whether the source version has changed since the original skill was created. Look for the version file matching the detected language (e.g., `pyproject.toml`, `package.json`, `Cargo.toml`). If the source version differs from the current `metadata.json` version, record `source_version_detected` in context for step-06 to use when updating `metadata.json`. No warning needed here — step-06 handles the version update.
|
|
36
|
+
|
|
37
|
+
## AST Tool Unavailability (Local Source)
|
|
38
|
+
|
|
39
|
+
If AST tool is unavailable at Forge/Deep tier with local source:
|
|
40
|
+
|
|
41
|
+
Warning message: "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
|
|
42
|
+
|
|
43
|
+
Degrade to Quick tier extraction. Note the degradation reason in context for the evidence report.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Tier Degradation Rules
|
|
2
|
+
|
|
3
|
+
## Remote Source at Forge/Deep Tier
|
|
4
|
+
|
|
5
|
+
When `source_repo` is a remote URL (GitHub URL or owner/repo format) and the tier is Forge or Deep:
|
|
6
|
+
|
|
7
|
+
- **ast-grep requires local files** — it cannot operate on remote URLs
|
|
8
|
+
|
|
9
|
+
**Ephemeral clone strategy (preferred):**
|
|
10
|
+
|
|
11
|
+
1. Check `git` availability (`git --version`). `git` is effectively guaranteed at Deep tier (via `gh` dependency) but NOT guaranteed at Forge tier.
|
|
12
|
+
2. If `git` is available: perform an ephemeral shallow clone to a system temp path (`{system_temp}/skf-ephemeral-{skill-name}-{timestamp}/`).
|
|
13
|
+
3. For create-skill: use `--depth 1 --single-branch --filter=blob:none`; if `include_patterns` are specified, apply mode selection: if `exclude_patterns` are absent, use cone mode (convert include_patterns to directory roots; use `--skip-checks` for individual file paths); if `exclude_patterns` are present, use `--no-cone` mode (pass gitignore-style patterns with `!`-prefixed excludes — includes first, then negations). See source-resolution-protocols.md for the full conversion rules.
|
|
14
|
+
4. For update-skill: use sparse-checkout with `--skip-checks` scoped to the changed files from the change manifest only (file paths require `--skip-checks`). No `--branch` flag — uses the remote default branch (must match the branch used during original create-skill run).
|
|
15
|
+
5. If clone succeeds: use the local clone path for AST extraction. All results are T1 with `[AST:...]` citations.
|
|
16
|
+
6. Cleanup: delete the temp directory after extraction inventory is built and all data is in context. The clone never persists beyond the extraction step.
|
|
17
|
+
|
|
18
|
+
**Fallback (clone fails or `git` unavailable):**
|
|
19
|
+
|
|
20
|
+
- The extraction step MUST warn the user explicitly before degrading
|
|
21
|
+
- **create-skill:** Warning must include actionable guidance — clone locally and update `source_repo` in the brief to the local path
|
|
22
|
+
- **update-skill:** Warning must include actionable guidance — clone locally, re-run [CS] Create Skill with the local path to regenerate provenance data, then re-run the update
|
|
23
|
+
- Extraction proceeds using Quick tier strategy (source reading via gh_bridge)
|
|
24
|
+
- All results labeled T1-low with `[SRC:...]` citations
|
|
25
|
+
- The degradation reason is recorded in the evidence report
|
|
26
|
+
|
|
27
|
+
Silent degradation is **forbidden**. The user must always know when AST extraction was skipped and why.
|
|
28
|
+
|
|
29
|
+
## AST Tool Unavailable at Forge/Deep Tier
|
|
30
|
+
|
|
31
|
+
When the tier is Forge or Deep but ast-grep is not functional:
|
|
32
|
+
|
|
33
|
+
- The extraction step MUST warn the user explicitly before degrading
|
|
34
|
+
- Warning must include actionable guidance: run [SF] Setup Forge to detect tools
|
|
35
|
+
- Extraction proceeds using Quick tier strategy
|
|
36
|
+
- All results labeled T1-low
|
|
37
|
+
- The degradation reason is recorded in the evidence report
|
|
38
|
+
|
|
39
|
+
## Per-File AST Failure
|
|
40
|
+
|
|
41
|
+
When ast-grep fails on an individual file (parse error, unsupported syntax):
|
|
42
|
+
|
|
43
|
+
- Fall back to source reading for **that file only**
|
|
44
|
+
- Other files continue with AST extraction
|
|
45
|
+
- The affected file's results are labeled T1-low; unaffected files retain T1
|
|
46
|
+
- Log a warning noting which file degraded and why
|
|
@@ -62,11 +62,15 @@ Load the existing skill and all its provenance data, detect whether this is an i
|
|
|
62
62
|
Provide either:
|
|
63
63
|
- A skill name (resolves to `{skills_output_folder}/{name}/`)
|
|
64
64
|
- A full path to the skill folder
|
|
65
|
+
- A skill name with `--from-test-report` to use the test report's gap findings instead of source drift detection
|
|
65
66
|
|
|
66
67
|
**Skill:** {user provides path or name}"
|
|
67
68
|
|
|
68
69
|
Resolve the path to an absolute skill folder location.
|
|
69
70
|
|
|
71
|
+
**If `--from-test-report` was provided (or user references a test report):**
|
|
72
|
+
Search for the test report at `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md`. If found, set `test_report_path` in context and `update_mode: gap-driven`. If not found, warn and continue with normal source drift mode.
|
|
73
|
+
|
|
70
74
|
### 2. Validate Required Artifacts
|
|
71
75
|
|
|
72
76
|
**Check SKILL.md exists:**
|
|
@@ -75,19 +79,21 @@ Resolve the path to an absolute skill folder location.
|
|
|
75
79
|
|
|
76
80
|
**Check metadata.json exists:**
|
|
77
81
|
- Load `{resolved_skill_path}/metadata.json`
|
|
78
|
-
- Extract: `
|
|
82
|
+
- Extract: `name`, `skill_type` (single or stack), `version`, `generation_date`, `confidence_tier`, `source_root`
|
|
79
83
|
- If missing: **ABORT** — "No metadata.json found. This skill may have been created manually. Run create-skill to generate provenance data."
|
|
80
84
|
|
|
81
85
|
**Detect skill type from metadata:**
|
|
82
86
|
- If `skill_type == "stack"`: flag as stack skill (multi-file update mode)
|
|
83
|
-
- If `skill_type == "
|
|
87
|
+
- If `skill_type == "single"` or absent: flag as single skill
|
|
84
88
|
|
|
85
89
|
### 3. Load Forge Tier Configuration
|
|
86
90
|
|
|
87
|
-
**Load `
|
|
88
|
-
- Extract: `
|
|
91
|
+
**Load `{sidecar_path}/forge-tier.yaml`:**
|
|
92
|
+
- Extract: `tier` (Quick, Forge, or Deep), available tools
|
|
89
93
|
- If missing: **ABORT** — "No forge-tier.yaml found. Run setup-forge first to detect available tools."
|
|
90
94
|
|
|
95
|
+
**Apply tier override:** Read `{sidecar_path}/preferences.yaml`. If `tier_override` is set and is a valid tier value (Quick, Forge, or Deep), use it instead of the detected tier.
|
|
96
|
+
|
|
91
97
|
**Determine analysis capabilities:**
|
|
92
98
|
- **Quick:** text pattern matching only → T1-low confidence
|
|
93
99
|
- **Forge:** AST structural extraction → T1 confidence
|
|
@@ -146,7 +152,7 @@ Please provide the current source code path:
|
|
|
146
152
|
| Property | Value |
|
|
147
153
|
|----------|-------|
|
|
148
154
|
| **Skill** | {skill_name} |
|
|
149
|
-
| **Type** | {
|
|
155
|
+
| **Type** | {single/stack} |
|
|
150
156
|
| **Version** | {version} |
|
|
151
157
|
| **Created** | {created date} |
|
|
152
158
|
| **Source** | {source_root} |
|
|
@@ -154,7 +160,7 @@ Please provide the current source code path:
|
|
|
154
160
|
| **Provenance Age** | {days} days since last extraction |
|
|
155
161
|
| **Exports** | {export_count} tracked exports |
|
|
156
162
|
| **[MANUAL] Sections** | {manual_count} preserved sections |
|
|
157
|
-
| **Mode** | {normal/degraded} |
|
|
163
|
+
| **Mode** | {normal/degraded/gap-driven} |
|
|
158
164
|
|
|
159
165
|
**Analysis plan:** {tier_description}
|
|
160
166
|
- {Quick: text pattern diff → T1-low findings}
|
|
@@ -188,7 +194,7 @@ ONLY WHEN [C] is selected and baseline has been established with all required ar
|
|
|
188
194
|
### ✅ SUCCESS:
|
|
189
195
|
|
|
190
196
|
- SKILL.md loaded and validated
|
|
191
|
-
- metadata.json loaded with
|
|
197
|
+
- metadata.json loaded with name, skill_type, version, source_root
|
|
192
198
|
- Forge tier loaded from sidecar/forge-tier.yaml
|
|
193
199
|
- Provenance map loaded (or degraded mode confirmed)
|
|
194
200
|
- [MANUAL] sections inventoried across all output files
|
|
@@ -56,6 +56,32 @@ Compare current source code state against the provenance map to produce a comple
|
|
|
56
56
|
|
|
57
57
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
58
58
|
|
|
59
|
+
### 0. Check for Test Report Input (Gap-Driven Mode)
|
|
60
|
+
|
|
61
|
+
**If `update_mode == "gap-driven"` (set in step-01 via `--from-test-report`):**
|
|
62
|
+
|
|
63
|
+
Load the test report at `{test_report_path}` and extract findings:
|
|
64
|
+
|
|
65
|
+
1. Read the **Gap Report** section — each gap entry has severity, category, and description
|
|
66
|
+
2. Read the **Coverage Analysis** section — each per-export row has documented/missing/mismatch status
|
|
67
|
+
3. Translate findings into change manifest format:
|
|
68
|
+
|
|
69
|
+
| Gap Severity | Gap Type | Change Category |
|
|
70
|
+
|-------------|----------|-----------------|
|
|
71
|
+
| Critical | Missing export documentation | NEW_EXPORT (undocumented public API) |
|
|
72
|
+
| High | Signature mismatch | MODIFIED_EXPORT (signature needs update) |
|
|
73
|
+
| Medium | Missing type/interface docs | NEW_EXPORT (undocumented type) |
|
|
74
|
+
| Medium | Stale documentation | MODIFIED_EXPORT (docs reference removed export) |
|
|
75
|
+
| Low | Missing metadata/examples | metadata update |
|
|
76
|
+
|
|
77
|
+
4. Build the change manifest from translated gaps — no file-level timestamp comparison needed since source hasn't changed
|
|
78
|
+
5. Set `gap_count` from the total number of translated entries
|
|
79
|
+
6. **Skip to section 5** (Display Change Summary) with the gap-derived manifest
|
|
80
|
+
|
|
81
|
+
"**Gap-driven update mode.** Translating {gap_count} test report findings into change manifest — source drift detection skipped."
|
|
82
|
+
|
|
83
|
+
**If normal mode:** Continue with source drift detection below.
|
|
84
|
+
|
|
59
85
|
### 1. Scan Current Source State
|
|
60
86
|
|
|
61
87
|
Read the source directory at `{source_root}` and build a current file inventory:
|
|
@@ -3,6 +3,9 @@ name: 'step-03-re-extract'
|
|
|
3
3
|
description: 'Tier-aware AST extraction on changed files only, producing fresh exports with confidence tiers'
|
|
4
4
|
|
|
5
5
|
nextStepFile: './step-04-merge.md'
|
|
6
|
+
extractionPatternsData: '../../create-skill/data/extraction-patterns.md'
|
|
7
|
+
tierDegradationRulesData: '../data/tier-degradation-rules.md'
|
|
8
|
+
remoteSourceResolutionData: '../data/remote-source-resolution.md'
|
|
6
9
|
---
|
|
7
10
|
|
|
8
11
|
# Step 3: Re-Extract Changed Exports
|
|
@@ -55,7 +58,26 @@ Perform tier-aware extraction on only the changed files identified in step 02, p
|
|
|
55
58
|
|
|
56
59
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
57
60
|
|
|
58
|
-
### 1.
|
|
61
|
+
### 1. Check for Docs-Only Mode
|
|
62
|
+
|
|
63
|
+
**If `source_type: "docs-only"` in the original brief or metadata:**
|
|
64
|
+
|
|
65
|
+
"**Docs-only skill detected.** This skill was generated from external documentation, not source code. Re-extraction will re-fetch the original `doc_urls` to check for updated content."
|
|
66
|
+
|
|
67
|
+
- Re-fetch each URL from `doc_urls` (from the brief or metadata) using whatever web fetching capability is available
|
|
68
|
+
- Extract updated API information with T3 `[EXT:{url}]` citations
|
|
69
|
+
- Build the updated extraction inventory from fetched content
|
|
70
|
+
- Skip all source code extraction below — proceed directly to the merge step (section 5 or equivalent)
|
|
71
|
+
|
|
72
|
+
**If `source_type: "source"` (default):** Continue with source extraction below.
|
|
73
|
+
|
|
74
|
+
### 1b. Determine Extraction Strategy by Tier
|
|
75
|
+
|
|
76
|
+
**Remote Source Resolution (Forge/Deep only):**
|
|
77
|
+
|
|
78
|
+
**MCP source access check (before ephemeral clone):** If MCP source-reading tools are available (zread, deepwiki, gh API, or similar) and `source_repo` is set in metadata.json, use MCP tools to fetch only the changed files from the change manifest. This avoids ephemeral clone overhead entirely. MCP provides full source file content equivalent to a local read. If the fetched content is written to a temp file and analyzed with ast-grep, label confidence as T1. If AST is unavailable (the common case for MCP-fetched content), use pattern-based extraction and label confidence as T1-low.
|
|
79
|
+
|
|
80
|
+
**If MCP unavailable:** Load and follow `{remoteSourceResolutionData}` for ephemeral clone setup, version reconciliation, and AST tool unavailability handling.
|
|
59
81
|
|
|
60
82
|
**Quick tier (text pattern matching):**
|
|
61
83
|
- Extract function/class/type names via regex patterns
|
|
@@ -63,7 +85,9 @@ Perform tier-aware extraction on only the changed files identified in step 02, p
|
|
|
63
85
|
- Confidence: T1-low (pattern-matched, not AST-verified)
|
|
64
86
|
|
|
65
87
|
**Forge tier (AST structural extraction):**
|
|
66
|
-
|
|
88
|
+
|
|
89
|
+
⚠️ **CRITICAL:** Load and follow the **AST Extraction Protocol** from `{extractionPatternsData}`. Use the decision tree based on the number of changed files: prefer MCP `find_code()` for small sets, `find_code_by_rule()` with scoped YAML rules for medium sets, and CLI `--json=stream` with line-by-line streaming for large sets. Never use `ast-grep --json` (without `=stream`) — it loads the entire result set into memory and will fail on large codebases.
|
|
90
|
+
|
|
67
91
|
- Extract: function signatures, type definitions, class members, exported constants
|
|
68
92
|
- Extract: parameter types, return types, JSDoc/docstring comments
|
|
69
93
|
- Confidence: T1 (AST-verified structural truth)
|
|
@@ -93,15 +117,26 @@ DO NOT BE LAZY — For EACH file in the change manifest with status MODIFIED, AD
|
|
|
93
117
|
|
|
94
118
|
**For MOVED files:** Re-extract at new location to update file:line references.
|
|
95
119
|
|
|
120
|
+
**Re-export tracing (Forge/Deep only):** After extracting changed files, check if any public exports from the package entry point (`__init__.py`, `index.ts`, `lib.rs`) are unresolved — particularly when a changed file is part of a module re-export chain. Follow the **Re-Export Tracing** protocol in `{extractionPatternsData}` to trace unresolved symbols to their actual definition files.
|
|
121
|
+
|
|
96
122
|
### 3. Deep Tier QMD Enrichment (Conditional)
|
|
97
123
|
|
|
98
124
|
**ONLY if forge_tier == Deep:**
|
|
99
125
|
|
|
126
|
+
Read the `qmd_collections` registry from `{sidecar_path}/forge-tier.yaml`.
|
|
127
|
+
|
|
128
|
+
Find the collection entry matching the current skill: look for an entry where `skill_name` matches the skill being updated AND `type` is `"extraction"`.
|
|
129
|
+
|
|
130
|
+
**If a matching extraction collection is found:**
|
|
100
131
|
Launch a subprocess that loads qmd_bridge and for each changed export:
|
|
101
|
-
1. Queries
|
|
132
|
+
1. Queries the `{skill_name}-extraction` collection for semantic context related to the export
|
|
102
133
|
2. Searches for usage patterns, documentation references, temporal history
|
|
103
134
|
3. Returns T2 evidence per export (usage frequency, context snippets, related concepts)
|
|
104
135
|
|
|
136
|
+
**If no matching collection found in registry:**
|
|
137
|
+
Log: "No QMD extraction collection found for {skill_name}. T2 enrichment skipped. Re-run [CS] Create Skill to generate the collection."
|
|
138
|
+
Continue without T2 enrichment — extraction still produces T1 results.
|
|
139
|
+
|
|
105
140
|
**If forge_tier != Deep:** Skip this section with notice: "QMD enrichment skipped (tier: {forge_tier})"
|
|
106
141
|
|
|
107
142
|
### 4. Compile Extraction Results
|
|
@@ -149,14 +184,8 @@ Extraction Results:
|
|
|
149
184
|
|
|
150
185
|
Display: "**Proceeding to merge...**"
|
|
151
186
|
|
|
152
|
-
#### Menu Handling Logic:
|
|
153
|
-
|
|
154
187
|
- After extraction results are compiled, immediately load, read entire file, then execute {nextStepFile}
|
|
155
|
-
|
|
156
|
-
#### EXECUTION RULES:
|
|
157
|
-
|
|
158
188
|
- This is an auto-proceed step with no user choices
|
|
159
|
-
- Proceed directly to next step after extraction completes
|
|
160
189
|
|
|
161
190
|
## CRITICAL STEP COMPLETION NOTE
|
|
162
191
|
|
|
@@ -35,7 +35,7 @@ Merge freshly extracted export data into the existing SKILL.md content while pre
|
|
|
35
35
|
|
|
36
36
|
- 🎯 Focus ONLY on merging extractions into existing skill content
|
|
37
37
|
- 🚫 FORBIDDEN to delete or modify [MANUAL] section content
|
|
38
|
-
- 🚫 FORBIDDEN to write files — merge produces
|
|
38
|
+
- 🚫 FORBIDDEN to write files — merge produces an edit plan for step 06 to execute
|
|
39
39
|
- 💬 If [MANUAL] conflicts detected: HALT and present to user for resolution
|
|
40
40
|
- 💬 If clean merge: auto-proceed without user interaction
|
|
41
41
|
|
|
@@ -50,7 +50,7 @@ Merge freshly extracted export data into the existing SKILL.md content while pre
|
|
|
50
50
|
|
|
51
51
|
- Available: extraction results from step 03, existing SKILL.md content, [MANUAL] inventory from step 01, change manifest from step 02
|
|
52
52
|
- Focus: content merge and [MANUAL] preservation
|
|
53
|
-
- Limits:
|
|
53
|
+
- Limits: merge planning only — no file writes (step 06 executes the edit plan)
|
|
54
54
|
- Dependencies: step 03 must have produced extraction results
|
|
55
55
|
|
|
56
56
|
## MANDATORY SEQUENCE
|
|
@@ -222,7 +222,7 @@ ONLY WHEN all merge operations are complete and any [MANUAL] conflicts have been
|
|
|
222
222
|
|
|
223
223
|
- ANY [MANUAL] content lost or modified without user consent
|
|
224
224
|
- Auto-resolving [MANUAL] conflicts without user input
|
|
225
|
-
- Writing files during merge (
|
|
225
|
+
- Writing files during merge planning (edits are executed in step 06)
|
|
226
226
|
- Applying merge out of priority order
|
|
227
227
|
- Not detecting orphaned [MANUAL] blocks on deleted exports
|
|
228
228
|
- Not handling stack skill multi-file merge
|