bmad-module-skill-forge 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +88 -44
- package/docs/agents.md +1 -1
- package/docs/architecture.md +49 -20
- package/docs/examples.md +13 -2
- package/docs/getting-started.md +15 -13
- package/docs/index.md +3 -1
- package/docs/workflows.md +2 -2
- package/package.json +5 -3
- package/src/forger/forge-tier.yaml +1 -1
- package/src/forger/preferences.yaml +8 -1
- package/src/knowledge/doc-fetcher.md +55 -0
- package/src/knowledge/overview.md +17 -16
- package/src/knowledge/progressive-capability.md +9 -1
- package/src/knowledge/provenance-tracking.md +31 -23
- package/src/knowledge/qmd-registry.md +132 -0
- package/src/knowledge/skf-knowledge-index.csv +2 -0
- package/src/workflows/analyze-source/data/skill-brief-schema.md +24 -4
- package/src/workflows/analyze-source/steps-c/step-01-init.md +19 -9
- package/src/workflows/analyze-source/steps-c/step-01b-continue.md +2 -2
- package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +10 -2
- package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +17 -2
- package/src/workflows/analyze-source/steps-c/step-05-recommend.md +2 -1
- package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +3 -3
- package/src/workflows/analyze-source/templates/analysis-report-template.md +1 -1
- package/src/workflows/analyze-source/validation-report.md +1 -1
- package/src/workflows/analyze-source/workflow-plan-analyze-source.md +1 -1
- package/src/workflows/analyze-source/workflow.md +1 -1
- package/src/workflows/audit-skill/steps-c/step-01-init.md +8 -5
- package/src/workflows/audit-skill/steps-c/step-02-re-index.md +12 -2
- package/src/workflows/audit-skill/validation-report.md +2 -2
- package/src/workflows/audit-skill/workflow-plan-audit-skill.md +1 -1
- package/src/workflows/brief-skill/data/scope-templates.md +60 -0
- package/src/workflows/brief-skill/data/skill-brief-schema.md +38 -3
- package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +20 -4
- package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +20 -64
- package/src/workflows/brief-skill/steps-c/step-04-confirm-brief.md +1 -1
- package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +60 -5
- package/src/workflows/brief-skill/workflow.md +2 -1
- package/src/workflows/create-skill/data/compile-assembly-rules.md +110 -0
- package/src/workflows/create-skill/data/extraction-patterns.md +248 -0
- package/src/workflows/create-skill/data/skill-sections.md +79 -25
- package/src/workflows/create-skill/data/source-resolution-protocols.md +130 -0
- package/src/workflows/create-skill/steps-c/step-01-load-brief.md +7 -4
- package/src/workflows/create-skill/steps-c/step-02-ecosystem-check.md +10 -7
- package/src/workflows/create-skill/steps-c/step-03-extract.md +49 -9
- package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +238 -0
- package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +220 -0
- package/src/workflows/create-skill/steps-c/step-04-enrich.md +40 -10
- package/src/workflows/create-skill/steps-c/step-05-compile.md +48 -81
- package/src/workflows/create-skill/steps-c/step-06-validate.md +149 -63
- package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +53 -3
- package/src/workflows/create-skill/steps-c/step-08-report.md +1 -1
- package/src/workflows/create-skill/workflow-plan-create-skill.md +42 -10
- package/src/workflows/create-skill/workflow.md +3 -2
- package/src/workflows/create-stack-skill/data/stack-skill-template.md +20 -7
- package/src/workflows/create-stack-skill/steps-c/step-01-init.md +2 -2
- package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +14 -0
- package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +28 -7
- package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +10 -7
- package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +66 -68
- package/src/workflows/create-stack-skill/validation-report.md +1 -1
- package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +4 -4
- package/src/workflows/export-skill/data/managed-section-format.md +1 -0
- package/src/workflows/export-skill/data/snippet-format.md +33 -14
- package/src/workflows/export-skill/steps-c/step-01-load-skill.md +16 -2
- package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +29 -22
- package/src/workflows/export-skill/steps-c/step-04-update-context.md +1 -1
- package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
- package/src/workflows/export-skill/validation-report.md +1 -1
- package/src/workflows/export-skill/workflow-plan-export-skill.md +8 -8
- package/src/workflows/quick-skill/data/skill-template.md +27 -7
- package/src/workflows/quick-skill/steps-c/step-04-compile.md +44 -13
- package/src/workflows/quick-skill/steps-c/step-05-validate.md +79 -16
- package/src/workflows/quick-skill/validation-report.md +4 -4
- package/src/workflows/quick-skill/workflow-plan-quick-skill.md +4 -4
- package/src/workflows/quick-skill/workflow.md +2 -0
- package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +12 -3
- package/src/workflows/setup-forge/steps-c/step-02-write-config.md +10 -2
- package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +111 -40
- package/src/workflows/setup-forge/steps-c/step-04-report.md +13 -0
- package/src/workflows/setup-forge/workflow.md +1 -0
- package/src/workflows/test-skill/data/scoring-rules.md +14 -6
- package/src/workflows/test-skill/data/source-access-protocol.md +37 -0
- package/src/workflows/test-skill/steps-c/step-01-init.md +38 -13
- package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -3
- package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +25 -5
- package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +18 -43
- package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +213 -0
- package/src/workflows/test-skill/steps-c/step-05-score.md +24 -27
- package/src/workflows/test-skill/steps-c/step-06-report.md +9 -4
- package/src/workflows/test-skill/templates/test-report-template.md +2 -1
- package/src/workflows/test-skill/validation-report.md +1 -1
- package/src/workflows/test-skill/workflow-plan-test-skill.md +7 -3
- package/src/workflows/test-skill/workflow.md +1 -1
- package/src/workflows/update-skill/data/remote-source-resolution.md +43 -0
- package/src/workflows/update-skill/steps-c/step-01-init.md +13 -7
- package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +26 -0
- package/src/workflows/update-skill/steps-c/step-03-re-extract.md +37 -9
- package/src/workflows/update-skill/steps-c/step-04-merge.md +3 -3
- package/src/workflows/update-skill/steps-c/step-05-validate.md +63 -67
- package/src/workflows/update-skill/steps-c/step-06-write.md +37 -14
- package/src/workflows/update-skill/steps-c/step-07-report.md +1 -1
- package/src/workflows/update-skill/validation-report.md +4 -4
- package/src/workflows/update-skill/workflow-plan-update-skill.md +4 -4
- package/src/workflows/update-skill/workflow.md +1 -1
- package/tools/cli/commands/status.js +4 -4
- package/tools/cli/commands/uninstall.js +1 -1
- package/tools/cli/commands/update.js +2 -2
- package/tools/cli/lib/ui.js +60 -16
- package/tools/cli/lib/version-check.js +2 -2
|
@@ -61,15 +61,40 @@ Auto-proceed silently. Display no message. Immediately load, read entire file, t
|
|
|
61
61
|
|
|
62
62
|
Continue to step 2.
|
|
63
63
|
|
|
64
|
-
### 2.
|
|
64
|
+
### 2. Collection Inventory Pre-check (Deep Tier Only)
|
|
65
65
|
|
|
66
|
-
|
|
66
|
+
Before searching, check which QMD collections are available:
|
|
67
67
|
|
|
68
|
-
|
|
68
|
+
1. Read the sidecar `forge-tier.yaml` to get registered `qmd_collections` entries
|
|
69
|
+
2. Identify which collections contain enrichment context — collections with `type` equal to `"temporal"` (issues, PRs, changelogs) or `"docs"` (fetched external documentation). Do NOT include `"extraction"` or `"brief"` type collections.
|
|
70
|
+
3. **If no enrichment collections exist** (no `"temporal"` or `"docs"` type collections): report this and auto-proceed. Display:
|
|
69
71
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
72
|
+
"**Enrichment: no enrichment collections available.**
|
|
73
|
+
Only brief-type or extraction-type QMD collections found — no temporal context (issues, PRs, changelogs) or docs collections indexed. Enrichment skipped (expected for first-run skill creation). Enrichment becomes available when temporal or docs context is indexed into QMD collections.
|
|
74
|
+
|
|
75
|
+
Proceeding to compilation..."
|
|
76
|
+
|
|
77
|
+
Then immediately load, read entire file, then execute `{nextStepFile}`.
|
|
78
|
+
|
|
79
|
+
4. **If temporal collections exist:** Continue to step 3.
|
|
80
|
+
|
|
81
|
+
### 3. QMD Enrichment Searches (Deep Tier Only)
|
|
82
|
+
|
|
83
|
+
For each major exported function — limited to the **top-level public API surface** (typically 10-20 functions that will appear in the context-snippet), not all extracted exports — search the temporal QMD collections for context:
|
|
84
|
+
|
|
85
|
+
**Search query construction:**
|
|
86
|
+
|
|
87
|
+
For each function, derive the **module context** from the extraction inventory's source file path (e.g., `src/graph/neo4j/index.ts` → module context `graph neo4j`). This context improves search relevance by scoping results to the function's subsystem without adding extra queries.
|
|
88
|
+
|
|
89
|
+
**Primary searches (BM25 — always runs, no GPU/VRAM dependency):**
|
|
90
|
+
|
|
91
|
+
1. **Issues/PRs:** `qmd_bridge.search("{module_context} {function_name}")` — find related discussions scoped by module context
|
|
92
|
+
2. **Changelog entries:** `qmd_bridge.search("{function_name} changelog")` — find version history, breaking changes (kept generic — changelogs rarely use module paths)
|
|
93
|
+
3. **Migration notes:** `qmd_bridge.search("{function_name} migration deprecated breaking")` — find deprecation or migration context using exact keyword matching
|
|
94
|
+
|
|
95
|
+
**Supplemental search (best-effort — requires GPU/VRAM, may fail):**
|
|
96
|
+
|
|
97
|
+
4. **Semantic migration context:** `qmd_bridge.vector_search("{module_context} {function_name} migration deprecated")` — adds semantic matches (synonyms, paraphrases) that BM25 keyword search may miss. Merge results with search #3, deduplicating by document ID. If `vector_search` fails (VRAM, GPU driver, model loading), discard silently — the BM25 results from search #3 provide baseline coverage.
|
|
73
98
|
|
|
74
99
|
**For each QMD result:**
|
|
75
100
|
|
|
@@ -84,7 +109,7 @@ For each major exported function in the extraction inventory, search QMD collect
|
|
|
84
109
|
- If individual search fails: skip that function's enrichment, continue with others
|
|
85
110
|
- If QMD returns no results for a function: no annotation added (absence is normal)
|
|
86
111
|
|
|
87
|
-
###
|
|
112
|
+
### 4. Annotate Extraction Inventory
|
|
88
113
|
|
|
89
114
|
Add enrichment annotations to the extraction inventory without modifying extraction data:
|
|
90
115
|
|
|
@@ -100,7 +125,7 @@ Add enrichment annotations to the extraction inventory without modifying extract
|
|
|
100
125
|
- T2-past annotations: {count}
|
|
101
126
|
- T2-future annotations: {count}
|
|
102
127
|
|
|
103
|
-
###
|
|
128
|
+
### 5. Report Enrichment (Deep Tier Only)
|
|
104
129
|
|
|
105
130
|
Display brief enrichment summary:
|
|
106
131
|
|
|
@@ -111,7 +136,7 @@ Display brief enrichment summary:
|
|
|
111
136
|
|
|
112
137
|
Proceeding to compilation..."
|
|
113
138
|
|
|
114
|
-
###
|
|
139
|
+
### 6. Menu Handling Logic
|
|
115
140
|
|
|
116
141
|
**Auto-proceed step — no user interaction.**
|
|
117
142
|
|
|
@@ -135,7 +160,9 @@ ONLY WHEN enrichment is complete (Deep tier) or the step is skipped (Quick/Forge
|
|
|
135
160
|
### ✅ SUCCESS:
|
|
136
161
|
|
|
137
162
|
- Quick/Forge tiers: skipped silently with no output, auto-proceeded
|
|
138
|
-
- Deep tier:
|
|
163
|
+
- Deep tier: collection inventory checked before searching
|
|
164
|
+
- Deep tier: no temporal collections → reported and auto-proceeded (not silently skipped)
|
|
165
|
+
- Deep tier: temporal collections exist → QMD searches performed per public API function (10-20, not all exports) with module context derived from extraction inventory source paths
|
|
139
166
|
- T2 annotations added with proper provenance citations
|
|
140
167
|
- Extraction data unmodified — enrichment is additive only
|
|
141
168
|
- QMD failures handled gracefully (skip and continue)
|
|
@@ -146,6 +173,9 @@ ONLY WHEN enrichment is complete (Deep tier) or the step is skipped (Quick/Forge
|
|
|
146
173
|
- Halting on QMD unavailability or search failures
|
|
147
174
|
- Modifying T1 extraction results with QMD data
|
|
148
175
|
- Displaying skip messages for Quick/Forge tiers (should be silent)
|
|
176
|
+
- Pre-emptively skipping all searches without checking the collection inventory
|
|
177
|
+
- Searching all 800+ exports instead of scoping to public API surface (10-20)
|
|
178
|
+
- Using bare function names without module context when source path is available in extraction inventory
|
|
149
179
|
- Beginning compilation or SKILL.md assembly in this step
|
|
150
180
|
- Not labeling QMD annotations as T2 confidence
|
|
151
181
|
|
|
@@ -3,6 +3,7 @@ name: 'step-05-compile'
|
|
|
3
3
|
description: 'Assemble SKILL.md, context-snippet, metadata, and references from extraction inventory'
|
|
4
4
|
nextStepFile: './step-06-validate.md'
|
|
5
5
|
skillSectionsData: '../data/skill-sections.md'
|
|
6
|
+
assemblyRulesData: '../data/compile-assembly-rules.md'
|
|
6
7
|
---
|
|
7
8
|
|
|
8
9
|
# Step 5: Compile
|
|
@@ -30,102 +31,72 @@ To assemble the complete skill content from the extraction inventory and enrichm
|
|
|
30
31
|
|
|
31
32
|
- 🎯 Focus ONLY on assembling content from extraction inventory + enrichment
|
|
32
33
|
- 🚫 FORBIDDEN to include any content without a provenance citation
|
|
33
|
-
-
|
|
34
|
+
- 💾 Sub-agents and compilation processes may write to a staging directory (e.g., `_bmad-output/{name}/`)
|
|
35
|
+
- 🚫 FORBIDDEN to write final files to `skills/` or `forge-data/` — that's step-07
|
|
34
36
|
- 🚫 FORBIDDEN to fabricate examples not found in source tests or docs
|
|
35
37
|
- ⚒️ Seed `<!-- [MANUAL] -->` markers for future update-skill compatibility
|
|
36
38
|
|
|
37
39
|
## EXECUTION PROTOCOLS:
|
|
38
40
|
|
|
39
41
|
- 🎯 Follow MANDATORY SEQUENCE exactly
|
|
40
|
-
- 💾 Build all content in context
|
|
42
|
+
- 💾 Build all content in context or write to a staging directory (`_bmad-output/{name}/`)
|
|
41
43
|
- 📖 Follow agentskills.io section structure from data file
|
|
42
|
-
- 🚫 Do not
|
|
44
|
+
- 🚫 Do not promote any output files to final `skills/` or `forge-data/` directories — that's step-07
|
|
43
45
|
|
|
44
46
|
## CONTEXT BOUNDARIES:
|
|
45
47
|
|
|
46
|
-
- Available: extraction_inventory, enrichment_annotations (if Deep), brief_data, tier
|
|
48
|
+
- Available: extraction_inventory, enrichment_annotations (if Deep), doc_fetch_inventory (if doc_urls), brief_data, tier
|
|
47
49
|
- Focus: Assembling structured content from verified data
|
|
48
|
-
-
|
|
50
|
+
- Docs-only mode: If `source_type: "docs-only"`, all content is T3 `[EXT:{url}]`. Overview must note "Generated from external documentation." metadata.json `source_authority` forced to `community`. No AST/source citations exist — only `[EXT:...]` citations.
|
|
51
|
+
- Limits: Do NOT write to final `skills/` or `forge-data/` directories, validate spec compliance, or report
|
|
49
52
|
- Dependencies: Extraction inventory from step-03 (enrichment from step-04 if Deep)
|
|
50
53
|
|
|
51
54
|
## MANDATORY SEQUENCE
|
|
52
55
|
|
|
53
56
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
|
|
54
57
|
|
|
55
|
-
### 1. Load
|
|
58
|
+
### 1. Load Data Files
|
|
56
59
|
|
|
57
|
-
Load `{skillSectionsData}` completely.
|
|
60
|
+
Load `{skillSectionsData}` and `{assemblyRulesData}` completely. These define the agentskills.io-compliant format and detailed assembly rules for all output artifacts.
|
|
58
61
|
|
|
59
62
|
### 2. Build SKILL.md Content
|
|
60
63
|
|
|
61
|
-
Assemble each section in order
|
|
62
|
-
|
|
63
|
-
**Frontmatter:**
|
|
64
|
-
```yaml
|
|
65
|
-
---
|
|
66
|
-
name: {brief.name}
|
|
67
|
-
version: {brief.version}
|
|
68
|
-
description: {brief.name} — {export_count} verified functions
|
|
69
|
-
author: {source_author}
|
|
70
|
-
---
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
**Section 1 — Overview:**
|
|
74
|
-
- What this skill provides
|
|
75
|
-
- Source repo, version, branch
|
|
76
|
-
- Forge tier used for compilation
|
|
77
|
-
- Export count and confidence summary
|
|
78
|
-
|
|
79
|
-
**Section 2 — Quick Start:**
|
|
80
|
-
- Select the 3-5 most commonly used functions (by import frequency or documentation prominence)
|
|
81
|
-
- Minimal usage examples — ONLY from source tests or official docs
|
|
82
|
-
- If no examples exist in source, show signature-only quick start
|
|
83
|
-
- Provenance citation for each function
|
|
84
|
-
|
|
85
|
-
**Section 3 — API Reference:**
|
|
86
|
-
- Group by module/file
|
|
87
|
-
- For each exported function:
|
|
88
|
-
- Full signature with types
|
|
89
|
-
- Parameters table (name, type, required/optional, description)
|
|
90
|
-
- Return type
|
|
91
|
-
- Usage example (from source tests/docs ONLY — omit if none found)
|
|
92
|
-
- Provenance citation: `[AST:{file}:L{line}]` or `[SRC:{file}:L{line}]`
|
|
93
|
-
- T2 annotations (Deep tier only): temporal context from enrichment
|
|
94
|
-
|
|
95
|
-
**Section 4 — Type Definitions:**
|
|
96
|
-
- All exported types, interfaces, enums
|
|
97
|
-
- Full type signatures with provenance citations
|
|
98
|
-
|
|
99
|
-
**Section 5 — Integration Patterns (Forge/Deep only):**
|
|
100
|
-
- Co-import patterns detected by ast_bridge
|
|
101
|
-
- Common usage combinations
|
|
102
|
-
- Skip for Quick tier (no co-import data available)
|
|
103
|
-
|
|
104
|
-
**Section 6 — Manual Sections:**
|
|
105
|
-
- Seed empty `<!-- [MANUAL] -->` markers:
|
|
106
|
-
```markdown
|
|
107
|
-
<!-- [MANUAL:additional-notes] -->
|
|
108
|
-
<!-- Add custom notes here. This section is preserved during skill updates. -->
|
|
109
|
-
<!-- [/MANUAL:additional-notes] -->
|
|
110
|
-
```
|
|
111
|
-
- Place after Quick Start and after API Reference sections
|
|
64
|
+
Assemble each section in order using the assembly rules data file (`{assemblyRulesData}`). The data file specifies frontmatter format, Tier 1 section details (Sections 1-8), Tier 2 section details (Sections 9-11), and assembly ordering rules. Follow it exactly.
|
|
112
65
|
|
|
113
66
|
### 3. Build context-snippet.md Content
|
|
114
67
|
|
|
115
|
-
|
|
68
|
+
Vercel-aligned indexed format for CLAUDE.md managed section (~80-120 tokens):
|
|
116
69
|
|
|
117
70
|
```markdown
|
|
118
|
-
{skill-name}
|
|
119
|
-
|
|
71
|
+
[{skill-name} v{version}]|root: skills/{skill-name}/
|
|
72
|
+
|IMPORTANT: {skill-name} v{version} — read SKILL.md before writing {skill-name} code. Do NOT rely on training data.
|
|
73
|
+
|quick-start:{SKILL.md#quick-start}
|
|
74
|
+
|api: {top exports with () for functions, comma-separated}
|
|
75
|
+
|key-types:{SKILL.md#key-types} — {inline summary of most important type values}
|
|
76
|
+
|gotchas: {2-3 most critical pitfalls or breaking changes, inline}
|
|
120
77
|
```
|
|
121
78
|
|
|
79
|
+
**Derivation rules:**
|
|
80
|
+
|
|
81
|
+
- **version**: From source detection (reconciled in step-03), not brief default
|
|
82
|
+
- **api**: Top 10 exports from extraction inventory, append `()` to function names
|
|
83
|
+
- **key-types**: Inline summary of most important enum/type values from Key Types section
|
|
84
|
+
- **gotchas**: Derived from T2-future annotations (breaking changes), async requirements, version-specific behavior changes. If no gotchas available, omit the gotchas line.
|
|
85
|
+
- **Section anchors** (`#quick-start`, `#key-types`): Must match actual heading slugs in the assembled SKILL.md
|
|
86
|
+
|
|
122
87
|
### 4. Build metadata.json Content
|
|
123
88
|
|
|
124
89
|
Following the structure from the skill-sections data file:
|
|
125
90
|
- Populate all fields from brief_data, extraction inventory, and tier
|
|
126
|
-
- Set `
|
|
91
|
+
- Set `generation_date` to current ISO-8601 timestamp
|
|
127
92
|
- Set `source_commit` from resolved source (if available)
|
|
128
|
-
- Set `stats` from extraction aggregate counts
|
|
93
|
+
- Set `stats` from extraction aggregate counts:
|
|
94
|
+
- `exports_documented`: count of exports with documentation in the assembled SKILL.md
|
|
95
|
+
- `exports_public_api`: count of exports from public entry points (`__init__.py`, `index.ts`, `lib.rs`, or equivalent)
|
|
96
|
+
- `exports_internal`: count of all other non-underscore-prefixed exports (internal modules, helpers, adapters)
|
|
97
|
+
- `exports_total`: `exports_public_api` + `exports_internal`
|
|
98
|
+
- `public_api_coverage`: `exports_documented / exports_public_api` (1.0 when all public API exports are documented; `null` if `exports_public_api` is 0)
|
|
99
|
+
- `total_coverage`: `exports_documented / exports_total` (may be low for large codebases — this is expected; `null` if `exports_total` is 0)
|
|
129
100
|
- Set `tool_versions` based on tier and available tools
|
|
130
101
|
|
|
131
102
|
### 5. Build references/ Content
|
|
@@ -140,37 +111,27 @@ Group functions logically by module, file, or functional area.
|
|
|
140
111
|
|
|
141
112
|
### 6. Build provenance-map.json Content
|
|
142
113
|
|
|
143
|
-
One entry per extracted
|
|
144
|
-
- claim text
|
|
145
|
-
- source_file and source_line
|
|
146
|
-
- confidence tier (T1, T1-low, T2)
|
|
147
|
-
- extraction_method (ast_bridge.scan_definitions, gh_bridge.read_file, qmd_bridge.search)
|
|
148
|
-
- ast_node_type (if AST-extracted)
|
|
114
|
+
One entry per extracted export: export_name, export_type, params[] (typed strings), return_type, source_file, source_line, confidence tier (T1/T1-low/T2), extraction_method, ast_node_type. See `{skillSectionsData}` for full schema.
|
|
149
115
|
|
|
150
116
|
### 7. Build evidence-report.md Content
|
|
151
117
|
|
|
152
|
-
Compilation audit trail:
|
|
153
|
-
- Generation date, forge tier, source info
|
|
154
|
-
- Tool versions used
|
|
155
|
-
- Extraction summary (files scanned, exports found, confidence breakdown)
|
|
156
|
-
- Validation results (populated in step-06)
|
|
157
|
-
- Warnings from extraction or enrichment
|
|
118
|
+
Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template.
|
|
158
119
|
|
|
159
120
|
### 8. Menu Handling Logic
|
|
160
121
|
|
|
161
122
|
**Auto-proceed step — no user interaction.**
|
|
162
123
|
|
|
163
|
-
After all content is assembled in context, immediately load, read entire file, then execute `{nextStepFile}`.
|
|
124
|
+
After all content is assembled in context (or written to the staging directory), immediately load, read entire file, then execute `{nextStepFile}`.
|
|
164
125
|
|
|
165
126
|
#### EXECUTION RULES:
|
|
166
127
|
|
|
167
128
|
- This is an auto-proceed assembly step with no user choices
|
|
168
|
-
- All content stays in context — no files are written yet
|
|
129
|
+
- All content stays in context or in the staging directory — no final files are written yet
|
|
169
130
|
- Proceed directly to validation after assembly is complete
|
|
170
131
|
|
|
171
132
|
## CRITICAL STEP COMPLETION NOTE
|
|
172
133
|
|
|
173
|
-
ONLY WHEN all
|
|
134
|
+
ONLY WHEN all 6 content artifacts (SKILL.md, context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md) are assembled in context will you proceed to load `{nextStepFile}` for spec validation. Note: extraction-rules.yaml is generated by step-07 from extraction data, not assembled here.
|
|
174
135
|
|
|
175
136
|
---
|
|
176
137
|
|
|
@@ -178,8 +139,11 @@ ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json,
|
|
|
178
139
|
|
|
179
140
|
### ✅ SUCCESS:
|
|
180
141
|
|
|
181
|
-
- Skill-sections data file loaded and followed
|
|
142
|
+
- Skill-sections data file and assembly rules data file loaded and followed
|
|
182
143
|
- SKILL.md assembled with all required sections in agentskills.io format
|
|
144
|
+
- Tier 1 sections assembled first and kept under 300 lines (excluding frontmatter)
|
|
145
|
+
- Tier 2 sections assembled after all Tier 1 sections
|
|
146
|
+
- Section 4b emitted only when Deep tier AND T2-future annotations exist; omitted entirely otherwise (no empty section)
|
|
183
147
|
- Every function entry has a provenance citation
|
|
184
148
|
- [MANUAL] markers seeded for update-skill compatibility
|
|
185
149
|
- context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md all assembled
|
|
@@ -190,9 +154,12 @@ ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json,
|
|
|
190
154
|
|
|
191
155
|
- Including functions or examples without provenance citations
|
|
192
156
|
- Fabricating usage examples not found in source
|
|
193
|
-
- Writing files to
|
|
157
|
+
- Writing files to final `skills/` or `forge-data/` directories (that's step-07)
|
|
194
158
|
- Missing required SKILL.md sections
|
|
159
|
+
- Emitting an empty Section 4b when no T2-future annotations exist
|
|
160
|
+
- Tier 1 content exceeding 300 lines without reduction
|
|
161
|
+
- Assembling Tier 2 sections before Tier 1 sections
|
|
195
162
|
- Not seeding [MANUAL] markers
|
|
196
|
-
- Not building all
|
|
163
|
+
- Not building all 6 content artifacts (extraction-rules.yaml is built by step-07)
|
|
197
164
|
|
|
198
165
|
**Master Rule:** Zero hallucination — every line of SKILL.md must trace to source code. Compile from data, not imagination.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: 'step-06-validate'
|
|
3
|
-
description: 'Validate compiled skill content against agentskills.io spec via
|
|
3
|
+
description: 'Validate compiled skill content against agentskills.io spec via skill-check'
|
|
4
4
|
nextStepFile: './step-07-generate-artifacts.md'
|
|
5
5
|
---
|
|
6
6
|
|
|
@@ -8,7 +8,7 @@ nextStepFile: './step-07-generate-artifacts.md'
|
|
|
8
8
|
|
|
9
9
|
## STEP GOAL:
|
|
10
10
|
|
|
11
|
-
To validate the compiled SKILL.md content against the agentskills.io specification using
|
|
11
|
+
To validate the compiled SKILL.md content against the agentskills.io specification using skill-check, auto-fix any validation failures, and confirm spec compliance before artifact generation.
|
|
12
12
|
|
|
13
13
|
## MANDATORY EXECUTION RULES (READ FIRST):
|
|
14
14
|
|
|
@@ -29,9 +29,10 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
|
|
|
29
29
|
|
|
30
30
|
- 🎯 Focus ONLY on validating compiled content against spec
|
|
31
31
|
- 🚫 FORBIDDEN to add new content — only fix spec compliance issues
|
|
32
|
-
-
|
|
32
|
+
- 💾 Validation and auto-fix modify files in the staging directory (`_bmad-output/{name}/`)
|
|
33
33
|
- 💬 If auto-fix fails, report issues clearly but proceed (warn, don't halt)
|
|
34
|
-
- ⚙️ If
|
|
34
|
+
- ⚙️ If skill-check unavailable: skip validation, add warning to evidence report
|
|
35
|
+
- ⚠️ Ignore non-zero exit codes from `skill-check` if the JSON output shows 0 errors — parse JSON output, not exit codes
|
|
35
36
|
|
|
36
37
|
## EXECUTION PROTOCOLS:
|
|
37
38
|
|
|
@@ -53,85 +54,169 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
|
|
|
53
54
|
|
|
54
55
|
### 1. Check Tool Availability
|
|
55
56
|
|
|
56
|
-
|
|
57
|
-
Continue to validation steps.
|
|
57
|
+
Run: `npx skill-check -h`
|
|
58
58
|
|
|
59
|
-
|
|
60
|
-
-
|
|
61
|
-
- Auto-proceed to next step
|
|
59
|
+
- If succeeds: Continue to automated validation (section 2)
|
|
60
|
+
- If fails: Perform manual fallback (section 3); add note to evidence-report: "Spec validation performed manually — skill-check tool unavailable"
|
|
62
61
|
|
|
63
|
-
|
|
62
|
+
**Important:** Do not assume availability — empirical check required.
|
|
64
63
|
|
|
65
|
-
|
|
64
|
+
### 2. Validate & Auto-Fix (skill-check check --fix)
|
|
66
65
|
|
|
67
|
-
|
|
68
|
-
- Required sections present (Overview, Quick Start, API Reference, Type Definitions)
|
|
69
|
-
- Section order follows agentskills.io standard
|
|
70
|
-
- Frontmatter contains required fields (name, version, description, author)
|
|
71
|
-
- Provenance citations present in API Reference entries
|
|
66
|
+
Run the external skill-check tool against the compiled skill staging directory:
|
|
72
67
|
|
|
73
|
-
|
|
68
|
+
```bash
|
|
69
|
+
npx skill-check check <staging-skill-dir> --fix --format json --no-security-scan
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
This performs frontmatter validation, description quality checks, body limit enforcement, local link resolution, file formatting, auto-fix of deterministic issues, and quality scoring (0-100) across five weighted categories.
|
|
73
|
+
|
|
74
|
+
**Parse the JSON output** for: `qualityScore` (0-100), `diagnostics[]` (remaining issues), `fixed[]` (auto-corrected issues).
|
|
75
|
+
|
|
76
|
+
**Note:** `skill-check` may return non-zero exit code even when `errorCount` is 0. Always rely on parsed JSON, not the shell exit code.
|
|
77
|
+
|
|
78
|
+
- **Score ≥ 70:** Record "Schema: PASS (score: {score}/100)" in evidence-report
|
|
79
|
+
- **Score < 70:** Log remaining diagnostics as warnings, record "Schema: WARN — score {score}/100, {count} remaining issues", proceed
|
|
80
|
+
- **Unfixable errors:** Record specific rule IDs and suggestions, proceed with warnings
|
|
81
|
+
|
|
82
|
+
### 3. Validate Frontmatter (Fallback)
|
|
83
|
+
|
|
84
|
+
**If skill-check was available:** Skip — already validated in step 2.
|
|
85
|
+
|
|
86
|
+
**If skill-check NOT available (fallback):** Perform manual frontmatter compliance check:
|
|
87
|
+
|
|
88
|
+
- [ ] Frontmatter present — file starts with `---` and has closing `---`
|
|
89
|
+
- [ ] `name` field — present, non-empty, lowercase alphanumeric + hyphens only, 1-64 chars
|
|
90
|
+
- [ ] `name` matches skill output directory name
|
|
91
|
+
- [ ] `description` field — present, non-empty, 1-1024 characters
|
|
92
|
+
- [ ] No unknown fields — only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` permitted
|
|
93
|
+
- [ ] `version` and `author` are NOT in frontmatter (they belong in metadata.json)
|
|
94
|
+
|
|
95
|
+
If fails: auto-fix (deterministic), re-validate once, record result. If passes: record "Frontmatter: PASS".
|
|
96
|
+
|
|
97
|
+
### 4. Split Oversized Body (if needed)
|
|
98
|
+
|
|
99
|
+
**If step 2 reported `body.max_lines` failure:**
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
npx skill-check split-body <staging-skill-dir> --write
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Then re-validate: `npx skill-check check <staging-skill-dir> --format json --no-security-scan`
|
|
106
|
+
|
|
107
|
+
**If skill-check unavailable or no body size issue:** Skip.
|
|
108
|
+
|
|
109
|
+
### 5. Security Scan
|
|
110
|
+
|
|
111
|
+
**If skill-check available:**
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
npx skill-check check <staging-skill-dir> --format json
|
|
115
|
+
```
|
|
74
116
|
|
|
75
|
-
|
|
76
|
-
1. Identify specific failures
|
|
77
|
-
2. Attempt auto-fix (structural adjustments only — never invent content)
|
|
78
|
-
3. Re-validate once
|
|
79
|
-
4. If still failing: record "Schema: FAIL — {specific issues}" in evidence-report, add warnings, but proceed
|
|
117
|
+
(Security scan enabled by default when `--no-security-scan` omitted. The scan uses [Snyk Agent Scan](https://github.com/snyk/agent-scan) to check for prompt injection risks, sensitive data exposure, and unsafe tool permissions.)
|
|
80
118
|
|
|
81
|
-
|
|
119
|
+
Record any security warnings in evidence-report. Security findings are advisory — they do not block artifact generation.
|
|
82
120
|
|
|
83
|
-
|
|
121
|
+
**If security scan fails due to missing SNYK_TOKEN:**
|
|
84
122
|
|
|
85
|
-
|
|
86
|
-
- `name` matches brief.name (kebab-case)
|
|
87
|
-
- `version` matches brief.version
|
|
88
|
-
- `description` follows format: "{name} — {count} verified functions"
|
|
89
|
-
- `author` is set
|
|
123
|
+
Display actionable guidance:
|
|
90
124
|
|
|
91
|
-
|
|
125
|
+
"Security scan requires a Snyk API token. **Important:** The Snyk API requires an Enterprise plan ([authentication docs](https://docs.snyk.io/snyk-api/authentication-for-api)). To enable:
|
|
126
|
+
1. Obtain a Snyk Enterprise account with API access
|
|
127
|
+
2. Get your API token from Account Settings → API Token
|
|
128
|
+
3. Set `SNYK_TOKEN=your-token` in your environment or `.env`
|
|
129
|
+
4. Re-run [SF] Setup Forge to update tool detection, then re-run [CS] Create Skill
|
|
92
130
|
|
|
93
|
-
|
|
94
|
-
1. Auto-fix frontmatter (these are deterministic fixes)
|
|
95
|
-
2. Re-validate once
|
|
96
|
-
3. Record result in evidence-report
|
|
131
|
+
If you don't have an Enterprise account, use `--no-security-scan` flag in skill-check to skip, or `--security-scan-runner pipx|uvx|local` to control the execution method. Security scanning is optional and does not affect tier level or block skill compilation."
|
|
97
132
|
|
|
98
|
-
|
|
133
|
+
Record: "Security scan skipped — SNYK_TOKEN not configured"
|
|
99
134
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
135
|
+
**If skill-check unavailable:** Skip with note: "Security scan skipped — skill-check tool unavailable"
|
|
136
|
+
|
|
137
|
+
### 6. Content Quality Review (tessl)
|
|
138
|
+
|
|
139
|
+
**If tessl available**, run: `npx -y tessl skill review <staging-skill-dir>`
|
|
140
|
+
|
|
141
|
+
Parse output for: `description_score`, `content_score`, `average_score`, `validation_result`, `judge_suggestions[]`.
|
|
142
|
+
|
|
143
|
+
- **Content score < 70%:** Record warning: "Content quality warning: tessl scored content at {score}%."
|
|
144
|
+
- **Unavailable:** Skip with note: "Content quality review skipped — tessl tool unavailable"
|
|
145
|
+
|
|
146
|
+
**Expected scoring for two-tier design:** The Skill Forge two-tier design (Tier 1 Key API Summary + Tier 2 Full API Reference) intentionally includes progressive disclosure. tessl's `conciseness` scorer may flag this as redundancy (typically scoring 2/3), which is expected behavior — not a defect. Content scores >= 60% are acceptable when both tiers are correctly applied with proper differentiation (see step-05 Tier 2 assembly rules). Do NOT consolidate Tier 1 and Tier 2 content to improve the score — the two-tier structure is a deliberate design choice for standalone usability.
|
|
147
|
+
|
|
148
|
+
tessl installs automatically via `npx`. A missing tool is not an error — graceful skip.
|
|
149
|
+
|
|
150
|
+
#### 6b. User Decision Gate (conditional)
|
|
151
|
+
|
|
152
|
+
**If tessl returned no suggestions OR tessl was unavailable:** Skip this gate — auto-proceed.
|
|
153
|
+
|
|
154
|
+
**If tessl returned suggestions**, present them to the user:
|
|
155
|
+
|
|
156
|
+
"**Content quality review: {score}%**
|
|
157
|
+
|
|
158
|
+
tessl suggestions:
|
|
159
|
+
{numbered list of judge_suggestions}
|
|
160
|
+
|
|
161
|
+
**Select an option:**
|
|
162
|
+
- **[S] Skip** — proceed with current content as-is (default)
|
|
163
|
+
- **[A] Apply structural fixes** — apply only structural suggestions (split sections, consolidate duplicates). No new content generated.
|
|
164
|
+
- **[R] Review all** — show each suggestion with proposed changes before applying"
|
|
165
|
+
|
|
166
|
+
#### Gate Rules:
|
|
167
|
+
|
|
168
|
+
- **Structural suggestions** (split reference section, consolidate duplicates, reorder sections) can be applied without zero-hallucination risk — they restructure existing content
|
|
169
|
+
- **Semantic suggestions** (add examples, add error handling, add validation checkpoints) introduce content not verified from source code. If the user chooses to apply these:
|
|
170
|
+
- Warn: "This adds content not verified from source code."
|
|
171
|
+
- Mark applied content with `<!-- [TESSL:auto-fix] -->` markers
|
|
172
|
+
- Cite as `[TESSL:suggestion]` in the provenance map with `confidence: "TESSL"` (below T3)
|
|
173
|
+
- Record in evidence report: "TESSL-suggested content applied: {count} items (unverified)"
|
|
174
|
+
- **If user selects [S]:** Record "tessl suggestions: skipped by user" in evidence report. Proceed to section 7.
|
|
175
|
+
- **If user selects [A]:** Apply structural fixes only, re-run tessl to capture updated score, record results. Proceed to section 7.
|
|
176
|
+
- **If user selects [R]:** Show each suggestion with the proposed change. For each, user confirms or skips. Apply confirmed changes, record results. Proceed to section 7.
|
|
177
|
+
|
|
178
|
+
### 7. Validate metadata.json
|
|
179
|
+
|
|
180
|
+
Cross-check metadata.json against extraction inventory:
|
|
181
|
+
- `stats.exports_documented` / `stats.exports_public_api` / `stats.exports_internal` / `stats.exports_total` are accurate
|
|
182
|
+
- `stats.public_api_coverage` and `stats.total_coverage` are correctly computed (null when denominator is 0)
|
|
104
183
|
- `confidence_t1`, `confidence_t2`, `confidence_t3` match actual counts
|
|
105
184
|
- `spec_version` is "1.3"
|
|
106
185
|
|
|
107
186
|
Auto-fix any discrepancies (these are computed values).
|
|
108
187
|
|
|
109
|
-
###
|
|
188
|
+
### 8. Update Evidence Report
|
|
110
189
|
|
|
111
|
-
Add validation results to
|
|
190
|
+
Add validation results to evidence-report content in context:
|
|
112
191
|
|
|
113
192
|
```markdown
|
|
114
193
|
## Validation Results
|
|
115
|
-
- Schema: {pass/fail}
|
|
194
|
+
- Schema: {pass/fail} (quality score: {score}/100)
|
|
116
195
|
- Frontmatter: {pass/fail}
|
|
196
|
+
- Body: {pass/fail} {split-body applied if applicable}
|
|
197
|
+
- Security: {pass/warn/skipped}
|
|
198
|
+
- Content Quality (tessl): {pass/warn/skipped} (score: {score}%)
|
|
117
199
|
- Metadata: {pass/fail}
|
|
118
200
|
|
|
119
|
-
##
|
|
120
|
-
- {
|
|
121
|
-
```
|
|
201
|
+
## Quality Score Breakdown
|
|
202
|
+
- Frontmatter (30%): {score} | Description (30%): {score} | Body (20%): {score} | Links (10%): {score} | File (10%): {score}
|
|
122
203
|
|
|
123
|
-
|
|
204
|
+
## Auto-Fixed Issues
|
|
205
|
+
- {list of issues automatically corrected by --fix}
|
|
206
|
+
|
|
207
|
+
## Remaining Warnings / Security Findings / Content Quality (tessl)
|
|
208
|
+
- {warnings, security results, tessl scores and suggestions — or "skipped"}
|
|
209
|
+
```
|
|
124
210
|
|
|
125
|
-
|
|
211
|
+
### 9. Menu Handling Logic
|
|
126
212
|
|
|
127
|
-
|
|
213
|
+
**Conditional interaction step.** If tessl produced suggestions, section 6b halts for user input. Otherwise, auto-proceed.
|
|
128
214
|
|
|
129
|
-
|
|
215
|
+
After validation completes (including any user decisions from section 6b), immediately load, read entire file, then execute `{nextStepFile}`.
|
|
130
216
|
|
|
131
|
-
- This is an auto-proceed validation step with no user choices
|
|
132
217
|
- Tool unavailability is a skip, not a halt
|
|
133
218
|
- Validation failures are warnings — proceed to artifact generation
|
|
134
|
-
-
|
|
219
|
+
- tessl gate only triggers when suggestions exist — no gate for clean reviews or unavailable tools
|
|
135
220
|
|
|
136
221
|
## CRITICAL STEP COMPLETION NOTE
|
|
137
222
|
|
|
@@ -143,20 +228,21 @@ ONLY WHEN validation is complete (or skipped) and evidence-report content is upd
|
|
|
143
228
|
|
|
144
229
|
### ✅ SUCCESS:
|
|
145
230
|
|
|
146
|
-
-
|
|
147
|
-
-
|
|
148
|
-
-
|
|
149
|
-
-
|
|
150
|
-
-
|
|
151
|
-
-
|
|
152
|
-
-
|
|
231
|
+
- `npx skill-check check --fix --format json` executed (or skipped with warning if unavailable)
|
|
232
|
+
- Quality score captured and recorded; auto-fix applied for deterministic issues
|
|
233
|
+
- Split-body applied if `body.max_lines` failed; security scan executed (or skipped with warning)
|
|
234
|
+
- `npx -y tessl skill review` executed (or skipped); content quality warning raised if score < 70%
|
|
235
|
+
- tessl suggestions presented to user when available; user decision recorded
|
|
236
|
+
- TESSL-applied content marked with `<!-- [TESSL:auto-fix] -->` and cited as `[TESSL:suggestion]`
|
|
237
|
+
- Metadata cross-check performed; evidence report updated with structured results
|
|
238
|
+
- Proceeded to step-07 (auto or after user gate)
|
|
153
239
|
|
|
154
240
|
### ❌ SYSTEM FAILURE:
|
|
155
241
|
|
|
156
|
-
- Halting
|
|
157
|
-
-
|
|
158
|
-
-
|
|
159
|
-
- Not recording
|
|
242
|
+
- Halting on validation failure or skill-check unavailability (should warn and proceed)
|
|
243
|
+
- Adding new content during validation without user approval via the tessl gate
|
|
244
|
+
- Applying semantic tessl suggestions without warning the user about unverified content
|
|
245
|
+
- Not recording quality score; skipping security scan without recording the skip
|
|
160
246
|
- Attempting more than one auto-fix cycle per failure
|
|
161
247
|
|
|
162
|
-
**Master Rule:** Validation informs, it does not block. Record results, fix what's deterministic, warn about the rest, and proceed.
|
|
248
|
+
**Master Rule:** Validation informs, it does not block. Record results, fix what's deterministic, scan for security issues, warn about the rest, and proceed.
|