bmad-module-skill-forge 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (110) hide show
  1. package/README.md +88 -44
  2. package/docs/agents.md +1 -1
  3. package/docs/architecture.md +49 -20
  4. package/docs/examples.md +13 -2
  5. package/docs/getting-started.md +15 -13
  6. package/docs/index.md +3 -1
  7. package/docs/workflows.md +2 -2
  8. package/package.json +5 -3
  9. package/src/forger/forge-tier.yaml +1 -1
  10. package/src/forger/preferences.yaml +8 -1
  11. package/src/knowledge/doc-fetcher.md +55 -0
  12. package/src/knowledge/overview.md +17 -16
  13. package/src/knowledge/progressive-capability.md +9 -1
  14. package/src/knowledge/provenance-tracking.md +31 -23
  15. package/src/knowledge/qmd-registry.md +132 -0
  16. package/src/knowledge/skf-knowledge-index.csv +2 -0
  17. package/src/workflows/analyze-source/data/skill-brief-schema.md +24 -4
  18. package/src/workflows/analyze-source/steps-c/step-01-init.md +19 -9
  19. package/src/workflows/analyze-source/steps-c/step-01b-continue.md +2 -2
  20. package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +10 -2
  21. package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +17 -2
  22. package/src/workflows/analyze-source/steps-c/step-05-recommend.md +2 -1
  23. package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +3 -3
  24. package/src/workflows/analyze-source/templates/analysis-report-template.md +1 -1
  25. package/src/workflows/analyze-source/validation-report.md +1 -1
  26. package/src/workflows/analyze-source/workflow-plan-analyze-source.md +1 -1
  27. package/src/workflows/analyze-source/workflow.md +1 -1
  28. package/src/workflows/audit-skill/steps-c/step-01-init.md +8 -5
  29. package/src/workflows/audit-skill/steps-c/step-02-re-index.md +12 -2
  30. package/src/workflows/audit-skill/validation-report.md +2 -2
  31. package/src/workflows/audit-skill/workflow-plan-audit-skill.md +1 -1
  32. package/src/workflows/brief-skill/data/scope-templates.md +60 -0
  33. package/src/workflows/brief-skill/data/skill-brief-schema.md +38 -3
  34. package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +20 -4
  35. package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +20 -64
  36. package/src/workflows/brief-skill/steps-c/step-04-confirm-brief.md +1 -1
  37. package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +60 -5
  38. package/src/workflows/brief-skill/workflow.md +2 -1
  39. package/src/workflows/create-skill/data/compile-assembly-rules.md +110 -0
  40. package/src/workflows/create-skill/data/extraction-patterns.md +248 -0
  41. package/src/workflows/create-skill/data/skill-sections.md +79 -25
  42. package/src/workflows/create-skill/data/source-resolution-protocols.md +130 -0
  43. package/src/workflows/create-skill/steps-c/step-01-load-brief.md +7 -4
  44. package/src/workflows/create-skill/steps-c/step-02-ecosystem-check.md +10 -7
  45. package/src/workflows/create-skill/steps-c/step-03-extract.md +49 -9
  46. package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +238 -0
  47. package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +220 -0
  48. package/src/workflows/create-skill/steps-c/step-04-enrich.md +40 -10
  49. package/src/workflows/create-skill/steps-c/step-05-compile.md +48 -81
  50. package/src/workflows/create-skill/steps-c/step-06-validate.md +149 -63
  51. package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +53 -3
  52. package/src/workflows/create-skill/steps-c/step-08-report.md +1 -1
  53. package/src/workflows/create-skill/workflow-plan-create-skill.md +42 -10
  54. package/src/workflows/create-skill/workflow.md +3 -2
  55. package/src/workflows/create-stack-skill/data/stack-skill-template.md +20 -7
  56. package/src/workflows/create-stack-skill/steps-c/step-01-init.md +2 -2
  57. package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +14 -0
  58. package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +28 -7
  59. package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +10 -7
  60. package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +66 -68
  61. package/src/workflows/create-stack-skill/validation-report.md +1 -1
  62. package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +4 -4
  63. package/src/workflows/export-skill/data/managed-section-format.md +1 -0
  64. package/src/workflows/export-skill/data/snippet-format.md +33 -14
  65. package/src/workflows/export-skill/steps-c/step-01-load-skill.md +16 -2
  66. package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +29 -22
  67. package/src/workflows/export-skill/steps-c/step-04-update-context.md +1 -1
  68. package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
  69. package/src/workflows/export-skill/validation-report.md +1 -1
  70. package/src/workflows/export-skill/workflow-plan-export-skill.md +8 -8
  71. package/src/workflows/quick-skill/data/skill-template.md +27 -7
  72. package/src/workflows/quick-skill/steps-c/step-04-compile.md +44 -13
  73. package/src/workflows/quick-skill/steps-c/step-05-validate.md +79 -16
  74. package/src/workflows/quick-skill/validation-report.md +4 -4
  75. package/src/workflows/quick-skill/workflow-plan-quick-skill.md +4 -4
  76. package/src/workflows/quick-skill/workflow.md +2 -0
  77. package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +12 -3
  78. package/src/workflows/setup-forge/steps-c/step-02-write-config.md +10 -2
  79. package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +111 -40
  80. package/src/workflows/setup-forge/steps-c/step-04-report.md +13 -0
  81. package/src/workflows/setup-forge/workflow.md +1 -0
  82. package/src/workflows/test-skill/data/scoring-rules.md +14 -6
  83. package/src/workflows/test-skill/data/source-access-protocol.md +37 -0
  84. package/src/workflows/test-skill/steps-c/step-01-init.md +38 -13
  85. package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -3
  86. package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +25 -5
  87. package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +18 -43
  88. package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +213 -0
  89. package/src/workflows/test-skill/steps-c/step-05-score.md +24 -27
  90. package/src/workflows/test-skill/steps-c/step-06-report.md +9 -4
  91. package/src/workflows/test-skill/templates/test-report-template.md +2 -1
  92. package/src/workflows/test-skill/validation-report.md +1 -1
  93. package/src/workflows/test-skill/workflow-plan-test-skill.md +7 -3
  94. package/src/workflows/test-skill/workflow.md +1 -1
  95. package/src/workflows/update-skill/data/remote-source-resolution.md +43 -0
  96. package/src/workflows/update-skill/steps-c/step-01-init.md +13 -7
  97. package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +26 -0
  98. package/src/workflows/update-skill/steps-c/step-03-re-extract.md +37 -9
  99. package/src/workflows/update-skill/steps-c/step-04-merge.md +3 -3
  100. package/src/workflows/update-skill/steps-c/step-05-validate.md +63 -67
  101. package/src/workflows/update-skill/steps-c/step-06-write.md +37 -14
  102. package/src/workflows/update-skill/steps-c/step-07-report.md +1 -1
  103. package/src/workflows/update-skill/validation-report.md +4 -4
  104. package/src/workflows/update-skill/workflow-plan-update-skill.md +4 -4
  105. package/src/workflows/update-skill/workflow.md +1 -1
  106. package/tools/cli/commands/status.js +4 -4
  107. package/tools/cli/commands/uninstall.js +1 -1
  108. package/tools/cli/commands/update.js +2 -2
  109. package/tools/cli/lib/ui.js +60 -16
  110. package/tools/cli/lib/version-check.js +2 -2
@@ -61,15 +61,40 @@ Auto-proceed silently. Display no message. Immediately load, read entire file, t
61
61
 
62
62
  Continue to step 2.
63
63
 
64
- ### 2. QMD Enrichment Searches (Deep Tier Only)
64
+ ### 2. Collection Inventory Pre-check (Deep Tier Only)
65
65
 
66
- For each major exported function in the extraction inventory, search QMD collections for temporal context:
66
+ Before searching, check which QMD collections are available:
67
67
 
68
- **Search targets per function:**
68
+ 1. Read the sidecar `forge-tier.yaml` to get registered `qmd_collections` entries
69
+ 2. Identify which collections contain enrichment context — collections with `type` equal to `"temporal"` (issues, PRs, changelogs) or `"docs"` (fetched external documentation). Do NOT include `"extraction"` or `"brief"` type collections.
70
+ 3. **If no enrichment collections exist** (no `"temporal"` or `"docs"` type collections): report this and auto-proceed. Display:
69
71
 
70
- 1. **Issues/PRs:** `qmd_bridge.search("{function_name}")` find related discussions, bug reports, feature requests
71
- 2. **Changelog entries:** `qmd_bridge.search("{function_name} changelog")` find version history, breaking changes
72
- 3. **Migration notes:** `qmd_bridge.vector_search("{function_name} migration deprecated")` — find deprecation or migration context
72
+ "**Enrichment: no enrichment collections available.**
73
+ Only brief-type or extraction-type QMD collections found — no temporal context (issues, PRs, changelogs) or docs collections indexed. Enrichment skipped (expected for first-run skill creation). Enrichment becomes available when temporal or docs context is indexed into QMD collections.
74
+
75
+ Proceeding to compilation..."
76
+
77
+ Then immediately load, read entire file, then execute `{nextStepFile}`.
78
+
79
+ 4. **If temporal collections exist:** Continue to step 3.
80
+
81
+ ### 3. QMD Enrichment Searches (Deep Tier Only)
82
+
83
+ For each major exported function — limited to the **top-level public API surface** (typically 10-20 functions that will appear in the context-snippet), not all extracted exports — search the temporal QMD collections for context:
84
+
85
+ **Search query construction:**
86
+
87
+ For each function, derive the **module context** from the extraction inventory's source file path (e.g., `src/graph/neo4j/index.ts` → module context `graph neo4j`). This context improves search relevance by scoping results to the function's subsystem without adding extra queries.
88
+
89
+ **Primary searches (BM25 — always runs, no GPU/VRAM dependency):**
90
+
91
+ 1. **Issues/PRs:** `qmd_bridge.search("{module_context} {function_name}")` — find related discussions scoped by module context
92
+ 2. **Changelog entries:** `qmd_bridge.search("{function_name} changelog")` — find version history, breaking changes (kept generic — changelogs rarely use module paths)
93
+ 3. **Migration notes:** `qmd_bridge.search("{function_name} migration deprecated breaking")` — find deprecation or migration context using exact keyword matching
94
+
95
+ **Supplemental search (best-effort — requires GPU/VRAM, may fail):**
96
+
97
+ 4. **Semantic migration context:** `qmd_bridge.vector_search("{module_context} {function_name} migration deprecated")` — adds semantic matches (synonyms, paraphrases) that BM25 keyword search may miss. Merge results with search #3, deduplicating by document ID. If `vector_search` fails (VRAM, GPU driver, model loading), discard silently — the BM25 results from search #3 provide baseline coverage.
73
98
 
74
99
  **For each QMD result:**
75
100
 
@@ -84,7 +109,7 @@ For each major exported function in the extraction inventory, search QMD collect
84
109
  - If individual search fails: skip that function's enrichment, continue with others
85
110
  - If QMD returns no results for a function: no annotation added (absence is normal)
86
111
 
87
- ### 3. Annotate Extraction Inventory
112
+ ### 4. Annotate Extraction Inventory
88
113
 
89
114
  Add enrichment annotations to the extraction inventory without modifying extraction data:
90
115
 
@@ -100,7 +125,7 @@ Add enrichment annotations to the extraction inventory without modifying extract
100
125
  - T2-past annotations: {count}
101
126
  - T2-future annotations: {count}
102
127
 
103
- ### 4. Report Enrichment (Deep Tier Only)
128
+ ### 5. Report Enrichment (Deep Tier Only)
104
129
 
105
130
  Display brief enrichment summary:
106
131
 
@@ -111,7 +136,7 @@ Display brief enrichment summary:
111
136
 
112
137
  Proceeding to compilation..."
113
138
 
114
- ### 5. Menu Handling Logic
139
+ ### 6. Menu Handling Logic
115
140
 
116
141
  **Auto-proceed step — no user interaction.**
117
142
 
@@ -135,7 +160,9 @@ ONLY WHEN enrichment is complete (Deep tier) or the step is skipped (Quick/Forge
135
160
  ### ✅ SUCCESS:
136
161
 
137
162
  - Quick/Forge tiers: skipped silently with no output, auto-proceeded
138
- - Deep tier: QMD searches performed per extracted function
163
+ - Deep tier: collection inventory checked before searching
164
+ - Deep tier: no temporal collections → reported and auto-proceeded (not silently skipped)
165
+ - Deep tier: temporal collections exist → QMD searches performed per public API function (10-20, not all exports) with module context derived from extraction inventory source paths
139
166
  - T2 annotations added with proper provenance citations
140
167
  - Extraction data unmodified — enrichment is additive only
141
168
  - QMD failures handled gracefully (skip and continue)
@@ -146,6 +173,9 @@ ONLY WHEN enrichment is complete (Deep tier) or the step is skipped (Quick/Forge
146
173
  - Halting on QMD unavailability or search failures
147
174
  - Modifying T1 extraction results with QMD data
148
175
  - Displaying skip messages for Quick/Forge tiers (should be silent)
176
+ - Pre-emptively skipping all searches without checking the collection inventory
177
+ - Searching all 800+ exports instead of scoping to public API surface (10-20)
178
+ - Using bare function names without module context when source path is available in extraction inventory
149
179
  - Beginning compilation or SKILL.md assembly in this step
150
180
  - Not labeling QMD annotations as T2 confidence
151
181
 
@@ -3,6 +3,7 @@ name: 'step-05-compile'
3
3
  description: 'Assemble SKILL.md, context-snippet, metadata, and references from extraction inventory'
4
4
  nextStepFile: './step-06-validate.md'
5
5
  skillSectionsData: '../data/skill-sections.md'
6
+ assemblyRulesData: '../data/compile-assembly-rules.md'
6
7
  ---
7
8
 
8
9
  # Step 5: Compile
@@ -30,102 +31,72 @@ To assemble the complete skill content from the extraction inventory and enrichm
30
31
 
31
32
  - 🎯 Focus ONLY on assembling content from extraction inventory + enrichment
32
33
  - 🚫 FORBIDDEN to include any content without a provenance citation
33
- - 🚫 FORBIDDEN to write files content stays in context until step-07
34
+ - 💾 Sub-agents and compilation processes may write to a staging directory (e.g., `_bmad-output/{name}/`)
35
+ - 🚫 FORBIDDEN to write final files to `skills/` or `forge-data/` — that's step-07
34
36
  - 🚫 FORBIDDEN to fabricate examples not found in source tests or docs
35
37
  - ⚒️ Seed `<!-- [MANUAL] -->` markers for future update-skill compatibility
36
38
 
37
39
  ## EXECUTION PROTOCOLS:
38
40
 
39
41
  - 🎯 Follow MANDATORY SEQUENCE exactly
40
- - 💾 Build all content in context do not write files
42
+ - 💾 Build all content in context or write to a staging directory (`_bmad-output/{name}/`)
41
43
  - 📖 Follow agentskills.io section structure from data file
42
- - 🚫 Do not write any output files — that's step-07
44
+ - 🚫 Do not promote any output files to final `skills/` or `forge-data/` directories — that's step-07
43
45
 
44
46
  ## CONTEXT BOUNDARIES:
45
47
 
46
- - Available: extraction_inventory, enrichment_annotations (if Deep), brief_data, tier
48
+ - Available: extraction_inventory, enrichment_annotations (if Deep), doc_fetch_inventory (if doc_urls), brief_data, tier
47
49
  - Focus: Assembling structured content from verified data
48
- - Limits: Do NOT write files, validate spec compliance, or report
50
+ - Docs-only mode: If `source_type: "docs-only"`, all content is T3 `[EXT:{url}]`. Overview must note "Generated from external documentation." metadata.json `source_authority` forced to `community`. No AST/source citations exist — only `[EXT:...]` citations.
51
+ - Limits: Do NOT write to final `skills/` or `forge-data/` directories, validate spec compliance, or report
49
52
  - Dependencies: Extraction inventory from step-03 (enrichment from step-04 if Deep)
50
53
 
51
54
  ## MANDATORY SEQUENCE
52
55
 
53
56
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
54
57
 
55
- ### 1. Load Section Structure
58
+ ### 1. Load Data Files
56
59
 
57
- Load `{skillSectionsData}` completely. This defines the agentskills.io-compliant format for all output artifacts.
60
+ Load `{skillSectionsData}` and `{assemblyRulesData}` completely. These define the agentskills.io-compliant format and detailed assembly rules for all output artifacts.
58
61
 
59
62
  ### 2. Build SKILL.md Content
60
63
 
61
- Assemble each section in order from the skill-sections data file:
62
-
63
- **Frontmatter:**
64
- ```yaml
65
- ---
66
- name: {brief.name}
67
- version: {brief.version}
68
- description: {brief.name} — {export_count} verified functions
69
- author: {source_author}
70
- ---
71
- ```
72
-
73
- **Section 1 — Overview:**
74
- - What this skill provides
75
- - Source repo, version, branch
76
- - Forge tier used for compilation
77
- - Export count and confidence summary
78
-
79
- **Section 2 — Quick Start:**
80
- - Select the 3-5 most commonly used functions (by import frequency or documentation prominence)
81
- - Minimal usage examples — ONLY from source tests or official docs
82
- - If no examples exist in source, show signature-only quick start
83
- - Provenance citation for each function
84
-
85
- **Section 3 — API Reference:**
86
- - Group by module/file
87
- - For each exported function:
88
- - Full signature with types
89
- - Parameters table (name, type, required/optional, description)
90
- - Return type
91
- - Usage example (from source tests/docs ONLY — omit if none found)
92
- - Provenance citation: `[AST:{file}:L{line}]` or `[SRC:{file}:L{line}]`
93
- - T2 annotations (Deep tier only): temporal context from enrichment
94
-
95
- **Section 4 — Type Definitions:**
96
- - All exported types, interfaces, enums
97
- - Full type signatures with provenance citations
98
-
99
- **Section 5 — Integration Patterns (Forge/Deep only):**
100
- - Co-import patterns detected by ast_bridge
101
- - Common usage combinations
102
- - Skip for Quick tier (no co-import data available)
103
-
104
- **Section 6 — Manual Sections:**
105
- - Seed empty `<!-- [MANUAL] -->` markers:
106
- ```markdown
107
- <!-- [MANUAL:additional-notes] -->
108
- <!-- Add custom notes here. This section is preserved during skill updates. -->
109
- <!-- [/MANUAL:additional-notes] -->
110
- ```
111
- - Place after Quick Start and after API Reference sections
64
+ Assemble each section in order using the assembly rules data file (`{assemblyRulesData}`). The data file specifies frontmatter format, Tier 1 section details (Sections 1-8), Tier 2 section details (Sections 9-11), and assembly ordering rules. Follow it exactly.
112
65
 
113
66
  ### 3. Build context-snippet.md Content
114
67
 
115
- Compressed 2-line format for CLAUDE.md managed section:
68
+ Vercel-aligned indexed format for CLAUDE.md managed section (~80-120 tokens):
116
69
 
117
70
  ```markdown
118
- {skill-name} -> skills/{skill-name}/
119
- exports: {comma-separated top 10 function names}
71
+ [{skill-name} v{version}]|root: skills/{skill-name}/
72
+ |IMPORTANT: {skill-name} v{version} read SKILL.md before writing {skill-name} code. Do NOT rely on training data.
73
+ |quick-start:{SKILL.md#quick-start}
74
+ |api: {top exports with () for functions, comma-separated}
75
+ |key-types:{SKILL.md#key-types} — {inline summary of most important type values}
76
+ |gotchas: {2-3 most critical pitfalls or breaking changes, inline}
120
77
  ```
121
78
 
79
+ **Derivation rules:**
80
+
81
+ - **version**: From source detection (reconciled in step-03), not brief default
82
+ - **api**: Top 10 exports from extraction inventory, append `()` to function names
83
+ - **key-types**: Inline summary of most important enum/type values from Key Types section
84
+ - **gotchas**: Derived from T2-future annotations (breaking changes), async requirements, version-specific behavior changes. If no gotchas available, omit the gotchas line.
85
+ - **Section anchors** (`#quick-start`, `#key-types`): Must match actual heading slugs in the assembled SKILL.md
86
+
122
87
  ### 4. Build metadata.json Content
123
88
 
124
89
  Following the structure from the skill-sections data file:
125
90
  - Populate all fields from brief_data, extraction inventory, and tier
126
- - Set `generated_at` to current ISO-8601 timestamp
91
+ - Set `generation_date` to current ISO-8601 timestamp
127
92
  - Set `source_commit` from resolved source (if available)
128
- - Set `stats` from extraction aggregate counts
93
+ - Set `stats` from extraction aggregate counts:
94
+ - `exports_documented`: count of exports with documentation in the assembled SKILL.md
95
+ - `exports_public_api`: count of exports from public entry points (`__init__.py`, `index.ts`, `lib.rs`, or equivalent)
96
+ - `exports_internal`: count of all other non-underscore-prefixed exports (internal modules, helpers, adapters)
97
+ - `exports_total`: `exports_public_api` + `exports_internal`
98
+ - `public_api_coverage`: `exports_documented / exports_public_api` (1.0 when all public API exports are documented; `null` if `exports_public_api` is 0)
99
+ - `total_coverage`: `exports_documented / exports_total` (may be low for large codebases — this is expected; `null` if `exports_total` is 0)
129
100
  - Set `tool_versions` based on tier and available tools
130
101
 
131
102
  ### 5. Build references/ Content
@@ -140,37 +111,27 @@ Group functions logically by module, file, or functional area.
140
111
 
141
112
  ### 6. Build provenance-map.json Content
142
113
 
143
- One entry per extracted claim:
144
- - claim text
145
- - source_file and source_line
146
- - confidence tier (T1, T1-low, T2)
147
- - extraction_method (ast_bridge.scan_definitions, gh_bridge.read_file, qmd_bridge.search)
148
- - ast_node_type (if AST-extracted)
114
+ One entry per extracted export: export_name, export_type, params[] (typed strings), return_type, source_file, source_line, confidence tier (T1/T1-low/T2), extraction_method, ast_node_type. See `{skillSectionsData}` for full schema.
149
115
 
150
116
  ### 7. Build evidence-report.md Content
151
117
 
152
- Compilation audit trail:
153
- - Generation date, forge tier, source info
154
- - Tool versions used
155
- - Extraction summary (files scanned, exports found, confidence breakdown)
156
- - Validation results (populated in step-06)
157
- - Warnings from extraction or enrichment
118
+ Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template.
158
119
 
159
120
  ### 8. Menu Handling Logic
160
121
 
161
122
  **Auto-proceed step — no user interaction.**
162
123
 
163
- After all content is assembled in context, immediately load, read entire file, then execute `{nextStepFile}`.
124
+ After all content is assembled in context (or written to the staging directory), immediately load, read entire file, then execute `{nextStepFile}`.
164
125
 
165
126
  #### EXECUTION RULES:
166
127
 
167
128
  - This is an auto-proceed assembly step with no user choices
168
- - All content stays in context — no files are written yet
129
+ - All content stays in context or in the staging directory — no final files are written yet
169
130
  - Proceed directly to validation after assembly is complete
170
131
 
171
132
  ## CRITICAL STEP COMPLETION NOTE
172
133
 
173
- ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md, extraction-rules.yaml) are assembled in context will you proceed to load `{nextStepFile}` for spec validation.
134
+ ONLY WHEN all 6 content artifacts (SKILL.md, context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md) are assembled in context will you proceed to load `{nextStepFile}` for spec validation. Note: extraction-rules.yaml is generated by step-07 from extraction data, not assembled here.
174
135
 
175
136
  ---
176
137
 
@@ -178,8 +139,11 @@ ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json,
178
139
 
179
140
  ### ✅ SUCCESS:
180
141
 
181
- - Skill-sections data file loaded and followed
142
+ - Skill-sections data file and assembly rules data file loaded and followed
182
143
  - SKILL.md assembled with all required sections in agentskills.io format
144
+ - Tier 1 sections assembled first and kept under 300 lines (excluding frontmatter)
145
+ - Tier 2 sections assembled after all Tier 1 sections
146
+ - Section 4b emitted only when Deep tier AND T2-future annotations exist; omitted entirely otherwise (no empty section)
183
147
  - Every function entry has a provenance citation
184
148
  - [MANUAL] markers seeded for update-skill compatibility
185
149
  - context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md all assembled
@@ -190,9 +154,12 @@ ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json,
190
154
 
191
155
  - Including functions or examples without provenance citations
192
156
  - Fabricating usage examples not found in source
193
- - Writing files to disk (that's step-07)
157
+ - Writing files to final `skills/` or `forge-data/` directories (that's step-07)
194
158
  - Missing required SKILL.md sections
159
+ - Emitting an empty Section 4b when no T2-future annotations exist
160
+ - Tier 1 content exceeding 300 lines without reduction
161
+ - Assembling Tier 2 sections before Tier 1 sections
195
162
  - Not seeding [MANUAL] markers
196
- - Not building all 7 content artifacts
163
+ - Not building all 6 content artifacts (extraction-rules.yaml is built by step-07)
197
164
 
198
165
  **Master Rule:** Zero hallucination — every line of SKILL.md must trace to source code. Compile from data, not imagination.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: 'step-06-validate'
3
- description: 'Validate compiled skill content against agentskills.io spec via skills_ref'
3
+ description: 'Validate compiled skill content against agentskills.io spec via skill-check'
4
4
  nextStepFile: './step-07-generate-artifacts.md'
5
5
  ---
6
6
 
@@ -8,7 +8,7 @@ nextStepFile: './step-07-generate-artifacts.md'
8
8
 
9
9
  ## STEP GOAL:
10
10
 
11
- To validate the compiled SKILL.md content against the agentskills.io specification using skills_ref, auto-fix any validation failures, and confirm spec compliance before artifact generation.
11
+ To validate the compiled SKILL.md content against the agentskills.io specification using skill-check, auto-fix any validation failures, and confirm spec compliance before artifact generation.
12
12
 
13
13
  ## MANDATORY EXECUTION RULES (READ FIRST):
14
14
 
@@ -29,9 +29,10 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
29
29
 
30
30
  - 🎯 Focus ONLY on validating compiled content against spec
31
31
  - 🚫 FORBIDDEN to add new content — only fix spec compliance issues
32
- - 🚫 FORBIDDEN to write files content stays in context until step-07
32
+ - 💾 Validation and auto-fix modify files in the staging directory (`_bmad-output/{name}/`)
33
33
  - 💬 If auto-fix fails, report issues clearly but proceed (warn, don't halt)
34
- - ⚙️ If skills_ref unavailable: skip validation, add warning to evidence report
34
+ - ⚙️ If skill-check unavailable: skip validation, add warning to evidence report
35
+ - ⚠️ Ignore non-zero exit codes from `skill-check` if the JSON output shows 0 errors — parse JSON output, not exit codes
35
36
 
36
37
  ## EXECUTION PROTOCOLS:
37
38
 
@@ -53,85 +54,169 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
53
54
 
54
55
  ### 1. Check Tool Availability
55
56
 
56
- **If skills_ref tool is available:**
57
- Continue to validation steps.
57
+ Run: `npx skill-check -h`
58
58
 
59
- **If skills_ref tool is NOT available:**
60
- - Add warning to evidence-report content: "Spec validation skippedskills_ref tool unavailable"
61
- - Auto-proceed to next step
59
+ - If succeeds: Continue to automated validation (section 2)
60
+ - If fails: Perform manual fallback (section 3); add note to evidence-report: "Spec validation performed manually skill-check tool unavailable"
62
61
 
63
- ### 2. Validate Schema
62
+ **Important:** Do not assume availability — empirical check required.
64
63
 
65
- Use `skills_ref.validate_schema()` against the compiled SKILL.md content.
64
+ ### 2. Validate & Auto-Fix (skill-check check --fix)
66
65
 
67
- **Check:**
68
- - Required sections present (Overview, Quick Start, API Reference, Type Definitions)
69
- - Section order follows agentskills.io standard
70
- - Frontmatter contains required fields (name, version, description, author)
71
- - Provenance citations present in API Reference entries
66
+ Run the external skill-check tool against the compiled skill staging directory:
72
67
 
73
- **If validation passes:** Record "Schema: PASS" in evidence-report content.
68
+ ```bash
69
+ npx skill-check check <staging-skill-dir> --fix --format json --no-security-scan
70
+ ```
71
+
72
+ This performs frontmatter validation, description quality checks, body limit enforcement, local link resolution, file formatting, auto-fix of deterministic issues, and quality scoring (0-100) across five weighted categories.
73
+
74
+ **Parse the JSON output** for: `qualityScore` (0-100), `diagnostics[]` (remaining issues), `fixed[]` (auto-corrected issues).
75
+
76
+ **Note:** `skill-check` may return non-zero exit code even when `errorCount` is 0. Always rely on parsed JSON, not the shell exit code.
77
+
78
+ - **Score ≥ 70:** Record "Schema: PASS (score: {score}/100)" in evidence-report
79
+ - **Score < 70:** Log remaining diagnostics as warnings, record "Schema: WARN — score {score}/100, {count} remaining issues", proceed
80
+ - **Unfixable errors:** Record specific rule IDs and suggestions, proceed with warnings
81
+
82
+ ### 3. Validate Frontmatter (Fallback)
83
+
84
+ **If skill-check was available:** Skip — already validated in step 2.
85
+
86
+ **If skill-check NOT available (fallback):** Perform manual frontmatter compliance check:
87
+
88
+ - [ ] Frontmatter present — file starts with `---` and has closing `---`
89
+ - [ ] `name` field — present, non-empty, lowercase alphanumeric + hyphens only, 1-64 chars
90
+ - [ ] `name` matches skill output directory name
91
+ - [ ] `description` field — present, non-empty, 1-1024 characters
92
+ - [ ] No unknown fields — only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` permitted
93
+ - [ ] `version` and `author` are NOT in frontmatter (they belong in metadata.json)
94
+
95
+ If fails: auto-fix (deterministic), re-validate once, record result. If passes: record "Frontmatter: PASS".
96
+
97
+ ### 4. Split Oversized Body (if needed)
98
+
99
+ **If step 2 reported `body.max_lines` failure:**
100
+
101
+ ```bash
102
+ npx skill-check split-body <staging-skill-dir> --write
103
+ ```
104
+
105
+ Then re-validate: `npx skill-check check <staging-skill-dir> --format json --no-security-scan`
106
+
107
+ **If skill-check unavailable or no body size issue:** Skip.
108
+
109
+ ### 5. Security Scan
110
+
111
+ **If skill-check available:**
112
+
113
+ ```bash
114
+ npx skill-check check <staging-skill-dir> --format json
115
+ ```
74
116
 
75
- **If validation fails:**
76
- 1. Identify specific failures
77
- 2. Attempt auto-fix (structural adjustments only — never invent content)
78
- 3. Re-validate once
79
- 4. If still failing: record "Schema: FAIL — {specific issues}" in evidence-report, add warnings, but proceed
117
+ (Security scan enabled by default when `--no-security-scan` omitted. The scan uses [Snyk Agent Scan](https://github.com/snyk/agent-scan) to check for prompt injection risks, sensitive data exposure, and unsafe tool permissions.)
80
118
 
81
- ### 3. Validate Frontmatter
119
+ Record any security warnings in evidence-report. Security findings are advisory — they do not block artifact generation.
82
120
 
83
- Use `skills_ref.validate_frontmatter()` against the SKILL.md frontmatter.
121
+ **If security scan fails due to missing SNYK_TOKEN:**
84
122
 
85
- **Check:**
86
- - `name` matches brief.name (kebab-case)
87
- - `version` matches brief.version
88
- - `description` follows format: "{name} — {count} verified functions"
89
- - `author` is set
123
+ Display actionable guidance:
90
124
 
91
- **If validation passes:** Record "Frontmatter: PASS" in evidence-report content.
125
+ "Security scan requires a Snyk API token. **Important:** The Snyk API requires an Enterprise plan ([authentication docs](https://docs.snyk.io/snyk-api/authentication-for-api)). To enable:
126
+ 1. Obtain a Snyk Enterprise account with API access
127
+ 2. Get your API token from Account Settings → API Token
128
+ 3. Set `SNYK_TOKEN=your-token` in your environment or `.env`
129
+ 4. Re-run [SF] Setup Forge to update tool detection, then re-run [CS] Create Skill
92
130
 
93
- **If validation fails:**
94
- 1. Auto-fix frontmatter (these are deterministic fixes)
95
- 2. Re-validate once
96
- 3. Record result in evidence-report
131
+ If you don't have an Enterprise account, use `--no-security-scan` flag in skill-check to skip, or `--security-scan-runner pipx|uvx|local` to control the execution method. Security scanning is optional and does not affect tier level or block skill compilation."
97
132
 
98
- ### 4. Validate metadata.json
133
+ Record: "Security scan skipped — SNYK_TOKEN not configured"
99
134
 
100
- Cross-check metadata.json content against extraction inventory:
101
- - `stats.exports_documented` matches actual documented exports
102
- - `stats.exports_total` matches total extracted exports
103
- - `stats.coverage` is accurate (documented / total)
135
+ **If skill-check unavailable:** Skip with note: "Security scan skipped — skill-check tool unavailable"
136
+
137
+ ### 6. Content Quality Review (tessl)
138
+
139
+ **If tessl available**, run: `npx -y tessl skill review <staging-skill-dir>`
140
+
141
+ Parse output for: `description_score`, `content_score`, `average_score`, `validation_result`, `judge_suggestions[]`.
142
+
143
+ - **Content score < 70%:** Record warning: "Content quality warning: tessl scored content at {score}%."
144
+ - **Unavailable:** Skip with note: "Content quality review skipped — tessl tool unavailable"
145
+
146
+ **Expected scoring for two-tier design:** The Skill Forge two-tier design (Tier 1 Key API Summary + Tier 2 Full API Reference) intentionally includes progressive disclosure. tessl's `conciseness` scorer may flag this as redundancy (typically scoring 2/3), which is expected behavior — not a defect. Content scores >= 60% are acceptable when both tiers are correctly applied with proper differentiation (see step-05 Tier 2 assembly rules). Do NOT consolidate Tier 1 and Tier 2 content to improve the score — the two-tier structure is a deliberate design choice for standalone usability.
147
+
148
+ tessl installs automatically via `npx`. A missing tool is not an error — graceful skip.
149
+
150
+ #### 6b. User Decision Gate (conditional)
151
+
152
+ **If tessl returned no suggestions OR tessl was unavailable:** Skip this gate — auto-proceed.
153
+
154
+ **If tessl returned suggestions**, present them to the user:
155
+
156
+ "**Content quality review: {score}%**
157
+
158
+ tessl suggestions:
159
+ {numbered list of judge_suggestions}
160
+
161
+ **Select an option:**
162
+ - **[S] Skip** — proceed with current content as-is (default)
163
+ - **[A] Apply structural fixes** — apply only structural suggestions (split sections, consolidate duplicates). No new content generated.
164
+ - **[R] Review all** — show each suggestion with proposed changes before applying"
165
+
166
+ #### Gate Rules:
167
+
168
+ - **Structural suggestions** (split reference section, consolidate duplicates, reorder sections) can be applied without zero-hallucination risk — they restructure existing content
169
+ - **Semantic suggestions** (add examples, add error handling, add validation checkpoints) introduce content not verified from source code. If the user chooses to apply these:
170
+ - Warn: "This adds content not verified from source code."
171
+ - Mark applied content with `<!-- [TESSL:auto-fix] -->` markers
172
+ - Cite as `[TESSL:suggestion]` in the provenance map with `confidence: "TESSL"` (below T3)
173
+ - Record in evidence report: "TESSL-suggested content applied: {count} items (unverified)"
174
+ - **If user selects [S]:** Record "tessl suggestions: skipped by user" in evidence report. Proceed to section 7.
175
+ - **If user selects [A]:** Apply structural fixes only, re-run tessl to capture updated score, record results. Proceed to section 7.
176
+ - **If user selects [R]:** Show each suggestion with the proposed change. For each, user confirms or skips. Apply confirmed changes, record results. Proceed to section 7.
177
+
178
+ ### 7. Validate metadata.json
179
+
180
+ Cross-check metadata.json against extraction inventory:
181
+ - `stats.exports_documented` / `stats.exports_public_api` / `stats.exports_internal` / `stats.exports_total` are accurate
182
+ - `stats.public_api_coverage` and `stats.total_coverage` are correctly computed (null when denominator is 0)
104
183
  - `confidence_t1`, `confidence_t2`, `confidence_t3` match actual counts
105
184
  - `spec_version` is "1.3"
106
185
 
107
186
  Auto-fix any discrepancies (these are computed values).
108
187
 
109
- ### 5. Update Evidence Report
188
+ ### 8. Update Evidence Report
110
189
 
111
- Add validation results to the evidence-report content in context:
190
+ Add validation results to evidence-report content in context:
112
191
 
113
192
  ```markdown
114
193
  ## Validation Results
115
- - Schema: {pass/fail}
194
+ - Schema: {pass/fail} (quality score: {score}/100)
116
195
  - Frontmatter: {pass/fail}
196
+ - Body: {pass/fail} {split-body applied if applicable}
197
+ - Security: {pass/warn/skipped}
198
+ - Content Quality (tessl): {pass/warn/skipped} (score: {score}%)
117
199
  - Metadata: {pass/fail}
118
200
 
119
- ## Warnings
120
- - {any warnings from validation}
121
- ```
201
+ ## Quality Score Breakdown
202
+ - Frontmatter (30%): {score} | Description (30%): {score} | Body (20%): {score} | Links (10%): {score} | File (10%): {score}
122
203
 
123
- ### 6. Menu Handling Logic
204
+ ## Auto-Fixed Issues
205
+ - {list of issues automatically corrected by --fix}
206
+
207
+ ## Remaining Warnings / Security Findings / Content Quality (tessl)
208
+ - {warnings, security results, tessl scores and suggestions — or "skipped"}
209
+ ```
124
210
 
125
- **Auto-proceed step no user interaction.**
211
+ ### 9. Menu Handling Logic
126
212
 
127
- After validation is complete (or skipped if tools unavailable), immediately load, read entire file, then execute `{nextStepFile}`.
213
+ **Conditional interaction step.** If tessl produced suggestions, section 6b halts for user input. Otherwise, auto-proceed.
128
214
 
129
- #### EXECUTION RULES:
215
+ After validation completes (including any user decisions from section 6b), immediately load, read entire file, then execute `{nextStepFile}`.
130
216
 
131
- - This is an auto-proceed validation step with no user choices
132
217
  - Tool unavailability is a skip, not a halt
133
218
  - Validation failures are warnings — proceed to artifact generation
134
- - Proceed directly to next step after validation completes
219
+ - tessl gate only triggers when suggestions exist — no gate for clean reviews or unavailable tools
135
220
 
136
221
  ## CRITICAL STEP COMPLETION NOTE
137
222
 
@@ -143,20 +228,21 @@ ONLY WHEN validation is complete (or skipped) and evidence-report content is upd
143
228
 
144
229
  ### ✅ SUCCESS:
145
230
 
146
- - Schema validation attempted (or skipped with warning if tool unavailable)
147
- - Frontmatter validation attempted
148
- - Metadata cross-check performed
149
- - Auto-fix applied for deterministic failures
150
- - Evidence report updated with validation results
151
- - Warnings recorded for any failures
152
- - Auto-proceeded to step-07
231
+ - `npx skill-check check --fix --format json` executed (or skipped with warning if unavailable)
232
+ - Quality score captured and recorded; auto-fix applied for deterministic issues
233
+ - Split-body applied if `body.max_lines` failed; security scan executed (or skipped with warning)
234
+ - `npx -y tessl skill review` executed (or skipped); content quality warning raised if score < 70%
235
+ - tessl suggestions presented to user when available; user decision recorded
236
+ - TESSL-applied content marked with `<!-- [TESSL:auto-fix] -->` and cited as `[TESSL:suggestion]`
237
+ - Metadata cross-check performed; evidence report updated with structured results
238
+ - Proceeded to step-07 (auto or after user gate)
153
239
 
154
240
  ### ❌ SYSTEM FAILURE:
155
241
 
156
- - Halting the workflow on validation failure (should warn and proceed)
157
- - Halting on skills_ref unavailability (should skip with warning)
158
- - Adding new content during validation (only structural fixes allowed)
159
- - Not recording validation results in evidence report
242
+ - Halting on validation failure or skill-check unavailability (should warn and proceed)
243
+ - Adding new content during validation without user approval via the tessl gate
244
+ - Applying semantic tessl suggestions without warning the user about unverified content
245
+ - Not recording quality score; skipping security scan without recording the skip
160
246
  - Attempting more than one auto-fix cycle per failure
161
247
 
162
- **Master Rule:** Validation informs, it does not block. Record results, fix what's deterministic, warn about the rest, and proceed.
248
+ **Master Rule:** Validation informs, it does not block. Record results, fix what's deterministic, scan for security issues, warn about the rest, and proceed.