bmad-module-skill-forge 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (115) hide show
  1. package/README.md +73 -159
  2. package/docs/agents.md +2 -2
  3. package/docs/architecture.md +155 -25
  4. package/docs/concepts.md +96 -0
  5. package/docs/examples.md +52 -2
  6. package/docs/getting-started.md +30 -14
  7. package/docs/index.md +16 -41
  8. package/docs/workflows.md +3 -3
  9. package/package.json +6 -4
  10. package/src/forger/forge-tier.yaml +1 -1
  11. package/src/forger/preferences.yaml +8 -1
  12. package/src/knowledge/doc-fetcher.md +55 -0
  13. package/src/knowledge/overview.md +17 -16
  14. package/src/knowledge/progressive-capability.md +9 -1
  15. package/src/knowledge/provenance-tracking.md +31 -23
  16. package/src/knowledge/qmd-registry.md +132 -0
  17. package/src/knowledge/skf-knowledge-index.csv +2 -0
  18. package/src/module.yaml +2 -2
  19. package/src/workflows/analyze-source/data/skill-brief-schema.md +24 -4
  20. package/src/workflows/analyze-source/steps-c/step-01-init.md +19 -9
  21. package/src/workflows/analyze-source/steps-c/step-01b-continue.md +2 -2
  22. package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +10 -2
  23. package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +17 -2
  24. package/src/workflows/analyze-source/steps-c/step-05-recommend.md +2 -1
  25. package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +3 -3
  26. package/src/workflows/analyze-source/templates/analysis-report-template.md +1 -1
  27. package/src/workflows/analyze-source/validation-report.md +1 -1
  28. package/src/workflows/analyze-source/workflow-plan-analyze-source.md +1 -1
  29. package/src/workflows/analyze-source/workflow.md +1 -1
  30. package/src/workflows/audit-skill/steps-c/step-01-init.md +8 -5
  31. package/src/workflows/audit-skill/steps-c/step-02-re-index.md +12 -2
  32. package/src/workflows/audit-skill/validation-report.md +2 -2
  33. package/src/workflows/audit-skill/workflow-plan-audit-skill.md +1 -1
  34. package/src/workflows/brief-skill/data/scope-templates.md +60 -0
  35. package/src/workflows/brief-skill/data/skill-brief-schema.md +38 -3
  36. package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +20 -4
  37. package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +20 -64
  38. package/src/workflows/brief-skill/steps-c/step-04-confirm-brief.md +1 -1
  39. package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +60 -5
  40. package/src/workflows/brief-skill/workflow.md +2 -1
  41. package/src/workflows/create-skill/data/compile-assembly-rules.md +110 -0
  42. package/src/workflows/create-skill/data/extraction-patterns.md +216 -3
  43. package/src/workflows/create-skill/data/skill-sections.md +81 -27
  44. package/src/workflows/create-skill/data/source-resolution-protocols.md +138 -0
  45. package/src/workflows/create-skill/data/tier-degradation-rules.md +46 -0
  46. package/src/workflows/create-skill/steps-c/step-01-load-brief.md +7 -4
  47. package/src/workflows/create-skill/steps-c/step-02-ecosystem-check.md +10 -7
  48. package/src/workflows/create-skill/steps-c/step-03-extract.md +51 -10
  49. package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +238 -0
  50. package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +220 -0
  51. package/src/workflows/create-skill/steps-c/step-04-enrich.md +40 -10
  52. package/src/workflows/create-skill/steps-c/step-05-compile.md +49 -82
  53. package/src/workflows/create-skill/steps-c/step-06-validate.md +152 -63
  54. package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +53 -3
  55. package/src/workflows/create-skill/steps-c/step-08-report.md +1 -1
  56. package/src/workflows/create-skill/workflow-plan-create-skill.md +42 -10
  57. package/src/workflows/create-skill/workflow.md +3 -2
  58. package/src/workflows/create-stack-skill/data/stack-skill-template.md +20 -7
  59. package/src/workflows/create-stack-skill/steps-c/step-01-init.md +2 -2
  60. package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +14 -0
  61. package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +28 -7
  62. package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +10 -7
  63. package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +66 -68
  64. package/src/workflows/create-stack-skill/validation-report.md +1 -1
  65. package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +4 -4
  66. package/src/workflows/export-skill/data/managed-section-format.md +1 -0
  67. package/src/workflows/export-skill/data/snippet-format.md +33 -14
  68. package/src/workflows/export-skill/steps-c/step-01-load-skill.md +16 -2
  69. package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +29 -22
  70. package/src/workflows/export-skill/steps-c/step-04-update-context.md +1 -1
  71. package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
  72. package/src/workflows/export-skill/validation-report.md +1 -1
  73. package/src/workflows/export-skill/workflow-plan-export-skill.md +8 -8
  74. package/src/workflows/export-skill/workflow.md +1 -1
  75. package/src/workflows/quick-skill/data/skill-template.md +27 -7
  76. package/src/workflows/quick-skill/steps-c/step-04-compile.md +44 -13
  77. package/src/workflows/quick-skill/steps-c/step-05-validate.md +79 -16
  78. package/src/workflows/quick-skill/validation-report.md +4 -4
  79. package/src/workflows/quick-skill/workflow-plan-quick-skill.md +4 -4
  80. package/src/workflows/quick-skill/workflow.md +2 -0
  81. package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +12 -3
  82. package/src/workflows/setup-forge/steps-c/step-02-write-config.md +10 -2
  83. package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +111 -40
  84. package/src/workflows/setup-forge/steps-c/step-04-report.md +13 -0
  85. package/src/workflows/setup-forge/workflow.md +1 -0
  86. package/src/workflows/test-skill/data/scoring-rules.md +14 -6
  87. package/src/workflows/test-skill/data/source-access-protocol.md +45 -0
  88. package/src/workflows/test-skill/steps-c/step-01-init.md +38 -13
  89. package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -3
  90. package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +25 -5
  91. package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +22 -47
  92. package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +219 -0
  93. package/src/workflows/test-skill/steps-c/step-05-score.md +25 -27
  94. package/src/workflows/test-skill/steps-c/step-06-report.md +9 -4
  95. package/src/workflows/test-skill/templates/test-report-template.md +2 -1
  96. package/src/workflows/test-skill/validation-report.md +1 -1
  97. package/src/workflows/test-skill/workflow-plan-test-skill.md +7 -3
  98. package/src/workflows/test-skill/workflow.md +1 -1
  99. package/src/workflows/update-skill/data/remote-source-resolution.md +43 -0
  100. package/src/workflows/update-skill/data/tier-degradation-rules.md +46 -0
  101. package/src/workflows/update-skill/steps-c/step-01-init.md +13 -7
  102. package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +26 -0
  103. package/src/workflows/update-skill/steps-c/step-03-re-extract.md +38 -9
  104. package/src/workflows/update-skill/steps-c/step-04-merge.md +3 -3
  105. package/src/workflows/update-skill/steps-c/step-05-validate.md +63 -67
  106. package/src/workflows/update-skill/steps-c/step-06-write.md +37 -14
  107. package/src/workflows/update-skill/steps-c/step-07-report.md +1 -1
  108. package/src/workflows/update-skill/validation-report.md +4 -4
  109. package/src/workflows/update-skill/workflow-plan-update-skill.md +4 -4
  110. package/src/workflows/update-skill/workflow.md +1 -1
  111. package/tools/cli/commands/status.js +4 -4
  112. package/tools/cli/commands/uninstall.js +1 -1
  113. package/tools/cli/commands/update.js +2 -2
  114. package/tools/cli/lib/ui.js +60 -16
  115. package/tools/cli/lib/version-check.js +2 -2
@@ -61,15 +61,40 @@ Auto-proceed silently. Display no message. Immediately load, read entire file, t
61
61
 
62
62
  Continue to step 2.
63
63
 
64
- ### 2. QMD Enrichment Searches (Deep Tier Only)
64
+ ### 2. Collection Inventory Pre-check (Deep Tier Only)
65
65
 
66
- For each major exported function in the extraction inventory, search QMD collections for temporal context:
66
+ Before searching, check which QMD collections are available:
67
67
 
68
- **Search targets per function:**
68
+ 1. Read the sidecar `forge-tier.yaml` to get registered `qmd_collections` entries
69
+ 2. Identify which collections contain enrichment context — collections with `type` equal to `"temporal"` (issues, PRs, changelogs) or `"docs"` (fetched external documentation). Do NOT include `"extraction"` or `"brief"` type collections.
70
+ 3. **If no enrichment collections exist** (no `"temporal"` or `"docs"` type collections): report this and auto-proceed. Display:
69
71
 
70
- 1. **Issues/PRs:** `qmd_bridge.search("{function_name}")` find related discussions, bug reports, feature requests
71
- 2. **Changelog entries:** `qmd_bridge.search("{function_name} changelog")` find version history, breaking changes
72
- 3. **Migration notes:** `qmd_bridge.vector_search("{function_name} migration deprecated")` — find deprecation or migration context
72
+ "**Enrichment: no enrichment collections available.**
73
+ Only brief-type or extraction-type QMD collections found — no temporal context (issues, PRs, changelogs) or docs collections indexed. Enrichment skipped (expected for first-run skill creation). Enrichment becomes available when temporal or docs context is indexed into QMD collections.
74
+
75
+ Proceeding to compilation..."
76
+
77
+ Then immediately load, read entire file, then execute `{nextStepFile}`.
78
+
79
+ 4. **If temporal collections exist:** Continue to step 3.
80
+
81
+ ### 3. QMD Enrichment Searches (Deep Tier Only)
82
+
83
+ For each major exported function — limited to the **top-level public API surface** (typically 10-20 functions that will appear in the context-snippet), not all extracted exports — search the temporal QMD collections for context:
84
+
85
+ **Search query construction:**
86
+
87
+ For each function, derive the **module context** from the extraction inventory's source file path (e.g., `src/graph/neo4j/index.ts` → module context `graph neo4j`). This context improves search relevance by scoping results to the function's subsystem without adding extra queries.
88
+
89
+ **Primary searches (BM25 — always runs, no GPU/VRAM dependency):**
90
+
91
+ 1. **Issues/PRs:** `qmd_bridge.search("{module_context} {function_name}")` — find related discussions scoped by module context
92
+ 2. **Changelog entries:** `qmd_bridge.search("{function_name} changelog")` — find version history, breaking changes (kept generic — changelogs rarely use module paths)
93
+ 3. **Migration notes:** `qmd_bridge.search("{function_name} migration deprecated breaking")` — find deprecation or migration context using exact keyword matching
94
+
95
+ **Supplemental search (best-effort — requires GPU/VRAM, may fail):**
96
+
97
+ 4. **Semantic migration context:** `qmd_bridge.vector_search("{module_context} {function_name} migration deprecated")` — adds semantic matches (synonyms, paraphrases) that BM25 keyword search may miss. Merge results with search #3, deduplicating by document ID. If `vector_search` fails (VRAM, GPU driver, model loading), discard silently — the BM25 results from search #3 provide baseline coverage.
73
98
 
74
99
  **For each QMD result:**
75
100
 
@@ -84,7 +109,7 @@ For each major exported function in the extraction inventory, search QMD collect
84
109
  - If individual search fails: skip that function's enrichment, continue with others
85
110
  - If QMD returns no results for a function: no annotation added (absence is normal)
86
111
 
87
- ### 3. Annotate Extraction Inventory
112
+ ### 4. Annotate Extraction Inventory
88
113
 
89
114
  Add enrichment annotations to the extraction inventory without modifying extraction data:
90
115
 
@@ -100,7 +125,7 @@ Add enrichment annotations to the extraction inventory without modifying extract
100
125
  - T2-past annotations: {count}
101
126
  - T2-future annotations: {count}
102
127
 
103
- ### 4. Report Enrichment (Deep Tier Only)
128
+ ### 5. Report Enrichment (Deep Tier Only)
104
129
 
105
130
  Display brief enrichment summary:
106
131
 
@@ -111,7 +136,7 @@ Display brief enrichment summary:
111
136
 
112
137
  Proceeding to compilation..."
113
138
 
114
- ### 5. Menu Handling Logic
139
+ ### 6. Menu Handling Logic
115
140
 
116
141
  **Auto-proceed step — no user interaction.**
117
142
 
@@ -135,7 +160,9 @@ ONLY WHEN enrichment is complete (Deep tier) or the step is skipped (Quick/Forge
135
160
  ### ✅ SUCCESS:
136
161
 
137
162
  - Quick/Forge tiers: skipped silently with no output, auto-proceeded
138
- - Deep tier: QMD searches performed per extracted function
163
+ - Deep tier: collection inventory checked before searching
164
+ - Deep tier: no temporal collections → reported and auto-proceeded (not silently skipped)
165
+ - Deep tier: temporal collections exist → QMD searches performed per public API function (10-20, not all exports) with module context derived from extraction inventory source paths
139
166
  - T2 annotations added with proper provenance citations
140
167
  - Extraction data unmodified — enrichment is additive only
141
168
  - QMD failures handled gracefully (skip and continue)
@@ -146,6 +173,9 @@ ONLY WHEN enrichment is complete (Deep tier) or the step is skipped (Quick/Forge
146
173
  - Halting on QMD unavailability or search failures
147
174
  - Modifying T1 extraction results with QMD data
148
175
  - Displaying skip messages for Quick/Forge tiers (should be silent)
176
+ - Pre-emptively skipping all searches without checking the collection inventory
177
+ - Searching all 800+ exports instead of scoping to public API surface (10-20)
178
+ - Using bare function names without module context when source path is available in extraction inventory
149
179
  - Beginning compilation or SKILL.md assembly in this step
150
180
  - Not labeling QMD annotations as T2 confidence
151
181
 
@@ -3,6 +3,7 @@ name: 'step-05-compile'
3
3
  description: 'Assemble SKILL.md, context-snippet, metadata, and references from extraction inventory'
4
4
  nextStepFile: './step-06-validate.md'
5
5
  skillSectionsData: '../data/skill-sections.md'
6
+ assemblyRulesData: '../data/compile-assembly-rules.md'
6
7
  ---
7
8
 
8
9
  # Step 5: Compile
@@ -30,103 +31,73 @@ To assemble the complete skill content from the extraction inventory and enrichm
30
31
 
31
32
  - 🎯 Focus ONLY on assembling content from extraction inventory + enrichment
32
33
  - 🚫 FORBIDDEN to include any content without a provenance citation
33
- - 🚫 FORBIDDEN to write files content stays in context until step-07
34
+ - 💾 Sub-agents and compilation processes may write to a staging directory (e.g., `_bmad-output/{name}/`)
35
+ - 🚫 FORBIDDEN to write final files to `skills/` or `forge-data/` — that's step-07
34
36
  - 🚫 FORBIDDEN to fabricate examples not found in source tests or docs
35
37
  - ⚒️ Seed `<!-- [MANUAL] -->` markers for future update-skill compatibility
36
38
 
37
39
  ## EXECUTION PROTOCOLS:
38
40
 
39
41
  - 🎯 Follow MANDATORY SEQUENCE exactly
40
- - 💾 Build all content in context do not write files
42
+ - 💾 Build all content in context or write to a staging directory (`_bmad-output/{name}/`)
41
43
  - 📖 Follow agentskills.io section structure from data file
42
- - 🚫 Do not write any output files — that's step-07
44
+ - 🚫 Do not promote any output files to final `skills/` or `forge-data/` directories — that's step-07
43
45
 
44
46
  ## CONTEXT BOUNDARIES:
45
47
 
46
- - Available: extraction_inventory, enrichment_annotations (if Deep), brief_data, tier
48
+ - Available: extraction_inventory, enrichment_annotations (if Deep), doc_fetch_inventory (if doc_urls), brief_data, tier
47
49
  - Focus: Assembling structured content from verified data
48
- - Limits: Do NOT write files, validate spec compliance, or report
50
+ - Docs-only mode: If `source_type: "docs-only"`, all content is T3 `[EXT:{url}]`. Overview must note "Generated from external documentation." metadata.json `source_authority` forced to `community`. No AST/source citations exist — only `[EXT:...]` citations.
51
+ - Limits: Do NOT write to final `skills/` or `forge-data/` directories, validate spec compliance, or report
49
52
  - Dependencies: Extraction inventory from step-03 (enrichment from step-04 if Deep)
50
53
 
51
54
  ## MANDATORY SEQUENCE
52
55
 
53
56
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
54
57
 
55
- ### 1. Load Section Structure
58
+ ### 1. Load Data Files
56
59
 
57
- Load `{skillSectionsData}` completely. This defines the agentskills.io-compliant format for all output artifacts.
60
+ Load `{skillSectionsData}` and `{assemblyRulesData}` completely. These define the agentskills.io-compliant format and detailed assembly rules for all output artifacts.
58
61
 
59
62
  ### 2. Build SKILL.md Content
60
63
 
61
- Assemble each section in order from the skill-sections data file:
62
-
63
- **Frontmatter:**
64
- ```yaml
65
- ---
66
- name: {brief.name}
67
- version: {brief.version}
68
- description: {brief.name} — {export_count} verified functions
69
- author: {source_author}
70
- ---
71
- ```
72
-
73
- **Section 1 — Overview:**
74
- - What this skill provides
75
- - Source repo, version, branch
76
- - Forge tier used for compilation
77
- - Export count and confidence summary
78
-
79
- **Section 2 — Quick Start:**
80
- - Select the 3-5 most commonly used functions (by import frequency or documentation prominence)
81
- - Minimal usage examples — ONLY from source tests or official docs
82
- - If no examples exist in source, show signature-only quick start
83
- - Provenance citation for each function
84
-
85
- **Section 3 — API Reference:**
86
- - Group by module/file
87
- - For each exported function:
88
- - Full signature with types
89
- - Parameters table (name, type, required/optional, description)
90
- - Return type
91
- - Usage example (from source tests/docs ONLY — omit if none found)
92
- - Provenance citation: `[AST:{file}:L{line}]` or `[SRC:{file}:L{line}]`
93
- - T2 annotations (Deep tier only): temporal context from enrichment
94
-
95
- **Section 4 — Type Definitions:**
96
- - All exported types, interfaces, enums
97
- - Full type signatures with provenance citations
98
-
99
- **Section 5 — Integration Patterns (Forge/Deep only):**
100
- - Co-import patterns detected by ast_bridge
101
- - Common usage combinations
102
- - Skip for Quick tier (no co-import data available)
103
-
104
- **Section 6 — Manual Sections:**
105
- - Seed empty `<!-- [MANUAL] -->` markers:
106
- ```markdown
107
- <!-- [MANUAL:additional-notes] -->
108
- <!-- Add custom notes here. This section is preserved during skill updates. -->
109
- <!-- [/MANUAL:additional-notes] -->
110
- ```
111
- - Place after Quick Start and after API Reference sections
64
+ Assemble each section in order using the assembly rules data file (`{assemblyRulesData}`). The data file specifies frontmatter format, Tier 1 section details (Sections 1-8), Tier 2 section details (Sections 9-11), and assembly ordering rules. Follow it exactly.
112
65
 
113
66
  ### 3. Build context-snippet.md Content
114
67
 
115
- Compressed 2-line format for CLAUDE.md managed section:
68
+ Vercel-aligned indexed format for CLAUDE.md managed section (~80-120 tokens):
116
69
 
117
70
  ```markdown
118
- {skill-name} -> skills/{skill-name}/
119
- exports: {comma-separated top 10 function names}
71
+ [{skill-name} v{version}]|root: skills/{skill-name}/
72
+ |IMPORTANT: {skill-name} v{version} read SKILL.md before writing {skill-name} code. Do NOT rely on training data.
73
+ |quick-start:{SKILL.md#quick-start}
74
+ |api: {top exports with () for functions, comma-separated}
75
+ |key-types:{SKILL.md#key-types} — {inline summary of most important type values}
76
+ |gotchas: {2-3 most critical pitfalls or breaking changes, inline}
120
77
  ```
121
78
 
79
+ **Derivation rules:**
80
+
81
+ - **version**: From source detection (reconciled in step-03), not brief default
82
+ - **api**: Top 10 exports from extraction inventory, append `()` to function names
83
+ - **key-types**: Inline summary of most important enum/type values from Key Types section
84
+ - **gotchas**: Derived from T2-future annotations (breaking changes), async requirements, version-specific behavior changes. If no gotchas available, omit the gotchas line.
85
+ - **Section anchors** (`#quick-start`, `#key-types`): Must match actual heading slugs in the assembled SKILL.md
86
+
122
87
  ### 4. Build metadata.json Content
123
88
 
124
89
  Following the structure from the skill-sections data file:
125
90
  - Populate all fields from brief_data, extraction inventory, and tier
126
- - Set `generated_at` to current ISO-8601 timestamp
91
+ - Set `generation_date` to current ISO-8601 timestamp
127
92
  - Set `source_commit` from resolved source (if available)
128
- - Set `stats` from extraction aggregate counts
129
- - Set `tool_versions` based on tier and available tools
93
+ - Set `stats` from extraction aggregate counts:
94
+ - `exports_documented`: count of exports with documentation in the assembled SKILL.md
95
+ - `exports_public_api`: count of exports from public entry points (`__init__.py`, `index.ts`, `lib.rs`, or equivalent)
96
+ - `exports_internal`: count of all other non-underscore-prefixed exports (internal modules, helpers, adapters)
97
+ - `exports_total`: `exports_public_api` + `exports_internal`
98
+ - `public_api_coverage`: `exports_documented / exports_public_api` (1.0 when all public API exports are documented; `null` if `exports_public_api` is 0)
99
+ - `total_coverage`: `exports_documented / exports_total` (may be low for large codebases — this is expected; `null` if `exports_total` is 0)
100
+ - Set `tool_versions` based on tier and available tools. Resolve `{skf_version}` from the SKF module's `package.json` `version` field (run `node -p "require('./node_modules/bmad-module-skill-forge/package.json').version"` or read the installed module's `package.json`). If unresolvable, fall back to `git describe --tags --abbrev=0` in the SKF module root. Never hardcode the version.
130
101
 
131
102
  ### 5. Build references/ Content
132
103
 
@@ -140,37 +111,27 @@ Group functions logically by module, file, or functional area.
140
111
 
141
112
  ### 6. Build provenance-map.json Content
142
113
 
143
- One entry per extracted claim:
144
- - claim text
145
- - source_file and source_line
146
- - confidence tier (T1, T1-low, T2)
147
- - extraction_method (ast_bridge.scan_definitions, gh_bridge.read_file, qmd_bridge.search)
148
- - ast_node_type (if AST-extracted)
114
+ One entry per extracted export: export_name, export_type, params[] (typed strings), return_type, source_file, source_line, confidence tier (T1/T1-low/T2), extraction_method, ast_node_type. See `{skillSectionsData}` for full schema.
149
115
 
150
116
  ### 7. Build evidence-report.md Content
151
117
 
152
- Compilation audit trail:
153
- - Generation date, forge tier, source info
154
- - Tool versions used
155
- - Extraction summary (files scanned, exports found, confidence breakdown)
156
- - Validation results (populated in step-06)
157
- - Warnings from extraction or enrichment
118
+ Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template. Use the same `{skf_version}` value resolved in section 4 when populating the Tool Versions block.
158
119
 
159
120
  ### 8. Menu Handling Logic
160
121
 
161
122
  **Auto-proceed step — no user interaction.**
162
123
 
163
- After all content is assembled in context, immediately load, read entire file, then execute `{nextStepFile}`.
124
+ After all content is assembled in context (or written to the staging directory), immediately load, read entire file, then execute `{nextStepFile}`.
164
125
 
165
126
  #### EXECUTION RULES:
166
127
 
167
128
  - This is an auto-proceed assembly step with no user choices
168
- - All content stays in context — no files are written yet
129
+ - All content stays in context or in the staging directory — no final files are written yet
169
130
  - Proceed directly to validation after assembly is complete
170
131
 
171
132
  ## CRITICAL STEP COMPLETION NOTE
172
133
 
173
- ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md, extraction-rules.yaml) are assembled in context will you proceed to load `{nextStepFile}` for spec validation.
134
+ ONLY WHEN all 6 content artifacts (SKILL.md, context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md) are assembled in context will you proceed to load `{nextStepFile}` for spec validation. Note: extraction-rules.yaml is generated by step-07 from extraction data, not assembled here.
174
135
 
175
136
  ---
176
137
 
@@ -178,8 +139,11 @@ ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json,
178
139
 
179
140
  ### ✅ SUCCESS:
180
141
 
181
- - Skill-sections data file loaded and followed
142
+ - Skill-sections data file and assembly rules data file loaded and followed
182
143
  - SKILL.md assembled with all required sections in agentskills.io format
144
+ - Tier 1 sections assembled first and kept under 300 lines (excluding frontmatter)
145
+ - Tier 2 sections assembled after all Tier 1 sections
146
+ - Section 4b emitted only when Deep tier AND T2-future annotations exist; omitted entirely otherwise (no empty section)
183
147
  - Every function entry has a provenance citation
184
148
  - [MANUAL] markers seeded for update-skill compatibility
185
149
  - context-snippet.md, metadata.json, references/, provenance-map.json, evidence-report.md all assembled
@@ -190,9 +154,12 @@ ONLY WHEN all 7 content artifacts (SKILL.md, context-snippet.md, metadata.json,
190
154
 
191
155
  - Including functions or examples without provenance citations
192
156
  - Fabricating usage examples not found in source
193
- - Writing files to disk (that's step-07)
157
+ - Writing files to final `skills/` or `forge-data/` directories (that's step-07)
194
158
  - Missing required SKILL.md sections
159
+ - Emitting an empty Section 4b when no T2-future annotations exist
160
+ - Tier 1 content exceeding 300 lines without reduction
161
+ - Assembling Tier 2 sections before Tier 1 sections
195
162
  - Not seeding [MANUAL] markers
196
- - Not building all 7 content artifacts
163
+ - Not building all 6 content artifacts (extraction-rules.yaml is built by step-07)
197
164
 
198
165
  **Master Rule:** Zero hallucination — every line of SKILL.md must trace to source code. Compile from data, not imagination.
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: 'step-06-validate'
3
- description: 'Validate compiled skill content against agentskills.io spec via skills_ref'
3
+ description: 'Validate compiled skill content against agentskills.io spec via skill-check'
4
4
  nextStepFile: './step-07-generate-artifacts.md'
5
5
  ---
6
6
 
@@ -8,7 +8,7 @@ nextStepFile: './step-07-generate-artifacts.md'
8
8
 
9
9
  ## STEP GOAL:
10
10
 
11
- To validate the compiled SKILL.md content against the agentskills.io specification using skills_ref, auto-fix any validation failures, and confirm spec compliance before artifact generation.
11
+ To validate the compiled SKILL.md content against the agentskills.io specification using skill-check, auto-fix any validation failures, and confirm spec compliance before artifact generation.
12
12
 
13
13
  ## MANDATORY EXECUTION RULES (READ FIRST):
14
14
 
@@ -29,9 +29,10 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
29
29
 
30
30
  - 🎯 Focus ONLY on validating compiled content against spec
31
31
  - 🚫 FORBIDDEN to add new content — only fix spec compliance issues
32
- - 🚫 FORBIDDEN to write files content stays in context until step-07
32
+ - 💾 Validation and auto-fix modify files in the staging directory (`_bmad-output/{name}/`)
33
33
  - 💬 If auto-fix fails, report issues clearly but proceed (warn, don't halt)
34
- - ⚙️ If skills_ref unavailable: skip validation, add warning to evidence report
34
+ - ⚙️ If skill-check unavailable: skip validation, add warning to evidence report
35
+ - ⚠️ Ignore non-zero exit codes from `skill-check` if the JSON output shows 0 errors — parse JSON output, not exit codes
35
36
 
36
37
  ## EXECUTION PROTOCOLS:
37
38
 
@@ -39,6 +40,7 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
39
40
  - 💾 Validation results are added to evidence-report content in context
40
41
  - 📖 Auto-fix pattern: validate → fix → re-validate (once)
41
42
  - 🚫 Maximum one auto-fix attempt per validation failure
43
+ - ⏸️ **Conditional interaction:** If tessl returns suggestions (section 6b), halt for user input. Otherwise auto-proceed. This is a conditional gate step, not a pure auto-proceed step.
42
44
 
43
45
  ## CONTEXT BOUNDARIES:
44
46
 
@@ -53,85 +55,171 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
53
55
 
54
56
  ### 1. Check Tool Availability
55
57
 
56
- **If skills_ref tool is available:**
57
- Continue to validation steps.
58
+ Run: `npx skill-check -h`
58
59
 
59
- **If skills_ref tool is NOT available:**
60
- - Add warning to evidence-report content: "Spec validation skippedskills_ref tool unavailable"
61
- - Auto-proceed to next step
60
+ - If succeeds: Continue to automated validation (section 2)
61
+ - If fails: Perform manual fallback (section 3); add note to evidence-report: "Spec validation performed manually skill-check tool unavailable"
62
62
 
63
- ### 2. Validate Schema
63
+ **Important:** Do not assume availability — empirical check required.
64
64
 
65
- Use `skills_ref.validate_schema()` against the compiled SKILL.md content.
65
+ ### 2. Validate & Auto-Fix (skill-check check --fix)
66
66
 
67
- **Check:**
68
- - Required sections present (Overview, Quick Start, API Reference, Type Definitions)
69
- - Section order follows agentskills.io standard
70
- - Frontmatter contains required fields (name, version, description, author)
71
- - Provenance citations present in API Reference entries
67
+ Run the external skill-check tool against the compiled skill staging directory:
72
68
 
73
- **If validation passes:** Record "Schema: PASS" in evidence-report content.
69
+ ```bash
70
+ npx skill-check check <staging-skill-dir> --fix --format json --no-security-scan
71
+ ```
72
+
73
+ This performs frontmatter validation, description quality checks, body limit enforcement, local link resolution, file formatting, auto-fix of deterministic issues, and quality scoring (0-100) across five weighted categories.
74
+
75
+ **Parse the JSON output** for: `qualityScore` (0-100), `diagnostics[]` (remaining issues), `fixed[]` (auto-corrected issues).
76
+
77
+ **Context sync after --fix:** If `fixed[]` is non-empty (i.e., `--fix` modified files on disk), re-read the modified SKILL.md to update the in-context copy. Verify the re-read content matches expectations before proceeding. This prevents silent divergence between the in-context SKILL.md and the on-disk version that step-07 will use for artifact generation.
78
+
79
+ **Note:** `skill-check` may return non-zero exit code even when `errorCount` is 0. Always rely on parsed JSON, not the shell exit code.
80
+
81
+ - **Score ≥ 70:** Record "Schema: PASS (score: {score}/100)" in evidence-report
82
+ - **Score < 70:** Log remaining diagnostics as warnings, record "Schema: WARN — score {score}/100, {count} remaining issues", proceed
83
+ - **Unfixable errors:** Record specific rule IDs and suggestions, proceed with warnings
84
+
85
+ ### 3. Validate Frontmatter (Fallback)
86
+
87
+ **If skill-check was available:** Skip — already validated in step 2.
88
+
89
+ **If skill-check NOT available (fallback):** Perform manual frontmatter compliance check:
90
+
91
+ - [ ] Frontmatter present — file starts with `---` and has closing `---`
92
+ - [ ] `name` field — present, non-empty, lowercase alphanumeric + hyphens only, 1-64 chars
93
+ - [ ] `name` matches skill output directory name
94
+ - [ ] `description` field — present, non-empty, 1-1024 characters
95
+ - [ ] No unknown fields — only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` permitted
96
+ - [ ] `version` and `author` are NOT in frontmatter (they belong in metadata.json)
97
+
98
+ If fails: auto-fix (deterministic), re-validate once, record result. If passes: record "Frontmatter: PASS".
99
+
100
+ ### 4. Split Oversized Body (if needed)
101
+
102
+ **If step 2 reported `body.max_lines` failure:**
103
+
104
+ ```bash
105
+ npx skill-check split-body <staging-skill-dir> --write
106
+ ```
107
+
108
+ Then re-validate: `npx skill-check check <staging-skill-dir> --format json --no-security-scan`
109
+
110
+ **If skill-check unavailable or no body size issue:** Skip.
111
+
112
+ ### 5. Security Scan
113
+
114
+ **If skill-check available:**
115
+
116
+ ```bash
117
+ npx skill-check check <staging-skill-dir> --format json
118
+ ```
119
+
120
+ (Security scan enabled by default when `--no-security-scan` omitted. The scan uses [Snyk Agent Scan](https://github.com/snyk/agent-scan) to check for prompt injection risks, sensitive data exposure, and unsafe tool permissions.)
121
+
122
+ Record any security warnings in evidence-report. Security findings are advisory — they do not block artifact generation.
123
+
124
+ **If security scan fails due to missing SNYK_TOKEN:**
125
+
126
+ Display actionable guidance:
127
+
128
+ "Security scan requires a Snyk API token. **Important:** The Snyk API requires an Enterprise plan ([authentication docs](https://docs.snyk.io/snyk-api/authentication-for-api)). To enable:
129
+ 1. Obtain a Snyk Enterprise account with API access
130
+ 2. Get your API token from Account Settings → API Token
131
+ 3. Set `SNYK_TOKEN=your-token` in your environment or `.env`
132
+ 4. Re-run [SF] Setup Forge to update tool detection, then re-run [CS] Create Skill
133
+
134
+ If you don't have an Enterprise account, use `--no-security-scan` flag in skill-check to skip, or `--security-scan-runner pipx|uvx|local` to control the execution method. Security scanning is optional and does not affect tier level or block skill compilation."
135
+
136
+ Record: "Security scan skipped — SNYK_TOKEN not configured"
137
+
138
+ **If skill-check unavailable:** Skip with note: "Security scan skipped — skill-check tool unavailable"
139
+
140
+ ### 6. Content Quality Review (tessl)
74
141
 
75
- **If validation fails:**
76
- 1. Identify specific failures
77
- 2. Attempt auto-fix (structural adjustments only — never invent content)
78
- 3. Re-validate once
79
- 4. If still failing: record "Schema: FAIL — {specific issues}" in evidence-report, add warnings, but proceed
142
+ **If tessl available**, run: `npx -y tessl skill review <staging-skill-dir>`
80
143
 
81
- ### 3. Validate Frontmatter
144
+ Parse output for: `description_score`, `content_score`, `average_score`, `validation_result`, `judge_suggestions[]`.
82
145
 
83
- Use `skills_ref.validate_frontmatter()` against the SKILL.md frontmatter.
146
+ - **Content score < 70%:** Record warning: "Content quality warning: tessl scored content at {score}%."
147
+ - **Unavailable:** Skip with note: "Content quality review skipped — tessl tool unavailable"
84
148
 
85
- **Check:**
86
- - `name` matches brief.name (kebab-case)
87
- - `version` matches brief.version
88
- - `description` follows format: "{name} — {count} verified functions"
89
- - `author` is set
149
+ > **EXPECTED BEHAVIOR — Two-Tier Scoring:** The Skill Forge two-tier design (Tier 1 Key API Summary + Tier 2 Full API Reference) intentionally includes progressive disclosure. tessl's `conciseness` scorer will flag this as redundancy (typically scoring 2/3), which is **expected behavior — not a defect**. tessl may also suggest removing `[MANUAL]` markers, moving Full API Reference to a separate file, or consolidating duplicate parameter documentation — **all three suggestions conflict with SKF design principles and must be dismissed.** Acceptable threshold: content scores >= 60% are normal for two-tier skills. Do NOT consolidate Tier 1 and Tier 2 content to improve the score — the two-tier structure is a deliberate design choice for standalone usability.
90
150
 
91
- **If validation passes:** Record "Frontmatter: PASS" in evidence-report content.
151
+ tessl installs automatically via `npx`. A missing tool is not an error — graceful skip.
92
152
 
93
- **If validation fails:**
94
- 1. Auto-fix frontmatter (these are deterministic fixes)
95
- 2. Re-validate once
96
- 3. Record result in evidence-report
153
+ #### 6b. User Decision Gate (conditional)
97
154
 
98
- ### 4. Validate metadata.json
155
+ **If tessl returned no suggestions OR tessl was unavailable:** Skip this gate — auto-proceed.
99
156
 
100
- Cross-check metadata.json content against extraction inventory:
101
- - `stats.exports_documented` matches actual documented exports
102
- - `stats.exports_total` matches total extracted exports
103
- - `stats.coverage` is accurate (documented / total)
157
+ **If tessl returned suggestions**, present them to the user:
158
+
159
+ "**Content quality review: {score}%**
160
+
161
+ tessl suggestions:
162
+ {numbered list of judge_suggestions}
163
+
164
+ **Select an option:**
165
+ - **[S] Skip** — proceed with current content as-is (default)
166
+ - **[A] Apply structural fixes** — apply only structural suggestions (split sections, consolidate duplicates). No new content generated.
167
+ - **[R] Review all** — show each suggestion with proposed changes before applying"
168
+
169
+ #### Gate Rules:
170
+
171
+ - **Structural suggestions** (split reference section, consolidate duplicates, reorder sections) can be applied without zero-hallucination risk — they restructure existing content
172
+ - **Semantic suggestions** (add examples, add error handling, add validation checkpoints) introduce content not verified from source code. If the user chooses to apply these:
173
+ - Warn: "This adds content not verified from source code."
174
+ - Mark applied content with `<!-- [TESSL:auto-fix] -->` markers
175
+ - Cite as `[TESSL:suggestion]` in the provenance map with `confidence: "TESSL"` (below T3)
176
+ - Record in evidence report: "TESSL-suggested content applied: {count} items (unverified)"
177
+ - **If user selects [S]:** Record "tessl suggestions: skipped by user" in evidence report. Proceed to section 7.
178
+ - **If user selects [A]:** Apply structural fixes only, re-run tessl to capture updated score, record results. Proceed to section 7.
179
+ - **If user selects [R]:** Show each suggestion with the proposed change. For each, user confirms or skips. Apply confirmed changes, record results. Proceed to section 7.
180
+
181
+ ### 7. Validate metadata.json
182
+
183
+ Cross-check metadata.json against extraction inventory:
184
+ - `stats.exports_documented` / `stats.exports_public_api` / `stats.exports_internal` / `stats.exports_total` are accurate
185
+ - `stats.public_api_coverage` and `stats.total_coverage` are correctly computed (null when denominator is 0)
104
186
  - `confidence_t1`, `confidence_t2`, `confidence_t3` match actual counts
105
187
  - `spec_version` is "1.3"
106
188
 
107
189
  Auto-fix any discrepancies (these are computed values).
108
190
 
109
- ### 5. Update Evidence Report
191
+ ### 8. Update Evidence Report
110
192
 
111
- Add validation results to the evidence-report content in context:
193
+ Add validation results to evidence-report content in context:
112
194
 
113
195
  ```markdown
114
196
  ## Validation Results
115
- - Schema: {pass/fail}
197
+ - Schema: {pass/fail} (quality score: {score}/100)
116
198
  - Frontmatter: {pass/fail}
199
+ - Body: {pass/fail} {split-body applied if applicable}
200
+ - Security: {pass/warn/skipped}
201
+ - Content Quality (tessl): {pass/warn/skipped} (score: {score}%)
117
202
  - Metadata: {pass/fail}
118
203
 
119
- ## Warnings
120
- - {any warnings from validation}
121
- ```
204
+ ## Quality Score Breakdown
205
+ - Frontmatter (30%): {score} | Description (30%): {score} | Body (20%): {score} | Links (10%): {score} | File (10%): {score}
122
206
 
123
- ### 6. Menu Handling Logic
207
+ ## Auto-Fixed Issues
208
+ - {list of issues automatically corrected by --fix}
209
+
210
+ ## Remaining Warnings / Security Findings / Content Quality (tessl)
211
+ - {warnings, security results, tessl scores and suggestions — or "skipped"}
212
+ ```
124
213
 
125
- **Auto-proceed step no user interaction.**
214
+ ### 9. Menu Handling Logic
126
215
 
127
- After validation is complete (or skipped if tools unavailable), immediately load, read entire file, then execute `{nextStepFile}`.
216
+ **Conditional interaction step.** If tessl produced suggestions, section 6b halts for user input. Otherwise, auto-proceed.
128
217
 
129
- #### EXECUTION RULES:
218
+ After validation completes (including any user decisions from section 6b), immediately load, read entire file, then execute `{nextStepFile}`.
130
219
 
131
- - This is an auto-proceed validation step with no user choices
132
220
  - Tool unavailability is a skip, not a halt
133
221
  - Validation failures are warnings — proceed to artifact generation
134
- - Proceed directly to next step after validation completes
222
+ - tessl gate only triggers when suggestions exist — no gate for clean reviews or unavailable tools
135
223
 
136
224
  ## CRITICAL STEP COMPLETION NOTE
137
225
 
@@ -143,20 +231,21 @@ ONLY WHEN validation is complete (or skipped) and evidence-report content is upd
143
231
 
144
232
  ### ✅ SUCCESS:
145
233
 
146
- - Schema validation attempted (or skipped with warning if tool unavailable)
147
- - Frontmatter validation attempted
148
- - Metadata cross-check performed
149
- - Auto-fix applied for deterministic failures
150
- - Evidence report updated with validation results
151
- - Warnings recorded for any failures
152
- - Auto-proceeded to step-07
234
+ - `npx skill-check check --fix --format json` executed (or skipped with warning if unavailable)
235
+ - Quality score captured and recorded; auto-fix applied for deterministic issues
236
+ - Split-body applied if `body.max_lines` failed; security scan executed (or skipped with warning)
237
+ - `npx -y tessl skill review` executed (or skipped); content quality warning raised if score < 70%
238
+ - tessl suggestions presented to user when available; user decision recorded
239
+ - TESSL-applied content marked with `<!-- [TESSL:auto-fix] -->` and cited as `[TESSL:suggestion]`
240
+ - Metadata cross-check performed; evidence report updated with structured results
241
+ - Proceeded to step-07 (auto or after user gate)
153
242
 
154
243
  ### ❌ SYSTEM FAILURE:
155
244
 
156
- - Halting the workflow on validation failure (should warn and proceed)
157
- - Halting on skills_ref unavailability (should skip with warning)
158
- - Adding new content during validation (only structural fixes allowed)
159
- - Not recording validation results in evidence report
245
+ - Halting on validation failure or skill-check unavailability (should warn and proceed)
246
+ - Adding new content during validation without user approval via the tessl gate
247
+ - Applying semantic tessl suggestions without warning the user about unverified content
248
+ - Not recording quality score; skipping security scan without recording the skip
160
249
  - Attempting more than one auto-fix cycle per failure
161
250
 
162
- **Master Rule:** Validation informs, it does not block. Record results, fix what's deterministic, warn about the rest, and proceed.
251
+ **Master Rule:** Validation informs, it does not block. Record results, fix what's deterministic, scan for security issues, warn about the rest, and proceed.