bmad-module-skill-forge 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (110) hide show
  1. package/README.md +88 -44
  2. package/docs/agents.md +1 -1
  3. package/docs/architecture.md +49 -20
  4. package/docs/examples.md +13 -2
  5. package/docs/getting-started.md +15 -13
  6. package/docs/index.md +3 -1
  7. package/docs/workflows.md +2 -2
  8. package/package.json +5 -3
  9. package/src/forger/forge-tier.yaml +1 -1
  10. package/src/forger/preferences.yaml +8 -1
  11. package/src/knowledge/doc-fetcher.md +55 -0
  12. package/src/knowledge/overview.md +17 -16
  13. package/src/knowledge/progressive-capability.md +9 -1
  14. package/src/knowledge/provenance-tracking.md +31 -23
  15. package/src/knowledge/qmd-registry.md +132 -0
  16. package/src/knowledge/skf-knowledge-index.csv +2 -0
  17. package/src/workflows/analyze-source/data/skill-brief-schema.md +24 -4
  18. package/src/workflows/analyze-source/steps-c/step-01-init.md +19 -9
  19. package/src/workflows/analyze-source/steps-c/step-01b-continue.md +2 -2
  20. package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +10 -2
  21. package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +17 -2
  22. package/src/workflows/analyze-source/steps-c/step-05-recommend.md +2 -1
  23. package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +3 -3
  24. package/src/workflows/analyze-source/templates/analysis-report-template.md +1 -1
  25. package/src/workflows/analyze-source/validation-report.md +1 -1
  26. package/src/workflows/analyze-source/workflow-plan-analyze-source.md +1 -1
  27. package/src/workflows/analyze-source/workflow.md +1 -1
  28. package/src/workflows/audit-skill/steps-c/step-01-init.md +8 -5
  29. package/src/workflows/audit-skill/steps-c/step-02-re-index.md +12 -2
  30. package/src/workflows/audit-skill/validation-report.md +2 -2
  31. package/src/workflows/audit-skill/workflow-plan-audit-skill.md +1 -1
  32. package/src/workflows/brief-skill/data/scope-templates.md +60 -0
  33. package/src/workflows/brief-skill/data/skill-brief-schema.md +38 -3
  34. package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +20 -4
  35. package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +20 -64
  36. package/src/workflows/brief-skill/steps-c/step-04-confirm-brief.md +1 -1
  37. package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +60 -5
  38. package/src/workflows/brief-skill/workflow.md +2 -1
  39. package/src/workflows/create-skill/data/compile-assembly-rules.md +110 -0
  40. package/src/workflows/create-skill/data/extraction-patterns.md +248 -0
  41. package/src/workflows/create-skill/data/skill-sections.md +79 -25
  42. package/src/workflows/create-skill/data/source-resolution-protocols.md +130 -0
  43. package/src/workflows/create-skill/steps-c/step-01-load-brief.md +7 -4
  44. package/src/workflows/create-skill/steps-c/step-02-ecosystem-check.md +10 -7
  45. package/src/workflows/create-skill/steps-c/step-03-extract.md +49 -9
  46. package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +238 -0
  47. package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +220 -0
  48. package/src/workflows/create-skill/steps-c/step-04-enrich.md +40 -10
  49. package/src/workflows/create-skill/steps-c/step-05-compile.md +48 -81
  50. package/src/workflows/create-skill/steps-c/step-06-validate.md +149 -63
  51. package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +53 -3
  52. package/src/workflows/create-skill/steps-c/step-08-report.md +1 -1
  53. package/src/workflows/create-skill/workflow-plan-create-skill.md +42 -10
  54. package/src/workflows/create-skill/workflow.md +3 -2
  55. package/src/workflows/create-stack-skill/data/stack-skill-template.md +20 -7
  56. package/src/workflows/create-stack-skill/steps-c/step-01-init.md +2 -2
  57. package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +14 -0
  58. package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +28 -7
  59. package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +10 -7
  60. package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +66 -68
  61. package/src/workflows/create-stack-skill/validation-report.md +1 -1
  62. package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +4 -4
  63. package/src/workflows/export-skill/data/managed-section-format.md +1 -0
  64. package/src/workflows/export-skill/data/snippet-format.md +33 -14
  65. package/src/workflows/export-skill/steps-c/step-01-load-skill.md +16 -2
  66. package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +29 -22
  67. package/src/workflows/export-skill/steps-c/step-04-update-context.md +1 -1
  68. package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
  69. package/src/workflows/export-skill/validation-report.md +1 -1
  70. package/src/workflows/export-skill/workflow-plan-export-skill.md +8 -8
  71. package/src/workflows/quick-skill/data/skill-template.md +27 -7
  72. package/src/workflows/quick-skill/steps-c/step-04-compile.md +44 -13
  73. package/src/workflows/quick-skill/steps-c/step-05-validate.md +79 -16
  74. package/src/workflows/quick-skill/validation-report.md +4 -4
  75. package/src/workflows/quick-skill/workflow-plan-quick-skill.md +4 -4
  76. package/src/workflows/quick-skill/workflow.md +2 -0
  77. package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +12 -3
  78. package/src/workflows/setup-forge/steps-c/step-02-write-config.md +10 -2
  79. package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +111 -40
  80. package/src/workflows/setup-forge/steps-c/step-04-report.md +13 -0
  81. package/src/workflows/setup-forge/workflow.md +1 -0
  82. package/src/workflows/test-skill/data/scoring-rules.md +14 -6
  83. package/src/workflows/test-skill/data/source-access-protocol.md +37 -0
  84. package/src/workflows/test-skill/steps-c/step-01-init.md +38 -13
  85. package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -3
  86. package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +25 -5
  87. package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +18 -43
  88. package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +213 -0
  89. package/src/workflows/test-skill/steps-c/step-05-score.md +24 -27
  90. package/src/workflows/test-skill/steps-c/step-06-report.md +9 -4
  91. package/src/workflows/test-skill/templates/test-report-template.md +2 -1
  92. package/src/workflows/test-skill/validation-report.md +1 -1
  93. package/src/workflows/test-skill/workflow-plan-test-skill.md +7 -3
  94. package/src/workflows/test-skill/workflow.md +1 -1
  95. package/src/workflows/update-skill/data/remote-source-resolution.md +43 -0
  96. package/src/workflows/update-skill/steps-c/step-01-init.md +13 -7
  97. package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +26 -0
  98. package/src/workflows/update-skill/steps-c/step-03-re-extract.md +37 -9
  99. package/src/workflows/update-skill/steps-c/step-04-merge.md +3 -3
  100. package/src/workflows/update-skill/steps-c/step-05-validate.md +63 -67
  101. package/src/workflows/update-skill/steps-c/step-06-write.md +37 -14
  102. package/src/workflows/update-skill/steps-c/step-07-report.md +1 -1
  103. package/src/workflows/update-skill/validation-report.md +4 -4
  104. package/src/workflows/update-skill/workflow-plan-update-skill.md +4 -4
  105. package/src/workflows/update-skill/workflow.md +1 -1
  106. package/tools/cli/commands/status.js +4 -4
  107. package/tools/cli/commands/uninstall.js +1 -1
  108. package/tools/cli/commands/update.js +2 -2
  109. package/tools/cli/lib/ui.js +60 -16
  110. package/tools/cli/lib/version-check.js +2 -2
@@ -2,7 +2,7 @@
2
2
  name: 'step-06-report'
3
3
  description: 'Generate gap report with remediation suggestions and finalize test report'
4
4
 
5
- outputFile: '{output_folder}/test-report-{skill_name}.md'
5
+ outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
6
6
  scoringRulesFile: '../data/scoring-rules.md'
7
7
  outputFormatsFile: '../data/output-section-formats.md'
8
8
  ---
@@ -17,7 +17,7 @@ Generate a detailed gap report listing every issue found during coverage and coh
17
17
 
18
18
  ### Universal Rules:
19
19
 
20
- - 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03 and 04
20
+ - 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03, 04, and 04b
21
21
  - 📖 CRITICAL: Read the complete step file before taking any action
22
22
  - ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
23
23
  - ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
@@ -69,6 +69,10 @@ Read `{outputFile}` and extract every issue found across all analysis sections:
69
69
  - Incomplete integration patterns (contextual mode)
70
70
  - Structural issues (naive mode — missing sections, broken examples)
71
71
 
72
+ **From External Validation (step 04b):**
73
+ - skill-check diagnostics (unresolved errors and warnings)
74
+ - tessl judge suggestions (content quality and actionability improvements)
75
+
72
76
  ### 2. Load Severity Rules
73
77
 
74
78
  Load `{scoringRulesFile}` for gap severity classification:
@@ -78,6 +82,7 @@ Load `{scoringRulesFile}` for gap severity classification:
78
82
  | **Critical** | Missing exported function/class documentation |
79
83
  | **High** | Signature mismatch between source and SKILL.md |
80
84
  | **Medium** | Missing type or interface documentation |
85
+ | **Medium** | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
81
86
  | **Low** | Missing optional metadata or examples |
82
87
  | **Info** | Style suggestions, non-blocking observations |
83
88
 
@@ -96,7 +101,7 @@ If no gaps found, append a clean pass message recommending **export-skill** work
96
101
  ### 5. Finalize Output Document
97
102
 
98
103
  Update `{outputFile}` frontmatter:
99
- - Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-05-score', 'step-06-report']`
104
+ - Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-04b-external-validators', 'step-05-score', 'step-06-report']`
100
105
 
101
106
  ### 6. Present Final Report
102
107
 
@@ -154,7 +159,7 @@ This is the final step of the test-skill workflow. When the user selects C, the
154
159
 
155
160
  ### ✅ SUCCESS:
156
161
 
157
- - Every gap traces to a finding from steps 03 or 04 (zero fabrication)
162
+ - Every gap traces to a finding from steps 03, 04, or 04b (zero fabrication)
158
163
  - Gaps classified by severity using scoring rules
159
164
  - Gaps ordered by severity (Critical first)
160
165
  - Every gap has a specific, actionable remediation suggestion
@@ -1,11 +1,12 @@
1
1
  ---
2
2
  workflowType: 'test-skill'
3
3
  skillName: ''
4
+ skillDir: ''
4
5
  testMode: ''
5
6
  forgeTier: ''
6
7
  testResult: ''
7
8
  score: ''
8
- threshold: ''
9
+ analysisConfidence: ''
9
10
  testDate: ''
10
11
  stepsCompleted: []
11
12
  nextWorkflow: ''
@@ -247,7 +247,7 @@ Each step has explicit CONTEXT BOUNDARIES with FORBIDDEN markers preventing scop
247
247
 
248
248
  ### Error Handling: Good
249
249
 
250
- - step-01: Three-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml)
250
+ - step-01: Three-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml). Checks `preferences.yaml` for `tier_override` after loading forge tier.
251
251
  - step-03/04: Subprocess fallback rules at multiple locations
252
252
  - step-06: Zero-gaps clean pass path
253
253
 
@@ -202,6 +202,7 @@ Cognitive completeness verification — Verifies that a skill is complete enough
202
202
  | 02 | detect-mode | Middle (Simple) | Auto-proceed | Determine naive vs contextual from skill metadata |
203
203
  | 03 | coverage-check | Validation Sequence | Auto-proceed | Compare documented exports against source API surface |
204
204
  | 04 | coherence-check | Validation Sequence (conditional) | Auto-proceed | Validate references/coherence (contextual=full, naive=simplified) |
205
+ | 04b | external-validators | Middle (Simple) | Auto-proceed | Run external validation tools (skill-check, tessl) and capture scores |
205
206
  | 05 | score | Middle (Simple) | Auto-proceed | Calculate completeness score from findings |
206
207
  | 06 | report | Final Step | C only | Generate gap report, finalize document, recommend next workflow |
207
208
 
@@ -210,7 +211,8 @@ Cognitive completeness verification — Verifies that a skill is complete enough
210
211
  ```
211
212
  User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
212
213
  [03] coverage-check (auto) → [04] coherence-check (auto) →
213
- [05] score (auto) → [06] report [C] finalize
214
+ [04b] external-validators (auto) → [05] score (auto)
215
+ [06] report → [C] finalize
214
216
  ```
215
217
 
216
218
  ### Data Flow
@@ -219,7 +221,8 @@ User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
219
221
  - Step 02: reads metadata type → sets {testMode} naive|contextual → appends Test Summary
220
222
  - Step 03: reads SKILL.md exports + source files → subprocess AST analysis → appends Coverage Analysis
221
223
  - Step 04: reads SKILL.md references → conditional depth by mode → appends Coherence Analysis
222
- - Step 05: reads coverage + coherence findingscalculates score → appends Completeness Score
224
+ - Step 04b: runs skill-check and tessl against skill directory captures external scores → appends External Validation
225
+ - Step 05: reads coverage + coherence + external validation findings → calculates score → appends Completeness Score
223
226
  - Step 06: reads all findings → generates remediation per gap → appends Gap Report → finalizes
224
227
 
225
228
  ### Subprocess Optimization
@@ -246,6 +249,7 @@ test-skill/
246
249
  │ ├── step-02-detect-mode.md
247
250
  │ ├── step-03-coverage-check.md
248
251
  │ ├── step-04-coherence-check.md
252
+ │ ├── step-04b-external-validators.md
249
253
  │ ├── step-05-score.md
250
254
  │ └── step-06-report.md
251
255
  └── templates/
@@ -261,7 +265,7 @@ Ferris (Audit mode): methodical, precise, evidence-based. Zero hallucination —
261
265
  - **Previous:** create-skill (required — SKILL.md must exist)
262
266
  - **Next:** export-skill (if pass) or update-skill (if fail)
263
267
  - **Input contract:** `{skills_output_folder}/{name}/SKILL.md` + `metadata.json` + `{sidecar_path}/forge-tier.yaml`
264
- - **Output contract:** `{output_folder}/test-report-{skill_name}.md` with frontmatter: workflowType, testResult, score, nextWorkflow
268
+ - **Output contract:** `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md` with frontmatter: workflowType, testResult, score, nextWorkflow
265
269
 
266
270
  ### Output Template (Structured)
267
271
 
@@ -49,7 +49,7 @@ installed_path: '{project-root}/_bmad/skf/workflows/test-skill'
49
49
 
50
50
  Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
51
51
 
52
- - `project_name`, `output_folder`, `user_name`, `communication_language`, `document_output_language`
52
+ - `project_name`, `user_name`, `communication_language`, `document_output_language`
53
53
  - `skills_output_folder`, `forge_data_folder`, `sidecar_path`
54
54
 
55
55
  ### 2. First Step Execution
@@ -0,0 +1,43 @@
1
+ # Remote Source Resolution (Forge/Deep Tier)
2
+
3
+ If `source_root` is a local path: proceed with the tier-appropriate strategy as normal.
4
+
5
+ If `source_root` (from metadata.json) is a remote URL (GitHub URL or owner/repo format) AND tier is Forge or Deep:
6
+
7
+ 1. **Check `git` availability:** Verify `git` is functional (`git --version`). If `git` is not available, skip to the fallback warning below.
8
+
9
+ 2. **Ephemeral sparse clone:** Clone only the changed files from the change manifest to a system temp path. Note: at this point in the flow, `{source_root}` is known to be a remote URL (the local-path case was already handled above).
10
+
11
+ ```
12
+ temp_path = {system_temp}/skf-ephemeral-{skill-name}-{timestamp}/
13
+ git clone --depth 1 --single-branch --filter=blob:none --sparse {source_root} {temp_path}
14
+ git -C {temp_path} sparse-checkout set --skip-checks {changed_files_from_manifest}
15
+ ```
16
+
17
+ **Note:** `--skip-checks` is required because `changed_files_from_manifest` contains individual file paths (e.g., `src/core/parser.py`), not directories. Without this flag, `git sparse-checkout set` rejects non-directory entries.
18
+
19
+ No `--branch` flag is used — the clone targets the remote's default branch, which must match the branch used during the original [CS] Create Skill run. This scopes the clone to only the files identified in step-02's change manifest, avoiding a full repository download.
20
+
21
+ 3. **If clone succeeds:** Update the working source path to `{temp_path}` for all subsequent AST operations in this step. Proceed with the **Forge tier** extraction strategy below. Mark `ephemeral_clone_active = true` for cleanup.
22
+
23
+ 4. **If clone fails (network error, auth failure, timeout):**
24
+
25
+ Warning message: "Ephemeral clone of `{source_root}` failed: {error}. Degrading to source reading (T1-low) for this run. For T1 (AST-verified) confidence, clone the repository locally and re-run [CS] Create Skill with the local path, then re-run this update."
26
+
27
+ Override the extraction strategy to Quick tier for this run. Note the degradation reason in context for the evidence report.
28
+
29
+ ## Ephemeral Clone Cleanup
30
+
31
+ After extraction is complete for all files in scope (whether successful or partially failed), before presenting the extraction summary, if `ephemeral_clone_active`, delete the `{temp_path}` directory. Log: "Ephemeral source clone cleaned up." This ensures cleanup runs even if some extractions failed, as long as the step itself is still executing.
32
+
33
+ ## Version Reconciliation
34
+
35
+ After the source path is accessible, check whether the source version has changed since the original skill was created. Look for the version file matching the detected language (e.g., `pyproject.toml`, `package.json`, `Cargo.toml`). If the source version differs from the current `metadata.json` version, record `source_version_detected` in context for step-06 to use when updating `metadata.json`. No warning needed here — step-06 handles the version update.
36
+
37
+ ## AST Tool Unavailability (Local Source)
38
+
39
+ If AST tool is unavailable at Forge/Deep tier with local source:
40
+
41
+ Warning message: "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
42
+
43
+ Degrade to Quick tier extraction. Note the degradation reason in context for the evidence report.
@@ -62,11 +62,15 @@ Load the existing skill and all its provenance data, detect whether this is an i
62
62
  Provide either:
63
63
  - A skill name (resolves to `{skills_output_folder}/{name}/`)
64
64
  - A full path to the skill folder
65
+ - A skill name with `--from-test-report` to use the test report's gap findings instead of source drift detection
65
66
 
66
67
  **Skill:** {user provides path or name}"
67
68
 
68
69
  Resolve the path to an absolute skill folder location.
69
70
 
71
+ **If `--from-test-report` was provided (or user references a test report):**
72
+ Search for the test report at `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md`. If found, set `test_report_path` in context and `update_mode: gap-driven`. If not found, warn and continue with normal source drift mode.
73
+
70
74
  ### 2. Validate Required Artifacts
71
75
 
72
76
  **Check SKILL.md exists:**
@@ -75,19 +79,21 @@ Resolve the path to an absolute skill folder location.
75
79
 
76
80
  **Check metadata.json exists:**
77
81
  - Load `{resolved_skill_path}/metadata.json`
78
- - Extract: `skill_name`, `skill_type` (individual or stack), `version`, `created`, `forge_tier`, `source_root`
82
+ - Extract: `name`, `skill_type` (single or stack), `version`, `generation_date`, `confidence_tier`, `source_root`
79
83
  - If missing: **ABORT** — "No metadata.json found. This skill may have been created manually. Run create-skill to generate provenance data."
80
84
 
81
85
  **Detect skill type from metadata:**
82
86
  - If `skill_type == "stack"`: flag as stack skill (multi-file update mode)
83
- - If `skill_type == "individual"` or absent: flag as individual skill
87
+ - If `skill_type == "single"` or absent: flag as single skill
84
88
 
85
89
  ### 3. Load Forge Tier Configuration
86
90
 
87
- **Load `sidecar/forge-tier.yaml`:**
88
- - Extract: `forge_tier` (Quick, Forge, or Deep), available tools
91
+ **Load `{sidecar_path}/forge-tier.yaml`:**
92
+ - Extract: `tier` (Quick, Forge, or Deep), available tools
89
93
  - If missing: **ABORT** — "No forge-tier.yaml found. Run setup-forge first to detect available tools."
90
94
 
95
+ **Apply tier override:** Read `{sidecar_path}/preferences.yaml`. If `tier_override` is set and is a valid tier value (Quick, Forge, or Deep), use it instead of the detected tier.
96
+
91
97
  **Determine analysis capabilities:**
92
98
  - **Quick:** text pattern matching only → T1-low confidence
93
99
  - **Forge:** AST structural extraction → T1 confidence
@@ -146,7 +152,7 @@ Please provide the current source code path:
146
152
  | Property | Value |
147
153
  |----------|-------|
148
154
  | **Skill** | {skill_name} |
149
- | **Type** | {individual/stack} |
155
+ | **Type** | {single/stack} |
150
156
  | **Version** | {version} |
151
157
  | **Created** | {created date} |
152
158
  | **Source** | {source_root} |
@@ -154,7 +160,7 @@ Please provide the current source code path:
154
160
  | **Provenance Age** | {days} days since last extraction |
155
161
  | **Exports** | {export_count} tracked exports |
156
162
  | **[MANUAL] Sections** | {manual_count} preserved sections |
157
- | **Mode** | {normal/degraded} |
163
+ | **Mode** | {normal/degraded/gap-driven} |
158
164
 
159
165
  **Analysis plan:** {tier_description}
160
166
  - {Quick: text pattern diff → T1-low findings}
@@ -188,7 +194,7 @@ ONLY WHEN [C] is selected and baseline has been established with all required ar
188
194
  ### ✅ SUCCESS:
189
195
 
190
196
  - SKILL.md loaded and validated
191
- - metadata.json loaded with skill_name, skill_type, version, source_root
197
+ - metadata.json loaded with name, skill_type, version, source_root
192
198
  - Forge tier loaded from sidecar/forge-tier.yaml
193
199
  - Provenance map loaded (or degraded mode confirmed)
194
200
  - [MANUAL] sections inventoried across all output files
@@ -56,6 +56,32 @@ Compare current source code state against the provenance map to produce a comple
56
56
 
57
57
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
58
58
 
59
+ ### 0. Check for Test Report Input (Gap-Driven Mode)
60
+
61
+ **If `update_mode == "gap-driven"` (set in step-01 via `--from-test-report`):**
62
+
63
+ Load the test report at `{test_report_path}` and extract findings:
64
+
65
+ 1. Read the **Gap Report** section — each gap entry has severity, category, and description
66
+ 2. Read the **Coverage Analysis** section — each per-export row has documented/missing/mismatch status
67
+ 3. Translate findings into change manifest format:
68
+
69
+ | Gap Severity | Gap Type | Change Category |
70
+ |-------------|----------|-----------------|
71
+ | Critical | Missing export documentation | NEW_EXPORT (undocumented public API) |
72
+ | High | Signature mismatch | MODIFIED_EXPORT (signature needs update) |
73
+ | Medium | Missing type/interface docs | NEW_EXPORT (undocumented type) |
74
+ | Medium | Stale documentation | MODIFIED_EXPORT (docs reference removed export) |
75
+ | Low | Missing metadata/examples | metadata update |
76
+
77
+ 4. Build the change manifest from translated gaps — no file-level timestamp comparison needed since source hasn't changed
78
+ 5. Set `gap_count` from the total number of translated entries
79
+ 6. **Skip to section 5** (Display Change Summary) with the gap-derived manifest
80
+
81
+ "**Gap-driven update mode.** Translating {gap_count} test report findings into change manifest — source drift detection skipped."
82
+
83
+ **If normal mode:** Continue with source drift detection below.
84
+
59
85
  ### 1. Scan Current Source State
60
86
 
61
87
  Read the source directory at `{source_root}` and build a current file inventory:
@@ -3,6 +3,8 @@ name: 'step-03-re-extract'
3
3
  description: 'Tier-aware AST extraction on changed files only, producing fresh exports with confidence tiers'
4
4
 
5
5
  nextStepFile: './step-04-merge.md'
6
+ extractionPatternsData: '../../create-skill/data/extraction-patterns.md'
7
+ remoteSourceResolutionData: '../data/remote-source-resolution.md'
6
8
  ---
7
9
 
8
10
  # Step 3: Re-Extract Changed Exports
@@ -55,7 +57,26 @@ Perform tier-aware extraction on only the changed files identified in step 02, p
55
57
 
56
58
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
57
59
 
58
- ### 1. Determine Extraction Strategy by Tier
60
+ ### 1. Check for Docs-Only Mode
61
+
62
+ **If `source_type: "docs-only"` in the original brief or metadata:**
63
+
64
+ "**Docs-only skill detected.** This skill was generated from external documentation, not source code. Re-extraction will re-fetch the original `doc_urls` to check for updated content."
65
+
66
+ - Re-fetch each URL from `doc_urls` (from the brief or metadata) using whatever web fetching capability is available
67
+ - Extract updated API information with T3 `[EXT:{url}]` citations
68
+ - Build the updated extraction inventory from fetched content
69
+ - Skip all source code extraction below — proceed directly to the merge step (section 5 or equivalent)
70
+
71
+ **If `source_type: "source"` (default):** Continue with source extraction below.
72
+
73
+ ### 1b. Determine Extraction Strategy by Tier
74
+
75
+ **Remote Source Resolution (Forge/Deep only):**
76
+
77
+ **MCP source access check (before ephemeral clone):** If MCP source-reading tools are available (zread, deepwiki, gh API, or similar) and `source_repo` is set in metadata.json, use MCP tools to fetch only the changed files from the change manifest. This avoids ephemeral clone overhead entirely. MCP provides full source file content equivalent to a local read. If the fetched content is written to a temp file and analyzed with ast-grep, label confidence as T1. If AST is unavailable (the common case for MCP-fetched content), use pattern-based extraction and label confidence as T1-low.
78
+
79
+ **If MCP unavailable:** Load and follow `{remoteSourceResolutionData}` for ephemeral clone setup, version reconciliation, and AST tool unavailability handling.
59
80
 
60
81
  **Quick tier (text pattern matching):**
61
82
  - Extract function/class/type names via regex patterns
@@ -63,7 +84,9 @@ Perform tier-aware extraction on only the changed files identified in step 02, p
63
84
  - Confidence: T1-low (pattern-matched, not AST-verified)
64
85
 
65
86
  **Forge tier (AST structural extraction):**
66
- - Use ast_bridge for full structural extraction
87
+
88
+ ⚠️ **CRITICAL:** Load and follow the **AST Extraction Protocol** from `{extractionPatternsData}`. Use the decision tree based on the number of changed files: prefer MCP `find_code()` for small sets, `find_code_by_rule()` with scoped YAML rules for medium sets, and CLI `--json=stream` with line-by-line streaming for large sets. Never use `ast-grep --json` (without `=stream`) — it loads the entire result set into memory and will fail on large codebases.
89
+
67
90
  - Extract: function signatures, type definitions, class members, exported constants
68
91
  - Extract: parameter types, return types, JSDoc/docstring comments
69
92
  - Confidence: T1 (AST-verified structural truth)
@@ -93,15 +116,26 @@ DO NOT BE LAZY — For EACH file in the change manifest with status MODIFIED, AD
93
116
 
94
117
  **For MOVED files:** Re-extract at new location to update file:line references.
95
118
 
119
+ **Re-export tracing (Forge/Deep only):** After extracting changed files, check if any public exports from the package entry point (`__init__.py`, `index.ts`, `lib.rs`) are unresolved — particularly when a changed file is part of a module re-export chain. Follow the **Re-Export Tracing** protocol in `{extractionPatternsData}` to trace unresolved symbols to their actual definition files.
120
+
96
121
  ### 3. Deep Tier QMD Enrichment (Conditional)
97
122
 
98
123
  **ONLY if forge_tier == Deep:**
99
124
 
125
+ Read the `qmd_collections` registry from `{sidecar_path}/forge-tier.yaml`.
126
+
127
+ Find the collection entry matching the current skill: look for an entry where `skill_name` matches the skill being updated AND `type` is `"extraction"`.
128
+
129
+ **If a matching extraction collection is found:**
100
130
  Launch a subprocess that loads qmd_bridge and for each changed export:
101
- 1. Queries QMD collection for semantic context related to the export
131
+ 1. Queries the `{skill_name}-extraction` collection for semantic context related to the export
102
132
  2. Searches for usage patterns, documentation references, temporal history
103
133
  3. Returns T2 evidence per export (usage frequency, context snippets, related concepts)
104
134
 
135
+ **If no matching collection found in registry:**
136
+ Log: "No QMD extraction collection found for {skill_name}. T2 enrichment skipped. Re-run [CS] Create Skill to generate the collection."
137
+ Continue without T2 enrichment — extraction still produces T1 results.
138
+
105
139
  **If forge_tier != Deep:** Skip this section with notice: "QMD enrichment skipped (tier: {forge_tier})"
106
140
 
107
141
  ### 4. Compile Extraction Results
@@ -149,14 +183,8 @@ Extraction Results:
149
183
 
150
184
  Display: "**Proceeding to merge...**"
151
185
 
152
- #### Menu Handling Logic:
153
-
154
186
  - After extraction results are compiled, immediately load, read entire file, then execute {nextStepFile}
155
-
156
- #### EXECUTION RULES:
157
-
158
187
  - This is an auto-proceed step with no user choices
159
- - Proceed directly to next step after extraction completes
160
188
 
161
189
  ## CRITICAL STEP COMPLETION NOTE
162
190
 
@@ -35,7 +35,7 @@ Merge freshly extracted export data into the existing SKILL.md content while pre
35
35
 
36
36
  - 🎯 Focus ONLY on merging extractions into existing skill content
37
37
  - 🚫 FORBIDDEN to delete or modify [MANUAL] section content
38
- - 🚫 FORBIDDEN to write files — merge produces in-memory result for step 06
38
+ - 🚫 FORBIDDEN to write files — merge produces an edit plan for step 06 to execute
39
39
  - 💬 If [MANUAL] conflicts detected: HALT and present to user for resolution
40
40
  - 💬 If clean merge: auto-proceed without user interaction
41
41
 
@@ -50,7 +50,7 @@ Merge freshly extracted export data into the existing SKILL.md content while pre
50
50
 
51
51
  - Available: extraction results from step 03, existing SKILL.md content, [MANUAL] inventory from step 01, change manifest from step 02
52
52
  - Focus: content merge and [MANUAL] preservation
53
- - Limits: in-memory merge only — no file writes
53
+ - Limits: merge planning only — no file writes (step 06 executes the edit plan)
54
54
  - Dependencies: step 03 must have produced extraction results
55
55
 
56
56
  ## MANDATORY SEQUENCE
@@ -222,7 +222,7 @@ ONLY WHEN all merge operations are complete and any [MANUAL] conflicts have been
222
222
 
223
223
  - ANY [MANUAL] content lost or modified without user consent
224
224
  - Auto-resolving [MANUAL] conflicts without user input
225
- - Writing files during merge (merge is in-memory only)
225
+ - Writing files during merge planning (edits are executed in step 06)
226
226
  - Applying merge out of priority order
227
227
  - Not detecting orphaned [MANUAL] blocks on deleted exports
228
228
  - Not handling stack skill multi-file merge
@@ -54,64 +54,76 @@ Validate the merged skill content against the agentskills.io specification, veri
54
54
 
55
55
  **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
56
56
 
57
- ### 1. Launch Parallel Validation Checks
57
+ ### 1. Check Tool Availability and Validation Timing
58
+
59
+ Run: `npx skill-check -h`
60
+
61
+ - If succeeds: skill-check is available for Checks A, E, F below
62
+ - If fails: Use manual fallback paths in those checks
63
+
64
+ **Important:** Do not assume availability — empirical check required.
65
+
66
+ **Validation timing note:** Step-04 produces an edit plan, not written files. Checks that require files on disk (skill-check Checks A, E, F) will be **deferred to post-write** — step-06 runs them after writing files. Structural checks (B, C, D) validate the planned merge content and run here.
67
+
68
+ ### 2. Launch Parallel Validation Checks
58
69
 
59
70
  Launch subprocesses in parallel for each validation category, aggregating results when complete:
60
71
 
61
- **Check A — Spec Compliance:**
62
- - Validate merged SKILL.md structure against agentskills.io specification (via skills_ref if available)
63
- - Verify required sections present: exports, usage patterns, conventions
64
- - Verify export entries have: name, type, signature, file:line reference
65
- - Flag missing or incomplete sections
72
+ **Check A — Spec Compliance (via skill-check):**
73
+
74
+ **If available**, run: `npx skill-check check <skill-dir> --fix --format json --no-security-scan`
75
+
76
+ Parse JSON output for quality score (0-100), auto-fixed issues, and remaining diagnostics. If `body.max_lines` reported, run: `npx skill-check split-body <skill-dir> --write`
77
+
78
+ **If unavailable**, perform manual check: validate merged SKILL.md structure, verify required sections (exports, usage patterns, conventions), verify export entries have name/type/signature/file:line reference, flag missing sections.
66
79
 
67
80
  **Check B — [MANUAL] Section Integrity:**
68
81
  - Compare [MANUAL] inventory from step 01 against merged content
69
- - Verify every [MANUAL] block from inventory exists in merged result
70
- - Verify [MANUAL] content is byte-identical (zero modification)
71
- - Flag any [MANUAL] blocks that were moved, truncated, or missing
82
+ - Verify every [MANUAL] block exists in merged result, byte-identical (zero modification)
83
+ - Flag any [MANUAL] blocks moved, truncated, or missing
72
84
 
73
85
  **Check C — Confidence Tier Consistency:**
74
86
  - Verify all re-extracted exports have confidence labels (T1/T1-low/T2)
75
- - Verify tier labels match forge tier capabilities:
76
- - Quick tier: only T1-low allowed
77
- - Forge tier: T1 expected (T1-low for degraded operations)
78
- - Deep tier: T1 + T2 expected
87
+ - Verify tier labels match forge tier: Quick=T1-low only, Forge=T1 (T1-low for degraded), Deep=T1+T2
79
88
  - Flag mismatched or missing tier labels
80
89
 
81
90
  **Check D — Provenance Completeness:**
82
- - Verify every export in merged SKILL.md has a provenance map entry
83
- - Verify file:line references point to actual source locations
91
+ - Verify every export has a provenance map entry with valid file:line references
84
92
  - Verify no stale references to old file paths or line numbers
85
93
  - Flag orphaned provenance entries (export removed but provenance remains)
86
94
 
87
- ### 2. Aggregate Validation Results
95
+ **Check E Diff Comparison (via skill-check):**
96
+
97
+ **If available** and previous skill version exists: `npx skill-check diff <original-skill-dir> <updated-skill-dir>`
98
+
99
+ Shows diagnostic changes between original and updated skill. Record diff results as informational context.
100
+
101
+ **If unavailable or no previous version:** Skip with note.
102
+
103
+ **Check F — Security Scan:**
104
+
105
+ **If available**, run: `npx skill-check check <skill-dir> --format json` (security scan enabled by default).
106
+
107
+ Record security findings as advisory warnings — they do not block the update.
108
+
109
+ **If unavailable:** Skip with note: "Security scan skipped — skill-check tool unavailable"
110
+
111
+ ### 3. Aggregate Validation Results
88
112
 
89
113
  Compile results from all checks:
90
114
 
91
115
  ```
92
116
  Validation Results:
93
- spec_compliance:
94
- status: PASS|WARN|FAIL
95
- findings: [{severity, description, location}]
96
-
97
- manual_integrity:
98
- status: PASS|WARN|FAIL
99
- sections_verified: [count]
100
- sections_intact: [count]
101
- findings: [{severity, description, section_name}]
102
-
103
- confidence_consistency:
104
- status: PASS|WARN|FAIL
105
- exports_checked: [count]
106
- findings: [{severity, description, export_name, expected_tier, actual_tier}]
107
-
108
- provenance_completeness:
109
- status: PASS|WARN|FAIL
110
- entries_checked: [count]
111
- findings: [{severity, description, export_name}]
117
+ spec_compliance: {status: PASS|WARN|FAIL, findings: [{severity, description, location}]}
118
+ manual_integrity: {status, sections_verified, sections_intact, findings}
119
+ confidence_consistency: {status, exports_checked, findings}
120
+ provenance_completeness: {status, entries_checked, findings}
121
+ diff_comparison: {status: PASS|SKIP, new_issues, fixed_issues, unchanged}
122
+ security_scan: {status: PASS|WARN|SKIP, findings}
123
+ quality_score: [0-100] # from skill-check, if available
112
124
  ```
113
125
 
114
- ### 3. For Stack Skills — Validate Reference Files
126
+ ### 4. For Stack Skills — Validate Reference Files
115
127
 
116
128
  **ONLY if skill_type == "stack":**
117
129
 
@@ -121,43 +133,28 @@ Repeat checks A-D for each reference file:
121
133
 
122
134
  **If skill_type != "stack":** Skip with notice.
123
135
 
124
- ### 4. Display Validation Summary
136
+ ### 5. Display Validation Summary
125
137
 
126
138
  "**Validation Results:**
127
139
 
128
140
  | Check | Status | Findings |
129
141
  |-------|--------|----------|
130
- | Spec Compliance | {PASS/WARN/FAIL} | {count} findings |
142
+ | Spec Compliance | {PASS/WARN/FAIL} | {count} findings (quality score: {score}/100) |
131
143
  | [MANUAL] Integrity | {PASS/WARN/FAIL} | {count} findings |
132
144
  | Confidence Tiers | {PASS/WARN/FAIL} | {count} findings |
133
145
  | Provenance | {PASS/WARN/FAIL} | {count} findings |
146
+ | Diff Comparison | {PASS/SKIP} | {new} new, {fixed} fixed |
147
+ | Security Scan | {PASS/WARN/SKIP} | {count} findings |
134
148
 
135
149
  **Overall: {ALL_PASS / WARNINGS_FOUND / FAILURES_FOUND}**"
136
150
 
137
- **If WARNINGS or FAILURES found:**
138
-
139
- List each finding with severity and description:
151
+ **If findings exist:** List each with severity, description, and location. Add: "Validation is advisory. Findings do not block the update."
140
152
 
141
- "**Findings:**
142
- 1. [{severity}] {description} — {location}
143
- 2. [{severity}] {description} — {location}
144
- ..."
145
-
146
- "**Note:** Validation is advisory. These findings are reported for your awareness but do not block the update. You may choose to address them after the update completes."
147
-
148
- ### 5. Present MENU OPTIONS
153
+ ### 6. Present MENU OPTIONS
149
154
 
150
155
  Display: "**Proceeding to write updated files...**"
151
156
 
152
- #### Menu Handling Logic:
153
-
154
- - After validation summary is displayed, immediately load, read entire file, then execute {nextStepFile}
155
-
156
- #### EXECUTION RULES:
157
-
158
- - This is an auto-proceed validation step with no user choices
159
- - Validation is advisory — findings are informational, not blocking
160
- - Proceed directly to next step after summary display
157
+ After validation summary is displayed, immediately load, read entire file, then execute {nextStepFile}. This is an auto-proceed step — validation is advisory, not blocking.
161
158
 
162
159
  ## CRITICAL STEP COMPLETION NOTE
163
160
 
@@ -169,19 +166,18 @@ ONLY WHEN all validation checks have completed and findings are displayed will y
169
166
 
170
167
  ### ✅ SUCCESS:
171
168
 
172
- - All four validation checks executed (spec, [MANUAL], confidence, provenance)
169
+ - All six checks executed (spec, [MANUAL], confidence, provenance, diff, security)
170
+ - `npx skill-check check --fix` and `diff` executed if available (or fallbacks used)
171
+ - Quality score captured; auto-fix applied for deterministic issues
173
172
  - Stack skill reference files validated if applicable
174
- - Findings reported with severity and specific locations
175
- - [MANUAL] integrity verified with byte-level comparison
176
- - Validation results displayed clearly
173
+ - Findings reported with severity and specific locations; [MANUAL] integrity verified
177
174
  - Auto-proceeds regardless of findings (advisory mode)
178
175
 
179
176
  ### ❌ SYSTEM FAILURE:
180
177
 
181
- - Skipping any of the four validation checks
182
- - Blocking the workflow on validation warnings
183
- - Not verifying [MANUAL] section integrity
184
- - Hallucinating validation findings not backed by actual comparison
185
- - Modifying merged content during validation
178
+ - Skipping any of the six checks; blocking on validation warnings
179
+ - Not verifying [MANUAL] integrity; hallucinating findings not backed by comparison
180
+ - Modifying merged content during validation (except via skill-check --fix)
181
+ - Not recording quality score when skill-check is available
186
182
 
187
183
  **Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.