bmad-module-skill-forge 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +88 -44
- package/docs/agents.md +1 -1
- package/docs/architecture.md +49 -20
- package/docs/examples.md +13 -2
- package/docs/getting-started.md +15 -13
- package/docs/index.md +3 -1
- package/docs/workflows.md +2 -2
- package/package.json +5 -3
- package/src/forger/forge-tier.yaml +1 -1
- package/src/forger/preferences.yaml +8 -1
- package/src/knowledge/doc-fetcher.md +55 -0
- package/src/knowledge/overview.md +17 -16
- package/src/knowledge/progressive-capability.md +9 -1
- package/src/knowledge/provenance-tracking.md +31 -23
- package/src/knowledge/qmd-registry.md +132 -0
- package/src/knowledge/skf-knowledge-index.csv +2 -0
- package/src/workflows/analyze-source/data/skill-brief-schema.md +24 -4
- package/src/workflows/analyze-source/steps-c/step-01-init.md +19 -9
- package/src/workflows/analyze-source/steps-c/step-01b-continue.md +2 -2
- package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +10 -2
- package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +17 -2
- package/src/workflows/analyze-source/steps-c/step-05-recommend.md +2 -1
- package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +3 -3
- package/src/workflows/analyze-source/templates/analysis-report-template.md +1 -1
- package/src/workflows/analyze-source/validation-report.md +1 -1
- package/src/workflows/analyze-source/workflow-plan-analyze-source.md +1 -1
- package/src/workflows/analyze-source/workflow.md +1 -1
- package/src/workflows/audit-skill/steps-c/step-01-init.md +8 -5
- package/src/workflows/audit-skill/steps-c/step-02-re-index.md +12 -2
- package/src/workflows/audit-skill/validation-report.md +2 -2
- package/src/workflows/audit-skill/workflow-plan-audit-skill.md +1 -1
- package/src/workflows/brief-skill/data/scope-templates.md +60 -0
- package/src/workflows/brief-skill/data/skill-brief-schema.md +38 -3
- package/src/workflows/brief-skill/steps-c/step-01-gather-intent.md +20 -4
- package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +20 -64
- package/src/workflows/brief-skill/steps-c/step-04-confirm-brief.md +1 -1
- package/src/workflows/brief-skill/steps-c/step-05-write-brief.md +60 -5
- package/src/workflows/brief-skill/workflow.md +2 -1
- package/src/workflows/create-skill/data/compile-assembly-rules.md +110 -0
- package/src/workflows/create-skill/data/extraction-patterns.md +248 -0
- package/src/workflows/create-skill/data/skill-sections.md +79 -25
- package/src/workflows/create-skill/data/source-resolution-protocols.md +130 -0
- package/src/workflows/create-skill/steps-c/step-01-load-brief.md +7 -4
- package/src/workflows/create-skill/steps-c/step-02-ecosystem-check.md +10 -7
- package/src/workflows/create-skill/steps-c/step-03-extract.md +49 -9
- package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +238 -0
- package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md +220 -0
- package/src/workflows/create-skill/steps-c/step-04-enrich.md +40 -10
- package/src/workflows/create-skill/steps-c/step-05-compile.md +48 -81
- package/src/workflows/create-skill/steps-c/step-06-validate.md +149 -63
- package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +53 -3
- package/src/workflows/create-skill/steps-c/step-08-report.md +1 -1
- package/src/workflows/create-skill/workflow-plan-create-skill.md +42 -10
- package/src/workflows/create-skill/workflow.md +3 -2
- package/src/workflows/create-stack-skill/data/stack-skill-template.md +20 -7
- package/src/workflows/create-stack-skill/steps-c/step-01-init.md +2 -2
- package/src/workflows/create-stack-skill/steps-c/step-04-parallel-extract.md +14 -0
- package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +28 -7
- package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +10 -7
- package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +66 -68
- package/src/workflows/create-stack-skill/validation-report.md +1 -1
- package/src/workflows/create-stack-skill/workflow-plan-create-stack-skill.md +4 -4
- package/src/workflows/export-skill/data/managed-section-format.md +1 -0
- package/src/workflows/export-skill/data/snippet-format.md +33 -14
- package/src/workflows/export-skill/steps-c/step-01-load-skill.md +16 -2
- package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +29 -22
- package/src/workflows/export-skill/steps-c/step-04-update-context.md +1 -1
- package/src/workflows/export-skill/steps-c/step-05-token-report.md +1 -1
- package/src/workflows/export-skill/validation-report.md +1 -1
- package/src/workflows/export-skill/workflow-plan-export-skill.md +8 -8
- package/src/workflows/quick-skill/data/skill-template.md +27 -7
- package/src/workflows/quick-skill/steps-c/step-04-compile.md +44 -13
- package/src/workflows/quick-skill/steps-c/step-05-validate.md +79 -16
- package/src/workflows/quick-skill/validation-report.md +4 -4
- package/src/workflows/quick-skill/workflow-plan-quick-skill.md +4 -4
- package/src/workflows/quick-skill/workflow.md +2 -0
- package/src/workflows/setup-forge/steps-c/step-01-detect-and-tier.md +12 -3
- package/src/workflows/setup-forge/steps-c/step-02-write-config.md +10 -2
- package/src/workflows/setup-forge/steps-c/step-03-auto-index.md +111 -40
- package/src/workflows/setup-forge/steps-c/step-04-report.md +13 -0
- package/src/workflows/setup-forge/workflow.md +1 -0
- package/src/workflows/test-skill/data/scoring-rules.md +14 -6
- package/src/workflows/test-skill/data/source-access-protocol.md +37 -0
- package/src/workflows/test-skill/steps-c/step-01-init.md +38 -13
- package/src/workflows/test-skill/steps-c/step-02-detect-mode.md +2 -3
- package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +25 -5
- package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +18 -43
- package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +213 -0
- package/src/workflows/test-skill/steps-c/step-05-score.md +24 -27
- package/src/workflows/test-skill/steps-c/step-06-report.md +9 -4
- package/src/workflows/test-skill/templates/test-report-template.md +2 -1
- package/src/workflows/test-skill/validation-report.md +1 -1
- package/src/workflows/test-skill/workflow-plan-test-skill.md +7 -3
- package/src/workflows/test-skill/workflow.md +1 -1
- package/src/workflows/update-skill/data/remote-source-resolution.md +43 -0
- package/src/workflows/update-skill/steps-c/step-01-init.md +13 -7
- package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +26 -0
- package/src/workflows/update-skill/steps-c/step-03-re-extract.md +37 -9
- package/src/workflows/update-skill/steps-c/step-04-merge.md +3 -3
- package/src/workflows/update-skill/steps-c/step-05-validate.md +63 -67
- package/src/workflows/update-skill/steps-c/step-06-write.md +37 -14
- package/src/workflows/update-skill/steps-c/step-07-report.md +1 -1
- package/src/workflows/update-skill/validation-report.md +4 -4
- package/src/workflows/update-skill/workflow-plan-update-skill.md +4 -4
- package/src/workflows/update-skill/workflow.md +1 -1
- package/tools/cli/commands/status.js +4 -4
- package/tools/cli/commands/uninstall.js +1 -1
- package/tools/cli/commands/update.js +2 -2
- package/tools/cli/lib/ui.js +60 -16
- package/tools/cli/lib/version-check.js +2 -2
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
name: 'step-06-report'
|
|
3
3
|
description: 'Generate gap report with remediation suggestions and finalize test report'
|
|
4
4
|
|
|
5
|
-
outputFile: '{
|
|
5
|
+
outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
|
|
6
6
|
scoringRulesFile: '../data/scoring-rules.md'
|
|
7
7
|
outputFormatsFile: '../data/output-section-formats.md'
|
|
8
8
|
---
|
|
@@ -17,7 +17,7 @@ Generate a detailed gap report listing every issue found during coverage and coh
|
|
|
17
17
|
|
|
18
18
|
### Universal Rules:
|
|
19
19
|
|
|
20
|
-
- 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03 and
|
|
20
|
+
- 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03, 04, and 04b
|
|
21
21
|
- 📖 CRITICAL: Read the complete step file before taking any action
|
|
22
22
|
- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
|
|
23
23
|
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
|
@@ -69,6 +69,10 @@ Read `{outputFile}` and extract every issue found across all analysis sections:
|
|
|
69
69
|
- Incomplete integration patterns (contextual mode)
|
|
70
70
|
- Structural issues (naive mode — missing sections, broken examples)
|
|
71
71
|
|
|
72
|
+
**From External Validation (step 04b):**
|
|
73
|
+
- skill-check diagnostics (unresolved errors and warnings)
|
|
74
|
+
- tessl judge suggestions (content quality and actionability improvements)
|
|
75
|
+
|
|
72
76
|
### 2. Load Severity Rules
|
|
73
77
|
|
|
74
78
|
Load `{scoringRulesFile}` for gap severity classification:
|
|
@@ -78,6 +82,7 @@ Load `{scoringRulesFile}` for gap severity classification:
|
|
|
78
82
|
| **Critical** | Missing exported function/class documentation |
|
|
79
83
|
| **High** | Signature mismatch between source and SKILL.md |
|
|
80
84
|
| **Medium** | Missing type or interface documentation |
|
|
85
|
+
| **Medium** | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
|
|
81
86
|
| **Low** | Missing optional metadata or examples |
|
|
82
87
|
| **Info** | Style suggestions, non-blocking observations |
|
|
83
88
|
|
|
@@ -96,7 +101,7 @@ If no gaps found, append a clean pass message recommending **export-skill** work
|
|
|
96
101
|
### 5. Finalize Output Document
|
|
97
102
|
|
|
98
103
|
Update `{outputFile}` frontmatter:
|
|
99
|
-
- Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-05-score', 'step-06-report']`
|
|
104
|
+
- Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-04b-external-validators', 'step-05-score', 'step-06-report']`
|
|
100
105
|
|
|
101
106
|
### 6. Present Final Report
|
|
102
107
|
|
|
@@ -154,7 +159,7 @@ This is the final step of the test-skill workflow. When the user selects C, the
|
|
|
154
159
|
|
|
155
160
|
### ✅ SUCCESS:
|
|
156
161
|
|
|
157
|
-
- Every gap traces to a finding from steps 03 or
|
|
162
|
+
- Every gap traces to a finding from steps 03, 04, or 04b (zero fabrication)
|
|
158
163
|
- Gaps classified by severity using scoring rules
|
|
159
164
|
- Gaps ordered by severity (Critical first)
|
|
160
165
|
- Every gap has a specific, actionable remediation suggestion
|
|
@@ -247,7 +247,7 @@ Each step has explicit CONTEXT BOUNDARIES with FORBIDDEN markers preventing scop
|
|
|
247
247
|
|
|
248
248
|
### Error Handling: Good
|
|
249
249
|
|
|
250
|
-
- step-01: Three-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml)
|
|
250
|
+
- step-01: Three-tier handling (hard stop for missing SKILL.md, warning for missing metadata.json, graceful default for missing forge-tier.yaml). Checks `preferences.yaml` for `tier_override` after loading forge tier.
|
|
251
251
|
- step-03/04: Subprocess fallback rules at multiple locations
|
|
252
252
|
- step-06: Zero-gaps clean pass path
|
|
253
253
|
|
|
@@ -202,6 +202,7 @@ Cognitive completeness verification — Verifies that a skill is complete enough
|
|
|
202
202
|
| 02 | detect-mode | Middle (Simple) | Auto-proceed | Determine naive vs contextual from skill metadata |
|
|
203
203
|
| 03 | coverage-check | Validation Sequence | Auto-proceed | Compare documented exports against source API surface |
|
|
204
204
|
| 04 | coherence-check | Validation Sequence (conditional) | Auto-proceed | Validate references/coherence (contextual=full, naive=simplified) |
|
|
205
|
+
| 04b | external-validators | Middle (Simple) | Auto-proceed | Run external validation tools (skill-check, tessl) and capture scores |
|
|
205
206
|
| 05 | score | Middle (Simple) | Auto-proceed | Calculate completeness score from findings |
|
|
206
207
|
| 06 | report | Final Step | C only | Generate gap report, finalize document, recommend next workflow |
|
|
207
208
|
|
|
@@ -210,7 +211,8 @@ Cognitive completeness verification — Verifies that a skill is complete enough
|
|
|
210
211
|
```
|
|
211
212
|
User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
|
|
212
213
|
[03] coverage-check (auto) → [04] coherence-check (auto) →
|
|
213
|
-
[
|
|
214
|
+
[04b] external-validators (auto) → [05] score (auto) →
|
|
215
|
+
[06] report → [C] finalize
|
|
214
216
|
```
|
|
215
217
|
|
|
216
218
|
### Data Flow
|
|
@@ -219,7 +221,8 @@ User provides skill path → [01] init (auto) → [02] detect-mode (auto) →
|
|
|
219
221
|
- Step 02: reads metadata type → sets {testMode} naive|contextual → appends Test Summary
|
|
220
222
|
- Step 03: reads SKILL.md exports + source files → subprocess AST analysis → appends Coverage Analysis
|
|
221
223
|
- Step 04: reads SKILL.md references → conditional depth by mode → appends Coherence Analysis
|
|
222
|
-
- Step
|
|
224
|
+
- Step 04b: runs skill-check and tessl against skill directory → captures external scores → appends External Validation
|
|
225
|
+
- Step 05: reads coverage + coherence + external validation findings → calculates score → appends Completeness Score
|
|
223
226
|
- Step 06: reads all findings → generates remediation per gap → appends Gap Report → finalizes
|
|
224
227
|
|
|
225
228
|
### Subprocess Optimization
|
|
@@ -246,6 +249,7 @@ test-skill/
|
|
|
246
249
|
│ ├── step-02-detect-mode.md
|
|
247
250
|
│ ├── step-03-coverage-check.md
|
|
248
251
|
│ ├── step-04-coherence-check.md
|
|
252
|
+
│ ├── step-04b-external-validators.md
|
|
249
253
|
│ ├── step-05-score.md
|
|
250
254
|
│ └── step-06-report.md
|
|
251
255
|
└── templates/
|
|
@@ -261,7 +265,7 @@ Ferris (Audit mode): methodical, precise, evidence-based. Zero hallucination —
|
|
|
261
265
|
- **Previous:** create-skill (required — SKILL.md must exist)
|
|
262
266
|
- **Next:** export-skill (if pass) or update-skill (if fail)
|
|
263
267
|
- **Input contract:** `{skills_output_folder}/{name}/SKILL.md` + `metadata.json` + `{sidecar_path}/forge-tier.yaml`
|
|
264
|
-
- **Output contract:** `{
|
|
268
|
+
- **Output contract:** `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md` with frontmatter: workflowType, testResult, score, nextWorkflow
|
|
265
269
|
|
|
266
270
|
### Output Template (Structured)
|
|
267
271
|
|
|
@@ -49,7 +49,7 @@ installed_path: '{project-root}/_bmad/skf/workflows/test-skill'
|
|
|
49
49
|
|
|
50
50
|
Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
|
|
51
51
|
|
|
52
|
-
- `project_name`, `
|
|
52
|
+
- `project_name`, `user_name`, `communication_language`, `document_output_language`
|
|
53
53
|
- `skills_output_folder`, `forge_data_folder`, `sidecar_path`
|
|
54
54
|
|
|
55
55
|
### 2. First Step Execution
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Remote Source Resolution (Forge/Deep Tier)
|
|
2
|
+
|
|
3
|
+
If `source_root` is a local path: proceed with the tier-appropriate strategy as normal.
|
|
4
|
+
|
|
5
|
+
If `source_root` (from metadata.json) is a remote URL (GitHub URL or owner/repo format) AND tier is Forge or Deep:
|
|
6
|
+
|
|
7
|
+
1. **Check `git` availability:** Verify `git` is functional (`git --version`). If `git` is not available, skip to the fallback warning below.
|
|
8
|
+
|
|
9
|
+
2. **Ephemeral sparse clone:** Clone only the changed files from the change manifest to a system temp path. Note: at this point in the flow, `{source_root}` is known to be a remote URL (the local-path case was already handled above).
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
temp_path = {system_temp}/skf-ephemeral-{skill-name}-{timestamp}/
|
|
13
|
+
git clone --depth 1 --single-branch --filter=blob:none --sparse {source_root} {temp_path}
|
|
14
|
+
git -C {temp_path} sparse-checkout set --skip-checks {changed_files_from_manifest}
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
**Note:** `--skip-checks` is required because `changed_files_from_manifest` contains individual file paths (e.g., `src/core/parser.py`), not directories. Without this flag, `git sparse-checkout set` rejects non-directory entries.
|
|
18
|
+
|
|
19
|
+
No `--branch` flag is used — the clone targets the remote's default branch, which must match the branch used during the original [CS] Create Skill run. This scopes the clone to only the files identified in step-02's change manifest, avoiding a full repository download.
|
|
20
|
+
|
|
21
|
+
3. **If clone succeeds:** Update the working source path to `{temp_path}` for all subsequent AST operations in this step. Proceed with the **Forge tier** extraction strategy below. Mark `ephemeral_clone_active = true` for cleanup.
|
|
22
|
+
|
|
23
|
+
4. **If clone fails (network error, auth failure, timeout):**
|
|
24
|
+
|
|
25
|
+
Warning message: "Ephemeral clone of `{source_root}` failed: {error}. Degrading to source reading (T1-low) for this run. For T1 (AST-verified) confidence, clone the repository locally and re-run [CS] Create Skill with the local path, then re-run this update."
|
|
26
|
+
|
|
27
|
+
Override the extraction strategy to Quick tier for this run. Note the degradation reason in context for the evidence report.
|
|
28
|
+
|
|
29
|
+
## Ephemeral Clone Cleanup
|
|
30
|
+
|
|
31
|
+
After extraction is complete for all files in scope (whether successful or partially failed), before presenting the extraction summary, if `ephemeral_clone_active`, delete the `{temp_path}` directory. Log: "Ephemeral source clone cleaned up." This ensures cleanup runs even if some extractions failed, as long as the step itself is still executing.
|
|
32
|
+
|
|
33
|
+
## Version Reconciliation
|
|
34
|
+
|
|
35
|
+
After the source path is accessible, check whether the source version has changed since the original skill was created. Look for the version file matching the detected language (e.g., `pyproject.toml`, `package.json`, `Cargo.toml`). If the source version differs from the current `metadata.json` version, record `source_version_detected` in context for step-06 to use when updating `metadata.json`. No warning needed here — step-06 handles the version update.
|
|
36
|
+
|
|
37
|
+
## AST Tool Unavailability (Local Source)
|
|
38
|
+
|
|
39
|
+
If AST tool is unavailable at Forge/Deep tier with local source:
|
|
40
|
+
|
|
41
|
+
Warning message: "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
|
|
42
|
+
|
|
43
|
+
Degrade to Quick tier extraction. Note the degradation reason in context for the evidence report.
|
|
@@ -62,11 +62,15 @@ Load the existing skill and all its provenance data, detect whether this is an i
|
|
|
62
62
|
Provide either:
|
|
63
63
|
- A skill name (resolves to `{skills_output_folder}/{name}/`)
|
|
64
64
|
- A full path to the skill folder
|
|
65
|
+
- A skill name with `--from-test-report` to use the test report's gap findings instead of source drift detection
|
|
65
66
|
|
|
66
67
|
**Skill:** {user provides path or name}"
|
|
67
68
|
|
|
68
69
|
Resolve the path to an absolute skill folder location.
|
|
69
70
|
|
|
71
|
+
**If `--from-test-report` was provided (or user references a test report):**
|
|
72
|
+
Search for the test report at `{forge_data_folder}/{skill_name}/test-report-{skill_name}.md`. If found, set `test_report_path` in context and `update_mode: gap-driven`. If not found, warn and continue with normal source drift mode.
|
|
73
|
+
|
|
70
74
|
### 2. Validate Required Artifacts
|
|
71
75
|
|
|
72
76
|
**Check SKILL.md exists:**
|
|
@@ -75,19 +79,21 @@ Resolve the path to an absolute skill folder location.
|
|
|
75
79
|
|
|
76
80
|
**Check metadata.json exists:**
|
|
77
81
|
- Load `{resolved_skill_path}/metadata.json`
|
|
78
|
-
- Extract: `
|
|
82
|
+
- Extract: `name`, `skill_type` (single or stack), `version`, `generation_date`, `confidence_tier`, `source_root`
|
|
79
83
|
- If missing: **ABORT** — "No metadata.json found. This skill may have been created manually. Run create-skill to generate provenance data."
|
|
80
84
|
|
|
81
85
|
**Detect skill type from metadata:**
|
|
82
86
|
- If `skill_type == "stack"`: flag as stack skill (multi-file update mode)
|
|
83
|
-
- If `skill_type == "
|
|
87
|
+
- If `skill_type == "single"` or absent: flag as single skill
|
|
84
88
|
|
|
85
89
|
### 3. Load Forge Tier Configuration
|
|
86
90
|
|
|
87
|
-
**Load `
|
|
88
|
-
- Extract: `
|
|
91
|
+
**Load `{sidecar_path}/forge-tier.yaml`:**
|
|
92
|
+
- Extract: `tier` (Quick, Forge, or Deep), available tools
|
|
89
93
|
- If missing: **ABORT** — "No forge-tier.yaml found. Run setup-forge first to detect available tools."
|
|
90
94
|
|
|
95
|
+
**Apply tier override:** Read `{sidecar_path}/preferences.yaml`. If `tier_override` is set and is a valid tier value (Quick, Forge, or Deep), use it instead of the detected tier.
|
|
96
|
+
|
|
91
97
|
**Determine analysis capabilities:**
|
|
92
98
|
- **Quick:** text pattern matching only → T1-low confidence
|
|
93
99
|
- **Forge:** AST structural extraction → T1 confidence
|
|
@@ -146,7 +152,7 @@ Please provide the current source code path:
|
|
|
146
152
|
| Property | Value |
|
|
147
153
|
|----------|-------|
|
|
148
154
|
| **Skill** | {skill_name} |
|
|
149
|
-
| **Type** | {
|
|
155
|
+
| **Type** | {single/stack} |
|
|
150
156
|
| **Version** | {version} |
|
|
151
157
|
| **Created** | {created date} |
|
|
152
158
|
| **Source** | {source_root} |
|
|
@@ -154,7 +160,7 @@ Please provide the current source code path:
|
|
|
154
160
|
| **Provenance Age** | {days} days since last extraction |
|
|
155
161
|
| **Exports** | {export_count} tracked exports |
|
|
156
162
|
| **[MANUAL] Sections** | {manual_count} preserved sections |
|
|
157
|
-
| **Mode** | {normal/degraded} |
|
|
163
|
+
| **Mode** | {normal/degraded/gap-driven} |
|
|
158
164
|
|
|
159
165
|
**Analysis plan:** {tier_description}
|
|
160
166
|
- {Quick: text pattern diff → T1-low findings}
|
|
@@ -188,7 +194,7 @@ ONLY WHEN [C] is selected and baseline has been established with all required ar
|
|
|
188
194
|
### ✅ SUCCESS:
|
|
189
195
|
|
|
190
196
|
- SKILL.md loaded and validated
|
|
191
|
-
- metadata.json loaded with
|
|
197
|
+
- metadata.json loaded with name, skill_type, version, source_root
|
|
192
198
|
- Forge tier loaded from sidecar/forge-tier.yaml
|
|
193
199
|
- Provenance map loaded (or degraded mode confirmed)
|
|
194
200
|
- [MANUAL] sections inventoried across all output files
|
|
@@ -56,6 +56,32 @@ Compare current source code state against the provenance map to produce a comple
|
|
|
56
56
|
|
|
57
57
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
58
58
|
|
|
59
|
+
### 0. Check for Test Report Input (Gap-Driven Mode)
|
|
60
|
+
|
|
61
|
+
**If `update_mode == "gap-driven"` (set in step-01 via `--from-test-report`):**
|
|
62
|
+
|
|
63
|
+
Load the test report at `{test_report_path}` and extract findings:
|
|
64
|
+
|
|
65
|
+
1. Read the **Gap Report** section — each gap entry has severity, category, and description
|
|
66
|
+
2. Read the **Coverage Analysis** section — each per-export row has documented/missing/mismatch status
|
|
67
|
+
3. Translate findings into change manifest format:
|
|
68
|
+
|
|
69
|
+
| Gap Severity | Gap Type | Change Category |
|
|
70
|
+
|-------------|----------|-----------------|
|
|
71
|
+
| Critical | Missing export documentation | NEW_EXPORT (undocumented public API) |
|
|
72
|
+
| High | Signature mismatch | MODIFIED_EXPORT (signature needs update) |
|
|
73
|
+
| Medium | Missing type/interface docs | NEW_EXPORT (undocumented type) |
|
|
74
|
+
| Medium | Stale documentation | MODIFIED_EXPORT (docs reference removed export) |
|
|
75
|
+
| Low | Missing metadata/examples | metadata update |
|
|
76
|
+
|
|
77
|
+
4. Build the change manifest from translated gaps — no file-level timestamp comparison needed since source hasn't changed
|
|
78
|
+
5. Set `gap_count` from the total number of translated entries
|
|
79
|
+
6. **Skip to section 5** (Display Change Summary) with the gap-derived manifest
|
|
80
|
+
|
|
81
|
+
"**Gap-driven update mode.** Translating {gap_count} test report findings into change manifest — source drift detection skipped."
|
|
82
|
+
|
|
83
|
+
**If normal mode:** Continue with source drift detection below.
|
|
84
|
+
|
|
59
85
|
### 1. Scan Current Source State
|
|
60
86
|
|
|
61
87
|
Read the source directory at `{source_root}` and build a current file inventory:
|
|
@@ -3,6 +3,8 @@ name: 'step-03-re-extract'
|
|
|
3
3
|
description: 'Tier-aware AST extraction on changed files only, producing fresh exports with confidence tiers'
|
|
4
4
|
|
|
5
5
|
nextStepFile: './step-04-merge.md'
|
|
6
|
+
extractionPatternsData: '../../create-skill/data/extraction-patterns.md'
|
|
7
|
+
remoteSourceResolutionData: '../data/remote-source-resolution.md'
|
|
6
8
|
---
|
|
7
9
|
|
|
8
10
|
# Step 3: Re-Extract Changed Exports
|
|
@@ -55,7 +57,26 @@ Perform tier-aware extraction on only the changed files identified in step 02, p
|
|
|
55
57
|
|
|
56
58
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
57
59
|
|
|
58
|
-
### 1.
|
|
60
|
+
### 1. Check for Docs-Only Mode
|
|
61
|
+
|
|
62
|
+
**If `source_type: "docs-only"` in the original brief or metadata:**
|
|
63
|
+
|
|
64
|
+
"**Docs-only skill detected.** This skill was generated from external documentation, not source code. Re-extraction will re-fetch the original `doc_urls` to check for updated content."
|
|
65
|
+
|
|
66
|
+
- Re-fetch each URL from `doc_urls` (from the brief or metadata) using whatever web fetching capability is available
|
|
67
|
+
- Extract updated API information with T3 `[EXT:{url}]` citations
|
|
68
|
+
- Build the updated extraction inventory from fetched content
|
|
69
|
+
- Skip all source code extraction below — proceed directly to the merge step (section 5 or equivalent)
|
|
70
|
+
|
|
71
|
+
**If `source_type: "source"` (default):** Continue with source extraction below.
|
|
72
|
+
|
|
73
|
+
### 1b. Determine Extraction Strategy by Tier
|
|
74
|
+
|
|
75
|
+
**Remote Source Resolution (Forge/Deep only):**
|
|
76
|
+
|
|
77
|
+
**MCP source access check (before ephemeral clone):** If MCP source-reading tools are available (zread, deepwiki, gh API, or similar) and `source_repo` is set in metadata.json, use MCP tools to fetch only the changed files from the change manifest. This avoids ephemeral clone overhead entirely. MCP provides full source file content equivalent to a local read. If the fetched content is written to a temp file and analyzed with ast-grep, label confidence as T1. If AST is unavailable (the common case for MCP-fetched content), use pattern-based extraction and label confidence as T1-low.
|
|
78
|
+
|
|
79
|
+
**If MCP unavailable:** Load and follow `{remoteSourceResolutionData}` for ephemeral clone setup, version reconciliation, and AST tool unavailability handling.
|
|
59
80
|
|
|
60
81
|
**Quick tier (text pattern matching):**
|
|
61
82
|
- Extract function/class/type names via regex patterns
|
|
@@ -63,7 +84,9 @@ Perform tier-aware extraction on only the changed files identified in step 02, p
|
|
|
63
84
|
- Confidence: T1-low (pattern-matched, not AST-verified)
|
|
64
85
|
|
|
65
86
|
**Forge tier (AST structural extraction):**
|
|
66
|
-
|
|
87
|
+
|
|
88
|
+
⚠️ **CRITICAL:** Load and follow the **AST Extraction Protocol** from `{extractionPatternsData}`. Use the decision tree based on the number of changed files: prefer MCP `find_code()` for small sets, `find_code_by_rule()` with scoped YAML rules for medium sets, and CLI `--json=stream` with line-by-line streaming for large sets. Never use `ast-grep --json` (without `=stream`) — it loads the entire result set into memory and will fail on large codebases.
|
|
89
|
+
|
|
67
90
|
- Extract: function signatures, type definitions, class members, exported constants
|
|
68
91
|
- Extract: parameter types, return types, JSDoc/docstring comments
|
|
69
92
|
- Confidence: T1 (AST-verified structural truth)
|
|
@@ -93,15 +116,26 @@ DO NOT BE LAZY — For EACH file in the change manifest with status MODIFIED, AD
|
|
|
93
116
|
|
|
94
117
|
**For MOVED files:** Re-extract at new location to update file:line references.
|
|
95
118
|
|
|
119
|
+
**Re-export tracing (Forge/Deep only):** After extracting changed files, check if any public exports from the package entry point (`__init__.py`, `index.ts`, `lib.rs`) are unresolved — particularly when a changed file is part of a module re-export chain. Follow the **Re-Export Tracing** protocol in `{extractionPatternsData}` to trace unresolved symbols to their actual definition files.
|
|
120
|
+
|
|
96
121
|
### 3. Deep Tier QMD Enrichment (Conditional)
|
|
97
122
|
|
|
98
123
|
**ONLY if forge_tier == Deep:**
|
|
99
124
|
|
|
125
|
+
Read the `qmd_collections` registry from `{sidecar_path}/forge-tier.yaml`.
|
|
126
|
+
|
|
127
|
+
Find the collection entry matching the current skill: look for an entry where `skill_name` matches the skill being updated AND `type` is `"extraction"`.
|
|
128
|
+
|
|
129
|
+
**If a matching extraction collection is found:**
|
|
100
130
|
Launch a subprocess that loads qmd_bridge and for each changed export:
|
|
101
|
-
1. Queries
|
|
131
|
+
1. Queries the `{skill_name}-extraction` collection for semantic context related to the export
|
|
102
132
|
2. Searches for usage patterns, documentation references, temporal history
|
|
103
133
|
3. Returns T2 evidence per export (usage frequency, context snippets, related concepts)
|
|
104
134
|
|
|
135
|
+
**If no matching collection found in registry:**
|
|
136
|
+
Log: "No QMD extraction collection found for {skill_name}. T2 enrichment skipped. Re-run [CS] Create Skill to generate the collection."
|
|
137
|
+
Continue without T2 enrichment — extraction still produces T1 results.
|
|
138
|
+
|
|
105
139
|
**If forge_tier != Deep:** Skip this section with notice: "QMD enrichment skipped (tier: {forge_tier})"
|
|
106
140
|
|
|
107
141
|
### 4. Compile Extraction Results
|
|
@@ -149,14 +183,8 @@ Extraction Results:
|
|
|
149
183
|
|
|
150
184
|
Display: "**Proceeding to merge...**"
|
|
151
185
|
|
|
152
|
-
#### Menu Handling Logic:
|
|
153
|
-
|
|
154
186
|
- After extraction results are compiled, immediately load, read entire file, then execute {nextStepFile}
|
|
155
|
-
|
|
156
|
-
#### EXECUTION RULES:
|
|
157
|
-
|
|
158
187
|
- This is an auto-proceed step with no user choices
|
|
159
|
-
- Proceed directly to next step after extraction completes
|
|
160
188
|
|
|
161
189
|
## CRITICAL STEP COMPLETION NOTE
|
|
162
190
|
|
|
@@ -35,7 +35,7 @@ Merge freshly extracted export data into the existing SKILL.md content while pre
|
|
|
35
35
|
|
|
36
36
|
- 🎯 Focus ONLY on merging extractions into existing skill content
|
|
37
37
|
- 🚫 FORBIDDEN to delete or modify [MANUAL] section content
|
|
38
|
-
- 🚫 FORBIDDEN to write files — merge produces
|
|
38
|
+
- 🚫 FORBIDDEN to write files — merge produces an edit plan for step 06 to execute
|
|
39
39
|
- 💬 If [MANUAL] conflicts detected: HALT and present to user for resolution
|
|
40
40
|
- 💬 If clean merge: auto-proceed without user interaction
|
|
41
41
|
|
|
@@ -50,7 +50,7 @@ Merge freshly extracted export data into the existing SKILL.md content while pre
|
|
|
50
50
|
|
|
51
51
|
- Available: extraction results from step 03, existing SKILL.md content, [MANUAL] inventory from step 01, change manifest from step 02
|
|
52
52
|
- Focus: content merge and [MANUAL] preservation
|
|
53
|
-
- Limits:
|
|
53
|
+
- Limits: merge planning only — no file writes (step 06 executes the edit plan)
|
|
54
54
|
- Dependencies: step 03 must have produced extraction results
|
|
55
55
|
|
|
56
56
|
## MANDATORY SEQUENCE
|
|
@@ -222,7 +222,7 @@ ONLY WHEN all merge operations are complete and any [MANUAL] conflicts have been
|
|
|
222
222
|
|
|
223
223
|
- ANY [MANUAL] content lost or modified without user consent
|
|
224
224
|
- Auto-resolving [MANUAL] conflicts without user input
|
|
225
|
-
- Writing files during merge (
|
|
225
|
+
- Writing files during merge planning (edits are executed in step 06)
|
|
226
226
|
- Applying merge out of priority order
|
|
227
227
|
- Not detecting orphaned [MANUAL] blocks on deleted exports
|
|
228
228
|
- Not handling stack skill multi-file merge
|
|
@@ -54,64 +54,76 @@ Validate the merged skill content against the agentskills.io specification, veri
|
|
|
54
54
|
|
|
55
55
|
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
56
56
|
|
|
57
|
-
### 1.
|
|
57
|
+
### 1. Check Tool Availability and Validation Timing
|
|
58
|
+
|
|
59
|
+
Run: `npx skill-check -h`
|
|
60
|
+
|
|
61
|
+
- If succeeds: skill-check is available for Checks A, E, F below
|
|
62
|
+
- If fails: Use manual fallback paths in those checks
|
|
63
|
+
|
|
64
|
+
**Important:** Do not assume availability — empirical check required.
|
|
65
|
+
|
|
66
|
+
**Validation timing note:** Step-04 produces an edit plan, not written files. Checks that require files on disk (skill-check Checks A, E, F) will be **deferred to post-write** — step-06 runs them after writing files. Structural checks (B, C, D) validate the planned merge content and run here.
|
|
67
|
+
|
|
68
|
+
### 2. Launch Parallel Validation Checks
|
|
58
69
|
|
|
59
70
|
Launch subprocesses in parallel for each validation category, aggregating results when complete:
|
|
60
71
|
|
|
61
|
-
**Check A — Spec Compliance:**
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
-
|
|
72
|
+
**Check A — Spec Compliance (via skill-check):**
|
|
73
|
+
|
|
74
|
+
**If available**, run: `npx skill-check check <skill-dir> --fix --format json --no-security-scan`
|
|
75
|
+
|
|
76
|
+
Parse JSON output for quality score (0-100), auto-fixed issues, and remaining diagnostics. If `body.max_lines` reported, run: `npx skill-check split-body <skill-dir> --write`
|
|
77
|
+
|
|
78
|
+
**If unavailable**, perform manual check: validate merged SKILL.md structure, verify required sections (exports, usage patterns, conventions), verify export entries have name/type/signature/file:line reference, flag missing sections.
|
|
66
79
|
|
|
67
80
|
**Check B — [MANUAL] Section Integrity:**
|
|
68
81
|
- Compare [MANUAL] inventory from step 01 against merged content
|
|
69
|
-
- Verify every [MANUAL] block
|
|
70
|
-
-
|
|
71
|
-
- Flag any [MANUAL] blocks that were moved, truncated, or missing
|
|
82
|
+
- Verify every [MANUAL] block exists in merged result, byte-identical (zero modification)
|
|
83
|
+
- Flag any [MANUAL] blocks moved, truncated, or missing
|
|
72
84
|
|
|
73
85
|
**Check C — Confidence Tier Consistency:**
|
|
74
86
|
- Verify all re-extracted exports have confidence labels (T1/T1-low/T2)
|
|
75
|
-
- Verify tier labels match forge tier
|
|
76
|
-
- Quick tier: only T1-low allowed
|
|
77
|
-
- Forge tier: T1 expected (T1-low for degraded operations)
|
|
78
|
-
- Deep tier: T1 + T2 expected
|
|
87
|
+
- Verify tier labels match forge tier: Quick=T1-low only, Forge=T1 (T1-low for degraded), Deep=T1+T2
|
|
79
88
|
- Flag mismatched or missing tier labels
|
|
80
89
|
|
|
81
90
|
**Check D — Provenance Completeness:**
|
|
82
|
-
- Verify every export
|
|
83
|
-
- Verify file:line references point to actual source locations
|
|
91
|
+
- Verify every export has a provenance map entry with valid file:line references
|
|
84
92
|
- Verify no stale references to old file paths or line numbers
|
|
85
93
|
- Flag orphaned provenance entries (export removed but provenance remains)
|
|
86
94
|
|
|
87
|
-
|
|
95
|
+
**Check E — Diff Comparison (via skill-check):**
|
|
96
|
+
|
|
97
|
+
**If available** and previous skill version exists: `npx skill-check diff <original-skill-dir> <updated-skill-dir>`
|
|
98
|
+
|
|
99
|
+
Shows diagnostic changes between original and updated skill. Record diff results as informational context.
|
|
100
|
+
|
|
101
|
+
**If unavailable or no previous version:** Skip with note.
|
|
102
|
+
|
|
103
|
+
**Check F — Security Scan:**
|
|
104
|
+
|
|
105
|
+
**If available**, run: `npx skill-check check <skill-dir> --format json` (security scan enabled by default).
|
|
106
|
+
|
|
107
|
+
Record security findings as advisory warnings — they do not block the update.
|
|
108
|
+
|
|
109
|
+
**If unavailable:** Skip with note: "Security scan skipped — skill-check tool unavailable"
|
|
110
|
+
|
|
111
|
+
### 3. Aggregate Validation Results
|
|
88
112
|
|
|
89
113
|
Compile results from all checks:
|
|
90
114
|
|
|
91
115
|
```
|
|
92
116
|
Validation Results:
|
|
93
|
-
spec_compliance:
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
sections_intact: [count]
|
|
101
|
-
findings: [{severity, description, section_name}]
|
|
102
|
-
|
|
103
|
-
confidence_consistency:
|
|
104
|
-
status: PASS|WARN|FAIL
|
|
105
|
-
exports_checked: [count]
|
|
106
|
-
findings: [{severity, description, export_name, expected_tier, actual_tier}]
|
|
107
|
-
|
|
108
|
-
provenance_completeness:
|
|
109
|
-
status: PASS|WARN|FAIL
|
|
110
|
-
entries_checked: [count]
|
|
111
|
-
findings: [{severity, description, export_name}]
|
|
117
|
+
spec_compliance: {status: PASS|WARN|FAIL, findings: [{severity, description, location}]}
|
|
118
|
+
manual_integrity: {status, sections_verified, sections_intact, findings}
|
|
119
|
+
confidence_consistency: {status, exports_checked, findings}
|
|
120
|
+
provenance_completeness: {status, entries_checked, findings}
|
|
121
|
+
diff_comparison: {status: PASS|SKIP, new_issues, fixed_issues, unchanged}
|
|
122
|
+
security_scan: {status: PASS|WARN|SKIP, findings}
|
|
123
|
+
quality_score: [0-100] # from skill-check, if available
|
|
112
124
|
```
|
|
113
125
|
|
|
114
|
-
###
|
|
126
|
+
### 4. For Stack Skills — Validate Reference Files
|
|
115
127
|
|
|
116
128
|
**ONLY if skill_type == "stack":**
|
|
117
129
|
|
|
@@ -121,43 +133,28 @@ Repeat checks A-D for each reference file:
|
|
|
121
133
|
|
|
122
134
|
**If skill_type != "stack":** Skip with notice.
|
|
123
135
|
|
|
124
|
-
###
|
|
136
|
+
### 5. Display Validation Summary
|
|
125
137
|
|
|
126
138
|
"**Validation Results:**
|
|
127
139
|
|
|
128
140
|
| Check | Status | Findings |
|
|
129
141
|
|-------|--------|----------|
|
|
130
|
-
| Spec Compliance | {PASS/WARN/FAIL} | {count} findings |
|
|
142
|
+
| Spec Compliance | {PASS/WARN/FAIL} | {count} findings (quality score: {score}/100) |
|
|
131
143
|
| [MANUAL] Integrity | {PASS/WARN/FAIL} | {count} findings |
|
|
132
144
|
| Confidence Tiers | {PASS/WARN/FAIL} | {count} findings |
|
|
133
145
|
| Provenance | {PASS/WARN/FAIL} | {count} findings |
|
|
146
|
+
| Diff Comparison | {PASS/SKIP} | {new} new, {fixed} fixed |
|
|
147
|
+
| Security Scan | {PASS/WARN/SKIP} | {count} findings |
|
|
134
148
|
|
|
135
149
|
**Overall: {ALL_PASS / WARNINGS_FOUND / FAILURES_FOUND}**"
|
|
136
150
|
|
|
137
|
-
**If
|
|
138
|
-
|
|
139
|
-
List each finding with severity and description:
|
|
151
|
+
**If findings exist:** List each with severity, description, and location. Add: "Validation is advisory. Findings do not block the update."
|
|
140
152
|
|
|
141
|
-
|
|
142
|
-
1. [{severity}] {description} — {location}
|
|
143
|
-
2. [{severity}] {description} — {location}
|
|
144
|
-
..."
|
|
145
|
-
|
|
146
|
-
"**Note:** Validation is advisory. These findings are reported for your awareness but do not block the update. You may choose to address them after the update completes."
|
|
147
|
-
|
|
148
|
-
### 5. Present MENU OPTIONS
|
|
153
|
+
### 6. Present MENU OPTIONS
|
|
149
154
|
|
|
150
155
|
Display: "**Proceeding to write updated files...**"
|
|
151
156
|
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
- After validation summary is displayed, immediately load, read entire file, then execute {nextStepFile}
|
|
155
|
-
|
|
156
|
-
#### EXECUTION RULES:
|
|
157
|
-
|
|
158
|
-
- This is an auto-proceed validation step with no user choices
|
|
159
|
-
- Validation is advisory — findings are informational, not blocking
|
|
160
|
-
- Proceed directly to next step after summary display
|
|
157
|
+
After validation summary is displayed, immediately load, read entire file, then execute {nextStepFile}. This is an auto-proceed step — validation is advisory, not blocking.
|
|
161
158
|
|
|
162
159
|
## CRITICAL STEP COMPLETION NOTE
|
|
163
160
|
|
|
@@ -169,19 +166,18 @@ ONLY WHEN all validation checks have completed and findings are displayed will y
|
|
|
169
166
|
|
|
170
167
|
### ✅ SUCCESS:
|
|
171
168
|
|
|
172
|
-
- All
|
|
169
|
+
- All six checks executed (spec, [MANUAL], confidence, provenance, diff, security)
|
|
170
|
+
- `npx skill-check check --fix` and `diff` executed if available (or fallbacks used)
|
|
171
|
+
- Quality score captured; auto-fix applied for deterministic issues
|
|
173
172
|
- Stack skill reference files validated if applicable
|
|
174
|
-
- Findings reported with severity and specific locations
|
|
175
|
-
- [MANUAL] integrity verified with byte-level comparison
|
|
176
|
-
- Validation results displayed clearly
|
|
173
|
+
- Findings reported with severity and specific locations; [MANUAL] integrity verified
|
|
177
174
|
- Auto-proceeds regardless of findings (advisory mode)
|
|
178
175
|
|
|
179
176
|
### ❌ SYSTEM FAILURE:
|
|
180
177
|
|
|
181
|
-
- Skipping any of the
|
|
182
|
-
-
|
|
183
|
-
-
|
|
184
|
-
-
|
|
185
|
-
- Modifying merged content during validation
|
|
178
|
+
- Skipping any of the six checks; blocking on validation warnings
|
|
179
|
+
- Not verifying [MANUAL] integrity; hallucinating findings not backed by comparison
|
|
180
|
+
- Modifying merged content during validation (except via skill-check --fix)
|
|
181
|
+
- Not recording quality score when skill-check is available
|
|
186
182
|
|
|
187
183
|
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|