bmad-module-skill-forge 0.2.0

package/src/workflows/audit-skill/steps-c/step-04-semantic-diff.md

@@ -0,0 +1,199 @@
+---
+name: 'step-04-semantic-diff'
+description: 'Compare QMD knowledge context for meaning-level changes (Deep tier only, skip at Quick/Forge)'
+
+nextStepFile: './step-05-severity-classify.md'
+outputFile: '{forge_data_folder}/{skill_name}/drift-report-{timestamp}.md'
+---
+
+# Step 4: Semantic Diff
+
+## STEP GOAL:
+
+Compare QMD knowledge context between the original skill creation and current state to detect meaning-level changes that structural diff cannot catch. This step executes ONLY at Deep tier — at Quick and Forge tiers, it appends a skip notice and auto-proceeds.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 🛑 NEVER generate content without user input
+- 📖 CRITICAL: Read the complete step file before taking any action
+- 🔄 CRITICAL: When loading next step with 'C', ensure entire file is read
+- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
+- ✅ YOU MUST ALWAYS SPEAK OUTPUT in your Agent communication style with the config `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are a skill auditor performing semantic analysis
+- ✅ Semantic findings have T2 confidence — evidence-backed but not structurally absolute
+- ✅ Distinguish clearly between structural facts (T1) and semantic observations (T2)
+
+### Step-Specific Rules:
+
+- 🎯 Focus only on semantic/meaning-level changes via QMD context
+- 🚫 FORBIDDEN to classify severity — that happens in Step 05
+- 🚫 FORBIDDEN to repeat structural findings from Step 03
+- 💬 Use subprocess Pattern 3 (data operations) when available for QMD queries
+- ⚙️ If subprocess unavailable, query QMD in main thread
+- 🔀 CONDITIONAL: Skip entire analysis at Quick/Forge tier — append skip notice only
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Check forge tier FIRST — skip if not Deep
+- 💾 Append ## Semantic Drift section (or skip notice) to {outputFile}
+- 📖 Update {outputFile} frontmatter stepsCompleted when complete
+- 🚫 Only semantic findings — no structural or severity analysis
+
+## CONTEXT BOUNDARIES:
+
+- Available: Forge tier, structural drift findings (from Step 03), QMD access (Deep tier only)
+- Focus: Meaning-level changes that complement structural diff
+- Limits: Do not re-analyze structural changes, do not classify severity
+- Dependencies: Steps 01-03 must be complete; QMD collection must exist for Deep tier
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
+
+### 1. Check Forge Tier
+
+**IF forge tier is Quick or Forge:**
+
+Append to {outputFile}:
+
+```markdown
+## Semantic Drift
+
+**Status:** Skipped — Semantic diff requires Deep tier (current tier: {tier})
+
+Semantic analysis compares QMD knowledge context for meaning-level changes that structural diff cannot detect. To enable semantic diff, run setup-forge with QMD available to unlock Deep tier.
+```
+
+Update frontmatter: append `'step-04-semantic-diff'` to `stepsCompleted`
+
+"**Semantic diff skipped (requires Deep tier). Proceeding to severity classification...**"
+
+→ Auto-proceed to {nextStepFile}
+
+**IF forge tier is Deep:**
+
+Continue to section 2.
+
+### 2. Query Original Knowledge Context
+
+Launch a subprocess (Pattern 3 — data operations) that:
+1. Loads the QMD collection for the project
+2. Queries for knowledge context around each export documented in the skill
+3. Retrieves: usage patterns, conventions, architectural context, dependency relationships
+4. Returns structured findings to parent
+
+**If subprocess unavailable:** Query QMD in main thread.
+
+### 3. Compare Knowledge Context
+
+For each export in the skill, compare original context (from skill creation) against current context (from QMD):
+
+**Detect:**
+- **New patterns:** Usage patterns that have emerged since skill was created
+- **Changed conventions:** Project conventions that have shifted (e.g., new error handling pattern)
+- **Dependency shifts:** Libraries or modules that exports now depend on differently
+- **Architectural changes:** Structural reorganization affecting how exports relate to each other
+- **Deprecated patterns:** Usage patterns documented in skill that are no longer followed
+
+For each finding, record:
+- What changed (description)
+- Evidence (QMD reference or source citation)
+- Affected exports
+- Confidence: T2
+
+### 4. Compile Semantic Drift Section
+
+Append to {outputFile}:
+
+```markdown
+## Semantic Drift
+
+**Method:** QMD knowledge context comparison (Deep tier)
+**QMD Collection:** {collection_name}
+
+### New Patterns Detected ({count})
+
+| Pattern | Description | Affected Exports | Evidence | Confidence |
+|---------|------------|-----------------|----------|------------|
+| {pattern} | {description} | {exports} | {evidence} | T2 |
+
+### Changed Conventions ({count})
+
+| Convention | Before | After | Affected Exports | Evidence | Confidence |
+|-----------|--------|-------|-----------------|----------|------------|
+| {convention} | {old} | {new} | {exports} | {evidence} | T2 |
+
+### Dependency Shifts ({count})
+
+| Export | Original Dependencies | Current Dependencies | Change | Confidence |
+|--------|---------------------|---------------------|--------|------------|
+| {export} | {old_deps} | {new_deps} | {description} | T2 |
+
+### Deprecated Patterns ({count})
+
+| Pattern | Documented In Skill | Current Status | Evidence | Confidence |
+|---------|-------------------|----------------|----------|------------|
+| {pattern} | {skill_reference} | {status} | {evidence} | T2 |
+
+### Summary
+
+| Category | Count |
+|----------|-------|
+| New patterns | {count} |
+| Changed conventions | {count} |
+| Dependency shifts | {count} |
+| Deprecated patterns | {count} |
+| **Total Semantic Items** | {total} |
+```
+
+### 5. Update Report and Auto-Proceed
+
+Update {outputFile} frontmatter:
+- Append `'step-04-semantic-diff'` to `stepsCompleted`
+
+### 6. Present MENU OPTIONS
+
+Display: "**Semantic diff complete. {total} semantic drift items found. Proceeding to severity classification...**"
+
+#### Menu Handling Logic:
+
+- After semantic diff section is appended and frontmatter updated, immediately load, read entire file, then execute {nextStepFile}
+
+#### EXECUTION RULES:
+
+- This is an auto-proceed analysis step with no user choices
+- Proceed directly to next step after completion
+
+## CRITICAL STEP COMPLETION NOTE
+
+ONLY WHEN the ## Semantic Drift section (or skip notice) has been appended to {outputFile} will you then load and read fully `{nextStepFile}` to execute and begin severity classification.
+
+---
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS
+
+### ✅ SUCCESS:
+
+- Forge tier checked FIRST — skip gracefully at Quick/Forge
+- Deep tier: QMD knowledge context queried and compared
+- All semantic findings have T2 confidence labels
+- Findings clearly distinguished from structural drift (T1)
+- Skip notice includes explanation of what semantic diff provides
+- Structured table format in drift report
+- Frontmatter stepsCompleted updated
+
+### ❌ SYSTEM FAILURE:
+
+- Not checking tier before attempting QMD analysis
+- Repeating structural findings from Step 03
+- Missing confidence tier labels on findings
+- Classifying severity in this step (severity is Step 05)
+- Failing to skip gracefully at non-Deep tiers
+- Hardcoded paths instead of frontmatter variables
+
+**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.

package/src/workflows/audit-skill/steps-c/step-05-severity-classify.md

@@ -0,0 +1,210 @@
+---
+name: 'step-05-severity-classify'
+description: 'Grade every drift finding by severity using classification rules'
+
+nextStepFile: './step-06-report.md'
+outputFile: '{forge_data_folder}/{skill_name}/drift-report-{timestamp}.md'
+severityRulesFile: '../data/severity-rules.md'
+---
+
+# Step 5: Severity Classification
+
+## STEP GOAL:
+
+Grade every drift finding from Steps 03 and 04 by severity level (CRITICAL/HIGH/MEDIUM/LOW) using the classification rules. Calculate the overall drift score and produce a categorized findings table with confidence tier labels.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 🛑 NEVER generate content without user input
+- 📖 CRITICAL: Read the complete step file before taking any action
+- 🔄 CRITICAL: When loading next step with 'C', ensure entire file is read
+- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
+- ✅ YOU MUST ALWAYS SPEAK OUTPUT in your Agent communication style with the config `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are a skill auditor performing severity assessment
+- ✅ Classification must be deterministic — same finding always gets same severity
+- ✅ Apply rules strictly — do not invent new severity categories or soften classifications
+
+### Step-Specific Rules:
+
+- 🎯 Focus only on classifying severity of existing findings
+- 🚫 FORBIDDEN to discover new drift items — only classify what Steps 03-04 found
+- 🚫 FORBIDDEN to suggest remediation — that happens in Step 06
+- 💬 Use subprocess Pattern 3 (data operations) to load severity rules and classify
+- ⚙️ If subprocess unavailable, load rules and classify in main thread
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Load {severityRulesFile} and apply to every finding
+- 💾 Append ## Severity Classification section to {outputFile}
+- 📖 Update {outputFile} frontmatter stepsCompleted when complete
+- 🚫 Only classification — no new findings, no remediation
+
+## CONTEXT BOUNDARIES:
+
+- Available: Structural drift findings (Step 03), semantic drift findings (Step 04 if Deep), severity rules
+- Focus: Applying deterministic severity classification to every finding
+- Limits: Do not discover new drift, do not suggest fixes
+- Dependencies: Steps 03 (and optionally 04) must be complete
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
+
+### 1. Load Severity Rules
+
+Launch a subprocess (Pattern 3 — data operations) that:
+1. Loads {severityRulesFile}
+2. Extracts classification criteria for each severity level
+3. Returns structured rules to parent
+
+**If subprocess unavailable:** Load {severityRulesFile} directly in main thread.
+
+**Rules summary:**
+- **CRITICAL:** Removed/renamed exports, changed signatures (breaking changes)
+- **HIGH:** New public API not in skill (>3), removed helpers used in patterns, deprecated APIs
+- **MEDIUM:** Implementation changes behind stable API, 1-3 new exports, moved functions
+- **LOW:** Style/convention changes, comments, whitespace, internal functions
+
+### 2. Collect All Findings
+
+Gather all drift items from the report:
+
+**From ## Structural Drift (Step 03):**
+- Added exports
+- Removed exports
+- Changed exports
+
+**From ## Semantic Drift (Step 04, if Deep tier):**
+- New patterns
+- Changed conventions
+- Dependency shifts
+- Deprecated patterns
+
+Count total findings to classify.
+
+### 3. Apply Severity Classification
+
+For EACH finding, apply the severity rules:
+
+**Structural findings classification:**
+- Removed export → CRITICAL (breaking: skill references something that no longer exists)
+- Changed signature → CRITICAL (breaking: skill documents wrong parameters/return type)
+- Renamed export → CRITICAL (breaking: skill references old name)
+- Moved export (same signature) → MEDIUM (non-breaking but location in skill is wrong)
+- Added export (>3 total) → HIGH (significant API surface not documented)
+- Added export (1-3 total) → MEDIUM (minor gap in coverage)
+
+**Semantic findings classification:**
+- Deprecated pattern still in skill → HIGH (skill teaches outdated approach)
+- Changed convention → MEDIUM (skill may use old style)
+- New pattern detected → MEDIUM (skill doesn't cover new approach)
+- Dependency shift → MEDIUM (skill may reference wrong dependencies)
+
+Record for each finding: original finding + assigned severity level.
+
+### 4. Calculate Overall Drift Score
+
+Apply scoring rules from {severityRulesFile}:
+
+| Score | Criteria |
+|-------|----------|
+| **CLEAN** | 0 findings at any level |
+| **MINOR** | LOW findings only, no MEDIUM+ |
+| **SIGNIFICANT** | Any MEDIUM or HIGH findings, no CRITICAL |
+| **CRITICAL** | Any CRITICAL findings present |
+
+### 5. Compile Severity Classification Section
+
+Append to {outputFile}:
+
+```markdown
+## Severity Classification
+
+**Overall Drift Score: {CLEAN / MINOR / SIGNIFICANT / CRITICAL}**
+
+### CRITICAL ({count})
+
+| # | Finding | Type | Detail | Location | Confidence |
+|---|---------|------|--------|----------|------------|
+| 1 | {finding} | {structural/semantic} | {detail} | {file}:{line} | {T1/T2} |
+
+### HIGH ({count})
+
+| # | Finding | Type | Detail | Location | Confidence |
+|---|---------|------|--------|----------|------------|
+| 1 | {finding} | {structural/semantic} | {detail} | {file}:{line} | {T1/T2} |
+
+### MEDIUM ({count})
+
+| # | Finding | Type | Detail | Location | Confidence |
+|---|---------|------|--------|----------|------------|
+| 1 | {finding} | {structural/semantic} | {detail} | {file}:{line} | {T1/T2} |
+
+### LOW ({count})
+
+| # | Finding | Type | Detail | Location | Confidence |
+|---|---------|------|--------|----------|------------|
+| 1 | {finding} | {structural/semantic} | {detail} | {file}:{line} | {T1/T2} |
+
+### Classification Summary
+
+| Severity | Count | Source |
+|----------|-------|--------|
+| CRITICAL | {count} | {structural: N, semantic: N} |
+| HIGH | {count} | {structural: N, semantic: N} |
+| MEDIUM | {count} | {structural: N, semantic: N} |
+| LOW | {count} | {structural: N, semantic: N} |
+| **Total** | {total} | |
+```
+
+### 6. Update Report and Auto-Proceed
+
+Update {outputFile} frontmatter:
+- Append `'step-05-severity-classify'` to `stepsCompleted`
+- Set `drift_score` to calculated overall score
+
+### 7. Present MENU OPTIONS
+
+Display: "**Severity classification complete. Overall drift score: {score}. Proceeding to report generation...**"
+
+#### Menu Handling Logic:
+
+- After severity classification section is appended and frontmatter updated, immediately load, read entire file, then execute {nextStepFile}
+
+#### EXECUTION RULES:
+
+- This is an auto-proceed analysis step with no user choices
+- Proceed directly to next step after completion
+
+## CRITICAL STEP COMPLETION NOTE
+
+ONLY WHEN the ## Severity Classification section has been appended to {outputFile} with all findings classified will you then load and read fully `{nextStepFile}` to execute and begin final report generation.
+
+---
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS
+
+### ✅ SUCCESS:
+
+- Severity rules loaded from {severityRulesFile}
+- Every finding from Steps 03-04 classified (none missed)
+- Classification is deterministic — follows rules strictly
+- Overall drift score calculated correctly
+- Structured tables with severity, source type, and confidence
+- Frontmatter drift_score and stepsCompleted updated
+
+### ❌ SYSTEM FAILURE:
+
+- Inventing new severity categories beyond CRITICAL/HIGH/MEDIUM/LOW
+- Missing any finding from classification (incomplete coverage)
+- Softening classifications that should be CRITICAL
+- Discovering new drift items (classification only, not discovery)
+- Suggesting remediation (that is Step 06)
+- Hardcoded paths instead of frontmatter variables
+
+**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.

package/src/workflows/audit-skill/steps-c/step-06-report.md

@@ -0,0 +1,217 @@
+---
+name: 'step-06-report'
+description: 'Finalize drift report with audit summary, remediation suggestions, and provenance metadata'
+
+outputFile: '{forge_data_folder}/{skill_name}/drift-report-{timestamp}.md'
+---
+
+# Step 6: Generate Report
+
+## STEP GOAL:
+
+Finalize the drift report by completing the Audit Summary with calculated metrics, generating actionable remediation suggestions for each drift finding, and adding provenance metadata. Present the final report to the user with a next-workflow recommendation.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 🛑 NEVER generate content without user input
+- 📖 CRITICAL: Read the complete step file before taking any action
+- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
+- ✅ YOU MUST ALWAYS SPEAK OUTPUT in your Agent communication style with the config `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are a skill auditor delivering the final audit report
+- ✅ Remediation suggestions must be specific and actionable
+- ✅ Every suggestion must reference the specific drift finding it addresses
+
+### Step-Specific Rules:
+
+- 🎯 Focus on completing the report — summary, remediation, provenance
+- 🚫 FORBIDDEN to discover new drift items or reclassify severity
+- 💬 Remediation suggestions must be practical: what to change, where, and why
+- 📋 This is the FINAL step — no nextStepFile
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Complete all remaining sections in {outputFile}
+- 💾 Update {outputFile} frontmatter to mark workflow complete
+- 📖 Present final summary to user
+- 🚫 Do not re-analyze — only synthesize and present
+
+## CONTEXT BOUNDARIES:
+
+- Available: Complete drift report with structural diff, semantic diff, severity classification
+- Focus: Synthesis, remediation guidance, and presentation
+- Limits: Do not add new findings or reclassify — only complete the report
+- Dependencies: Steps 01-05 must be complete
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
+
+### 1. Complete Audit Summary
+
+Update the ## Audit Summary section at the top of {outputFile} with final calculated values:
+
+- Fill in severity count table from Step 05 classification summary
+- Set overall drift score
+- Add total findings count
+
+### 2. Generate Remediation Suggestions
+
+For EACH classified drift finding, generate a specific remediation suggestion:
+
+**CRITICAL findings remediation:**
+- Removed export → "Remove reference to `{export_name}` from SKILL.md section {section}. Export no longer exists at `{file}:{line}`."
+- Changed signature → "Update `{export_name}` signature in SKILL.md from `{old_signature}` to `{new_signature}`. See `{file}:{line}`."
+- Renamed export → "Replace `{old_name}` with `{new_name}` throughout SKILL.md. Renamed at `{file}:{line}`."
+
+**HIGH findings remediation:**
+- New public API (>3) → "Add documentation for {count} new exports to SKILL.md: {export_list}. Consider running update-skill workflow."
+- Deprecated API → "Mark `{export_name}` as deprecated in SKILL.md. Current replacement: `{replacement}` at `{file}:{line}`."
+
+**MEDIUM findings remediation:**
+- Moved function → "Update file reference for `{export_name}` from `{old_file}` to `{new_file}:{line}`."
+- New exports (1-3) → "Consider adding `{export_names}` to SKILL.md for completeness."
+- Changed convention → "Review convention documentation in SKILL.md for currency."
+
+**LOW findings remediation:**
+- Style changes → "Optional: Update style references in SKILL.md to reflect current conventions."
+
+Append to {outputFile}:
+
+```markdown
+## Remediation Suggestions
+
+### Priority Actions (CRITICAL + HIGH)
+
+| # | Severity | Finding | Remediation | Effort |
+|---|----------|---------|-------------|--------|
+| 1 | {severity} | {finding} | {specific action} | {low/medium/high} |
+
+### Recommended Updates (MEDIUM)
+
+| # | Finding | Remediation | Effort |
+|---|---------|-------------|--------|
+| 1 | {finding} | {specific action} | {low/medium} |
+
+### Optional Improvements (LOW)
+
+| # | Finding | Remediation |
+|---|---------|-------------|
+| 1 | {finding} | {specific action} |
+
+### Workflow Recommendation
+
+{IF any CRITICAL or HIGH findings:}
+**Recommended:** Run `[US] Update Skill` workflow to apply priority remediations automatically.
+
+{IF only MEDIUM or LOW findings:}
+**Optional:** Minor drift detected. Manual updates sufficient, or run `[US] Update Skill` for automated remediation.
+
+{IF CLEAN:}
+**No action needed.** Skill is current with source code.
+```
+
+### 3. Add Provenance Section
+
+Append to {outputFile}:
+
+```markdown
+## Provenance
+
+| Field | Value |
+|-------|-------|
+| **Audit Date** | {current_date} |
+| **Audited By** | Ferris (Audit mode) |
+| **Forge Tier** | {tier} |
+| **Tools Used** | {tool_list based on tier} |
+| **Source Path** | {source_path} |
+| **Skill Path** | {skill_path} |
+| **Provenance Map** | {provenance_map_path} |
+| **Provenance Age** | {days} days |
+| **Mode** | {normal / degraded} |
+
+**Confidence Legend:**
+- **T1:** AST extraction — high reliability, structural truth
+- **T1-low:** Text pattern matching — moderate reliability
+- **T2:** QMD temporal context — evidence-backed semantic analysis
+```
+
+### 4. Update Report Frontmatter
+
+Update {outputFile} frontmatter:
+- Append `'step-06-report'` to `stepsCompleted`
+- Set `drift_score` to final calculated score
+- Set `nextWorkflow` to `'update-skill'` if CRITICAL or HIGH findings, otherwise leave empty
+
+### 5. Present Final Report Summary
+
+"**Audit Complete: {skill_name}**
+
+---
+
+**Overall Drift Score: {CLEAN / MINOR / SIGNIFICANT / CRITICAL}**
+
+| Severity | Count |
+|----------|-------|
+| CRITICAL | {count} |
+| HIGH | {count} |
+| MEDIUM | {count} |
+| LOW | {count} |
+| **Total** | {total} |
+
+**Report saved to:** `{outputFile}`
+
+{IF CRITICAL/HIGH findings:}
+**Action Required:** {count} priority items need remediation. Recommend running `[US] Update Skill` workflow.
+
+{IF MEDIUM/LOW only:}
+**Minor Drift:** Skill is functional but could benefit from updates. See report for details.
+
+{IF CLEAN:}
+**All Clear:** No drift detected. Skill accurately reflects current source code.
+
+---
+
+**Next Steps:**
+{IF findings exist:}
+1. **[US] Update Skill** — Automatically apply remediations from this drift report
+2. **Review report** — Manual review at `{outputFile}`
+
+{IF CLEAN:}
+1. **No action needed** — Skill is current
+2. **[EX] Export Skill** — Skill is ready for distribution
+
+---
+
+**Audit workflow complete.**"
+
+---
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS
+
+### ✅ SUCCESS:
+
+- Audit Summary completed with accurate metrics
+- Remediation suggestion for every CRITICAL and HIGH finding
+- Each remediation is specific: what to change, where, and why
+- Provenance section documents complete audit trail
+- Overall drift score correctly reflects findings
+- Frontmatter updated with completion status and nextWorkflow
+- Final summary presented clearly to user
+- Report saved to correct output path
+
+### ❌ SYSTEM FAILURE:
+
+- Missing remediation for any CRITICAL or HIGH finding
+- Vague remediation suggestions (e.g., "update the skill" without specifics)
+- Discovering new findings in this step
+- Reclassifying severity
+- Not presenting final summary to user
+- Missing provenance section
+- Hardcoded paths instead of frontmatter variables
+
+**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.