bmad-module-skill-forge 0.9.0 → 0.10.1-alpha.0

package/src/workflows/setup-forge/workflow.md

@@ -1,51 +0,0 @@
----
-name: setup-forge
-description: Initialize forge environment, detect tools, set tier, verify QMD collection hygiene
-web_bundle: true
-installed_path: '{project-root}/_bmad/skf/workflows/skillforge/setup-forge'
----
-
-# Setup Forge
-
-**Goal:** Initialize the forge environment by detecting available tools, determining the capability tier (Quick/Forge/Forge+/Deep), writing persistent configuration, and optionally indexing the project for deep search.
-
-**Your Role:** You are a system executor performing environment resolution. This is a fully autonomous workflow — no user interaction is required during execution. Run each step in sequence, write configuration files, and report results at completion.
-
-## WORKFLOW ARCHITECTURE
-
-### Core Principles
-
-- **Micro-file Design**: Each step of the overall goal is a self contained instruction file that you will adhere to 1 file as directed at a time
-- **Just-In-Time Loading**: Only 1 current step file will be loaded, read, and executed to completion - never load future step files until told to do so
-- **Sequential Enforcement**: Sequence within the step files must be completed in order, no skipping or optimization allowed
-- **Fully Autonomous**: All steps auto-proceed with no user interaction until the final report
-
-### Step Processing Rules
-
-1. **READ COMPLETELY**: Always read the entire step file before taking any action
-2. **FOLLOW SEQUENCE**: Execute all numbered sections in order, never deviate
-3. **AUTO-PROCEED**: Each step proceeds to the next automatically after completion
-4. **LOAD NEXT**: When directed, load, read entire file, then execute the next step file
-
-### Critical Rules (NO EXCEPTIONS)
-
-- 🛑 **NEVER** load multiple step files simultaneously
-- 📖 **ALWAYS** read entire step file before execution
-- 🚫 **NEVER** skip steps or optimize the sequence
-- 🎯 **ALWAYS** follow the exact instructions in the step file
-- 📋 **NEVER** create mental todo lists from future steps
-
----
-
-## INITIALIZATION SEQUENCE
-
-### 1. Module Configuration Loading
-
-Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
-
-- `project_name`, `output_folder`, `user_name`, `communication_language`, `document_output_language`
-- `skills_output_folder`, `forge_data_folder`, `sidecar_path`
-
-### 2. First Step Execution
-
-Load, read the full file and then execute ./steps-c/step-01-detect-and-tier.md to begin the workflow.

package/src/workflows/test-skill/data/scoring-rules.md

@@ -1,123 +0,0 @@
-# Scoring Rules
-
-## Default Threshold
-
-**Pass threshold:** 80%
-
-## Category Weights
-
-| Category | Weight | Description |
-|----------|--------|-------------|
-| Export Coverage | 36% | Percentage of source exports documented in SKILL.md |
-| Signature Accuracy | 22% | Documented signatures match actual source signatures |
-| Type Coverage | 14% | Types and interfaces referenced are complete |
-| Coherence (contextual) | 18% | Cross-references valid, integration patterns complete |
-| Coherence (naive) | 0% | Not applicable — weight redistributed to other categories |
-| External Validation | 10% | Average of skill-check quality score + tessl average score (redistributed if unavailable) |
-
-## Naive Mode Weight Redistribution
-
-The following weights replace the default table for naive mode. The 18% coherence weight from the default table has been proportionally redistributed into these values. Do not apply a second redistribution.
-
-When running in naive mode (no coherence category):
-- Export Coverage: 45%
-- Signature Accuracy: 25%
-- Type Coverage: 20%
-- External Validation: 10%
-
-## External Validation Unavailable
-
-When neither skill-check nor tessl is available, redistribute the 10% external validation weight proportionally to the other active categories. When only one tool is available, use that tool's score as the external validation score.
-
-## tessl and Split-Body Interaction
-
-tessl evaluates SKILL.md body content only — it does not read `references/*.md` files. After split-body extraction, the tessl content score will drop significantly (e.g., 65% to 38%) because Tier 2 content is no longer inline. This is expected behavior and does not reflect actual content quality. When reporting scores for a split-body skill, note: "tessl content score reflects post-split inline content only. Use the pre-split tessl score as the content quality baseline."
-
-## Tier-Dependent Scoring
-
-### Quick Tier (no tools)
-- Export Coverage: file/structure existence check only
-- Signature Accuracy: skipped (no AST)
-- Type Coverage: skipped (no AST)
-- Score based on: structural completeness only
-- Weight redistribution: skipped categories' weights (Signature Accuracy 22% + Type Coverage 14%) redistributed proportionally to remaining active categories
-
-### Docs-Only Mode (Quick tier, all [EXT:...] citations)
-
-When `docs_only_mode: true` is set by step-03 (indicating a Quick tier skill where all SKILL.md citations are `[EXT:...]` format with no local source code):
-
-- **Signature Accuracy:** Not scored (no source to compare against)
-- **Type Coverage:** Not scored (no source to compare against)
-- **Weight redistribution:** Same as Quick tier — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories
-- **Export Coverage basis:** Documentation completeness rather than source coverage. Score = (documented_items_with_complete_descriptions / total_documented_items) * 100. A "complete" item has: description, parameters (if function/method), and return type (if function/method).
-- **Coherence:** Standard rules for the detected mode (naive or contextual) apply unchanged
-
-This is functionally identical to Quick tier weight redistribution but with a different coverage denominator (self-consistency instead of source comparison).
-
-### State 2 Source Access (Any Tier, Provenance-Map Only)
-
-When source is not locally available and analysis resolves to State 2 (provenance-map baseline per source-access-protocol.md):
-
-- **Signature Accuracy:** N/A — provenance-map stores parameters as flat string arrays; verification is string comparison only, not semantic AST verification. Type aliases (`str` vs `String`, `list` vs `List[Any]`) cannot be resolved without live source.
-- **Type Coverage:** N/A — cannot verify type completeness without local source access for AST re-parsing.
-- **Weight redistribution:** Same as Quick tier — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories (Export Coverage, Coherence, External Validation).
-- **Applies regardless of detected tier** (including Forge, Forge+, Deep) whenever `analysis_confidence` is `provenance-map` and local source is unavailable.
-- **Export Coverage denominator:** Uses the union of provenance-map entry names and metadata.json `exports[]` names (per source-access-protocol.md State 2 rules).
-
-Note: When provenance-map entries are predominantly T1 (AST-verified at compilation time), the coverage and name-matching data is already at highest confidence. The N/A categories reflect the inability to re-verify at test time, not low-quality extraction data.
-
-### Forge Tier (ast-grep)
-- Export Coverage: AST-backed export comparison
-- Signature Accuracy: AST-verified signature matching
-- Type Coverage: AST-verified type completeness
-- Full scoring formula applied
-
-### Forge+ Tier (ast-grep + ccc)
-- Same scoring as Forge tier — ccc provides pre-ranking but does not change scoring weights
-- Improved extraction coverage (from ccc pre-discovery) may increase T1 count, but scoring formula is identical to Forge
-- Full scoring formula applied
-
-### Deep Tier (ast-grep + gh + QMD)
-- All Forge tier checks plus:
-- Cross-repository reference verification
-- QMD knowledge enrichment for coherence
-- Full scoring formula with maximum depth
-- **Migration & Deprecation Warnings section:** If T2-future annotations exist in the enrichment data, verify that Section 4b is present in SKILL.md Tier 1 and that each warning traces to a T2 provenance citation. If no T2-future annotations exist, Section 4b should be absent (not empty). Presence/absence mismatch is a Medium severity gap.
-
-## Score Calculation
-
-```
-score = sum(category_weight * category_score) for each category
-category_score = (items_passing / items_total) * 100
-```
-
-## Coherence Score Aggregation (Contextual Mode)
-
-```
-reference_validity = (valid_references / total_references) * 100
-integration_completeness = (complete_patterns / total_patterns) * 100
-combined_coherence = (reference_validity * 0.6) + (integration_completeness * 0.4)
-```
-
-If no integration patterns exist, combined coherence equals reference validity.
-
-## Result Determination
-
-- score >= threshold → PASS
-- score < threshold → FAIL
-
-## Gap Severity
-
-| Severity | Criteria |
-|----------|----------|
-| Critical | Missing exported function/class documentation |
-| High | Signature mismatch between source and SKILL.md |
-| Medium | Missing type or interface documentation |
-| Medium | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
-| Medium | Script/asset directory exists but no Scripts & Assets section in SKILL.md |
-| Medium | Scripts & Assets section references file not found in scripts/ or assets/ directory |
-| Low | Script/asset file present without provenance entry in provenance-map.json file_entries |
-| Low | Missing optional metadata or examples |
-| Low | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps) |
-| Info | Style suggestions, non-blocking observations |
-| Info | Discovery testing not performed — realistic prompt testing recommended before export |

package/src/workflows/test-skill/data/source-access-protocol.md

@@ -1,51 +0,0 @@
-# Source Access Protocol
-
-## Source API Surface Definition
-
-**Source API surface** = the package's top-level public exports. These are the symbols reachable from the primary entry point without importing internal modules:
-
-- **Python:** symbols exported in `__init__.py` (including re-exports) — exclude private (`_prefixed`) names
-- **TypeScript/JavaScript:** named exports from `index.ts` / `index.js` — exclude unexported locals
-- **Go:** exported identifiers (capitalized) from the package's public-facing files
-- **Rust:** items in `pub use` from `lib.rs` or `mod.rs`
-
-Internal module symbols are **excluded** from the coverage denominator unless they are explicitly documented in SKILL.md (in which case they count as documented extras, not missing coverage).
-
-This matches the extraction-patterns.md convention used during skill creation: coverage measures how well SKILL.md documents what users actually import, not the entire internal codebase.
-
-## Source Access Resolution
-
-Before analysis, determine source access level. Walk through these states in order — use the first that succeeds:
-
-**State 1 — Local source available:**
-Check if `{source_path}` (from metadata.json `source_root`) exists on disk. If yes → full analysis at detected tier (AST + signatures). Set `analysis_confidence: full`.
-
-**State 2 — Local absent, provenance-map exists:**
-Check `{forge_data_folder}/{skill_name}/provenance-map.json`. If present AND contains at least 1 entry, use it as the baseline export inventory — each entry contains structured fields: `export_name`, `export_type`, `params[]`, `return_type`, `source_file`, `source_line`, `confidence`, and `ast_node_type`. Cross-reference against SKILL.md documented exports for name-matching and param-by-param coverage. Signature verification compares SKILL.md's documented params/return types against provenance-map entries directly.
-
-**Cross-reference with metadata.json:** After loading provenance-map entries, compare the entry count against `metadata.json`'s `exports[]` array length and `stats.exports_public_api` count. If metadata reports more exports than provenance-map entries:
-- Compute `gap = metadata.exports.length - provenance_map.entries.length`
-- Report: "Provenance-map contains {pmap_count} entries but metadata.json lists {meta_count} exports ({gap} gap). Coverage denominator uses the union."
-- Build the coverage denominator from the **union** of provenance-map entry names and metadata.json `exports[]` names. Exports present in metadata but absent from provenance-map are counted as "missing documentation" in the coverage calculation.
-- If metadata.json is unavailable or has no `exports[]` array, use provenance-map count alone with a note: "Coverage denominator is provenance-map only — may undercount if extraction was incomplete." If remote reading tools are available (zread, deepwiki, gh API, or similar), supplement by reading the entry point file for live signature verification. Set `analysis_confidence: provenance-map`.
-
-**State 2 limitations:** Signature verification at State 2 is **string comparison only**, not semantic. Provenance-map stores parameters as flat string arrays (e.g., `["data: Union[BinaryIO, list, str]"]`), so `str` vs `String` or `list` vs `List[Any]` would be treated as mismatches even when semantically equivalent. For full type-aware verification (handling type aliases, generic equivalence), State 1 (local source) with AST re-parsing is required. When the SKILL.md was compiled from the same provenance-map (typical for create-then-test flows), most strings will match. However, enrichment (step-04) and doc-fetching (step-03c) during compilation may alter parameter descriptions, add type annotations, or normalize signatures, causing mismatches even in create-then-test flows. Expect some string-level mismatches and treat them as compilation artifacts, not source drift signals, until signature fidelity is enforced by step-05's Signature Fidelity Rule (see `signature_source` field in provenance-map entries).
-
-**State 3 — No provenance-map, metadata exports exist (quick-skill path):**
-If no provenance-map.json exists (typical for quick-skill output), fall back to `metadata.json`'s `exports[]` array for the export name list. Coverage check becomes a self-consistency comparison: are all names in `exports[]` documented in SKILL.md with description, parameters, and return type? Signatures cannot be verified. If remote reading tools are available, supplement by reading the entry point for live export comparison. Set `analysis_confidence: metadata-only`.
-
-**State 4 — No local source, no forge-data, remote tools available:**
-If neither provenance-map nor metadata exports provide a usable baseline, but remote reading tools (zread, deepwiki, gh API, or similar) are available and `source_repo` is set in metadata.json, read the entry point remotely to build the export inventory from scratch. Name-matching only — no AST. Set `analysis_confidence: remote-only`.
-
-**State 5 — No source access at all:**
-If none of the above succeed, fall through to docs-only mode (as defined in step-01-init.md Section 0: pre-analysis source type detection). Set `analysis_confidence: docs-only`. Warn: "**No source access available.** Coverage check evaluates documentation self-consistency only. Re-run with local clone or remote access for source-backed verification."
-
-Set `analysis_confidence` in context for use in Section 2 analysis depth, step-05 output, and step-05 scoring.
-
-**Confidence tier mapping:** `full` = T1, `provenance-map` = T1, `metadata-only` = T1-low, `remote-only` = T1-low, `docs-only` = T3. This aligns with the T1/T1-low/T2/T3 scale used across all SKF workflows.
-
-**Degradation notice rules:** When `analysis_confidence` is `provenance-map`, check the `confidence` field of provenance-map entries before emitting a degradation recommendation:
-
-- **All/most entries T1 (AST-verified):** The provenance-map data is already at highest confidence. Do NOT recommend re-running with a local clone — it would produce identical results. Use: "Resolved via: provenance-map (T1 AST-verified at compilation time). Local clone not required — provenance data is already at highest confidence."
-- **Mixed T1/T1-low entries:** Report the breakdown. Recommend local clone only for the T1-low entries: "Resolved via: provenance-map ({n} T1, {m} T1-low). Re-run with local clone to upgrade T1-low entries to full AST verification."
-- **All/most entries T1-low or lower:** Keep the standard recommendation: "Re-run with local clone for full AST-backed verification."

package/src/workflows/test-skill/steps-c/step-01-init.md

@@ -1,217 +0,0 @@
----
-name: 'step-01-init'
-description: 'Discover skill, load forge tier state, validate inputs, create test report'
-
-nextStepFile: './step-02-detect-mode.md'
-outputFile: '{forge_version}/test-report-{skill_name}.md'
-templateFile: '../templates/test-report-template.md'
-sidecarFile: '{sidecar_path}/forge-tier.yaml'
-skillsOutputFolder: '{skills_output_folder}'
----
-
-# Step 1: Initialize Test
-
-## STEP GOAL:
-
-Discover and validate the target skill, load forge tier state to determine analysis depth, and create the test report document from template.
-
-## MANDATORY EXECUTION RULES (READ FIRST):
-
-### Universal Rules:
-
-- 🛑 NEVER proceed if required inputs are missing — halt with clear error
-- 📖 CRITICAL: Read the complete step file before taking any action
-- 🔄 CRITICAL: When loading next step, ensure entire file is read
-- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
-- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
-
-### Role Reinforcement:
-
-- ✅ You are a skill auditor operating in Ferris's Audit mode
-- ✅ If you already have been given a name, communication_style and identity, continue to use those while playing this new role
-- ✅ Methodical, precise, evidence-based — zero hallucination
-- ✅ Every finding must trace to actual artifacts
-
-### Step-Specific Rules:
-
-- 🎯 Focus only on input discovery and validation — do NOT begin analysis
-- 🚫 FORBIDDEN to start coverage or coherence checks in this step
-- 💬 Report what was found clearly and concisely
-- 📋 Validate all required files exist before proceeding
-
-## EXECUTION PROTOCOLS:
-
-- 🎯 Discover skill path from user input or batch list
-- 💾 Create output document from {templateFile}
-- 📖 Load forge tier state from {sidecarFile}
-- 🚫 FORBIDDEN to proceed if SKILL.md is missing
-
-## CONTEXT BOUNDARIES:
-
-- Available: User-provided skill path, SKF module config
-- Focus: Input validation and tier detection only
-- Limits: Do not analyze skill content — just confirm it exists and is loadable
-- Dependencies: setup-forge must have run (forge-tier.yaml must exist)
-
-## MANDATORY SEQUENCE
-
-**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
-
-### 1. Receive Skill Path
-
-If skill path was provided as workflow argument, use it directly.
-
-If no path provided, ask:
-
-"**Which skill would you like to test?**
-
-Provide the skill path or name. I'll search in `{skillsOutputFolder}`.
-
-**Path or name:**"
-
-### 2. Validate Skill Exists (version-aware)
-
-Resolve the skill path using version-aware resolution (see [knowledge/version-paths.md](../../../knowledge/version-paths.md)):
-
-1. Read `{skillsOutputFolder}/.export-manifest.json` and look up the skill name in `exports` to get `active_version`
-2. If found: resolve to `{skill_package}` = `{skillsOutputFolder}/{skill_name}/{active_version}/{skill_name}/`
-3. If not in manifest: check for `active` symlink at `{skillsOutputFolder}/{skill_name}/active` — resolve to `{skill_group}/active/{skill_name}/`
-4. If neither: fall back to flat path `{skillsOutputFolder}/{skill_name}/`. If SKILL.md exists at the flat path, auto-migrate per `knowledge/version-paths.md` migration rules
-5. Store the resolved path as `{resolved_skill_package}`
-
-Check that the skill package contains required files:
-
-**Required files:**
-- `{resolved_skill_package}/SKILL.md` — the skill documentation
-- `{resolved_skill_package}/metadata.json` — skill metadata
-
-**If SKILL.md missing:**
-"**Error: SKILL.md not found at `{resolved_skill_package}/SKILL.md`**
-
-This skill has not been created yet. Run the **create-skill** workflow first."
-
-HALT — do not proceed.
-
-**If metadata.json missing:**
-"**Warning:** metadata.json not found. Proceeding with limited metadata. Some checks may be skipped."
-
-### 3. Validate Frontmatter Compliance
-
-Before proceeding to content analysis, verify that SKILL.md has valid agentskills.io frontmatter. This catches compliance issues early, regardless of which workflow generated the skill.
-
-**Check the following:**
-- [ ] SKILL.md begins with `---` and has a closing `---` (frontmatter present)
-- [ ] `name` field exists, non-empty, lowercase alphanumeric + hyphens, 1-64 chars
-- [ ] `name` matches the skill directory name
-- [ ] `description` field exists, non-empty, 1-1024 chars
-- [ ] No unknown frontmatter fields (only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` are permitted)
-
-**If frontmatter is missing or invalid:**
-"**Warning: SKILL.md frontmatter is non-compliant with agentskills.io specification.**
-
-{list specific issues}
-
-This skill will fail `npx skills add` and `npx skill-check check`. Consider fixing frontmatter before proceeding (run `npx skill-check check <skill-dir> --fix` to auto-fix deterministic issues)."
-
-Log each issue as a pre-check finding. Continue with testing — frontmatter issues will be reported in the gap report alongside coverage/coherence findings.
-
-### 4. Load Forge Tier State
-
-Read `{sidecarFile}` to determine available analysis depth.
-
-**If forge-tier.yaml exists:**
-- Read `tier` value (Quick, Forge, Forge+, or Deep)
-- Read tool availability flags (ast_grep, gh_cli, qmd)
-
-**If forge-tier.yaml missing:**
-"**Cannot proceed.** forge-tier.yaml not found at `{sidecarFile}`. Please run the **setup-forge** workflow first to configure your forge tier (Quick/Forge/Forge+/Deep)."
-
-HALT — do not proceed.
-
-### 4b. Apply Tier Override (if set)
-
-Read `{sidecar_path}/preferences.yaml`. If `tier_override` is set and is a valid tier value (Quick, Forge, Forge+, or Deep), update `detected_tier` to the override value for use in subsequent steps and output documents.
-
-### 5. Load Skill Metadata
-
-Read `metadata.json` to extract:
-- `name` — display name
-- `skill_type` — single or stack (needed for mode detection)
-- `source_path` — path to source code (if present)
-- `generation_date` — when skill was generated
-- `confidence_tier` — tier used during creation
-
-If source path override was provided as optional input, use that instead.
-
-### 6. Create Output Document
-
-Create `{outputFile}` from `{templateFile}` with initial frontmatter:
-
-```yaml
----
-workflowType: 'test-skill'
-skillName: '{skill_name}'
-skillDir: '{skill_path}'
-testMode: ''
-forgeTier: '{detected_tier}'
-testResult: ''
-score: ''
-threshold: ''
-analysisConfidence: ''
-testDate: '{current_date}'
-stepsCompleted: ['step-01-init']
-nextWorkflow: ''
----
-```
-
-### 7. Report Initialization Status
-
-"**Test initialization complete.**
-
-**Skill:** {skill_name}
-**Path:** {skill_path}
-**Type:** {skill_type}
-**Forge Tier:** {detected_tier}
-**Source:** {source_path}
-
-**Proceeding to mode detection...**"
-
-### 8. Auto-Proceed
-
-Display: "**Proceeding to mode detection...**"
-
-#### Menu Handling Logic:
-
-- After initialization is complete, update {outputFile} frontmatter stepsCompleted, then immediately load, read entire file, then execute {nextStepFile}
-
-#### EXECUTION RULES:
-
-- This is an auto-proceed initialization step with no user choices
-- Proceed directly to next step after setup
-
-## CRITICAL STEP COMPLETION NOTE
-
-ONLY WHEN all required files are validated, forge tier is loaded, output document is created, and initialization status is reported, will you then load and read fully `{nextStepFile}` to execute mode detection.
-
----
-
-## 🚨 SYSTEM SUCCESS/FAILURE METRICS
-
-### ✅ SUCCESS:
-
-- Skill path validated (SKILL.md exists)
-- Forge tier state loaded (HALT if missing), override applied if set in preferences.yaml
-- Skill metadata extracted
-- Output document created from template with initial frontmatter
-- Initialization status reported clearly
-- Auto-proceeded to step 02
-
-### ❌ SYSTEM FAILURE:
-
-- Proceeding without validating SKILL.md exists
-- Not loading forge tier state
-- Not creating output document before proceeding
-- Starting analysis in this step (coverage/coherence belongs to later steps)
-- Hardcoding paths instead of using frontmatter variables
-
-**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.