bmad-module-skill-forge 0.2.0 → 0.3.0

package/src/workflows/test-skill/data/source-access-protocol.md

@@ -0,0 +1,37 @@
+# Source Access Protocol
+
+## Source API Surface Definition
+
+**Source API surface** = the package's top-level public exports. These are the symbols reachable from the primary entry point without importing internal modules:
+
+- **Python:** symbols exported in `__init__.py` (including re-exports) — exclude private (`_prefixed`) names
+- **TypeScript/JavaScript:** named exports from `index.ts` / `index.js` — exclude unexported locals
+- **Go:** exported identifiers (capitalized) from the package's public-facing files
+- **Rust:** items in `pub use` from `lib.rs` or `mod.rs`
+
+Internal module symbols are **excluded** from the coverage denominator unless they are explicitly documented in SKILL.md (in which case they count as documented extras, not missing coverage).
+
+This matches the extraction-patterns.md convention used during skill creation: coverage measures how well SKILL.md documents what users actually import, not the entire internal codebase.
+
+## Source Access Resolution
+
+Before analysis, determine source access level. Walk through these states in order — use the first that succeeds:
+
+**State 1 — Local source available:**
+Check if `{source_path}` (from metadata.json `source_root`) exists on disk. If yes → full analysis at detected tier (AST + signatures). Set `analysis_confidence: full`.
+
+**State 2 — Local absent, provenance-map exists:**
+Check `{forge_data_folder}/{skill_name}/provenance-map.json`. If present, use it as the baseline export inventory — each entry contains structured fields: `export_name`, `export_type`, `params[]`, `return_type`, `source_file`, `source_line`, `confidence`, and `ast_node_type`. Cross-reference against SKILL.md documented exports for name-matching and param-by-param coverage. Signature verification compares SKILL.md's documented params/return types against provenance-map entries directly. If remote reading tools are available (zread, deepwiki, gh API, or similar), supplement by reading the entry point file for live signature verification. Set `analysis_confidence: provenance-map`.
+
+**State 3 — No provenance-map, metadata exports exist (quick-skill path):**
+If no provenance-map.json exists (typical for quick-skill output), fall back to `metadata.json`'s `exports[]` array for the export name list. Coverage check becomes a self-consistency comparison: are all names in `exports[]` documented in SKILL.md with description, parameters, and return type? Signatures cannot be verified. If remote reading tools are available, supplement by reading the entry point for live export comparison. Set `analysis_confidence: metadata-only`.
+
+**State 4 — No local source, no forge-data, remote tools available:**
+If neither provenance-map nor metadata exports provide a usable baseline, but remote reading tools (zread, deepwiki, gh API, or similar) are available and `source_repo` is set in metadata.json, read the entry point remotely to build the export inventory from scratch. Name-matching only — no AST. Set `analysis_confidence: remote-only`.
+
+**State 5 — No source access at all:**
+If none of the above succeed, fall through to docs-only mode (section 0 already handles this). Set `analysis_confidence: docs-only`. Warn: "**No source access available.** Coverage check evaluates documentation self-consistency only. Re-run with local clone or remote access for source-backed verification."
+
+Set `analysis_confidence` in context for use in Section 2 analysis depth, step-05 output, and step-05 scoring.
+
+**Confidence tier mapping:** `full` = T1, `provenance-map` = T1, `metadata-only` = T1-low, `remote-only` = T1-low, `docs-only` = T3. This aligns with the T1/T1-low/T2/T3 scale used across all SKF workflows.

package/src/workflows/test-skill/steps-c/step-01-init.md

@@ -3,7 +3,7 @@ name: 'step-01-init'
 description: 'Discover skill, load forge tier state, validate inputs, create test report'
 
 nextStepFile: './step-02-detect-mode.md'
-outputFile: '{output_folder}/test-report-{skill_name}.md'
+outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
 templateFile: '../templates/test-report-template.md'
 sidecarFile: '{sidecar_path}/forge-tier.yaml'
 skillsOutputFolder: '{skills_output_folder}'
@@ -87,12 +87,32 @@ HALT — do not proceed.
 **If metadata.json missing:**
 "**Warning:** metadata.json not found. Proceeding with limited metadata. Some checks may be skipped."
 
-### 3. Load Forge Tier State
+### 3. Validate Frontmatter Compliance
+
+Before proceeding to content analysis, verify that SKILL.md has valid agentskills.io frontmatter. This catches compliance issues early, regardless of which workflow generated the skill.
+
+**Check the following:**
+- [ ] SKILL.md begins with `---` and has a closing `---` (frontmatter present)
+- [ ] `name` field exists, non-empty, lowercase alphanumeric + hyphens, 1-64 chars
+- [ ] `name` matches the skill directory name
+- [ ] `description` field exists, non-empty, 1-1024 chars
+- [ ] No unknown frontmatter fields (only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` are permitted)
+
+**If frontmatter is missing or invalid:**
+"**Warning: SKILL.md frontmatter is non-compliant with agentskills.io specification.**
+
+{list specific issues}
+
+This skill will fail `npx skills add` and `npx skill-check check`. Consider fixing frontmatter before proceeding (run `npx skill-check check <skill-dir> --fix` to auto-fix deterministic issues)."
+
+Log each issue as a pre-check finding. Continue with testing — frontmatter issues will be reported in the gap report alongside coverage/coherence findings.
+
+### 4. Load Forge Tier State
 
 Read `{sidecarFile}` to determine available analysis depth.
 
 **If forge-tier.yaml exists:**
-- Read `detected_tier` value (Quick, Forge, or Deep)
+- Read `tier` value (Quick, Forge, or Deep)
 - Read tool availability flags (ast_grep, gh_cli, qmd)
 
 **If forge-tier.yaml missing:**
@@ -102,18 +122,22 @@ Run **setup-forge** to enable deeper analysis with AST-backed verification."
 
 Default to Quick tier and continue.
 
-### 4. Load Skill Metadata
+### 4b. Apply Tier Override (if set)
+
+Read `{sidecar_path}/preferences.yaml`. If `tier_override` is set and is a valid tier value (Quick, Forge, or Deep), update `detected_tier` to the override value for use in subsequent steps and output documents.
+
+### 5. Load Skill Metadata
 
 Read `metadata.json` to extract:
-- `skill_name` — display name
-- `skill_type` — individual or stack (needed for mode detection)
+- `name` — display name
+- `skill_type` — single or stack (needed for mode detection)
 - `source_path` — path to source code (if present)
-- `created_date` — when skill was created
-- `tier` — tier used during creation
+- `generation_date` — when skill was generated
+- `confidence_tier` — tier used during creation
 
 If source path override was provided as optional input, use that instead.
 
-### 5. Create Output Document
+### 6. Create Output Document
 
 Create `{outputFile}` from `{templateFile}` with initial frontmatter:
 
@@ -121,18 +145,19 @@ Create `{outputFile}` from `{templateFile}` with initial frontmatter:
 ---
 workflowType: 'test-skill'
 skillName: '{skill_name}'
+skillDir: '{skill_path}'
 testMode: ''
 forgeTier: '{detected_tier}'
 testResult: ''
 score: ''
-threshold: ''
+analysisConfidence: ''
 testDate: '{current_date}'
 stepsCompleted: ['step-01-init']
 nextWorkflow: ''
 ---
 ```
 
-### 6. Report Initialization Status
+### 7. Report Initialization Status
 
 "**Test initialization complete.**
 
@@ -144,7 +169,7 @@ nextWorkflow: ''
 
 **Proceeding to mode detection...**"
 
-### 7. Auto-Proceed
+### 8. Auto-Proceed
 
 Display: "**Proceeding to mode detection...**"
 
@@ -168,7 +193,7 @@ ONLY WHEN all required files are validated, forge tier is loaded, output documen
 ### ✅ SUCCESS:
 
 - Skill path validated (SKILL.md exists)
-- Forge tier state loaded (or gracefully defaulted to Quick)
+- Forge tier state loaded (or gracefully defaulted to Quick), override applied if set in preferences.yaml
 - Skill metadata extracted
 - Output document created from template with initial frontmatter
 - Initialization status reported clearly

package/src/workflows/test-skill/steps-c/step-02-detect-mode.md

@@ -3,7 +3,7 @@ name: 'step-02-detect-mode'
 description: 'Determine naive (individual) vs contextual (stack) testing mode'
 
 nextStepFile: './step-03-coverage-check.md'
-outputFile: '{output_folder}/test-report-{skill_name}.md'
+outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
 ---
 
 # Step 2: Detect Mode
@@ -59,7 +59,7 @@ Examine the skill metadata to determine whether this is an individual skill (nai
 Read the skill metadata (loaded in step 01) and check:
 
 **Individual Skill indicators:**
-- `skill_type: 'individual'` in metadata
+- `skill_type: 'single'` in metadata
 - Single source library/package
 - No cross-references to other skills
 - Self-contained API surface
@@ -99,7 +99,6 @@ Append the **Test Summary** section to `{outputFile}`:
 **Skill:** {skill_name}
 **Test Mode:** {naive|contextual}
 **Forge Tier:** {detected_tier}
-**Threshold:** {threshold}%
 
 **Mode Rationale:** {brief explanation of why this mode was selected}
 

package/src/workflows/test-skill/steps-c/step-03-coverage-check.md

@@ -3,8 +3,9 @@ name: 'step-03-coverage-check'
 description: 'Compare documented exports in SKILL.md against actual source API surface'
 
 nextStepFile: './step-04-coherence-check.md'
-outputFile: '{output_folder}/test-report-{skill_name}.md'
+outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
 scoringRulesFile: '../data/scoring-rules.md'
+sourceAccessProtocol: '../data/source-access-protocol.md'
 ---
 
 # Step 3: Coverage Check
@@ -32,7 +33,7 @@ Compare the exports, functions, classes, types, and interfaces documented in SKI
 
 ### Step-Specific Rules:
 
-- 🎯 Use subprocess optimization for per-file AST analysis (Pattern 2) when available
+- 🎯 Use subprocess optimization for per-file AST analysis when available
 - 💬 Subprocess returns structured findings only, not full file contents
 - 🚫 DO NOT BE LAZY — For EACH source file, launch a subprocess for deep analysis
 - ⚙️ If subprocess unavailable, perform analysis in main thread sequentially
@@ -56,6 +57,22 @@ Compare the exports, functions, classes, types, and interfaces documented in SKI
 
 **CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
 
+### 0. Check for Docs-Only Mode
+
+**If metadata.json indicates `confidence_tier: "Quick"` and all SKILL.md citations are `[EXT:...]` format (docs-only skill):**
+
+Coverage scoring adapts: instead of comparing SKILL.md against source code exports, compare SKILL.md documented items against themselves for internal completeness (every documented function has a description, parameters, and return type). Score based on documentation completeness rather than source coverage. Set `docs_only_mode: true` in context for step-05 scoring.
+
+"**Docs-only skill detected.** Coverage check evaluates documentation completeness rather than source code coverage."
+
+**If source-based skill:** Continue with standard coverage check below.
+
+### 0b. Load Source Access Protocol
+
+Load `{sourceAccessProtocol}` and follow both sections:
+1. **Source API Surface Definition** — determines what counts as the public API for coverage denominator
+2. **Source Access Resolution** — 5-state waterfall to determine how source files will be read and sets `analysis_confidence`
+
 ### 1. Extract Documented Exports from SKILL.md
 
 Read SKILL.md and extract all documented items:
@@ -70,14 +87,16 @@ Build the **documented inventory** — a list of everything the SKILL.md claims
 
 ### 2. Analyze Source Code (Tier-Dependent)
 
+Start from the package entry point (see 0b) and identify the public API surface. Then analyze those exports at the appropriate tier depth.
+
 **Quick Tier (no tools):**
-- Read source files directly
-- Identify exported items by scanning for `export` keywords, `module.exports`, or language-specific export patterns
+- Read the entry point file(s) directly
+- Identify public exports by scanning for `export` keywords, `module.exports`, `__init__.py` imports, or language-specific export patterns
 - Compare against documented inventory by name matching
 - Cannot verify signatures — note as "unverified" in report
 
 **Forge Tier (ast-grep available):**
-DO NOT BE LAZY — For EACH source file in the skill's source path, launch a subprocess that:
+DO NOT BE LAZY — For EACH source file that defines public API exports, launch a subprocess that:
 1. Uses ast-grep to extract all exported symbols with their full signatures
 2. Matches each export against the documented inventory
 3. Returns structured findings:
@@ -140,6 +159,7 @@ Append the **Coverage Analysis** section to `{outputFile}`:
 ## Coverage Analysis
 
 **Tier:** {forge_tier}
+**Source Access:** {analysis_confidence} (full | provenance-map | metadata-only | remote-only | docs-only)
 **Source Path:** {source_path}
 **Files Analyzed:** {count}
 

package/src/workflows/test-skill/steps-c/step-04-coherence-check.md

@@ -2,8 +2,8 @@
 name: 'step-04-coherence-check'
 description: 'Validate internal consistency — references, types, and integration patterns'
 
-nextStepFile: './step-05-score.md'
-outputFile: '{output_folder}/test-report-{skill_name}.md'
+nextStepFile: './step-04b-external-validators.md'
+outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
 outputFormatsFile: '../data/output-section-formats.md'
 ---
 
@@ -32,7 +32,7 @@ Validate internal consistency of the skill documentation. In contextual mode (st
 
 ### Step-Specific Rules:
 
-- 🎯 Use subprocess optimization: Pattern 1 (grep for references across files), Pattern 2 (per-reference deep validation)
+- 🎯 Use subprocess optimization: grep for references across files, then per-reference deep validation
 - 💬 Subprocess returns structured findings only, not full file contents
 - 🚫 DO NOT BE LAZY — For EACH reference in contextual mode, launch a subprocess to validate the target
 - ⚙️ If subprocess unavailable, perform validation in main thread sequentially
@@ -47,7 +47,7 @@ Validate internal consistency of the skill documentation. In contextual mode (st
 
 ## CONTEXT BOUNDARIES:
 
-- Available: SKILL.md, source files, testMode, forge tier, coverage results from step 03
+- Available: SKILL.md, source files, testMode, forge tier, coverage results from step 03, evidence-report.md from skill's forge-data directory (Deep tier only)
 - Focus: Internal consistency and reference validation only — coverage was step 03
 - Limits: Do NOT recalculate coverage scores — use results from step 03
 - Dependencies: step-03 must have completed coverage analysis
@@ -102,22 +102,7 @@ Scan SKILL.md for all cross-references:
 - Type imports (`import { Type } from './module'`)
 - Integration pattern references (middleware chains, plugin hooks, shared state)
 
-Launch a subprocess that:
-1. Runs grep/regex across SKILL.md for reference patterns (file paths, import statements, skill names)
-2. Returns all found references with their line numbers
-
-```json
-{
-  "references_found": [
-    {"line": 15, "type": "file_path", "target": "./shared/types.ts"},
-    {"line": 42, "type": "skill_ref", "target": "auth-skill"},
-    {"line": 78, "type": "type_import", "target": "SharedConfig from ./config"}
-  ],
-  "total_references": 3
-}
-```
-
-If subprocess unavailable, scan SKILL.md in main thread.
+Launch a subprocess to grep/regex SKILL.md for reference patterns and return all found references with line numbers as structured JSON (`references_found[]` with line, type, target fields). If subprocess unavailable, scan in main thread.
 
 ### 4. Contextual Mode: Validate Each Reference
 
@@ -129,29 +114,7 @@ DO NOT BE LAZY — For EACH reference found, launch a subprocess that:
    - Type imports: type is actually exported from the referenced module
    - Skill references: referenced skill exists in skills output folder
    - Integration patterns: documented pattern matches actual implementation
-3. Returns structured validation:
-
-```json
-{
-  "reference": "./shared/types.ts",
-  "line": 15,
-  "target_exists": true,
-  "type_match": true,
-  "signature_match": true,
-  "issues": []
-}
-```
-
-Or for failures:
-
-```json
-{
-  "reference": "auth-skill",
-  "line": 42,
-  "target_exists": false,
-  "issues": ["Referenced skill 'auth-skill' not found in skills output folder"]
-}
-```
+3. Returns structured validation JSON per reference (reference, line, target_exists, type_match, signature_match, issues[])
 
 If subprocess unavailable, validate each reference in main thread.
 
@@ -180,6 +143,18 @@ Build integration completeness findings:
 }
 ```
 
+### 5b. Contextual Mode + Deep Tier: Section 4b Verification
+
+**Only execute if forge tier is Deep.** Skip silently for Quick/Forge tiers.
+
+Check whether SKILL.md contains a "Migration & Deprecation Warnings" section (Section 4b). Then check the skill's `evidence-report.md` (at `forge-data/{skill_name}/evidence-report.md`) for T2-future annotation counts.
+
+- **If T2-future annotations > 0 AND Section 4b is absent:** Flag as Medium severity gap: "Migration section missing — T2-future annotations exist but Section 4b is not present in SKILL.md Tier 1."
+- **If T2-future annotations = 0 AND Section 4b is present:** Flag as Medium severity gap: "Migration section unexpected — Section 4b is present but no T2-future annotations were produced."
+- **If evidence-report.md is unavailable:** Skip this check silently. Note: "Section 4b verification skipped — evidence-report.md not found."
+
+Add findings to the coherence analysis results.
+
 ### 6. Append Coherence Analysis to Output
 
 Load `{outputFormatsFile}` and use the appropriate Coherence Analysis section format (naive or contextual) to append findings to `{outputFile}`.

package/src/workflows/test-skill/steps-c/step-04b-external-validators.md

@@ -0,0 +1,213 @@
+---
+name: 'step-04b-external-validators'
+description: 'Run external validation tools (skill-check, tessl) and capture scores for scoring step'
+
+nextStepFile: './step-05-score.md'
+outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
+---
+
+# Step 4b: External Validators
+
+## STEP GOAL:
+
+Run external validation tools (`skill-check` and `tessl`) against the skill directory, capture their scores and findings, and append results to the test report. These tools catch complementary issues that internal coverage and coherence checks miss: `skill-check` validates spec compliance while `tessl` evaluates content quality and actionability.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 📖 CRITICAL: Read the complete step file before taking any action
+- 🎯 ALWAYS follow the exact instructions in the step file
+- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a tool you do not have access to, you MUST still achieve the outcome in your main context thread
+- ✅ YOU MUST ALWAYS SPEAK OUTPUT in your Agent communication style with the config `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are a skill auditor running external quality gates
+- ✅ Both tools are optional — unavailability is a skip, never a halt
+- ✅ Report scores factually — do not editorialize on the results
+
+### Step-Specific Rules:
+
+- 🎯 Run each tool independently — one tool's failure does not affect the other
+- 🚫 FORBIDDEN to halt the workflow if either tool is unavailable or fails
+- 🚫 FORBIDDEN to modify skill content — this step only measures and reports
+- ⚠️ Ignore non-zero exit codes if the tool produces parseable output
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow MANDATORY SEQUENCE exactly
+- 💾 Append External Validation section to {outputFile}
+- 📖 Update stepsCompleted in {outputFile}
+- 📖 Store scores in context for step-05 scoring
+
+## CONTEXT BOUNDARIES:
+
+- Available: skill directory path, testMode, forge tier from previous steps
+- Focus: External tool execution and score capture only
+- Limits: Do NOT modify skill content or recalculate internal scores
+- Dependencies: steps 03 and 04 must have completed
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+### 1. Resolve Skill Directory
+
+Read {outputFile} frontmatter to get the skill directory path (`skillDir`).
+
+### 2. Run skill-check
+
+**Check availability:**
+
+```bash
+npx skill-check -h
+```
+
+If unavailable, record `skill_check_score: N/A` and skip to section 3.
+
+**Run validation:**
+
+```bash
+npx skill-check check {skillDir} --format json --no-security-scan
+```
+
+**Parse JSON output** to extract:
+- `qualityScore` — overall score (0-100)
+- `diagnostics[]` — any remaining issues
+- `errorCount` and `warningCount`
+
+**Note:** `skill-check` may return a non-zero exit code even when `errorCount` is 0. Always rely on the parsed JSON output, not the shell exit code.
+
+Store in context: `skill_check_score`, `skill_check_diagnostics`
+
+**If skill-check fails entirely:** Record `skill_check_score: N/A`, log warning, continue.
+
+### 3. Run tessl
+
+**Check availability:**
+
+```bash
+npx -y tessl --version
+```
+
+If unavailable, record `tessl_score: N/A` and skip to section 4.
+
+**Run review:**
+
+```bash
+npx -y tessl skill review {skillDir}
+```
+
+**Parse the output** to extract:
+- `description_score` — percentage (e.g., 100%)
+- `content_score` — percentage (e.g., 45%)
+- `average_score` — percentage (e.g., 73%)
+- `validation_result` — PASSED/FAILED
+- `judge_suggestions[]` — list of improvement suggestions
+
+The tessl output is human-readable text, not JSON. Parse the percentage values from lines like "Description: 100%", "Content: 45%", "Average Score: 73%".
+
+Store in context: `tessl_description_score`, `tessl_content_score`, `tessl_average_score`, `tessl_suggestions`
+
+**If tessl content score < 70%:** Flag a warning:
+
+"**Content quality warning:** tessl scored content at {score}%. This often indicates SKILL.md lacks inline actionable content (e.g., after split-body). Consider inlining Quick Start and common workflows directly in SKILL.md."
+
+**If tessl fails entirely:** Record `tessl_score: N/A`, log warning, continue.
+
+### 4. Calculate Combined External Score
+
+**If both tools ran:**
+
+```
+external_score = (skill_check_score + tessl_average_score) / 2
+```
+
+Note: `skill_check_score` is 0-100, `tessl_average_score` is 0-100%. Both are on the same scale.
+
+**If only one tool ran:** Use that tool's score as the external score.
+
+**If neither tool ran:** Record `external_score: N/A`. The scoring step will redistribute the external validation weight.
+
+### 5. Append External Validation to Output
+
+Append to `{outputFile}`:
+
+```markdown
+## External Validation
+
+### skill-check
+- **Available:** {yes/no}
+- **Quality Score:** {score}/100
+- **Errors:** {count}
+- **Warnings:** {count}
+- **Diagnostics:** {list or "none"}
+
+### tessl
+- **Available:** {yes/no}
+- **Validation:** {PASSED/FAILED}
+- **Description Score:** {score}%
+- **Content Score:** {score}%
+- **Average Score:** {score}%
+- **Suggestions:**
+{bulleted list of judge suggestions}
+
+### Combined External Score
+- **External Validation Score:** {external_score}%
+- **Tools used:** {list of tools that ran}
+```
+
+### 6. Report Results
+
+"**External validation complete.**
+
+| Tool | Score | Status |
+|------|-------|--------|
+| skill-check | {score}/100 | {available/skipped} |
+| tessl | {average}% | {available/skipped} |
+| **Combined** | **{external_score}%** | |
+
+{content quality warning if tessl content < 70%}
+
+**Proceeding to scoring...**"
+
+### 7. Auto-Proceed
+
+#### Menu Handling Logic:
+
+- After external validation is complete, update {outputFile} frontmatter stepsCompleted, then immediately load, read entire file, then execute {nextStepFile}
+
+#### EXECUTION RULES:
+
+- This is an auto-proceed validation step with no user choices
+- Tool unavailability is a skip, not a halt
+- Proceed directly to next step after external validation
+
+## CRITICAL STEP COMPLETION NOTE
+
+ONLY WHEN external validation results are appended to {outputFile} and scores are stored in context will you proceed to load `{nextStepFile}` for scoring.
+
+---
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS
+
+### ✅ SUCCESS:
+
+- skill-check executed and quality score captured (or skipped with N/A if unavailable)
+- tessl executed and description/content/average scores captured (or skipped with N/A)
+- Content quality warning raised if tessl content < 70%
+- Combined external score calculated
+- External Validation section appended to output
+- Scores stored in context for step-05
+- Auto-proceeded to step-05
+
+### ❌ SYSTEM FAILURE:
+
+- Halting the workflow because a tool is unavailable or fails
+- Not recording N/A when a tool is skipped (step-05 needs to know)
+- Modifying skill content based on validation results
+- Not parsing tessl content score (the key insight this tool provides)
+- Fabricating scores not from actual tool output
+
+**Master Rule:** External validators inform — they do not block. Run what's available, capture the scores, report factually, and proceed.

package/src/workflows/test-skill/steps-c/step-05-score.md

@@ -3,7 +3,7 @@ name: 'step-05-score'
 description: 'Calculate completeness score from coverage and coherence findings'
 
 nextStepFile: './step-06-report.md'
-outputFile: '{output_folder}/test-report-{skill_name}.md'
+outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
 scoringRulesFile: '../data/scoring-rules.md'
 ---
 
@@ -11,7 +11,7 @@ scoringRulesFile: '../data/scoring-rules.md'
 
 ## STEP GOAL:
 
-Calculate the overall completeness score by aggregating coverage and coherence category scores with the appropriate weight distribution (naive or contextual), apply the pass/fail threshold, and determine the test result.
+Calculate the overall completeness score by aggregating coverage, coherence, and external validation category scores with the appropriate weight distribution (naive or contextual), apply the pass/fail threshold, and determine the test result.
 
 ## MANDATORY EXECUTION RULES (READ FIRST):
 
@@ -46,10 +46,10 @@ Calculate the overall completeness score by aggregating coverage and coherence c
 
 ## CONTEXT BOUNDARIES:
 
-- Available: Coverage Analysis (step 03) and Coherence Analysis (step 04) in {outputFile}
+- Available: Coverage Analysis (step 03), Coherence Analysis (step 04), and External Validation (step 04b) in {outputFile}
 - Focus: Score calculation and pass/fail determination only
 - Limits: Do NOT generate gap remediation — that's step 06
-- Dependencies: steps 03 and 04 must have appended their analysis sections
+- Dependencies: steps 03, 04, and 04b must have appended their analysis sections
 
 ## MANDATORY SEQUENCE
 
@@ -76,33 +76,15 @@ Read `{outputFile}` and extract the category scores calculated in previous steps
 - Combined Coherence: {percentage}% (contextual mode only)
 - Or: not scored (naive mode — weight redistributed)
 
+**From External Validation (step 04b):**
+- External Validation Score: {percentage}% (combined skill-check + tessl average)
+- Or: N/A (if neither tool was available — weight redistributed to other categories)
+
 ### 3. Apply Weight Distribution
 
 **Read testMode from {outputFile} frontmatter.**
 
-**IF naive mode — use redistributed weights:**
-
-| Category | Score | Weight | Weighted |
-|----------|-------|--------|----------|
-| Export Coverage | {N}% | 50% | {N * 0.50}% |
-| Signature Accuracy | {N}% | 30% | {N * 0.30}% |
-| Type Coverage | {N}% | 20% | {N * 0.20}% |
-| **Total** | | **100%** | **{sum}%** |
-
-**IF contextual mode — use full weights:**
-
-| Category | Score | Weight | Weighted |
-|----------|-------|--------|----------|
-| Export Coverage | {N}% | 40% | {N * 0.40}% |
-| Signature Accuracy | {N}% | 25% | {N * 0.25}% |
-| Type Coverage | {N}% | 15% | {N * 0.15}% |
-| Coherence | {N}% | 20% | {N * 0.20}% |
-| **Total** | | **100%** | **{sum}%** |
-
-**Quick tier adjustment:**
-If Signature Accuracy and Type Coverage are N/A (Quick tier, no AST):
-- Naive: Export Coverage gets 100% weight
-- Contextual: Export Coverage 60%, Coherence 40%
+Apply the weight distribution from `{scoringRulesFile}` for the detected mode (naive or contextual). The scoring rules define category weights, external validation redistribution when unavailable, and Quick tier adjustments. Calculate the weighted score for each category and sum for the total.
 
 ### 4. Determine Pass/Fail
 
@@ -139,6 +121,7 @@ Append the **Completeness Score** section to `{outputFile}`:
 | Signature Accuracy | {N}% | {W}% | {WS}% |
 | Type Coverage | {N}% | {W}% | {WS}% |
 | Coherence | {N}% | {W}% | {WS}% |
+| External Validation | {N}% | {W}% | {WS}% |
 | **Total** | | **100%** | **{total}%** |
 
 ### Result
@@ -149,6 +132,18 @@ Append the **Completeness Score** section to `{outputFile}`:
 
 **Weight Distribution:** {naive (redistributed) | contextual (full)}
 **Tier Adjustment:** {none | Quick tier — signature and type coverage not scored}
+**External Validators:** {both available | skill-check only | tessl only | none — weight redistributed}
+**Analysis Confidence:** {full | provenance-map | metadata-only | remote-only | docs-only}
+```
+
+If `analysis_confidence` is not `full`, append a degradation notice:
+
+```markdown
+### Access Degradation Notice
+
+**Resolved via:** {analysis_confidence}
+**Impact:** {describe limitation — e.g., "Signature checks limited to name-matching. Source file:line citations from provenance-map, not live AST."}
+**Recommendation:** Re-run with local clone for full AST-backed verification.
 ```
 
 ### 7. Update Output Frontmatter
@@ -157,6 +152,7 @@ Update `{outputFile}` frontmatter:
 - `testResult: '{pass|fail}'`
 - `score: '{total}%'`
 - `threshold: '{threshold}%'`
+- `analysisConfidence: '{analysis_confidence}'`
 - `nextWorkflow: '{export-skill|update-skill}'`
 - Append `'step-05-score'` to `stepsCompleted`
 
@@ -172,6 +168,7 @@ Update `{outputFile}` frontmatter:
 | Signature Accuracy | {N}% | {WS}% |
 | Type Coverage | {N}% | {WS}% |
 | Coherence | {N}% | {WS}% |
+| External Validation | {N}% | {WS}% |
 
 **Threshold:** {threshold}%
 **Recommendation:** {export-skill if pass | update-skill if fail}