bmad-module-skill-forge 2.2.0 → 3.0.0

package/src/skf-test-skill/references/score.md

@@ -1,9 +1,11 @@
 ---
 nextStepFile: 'report.md'
-outputFile: '{forge_version}/test-report-{skill_name}-{run_id}.md'
-scoringRulesFile: '{scoringRulesPath}'
+outputFile: '{report_file}'
 sourceAccessProtocol: 'references/source-access-protocol.md'
 scoringScript: 'scripts/compute-score.py'
+# §4b.1 reads the run's gaps from the gap ledger the stages recorded them in.
+ledgerFile: '{forge_version}/test-findings-{run_id}.json'
+gapLedgerScript: 'scripts/gap-ledger.py'
 ---
 
 <!-- Config: communicate in {communication_language}. -->
@@ -14,126 +16,96 @@ scoringScript: 'scripts/compute-score.py'
 
 Calculate the overall completeness score by aggregating coverage, coherence, and external validation category scores with the appropriate weight distribution (naive or contextual), apply the pass/fail threshold, and determine the test result.
 
-### 1. Load Scoring Rules
+**Halt envelope.** Every HALT in this step names its `halt_reason` and phase and carries exit code 1. It releases the run lock first, whatever the release prints: from `{project-root}`, run `uv run {runLockHelper} release --lock "{forge_version}/.test-skill.lock" --owner "{run_owner}"` (SKILL.md Workflow Rules). In headless mode it then writes `{"phase": "<phase>", "reason": "<the halt message>", "halt_reason": "<halt_reason>", "skill_name": "{skill_name}", "report_path": "{report_file}"}`, adding `"path"` when the halt names one, to `{run_dir}/halt.json` and runs:
 
-Load `{scoringRulesFile}` to get:
-- Category weights (naive vs contextual distribution)
-- Tier-dependent scoring adjustments
-
-**Resolve the pass threshold (precedence: CLI > pipeline default > scalar > bundled fallback):**
-
-1. If the workflow received `--threshold=<N>` on invocation, use that integer as `effective_threshold` (CLI wins). Set `threshold_source` = `"CLI override ({N}%)"`.
-2. Else if `{pipeline_default_threshold}` is set in workflow context (resolved by init.md §1b from the per-pipeline threshold lookup table when `{pipeline_alias}` is present), use it as `effective_threshold`. Set `threshold_source` = `"pipeline default ({pipeline_alias} → {N}%)"`.
-3. Else if the resolved `{defaultThreshold}` workflow-context variable (from SKILL.md On Activation §3 — `workflow.default_threshold` scalar, default `80`) is set, use it as `effective_threshold`. Set `threshold_source` = `"workflow default ({N}%)"`.
-4. Else fall back to `80` (the bundled default — this branch should be unreachable when SKILL.md resolution ran correctly, but keeps the step robust if customize.toml resolution failed silently). Set `threshold_source` = `"bundled fallback (80%)"`.
+```bash
+uv run {emitEnvelopeHelper} emit-halt --workflow skf-test-skill --run-dir "{run_dir}" --target stderr < "{run_dir}/halt.json"
+```
 
-Store `threshold_source` in workflow context for use in the score report section.
+Display the line it prints, as the run's last line, then stop. If the emitter exits non-zero or prints no line, display the halt message alone. An interactive HALT displays its message and emits nothing.
 
-Pass `effective_threshold` into the scoring-input JSON's `threshold` field in §3a (the compute-score.py script already honors this field). The CLI flag, pipeline default, and the scalar all feed the same downstream field; the script does not need to know which layer supplied the value.
+### 1. Read the Pass Threshold
 
-**Docs-only mode check:** If the Coverage Analysis section in `{outputFile}` notes docs-only mode (set by step 3 for skills with all `[EXT:...]` citations and no local source), apply Quick-tier weight redistribution: Signature Accuracy and Type Coverage are not scored, their weights (22% + 14%) are redistributed proportionally to remaining active categories. Coverage score is based on documentation completeness rather than source coverage (as calculated by step 3).
+`{scoringScript}` owns the category weights, their redistribution and every verdict rule; this step sets its flags, hands it the score files and reads its output.
 
-### 2. Read Category Scores from Output
+**The pass threshold was resolved once, at init** (init.md §1b, precedence: CLI > pipeline default > scalar > bundled fallback), and init.md §6c wrote it into the report. Read `threshold` and `thresholdSource` from the `{outputFile}` frontmatter and bind `effective_threshold` ← `threshold` as a number (without its `%`) and `threshold_source` ← `thresholdSource`. Never resolve the precedence again here: the frontmatter keeps what a long run's context may lose.
 
-Read `{outputFile}` and extract the category scores calculated in previous steps:
+Pass `effective_threshold` into the scoring-input JSON's `threshold` field in §3a. The CLI flag, pipeline default, and the scalar all feed the same downstream field; the script does not need to know which layer supplied the value.
 
-**From Coverage Analysis (step 03):**
-- Export Coverage: {percentage}%
-- Signature Accuracy: {percentage}% or N/A (Quick tier)
-- Type Coverage: {percentage}% or N/A (Quick tier)
+### 2. The Category Score Files
 
-**From Coherence Analysis (step 04):**
-- Combined Coherence: {percentage}% (contextual mode only)
-- Or: not scored (naive mode — weight redistributed)
+The category scores are never read back out of the report: each is the output file of the script that computed it, in the run folder `{run_dir}` (init.md §6b), and §3b hands the files to the scoring script by path:
 
-**From External Validation (step 04b):**
-- External Validation Score: {percentage}% (combined skill-check + tessl average)
-- Or: N/A (if neither tool was available — weight redistributed to other categories)
+- `coverage.json`: Export Coverage (coverage-check §2c)
+- `signatures.json`: Signature Accuracy and Type Coverage (coverage-check §2b; there is none at Quick tier, for a docs-only skill, at States 2 to 5, or for a stack or a reference app)
+- `coherence.json`: the combined coherence (coherence-check §5c; contextual mode only)
+- `external.json`: the external validation score (external-validators §4; its `externalScore` is null when neither validator scored)
+- `surface.json`: at State 2, the provenance-map and metadata counts behind the State 2 undercount deduction (coverage-check §2)
 
 ### 3. Apply Weight Distribution
 
 **Read testMode from {outputFile} frontmatter.**
 
-#### 2b. Apply State 2 Undercount Deduction (pre-script)
-
-If `analysis_confidence == 'provenance-map'` (State 2) AND `metadata.json.skill_type != "stack"` AND step 3 recorded a provenance vs metadata divergence > 5% (see §4b in step 3), apply a 10-point deduction to `exportCoverage` BEFORE building the scoring input:
-
-```
-exportCoverage_adjusted = max(0, exportCoverage - 10)
-```
-
-Record in the report: `scoring_notes: State 2 undercount risk acknowledged — 10% deduction applied to Export Coverage (raw: {N}%, adjusted: {M}%)`. Use the adjusted value as the `exportCoverage` field in §3a below. The deduction is deterministic and does not change category weights or active-category counting.
-
-Stack skills are exempt (the `metadata.json.skill_type != "stack"` guard above; `skill_type` is loaded in step 01 and also surfaces as `stackSkill` in §3a): a stack's own barrel is empty by design, so a provenance-vs-metadata divergence on a stack reflects the barrel-vs-constituent surface difference (suppressed by §4b's stack-skill branch in step 3), not extraction undercount — deducting for it would penalize a correctly built stack.
-
 #### 3a. Construct Scoring Input JSON
 
-Build a JSON object from the data gathered in steps 1-2:
+Build the flags the script reads from workflow context, and `toolingStatus` from the `{outputFile}` frontmatter (init.md §6c wrote it), never from memory:
 
 ```json
 {
   "mode": "{testMode: contextual or naive}",
   "tier": "{forge_tier: Quick, Forge, Forge+, or Deep}",
-  "docsOnly": "{true if docs_only_mode detected in step 03, else false}",
+  "docsOnly": "{true if docs_only_mode is set (coverage-check §0), else false}",
   "state2": "{true if analysis_confidence is provenance-map, else false}",
   "stackSkill": "{true if metadata.json.skill_type == 'stack', else false}",
   "referenceApp": "{true if metadata.json.scope_type == 'reference-app', else false}",
-  "scores": {
-    "exportCoverage": "{export_coverage_percentage}",
-    "signatureAccuracy": "{signature_accuracy_percentage or null if N/A}",
-    "typeCoverage": "{type_coverage_percentage or null if N/A}",
-    "coherence": "{combined_coherence_percentage or null if naive mode}",
-    "externalValidation": "{external_validation_score or null if N/A}"
-  },
-  "threshold": "{effective_threshold from §1 — CLI --threshold wins, then pipeline default, then workflow.default_threshold scalar, then 80}",
-  "analysisConfidence": "{resolved analysis confidence — 'degraded' when python3/frontmatter validator missing; else full/provenance-map/metadata-only/remote-only/docs-only; omit if unknown}",
-  "toolingStatus": "{a missing-helper marker like 'python3-missing' or 'frontmatter-validator-missing' when a helper is unavailable, else omit}"
+  "threshold": "{effective_threshold from §1}",
+  "analysisConfidence": "{analysis_confidence: full, provenance-map, metadata-only, remote-only or docs-only}",
+  "toolingStatus": "{toolingStatus from the {outputFile} frontmatter: ok, or frontmatter-validator-timeout}"
 }
 ```
 
-**Important:** Score values must be numbers (not strings). Use `null` (not `"N/A"`) for categories that were not scored. `analysisConfidence` and `toolingStatus` are optional strings — pass them so the script can apply the post-score tooling cap (§3d) deterministically; the script fires Cap 1 when `analysisConfidence == "degraded"` or `toolingStatus` contains `missing`. Read `metadata.json.skill_type` from `{resolved_skill_package}/metadata.json`; if the value is `"stack"`, set `stackSkill: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (the categories will be redistributed per `{scoringRulesFile}` Stack Skills rule). Likewise read `metadata.json.scope_type`; if the value is `"reference-app"`, set `referenceApp: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (redistributed per `{scoringRulesFile}` Reference-App rule — a reference app documents wiring patterns, not library export signatures).
+**Important:** the flags are bare booleans (`true`, not `"true"`) and the threshold a number. Always pass `toolingStatus`: any value other than `ok` fires Cap 1 (§3d). `analysisConfidence` names the source access only, never tooling health. Read `metadata.json.skill_type` and `metadata.json.scope_type` from `{resolved_skill_package}/metadata.json`: a stack (`stackSkill`) and a reference app (`referenceApp`, which documents wiring patterns, not library export signatures) have no Signature Accuracy or Type Coverage, and the script redistributes their weights. With no local source (`analysisConfidence` `metadata-only` or `remote-only`, States 3 and 4) it skips both categories at any tier, as it does at Quick tier, for `docsOnly` and for `state2`.
 
 #### 3b. Run the Scoring Script
 
+Pass the §3a JSON and the score files that exist: `--signatures` when coverage-check scored the signatures, `--coherence` in contextual mode, and `--surface` at State 2:
+
 ```bash
-echo '<JSON>' | uv run {scoringScript} --stdin
+uv run {scoringScript} --json-input '<the §3a JSON>' --coverage "{run_dir}/coverage.json" [--signatures "{run_dir}/signatures.json"] [--coherence "{run_dir}/coherence.json"] --external "{run_dir}/external.json" [--surface "{run_dir}/surface.json"] > "{run_dir}/score.json"
 ```
 
-Where `{scoringScript}` is the path resolved from the frontmatter variable (relative to the skill root, i.e., the skf-test-skill/ directory). The script also accepts the JSON as a positional argument (`uv run {scoringScript} '<JSON>'`) or via `--json-input '<JSON>'`; `--stdin` is preferred since it avoids shell-quote escaping of nested JSON.
-
-Parse the JSON output. The script returns:
-- `weights` — final redistributed weights per category
-- `weightedScores` — weighted contribution per category
-- `totalScore` — the overall completeness score
-- `threshold` — the threshold used
-- `result` — `"PASS"`, `"FAIL"`, or **`"INCONCLUSIVE"`** (minimum-evidence floor — see `{scoringRulesFile}`)
-- `activeCategories` — list of categories that were scored
-- `skippedCategories` — list of categories that were skipped
-- `skipReasons` — why each category was skipped
-- `weightSum` — sum of final weights (should be ~100)
-- `inconclusiveReasons` — only present when `result == "INCONCLUSIVE"`; explains which floor clause tripped
-- **Verdict-override group** — present as an atomic set of four keys only when a post-score cap or the threshold fallback engaged (§3d/§4b are applied by the script, not re-derived here):
-  - `effectiveResult` — the **final verdict** after caps + fallback (`"PASS"` / `"FAIL"`); read this as the outcome for §4–§8
-  - `capReason` — string describing the cap(s) that fired, or `null`
-  - `thresholdFallback` — `true` when the FAIL→PASS-at-80-floor fallback fired
-  - `originalThreshold` — the pre-fallback threshold when `thresholdFallback` is `true`, else `null`
+Where `{scoringScript}` is the path resolved from the frontmatter variable (relative to the skill root, i.e., the skf-test-skill/ directory).
+
+Read its output from `{run_dir}/score.json`, where the report step's result contract reads `activeCategories` and `inconclusiveReasons` too. The script returns:
+- `weights`: final redistributed weights per category
+- `weightedScores`: weighted contribution per category
+- `totalScore`: the overall completeness score
+- `threshold`: the threshold used
+- `result`: `"PASS"`, `"FAIL"`, or **`"INCONCLUSIVE"`** (the minimum-evidence floor)
+- `activeCategories`: list of categories that were scored
+- `skippedCategories`: list of categories that were skipped
+- `skipReasons`: why each category was skipped
+- `weightSum`: sum of final weights (should be ~100)
+- `inconclusiveReasons`: only present when `result == "INCONCLUSIVE"`; explains which floor clause tripped
+- `state2Deduction` and `scoringNotes`: present when the State 2 undercount deduction was weighed (`applied` says whether it lowered Export Coverage) and when a note applies
+- **Verdict-override group**: present as an atomic set of four keys only when a post-score cap or the threshold fallback engaged (§3d/§4b are applied by the script, not re-derived here):
+  - `effectiveResult`: the **final verdict** after caps + fallback (`"PASS"` / `"FAIL"`); read this as the outcome for §4 to §8
+  - `capReason`: string describing the cap(s) that fired, or `null`
+  - `thresholdFallback`: `true` when the FAIL→PASS-at-80-floor fallback fired
+  - `originalThreshold`: the pre-fallback threshold when `thresholdFallback` is `true`, else `null`
   - When the group is **absent**, no cap or fallback engaged and `result` is the final verdict.
 
-Use these values for Section 4 (pass/fail/inconclusive) and Section 6 (output formatting). **The script owns the minimum-evidence floor, both post-score caps, and the threshold fallback — everywhere below, read its emitted fields (`result`, `effectiveResult`, `capReason`, `thresholdFallback`) and never recompute a verdict, cap, or threshold decision.** The final verdict is `effectiveResult` when the override group is present, otherwise `result` (INCONCLUSIVE is never overridden).
-
-#### 3c. Fallback (if script execution fails)
-
-**First distinguish "the script could not run" from "the script rejected the input" — they take opposite paths.**
+Use these values for Section 4 (pass/fail/inconclusive) and Section 6 (output formatting). **The script owns the minimum-evidence floor, the State 2 deduction, both post-score caps, and the threshold fallback: everywhere below, read its emitted fields (`result`, `effectiveResult`, `capReason`, `thresholdFallback`, `scoringNotes`) and never recompute a verdict, cap, deduction or threshold decision.** The final verdict is `effectiveResult` when the override group is present, otherwise `result` (INCONCLUSIVE is never overridden).
 
-A `{"error": ..., "code": "INVALID_INPUT"}` envelope on stdout means the script ran fine and refused what it was given: a field is missing, mistyped, or out of range (exit 2), or the payload was not parseable JSON at all (exit 1). Either way the *input* is wrong, not the script. **Correct the §3a input and re-run.** Do not fall through to the manual redistribution below — it would hand-compute a total from the very numbers the script just refused, so an out-of-range `exportCoverage` would silently become a score. If the input cannot be corrected, report `score: scoring input rejected — {error}` and leave the score unset rather than emitting a computed total.
+#### 3c. If the Script Refuses the Input or Does Not Run
 
-The fallback below applies only when the script genuinely could not run — missing file, no `uv`, unreadable interpreter — that is, **no envelope on stdout at all**.
+A `{"error": ..., "code": "INVALID_INPUT"}` envelope in `{run_dir}/score.json` (exit 2, or exit 1 for a payload that is not JSON) means the script ran and refused what it was given: a flag is missing, mistyped or out of range, or a score file is missing its score or holds a refused result. **Correct the §3a input, or re-run the step that writes the refused file, and run it again.** Never compute a total by hand from the numbers the script refused.
 
-If the script is unavailable, redistribute each skipped category's weight (the `null`-scored categories in the §3a JSON — naive mode already zeroes coherence, and Quick-tier/docsOnly/state2/stackSkill/referenceApp already null out Signature Accuracy + Type Coverage) proportionally across the active categories, then report `total = Σ(weight/100 × category_score)` using the detected mode's weight table in `{scoringRulesFile}`. Report: "**Note:** Scoring script unavailable — calculated manually per scoring-rules.md."
+No envelope at all (the script file is missing, or it cannot start): HALT with its stderr (`halt_reason: "helper-failed"`, phase `score:compute`). A quality gate does not score by hand, so the run stops without a score rather than with an unchecked one.
 
 ### 3d. Read Post-Score Caps (applied by the script)
 
-The script applies two post-score caps — **Cap 1** (tooling degraded) and **Cap 2** (docs-only with no external validators) — and returns the settled outcome as `capReason` + `effectiveResult` (§3b). Read those fields; never recompute a cap. If `capReason` is non-null, record `scoring_notes: {capReason}` in the report. A fired cap forces the script's `PASS` into `FAIL`, never touches an INCONCLUSIVE verdict, and may still be re-flipped to PASS by the threshold fallback (§4b) — `effectiveResult` reflects that settled outcome. (Both caps exist because a degraded-tooling or docs-only-without-validators run has too thin an evidence base to trust a PASS; §3a passes the fields Cap 1 reads.)
+The script settles the verdict in the one order its docstring states, with two post-score caps: **Cap 1** (any `toolingStatus` other than `ok`) and **Cap 2** (docs-only with no external validation score). It returns the settled outcome as `capReason` + `effectiveResult` (§3b). Read those fields; never recompute a cap. If `capReason` is non-null, record `scoring_notes: {capReason}` in the report. A fired cap forces the script's `PASS` into `FAIL`, never touches an INCONCLUSIVE verdict, and is never re-flipped by the threshold fallback (§4b): a capped run stays FAIL whatever the threshold. (Both caps exist because a degraded-tooling or docs-only-without-validators run has too thin an evidence base to trust a PASS; §3a passes the field Cap 1 reads.)
 
 ### 4. Determine Result (PASS / FAIL / INCONCLUSIVE)
 
@@ -144,7 +116,7 @@ IF result == "INCONCLUSIVE" — minimum-evidence floor tripped; not PASS, not FA
 ELSE settled verdict = effectiveResult if the override group is present, else result   (PASS or FAIL)
 ```
 
-**INCONCLUSIVE floor clauses** (see `{scoringRulesFile}`):
+**INCONCLUSIVE floor clauses** (the script's minimum-evidence floor):
 - `active_categories < 2` (after all redistribution), OR
 - `tier == "Quick"` AND Export Coverage is the sole scoring contributor
 
@@ -152,7 +124,7 @@ ELSE settled verdict = effectiveResult if the override group is present, else re
 
 After §4 determines the result but before §5 recommends the next workflow, the script's threshold fallback may already have converted a FAIL into a PASS at the 80% floor. This step documents that quality compromise in an evidence report.
 
-The rule the script encodes (documented for interpretation): when `result == "FAIL"` AND `totalScore >= 80` AND `effective_threshold > 80`, the script overrides `result` to `"PASS"` at the 80% floor; an `INCONCLUSIVE` verdict is never overridden. Read the script's fields:
+Read the script's fields:
 
 - `thresholdFallback == true` → the fallback fired; `effectiveResult` is `"PASS"` and `originalThreshold` holds the pre-fallback threshold.
 - `thresholdFallback` absent or `false` → no fallback; the settled verdict from §4 stands.
@@ -164,17 +136,21 @@ The rule the script encodes (documented for interpretation): when `result == "FA
 3. Set `effective_threshold = 80` for use by §5/§6/§7/§8.
 4. Generate the evidence report (§4b.1 below).
 
-The cap↔fallback interaction (a cap forcing FAIL that the fallback then re-flips to PASS at the 80 floor) is resolved inside the script — `effectiveResult` reflects the settled outcome. The `{totalScore}` used in the report below is the script's raw `totalScore`.
+The `{totalScore}` used in the report below is the script's raw `totalScore`.
 
 #### 4b.1 Generate Evidence Report
 
 Write the evidence report to `{forge_version}/evidence-report-fallback.md`. The report documents the quality compromise for audit purposes.
 
-**Read gap entries:** extract findings from the Coverage Analysis and Coherence Analysis sections in `{outputFile}` — list each gap with severity (Critical through Low).
+**Read the gaps from the gap ledger**, not from the report's sections: the stages recorded every gap they found in `{ledgerFile}`, and the hard gate passed, so none is Critical or High. The script orders and counts them (`{gapLedgerScript}` resolves relative to the skill root):
 
-**Check for prior remediation:** glob `{forge_version}/test-report-{skill_name}-*.md` for a prior test report. If found, note the path — this implies remediation was attempted between runs. If not found, note "first test run — no prior remediation cycle".
+```bash
+uv run {gapLedgerScript} render --ledger "{ledgerFile}"
+```
+
+Exit 0: write its output unchanged under **Findings Preventing Higher Threshold**: the totals, the Remediation Summary and one entry per gap, by severity. A render that exits 1 prints its error on stderr: write that line there instead and continue, since the evidence report does not decide the verdict.
 
-**Check for post-score cap:** if Cap 1 or Cap 2 (§3d) fired, note the cap reason in the remediation section: "Score capped due to {cap_reason} — address tooling to test at the higher threshold."
+**Check for prior remediation:** glob `{forge_version}/test-report-{skill_name}-*.md` for a prior test report. If found, note the path — this implies remediation was attempted between runs. If not found, note "first test run — no prior remediation cycle".
 
 **Evidence report template:**
 
@@ -196,10 +172,7 @@ Write the evidence report to `{forge_version}/evidence-report-fallback.md`. The
 
 ## Findings Preventing Higher Threshold
 
-{For each gap entry from Coverage Analysis and Coherence Analysis:}
-- **{GAP-NNN}: {title}** — Severity: {severity}, Category: {category}
-
-{Count: N critical, M high, P medium, Q low, R info findings}
+{the output of the render command above, unchanged}
 
 ## Remediation Context
 
@@ -209,12 +182,9 @@ A prior test run was found at `{prior_report_path}`, indicating remediation was
 {If no prior test report:}
 No prior test report found for this skill version — this is the first test run.
 
-{If post-score cap was active:}
-**Note:** Score was capped due to {cap_reason}. Address tooling limitations to test at the higher threshold.
-
 ## Conclusion
 
-Skill accepted at 80% floor (original target: {original_threshold}%). The {N} findings above prevented meeting the higher threshold. Review and address findings before the next pipeline run to achieve the {original_threshold}% target.
+Skill accepted at 80% floor (original target: {original_threshold}%). The findings above prevented meeting the higher threshold. Review and address findings before the next pipeline run to achieve the {original_threshold}% target.
 ```
 
 Record `evidence_report_path: '{forge_version}/evidence-report-fallback.md'` in workflow context for use by §6/§7/§8 and by report.md.
@@ -225,10 +195,11 @@ Based on the **settled verdict** (§4 — `effectiveResult` when the override gr
 
 **IF PASS:**
 - `nextWorkflow: 'export-skill'` — skill is ready for export
-- **Drift override:** if workflow context carries
-  `allow_workspace_drift: true` (set in step 1 §5b when the user passed
+- **Drift override:** if the `{outputFile}` frontmatter's `workspaceDrift`
+  is `overridden` (init.md §5b wrote it when the user passed
   `--allow-workspace-drift` AND the workspace HEAD did not match
-  `metadata.source_commit`), the PASS is a **conditional PASS**:
+  `metadata.source_commit`; the flag on a clean tree leaves it `ok`), the
+  PASS is a **conditional PASS**:
   - Write `testResult: 'pass-with-drift'` to the output frontmatter instead of
     bare `'pass'`. The result contract (§4c of step 6) mirrors the same
     value.
@@ -245,9 +216,9 @@ Based on the **settled verdict** (§4 — `effectiveResult` when the override gr
 **IF INCONCLUSIVE:**
 - `nextWorkflow: 'manual-review'` — evidence base is insufficient to grade the skill automatically. The test report records `inconclusiveReasons` from the scoring script. Surface to the user — do not auto-recommend export or update.
 
-### 6. Append Completeness Score to Output
+### 6. Write the Completeness Score Section
 
-Append the **Completeness Score** section to `{outputFile}`:
+Write the **Completeness Score** section in place of the template's `## Completeness Score` heading and the placeholder comment under it, in `{outputFile}`:
 
 ```markdown
 ## Completeness Score
@@ -277,11 +248,14 @@ Append the **Completeness Score** section to `{outputFile}`:
 **Threshold Fallback:** scored {totalScore}% against {original_threshold}% target — accepted at 80% floor. Evidence report: {evidence_report_path}
 **Weight Distribution:** {naive (redistributed) | contextual (full)}
 **Tier Adjustment:** {none | Quick tier — signature and type coverage not scored}
-**External Validators:** {both available | skill-check only | tessl only | none — weight redistributed}
+**External Validators:** {skill-check and Tessl Review | skill-check only | Tessl Review only | none — weight redistributed} (from `{external_tools_used}`; `tessl` is Tessl Review)
 **Analysis Confidence:** {full | provenance-map | metadata-only | remote-only | docs-only}
+**Tooling Status:** {ok | frontmatter-validator-timeout}
+{If the script returned `scoringNotes` or a `capReason`:}
+**Scoring Notes:** {each entry of `scoringNotes`, and `capReason`}
 ```
 
-If `analysis_confidence` is not `full`, append a degradation notice. **The notice must be confidence-aware** — see the degradation notice rules in `{sourceAccessProtocol}`:
+If `analysis_confidence` is not `full`, add a degradation notice at the end of the Completeness Score section. **The notice must be confidence-aware** (see the degradation notice rules in `{sourceAccessProtocol}`):
 
 ```markdown
 ### Access Degradation Notice
@@ -294,12 +268,11 @@ If `analysis_confidence` is not `full`, append a degradation notice. **The notic
 ### 7. Update Output Frontmatter
 
 Update `{outputFile}` frontmatter:
-- `testResult: '{pass|pass-with-drift|fail|inconclusive}'` (lowercase; mirrors the **settled verdict** — `effectiveResult` when the override group is present, else `result` — with `pass-with-drift` substituted for `pass` when `allow_workspace_drift` was set and drift was observed — see §5 drift override)
+- `testResult: '{pass|pass-with-drift|fail|inconclusive}'` (lowercase; mirrors the **settled verdict**, `effectiveResult` when the override group is present, else `result`, with `pass-with-drift` substituted for `pass` when the frontmatter's `workspaceDrift` is `overridden`: see the §5 drift override)
 - `score: '{total}%'`
-- `threshold: '{threshold}%'`
-- `thresholdSource: '{threshold_source}'`
+- `threshold: '{threshold}%'`: the threshold the verdict was settled against (80 after a threshold fallback); `thresholdSource` keeps what init.md wrote
 - When `threshold_fallback` is true, add: `thresholdFallback: true`, `originalThreshold: '{original_threshold}%'`, `evidenceReportPath: '{evidence_report_path}'`
-- `analysisConfidence: '{full|degraded|provenance-map|metadata-only|remote-only|docs-only}'`
+- `analysisConfidence: '{full|provenance-map|metadata-only|remote-only|docs-only}'`
 - `nextWorkflow: '{export-skill|update-skill|manual-review}'`
 - Append `'score'` to `stepsCompleted`
 
@@ -311,5 +284,5 @@ Report the completeness score to the user: the total percentage and PASS/FAIL ve
 
 Then proceed to the gap report.
 
-Update stepsCompleted, then load and execute {nextStepFile}.
+`stepsCompleted` now ends with `'score'` (§7 appended it), so load and execute {nextStepFile}.
 

package/src/skf-test-skill/references/scoring-rules.md

@@ -2,164 +2,50 @@
 
 # Scoring Rules
 
-## Default Threshold
+## Where the Scoring Rules Live
 
-**Pass threshold:** 80%
+Scripts compute every score, so this file keeps no copy of their rules:
 
-## Category Weights
+- `scripts/compute-score.py` owns the category weights (contextual and naive), the redistribution of each skipped category's weight (Quick tier, docs-only, State 2, no local source at States 3 and 4, a stack, a reference app, no external validation score), the State 2 undercount deduction, the minimum-evidence floor (INCONCLUSIVE), the two caps and the threshold fallback, in the order its docstring states. score.md sets the flags it reads and reads its verdict.
+- `scripts/aggregate-coherence.py` owns the contextual coherence split between reference validity and integration completeness; coherence-check.md §5 decides which integration patterns are complete.
+- `scripts/reconcile-coverage.py` computes Export Coverage on every branch; a docs-only skill takes its `docsOnly` branch, the documentation completeness ratio coverage-check.md §2c names. `scripts/score-signatures.py` computes Signature Accuracy and Type Coverage.
 
-| Category               | Weight | Description                                                                               |
-|------------------------|--------|-------------------------------------------------------------------------------------------|
-| Export Coverage        | 36%    | Percentage of source exports documented in SKILL.md                                       |
-| Signature Accuracy     | 22%    | Documented signatures match actual source signatures                                      |
-| Type Coverage          | 14%    | Types and interfaces referenced are complete                                              |
-| Coherence (contextual) | 18%    | Cross-references valid, integration patterns complete                                     |
-| Coherence (naive)      | 0%     | Not applicable — weight redistributed to other categories                                 |
-| External Validation    | 10%    | Average of skill-check quality score + tessl average score (redistributed if unavailable) |
-
-## Naive Mode Weight Redistribution
-
-The following weights replace the default table for naive mode. The 18% coherence weight from the default table has been proportionally redistributed into these values. Do not re-redistribute for coherence (already handled in this table). Quick-tier redistribution (zeroing Signature Accuracy and Type Coverage) still applies as an additional step.
-
-When running in naive mode (no coherence category):
-- Export Coverage: 45%
-- Signature Accuracy: 25%
-- Type Coverage: 20%
-- External Validation: 10%
-
-## External Validation Unavailable
-
-When neither skill-check nor tessl is available, redistribute the 10% external validation weight proportionally to the other active categories. When only one tool is available, use that tool's score as the external validation score.
-
-## tessl and Split-Body Interaction
-
-tessl evaluates SKILL.md body content only — it does not read `references/*.md` files. After split-body extraction, the tessl content score will drop significantly (e.g., 65% to 38%) because Tier 2 content is no longer inline. This is expected behavior and does not reflect actual content quality. When reporting scores for a split-body skill, note: "tessl content score reflects post-split inline content only. Use the pre-split tessl score as the content quality baseline."
-
-## Tier-Dependent Scoring
-
-### Quick Tier (no tools)
-- Export Coverage: file/structure existence check only
-- Signature Accuracy: skipped (no AST)
-- Type Coverage: skipped (no AST)
-- Score based on: structural completeness only
-- Weight redistribution: skipped categories' weights (Signature Accuracy 22% + Type Coverage 14%) redistributed proportionally to remaining active categories
-
-### Docs-Only Mode (all [EXT:...] citations, any tier)
-
-When `docs_only_mode: true` is set by step 3 (indicating a skill where all SKILL.md citations are `[EXT:...]` format with no local source code):
-
-- **Signature Accuracy:** Not scored (no source to compare against)
-- **Type Coverage:** Not scored (no source to compare against)
-- **Weight redistribution:** Same as Quick tier — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories
-- **Export Coverage basis:** Documentation completeness rather than source coverage. Score = (documented_items_with_complete_descriptions / total_documented_items) * 100. A "complete" item has: description, parameters (if function/method), and return type (if function/method).
-- **Coherence:** Standard rules for the detected mode (naive or contextual) apply unchanged
-
-This is functionally identical to Quick tier weight redistribution but with a different coverage denominator (self-consistency instead of source comparison).
-
-**External-validator requirement for docs-only:** docs-only mode removes two categories (Signature Accuracy, Type Coverage) from scoring. If External Validation is also unavailable, the evidence base collapses to Coverage alone (naive) or Coverage + Coherence (contextual) — which in the naive/Quick case trips the minimum-evidence floor (INCONCLUSIVE). To keep docs-only skills gradable when external validators are present but still deterministic when they are missing: **when `docsOnly: true` AND `externalValidation is null`, step 5 must cap `totalScore` at `threshold - 1` (forcing FAIL) before the INCONCLUSIVE floor is evaluated.** This prevents a docs-only skill from PASSing with only one or two redistributed categories carrying all the weight. Implement in step 5 §4 as a pre-compare cap, recorded in the report as `scoring_notes: docs-only without external validators — capped below threshold`.
-
-### Stack Skills (Any Tier)
-
-When `metadata.json.skill_type == "stack"` (set `stackSkill: true` in the scoring input):
-
-- **Signature Accuracy:** N/A — a stack skill's "signature surface" is the external library API it composes (pydantic, SQLAlchemy, FastAPI, etc.), not a proprietary surface the skill authors. Grading signatures against a surface the skill does not own produces meaningless numbers.
-- **Type Coverage:** N/A — same rationale; the type surface belongs to the external libraries.
-- **Weight redistribution:** Same as Quick tier / docs-only / State 2 — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories (Export Coverage, Coherence, External Validation).
-- **Applies regardless of detected tier** (Quick, Forge, Forge+, Deep) and is independent of `docsOnly` and `state2`. A stack skill can also be docs-only or State 2; the skip reasons combine additively (e.g. `"stack skill (external type surface) + State 2 (provenance-map)"`).
-- **Detection:** step 5 reads `metadata.json.skill_type` from the skill package. If the value is `"stack"`, set `stackSkill: true` in the scoring input JSON.
-
-### Reference-App Skills (Any Tier)
-
-When `metadata.json.scope_type == "reference-app"` (set `referenceApp: true` in the scoring input):
-
-- **Signature Accuracy:** N/A — a reference app documents wiring patterns (how surfaces are composed), not a library export surface the skill authors. There are no public-export signatures to grade against; the Pattern Surface replaces the Key API Summary (see `skf-create-skill` Reference-App Assembly Overrides).
-- **Type Coverage:** N/A — same rationale; a reference app has no library type surface to cover. Coverage is measured as pattern-surface coverage (`stats.pattern_surfaces_documented`), not export/type coverage.
-- **Weight redistribution:** Same as Quick tier / docs-only / State 2 / stack — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories (Export Coverage, Coherence, External Validation).
-- **Applies regardless of detected tier** (Quick, Forge, Forge+, Deep) and is independent of `docsOnly` and `state2`; skip reasons combine additively. `referenceApp` and `stackSkill` are distinct scope/type signals and should not both be set for the same skill.
-- **Detection:** step 5 reads `metadata.json.scope_type` from the skill package. If the value is `"reference-app"`, set `referenceApp: true` in the scoring input JSON. The skip reason recorded is `"reference-app (no library export signatures)"`.
-
-### State 2 Source Access (Any Tier, Provenance-Map Only)
-
-When source is not locally available and analysis resolves to State 2 (provenance-map baseline per source-access-protocol.md):
-
-- **Signature Accuracy:** N/A — provenance-map stores parameters as flat string arrays; verification is string comparison only, not semantic AST verification. Type aliases (`str` vs `String`, `list` vs `List[Any]`) cannot be resolved without live source.
-- **Type Coverage:** N/A — cannot verify type completeness without local source access for AST re-parsing.
-- **Weight redistribution:** Same as Quick tier — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories (Export Coverage, Coherence, External Validation).
-- **Applies regardless of detected tier** (including Forge, Forge+, Deep) whenever `analysis_confidence` is `provenance-map` and local source is unavailable.
-- **Export Coverage denominator:** Uses the union of provenance-map entry names and metadata.json `exports[]` names (per source-access-protocol.md State 2 rules).
-
-Note: When provenance-map entries are predominantly T1 (AST-verified at compilation time), the coverage and name-matching data is already at highest confidence. The N/A categories reflect the inability to re-verify at test time, not low-quality extraction data.
-
-**State 2 undercount risk acknowledgement:** provenance-map is a cached extraction snapshot — if the source has evolved since extraction, public API adds/removes will not surface in Export Coverage (denominator is frozen to the provenance-map union). When `state2: true` AND step 3 records any provenance vs metadata divergence (e.g. union > either source by >5%), apply a flat **10% deduction** to `exportCoverage` before calling the scoring script, AND set `analysis_confidence: provenance-map` (already set) with a report note: `scoring_notes: State 2 undercount risk acknowledged — 10% deduction applied to Export Coverage`. Rationale: the skill cannot be reliably scored on a frozen denominator when the cache is known to disagree with its own metadata; prefer understating over overstating.
-
-### Forge Tier (ast-grep)
-- Export Coverage: AST-backed export comparison
-- Signature Accuracy: AST-verified signature matching
-- Type Coverage: AST-verified type completeness
-- Full scoring formula applied
-
-### Forge+ Tier (ast-grep + ccc)
-- Same scoring as Forge tier — ccc provides pre-ranking but does not change scoring weights
-- Improved extraction coverage (from ccc pre-discovery) may increase T1 count, but scoring formula is identical to Forge
-- Full scoring formula applied
-
-### Deep Tier (ast-grep + gh + QMD)
-- All Forge tier checks plus:
-- Cross-repository reference verification
-- QMD knowledge enrichment for coherence
-- Full scoring formula with maximum depth
-- **Migration & Deprecation Warnings section:** If T2-future annotations exist in the enrichment data, verify that Section 4b is present in SKILL.md Tier 1 and that each warning traces to a T2 provenance citation. If no T2-future annotations exist, Section 4b should normally be absent (not empty). Presence/absence mismatch is a Medium severity gap — with one Info-severity exception for historical-migration content (completed package renames, consolidated import paths, shipped API cutovers that remain load-bearing for training-data drift remediation). See `references/coherence-check.md` §2b/§5b for the three-case rule.
-
-## Score Calculation
-
-```
-score = sum(category_weight * category_score) for each category
-category_score = (items_passing / items_total) * 100
-```
-
-## Coherence Score Aggregation (Contextual Mode)
-
-```
-reference_validity = (valid_references / total_references) * 100
-integration_completeness = (complete_patterns / total_patterns) * 100
-combined_coherence = (reference_validity * 0.6) + (integration_completeness * 0.4)
-```
-
-If no integration patterns exist, combined coherence equals reference validity.
-
-This is the documented contract. The tally + weighted mean is computed by `scripts/aggregate-coherence.py` (invoked from coherence-check.md §5c), not by hand; the per-reference validity judgment (§4) and pattern-completeness judgment (§5) stay in the prompt, only the arithmetic is scripted.
-
-## Result Determination
-
-Three-state gate — **PASS / FAIL / INCONCLUSIVE**. `INCONCLUSIVE` is not PASS and not FAIL; it signals insufficient evidence to grade the skill. Downstream workflows must treat `INCONCLUSIVE` as a hard gate — do not export, do not auto-retry, surface to the human.
-
-- **Minimum-Evidence Floor (applies before PASS/FAIL comparison):**
-  - `active_categories` = count of categories with a non-zero final weight *after* all redistribution (Quick tier, docs-only, State 2, external-validator-unavailable). Categories with a redistributed weight of 0 do not count as active, even if they received a score.
-  - **If `active_categories < 2`** → force `result: INCONCLUSIVE` with rationale `"insufficient evidence: only {N} active category"`. A single active category cannot cross-validate itself and a PASS would be a false signal.
-  - **If `tier == "Quick"` AND the sole active contributor is Export Coverage** → force `result: INCONCLUSIVE` with rationale `"Quick tier: Export Coverage alone is insufficient evidence — add a second active category by upgrading tier or enabling external validators"`. This catches the degenerate case where every signature/type/coherence/external category gets redistributed to 0 and Export Coverage is doing all the work.
-  - The floor is enforced by `scripts/compute-score.py`. The step 5 scoring step reads `result` from the script output and writes it into the test report frontmatter unchanged.
-
-- Otherwise:
-  - score >= threshold → PASS
-  - score < threshold → FAIL
-
-The floor is intentionally conservative: skf-test-skill grades other skills, so a false PASS has catastrophic downstream effects (polluted exports, misleading feasibility data). Falling back to INCONCLUSIVE is always preferred over a low-evidence PASS.
+The pass threshold is `{defaultThreshold}`, unless `--threshold` or the pipeline's default overrides it (score.md §1).
 
 ## Gap Severity
 
-| Severity | Criteria                                                                                                       |
-|----------|----------------------------------------------------------------------------------------------------------------|
-| Critical | Missing exported function/class documentation                                                                  |
-| High     | Signature mismatch between source and SKILL.md                                                                 |
-| Medium   | Missing type or interface documentation                                                                        |
-| Medium   | Migration section present/absent mismatch with T2-future annotation data (Deep tier)                           |
-| Medium   | Metadata drift — intra-cluster export counts diverge (barrel: `stats.exports_public_api` vs `exports[].length`; or documented-surface: `stats.exports_documented` vs provenance-map entry count; >10% divergence) |
-| Medium   | Denominator inflation — stratified-scope `scope.include` union exceeds provenance-map entry count by >25% (brief missing `scope.tier_a_include`) |
-| Medium   | Script/asset directory exists but no Scripts & Assets section in SKILL.md                                      |
-| Medium   | Scripts & Assets section references file not found in scripts/ or assets/ directory                            |
-| Low      | Script/asset file present without provenance entry in provenance-map.json file_entries                         |
-| Low      | Missing optional metadata or examples                                                                          |
-| Low      | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps) |
-| Info     | Style suggestions, non-blocking observations                                                                   |
-| Info     | Discovery testing not performed — realistic prompt testing recommended before export                           |
-| Info     | Multi-denominator reporting — barrel vs documented-surface clusters diverge by design (>10% cross-cluster)     |
+This table is the single source of truth for classifying a gap. Every stage records each gap it finds in the gap ledger (`scripts/gap-ledger.py`, `{forge_version}/test-findings-{run_id}.json`) with the severity and the category of the row it matches. The hard gate (step 4c) blocks a run on any Critical or High gap, before scoring. A Medium, Low or Info gap never blocks. Scores come from the category metrics (a missing export lowers Export Coverage), and the pass threshold decides the verdict. Discovery testing runs after the gate, so a discovery gap of any severity is counted in the Gap Report and blocks nothing.
+
+| Severity | Category                    | Criteria |
+|----------|-----------------------------|----------|
+| Critical | `signature-mismatch`        | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type (coverage-check §2) |
+| Critical | `fabricated-signature`      | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name (coverage-check §2c) |
+| Critical | `broken-reference`          | Broken reference (contextual mode): a reference whose target does not exist, such as a file not at the stated path, a skill not in the skills output folder, a type no module declares, or a `scripts/` or `assets/` file the skill names (coherence-check §4) |
+| High     | `inaccurate-reference`      | Inaccurate reference (contextual mode): the target exists but does not match, such as a type the cited module does not export, a signature that differs, or an integration pattern that does not match the implementation (coherence-check §4) |
+| High     | `reference-escape`          | A reference whose real path leaves the skill, its source and, for a stack, the skills output folder (coherence-check §4) |
+| High     | `split-body-mismatch`       | The SKILL.md body and a `references/` file document one export differently (coverage-check §1b) |
+| High     | `numerator-inflation`       | `stats.exports_documented` equals the denominator while declared exports are absent from SKILL.md and `references/` (coverage-check §4b) |
+| High     | `structural`                | Naive mode: a missing required section, an unbalanced code fence, or a documented async or sync behaviour its example contradicts (coherence-check §2.1, §2.2, §2.5) |
+| High     | `discovery`                 | Discovery test: at most 1 of 3 realistic prompts routed to the skill (report §4b.3) |
+| Medium   | `missing-export`            | Missing export: an exported function or class the skill does not document. It lowers Export Coverage, and the threshold decides the verdict |
+| Medium   | `missing-type`              | Missing type or interface documentation |
+| Medium   | `stale-documentation`       | A documented export the enumerated source surface lacks that is not a fabricated signature: its cited file defines it, or no cited line could be checked (coverage-check §2c) |
+| Medium   | `migration-section`         | Migration section present/absent mismatch with T2-future annotation data (Deep tier; Case 1 and Case 3 of `migration-section-rules.md`) |
+| Medium   | `metadata-drift`            | Metadata drift: intra-cluster export counts diverge by more than 10% (barrel: `stats.exports_public_api` vs `exports[].length`; documented surface: `stats.exports_documented` vs the provenance-map named-export count), or denominator deflation: the re-derived source barrel exceeds `effective_denominator` by more than 25% and the brief has no `scope.tier_a_include` (coverage-check §4 and §4b) |
+| Medium   | `denominator-inflation`     | Denominator inflation: the stratified-scope `scope.include` union exceeds the provenance-map entry count by more than 25% (brief missing `scope.tier_a_include`) |
+| Medium   | `scripts-assets`            | A `scripts/` or `assets/` directory exists but SKILL.md has no Scripts & Assets section (coherence-check §2.7 and §4) |
+| Medium   | `integration-pattern`       | Incomplete integration pattern (contextual mode): one entry of coherence-check §5's `incomplete_patterns` |
+| Medium   | `structural`                | Naive mode: an opening code fence without a language tag, an exported function or method no usage-family section or reference file names, or a table row whose column count differs from its header (coherence-check §2.3, §2.4, §2.6) |
+| Medium   | `discovery`                 | Discovery test: 2 of 3 realistic prompts routed to the skill (report §4b.3) |
+| Low      | `scripts-assets-provenance` | Script/asset file present without provenance entry in provenance-map.json file_entries |
+| Low      | `provenance-line`           | Provenance line is not the definition of an export (`line-not-definition`, coverage-check §4c) |
+| Low      | `migration-section`         | Case 3 of `migration-section-rules.md` that the reviewer downgraded with an inline justification |
+| Low      | `metadata`                  | Missing optional metadata or examples |
+| Low      | `description`               | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps), a Tessl Review description suggestion included |
+| Low      | `external-validator`        | A skill-check diagnostic or a Tessl Review validation finding the run left unresolved (external-validators §5b) |
+| Info     | `observation`               | Style suggestions, non-blocking observations |
+| Info     | `provenance-unverified`     | Provenance line not verified: the line-check rules found no definition line (coverage-check §4c) |
+| Info     | `migration-section`         | Historical migration content in Section 4b with no T2-future annotation (Case 2 of `migration-section-rules.md`) |
+| Info     | `multi-denominator`         | Multi-denominator reporting: barrel and documented-surface clusters diverge by design (>10% cross-cluster) |
+| Info     | `discovery`                 | Discovery testing not performed (`--no-discovery`, a catalog of fewer than 2 skills, or no subagents to route the prompts): realistic prompt testing recommended before export (report §4b) |
+| Info     | `external-validator`        | A Tessl Review content suggestion (advisory; one marked `(not applicable: …)` is not recorded) |