bmad-module-skill-forge 2.2.0 → 4.0.0

package/docs/verifying-a-skill.md

@@ -1,53 +1,66 @@
 ---
 title: Verifying a Skill
-description: How to audit any SKF-compiled skill — walk every instruction back to an upstream commit and line number in under 60 seconds.
+description: How to audit an SKF skill by tracing its instructions back to an upstream commit and line number in under 60 seconds.
 ---
 
-**Nothing is made up.** Every instruction in every skill traces back to a specific file, a specific line, and a specific commit in the upstream source. If a skill claims a function exists, you can open the real source tree at the pinned commit and see it with your own eyes. If the claim and the source disagree — that's a bug, and SKF treats it as one.
+**Nothing is made up.** Every instruction in a skill compiled from source with Create Skill or Stack Skill traces back to a specific file, a specific line, and a specific commit in the upstream source. If a skill claims a function exists, you can open the real source tree at the pinned commit and see it with your own eyes. If the claim and the source disagree, that's a bug, and SKF treats it as one.
 
 ---
 
 ## The three-step audit
 
-Pick any symbol in any SKF-compiled skill. You can trace it to the exact line of upstream source in under 60 seconds.
+Pick any symbol in a skill compiled from source. You can trace it to the exact line of upstream source in under 60 seconds.
+
+This audit needs the provenance map that Create Skill and Stack Skill write. Quick Skill writes none, so its skills name the upstream repository but not the line behind each symbol. A docs-only skill cites documentation links (`[EXT:...]`) instead of source lines, so it has no commit to check. A stack skill built from code has no `source_commit` or `source_repo` in its `metadata.json`. Look in its `provenance-map.json` (step 2) instead, where `source_repo` lists each repository and `source_commit` lists one commit per repository. A stack skill composed from other skills has no commit of its own and traces back to those skills instead.
 
 ### 1. Open the skill's `metadata.json`
 
 Every skill ships a `metadata.json` next to its `SKILL.md`. Note two fields:
 
-- `source_commit` — the exact commit SHA the skill was compiled from
-- `source_repo` — the upstream repository
+- `source_commit`: the exact commit SHA the skill was compiled from
+- `source_repo`: the upstream repository
 
 This is the anchor. Everything else traces back to this commit.
 
 ### 2. Open the skill's `provenance-map.json`
 
-Provenance maps live in `forge-data/{skill}/{version}/provenance-map.json` alongside each compiled skill. Find your symbol. Every entry carries its own `source_file` and `source_line`:
+Provenance maps live in `forge-data/{skill}/{version}/provenance-map.json` in the project that compiled the skill. They are part of the audit trail and do not ship inside the skill package, so for a published skill, look in the repository that publishes it (oh-my-skills keeps its `forge-data/` folder public). Find your symbol. Every entry carries its own `source_file` and `source_line`, and its `confidence` and `extraction_method` name the tool that read it. The skill's `SKILL.md` also cites each signature inline, as `[AST:file:Lnn]` for an export ast-grep matched or `[SRC:file:Lnn]` for one read by eye, so you can often go straight to step 3. The entry for an export ast-grep matched looks like this:
 
 ```json
 {
   "export_name": "search",
-  "export_type": "function",
-  "params": ["query_text: str", "query_type: SearchType = GRAPH_COMPLETION", "top_k: int = 10"],
+  "export_type": "async_function",
+  "source_library": "cognee",
+  "params": ["query_text: str", "query_type: SearchType = SearchType.GRAPH_COMPLETION"],
   "return_type": "List[SearchResult]",
   "source_file": "cognee/api/v1/search/search.py",
   "source_line": 27,
   "confidence": "T1",
-  "extraction_method": "ast-grep"
+  "extraction_method": "ast-grep",
+  "ast_node_type": "function_definition",
+  "signature_source": "T1"
 }
 ```
 
-The snippet above is a real entry from [`forge-data/oms-cognee/1.0.0/provenance-map.json`](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-cognee/1.0.0/provenance-map.json). Line number is not rounded. Confidence tier is explicit. Extraction method is named. Nothing is paraphrased.
+This is the entry SKF 3.0.0 writes for cognee's `search` when ast-grep matches it, shortened to its first two parameters (`search` takes 20). It points at line 27, where `search` is defined at cognee v1.0.0's commit (`3c048aa4`), and its labels follow the tool that read it: an ast-grep match gives `T1`, `ast-grep`, the node kind its rule matched and an `[AST:]` citation. An export read by eye instead, because ast-grep could not parse its file, an ast-grep call that failed or timed out left its file unread, or no rule matched it, gets `T1-low`, `source-read`, no node kind and an `[SRC:]` citation, at every forge tier.
+
+The published entry in oh-my-skills' [`forge-data/oms-cognee/1.0.0/provenance-map.json`](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-cognee/1.0.0/provenance-map.json) is a good one to check by hand, because it was compiled before labels followed the tool. It records `search` as `source-read` but labels it `T1`, and it points at line 26, the blank line above the definition, while the skill's `SKILL.md` cites `[AST:cognee/api/v1/search/search.py:L27]`. SKF 3.0.0's [line, label and citation checks](#workflow-time-enforcement) flag all three when Create Skill or Update Skill writes such a map: a source read is `T1-low` with an `[SRC:]` citation, and its line is 27, the `def` line.
 
 ### 3. Visit the upstream repo at the pinned commit
 
 Open `{source_repo}` at `{source_commit}`, jump to `{source_file}` line `{source_line}`. The signature in `SKILL.md` should match what you see in the source.
 
-If it doesn't, **that's a bug**. [Open an issue](https://github.com/armelhbobdad/bmad-module-skill-forge/issues/new/choose). SKF will republish the skill with a new commit SHA and a new provenance map. Falsifiability isn't a feature — it's the whole deal.
+If it doesn't, **that's a bug**. For a skill you built, Test Skill (`@Ferris TS`) rates a signature mismatch as a Critical gap, which stops the test at its [hard gate](#hard-gate), and Update Skill (`@Ferris US`) with `--from-test-report` repairs the gaps a test report lists. If you think SKF extracted it wrong, [open an issue](https://github.com/armelhbobdad/bmad-module-skill-forge/issues/new/choose). Falsifiability isn't a feature. It's the whole deal.
 
 ### Workflow-time enforcement
 
-The same anchor is enforced automatically by `skf-test-skill` and by gap-driven `skf-update-skill`. Before either workflow reads source at a recorded `source_line`, it runs `git rev-parse HEAD` on the local workspace and compares it to `metadata.source_commit`. If the workspace has drifted, the workflow halts with a `halted-for-workspace-drift` status and tells you the exact `git checkout {source_ref}` to re-sync — so spot-checks can never silently verify against the wrong tree. Pass `--allow-workspace-drift` to opt in to reading the current HEAD anyway; the override is recorded in the final report rather than hidden.
+SKF checks the same anchor while it works, so a skill and its citations always describe one commit.
+
+- **Create Skill (`@Ferris CS`)**, at Forge tier and above, reads a remote source in a private checkout at the commit it resolves from the brief: the ref its `target_ref` names, or the tag that matches its version. It records that commit as the skill's `source_commit`, and it moves SKF's shared clone of the repository to the same commit. If it could not move the clone, it says why, and the run skips ccc discovery and the ccc index registration.
+- **Update Skill (`@Ferris US`)**, for a skill built from a remote repository, reads the source in a private checkout at the commit that the skill's `source_ref` (or the `--target-ref` you pass) points to now. When it writes, it records that commit as the new `source_commit`. If it could not move SKF's shared clone to that commit, its report says why and how to move it.
+- **Test Skill (`@Ferris TS`)**, and **Update Skill with `--from-test-report`** (a repair driven by a test report's gaps), first compare the commit your local source is on (`git rev-parse HEAD`) with `metadata.source_commit`. If they differ, they stop before reading any source: Test Skill with `workspace-drift`, Update Skill with `halted-for-workspace-drift`. The message names the commit to check out, so a spot-check never quietly reads the wrong tree.
+- To read the current commit anyway, pass `--allow-workspace-drift`. The final report records the override. In Test Skill, a pass under this override is recorded as `pass-with-drift`, and SKF recommends Update Skill instead of export. Update Skill with `--from-test-report` takes nothing from the current commit: it never moves a provenance line to it or records one from it, and it reads no signature, parameter list, return type or node kind there. Every new or changed export in the test report, whatever its severity, needs one of those from the source, so it stops the run with `halted-for-workspace-drift` before it changes the skill, and the message lists what each gap needs. A rescope (a gap that takes an export out of the skill's scope) stops the run the same way, because recounting the public API needs the pinned tree: the run has not written the scope amendment to the skill brief yet, so the brief stays as it was and a re-run asks again. A report whose gaps are only provenance line fixes, structural fixes or metadata patches still runs: a line it would have moved is recorded as `unknown` instead, and a node kind it would have looked up in the source stays as it is, each with a warning, while the stats keep the public API counts `metadata.json` records.
+- **Line, label and citation checks.** When the source is on disk, Create Skill and Update Skill check that each provenance `source_line` in a Python, TypeScript or JavaScript file is the line that defines its export (the `def` or declaration line, not a decorator or blank line above it). With or without the source, they check that each `[AST:]` / `[SRC:]` citation matches how its entry was extracted: ast-grep gives `[AST:]`, reading the source by eye gives `[SRC:]`. Each entry's labels must also match its `extraction_method`: an ast-grep entry is `T1` with a node kind, and an entry read by eye is `T1-low` with no node kind and a `signature_source` other than `T1`. They relabel an entry that disagrees, so a source read labeled `T1` becomes `T1-low`. They also ask the ast-grep CLI whether each ast-grep entry's `ast_node_type` is a node kind of its file's language, and replace a kind it does not know with the kind the matching recipe declares. They fix each finding that has one answer and list the rest as warnings in the evidence report. With the local source at hand, Test Skill reports each wrong line as a Low gap, and Update Skill with `--from-test-report` moves the entry to its definition line. When the checks find no line that defines an export, which can mean a shape they do not cover, the entry is left as it is and Test Skill asks you to check it by hand.
 
 ---
 
@@ -61,7 +74,7 @@ Every file in the per-skill output carries a specific job. Here's the lookup tab
 | Which symbols are documented and where did each come from? | `forge-data/{name}/{version}/provenance-map.json` |
 | What AST patterns were used for extraction? | `forge-data/{name}/{version}/extraction-rules.yaml` |
 | What signatures, types, and examples did the extractor actually capture? | `forge-data/{name}/{version}/evidence-report.md` |
-| How was the skill scored? Show me the math. | `forge-data/{name}/{version}/test-report-{name}-{run_id}.md` (per-run, run-id-suffixed — newest wins) |
+| How was the skill scored? Show me the math. | `forge-data/{name}/{version}/test-report-{name}-{run_id}.md` (one report per run, named with the run ID; the newest one counts. Test Skill gives a report this name only once its checks pass, so a run that halts leaves just its hidden in-progress file, `.skf-test-report-{name}-{run_id}.md`, and the last finished report still counts. Older reports, like the oh-my-skills ones linked below, are named `test-report-{name}.md`) |
 | How was the skill scoped, and what was deliberately left out? | `forge-data/{name}/skill-brief.yaml` |
 
 Everything a reader needs to reconstruct the compilation is in the two sibling directories: `skills/` ships to consumers, `forge-data/` is the audit trail.
@@ -70,22 +83,22 @@ Everything a reader needs to reconstruct the compilation is in the two sibling d
 
 ## The scores, including the ones we lose
 
-Completeness scoring is never 100%. The [scoring formula](#how-the-score-is-computed) is deterministic and the pass threshold is **80%** — but every test report also logs the specific edges where a skill falls short, so the numbers aren't marketing.
+The [scoring formula](#how-the-score-is-computed) is deterministic and the pass threshold is **80%**. Every test report also logs the specific edges where a skill falls short, so the numbers aren't marketing.
 
-Take oh-my-skills' four reference skills as an example. Their scores range from **99.0% to 99.49%** — none are perfect, and every test report names the specific drift it found:
+Take oh-my-skills' four reference skills as an example. Their scores range from **99.0% to 99.49%**. None is perfect, and every test report names the specific drift it found:
 
 | Skill | Score | What the report discloses |
 |---|---|---|
 | [oms-cocoindex](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-cocoindex/0.3.37/test-report-oms-cocoindex.md) | **99.0%** | 114/114 provenance entries; 55 public-API denominator from `__init__.py` `__all__`; 20/20 sampled signatures matched. Two denominators (barrel vs. full surface) both disclosed with rationale. |
 | [oms-cognee](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-cognee/1.0.0/test-report-oms-cognee.md) | **99.0%** | 34/34 exports documented; denominator is the `cognee/__init__.py` barrel (61 lines, 34 public re-exports) at pinned commit `3c048aa4` (v1.0.0). |
-| [oms-storybook-react-vite](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-storybook-react-vite/10.3.5/test-report-oms-storybook-react-vite.md) | **99.49%** | 215/216 documented — the missing 1 entry is logged openly as **GAP-004**, a canonical surface count drift from the stated denominator. |
+| [oms-storybook-react-vite](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-storybook-react-vite/10.3.5/test-report-oms-storybook-react-vite.md) | **99.49%** | 215/216 documented. **GAP-004** logs the one-count gap openly: all 215 exports in the provenance map are documented, and the 216th is a stale count in `metadata.json`, not a missing export. |
 | [oms-uitripled](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-uitripled/0.1.0/test-report-oms-uitripled.md) | **99.45%** | 34-entry denominator (not 11, not 25) with the full reconciliation reasoning in the report. |
 
-Perfection is suspicious. Visible fallibility is trustworthy. SKF writes down the edges it can't score cleanly — so you can read them and decide for yourself whether the remaining coverage is enough for your use case.
+Perfection is suspicious. Visible fallibility is trustworthy. SKF writes down the edges it can't score cleanly, so you can read them and decide for yourself whether the remaining coverage is enough for your use case.
 
-### GAP-004: a worked example of the 1% that fails
+### GAP-004: a worked example of a logged gap
 
-The [`oms-storybook-react-vite` test report](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-storybook-react-vite/10.3.5/test-report-oms-storybook-react-vite.md) scores **215/216** — not 216/216. The missing 1 entry is logged as **GAP-004**: a canonical export surface count (via the provenance map) diverges from the stated denominator in metadata.json. The report names the gap, shows the math, and leaves the drift visible for the next recompilation pass. Nothing was hidden.
+The [`oms-storybook-react-vite` test report](https://github.com/armelhbobdad/oh-my-skills/blob/main/forge-data/oms-storybook-react-vite/10.3.5/test-report-oms-storybook-react-vite.md) scores Export Coverage at **215/216**, not 216/216. The report logs the gap as **GAP-004**: once duplicate entries are folded, the provenance map holds 215 unique exports, all documented, while `metadata.json` still states a denominator of 216. The report names the gap, shows the math, and leaves the drift visible for the next recompilation pass. Nothing was hidden.
 
 That's the pattern SKF asks you to trust: when scoring can't reach 100%, the report says so, cites the line, and leaves a fingerprint for the next audit.
 
@@ -93,11 +106,11 @@ That's the pattern SKF asks you to trust: when scoring can't reach 100%, the rep
 
 ## How the Score Is Computed
 
-The Test Skill workflow (`@Ferris TS`) calculates the completeness score — a weighted measure of how thoroughly and accurately a skill documents its target. This score is the quality gate: pass and the skill is ready for export; fail and it routes to update-skill for remediation.
+The Test Skill workflow (`@Ferris TS`) calculates the completeness score: a weighted measure of how thoroughly and accurately a skill documents its target. This score is the quality gate. A pass means the skill is ready for export. A fail routes it to Update Skill for repair. When there is too little evidence to grade the skill, the result is INCONCLUSIVE, and SKF asks you to review it by hand.
 
 ### Categories and weights
 
-The score is the weighted sum of five categories:
+The score is a weighted sum of up to five categories. The weights below are the starting point. No single run scores all five: naive mode drops Coherence, and contextual mode drops Signature Accuracy and Type Coverage.
 
 | Category | Weight | What it measures |
 |---|---|---|
@@ -105,7 +118,7 @@ The score is the weighted sum of five categories:
 | **Signature Accuracy** | 22% | Documented function signatures match actual source signatures (parameter names, types, order, return types) |
 | **Type Coverage** | 14% | Types and interfaces referenced in exports are fully documented |
 | **Coherence** | 18% | Cross-references resolve, integration patterns are complete (contextual mode only) |
-| **External Validation** | 10% | Average of skill-check quality score (0–100) and tessl content score (0–100%) |
+| **External Validation** | 10% | skill-check quality score (0–100), averaged with the Tessl Review score (0–100) when you opt in to Tessl Review |
 
 ### Formula
 
@@ -123,18 +136,20 @@ coherence = (reference_validity × 0.6) + (integration_completeness × 0.4)
 
 If no integration patterns exist, coherence equals reference validity alone.
 
-**External validation** averages the two tools when both are available. When only one tool is available, that tool's score is used. When neither is available, the 10% weight is redistributed proportionally to the other active categories.
+**External validation** averages skill-check and Tessl Review when both produce a score. When only one does, that score is used. When neither does, the 10% weight is redistributed proportionally to the other active categories.
+
+**Tessl Review is optional.** It runs only after you set `tessl_review_workspace` in `_bmad/_memory/forger-sidecar/preferences.yaml` to the name of one of your Tessl workspaces (after `tessl login`, or with `TESSL_TOKEN` set). Create Skill and Test Skill then send the skill's `SKILL.md`, `references/`, `scripts/` and `assets/` to Tessl, where each review stays in that workspace's history, and each fresh review spends Tessl credits; each workflow run can spend one. A review takes Tessl about two minutes: the workflow checks on it in calls of under two minutes each, for up to about ten minutes, and records `timeout` if it has not finished by then. SKF never applies Tessl's suggestions: it lists them in the evidence and test reports, and you act on a description suggestion by editing the brief.
 
 ### Deterministic scoring
 
-The weight redistribution and score aggregation are computed by a deterministic Python script ([`compute-score.py`](https://github.com/armelhbobdad/bmad-module-skill-forge/blob/main/src/skf-test-skill/scripts/compute-score.py)). The LLM extracts category scores from the test report, constructs a JSON input, invokes the script, and uses its output for the final score. Same inputs always produce the same score. If the script is unavailable, the LLM falls back to manual calculation using the same formulas.
+The weight redistribution and score aggregation are computed by a deterministic Python script ([`compute-score.py`](https://github.com/armelhbobdad/bmad-module-skill-forge/blob/main/src/skf-test-skill/scripts/compute-score.py)). It reads, by path, the score files the earlier steps' scripts wrote in the run folder, such as the Signature Accuracy and Type Coverage that `score-signatures.py` scored and the export surface `load-coverage-inputs.py` built, so no score is copied out of the test report by hand. Same inputs always produce the same score. When a scoring script cannot run, Test Skill halts without a score rather than scoring by hand.
 
 ### Naive vs contextual mode
 
 Test Skill runs in one of two modes, detected automatically:
 
-- **Contextual mode** (stack skills) — all five categories scored with the default weights above.
-- **Naive mode** (individual skills) — Coherence is not scored. Its 18% weight is redistributed:
+- **Contextual mode** (stack skills): Coherence is scored, but Signature Accuracy and Type Coverage are not, because a stack skill's signatures and types belong to the libraries it combines. Their 36% is shared out in proportion, so Export Coverage weighs about 56%, Coherence about 28% and External Validation about 16%.
+- **Naive mode** (individual skills): Coherence is not scored. Its 18% weight is redistributed:
 
 | Category | Naive Weight |
 |---|---|
@@ -145,73 +160,88 @@ Test Skill runs in one of two modes, detected automatically:
 
 ### Tier adjustments
 
-Your forge tier determines which categories can be scored:
+Some categories cannot be scored for every skill. These conditions skip Signature Accuracy and Type Coverage:
 
-| Tier | Skipped Categories | Reason |
+| Condition | Skipped Categories | Reason |
 |---|---|---|
-| **Quick** | Signature Accuracy, Type Coverage | No AST parsing available |
-| **Docs-only** | Signature Accuracy, Type Coverage | No source code to compare against |
-| **Provenance-map** (State 2) | Signature Accuracy, Type Coverage | String comparison only, no semantic AST verification |
-| **Forge / Forge+ / Deep** | None | Full AST-backed scoring |
+| **Quick** tier | Signature Accuracy, Type Coverage | No AST parsing available |
+| **Docs-only** skill (every citation is a documentation link, `[EXT:...]`) | Signature Accuracy, Type Coverage | No source code to compare against |
+| **Source not on disk**: Test Skill works from the saved provenance map (`analysisConfidence` `provenance-map`), from the export list in the skill's `metadata.json` (`metadata-only`) or from a remote read of the source (`remote-only`) | Signature Accuracy, Type Coverage | Names can be matched, but signatures cannot be re-checked without the source |
+| **Stack skill**, any tier | Signature Accuracy, Type Coverage | The signatures and types belong to the libraries the stack combines |
+| **Reference-app skill**, any tier | Signature Accuracy, Type Coverage | It documents how parts are wired together, not a library's exports |
+| **Forge / Forge+ / Deep** individual skill with its source on disk | None | Full AST-backed scoring |
 
-When categories are skipped, their combined weight is redistributed proportionally to the remaining active categories. A Quick-tier skill and a Deep-tier skill both pass at the same 80% threshold — the score reflects what your tier can actually measure.
+When categories are skipped, their combined weight is redistributed proportionally to the remaining active categories. A Quick-tier skill and a Deep-tier skill both pass at the same 80% threshold. The score reflects what your tier can actually measure.
 
 ### Hard gate
 
-Before scoring, a hard gate scans all findings for **Critical** and **High** severity. If any are present, the pipeline blocks immediately — no score is computed and the skill cannot pass regardless of coverage percentage. Medium, Low, and Info findings pass through to scoring.
+Before scoring, a hard gate reads the run's gap ledger, `test-findings-{run_id}.json` in the skill's version folder under `forge-data/`, where the coverage, coherence and external validation steps record every gap they find with its severity. If any gap is **Critical** or **High**, Test Skill stops there: no score is computed, and the result is FAIL whatever the coverage. A blocked run still completes its record. It writes the Gap Report, with the blocking gaps first, and a FAIL result record whose `halt_reason` is `hard-gate-blocked`, runs the `on_complete` hook and the health check, then exits with code 2. SKF recommends `@Ferris US <name> --from-test-report` to repair the blocking gaps. Medium, Low and Info gaps pass through to scoring: a missing export, for example, is Medium, so it lowers Export Coverage and the threshold decides the verdict.
 
 ### Pass/fail
 
 ```
-threshold = pipeline_default OR custom_threshold OR 80% (default)
+threshold = --threshold=<N> if you passed it, else the pipeline default, else default_threshold (80 unless your team changed it)
 
 score >= threshold  →  PASS  →  Recommend export-skill
 score <  threshold  →  FAIL  →  Recommend update-skill
+too little evidence  →  INCONCLUSIVE  →  Review by hand
 ```
 
-The default threshold is **80%**. Pipeline aliases declare their own defaults: `forge-auto` targets **90%**, `forge` and `forge-quick` target **80%**. You can override any default with a custom threshold (e.g., `TS[min:85]`).
+INCONCLUSIVE means fewer than two categories were left to score, or a Quick-tier skill had only Export Coverage to go on. It is neither a pass nor a fail: SKF does not export the skill or retry, and it leaves the decision to you. To get a verdict, add evidence and test again: install skill-check so `npx skill-check` runs without a download, or install ast-grep and re-run `@Ferris SF` to move up a tier.
+
+The default threshold is **80%**. Pipeline aliases declare their own defaults: `forge-auto` targets **90%**, `forge` and `forge-quick` target **80%**. To set the threshold for one run, pass `--threshold=85` to Test Skill, or write `TS[min:85]` inside a pipeline sequence. To change the default for your whole team, set `default_threshold` in `_bmad/custom/skf-test-skill.toml`. That file is read only in a project where the BMAD Method installer has added its customization script. With SKF installed alone, pass `--threshold` on each run instead (see [Customizing a Workflow](/docs/workflows.md#customizing-a-workflow)).
 
-When a skill scores between 80% and its target threshold (e.g., 82% against a 90% target), the soft gate falls back to the 80% floor — the skill passes, but an evidence report is written to `forge-data/{skill}/{version}/evidence-report-fallback.md` documenting the quality compromise.
+Test Skill does not run without its tools: a missing `uv` or `python3` halts the run before it tests anything (`runtime-missing`). Two caps turn a pass into a fail. The first fires whenever the report's `toolingStatus` is not `ok`, for example when the frontmatter validator timed out (`frontmatter-validator-timeout`). A docs-only stack skill that neither skill-check nor Tessl Review scored is capped the same way, because too little is left to grade it. A docs-only individual skill in that state has only Export Coverage left, so it ends INCONCLUSIVE instead. A capped run stays FAIL at any threshold: the fallback below never accepts it.
+
+When no cap fired and a skill scores between 80% and its target threshold (for example, 82% against a 90% target), it still passes at the 80% floor, and Test Skill writes an evidence report to `forge-data/{skill}/{version}/evidence-report-fallback.md` that records the shortfall.
+
+A run that found drift and went on under `--allow-workspace-drift` ends **pass-with-drift**, not PASS, when it reaches the threshold. It read a different commit than the skill cites, so test again on the pinned commit before you export.
 
 ### Gap severities
 
-When the score is calculated, each finding is classified by severity to guide remediation:
+Each step rates every gap it records against the Gap Severity table in Test Skill's [`scoring-rules.md`](https://github.com/armelhbobdad/bmad-module-skill-forge/blob/main/src/skf-test-skill/references/scoring-rules.md), so a gap gets the same severity on every run. Critical and High gaps found before scoring block the run at the [hard gate](#hard-gate). The others lower the score or guide remediation:
 
 | Severity | Examples |
 |---|---|
-| **Critical** | Missing exported function/class documentation |
-| **High** | Signature mismatch between source and `SKILL.md` |
-| **Medium** | Missing type/interface documentation; scripts/assets directory inconsistencies |
-| **Low** | Missing optional metadata or examples; description optimization opportunities |
-| **Info** | Style suggestions; discovery testing recommendations |
+| **Critical** | A wrong signature, or a fabricated one (a documented export that the source does not define at the line the skill cites); a broken reference in a stack skill (a file, skill, type, or `scripts/` or `assets/` file that does not exist) |
+| **High** | An inaccurate reference in a stack skill (the target exists but does not match), or a reference whose real path leads outside the skill and its sources; a split-body mismatch (`SKILL.md` and a `references/` file document one export differently); `metadata.json` stats that count every export as documented while some are missing from the skill; in an individual skill, a missing required section, an unbalanced code fence, or an example that awaits a sync function or skips the `await` of an async one |
+| **Medium** | A missing export or type (a missing export lowers Export Coverage); stale documentation (a documented export outside the coverage surface that the run did not match to one declaration in the source, or that a brief scopes out); an incomplete integration pattern in a stack skill; in an individual skill, a code fence with no language tag, a function no example names, or a table row whose column count differs from its header |
+| **Low** | Gaps in metadata, the description or provenance lines, such as missing optional metadata or a provenance line that is not the definition line |
+| **Info** | Observations such as style suggestions or a documented extra (a documented name outside the coverage surface that the source still declares); discovery testing not performed |
+
+Only some runs can find a documented extra: the source must be Python or TypeScript/JavaScript, and the run needs the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden`, a provenance map and a coverage surface that a complete extraction built. On any other run, every documented export outside the coverage surface is stale documentation.
+
+A declaration the run finds only in an `example`, `demo`, `demos`, `sample`, `samples`, `scripts`, `playground`, `bench`, `benchmark` or `migrations` folder right below a package's root, or in an `e2e` or `__fixtures__` folder anywhere, does not count, so a name declared only in `scripts/foo.py` stays stale documentation unless the provenance map cites a file in that folder. Nor does a module named after the name count (such as `client.py` for a documented `client()`) when the skill documents the name as a function, hook, class, interface, enum or type, unless the provenance map records the name as a module or package, or the module is a TypeScript or JavaScript file that is itself the export (`export default`, `module.exports =`).
+
+Discovery testing runs after the hard gate, so a discovery gap (Info, Medium or High) is counted in the Gap Report and blocks nothing.
 
 ### Score report output
 
-The test report includes a score breakdown table showing each category's raw score, weight, and weighted contribution:
+The test report includes a score breakdown table showing each category's raw score, weight, and weighted contribution. This example is an individual skill (naive mode) at Deep tier:
 
 | Category | Score | Weight | Weighted |
 |---|---|---|---|
-| Export Coverage | 92% | 36% | 33.1% |
-| Signature Accuracy | 85% | 22% | 18.7% |
-| Type Coverage | 100% | 14% | 14.0% |
-| Coherence | 80% | 18% | 14.4% |
+| Export Coverage | 92% | 45% | 41.4% |
+| Signature Accuracy | 85% | 25% | 21.25% |
+| Type Coverage | 100% | 20% | 20.0% |
+| Coherence | not scored (naive mode) | 0% | 0% |
 | External Validation | 78% | 10% | 7.8% |
-| **Total** | | **100%** | **88.0%** |
+| **Total** | | **100%** | **90.45%** |
 
-The report also records `analysisConfidence` (full, provenance-map, metadata-only, remote-only, or docs-only) and includes a degradation notice when source access was limited.
+The report also records `analysisConfidence`, how Test Skill read the source (`full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`), and includes a degradation notice when source access was limited. It records the health of the tools the run used apart from that, as `toolingStatus`: `ok`, or `frontmatter-validator-timeout`.
 
 ---
 
 ## Build-time drift detection (for docs themselves)
 
-The SKF docs you're reading right now are themselves verified against oh-my-skills. A `docs/_data/pinned.yaml` anchor file records the exact version, commit SHA, and confidence tier of every reference skill. A Node validator (`tools/validate-docs-drift.js`) runs as part of `npm run quality` and:
+The library version numbers in the SKF docs you're reading right now are checked against oh-my-skills. A `docs/_data/pinned.yaml` anchor file records the exact version, commit SHA, and confidence tier of every reference skill. A Node validator (`tools/validate-docs-drift.js`) runs as part of `npm run quality` and:
 
-1. **Confirms canonical truth** — every anchor in `pinned.yaml` is cross-checked against the actual `metadata.json` in oh-my-skills. Version, commit, tier, and authority must match.
-2. **Scans docs for stale prose** — every `.md` file is grepped for `<library> v?<x.y.z>` patterns and any version that disagrees with `pinned.yaml` is flagged with file + line number.
+1. **Confirms canonical truth.** Every anchor in `pinned.yaml` is cross-checked against the actual `metadata.json` in oh-my-skills: version, commit, tier, and authority must match. The `skf_version` in `pinned.yaml` must also match the version in SKF's own `package.json`.
+2. **Scans docs for stale prose.** Every published `.md` page under `docs/` is searched for `<library> v?<x.y.z>` patterns, and any version that disagrees with `pinned.yaml` is flagged with its file and line number.
 
-If the validator flags drift, the CI fails before the docs get merged. It's the same "nothing is made up" contract SKF applies to skills, applied to the docs that describe SKF. When the anchor file is updated to reflect a new oh-my-skills release, the prose must update too — otherwise `npm run docs:validate-drift` blocks the merge.
+CI does not run this check. It needs a local clone of oh-my-skills, so run it yourself before you merge any change to the docs or to `pinned.yaml`. It's the same "nothing is made up" contract SKF applies to skills, applied to the docs that describe SKF. When the anchor file is updated to reflect a new oh-my-skills release, the prose must update too, or `npm run docs:validate-drift` fails.
 
-You can run it yourself from the SKF repo root:
+Run it from the SKF repo root. By default it reads oh-my-skills from `../oh-my-skills`, a clone next to your SKF checkout, and it fails if no clone is there:
 
 ```bash
 npm run docs:validate-drift
@@ -226,10 +256,10 @@ OMS=/path/to/your/oh-my-skills npm run docs:validate-drift
 Clean output looks like this:
 
 ```
-OK: 4 skills checked against /home/you/oh-my-skills, no drift.
+OK: skf_version x.y.z matches package.json; 4 skills checked against /home/you/oh-my-skills, no drift.
 ```
 
-Dirty output cites exact file:line locations so the fix is mechanical.
+Dirty output lists each finding. A stale version in the docs is cited by file and line, and an anchor mismatch names the skill and the value its `metadata.json` holds, so the fix is mechanical.
 
 ---
 

package/docs/why-skf.md

@@ -1,17 +1,17 @@
 ---
 title: Why Skill Forge?
-description: The strategic case for SKF — the problem it solves, how it compares to alternatives, who it's for, and who it isn't.
+description: The strategic case for SKF, covering the problem it solves, how it compares to alternatives, who it's for, and who it isn't.
 ---
 
-Skill Forge is the only AI-skills toolchain where every claim your agent reads points back to a real upstream location — a `file:line` at a pinned commit when source is available, or a documentation URL when it isn't. Not "sourced from training data." Not "retrieved from context." **Cited.** You can open the upstream repo (or the doc page) and see the function exists — in under a minute. That's the wedge. This page explains why it matters, how SKF compares to alternatives, and who it's for.
+Skill Forge compiles AI skills where each claim your agent reads points back to a real upstream location: a `file:line` at a pinned commit when source is available, or a documentation URL when it isn't. Not "sourced from training data." Not "retrieved from context." **Cited.** You can open the upstream repo (or the doc page) and see the function exists in under a minute. That is the core idea. This page explains why it matters, how SKF compares to alternatives, and who it's for.
 
 ---
 
 ## The problem you're hiring SKF to solve
 
-Your AI agents read your codebase through the lens of whatever happened to be in their training data. When that training data is wrong, stale, or incomplete, your agent invents — function names that don't exist, parameter types that don't match, config options removed two versions ago. You catch some of it in review. You ship some of it by accident. Every sprint, your team spends hours untangling code that only compiles in the AI's imagination.
+Your AI agents read your codebase through the lens of whatever happened to be in their training data. When that training data is wrong, stale, or incomplete, your agent invents: function names that don't exist, parameter types that don't match, config options removed two versions ago. You catch some of it in review. You ship some of it by accident. Every sprint, your team spends hours untangling code that only compiles in the AI's imagination.
 
-SKF treats this as a citation problem, not a model problem. If a skill claims `cognee.search()` takes `query_text` as its first parameter, SKF points to `cognee/api/v1/search/search.py:L26` at commit `3c048aa4` in the upstream repo. That's the whole pitch: **nothing is made up, and everything is falsifiable in 60 seconds.**
+SKF treats this as a citation problem, not a model problem. If a skill claims `cognee.search()` takes `query_text` as its first parameter, SKF points to `cognee/api/v1/search/search.py:L27` at commit `3c048aa4` in the upstream repo. That's the whole pitch: **nothing is made up, and everything is falsifiable in 60 seconds.**
 
 ---
 
@@ -21,12 +21,12 @@ SKF treats this as a citation problem, not a model problem. If a skill claims `c
 
 | Approach | What it does well | Where it falls short |
 |----------|-------------------|----------------------|
-| Skill scaffolding (`npx skills init`) | Generates a spec-compliant skill file | The file is empty — you still have to write every instruction by hand |
+| Skill scaffolding (`npx skills init`) | Creates a SKILL.md template in the standard skill format | The template holds only placeholder text, so you still write every instruction by hand |
 | LLM summarization | Understands context and intent | Generates plausible-sounding content that may not match the actual API |
-| RAG / context stuffing | Retrieves relevant code snippets | Returns fragments without synthesis — no coherent skill output |
+| RAG (retrieval) or pasting code into the prompt | Finds relevant code snippets | Hands over loose fragments, not a finished skill your agent can follow |
 | Manual authoring | High initial accuracy | Drifts as the source code changes, doesn't scale across dependencies |
 | IDE built-in context (Copilot, Cursor) | Convenient, zero setup | Uses generic training data, not your project's specific integration patterns |
-| **Skill Forge** | **Every instruction cites upstream — `file:line@SHA` for source skills, doc URL for docs-only skills. Falsifiable in 60 seconds.** | **Coverage depends on which tools you've installed (Quick / Forge / Forge+ / Deep tiers).** |
+| **Skill Forge** | **Every instruction cites upstream: a source file and line at a pinned commit for source skills, a doc URL for docs-only skills. Falsifiable in 60 seconds.** | **Coverage depends on which tools you've installed, which set your capability tier (Quick / Forge / Forge+ / Deep). Quick Skill (QS) makes a best-effort skill with no provenance map.** |
 
 </div>
 
@@ -36,15 +36,17 @@ SKF treats this as a citation problem, not a model problem. If a skill claims `c
 
 Pick any symbol in any SKF-compiled skill. Three clicks:
 
-1. Open the skill's `metadata.json` — it names the upstream repo and the exact commit SHA.
-2. Open the skill's `provenance-map.json` — find your symbol; it lists the file and line.
+1. Open the skill's `metadata.json`: it names the upstream repo (`source_repo`) and the exact commit (`source_commit`).
+2. Open `provenance-map.json` in the skill's forge folder (`forge-data/{skill}/{version}/`, next to its test report) and find your symbol: it lists the file and line.
 3. Visit the upstream repo at that commit and that line. The signature in the skill should match.
 
-For docs-only skills, the audit shape is the same — `provenance-map.json` still lists every symbol — but entries cite `[EXT:{url}]` instead of `file:line@SHA`, and step 3 becomes "open the doc URL and confirm the signature matches."
+For docs-only skills, the audit shape is the same, with two differences: `metadata.json` records no commit, and the entries in `provenance-map.json` cite `[EXT:{url}]` instead of a file and line. Step 3 becomes "open the doc URL and confirm the signature matches."
 
-If it doesn't, **that's a bug.** [Open an issue](https://github.com/armelhbobdad/bmad-module-skill-forge/issues/new/choose) and SKF republishes the skill with a new commit SHA and a new provenance map. No other AI-skills tool treats disagreement between claim and source as a defect. SKF does.
+If it doesn't, **that's a bug.** For a skill from oh-my-skills, [open an issue there](https://github.com/armelhbobdad/oh-my-skills/issues) with the file and line. The fix ships publicly with a new commit SHA and a new provenance map. If a skill you compiled with SKF gets it wrong, [open a bug report](https://github.com/armelhbobdad/bmad-module-skill-forge/issues/new/choose) on SKF. SKF treats any disagreement between claim and source as a defect.
 
-See the [Verifying a Skill](/docs/verifying-a-skill.md) page for the full three-step audit on real skills, the test reports that log *exactly* where coverage falls short, and the scoring formula behind the 80% pass threshold.
+Quick Skill (QS) is the exception. It makes a fast, best-effort skill with a `metadata.json` but no provenance map, so run Create Skill (CS) from a brief when you need this audit trail.
+
+See the [Verifying a Skill](/docs/verifying-a-skill.md) page for the full three-step audit on real skills, the test reports that log *exactly* where coverage falls short, and the scoring formula behind the default 80% pass threshold.
 
 ---
 
@@ -57,31 +59,31 @@ Your agent just hallucinated a method that doesn't exist, again. You want this t
 
 ### The BMAD user
 
-You already use BMAD Method, BMM phases, TEA, or BMB, and you want to know where SKF fits.
+You already use the BMAD Method, with its BMM phases (analysis, planning, solutioning, implementation) or its TEA (Test Architect) and BMB (BMAD Builder) modules, and you want to know where SKF fits.
 → Read [BMAD Synergy](/docs/bmad-synergy.md) for the phase-by-phase integration playbook.
 
 ### The skeptic
 
 "AI docs for AI" sounds like the problem pretending to be the solution. You want receipts before you install anything.
-→ Start with [Verifying a Skill](/docs/verifying-a-skill.md) — the three-step audit on real skills, including the 1% that fails.
+→ Start with [Verifying a Skill](/docs/verifying-a-skill.md): the three-step audit on real skills, including the gaps their own test reports log.
 
 ### The OSS maintainer
 
-You want to ship verified skills alongside your library releases — `npx skills publish`-ready, drift-detectable, version-pinned.
+You want to ship verified skills alongside your library releases: ready for `npx skills publish`, drift-detectable, and version-pinned.
 → See [Examples → OSS Maintainer Publishing Official Skills](/docs/examples.md#scenario-h-oss-maintainer-publishing-official-skills).
 
 ### The team lead evaluating adoption
 
-You're considering running SKF across a brownfield platform. You need to know about rollback safety, `[MANUAL]` section preservation, and the health-check feedback loop before committing.
-→ Start with [Architecture](/docs/architecture.md), then [Workflows → Workflow Health Check](/docs/workflows.md#terminal-step-health-check).
+You're considering running SKF across a brownfield platform (an existing, already-running codebase). Before you adopt it, you need to know that an update writes a new version beside the previous one instead of overwriting it, that hand-written `[MANUAL]` sections survive regeneration, and how the health check reports workflow problems.
+→ Start with [Architecture](/docs/architecture.md), then [Workflows → Update Skill](/docs/workflows.md#update-skill-us) and [Workflows → Terminal Step: Health Check](/docs/workflows.md#terminal-step-health-check).
 
 ---
 
 ## Not for you if…
 
 - You want docs that hand-hold through every happy path with screenshots and emojis. SKF is a citation machine, not a tutorial series.
-- You don't have Node.js ≥ 22 and Python ≥ 3.10 installed. SKF is a Node/Python toolchain at its core.
-- You have neither source code nor published documentation for the target. SKF compiles from one or both — a source repo (citations as `file:line@SHA`) or doc URLs (citations as `[EXT:{url}]`). A vague description with no upstream artifact to cite isn't enough.
+- You don't have Node.js ≥ 22, Python ≥ 3.11, and `uv` installed. SKF is a Node/Python toolchain at its core, and setup stops without `uv`.
+- You have neither source code nor published documentation for the target. SKF compiles from one or both: a source repo (each citation names a file and line at a pinned commit) or doc URLs (citations as `[EXT:{url}]`). A vague description with no upstream artifact to cite isn't enough.
 
 Everything else is downstream of one question: *are the instructions your AI reads provably true?* If yes, SKF isn't adding value. If you can't be sure, SKF is the tool.
 
@@ -89,6 +91,6 @@ Everything else is downstream of one question: *are the instructions your AI rea
 
 ## Next
 
-- **[Install SKF](/docs/getting-started.md#install)** — Node ≥ 22, Python ≥ 3.10, `uv`, one `npx` command
-- **[Audit a skill in 60 seconds](/docs/verifying-a-skill.md)** — see the receipts before you install
-- **[Browse real skills](https://github.com/armelhbobdad/oh-my-skills)** — four Deep-tier skills, all shipping their audit trails
+- **[Install SKF](/docs/getting-started.md#install)**: Node ≥ 22, Python ≥ 3.11, `uv`, and one `npx` command
+- **[Audit a skill in 60 seconds](/docs/verifying-a-skill.md)**: see the receipts before you install
+- **[Browse real skills](https://github.com/armelhbobdad/oh-my-skills)**: four Deep-tier skills, all shipping their audit trails