bmad-module-skill-forge 4.0.0 → 5.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +3 -3
- package/docs/_data/pinned.yaml +19 -13
- package/docs/architecture.md +2 -2
- package/docs/concepts.md +4 -4
- package/docs/examples.md +32 -25
- package/docs/getting-started.md +3 -3
- package/docs/how-it-works.md +2 -2
- package/docs/index.md +1 -1
- package/docs/skill-model.md +41 -39
- package/docs/verifying-a-skill.md +41 -20
- package/docs/why-skf.md +1 -1
- package/docs/workflows.md +2 -2
- package/package.json +1 -1
- package/src/shared/scripts/schemas/skf-update-result-envelope.v1.json +11 -3
- package/src/shared/scripts/schemas/skill-brief.v1.json +1 -1
- package/src/shared/scripts/skf-build-change-manifest.py +881 -168
- package/src/shared/scripts/skf-count-tokens.py +54 -10
- package/src/shared/scripts/skf-detect-workspaces.py +67 -0
- package/src/shared/scripts/skf-emit-result-envelope.py +6 -5
- package/src/shared/scripts/skf-extract-public-api.py +439 -107
- package/src/shared/scripts/skf-extraction-inventory.py +10 -6
- package/src/shared/scripts/skf-extraction-snapshot.py +6 -6
- package/src/shared/scripts/skf-parse-gaps.py +135 -26
- package/src/shared/scripts/skf-render-metadata-stats.py +1 -1
- package/src/shared/scripts/skf-resolve-customization.py +197 -0
- package/src/shared/scripts/skf-shard-body.py +239 -41
- package/src/shared/scripts/skf-validate-output.py +12 -3
- package/src/shared/scripts/skf-verify-provenance-completeness.py +470 -38
- package/src/shared/tool-requirements.yaml +8 -2
- package/src/skf-analyze-source/SKILL.md +2 -2
- package/src/skf-analyze-source/assets/skill-brief-schema.md +1 -1
- package/src/skf-audit-skill/SKILL.md +2 -2
- package/src/skf-audit-skill/references/re-index.md +2 -2
- package/src/skf-brief-skill/SKILL.md +2 -2
- package/src/skf-brief-skill/assets/skill-brief-schema.md +2 -2
- package/src/skf-brief-skill/references/scope-definition.md +4 -4
- package/src/skf-campaign/SKILL.md +2 -2
- package/src/skf-campaign/references/step-05-skill-loop.md +6 -1
- package/src/skf-create-skill/SKILL.md +2 -2
- package/src/skf-create-skill/assets/skill-sections.md +1 -1
- package/src/skf-create-skill/references/extract.md +2 -1
- package/src/skf-create-skill/references/extraction-patterns.md +3 -3
- package/src/skf-create-skill/references/report.md +6 -1
- package/src/skf-create-stack-skill/SKILL.md +2 -2
- package/src/skf-create-stack-skill/references/validate.md +1 -1
- package/src/skf-drop-skill/SKILL.md +2 -2
- package/src/skf-export-skill/SKILL.md +2 -2
- package/src/skf-export-skill/assets/snippet-format.md +5 -5
- package/src/skf-export-skill/references/generate-snippet.md +22 -15
- package/src/skf-export-skill/references/load-skill.md +1 -1
- package/src/skf-export-skill/references/token-report.md +29 -9
- package/src/skf-forger/SKILL.md +2 -2
- package/src/skf-quick-skill/SKILL.md +2 -2
- package/src/skf-refine-architecture/SKILL.md +6 -8
- package/src/skf-rename-skill/SKILL.md +2 -2
- package/src/skf-setup/SKILL.md +2 -2
- package/src/skf-test-skill/SKILL.md +2 -2
- package/src/skf-test-skill/assets/output-section-formats.md +6 -4
- package/src/skf-test-skill/references/coherence-check.md +4 -3
- package/src/skf-test-skill/references/coverage-check-branches.md +61 -0
- package/src/skf-test-skill/references/coverage-check-tiers.md +1 -1
- package/src/skf-test-skill/references/coverage-check.md +45 -69
- package/src/skf-test-skill/references/init.md +2 -1
- package/src/skf-test-skill/references/migration-section-rules.md +26 -14
- package/src/skf-test-skill/references/report.md +2 -2
- package/src/skf-test-skill/references/scoring-rules.md +5 -4
- package/src/skf-test-skill/references/source-access-protocol.md +6 -6
- package/src/skf-test-skill/scripts/gap-ledger.py +378 -163
- package/src/skf-test-skill/scripts/load-coverage-inputs.py +171 -70
- package/src/skf-test-skill/scripts/locate-export-segments.py +101 -20
- package/src/skf-test-skill/scripts/reconcile-coverage.py +192 -39
- package/src/skf-test-skill/scripts/score-signatures.py +329 -54
- package/src/skf-test-skill/scripts/stage-helper-payload.py +64 -13
- package/src/skf-test-skill/scripts/validate-inventory.py +108 -9
- package/src/skf-test-skill/templates/test-report-template.md +1 -0
- package/src/skf-update-skill/SKILL.md +2 -2
- package/src/skf-update-skill/references/detect-changes.md +1 -1
- package/src/skf-update-skill/references/gap-driven.md +36 -21
- package/src/skf-update-skill/references/merge.md +15 -10
- package/src/skf-update-skill/references/re-extract.md +3 -1
- package/src/skf-update-skill/references/report.md +5 -4
- package/src/skf-update-skill/references/write.md +39 -18
- package/src/skf-verify-stack/SKILL.md +2 -2
- package/tools/cli/lib/tool-check.js +158 -2
|
@@ -4,6 +4,7 @@ outputFile: '{report_file}'
|
|
|
4
4
|
scoringRulesFile: 'references/scoring-rules.md'
|
|
5
5
|
sourceAccessProtocol: 'references/source-access-protocol.md'
|
|
6
6
|
coverageTiersFile: 'references/coverage-check-tiers.md'
|
|
7
|
+
coverageBranchesFile: 'references/coverage-check-branches.md'
|
|
7
8
|
validateInventoryScript: 'scripts/validate-inventory.py'
|
|
8
9
|
coverageInputsScript: 'scripts/load-coverage-inputs.py'
|
|
9
10
|
scoreSignaturesScript: 'scripts/score-signatures.py'
|
|
@@ -33,7 +34,7 @@ detectWorkspacesProbeOrder:
|
|
|
33
34
|
|
|
34
35
|
## STEP GOAL:
|
|
35
36
|
|
|
36
|
-
Compare
|
|
37
|
+
Compare what SKILL.md documents (exports, functions, classes, types, interfaces) with the source API surface: missing documentation, undocumented exports, signature mismatches, at the forge tier's depth.
|
|
37
38
|
|
|
38
39
|
**Halt envelope.** Every HALT in this step names its `halt_reason` and phase and carries exit code 1. It releases the run lock first, whatever the release prints: from `{project-root}`, run `uv run {runLockHelper} release --lock "{forge_version}/.test-skill.lock" --owner "{run_owner}"` (SKILL.md Workflow Rules). In headless mode it then writes `{"phase": "<phase>", "reason": "<the halt message>", "halt_reason": "<halt_reason>", "skill_name": "{skill_name}", "report_path": "{report_file}"}`, adding `"path"` when the halt names one, to `{run_dir}/halt.json` and runs:
|
|
39
40
|
|
|
@@ -43,11 +44,11 @@ uv run {emitEnvelopeHelper} emit-halt --workflow skf-test-skill --run-dir "{run_
|
|
|
43
44
|
|
|
44
45
|
Display the line it prints, as the run's last line, then stop. If the emitter exits non-zero or prints no line, display the halt message alone. An interactive HALT displays its message and emits nothing.
|
|
45
46
|
|
|
46
|
-
**Run files.** Every script below writes its result into the run folder `{run_dir}` that init.md §6b created,
|
|
47
|
+
**Run files.** Every script below writes its result into the run folder `{run_dir}` that init.md §6b created, where the next one reads it by path. A subagent's response is saved there exactly as it came back, fence and all (the subagent writes it to the path it is given, or the parent saves it, with the Write tool), and passed by path, so a quote or an apostrophe in it never meets a shell. Every count, name list and score a script computes is read from the file it wrote, never re-derived by hand or copied between commands. Each command runs from `{project-root}`; a `{...Script}` path resolves relative to the skill root.
|
|
47
48
|
|
|
48
49
|
### 0. Check for Docs-Only Mode
|
|
49
50
|
|
|
50
|
-
A skill is docs-only when every citation it writes is `[EXT:...]`,
|
|
51
|
+
A skill is docs-only when every citation it writes is `[EXT:...]`, none local (`[AST:...]`, `[SRC:...]`, or a stack's `[from skill: ...]`). The citation census decides it:
|
|
51
52
|
|
|
52
53
|
```bash
|
|
53
54
|
uv run {coverageInputsScript} census --skill-dir "{resolved_skill_package}" --output "{run_dir}/census.json"
|
|
@@ -63,21 +64,21 @@ A source-based skill continues with §0b.
|
|
|
63
64
|
|
|
64
65
|
### 0b. Load Source Access Protocol
|
|
65
66
|
|
|
66
|
-
Load `{sourceAccessProtocol}` and follow both sections: **Source API Surface Definition** (the
|
|
67
|
+
Load `{sourceAccessProtocol}` and follow both sections: **Source API Surface Definition** (the denominator clause §2b picks) and **Source Access Resolution** (the 5-state waterfall that decides how the source is read and sets `analysis_confidence`; State 5 makes the run docs-only).
|
|
67
68
|
|
|
68
|
-
**Workspace layout.** The Source API Surface Definition's monorepo tests read `{source_is_monorepo}
|
|
69
|
+
**Workspace layout.** The Source API Surface Definition's monorepo tests read `{source_is_monorepo}`. At State 1 (local source), resolve `{detectWorkspacesHelper}` ← first existing path in `{detectWorkspacesProbeOrder}` and run (`{stageHelperPayloadScript}` pipes it the tree and root manifests from disk; leave `--brief` out with no brief):
|
|
69
70
|
|
|
70
71
|
```bash
|
|
71
|
-
uv run {stageHelperPayloadScript} detect-workspaces --source-root "{source_path}" | uv run {detectWorkspacesHelper}
|
|
72
|
+
uv run {stageHelperPayloadScript} detect-workspaces --source-root "{source_path}" --brief "{forge_data_folder}/{skill_name}/skill-brief.yaml" | uv run {detectWorkspacesHelper}
|
|
72
73
|
```
|
|
73
74
|
|
|
74
|
-
Bind `{source_is_monorepo}` ← `is_monorepo` and `{source_workspace_kind}` ← `manifest_kind`. At States 2 to 5, or when the helper does not resolve or the command exits non-zero, the layout is unknown: bind `{source_is_monorepo}` ← false. §5 records the layout.
|
|
75
|
+
A workspace whose members `scope.exclude` drops is no workspace. Bind `{source_is_monorepo}` ← `is_monorepo` and `{source_workspace_kind}` ← `manifest_kind`. At States 2 to 5, or when the helper does not resolve or the command exits non-zero, the layout is unknown: bind `{source_is_monorepo}` ← false. §5 records the layout.
|
|
75
76
|
|
|
76
|
-
**Clause flags.** Every `surface` command in §2 takes `--monorepo` when `{source_is_monorepo}` is true, and the flags the Source API Surface Definition leaves to you.
|
|
77
|
+
**Clause flags.** Every `surface` command, in §2 or in a branch file it loads, takes `--monorepo` when `{source_is_monorepo}` is true, and the flags the Source API Surface Definition leaves to you.
|
|
77
78
|
|
|
78
79
|
### 1. Extract Documented Exports from SKILL.md
|
|
79
80
|
|
|
80
|
-
Delegate the reading of the skill under test to a subagent, given the path to SKILL.md and to its `references/` directory when one exists
|
|
81
|
+
Delegate the reading of the skill under test to a subagent, given `{run_dir}` and the path to SKILL.md and to its `references/` directory when one exists: it reads SKILL.md, and every `references/*.md` file too when SKILL.md's `## Full` headings are absent or stubs. For each export both the SKILL.md body and a reference file document, it compares the two: parameters (name, type, order, optionality), the return type, and descriptions that contradict (brief against detailed is fine, as is a form the skill labels partial that leaves out only optional members). It writes only this JSON, with no prose and no commentary, to `{run_dir}/inventory-response.txt` with the Write tool, never a shell, and returns that path, or returns the JSON itself when it cannot write files:
|
|
81
82
|
|
|
82
83
|
```json
|
|
83
84
|
{
|
|
@@ -85,7 +86,7 @@ Delegate the reading of the skill under test to a subagent, given the path to SK
|
|
|
85
86
|
{"name": "functionName", "kind": "function", "params": "...", "return_type": "...", "description": "..."},
|
|
86
87
|
{"name": "ClassName", "kind": "class", "methods": ["..."], "properties": ["..."], "description": "..."},
|
|
87
88
|
{"name": "TypeName", "kind": "type", "fields": ["..."], "description": "..."},
|
|
88
|
-
{"name": "useHook", "kind": "hook", "usage_signature": "...", "description": "..."}
|
|
89
|
+
{"name": "useHook", "kind": "hook", "usage_signature": "...", "description": "...", "signature_form": "partial"}
|
|
89
90
|
],
|
|
90
91
|
"references": ["references/api-reference.md"],
|
|
91
92
|
"cross_check_mismatches": [
|
|
@@ -94,29 +95,28 @@ Delegate the reading of the skill under test to a subagent, given the path to SK
|
|
|
94
95
|
}
|
|
95
96
|
```
|
|
96
97
|
|
|
97
|
-
Tell the subagent the contract the validator checks: each `kind` is one of `function`, `class`, `type`, `constant`, `hook`, `interface`, `method`, `struct`, `enum`, `trait`, `macro` or `adapter`; every entry carries the `description` the skill gives it (an empty string when it gives none); a function or method carries `params` (`""` or `[]` when it takes none) and `return_type` as the skill documents them, and leaves a key out when the skill documents nothing for it.
|
|
98
|
+
Tell the subagent the contract the validator checks: each `kind` is one of `function`, `class`, `type`, `constant`, `hook`, `interface`, `method`, `struct`, `enum`, `trait`, `macro` or `adapter`; every entry carries the `description` the skill gives it (an empty string when it gives none); a function or method carries `params` (`""` or `[]` when it takes none) and `return_type` as the skill documents them, and leaves a key out when the skill documents nothing for it. An entry whose signature the skill labels partial (simplified, key parameters, a `...` standing in for omitted members, not rest or variadic syntax) carries `signature_form: "partial"`, any other `"full"` or no key: copy the label, never judge the form.
|
|
98
99
|
|
|
99
|
-
|
|
100
|
+
When the subagent returned the JSON rather than the path, or the file is absent or empty afterwards, save its response, as it came back, to `{run_dir}/inventory-response.txt` with the Write tool. The documented inventory is the one §1a validates: do not load SKILL.md or the reference files into the parent context. Without subagents, the parent builds the same JSON from the same files and saves it to that file; §1a's checks still run on it.
|
|
100
101
|
|
|
101
102
|
#### 1a. Parent-Side Schema Validation + Spot-Check
|
|
102
103
|
|
|
103
|
-
Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped) and spot-checks sampled names against SKILL.md and the reference files
|
|
104
|
+
Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped) and spot-checks sampled names against SKILL.md and the subagent's listed reference files. The look-up is the script's, never a grep: an export named `$state` or `a.b` is neither a shell variable nor a regex.
|
|
104
105
|
|
|
105
106
|
```bash
|
|
106
107
|
uv run {validateInventoryScript} --input "{run_dir}/inventory-response.txt" --skill-package "{resolved_skill_package}" --output "{run_dir}/inventory.json"
|
|
107
108
|
```
|
|
108
109
|
|
|
109
110
|
- `valid` true (exit 0): every later command reads `{run_dir}/inventory.json`.
|
|
110
|
-
- `valid` false (exit 2)
|
|
111
|
-
- Exit 1 (no input, a file that cannot be read, or a package with no readable SKILL.md): correct the file and run it again.
|
|
111
|
+
- `valid` false (exit 2), or exit 1: load `{coverageBranchesFile}` and follow its **Inventory Failures** section. Invalid a second time: HALT (`halt_reason: "inventory-invalid"`, phase `coverage-check:inventory`): "coverage-check: the subagent inventory failed {schema validation | the ground-truth spot-check} twice: {the violations joined}."
|
|
112
112
|
|
|
113
113
|
### 1b. Cross-Check Split-Body Consistency
|
|
114
114
|
|
|
115
|
-
Only when the validated inventory's `references` is non-empty.
|
|
115
|
+
Only when the validated inventory's `references` is non-empty. Each `cross_check_mismatches` entry is a High `split-body-mismatch` gap. §5b records it titled `Split-body mismatch: {export} ({reference_file})`, with its `export` and its Source at the side the source contradicts (the reference file when no source settles it): update-skill edits that file to match the other without reading the source.
|
|
116
116
|
|
|
117
117
|
### 2. Analyze Source Code (Tier-Dependent)
|
|
118
118
|
|
|
119
|
-
Identify the public API surface from the package entry points (
|
|
119
|
+
Identify the public API surface from the package entry points (0b), as deep as the tier allows. Whichever branch runs, `{coverageInputsScript} surface` writes it to `{run_dir}/surface.json`, which §2b, §2c, §4 and step 5 read. Names read by eye reach it as per-file results, `{"file": "<path>", "exports_found": [<each name>], "signature_mismatches": []}` for each file, saved to `{run_dir}/per-file-<n>.json` with the Write tool and given to `surface` with one `--per-file` each.
|
|
120
120
|
|
|
121
121
|
**Exits.** Every `{coverageInputsScript}` and `{scoreSignaturesScript}` command in §2 and §2b exits 0, or exits 1 with its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:surface`). `surface` with a `--per-file`, and `score`, also exit 2 when a subagent response breaks the schema: re-dispatch that subagent once with its `violations[]` appended (or correct a result the main thread wrote), save the new response over the old one and run the command again; a second failure HALTs the same way.
|
|
122
122
|
|
|
@@ -141,35 +141,31 @@ uv run {coverageInputsScript} surface --extraction "{run_dir}/extract-full.json"
|
|
|
141
141
|
The surface holds the names the entry points export, less those defined in a file the brief scopes out (`excluded.outsideScope`); the recipes' internal matches are never part of it. A Python `__init__.py` nested in the folder of another package that exports a name is no barrel: its names (`excluded.nestedEntries`) leave only the `all` and `root` sets and the umbrella ratio, and `plan` still compares their signatures. When a `scope.include` glob matches no file (`guards.staleScope.fires`), the root exports defined in a file no glob covers and the brief does not exclude are back in it (`guards.staleScope.restored`). A stale `scope.tier_a_include` glob fires the guard too (`guards.staleScope.unmatchedTierAInclude`) and counts no name. §5's render lists `excluded.outsideScope`, `excluded.nestedEntries` and the restored names. Decide what to do from the JSON, not from the exit code:
|
|
142
142
|
|
|
143
143
|
- `extraction.fallback.needed` is true, or the runner printed no JSON on an exit other than 2: load `{coverageTiersFile}` and run its **Fallback Per-File Scan** in place of the rest of this tier.
|
|
144
|
-
- `extraction.
|
|
145
|
-
-
|
|
144
|
+
- `extraction.readByEye` is non-empty (a file the run could not parse or decode, or a failed call left unread): keep the surface, read by eye only the files it names, save a per-file result for each, and run `surface` again with them, once (the second call starts no new pass).
|
|
145
|
+
- In the Coverage Analysis section, list each `readByEye` file as read by eye, each `extraction.unread` file as unread.
|
|
146
|
+
- `extractionGaps[]` names exports no recipe found: they are on the surface; read their signature by eye at their `file` and `line`.
|
|
146
147
|
|
|
147
148
|
**Compare the signatures.** A script lists the comparisons; subagents judge them:
|
|
148
149
|
|
|
149
150
|
```bash
|
|
150
|
-
uv run {scoreSignaturesScript} plan --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --output "{run_dir}/signature-plan.json"
|
|
151
|
+
uv run {scoreSignaturesScript} plan --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --skill-dir "{resolved_skill_package}" --output "{run_dir}/signature-plan.json"
|
|
151
152
|
```
|
|
152
153
|
|
|
153
|
-
For each `files[]` entry, delegate one subagent with its checks. For each check it reads the full declaration at `{file}:{line}` (all of it: `signatureLine` is one physical line, and a multi-line signature cut there
|
|
154
|
+
For each `files[]` entry, delegate one subagent with its checks. For each check it reads the full declaration at `{file}:{line}` (all of it: `signatureLine` is one physical line, and a multi-line signature cut there reads as a mismatch), compares it with the check's `documented` signature (params: name, type, order, optionality; and the return type), and returns only this JSON, with no prose and no markdown fences:
|
|
154
155
|
|
|
155
156
|
```json
|
|
156
|
-
{"file": "src/utils.ts", "signature_mismatches": [{"name": "formatDate", "line": 42, "source_sig": "(date: Date, format?: string) => string", "documented_sig": "(date: Date) => string", "issue": "missing optional parameter 'format'"}]}
|
|
157
|
+
{"file": "src/utils.ts", "signature_mismatches": [{"name": "formatDate", "line": 42, "source_sig": "(date: Date, format?: string) => string", "documented_sig": "(date: Date) => string", "issue": "missing optional parameter 'format'", "difference": "omitted-optional"}]}
|
|
157
158
|
```
|
|
158
159
|
|
|
159
|
-
|
|
160
|
+
A mismatch carries `difference` when the two differ only in one of these ways, else no key: `parameter-name` (only parameter names differ: types, order, optionality, defaults and return type equal; a destructured property is part of a type) or `omitted-optional` (the documented form leaves out only members the source marks optional). A parameter or field with a default or a `?`, or a TS/JS rest parameter, is optional; a Rust `Option<T>` field a struct literal still requires is not. Tag it in any language and form: `score` (§2b) checks the facts. The example above, tagged `omitted-optional`, stays Critical against a full form.
|
|
160
161
|
|
|
161
|
-
|
|
162
|
+
A check with `candidates` reads each and reports a mismatch only when the documented signature matches none: one, at the closest candidate, its `file` and `line` on the mismatch object (the result's `file` stays its `files[]` entry) and its `difference` tagged against it. A declaration opening an overload set (Python `@overload` stubs, TS bodiless signatures) is read through its implementation: a documented signature matching the implementation or any overload is no mismatch.
|
|
162
163
|
|
|
163
|
-
|
|
164
|
-
- **State 2** (`{forge_provenance_map}`, the provenance map init.md §2 bound): the union of its named exports and the `metadata.json` `exports[]` names, with `--fold`, `--keep` and `--fold-prefix` as the Source API Surface Definition's canonicalization says:
|
|
164
|
+
Save each response to `{run_dir}/signatures-<n>.txt` with the Write tool. Without subagents, compare in the main thread and save the same JSON.
|
|
165
165
|
|
|
166
|
-
|
|
167
|
-
uv run {coverageInputsScript} surface --provenance "{forge_provenance_map}" --metadata "{resolved_skill_package}/metadata.json" [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] [--fold] [--keep <variant>] [--fold-prefix <prefix>] [--monorepo] [--pattern-reference-intent] [--curated-subset] --output "{run_dir}/surface.json"
|
|
168
|
-
```
|
|
166
|
+
**Deep Tier (ast-grep + gh + QMD):** the Forge tier checks, plus: check with the gh CLI that the source repository matches the documented version, check type definitions against their source declarations, and trace re-exports to their origin.
|
|
169
167
|
|
|
170
|
-
|
|
171
|
-
- **State 3**: the `metadata.json` `exports[]` names: `surface --metadata "{resolved_skill_package}/metadata.json" [--monorepo] [--pattern-reference-intent] [--curated-subset] --output "{run_dir}/surface.json"`.
|
|
172
|
-
- **State 4**: the names read remotely by eye, as per-file results: `surface --per-file "{run_dir}/per-file-<n>.json" [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] --metadata "{resolved_skill_package}/metadata.json" [--monorepo] [--pattern-reference-intent] [--curated-subset] [--barrel <state>] --output "{run_dir}/surface.json"`.
|
|
168
|
+
**States 2 to 4 (no local source):** the surface is what the skill recorded, so there is no signature to score: load `{coverageBranchesFile}` and follow its **States 2 to 4** section.
|
|
173
169
|
|
|
174
170
|
### 2b. Resolve the Denominator and Guard Zero Exports
|
|
175
171
|
|
|
@@ -179,7 +175,7 @@ Load the metadata and provenance-map counts once, for §2c, §4b and the numerat
|
|
|
179
175
|
uv run {coverageInputsScript} metadata --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] --output "{run_dir}/coverage-inputs.json"
|
|
180
176
|
```
|
|
181
177
|
|
|
182
|
-
**Pick the denominator.** `surface.json` `denominator` holds the Source API Surface Definition clause (§0b) `surface` picked: its §2c `branch` (`barrel`, `scalar` or `stack`), `surfaceSet` (the set the barrel branch counts and the signature score reads) and `ladderStep`. When `absentSet` names a set the surface lacks (the scope sets need the brief's globs and names that carry their files, `subpaths` and `root` an extraction), `surfaceSet` is `all` and the `Denominator:` line (§4) adds `({absentSet} set absent: all set used)`. When `rebuild` is `provenance-fold`, rerun State 2's command
|
|
178
|
+
**Pick the denominator.** `surface.json` `denominator` holds the Source API Surface Definition clause (§0b) `surface` picked: its §2c `branch` (`barrel`, `scalar` or `stack`), `surfaceSet` (the set the barrel branch counts, as the scalar one does when `setCounted` is true, and the signature score reads) and `ladderStep`. When `absentSet` names a set the surface lacks (the scope sets need the brief's globs and names that carry their files, `subpaths` and `root` an extraction), `surfaceSet` is `all` and the `Denominator:` line (§4) adds `({absentSet} set absent: all set used)`. When `rebuild` is `provenance-fold`, load `{coverageBranchesFile}` and rerun State 2's command from its **States 2 to 4** section without `--metadata` and with `--fold` and `--pattern-reference-intent`, running nothing else from that section, and read the `denominator` it writes.
|
|
183
179
|
|
|
184
180
|
- **A stack skill** (`branch` `stack`): `coverage-inputs.json` `stack.denominator`. Its own barrel is empty by design, and its `[from skill: …]` citations never make it docs-only. **If `stack.denominator` is 0**: HALT (`halt_reason: "indeterminate-surface"`, phase `coverage-check:denominator`) with `Error: stack composition surface empty: {skill_name} cites no contracts, libraries, or integration pairs, so Export Coverage is undefined. Verify the stack was compiled from at least one constituent skill.` Do not write the Coverage Analysis section; this is an indeterminate state, not a FAIL.
|
|
185
181
|
- **A docs-only skill**: the `docsOnly` branch, over the inventory.
|
|
@@ -190,7 +186,7 @@ uv run {coverageInputsScript} metadata --metadata "{resolved_skill_package}/meta
|
|
|
190
186
|
uv run {numeratorVerifyScript} --inputs "{run_dir}/coverage-inputs.json" --skill-dir "{resolved_skill_package}" --output "{run_dir}/numerator.json"
|
|
191
187
|
```
|
|
192
188
|
|
|
193
|
-
§2c's scalar branch
|
|
189
|
+
Unless §2c's scalar branch counts its set, it takes the `verified` count as the numerator when `inflated` is true; §4b records the gap. Exit 1 or 2: HALT with its message (`halt_reason: "helper-failed"`, phase `coverage-check:numerator`).
|
|
194
190
|
|
|
195
191
|
**If the chosen denominator is 0 AND `docs_only_mode == false` AND `metadata.json.skill_type != "stack"`:** HALT (`halt_reason: "indeterminate-surface"`, phase `coverage-check:denominator`) with:
|
|
196
192
|
|
|
@@ -208,14 +204,14 @@ Do not write the Coverage Analysis section or attach a score: this is an indeter
|
|
|
208
204
|
**Score the signatures** (Forge, Forge+ and Deep tier at State 1, once §2 compared them). Give `{scoreSignaturesScript}` `denominator.surfaceSet` as `--surface-set`. Type Coverage then counts that set's types alone, so a type the clause leaves out (a stratified skill's deferred Tier B) lowers neither Export Coverage nor Type Coverage. Give it one `--results` per saved response (the fallback scan's `per-file-<n>.txt`, else the `signatures-<n>.txt`):
|
|
209
205
|
|
|
210
206
|
```bash
|
|
211
|
-
uv run {scoreSignaturesScript} score --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --results "{run_dir}/signatures-<n>.txt" --output "{run_dir}/signatures.json" --gaps-output "{run_dir}/signature-gaps.json"
|
|
207
|
+
uv run {scoreSignaturesScript} score --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --plan "{run_dir}/signature-plan.json" --results "{run_dir}/signatures-<n>.txt" --output "{run_dir}/signatures.json" --gaps-output "{run_dir}/signature-gaps.json"
|
|
212
208
|
```
|
|
213
209
|
|
|
214
|
-
It
|
|
210
|
+
It grades each mismatch of a compared name from facts it checks, never a `difference` alone, into `signature-gaps.json`: a Low `signature-mismatch` gap for a `parameter-name` difference in a file the plan offered when it is TS/JS, Go, Rust, Java, C or C++, an Info `observation` titled `Partial signature: {name}` for an `omitted-optional` one on a `partial` form, a Critical `signature-mismatch` gap otherwise (Python included). A signature gap is titled `Signature mismatch: {name}`, with `export` `{name}`, its Source at `{file}:{line}` and both signatures in its Issue; §5b appends that file to the ledger. A Low or Info one counts as matching, and §5 renders it so. §5's render lists its `warnings` (a mismatch for a name never compared, a category with nothing to compare).
|
|
215
211
|
|
|
216
212
|
### 2c. Reconcile Documented vs Source Surface (Deterministic Intersection)
|
|
217
213
|
|
|
218
|
-
`{reconcileScript}` reconciles the §1 inventory with the source surface and writes `{run_dir}/coverage.json`, whose `exportCoverage` is the Export Coverage
|
|
214
|
+
`{reconcileScript}` reconciles the §1 inventory with the source surface and writes `{run_dir}/coverage.json`, whose `exportCoverage` is the Export Coverage step 5 hands to the scoring script. Run the branch §2b picked:
|
|
219
215
|
|
|
220
216
|
1. **Barrel**, with the name set §2b picked as `--surface-set`:
|
|
221
217
|
|
|
@@ -223,19 +219,9 @@ It writes each mismatch of a compared name to `signature-gaps.json` as a Critica
|
|
|
223
219
|
uv run {reconcileScript} --denominator-source barrel --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --output "{run_dir}/coverage.json"
|
|
224
220
|
```
|
|
225
221
|
|
|
226
|
-
2. **Scalar
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
uv run {reconcileScript} --denominator-source scalar --coverage-inputs "{run_dir}/coverage-inputs.json" --inventory "{run_dir}/inventory.json" --skill-dir "{resolved_skill_package}" --verified "{run_dir}/numerator.json" --output "{run_dir}/coverage.json"
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
3. **Stack** (`metadata.json.skill_type == "stack"`):
|
|
222
|
+
2. **Scalar or stack**: load `{coverageBranchesFile}` and follow its **Scalar and Stack Branches** section, which runs the branch's command.
|
|
233
223
|
|
|
234
|
-
|
|
235
|
-
uv run {reconcileScript} --denominator-source stack --coverage-inputs "{run_dir}/coverage-inputs.json" --skill-dir "{resolved_skill_package}" --output "{run_dir}/coverage.json"
|
|
236
|
-
```
|
|
237
|
-
|
|
238
|
-
4. **Docs-only**: Export Coverage is the documentation completeness ratio, complete items over documented items, and `incomplete[]` lists each incomplete item with the fields it lacks:
|
|
224
|
+
3. **Docs-only**: Export Coverage is the documentation completeness ratio, complete over documented items, and `incomplete[]` lists each incomplete item with the fields it lacks:
|
|
239
225
|
|
|
240
226
|
```bash
|
|
241
227
|
uv run {reconcileScript} --denominator-source docsOnly --inventory "{run_dir}/inventory.json" --output "{run_dir}/coverage.json"
|
|
@@ -243,34 +229,22 @@ It writes each mismatch of a compared name to `signature-gaps.json` as a Critica
|
|
|
243
229
|
|
|
244
230
|
Exit 2 printed an `INVALID_INPUT` envelope: correct the input it names and run it again. Exit 1: HALT with its stderr (`halt_reason: "helper-failed"`, phase `coverage-check:reconcile`).
|
|
245
231
|
|
|
246
|
-
**A surplus is a signal, not a pass.** The scalar and stack branches count the numerator and the denominator from independent sets, so `documented` can exceed `denominator`: the script then reports `numeratorSurplus` above 0 and `coverageCapped: true`, holds `exportCoverage` at 100 and keeps the raw ratio in `coverageUncapped`, and §5's render states the surplus beside both counts. On a skill with no `scope.tier_a_include` glob that matches a file, a large surplus is the deflated-denominator signature: check `guards.deflation` before accepting the 100.
|
|
247
|
-
|
|
248
232
|
**Classify what the script found.** §5b records each of these gaps from the run files:
|
|
249
233
|
|
|
250
|
-
- **Missing names** (barrel
|
|
251
|
-
- **
|
|
252
|
-
- **Stale names** (barrel branch: `stale`): each is a documented name the enumerated source surface lacks. Where ast-grep read the source at the pinned commit (Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` in the `{outputFile}` frontmatter) and init.md §2 bound `{forge_provenance_map}`, check the lines the skill cites for them: resolve `{verifyProvenanceCompletenessHelper}` ← first existing path in `{verifyProvenanceCompletenessProbeOrder}` and run once:
|
|
234
|
+
- **Missing names** (`missing`, barrel or scalar): Medium `missing-export` gaps (Medium `missing-type` for a name `signatures.json` lists in `missingTypes`), each at the file `surface.json` records for it, so update-skill can re-extract the export from it; when `guards.inflation.fires` on a stratified-scope or multi-entry `scope.include` set, one Medium `denominator-inflation` gap beside the guard's at the brief lists them instead. A missing export does not block: it lowers Export Coverage.
|
|
235
|
+
- **Stale names** (`stale` when `staleApplicable` is true): each is a documented name the enumerated source surface lacks. Where ast-grep read the source at the pinned commit (Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` in the `{outputFile}` frontmatter) and init.md §2 bound `{forge_provenance_map}`, check the lines the skill cites for them: resolve `{verifyProvenanceCompletenessHelper}` ← first existing path in `{verifyProvenanceCompletenessProbeOrder}` and run once:
|
|
253
236
|
|
|
254
237
|
```bash
|
|
255
238
|
uv run {verifyProvenanceCompletenessHelper} classify-stale --names "{run_dir}/coverage.json" --provenance "{forge_provenance_map}" --source-root "{source_path}" --inventory "{run_dir}/inventory.json" -o "{run_dir}/stale.json"
|
|
256
239
|
```
|
|
257
240
|
|
|
258
|
-
A name it marks `fabricated: true` (every map entry of the name cites a missing file or one that does not define it) is a Critical `fabricated-signature` gap at its `source`. A name with a `defined_at` (the source still declares it there, an import never counting, a module so named counting unless the name's inventory kind rules it out), or whose `declared_in` lists several files of which the skill's `[AST:]`/`[SRC:]` citations on lines naming it cite exactly one, is a documented extra: an Info `observation` gap at that declaration, when `surface.json` has an `extraction` and its `excluded.outsideScope` does not list the name (a brief scoped its file out). Any other stale name (no `defined_at` and no single cited declaring file, one scoped out, no `extraction`, a dotted name, the conditions above not met, a helper that does not resolve or exits non-zero) is a Medium `stale-documentation` gap.
|
|
241
|
+
A name it marks `fabricated: true` (every map entry of the name cites a missing file or one that does not define it) is a Critical `fabricated-signature` gap at its `source`. A name with a `defined_at` (the source still declares it there, an import never counting, a module so named counting unless the name's inventory kind rules it out), or whose `declared_in` lists several files of which the skill's `[AST:]`/`[SRC:]` citations on lines naming it cite exactly one, is a documented extra: an Info `observation` gap at that declaration, when `surface.json` has an `extraction` and its `excluded.outsideScope` does not list the name (a brief scoped its file out). Any other stale name (no `defined_at` and no single cited declaring file, one scoped out, no `extraction`, a dotted name, the conditions above not met, a helper that does not resolve or exits non-zero) is a Medium `stale-documentation` gap; one the brief's globs scope out says which.
|
|
259
242
|
|
|
260
243
|
### 4. The Denominator Record
|
|
261
244
|
|
|
262
|
-
Record the denominator source in the Coverage Analysis section with the annotation `{sourceAccessProtocol}` gives `denominator.clause` (§4b gives a stack's and a reference app's): `Denominator: {barrel | empty barrel (…) | stratified (…) | multi-entry (…) | specific-modules (…) | pattern-reference (…) | stack composition (…) | pattern-surface (…) | docs-only completeness}`, and the `canonical.summary` line when `surface.json` has a fold. When `denominator.clause` is `stratified-scope` or `multi-entry`, append
|
|
245
|
+
Record the denominator source in the Coverage Analysis section with the annotation `{sourceAccessProtocol}` gives `denominator.clause` (§4b gives a stack's and a reference app's): `Denominator: {barrel | empty barrel (…) | stratified (…) | multi-entry (…) | specific-modules (…) | pattern-reference (…) | stack composition (…) | pattern-surface (…) | docs-only completeness}`, and the `canonical.summary` line when `surface.json` has a fold. When `denominator.clause` is `stratified-scope` or `multi-entry`, load `{coverageBranchesFile}` and append the block of its **Denominator Candidates** section after the `Denominator:` line, with every count from `surface.json` `candidates` (`absent` for a null one).
|
|
263
246
|
|
|
264
|
-
|
|
265
|
-
**Denominator Candidates** (stratified-scope audit trail):
|
|
266
|
-
- `stats.effective_denominator`: {statsEffectiveDenominator | absent} {← chosen on the scalar branch}
|
|
267
|
-
- `scope.tier_a_include` union: {tierAIncludeUnion | absent} {← chosen if the barrel branch's set}
|
|
268
|
-
- `scope.include` union: {scopeIncludeUnion | absent} {← chosen if the barrel branch's set}
|
|
269
|
-
- exports-map subpath union: {subpathUnion | absent} {← chosen if the barrel branch's set}
|
|
270
|
-
- root barrel: {rootBarrel | absent} {secondary candidate: root-barrel-vs-subpath-union audit}
|
|
271
|
-
```
|
|
272
|
-
|
|
273
|
-
The guards raise coverage gaps too, which §5b records from `surface.json`: `guards.deflation.fires` is a Medium `metadata-drift` gap, `guards.inflation.fires` a Medium `denominator-inflation` gap and `guards.staleScope.fires` a Medium `brief-scope-stale` gap at the brief.
|
|
247
|
+
The guards raise coverage gaps too, which §5b records from `surface.json`: `guards.deflation.fires` is a Medium `metadata-drift` gap (on its `tier_a_include` basis, a stale stored count to recount), `guards.inflation.fires` a Medium `denominator-inflation` gap (with deflation, both name one fix: the brief's tier A list) and `guards.staleScope.fires` a Medium `brief-scope-stale` gap at the brief.
|
|
274
248
|
|
|
275
249
|
### 4b. Metadata Export-Count Coherence Cross-Check
|
|
276
250
|
|
|
@@ -282,11 +256,11 @@ uv run {coherenceScript} --inputs "{run_dir}/coverage-inputs.json" > "{run_dir}/
|
|
|
282
256
|
|
|
283
257
|
A non-zero exit leaves the script's error, or nothing, in that file: delete it, so §5b skips its call and records no count finding, and note the error in the Coverage Analysis section. §5b records each `findings[]` entry: a Medium entry as a Medium `metadata-drift` gap and an Info entry as an Info `multi-denominator` gap. They describe data quality and change no denominator. Record the denominator of a stack (`Denominator: stack composition ({N} cited contracts)` or `({N} libraries + integration pairs)`, from `stack.basis`) and of a reference app (`Denominator: pattern-surface ({pattern_surfaces_documented})`).
|
|
284
258
|
|
|
285
|
-
**The numerator ground truth §2b ran.** `numerator.json` `inflated: true` (`verified < declared`) is a High `numerator-inflation` gap listing its `absent[]` names, which §5b records; on
|
|
259
|
+
**The numerator ground truth §2b ran.** `numerator.json` `inflated: true` (`verified < declared`) is a High `numerator-inflation` gap listing its `absent[]` names, which §5b records; on a scalar branch that counts no set those names also replace the missing-count gap in the Scalar and Stack Branches section of coverage-check-branches.md. `inflated: false` or `skipped: true` is no finding.
|
|
286
260
|
|
|
287
261
|
### 4c. Provenance Line Check
|
|
288
262
|
|
|
289
|
-
Check that the provenance map records each export at the line that defines it (the `def` or declaration line
|
|
263
|
+
Check that the provenance map records each export at the line that defines it (the `def` or declaration line, not a decorator or blank line above it), so update-skill can move a wrong line. Run this section only when `analysis_confidence` is `full`, `{forge_version}/provenance-map.json` exists, and the `{outputFile}` frontmatter's `workspaceDrift` is not `overridden` (init.md §5b: the drifted tree is not the pinned commit). Otherwise skip to section 5.
|
|
290
264
|
|
|
291
265
|
Resolve `{verifyProvenanceCompletenessHelper}` ← first existing path in `{verifyProvenanceCompletenessProbeOrder}`. If neither path exists, skip to section 5: the check is advisory. Otherwise run:
|
|
292
266
|
|
|
@@ -340,6 +314,8 @@ uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check -
|
|
|
340
314
|
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from provenance-line --input "{run_dir}/provenance-verify.json"
|
|
341
315
|
```
|
|
342
316
|
|
|
317
|
+
**A split-body mismatch's Source** is a line inside the skill package. At State 1 and the Forge tier or above, with `workspaceDrift` not `overridden`, set the two forms its `issue` gives against the export's declaration at the `file` and `line` `surface.json` gives, when that one declaration decides (no overload set, no second declaring file): when the reference agrees and the body does not, the Source is `SKILL.md:{skill_md_line}`, else `{reference_file}:{reference_line}`. Its remediation names the file to edit and the file and line whose form it takes, citing the declaration's `file:line`; when both forms agree with it, it says so and names what differs; when neither does, or nothing decides, it says no source settled it.
|
|
318
|
+
|
|
343
319
|
Then write the records no script wrote, §1b's split-body mismatches, in the Ledger Record Format of `{outputFormatsFile}`, each naming its export in `export` and in its title: the ledger tells two gaps apart by their title, Source and `export`, and update-skill takes the export's name from `export`. Write them as one JSON array on the lines between the two markers, exactly as they are: the quoted marker hands them to the script unchanged, quotes, apostrophes and `$` included. Run the command even when the array is empty (`[]`): the hard gate refuses to decide until every stage before it has recorded, with gaps or without.
|
|
344
320
|
|
|
345
321
|
```bash
|
|
@@ -356,6 +332,6 @@ Rely on the JSON of each command:
|
|
|
356
332
|
|
|
357
333
|
### 6. Report Coverage Results
|
|
358
334
|
|
|
359
|
-
Report
|
|
335
|
+
Report to the user from §5's render: the {forge_tier} tier, the **Files Analyzed** count, the three category scores and the issue count; the Coverage Analysis section has the details.
|
|
360
336
|
|
|
361
337
|
Append `'coverage-check'` to `stepsCompleted` in the `{outputFile}` frontmatter, then load and execute {nextStepFile}.
|
|
@@ -313,13 +313,14 @@ thresholdSource: '{threshold_source}'
|
|
|
313
313
|
analysisConfidence: ''
|
|
314
314
|
toolingStatus: '{ok|frontmatter-validator-timeout}'
|
|
315
315
|
workspaceDrift: '{not-checked|ok|overridden}'
|
|
316
|
+
health_check_dispatched: false
|
|
316
317
|
testDate: '{run_id timestamp ISO-8601 UTC}'
|
|
317
318
|
stepsCompleted: ['init']
|
|
318
319
|
nextWorkflow: ''
|
|
319
320
|
---
|
|
320
321
|
```
|
|
321
322
|
|
|
322
|
-
`toolingStatus` is the `tooling_status` §3b set: score.md §3a reads it back for Cap 1. `threshold` and `thresholdSource` are the pass threshold §1b resolved and its source: score.md §1 reads them back, and score.md §7 rewrites `threshold` with the threshold the verdict was settled against. `analysisConfidence` stays empty until score.md §7 writes the source access coverage-check resolved: `full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`.
|
|
323
|
+
Replace the template's frontmatter with this block (the two hold the same keys). `toolingStatus` is the `tooling_status` §3b set: score.md §3a reads it back for Cap 1. `threshold` and `thresholdSource` are the pass threshold §1b resolved and its source: score.md §1 reads them back, and score.md §7 rewrites `threshold` with the threshold the verdict was settled against. `analysisConfidence` stays empty until score.md §7 writes the source access coverage-check resolved: `full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`.
|
|
323
324
|
|
|
324
325
|
### 7. Report Initialization Status
|
|
325
326
|
|
|
@@ -16,7 +16,8 @@ Execute this check only when both conditions are met:
|
|
|
16
16
|
the flat copy an older skill may still keep); the detection contract below
|
|
17
17
|
reads it.
|
|
18
18
|
|
|
19
|
-
If either condition fails, skip
|
|
19
|
+
If either condition fails, skip the check with no note and proceed to the next
|
|
20
|
+
section: a missing evidence report is not Case 4.
|
|
20
21
|
|
|
21
22
|
The check runs regardless of naive/contextual mode. T2-future annotations are a
|
|
22
23
|
property of the source code and enrichment data, not the skill type.
|
|
@@ -54,23 +55,26 @@ narrative body.
|
|
|
54
55
|
**Detection contract.** Read the frontmatter deterministically:
|
|
55
56
|
|
|
56
57
|
```bash
|
|
57
|
-
# Extract t2_future_count from frontmatter. Requires a `---` delimiter pair
|
|
58
|
-
|
|
58
|
+
# Extract t2_future_count from frontmatter. Requires a `---` delimiter pair;
|
|
59
|
+
# a trailing CR is stripped first, so a CRLF report reads the same.
|
|
60
|
+
awk '{sub(/\r$/,"")} /^---$/{c++;next} c==1 && /^t2_future_count:/{print $2; exit}' \
|
|
59
61
|
{forge_evidence_report}
|
|
60
62
|
```
|
|
61
63
|
|
|
62
|
-
- **Frontmatter missing OR `t2_future_count` absent
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
invert Case-1 vs Case-2/3
|
|
67
|
-
|
|
68
|
-
|
|
64
|
+
- **Frontmatter missing, OR `t2_future_count` absent or not a non-negative
|
|
65
|
+
integer** (empty, `null`, quoted) → treat as Case 4 (see below). Do not
|
|
66
|
+
fall back to grepping prose (`grep "T2-future"`): prose drift (heading
|
|
67
|
+
renames, alt phrasings like "forward-looking annotations", capitalization
|
|
68
|
+
variance) silently breaks detection and can invert Case-1 vs Case-2/3
|
|
69
|
+
severity.
|
|
70
|
+
- **`t2_future_count` a non-negative integer** (digits only) → use its value
|
|
71
|
+
for the Case Rules below.
|
|
69
72
|
|
|
70
73
|
The pinned field is emitted by `skf-create-skill/references/compile.md`
|
|
71
74
|
§7 (frontmatter-pinned fields), which always writes `t2_future_count: N`
|
|
72
75
|
(including 0). Legacy skills whose `evidence-report.md` predates the pinned
|
|
73
|
-
field land in Case 4
|
|
76
|
+
field land in Case 4, and so does every stack skill: create-stack-skill's
|
|
77
|
+
evidence report has no frontmatter.
|
|
74
78
|
|
|
75
79
|
### Case 1 — T2-future > 0 AND Section 4b absent
|
|
76
80
|
|
|
@@ -106,11 +110,19 @@ Flag as **Medium** severity gap:
|
|
|
106
110
|
Reviewer may downgrade to Low with inline justification on a case-by-case
|
|
107
111
|
basis.
|
|
108
112
|
|
|
109
|
-
### Case 4
|
|
113
|
+
### Case 4: evidence report has no pinned count
|
|
110
114
|
|
|
111
|
-
|
|
115
|
+
The evidence report exists but pins no usable `t2_future_count`: the field is
|
|
116
|
+
absent or not a non-negative integer, or the report has no frontmatter at all
|
|
117
|
+
(a create-skill report older than the field, or any create-stack-skill
|
|
118
|
+
report). Skip the Case Rules and raise no gap. Record this note on the
|
|
119
|
+
Coherence Analysis section's **Migration Section:** line (`{outputFormatsFile}`
|
|
120
|
+
gives it in both formats):
|
|
112
121
|
|
|
113
|
-
> "Section 4b verification skipped
|
|
122
|
+
> "Section 4b verification skipped: evidence-report.md pins no
|
|
123
|
+
> `t2_future_count`. For a skill create-skill made, re-run create-skill to
|
|
124
|
+
> regenerate the report with the field (create-stack-skill does not write
|
|
125
|
+
> it)."
|
|
114
126
|
|
|
115
127
|
## Output
|
|
116
128
|
|
|
@@ -221,7 +221,7 @@ uv run {resultContextScript} --report "{report_file}" --report-path "{publishedR
|
|
|
221
221
|
|
|
222
222
|
Bind `{emit_target}` ← `target` from its JSON (`stderr` for a run the hard gate blocked, else `stdout`). Exit 2: correct the command and run it again. Exit 1: HALT with its `error` (`halt_reason: "helper-failed"`, phase `report:result-contract`); the report keeps its in-progress name, so export-skill never reads a verdict the scoring output did not settle.
|
|
223
223
|
|
|
224
|
-
**Publish the report.** Set `health_check_dispatched
|
|
224
|
+
**Publish the report.** Set `health_check_dispatched`, already in the `{outputFile}` frontmatter: `false` when `no_health_check` is true (the `--no-health-check` flag), else `true`, since the health check now runs with no menu before it. The checks passed, so the report takes its public name, the one export-skill and update-skill read, a blocked run's included. From `{project-root}`, run:
|
|
225
225
|
|
|
226
226
|
```bash
|
|
227
227
|
mv "{report_file}" "{publishedReportFile}"
|
|
@@ -289,7 +289,7 @@ The report is final once §4c published it and wrote the result files: no later
|
|
|
289
289
|
**update-skill** — The skill scored above threshold, but `--allow-workspace-drift` was in effect: the test ran against workspace HEAD, not `metadata.source_commit`. A conditional PASS is not trustworthy enough to export. Re-sync the source tree to the pinned commit (or re-extract against current HEAD) and re-run test-skill without the drift override before exporting.
|
|
290
290
|
|
|
291
291
|
{IF FAIL after scoring:}
|
|
292
|
-
**update-skill `--from-test-report`**: This skill needs remediation. Review the Gap Report above, then run `@Ferris US {skill_name} --from-test-report`: Update Skill
|
|
292
|
+
**update-skill `--from-test-report`**: This skill needs remediation. Review the Gap Report above, then run `@Ferris US {skill_name} --from-test-report`: Update Skill repairs the gaps it routes, at the skill's pinned commit, and lists each other gap as not repaired, with the reason. A `stale-documentation` or `metadata-drift` gap needs what its remediation names: a person, Create Skill or the skill brief. Test it again afterwards.
|
|
293
293
|
|
|
294
294
|
{IF the hard gate blocked the run:}
|
|
295
295
|
**update-skill `--from-test-report`**: The hard gate stopped this run before scoring on {blocking_gaps} Critical or High gap(s), listed first in the Gap Report. Run `@Ferris US {skill_name} --from-test-report` to repair them, then test again.
|
|
@@ -18,7 +18,7 @@ This table is the single source of truth for classifying a gap. Every stage reco
|
|
|
18
18
|
|
|
19
19
|
| Severity | Category | Criteria |
|
|
20
20
|
|----------|-----------------------------|----------|
|
|
21
|
-
| Critical | `signature-mismatch` | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type (coverage-check §2) |
|
|
21
|
+
| Critical | `signature-mismatch` | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type, in any way but the two the Low `signature-mismatch` row and the Info `observation` row grade, such as a renamed Python parameter (the script cannot see which parameters a caller must pass by position), an optional parameter left out of a form the skill does not label partial, or a mismatch with no `difference` (coverage-check §2 and §2b) |
|
|
22
22
|
| Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name (coverage-check §2c) |
|
|
23
23
|
| Critical | `broken-reference` | Broken reference (contextual mode): a reference whose target does not exist, such as a file not at the stated path, a skill not in the skills output folder, a type no module declares, or a `scripts/` or `assets/` file the skill names (coherence-check §4) |
|
|
24
24
|
| High | `inaccurate-reference` | Inaccurate reference (contextual mode): the target exists but does not match, such as a type the cited module does not export, a signature that differs, or an integration pattern that does not match the implementation (coherence-check §4) |
|
|
@@ -31,20 +31,21 @@ This table is the single source of truth for classifying a gap. Every stage reco
|
|
|
31
31
|
| Medium | `missing-type` | Missing type or interface documentation |
|
|
32
32
|
| Medium | `stale-documentation` | A documented export the enumerated source surface lacks that is not a fabricated signature and not a documented extra: classify-stale found no single file that declares it (an import, a re-export or an indented local does not count, nor does a module so named when the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, as the Info `observation` row says) and the skill's citations on the lines that name it cite no single one of the files that do, it is a dotted name, the brief scoped its file out (`excluded.outsideScope`), or the run could not look: the source is not Python or TS/JS, classify-stale did not run (it needs the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map), or `surface.json` has no complete `extraction` (coverage-check §2c) |
|
|
33
33
|
| Medium | `migration-section` | Migration section present/absent mismatch with T2-future annotation data (Deep tier; Case 1 and Case 3 of `migration-section-rules.md`) |
|
|
34
|
-
| Medium | `metadata-drift` | Metadata drift: intra-cluster export counts diverge by more than 10% (barrel: `stats.exports_public_api` vs `exports[].length`; documented surface: `stats.exports_documented` vs the provenance-map named-export count), or denominator deflation: the re-derived source barrel exceeds `effective_denominator` by more than 25% and the brief has no `scope.tier_a_include` glob that counts a public name (coverage-check §4 and §4b) |
|
|
35
|
-
| Medium | `denominator-inflation` | Denominator inflation: the stratified-scope `scope.include` union exceeds the provenance-map entry count by more than 25% (the brief has no `scope.tier_a_include` glob that counts a public name) |
|
|
34
|
+
| Medium | `metadata-drift` | Metadata drift: intra-cluster export counts diverge by more than 10% (barrel: `stats.exports_public_api` vs `exports[].length`; documented surface: `stats.exports_documented` vs the provenance-map named-export count), or denominator deflation: the re-derived source barrel exceeds `effective_denominator` by more than 25% and the brief has no `scope.tier_a_include` glob that counts a public name, or denominator drift: on a stratified-scope, specific-modules or multi-entry skill, the brief's `scope.tier_a_include` set differs from `effective_denominator` by more than 25% either way, or counts a name against an `effective_denominator` of 0, a stale count to recount (coverage-check §4 and §4b) |
|
|
35
|
+
| Medium | `denominator-inflation` | Denominator inflation: the stratified-scope `scope.include` union exceeds the provenance-map entry count by more than 25% (the brief has no `scope.tier_a_include` glob that counts a public name). On a stratified-scope or multi-entry barrel over that union, the names it counts and the skill does not document are one more such gap beside it at the brief, listing at most 50 and deferring to its fix, instead of a `missing-export` gap each; Export Coverage still counts them |
|
|
36
36
|
| Medium | `brief-scope-stale` | Stale brief scope: a `scope.include` glob matches no file in the source tested, so the surface restores the root exports defined in a file no glob covers that the brief does not exclude, or a `scope.tier_a_include` glob matches no file in the source tested, so it counts no name (coverage-check §4) |
|
|
37
37
|
| Medium | `scripts-assets` | A `scripts/` or `assets/` directory exists but SKILL.md has no Scripts & Assets section (coherence-check §2.7 and §4) |
|
|
38
38
|
| Medium | `integration-pattern` | Incomplete integration pattern (contextual mode): one entry of coherence-check §5's `incomplete_patterns` |
|
|
39
39
|
| Medium | `structural` | Naive mode: an opening code fence without a language tag, an exported function or method no usage-family section or reference file names, or a table row whose column count differs from its header (coherence-check §2.3, §2.4, §2.6) |
|
|
40
40
|
| Medium | `discovery` | Discovery test: 2 of 3 realistic prompts routed to the skill (report §4b.3) |
|
|
41
|
+
| Low | `signature-mismatch` | A renamed parameter callers cannot see: a mismatch the signature judge tags `parameter-name` (only parameter names differ; types, order, optionality, defaults and return type are equal) at the file the surface records for the export, the one the plan sent the check to or a candidate the judge compared with, when it is a `.ts`, `.tsx`, `.mts`, `.cts`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.go`, `.rs`, `.java`, `.c`, `.h`, `.cc`, `.cpp`, `.cxx` or `.hpp` file, a language with no keyword arguments (not a Vue component, whose props callers set by name), as score-signatures.py checks. It counts as matching in Signature Accuracy (coverage-check §2 and §2b) |
|
|
41
42
|
| Low | `scripts-assets-provenance` | Script/asset file present without provenance entry in provenance-map.json file_entries |
|
|
42
43
|
| Low | `provenance-line` | Provenance line is not the definition of an export (`line-not-definition`, coverage-check §4c) |
|
|
43
44
|
| Low | `migration-section` | Case 3 of `migration-section-rules.md` that the reviewer downgraded with an inline justification |
|
|
44
45
|
| Low | `metadata` | Missing optional metadata or examples |
|
|
45
46
|
| Low | `description` | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps), a Tessl Review description suggestion included |
|
|
46
47
|
| Low | `external-validator` | A skill-check diagnostic or a Tessl Review validation finding the run left unresolved (external-validators §5b) |
|
|
47
|
-
| Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c) |
|
|
48
|
+
| Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c). Also a partial signature: a mismatch the signature judge tags `omitted-optional` (the documented form leaves out only members the source marks optional) on a form the skill labels partial (simplified, key parameters, a `...` that stands in for omitted members, not rest or variadic syntax), which the inventory marks `signature_form: "partial"`, as score-signatures.py checks: `Partial signature: {name}`, at the declaration, counted as matching in Signature Accuracy (coverage-check §2 and §2b) |
|
|
48
49
|
| Info | `provenance-unverified` | Provenance line not verified: the line-check rules found no definition line (coverage-check §4c) |
|
|
49
50
|
| Info | `migration-section` | Historical migration content in Section 4b with no T2-future annotation (Case 2 of `migration-section-rules.md`) |
|
|
50
51
|
| Info | `multi-denominator` | Multi-denominator reporting: barrel and documented-surface clusters diverge by design (>10% cross-cluster) |
|
|
@@ -11,17 +11,17 @@
|
|
|
11
11
|
- **Go:** exported identifiers (capitalized) from the package's public-facing files
|
|
12
12
|
- **Rust:** items in `pub use` from `lib.rs` or `mod.rs`
|
|
13
13
|
|
|
14
|
-
The clauses below decide which part of that surface is the coverage denominator. `load-coverage-inputs.py surface` picks the clause from the brief (`{forge_data_folder}/{skill_name}/skill-brief.yaml`), `{source_is_monorepo}` (coverage-check.md §0b, passed as `--monorepo`), the extraction and `metadata.json`, and writes it to `surface.json` `denominator` with the branch, the name set, the rung it takes (coverage-check.md §2b) and the entry points' state it read (`barrel`), beside the counts each clause names (`sets`, `candidates` and `guards`). Three of its inputs are yours to pass as flags, the last two only where no file gives them:
|
|
14
|
+
The clauses below decide which part of that surface is the coverage denominator. `load-coverage-inputs.py surface` picks the clause from the brief (`{forge_data_folder}/{skill_name}/skill-brief.yaml`), `{source_is_monorepo}` (coverage-check.md §0b, passed as `--monorepo`), the extraction and `metadata.json`, and writes it to `surface.json` `denominator` with the branch, the name set, the rung it takes (coverage-check.md §2b) and the entry points' state it read (`barrel`), beside the counts each clause names (`sets`, `candidates` and `guards`). A workspace member counts toward `{source_is_monorepo}` only through the files the brief keeps: excluding its source files drops it, excluding only its manifest does not, and a workspace with no member left is no workspace. Three of its inputs are yours to pass as flags, the last two only where no file gives them:
|
|
15
15
|
|
|
16
16
|
- **Pattern-reference intent** (`--pattern-reference-intent`): `scope.notes` says the skill demonstrates an integration pattern rather than a library API ("Reference app, not a library", "pattern-reference", "embedded-pattern skill", "skill value is the … pattern"; the author's words are authoritative).
|
|
17
17
|
- **A curated monorepo subset** (`--curated-subset`), where the extraction has no `counts.curated_subset` (Quick tier, the fallback per-file scan, States 2 to 4): `{source_is_monorepo}` is true and the brief's `scope.include` lists a curated subset rather than the full workspace.
|
|
18
18
|
- **The entry points' state** (`--barrel`), where no complete extraction or quick output gives it (the fallback per-file scan when the runner listed entry points, Quick tier read by eye, State 4): `none` when no barrel file is at any recognized entry-point path (`__init__.py`, `index.ts`/`index.js`, `lib.rs`, `mod.rs`), `empty` when the primary entry point exports nothing (`export {};`, an empty `__init__.py`, a `lib.rs` with no `pub use`), else `barrel`.
|
|
19
19
|
|
|
20
|
-
The recipe runner already re-derives each barrel (multi-line re-export blocks, star re-exports and a Rust `pub use` chain included) and counts each name once,
|
|
20
|
+
The recipe runner already re-derives each barrel (multi-line re-export blocks, star re-exports and a Rust `pub use` chain included) and counts each name once, however many in-scope files define it, so a barrel that only re-exports adds nothing of its own. Each of those files gives the name a surface record of its own (the runner's `entry_point_diff.other_definitions` beside its `public` row), so a set holds the name when any of its files matches, and the signature plan can compare the declaration the skill cites. Without the runner's names (Quick tier, the fallback per-file scan, States 2 to 4) the script builds the `scope.include` and `tier_a_include` sets from the files the names carry and the brief's globs (the runner's JSON the fallback scan passes adds only its scope and stale globs), and there is no `subpaths` or `root` set (coverage-check.md §2b). Never re-count a barrel by hand. A `scope.tier_a_include` glob counts a name only in a file a `scope.include` glob also matches, and one that matches no file (`guards.staleScope.unmatchedTierAInclude`, a brief older than the source) counts none: when the tier A globs count no name, `surface.json` has no `tier_a_include` set. Each clause, by its `denominator.clause`, with what stays yours:
|
|
21
21
|
|
|
22
22
|
- **Empty-barrel packages (`empty-barrel`; copy-paste or subpath-only distribution):** the primary entry point is empty, so the denominator is the `scope.include` set. The brief's `scope.notes` should say the distribution model is by design; when it does, treat the empty barrel as intended rather than a bug. With no brief and an empty barrel, set `analysis_confidence: docs-only` and report that the source API surface could not be determined. Annotate: `Empty barrel, denominator: scope.include union ({N} exports)`.
|
|
23
23
|
|
|
24
|
-
- **Stratified-scope monorepo packages (`stratified-scope`; curated subsets of multi-package repos):** the denominator reflects only the authored surface, not the monorepo's global export count: the `stats.effective_denominator` Create Skill stored, unless `guards.deflation.fires` (a Medium `metadata-drift` gap, coverage-check.md §4: a legitimate scope reduction goes in the brief, through `scope.tier_a_include` or `scope.exclude`, never into `metadata.stats`), then the `tier_a_include` and `scope.include` sets. With no surface read from the source (`guards.deflation.applicable` false), annotate the report `effective_denominator used unverified: no source access to re-derive the barrel`. When `guards.inflation.fires` (the coarse globs sweep in internal infrastructure, a Medium `denominator-inflation` gap) and `guards.umbrella.umbrella` is true
|
|
24
|
+
- **Stratified-scope monorepo packages (`stratified-scope`; curated subsets of multi-package repos):** the denominator reflects only the authored surface, not the monorepo's global export count: the `stats.effective_denominator` Create Skill stored, unless `guards.deflation.fires` (a Medium `metadata-drift` gap, coverage-check.md §4: a legitimate scope reduction goes in the brief, through `scope.tier_a_include` or `scope.exclude`, never into `metadata.stats`), then the `tier_a_include` set when the guard measured the stored count against it (`guards.deflation.basis` `tier_a_include`, in either direction: a stale count, which re-running Create Skill recounts, storing the runner's tier A count when `effective_denominator_decision.store` holds and `scope.notes` documents the stratification and otherwise dropping the stored count; on an incomplete extraction a set below the stored count leaves the stored count), else the `scope.include` set; with no stored count, the `tier_a_include` and `scope.include` sets. Never set `stats.effective_denominator` by hand: Update Skill's recount drops it. With no surface read from the source (`guards.deflation.applicable` false), annotate the report `effective_denominator used unverified: no source access to re-derive the barrel`. When `guards.inflation.fires` (the coarse globs sweep in internal infrastructure, a Medium `denominator-inflation` gap), the `scope.include` union is itself inflated: when the deflation guard fires too, that union stays the denominator and both gaps ask for a `scope.tier_a_include` list of the files that define the authored names (when `guards.umbrella.umbrella` is true, those files rather than the barrel). When the barrel branch counts that union, here or on the multi-entry ladder, the names it counts and the skill does not document are one more `denominator-inflation` gap beside the guard's at the brief, which defers to the guard's fix, not a `missing-export` gap each. Leave `analysis_confidence` unchanged. Annotate the report: `Stratified scope, denominator: {effective_denominator | tier_a_include union | scope.include union} ({N} files matched, {M} exports union)`. `scope.notes` should document the stratification (for example "Tier A: fully documented; Tier B: deferred to references; Tier C: excluded"); when it does, treat the curated subset as intended, not a scope gap.
|
|
25
25
|
|
|
26
26
|
- **Standard barrel (`standard-barrel`, the `all` set):** `full-library` skills, single-package repositories and briefs where the full monorepo is intentionally in scope. A `scope.include` glob that matches no file (a brief written for an older layout of the source) does not shrink that barrel: `guards.staleScope` restores the root exports defined in a file no glob covers, unless `scope.exclude` or a `scope-expansion` amendment whose latest action is `skipped` or `demoted-include` matches the file. The stale glob is a Medium `brief-scope-stale` gap (coverage-check.md §4), which also names each `scope.tier_a_include` glob that matches no file.
|
|
27
27
|
|
|
@@ -29,15 +29,15 @@ The recipe runner already re-derives each barrel (multi-line re-export blocks, s
|
|
|
29
29
|
|
|
30
30
|
- **Single-crate curated subset (`specific-modules`):** a single-package repo whose brief carves a subset of the crate's public surface with `scope.include` and `scope.exclude`: the in-scope reachable barrel, or the `stats.effective_denominator` Create Skill stored for it, under the same deflation guard. Annotate: `Specific-modules subset, denominator: in-scope reachable barrel ({N} items from {M} modules, after scope.include/exclude filtering)`.
|
|
31
31
|
|
|
32
|
-
- **Multi-entry (exports-map) packages (`multi-entry`):** an in-scope `package.json` declares an `exports` map with more than one non-root subpath (`extraction.arms.multi_subpath_exports`). Installers reach every published subpath, so the denominator is the `subpaths` set (wildcard subpaths are left out: their surface is unbounded), or a stored count or `tier_a_include` set ahead of it. The root-barrel count (`candidates.rootBarrel`) is the secondary candidate in coverage-check.md §4
|
|
32
|
+
- **Multi-entry (exports-map) packages (`multi-entry`):** an in-scope `package.json` declares an `exports` map with more than one non-root subpath (`extraction.arms.multi_subpath_exports`). Installers reach every published subpath, so the denominator is the `subpaths` set (wildcard subpaths are left out: their surface is unbounded), or a stored count or `tier_a_include` set ahead of it, the stored count under the same deflation guard, whose fire counts the `tier_a_include` set when the surface has it, else the `scope.include` set. The root-barrel count (`candidates.rootBarrel`) is the secondary candidate in the Denominator Candidates block coverage-check.md §4 appends from coverage-check-branches.md. Annotate: `Multi-entry (exports-map), denominator: {effective_denominator | tier_a_include union | scope.include union | subpath union} ({N} subpaths resolved, {M} exports union; root barrel: {R})`.
|
|
33
33
|
|
|
34
|
-
Internal module symbols are **excluded** from the coverage denominator unless they are explicitly documented in SKILL.md (in which case they count as documented extras, not missing coverage): the surface holds only the names the entry points export, and the extraction's `internal` names never reach it. A documented extra reaches coverage-check.md §2c as a stale name. In a Python or TS/JS source, on a run where §2c runs `classify-stale` (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and whose `surface.json` came from a complete extraction, it is an Info `observation` at the line where `classify-stale` finds the source still declares it (`defined_at`, or among several declaring files the one the skill cites on a line naming it), which update-skill never routes. Otherwise, and for a name a brief scopes out (`excluded.outsideScope`), it stays a Medium `stale-documentation` gap.
|
|
34
|
+
Internal module symbols are **excluded** from the coverage denominator unless they are explicitly documented in SKILL.md (in which case they count as documented extras, not missing coverage): the surface holds only the names the entry points export, and the extraction's `internal` names never reach it. A documented extra reaches coverage-check.md §2c as a stale name. In a Python or TS/JS source, on a run where §2c runs `classify-stale` (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and whose `surface.json` came from a complete extraction, it is an Info `observation` at the line where `classify-stale` finds the source still declares it (`defined_at`, or among several declaring files the one the skill cites on a line naming it), which update-skill never routes. Otherwise, and for a name a brief scopes out (`excluded.outsideScope`), it stays a Medium `stale-documentation` gap. When the brief is what scopes the file out (a `scope.exclude` glob matches it, or no `scope.include` glob covers it), the gap names that file and line and the glob: to keep the documentation, narrow the exclude glob (removing it takes a `demoted-exclude` amendment) or widen `scope.include`; else remove that documentation.
|
|
35
35
|
|
|
36
36
|
This matches the extraction-patterns.md convention used during skill creation: coverage measures how well SKILL.md documents what users actually import, not the entire internal codebase.
|
|
37
37
|
|
|
38
38
|
### Provenance-map canonicalization
|
|
39
39
|
|
|
40
|
-
A provenance map may hold **bookkeeping variants** of one export, artifacts of how the library structures its registry (Storybook's component-plus-story decomposition, accessibility renderer shadowing, exact-match renderer disambiguation). Counting them as separate exports inflates the denominator with names that are structural duplicates of a documented base export. When a clause takes its denominator from the provenance map, pass `--fold` to `surface`: it folds `X_def`, `X_exact` and `a11y_X` back to `X`, and `<prefix>X` back to `X` for each renderer prefix you pass as `--fold-prefix` (`mobile_`, `web_`, `ssr_`, only folded when `X` is an entry too), and reports the fold summary in `canonical.summary` for the report.
|
|
40
|
+
A provenance map may hold **bookkeeping variants** of one export, artifacts of how the library structures its registry (Storybook's component-plus-story decomposition, accessibility renderer shadowing, exact-match renderer disambiguation). Counting them as separate exports inflates the denominator with names that are structural duplicates of a documented base export. When a clause takes its denominator from the provenance map, pass `--fold` to `surface`: it folds `X_def`, `X_exact` and `a11y_X` back to `X`, and `<prefix>X` back to `X` for each renderer prefix you pass as `--fold-prefix` (`mobile_`, `web_`, `ssr_`, only folded when `X` is an entry too), and reports the fold summary in `canonical.summary` for the report. Test-skill defines this fold convention: the base form is authoritative, and the variant is a sibling record.
|
|
41
41
|
|
|
42
42
|
A map whose raw count equals `stats.effective_denominator` is already canonical: `surface` folds nothing (`canonical.skipped`). A non-zero `canonical.residual` after folding (raw 222, canonical 215, `effective_denominator` 216) is an unexplained-reconciliation note; do not invent fold rules to close it. One judgment stays here: a variant the library's public surface genuinely distinguishes (an `a11y_Checkbox` documented and installed on its own) is a real export: pass it as `--keep`. Check SKILL.md for explicit documentation of the variant first; when in doubt, keep it and report both forms.
|
|
43
43
|
|