bmad-module-skill-forge 5.0.0 → 7.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/docs/_data/pinned.yaml +1 -1
- package/docs/agents.md +1 -1
- package/docs/architecture.md +3 -3
- package/docs/campaign.md +1 -1
- package/docs/troubleshooting.md +9 -3
- package/docs/verifying-a-skill.md +9 -9
- package/docs/workflows.md +7 -7
- package/package.json +1 -1
- package/src/knowledge/tool-resolution.md +6 -6
- package/src/shared/references/headless-gate-convention.md +4 -1
- package/src/shared/references/output-contract-schema.md +3 -3
- package/src/shared/references/pipeline-contracts.md +2 -1
- package/src/shared/scripts/schemas/skf-test-result-envelope.v1.json +1 -1
- package/src/shared/scripts/skf-build-change-manifest.py +332 -54
- package/src/shared/scripts/skf-classify-changed-files.py +15 -10
- package/src/shared/scripts/skf-emit-result-envelope.py +58 -15
- package/src/shared/scripts/skf-extract-public-api.py +33 -13
- package/src/shared/scripts/skf-parse-gaps.py +3 -2
- package/src/shared/scripts/skf-quick-batch.py +16 -3
- package/src/shared/scripts/skf-verify-provenance-completeness.py +317 -44
- package/src/skf-analyze-source/SKILL.md +1 -0
- package/src/skf-analyze-source/customize.toml +3 -2
- package/src/skf-analyze-source/references/auto-docs-only.md +2 -2
- package/src/skf-analyze-source/references/generate-briefs.md +5 -3
- package/src/skf-analyze-source/references/headless-contract.md +3 -3
- package/src/skf-analyze-source/references/step-auto-scope.md +2 -2
- package/src/skf-audit-skill/SKILL.md +2 -1
- package/src/skf-audit-skill/references/report.md +1 -1
- package/src/skf-brief-skill/SKILL.md +1 -0
- package/src/skf-brief-skill/references/invocation-contract.md +2 -2
- package/src/skf-campaign/SKILL.md +1 -0
- package/src/skf-campaign/references/campaign-contracts.md +3 -1
- package/src/skf-campaign/references/step-05-skill-loop.md +2 -2
- package/src/skf-campaign/references/step-07-capstone.md +3 -3
- package/src/skf-campaign/references/step-08-verify.md +3 -3
- package/src/skf-campaign/references/step-09-refine.md +3 -3
- package/src/skf-campaign/references/step-10-export.md +3 -2
- package/src/skf-campaign/scripts/campaign-state.py +22 -5
- package/src/skf-create-skill/SKILL.md +3 -2
- package/src/skf-create-skill/customize.toml +4 -4
- package/src/skf-create-skill/references/batch-mode.md +3 -1
- package/src/skf-create-skill/references/ccc-index-check.md +1 -1
- package/src/skf-create-skill/references/component-extraction.md +1 -1
- package/src/skf-create-skill/references/extract.md +1 -1
- package/src/skf-create-skill/references/generate-artifacts.md +1 -1
- package/src/skf-create-skill/references/report.md +9 -4
- package/src/skf-create-skill/references/sub/ccc-discover.md +1 -1
- package/src/skf-create-skill/references/validate.md +2 -2
- package/src/skf-create-stack-skill/SKILL.md +1 -0
- package/src/skf-create-stack-skill/references/generate-output.md +1 -1
- package/src/skf-create-stack-skill/references/invocation-contract.md +1 -1
- package/src/skf-create-stack-skill/references/parallel-extract.md +2 -2
- package/src/skf-create-stack-skill/references/report.md +1 -1
- package/src/skf-create-stack-skill/references/validate.md +12 -4
- package/src/skf-drop-skill/SKILL.md +1 -0
- package/src/skf-drop-skill/references/invocation-contract.md +1 -1
- package/src/skf-drop-skill/references/report.md +1 -1
- package/src/skf-export-skill/SKILL.md +1 -0
- package/src/skf-export-skill/references/invocation-contract.md +1 -1
- package/src/skf-export-skill/references/summary.md +1 -1
- package/src/skf-forger/SKILL.md +1 -0
- package/src/skf-forger/references/pipeline-mode.md +2 -2
- package/src/skf-quick-skill/SKILL.md +2 -1
- package/src/skf-quick-skill/references/batch-mode.md +2 -0
- package/src/skf-quick-skill/references/finalize.md +1 -1
- package/src/skf-quick-skill/references/halt-contract.md +2 -2
- package/src/skf-quick-skill/references/write-and-validate.md +20 -10
- package/src/skf-refine-architecture/SKILL.md +2 -1
- package/src/skf-refine-architecture/customize.toml +1 -1
- package/src/skf-refine-architecture/references/report.md +1 -1
- package/src/skf-rename-skill/SKILL.md +1 -0
- package/src/skf-rename-skill/references/invocation-contract.md +2 -2
- package/src/skf-setup/SKILL.md +1 -0
- package/src/skf-setup/references/ccc-index.md +1 -1
- package/src/skf-setup/references/invocation-contract.md +1 -1
- package/src/skf-test-skill/SKILL.md +1 -0
- package/src/skf-test-skill/assets/output-section-formats.md +4 -4
- package/src/skf-test-skill/references/coherence-check.md +8 -6
- package/src/skf-test-skill/references/coverage-check-branches.md +23 -3
- package/src/skf-test-skill/references/coverage-check-tiers.md +3 -3
- package/src/skf-test-skill/references/coverage-check.md +30 -43
- package/src/skf-test-skill/references/init.md +1 -1
- package/src/skf-test-skill/references/inventory-contract.md +27 -0
- package/src/skf-test-skill/references/invocation-contract.md +3 -3
- package/src/skf-test-skill/references/report.md +3 -3
- package/src/skf-test-skill/references/scoring-rules.md +7 -7
- package/src/skf-test-skill/references/signature-comparison-rules.md +74 -0
- package/src/skf-test-skill/references/source-access-protocol.md +12 -8
- package/src/skf-test-skill/references/step-hard-gate.md +1 -0
- package/src/skf-test-skill/scripts/build-result-context.py +20 -3
- package/src/skf-test-skill/scripts/gap-ledger.py +78 -13
- package/src/skf-test-skill/scripts/load-coverage-inputs.py +575 -9
- package/src/skf-test-skill/scripts/locate-export-segments.py +591 -63
- package/src/skf-test-skill/scripts/score-signatures.py +937 -76
- package/src/skf-test-skill/scripts/validate-inventory.py +325 -30
- package/src/skf-update-skill/SKILL.md +1 -0
- package/src/skf-update-skill/customize.toml +1 -1
- package/src/skf-update-skill/references/gap-driven.md +28 -22
- package/src/skf-update-skill/references/invocation-contract.md +1 -1
- package/src/skf-update-skill/references/merge.md +4 -4
- package/src/skf-update-skill/references/report.md +11 -4
- package/src/skf-update-skill/references/write.md +6 -5
- package/src/skf-verify-stack/SKILL.md +2 -1
|
@@ -3,6 +3,7 @@ nextStepFile: 'external-validators.md'
|
|
|
3
3
|
outputFile: '{report_file}'
|
|
4
4
|
outputFormatsFile: 'assets/output-section-formats.md'
|
|
5
5
|
scoringRulesFile: 'references/scoring-rules.md'
|
|
6
|
+
signatureRulesFile: 'references/signature-comparison-rules.md'
|
|
6
7
|
coherenceAggregationScript: 'scripts/aggregate-coherence.py'
|
|
7
8
|
locateExportSegmentsScript: 'scripts/locate-export-segments.py'
|
|
8
9
|
migrationSectionRules: 'references/migration-section-rules.md'
|
|
@@ -77,24 +78,24 @@ uv run {scanSkillMdStructureHelper} usage-scope "{resolved_skill_package}/SKILL.
|
|
|
77
78
|
The script decides split-body from the `references/` folder alone; whether the `## Full*` sections are stubs is your judgment, so pass `--body single` when a `references/` directory exists but those sections carry real content. Read `zero_usage[]` from the output:
|
|
78
79
|
- **Zero occurrences across the entire scope → Medium severity** finding, one per `zero_usage[]` entry: `naive-coherence: exported {kind} \`{name}\` is not referenced in any usage-family section or reference file`. This catches the "documented but unused" failure mode that trivially fails discovery testing. Like a missing export, it is a completeness gap, so it does not block the hard gate.
|
|
79
80
|
|
|
80
|
-
**2.5 Async/sync consistency.** `{locateExportSegmentsScript}` reports the facts: for each export the step 3 inventory lists with `kind` `function` or `method`, where the skill describes it (`descriptions[]`: the heading sections, table rows, list items and paragraphs that name it in code, and the signature lines that declare it) and each call to it in fenced code, with whether the call awaits it (`awaited`) and how (`awaitedVia`): `direct` when the await applies to the call itself (`directlyAwaited`), `chain` when it applies to a member, subscript, call or group around the call, `consumer` for an async loop or context manager, and `argument` or `runner` when the call sits in the arguments of an awaited call or of a runner, whose callee `enclosingCallee` names. It reads the names from the validated inventory file itself, keeping the `function` and `method` entries, so no name is copied by hand. Skip this check when the inventory has no
|
|
81
|
+
**2.5 Async/sync consistency.** `{locateExportSegmentsScript}` reports the facts: for each export the step 3 inventory lists with `kind` `function` or `method`, and each method a class, interface, struct, trait or type entry lists in `methods`, with that entry as its `owner`, where the skill describes it (`descriptions[]`: the heading sections, table rows, list items and paragraphs that name it in code, and the signature lines that declare it) and each call to it in fenced code, with whether the call awaits it (`awaited`) and how (`awaitedVia`): `direct` when the await applies to the call itself (`directlyAwaited`), `chain` when it applies to a member, subscript, call or group around the call, `consumer` for an async loop or context manager, and `argument` or `runner` when the call sits in the arguments of an awaited call or of a runner, whose callee `enclosingCallee` names. A method with an `owner` (a `method` entry that gives one, or an item of an entry's `methods`) counts only the calls on that owner or on a variable the same example binds to it (`receiverMatch` `owner` for `cognee.agents.get(...)`, `bound` for `gen.next_uuid()` after `gen = UuidGenerator()`); a call on another owner (`cognee.config.get(...)`) is not its call, and a call with no receiver or on one it cannot tie to the owner is listed in `unverifiedCalls[]` (for a method with no tied call, in `notCalledUnverified[]`). It reads the names from the validated inventory file itself, keeping the `function` and `method` entries and the `methods` of each class, interface, struct, trait or type entry, so no name is copied by hand. Skip this check when the inventory has no `function` or `method` entry and no class, interface, struct, trait or type entry with `methods`; otherwise run the script (it resolves relative to the skill root):
|
|
81
82
|
|
|
82
83
|
```bash
|
|
83
84
|
uv run {locateExportSegmentsScript} --inventory "{run_dir}/inventory.json" --skill-dir "{resolved_skill_package}"
|
|
84
85
|
```
|
|
85
86
|
|
|
86
|
-
The script judges no meaning. For each `exports[]` entry, read its `descriptions[]` and decide what they assert about **this** export. The word `async` in a description decides nothing:
|
|
87
|
+
The script judges no meaning. For each `exports[]` entry, read its `descriptions[]` and decide what they assert about **this** export (with an `owner`, the method on that owner). The word `async` in a description decides nothing:
|
|
87
88
|
|
|
88
89
|
- **async**: the caller must await what it returns (a promise, a coroutine or a future), as its signature or its description says. A Kotlin `suspend` function needs no `await`, so it makes no async claim here.
|
|
89
90
|
- **sync**: it returns its result directly, for example as the blocking counterpart of an async export.
|
|
90
|
-
- **no claim**: nothing says how it returns. A description that names a paired sync or async variant (`readFileSync` as "the sync counterpart of the async `readFile`"), or says async about another API, asserts nothing about this export.
|
|
91
|
+
- **no claim**: nothing says how it returns. A description that names a paired sync or async variant (`readFileSync` as "the sync counterpart of the async `readFile`"), or says async about another API, such as a same-named method of another owner (`config.get` for `agents.get`), asserts nothing about this export.
|
|
91
92
|
|
|
92
93
|
Then, for each export:
|
|
93
94
|
|
|
94
95
|
- **async**, no call is awaited (`awaitedCount` is 0) and no unawaited call hands its result on → **High severity** finding: `naive-coherence: \`{name}\` described as async but example lacks \`await\`` (cite the first call's `file` and `line`). A call hands its result on when its `chainedMember` waits for it (`then`, `catch`, `join`, `get`), the example returns it (`return fetchData(x)` in an async function) or stores it and awaits it later: read the call lines before raising the finding.
|
|
95
96
|
- **sync** and `directlyAwaitedCount` is above 0 → **High severity** finding: `naive-coherence: \`{name}\` described as sync but example awaits it` (cite the first directly awaited call).
|
|
96
97
|
- **sync**, `directlyAwaitedCount` is 0 and a call sits in the arguments of a call that needs awaitable arguments → the same **High severity** finding (cite the first such call's `file` and `line`): its `awaitedVia` is `runner` (a runner such as Python `asyncio.run` or `loop.run_until_complete`, Rust `block_on`), or `argument` with an `enclosingCallee` of Python `asyncio.gather`, `asyncio.wait` or `asyncio.wait_for`. JavaScript `Promise.all`, `Promise.race` and `Promise.allSettled` accept plain values, so a call in their arguments raises nothing, and neither does any other call that is `awaited` but not `directlyAwaited` (`awaitedVia` `chain`, `consumer`, or `argument` of another callee).
|
|
97
|
-
- Otherwise no finding. The names in `notCalled[]`
|
|
98
|
+
- Otherwise no finding. The names in `notCalled[]` (a method as `owner.name`) have no call an example ties to them, so they raise none. A call in `unverifiedCalls[]` or `notCalledUnverified[]` raises none either. For an export with an `owner`, its finding's title writes `{owner}.{name}` for `{name}`, and so does its ledger record's `export`.
|
|
98
99
|
|
|
99
100
|
**2.6 Table syntax.** Read `table_drift[]` from the second JSON blob (§2.0). The script normalizes escaped pipes (`\|`, used inside TypeScript union types such as `string \| undefined`) before splitting and compares each row against its header's column count, so a plain `split on |` false-positive cannot occur here. For each entry, emit a **Medium severity** finding: `naive-coherence: table row at line {entry.line} has {entry.actual_cols} columns; header has {entry.expected_cols}` (the `entry.section` and `entry.row` fields populate the detail when present).
|
|
100
101
|
|
|
@@ -144,16 +145,17 @@ SKILL.md's cross-references come in two kinds, and each has one owner:
|
|
|
144
145
|
|
|
145
146
|
The scanned references need no further check: `{run_dir}/references.json` holds their statuses.
|
|
146
147
|
|
|
147
|
-
Delegate the references §3's subagent found, in one batch, to a subagent that, for each one:
|
|
148
|
+
Delegate the references §3's subagent found, in one batch, to a subagent given the absolute path of `{signatureRulesFile}` (under `{skill-root}`), that, for each one:
|
|
148
149
|
|
|
149
150
|
1. Checks if the target exists (the skill exists in the skills output folder, the type is declared, the integration point is in the code)
|
|
150
151
|
2. If the target exists, validates the reference is accurate:
|
|
151
152
|
- Type imports: type is actually exported from the referenced module
|
|
152
153
|
- Skill references: referenced skill exists in skills output folder
|
|
153
154
|
- Integration patterns: documented pattern matches actual implementation
|
|
155
|
+
- Signatures: a documented signature matches its declaration by the rules of `{signatureRulesFile}` (`signature_match`)
|
|
154
156
|
3. Returns only a JSON array, one object per reference, with no prose, no commentary and no markdown fences: `{"reference": "...", "line": N, "target_exists": <bool>, "type_match": <bool>, "signature_match": <bool>, "issues": ["..."]}`
|
|
155
157
|
|
|
156
|
-
Save its response, as it came back, to `{run_dir}/judged-references.json` with the Write tool (no file when §3 found no such reference). If subagent unavailable, validate each reference in main thread and save the same array. §5c counts the valid references from both files, and classifies the invalid ones.
|
|
158
|
+
Save its response, as it came back, to `{run_dir}/judged-references.json` with the Write tool (no file when §3 found no such reference). If subagent unavailable, load `{signatureRulesFile}`, validate each reference in main thread by it and save the same array. §5c counts the valid references from both files, and classifies the invalid ones.
|
|
157
159
|
|
|
158
160
|
When `scripts_assets.missing` in `{run_dir}/references.json` is true (a `scripts/` or `assets/` folder beside SKILL.md, and no Scripts & Assets section), it is a Medium `scripts-assets` gap, as in naive mode §2.7; §6 records it from that file.
|
|
159
161
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Coverage Check: Branch Sections
|
|
2
2
|
|
|
3
|
-
Load this file only from the coverage-check.md sentence that names one of its sections, and follow that section alone: **Inventory Failures** from coverage-check.md §1a when the inventory check fails; **States 2 to 4** from coverage-check.md §2 when the source is not local, and only its State 2 command from coverage-check.md §2b when `rebuild` is `provenance-fold`; **Scalar and Stack Branches** from coverage-check.md §2c on the scalar or stack branch; **Denominator Candidates** from coverage-check.md §4 when `denominator.clause` is `stratified-scope` or `multi-entry
|
|
3
|
+
Load this file only from the coverage-check.md sentence that names one of its sections, and follow that section alone: **Inventory Failures** from coverage-check.md §1a when the inventory check fails; **States 2 to 4** from coverage-check.md §2 when the source is not local, and only its State 2 command from coverage-check.md §2b when `rebuild` is `provenance-fold`; **Scalar and Stack Branches** from coverage-check.md §2c on the scalar or stack branch; **Denominator Candidates** from coverage-check.md §4 when `denominator.clause` is `stratified-scope` or `multi-entry`; **Metadata Count Cross-Check** from coverage-check.md §4b; **Docs-Only Run** from coverage-check.md §0 when the run is docs-only; **Split-Body Sources** from coverage-check.md §5b when §1b found a split-body mismatch. Load it once: when another sentence names one of its sections, read that section from the copy already loaded. A bare § in a section is coverage-check.md's. coverage-check.md's frontmatter bindings, its run files, its per-file results and its §2 **Exits** apply here unchanged. When the section is done, continue in coverage-check.md right after the sentence that loaded it.
|
|
4
4
|
|
|
5
5
|
## Inventory Failures
|
|
6
6
|
|
|
@@ -10,8 +10,8 @@ Load this file only from the coverage-check.md sentence that names one of its se
|
|
|
10
10
|
rm -f "{run_dir}/inventory-response.txt"
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
Re-dispatch the §1 subagent once, with the script's `violations[]` appended to its instructions (an absent name as a name the skill does not write): it writes its new response to that file, or returns it for you to save there, as §1 says. Run the §1a command again.
|
|
14
|
-
- **Exit 1** (no input, a file that cannot be read, or a package with no readable SKILL.md): correct the file and run it again.
|
|
13
|
+
Re-dispatch the §1 subagent once, with the script's `violations[]` appended to its instructions (an absent name as a name the skill does not write, a floor name as one it must keep): it writes its new response to that file, or returns it for you to save there, as §1 says. Run the §1a command again.
|
|
14
|
+
- **Exit 1** (no input, a file that cannot be read, or a package with no readable SKILL.md): correct the file and run it again. An error that names `--floor` (the floor file missing or malformed): run §1's `floor` command again, whose exit 1 halts the run `helper-failed` as §1 says, then the §1a command.
|
|
15
15
|
|
|
16
16
|
## States 2 to 4
|
|
17
17
|
|
|
@@ -59,3 +59,23 @@ coverage-check.md §5b records this gap from the run files:
|
|
|
59
59
|
- exports-map subpath union: {subpathUnion | absent} {← chosen if the barrel branch's set}
|
|
60
60
|
- root barrel: {rootBarrel | absent} {secondary candidate: root-barrel-vs-subpath-union audit}
|
|
61
61
|
```
|
|
62
|
+
|
|
63
|
+
## Metadata Count Cross-Check
|
|
64
|
+
|
|
65
|
+
`{coherenceScript}` compares the `coverage-inputs.json` counts and owns every skip (a stack, a reference app, a cluster with fewer than two counts, drift within its threshold):
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
uv run {coherenceScript} --inputs "{run_dir}/coverage-inputs.json" > "{run_dir}/metadata-coherence.json"
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
A non-zero exit leaves the script's error, or nothing, in that file: delete it, so §5b skips its call and records no count finding, and note the error in the Coverage Analysis section. §5b records each `findings[]` entry: a Medium entry as a Medium `metadata-drift` gap and an Info entry as an Info `multi-denominator` gap. They describe data quality and change no denominator. Record the denominator of a stack (`Denominator: stack composition ({N} cited contracts)` or `({N} libraries + integration pairs)`, from `stack.basis`) and of a reference app (`Denominator: pattern-surface ({pattern_surfaces_documented})`).
|
|
72
|
+
|
|
73
|
+
## Docs-Only Run
|
|
74
|
+
|
|
75
|
+
**A docs-only run** measures documentation completeness as its Export Coverage, with `analysis_confidence` `docs-only`. It runs only §1, §1a and §1b, the docs-only guard of §2b, §2c's `docsOnly` branch, §4's `Denominator: docs-only completeness` annotation, §5, §5b and §6. Step 5 passes `docsOnly: true` to the scoring script, which skips Signature Accuracy and Type Coverage at any tier.
|
|
76
|
+
|
|
77
|
+
"**Docs-only skill detected.** Coverage check evaluates documentation completeness rather than source code coverage."
|
|
78
|
+
|
|
79
|
+
## Split-Body Sources
|
|
80
|
+
|
|
81
|
+
The Source is `{reference_file}:{reference_line}`, of two references the one that sorts last, unless the pinned source settles it for the other side. At State 1 and the Forge tier or above, with `workspaceDrift` not `overridden`, load `{signatureRulesFile}` and set the two forms its `issue` gives against the export's declaration at the `file` and `line` `surface.json` gives, by its rules, when that one declaration decides (no overload set, no second declaring file): when `{reference_file}`'s form agrees and the other side's does not, the Source is the other side, `SKILL.md:{skill_md_line}` or `{other_file}:{other_line}`. Its remediation names the file to edit and the file and line whose form it takes, citing the declaration's `file:line`; when both forms agree with it, it says so and names what differs; when neither does, or nothing decides, it says no source settled it and names `{reference_file}` to edit, taking the other side's form (`{other_file}`'s, for two references).
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Coverage Check: Tier Branches
|
|
2
2
|
|
|
3
|
-
Load this file only from coverage-check.md §2, and follow the one section §2 names: **Quick Tier** at Quick tier, **Fallback Per-File Scan** when the Forge-tier recipe runner cannot be used. coverage-check.md's frontmatter bindings, its run files, its per-file results and its §2 **Exits** apply here unchanged. When the section is done, continue in coverage-check.md §2 after the tier that loaded it (at Deep tier, its
|
|
3
|
+
Load this file only from coverage-check.md §2, and follow the one section §2 names: **Quick Tier** at Quick tier, **Fallback Per-File Scan** when the Forge-tier recipe runner cannot be used. coverage-check.md's frontmatter bindings, its run files, its per-file results and its §2 **Exits** apply here unchanged. When the section is done, continue in coverage-check.md §2 after the tier that loaded it (at Deep tier, its **Deep Tier** paragraph, which runs no check of its own but records what no script checks; otherwise States 2 to 4 and then §2b).
|
|
4
4
|
|
|
5
5
|
## Quick Tier
|
|
6
6
|
|
|
@@ -20,13 +20,13 @@ Load this file only from coverage-check.md §2, and follow the one section §2 n
|
|
|
20
20
|
|
|
21
21
|
## Fallback Per-File Scan
|
|
22
22
|
|
|
23
|
-
The recipe runner cannot be used, so subagents read the files in scope one at a time. List every documented signature first: `plan` without `--surface` writes the map as `documentedSignatures`:
|
|
23
|
+
The recipe runner cannot be used, so subagents read the files in scope one at a time. List every documented signature first: `plan` without `--surface` writes the map as `documentedSignatures`, and each name's other documented forms as `documentedAlternatives`:
|
|
24
24
|
|
|
25
25
|
```bash
|
|
26
26
|
uv run {scoreSignaturesScript} plan --inventory "{run_dir}/inventory.json" --output "{run_dir}/signature-plan.json"
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
For each source file that defines public API exports, delegate a subagent that gets that map with the file. It extracts the symbols the file exports, with their full signatures, with ast-grep (or reads the file when `extraction.fallback.reason` says no ast-grep can run; list those files in the Coverage Analysis section as read by eye), leaves out a name the file only passes on from another module of the source being tested (`export { x } from './x'`, `export * from`, an `__init__.py` import, a Rust `pub use`), which that module's own result counts, keeps a name it passes on from another package (`export { z } from 'zod'`), which no other result counts, compares each with the map's entry (
|
|
29
|
+
For each source file that defines public API exports, delegate a subagent that gets that map with the file and the absolute path of `{signatureRulesFile}` (under `{skill-root}`). It gets the `documentedAlternatives` map too, whose forms it compares as that file says. It extracts the symbols the file exports, with their full signatures, with ast-grep (or reads the file when `extraction.fallback.reason` says no ast-grep can run; list those files in the Coverage Analysis section as read by eye), leaves out a name the file only passes on from another module of the source being tested (`export { x } from './x'`, `export * from`, an `__init__.py` import, a Rust `pub use`), which that module's own result counts, keeps a name it passes on from another package (`export { z } from 'zod'`), which no other result counts, compares each with the map's entry by the rules of that file (overload sets and `difference` tags included), gives each mismatch the `line` that defines the export, and returns only `{"file", "exports_found": [...], "types_found": [<the exported interfaces, type aliases, enums and classes>], "signature_mismatches": [...]}`, each mismatch in the shape §2's signature comparison returns. Save each response to `{run_dir}/per-file-<n>.txt` with the Write tool and build the surface from them, one `--per-file` each and `--fallback-scan` (each names the file that defines its exports, so a `scope.tier_a_include` list that counts none of them does not turn the guards off), with the same baselines as at Quick tier, and `--extraction` when the runner's JSON sent you here (`extraction.fallback.needed`): it adds no name, only its scope and the globs it lists as matching no file, so the guards still run:
|
|
30
30
|
|
|
31
31
|
```bash
|
|
32
32
|
uv run {coverageInputsScript} surface --per-file "{run_dir}/per-file-<n>.txt" --fallback-scan [--extraction "{run_dir}/extract-full.json"] [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] [--monorepo] [--pattern-reference-intent] [--curated-subset] [--barrel <state>] --output "{run_dir}/surface.json"
|
|
@@ -2,6 +2,8 @@
|
|
|
2
2
|
nextStepFile: 'coherence-check.md'
|
|
3
3
|
outputFile: '{report_file}'
|
|
4
4
|
scoringRulesFile: 'references/scoring-rules.md'
|
|
5
|
+
signatureRulesFile: 'references/signature-comparison-rules.md'
|
|
6
|
+
inventoryContractFile: 'references/inventory-contract.md'
|
|
5
7
|
sourceAccessProtocol: 'references/source-access-protocol.md'
|
|
6
8
|
coverageTiersFile: 'references/coverage-check-tiers.md'
|
|
7
9
|
coverageBranchesFile: 'references/coverage-check-branches.md'
|
|
@@ -56,9 +58,7 @@ uv run {coverageInputsScript} census --skill-dir "{resolved_skill_package}" --ou
|
|
|
56
58
|
|
|
57
59
|
Bind `{docs_only_mode}` ← `docsOnly`. Exit 1 printed its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:census`). §0b sets `{docs_only_mode}` too when source access reaches State 5.
|
|
58
60
|
|
|
59
|
-
**A docs-only run
|
|
60
|
-
|
|
61
|
-
"**Docs-only skill detected.** Coverage check evaluates documentation completeness rather than source code coverage."
|
|
61
|
+
**A docs-only run**: load `{coverageBranchesFile}` and follow its **Docs-Only Run** section.
|
|
62
62
|
|
|
63
63
|
A source-based skill continues with §0b.
|
|
64
64
|
|
|
@@ -78,41 +78,32 @@ A workspace whose members `scope.exclude` drops is no workspace. Bind `{source_i
|
|
|
78
78
|
|
|
79
79
|
### 1. Extract Documented Exports from SKILL.md
|
|
80
80
|
|
|
81
|
-
|
|
81
|
+
First write the inventory's floor, the names the skill documents in a fixed form, which the inventory must keep:
|
|
82
82
|
|
|
83
|
-
```
|
|
84
|
-
{
|
|
85
|
-
"exports": [
|
|
86
|
-
{"name": "functionName", "kind": "function", "params": "...", "return_type": "...", "description": "..."},
|
|
87
|
-
{"name": "ClassName", "kind": "class", "methods": ["..."], "properties": ["..."], "description": "..."},
|
|
88
|
-
{"name": "TypeName", "kind": "type", "fields": ["..."], "description": "..."},
|
|
89
|
-
{"name": "useHook", "kind": "hook", "usage_signature": "...", "description": "...", "signature_form": "partial"}
|
|
90
|
-
],
|
|
91
|
-
"references": ["references/api-reference.md"],
|
|
92
|
-
"cross_check_mismatches": [
|
|
93
|
-
{"export": "functionName", "skill_md_line": 42, "reference_file": "references/api-reference.md", "reference_line": 18, "issue": "SKILL.md shows (date: Date) => string, the reference (date: Date, format?: string) => string"}
|
|
94
|
-
]
|
|
95
|
-
}
|
|
83
|
+
```bash
|
|
84
|
+
uv run {coverageInputsScript} floor --skill-dir "{resolved_skill_package}" --output "{run_dir}/inventory-floor.json"
|
|
96
85
|
```
|
|
97
86
|
|
|
98
|
-
|
|
87
|
+
Exit 1 printed its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:inventory`).
|
|
99
88
|
|
|
100
|
-
|
|
89
|
+
Delegate the reading of the skill under test to a subagent, given `{run_dir}` and the path to SKILL.md and to its `references/` directory when one exists: it reads SKILL.md, and every `references/*.md` file too when SKILL.md's `## Full` headings are absent or stubs. It is given `{run_dir}/inventory-floor.json` too, and lists each name there with an origin in a file it reads, choosing its kind from the name's `kindLabel`, the skill's own label, when it has one. An origin's `signature` is that place's entry, its non-blank `params` and `return_type` kept as given; a blank one is the reader's. For each export two or more of the files it reads document, it compares the SKILL.md body with each reference file, and two reference files with each other only when the body does not settle them (it does not document the export, or agrees with both), by the rules of `{signatureRulesFile}`, whose absolute path (under `{skill-root}`) it is given: parameters, the return type, and descriptions that contradict (brief against detailed is fine). That file alone says what a form may leave out. It writes only this JSON, with no prose and no commentary, to `{run_dir}/inventory-response.txt` with the Write tool, never a shell, and returns that path, or returns the JSON itself when it cannot write files. The JSON's shape and the contract the validator checks are in `{inventoryContractFile}`, whose absolute path (under `{skill-root}`) the subagent is given.
|
|
90
|
+
|
|
91
|
+
When the subagent returned the JSON rather than the path, or the file is absent or empty afterwards, save its response, as it came back, to `{run_dir}/inventory-response.txt` with the Write tool. The documented inventory is the one §1a validates: do not load SKILL.md or the reference files into the parent context. Without subagents, the parent loads `{inventoryContractFile}` and builds the same JSON from the same files and `inventory-floor.json` and saves it to that file; §1a's checks still run on it.
|
|
101
92
|
|
|
102
93
|
#### 1a. Parent-Side Schema Validation + Spot-Check
|
|
103
94
|
|
|
104
|
-
Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped)
|
|
95
|
+
Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped), spot-checks sampled names against SKILL.md and the subagent's listed reference files, and refuses it when it leaves out a floor name those files write, or an origin's `signature`. The look-up is the script's, never a grep: an export named `$state` or `a.b` is neither a shell variable nor a regex.
|
|
105
96
|
|
|
106
97
|
```bash
|
|
107
|
-
uv run {validateInventoryScript} --input "{run_dir}/inventory-response.txt" --skill-package "{resolved_skill_package}" --output "{run_dir}/inventory.json"
|
|
98
|
+
uv run {validateInventoryScript} --input "{run_dir}/inventory-response.txt" --skill-package "{resolved_skill_package}" --output "{run_dir}/inventory.json" --floor "{run_dir}/inventory-floor.json"
|
|
108
99
|
```
|
|
109
100
|
|
|
110
101
|
- `valid` true (exit 0): every later command reads `{run_dir}/inventory.json`.
|
|
111
|
-
- `valid` false (exit 2), or exit 1: load `{coverageBranchesFile}` and follow its **Inventory Failures** section. Invalid a second time: HALT (`halt_reason: "inventory-invalid"`, phase `coverage-check:inventory`): "coverage-check: the subagent inventory failed {schema validation | the ground-truth spot-check} twice: {the violations joined}."
|
|
102
|
+
- `valid` false (exit 2), or exit 1: load `{coverageBranchesFile}` and follow its **Inventory Failures** section. Invalid a second time: HALT (`halt_reason: "inventory-invalid"`, phase `coverage-check:inventory`): "coverage-check: the subagent inventory failed {schema validation | the ground-truth spot-check | the floor check} twice: {the violations joined}."
|
|
112
103
|
|
|
113
104
|
### 1b. Cross-Check Split-Body Consistency
|
|
114
105
|
|
|
115
|
-
Only when the validated inventory's `references` is non-empty. Each `cross_check_mismatches` entry is a High `split-body-mismatch` gap. §5b records it titled `Split-body mismatch: {export} ({reference_file})`, with its `export` and its Source at the side the source contradicts (
|
|
106
|
+
Only when the validated inventory's `references` is non-empty. Each `cross_check_mismatches` entry is a High `split-body-mismatch` gap. §5b records it titled `Split-body mismatch: {export} ({reference_file})`, with its `export` and its Source at the side the source contradicts (`{reference_file}` when no source settles it): update-skill edits that file to match the other without reading the source. An entry for two references the body does not settle is titled `Split-body mismatch: {export} ({other_file}, {reference_file})`.
|
|
116
107
|
|
|
117
108
|
### 2. Analyze Source Code (Tier-Dependent)
|
|
118
109
|
|
|
@@ -138,7 +129,7 @@ Build the surface from it, with the metadata, the brief when the skill has one a
|
|
|
138
129
|
uv run {coverageInputsScript} surface --extraction "{run_dir}/extract-full.json" --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] [--monorepo] [--pattern-reference-intent] [--curated-subset] --output "{run_dir}/surface.json"
|
|
139
130
|
```
|
|
140
131
|
|
|
141
|
-
The surface holds
|
|
132
|
+
The surface holds what `{sourceAccessProtocol}` **What the Surface Holds** says (its exclusions and the stale-scope guard). §5's render lists `excluded.outsideScope`, `excluded.nestedEntries` and the restored names. Decide what to do from the JSON, not from the exit code:
|
|
142
133
|
|
|
143
134
|
- `extraction.fallback.needed` is true, or the runner printed no JSON on an exit other than 2: load `{coverageTiersFile}` and run its **Fallback Per-File Scan** in place of the rest of this tier.
|
|
144
135
|
- `extraction.readByEye` is non-empty (a file the run could not parse or decode, or a failed call left unread): keep the surface, read by eye only the files it names, save a per-file result for each, and run `surface` again with them, once (the second call starts no new pass).
|
|
@@ -151,19 +142,21 @@ The surface holds the names the entry points export, less those defined in a fil
|
|
|
151
142
|
uv run {scoreSignaturesScript} plan --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --skill-dir "{resolved_skill_package}" --output "{run_dir}/signature-plan.json"
|
|
152
143
|
```
|
|
153
144
|
|
|
154
|
-
For each `files[]` entry, delegate one subagent with its checks. For each check it reads the full declaration at `{file}:{line}` (all of it: `signatureLine` is one physical line, and a multi-line signature cut there reads as a mismatch), compares it with the check's `documented` signature
|
|
145
|
+
For each `files[]` entry, delegate one subagent with its checks and the absolute path of `{signatureRulesFile}` (under `{skill-root}`). For each check it reads the full declaration at `{file}:{line}` (all of it: `signatureLine` is one physical line, and a multi-line signature cut there reads as a mismatch), compares it with the check's `documented` signature by the rules of that file, and returns only this JSON, with no prose and no markdown fences:
|
|
155
146
|
|
|
156
147
|
```json
|
|
157
148
|
{"file": "src/utils.ts", "signature_mismatches": [{"name": "formatDate", "line": 42, "source_sig": "(date: Date, format?: string) => string", "documented_sig": "(date: Date) => string", "issue": "missing optional parameter 'format'", "difference": "omitted-optional"}]}
|
|
158
149
|
```
|
|
159
150
|
|
|
160
|
-
|
|
151
|
+
Tag a mismatch's `difference` as `{signatureRulesFile}` says: `score` (§2b) checks the facts. One tagged `annotation-contradicted` also gives `evidence_line`, the body line that returns the documented type.
|
|
152
|
+
|
|
153
|
+
A check with `candidates` reads each and reports a mismatch only when the documented signature matches none: one, at the closest candidate, its `file` and `line` on the mismatch object (the result's `file` stays its `files[]` entry) and its `difference` tagged against it.
|
|
161
154
|
|
|
162
|
-
A check with `
|
|
155
|
+
A check with `alternatives` (the other forms the skill documents for the name) is compared with each of them too, as `{signatureRulesFile}` says, each mismatch naming its form as `form`.
|
|
163
156
|
|
|
164
|
-
Save each response to `{run_dir}/signatures-<n>.txt` with the Write tool. Without subagents, compare in the main thread and save the same JSON.
|
|
157
|
+
Save each response to `{run_dir}/signatures-<n>.txt` with the Write tool. Without subagents, load `{signatureRulesFile}`, compare in the main thread by its rules and save the same JSON.
|
|
165
158
|
|
|
166
|
-
**Deep Tier (ast-grep + gh + QMD):** the Forge tier checks,
|
|
159
|
+
**Deep Tier (ast-grep + gh + QMD):** the Forge tier checks, and no coverage check of its own: init.md §5b's drift guard checks the local tree against the pinned commit, at any tier, when both exist; the recipe runner traces each re-export to the file that defines it when it runs (the fallback per-file scan does not); and the Forge tier's signature check compares each documented signature with its declaration. A documentation error you see that no script records (an import path whose module does not export the name, a type member that differs from its declaration) goes in coverage-check §5b's hand-written records as an Info `observation` titled `Not checked by script: {name}`, with `export` `{name}`, its Source at the documenting line and the remediation "Correct the documentation to match the source; no script checks this."
|
|
167
160
|
|
|
168
161
|
**States 2 to 4 (no local source):** the surface is what the skill recorded, so there is no signature to score: load `{coverageBranchesFile}` and follow its **States 2 to 4** section.
|
|
169
162
|
|
|
@@ -201,13 +194,13 @@ Export Coverage is undefined, and scoring would yield a vacuous PASS. Fix one of
|
|
|
201
194
|
|
|
202
195
|
Do not write the Coverage Analysis section or attach a score: this is an indeterminate state, not a FAIL. **If `docs_only_mode == true` and the validated inventory documents no item:** HALT the same way with "docs-only skill declares zero items: no API surface to test".
|
|
203
196
|
|
|
204
|
-
**Score the signatures** (Forge, Forge+ and Deep tier at State 1, once §2 compared them). Give `{scoreSignaturesScript}` `denominator.surfaceSet` as `--surface-set`. Type Coverage then counts that set's types alone, so a type the clause leaves out (a stratified skill's deferred Tier B) lowers neither Export Coverage nor Type Coverage. Give it one `--results` per saved response (the fallback scan's `per-file-<n>.txt`, else the `signatures-<n>.txt`):
|
|
197
|
+
**Score the signatures** (Forge, Forge+ and Deep tier at State 1, once §2 compared them). Give `{scoreSignaturesScript}` `denominator.surfaceSet` as `--surface-set`. Type Coverage then counts that set's types alone, so a type the clause leaves out (a stratified skill's deferred Tier B) lowers neither Export Coverage nor Type Coverage. Give it one `--results` per saved response (the fallback scan's `per-file-<n>.txt`, else the `signatures-<n>.txt`), and `--source-root` whenever the source is local (State 1, Forge tier or above), under a drift override too, since the judges read that tree:
|
|
205
198
|
|
|
206
199
|
```bash
|
|
207
|
-
uv run {scoreSignaturesScript} score --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --plan "{run_dir}/signature-plan.json" --results "{run_dir}/signatures-<n>.txt" --output "{run_dir}/signatures.json" --gaps-output "{run_dir}/signature-gaps.json"
|
|
200
|
+
uv run {scoreSignaturesScript} score --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --plan "{run_dir}/signature-plan.json" [--source-root "{source_path}"] --results "{run_dir}/signatures-<n>.txt" --output "{run_dir}/signatures.json" --gaps-output "{run_dir}/signature-gaps.json"
|
|
208
201
|
```
|
|
209
202
|
|
|
210
|
-
It grades each mismatch of a compared name from facts it checks, never a `difference` alone, into `signature-gaps.json`: a Low `signature-mismatch` gap for a `parameter-name` difference in a file the plan offered when it is TS/JS, Go, Rust, Java, C or C++, an Info `observation` titled `Partial signature: {name}` for an `omitted-optional` one on a `partial` form, a Critical `signature-mismatch` gap otherwise (Python included). A signature gap is titled `Signature mismatch: {name}`, with `export` `{name}`, its Source at `{file}:{line}` and both signatures in its Issue; §5b appends that file to the ledger. A Low or Info one counts as matching, and §5 renders it so. §5's render lists its `warnings` (a mismatch for a name never compared, a category with nothing to compare).
|
|
203
|
+
It grades each mismatch of a compared name from facts it checks, never a `difference` alone, into `signature-gaps.json`: a Low `signature-mismatch` gap for a `parameter-name` difference in a file the plan offered when it is TS/JS, Go, Rust, Java, C or C++, an Info `observation` titled `Partial signature: {name}` for an `omitted-optional` one on a `partial` form, an Info `observation` titled `Annotation contradicted: {name}` (no change to the skill: report the annotation upstream) for an `annotation-contradicted` one whose facts check (a documented return type, the same parameters on both sides, and an `evidence_line` between the declaration and the next one, whose line returns in the `--source-root` file), a Critical `signature-mismatch` gap otherwise (Python included). A signature gap is titled `Signature mismatch: {name}`, with `export` `{name}`, its Source at `{file}:{line}` and both signatures in its Issue (for a name documented in several forms, each gap's title ends `(form {n} of {m})` and its Issue names that form); §5b appends that file to the ledger. A Low or Info one counts as matching, and §5 renders it so. §5's render lists its `warnings` (a mismatch for a name never compared, a category with nothing to compare).
|
|
211
204
|
|
|
212
205
|
### 2c. Reconcile Documented vs Source Surface (Deterministic Intersection)
|
|
213
206
|
|
|
@@ -248,13 +241,7 @@ The guards raise coverage gaps too, which §5b records from `surface.json`: `gua
|
|
|
248
241
|
|
|
249
242
|
### 4b. Metadata Export-Count Coherence Cross-Check
|
|
250
243
|
|
|
251
|
-
`{
|
|
252
|
-
|
|
253
|
-
```bash
|
|
254
|
-
uv run {coherenceScript} --inputs "{run_dir}/coverage-inputs.json" > "{run_dir}/metadata-coherence.json"
|
|
255
|
-
```
|
|
256
|
-
|
|
257
|
-
A non-zero exit leaves the script's error, or nothing, in that file: delete it, so §5b skips its call and records no count finding, and note the error in the Coverage Analysis section. §5b records each `findings[]` entry: a Medium entry as a Medium `metadata-drift` gap and an Info entry as an Info `multi-denominator` gap. They describe data quality and change no denominator. Record the denominator of a stack (`Denominator: stack composition ({N} cited contracts)` or `({N} libraries + integration pairs)`, from `stack.basis`) and of a reference app (`Denominator: pattern-surface ({pattern_surfaces_documented})`).
|
|
244
|
+
Load `{coverageBranchesFile}` and follow its **Metadata Count Cross-Check** section.
|
|
258
245
|
|
|
259
246
|
**The numerator ground truth §2b ran.** `numerator.json` `inflated: true` (`verified < declared`) is a High `numerator-inflation` gap listing its `absent[]` names, which §5b records; on a scalar branch that counts no set those names also replace the missing-count gap in the Scalar and Stack Branches section of coverage-check-branches.md. `inflated: false` or `skipped: true` is no finding.
|
|
260
247
|
|
|
@@ -272,9 +259,9 @@ uv run {verifyProvenanceCompletenessHelper} verify \
|
|
|
272
259
|
-o "{run_dir}/provenance-verify.json"
|
|
273
260
|
```
|
|
274
261
|
|
|
275
|
-
Rely on the file, not the exit code (exit 1 means findings); on exit 2 the helper writes no file: skip to section 5. §5b records each `stale[]` item
|
|
262
|
+
Rely on the file, not the exit code (exit 1 means findings); on exit 2 the helper writes no file: skip to section 5. §5b records each `stale[]` item by its `reason`: `line-not-definition` is a Low `provenance-line` gap when its `definition_lines` holds a line, else an Info `provenance-unverified` gap (the rules may not cover its shape: the export is unverified, not gone); `file-missing` (the pinned source lacks the file) and `line-out-of-bounds` (the line is past its end, or below 1) are Low `provenance-line` gaps; `line-invalid` (not a line number) is an Info `provenance-unverified` gap. A bookkeeping variant (`X_def`, `X_exact`, `a11y_X`) the cited file never names is checked against `X`. Its Source is exactly the `file:line` the map records (the file alone for `line-invalid`), the citation update-skill's rule R5 reads. A name §2c's `fabricated-signature` gap covers takes no such gap.
|
|
276
263
|
|
|
277
|
-
This section reports none of the helper's
|
|
264
|
+
This section reports none of the helper's `missing`, `orphaned` or `citations` findings, and no gap it records blocks the gate.
|
|
278
265
|
|
|
279
266
|
### 5. Write the Coverage Analysis Section
|
|
280
267
|
|
|
@@ -314,9 +301,9 @@ uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check -
|
|
|
314
301
|
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from provenance-line --input "{run_dir}/provenance-verify.json"
|
|
315
302
|
```
|
|
316
303
|
|
|
317
|
-
**A split-body mismatch's Source** is a line inside the skill package
|
|
304
|
+
**A split-body mismatch's Source** is a line inside the skill package: when §1b found one, load `{coverageBranchesFile}` and follow its **Split-Body Sources** section.
|
|
318
305
|
|
|
319
|
-
Then write the records no script wrote, §1b's split-body mismatches, in the Ledger Record Format of `{outputFormatsFile}
|
|
306
|
+
Then write the records no script wrote, §1b's split-body mismatches, in the Ledger Record Format of `{outputFormatsFile}` (at the Deep tier, §2's `Not checked by script: {name}` observations too), each naming its export in `export` and in its title: the ledger tells two gaps apart by their title, Source and `export`, and update-skill takes the export's name from `export`. Write them as one JSON array on the lines between the two markers, exactly as they are: the quoted marker hands them to the script unchanged, quotes, apostrophes and `$` included. Run the command even when the array is empty (`[]`): the hard gate refuses to decide until every stage before it has recorded, with gaps or without.
|
|
320
307
|
|
|
321
308
|
```bash
|
|
322
309
|
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check <<'SKF_GAPS'
|
|
@@ -320,7 +320,7 @@ nextWorkflow: ''
|
|
|
320
320
|
---
|
|
321
321
|
```
|
|
322
322
|
|
|
323
|
-
Replace the template's frontmatter with this block (the two hold the same keys). `toolingStatus` is the `tooling_status` §3b set: score.md §3a reads it back for Cap 1. `threshold` and `thresholdSource` are the pass threshold §1b resolved and its source: score.md §1 reads them back, and score.md §7 rewrites `threshold` with the threshold the verdict was settled against. `analysisConfidence` stays empty until score.md §7 writes the source access coverage-check resolved: `full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`.
|
|
323
|
+
Replace the template's frontmatter with this block (the two hold the same keys). `toolingStatus` is the `tooling_status` §3b set: score.md §3a reads it back for Cap 1. `threshold` and `thresholdSource` are the pass threshold §1b resolved and its source: score.md §1 reads them back, and score.md §7 rewrites `threshold` with the threshold the verdict was settled against. `analysisConfidence` stays empty until score.md §7, or step-hard-gate.md §3 on a run the hard gate blocks, writes the source access coverage-check resolved: `full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`. report.md §4c halts on a report that holds none of them.
|
|
324
324
|
|
|
325
325
|
### 7. Report Initialization Status
|
|
326
326
|
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Documented Inventory Contract
|
|
2
|
+
|
|
3
|
+
The JSON coverage-check.md §1's reader writes to `{run_dir}/inventory-response.txt`: every export the skill documents, in this shape.
|
|
4
|
+
|
|
5
|
+
```json
|
|
6
|
+
{
|
|
7
|
+
"exports": [
|
|
8
|
+
{"name": "functionName", "kind": "function", "params": "...", "return_type": "...", "description": "..."},
|
|
9
|
+
{"name": "ClassName", "kind": "class", "methods": ["..."], "properties": ["..."], "description": "..."},
|
|
10
|
+
{"name": "TypeName", "kind": "type", "fields": ["..."], "description": "..."},
|
|
11
|
+
{"name": "useHook", "kind": "hook", "usage_signature": "...", "description": "...", "signature_form": "partial"},
|
|
12
|
+
{"name": "ComponentName", "kind": "constant", "usage_signature": "<ComponentName prop={value} />", "description": "..."},
|
|
13
|
+
{"name": "Card", "kind": "function", "params": "{ title }", "description": "..."},
|
|
14
|
+
{"name": "Card", "kind": "function", "params": "{ title: string; footer?: ReactNode }", "description": "..."},
|
|
15
|
+
{"name": "Note", "kind": "class", "params": "each with the base fields id, scope=Scope(); content, title=None", "description": "..."}
|
|
16
|
+
],
|
|
17
|
+
"references": ["references/api-reference.md"],
|
|
18
|
+
"cross_check_mismatches": [
|
|
19
|
+
{"export": "functionName", "skill_md_line": 42, "reference_file": "references/api-reference.md", "reference_line": 18, "issue": "SKILL.md shows (date: Date) => string, references/api-reference.md (date: Date, format?: string) => string"},
|
|
20
|
+
{"export": "runTasks", "other_file": "references/api-reference.md", "other_line": 40, "reference_file": "references/pipelines.md", "reference_line": 12, "issue": "references/api-reference.md shows (tasks: Task[], data: unknown) => void, references/pipelines.md (tasks: Task[]) => void"}
|
|
21
|
+
]
|
|
22
|
+
}
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
The contract `validate-inventory.py` checks (coverage-check.md §1a): each `kind` is one of `function`, `class`, `type`, `constant`, `hook`, `interface`, `method`, `struct`, `enum`, `trait`, `macro` or `adapter`; every entry carries the `description` the skill gives it (an empty string when it gives none); a function or method carries `params` (`""` or `[]` when it takes none) and `return_type` as the skill documents them, and leaves a key out when the skill documents nothing for it. A name written with empty parentheses in running text or a description cell is a mention, not a form: its entry leaves `params` and `usage_signature` out (a cell reading "`instrument()` wraps functions"). A description cell is any table cell but an export table's first cell and a cell whose column writes the call or signature, by its role (Signature, Props, Params, Parameters, Arguments, Usage, Syntax, Example and the like); a heading or an export table's first cell that writes `name()` is a form, recorded as written. A `method`'s `name` is the bare method name and its `owner` the class or object it is called on (`agents` for `cognee.agents.get`, where the skill documents `agents` as a class; the class for one reached through a property, `client.messages`); `owner` is not validated. A function the skill documents in a module is a `function` named bare, never a `method`, even when it is called through the module's name (`register_sql_connection` for `cognee.tools.register_sql_connection`, where the skill documents `tools` as a module). Each `methods` item starts with the method's name, optionally after `async` (`async next_uuid(dep=None)`). A `const` holding a component (a `forwardRef`, `memo` or other wrapper call, or an alias like `const Tabs = TabsPrimitive.Root`) is a `constant` with the declaration or usage form the skill writes as `usage_signature`; a data constant carries no `usage_signature`, `params` or `return_type`. A name documented in several places is listed once per place, each entry with that place's form (`Card` above), and keeps one `kind` in every place. A place `inventory-floor.json` gives a `signature` (a cited function declaration, a Signature cell, or a heading or an export table's first cell that writes one call) has that form as its entry, whatever its kind, its non-blank `params` and `return_type` kept as given (a `params` may open with the shared text the next sentence gives, then `; `); a blank one is the reader's, and a form the skill writes only in prose is an entry of its own. Where the skill documents several names and states members they share once, in the same place (a table row, a paragraph) or over a table for all its rows (a sentence or caption just above it, or its column header), each name's entry carries that shared text in its `params` (in its `usage_signature`, for a usage form), never in a `methods` item, which still starts with the method's name: the shared text first, then `; `, then the name's own members as the skill writes them, without the name, never the name's own members alone (`Note` above, from a row that documents `Note` and `Task`, each with the base fields `id, scope=Scope()`, as `Note(content, title=None)` and `Task(title, done=False)`). An entry whose signature the skill labels partial (simplified, key parameters, a `...` standing in for omitted members, not rest or variadic syntax; a label in a table's column header, such as `Props (simplified)`, labels each of its rows) carries `signature_form: "partial"`, any other `"full"` or no key: copy the label, never judge the form.
|
|
26
|
+
|
|
27
|
+
A `cross_check_mismatches` entry gives its `export`, `reference_file`, `reference_line` and an `issue` that names each form by its file's path, and the pair's other side: the SKILL.md body's line as `skill_md_line`, or, for two references, the one that sorts first in codepoint order, by path and then line, as `other_file` and `other_line`, with the one that sorts last as `reference_file` and `reference_line`. Each file is its path in the package, such as `references/api-reference.md`.
|
|
@@ -15,7 +15,7 @@ The full argument set, gate map, outputs, exit codes and `SKF_TEST_RESULT_JSON`
|
|
|
15
15
|
|
|
16
16
|
## Exit Codes
|
|
17
17
|
|
|
18
|
-
Every terminal state settles one exit code, so a headless automator branches on the verdict, and on any HARD HALT, without grepping message text. A skill run cannot set the process exit status of the agent that runs it (`claude -p` exits 0 whatever the run decided), so automators read `exit_code` from the `SKF_TEST_RESULT_JSON` line or
|
|
18
|
+
Every terminal state settles one exit code, so a headless automator branches on the verdict, and on any HARD HALT, without grepping message text. A skill run cannot set the process exit status of the agent that runs it (`claude -p` exits 0 whatever the run decided), so automators read `exit_code` from the run's own `SKF_TEST_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and `skf-test-skill-result-latest.json` may then describe an earlier run, so it is never read in its place.
|
|
19
19
|
|
|
20
20
|
| Code | Meaning | Raised by |
|
|
21
21
|
| ---- | -------------------- | ------------------------------------------------------------------------------------------ |
|
|
@@ -48,7 +48,7 @@ The emitter prints a finished run's line on stdout and a halt's or a blocked run
|
|
|
48
48
|
| `forge-tier-missing` | init.md §4 |
|
|
49
49
|
| `workspace-drift` | init.md §5b |
|
|
50
50
|
| `another-run-active` | init.md §6b, and the lock renewal in report.md §4c |
|
|
51
|
-
| `inventory-invalid` | coverage-check.md §1a: the documented inventory failed its schema check
|
|
51
|
+
| `inventory-invalid` | coverage-check.md §1a: the documented inventory failed its schema check, spot-check or floor check twice |
|
|
52
52
|
| `indeterminate-surface` | coverage-check.md §2b: zero exports, an empty stack surface or a docs-only skill that documents nothing, so no score can be attached |
|
|
53
53
|
| `step-completeness-violation` | step 4c §2 (a stage recorded nothing in the gap ledger) and report.md §4c |
|
|
54
54
|
| `report-anchor-missing` | report.md §4c |
|
|
@@ -59,7 +59,7 @@ When the emitter itself is not installed (init.md §0), no line can be printed:
|
|
|
59
59
|
|
|
60
60
|
## Result Files
|
|
61
61
|
|
|
62
|
-
report.md §4c runs the emitter with `--result-dir "{forge_version}"`. It writes the per-run record `skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name, `-2`, `-3` appended when a run already took that second's name) and then its copy `skf-test-skill-result-latest.json`, the
|
|
62
|
+
report.md §4c runs the emitter with `--result-dir "{forge_version}"`. It writes the per-run record `skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name, `-2`, `-3` appended when a run already took that second's name) and then its copy `skf-test-skill-result-latest.json`, the record of the last run whose emitter wrote this copy, at a stable path (a copy, not a symlink), each atomically. The record holds the envelope's fields, plus `skill`, `outputs[]` (the published report), `summary` (`score`, `threshold`, `result` (`PASS`, `PASS_WITH_DRIFT`, `FAIL` or `INCONCLUSIVE`), `testMode`, `activeCategories[]`, `inconclusiveReasons[]` when present, `threshold_fallback`, `original_threshold` and `evidence_report_path` when the threshold fallback fired, `gapCounts` with the gap ledger's gaps by severity, and `hardGate`), `runId` and `healthCheckDispatched`, and the emitter stamps `timestamp`, `run_id`, `headless_decisions` and `warnings` into it. A run the hard gate blocked writes it too, as a FAIL record, so its `skf-test-skill-result-latest.json` holds that FAIL, not an earlier run's verdict. A HALT writes no result file, and neither does a run cut off before report.md §4c: the copy then still holds an earlier run's verdict.
|
|
63
63
|
|
|
64
64
|
## Emitting a Halt
|
|
65
65
|
|
|
@@ -213,13 +213,13 @@ uv run {runLockHelper} acquire --lock "{forge_version}/.test-skill.lock" --owner
|
|
|
213
213
|
|
|
214
214
|
The run lock renewed here stays held until §7 releases it, so no other run publishes over this run's report or result files meanwhile.
|
|
215
215
|
|
|
216
|
-
**Build the result payload** before the report takes its public name. Every value in it comes from the run's own records: `{resultContextScript}` (it resolves relative to the skill root) reads the report frontmatter, the gap ledger and the scoring output, checks the verdict fields score.md §7 wrote (`testResult`, `score`, `threshold` and the fallback fields) against the scoring output, derives `verdict`, `exit_code` and `next_workflow` from `testResult`, and writes the shared emitter's payload, which names the report by its public name. Pass `--score` for a scored run (a run the hard gate blocked has no scoring output), `--no-health-check` when that flag was given, and `--warning` when SKILL.md On Activation step 3 kept a `{customization_resolver_unavailable}` reason: first write `customization_resolver_unavailable: {customization_resolver_unavailable}` to `{run_dir}/resolver-warning.txt` with a file write, never `echo` (the reason can hold quotes, backticks or `$( )`), and pass that file's text (the emitter adds the warnings a stage recorded in the run's sink):
|
|
216
|
+
**Build the result payload** before the report takes its public name. Every value in it comes from the run's own records: `{resultContextScript}` (it resolves relative to the skill root) reads the report frontmatter, the gap ledger and the scoring output, checks the verdict fields score.md §7 wrote (`testResult`, `score`, `threshold` and the fallback fields) against the scoring output, checks that `analysisConfidence` names a source access (`full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`), a blocked run's included, derives `verdict`, `exit_code` and `next_workflow` from `testResult`, and writes the shared emitter's payload, which names the report by its public name. Pass `--score` for a scored run (a run the hard gate blocked has no scoring output), `--no-health-check` when that flag was given, and `--warning` when SKILL.md On Activation step 3 kept a `{customization_resolver_unavailable}` reason: first write `customization_resolver_unavailable: {customization_resolver_unavailable}` to `{run_dir}/resolver-warning.txt` with a file write, never `echo` (the reason can hold quotes, backticks or `$( )`), and pass that file's text (the emitter adds the warnings a stage recorded in the run's sink):
|
|
217
217
|
|
|
218
218
|
```bash
|
|
219
219
|
uv run {resultContextScript} --report "{report_file}" --report-path "{publishedReportFile}" --ledger "{ledgerFile}" [--score "{run_dir}/score.json"] [--no-health-check] [--warning "$(cat "{run_dir}/resolver-warning.txt")"] --output "{run_dir}/result-context.json"
|
|
220
220
|
```
|
|
221
221
|
|
|
222
|
-
Bind `{emit_target}` ← `target` from its JSON (`stderr` for a run the hard gate blocked, else `stdout`). Exit 2: correct the command and run it again. Exit 1: HALT with its `error` (`halt_reason: "helper-failed"`, phase `report:result-contract`); the report keeps its in-progress name, so export-skill never reads a verdict the scoring output did not settle.
|
|
222
|
+
Bind `{emit_target}` ← `target` from its JSON (`stderr` for a run the hard gate blocked, else `stdout`). Exit 2: correct the command and run it again. Exit 1: HALT with its `error` (`halt_reason: "helper-failed"`, phase `report:result-contract`), except when the `error` names `analysisConfidence`: then write `analysisConfidence: '{analysis_confidence}'`, the value source-access-protocol.md resolved (the Coverage Analysis section's **Source Access:** line records it), to the frontmatter and run it again, halting only when it exits 1 again; the report keeps its in-progress name, so export-skill never reads a verdict the scoring output did not settle.
|
|
223
223
|
|
|
224
224
|
**Publish the report.** Set `health_check_dispatched`, already in the `{outputFile}` frontmatter: `false` when `no_health_check` is true (the `--no-health-check` flag), else `true`, since the health check now runs with no menu before it. The checks passed, so the report takes its public name, the one export-skill and update-skill read, a blocked run's included. From `{project-root}`, run:
|
|
225
225
|
|
|
@@ -235,7 +235,7 @@ and bind `{report_file}` ← `{publishedReportFile}`. If the move fails, HALT (`
|
|
|
235
235
|
uv run {emitEnvelopeHelper} emit --workflow skf-test-skill --run-dir "{run_dir}" --result-dir "{forge_version}" --target {emit_target} < "{run_dir}/result-context.json"
|
|
236
236
|
```
|
|
237
237
|
|
|
238
|
-
It writes `{forge_version}/skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name) and then its `skf-test-skill-result-latest.json` copy (
|
|
238
|
+
It writes `{forge_version}/skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name) and then its `skf-test-skill-result-latest.json` copy (the record of the last run whose emitter wrote this copy, at a stable path: a copy, not a symlink; a caller reads its own run's `SKF_TEST_RESULT_JSON` line), each atomically, so a partial write is never observable; stamps the timestamp, `run_id`, `headless_decisions` and `warnings` into both; and prints the `SKF_TEST_RESULT_JSON:` line. Bind `{result_line}` ← that line and `{result_path}` ← its `result_path`. A run the hard gate blocked writes the record too, as a FAIL (`status: "error"`, `verdict: "FAIL"`, `halt_reason: "hard-gate-blocked"`, `exit_code` 2), so after a blocked run `skf-test-skill-result-latest.json` holds its FAIL, never an earlier run's verdict. A result file that could not be written leaves `result_path` null and a `result_file_write_failed` warning in the line, and the run still finishes. If the emitter exits non-zero, run it once more; if it fails again, HALT (`halt_reason: "write-failed"`, phase `report:result-contract`) with its stderr: the run's result contract could not be written. `references/invocation-contract.md` lists the record's fields.
|
|
239
239
|
|
|
240
240
|
Once the result files are written, append `'report'` to `stepsCompleted` in the `{outputFile}` frontmatter (it stays out of the expected set above).
|
|
241
241
|
|
|
@@ -18,12 +18,12 @@ This table is the single source of truth for classifying a gap. Every stage reco
|
|
|
18
18
|
|
|
19
19
|
| Severity | Category | Criteria |
|
|
20
20
|
|----------|-----------------------------|----------|
|
|
21
|
-
| Critical | `signature-mismatch` | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type, in any way but
|
|
22
|
-
| Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name (coverage-check §2c) |
|
|
21
|
+
| Critical | `signature-mismatch` | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type, as signature-comparison-rules.md reads each form (a component by its props; an annotation either side leaves out is not compared), in any way but those the Low `signature-mismatch` row and the Info `observation` row grade, such as a renamed Python parameter (the script cannot see which parameters a caller must pass by position), an optional parameter left out of a form the skill does not label partial, or a mismatch with no `difference` (coverage-check §2 and §2b) |
|
|
22
|
+
| Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name, and no item of an export list (`export { L as NAME }`, `export type { ... }`) or of a Python `from X import` clause elsewhere renames to it a name that file declares; any other rename (`export { default as NAME }`, `const NAME = L`, `type NAME = L`, an import then an export, Python `NAME = L`) stays fabricated (coverage-check §2c) |
|
|
23
23
|
| Critical | `broken-reference` | Broken reference (contextual mode): a reference whose target does not exist, such as a file not at the stated path, a skill not in the skills output folder, a type no module declares, or a `scripts/` or `assets/` file the skill names (coherence-check §4) |
|
|
24
24
|
| High | `inaccurate-reference` | Inaccurate reference (contextual mode): the target exists but does not match, such as a type the cited module does not export, a signature that differs, or an integration pattern that does not match the implementation (coherence-check §4) |
|
|
25
25
|
| High | `reference-escape` | A reference whose real path leaves the skill, its source and, for a stack, the skills output folder (coherence-check §4) |
|
|
26
|
-
| High | `split-body-mismatch` |
|
|
26
|
+
| High | `split-body-mismatch` | Two files of the skill (the SKILL.md body or a `references/` file) document one export differently (coverage-check §1b) |
|
|
27
27
|
| High | `numerator-inflation` | A declared count (`stats.exports_documented`, or the provenance map's distinct entry names) equals the denominator while declared exports are absent from SKILL.md and `references/` (coverage-check §4b) |
|
|
28
28
|
| High | `structural` | Naive mode: a missing required section, an unbalanced code fence, or a documented async or sync behaviour its example contradicts (coherence-check §2.1, §2.2, §2.5) |
|
|
29
29
|
| High | `discovery` | Discovery test: at most 1 of 3 realistic prompts routed to the skill (report §4b.3) |
|
|
@@ -38,15 +38,15 @@ This table is the single source of truth for classifying a gap. Every stage reco
|
|
|
38
38
|
| Medium | `integration-pattern` | Incomplete integration pattern (contextual mode): one entry of coherence-check §5's `incomplete_patterns` |
|
|
39
39
|
| Medium | `structural` | Naive mode: an opening code fence without a language tag, an exported function or method no usage-family section or reference file names, or a table row whose column count differs from its header (coherence-check §2.3, §2.4, §2.6) |
|
|
40
40
|
| Medium | `discovery` | Discovery test: 2 of 3 realistic prompts routed to the skill (report §4b.3) |
|
|
41
|
-
| Low | `signature-mismatch` | A renamed parameter callers cannot see: a mismatch the signature judge tags `parameter-name` (
|
|
41
|
+
| Low | `signature-mismatch` | A renamed parameter callers cannot see: a mismatch the signature judge tags `parameter-name` (defined in signature-comparison-rules.md, Difference tags) at the file the surface records for the export, the one the plan sent the check to or a candidate the judge compared with, when it is a `.ts`, `.tsx`, `.mts`, `.cts`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.go`, `.rs`, `.java`, `.c`, `.h`, `.cc`, `.cpp`, `.cxx` or `.hpp` file, a language with no keyword arguments (not a Vue component, whose props callers set by name), as score-signatures.py checks. It counts as matching in Signature Accuracy (coverage-check §2 and §2b) |
|
|
42
42
|
| Low | `scripts-assets-provenance` | Script/asset file present without provenance entry in provenance-map.json file_entries |
|
|
43
|
-
| Low | `provenance-line` | Provenance line is not the definition of an export (`line-not-definition
|
|
43
|
+
| Low | `provenance-line` | Provenance line is not the definition of an export (`line-not-definition`), or its citation does not resolve: the pinned source lacks its file (`file-missing`) or the line is past the file's end, or below 1 (`line-out-of-bounds`) (coverage-check §4c) |
|
|
44
44
|
| Low | `migration-section` | Case 3 of `migration-section-rules.md` that the reviewer downgraded with an inline justification |
|
|
45
45
|
| Low | `metadata` | Missing optional metadata or examples |
|
|
46
46
|
| Low | `description` | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps), a Tessl Review description suggestion included |
|
|
47
47
|
| Low | `external-validator` | A skill-check diagnostic or a Tessl Review validation finding the run left unresolved (external-validators §5b) |
|
|
48
|
-
| Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c). Also a partial signature: a mismatch the signature judge tags `omitted-optional` (
|
|
49
|
-
| Info | `provenance-unverified` | Provenance line not verified: the line-check rules found no definition line (coverage-check §4c) |
|
|
48
|
+
| Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it; an `aliased` name, which an export-list item or a Python `from X import` item renames from a name its cited file declares, at that item's line, a multi-line list's item at its own line), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c). Also a partial signature: a mismatch the signature judge tags `omitted-optional` (defined in signature-comparison-rules.md, Difference tags) on a form the skill labels partial (simplified, key parameters, a `...` that stands in for omitted members, not rest or variadic syntax), which the inventory marks `signature_form: "partial"`, as score-signatures.py checks: `Partial signature: {name}`, at the declaration, counted as matching in Signature Accuracy (coverage-check §2 and §2b). Also a return annotation the body contradicts: a mismatch the signature judge tags `annotation-contradicted` (defined in signature-comparison-rules.md, Difference tags) whose documented signature states a return type and writes the source's parameters, and whose `evidence_line`, the body line that returns the documented type, comes after the declaration and before the next one and, with the local source, starts with `return` (or is a one-line declaration's own expression body), as score-signatures.py checks: `Annotation contradicted: {name}`, at the declaration, counted as matching in Signature Accuracy, with no change to the skill (the annotation is reported upstream) (coverage-check §2 and §2b) Also a documentation error a Deep-tier run sees that no script checks (an import path whose module does not export the name, a type member that differs from its declaration): `Not checked by script: {name}`, at the documenting line, written by hand (coverage-check §2 and §5b) |
|
|
49
|
+
| Info | `provenance-unverified` | Provenance line not verified: the line-check rules found no definition line, or the recorded line is not a line number (`line-invalid`) (coverage-check §4c) |
|
|
50
50
|
| Info | `migration-section` | Historical migration content in Section 4b with no T2-future annotation (Case 2 of `migration-section-rules.md`) |
|
|
51
51
|
| Info | `multi-denominator` | Multi-denominator reporting: barrel and documented-surface clusters diverge by design (>10% cross-cluster) |
|
|
52
52
|
| Info | `discovery` | Discovery testing not performed (`--no-discovery`, a catalog of fewer than 2 skills, or no subagents to route the prompts): realistic prompt testing recommended before export (report §4b) |
|