bmad-module-skill-forge 2.2.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +49 -49
- package/docs/404.md +10 -8
- package/docs/_data/pinned.yaml +22 -13
- package/docs/_internal/STABILITY.md +58 -35
- package/docs/agents.md +39 -53
- package/docs/architecture.md +65 -51
- package/docs/bmad-synergy.md +53 -51
- package/docs/campaign.md +127 -75
- package/docs/concepts.md +34 -28
- package/docs/examples.md +70 -68
- package/docs/forge-auto.md +34 -31
- package/docs/getting-started.md +118 -57
- package/docs/how-it-works.md +25 -21
- package/docs/index.md +15 -15
- package/docs/skill-model.md +70 -59
- package/docs/troubleshooting.md +143 -18
- package/docs/verifying-a-skill.md +87 -57
- package/docs/why-skf.md +24 -22
- package/docs/workflows.md +274 -110
- package/package.json +18 -10
- package/src/forger/preferences.yaml +8 -1
- package/src/knowledge/agentskills-spec.md +2 -0
- package/src/knowledge/ccc-bridge.md +40 -24
- package/src/knowledge/confidence-tiers.md +9 -5
- package/src/knowledge/overview.md +3 -3
- package/src/knowledge/progressive-capability.md +8 -8
- package/src/knowledge/provenance-tracking.md +9 -5
- package/src/knowledge/qmd-registry.md +3 -3
- package/src/knowledge/skf-knowledge-index.csv +2 -2
- package/src/knowledge/skill-lifecycle.md +18 -11
- package/src/knowledge/split-body-strategy.md +0 -3
- package/src/knowledge/tool-resolution.md +4 -2
- package/src/knowledge/version-paths.md +101 -15
- package/src/module-help.csv +3 -3
- package/src/module.yaml +10 -0
- package/src/shared/data/ast-grep-recipes.yaml +608 -0
- package/src/shared/data/ide-context-files.json +30 -0
- package/src/shared/health-check.md +31 -59
- package/src/shared/references/description-guard-protocol.md +12 -9
- package/src/shared/references/feasibility-report-schema.md +19 -13
- package/src/shared/references/headless-gate-convention.md +55 -3
- package/src/shared/references/output-contract-schema.md +19 -8
- package/src/shared/references/pipeline-contracts.md +94 -29
- package/src/shared/references/tessl-review.md +77 -0
- package/src/shared/scripts/schemas/skf-analyze-result-envelope.v1.json +194 -0
- package/src/shared/scripts/schemas/skf-audit-result-envelope.v1.json +192 -0
- package/src/shared/scripts/schemas/skf-brief-result-envelope.v1.json +32 -4
- package/src/shared/scripts/schemas/skf-campaign-result-envelope.v1.json +174 -0
- package/src/shared/scripts/schemas/skf-create-skill-result-envelope.v1.json +284 -0
- package/src/shared/scripts/schemas/skf-drop-skill-result-envelope.v1.json +164 -0
- package/src/shared/scripts/schemas/skf-export-result-envelope.v1.json +160 -0
- package/src/shared/scripts/schemas/skf-quick-skill-result-envelope.v1.json +261 -0
- package/src/shared/scripts/schemas/skf-refine-architecture-result-envelope.v1.json +192 -0
- package/src/shared/scripts/schemas/skf-rename-skill-result-envelope.v1.json +179 -0
- package/src/shared/scripts/schemas/skf-setup-result-envelope.v1.json +27 -11
- package/src/shared/scripts/schemas/skf-stack-result-envelope.v1.json +176 -0
- package/src/shared/scripts/schemas/skf-test-result-envelope.v1.json +173 -0
- package/src/shared/scripts/schemas/skf-update-result-envelope.v1.json +19 -5
- package/src/shared/scripts/schemas/skf-verify-stack-result-envelope.v1.json +180 -0
- package/src/shared/scripts/schemas/skill-brief.v1.json +34 -0
- package/src/shared/scripts/schemas/workspace-detection.v1.json +1 -1
- package/src/shared/scripts/skf-atomic-write.py +233 -64
- package/src/shared/scripts/skf-build-change-manifest.py +2753 -28
- package/src/shared/scripts/skf-ccc-git-hygiene.py +532 -0
- package/src/shared/scripts/skf-check-unit-records.py +184 -0
- package/src/shared/scripts/skf-check-workspace-drift.py +538 -9
- package/src/shared/scripts/skf-classify-changed-files.py +772 -0
- package/src/shared/scripts/skf-comention-pairs.py +1333 -123
- package/src/shared/scripts/skf-compare-file-hashes.py +267 -105
- package/src/shared/scripts/skf-count-imports.py +1867 -0
- package/src/shared/scripts/skf-count-tokens.py +1 -1
- package/src/shared/scripts/skf-derive-assembly-shape.py +21 -3
- package/src/shared/scripts/skf-description-guard.py +99 -11
- package/src/shared/scripts/skf-detect-docs.py +694 -2
- package/src/shared/scripts/skf-detect-language.py +486 -107
- package/src/shared/scripts/skf-detect-registry.py +887 -0
- package/src/shared/scripts/skf-detect-scripts-assets.py +197 -55
- package/src/shared/scripts/skf-detect-tools.py +209 -27
- package/src/shared/scripts/skf-detect-workspaces.py +506 -30
- package/src/shared/scripts/skf-disqualify-candidates.py +821 -182
- package/src/shared/scripts/skf-emit-brief-result-envelope.py +69 -191
- package/src/shared/scripts/skf-emit-result-envelope.py +1604 -106
- package/src/shared/scripts/skf-enumerate-stack-skills.py +986 -117
- package/src/shared/scripts/skf-extract-public-api.py +5247 -96
- package/src/shared/scripts/skf-extraction-inventory.py +1280 -0
- package/src/shared/scripts/skf-extraction-snapshot.py +664 -0
- package/src/shared/scripts/skf-fetch-temporal.py +674 -0
- package/src/shared/scripts/skf-find-cycles.py +97 -29
- package/src/shared/scripts/skf-find-test-report.py +476 -0
- package/src/shared/scripts/skf-forge-tier-rw.py +469 -205
- package/src/shared/scripts/skf-github-fetch.py +365 -0
- package/src/shared/scripts/skf-github-probe.py +700 -0
- package/src/shared/scripts/skf-hash-content.py +180 -20
- package/src/shared/scripts/skf-language-corpora.py +1 -1
- package/src/shared/scripts/skf-load-provenance.py +161 -27
- package/src/shared/scripts/skf-manifest-ops.py +1 -1
- package/src/shared/scripts/skf-merge-ccc-exclusions.py +2307 -223
- package/src/shared/scripts/skf-merge-doc-urls.py +143 -20
- package/src/shared/scripts/skf-names-present.py +221 -0
- package/src/shared/scripts/skf-pair-intersect.py +119 -28
- package/src/shared/scripts/skf-parse-gaps.py +1384 -0
- package/src/shared/scripts/skf-preapply.py +1 -1
- package/src/shared/scripts/skf-preflight.py +176 -28
- package/src/shared/scripts/skf-promote-staged.py +430 -0
- package/src/shared/scripts/skf-provenance-gap-dispatch.py +64 -8
- package/src/shared/scripts/skf-qmd-classify-collections.py +229 -45
- package/src/shared/scripts/skf-quick-batch.py +774 -0
- package/src/shared/scripts/skf-rebuild-managed-sections.py +972 -88
- package/src/shared/scripts/skf-recommend-scope-type.py +323 -156
- package/src/shared/scripts/skf-render-metadata-stats.py +156 -12
- package/src/shared/scripts/skf-render-quick-metadata.py +132 -28
- package/src/shared/scripts/skf-render-stack-metadata.py +976 -0
- package/src/shared/scripts/skf-resolve-authoritative-files.py +155 -14
- package/src/shared/scripts/skf-resolve-package.py +523 -106
- package/src/shared/scripts/skf-rewrite-skill-name.py +433 -47
- package/src/shared/scripts/skf-run-lock.py +577 -0
- package/src/shared/scripts/skf-scan-manifests.py +998 -57
- package/src/shared/scripts/skf-scan-skill-md-structure.py +1341 -37
- package/src/shared/scripts/skf-severity-classify.py +712 -108
- package/src/shared/scripts/skf-shape-detect.py +298 -204
- package/src/shared/scripts/skf-shard-body.py +9 -8
- package/src/shared/scripts/skf-skill-inventory.py +2412 -89
- package/src/shared/scripts/skf-skills-module.py +472 -0
- package/src/shared/scripts/skf-source-tree.py +1796 -0
- package/src/shared/scripts/skf-structural-diff.py +787 -121
- package/src/shared/scripts/skf-tessl-review.py +706 -0
- package/src/shared/scripts/skf-update-active-symlink.py +120 -67
- package/src/shared/scripts/skf-update-run-state.py +624 -0
- package/src/shared/scripts/skf-validate-brief-inputs.py +92 -29
- package/src/shared/scripts/skf-validate-brief-schema.py +23 -2
- package/src/shared/scripts/skf-validate-feasibility-report.py +577 -66
- package/src/shared/scripts/skf-validate-frontmatter.py +103 -26
- package/src/shared/scripts/skf-validate-output.py +712 -64
- package/src/shared/scripts/skf-validate-pins.py +1 -1
- package/src/shared/scripts/skf-verify-no-trace.py +116 -10
- package/src/shared/scripts/skf-verify-provenance-completeness.py +3945 -152
- package/src/shared/scripts/skf-write-skill-brief.py +406 -25
- package/src/shared/tool-requirements.yaml +201 -0
- package/src/skf-analyze-source/SKILL.md +41 -36
- package/src/skf-analyze-source/assets/skill-brief-schema.md +7 -8
- package/src/skf-analyze-source/customize.toml +36 -24
- package/src/skf-analyze-source/references/auto-docs-only.md +58 -20
- package/src/skf-analyze-source/references/continue.md +2 -13
- package/src/skf-analyze-source/references/discover-additional-source.md +147 -0
- package/src/skf-analyze-source/references/generate-briefs.md +121 -55
- package/src/skf-analyze-source/references/headless-contract.md +42 -15
- package/src/skf-analyze-source/references/identify-units.md +96 -110
- package/src/skf-analyze-source/references/init.md +73 -71
- package/src/skf-analyze-source/references/map-and-detect.md +123 -105
- package/src/skf-analyze-source/references/map-unit-exports.md +68 -0
- package/src/skf-analyze-source/references/recommend.md +51 -70
- package/src/skf-analyze-source/references/scan-project.md +60 -84
- package/src/skf-analyze-source/references/scan-root.md +39 -0
- package/src/skf-analyze-source/references/step-auto-scope-coexistence.md +47 -0
- package/src/skf-analyze-source/references/step-auto-scope-corpora.md +53 -0
- package/src/skf-analyze-source/references/step-auto-scope-split.md +91 -0
- package/src/skf-analyze-source/references/step-auto-scope.md +208 -293
- package/src/skf-analyze-source/references/step-shape-detect.md +14 -12
- package/src/skf-analyze-source/references/unit-detection-heuristics.md +37 -42
- package/src/skf-audit-skill/SKILL.md +43 -51
- package/src/skf-audit-skill/assets/drift-report-template.md +28 -3
- package/src/skf-audit-skill/customize.toml +41 -20
- package/src/skf-audit-skill/references/constituent-freshness.md +91 -0
- package/src/skf-audit-skill/references/doc-drift.md +141 -0
- package/src/skf-audit-skill/references/headless-contract.md +29 -0
- package/src/skf-audit-skill/references/health-check.md +2 -0
- package/src/skf-audit-skill/references/init.md +149 -210
- package/src/skf-audit-skill/references/re-index.md +103 -110
- package/src/skf-audit-skill/references/report.md +96 -33
- package/src/skf-audit-skill/references/semantic-diff.md +34 -25
- package/src/skf-audit-skill/references/severity-classify.md +79 -65
- package/src/skf-audit-skill/references/severity-rules.md +15 -6
- package/src/skf-audit-skill/references/structural-diff.md +80 -90
- package/src/skf-audit-skill/references/upstream-checkout.md +77 -0
- package/src/skf-audit-skill/scripts/render-drift-tables.py +712 -0
- package/src/skf-brief-skill/SKILL.md +37 -26
- package/src/skf-brief-skill/assets/description-voice-examples.md +1 -1
- package/src/skf-brief-skill/assets/scope-templates.md +18 -25
- package/src/skf-brief-skill/assets/skill-brief-schema.md +30 -23
- package/src/skf-brief-skill/customize.toml +48 -20
- package/src/skf-brief-skill/references/analyze-target.md +144 -127
- package/src/skf-brief-skill/references/confirm-brief.md +11 -9
- package/src/skf-brief-skill/references/draft-checkpoint.md +36 -17
- package/src/skf-brief-skill/references/gather-intent-ratify.md +68 -0
- package/src/skf-brief-skill/references/gather-intent.md +87 -182
- package/src/skf-brief-skill/references/headless-args.md +84 -16
- package/src/skf-brief-skill/references/health-check.md +1 -1
- package/src/skf-brief-skill/references/invocation-contract.md +18 -13
- package/src/skf-brief-skill/references/portfolio-similarity-check.md +1 -1
- package/src/skf-brief-skill/references/qmd-collection-registration.md +7 -10
- package/src/skf-brief-skill/references/scope-definition.md +123 -47
- package/src/skf-brief-skill/references/step-auto-brief.md +26 -100
- package/src/skf-brief-skill/references/step-auto-validate.md +20 -121
- package/src/skf-brief-skill/references/version-resolution.md +7 -10
- package/src/skf-brief-skill/references/write-brief.md +66 -57
- package/src/skf-campaign/SKILL.md +31 -34
- package/src/skf-campaign/assets/campaign-state-schema.json +25 -4
- package/src/skf-campaign/customize.toml +47 -16
- package/src/skf-campaign/references/campaign-contracts.md +39 -29
- package/src/skf-campaign/references/campaign-directive-spec.md +4 -2
- package/src/skf-campaign/references/health-check.md +7 -16
- package/src/skf-campaign/references/step-01-setup.md +46 -84
- package/src/skf-campaign/references/step-02-strategy.md +31 -30
- package/src/skf-campaign/references/step-03-pins.md +26 -18
- package/src/skf-campaign/references/step-04-provenance.md +24 -23
- package/src/skf-campaign/references/step-05-skill-loop.md +67 -48
- package/src/skf-campaign/references/step-06-batch.md +41 -34
- package/src/skf-campaign/references/step-07-capstone.md +40 -25
- package/src/skf-campaign/references/step-08-verify.md +28 -29
- package/src/skf-campaign/references/step-09-refine.md +29 -30
- package/src/skf-campaign/references/step-10-export.md +48 -36
- package/src/skf-campaign/references/step-11-maintenance.md +31 -34
- package/src/skf-campaign/references/step-resume.md +37 -68
- package/src/skf-campaign/scripts/campaign-deps.py +57 -18
- package/src/skf-campaign/scripts/campaign-parse-manifest.py +361 -54
- package/src/skf-campaign/scripts/campaign-provenance.py +96 -53
- package/src/skf-campaign/scripts/campaign-quality-gate.py +592 -0
- package/src/skf-campaign/scripts/campaign-render-batch.py +257 -49
- package/src/skf-campaign/scripts/campaign-render-kickoff.py +343 -50
- package/src/skf-campaign/scripts/campaign-report.py +204 -37
- package/src/skf-campaign/scripts/campaign-state.py +1128 -0
- package/src/skf-campaign/scripts/campaign-status.py +10 -74
- package/src/skf-campaign/scripts/campaign-validate-pins.py +30 -3
- package/src/skf-campaign/scripts/campaign-validate-state.py +34 -4
- package/src/skf-campaign/templates/campaign-brief-template.yaml +4 -5
- package/src/skf-campaign/templates/campaign-report-template.md +5 -1
- package/src/skf-campaign/templates/kickoff-template.md +0 -14
- package/src/skf-create-skill/SKILL.md +27 -20
- package/src/skf-create-skill/assets/compile-assembly-rules.md +11 -8
- package/src/skf-create-skill/assets/skill-sections.md +21 -11
- package/src/skf-create-skill/customize.toml +20 -8
- package/src/skf-create-skill/references/authoritative-files-protocol.md +36 -58
- package/src/skf-create-skill/references/batch-mode.md +81 -0
- package/src/skf-create-skill/references/ccc-index-check.md +27 -0
- package/src/skf-create-skill/references/compile.md +119 -70
- package/src/skf-create-skill/references/component-extraction.md +113 -91
- package/src/skf-create-skill/references/enrich.md +41 -23
- package/src/skf-create-skill/references/entry-points-by-hand.md +37 -0
- package/src/skf-create-skill/references/extract.md +204 -162
- package/src/skf-create-skill/references/extraction-patterns-by-hand.md +141 -0
- package/src/skf-create-skill/references/extraction-patterns-tracing.md +11 -11
- package/src/skf-create-skill/references/extraction-patterns.md +588 -201
- package/src/skf-create-skill/references/generate-artifacts.md +145 -93
- package/src/skf-create-skill/references/health-check.md +1 -1
- package/src/skf-create-skill/references/load-brief.md +53 -36
- package/src/skf-create-skill/references/relabel-rule.md +12 -0
- package/src/skf-create-skill/references/report.md +37 -57
- package/src/skf-create-skill/references/source-resolution-protocols.md +58 -169
- package/src/skf-create-skill/references/step-auto-shard.md +20 -93
- package/src/skf-create-skill/references/step-doc-rot.md +44 -60
- package/src/skf-create-skill/references/step-doc-sources.md +28 -19
- package/src/skf-create-skill/references/sub/ccc-discover.md +18 -7
- package/src/skf-create-skill/references/sub/fetch-docs.md +106 -59
- package/src/skf-create-skill/references/sub/fetch-temporal.md +53 -166
- package/src/skf-create-skill/references/tier-degradation-rules.md +18 -9
- package/src/skf-create-skill/references/validate.md +173 -99
- package/src/skf-create-skill/scripts/scan-doc-rot.py +232 -99
- package/src/skf-create-stack-skill/SKILL.md +29 -56
- package/src/skf-create-stack-skill/assets/metadata-contract.md +51 -0
- package/src/skf-create-stack-skill/assets/provenance-map-schema.md +15 -15
- package/src/skf-create-stack-skill/assets/stack-skill-template.md +23 -57
- package/src/skf-create-stack-skill/customize.toml +23 -12
- package/src/skf-create-stack-skill/references/compile-stack.md +57 -66
- package/src/skf-create-stack-skill/references/compose-mode-rules.md +24 -44
- package/src/skf-create-stack-skill/references/detect-integrations.md +92 -95
- package/src/skf-create-stack-skill/references/detect-manifests.md +46 -71
- package/src/skf-create-stack-skill/references/generate-output.md +138 -128
- package/src/skf-create-stack-skill/references/init.md +90 -32
- package/src/skf-create-stack-skill/references/integration-patterns.md +1 -17
- package/src/skf-create-stack-skill/references/invocation-contract.md +34 -0
- package/src/skf-create-stack-skill/references/manifest-patterns.md +32 -20
- package/src/skf-create-stack-skill/references/parallel-extract.md +112 -38
- package/src/skf-create-stack-skill/references/rank-and-confirm.md +57 -75
- package/src/skf-create-stack-skill/references/report.md +39 -33
- package/src/skf-create-stack-skill/references/validate.md +20 -39
- package/src/skf-drop-skill/SKILL.md +28 -51
- package/src/skf-drop-skill/customize.toml +44 -30
- package/src/skf-drop-skill/references/execute.md +83 -146
- package/src/skf-drop-skill/references/health-check.md +2 -0
- package/src/skf-drop-skill/references/invocation-contract.md +64 -0
- package/src/skf-drop-skill/references/report.md +46 -30
- package/src/skf-drop-skill/references/select.md +122 -124
- package/src/skf-drop-skill/scripts/dir-sizes.py +54 -26
- package/src/skf-drop-skill/scripts/drop-roster.py +322 -0
- package/src/skf-export-skill/SKILL.md +40 -62
- package/src/skf-export-skill/assets/managed-section-format.md +14 -50
- package/src/skf-export-skill/assets/snippet-format.md +3 -1
- package/src/skf-export-skill/customize.toml +42 -13
- package/src/skf-export-skill/references/generate-snippet.md +54 -46
- package/src/skf-export-skill/references/health-check.md +2 -0
- package/src/skf-export-skill/references/invocation-contract.md +15 -0
- package/src/skf-export-skill/references/load-skill.md +88 -62
- package/src/skf-export-skill/references/manifest-rebuild.md +9 -10
- package/src/skf-export-skill/references/multi-skill-mode.md +4 -4
- package/src/skf-export-skill/references/orphan-context-detection.md +13 -5
- package/src/skf-export-skill/references/orphan-row-detection.md +31 -22
- package/src/skf-export-skill/references/package.md +15 -73
- package/src/skf-export-skill/references/preflight-snippet-root-probe.md +51 -70
- package/src/skf-export-skill/references/result-envelope.md +31 -16
- package/src/skf-export-skill/references/summary.md +41 -17
- package/src/skf-export-skill/references/token-report.md +4 -12
- package/src/skf-export-skill/references/update-context.md +118 -237
- package/src/skf-forger/SKILL.md +61 -37
- package/src/skf-forger/bmad-skill-manifest.yaml +4 -1
- package/src/skf-forger/customize.toml +43 -0
- package/src/skf-forger/references/pipeline-mode.md +73 -18
- package/src/skf-forger/scripts/forge-status.py +299 -0
- package/src/skf-forger/scripts/parse-pipeline.py +322 -54
- package/src/skf-forger/scripts/pipeline-gate.py +428 -0
- package/src/skf-forger/scripts/pipeline-journal.py +1052 -0
- package/src/skf-quick-skill/SKILL.md +54 -53
- package/src/skf-quick-skill/assets/skill-template.md +4 -2
- package/src/skf-quick-skill/customize.toml +51 -20
- package/src/skf-quick-skill/references/batch-mode.md +74 -21
- package/src/skf-quick-skill/references/compile.md +30 -31
- package/src/skf-quick-skill/references/ecosystem-check.md +7 -57
- package/src/skf-quick-skill/references/finalize.md +69 -18
- package/src/skf-quick-skill/references/halt-contract.md +41 -29
- package/src/skf-quick-skill/references/health-check.md +5 -3
- package/src/skf-quick-skill/references/quick-extract.md +114 -45
- package/src/skf-quick-skill/references/registry-resolution.md +9 -64
- package/src/skf-quick-skill/references/resolve-target.md +116 -66
- package/src/skf-quick-skill/references/write-and-validate.md +79 -40
- package/src/skf-refine-architecture/SKILL.md +43 -53
- package/src/skf-refine-architecture/customize.toml +49 -20
- package/src/skf-refine-architecture/references/compile.md +97 -54
- package/src/skf-refine-architecture/references/exit-codes.md +30 -0
- package/src/skf-refine-architecture/references/finding-storage.md +21 -0
- package/src/skf-refine-architecture/references/gap-analysis.md +101 -38
- package/src/skf-refine-architecture/references/improvements.md +14 -20
- package/src/skf-refine-architecture/references/init.md +98 -57
- package/src/skf-refine-architecture/references/issue-detection.md +32 -39
- package/src/skf-refine-architecture/references/refinement-rules.md +37 -48
- package/src/skf-refine-architecture/references/report.md +65 -54
- package/src/skf-refine-architecture/scripts/skf-check-preservation.py +1619 -0
- package/src/skf-rename-skill/SKILL.md +19 -54
- package/src/skf-rename-skill/customize.toml +32 -26
- package/src/skf-rename-skill/references/execute.md +230 -184
- package/src/skf-rename-skill/references/exit-codes.md +7 -7
- package/src/skf-rename-skill/references/health-check.md +1 -7
- package/src/skf-rename-skill/references/invocation-contract.md +25 -0
- package/src/skf-rename-skill/references/report.md +63 -52
- package/src/skf-rename-skill/references/select.md +197 -115
- package/src/skf-rename-skill/scripts/skf-validate-rename-name.py +161 -21
- package/src/skf-setup/SKILL.md +26 -26
- package/src/skf-setup/customize.toml +14 -10
- package/src/skf-setup/references/auto-index.md +50 -63
- package/src/skf-setup/references/ccc-index.md +29 -90
- package/src/skf-setup/references/detect-and-tier.md +58 -67
- package/src/skf-setup/references/health-check.md +2 -0
- package/src/skf-setup/references/invocation-contract.md +14 -0
- package/src/skf-setup/references/report.md +43 -166
- package/src/skf-setup/references/tier-rules.md +5 -5
- package/src/skf-setup/references/write-config.md +34 -57
- package/src/skf-test-skill/SKILL.md +17 -57
- package/src/skf-test-skill/assets/output-section-formats.md +33 -34
- package/src/skf-test-skill/customize.toml +27 -13
- package/src/skf-test-skill/references/coherence-check.md +134 -62
- package/src/skf-test-skill/references/coverage-check-tiers.md +35 -0
- package/src/skf-test-skill/references/coverage-check.md +209 -264
- package/src/skf-test-skill/references/detect-mode.md +5 -5
- package/src/skf-test-skill/references/external-validators.md +94 -82
- package/src/skf-test-skill/references/health-check.md +6 -7
- package/src/skf-test-skill/references/init.md +164 -91
- package/src/skf-test-skill/references/invocation-contract.md +66 -0
- package/src/skf-test-skill/references/migration-section-rules.md +5 -2
- package/src/skf-test-skill/references/report.md +164 -142
- package/src/skf-test-skill/references/score.md +80 -107
- package/src/skf-test-skill/references/scoring-rules.md +42 -155
- package/src/skf-test-skill/references/source-access-protocol.md +18 -94
- package/src/skf-test-skill/references/step-hard-gate.md +51 -33
- package/src/skf-test-skill/scripts/aggregate-coherence.py +218 -30
- package/src/skf-test-skill/scripts/build-result-context.py +380 -0
- package/src/skf-test-skill/scripts/check-metadata-coherence.py +37 -4
- package/src/skf-test-skill/scripts/combine-external-scores.py +23 -6
- package/src/skf-test-skill/scripts/compute-score.py +223 -45
- package/src/skf-test-skill/scripts/gap-ledger.py +1666 -0
- package/src/skf-test-skill/scripts/hard-gate.py +203 -0
- package/src/skf-test-skill/scripts/load-coverage-inputs.py +1198 -0
- package/src/skf-test-skill/scripts/locate-export-segments.py +835 -0
- package/src/skf-test-skill/scripts/reconcile-coverage.py +547 -79
- package/src/skf-test-skill/scripts/score-signatures.py +479 -0
- package/src/skf-test-skill/scripts/stage-helper-payload.py +148 -0
- package/src/skf-test-skill/scripts/validate-inventory.py +311 -16
- package/src/skf-test-skill/scripts/verify-declared-numerator.py +133 -56
- package/src/skf-test-skill/templates/test-report-template.md +21 -15
- package/src/skf-update-skill/SKILL.md +38 -25
- package/src/skf-update-skill/customize.toml +27 -12
- package/src/skf-update-skill/references/detect-changes.md +253 -223
- package/src/skf-update-skill/references/gap-driven.md +336 -0
- package/src/skf-update-skill/references/health-check.md +31 -5
- package/src/skf-update-skill/references/init.md +254 -84
- package/src/skf-update-skill/references/invocation-contract.md +12 -0
- package/src/skf-update-skill/references/manual-section-rules.md +6 -23
- package/src/skf-update-skill/references/merge.md +182 -49
- package/src/skf-update-skill/references/re-extract.md +53 -262
- package/src/skf-update-skill/references/report.md +102 -35
- package/src/skf-update-skill/references/write.md +357 -155
- package/src/skf-update-skill/scripts/skf-new-file-diff.py +183 -46
- package/src/skf-verify-stack/SKILL.md +23 -42
- package/src/skf-verify-stack/assets/feasibility-report-template.md +11 -5
- package/src/skf-verify-stack/customize.toml +55 -22
- package/src/skf-verify-stack/references/coverage-patterns.md +17 -8
- package/src/skf-verify-stack/references/coverage.md +67 -64
- package/src/skf-verify-stack/references/exit-codes.md +21 -8
- package/src/skf-verify-stack/references/init.md +107 -62
- package/src/skf-verify-stack/references/integration-verification-rules.md +12 -13
- package/src/skf-verify-stack/references/integrations.md +106 -65
- package/src/skf-verify-stack/references/report.md +59 -42
- package/src/skf-verify-stack/references/requirements.md +32 -9
- package/src/skf-verify-stack/references/synthesize.md +53 -39
- package/src/skf-verify-stack/scripts/skf-coverage-tally.py +398 -52
- package/src/skf-verify-stack/scripts/skf-previous-report.py +330 -0
- package/src/skf-verify-stack/scripts/skf-report-delta.py +453 -49
- package/src/skf-verify-stack/scripts/skf-verdict-rollup.py +240 -27
- package/tools/cli/commands/install.js +11 -4
- package/tools/cli/commands/status.js +15 -3
- package/tools/cli/commands/update.js +11 -4
- package/tools/cli/lib/tool-check.js +320 -0
- package/tools/cli/lib/version-check.js +22 -9
- package/tools/cli/skf-cli.js +4 -1
- package/.gitattributes +0 -16
- package/.nvmrc +0 -1
- package/docs/_internal/RELEASING.md +0 -679
- package/release-audits/v1.0.0-launch-audit.md +0 -728
- package/src/skf-audit-skill/references/step-doc-drift.md +0 -177
- package/src/skf-brief-skill/references/headless-source-authority-detection.md +0 -26
- package/src/skf-campaign/manifest.yaml +0 -15
- package/src/skf-create-skill/assets/tessl-dismissal-rules.md +0 -106
- package/src/skf-create-skill/references/ecosystem-check.md +0 -74
- package/src/skf-drop-skill/references/headless-contract.md +0 -18
- package/src/skf-update-skill/references/merge-conflict-rules.md +0 -15
- package/src/skf-update-skill/references/remote-source-resolution.md +0 -98
- package/src/skf-update-skill/references/validate.md +0 -124
- package/tools/validate-docs-drift.js +0 -233
- package/tools/validate-docs-links.js +0 -665
|
@@ -1,11 +1,30 @@
|
|
|
1
1
|
---
|
|
2
2
|
nextStepFile: 'coherence-check.md'
|
|
3
|
-
outputFile: '{
|
|
4
|
-
scoringRulesFile: '
|
|
3
|
+
outputFile: '{report_file}'
|
|
4
|
+
scoringRulesFile: 'references/scoring-rules.md'
|
|
5
5
|
sourceAccessProtocol: 'references/source-access-protocol.md'
|
|
6
|
+
coverageTiersFile: 'references/coverage-check-tiers.md'
|
|
7
|
+
validateInventoryScript: 'scripts/validate-inventory.py'
|
|
8
|
+
coverageInputsScript: 'scripts/load-coverage-inputs.py'
|
|
9
|
+
scoreSignaturesScript: 'scripts/score-signatures.py'
|
|
6
10
|
reconcileScript: 'scripts/reconcile-coverage.py'
|
|
7
11
|
coherenceScript: 'scripts/check-metadata-coherence.py'
|
|
8
12
|
numeratorVerifyScript: 'scripts/verify-declared-numerator.py'
|
|
13
|
+
stageHelperPayloadScript: 'scripts/stage-helper-payload.py'
|
|
14
|
+
outputFormatsFile: 'assets/output-section-formats.md'
|
|
15
|
+
ledgerFile: '{forge_version}/test-findings-{run_id}.json'
|
|
16
|
+
gapLedgerScript: 'scripts/gap-ledger.py'
|
|
17
|
+
# Each `<name>ProbeOrder` resolves `{<name>Helper}` to its first existing path:
|
|
18
|
+
# the installed SKF module first, the src/ dev checkout second.
|
|
19
|
+
verifyProvenanceCompletenessProbeOrder:
|
|
20
|
+
- '{project-root}/_bmad/skf/shared/scripts/skf-verify-provenance-completeness.py'
|
|
21
|
+
- '{project-root}/src/shared/scripts/skf-verify-provenance-completeness.py'
|
|
22
|
+
extractPublicApiProbeOrder:
|
|
23
|
+
- '{project-root}/_bmad/skf/shared/scripts/skf-extract-public-api.py'
|
|
24
|
+
- '{project-root}/src/shared/scripts/skf-extract-public-api.py'
|
|
25
|
+
detectWorkspacesProbeOrder:
|
|
26
|
+
- '{project-root}/_bmad/skf/shared/scripts/skf-detect-workspaces.py'
|
|
27
|
+
- '{project-root}/src/shared/scripts/skf-detect-workspaces.py'
|
|
9
28
|
---
|
|
10
29
|
|
|
11
30
|
<!-- Config: communicate in {communication_language}. -->
|
|
@@ -16,362 +35,282 @@ numeratorVerifyScript: 'scripts/verify-declared-numerator.py'
|
|
|
16
35
|
|
|
17
36
|
Compare the exports, functions, classes, types, and interfaces documented in SKILL.md against the actual source code API surface. Identify missing documentation, undocumented exports, and signature mismatches. Analysis depth scales with forge tier.
|
|
18
37
|
|
|
38
|
+
**Halt envelope.** Every HALT in this step names its `halt_reason` and phase and carries exit code 1. It releases the run lock first, whatever the release prints: from `{project-root}`, run `uv run {runLockHelper} release --lock "{forge_version}/.test-skill.lock" --owner "{run_owner}"` (SKILL.md Workflow Rules). In headless mode it then writes `{"phase": "<phase>", "reason": "<the halt message>", "halt_reason": "<halt_reason>", "skill_name": "{skill_name}", "report_path": "{report_file}"}`, adding `"path"` when the halt names one, to `{run_dir}/halt.json` and runs:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
uv run {emitEnvelopeHelper} emit-halt --workflow skf-test-skill --run-dir "{run_dir}" --target stderr < "{run_dir}/halt.json"
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Display the line it prints, as the run's last line, then stop. If the emitter exits non-zero or prints no line, display the halt message alone. An interactive HALT displays its message and emits nothing.
|
|
45
|
+
|
|
46
|
+
**Run files.** Every script below writes its result into the run folder `{run_dir}` that init.md §6b created, and the next one reads it from there by path. A subagent's response is saved there with the Write tool exactly as it came back, fence and all, and passed by path, so a quote or an apostrophe in it never meets a shell. Every count, name list and score a script computes is read from the file it wrote, never re-derived by hand or copied from one command into another. Each command runs from `{project-root}`; a `{...Script}` path resolves relative to the skill root.
|
|
47
|
+
|
|
19
48
|
### 0. Check for Docs-Only Mode
|
|
20
49
|
|
|
21
|
-
|
|
50
|
+
A skill is docs-only when every citation it writes is `[EXT:...]`, with no local source citation (`[AST:...]`, `[SRC:...]`, or a stack's `[from skill: ...]`). The citation census decides it:
|
|
22
51
|
|
|
23
|
-
|
|
52
|
+
```bash
|
|
53
|
+
uv run {coverageInputsScript} census --skill-dir "{resolved_skill_package}" --output "{run_dir}/census.json"
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Bind `{docs_only_mode}` ← `docsOnly`. Exit 1 printed its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:census`). §0b sets `{docs_only_mode}` too when source access reaches State 5.
|
|
24
57
|
|
|
25
|
-
**
|
|
58
|
+
**A docs-only run** measures documentation completeness as its Export Coverage, with `analysis_confidence` `docs-only`. It runs only §1, §1a and §1b, the docs-only guard of §2b, §2c's `docsOnly` branch, §4's `Denominator: docs-only completeness` annotation, §5, §5b and §6. Step 5 passes `docsOnly: true` to the scoring script, which skips Signature Accuracy and Type Coverage at any tier.
|
|
26
59
|
|
|
27
60
|
"**Docs-only skill detected.** Coverage check evaluates documentation completeness rather than source code coverage."
|
|
28
61
|
|
|
29
|
-
|
|
62
|
+
A source-based skill continues with §0b.
|
|
30
63
|
|
|
31
64
|
### 0b. Load Source Access Protocol
|
|
32
65
|
|
|
33
|
-
Load `{sourceAccessProtocol}` and follow both sections:
|
|
34
|
-
1. **Source API Surface Definition** — determines what counts as the public API for coverage denominator
|
|
35
|
-
2. **Source Access Resolution** — 5-state waterfall to determine how source files will be read and sets `analysis_confidence`
|
|
66
|
+
Load `{sourceAccessProtocol}` and follow both sections: **Source API Surface Definition** (the clause that decides the coverage denominator, picked in §2b) and **Source Access Resolution** (the 5-state waterfall that decides how source files are read and sets `analysis_confidence`; State 5 makes the run docs-only).
|
|
36
67
|
|
|
37
|
-
|
|
68
|
+
**Workspace layout.** The Source API Surface Definition's monorepo tests read `{source_is_monorepo}`, which `{detectWorkspacesHelper}` decides. At State 1 (local source), resolve `{detectWorkspacesHelper}` ← first existing path in `{detectWorkspacesProbeOrder}` and run (`{stageHelperPayloadScript}` reads the source tree and the root manifests from disk and pipes them to the helper):
|
|
38
69
|
|
|
39
|
-
|
|
70
|
+
```bash
|
|
71
|
+
uv run {stageHelperPayloadScript} detect-workspaces --source-root "{source_path}" | uv run {detectWorkspacesHelper}
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Bind `{source_is_monorepo}` ← `is_monorepo` and `{source_workspace_kind}` ← `manifest_kind`. At States 2 to 5, or when the helper does not resolve or the command exits non-zero, the layout is unknown: bind `{source_is_monorepo}` ← false. §5 records the layout.
|
|
75
|
+
|
|
76
|
+
**Clause flags.** Every `surface` command in §2 takes `--monorepo` when `{source_is_monorepo}` is true, and the flags the Source API Surface Definition leaves to you.
|
|
77
|
+
|
|
78
|
+
### 1. Extract Documented Exports from SKILL.md
|
|
40
79
|
|
|
41
|
-
Delegate reading of the skill under test to a subagent
|
|
42
|
-
1. Read SKILL.md
|
|
43
|
-
2. If a `references/` directory exists alongside SKILL.md and SKILL.md's `## Full` headings are absent or stubs, also read all `references/*.md` files
|
|
44
|
-
3. only return this compact JSON inventory — no prose, no extra commentary:
|
|
80
|
+
Delegate the reading of the skill under test to a subagent, given the path to SKILL.md and to its `references/` directory when one exists. The subagent reads SKILL.md, and every `references/*.md` file as well when SKILL.md's `## Full` headings are absent or stubs. For each export documented both in the SKILL.md body and in a reference file, it compares the two: parameters (name, type, order, optionality), the return type, and descriptions that contradict (brief against detailed is fine). It returns only this JSON, with no prose and no commentary:
|
|
45
81
|
|
|
46
82
|
```json
|
|
47
83
|
{
|
|
48
84
|
"exports": [
|
|
49
85
|
{"name": "functionName", "kind": "function", "params": "...", "return_type": "...", "description": "..."},
|
|
50
|
-
{"name": "ClassName", "kind": "class", "methods": ["..."], "properties": ["..."]},
|
|
51
|
-
{"name": "TypeName", "kind": "type", "fields": ["..."]},
|
|
52
|
-
{"name": "
|
|
53
|
-
{"name": "useHook", "kind": "hook", "usage_signature": "..."}
|
|
86
|
+
{"name": "ClassName", "kind": "class", "methods": ["..."], "properties": ["..."], "description": "..."},
|
|
87
|
+
{"name": "TypeName", "kind": "type", "fields": ["..."], "description": "..."},
|
|
88
|
+
{"name": "useHook", "kind": "hook", "usage_signature": "...", "description": "..."}
|
|
54
89
|
],
|
|
55
|
-
"
|
|
56
|
-
"references": ["references/api-reference.md", "references/type-definitions.md"],
|
|
90
|
+
"references": ["references/api-reference.md"],
|
|
57
91
|
"cross_check_mismatches": [
|
|
58
|
-
{
|
|
59
|
-
"export": "functionName",
|
|
60
|
-
"skill_md_line": 42,
|
|
61
|
-
"reference_file": "references/api-reference.md",
|
|
62
|
-
"reference_line": 18,
|
|
63
|
-
"issue": "description of the signature mismatch"
|
|
64
|
-
}
|
|
92
|
+
{"export": "functionName", "skill_md_line": 42, "reference_file": "references/api-reference.md", "reference_line": 18, "issue": "SKILL.md shows (date: Date) => string, the reference (date: Date, format?: string) => string"}
|
|
65
93
|
]
|
|
66
94
|
}
|
|
67
95
|
```
|
|
68
96
|
|
|
69
|
-
|
|
97
|
+
Tell the subagent the contract the validator checks: each `kind` is one of `function`, `class`, `type`, `constant`, `hook`, `interface`, `method`, `struct`, `enum`, `trait`, `macro` or `adapter`; every entry carries the `description` the skill gives it (an empty string when it gives none); a function or method carries `params` (`""` or `[]` when it takes none) and `return_type` as the skill documents them, and leaves a key out when the skill documents nothing for it.
|
|
70
98
|
|
|
71
|
-
|
|
99
|
+
Save the subagent's response, as it came back, to `{run_dir}/inventory-response.txt` with the Write tool. The documented inventory is the one §1a validates: do not load SKILL.md or the reference files into the parent context. Without subagents, the parent reads the same files itself, builds the same JSON and saves it to the same file; §1a's checks still run on it.
|
|
72
100
|
|
|
73
101
|
#### 1a. Parent-Side Schema Validation + Spot-Check
|
|
74
102
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
**Schema validation (required keys + types) — delegated to `scripts/validate-inventory.py`.** Fence-stripping, JSON parsing, and the required-keys / per-entry-type / `kind`-enum / mismatch-field contract are structural validation with one correct verdict per input, so they run in the script, not in-prompt. Pipe the subagent's **raw** response (fence and all) to it exactly as §2c pipes the reconcile input:
|
|
103
|
+
Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped) and spot-checks sampled names against SKILL.md and the reference files the subagent listed. The look-up is the script's, never a grep: an export named `$state` or `a.b` is neither a shell variable nor a regex.
|
|
78
104
|
|
|
79
105
|
```bash
|
|
80
|
-
|
|
106
|
+
uv run {validateInventoryScript} --input "{run_dir}/inventory-response.txt" --skill-package "{resolved_skill_package}" --output "{run_dir}/inventory.json"
|
|
81
107
|
```
|
|
82
108
|
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
-
|
|
86
|
-
- Required keys present with correct types: `exports` (list), `cross_check_mismatches` (list — may be empty). Note: the parent already knows the skill name from workflow context (`{resolved_skill_package}` from step 1) — the subagent is not required to echo it back, and doing so introduces a contract-drift surface without improving verification.
|
|
87
|
-
- Each `exports[]` entry is a dict with at minimum `name` (non-empty string) and `kind` (one of `function|class|type|constant|hook|interface|method|struct|enum|trait|macro|adapter`). The enum spans the constructs SKF actually documents across languages and skill types: JS/TS (`function`/`class`/`type`/`constant`/`hook`/`interface`/`method`), Rust public-API items (`struct`/`enum`/`trait`/`macro` — alongside the shared `type`/`constant`/`function`), and stack-composition scaffolds (`adapter`). Malformed entries are counted in `rejectedCount`.
|
|
88
|
-
- Each non-empty `cross_check_mismatches[]` entry carries `export`, `skill_md_line`, `reference_file`, `reference_line`, `issue`.
|
|
109
|
+
- `valid` true (exit 0): every later command reads `{run_dir}/inventory.json`.
|
|
110
|
+
- `valid` false (exit 2): re-dispatch the §1 subagent once, with the script's `violations[]` appended to its instructions (an absent name as a name the skill does not write), save the new response over the old one and run the command again. Invalid a second time: HALT (`halt_reason: "inventory-invalid"`, phase `coverage-check:inventory`): "coverage-check: the subagent inventory failed {schema validation | the ground-truth spot-check} twice: {the violations joined}."
|
|
111
|
+
- Exit 1 (no input, a file that cannot be read, or a package with no readable SKILL.md): correct the file and run it again.
|
|
89
112
|
|
|
90
|
-
|
|
113
|
+
### 1b. Cross-Check Split-Body Consistency
|
|
91
114
|
|
|
92
|
-
|
|
115
|
+
Only when the validated inventory's `references` is non-empty. The SKILL.md body is authoritative: each `cross_check_mismatches` entry is a High `split-body-mismatch` gap. §5b records it titled `Split-body mismatch: {export}`, with its `export` and its Source at the reference file's line (`{reference_file}:{reference_line}`), inside the skill package: the reference file is the one to update, and update-skill repairs it without reading the source.
|
|
93
116
|
|
|
94
|
-
|
|
95
|
-
2. Otherwise, sample `min(3, exportsCount)` exports deterministically — by default take indices `[0, len//2, len-1]` (first, middle, last) from `inventory.exports` after a stable sort by `name`.
|
|
96
|
-
3. For each sampled export, grep for the name across SKILL.md **and every reference file the subagent listed in `inventory.references`** (the documented surface of a split-body skill spans both): `grep -n "{export.name}" {resolved_skill_package}/SKILL.md {resolved_skill_package}/{each references[] path}` in the parent context. The name must appear at least once somewhere in that file set. Greping SKILL.md alone would false-HALT a split-body skill whose sampled export is documented only in a `references/*.md` file (a legitimate placement per §1 step 2 and the split-body note below).
|
|
97
|
-
4. If a sampled name returns zero matches across SKILL.md **and** all listed reference files, HALT "coverage-check: subagent inventory failed ground-truth spot-check — `{name}` claimed as export but absent from SKILL.md and the listed reference files".
|
|
117
|
+
### 2. Analyze Source Code (Tier-Dependent)
|
|
98
118
|
|
|
99
|
-
|
|
119
|
+
Identify the public API surface from the package entry points (see 0b), at the depth the tier allows. Whichever branch runs, `{coverageInputsScript} surface` writes it to `{run_dir}/surface.json`, which §2b, §2c, §4 and step 5 read. Names read by eye reach it as per-file results, `{"file": "<path>", "exports_found": [<each name>], "signature_mismatches": []}` for each file, saved to `{run_dir}/per-file-<n>.json` with the Write tool and given to `surface` with one `--per-file` each.
|
|
100
120
|
|
|
101
|
-
**
|
|
121
|
+
**Exits.** Every `{coverageInputsScript}` and `{scoreSignaturesScript}` command in §2 and §2b exits 0, or exits 1 with its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:surface`). `surface` with a `--per-file`, and `score`, also exit 2 when a subagent response breaks the schema: re-dispatch that subagent once with its `violations[]` appended (or correct a result the main thread wrote), save the new response over the old one and run the command again; a second failure HALTs the same way.
|
|
102
122
|
|
|
103
|
-
|
|
123
|
+
**Quick Tier (no AST tools):** load `{coverageTiersFile}` and follow its **Quick Tier** section. Signatures cannot be verified at this tier (§5's render marks them unverified).
|
|
104
124
|
|
|
105
|
-
**
|
|
125
|
+
**Forge Tier (ast-grep available):**
|
|
106
126
|
|
|
107
|
-
|
|
127
|
+
At State 1, run the recipe runner once over the source. Resolve `{extractPublicApiHelper}` ← first existing path in `{extractPublicApiProbeOrder}` and run it (leave `--brief` out when the skill has no brief; `--head-cap 0` keeps every match, so no export is cut and read as missing):
|
|
108
128
|
|
|
109
|
-
|
|
110
|
-
-
|
|
111
|
-
|
|
129
|
+
```bash
|
|
130
|
+
uv run {extractPublicApiHelper} --mode full --source-root "{source_path}" --brief "{forge_data_folder}/{skill_name}/skill-brief.yaml" --tier {detected_tier} --head-cap 0 --output "{run_dir}/extract-full.json"
|
|
131
|
+
```
|
|
112
132
|
|
|
113
|
-
|
|
133
|
+
Exit 2 is an input error (a brief the runner cannot read, say) or a failure the runner did not foresee, named in one line on stderr with no JSON: HALT with that line (`halt_reason: "helper-failed"`, phase `coverage-check:surface`). No Fallback Per-File Scan stands in for it: every flag here is fixed, so the error is a defect to surface.
|
|
114
134
|
|
|
115
|
-
|
|
135
|
+
Build the surface from it, with the metadata, the brief when the skill has one and, when init.md §2 bound one, the provenance map (the baselines of the §2b candidates and guards; they add no name):
|
|
116
136
|
|
|
117
|
-
```
|
|
118
|
-
{
|
|
119
|
-
"cross_check_mismatches": [
|
|
120
|
-
{
|
|
121
|
-
"export": "formatDate",
|
|
122
|
-
"skill_md_line": 42,
|
|
123
|
-
"reference_file": "references/api-reference.md",
|
|
124
|
-
"reference_line": 18,
|
|
125
|
-
"issue": "SKILL.md shows (date: Date) => string, reference shows (date: Date, format?: string) => string"
|
|
126
|
-
}
|
|
127
|
-
],
|
|
128
|
-
"exports_cross_checked": 12,
|
|
129
|
-
"mismatches_found": 1
|
|
130
|
-
}
|
|
137
|
+
```bash
|
|
138
|
+
uv run {coverageInputsScript} surface --extraction "{run_dir}/extract-full.json" --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] [--monorepo] [--pattern-reference-intent] [--curated-subset] --output "{run_dir}/surface.json"
|
|
131
139
|
```
|
|
132
140
|
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
### 2. Analyze Source Code (Tier-Dependent)
|
|
141
|
+
The surface holds the names the entry points export, less those defined in a file the brief scopes out (`excluded.outsideScope`); the recipes' internal matches are never part of it. A Python `__init__.py` nested in the folder of another package that exports a name is no barrel: its names (`excluded.nestedEntries`) leave only the `all` and `root` sets and the umbrella ratio, and `plan` still compares their signatures. When a `scope.include` glob matches no file (`guards.staleScope.fires`), the root exports defined in a file no glob covers and the brief does not exclude are back in it (`guards.staleScope.restored`). A stale `scope.tier_a_include` glob fires the guard too (`guards.staleScope.unmatchedTierAInclude`) and counts no name. §5's render lists `excluded.outsideScope`, `excluded.nestedEntries` and the restored names. Decide what to do from the JSON, not from the exit code:
|
|
136
142
|
|
|
137
|
-
|
|
143
|
+
- `extraction.fallback.needed` is true, or the runner printed no JSON on an exit other than 2: load `{coverageTiersFile}` and run its **Fallback Per-File Scan** in place of the rest of this tier.
|
|
144
|
+
- `extraction.status` is `incomplete` (exit 1): keep the surface, read by eye only the files `extraction.readByEye` names, save a per-file result for each, and run `surface` again with them.
|
|
145
|
+
- `extractionGaps[]` names exports no recipe found: they are on the surface, and their signature is read by eye at their `file` and `line`.
|
|
138
146
|
|
|
139
|
-
**
|
|
140
|
-
- Read the entry point file(s) directly
|
|
141
|
-
- Identify public exports by scanning for `export` keywords, `module.exports`, `__init__.py` imports, or language-specific export patterns
|
|
142
|
-
- Compare against documented inventory by name matching
|
|
143
|
-
- Cannot verify signatures — note as "unverified" in report
|
|
147
|
+
**Compare the signatures.** A script lists the comparisons; subagents judge them:
|
|
144
148
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
149
|
+
```bash
|
|
150
|
+
uv run {scoreSignaturesScript} plan --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --output "{run_dir}/signature-plan.json"
|
|
151
|
+
```
|
|
148
152
|
|
|
149
|
-
For
|
|
150
|
-
1. Uses ast-grep to extract all exported symbols with their full signatures (the `source_sig`)
|
|
151
|
-
2. Matches each export against the `documented_signatures` map supplied by the parent, comparing params (name, type, order, optionality) and return type
|
|
152
|
-
3. Returns only the JSON object below — no prose, no commentary, no markdown fences:
|
|
153
|
+
For each `files[]` entry, delegate one subagent with its checks. For each check it reads the full declaration at `{file}:{line}` (all of it: `signatureLine` is one physical line, and a multi-line signature cut there would read as a mismatch), compares it with the check's `documented` signature (params: name, type, order, optionality; and the return type), and returns only this JSON, with no prose and no markdown fences:
|
|
153
154
|
|
|
154
155
|
```json
|
|
155
|
-
{
|
|
156
|
-
"file": "src/utils.ts",
|
|
157
|
-
"exports_found": ["formatDate", "parseConfig", "ConfigType"],
|
|
158
|
-
"exports_documented": ["formatDate", "parseConfig"],
|
|
159
|
-
"missing_docs": ["ConfigType"],
|
|
160
|
-
"signature_mismatches": [
|
|
161
|
-
{
|
|
162
|
-
"name": "formatDate",
|
|
163
|
-
"source_sig": "(date: Date, format?: string) => string",
|
|
164
|
-
"documented_sig": "(date: Date) => string",
|
|
165
|
-
"issue": "missing optional parameter 'format'"
|
|
166
|
-
}
|
|
167
|
-
]
|
|
168
|
-
}
|
|
156
|
+
{"file": "src/utils.ts", "signature_mismatches": [{"name": "formatDate", "line": 42, "source_sig": "(date: Date, format?: string) => string", "documented_sig": "(date: Date) => string", "issue": "missing optional parameter 'format'"}]}
|
|
169
157
|
```
|
|
170
158
|
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
**Deep Tier (ast-grep + gh + QMD):**
|
|
174
|
-
- All Forge tier checks, plus:
|
|
175
|
-
- Use gh CLI to verify source repository matches documented version
|
|
176
|
-
- Cross-check type definitions against their source declarations
|
|
177
|
-
- Verify re-exported symbols trace to their original source
|
|
159
|
+
Save each response to `{run_dir}/signatures-<n>.txt` with the Write tool. Without subagents, compare in the main thread and save the same JSON.
|
|
178
160
|
|
|
179
|
-
|
|
161
|
+
**Deep Tier (ast-grep + gh + QMD):** all Forge tier checks, plus: verify with the gh CLI that the source repository matches the documented version, cross-check type definitions against their source declarations, and trace re-exported symbols to their original source.
|
|
180
162
|
|
|
181
|
-
|
|
163
|
+
**States 2 to 4 (no local source):** the surface is what the skill recorded, so there is no signature to score:
|
|
164
|
+
- **State 2** (`{forge_provenance_map}`, the provenance map init.md §2 bound): the union of its named exports and the `metadata.json` `exports[]` names, with `--fold`, `--keep` and `--fold-prefix` as the Source API Surface Definition's canonicalization says:
|
|
182
165
|
|
|
183
|
-
|
|
166
|
+
```bash
|
|
167
|
+
uv run {coverageInputsScript} surface --provenance "{forge_provenance_map}" --metadata "{resolved_skill_package}/metadata.json" [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] [--fold] [--keep <variant>] [--fold-prefix <prefix>] [--monorepo] [--pattern-reference-intent] [--curated-subset] --output "{run_dir}/surface.json"
|
|
168
|
+
```
|
|
184
169
|
|
|
185
|
-
|
|
186
|
-
|
|
170
|
+
Step 5's scoring script reads its `state2` counts, and the report records its fold summary, `canonical.summary`.
|
|
171
|
+
- **State 3**: the `metadata.json` `exports[]` names: `surface --metadata "{resolved_skill_package}/metadata.json" [--monorepo] [--pattern-reference-intent] [--curated-subset] --output "{run_dir}/surface.json"`.
|
|
172
|
+
- **State 4**: the names read remotely by eye, as per-file results: `surface --per-file "{run_dir}/per-file-<n>.json" [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] --metadata "{resolved_skill_package}/metadata.json" [--monorepo] [--pattern-reference-intent] [--curated-subset] [--barrel <state>] --output "{run_dir}/surface.json"`.
|
|
187
173
|
|
|
188
|
-
|
|
174
|
+
### 2b. Resolve the Denominator and Guard Zero Exports
|
|
189
175
|
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
**If `total_exports == 0` AND `docs_only_mode == false` AND `metadata.json.skill_type != "stack"`:** HALT with:
|
|
176
|
+
Load the metadata and provenance-map counts once, for §2c, §4b and the numerator check below (pass `--provenance` when init.md §2 bound `{forge_provenance_map}`):
|
|
193
177
|
|
|
178
|
+
```bash
|
|
179
|
+
uv run {coverageInputsScript} metadata --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] --output "{run_dir}/coverage-inputs.json"
|
|
194
180
|
```
|
|
195
|
-
Error: indeterminate API surface — 0 exports discovered in source for {skill_name}.
|
|
196
181
|
|
|
197
|
-
|
|
198
|
-
Export Coverage is undefined (division by zero) and downstream scoring
|
|
199
|
-
would yield a vacuous PASS.
|
|
182
|
+
**Pick the denominator.** `surface.json` `denominator` holds the Source API Surface Definition clause (§0b) `surface` picked: its §2c `branch` (`barrel`, `scalar` or `stack`), `surfaceSet` (the set the barrel branch counts and the signature score reads) and `ladderStep`. When `absentSet` names a set the surface lacks (the scope sets need the brief's globs and names that carry their files, `subpaths` and `root` an extraction), `surfaceSet` is `all` and the `Denominator:` line (§4) adds `({absentSet} set absent: all set used)`. When `rebuild` is `provenance-fold`, rerun State 2's command with no `--metadata`, with `--fold` and `--pattern-reference-intent`, and read its `denominator`.
|
|
200
183
|
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
- Add `[EXT:]` citations if this is actually a docs-only skill
|
|
204
|
-
- Verify the skill's source_path / source_ref resolve to the intended tree
|
|
205
|
-
```
|
|
184
|
+
- **A stack skill** (`branch` `stack`): `coverage-inputs.json` `stack.denominator`. Its own barrel is empty by design, and its `[from skill: …]` citations never make it docs-only. **If `stack.denominator` is 0**: HALT (`halt_reason: "indeterminate-surface"`, phase `coverage-check:denominator`) with `Error: stack composition surface empty: {skill_name} cites no contracts, libraries, or integration pairs, so Export Coverage is undefined. Verify the stack was compiled from at least one constituent skill.` Do not write the Coverage Analysis section; this is an indeterminate state, not a FAIL.
|
|
185
|
+
- **A docs-only skill**: the `docsOnly` branch, over the inventory.
|
|
206
186
|
|
|
207
|
-
|
|
187
|
+
**Numerator ground truth.** A declared count equal to the denominator (`stats.exports_documented == stats.effective_denominator`, the inflation signature) may be padded to a tautological 100%. Run the check on every source-based skill; without the signature (a stack and a reference app carry none) it reports `skipped`:
|
|
208
188
|
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
189
|
+
```bash
|
|
190
|
+
uv run {numeratorVerifyScript} --inputs "{run_dir}/coverage-inputs.json" --skill-dir "{resolved_skill_package}" --output "{run_dir}/numerator.json"
|
|
191
|
+
```
|
|
212
192
|
|
|
213
|
-
|
|
193
|
+
§2c's scalar branch takes its `verified` count as the numerator when `inflated` is true, and §4b records the gap. Exit 1 or 2: HALT with its message (`halt_reason: "helper-failed"`, phase `coverage-check:numerator`).
|
|
214
194
|
|
|
215
|
-
**
|
|
195
|
+
**If the chosen denominator is 0 AND `docs_only_mode == false` AND `metadata.json.skill_type != "stack"`:** HALT (`halt_reason: "indeterminate-surface"`, phase `coverage-check:denominator`) with:
|
|
216
196
|
|
|
217
|
-
|
|
197
|
+
```
|
|
198
|
+
Error: indeterminate API surface: 0 exports discovered in source for {skill_name}.
|
|
218
199
|
|
|
219
|
-
|
|
200
|
+
Export Coverage is undefined, and scoring would yield a vacuous PASS. Fix one of:
|
|
201
|
+
- Set `scope.include` in the brief to point at the package's entry point(s)
|
|
202
|
+
- Add `[EXT:]` citations if this is actually a docs-only skill
|
|
203
|
+
- Verify the skill's source_root / source_ref resolve to the intended tree
|
|
204
|
+
```
|
|
220
205
|
|
|
221
|
-
|
|
206
|
+
Do not write the Coverage Analysis section or attach a score: this is an indeterminate state, not a FAIL. **If `docs_only_mode == true` and the validated inventory documents no item:** HALT the same way with "docs-only skill declares zero items: no API surface to test".
|
|
222
207
|
|
|
223
|
-
**
|
|
208
|
+
**Score the signatures** (Forge, Forge+ and Deep tier at State 1, once §2 compared them). Give `{scoreSignaturesScript}` `denominator.surfaceSet` as `--surface-set`. Type Coverage then counts that set's types alone, so a type the clause leaves out (a stratified skill's deferred Tier B) lowers neither Export Coverage nor Type Coverage. Give it one `--results` per saved response (the fallback scan's `per-file-<n>.txt`, else the `signatures-<n>.txt`):
|
|
224
209
|
|
|
225
210
|
```bash
|
|
226
|
-
|
|
211
|
+
uv run {scoreSignaturesScript} score --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --results "{run_dir}/signatures-<n>.txt" --output "{run_dir}/signatures.json" --gaps-output "{run_dir}/signature-gaps.json"
|
|
227
212
|
```
|
|
228
213
|
|
|
229
|
-
|
|
214
|
+
It writes each mismatch of a compared name to `signature-gaps.json` as a Critical `signature-mismatch` gap titled `Signature mismatch: {name}`, with `export` `{name}`, its Source at `{file}:{line}` and both signatures in its Issue; §5b appends that file to the ledger. §5's render lists its `warnings` (a mismatch for a name never compared, a category with nothing to compare).
|
|
230
215
|
|
|
231
|
-
|
|
232
|
-
{
|
|
233
|
-
"denominatorSource": "barrel | scalar | stack",
|
|
234
|
-
"exports": [ /* §1 inventory exports[] — used for barrel + scalar; kind:"method" entries are excluded automatically */ ],
|
|
235
|
-
"barrelSet": [ /* barrel: resolved enumerated name set, when §4 supplies one */ ],
|
|
236
|
-
"perFileResults": [ /* barrel: the §2 per-file results, unioned into barrel_set when no barrelSet */ ],
|
|
237
|
-
"denominatorValue": 0,
|
|
238
|
-
"compositionNames": [ /* stack: composition-surface names to grep */ ],
|
|
239
|
-
"skillPackagePath": "{resolved_skill_package}"
|
|
240
|
-
}
|
|
241
|
-
```
|
|
216
|
+
### 2c. Reconcile Documented vs Source Surface (Deterministic Intersection)
|
|
242
217
|
|
|
243
|
-
|
|
218
|
+
`{reconcileScript}` reconciles the §1 inventory with the source surface and writes `{run_dir}/coverage.json`, whose `exportCoverage` is the Export Coverage score step 5 hands to the scoring script. Run the branch §2b picked:
|
|
244
219
|
|
|
245
|
-
|
|
246
|
-
- `missing` / `missingCount` — source names not documented (barrel enumerates the names; scalar/stack give only the residual count). Never negative.
|
|
247
|
-
- `stale` / `staleCount` — documented names not in source (barrel only; empty with `staleApplicable: false` for scalar/stack)
|
|
248
|
-
- `denominator` — `|barrel_set|` (barrel) or the resolved scalar/stack denominator
|
|
249
|
-
- `exportCoverage` — `documented / denominator * 100`, already rounded; capped at 100 on the scalar/stack branches
|
|
250
|
-
- `numeratorSurplus` / `coverageUncapped` / `coverageCapped` — scalar/stack only; see the surplus note below
|
|
220
|
+
1. **Barrel**, with the name set §2b picked as `--surface-set`:
|
|
251
221
|
|
|
252
|
-
|
|
222
|
+
```bash
|
|
223
|
+
uv run {reconcileScript} --denominator-source barrel --inventory "{run_dir}/inventory.json" --surface "{run_dir}/surface.json" --surface-set <set> --output "{run_dir}/coverage.json"
|
|
224
|
+
```
|
|
253
225
|
|
|
254
|
-
|
|
226
|
+
2. **Scalar**, over the `coverage-inputs.json` scalar; `--verified` makes §2b's `verified` count the numerator when the count was inflated:
|
|
255
227
|
|
|
256
|
-
|
|
228
|
+
```bash
|
|
229
|
+
uv run {reconcileScript} --denominator-source scalar --coverage-inputs "{run_dir}/coverage-inputs.json" --inventory "{run_dir}/inventory.json" --skill-dir "{resolved_skill_package}" --verified "{run_dir}/numerator.json" --output "{run_dir}/coverage.json"
|
|
230
|
+
```
|
|
257
231
|
|
|
258
|
-
|
|
232
|
+
3. **Stack** (`metadata.json.skill_type == "stack"`):
|
|
259
233
|
|
|
260
|
-
|
|
234
|
+
```bash
|
|
235
|
+
uv run {reconcileScript} --denominator-source stack --coverage-inputs "{run_dir}/coverage-inputs.json" --skill-dir "{resolved_skill_package}" --output "{run_dir}/coverage.json"
|
|
236
|
+
```
|
|
261
237
|
|
|
262
|
-
|
|
263
|
-
|--------|------|-----------|-----------------|-----------|--------|
|
|
264
|
-
| {name} | function/class/type | yes/no | yes/no/unverified | src/file.ts:42 | PASS/FAIL/WARN |
|
|
238
|
+
4. **Docs-only**: Export Coverage is the documentation completeness ratio, complete items over documented items, and `incomplete[]` lists each incomplete item with the fields it lacks:
|
|
265
239
|
|
|
266
|
-
|
|
267
|
-
-
|
|
268
|
-
|
|
269
|
-
- Missing documentation: `missingCount`
|
|
270
|
-
- Signature mismatches: {N}
|
|
271
|
-
- Undocumented in SKILL.md but not in source (stale docs): `staleCount`
|
|
240
|
+
```bash
|
|
241
|
+
uv run {reconcileScript} --denominator-source docsOnly --inventory "{run_dir}/inventory.json" --output "{run_dir}/coverage.json"
|
|
242
|
+
```
|
|
272
243
|
|
|
273
|
-
|
|
244
|
+
Exit 2 printed an `INVALID_INPUT` envelope: correct the input it names and run it again. Exit 1: HALT with its stderr (`halt_reason: "helper-failed"`, phase `coverage-check:reconcile`).
|
|
274
245
|
|
|
275
|
-
|
|
246
|
+
**A surplus is a signal, not a pass.** The scalar and stack branches count the numerator and the denominator from independent sets, so `documented` can exceed `denominator`: the script then reports `numeratorSurplus` above 0 and `coverageCapped: true`, holds `exportCoverage` at 100 and keeps the raw ratio in `coverageUncapped`, and §5's render states the surplus beside both counts. On a skill with no `scope.tier_a_include` glob that matches a file, a large surplus is the deflated-denominator signature: check `guards.deflation` before accepting the 100.
|
|
276
247
|
|
|
277
|
-
|
|
248
|
+
**Classify what the script found.** §5b records each of these gaps from the run files:
|
|
278
249
|
|
|
279
|
-
- **
|
|
280
|
-
- **
|
|
281
|
-
- **
|
|
250
|
+
- **Missing names** (barrel branch: `missing`): Medium `missing-export` gaps (Medium `missing-type` for a name `signatures.json` lists in `missingTypes`), each at the file `surface.json` records for it, so update-skill can re-extract the export from it. A missing export does not block: it lowers Export Coverage.
|
|
251
|
+
- **Missing count** (scalar or stack branch, `missingCount` above 0): one Medium `missing-export` gap for the count. When §2b's numerator ground-truth check found the count inflated, its `absent[]` names replace this gap, one each: update-skill documents a named export, not a count.
|
|
252
|
+
- **Stale names** (barrel branch: `stale`): each is a documented name the enumerated source surface lacks. Where ast-grep read the source at the pinned commit (Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` in the `{outputFile}` frontmatter) and init.md §2 bound `{forge_provenance_map}`, check the lines the skill cites for them: resolve `{verifyProvenanceCompletenessHelper}` ← first existing path in `{verifyProvenanceCompletenessProbeOrder}` and run once:
|
|
282
253
|
|
|
283
|
-
|
|
254
|
+
```bash
|
|
255
|
+
uv run {verifyProvenanceCompletenessHelper} classify-stale --names "{run_dir}/coverage.json" --provenance "{forge_provenance_map}" --source-root "{source_path}" --inventory "{run_dir}/inventory.json" -o "{run_dir}/stale.json"
|
|
256
|
+
```
|
|
284
257
|
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
`Denominator
|
|
290
|
-
all three values — the chosen one explicitly marked and the other two recorded
|
|
291
|
-
as-observed (or `absent` when the candidate was not present for this skill):
|
|
258
|
+
A name it marks `fabricated: true` (every map entry of the name cites a missing file or one that does not define it) is a Critical `fabricated-signature` gap at its `source`. A name with a `defined_at` (the source still declares it there, an import never counting, a module so named counting unless the name's inventory kind rules it out), or whose `declared_in` lists several files of which the skill's `[AST:]`/`[SRC:]` citations on lines naming it cite exactly one, is a documented extra: an Info `observation` gap at that declaration, when `surface.json` has an `extraction` and its `excluded.outsideScope` does not list the name (a brief scoped its file out). Any other stale name (no `defined_at` and no single cited declaring file, one scoped out, no `extraction`, a dotted name, the conditions above not met, a helper that does not resolve or exits non-zero) is a Medium `stale-documentation` gap.
|
|
259
|
+
|
|
260
|
+
### 4. The Denominator Record
|
|
261
|
+
|
|
262
|
+
Record the denominator source in the Coverage Analysis section with the annotation `{sourceAccessProtocol}` gives `denominator.clause` (§4b gives a stack's and a reference app's): `Denominator: {barrel | empty barrel (…) | stratified (…) | multi-entry (…) | specific-modules (…) | pattern-reference (…) | stack composition (…) | pattern-surface (…) | docs-only completeness}`, and the `canonical.summary` line when `surface.json` has a fold. When `denominator.clause` is `stratified-scope` or `multi-entry`, append this block after the `Denominator:` line, with every count from `surface.json` `candidates` (`absent` for a null one):
|
|
292
263
|
|
|
293
264
|
```markdown
|
|
294
265
|
**Denominator Candidates** (stratified-scope audit trail):
|
|
295
|
-
- `stats.effective_denominator`: {
|
|
296
|
-
- `scope.tier_a_include` union: {
|
|
297
|
-
- `scope.include` union: {
|
|
298
|
-
- exports-map subpath union: {
|
|
299
|
-
- root barrel: {
|
|
266
|
+
- `stats.effective_denominator`: {statsEffectiveDenominator | absent} {← chosen on the scalar branch}
|
|
267
|
+
- `scope.tier_a_include` union: {tierAIncludeUnion | absent} {← chosen if the barrel branch's set}
|
|
268
|
+
- `scope.include` union: {scopeIncludeUnion | absent} {← chosen if the barrel branch's set}
|
|
269
|
+
- exports-map subpath union: {subpathUnion | absent} {← chosen if the barrel branch's set}
|
|
270
|
+
- root barrel: {rootBarrel | absent} {secondary candidate: root-barrel-vs-subpath-union audit}
|
|
300
271
|
```
|
|
301
272
|
|
|
302
|
-
|
|
303
|
-
against the other two without re-running the extraction. A future reviewer who
|
|
304
|
-
suspects denominator gaming has the evidence inline. The `multi-entry` clause
|
|
305
|
-
requires the root-barrel named-export count in the `root barrel` row so the
|
|
306
|
-
root-barrel-vs-subpath-union choice is auditable.
|
|
273
|
+
The guards raise coverage gaps too, which §5b records from `surface.json`: `guards.deflation.fires` is a Medium `metadata-drift` gap, `guards.inflation.fires` a Medium `denominator-inflation` gap and `guards.staleScope.fires` a Medium `brief-scope-stale` gap at the brief.
|
|
307
274
|
|
|
308
275
|
### 4b. Metadata Export-Count Coherence Cross-Check
|
|
309
276
|
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
**Stack-skill branch (`metadata.json.skill_type == "stack"`):** Skip the intra-cluster and cross-cluster count comparisons, the `confidence_distribution` sum check, AND the numerator ground-truth check below — none apply to a stack. For a stack the three counts measure intentionally *different* surfaces, so comparing them yields only false drift: `exports_documented` / `exports[]` measure the stack's own barrel (empty by design → `0`), the provenance-map enumerates the cited *constituent* contracts, and `confidence_distribution` bins those constituents — so for a stack treat `confidence_distribution` as a per-constituent count (it sums to the constituent count, not to `exports_documented`) and do not assert it against `exports_documented`. The numerator-inflation arm below targets an *individual* skill whose `exports_documented` was padded to equal `effective_denominator`; a stack's numerator is instead computed by full-grep in §2c's stack-skill branch and stack `metadata.json.stats` carries no `effective_denominator`, so the arm is not run. Record the denominator using the §2b source — `Denominator: stack composition ({N} cited contracts)` when the provenance-map supplied it, or `Denominator: stack composition ({N} libraries + integration pairs)` when the composition surface did — then proceed.
|
|
313
|
-
|
|
314
|
-
**Reference-app branch (`metadata.json.scope_type == "reference-app"`):** Skip the intra-cluster and cross-cluster count comparisons, the `confidence_distribution` sum check, AND the numerator ground-truth check below — none apply to a reference app, whose counts measure intentionally *different* surfaces. A reference app documents wiring/construct **pattern surfaces**, not a library export barrel, so `metadata.json.exports[]` is empty by design (`0`), `stats.exports_public_api` / `stats.pattern_surfaces_documented` count the documented pattern surfaces, and `confidence_distribution` bins the per-citation provenance entries (it sums to the citation count, not to `pattern_surfaces_documented`). Comparing these yields only false drift: a spurious Cluster-A "barrel drift" (`exports_public_api` vs `exports[].length == 0`) and a Cluster-B "documented-surface drift" (`pattern_surfaces_documented` vs the larger `confidence_distribution` sum). The numerator-inflation arm also does not apply — a reference app carries no `effective_denominator` (see the `skf-create-skill` reference-app carve-out), so the `exports_documented == effective_denominator` signature never fires. Record the denominator as `Denominator: pattern-surface ({pattern_surfaces_documented})` and proceed. (`referenceApp` and a stack skill are distinct signals — a skill is one or the other, never both, so only one of these two branches applies.)
|
|
315
|
-
|
|
316
|
-
**Collect available counts (skip any that are absent) and bin them into two clusters:**
|
|
317
|
-
|
|
318
|
-
**Cluster A — public-barrel surface** (what `__init__.py` / `index.ts` / `lib.rs` re-exports):
|
|
319
|
-
|
|
320
|
-
1. `metadata.json.stats.exports_public_api` — the declared public API count
|
|
321
|
-
2. `metadata.json.exports[]` array length — the enumerated public export list
|
|
322
|
-
|
|
323
|
-
**Cluster B — documented surface** (what was extracted and documented, including methods and submodule members):
|
|
324
|
-
|
|
325
|
-
3. `metadata.json.stats.exports_documented` — the declared documented count
|
|
326
|
-
4. Provenance-map **named-export count** (if `{forge_data_folder}/{skill_name}/provenance-map.json` exists) — pass the raw `export_name` values as `provenanceExportNames`; the script counts top-level named exports, **excluding entries whose `export_name` contains `::`** (impl-block methods like `Type::method`, which roll up under an already-counted type and are not separate barrel exports). Comparing the raw entry count instead false-positives on any method-enumerating provenance map (common for Rust/TS type-heavy skills): e.g. a map with 88 named exports + 48 `Type::method` entries reports 136 against `exports_documented` ≈ 92 → a spurious ~32% Cluster-B drift, while the comparable named-export count (88) agrees within ~4%.
|
|
327
|
-
5. `confidence_distribution` (`t1`, `t1_low`, `t2`, `t3`, when present in `metadata.json.stats`) — pass the tiers as `confidenceDistribution`; the script sums them. Every extracted/documented export is binned into exactly one confidence tier, so the distribution must sum to the documented-surface total; a divergence (e.g., distribution sums to 91 while `exports_documented` is 85) is an internal-consistency defect even when the two clusters look fine
|
|
328
|
-
|
|
329
|
-
Cluster assignment is canonical: `skf-create-skill` step 5 derives `exports_public_api` from entry-point validation and writes the `exports[]` array from the same barrel surface (see `skf-create-skill/references/compile.md:105`), while `exports_documented` tracks the broader documented surface that the provenance-map also enumerates.
|
|
330
|
-
|
|
331
|
-
**Delegate the drift arithmetic to `uv run {coherenceScript}`.** The intra-cluster / cross-cluster `>10%` comparisons are deterministic count arithmetic with one correct answer per input, so the script — not the prompt — owns the binning, the divergence percentages, and every skip condition (a cluster with fewer than two present counts, and clusters that agree within the threshold, are skipped inside the script). Build the input from the counts collected above and run it (`{coherenceScript}` resolves relative to the skill root):
|
|
277
|
+
`{coherenceScript}` compares the `coverage-inputs.json` counts and owns every skip (a stack, a reference app, a cluster with fewer than two counts, drift within its threshold):
|
|
332
278
|
|
|
333
279
|
```bash
|
|
334
|
-
|
|
280
|
+
uv run {coherenceScript} --inputs "{run_dir}/coverage-inputs.json" > "{run_dir}/metadata-coherence.json"
|
|
335
281
|
```
|
|
336
282
|
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
```json
|
|
340
|
-
{
|
|
341
|
-
"skillType": "{metadata.json.skill_type or null}",
|
|
342
|
-
"scopeType": "{metadata.json.scope_type or null}",
|
|
343
|
-
"clusterA": {"exports_public_api": 0, "exports_length": 0},
|
|
344
|
-
"clusterB": {"exports_documented": 0},
|
|
345
|
-
"provenanceExportNames": [ /* raw provenance-map export_name values — the script excludes `::` impl-block methods and counts the rest */ ],
|
|
346
|
-
"confidenceDistribution": {"t1": 0, "t1_low": 0, "t2": 0, "t3": 0}
|
|
347
|
-
}
|
|
348
|
-
```
|
|
283
|
+
A non-zero exit leaves the script's error, or nothing, in that file: delete it, so §5b skips its call and records no count finding, and note the error in the Coverage Analysis section. §5b records each `findings[]` entry: a Medium entry as a Medium `metadata-drift` gap and an Info entry as an Info `multi-denominator` gap. They describe data quality and change no denominator. Record the denominator of a stack (`Denominator: stack composition ({N} cited contracts)` or `({N} libraries + integration pairs)`, from `stack.basis`) and of a reference app (`Denominator: pattern-surface ({pattern_surfaces_documented})`).
|
|
349
284
|
|
|
350
|
-
The
|
|
285
|
+
**The numerator ground truth §2b ran.** `numerator.json` `inflated: true` (`verified < declared`) is a High `numerator-inflation` gap listing its `absent[]` names, which §5b records; on the scalar branch those names also replace §2c's missing-count gap. `inflated: false` or `skipped: true` is no finding.
|
|
351
286
|
|
|
352
|
-
|
|
353
|
-
- An **Info** `multi-denominator reporting — barrel vs documented surface` is expected for skills whose documented surface intentionally exceeds the barrel (methods, submodule members, re-exported classes) — it is not drift. The note exists so the test report makes the dual-denominator design visible and auditable without demanding action.
|
|
287
|
+
### 4c. Provenance Line Check
|
|
354
288
|
|
|
355
|
-
(
|
|
289
|
+
Check that the provenance map records each export at the line that defines it (the `def` or declaration line itself, not a decorator or blank line above it), so update-skill can move a wrong line. Run this section only when `analysis_confidence` is `full`, `{forge_version}/provenance-map.json` exists, and the `{outputFile}` frontmatter's `workspaceDrift` is not `overridden` (init.md §5b: the drifted tree is not the pinned commit). Otherwise skip to section 5.
|
|
356
290
|
|
|
357
|
-
|
|
291
|
+
Resolve `{verifyProvenanceCompletenessHelper}` ← first existing path in `{verifyProvenanceCompletenessProbeOrder}`. If neither path exists, skip to section 5: the check is advisory. Otherwise run:
|
|
358
292
|
|
|
359
293
|
```bash
|
|
360
|
-
|
|
294
|
+
uv run {verifyProvenanceCompletenessHelper} verify \
|
|
295
|
+
--metadata {resolved_skill_package}/metadata.json \
|
|
296
|
+
--provenance {forge_version}/provenance-map.json \
|
|
297
|
+
--source-root {source_path} \
|
|
298
|
+
-o "{run_dir}/provenance-verify.json"
|
|
361
299
|
```
|
|
362
300
|
|
|
363
|
-
|
|
301
|
+
Rely on the file, not the exit code (exit 1 means findings); on exit 2 the helper writes no file: skip to section 5. §5b records each `stale[]` item whose `reason` is `line-not-definition`: a Low `provenance-line` gap when its `definition_lines` holds a line, else an Info `provenance-unverified` gap (the rules may not cover its shape: the export is unverified, not gone). Its Source is exactly the `file:line` the map records, the citation update-skill's rule R5 reads.
|
|
364
302
|
|
|
365
|
-
|
|
366
|
-
- `inflated: true` (`verified < declared`) → emit a **High**-severity gap `numerator inflation — {declared − verified} of {declared} declared exports absent from SKILL.md/references` listing the script's `absent[]` names, and use `verified` as the Export Coverage numerator (overriding `exports_documented`). A numerator padded to equal the denominator otherwise produces a tautological 100% that passes the gate.
|
|
303
|
+
This section reports none of the helper's other findings, and neither gap blocks the gate.
|
|
367
304
|
|
|
368
|
-
|
|
305
|
+
### 5. Write the Coverage Analysis Section
|
|
369
306
|
|
|
370
|
-
|
|
307
|
+
Write the **Coverage Analysis** section in place of the template's `## Coverage Analysis` heading and the placeholder comment under it, in `{outputFile}`. `{reconcileScript}` renders its counts, per-export table and category scores from the run files (leave out `--surface` on a docs-only run and `--signatures` when §2b scored none):
|
|
371
308
|
|
|
372
|
-
|
|
309
|
+
```bash
|
|
310
|
+
uv run {reconcileScript} render --coverage "{run_dir}/coverage.json" [--surface "{run_dir}/surface.json"] [--signatures "{run_dir}/signatures.json"]
|
|
311
|
+
```
|
|
373
312
|
|
|
374
|
-
|
|
313
|
+
Exit 0: write its output unchanged where the section below places it. Exit 2 is a usage error: correct the command and run it again. Exit 1 printed its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:render`).
|
|
375
314
|
|
|
376
315
|
```markdown
|
|
377
316
|
## Coverage Analysis
|
|
@@ -379,38 +318,44 @@ Append the **Coverage Analysis** section to `{outputFile}`:
|
|
|
379
318
|
**Tier:** {forge_tier}
|
|
380
319
|
**Source Access:** {analysis_confidence} (full | provenance-map | metadata-only | remote-only | docs-only)
|
|
381
320
|
**Source Path:** {source_path}
|
|
382
|
-
**
|
|
383
|
-
**Denominator:** {
|
|
321
|
+
**Workspace Layout:** {source_workspace_kind | single package | unknown (no local source, or the §0b helper failed)}
|
|
322
|
+
**Denominator:** {the §4 annotation}
|
|
323
|
+
{the Denominator Candidates block, when §4 adds one}
|
|
384
324
|
|
|
385
|
-
|
|
325
|
+
{the render output}
|
|
386
326
|
|
|
387
|
-
|
|
388
|
-
|
|
389
|
-
| ... per-export rows ... |
|
|
327
|
+
{the notes §2 to §4b add to this section}
|
|
328
|
+
```
|
|
390
329
|
|
|
391
|
-
### Coverage
|
|
330
|
+
### 5b. Record the Coverage Gaps
|
|
392
331
|
|
|
393
|
-
|
|
394
|
-
- **Documented:** {N} ({percentage}%)
|
|
395
|
-
- **Missing Documentation:** {N}
|
|
396
|
-
- **Signature Mismatches:** {N}
|
|
397
|
-
- **Stale Documentation:** {N}
|
|
398
|
-
- **Numerator Surplus:** {N} — omit this row unless `coverageCapped: true`; when present, also give the uncapped percentage
|
|
332
|
+
Record every gap this step found in the gap ledger `{ledgerFile}`: the hard gate (step 4c) decides from it, and the report step renders the Gap Report from it. `{gapLedgerScript}` writes the record of each gap a script found from that script's file: §2b's signature mismatches, §2c's missing names, missing count and stale names, §4's guard gaps, §4b's count findings and numerator gap, and §4c's provenance line gaps, each with the severity and category of its row in the Gap Severity table of `{scoringRulesFile}` and a fixed title, Source, `export` and Remediation. Run each command whose `--input` file this run wrote, leaving out a bracketed flag whose file does not exist: a run file this run did not write, or `--provenance` when init.md §2 bound no `{forge_provenance_map}`:
|
|
399
333
|
|
|
400
|
-
|
|
334
|
+
```bash
|
|
335
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --input "{run_dir}/signature-gaps.json"
|
|
336
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from coverage --input "{run_dir}/coverage.json" [--surface "{run_dir}/surface.json"] [--signatures "{run_dir}/signatures.json"] [--numerator "{run_dir}/numerator.json"] [--stale "{run_dir}/stale.json"] [--provenance "{forge_provenance_map}"] --skill-dir "{resolved_skill_package}" --metadata "{resolved_skill_package}/metadata.json"
|
|
337
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from guards --input "{run_dir}/surface.json" --metadata "{resolved_skill_package}/metadata.json"
|
|
338
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from numerator --input "{run_dir}/numerator.json" --metadata "{resolved_skill_package}/metadata.json"
|
|
339
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from metadata-coherence --input "{run_dir}/metadata-coherence.json" --metadata "{resolved_skill_package}/metadata.json"
|
|
340
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from provenance-line --input "{run_dir}/provenance-verify.json"
|
|
341
|
+
```
|
|
401
342
|
|
|
402
|
-
|
|
403
|
-
|----------|-------|
|
|
404
|
-
| Export Coverage | {N}% |
|
|
405
|
-
| Signature Accuracy | {N}% or N/A |
|
|
406
|
-
| Type Coverage | {N}% or N/A |
|
|
343
|
+
Then write the records no script wrote, §1b's split-body mismatches, in the Ledger Record Format of `{outputFormatsFile}`, each naming its export in `export` and in its title: the ledger tells two gaps apart by their title, Source and `export`, and update-skill takes the export's name from `export`. Write them as one JSON array on the lines between the two markers, exactly as they are: the quoted marker hands them to the script unchanged, quotes, apostrophes and `$` included. Run the command even when the array is empty (`[]`): the hard gate refuses to decide until every stage before it has recorded, with gaps or without.
|
|
407
344
|
|
|
408
|
-
|
|
345
|
+
```bash
|
|
346
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check <<'SKF_GAPS'
|
|
347
|
+
<the records, one JSON array>
|
|
348
|
+
SKF_GAPS
|
|
409
349
|
```
|
|
410
350
|
|
|
411
|
-
|
|
351
|
+
Rely on the JSON of each command:
|
|
412
352
|
|
|
413
|
-
|
|
353
|
+
- Exit 0: the records are in the ledger. `appended` names the id each new record received, and `duplicates` the ones a rerun of this step had already recorded.
|
|
354
|
+
- Exit 2 (`INVALID_RECORD` or `INVALID_INPUT`): nothing was written. Correct each record `errors[]` names (its `index` counts from 0), or the file or flag the `error` names, and run the command again.
|
|
355
|
+
- Exit 1: HALT with the script's `error` (`halt_reason: "helper-failed"`, phase `coverage-check:ledger`).
|
|
356
|
+
|
|
357
|
+
### 6. Report Coverage Results
|
|
414
358
|
|
|
415
|
-
|
|
359
|
+
Report the coverage result to the user from §5's render: the {forge_tier} tier, its **Files Analyzed** count, the three category scores and the issue count. Full details are in the Coverage Analysis section.
|
|
416
360
|
|
|
361
|
+
Append `'coverage-check'` to `stepsCompleted` in the `{outputFile}` frontmatter, then load and execute {nextStepFile}.
|