bmad-module-skill-forge 1.6.0 → 1.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/docs/_data/pinned.yaml +1 -1
- package/docs/workflows.md +1 -1
- package/package.json +2 -2
- package/src/knowledge/version-paths.md +4 -3
- package/src/shared/health-check.md +1 -1
- package/src/shared/scripts/schemas/skill-brief.v1.json +5 -0
- package/src/shared/scripts/skf-atomic-write.py +4 -1
- package/src/shared/scripts/skf-detect-language.py +34 -0
- package/src/shared/scripts/skf-detect-workspaces.py +54 -0
- package/src/shared/scripts/skf-enumerate-stack-skills.py +107 -16
- package/src/shared/scripts/skf-forge-tier-rw.py +4 -1
- package/src/shared/scripts/skf-manifest-ops.py +8 -4
- package/src/shared/scripts/skf-merge-ccc-exclusions.py +4 -1
- package/src/shared/scripts/skf-qmd-classify-collections.py +29 -3
- package/src/shared/scripts/skf-rebuild-managed-sections.py +72 -7
- package/src/shared/scripts/skf-scan-skill-md-structure.py +13 -3
- package/src/shared/scripts/skf-validate-brief-inputs.py +79 -18
- package/src/shared/scripts/skf-validate-brief-schema.py +47 -1
- package/src/shared/scripts/skf-validate-frontmatter.py +60 -2
- package/src/shared/scripts/skf-write-skill-brief.py +97 -17
- package/src/skf-analyze-source/assets/skill-brief-schema.md +10 -2
- package/src/skf-analyze-source/references/generate-briefs.md +30 -9
- package/src/skf-analyze-source/references/recommend.md +1 -1
- package/src/skf-brief-skill/SKILL.md +3 -3
- package/src/skf-brief-skill/assets/skill-brief-schema.md +6 -1
- package/src/skf-brief-skill/references/analyze-target.md +22 -6
- package/src/skf-brief-skill/references/confirm-brief.md +4 -0
- package/src/skf-brief-skill/references/gather-intent.md +14 -4
- package/src/skf-brief-skill/references/headless-args.md +3 -2
- package/src/skf-brief-skill/references/portfolio-similarity-check.md +16 -8
- package/src/skf-brief-skill/references/scope-definition.md +20 -2
- package/src/skf-brief-skill/references/version-resolution.md +1 -1
- package/src/skf-brief-skill/references/write-brief.md +8 -2
- package/src/skf-create-skill/assets/compile-assembly-rules.md +19 -0
- package/src/skf-create-skill/assets/skill-sections.md +1 -0
- package/src/skf-create-skill/assets/tessl-dismissal-rules.md +18 -5
- package/src/skf-create-skill/references/compile.md +5 -2
- package/src/skf-create-skill/references/extract.md +8 -3
- package/src/skf-create-skill/references/extraction-patterns.md +14 -5
- package/src/skf-create-skill/references/generate-artifacts.md +5 -1
- package/src/skf-create-skill/references/source-resolution-protocols.md +5 -2
- package/src/skf-create-skill/references/validate.md +4 -3
- package/src/skf-create-stack-skill/assets/provenance-map-schema.md +3 -3
- package/src/skf-create-stack-skill/assets/stack-skill-template.md +56 -0
- package/src/skf-create-stack-skill/references/compile-stack.md +13 -2
- package/src/skf-create-stack-skill/references/compose-mode-rules.md +2 -0
- package/src/skf-create-stack-skill/references/detect-integrations.md +5 -2
- package/src/skf-create-stack-skill/references/generate-output.md +49 -9
- package/src/skf-create-stack-skill/references/validate.md +8 -5
- package/src/skf-export-skill/references/load-skill.md +5 -4
- package/src/skf-export-skill/references/manifest-rebuild.md +1 -1
- package/src/skf-export-skill/references/token-report.md +2 -2
- package/src/skf-export-skill/references/update-context.md +25 -11
- package/src/skf-quick-skill/assets/skill-template.md +2 -2
- package/src/skf-quick-skill/references/compile.md +4 -2
- package/src/skf-quick-skill/references/write-and-validate.md +2 -2
- package/src/skf-refine-architecture/SKILL.md +1 -1
- package/src/skf-refine-architecture/references/gap-analysis.md +30 -4
- package/src/skf-refine-architecture/references/init.md +2 -0
- package/src/skf-refine-architecture/references/issue-detection.md +8 -3
- package/src/skf-test-skill/references/coherence-check.md +4 -4
- package/src/skf-test-skill/references/coverage-check.md +58 -16
- package/src/skf-test-skill/references/external-validators.md +7 -7
- package/src/skf-test-skill/references/init.md +5 -4
- package/src/skf-test-skill/references/score.md +6 -3
- package/src/skf-test-skill/references/scoring-rules.md +12 -0
- package/src/skf-test-skill/references/source-access-protocol.md +19 -3
- package/src/skf-test-skill/scripts/compute-score.py +4 -1
- package/src/skf-update-skill/references/detect-changes.md +15 -2
- package/src/skf-update-skill/references/merge.md +22 -0
- package/src/skf-update-skill/references/re-extract.md +19 -7
- package/src/skf-update-skill/references/write.md +3 -0
- package/src/skf-verify-stack/SKILL.md +2 -2
- package/src/skf-verify-stack/assets/feasibility-report-template.md +1 -1
- package/src/skf-verify-stack/references/coverage.md +35 -6
- package/src/skf-verify-stack/references/init.md +6 -2
- package/src/skf-verify-stack/references/integration-verification-rules.md +1 -1
- package/src/skf-verify-stack/references/integrations.md +7 -3
- package/src/skf-verify-stack/references/report.md +5 -1
- package/src/skf-verify-stack/references/requirements.md +3 -1
- package/src/skf-verify-stack/references/synthesize.md +11 -3
- package/tools/cli/lib/installer.js +1 -0
|
@@ -24,10 +24,10 @@ Before running external validators, check if `{forge_data_folder}/{skill_name}/e
|
|
|
24
24
|
**Pre-check (untracked or staged-only file):** Run `git ls-files --error-unmatch {skillDir}/SKILL.md 2>/dev/null`.
|
|
25
25
|
- If the command fails (exit code non-zero) or git is not available, the file is either **untracked** (new, never committed) or we're in a **non-git environment**:
|
|
26
26
|
- Check if `{skillDir}/metadata.json` exists and has a `generation_date` field
|
|
27
|
-
- Compare `metadata.json` `generation_date` against the evidence report's generation date (from its frontmatter or `## Validation Results` timestamp)
|
|
28
|
-
-
|
|
29
|
-
-
|
|
30
|
-
- Note: "Staleness check: SKILL.md is untracked/non-git — using metadata.json timestamp comparison."
|
|
27
|
+
- Compare `metadata.json` `generation_date` against the evidence report's generation date (from its frontmatter `generated` field or the `## Validation Results` timestamp)
|
|
28
|
+
- **Precision guard (mirror of the git-path Primary-cross check):** date-granularity equality is NOT proof of same-session generation. A same-day `update-skill` that regenerates SKILL.md *after* the cached evidence report was produced yields the same calendar date (e.g. `metadata.generation_date: 2026-05-23T00:00:00Z` vs evidence `generated: 2026-05-23`), so reusing on date-equality alone would publish pre-update scores for post-update content. Auto-reuse is safe **only** when BOTH timestamps carry a real time-of-day component — neither a date-only string (`2026-05-23`) nor a midnight-coerced `…T00:00:00Z` — AND they match to the minute. In that case auto-reuse: the evidence report was generated from the same SKILL.md content.
|
|
29
|
+
- Otherwise — if either timestamp is date-only or midnight-coerced, if they differ, or if `metadata.json` is missing or has no `generation_date` — treat as stale and proceed to section 2 for a fresh run. Forcing a fresh run on ambiguous precision matches the git path's bias toward freshness over reusing possibly-stale scores.
|
|
30
|
+
- Note: "Staleness check: SKILL.md is untracked/non-git — using metadata.json timestamp comparison (date-only/midnight timestamps force a fresh run)."
|
|
31
31
|
- If the command succeeds (file is tracked by git), continue to Primary check below.
|
|
32
32
|
|
|
33
33
|
**Primary (git-tracked):** Run `git log -1 --format=%cI -- {skillDir}/SKILL.md` to get the last commit date of SKILL.md. Compare against the evidence report's generation date (from its frontmatter or the `## Validation Results` timestamp). If SKILL.md's last commit is newer, results are stale.
|
|
@@ -66,11 +66,11 @@ timeout 120s npx skill-check check {skillDir} --format json --no-security-scan
|
|
|
66
66
|
If the command exits non-zero AND the exit code is `124` (GNU timeout's signal for the 120s wall-clock expiring), record `skill_check_score: N/A` with reason `timeout-120s`, log a warning, and skip to section 3. Other non-zero exits fall through to the regular JSON-parse path per the note below.
|
|
67
67
|
|
|
68
68
|
**Parse JSON output** to extract:
|
|
69
|
-
- `
|
|
69
|
+
- `scores[].score` — overall score (0-100); match the entry by `relativePath` (or `skillId`) to the validated skill dir. Older skill-check builds exposed this as a top-level `qualityScore` — fall back to that if `scores[]` is absent.
|
|
70
70
|
- `diagnostics[]` — any remaining issues
|
|
71
|
-
- `errorCount` and `warningCount`
|
|
71
|
+
- `summary.errorCount` and `summary.warningCount` — issue counts (the counts live under `summary`, not at the top level)
|
|
72
72
|
|
|
73
|
-
**Note:** `skill-check` may return a non-zero exit code even when `errorCount` is 0. Always rely on the parsed JSON output, not the shell exit code.
|
|
73
|
+
**Note:** `skill-check` may return a non-zero exit code even when `summary.errorCount` is 0. Always rely on the parsed JSON output, not the shell exit code.
|
|
74
74
|
|
|
75
75
|
Store in context: `skill_check_score`, `skill_check_diagnostics`
|
|
76
76
|
|
|
@@ -45,10 +45,11 @@ Provide the skill path or name. I'll search in `{skillsOutputFolder}`.
|
|
|
45
45
|
Resolve the skill path using version-aware resolution (see `{versionPathsKnowledge}`):
|
|
46
46
|
|
|
47
47
|
1. Read `{skillsOutputFolder}/.export-manifest.json` and look up the skill name in `exports` to get `active_version`
|
|
48
|
-
2. If
|
|
49
|
-
3. If
|
|
50
|
-
4. If
|
|
51
|
-
5.
|
|
48
|
+
2. **Manifest-lag guard.** If the skill is in the manifest, also read the `active` symlink target at `{skillsOutputFolder}/{skill_name}/active`. If that symlink resolves to a *different* version than `active_version`, prefer the **symlink target** as `{resolved_version}` and emit an Info note: "manifest active_version {M} lags the active symlink {N} — testing the symlink target (the just-forged version); run export-skill to reconcile the manifest." This is the canonical SS→TS→EX case: create-stack-skill flipped `active` to the new version, but the manifest only advances when export-skill runs — so a bare manifest-first resolution would test the *previously exported* version and report a PASS for the wrong version (silent false confidence). When the symlink matches `active_version` (or no `active` symlink exists), use `active_version`. See `{versionPathsKnowledge}` "Reading Workflows".
|
|
49
|
+
3. If found: resolve to `{skill_package}` = `{skillsOutputFolder}/{skill_name}/{resolved_version}/{skill_name}/`
|
|
50
|
+
4. If not in manifest: check for `active` symlink at `{skillsOutputFolder}/{skill_name}/active` — resolve to `{skill_group}/active/{skill_name}/`
|
|
51
|
+
5. If neither: fall back to flat path `{skillsOutputFolder}/{skill_name}/`. If SKILL.md exists at the flat path, auto-migrate per `{versionPathsKnowledge}` migration rules
|
|
52
|
+
6. Store the resolved path as `{resolved_skill_package}`
|
|
52
53
|
|
|
53
54
|
Check that the skill package contains required files:
|
|
54
55
|
|
|
@@ -53,7 +53,7 @@ Read `{outputFile}` and extract the category scores calculated in previous steps
|
|
|
53
53
|
|
|
54
54
|
#### 2b. Apply State 2 Undercount Deduction (pre-script)
|
|
55
55
|
|
|
56
|
-
If `analysis_confidence == 'provenance-map'` (State 2) AND step 3 recorded a provenance vs metadata divergence > 5% (see §4b in step 3), apply a 10-point deduction to `exportCoverage` BEFORE building the scoring input:
|
|
56
|
+
If `analysis_confidence == 'provenance-map'` (State 2) AND `metadata.json.skill_type != "stack"` AND step 3 recorded a provenance vs metadata divergence > 5% (see §4b in step 3), apply a 10-point deduction to `exportCoverage` BEFORE building the scoring input:
|
|
57
57
|
|
|
58
58
|
```
|
|
59
59
|
exportCoverage_adjusted = max(0, exportCoverage - 10)
|
|
@@ -61,6 +61,8 @@ exportCoverage_adjusted = max(0, exportCoverage - 10)
|
|
|
61
61
|
|
|
62
62
|
Record in the report: `scoring_notes: State 2 undercount risk acknowledged — 10% deduction applied to Export Coverage (raw: {N}%, adjusted: {M}%)`. Use the adjusted value as the `exportCoverage` field in §3a below. The deduction is deterministic and does not change category weights or active-category counting.
|
|
63
63
|
|
|
64
|
+
Stack skills are exempt (the `metadata.json.skill_type != "stack"` guard above; `skill_type` is loaded in step 01 and also surfaces as `stackSkill` in §3a): a stack's own barrel is empty by design, so a provenance-vs-metadata divergence on a stack reflects the barrel-vs-constituent surface difference (suppressed by §4b's stack-skill branch in step 3), not extraction undercount — deducting for it would penalize a correctly built stack.
|
|
65
|
+
|
|
64
66
|
#### 3a. Construct Scoring Input JSON
|
|
65
67
|
|
|
66
68
|
Build a JSON object from the data gathered in steps 1-2:
|
|
@@ -72,6 +74,7 @@ Build a JSON object from the data gathered in steps 1-2:
|
|
|
72
74
|
"docsOnly": "{true if docs_only_mode detected in step 03, else false}",
|
|
73
75
|
"state2": "{true if analysis_confidence is provenance-map, else false}",
|
|
74
76
|
"stackSkill": "{true if metadata.json.skill_type == 'stack', else false}",
|
|
77
|
+
"referenceApp": "{true if metadata.json.scope_type == 'reference-app', else false}",
|
|
75
78
|
"scores": {
|
|
76
79
|
"exportCoverage": "{export_coverage_percentage}",
|
|
77
80
|
"signatureAccuracy": "{signature_accuracy_percentage or null if N/A}",
|
|
@@ -83,7 +86,7 @@ Build a JSON object from the data gathered in steps 1-2:
|
|
|
83
86
|
}
|
|
84
87
|
```
|
|
85
88
|
|
|
86
|
-
**Important:** Score values must be numbers (not strings). Use `null` (not `"N/A"`) for categories that were not scored. Read `metadata.json.skill_type` from `{resolved_skill_package}/metadata.json`; if the value is `"stack"`, set `stackSkill: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (the categories will be redistributed per `{scoringRulesFile}` Stack Skills rule).
|
|
89
|
+
**Important:** Score values must be numbers (not strings). Use `null` (not `"N/A"`) for categories that were not scored. Read `metadata.json.skill_type` from `{resolved_skill_package}/metadata.json`; if the value is `"stack"`, set `stackSkill: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (the categories will be redistributed per `{scoringRulesFile}` Stack Skills rule). Likewise read `metadata.json.scope_type`; if the value is `"reference-app"`, set `referenceApp: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (redistributed per `{scoringRulesFile}` Reference-App rule — a reference app documents wiring patterns, not library export signatures).
|
|
87
90
|
|
|
88
91
|
#### 3b. Run the Scoring Script
|
|
89
92
|
|
|
@@ -112,7 +115,7 @@ Use these values for Section 4 (pass/fail/inconclusive) and Section 6 (output fo
|
|
|
112
115
|
If the script is unavailable or returns an error, fall back to manual calculation:
|
|
113
116
|
|
|
114
117
|
1. Select the weight table from `{scoringRulesFile}` for the detected mode (naive or contextual)
|
|
115
|
-
2. Determine skip conditions: Quick tier/docsOnly/state2/stackSkill skip Signature Accuracy + Type Coverage; naive mode coherence is already 0; null external validation means skip it
|
|
118
|
+
2. Determine skip conditions: Quick tier/docsOnly/state2/stackSkill/referenceApp skip Signature Accuracy + Type Coverage; naive mode coherence is already 0; null external validation means skip it
|
|
116
119
|
3. For each skipped category, set its weight to 0
|
|
117
120
|
4. Compute sum of active category weights
|
|
118
121
|
5. For each active category: `new_weight = old_weight / sum_active * 100`
|
|
@@ -70,6 +70,18 @@ When `metadata.json.skill_type == "stack"` (set `stackSkill: true` in the scorin
|
|
|
70
70
|
|
|
71
71
|
Equivalence-class note: stack skills with `docsOnly:false` / `state2:false` map to the same equivalence class as State 2 contextual rows (class B) or State 2 naive rows (class D) — the redistribution math is identical; only the `skipReasons` string changes.
|
|
72
72
|
|
|
73
|
+
### Reference-App Skills (Any Tier)
|
|
74
|
+
|
|
75
|
+
When `metadata.json.scope_type == "reference-app"` (set `referenceApp: true` in the scoring input):
|
|
76
|
+
|
|
77
|
+
- **Signature Accuracy:** N/A — a reference app documents wiring patterns (how surfaces are composed), not a library export surface the skill authors. There are no public-export signatures to grade against; the Pattern Surface replaces the Key API Summary (see `skf-create-skill` Reference-App Assembly Overrides).
|
|
78
|
+
- **Type Coverage:** N/A — same rationale; a reference app has no library type surface to cover. Coverage is measured as pattern-surface coverage (`stats.pattern_surfaces_documented`), not export/type coverage.
|
|
79
|
+
- **Weight redistribution:** Same as Quick tier / docs-only / State 2 / stack — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories (Export Coverage, Coherence, External Validation).
|
|
80
|
+
- **Applies regardless of detected tier** (Quick, Forge, Forge+, Deep) and is independent of `docsOnly` and `state2`; skip reasons combine additively. `referenceApp` and `stackSkill` are distinct scope/type signals and should not both be set for the same skill.
|
|
81
|
+
- **Detection:** step 5 reads `metadata.json.scope_type` from the skill package. If the value is `"reference-app"`, set `referenceApp: true` in the scoring input JSON. The skip reason recorded is `"reference-app (no library export signatures)"`.
|
|
82
|
+
|
|
83
|
+
Equivalence-class note: reference-app skills map to the same redistribution equivalence class as stack skills — the math is identical; only the `skipReasons` string changes.
|
|
84
|
+
|
|
73
85
|
### State 2 Source Access (Any Tier, Provenance-Map Only)
|
|
74
86
|
|
|
75
87
|
When source is not locally available and analysis resolves to State 2 (provenance-map baseline per source-access-protocol.md):
|
|
@@ -16,12 +16,28 @@
|
|
|
16
16
|
|
|
17
17
|
**Resolution order:**
|
|
18
18
|
|
|
19
|
-
1. **Prefer `metadata.json.stats.effective_denominator`** when present. `skf-create-skill` step 5 §4 writes this field for stratified-scope skills. When set, use it directly as the `exports_public_api` count for coverage scoring
|
|
19
|
+
1. **Prefer `metadata.json.stats.effective_denominator`** when present. `skf-create-skill` step 5 §4 writes this field for stratified-scope skills. When set, use it directly as the `exports_public_api` count for coverage scoring — **subject to the deflation guard below**.
|
|
20
|
+
|
|
21
|
+
**Denominator deflation guard (when `effective_denominator` is used).** `effective_denominator` is a stored count, and a gap-driven `skf-update-skill` run can lower it — legitimately (a real rescope expressed in the brief) or by gaming (deleting in-scope public exports from `metadata`/provenance and setting `effective_denominator` to the documented total to force 100% coverage). When source is readable (State 1, or State ≥ 2 with remote tools), validate it: re-derive the public surface from `scope.include` filtered by `scope.exclude` — the same union step 2 computes — and compare. If the re-derived source barrel exceeds `effective_denominator` by **more than 25%** AND the brief carries no `scope.tier_a_include` (the only legitimate narrowing mechanism), treat `effective_denominator` as **deflated**: use the re-derived source-barrel count as `total_exports` instead, and emit a **Medium**-severity gap `denominator deflation — effective_denominator below source public surface without tier_a_include` reporting both counts (`effective_denominator: {N}`, `re-derived source barrel: {M}`, `{percent}% below source`). A legitimate scope reduction is expressed in the brief — via `scope.tier_a_include`, or by adding the removed exports to `scope.exclude` so the re-derived barrel shrinks to match (the mechanism `skf-update-skill` gap-driven rescope uses) — **never** by editing `metadata.stats` to equal the documented count. This is the directional counterpart to the inflation check in step 3: inflation fires when the coarse surface is too large, deflation when the stored denominator is suspiciously small. When source is unavailable, skip the guard and annotate the report: `effective_denominator used unverified — no source access to re-derive the barrel`.
|
|
22
|
+
|
|
23
|
+
**Rust barrel re-derivation (when ast-grep `pub`-fn extraction is unreliable).** The Rust-skill condition documents the ast-grep `pub-fn` extractor as unreliable, so re-deriving a crate's barrel surface — for this guard or for the live fallback below — falls back to grep. A naive `pub`-token sweep over-counts badly: it pulls in `pub(crate)`/`pub(super)` internals, impl-block associated consts and methods, test modules, and foreign re-export files (such sweeps routinely land 30–40% above the true barrel). Enumerate deterministically instead:
|
|
24
|
+
|
|
25
|
+
- Trace the `pub use` re-export chain from the crate root (`lib.rs` / `mod.rs`) to find which modules the barrel actually re-exports; count from those, not from every file containing a `pub` token.
|
|
26
|
+
- Count only **unrestricted, module-level (column-0) `pub`** item declarations: `pub struct` / `pub enum` / `pub trait` / `pub type` / `pub const` / `pub fn` (free functions) that start at column 0.
|
|
27
|
+
- Exclude `pub(crate)` / `pub(super)` / `pub(in …)` (crate-internal, not barrel surface); anything indented (impl-block associated consts and methods are not free barrel items); `*tests.rs` and `#[cfg(test)]` modules; and files listed under `scope.exclude` (e.g. foreign re-export files).
|
|
28
|
+
|
|
29
|
+
**TypeScript / JavaScript barrel re-derivation (multi-line re-export blocks).** When re-deriving a TS/JS barrel — for the deflation guard above or the live fallback below — a line-oriented `^export` regex undercounts: a named re-export block spans continuation lines (`export type {\n A,\n B,\n} from "./x"`), and a single-line match captures only the opening line, dropping every member listed on the continuation lines. Enumerate the full block instead:
|
|
30
|
+
|
|
31
|
+
- For each `export { … }` / `export type { … }` (including the `import`-then-`export` re-export form), accumulate named members across continuation lines until the closing `}` — count every identifier in the brace list, not only those on the opening line. A renamed re-export (`export { A as B }`) counts once under its public name (`B`).
|
|
32
|
+
- Resolve `export * from "…"` to the target module's own public names — a star re-export contributes that module's surface, not a single entry.
|
|
33
|
+
- Prefer a `tsc` / `ts-morph` / AST entry-point resolution over the line regex when a TS toolchain is reachable; it resolves multi-line blocks and star re-exports natively. Fall back to the brace-accumulation grep only when no AST tool is available — the same grep-fallback posture the Rust note above takes.
|
|
20
34
|
2. **Fall back to live re-derivation** when `effective_denominator` is absent (older skills, quick-tier output, or skills compiled before this rule existed). Read the brief's scope globs from `forge-data/{skill_name}/skill-brief.yaml`, resolve them against `source_path`, filter out files matching `scope.exclude`, and compute the source API surface as the **union of named exports across the matched files only**. The skill brief's `scope.notes` field should document the stratification strategy (e.g., "Tier A: fully documented; Tier B: deferred to references; Tier C: excluded") — when present, treat it as confirmation that the curated subset is by design, not a scope gap.
|
|
21
35
|
|
|
22
36
|
**Honor `scope.tier_a_include` when present.** When re-deriving, prefer the brief-level `scope.tier_a_include` narrow include list over the coarse `scope.include`. `tier_a_include` is an optional brief field that lists only the authoring surface the brief actually intends to document (tier A), letting the denominator match the brief's authoring-vs-installing intent even when `scope.include` uses coarse globs that also match internal infrastructure. When `tier_a_include` is present, resolve its globs (still filtered by `scope.exclude`), compute the union across those files, and use that count as the denominator. When absent, fall back to resolving `scope.include`.
|
|
23
37
|
|
|
24
|
-
|
|
38
|
+
**Exclude umbrella barrel files from a `tier_a_include` re-derivation.** A barrel file (`lib.rs`, `mod.rs`, `index.ts`/`index.js`, `__init__.py`) whose public surface is dominated by `pub use` / re-export leaves re-exports the *entire* crate/package surface. Including such a file in the resolved `tier_a_include` set makes the union **grow** rather than shrink — defeating the narrowing and producing a denominator larger than `scope.include` would have. Detect an umbrella barrel as a file whose exported names are mostly (>50%) re-exports of symbols defined in other files. When a `tier_a_include` file set resolves to include an umbrella barrel, drop the barrel and count only the concrete-definition files; if dropping it leaves no meaningful surface, prefer `effective_denominator` (priority 1) instead of the inflated `tier_a_include` union.
|
|
39
|
+
|
|
40
|
+
3. **Denominator inflation check (absent `tier_a_include`).** If re-derivation used `scope.include` because no `tier_a_include` was provided, compare the resulting union count against the provenance-map entry count (when provenance-map exists). If the `scope.include` union is more than 25% larger than the provenance-map entry count, the coarse globs are almost certainly sweeping in internal infrastructure that the brief did not intend to document. Emit a **Medium**-severity gap titled `denominator inflation — coarse scope.include union exceeds authored surface` that points the user at the brief for rescoping via `scope.tier_a_include`. **When the package's barrel is an umbrella re-export file** (public surface mostly `pub use` / re-export leaves — see the umbrella-barrel note in step 2), recommend `stats.effective_denominator` (priority 1) as the **primary** remediation instead of `tier_a_include`: a `tier_a_include` that lists the umbrella barrel would recount *larger* than `scope.include`, not smaller, so it cannot clear the inflation. Report both counts (`scope.include union: {N}`, `provenance-map entries: {M}`, `{percent}% inflation`) and state that the coverage score it produced is driven by denominator inflation rather than documentation gaps. The check is skipped when provenance-map is unavailable (there is no baseline to compare against).
|
|
25
41
|
|
|
26
42
|
Leave `analysis_confidence` unchanged (still `full` or `provenance-map` per the waterfall) — stratified scope does not degrade confidence, only the denominator. Annotate the coverage report with: `Stratified scope — denominator: {effective_denominator | tier_a_include union | scope.include union} ({N} files matched, {M} exports union)`.
|
|
27
43
|
|
|
@@ -40,7 +56,7 @@
|
|
|
40
56
|
|
|
41
57
|
**Confidence:** leave `analysis_confidence` unchanged (still `full` or `provenance-map` per the waterfall). Pattern-reference does not degrade confidence — the surface is smaller than a library barrel, not lower quality. Annotate the coverage report with: `Pattern-reference — denominator: {tier_a_include union | canonicalized provenance-map count} ({N} pattern surfaces)`.
|
|
42
58
|
|
|
43
|
-
**When this clause does NOT apply:** any repo with a non-empty barrel file, any monorepo (use the stratified-scope clause), or any single-package repo whose `scope.type` is explicitly `public-api` / `specific-modules` / `component-library` / `docs-only` (those scope types have their own denominator semantics). Also does NOT apply
|
|
59
|
+
**When this clause does NOT apply:** any repo with a non-empty barrel file, any monorepo (use the stratified-scope clause), or any single-package repo whose `scope.type` is explicitly `public-api` / `specific-modules` / `component-library` / `docs-only` (those scope types have their own denominator semantics). Also does NOT apply when `scope.type: "reference-app"` — that scope type carries its own pattern-surface denominator semantics (the brief speaks for itself), so this clause's filesystem trigger is moot.
|
|
44
60
|
|
|
45
61
|
Internal module symbols are **excluded** from the coverage denominator unless they are explicitly documented in SKILL.md (in which case they count as documented extras, not missing coverage).
|
|
46
62
|
|
|
@@ -153,6 +153,7 @@ def compute_score(inp):
|
|
|
153
153
|
docs_only = inp.get("docsOnly") is True
|
|
154
154
|
state2 = inp.get("state2") is True
|
|
155
155
|
stack_skill = inp.get("stackSkill") is True
|
|
156
|
+
reference_app = inp.get("referenceApp") is True
|
|
156
157
|
threshold = inp.get("threshold") if inp.get("threshold") is not None else DEFAULT_THRESHOLD
|
|
157
158
|
scores = inp["scores"]
|
|
158
159
|
|
|
@@ -161,7 +162,7 @@ def compute_score(inp):
|
|
|
161
162
|
|
|
162
163
|
# 3. Determine skip set
|
|
163
164
|
skip_reasons = {}
|
|
164
|
-
skip_sig_type = tier == "Quick" or docs_only or state2 or stack_skill
|
|
165
|
+
skip_sig_type = tier == "Quick" or docs_only or state2 or stack_skill or reference_app
|
|
165
166
|
|
|
166
167
|
if skip_sig_type:
|
|
167
168
|
reasons = []
|
|
@@ -173,6 +174,8 @@ def compute_score(inp):
|
|
|
173
174
|
reasons.append("State 2 (provenance-map)")
|
|
174
175
|
if stack_skill:
|
|
175
176
|
reasons.append("stack skill (external type surface)")
|
|
177
|
+
if reference_app:
|
|
178
|
+
reasons.append("reference-app (no library export signatures)")
|
|
176
179
|
reason = " + ".join(reasons)
|
|
177
180
|
skip_reasons["signatureAccuracy"] = reason
|
|
178
181
|
skip_reasons["typeCoverage"] = reason
|
|
@@ -54,17 +54,30 @@ Load the test report at `{test_report_path}` and extract findings:
|
|
|
54
54
|
|
|
55
55
|
| Gap Severity | Gap Type | Change Category |
|
|
56
56
|
|-------------|----------|-----------------|
|
|
57
|
-
| Critical | Missing export documentation | NEW_EXPORT (undocumented public API) |
|
|
57
|
+
| Critical | Missing export documentation | NEW_EXPORT (undocumented public API) — unless the remediation says the export is internal / out of scope; then DELETED_EXPORT (rescope), see rule R1 |
|
|
58
58
|
| High | Signature mismatch | MODIFIED_EXPORT (signature needs update) |
|
|
59
59
|
| Medium | Missing type/interface docs | NEW_EXPORT (undocumented type) |
|
|
60
60
|
| Medium | Stale documentation | MODIFIED_EXPORT (docs reference removed export) |
|
|
61
|
-
|
|
|
61
|
+
| Critical/Medium | Coverage gap whose remediation is **removal** (export is internal, `#[doc(hidden)]`, or explicitly out of scope) | DELETED_EXPORT (rescope) — see rule R1 |
|
|
62
|
+
| Medium/High | Structural / coherence drift (output file only, no source change) | STRUCTURAL_FIX — see rule R2 |
|
|
63
|
+
| Medium | Export documented in SKILL.md/references but **missing from the provenance-map** | NEW_EXPORT (provenance-completeness) — see rule R3 |
|
|
64
|
+
| Low | Missing metadata/examples | metadata update — see rule R4 |
|
|
65
|
+
|
|
66
|
+
**Translation rules (referenced by the table above):**
|
|
67
|
+
|
|
68
|
+
- **R1 — DELETED_EXPORT (rescope).** A coverage gap is a rescope only when its remediation text names removal (`rescope`, `remove from surface`, `out of scope`) or the export is upstream `#[doc(hidden)]` / internal. Default a bare "missing export documentation" gap to NEW_EXPORT (document it); choose rescope only on that explicit signal. **Interactive:** prompt per qualifying gap — "[D] Document the export / [R] Rescope (remove from the public surface)". **Headless:** default to **document** (NEW_EXPORT); choose rescope only when the remediation explicitly says removal *and* the export is internal/`#[doc(hidden)]`. A rescope is honest only if the reduction is expressed in the brief's scope: append a `scope.amendments[]` entry (`category: "scope-expansion"`, `action: "excluded"`) **and** add the export's source path to `brief.scope.exclude`, then route the entry to merge Priority 1 (removal) so step 4/6 remove it and recompute stats from the amended scope. Never close a coverage gap by editing `metadata.stats` to equal the documented count — that is denominator deflation and `skf-test-skill` will reject it.
|
|
69
|
+
- **R2 — STRUCTURAL_FIX.** A coherence finding from `skf-scan-skill-md-structure.py` (e.g., `table_drift`, `unbalanced_fences`, a broken intra-skill anchor) that touches the generated output file only, with no source change and no provenance entry change. Carries `remediation` text describing the surgical markdown edit. Routes to merge Priority 8 (generated-markdown edit only); it never adds, modifies, or removes a provenance `entries[]` row.
|
|
70
|
+
- **R3 — provenance-completeness.** An export documented in SKILL.md/`references/` but absent from the provenance-map. These are documented-and-known, so `unknown` is never correct: route to re-extract §0a source resolution **regardless of severity**, gated only on `source_root` being pinned and readable (see re-extract.md §0).
|
|
71
|
+
- **R4 — metadata update.** A metadata-coherence patch that changes no export's source (e.g., reconcile a divergent `stats` count). Routes to merge Priority 8b and is applied by write.md §2 *before* the automatic stat recount (see merge.md §3 / write.md §2).
|
|
62
72
|
|
|
63
73
|
4. Build the change manifest from translated gaps — no file-level timestamp comparison needed since source hasn't changed. For each manifest entry, propagate these fields from the test report finding so step 3 can resolve the export against live source:
|
|
64
74
|
|
|
65
75
|
- **`severity`** — the Gap Report severity (`Critical`, `High`, `Medium`, `Low`, `Info`). Step-03 §0 and step 6 §3 gate the null-citation fallback on severity: Critical/High gaps must produce AST provenance, Medium/Low/Info gaps may degrade to `unknown`.
|
|
66
76
|
- **`source_citation: {file, line}`** — populated only when the finding's `Source:` field is a `file:line` pair (e.g., a Gap Report row that cites `packages/utils/src/builder-utils.ts:33`). Step-03 §0 uses this field to perform a live spot-check against source rather than flagging the export as `unknown`. Omit when the `Source:` field is a region reference (e.g., `@storybook/addon-docs control primitives`) or missing.
|
|
67
77
|
- **`remediation_paths: [path, ...]`** — path-like tokens extracted from the finding's `Remediation:` text: any substring matching a recognized source file extension (`.ts`, `.tsx`, `.js`, `.jsx`, `.mjs`, `.cjs`, `.py`, `.rs`, `.go`, `.java`, `.rb`, `.c`, `.h`, `.cpp`), or a directory/glob fragment under the project's source root. Include every matching path verbatim. Step-03 §0a uses this list as the source set for its Targeted Re-Extraction Branch when `source_citation` is absent and severity is Critical/High. Omit the field when the Remediation text names no paths — the entry then falls through to `unknown` or to §0a's halt, depending on severity.
|
|
78
|
+
- **`change_category`** — the Change Category resolved from the table above (`NEW_EXPORT`, `MODIFIED_EXPORT`, `DELETED_EXPORT`, `STRUCTURAL_FIX`, or `metadata update`). Step 3 §1a partitions on this field; merge.md §3 dispatches on it.
|
|
79
|
+
- **`remediation`** — the finding's full `Remediation:` text, verbatim. Required for `STRUCTURAL_FIX` (the surgical markdown edit), `metadata update` (the patch description), and `DELETED_EXPORT` rescope (the removal rationale recorded in the `scope.amendments[]` entry). For `NEW_EXPORT` / `MODIFIED_EXPORT` it is informational.
|
|
80
|
+
- **`provenance_completeness: true`** — set on a `NEW_EXPORT` entry whose gap is rule R3 (documented in SKILL.md/`references/` but absent from the provenance-map). Step 3 §0 routes these to §0a regardless of severity.
|
|
68
81
|
5. Set `gap_count` from the total number of translated entries
|
|
69
82
|
6. **Skip to section 5** (Display Change Summary) with the gap-derived manifest
|
|
70
83
|
|
|
@@ -42,6 +42,7 @@ Follow the merge priority order from {mergeConflictRulesFile}:
|
|
|
42
42
|
- Check if deleted export has attached [MANUAL] blocks
|
|
43
43
|
- If [MANUAL] attached: flag as ORPHAN conflict (do not remove)
|
|
44
44
|
- If no [MANUAL]: remove generated content cleanly
|
|
45
|
+
- **Gap-driven rescopes** (`DELETED_EXPORT` from detect-changes §0 rule R1, verification `rescoped`) are processed here with the same removal. Step 6 also removes the provenance `entries[]` row and recomputes `stats` from the amended `brief.scope` (write.md §2/§3). The brief's `scope.amendments[]` (`action: "excluded"`) + `scope.exclude` entry must already exist — step 3 §0 HALTs otherwise, so no unscoped removal reaches here.
|
|
45
46
|
|
|
46
47
|
**Priority 2 — Process MOVED exports:**
|
|
47
48
|
- Update file:line citations in generated content
|
|
@@ -97,6 +98,27 @@ For each entry in the in-context `promoted_docs_new[]` list:
|
|
|
97
98
|
|
|
98
99
|
**If `promoted_docs_new[]` is empty:** skip Priority 7 silently. No report entry.
|
|
99
100
|
|
|
101
|
+
**Priority 8 — Process STRUCTURAL_FIX entries (gap-driven, from detect-changes §0 rule R2):**
|
|
102
|
+
|
|
103
|
+
For each `STRUCTURAL_FIX` entry forwarded by step 3 §0/1a:
|
|
104
|
+
|
|
105
|
+
- Apply the surgical edit described in the entry's `remediation` text to the **generated output file only** (e.g., escape an unescaped `|` inside a code span, balance a fence, repair a broken intra-skill anchor in SKILL.md or a `references/*.md`).
|
|
106
|
+
- Do **not** add, modify, or remove any provenance `entries[]` row — STRUCTURAL_FIX never touches the provenance map.
|
|
107
|
+
- Preserve any [MANUAL] blocks; if the fix location overlaps a [MANUAL] block, flag as a POSITION conflict instead of editing.
|
|
108
|
+
- Record in the update report: `"Structural fix: {remediation summary} at {file}:{line}"`.
|
|
109
|
+
|
|
110
|
+
**If no STRUCTURAL_FIX entries:** skip Priority 8 silently.
|
|
111
|
+
|
|
112
|
+
**Priority 8b — Process metadata-update entries (gap-driven, from detect-changes §0 rule R4):**
|
|
113
|
+
|
|
114
|
+
For each `metadata update` entry forwarded by step 3 §0/1a:
|
|
115
|
+
|
|
116
|
+
- Queue the surgical metadata patch described in the entry's `remediation` (e.g., reconcile a divergent `stats` count, add an explanatory stat) in workflow context as `metadata_patches[]` for write.md §2 to apply **before** its automatic stat recount.
|
|
117
|
+
- Touch no provenance `entries[]` row and no generated markdown — this priority only stages the patch; write.md §2 applies it.
|
|
118
|
+
- Record in the update report: `"Metadata patch queued: {remediation summary}"`.
|
|
119
|
+
|
|
120
|
+
**If no metadata-update entries:** skip Priority 8b silently.
|
|
121
|
+
|
|
100
122
|
### 4. Check for Conflicts
|
|
101
123
|
|
|
102
124
|
Scan all merge operations for flagged conflicts:
|
|
@@ -66,14 +66,21 @@ The helper emits a result envelope:
|
|
|
66
66
|
- **`mismatch`** (helper exit 2): HALT immediately with exit status `halted-for-workspace-drift`. Display the helper's `halt_message` verbatim — it already substitutes `{pinned_commit}`, `{source_ref or "unset"}`, `{source_root}`, `{head_sha}`, and the suggested `git checkout` command. Do not proceed to bullet 1. Step-04 merge has not run; no partial writes.
|
|
67
67
|
|
|
68
68
|
1. Use the provenance map already loaded in step 1 (at `{forge_version}/provenance-map.json`) — do not re-read
|
|
69
|
-
2.
|
|
69
|
+
2. **Partition by change category, then iterate the export-bearing entries.** Entries that do not name an export skip the per-export verification below — they have no symbol to resolve against source:
|
|
70
|
+
- **`STRUCTURAL_FIX`** (detect-changes §0 rule R2): forward verbatim to the merge step (its `remediation` text describes a generated-markdown edit). No spot-check, no provenance lookup, no `entries[]` change.
|
|
71
|
+
- **`metadata update`** (rule R4): forward the metadata-patch payload to the merge step. No spot-check, no provenance lookup.
|
|
72
|
+
|
|
73
|
+
Carry both through workflow context to merge.md unchanged. Then, for each export-bearing entry (`NEW_EXPORT`, `MODIFIED_EXPORT`, `DELETED_EXPORT`):
|
|
74
|
+
- **If the entry is `DELETED_EXPORT` (rescope, rule R1):** do not resolve against source — the export is being removed from the public surface. Record `verification: rescoped` and flag for merge Priority 1 (removal). Confirm the brief carries the matching `scope.amendments[]` (`action: "excluded"`) + `scope.exclude` entry that step 2 R1 wrote; if absent, HALT — a rescope without a brief scope amendment is denominator deflation and must not be written.
|
|
70
75
|
- Look up the export by `name` in `provenance_map.exports` — read `source_file` and `source_line`
|
|
71
76
|
- **If export not found in provenance map:**
|
|
72
77
|
- **If the manifest entry has a `source_citation` (propagated from the test report by step 2 §0 bullet 4):** read the file at that citation's `file:line ± 5` lines and verify the symbol name still appears within that window. Record a full `verified` / `moved` / `missing` entry using the citation as the starting location — same spot-check logic as the "export found" branch below, keyed on the manifest-supplied citation instead of the provenance map. The export is still flagged `NEW_EXPORT` for the merge step; this branch only upgrades the provenance entry from `unknown` to a live spot-check result so step 6 writes `source_file` / `source_line` instead of `null`.
|
|
78
|
+
- **If the entry is a provenance-completeness gap (rule R3 — documented in SKILL.md/`references/` but missing from the provenance-map) AND `source_root` is pinned and readable:** route this entry to §0a (Targeted Re-Extraction Branch) **regardless of severity**. These exports are documented-and-known; `unknown` is never the correct outcome for them. §0a resolves them against pinned source and records a full `re-extracted` provenance entry. This route takes precedence over the severity-gated branches below.
|
|
73
79
|
- **If the manifest entry has no `source_citation` but has a non-empty `remediation_paths[]` AND `severity` is `Critical` or `High`:** route this entry to §0a (Targeted Re-Extraction Branch). §0a scans the remediation paths with the tier-appropriate extractor and, on success, records a full verification record with `verification: re-extracted`, a live `provenance_citation`, and full signature/params/return-type fields for the merge step to consume as a NEW_EXPORT. On failure, §0a halts the workflow — Critical/High gaps are not allowed to degrade to `unknown`. See §0a for the procedure, the consolidated halt protocol, and the output record shape.
|
|
74
|
-
- **If the manifest entry has no `source_citation` AND (`remediation_paths[]` is empty OR `severity` is `Medium`, `Low`, or `Info`):** record as new (`provenance_citation: unknown`) — no spot-check possible; flag for merge step to handle as `NEW_EXPORT`. Step-06 §3 only accepts null `source_file` / `source_line` for these lower-severity unknowns; a Critical/High unknown reaching step 6 indicates §0a was skipped or bypassed and is a workflow bug.
|
|
80
|
+
- **If the manifest entry has no `source_citation` AND (`remediation_paths[]` is empty OR `severity` is `Medium`, `Low`, or `Info`) AND it is not a provenance-completeness gap:** record as new (`provenance_citation: unknown`) — no spot-check possible; flag for merge step to handle as `NEW_EXPORT`. Step-06 §3 only accepts null `source_file` / `source_line` for these lower-severity unknowns; a Critical/High unknown reaching step 6 indicates §0a was skipped or bypassed and is a workflow bug.
|
|
75
81
|
- **If export found:** read the source file at `source_line ± 5` lines and verify the symbol name still appears within that window
|
|
76
|
-
- Record verification outcome: `verified` (symbol at recorded line), `moved` (symbol found elsewhere in same file — record new line), `missing` (symbol not found in file), `re-extracted` (resolved via §0a from `remediation_paths[]`), or `unknown` (no usable provenance data)
|
|
82
|
+
- Record verification outcome: `verified` (symbol at recorded line), `moved` (symbol found elsewhere in same file — record new line), `missing` (symbol not found in file), `re-extracted` (resolved via §0a from `remediation_paths[]` or rule R3), `rescoped` (DELETED_EXPORT — flagged for removal), or `unknown` (no usable provenance data)
|
|
83
|
+
- **Public-reachability gate (`NEW_EXPORT` only):** before an entry is flagged `NEW_EXPORT` for the merge step, confirm the symbol is reachable as **public API** — not merely that a `pub` / `export` / top-level definition exists at the citation. The spot-check already read `source_file` at `source_line ± 5`; extend that read to resolve the symbol's module path and confirm at least one of: (a) it is re-exported from the package entry-point barrel (`lib.rs` `pub use`, `index.ts` / `index.js` export, `__init__.py` import or `__all__`), or (b) every ancestor module on the path from that barrel is public (Rust `pub mod`, an exported TS namespace, a non-underscore Python package). A `pub(crate)` / `pub(super)` item, or one under a private module (e.g. `mod authority;` declared without `pub`), is **not** reachable. **On failure:** do not document it as public — drop it from the export-bearing set so it is never written to the documented `exports[]` array or SKILL.md, and re-queue it as a `metadata update` (rule R4) recording `reclassified: internal-unreachable` for the evidence report. Do **not** hand-set any `stats` count: write.md §2's automatic recount derives `exports_internal` / `exports_total` from the merged surface and owns those fields. This honors the workflow rule that documented API must be importable by users — a `pub`-but-internal symbol would otherwise inflate the documented public surface with a type users cannot import.
|
|
77
84
|
3. Build a minimal extraction results block matching section 4's shape, with `mode: gap-driven` and per-export verification records:
|
|
78
85
|
|
|
79
86
|
```
|
|
@@ -89,10 +96,10 @@ The helper emits a result envelope:
|
|
|
89
96
|
Per-export verification:
|
|
90
97
|
{export_name}:
|
|
91
98
|
provenance_citation: {source_file}:{source_line}
|
|
92
|
-
verification: verified|moved|missing|unknown|re-extracted
|
|
99
|
+
verification: verified|moved|missing|unknown|re-extracted|rescoped
|
|
93
100
|
new_location: {source_file}:{new_line} # set when moved OR re-extracted
|
|
94
101
|
resolution_source: remediation-paths # set only when verification == re-extracted
|
|
95
|
-
gap_category: NEW_EXPORT|MODIFIED_EXPORT|metadata_update
|
|
102
|
+
gap_category: NEW_EXPORT|MODIFIED_EXPORT|DELETED_EXPORT|metadata_update
|
|
96
103
|
|
|
97
104
|
Per-file extractions: # populated only when §0a produced re-extracted records
|
|
98
105
|
{file_path}:
|
|
@@ -110,7 +117,7 @@ The helper emits a result envelope:
|
|
|
110
117
|
4. Set `no_reextraction: true` in workflow context — step 6 will use this flag to skip stale `source_file`/`source_line`/`confidence` field updates for `verified` exports. `moved` exports get updated citations; `re-extracted` exports get full fresh provenance from §0a's extraction records (see step 6 §3). The flag is a global gap-driven marker, not a per-entry one — step 6 dispatches on each verification outcome independently.
|
|
111
118
|
5. **Skip all remaining sections of step 3** — sections 1–5 are source-drift extraction paths that do not apply. Display the summary below and load `{nextStepFile}` to proceed directly to the merge step.
|
|
112
119
|
|
|
113
|
-
"**Gap-driven re-extraction.** Verified {verified_count}/{gap_count} citations against live source. Moved: {moved_count}. Missing: {missing_count}. Re-extracted (via remediation paths, §0a): {re_extracted_count}. Unknown (not in provenance map): {unknown_count}. Proceeding to merge."
|
|
120
|
+
"**Gap-driven re-extraction.** Verified {verified_count}/{gap_count} citations against live source. Moved: {moved_count}. Missing: {missing_count}. Re-extracted (via remediation paths or provenance-completeness, §0a): {re_extracted_count}. Rescoped (removed from surface): {rescoped_count}. Unknown (not in provenance map): {unknown_count}. Proceeding to merge."
|
|
114
121
|
|
|
115
122
|
**If normal mode (`update_mode` unset or not `gap-driven`):** Continue with docs-only check and source extraction below.
|
|
116
123
|
|
|
@@ -118,7 +125,10 @@ The helper emits a result envelope:
|
|
|
118
125
|
|
|
119
126
|
**Do not execute this section sequentially.** It is a helper procedure invoked by §0 bullet 2 when specific conditions are met (see below). Normal-mode runs, and gap-driven runs where every entry has a `source_citation` or qualifies as `Medium`/`Low`/`Info` unknown, skip this section entirely. §0's "skip sections 1–5" instruction does not apply here — §0a is addressed by name from §0, not by sequential fall-through.
|
|
120
127
|
|
|
121
|
-
**Used by:** §0 bullet 2,
|
|
128
|
+
**Used by:** §0 bullet 2, in either of two cases:
|
|
129
|
+
|
|
130
|
+
- a manifest entry has no `source_citation`, has non-empty `remediation_paths[]`, and `severity` is `Critical` or `High`; **or**
|
|
131
|
+
- a provenance-completeness gap (rule R3 — documented but missing from the provenance-map), routed here **regardless of severity** as long as `source_root` is pinned and readable. For this case, treat the export's documented `source` reference (or `remediation_paths[]` when present) as the path set to scan.
|
|
122
132
|
|
|
123
133
|
**Purpose:** produce AST-backed provenance for citation-less Critical/High gaps so step 6 §3 never writes `source_file: null` for blocking findings. Honors the workflow-level rule **Never hallucinate — every statement must have AST provenance** against the most common gap-driven trigger — a failing test report whose Gap Report `Source:` field is a region reference (e.g., `@storybook/addon-docs control primitives`) rather than a `file:line` pair. Gap-driven mode skips §1 through §5, so §0a is also the only place §1b's source-access and extraction machinery is invoked during gap-driven runs.
|
|
124
134
|
|
|
@@ -138,6 +148,8 @@ The helper emits a result envelope:
|
|
|
138
148
|
- `resolution_source: remediation-paths`
|
|
139
149
|
- `confidence: T1` (AST-extracted) or `T1-low` (pattern-matched fallback)
|
|
140
150
|
- the full extraction signature (type, params, return_type, docstring) — mirror the shape of §4's per-file extraction record so step 4 Priority 5 can merge it with the same code path used in normal mode.
|
|
151
|
+
|
|
152
|
+
Then apply the §0 bullet 2 **public-reachability gate** to each matched symbol before recording it as a NEW_EXPORT: §0a has full source access, so resolve the symbol's module path and confirm barrel re-export or a fully-`pub` module chain. If it is unreachable (`pub(crate)` / private module), do **not** record `re-extracted` — drop it from the export-bearing set and re-queue the entry as a `metadata update` (rule R4, `reclassified: internal-unreachable`), exactly as the gate specifies. A re-extracted symbol that is not public API is not a documentable NEW_EXPORT. This is a resolution, not an `unresolved[]` failure (step 5) — the symbol was found, just not public — so it does not trigger the Critical/High HALT.
|
|
141
153
|
5. **Track failures across all qualifying entries.** Collect every entry whose symbol was not found in any scanned remediation path into an `unresolved[]` list. After processing every qualifying entry, if `unresolved[]` is non-empty: HALT with a consolidated report listing every unresolved entry (`name`, `severity`, `remediation_paths`, `files_scanned`, `exports_found_in_scan`). Template:
|
|
142
154
|
|
|
143
155
|
```
|
|
@@ -59,6 +59,8 @@ SKILL.md was written in step 4 section 6b. Verify the write landed intact before
|
|
|
59
59
|
### 2. Write Updated metadata.json
|
|
60
60
|
|
|
61
61
|
Update `{skill_package}/metadata.json`:
|
|
62
|
+
- **First, apply any queued `metadata_patches[]`** (staged by merge Priority 8b from gap-driven `metadata update` entries): apply each surgical patch described in the gap's remediation (reconcile a divergent count, add an explanatory stat, etc.) *before* the automatic recount below, so the recount overrides only the fields it owns and the patch survives for any field it does not. If a patch and the recount disagree on a field the recount owns (e.g., `exports_documented`), the recount wins — log the divergence so a still-stale stat surfaces in the report rather than being silently overwritten.
|
|
63
|
+
- **For gap-driven rescopes** (`DELETED_EXPORT` / verification `rescoped`): the removed exports are already dropped from the `exports` array below, and `stats` recompute from that reduced surface — never set a `stats` count by hand to match the documented total. The reduction is justified by the `brief.scope.exclude` + `scope.amendments[]` (`action: "excluded"`) written in step 2; the recount simply reflects the smaller surface.
|
|
62
64
|
- Update `version`: **if `update_mode == "gap-driven"`, do not bump — the skill is being repaired against the same source commit, so leave `version` unchanged and update only `generation_date` / `last_update` below.** This keeps metadata `version` consistent with the on-disk `{skill_package}` path, which step 4 §6b also leaves unchanged in gap-driven mode (see step 4 §6b's "If the source version detected during step 3 differs..." carve-out — in gap-driven mode no source version is detected, so step 4 writes into the existing version directory). Otherwise, if a source version was detected during re-extraction and differs from the current metadata version, use the source version; otherwise increment patch version
|
|
63
65
|
- Update `generation_date` timestamp to current ISO-8601 date
|
|
64
66
|
- Update `exports` array to reflect current export list
|
|
@@ -85,6 +87,7 @@ Dispatch per-entry on the verification outcome recorded by step 3 — gap-driven
|
|
|
85
87
|
- **`moved` exports**: update `source_line` (and `source_file` if different) to the new location recorded by the spot-check. Do not touch other fields.
|
|
86
88
|
- **`re-extracted` exports** (resolved via step 3 §0a's Targeted Re-Extraction Branch from `remediation_paths[]`): write a full entry — `source_file`, `source_line`, `confidence`, `extraction_method`, `ast_node_type`, `params[]`, `return_type` — from §0a's fresh AST extraction record. This is the only gap-driven outcome that produces normal-mode-quality provenance; do NOT fall through to the byte-identical preservation above.
|
|
87
89
|
- **`unknown` exports** (not in provenance map; no `source_citation`; `severity` is `Medium`, `Low`, or `Info`, OR `remediation_paths[]` was empty and §0a did not halt): add new entries with fields populated from step 4 merge output. `source_file`/`source_line` may be `null` here — leave these fields unset rather than writing stale values. **This path is only acceptable for `severity` in `Medium`, `Low`, or `Info`.** A Critical/High `unknown` reaching this branch indicates step 3 §0a was skipped or bypassed and is a workflow bug — step 3 §0a should have halted with status `halted-for-remediation-path` before step 6 ran. If you encounter one, halt with a pointer to §0a rather than writing null citations for a blocking gap.
|
|
90
|
+
- **`rescoped` exports** (`DELETED_EXPORT`, removed from the public surface per detect-changes §0 rule R1): remove the entry from the provenance map — identical to the normal-mode "For deleted exports" path below. The reduction's audit trail is the `brief.scope.exclude` + `scope.amendments[]` (`action: "excluded"`) entry written in step 2; do not record the removal by editing `stats` directly — §2 recomputes `stats` from the reduced `exports` array.
|
|
88
91
|
- Skip the "For each export in the updated skill" bullets below — they apply only to normal re-extraction mode.
|
|
89
92
|
|
|
90
93
|
**For each export in the updated skill (normal mode only):**
|
|
@@ -9,7 +9,7 @@ description: Pre-code stack feasibility verification against architecture and PR
|
|
|
9
9
|
|
|
10
10
|
Cross-references generated skills against architecture and PRD documents to produce a feasibility report with evidence-backed integration verdicts, coverage analysis, and requirements mapping. This is a read-only workflow — it never modifies skills or input documents, only reads and produces a feasibility report. Every verdict must cite specific APIs, types, or function signatures from the generated skills.
|
|
11
11
|
|
|
12
|
-
**Schema contract:** This skill is the PRODUCER of the feasibility report schema defined in `src/shared/references/feasibility-report-schema.md
|
|
12
|
+
**Schema contract:** This skill is the PRODUCER of the feasibility report schema defined under the SKF shared references (`_bmad/skf/shared/references/feasibility-report-schema.md` in installed mode; `src/shared/references/feasibility-report-schema.md` in a dev checkout). All report outputs emit `schemaVersion: "1.0"` in frontmatter, use only the defined verdict tokens (`Verified|Plausible|Risky|Blocked` per pair; `FEASIBLE|CONDITIONALLY_FEASIBLE|NOT_FEASIBLE` overall), follow the fixed section-heading order, and are written through `src/shared/scripts/skf-atomic-write.py write` to both the timestamped file and the stable `-latest.md` copy.
|
|
13
13
|
|
|
14
14
|
## Conventions
|
|
15
15
|
|
|
@@ -54,7 +54,7 @@ These rules apply to every step in this workflow:
|
|
|
54
54
|
| **Inputs** | architecture_doc_path [required], prd_path [optional], previous_report_path [optional] |
|
|
55
55
|
| **Flags** | `--headless` / `-H` (auto-resolve all gates); `--architecture-doc <path>` (skip step 1 prompt for the required input); `--prd <path>` (skip step 1 prompt for the optional PRD); `--previous-report <path>` (skip step 1 prompt for delta comparison) |
|
|
56
56
|
| **Gates** | step 1: Input Gate [use args] | step 6: Confirm Gate [C] |
|
|
57
|
-
| **Outputs** | `feasibility-report-{projectSlug}-{timestamp}.md` and `feasibility-report-{projectSlug}-latest.md` (copy, not symlink) per `
|
|
57
|
+
| **Outputs** | `feasibility-report-{projectSlug}-{timestamp}.md` and `feasibility-report-{projectSlug}-latest.md` (copy, not symlink) per the SKF shared feasibility report schema (`_bmad/skf/shared/references/feasibility-report-schema.md`; `src/shared/references/…` in a dev checkout) — with integration verdicts, coverage analysis, recommendations, and evidence sources; plus `verify-stack-result-{timestamp}.json` and `verify-stack-result-latest.json` |
|
|
58
58
|
| **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true. Per-flag args (`--architecture-doc`, `--prd`, `--previous-report`) consumed at the gates that would otherwise prompt. |
|
|
59
59
|
| **Exit codes** | See "Exit Codes" below |
|
|
60
60
|
|
|
@@ -37,7 +37,7 @@ deltaUnchanged: null
|
|
|
37
37
|
**PRD Document:** {prdDoc}
|
|
38
38
|
**Skills Analyzed:** {skillsAnalyzed}
|
|
39
39
|
|
|
40
|
-
> Schema contract: `
|
|
40
|
+
> Schema contract: `{feasibilitySchemaRef}` (schemaVersion `1.0`). Consumers MUST halt on `schemaVersion` mismatch.
|
|
41
41
|
|
|
42
42
|
## Executive Summary
|
|
43
43
|
|
|
@@ -1,7 +1,9 @@
|
|
|
1
1
|
---
|
|
2
2
|
nextStepFile: 'integrations.md'
|
|
3
3
|
coveragePatternsData: '{coveragePatternsPath}'
|
|
4
|
-
|
|
4
|
+
feasibilitySchemaProbeOrder:
|
|
5
|
+
- '{project-root}/_bmad/skf/shared/references/feasibility-report-schema.md'
|
|
6
|
+
- '{project-root}/src/shared/references/feasibility-report-schema.md'
|
|
5
7
|
atomicWriteProbeOrder:
|
|
6
8
|
- '{project-root}/_bmad/skf/shared/scripts/skf-atomic-write.py'
|
|
7
9
|
- '{project-root}/src/shared/scripts/skf-atomic-write.py'
|
|
@@ -64,9 +66,18 @@ For each referenced technology in the list:
|
|
|
64
66
|
4. Compare the resulting basenames/segments against the tech tokens via case-insensitive equality (no substring/fuzzy matching)
|
|
65
67
|
5. A match on either `source_repo` basename or `source_root` last segment counts as a hit
|
|
66
68
|
|
|
69
|
+
**Detect a deliberate-removal signal (from the architecture document):** Before assigning Covered/Missing, check whether the referenced technology is explicitly marked for removal or replacement in the architecture document itself. Be conservative — recognize a removal signal ONLY when one of these is present, and when in doubt do NOT treat it as removal:
|
|
70
|
+
- The technology is listed under a section whose heading matches (case-insensitive) one of: "deprecated", "removed", "legacy", "migrating away", "being replaced", "to be removed", "sunset", "retiring".
|
|
71
|
+
- The technology's own mention carries an inline removal annotation, e.g. "(deprecated)", "(being replaced by …)", "(removing)", "(to be removed)", "(legacy)".
|
|
72
|
+
|
|
73
|
+
Record the cited section heading or annotation text as evidence for every technology flagged this way.
|
|
74
|
+
|
|
67
75
|
**Assign verdict:**
|
|
68
76
|
- **Covered** — a matching skill exists in the inventory
|
|
69
|
-
- **
|
|
77
|
+
- **Replaced** — no matching skill exists AND a deliberate-removal signal (above) was found; the technology is intentionally being removed/replaced, so no skill should exist for it
|
|
78
|
+
- **Missing** — no matching skill found and no removal signal
|
|
79
|
+
|
|
80
|
+
**Replaced** technologies are EXCLUDED from the coverage denominator — they are not a gap to close. The denominator (`live_count`) is the count of referenced technologies that are **Covered** or **Missing**; **Replaced** technologies do not count toward it.
|
|
70
81
|
|
|
71
82
|
Build the coverage matrix as a structured table.
|
|
72
83
|
|
|
@@ -90,9 +101,9 @@ Extra and Orphan skills are informational only. They do not affect the coverage
|
|
|
90
101
|
|
|
91
102
|
| Technology | Source Section | Skill Match | Verdict |
|
|
92
103
|
|------------|---------------|-------------|---------|
|
|
93
|
-
| {tech_name} | {section_heading} | {skill_name or '—'} | {Covered / Missing} |
|
|
104
|
+
| {tech_name} | {section_heading} | {skill_name or '—'} | {Covered / Missing / Replaced} |
|
|
94
105
|
|
|
95
|
-
**Coverage: {covered_count}/{
|
|
106
|
+
**Coverage: {covered_count}/{live_count} ({percentage}%)** — `live_count` excludes technologies marked **Replaced** (being removed/replaced); they are not a coverage gap.
|
|
96
107
|
|
|
97
108
|
{IF 100% coverage AND no Extra skills:}
|
|
98
109
|
**All referenced technologies have a matching skill. No extra skills detected.**
|
|
@@ -102,6 +113,11 @@ Extra and Orphan skills are informational only. They do not affect the coverage
|
|
|
102
113
|
{For each missing technology:}
|
|
103
114
|
- `{tech_name}` → Run **[CS] Create Skill** or **[QS] Quick Skill** for `{tech_name}`, then re-run **[VS]**
|
|
104
115
|
|
|
116
|
+
{IF any Replaced:}
|
|
117
|
+
**Replaced / Being Removed (informational — no skill needed):**
|
|
118
|
+
{For each replaced technology:}
|
|
119
|
+
- `{tech_name}` — marked for removal/replacement in the architecture document ({cited_removal_evidence}); excluded from coverage. No skill should be created; if the technology is not actually being removed, correct the architecture document and re-run **[VS]**.
|
|
120
|
+
|
|
105
121
|
{IF any Extra:}
|
|
106
122
|
**Extra Skills (informational):**
|
|
107
123
|
{For each extra skill:}
|
|
@@ -111,17 +127,30 @@ Extra and Orphan skills are informational only. They do not affect the coverage
|
|
|
111
127
|
|
|
112
128
|
**Resolve `{atomicWriteHelper}`** from `{atomicWriteProbeOrder}`; first existing path wins. HALT if no candidate exists.
|
|
113
129
|
|
|
130
|
+
**Resolve `{feasibilitySchemaRef}`** from `{feasibilitySchemaProbeOrder}`; first existing path wins (installed SKF module path first, dev-checkout `src/` fallback).
|
|
131
|
+
|
|
114
132
|
Write the **Coverage Analysis** section to `{outputFile}` (see `{feasibilitySchemaRef}` — section headings are fixed and ordered: `## Executive Summary`, `## Coverage Analysis`, `## Integration Verdicts`, `## Recommendations`, `## Evidence Sources`):
|
|
115
133
|
- Include the full coverage table
|
|
116
134
|
- Include coverage percentage
|
|
117
135
|
- Include missing skill recommendations
|
|
136
|
+
- Include the Replaced (being removed/replaced) subdivision from section 3, with the cited removal evidence — these are NOT gaps and carry no [CS]/[QS] recommendation
|
|
118
137
|
- Include the Extra (unreferenced) and Orphan (source_repo unresolvable) subdivisions from section 4
|
|
119
|
-
- Update frontmatter: append `'coverage'` to `stepsCompleted`; set `coveragePercentage` (integer 0..100)
|
|
138
|
+
- Update frontmatter: append `'coverage'` to `stepsCompleted`; set `coveragePercentage` (integer 0..100) computed as `covered_count / live_count` (Replaced technologies excluded from the denominator; if `live_count` is 0, set `coveragePercentage: 0`)
|
|
120
139
|
- Pipe the updated full content through `python3 {atomicWriteHelper} write --target {outputFile}` and again with `--target {outputFileLatest}`
|
|
121
140
|
|
|
122
141
|
### 7. Auto-Proceed to Next Step
|
|
123
142
|
|
|
124
|
-
{IF
|
|
143
|
+
{IF live_count is 0 — every referenced technology is marked Replaced:}
|
|
144
|
+
"**⚠️ Every referenced technology is marked for removal/replacement — there is no live technology to verify.** Coverage is reported as 0%; the architecture document describes only technologies being removed, so the stack cannot be assessed as described.
|
|
145
|
+
|
|
146
|
+
**Recommended:** Update the architecture document to describe the technologies that remain after the pivot, then re-run [VS].
|
|
147
|
+
|
|
148
|
+
**Select:** [X] Halt workflow (recommended) | [C] Continue anyway"
|
|
149
|
+
|
|
150
|
+
- IF X: "**Workflow halted.** Update the architecture document and re-run [VS] when ready." — END workflow
|
|
151
|
+
- IF C: "**Continuing — the analysis covers only technologies marked for removal and will be limited.**" Load, read the full file and then execute `{nextStepFile}`.
|
|
152
|
+
|
|
153
|
+
{IF coveragePercentage is 0% AND live_count > 0:}
|
|
125
154
|
"**⚠️ 0% coverage — no matching skills found for any referenced technology.** All subsequent analysis (integration, requirements) will be vacuous and produce empty tables.
|
|
126
155
|
|
|
127
156
|
**Recommended:** Generate skills with [CS] or [QS] for your architecture technologies, then re-run [VS].
|
|
@@ -1,6 +1,8 @@
|
|
|
1
1
|
---
|
|
2
2
|
nextStepFile: 'coverage.md'
|
|
3
|
-
|
|
3
|
+
feasibilitySchemaProbeOrder:
|
|
4
|
+
- '{project-root}/_bmad/skf/shared/references/feasibility-report-schema.md'
|
|
5
|
+
- '{project-root}/src/shared/references/feasibility-report-schema.md'
|
|
4
6
|
atomicWriteProbeOrder:
|
|
5
7
|
- '{project-root}/_bmad/skf/shared/scripts/skf-atomic-write.py'
|
|
6
8
|
- '{project-root}/src/shared/scripts/skf-atomic-write.py'
|
|
@@ -113,6 +115,8 @@ Each helper-emitted entry includes: `skill_name`, `version`, `language`, `confid
|
|
|
113
115
|
|
|
114
116
|
**Resolve `{atomicWriteHelper}`** from `{atomicWriteProbeOrder}`; first existing path wins. HALT if no candidate exists.
|
|
115
117
|
|
|
118
|
+
**Resolve `{feasibilitySchemaRef}`** from `{feasibilitySchemaProbeOrder}`; first existing path wins (installed SKF module path first, dev-checkout `src/` fallback). HALT if no candidate exists.
|
|
119
|
+
|
|
116
120
|
This skill is the PRODUCER of the feasibility report schema defined in `{feasibilitySchemaRef}`. All outputs MUST conform to that schema — in particular: `schemaVersion: "1.0"`, the defined verdict token set (`Verified|Plausible|Risky|Blocked`; overall `FEASIBLE|CONDITIONALLY_FEASIBLE|NOT_FEASIBLE`), the filename pattern, and the section-heading order.
|
|
117
121
|
|
|
118
122
|
**Filename variables** (already computed at activation — do not re-derive):
|
|
@@ -121,7 +125,7 @@ This skill is the PRODUCER of the feasibility report schema defined in `{feasibi
|
|
|
121
125
|
- `outputFile` resolves to `{outputFolderPath}/feasibility-report-{project_slug}-{timestamp}.md` per the stage frontmatter template
|
|
122
126
|
- `outputFileLatest` resolves to `{outputFolderPath}/feasibility-report-{project_slug}-latest.md` (a copy, not a symlink — per schema)
|
|
123
127
|
|
|
124
|
-
**Load** `{reportTemplatePath}` (the customize-aware template path resolved in SKILL.md On Activation §4) and stage the initial content.
|
|
128
|
+
**Load** `{reportTemplatePath}` (the customize-aware template path resolved in SKILL.md On Activation §4) and stage the initial content. Substitute the template's `Schema contract:` line `{feasibilitySchemaRef}` placeholder with the resolved schema path so every emitted report cites a path that exists on the running machine.
|
|
125
129
|
|
|
126
130
|
**Populate frontmatter (per shared schema — required keys):**
|
|
127
131
|
- `schemaVersion: "1.0"`
|
|
@@ -8,7 +8,7 @@ Rules for cross-referencing API surfaces between two skills to determine integra
|
|
|
8
8
|
|
|
9
9
|
## Verdict Definitions
|
|
10
10
|
|
|
11
|
-
Token set is defined canonically in `src/shared/references/feasibility-report-schema.md` — the table below restates the same set with this skill's evidence obligations. Tokens are case-sensitive (`Verified`, `Plausible`, `Risky`, `Blocked`); emitting any other token is a schema violation.
|
|
11
|
+
Token set is defined canonically in the SKF shared feasibility report schema (`_bmad/skf/shared/references/feasibility-report-schema.md` in installed mode; `src/shared/references/feasibility-report-schema.md` in a dev checkout) — the table below restates the same set with this skill's evidence obligations. Tokens are case-sensitive (`Verified`, `Plausible`, `Risky`, `Blocked`); emitting any other token is a schema violation.
|
|
12
12
|
|
|
13
13
|
| Verdict | Meaning | Required Evidence |
|
|
14
14
|
|---------------|------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
|
@@ -2,7 +2,9 @@
|
|
|
2
2
|
nextStepFile: 'requirements.md'
|
|
3
3
|
integrationRulesData: '{integrationRulesPath}'
|
|
4
4
|
coveragePatternsData: '{coveragePatternsPath}'
|
|
5
|
-
|
|
5
|
+
feasibilitySchemaProbeOrder:
|
|
6
|
+
- '{project-root}/_bmad/skf/shared/references/feasibility-report-schema.md'
|
|
7
|
+
- '{project-root}/src/shared/references/feasibility-report-schema.md'
|
|
6
8
|
atomicWriteProbeOrder:
|
|
7
9
|
- '{project-root}/_bmad/skf/shared/scripts/skf-atomic-write.py'
|
|
8
10
|
- '{project-root}/src/shared/scripts/skf-atomic-write.py'
|
|
@@ -26,6 +28,8 @@ Cross-reference API surfaces between library pairs that the architecture documen
|
|
|
26
28
|
|
|
27
29
|
## MANDATORY SEQUENCE
|
|
28
30
|
|
|
31
|
+
**Resolve `{feasibilitySchemaRef}`** from `{feasibilitySchemaProbeOrder}`; first existing path wins (installed SKF module path first, dev-checkout `src/` fallback) — used by the verdict-cap references and the append step below.
|
|
32
|
+
|
|
29
33
|
### 1. Load Integration Verification Rules
|
|
30
34
|
|
|
31
35
|
Load `{integrationRulesData}` for the cross-reference verification protocol.
|
|
@@ -78,7 +82,7 @@ For each library in an integration pair, delegate SKILL.md reading to a parallel
|
|
|
78
82
|
- `data_formats_inferred`: best-effort prose scan — format tokens mentioned in SKILL.md descriptions/examples. NOT a declared field in `metadata.json`
|
|
79
83
|
- If a field has no matches, return an empty array `[]`
|
|
80
84
|
|
|
81
|
-
**CRITICAL — these fields are inferred, not declared.** `protocols` and `data_formats` do not exist in any skill's `metadata.json`. Treat them as weak evidence from prose scanning only. When either list is used to justify compatibility in Check 2, the per-pair verdict MUST be capped at `Plausible` (see the schema's producer obligations — `
|
|
85
|
+
**CRITICAL — these fields are inferred, not declared.** `protocols` and `data_formats` do not exist in any skill's `metadata.json`. Treat them as weak evidence from prose scanning only. When either list is used to justify compatibility in Check 2, the per-pair verdict MUST be capped at `Plausible` (see the schema's producer obligations — `{feasibilitySchemaRef}`).
|
|
82
86
|
|
|
83
87
|
**Schema validation (parent):** Each subagent response must contain the required keys (`skill_name`, `language`, `exports`). Reject responses missing required keys and exclude that skill from pair evaluation; HALT if more than **20%** (same failure-budget threshold as step 1 §2; see the justification there) of subagent calls return malformed JSON.
|
|
84
88
|
|
|
@@ -122,7 +126,7 @@ For each integration pair `{library_a, library_b}`, apply the verification proto
|
|
|
122
126
|
- If a literal citation is found in at least one direction → Check 4 PASSES; record the exact cited substring and its location as evidence
|
|
123
127
|
- If neither skill literally cites the other → Check 4 FAILS (weak/missing evidence); per-pair verdict MUST be capped at `Plausible` regardless of Checks 1–3 outcomes
|
|
124
128
|
|
|
125
|
-
**Assign verdict per pair (per `
|
|
129
|
+
**Assign verdict per pair (per `{feasibilitySchemaRef}`):**
|
|
126
130
|
- **Verified** — Checks 1, 3 pass with declared evidence AND Check 4 passes with a literal substring/name citation recorded in the evidence block. Check 2 is best-effort only; it cannot by itself promote a pair to `Verified`.
|
|
127
131
|
- **Plausible** — Checks pass, but at least one relies on inferred evidence (e.g., Check 2 prose scan) OR Check 4 is weak/missing. This is the cap whenever Check 4 fails.
|
|
128
132
|
- **Risky** — At least one check flags an incompatibility that a workaround may resolve (bridge layer, adapter, serialization shim).
|