bmad-module-skill-forge 2.0.0 → 2.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (235) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/.nvmrc +1 -1
  3. package/docs/_data/pinned.yaml +1 -1
  4. package/package.json +2 -2
  5. package/src/knowledge/skill-lifecycle.md +1 -1
  6. package/src/module-help.csv +2 -2
  7. package/src/shared/scripts/skf-comention-pairs.py +350 -0
  8. package/src/shared/scripts/skf-count-tokens.py +222 -0
  9. package/src/shared/scripts/skf-detect-docs.py +232 -1
  10. package/src/shared/scripts/skf-detect-language.py +77 -0
  11. package/src/shared/scripts/skf-detect-tools.py +105 -2
  12. package/src/shared/scripts/skf-enumerate-stack-skills.py +111 -2
  13. package/src/shared/scripts/skf-find-cycles.py +246 -0
  14. package/src/shared/scripts/skf-hash-content.py +487 -2
  15. package/src/shared/scripts/skf-manifest-ops.py +67 -1
  16. package/src/shared/scripts/skf-rebuild-managed-sections.py +180 -0
  17. package/src/shared/scripts/skf-recommend-scope-type.py +1 -2
  18. package/src/shared/scripts/skf-render-metadata-stats.py +499 -0
  19. package/src/shared/scripts/skf-rewrite-skill-name.py +308 -0
  20. package/src/shared/scripts/skf-shard-body.py +461 -0
  21. package/src/shared/scripts/skf-skill-inventory.py +139 -3
  22. package/src/shared/scripts/skf-structural-diff.py +307 -67
  23. package/src/shared/scripts/skf-validate-brief-schema.py +39 -1
  24. package/src/shared/scripts/skf-validate-feasibility-report.py +234 -0
  25. package/src/shared/scripts/skf-validate-frontmatter.py +17 -12
  26. package/src/shared/scripts/skf-validate-output.py +443 -10
  27. package/src/shared/scripts/skf-validate-pins.py +1 -1
  28. package/src/shared/scripts/skf-verify-no-trace.py +214 -0
  29. package/src/shared/scripts/skf-verify-provenance-completeness.py +439 -0
  30. package/src/skf-analyze-source/SKILL.md +17 -32
  31. package/src/skf-analyze-source/customize.toml +2 -0
  32. package/src/skf-analyze-source/references/auto-docs-only.md +116 -0
  33. package/src/skf-analyze-source/references/continue.md +8 -13
  34. package/src/skf-analyze-source/references/generate-briefs.md +28 -12
  35. package/src/skf-analyze-source/references/headless-contract.md +33 -0
  36. package/src/skf-analyze-source/references/health-check.md +1 -2
  37. package/src/skf-analyze-source/references/identify-units.md +25 -36
  38. package/src/skf-analyze-source/references/init.md +12 -25
  39. package/src/skf-analyze-source/references/map-and-detect.md +8 -20
  40. package/src/skf-analyze-source/references/recommend.md +4 -14
  41. package/src/skf-analyze-source/references/scan-project.md +3 -11
  42. package/src/skf-analyze-source/references/step-auto-scope.md +42 -180
  43. package/src/skf-analyze-source/references/unit-detection-heuristics.md +2 -2
  44. package/src/skf-audit-skill/SKILL.md +9 -6
  45. package/src/skf-audit-skill/customize.toml +2 -0
  46. package/src/skf-audit-skill/references/init.md +58 -32
  47. package/src/skf-audit-skill/references/re-index.md +15 -34
  48. package/src/skf-audit-skill/references/report.md +11 -58
  49. package/src/skf-audit-skill/references/semantic-diff.md +4 -22
  50. package/src/skf-audit-skill/references/severity-classify.md +53 -87
  51. package/src/skf-audit-skill/references/step-doc-drift.md +49 -19
  52. package/src/skf-audit-skill/references/structural-diff.md +43 -68
  53. package/src/skf-brief-skill/SKILL.md +7 -36
  54. package/src/skf-brief-skill/assets/scope-templates.md +1 -1
  55. package/src/skf-brief-skill/customize.toml +14 -0
  56. package/src/skf-brief-skill/references/analyze-target.md +6 -14
  57. package/src/skf-brief-skill/references/confirm-brief.md +2 -14
  58. package/src/skf-brief-skill/references/draft-checkpoint.md +8 -8
  59. package/src/skf-brief-skill/references/gather-intent.md +19 -22
  60. package/src/skf-brief-skill/references/headless-args.md +1 -1
  61. package/src/skf-brief-skill/references/health-check.md +1 -22
  62. package/src/skf-brief-skill/references/invocation-contract.md +37 -0
  63. package/src/skf-brief-skill/references/scope-definition.md +4 -14
  64. package/src/skf-brief-skill/references/step-auto-brief.md +1 -5
  65. package/src/skf-brief-skill/references/step-auto-validate.md +1 -5
  66. package/src/skf-brief-skill/references/version-resolution.md +1 -1
  67. package/src/skf-brief-skill/references/write-brief.md +17 -10
  68. package/src/skf-campaign/SKILL.md +15 -84
  69. package/src/skf-campaign/customize.toml +2 -0
  70. package/src/skf-campaign/manifest.yaml +4 -0
  71. package/src/skf-campaign/references/campaign-contracts.md +67 -0
  72. package/src/skf-campaign/references/campaign-directive-spec.md +14 -103
  73. package/src/skf-campaign/references/step-01-setup.md +4 -5
  74. package/src/skf-campaign/references/step-02-strategy.md +8 -3
  75. package/src/skf-campaign/references/step-03-pins.md +1 -1
  76. package/src/skf-campaign/references/step-04-provenance.md +1 -1
  77. package/src/skf-campaign/references/step-05-skill-loop.md +1 -1
  78. package/src/skf-campaign/references/step-06-batch.md +7 -6
  79. package/src/skf-campaign/references/step-11-maintenance.md +2 -2
  80. package/src/skf-campaign/references/step-resume.md +9 -8
  81. package/src/skf-campaign/scripts/campaign-deps.py +10 -1
  82. package/src/skf-campaign/scripts/campaign-render-batch.py +199 -0
  83. package/src/skf-campaign/scripts/campaign-status.py +162 -0
  84. package/src/skf-campaign/templates/campaign-brief-template.yaml +1 -1
  85. package/src/skf-create-skill/SKILL.md +14 -3
  86. package/src/skf-create-skill/assets/compile-assembly-rules.md +17 -38
  87. package/src/skf-create-skill/assets/skill-sections.md +9 -100
  88. package/src/skf-create-skill/assets/tessl-dismissal-rules.md +4 -13
  89. package/src/skf-create-skill/customize.toml +13 -0
  90. package/src/skf-create-skill/references/authoritative-files-protocol.md +7 -7
  91. package/src/skf-create-skill/references/compile.md +34 -47
  92. package/src/skf-create-skill/references/component-extraction.md +8 -35
  93. package/src/skf-create-skill/references/ecosystem-check.md +9 -25
  94. package/src/skf-create-skill/references/enrich.md +4 -17
  95. package/src/skf-create-skill/references/extract.md +14 -29
  96. package/src/skf-create-skill/references/extraction-patterns-tracing.md +1 -1
  97. package/src/skf-create-skill/references/extraction-patterns.md +7 -7
  98. package/src/skf-create-skill/references/generate-artifacts.md +9 -23
  99. package/src/skf-create-skill/references/health-check.md +1 -1
  100. package/src/skf-create-skill/references/load-brief.md +16 -18
  101. package/src/skf-create-skill/references/report.md +12 -15
  102. package/src/skf-create-skill/references/source-resolution-protocols.md +3 -3
  103. package/src/skf-create-skill/references/step-auto-shard.md +52 -25
  104. package/src/skf-create-skill/references/step-doc-rot.md +30 -12
  105. package/src/skf-create-skill/references/step-doc-sources.md +2 -2
  106. package/src/skf-create-skill/references/sub/ccc-discover.md +2 -13
  107. package/src/skf-create-skill/references/sub/fetch-docs.md +8 -50
  108. package/src/skf-create-skill/references/sub/fetch-temporal.md +5 -20
  109. package/src/skf-create-skill/references/tier-degradation-rules.md +6 -8
  110. package/src/skf-create-skill/references/validate.md +59 -28
  111. package/src/skf-create-skill/scripts/scan-doc-rot.py +245 -0
  112. package/src/skf-create-stack-skill/SKILL.md +15 -9
  113. package/src/skf-create-stack-skill/assets/provenance-map-schema.md +1 -1
  114. package/src/skf-create-stack-skill/customize.toml +10 -0
  115. package/src/skf-create-stack-skill/references/compile-stack.md +7 -8
  116. package/src/skf-create-stack-skill/references/compose-mode-rules.md +2 -13
  117. package/src/skf-create-stack-skill/references/detect-integrations.md +49 -54
  118. package/src/skf-create-stack-skill/references/detect-manifests.md +37 -30
  119. package/src/skf-create-stack-skill/references/generate-output.md +24 -75
  120. package/src/skf-create-stack-skill/references/init.md +9 -81
  121. package/src/skf-create-stack-skill/references/manifest-patterns.md +5 -35
  122. package/src/skf-create-stack-skill/references/parallel-extract.md +12 -27
  123. package/src/skf-create-stack-skill/references/rank-and-confirm.md +6 -6
  124. package/src/skf-create-stack-skill/references/report.md +27 -62
  125. package/src/skf-create-stack-skill/references/validate.md +35 -31
  126. package/src/skf-drop-skill/SKILL.md +20 -23
  127. package/src/skf-drop-skill/customize.toml +10 -0
  128. package/src/skf-drop-skill/references/execute.md +51 -31
  129. package/src/skf-drop-skill/references/headless-contract.md +18 -0
  130. package/src/skf-drop-skill/references/health-check.md +1 -13
  131. package/src/skf-drop-skill/references/report.md +42 -10
  132. package/src/skf-drop-skill/references/select.md +88 -48
  133. package/src/skf-drop-skill/scripts/dir-sizes.py +116 -0
  134. package/src/skf-export-skill/SKILL.md +17 -14
  135. package/src/skf-export-skill/assets/managed-section-format.md +2 -5
  136. package/src/skf-export-skill/customize.toml +10 -6
  137. package/src/skf-export-skill/references/generate-snippet.md +21 -35
  138. package/src/skf-export-skill/references/load-skill.md +44 -42
  139. package/src/skf-export-skill/references/multi-skill-mode.md +1 -1
  140. package/src/skf-export-skill/references/orphan-context-detection.md +11 -3
  141. package/src/skf-export-skill/references/package.md +27 -33
  142. package/src/skf-export-skill/references/preflight-snippet-root-probe.md +24 -5
  143. package/src/skf-export-skill/references/result-envelope.md +28 -0
  144. package/src/skf-export-skill/references/summary.md +11 -5
  145. package/src/skf-export-skill/references/token-report.md +42 -24
  146. package/src/skf-export-skill/references/update-context.md +32 -33
  147. package/src/skf-forger/SKILL.md +27 -39
  148. package/src/skf-forger/references/pipeline-mode.md +34 -0
  149. package/src/skf-forger/scripts/parse-pipeline.py +270 -0
  150. package/src/skf-quick-skill/SKILL.md +10 -60
  151. package/src/skf-quick-skill/customize.toml +9 -0
  152. package/src/skf-quick-skill/references/batch-mode.md +6 -4
  153. package/src/skf-quick-skill/references/compile.md +15 -52
  154. package/src/skf-quick-skill/references/ecosystem-check.md +5 -20
  155. package/src/skf-quick-skill/references/finalize.md +12 -6
  156. package/src/skf-quick-skill/references/halt-contract.md +54 -0
  157. package/src/skf-quick-skill/references/health-check.md +3 -4
  158. package/src/skf-quick-skill/references/quick-extract.md +7 -18
  159. package/src/skf-quick-skill/references/registry-resolution.md +1 -23
  160. package/src/skf-quick-skill/references/resolve-target.md +27 -34
  161. package/src/skf-quick-skill/references/write-and-validate.md +10 -25
  162. package/src/skf-refine-architecture/SKILL.md +14 -7
  163. package/src/skf-refine-architecture/customize.toml +9 -0
  164. package/src/skf-refine-architecture/references/compile.md +9 -7
  165. package/src/skf-refine-architecture/references/gap-analysis.md +21 -50
  166. package/src/skf-refine-architecture/references/health-check.md +1 -1
  167. package/src/skf-refine-architecture/references/improvements.md +9 -25
  168. package/src/skf-refine-architecture/references/init.md +9 -16
  169. package/src/skf-refine-architecture/references/issue-detection.md +10 -25
  170. package/src/skf-refine-architecture/references/refinement-rules.md +4 -16
  171. package/src/skf-refine-architecture/references/report.md +13 -10
  172. package/src/skf-rename-skill/SKILL.md +11 -21
  173. package/src/skf-rename-skill/customize.toml +10 -0
  174. package/src/skf-rename-skill/references/execute.md +107 -78
  175. package/src/skf-rename-skill/references/exit-codes.md +12 -0
  176. package/src/skf-rename-skill/references/report.md +16 -8
  177. package/src/skf-rename-skill/references/select.md +69 -42
  178. package/src/skf-rename-skill/scripts/skf-validate-rename-name.py +161 -0
  179. package/src/skf-setup/SKILL.md +20 -12
  180. package/src/skf-setup/customize.toml +10 -0
  181. package/src/skf-setup/references/auto-index.md +9 -10
  182. package/src/skf-setup/references/ccc-index.md +14 -15
  183. package/src/skf-setup/references/detect-and-tier.md +6 -8
  184. package/src/skf-setup/references/report.md +22 -31
  185. package/src/skf-setup/references/tier-rules.md +2 -32
  186. package/src/skf-setup/references/write-config.md +9 -10
  187. package/src/skf-test-skill/SKILL.md +10 -8
  188. package/src/skf-test-skill/customize.toml +2 -0
  189. package/src/skf-test-skill/references/coherence-check.md +19 -31
  190. package/src/skf-test-skill/references/coverage-check.md +109 -75
  191. package/src/skf-test-skill/references/detect-mode.md +10 -41
  192. package/src/skf-test-skill/references/external-validators.md +11 -19
  193. package/src/skf-test-skill/references/init.md +38 -20
  194. package/src/skf-test-skill/references/migration-section-rules.md +4 -4
  195. package/src/skf-test-skill/references/report.md +60 -40
  196. package/src/skf-test-skill/references/score.md +32 -64
  197. package/src/skf-test-skill/references/scoring-rules.md +6 -62
  198. package/src/skf-test-skill/references/source-access-protocol.md +6 -6
  199. package/src/skf-test-skill/references/step-hard-gate.md +1 -3
  200. package/src/skf-test-skill/scripts/aggregate-coherence.py +276 -0
  201. package/src/skf-test-skill/scripts/check-metadata-coherence.py +359 -0
  202. package/src/skf-test-skill/scripts/combine-external-scores.py +190 -0
  203. package/src/skf-test-skill/scripts/compute-score.py +131 -1
  204. package/src/skf-test-skill/scripts/reconcile-coverage.py +327 -0
  205. package/src/skf-test-skill/scripts/validate-inventory.py +251 -0
  206. package/src/skf-test-skill/scripts/verify-declared-numerator.py +192 -0
  207. package/src/skf-update-skill/SKILL.md +17 -5
  208. package/src/skf-update-skill/customize.toml +14 -0
  209. package/src/skf-update-skill/references/detect-changes.md +19 -37
  210. package/src/skf-update-skill/references/health-check.md +3 -9
  211. package/src/skf-update-skill/references/init.md +39 -32
  212. package/src/skf-update-skill/references/manual-section-rules.md +1 -2
  213. package/src/skf-update-skill/references/merge-conflict-rules.md +1 -52
  214. package/src/skf-update-skill/references/merge.md +13 -57
  215. package/src/skf-update-skill/references/re-extract.md +12 -24
  216. package/src/skf-update-skill/references/report.md +14 -7
  217. package/src/skf-update-skill/references/validate.md +32 -30
  218. package/src/skf-update-skill/references/write.md +86 -51
  219. package/src/skf-update-skill/scripts/skf-new-file-diff.py +167 -0
  220. package/src/skf-verify-stack/SKILL.md +11 -21
  221. package/src/skf-verify-stack/customize.toml +12 -3
  222. package/src/skf-verify-stack/references/coverage-patterns.md +1 -29
  223. package/src/skf-verify-stack/references/coverage.md +21 -10
  224. package/src/skf-verify-stack/references/exit-codes.md +14 -0
  225. package/src/skf-verify-stack/references/health-check.md +1 -1
  226. package/src/skf-verify-stack/references/init.md +20 -25
  227. package/src/skf-verify-stack/references/integration-verification-rules.md +10 -19
  228. package/src/skf-verify-stack/references/integrations.md +44 -49
  229. package/src/skf-verify-stack/references/report.md +28 -34
  230. package/src/skf-verify-stack/references/requirements.md +3 -3
  231. package/src/skf-verify-stack/references/synthesize.md +36 -39
  232. package/src/skf-verify-stack/scripts/skf-coverage-tally.py +196 -0
  233. package/src/skf-verify-stack/scripts/skf-report-delta.py +256 -0
  234. package/src/skf-verify-stack/scripts/skf-verdict-rollup.py +258 -0
  235. package/src/skf-rename-skill/references/rebuild-context.md +0 -110
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  nextStepFile: 'detect-mode.md'
3
3
  outputFile: '{forge_version}/test-report-{skill_name}-{run_id}.md'
4
- templateFile: 'templates/test-report-template.md'
4
+ templateFile: '{testReportTemplatePath}'
5
5
  sidecarFile: '{sidecar_path}/forge-tier.yaml'
6
6
  skillsOutputFolder: '{skills_output_folder}'
7
7
  # frontmatterScript resolves deterministically by probing two candidate
@@ -77,6 +77,12 @@ Check that the skill package contains required files:
77
77
 
78
78
  This skill has not been created yet. Run the **create-skill** workflow first."
79
79
 
80
+ **Headless envelope (if `{headless_mode}`):** emit to **stderr**:
81
+
82
+ ```
83
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":null,"next_workflow":null,"exit_code":1,"halt_reason":"target-inaccessible"}
84
+ ```
85
+
80
86
  HALT — do not proceed.
81
87
 
82
88
  **If metadata.json missing:**
@@ -89,9 +95,9 @@ HALT — do not proceed.
89
95
  1. `{project-root}/_bmad/skf/shared/scripts/skf-validate-frontmatter.py` (installed module layout)
90
96
  2. `{project-root}/src/shared/scripts/skf-validate-frontmatter.py` (development-tree layout)
91
97
 
92
- Use the FIRST path that exists as `{frontmatterScript}`. There is no manual fallback.
98
+ Use the first path that exists as `{frontmatterScript}`. There is no manual fallback.
93
99
 
94
- **If neither path exists, HALT** with the diagnostic below. test-skill is a quality gate; without the deterministic validator it cannot produce a trustworthy frontmatter verdict, and silent manual checks have been known to miss subtle spec drift. The missing helper must be restored before testing continues:
100
+ **If neither path exists, HALT** with the diagnostic below. test-skill is a quality gate; without the deterministic validator it cannot produce a trustworthy frontmatter verdict, and a silent manual check can miss subtle spec drift. The missing helper must be restored before testing continues:
95
101
 
96
102
  ```
97
103
  Error: cannot locate skf-validate-frontmatter.py at either of:
@@ -104,7 +110,9 @@ SKF module (`skf init`) or run from a development checkout with src/ present.
104
110
 
105
111
  Do not proceed. No partial test report is written.
106
112
 
107
- **3b. Run the validator (30s timeout deterministic validator should finish in <1s; the cap only guards against runaway python).** Invoke via `uv run` so the script's PEP 723 PyYAML dependency resolves automatically; bare `python3` would fail with `ModuleNotFoundError: No module named 'yaml'` on a fresh interpreter (`docs/getting-started.md` documents uv as the runtime prereq for exactly this).
113
+ **3b. Python runtime probe (before invoking the validator).** Confirm both `python3` and `uv` are on `$PATH` (`command -v python3` and `command -v uv`). Both are required: `uv run` shells through to `python3` and honors the script's PEP 723 PyYAML dependency declaration that bare `python3` ignores (bare `python3` fails with `ModuleNotFoundError: No module named 'yaml'` on a fresh interpreter `docs/getting-started.md` documents uv as the runtime prereq for exactly this). If either is missing, set `analysis_confidence: degraded` in workflow context and carry a **score cap** into step 5: `capped_score = threshold - 1` → forces auto-FAIL until the runtime is restored. Record the reason in evidence-report and the test report frontmatter (`analysisConfidence: degraded`, `toolingStatus: python3-missing` or `uv-missing` as appropriate). `uv` is a documented runtime prerequisite — see `docs/getting-started.md` for install instructions.
114
+
115
+ **3c. Run the validator (30s timeout — the deterministic validator should finish in <1s; the cap only guards against runaway python).**
108
116
 
109
117
  ```bash
110
118
  timeout 30s uv run {frontmatterScript} {resolved_skill_package}/SKILL.md --skill-dir-name {skill_name}
@@ -119,7 +127,11 @@ Parse the JSON output. Treat each `status` value explicitly:
119
127
 
120
128
  - `status: "pass"` — continue silently.
121
129
  - `status: "warn"` — display the warning below, log each issue as a pre-check finding, and continue with testing. Frontmatter issues surface in the gap report alongside coverage/coherence findings.
122
- - `status: "fail"` — **HALT with auto-FAIL.** Frontmatter failure indicates the skill will be rejected by `npx skills add` and `npx skill-check check`; shipping it would produce a false PASS downstream. Write the halt note into the evidence-report and exit the workflow. If `{headless_mode}`, set `testResult: fail` in the output frontmatter before exiting so the result contract records the terminal state.
130
+ - `status: "fail"` — **HALT with auto-FAIL.** Frontmatter failure means the skill will be rejected by `npx skills add` and `npx skill-check check`; shipping it would produce a false PASS downstream. Write the halt note into evidence-report and exit non-zero. **Headless envelope (if `{headless_mode}`):** emit to **stderr** before halting. The output document does not exist yet (created in §6), so `report_path` is `null` matching the other pre-report init HALTs (target-inaccessible, forge-tier-missing, workspace-drift, another-run-active). A frontmatter-invalid target is the most common failure this gate exists to catch, so a headless orchestrator must be able to branch on it (route to update-skill) rather than see an unlabelled non-zero exit:
131
+
132
+ ```
133
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":null,"next_workflow":null,"exit_code":1,"halt_reason":"frontmatter-invalid"}
134
+ ```
123
135
 
124
136
  ```
125
137
  **Warning/Error: SKILL.md frontmatter is non-compliant with agentskills.io specification.**
@@ -129,8 +141,6 @@ Parse the JSON output. Treat each `status` value explicitly:
129
141
  This skill will fail `npx skills add` and `npx skill-check check`. {If warn:} Consider fixing frontmatter before proceeding (run `npx skill-check check <skill-dir> --fix` to auto-fix deterministic issues). {If fail:} test-skill cannot proceed — halt and repair frontmatter, then re-run.
130
142
  ```
131
143
 
132
- **3c. Python runtime probe.** Before the first invocation, confirm both `python3` AND `uv` are on `$PATH` (`command -v python3` and `command -v uv`). Both are required: `uv run` shells through to `python3` under the hood AND honors the script's PEP 723 PyYAML dependency declaration that bare `python3` ignores. If either is missing, set `analysis_confidence: degraded` in workflow context and carry a **score cap** into step 5: `capped_score = threshold - 1` → forces auto-FAIL until the runtime is restored. Record the reason in evidence-report and the test report frontmatter (`analysisConfidence: degraded`, `toolingStatus: python3-missing` or `uv-missing` as appropriate). `uv` is a documented runtime prerequisite — see `docs/getting-started.md` for install instructions.
133
-
134
144
  ### 4. Load Forge Tier State
135
145
 
136
146
  **`--tier=<...>` flag bypass (precedes the sidecar probe).** If `tier_flag` is set in workflow context (from §1's `--tier=<Quick|Forge|Forge+|Deep>` flag), validate the value against the allowed set. On valid match: set `detected_tier` directly to the flag's value, leave `ast_grep`/`gh_cli`/`qmd` availability flags unset (downstream steps treat unset as "unknown" — analysis proceeds without tool-specific enrichment), log Info note "tier — supplied via --tier flag, sidecar bypassed", and SKIP the sidecar probe and HALT below (jump straight to §4b "Apply Tier Override"). On invalid value (not one of the four), HALT with "Error: --tier=<value> is not one of Quick, Forge, Forge+, Deep".
@@ -144,6 +154,12 @@ This skill will fail `npx skills add` and `npx skill-check check`. {If warn:} Co
144
154
  **If forge-tier.yaml missing:**
145
155
  "**Cannot proceed.** forge-tier.yaml not found at `{sidecarFile}`. Please run the **setup** workflow first to configure your forge tier (Quick/Forge/Forge+/Deep), or re-run with `--tier=<Quick|Forge|Forge+|Deep>` to bypass the sidecar."
146
156
 
157
+ **Headless envelope (if `{headless_mode}`):** emit to **stderr**:
158
+
159
+ ```
160
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":null,"next_workflow":null,"exit_code":1,"halt_reason":"forge-tier-missing"}
161
+ ```
162
+
147
163
  HALT — do not proceed.
148
164
 
149
165
  ### 4b. Apply Tier Override (if set)
@@ -169,11 +185,11 @@ Test-skill reads `source_path` during coverage and coherence analysis. If the lo
169
185
 
170
186
  - Resolve `pinned_commit` from `metadata.source_commit`.
171
187
  - **If `pinned_commit` is null, empty, or `"local"`:** skip the guard; log `workspace_drift_check: skipped (no pinned commit)` and continue to section 6.
172
- - **If `pinned_commit` is a per-repo map (stack skills):** iterate each `{repo_path: commit}` entry — for each repo run `git -C "{repo_path}" rev-parse HEAD` and compare to its pinned commit (accept full-SHA or short-SHA-prefix match). If ANY repo diverges and the user did not pass `--allow-workspace-drift`, HALT with exit status `halted-for-workspace-drift` listing every mismatched repo. On all-match: log `workspace_drift_check: ok (stack, {N} repos verified)` and continue to section 6. This guard MUST iterate every repo — do not skip stack skills.
188
+ - **If `pinned_commit` is a per-repo map (stack skills):** iterate each `{repo_path: commit}` entry — for each repo run `git -C "{repo_path}" rev-parse HEAD` and compare to its pinned commit (accept full-SHA or short-SHA-prefix match). If ANY repo diverges and the user did not pass `--allow-workspace-drift`, HALT with exit status `workspace-drift` listing every mismatched repo (in `{headless_mode}`, emit the same `workspace-drift` stderr envelope shown in the single-tree branch below before halting). On all-match: log `workspace_drift_check: ok (stack, {N} repos verified)` and continue to section 6. This guard must iterate every repo — do not skip stack skills.
173
189
  - **If `source_path` is not a git working tree** (bare checkout, tarball extract, docs-only source) — detect by `git -C "{source_path}" rev-parse --is-inside-work-tree`, non-zero exit means skip: log `workspace_drift_check: skipped (not a git working tree)` and continue to section 6.
174
190
  - **Otherwise** run `git -C "{source_path}" rev-parse HEAD` and compare to `pinned_commit`. Accept full-SHA or short-SHA-prefix match (stored pins are often 8-char short hashes — see `src/knowledge/provenance-tracking.md`).
175
191
  - **On match:** log `workspace_drift_check: ok ({short_sha})` and continue.
176
- - **On mismatch, AND the user did not pass `--allow-workspace-drift`:** HALT with exit status `halted-for-workspace-drift`. Display:
192
+ - **On mismatch, AND the user did not pass `--allow-workspace-drift`:** HALT with exit status `workspace-drift`. Display:
177
193
 
178
194
  ```
179
195
  Workspace HEAD does not match the commit this skill was pinned against.
@@ -191,14 +207,24 @@ Test-skill reads `source_path` during coverage and coherence analysis. If the lo
191
207
  current workspace (accepts that findings reflect HEAD, not the pin).
192
208
  ```
193
209
 
210
+ **Headless envelope (if `{headless_mode}`):** emit to **stderr**:
211
+
212
+ ```
213
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":null,"next_workflow":null,"exit_code":1,"halt_reason":"workspace-drift"}
214
+ ```
215
+
194
216
  Do not proceed. The test report has not been created; no partial writes.
195
217
  - **On mismatch WITH `--allow-workspace-drift`:** log `workspace_drift_check: overridden (pinned={pinned_commit}, head={head_sha})`, carry the warning into the final report frontmatter (`workspaceDrift: overridden`), and set `allow_workspace_drift: true` in workflow context (consumed by step 5 §5 drift override — a PASS under drift is demoted to `pass-with-drift` and `nextWorkflow` is forced to `update-skill`, never `export-skill`). Continue.
196
218
 
197
219
  ### 6. Create Output Document
198
220
 
199
- **6a. Generate `{run_id}`**: a per-run identifier of the form `{YYYYMMDDTHHmmssZ}-{pid}-{rand4}` (UTC timestamp + process PID + 4-char random hex). Store in workflow context. All per-run artifacts in this and subsequent steps MUST carry this suffix; step 6 verifies `testDate` in the resulting report matches the run's stamp and fail-fast otherwise.
221
+ **6a. Generate `{run_id}`**: a per-run identifier of the form `{YYYYMMDDTHHmmssZ}-{pid}-{rand4}` (UTC timestamp + process PID + 4-char random hex). Store in workflow context. All per-run artifacts in this and subsequent steps must carry this suffix; step 6 verifies `testDate` in the resulting report matches the run's stamp and fail-fast otherwise.
200
222
 
201
- **6b. Acquire the per-skill test lock**: `flock {forge_version}/.test-skill.lock` for the duration of this run to serialize concurrent `skf-test-skill` invocations against the same skill. If the lock is already held by another run, HALT with "another test-skill run is active for {skill_name}".
223
+ **6b. Acquire the per-skill test lock**: `flock {forge_version}/.test-skill.lock` for the duration of this run to serialize concurrent `skf-test-skill` invocations against the same skill. If the lock is already held by another run, HALT with "another test-skill run is active for {skill_name}". **Headless envelope (if `{headless_mode}`):** emit to **stderr** before halting:
224
+
225
+ ```
226
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":null,"next_workflow":null,"exit_code":1,"halt_reason":"another-run-active"}
227
+ ```
202
228
 
203
229
  **6c. Create `{outputFile}` from `{templateFile}`** — use `{forge_version}/test-report-{skill_name}-{run_id}.md` Initial frontmatter:
204
230
 
@@ -224,15 +250,7 @@ nextWorkflow: ''
224
250
 
225
251
  ### 7. Report Initialization Status
226
252
 
227
- "**Test initialization complete.**
228
-
229
- **Skill:** {skill_name}
230
- **Path:** {skill_path}
231
- **Type:** {skill_type}
232
- **Forge Tier:** {detected_tier}
233
- **Source:** {source_path}
234
-
235
- **Proceeding to mode detection...**"
253
+ Report initialization to the user: the resolved skill name, path, type, forge tier, and source path. Then proceed to mode detection.
236
254
 
237
255
  Update stepsCompleted, then load and execute {nextStepFile}.
238
256
 
@@ -8,7 +8,7 @@
8
8
 
9
9
  ## Gate Check
10
10
 
11
- Execute this check ONLY IF both conditions are met:
11
+ Execute this check only when both conditions are met:
12
12
 
13
13
  1. Forge tier is **Deep** (tool-gated)
14
14
  2. `{forge_data_folder}/{skill_name}/evidence-report.md` exists (data-gated)
@@ -48,7 +48,7 @@ Check whether SKILL.md contains a "Migration & Deprecation Warnings" section
48
48
  pinned `t2_future_count` field — this is the authoritative count, not the
49
49
  narrative body.
50
50
 
51
- **Detection contract (MANDATORY).** Read the frontmatter deterministically:
51
+ **Detection contract.** Read the frontmatter deterministically:
52
52
 
53
53
  ```bash
54
54
  # Extract t2_future_count from frontmatter. Requires a `---` delimiter pair.
@@ -57,7 +57,7 @@ awk '/^---$/{c++;next} c==1 && /^t2_future_count:/{print $2; exit}' \
57
57
  ```
58
58
 
59
59
  - **Frontmatter missing OR `t2_future_count` absent** → treat as Case 4 (see
60
- below) and skip silently. Do NOT fall back to grepping prose (`grep "T2-future"`) —
60
+ below) and skip silently. Do not fall back to grepping prose (`grep "T2-future"`) —
61
61
  prose drift (heading renames, alt phrasings like "forward-looking
62
62
  annotations", capitalization variance) silently breaks detection and can
63
63
  invert Case-1 vs Case-2/3 severity.
@@ -65,7 +65,7 @@ awk '/^---$/{c++;next} c==1 && /^t2_future_count:/{print $2; exit}' \
65
65
  below.
66
66
 
67
67
  The pinned field is emitted by `skf-create-skill/references/compile.md`
68
- §7 (frontmatter-pinned fields), which ALWAYS writes `t2_future_count: N`
68
+ §7 (frontmatter-pinned fields), which always writes `t2_future_count: N`
69
69
  (including 0). Legacy skills whose `evidence-report.md` predates the pinned
70
70
  field land in Case 4.
71
71
 
@@ -2,8 +2,8 @@
2
2
  nextStepFile: 'health-check.md'
3
3
 
4
4
  outputFile: '{forge_version}/test-report-{skill_name}-{run_id}.md'
5
- scoringRulesFile: 'references/scoring-rules.md'
6
- outputFormatsFile: 'assets/output-section-formats.md'
5
+ scoringRulesFile: '{scoringRulesPath}'
6
+ outputFormatsFile: '{outputFormatsPath}'
7
7
  # outputContractSchema and healthCheck resolve relative to the SKF module root
8
8
  # (`{project-root}/_bmad/skf/` when installed, `{project-root}/src/` during
9
9
  # development), NOT relative to this step file. Both paths are probed in
@@ -23,7 +23,7 @@ atomicWriteProbeOrder:
23
23
 
24
24
  ## STEP GOAL:
25
25
 
26
- Generate a detailed gap report listing every issue found during coverage and coherence analysis, assign severity to each gap, provide specific actionable remediation suggestions, and finalize the test report document. Do not recalculate scores — that ran in step 5. This step chains to the local health-check step via `{nextStepFile}` after completion; the user-facing report is NOT the terminal step.
26
+ Generate a detailed gap report listing every issue found during coverage and coherence analysis, assign severity to each gap, provide specific actionable remediation suggestions, and finalize the test report document. Do not recalculate scores — that ran in step 5. This step chains to the local health-check step via `{nextStepFile}` after completion; the user-facing report is not the terminal step.
27
27
 
28
28
  ### 1. Collect All Issues
29
29
 
@@ -46,27 +46,13 @@ Read `{outputFile}` and extract every issue found across all analysis sections:
46
46
 
47
47
  ### 2. Load Severity Rules
48
48
 
49
- Load `{scoringRulesFile}` for gap severity classification:
50
-
51
- | Severity | Criteria |
52
- |----------|----------|
53
- | **Critical** | Missing exported function/class documentation |
54
- | **High** | Signature mismatch between source and SKILL.md |
55
- | **Medium** | Missing type or interface documentation |
56
- | **Medium** | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
57
- | **Medium** | Script/asset directory exists but no Scripts & Assets section in SKILL.md |
58
- | **Medium** | Scripts & Assets section references file not found in scripts/ or assets/ directory |
59
- | **Low** | Script/asset file present without provenance entry in provenance-map.json file_entries |
60
- | **Low** | Missing optional metadata or examples |
61
- | **Low** | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps) |
62
- | **Info** | Style suggestions, non-blocking observations |
63
- | **Info** | Discovery testing not performed — realistic prompt testing recommended before export |
49
+ Load the **Gap Severity** table from `{scoringRulesFile}` — it is the single source of truth for classifying every gap (Critical / High / Medium / Low / Info). Classify each issue directly against that table in §3; do not restate its rows here, so the table cannot drift between this step and the scoring step that already consumes it.
64
50
 
65
51
  ### 3. Classify and Order Gaps
66
52
 
67
53
  Load `{outputFormatsFile}` for gap entry format and remediation quality rules.
68
54
 
69
- For each issue, assign severity from `{scoringRulesFile}` and generate a specific remediation following the quality rules in `{outputFormatsFile}`. Remediation suggestions MUST reference specific files, exports, and line numbers. Order gaps by severity: Critical → High → Medium → Low → Info.
55
+ For each issue, assign severity from `{scoringRulesFile}` and generate a specific remediation following the quality rules in `{outputFormatsFile}`. Remediation suggestions must reference specific files, exports, and line numbers. Order gaps by severity: Critical → High → Medium → Low → Info.
70
56
 
71
57
  ### 4. Generate Remediation Summary and Append Gap Report
72
58
 
@@ -74,11 +60,11 @@ Load the Gap Report section format from `{outputFormatsFile}`. Count gaps by sev
74
60
 
75
61
  If no gaps found, append a clean pass message recommending **export-skill** workflow.
76
62
 
77
- ### 4b. Discovery Testing (MANDATORY)
63
+ ### 4b. Discovery Testing
78
64
 
79
65
  **`--no-discovery` flag bypass (precedes the precondition check).** If `no_discovery: true` is set in workflow context (from §1 of `init.md` — `--no-discovery` flag on invocation), record an Info-severity note in the Discovery Quality subsection: `discovery — skipped: --no-discovery flag set`, log the bypass, and SKIP §4b.1–§4b.3. Proceed to §4b.4 (description optimization) only if tessl/skill-check flagged description issues; otherwise skip directly to §4c.
80
66
 
81
- After gap enumeration, perform minimum-viable discovery testing. This is a **Medium-weight** check contributing to the Discovery Quality subsection — no longer advisory boilerplate.
67
+ After gap enumeration, perform minimum-viable discovery testing. This is a **Medium-weight** check contributing to the Discovery Quality subsection.
82
68
 
83
69
  **4b.0 Precondition — catalog size check:**
84
70
 
@@ -100,13 +86,15 @@ If SKILL.md does not contain enough organic examples, synthesize 3 from the skil
100
86
 
101
87
  **4b.2 Spawn a discovery subagent:**
102
88
 
89
+ **Subagents-unavailable guard (precedes the spawn).** If subagents cannot be spawned in this environment (e.g. a headless/CI pipeline with no subagent capability), do not fall back to answering the routing in the main thread — the main thread knows which skill is under test, so it would self-route to `3/3 PASS` and inflate the Discovery score, the exact false confidence §4b.0 warns against. Instead record an Info-severity note in the Discovery Quality subsection: `discovery — skipped: subagents unavailable, routing test requires isolated context`, exclude the discovery check from Discovery Quality scoring (do not count it PASS or FAIL), and skip to §4c.
90
+
103
91
  For each of the 3 prompts, spawn an isolated subagent with NO prior context about which skill is under test. Provide only:
104
92
  1. A compact list of ALL skills available in `{skillsOutputFolder}` (name + description line from each skill's SKILL.md frontmatter)
105
93
  2. The prompt text
106
94
 
107
95
  Instruction to the subagent:
108
96
 
109
- > "You are an agent selecting the best skill to handle a user request. Here is the catalog: {catalog}. The user says: '{prompt}'. Return JSON: `{\"selected_skill\": \"<name>\", \"confidence\": \"<high|medium|low>\", \"reasoning\": \"<one sentence>\"}`. If no skill fits, return `{\"selected_skill\": null, ...}`. Return ONLY JSON."
97
+ > "You are an agent selecting the best skill to handle a user request. Here is the catalog: {catalog}. The user says: '{prompt}'. Return JSON: `{\"selected_skill\": \"<name>\", \"confidence\": \"<high|medium|low>\", \"reasoning\": \"<one sentence>\"}`. If no skill fits, return `{\"selected_skill\": null, ...}`. Return only JSON."
110
98
 
111
99
  **4b.3 Evaluate discovery results:**
112
100
 
@@ -133,13 +121,17 @@ Realistic prompt patterns for synthesis (§4b.1 fallback):
133
121
 
134
122
  ### 4c. Result Contract (atomic write)
135
123
 
136
- **Resolve `{atomicWriteHelper}`:** probe `{atomicWriteProbeOrder}`. HALT if neither candidate exists — the contract is a downstream-consumer protocol and must never be written non-atomically.
124
+ **Resolve `{atomicWriteHelper}`:** probe `{atomicWriteProbeOrder}`. HALT if neither candidate exists — the contract is a downstream-consumer protocol and must never be written non-atomically. **Headless envelope (if `{headless_mode}`):** emit to **stderr** before halting:
125
+
126
+ ```
127
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":"{outputFile}","next_workflow":null,"exit_code":1,"halt_reason":"atomic-writer-missing"}
128
+ ```
137
129
 
138
130
  Write the result contract per `{outputContractSchema}`:
139
131
  - Per-run record: `{forge_version}/skf-test-skill-result-{run_id}.json` (the `{run_id}` set in step 1 §6a — already carries UTC timestamp + PID + random suffix, so no same-second collision).
140
132
  - Latest copy: `{forge_version}/skf-test-skill-result-latest.json` (stable path for pipeline consumers — copy, not symlink).
141
133
 
142
- Both writes MUST go through the atomic writer so partial writes are never observable:
134
+ Both writes must go through the atomic writer so partial writes are never observable:
143
135
 
144
136
  ```bash
145
137
  # Build the JSON payload in memory, then:
@@ -149,7 +141,7 @@ cat payload.json | python3 {atomicWriteHelper} write --target {forge_version}/sk
149
141
 
150
142
  Payload contents:
151
143
  - `outputs[]` — include the test report path at `{outputFile}` with its `{run_id}` suffix
152
- - `summary` — `score`, `threshold`, `result` (`"PASS"`, `"PASS_WITH_DRIFT"`, `"FAIL"`, or **`"INCONCLUSIVE"`**), `testMode` (naive/contextual), `activeCategories[]`, `inconclusiveReasons[]` (when present). `PASS_WITH_DRIFT` is set when the workflow observed workspace drift and the user passed `--allow-workspace-drift` — see step 5 §5 drift override. Downstream consumers MUST treat `PASS_WITH_DRIFT` as a non-exportable result: re-run against the pinned commit before export. When threshold fallback occurred, add `threshold_fallback: true`, `original_threshold: {N}`, and `evidence_report_path: '{path}'` to the summary — these fields are absent (not `false`/`null`) when no fallback occurred.
144
+ - `summary` — `score`, `threshold`, `result` (`"PASS"`, `"PASS_WITH_DRIFT"`, `"FAIL"`, or **`"INCONCLUSIVE"`**), `testMode` (naive/contextual), `activeCategories[]`, `inconclusiveReasons[]` (when present). `PASS_WITH_DRIFT` is set when the workflow observed workspace drift and the user passed `--allow-workspace-drift` — see step 5 §5 drift override. Downstream consumers must treat `PASS_WITH_DRIFT` as a non-exportable result: re-run against the pinned commit before export. When threshold fallback occurred, add `threshold_fallback: true`, `original_threshold: {N}`, and `evidence_report_path: '{path}'` to the summary — these fields are absent (not `false`/`null`) when no fallback occurred.
153
145
  - `runId` — the workflow's `{run_id}` for downstream correlation
154
146
  - `healthCheckDispatched` — boolean, set by §7 after the dispatch decision
155
147
 
@@ -178,13 +170,19 @@ Run it with a bounded timeout (default 60s). On success: log Info note "on_compl
178
170
  'report']
179
171
  ```
180
172
 
181
- If any expected entry is missing, HALT with "step completeness violation — missing {list}; workflow state is inconsistent, do not finalize the report". Only append `'report'` and write back after the check passes.
173
+ If any expected entry is missing, HALT with "step completeness violation — missing {list}; workflow state is inconsistent, do not finalize the report". **Headless envelope (if `{headless_mode}`):** emit to **stderr** before halting:
174
+
175
+ ```
176
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":"{outputFile}","next_workflow":null,"exit_code":1,"halt_reason":"step-completeness-violation"}
177
+ ```
178
+
179
+ Only append `'report'` and write back after the check passes.
182
180
 
183
181
  **Section anchor presence check (companion to stepsCompleted).** The
184
182
  report template ships six canonical H2 anchors — one per populating step. An
185
183
  off-sequence run (e.g. a step wrote its section into the wrong anchor, or a
186
184
  subagent truncated the file) can leave `stepsCompleted` intact while a section
187
- is missing. `grep -n` each anchor below against `{outputFile}`; each MUST
185
+ is missing. `grep -n` each anchor below against `{outputFile}`; each must
188
186
  return ≥1 match:
189
187
 
190
188
  ```
@@ -197,12 +195,18 @@ return ≥1 match:
197
195
  ```
198
196
 
199
197
  On any miss, HALT with "report anchor missing: {anchor} — section was not
200
- appended by its owning step". Do NOT append `'report'` and do NOT
201
- write the result contract. The template in `templates/test-report-template.md`
198
+ appended by its owning step". **Headless envelope (if `{headless_mode}`):** emit to **stderr** before halting:
199
+
200
+ ```
201
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":"{outputFile}","next_workflow":null,"exit_code":1,"halt_reason":"report-anchor-missing"}
202
+ ```
203
+
204
+ Do not append `'report'` and do not
205
+ write the result contract. The template at `{testReportTemplatePath}`
202
206
  declares these anchors as TBD placeholders; a miss means a step silently
203
207
  skipped its append.
204
208
 
205
- **INCONCLUSIVE as gate:** if `testResult == 'inconclusive'` (from step 5), the report final presentation (§6) and result contract (§4c) have already been written with that verdict. Do NOT auto-map INCONCLUSIVE to PASS or FAIL. Recommend `manual-review`. The step must still complete (health-check runs unconditionally) — INCONCLUSIVE is a report-time signal, not a workflow abort.
209
+ **INCONCLUSIVE as gate:** if `testResult == 'inconclusive'` (from step 5), the report final presentation (§6) and result contract (§4c) have already been written with that verdict. Do not auto-map INCONCLUSIVE to PASS or FAIL. Recommend `manual-review`. The step must still complete (health-check runs unconditionally) — INCONCLUSIVE is a report-time signal, not a workflow abort.
206
210
 
207
211
  ### 6. Present Final Report
208
212
 
@@ -238,7 +242,7 @@ skipped its append.
238
242
  **update-skill** — This skill needs remediation. Review the gap report above and run the update-skill workflow to address the {N} blocking issues (Critical + High).
239
243
 
240
244
  {IF INCONCLUSIVE:}
241
- **manual-review** — The evidence base was too thin to grade automatically. See `inconclusiveReasons` in the Completeness Score section. Typical fixes: upgrade forge tier, enable external validators, or re-extract with a wider scope. Do NOT export.
245
+ **manual-review** — The evidence base was too thin to grade automatically. See `inconclusiveReasons` in the Completeness Score section. Typical fixes: upgrade forge tier, enable external validators, or re-extract with a wider scope. Do not export.
242
246
 
243
247
  ---
244
248
 
@@ -246,19 +250,29 @@ skipped its append.
246
250
 
247
251
  **Test report finalized.**"
248
252
 
249
- ### 6b. Headless Exit Status
253
+ ### 6b. Determine Headless Exit Code
254
+
255
+ This step only determines the terminal exit code — it does not exit. Both modes then reach §7 (headless auto-proceeds past the menu; non-headless goes through the [C] menu), and the terminal process-exit with this code happens in §7 after the health-check dispatch.
256
+
257
+ If `{headless_mode}`, map `testResult` to the code the workflow will exit with in §7 and store it as `{headless_exit_code}` in workflow context:
258
+ - `testResult: 'pass'` → exit code 0
259
+ - `testResult: 'pass-with-drift'` → exit code 4 (distinct from clean pass — see the pass-with-drift row in SKILL.md Exit Codes; exiting 0 under a drift override would wrongly signal a clean pass)
260
+ - `testResult: 'fail'` → exit code 2 (the result contract was written in §4c — never exit before it)
261
+ - `testResult: 'inconclusive'` → exit code 3 (distinct from fail so orchestrators can route to manual-review queues)
250
262
 
251
- If `{headless_mode}`:
252
- - `testResult: 'pass'` → `exit 0`
253
- - `testResult: 'pass-with-drift'` → `exit 4` (distinct from clean pass — orchestrators MUST route to re-test-against-pinned-commit queues and refuse export; never exit 0 under drift override)
254
- - `testResult: 'fail'` → `exit 2` (after the result contract has been written in §4c — never before)
255
- - `testResult: 'inconclusive'` → `exit 3` (distinct from fail so orchestrators can route to manual-review queues)
263
+ ### 6c. Emit Headless Result Envelope (stdout)
256
264
 
257
- Non-headless mode always drops to the menu in §7.
265
+ If `{headless_mode}`, emit the terminal result envelope to **stdout** as a single line before chaining to §7 — this is the branchable record a headless orchestrator reads for the happy path (PASS / FAIL / INCONCLUSIVE / pass-with-drift). The SKILL.md Result Contract owns the shape and the field rules; the on-disk copy written in §4c is the richer form. Build it from the settled verdict and the values already in the output frontmatter:
266
+
267
+ ```
268
+ SKF_TEST_RESULT_JSON: {"status":"success","skill_name":"{skill_name}","verdict":"{PASS|FAIL|INCONCLUSIVE|pass-with-drift}","score":{score},"threshold":{threshold},"report_path":"{outputFile}","next_workflow":{export-skill when PASS | update-skill when FAIL or pass-with-drift | null when INCONCLUSIVE},"exit_code":{headless_exit_code},"halt_reason":null}
269
+ ```
270
+
271
+ `verdict` is uppercase for `pass`/`fail`/`inconclusive` (→ `PASS`/`FAIL`/`INCONCLUSIVE`) and the literal `pass-with-drift`. When threshold fallback occurred (frontmatter `thresholdFallback: true`), add `"threshold_fallback":true` and `"original_threshold":{originalThreshold}`; omit both otherwise. Non-headless runs skip this emission — the §6 presentation is their terminal output.
258
272
 
259
273
  ### 7. Health-Check Dispatch + MENU OPTIONS
260
274
 
261
- **`--no-health-check` flag bypass (precedes the health-check resolution).** If `no_health_check: true` is set in workflow context (from §1 of `init.md` — `--no-health-check` flag on invocation), set `health_check_dispatched: false` in the output report frontmatter and mirror `healthCheckDispatched: false` into the result contract written in §4c (re-write atomically via `{atomicWriteHelper}`). Log Info note "health-check — skipped: --no-health-check flag set" and EXIT THE WORKFLOW: in `{headless_mode}`, exit with the code already determined in §6b; non-headless, simply terminate after the §6 presentation. Do NOT resolve `{healthCheckFile}`, do NOT display the menu, do NOT chain to `{nextStepFile}`. The §6b exit branch is the terminal state under this flag.
275
+ **`--no-health-check` flag bypass (precedes the health-check resolution).** If `no_health_check: true` is set in workflow context (from §1 of `init.md` — `--no-health-check` flag on invocation), set `health_check_dispatched: false` in the output report frontmatter and mirror `healthCheckDispatched: false` into the result contract written in §4c (re-write atomically via `{atomicWriteHelper}`). Log Info note "health-check — skipped: --no-health-check flag set" and exit the workflow: in `{headless_mode}`, exit with `{headless_exit_code}` (determined in §6b); non-headless, simply terminate after the §6 presentation. Do not resolve `{healthCheckFile}`, do not display the menu, do not chain to `{nextStepFile}`. This flag is the one path where §7 does not dispatch the health-check.
262
276
 
263
277
  Resolve `{healthCheckFile}`: probe `{healthCheckProbeOrder}` in order. **HALT** if neither candidate exists — the health-check is the true terminal step; without it the workflow cannot complete honestly:
264
278
 
@@ -271,6 +285,12 @@ test-skill delegates its terminal step to the shared health-check. Install
271
285
  the SKF module or run from a development checkout with src/ present.
272
286
  ```
273
287
 
288
+ **Headless envelope (if `{headless_mode}`):** emit to **stderr** before halting:
289
+
290
+ ```
291
+ SKF_TEST_RESULT_JSON: {"status":"error","skill_name":"{skill_name}","verdict":null,"score":null,"threshold":null,"report_path":"{outputFile}","next_workflow":null,"exit_code":1,"halt_reason":"health-check-missing"}
292
+ ```
293
+
274
294
  Before displaying the menu, write the dispatch decision into the output report frontmatter (so the artifact records whether the health-check ran):
275
295
 
276
296
  - `health_check_dispatched: true` — when the C menu choice will be taken (headless, or user will select C)
@@ -280,5 +300,5 @@ Also mirror the boolean into the `healthCheckDispatched` field of the result con
280
300
 
281
301
  Display: "**Test complete.** [C] Finish"
282
302
 
283
- On [C] (or auto-proceed in `{headless_mode}` — log: "headless: auto-continue past report menu"): set `health_check_dispatched: true` in frontmatter, then load and execute `{nextStepFile}` (the local health-check dispatcher). The test report document at `{outputFile}` contains the full analysis: Test Summary, Coverage Analysis, Coherence Analysis, Completeness Score, and Gap Report.
303
+ On [C] (or auto-proceed in `{headless_mode}` — log: "headless: auto-continue past report menu"): set `health_check_dispatched: true` in frontmatter, then load and execute `{nextStepFile}` (the local health-check dispatcher). The test report document at `{outputFile}` contains the full analysis: Test Summary, Coverage Analysis, Coherence Analysis, Completeness Score, and Gap Report. In `{headless_mode}`, once the dispatched health-check completes, the workflow makes its terminal process-exit with `{headless_exit_code}` (determined in §6b) — this is the single terminal exit for the headless happy path.
284
304
 
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  nextStepFile: 'report.md'
3
3
  outputFile: '{forge_version}/test-report-{skill_name}-{run_id}.md'
4
- scoringRulesFile: 'references/scoring-rules.md'
4
+ scoringRulesFile: '{scoringRulesPath}'
5
5
  sourceAccessProtocol: 'references/source-access-protocol.md'
6
6
  scoringScript: 'scripts/compute-score.py'
7
7
  ---
@@ -85,11 +85,13 @@ Build a JSON object from the data gathered in steps 1-2:
85
85
  "coherence": "{combined_coherence_percentage or null if naive mode}",
86
86
  "externalValidation": "{external_validation_score or null if N/A}"
87
87
  },
88
- "threshold": "{effective_threshold from §1 — CLI --threshold wins, then pipeline default, then workflow.default_threshold scalar, then 80}"
88
+ "threshold": "{effective_threshold from §1 — CLI --threshold wins, then pipeline default, then workflow.default_threshold scalar, then 80}",
89
+ "analysisConfidence": "{resolved analysis confidence — 'degraded' when python3/frontmatter validator missing; else full/provenance-map/metadata-only/remote-only/docs-only; omit if unknown}",
90
+ "toolingStatus": "{a missing-helper marker like 'python3-missing' or 'frontmatter-validator-missing' when a helper is unavailable, else omit}"
89
91
  }
90
92
  ```
91
93
 
92
- **Important:** Score values must be numbers (not strings). Use `null` (not `"N/A"`) for categories that were not scored. Read `metadata.json.skill_type` from `{resolved_skill_package}/metadata.json`; if the value is `"stack"`, set `stackSkill: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (the categories will be redistributed per `{scoringRulesFile}` Stack Skills rule). Likewise read `metadata.json.scope_type`; if the value is `"reference-app"`, set `referenceApp: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (redistributed per `{scoringRulesFile}` Reference-App rule — a reference app documents wiring patterns, not library export signatures).
94
+ **Important:** Score values must be numbers (not strings). Use `null` (not `"N/A"`) for categories that were not scored. `analysisConfidence` and `toolingStatus` are optional strings — pass them so the script can apply the post-score tooling cap (§3d) deterministically; the script fires Cap 1 when `analysisConfidence == "degraded"` or `toolingStatus` contains `missing`. Read `metadata.json.skill_type` from `{resolved_skill_package}/metadata.json`; if the value is `"stack"`, set `stackSkill: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (the categories will be redistributed per `{scoringRulesFile}` Stack Skills rule). Likewise read `metadata.json.scope_type`; if the value is `"reference-app"`, set `referenceApp: true` and pass `null` for `signatureAccuracy` and `typeCoverage` (redistributed per `{scoringRulesFile}` Reference-App rule — a reference app documents wiring patterns, not library export signatures).
93
95
 
94
96
  #### 3b. Run the Scoring Script
95
97
 
@@ -110,75 +112,54 @@ Parse the JSON output. The script returns:
110
112
  - `skipReasons` — why each category was skipped
111
113
  - `weightSum` — sum of final weights (should be ~100)
112
114
  - `inconclusiveReasons` — only present when `result == "INCONCLUSIVE"`; explains which floor clause tripped
115
+ - **Verdict-override group** — present as an atomic set of four keys only when a post-score cap or the threshold fallback engaged (§3d/§4b are applied by the script, not re-derived here):
116
+ - `effectiveResult` — the **final verdict** after caps + fallback (`"PASS"` / `"FAIL"`); read this as the outcome for §4–§8
117
+ - `capReason` — string describing the cap(s) that fired, or `null`
118
+ - `thresholdFallback` — `true` when the FAIL→PASS-at-80-floor fallback fired
119
+ - `originalThreshold` — the pre-fallback threshold when `thresholdFallback` is `true`, else `null`
120
+ - When the group is **absent**, no cap or fallback engaged and `result` is the final verdict.
113
121
 
114
- Use these values for Section 4 (pass/fail/inconclusive) and Section 6 (output formatting). **Never override the script's `result` value** floor enforcement is centralized in the script for determinism.
122
+ Use these values for Section 4 (pass/fail/inconclusive) and Section 6 (output formatting). **The script owns the minimum-evidence floor, both post-score caps, and the threshold fallback — everywhere below, read its emitted fields (`result`, `effectiveResult`, `capReason`, `thresholdFallback`) and never recompute a verdict, cap, or threshold decision.** The final verdict is `effectiveResult` when the override group is present, otherwise `result` (INCONCLUSIVE is never overridden).
115
123
 
116
124
  #### 3c. Fallback (if script execution fails)
117
125
 
118
- If the script is unavailable or returns an error, fall back to manual calculation:
126
+ If the script is unavailable or errors, redistribute each skipped category's weight (the `null`-scored categories in the §3a JSON — naive mode already zeroes coherence, and Quick-tier/docsOnly/state2/stackSkill/referenceApp already null out Signature Accuracy + Type Coverage) proportionally across the active categories, then report `total = Σ(weight/100 × category_score)` using the detected mode's weight table in `{scoringRulesFile}`. Report: "**Note:** Scoring script unavailable — calculated manually per scoring-rules.md."
119
127
 
120
- 1. Select the weight table from `{scoringRulesFile}` for the detected mode (naive or contextual)
121
- 2. Determine skip conditions: Quick tier/docsOnly/state2/stackSkill/referenceApp skip Signature Accuracy + Type Coverage; naive mode coherence is already 0; null external validation means skip it
122
- 3. For each skipped category, set its weight to 0
123
- 4. Compute sum of active category weights
124
- 5. For each active category: `new_weight = old_weight / sum_active * 100`
125
- 6. `weighted_score = new_weight / 100 * category_score`
126
- 7. `total = sum of all weighted_scores`
128
+ ### 3d. Read Post-Score Caps (applied by the script)
127
129
 
128
- Report: "**Note:** Scoring script unavailablecalculated manually per scoring-rules.md."
129
-
130
- ### 3d. Apply Post-Score Caps (BEFORE PASS/FAIL determination)
131
-
132
- The scoring script returns `totalScore` and a preliminary `result`. Two caps may override the script's PASS into FAIL — applied in this order, and ONLY if the script did NOT return INCONCLUSIVE (floor always takes precedence):
133
-
134
- **Cap 1 — Tooling degraded:** if `analysisConfidence == "degraded"` or `toolingStatus` indicates a missing helper (`python3-missing`, `frontmatter-validator-missing`), set `effective_score = min(totalScore, threshold - 1)` and record `scoring_notes: tooling degraded — capped below threshold until helper restored`.
135
-
136
- **Cap 2 — Docs-only without external validators:** if `docsOnly == true` AND `externalValidation` score was null (neither skill-check nor tessl ran), set `effective_score = min(effective_score, threshold - 1)` and record `scoring_notes: docs-only without external validators — capped below threshold`.
137
-
138
- If either cap fired and `result` was `PASS`, change it to `FAIL`. If `result` was already `INCONCLUSIVE`, leave it — the evidence-floor verdict is never overridden by a cap.
130
+ The script applies two post-score caps — **Cap 1** (tooling degraded) and **Cap 2** (docs-only with no external validators) — and returns the settled outcome as `capReason` + `effectiveResult` (§3b). Read those fields; never recompute a cap. If `capReason` is non-null, record `scoring_notes: {capReason}` in the report. A fired cap forces the script's `PASS` into `FAIL`, never touches an INCONCLUSIVE verdict, and may still be re-flipped to PASS by the threshold fallback (§4b) `effectiveResult` reflects that settled outcome. (Both caps exist because a degraded-tooling or docs-only-without-validators run has too thin an evidence base to trust a PASS; §3a passes the fields Cap 1 reads.)
139
131
 
140
132
  ### 4. Determine Result (PASS / FAIL / INCONCLUSIVE)
141
133
 
142
- The scoring script enforces the minimum-evidence floor BEFORE comparing score vs threshold. Three possible outcomes:
134
+ The scoring script enforces the minimum-evidence floor BEFORE comparing score vs threshold, then applies the post-score caps (§3d) and threshold fallback (§4b). The **settled verdict** is `effectiveResult` when the override group is present, otherwise `result`:
143
135
 
144
136
  ```
145
- IF result == "INCONCLUSIVE" — minimum-evidence floor tripped; not PASS, not FAIL
146
- IF score >= threshold → PASS
147
- IF score < threshold → FAIL
137
+ IF result == "INCONCLUSIVE" — minimum-evidence floor tripped; not PASS, not FAIL (never overridden)
138
+ ELSE settled verdict = effectiveResult if the override group is present, else result (PASS or FAIL)
148
139
  ```
149
140
 
150
141
  **INCONCLUSIVE floor clauses** (see `{scoringRulesFile}`):
151
142
  - `active_categories < 2` (after all redistribution), OR
152
143
  - `tier == "Quick"` AND Export Coverage is the sole scoring contributor
153
144
 
154
- **Tooling-degraded cap:** If the `analysisConfidence` in output frontmatter is `degraded` (python3 missing, frontmatter validator missing, or other degraded state flagged in step 1), the step MUST cap the score at `threshold - 1` BEFORE the PASS/FAIL comparison. This forces a deterministic FAIL until tooling is restored. Do NOT override an INCONCLUSIVE result with the cap — INCONCLUSIVE remains the verdict.
155
-
156
145
  ### 4b. Threshold Fallback and Evidence Report
157
146
 
158
- After §4 determines the result but before §5 recommends the next workflow, check whether a threshold fallback applies. The fallback converts a FAIL into a PASS at the 80% floor when the score is between 80% and the effective threshold, and documents the quality compromise in an evidence report.
159
-
160
- **Fallback trigger conditions:**
147
+ After §4 determines the result but before §5 recommends the next workflow, the script's threshold fallback may already have converted a FAIL into a PASS at the 80% floor. This step documents that quality compromise in an evidence report.
161
148
 
162
- ```
163
- IF result == "FAIL"
164
- AND totalScore >= 80
165
- AND effective_threshold > 80
166
- THEN
167
- → THRESHOLD FALLBACK triggered
168
- ```
149
+ The rule the script encodes (documented for interpretation): when `result == "FAIL"` AND `totalScore >= 80` AND `effective_threshold > 80`, the script overrides `result` to `"PASS"` at the 80% floor; an `INCONCLUSIVE` verdict is never overridden. Read the script's fields:
169
150
 
170
- **Do NOT trigger fallback when:**
171
- - `result == "INCONCLUSIVE"` the evidence-floor verdict is never overridden by fallback
172
- - `totalScore < 80` — that is a genuine FAIL (floor not met)
173
- - `effective_threshold == 80` — there is nothing to fall back from
151
+ - `thresholdFallback == true` → the fallback fired; `effectiveResult` is `"PASS"` and `originalThreshold` holds the pre-fallback threshold.
152
+ - `thresholdFallback` absent or `false` no fallback; the settled verdict from §4 stands.
174
153
 
175
- **When fallback triggers:**
154
+ **When `thresholdFallback` is true:**
176
155
 
177
- 1. Record `threshold_fallback: true`, `original_threshold: {effective_threshold}`, `fallback_threshold: 80` in workflow context.
178
- 2. Override `result` to `"PASS"`.
156
+ 1. Record `threshold_fallback: true`, `original_threshold: {originalThreshold}`, `fallback_threshold: 80` in workflow context.
157
+ 2. Use `effectiveResult` (`"PASS"`) as the settled verdict — do not recompute it.
179
158
  3. Set `effective_threshold = 80` for use by §5/§6/§7/§8.
180
159
  4. Generate the evidence report (§4b.1 below).
181
160
 
161
+ The cap↔fallback interaction (a cap forcing FAIL that the fallback then re-flips to PASS at the 80 floor) is resolved inside the script — `effectiveResult` reflects the settled outcome. The `{totalScore}` used in the report below is the script's raw `totalScore`.
162
+
182
163
  #### 4b.1 Generate Evidence Report
183
164
 
184
165
  Write the evidence report to `{forge_version}/evidence-report-fallback.md`. The report documents the quality compromise for audit purposes.
@@ -234,7 +215,7 @@ Record `evidence_report_path: '{forge_version}/evidence-report-fallback.md'` in
234
215
 
235
216
  ### 5. Determine Next Workflow Recommendation
236
217
 
237
- Based on test result:
218
+ Based on the **settled verdict** (§4 — `effectiveResult` when the override group is present, else `result`; INCONCLUSIVE is never overridden):
238
219
 
239
220
  **IF PASS:**
240
221
  - `nextWorkflow: 'export-skill'` — skill is ready for export
@@ -301,13 +282,13 @@ If `analysis_confidence` is not `full`, append a degradation notice. **The notic
301
282
 
302
283
  **Resolved via:** {analysis_confidence} {confidence breakdown if provenance-map, e.g., "(T1 AST-verified at compilation time)" or "(12 T1, 3 T1-low)"}
303
284
  **Impact:** {describe limitation — e.g., "Signature checks limited to name-matching. Source file:line citations from provenance-map, not live AST." — or "Provenance data is at highest confidence; no limitation." for all-T1 provenance-map}
304
- **Recommendation:** {confidence-dependent — see {sourceAccessProtocol} degradation notice rules. Do NOT recommend local clone when provenance-map entries are already T1.}
285
+ **Recommendation:** {confidence-dependent — see {sourceAccessProtocol} degradation notice rules. Do not recommend local clone when provenance-map entries are already T1.}
305
286
  ```
306
287
 
307
288
  ### 7. Update Output Frontmatter
308
289
 
309
290
  Update `{outputFile}` frontmatter:
310
- - `testResult: '{pass|pass-with-drift|fail|inconclusive}'` (lowercase; mirrors script `result`, with `pass-with-drift` substituted for `pass` when `allow_workspace_drift` was set and drift was observed — see §5 drift override)
291
+ - `testResult: '{pass|pass-with-drift|fail|inconclusive}'` (lowercase; mirrors the **settled verdict** — `effectiveResult` when the override group is present, else `result` with `pass-with-drift` substituted for `pass` when `allow_workspace_drift` was set and drift was observed — see §5 drift override)
311
292
  - `score: '{total}%'`
312
293
  - `threshold: '{threshold}%'`
313
294
  - `thresholdSource: '{threshold_source}'`
@@ -318,24 +299,11 @@ Update `{outputFile}` frontmatter:
318
299
 
319
300
  ### 8. Report Score
320
301
 
321
- "**Completeness score calculated.**
322
-
323
- **{skill_name}:** **{total}%** — **{PASS|FAIL}**
302
+ Report the completeness score to the user: the total percentage and PASS/FAIL verdict, the per-category weighted breakdown (already appended to the report in §6), the threshold, and the recommended next workflow (export-skill on pass, update-skill on fail). When `threshold_fallback` is true, include the fallback notice:
324
303
 
325
- | Category | Score | Weighted |
326
- |----------|-------|----------|
327
- | Export Coverage | {N}% | {WS}% |
328
- | Signature Accuracy | {N}% | {WS}% |
329
- | Type Coverage | {N}% | {WS}% |
330
- | Coherence | {N}% | {WS}% |
331
- | External Validation | {N}% | {WS}% |
332
-
333
- **Threshold:** {threshold}%
334
- {If threshold_fallback is true:}
335
304
  **Threshold fallback:** scored {totalScore}% against {original_threshold}% target — accepted at 80% floor. Evidence report: {evidence_report_path}
336
- **Recommendation:** {export-skill if pass | update-skill if fail}
337
305
 
338
- **Proceeding to gap report...**"
306
+ Then proceed to the gap report.
339
307
 
340
308
  Update stepsCompleted, then load and execute {nextStepFile}.
341
309