bmad-module-skill-forge 7.0.0 → 8.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +1 -1
- package/docs/_data/pinned.yaml +9 -10
- package/docs/campaign.md +2 -2
- package/docs/skill-model.md +3 -3
- package/docs/verifying-a-skill.md +17 -18
- package/docs/workflows.md +8 -8
- package/package.json +1 -1
- package/src/shared/references/description-guard-protocol.md +12 -10
- package/src/shared/scripts/schemas/skf-analyze-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-audit-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-campaign-result-envelope.v1.json +38 -0
- package/src/shared/scripts/schemas/skf-create-skill-result-envelope.v1.json +5 -0
- package/src/shared/scripts/schemas/skf-drop-skill-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-export-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-quick-skill-result-envelope.v1.json +8 -3
- package/src/shared/scripts/schemas/skf-refine-architecture-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-stack-result-envelope.v1.json +6 -1
- package/src/shared/scripts/schemas/skf-test-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-update-result-envelope.v1.json +7 -2
- package/src/shared/scripts/schemas/skf-verify-stack-result-envelope.v1.json +1 -1
- package/src/shared/scripts/skf-parse-gaps.py +4 -1
- package/src/shared/scripts/skf-quick-batch.py +8 -4
- package/src/shared/scripts/skf-verify-provenance-completeness.py +957 -183
- package/src/skf-analyze-source/SKILL.md +1 -1
- package/src/skf-audit-skill/SKILL.md +1 -1
- package/src/skf-audit-skill/customize.toml +4 -3
- package/src/skf-audit-skill/references/report.md +1 -1
- package/src/skf-brief-skill/SKILL.md +1 -1
- package/src/skf-campaign/SKILL.md +1 -1
- package/src/skf-campaign/assets/campaign-state-schema.json +58 -0
- package/src/skf-campaign/references/campaign-contracts.md +1 -1
- package/src/skf-campaign/references/step-06-batch.md +2 -2
- package/src/skf-campaign/references/step-07-capstone.md +1 -1
- package/src/skf-campaign/references/step-08-verify.md +1 -1
- package/src/skf-campaign/references/step-09-refine.md +1 -1
- package/src/skf-campaign/references/step-11-maintenance.md +1 -1
- package/src/skf-campaign/scripts/campaign-render-batch.py +43 -10
- package/src/skf-campaign/scripts/campaign-report.py +60 -4
- package/src/skf-campaign/scripts/campaign-state.py +67 -19
- package/src/skf-campaign/templates/campaign-report-template.md +4 -0
- package/src/skf-create-skill/SKILL.md +1 -1
- package/src/skf-create-skill/assets/skill-sections.md +2 -2
- package/src/skf-create-skill/references/report.md +2 -2
- package/src/skf-create-skill/references/validate.md +49 -29
- package/src/skf-create-stack-skill/SKILL.md +1 -1
- package/src/skf-create-stack-skill/references/invocation-contract.md +1 -1
- package/src/skf-create-stack-skill/references/report.md +3 -3
- package/src/skf-create-stack-skill/references/validate.md +9 -4
- package/src/skf-drop-skill/SKILL.md +1 -1
- package/src/skf-drop-skill/references/invocation-contract.md +1 -1
- package/src/skf-export-skill/SKILL.md +1 -1
- package/src/skf-export-skill/references/summary.md +1 -1
- package/src/skf-forger/SKILL.md +1 -1
- package/src/skf-quick-skill/SKILL.md +1 -1
- package/src/skf-quick-skill/references/batch-mode.md +28 -2
- package/src/skf-quick-skill/references/compile.md +2 -0
- package/src/skf-quick-skill/references/finalize.md +3 -2
- package/src/skf-quick-skill/references/halt-contract.md +3 -3
- package/src/skf-quick-skill/references/quick-extract.md +2 -1
- package/src/skf-quick-skill/references/write-and-validate.md +11 -5
- package/src/skf-refine-architecture/SKILL.md +1 -1
- package/src/skf-refine-architecture/customize.toml +1 -1
- package/src/skf-refine-architecture/references/report.md +2 -2
- package/src/skf-rename-skill/SKILL.md +1 -1
- package/src/skf-rename-skill/references/report.md +1 -1
- package/src/skf-setup/SKILL.md +1 -1
- package/src/skf-test-skill/SKILL.md +1 -1
- package/src/skf-test-skill/customize.toml +10 -0
- package/src/skf-test-skill/references/coverage-check-tiers.md +3 -3
- package/src/skf-test-skill/references/coverage-check.md +2 -2
- package/src/skf-test-skill/references/init.md +1 -1
- package/src/skf-test-skill/references/invocation-contract.md +1 -1
- package/src/skf-test-skill/references/report.md +3 -2
- package/src/skf-test-skill/references/scoring-rules.md +2 -2
- package/src/skf-test-skill/references/signature-comparison-rules.md +4 -1
- package/src/skf-test-skill/scripts/gap-ledger.py +218 -46
- package/src/skf-test-skill/scripts/load-coverage-inputs.py +12 -6
- package/src/skf-test-skill/templates/test-report-template.md +4 -3
- package/src/skf-update-skill/SKILL.md +1 -1
- package/src/skf-update-skill/references/gap-driven.md +1 -1
- package/src/skf-update-skill/references/invocation-contract.md +1 -1
- package/src/skf-update-skill/references/merge.md +1 -1
- package/src/skf-update-skill/references/report.md +18 -7
- package/src/skf-update-skill/references/write.md +24 -12
- package/src/skf-verify-stack/SKILL.md +1 -1
- package/src/skf-verify-stack/references/report.md +1 -1
|
@@ -15,6 +15,9 @@ skillInventoryProbeOrder:
|
|
|
15
15
|
atomicWriteProbeOrder:
|
|
16
16
|
- '{project-root}/_bmad/skf/shared/scripts/skf-atomic-write.py'
|
|
17
17
|
- '{project-root}/src/shared/scripts/skf-atomic-write.py'
|
|
18
|
+
hashContentProbeOrder:
|
|
19
|
+
- '{project-root}/_bmad/skf/shared/scripts/skf-hash-content.py'
|
|
20
|
+
- '{project-root}/src/shared/scripts/skf-hash-content.py'
|
|
18
21
|
---
|
|
19
22
|
|
|
20
23
|
<!-- Config: communicate in {communication_language}. Generated SKILL.md text in {document_output_language}. -->
|
|
@@ -116,18 +119,22 @@ Run: `npx skill-check -h`
|
|
|
116
119
|
|
|
117
120
|
### 4. Validate SKILL.md via skill-check (if available)
|
|
118
121
|
|
|
119
|
-
**If `npx skill-check` is available**, run validation and its security scan in one call against the skill package written in section 2. `--fix` can also rewrite the frontmatter `description` the user approved in step 4, which would leave SKILL.md out of step with `metadata.json`, so the call runs inside the description guard. Resolve `{descriptionGuardHelper}` ← first existing path in `{descriptionGuardProbeOrder}`, then run the
|
|
122
|
+
**If `npx skill-check` is available**, run validation and its security scan in one call against the skill package written in section 2. `--fix` can also rewrite the frontmatter `description` the user approved in step 4, which would leave SKILL.md out of step with `metadata.json`, so the call runs inside the description guard. Resolve `{descriptionGuardHelper}` ← first existing path in `{descriptionGuardProbeOrder}` and `{hashContentHelper}` ← first existing path in `{hashContentProbeOrder}`, then run the five commands in order. `verify-restore` runs whatever skill-check exits; note that exit code:
|
|
120
123
|
|
|
121
124
|
```bash
|
|
125
|
+
uv run {hashContentHelper} hash "{skill_package}/SKILL.md" > "{run_dir}/skill-md-hash-before.json"
|
|
122
126
|
uv run {descriptionGuardHelper} capture "{skill_package}/SKILL.md" > "{run_dir}/description-guard.json"
|
|
123
127
|
npx skill-check check "{skill_package}" --fix --format json > "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"; echo "skill-check exited $?"
|
|
124
128
|
uv run {descriptionGuardHelper} verify-restore "{skill_package}/SKILL.md" --captured-file "{run_dir}/description-guard.json"
|
|
129
|
+
uv run {hashContentHelper} hash "{skill_package}/SKILL.md" > "{run_dir}/skill-md-hash-after.json"
|
|
125
130
|
```
|
|
126
131
|
|
|
127
|
-
**If no "skill-check exited" line came back**, the shell tool stopped the block before skill-check finished (a cold `uvx mcp-scan` install can outlast its timeout): run the `verify-restore` command on its own before reading `{run_dir}/skill-check.json`, so a description `--fix` rewrote is restored.
|
|
132
|
+
**If no "skill-check exited" line came back**, the shell tool stopped the block before skill-check finished (a cold `uvx mcp-scan` install can outlast its timeout): run the `verify-restore` command on its own before reading `{run_dir}/skill-check.json`, so a description `--fix` rewrote is restored, then run the second `hash` command.
|
|
128
133
|
|
|
129
134
|
**Guard outputs.** `capture` stages the approved description in `{run_dir}/description-guard.json` and `verify-restore` reads it there, so it never passes through a shell string. Bind `{guard_restored}` ← `restored` and `{guard_diff_kind}` ← `diff_kind` from `verify-restore`. When `{guard_restored}` is true, the approved description is back in SKILL.md, matching `metadata.json`: record the validation note "description restored after `skill-check --fix` ({guard_diff_kind})", which §6 lists with the auto-fixed issues. `verify-restore` refuses an empty or whitespace-only captured description, and a file it cannot read (exit 1, SKILL.md untouched): re-run it with `--captured-file "{skill_package}/metadata.json"`, whose `description` step 4 rendered, never with an empty value, and if that exits 1 too, record a high-severity issue: the guard could not check the description. When it exits 2 with JSON on stdout (the JSON carries `restore_error`), the restore could not be written, so record a high-severity issue: SKILL.md's description no longer matches `metadata.json`.
|
|
130
135
|
|
|
136
|
+
**Rewrite note.** skill-check's JSON lists no fix, so the two `hash` commands tell whether `--fix` changed SKILL.md. Bind `{skill_md_changed}` ← true when the `content_hash` in `{run_dir}/skill-md-hash-before.json` differs from the one in `{run_dir}/skill-md-hash-after.json`, taken after the last `verify-restore` (run the second `hash` command again after a `verify-restore` re-run). When no `{hashContentProbeOrder}` path exists (leave both `hash` commands out) or either file holds no `content_hash`, SKILL.md was not hashed: bind `{skill_md_changed}` ← true as well. When `{skill_md_changed}` is true, record the validation note "skill-check --fix rewrote SKILL.md", or "SKILL.md may have been rewritten (not hashed) by skill-check --fix" when it was not hashed, which §6 lists with the auto-fixed issues. When the block above did not run at all (§3 found no skill-check, or the guard below is missing), `--fix` did not run: hash nothing and bind `{skill_md_changed}` ← false.
|
|
137
|
+
|
|
131
138
|
**If no `{descriptionGuardProbeOrder}` path exists**, never run `--fix` unguarded. Run `npx skill-check check "{skill_package}" --format json` instead, which validates and scans without fixing, with the same capture (`> "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"`, its exit code noted), and log the validation issue "description guard unavailable (`skf-description-guard.py` missing): skill-check ran without `--fix`".
|
|
132
139
|
|
|
133
140
|
**If `{run_dir}/skill-check.json` does not start with a JSON object**, skill-check stopped before it validated (exit 2 on a config, option or plugin error): log "skill-check failed: {the stderr line that names the cause, or its last line}" and "security scan skipped: skill-check failed ({that line})", keep `{quality_score}` null, bind `{security_result}` ← `skipped`, and run the frontmatter validator below, as when skill-check is not available. Otherwise stdout can go on after the JSON (a `Wrote <path>` line when the project config sets a report path), so read only the JSON object at its start.
|
|
@@ -135,7 +142,6 @@ uv run {descriptionGuardHelper} verify-restore "{skill_package}/SKILL.md" --capt
|
|
|
135
142
|
**Parse the JSON in `{run_dir}/skill-check.json`** to extract:
|
|
136
143
|
- `scores[].score` — overall score (0-100); match the entry by `relativePath`/`skillId`
|
|
137
144
|
- `diagnostics[]` — remaining issues after auto-fix
|
|
138
|
-
- `fixed[]` — issues automatically corrected
|
|
139
145
|
- `security[]` (when present) — security findings, recorded as advisory warnings (security issues do not block output)
|
|
140
146
|
|
|
141
147
|
Bind `{quality_score}` ← that score, and record the remaining diagnostics and security findings as validation issues. `{quality_score}` is null when skill-check did not run.
|
|
@@ -172,7 +178,7 @@ The validator emits JSON with `result` (PASS/FAIL), `validation.skill_md.body[]`
|
|
|
172
178
|
|
|
173
179
|
**SKILL.md:** {pass/issues found}{ (quality score: {quality_score}/100), only when `{quality_score}` is not null}
|
|
174
180
|
{list any issues}
|
|
175
|
-
{list
|
|
181
|
+
{list the auto-fixed issues: the §4 rewrite note when `{skill_md_changed}` is true, and the §4 description-guard note when the guard restored the description}
|
|
176
182
|
|
|
177
183
|
**context-snippet.md:** {pass/issues found}
|
|
178
184
|
{list any issues}
|
|
@@ -190,7 +196,7 @@ These issues are advisory for community-tier skills: fix them in the written fil
|
|
|
190
196
|
|
|
191
197
|
**Proceeding to finalize...**"
|
|
192
198
|
|
|
193
|
-
Set `validation_result` with pass/fail status, quality score, and issues list, and `{validation_issues}` ← `{"skill_md": <n>, "context_snippet": <n>, "metadata": <n>, "security": <n>}`, each `<n>` the number of entries the validators returned for that heading: `skill_md` counts skill-check's `diagnostics[]` (without skill-check, the frontmatter validator's `issues[]`) plus `validation.skill_md.body[]`; `context_snippet` counts `validation.context_snippet.issues[]`; `metadata` counts `validation.metadata.issues[]`; `security` counts skill-check's `security[]` (or, without one, the finding codes §4 read from the tail, each occurrence one finding, as skill-check's own scan summary counts them). A check that did not run counts 0, and
|
|
199
|
+
Set `validation_result` with pass/fail status, quality score, and issues list, and `{validation_issues}` ← `{"skill_md": <n>, "context_snippet": <n>, "metadata": <n>, "security": <n>}`, each `<n>` the number of entries the validators returned for that heading: `skill_md` counts skill-check's `diagnostics[]` (without skill-check, the frontmatter validator's `issues[]`) plus `validation.skill_md.body[]`; `context_snippet` counts `validation.context_snippet.issues[]`; `metadata` counts `validation.metadata.issues[]`; `security` counts skill-check's `security[]` (or, without one, the finding codes §4 read from the tail, each occurrence one finding, as skill-check's own scan summary counts them). A check that did not run counts 0, and the §4 rewrite note and the guard note count nothing. Step 6 §3's summary carries `{quality_score}`, `{validation_issues}` and `{security_result}`.
|
|
194
200
|
|
|
195
201
|
### 7. Auto-Proceed to Finalize
|
|
196
202
|
|
|
@@ -77,7 +77,7 @@ When `{headless_mode}` is true, step 6 prints one `SKF_REFINE_ARCHITECTURE_RESUL
|
|
|
77
77
|
uv run {resolveCustomizationHelper} --skill {skill-root} --project-root {project-root} --key workflow
|
|
78
78
|
```
|
|
79
79
|
|
|
80
|
-
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-refine-architecture.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. In that case keep the reason as `{customization_resolver_unavailable}` for §5 to record.
|
|
80
|
+
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-refine-architecture.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `SKF helper skf-resolve-customization.py not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. In that case keep the reason as `{customization_resolver_unavailable}` for §5 to record.
|
|
81
81
|
|
|
82
82
|
Bind each path scalar, taking the bundled default when the merged value is empty or absent:
|
|
83
83
|
|
|
@@ -80,7 +80,7 @@ output_folder_path = ""
|
|
|
80
80
|
|
|
81
81
|
# Optional post-completion hook: a shell command. When non-empty, step 6 runs
|
|
82
82
|
# it with `--result-path=<path>` (the per-run result JSON) once the refined
|
|
83
|
-
# architecture is promoted and
|
|
83
|
+
# architecture is promoted and the per-run result file is written, before the run
|
|
84
84
|
# finishes. Use to index the refined doc, notify, or chain the next workflow.
|
|
85
85
|
# A failure shows as a warning line in step 6's report but never fails the run.
|
|
86
86
|
|
|
@@ -95,11 +95,11 @@ The shared emitter writes the result contract (per `shared/references/output-con
|
|
|
95
95
|
uv run {emitEnvelopeHelper} emit --workflow skf-refine-architecture --run-dir "{run_dir}" --result-dir "{outputFolderPath}" < "{run_dir}/result-context.json"
|
|
96
96
|
```
|
|
97
97
|
|
|
98
|
-
It writes `{outputFolderPath}/refine-architecture-result-{YYYYMMDD-HHmmss}.json` and its `refine-architecture-result-latest.json` copy (a copy, not a symlink: the record of the last run whose emitter wrote this copy, at a stable path; a caller reads its own run's `SKF_REFINE_ARCHITECTURE_RESULT_JSON` line), stamping the time, the run id, the run's warnings and the auto-decisions the gates recorded, and prints the `SKF_REFINE_ARCHITECTURE_RESULT_JSON:` line on stdout. When `{headless_mode}` is true, display the line verbatim. Bind `{result_path}` ← the line's `result_path`: a
|
|
98
|
+
It writes `{outputFolderPath}/refine-architecture-result-{YYYYMMDD-HHmmss}.json` and its `refine-architecture-result-latest.json` copy (a copy, not a symlink: the record of the last run whose emitter wrote this copy, at a stable path; a caller reads its own run's `SKF_REFINE_ARCHITECTURE_RESULT_JSON` line), stamping the time, the run id, the run's warnings and the auto-decisions the gates recorded, and prints the `SKF_REFINE_ARCHITECTURE_RESULT_JSON:` line on stdout. When `{headless_mode}` is true, display the line verbatim. Bind `{result_path}` ← the line's `result_path`: a per-run record that could not be written leaves it null, and a `-latest` copy that could not be written leaves it set; either adds a `result_file_write_failed` warning to the line, and the run still finishes. If the emitter exits non-zero, run §1's `context` command and the emitter once more; if it fails again, HALT (exit code 4, `halt_reason: "write-failed"`) at phase `report:result-contract`: the run's result contract could not be written.
|
|
99
99
|
|
|
100
100
|
### 5. Post-Completion Hook (optional)
|
|
101
101
|
|
|
102
|
-
Skip this section when `{onCompleteCommand}` (resolved at SKILL.md On Activation §4 from `workflow.on_complete`) is empty, the bundled default. When `{result_path}` is null, skip it too, with one line saying no result file was written for the hook to read. Otherwise invoke it now, after the emitter wrote
|
|
102
|
+
Skip this section when `{onCompleteCommand}` (resolved at SKILL.md On Activation §4 from `workflow.on_complete`) is empty, the bundled default. When `{result_path}` is null, skip it too, with one line saying no result file was written for the hook to read. Otherwise invoke it now, after the emitter wrote the per-run record, so a notifier, indexer or chained pipeline sees a complete record:
|
|
103
103
|
|
|
104
104
|
```bash
|
|
105
105
|
{onCompleteCommand} --result-path="{result_path}"
|
|
@@ -70,7 +70,7 @@ Headless callers: the inputs, flags, gate map, outputs, concurrency rule and `SK
|
|
|
70
70
|
uv run {resolveCustomizationHelper} --skill {skill-root} --project-root {project-root} --key workflow
|
|
71
71
|
```
|
|
72
72
|
|
|
73
|
-
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-rename-skill.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. In that case keep the reason as `{customization_resolver_unavailable}` (unset when the resolver ran) for select.md §1 to record once it has created `{run_dir}`.
|
|
73
|
+
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-rename-skill.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `SKF helper skf-resolve-customization.py not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. In that case keep the reason as `{customization_resolver_unavailable}` (unset when the resolver ran) for select.md §1 to record once it has created `{run_dir}`.
|
|
74
74
|
|
|
75
75
|
Apply the scalar fallback now so stage files don't have to repeat the conditional logic. For each of the two scalars, if the merged value is empty or absent, the bundled default applies:
|
|
76
76
|
|
|
@@ -78,7 +78,7 @@ If `{onCompleteCommand}` is non-empty (resolved at SKILL.md On Activation §3 fr
|
|
|
78
78
|
{onCompleteCommand} --result-path="{result_path}"
|
|
79
79
|
```
|
|
80
80
|
|
|
81
|
-
Log success/failure but never fail the workflow on a hook error: the rename is already committed. The hook runs after the
|
|
81
|
+
Log success/failure but never fail the workflow on a hook error: the rename is already committed. The hook runs after the emitter wrote the per-run record (`result_path` not null) so an audit-log emit, registry re-index, or notifier sees the completed rename. When `{onCompleteCommand}` is empty (bundled default), skip the invocation entirely; when `{result_path}` is null, skip it with one line saying no result file was written for it to read.
|
|
82
82
|
|
|
83
83
|
### 4. Release the Run Lock and the Run Folder
|
|
84
84
|
|
package/src/skf-setup/SKILL.md
CHANGED
|
@@ -72,7 +72,7 @@ You are a system executor: run each step in sequence, write the configuration fi
|
|
|
72
72
|
uv run {resolveCustomizationHelper} --skill {skill-root} --project-root {project-root} --key workflow
|
|
73
73
|
```
|
|
74
74
|
|
|
75
|
-
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-setup.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. Bind `{customization_resolver_unavailable}` ← that reason (null when the resolver ran), which `references/report.md` and every later halt hand to the emitter; print the warning only when `{quiet_mode}` is false.
|
|
75
|
+
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-setup.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `SKF helper skf-resolve-customization.py not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. Bind `{customization_resolver_unavailable}` ← that reason (null when the resolver ran), which `references/report.md` and every later halt hand to the emitter; print the warning only when `{quiet_mode}` is false.
|
|
76
76
|
|
|
77
77
|
Apply the resolved values, with no narration of your own when `{quiet_mode}` is true: run each `workflow.activation_steps_prepend` entry now; treat each `workflow.persistent_facts` entry as standing context (a `file:` entry loads its file or glob contents); bind `{onCompleteCommand}` ← `workflow.on_complete` (empty: no hook) for `references/report.md` §5; then run each `workflow.activation_steps_append` entry, before the first stage.
|
|
78
78
|
|
|
@@ -70,7 +70,7 @@ Headless callers: the full argument set (`Inputs`), the outputs, the exit codes
|
|
|
70
70
|
uv run {resolveCustomizationHelper} --skill {skill-root} --project-root {project-root} --key workflow
|
|
71
71
|
```
|
|
72
72
|
|
|
73
|
-
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-test-skill.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. In that case keep the reason as `{customization_resolver_unavailable}`: report.md §4c hands it to the result as a warning.
|
|
73
|
+
It merges the bundled `{skill-root}/customize.toml` with `{project-root}/_bmad/custom/skf-test-skill.toml` (team overrides, committed) and `.user.toml` (personal overrides, gitignored). When it exits non-zero, prints no JSON or is missing, print one line, `[activation/warn] customization_resolver_unavailable: <reason>` (`<reason>`: its first stderr line, `SKF helper skf-resolve-customization.py not found` when the script is missing, `no JSON` when it printed none). If the resolver cannot run, read `{skill-root}/customize.toml` alone and use its bundled defaults: the `{project-root}/_bmad/custom/` overrides do not apply to this run. In that case keep the reason as `{customization_resolver_unavailable}`: report.md §4c hands it to the result as a warning.
|
|
74
74
|
|
|
75
75
|
Apply the scalar fallbacks now so stage files don't have to repeat the conditional logic. For each scalar, if the merged value is empty or absent, use the bundled default:
|
|
76
76
|
|
|
@@ -48,6 +48,16 @@ persistent_facts = [
|
|
|
48
48
|
#
|
|
49
49
|
# Lift the canonical asset paths so orgs can substitute house-style copies
|
|
50
50
|
# without forking the skill. Empty string = use the bundled default.
|
|
51
|
+
#
|
|
52
|
+
# The test report template init copies to the run's report. Default:
|
|
53
|
+
# templates/test-report-template.md. A house-style copy may reword the prose,
|
|
54
|
+
# and must keep:
|
|
55
|
+
# - the title line `# Test Report: {{skillName}}`, which init fills with
|
|
56
|
+
# the skill name;
|
|
57
|
+
# - the six `## ...` section headings, each with the `<!-- Populated by`
|
|
58
|
+
# comment under it, since each stage replaces its own placeholder.
|
|
59
|
+
# The report check halts the run (`report-anchor-missing`) on a report
|
|
60
|
+
# without them.
|
|
51
61
|
|
|
52
62
|
test_report_template_path = ""
|
|
53
63
|
|
|
@@ -4,13 +4,13 @@ Load this file only from coverage-check.md §2, and follow the one section §2 n
|
|
|
4
4
|
|
|
5
5
|
## Quick Tier
|
|
6
6
|
|
|
7
|
-
- **A skill quick-skill built** (`metadata.json` `generated_by` is `quick-skill`), or **any skill in Python, JavaScript or TypeScript**, **from local source** (State 1): parse the entry points with `{extractPublicApiHelper}` `--mode quick`. quick-skill built its export list with that parser, so both sides count one surface, and in those three languages it gives each name the file that defines it (`defined_in`), so the brief's globs place the name as the recipe runner does. Resolve it ← first existing path in `{extractPublicApiProbeOrder}` and run it once per package in scope, with its manifest and one `--entry-file` for each entry-point file the Source API Surface Definition makes its surface, each path relative to `{source_path}` (`<n>` counts the packages from 1):
|
|
7
|
+
- **A skill quick-skill built** (`metadata.json` `generated_by` is `quick-skill`), or **any skill in Python, JavaScript or TypeScript**, **from local source** (State 1): parse the entry points with `{extractPublicApiHelper}` `--mode quick`. quick-skill built its export list with that parser, so both sides count one surface (except names quick-skill took from a README for a star re-export from another package: the parser gives that star no name, so those names read as stale), and in those three languages it gives each name the file that defines it (`defined_in`), so the brief's globs place the name as the recipe runner does. Resolve it ← first existing path in `{extractPublicApiProbeOrder}` and run it once per package in scope, with its manifest and one `--entry-file` for each entry-point file the Source API Surface Definition makes its surface, each path relative to `{source_path}` (`<n>` counts the packages from 1):
|
|
8
8
|
|
|
9
9
|
```bash
|
|
10
10
|
uv run {extractPublicApiHelper} --mode quick --language <language> --source-root "{source_path}" --manifest-file <manifest path> --entry-file <entry path> > "{run_dir}/quick-<n>.json"
|
|
11
11
|
```
|
|
12
12
|
|
|
13
|
-
Its `warnings` name each statement whose names the entry file alone cannot give. Read by eye only the statements its warnings name, save the names they give as a per-file result for the entry file that holds the statement, `{"file": "<entry path>", "exports_found": [<each name>], "signature_mismatches": []}`, and list those names in the Coverage Analysis section as read by eye. Give `surface` one `--quick` per package, one `--per-file` per saved result, the brief when the skill has one (its globs give the `scope.include` and `tier_a_include` sets), the metadata and, when init.md §2 bound one, the provenance map (the baselines of the §2b candidates and guards; they add no name):
|
|
13
|
+
Its `warnings` name each statement whose names the entry file alone cannot give. Read by eye only the statements its warnings name, save the names they give as a per-file result for the entry file that holds the statement, `{"file": "<entry path>", "exports_found": [<each name>], "signature_mismatches": []}`, and list those names in the Coverage Analysis section as read by eye. A warning on an `export *` or `export type *` from another package (a specifier that is no relative path; the warning writes either as `export *`) gives no name, as the recipe runner leaves such a star unresolved: list the statement as the source writes it, `<line>: <statement>`, in the `unresolved_stars` of that entry file's per-file result, and in the Coverage Analysis section as unresolved. Give `surface` one `--quick` per package, one `--per-file` per saved result that lists a name, the brief when the skill has one (its globs give the `scope.include` and `tier_a_include` sets), the metadata and, when init.md §2 bound one, the provenance map (the baselines of the §2b candidates and guards; they add no name):
|
|
14
14
|
|
|
15
15
|
```bash
|
|
16
16
|
uv run {coverageInputsScript} surface --quick "{run_dir}/quick-<n>.json" [--per-file "{run_dir}/per-file-<n>.json"] [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] [--monorepo] [--pattern-reference-intent] [--curated-subset] [--barrel <state>] --output "{run_dir}/surface.json"
|
|
@@ -26,7 +26,7 @@ The recipe runner cannot be used, so subagents read the files in scope one at a
|
|
|
26
26
|
uv run {scoreSignaturesScript} plan --inventory "{run_dir}/inventory.json" --output "{run_dir}/signature-plan.json"
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
For each source file that defines public API exports, delegate a subagent that gets that map with the file and the absolute path of `{signatureRulesFile}` (under `{skill-root}`). It gets the `documentedAlternatives` map too, whose forms it compares as that file says. It extracts the symbols the file exports, with their full signatures, with ast-grep (or reads the file when `extraction.fallback.reason` says no ast-grep can run; list those files in the Coverage Analysis section as read by eye), leaves out a name the file only passes on from another module of the source being tested (`export { x } from './x'`, `export * from`, an `__init__.py` import, a Rust `pub use`), which that module's own result counts, keeps a name it passes on from another package (`export { z } from 'zod'`), which no other result counts, compares each with the map's entry by the rules of that file (overload sets and `difference` tags included), gives each mismatch the `line` that defines the export, and returns only `{"file", "exports_found": [...], "types_found": [<the exported interfaces, type aliases, enums and classes>], "signature_mismatches": [...]}`, each mismatch in the shape §2's signature comparison returns. Save each response to `{run_dir}/per-file-<n>.txt` with the Write tool and build the surface from them, one `--per-file` each and `--fallback-scan` (each names the file that defines its exports, so a `scope.tier_a_include` list that counts none of them does not turn the guards off), with the same baselines as at Quick tier, and `--extraction` when the runner's JSON sent you here (`extraction.fallback.needed`): it adds no name, only its scope and the globs it lists as matching no file, so the guards still run:
|
|
29
|
+
For each source file that defines public API exports, delegate a subagent that gets that map with the file and the absolute path of `{signatureRulesFile}` (under `{skill-root}`). It gets the `documentedAlternatives` map too, whose forms it compares as that file says. It extracts the symbols the file exports, with their full signatures, with ast-grep (or reads the file when `extraction.fallback.reason` says no ast-grep can run; list those files in the Coverage Analysis section as read by eye), leaves out a name the file only passes on from another module of the source being tested (`export { x } from './x'`, `export * from`, an `__init__.py` import, a Rust `pub use`), which that module's own result counts, keeps a name it passes on from another package (`export { z } from 'zod'`), which no other result counts, gives no name for an `export * from` or `export type * from` another package (a specifier that is no relative path; `export * as ns` still gives `ns`), which the recipe runner leaves unresolved, listing each such statement as `<line>: <statement>` in `unresolved_stars`, compares each with the map's entry by the rules of that file (overload sets and `difference` tags included), gives each mismatch the `line` that defines the export, and returns only `{"file", "exports_found": [...], "types_found": [<the exported interfaces, type aliases, enums and classes>], "signature_mismatches": [...], "unresolved_stars": [...]}`, each mismatch in the shape §2's signature comparison returns. List each response's `unresolved_stars` in the Coverage Analysis section as unresolved. Save each response to `{run_dir}/per-file-<n>.txt` with the Write tool and build the surface from them, one `--per-file` each and `--fallback-scan` (each names the file that defines its exports, so a `scope.tier_a_include` list that counts none of them does not turn the guards off), with the same baselines as at Quick tier, and `--extraction` when the runner's JSON sent you here (`extraction.fallback.needed`): it adds no name, only its scope and the globs it lists as matching no file, so the guards still run:
|
|
30
30
|
|
|
31
31
|
```bash
|
|
32
32
|
uv run {coverageInputsScript} surface --per-file "{run_dir}/per-file-<n>.txt" --fallback-scan [--extraction "{run_dir}/extract-full.json"] [--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] --metadata "{resolved_skill_package}/metadata.json" [--provenance "{forge_provenance_map}"] [--monorepo] [--pattern-reference-intent] [--curated-subset] [--barrel <state>] --output "{run_dir}/surface.json"
|
|
@@ -107,7 +107,7 @@ Only when the validated inventory's `references` is non-empty. Each `cross_check
|
|
|
107
107
|
|
|
108
108
|
### 2. Analyze Source Code (Tier-Dependent)
|
|
109
109
|
|
|
110
|
-
Identify the public API surface from the package entry points (0b), as deep as the tier allows. Whichever branch runs, `{coverageInputsScript} surface` writes it to `{run_dir}/surface.json`, which §2b, §2c, §4 and step 5 read. Names read by eye reach it as per-file results, `{"file": "<path>", "exports_found": [<each name>], "signature_mismatches": []}` for each file, saved to `{run_dir}/per-file-<n>.json` with the Write tool and given to `surface` with one `--per-file` each.
|
|
110
|
+
Identify the public API surface from the package entry points (0b), as deep as the tier allows. Whichever branch runs, `{coverageInputsScript} surface` writes it to `{run_dir}/surface.json`, which §2b, §2c, §4 and step 5 read. Names read by eye reach it as per-file results, `{"file": "<path>", "exports_found": [<each name>], "signature_mismatches": []}` for each file, saved to `{run_dir}/per-file-<n>.json` with the Write tool and given to `surface` with one `--per-file` each. An `export * from` or `export type * from` another package adds no name: list `<line>: <statement>` in `unresolved_stars` and Coverage Analysis.
|
|
111
111
|
|
|
112
112
|
**Exits.** Every `{coverageInputsScript}` and `{scoreSignaturesScript}` command in §2 and §2b exits 0, or exits 1 with its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:surface`). `surface` with a `--per-file`, and `score`, also exit 2 when a subagent response breaks the schema: re-dispatch that subagent once with its `violations[]` appended (or correct a result the main thread wrote), save the new response over the old one and run the command again; a second failure HALTs the same way.
|
|
113
113
|
|
|
@@ -298,7 +298,7 @@ uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check -
|
|
|
298
298
|
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from guards --input "{run_dir}/surface.json" --metadata "{resolved_skill_package}/metadata.json"
|
|
299
299
|
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from numerator --input "{run_dir}/numerator.json" --metadata "{resolved_skill_package}/metadata.json"
|
|
300
300
|
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from metadata-coherence --input "{run_dir}/metadata-coherence.json" --metadata "{resolved_skill_package}/metadata.json"
|
|
301
|
-
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from provenance-line --input "{run_dir}/provenance-verify.json"
|
|
301
|
+
uv run {gapLedgerScript} append --ledger "{ledgerFile}" --stage coverage-check --from provenance-line --input "{run_dir}/provenance-verify.json" [--stale "{run_dir}/stale.json"]
|
|
302
302
|
```
|
|
303
303
|
|
|
304
304
|
**A split-body mismatch's Source** is a line inside the skill package: when §1b found one, load `{coverageBranchesFile}` and follow its **Split-Body Sources** section.
|
|
@@ -320,7 +320,7 @@ nextWorkflow: ''
|
|
|
320
320
|
---
|
|
321
321
|
```
|
|
322
322
|
|
|
323
|
-
Replace the template's frontmatter with this block (the two hold the same keys). `toolingStatus` is the `tooling_status` §3b set: score.md §3a reads it back for Cap 1. `threshold` and `thresholdSource` are the pass threshold §1b resolved and its source: score.md §1 reads them back, and score.md §7 rewrites `threshold` with the threshold the verdict was settled against. `analysisConfidence` stays empty until score.md §7, or step-hard-gate.md §3 on a run the hard gate blocks, writes the source access coverage-check resolved: `full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`. report.md §4c halts on a report that holds none of them.
|
|
323
|
+
Replace the template's frontmatter with this block (the two hold the same keys). Then replace `{{skillName}}` in the template's H1 with `{skill_name}`, so the report's title is `# Test Report: {skill_name}`: report.md §4c halts on a report without that line. `toolingStatus` is the `tooling_status` §3b set: score.md §3a reads it back for Cap 1. `threshold` and `thresholdSource` are the pass threshold §1b resolved and its source: score.md §1 reads them back, and score.md §7 rewrites `threshold` with the threshold the verdict was settled against. `analysisConfidence` stays empty until score.md §7, or step-hard-gate.md §3 on a run the hard gate blocks, writes the source access coverage-check resolved: `full`, `provenance-map`, `metadata-only`, `remote-only` or `docs-only`. report.md §4c halts on a report that holds none of them.
|
|
324
324
|
|
|
325
325
|
### 7. Report Initialization Status
|
|
326
326
|
|
|
@@ -59,7 +59,7 @@ When the emitter itself is not installed (init.md §0), no line can be printed:
|
|
|
59
59
|
|
|
60
60
|
## Result Files
|
|
61
61
|
|
|
62
|
-
report.md §4c runs the emitter with `--result-dir "{forge_version}"`. It writes the per-run record `skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name, `-2`, `-3` appended when a run already took that second's name) and then its copy `skf-test-skill-result-latest.json`, the record of the last run whose emitter wrote this copy, at a stable path (a copy, not a symlink), each atomically. The record holds the envelope's fields, plus `skill`, `outputs[]` (the published report), `summary` (`score`, `threshold`, `result` (`PASS`, `PASS_WITH_DRIFT`, `FAIL` or `INCONCLUSIVE`), `testMode`, `activeCategories[]`, `inconclusiveReasons[]` when present, `threshold_fallback`, `original_threshold` and `evidence_report_path` when the threshold fallback fired, `gapCounts` with the gap ledger's gaps by severity, and `hardGate`), `runId` and `healthCheckDispatched`, and the emitter stamps `timestamp`, `run_id`, `headless_decisions` and `warnings` into it. A run the hard gate blocked writes it too, as a FAIL record, so its `skf-test-skill-result-latest.json` holds that FAIL, not an earlier run's verdict. A HALT writes no result file, and neither does a run cut off before report.md §4c: the copy then still holds an earlier run's verdict.
|
|
62
|
+
report.md §4c runs the emitter with `--result-dir "{forge_version}"`. It writes the per-run record `skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name, `-2`, `-3` appended when a run already took that second's name) and then its copy `skf-test-skill-result-latest.json`, the record of the last run whose emitter wrote this copy, at a stable path (a copy, not a symlink), each atomically. The record holds the envelope's fields, plus `skill`, `outputs[]` (the published report), `summary` (`score`, `threshold`, `result` (`PASS`, `PASS_WITH_DRIFT`, `FAIL` or `INCONCLUSIVE`), `testMode`, `activeCategories[]`, `inconclusiveReasons[]` when present, `threshold_fallback`, `original_threshold` and `evidence_report_path` when the threshold fallback fired, `gapCounts` with the gap ledger's gaps by severity, and `hardGate`), `runId` and `healthCheckDispatched`, and the emitter stamps `timestamp`, `run_id`, `headless_decisions` and `warnings` into it. A run the hard gate blocked writes it too, as a FAIL record, so its `skf-test-skill-result-latest.json` holds that FAIL, not an earlier run's verdict, unless the run's line holds a `result_file_write_failed` warning (the emitter stops at the first write that fails). A HALT writes no result file, and neither does a run cut off before report.md §4c: the copy then still holds an earlier run's verdict.
|
|
63
63
|
|
|
64
64
|
## Emitting a Halt
|
|
65
65
|
|
|
@@ -186,9 +186,10 @@ for heading in '## Test Summary' '## Coverage Analysis' '## Coherence Analysis'
|
|
|
186
186
|
printf '%s: ' "$heading"; grep -cxF "$heading" "{outputFile}"
|
|
187
187
|
done
|
|
188
188
|
printf 'placeholders: '; grep -c '^<!-- Populated by' "{outputFile}"
|
|
189
|
+
printf 'title: '; grep -cxF '# Test Report: {skill_name}' "{outputFile}"
|
|
189
190
|
```
|
|
190
191
|
|
|
191
|
-
Each heading must print 1, standing alone on exactly one line, and `
|
|
192
|
+
Each heading must print 1, standing alone on exactly one line, `placeholders` must print 0, and `title` must print 1 (the H1 init.md §6c filled). A run the hard gate blocked has all six: step 4c §3 wrote the Completeness Score section in scoring's place. On a heading whose count is not 1, a placeholder left, or a `title` count other than 1, HALT (`halt_reason: "report-anchor-missing"`, phase `report:anchors`) with "report anchor missing: {each heading whose count is not 1, with its count}, {placeholder_count} placeholder comments left, or a `title` count of {title_count} (the report does not hold `# Test Report: {skill_name}` once); the section or title was not written once by its owning step".
|
|
192
193
|
|
|
193
194
|
**Find the health check** unless `no_health_check` is true (the `--no-health-check` flag): resolve `{healthCheckFile}` ← first existing path in `{healthCheckProbeOrder}`. It ends the run (§7), so check it now, while a HALT still leaves the previous run's report and result files in place. If neither candidate exists, HALT (`halt_reason: "health-check-missing"`, phase `report:health-check`):
|
|
194
195
|
|
|
@@ -235,7 +236,7 @@ and bind `{report_file}` ← `{publishedReportFile}`. If the move fails, HALT (`
|
|
|
235
236
|
uv run {emitEnvelopeHelper} emit --workflow skf-test-skill --run-dir "{run_dir}" --result-dir "{forge_version}" --target {emit_target} < "{run_dir}/result-context.json"
|
|
236
237
|
```
|
|
237
238
|
|
|
238
|
-
It writes `{forge_version}/skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name) and then its `skf-test-skill-result-latest.json` copy (the record of the last run whose emitter wrote this copy, at a stable path: a copy, not a symlink; a caller reads its own run's `SKF_TEST_RESULT_JSON` line), each atomically, so a partial write is never observable; stamps the timestamp, `run_id`, `headless_decisions` and `warnings` into both; and prints the `SKF_TEST_RESULT_JSON:` line. Bind `{result_line}` ← that line and `{result_path}` ← its `result_path`. A run the hard gate blocked writes the record too, as a FAIL (`status: "error"`, `verdict: "FAIL"`, `halt_reason: "hard-gate-blocked"`, `exit_code` 2), so after a blocked run `skf-test-skill-result-latest.json` holds its FAIL, never an earlier run's verdict. A
|
|
239
|
+
It writes `{forge_version}/skf-test-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name) and then its `skf-test-skill-result-latest.json` copy (the record of the last run whose emitter wrote this copy, at a stable path: a copy, not a symlink; a caller reads its own run's `SKF_TEST_RESULT_JSON` line), each atomically, so a partial write is never observable; stamps the timestamp, `run_id`, `headless_decisions` and `warnings` into both; and prints the `SKF_TEST_RESULT_JSON:` line. Bind `{result_line}` ← that line and `{result_path}` ← its `result_path`. A run the hard gate blocked writes the record too, as a FAIL (`status: "error"`, `verdict: "FAIL"`, `halt_reason: "hard-gate-blocked"`, `exit_code` 2), so after a blocked run `skf-test-skill-result-latest.json` holds its FAIL, never an earlier run's verdict, unless the run's line holds a `result_file_write_failed` warning (the emitter stops at the first write that fails). A per-run record that could not be written leaves `result_path` null, and a `-latest` copy that could not be written leaves it set; either adds a `result_file_write_failed` warning to the line, and the run still finishes. If the emitter exits non-zero, run it once more; if it fails again, HALT (`halt_reason: "write-failed"`, phase `report:result-contract`) with its stderr: the run's result contract could not be written. `references/invocation-contract.md` lists the record's fields.
|
|
239
240
|
|
|
240
241
|
Once the result files are written, append `'report'` to `stepsCompleted` in the `{outputFile}` frontmatter (it stays out of the expected set above).
|
|
241
242
|
|
|
@@ -19,7 +19,7 @@ This table is the single source of truth for classifying a gap. Every stage reco
|
|
|
19
19
|
| Severity | Category | Criteria |
|
|
20
20
|
|----------|-----------------------------|----------|
|
|
21
21
|
| Critical | `signature-mismatch` | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type, as signature-comparison-rules.md reads each form (a component by its props; an annotation either side leaves out is not compared), in any way but those the Low `signature-mismatch` row and the Info `observation` row grade, such as a renamed Python parameter (the script cannot see which parameters a caller must pass by position), an optional parameter left out of a form the skill does not label partial, or a mismatch with no `difference` (coverage-check §2 and §2b) |
|
|
22
|
-
| Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name, and no item of an export list (`export { L as NAME }`, `export type { ... }`) or of a Python `from X import` clause elsewhere renames to it a name that file declares
|
|
22
|
+
| Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name, and no item of an export list (`export { L as NAME }`, `export type { ... }`) or of a Python `from X import` clause elsewhere renames to it a name that file declares, nor does any other rename the `aliased` clause of the Info `observation` row clears, directly or through a chain of renames; a rename that clause does not clear stays fabricated (coverage-check §2c) |
|
|
23
23
|
| Critical | `broken-reference` | Broken reference (contextual mode): a reference whose target does not exist, such as a file not at the stated path, a skill not in the skills output folder, a type no module declares, or a `scripts/` or `assets/` file the skill names (coherence-check §4) |
|
|
24
24
|
| High | `inaccurate-reference` | Inaccurate reference (contextual mode): the target exists but does not match, such as a type the cited module does not export, a signature that differs, or an integration pattern that does not match the implementation (coherence-check §4) |
|
|
25
25
|
| High | `reference-escape` | A reference whose real path leaves the skill, its source and, for a stack, the skills output folder (coherence-check §4) |
|
|
@@ -45,7 +45,7 @@ This table is the single source of truth for classifying a gap. Every stage reco
|
|
|
45
45
|
| Low | `metadata` | Missing optional metadata or examples |
|
|
46
46
|
| Low | `description` | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps), a Tessl Review description suggestion included |
|
|
47
47
|
| Low | `external-validator` | A skill-check diagnostic or a Tessl Review validation finding the run left unresolved (external-validators §5b) |
|
|
48
|
-
| Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it; an `aliased` name, which an export-list item
|
|
48
|
+
| Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it; an `aliased` name, which a line elsewhere renames from the declaration its cited file holds, directly or through a chain of renames (an export-list item `L as NAME` (`export { default as NAME } from` included, a whole-string side unquoted, a comment allowed before `export`), a plain `export { NAME }` of an import that renames (`import { L as NAME }`, `import NAME from`), a `const`, `let`, `var` or `type NAME = L` with a bare identifier `L` (a `const`, `let` or `var` annotated with a one-line type that holds no `;` and no `=` but in `=>` included, and a generic `type NAME<T> = L<T>` that passes its bare parameters on in the same order), an `export default L` at column 0 (a rename of `L` to `default`) whose bare identifier `L` an import or a declaration of the same file binds, the statement ending on its line, a Python `from X import L as NAME` at column 0, one level in a module-level `try`, `except`, `else`, `if`, `elif`, `with` or `finally` block or as such a block header's one-line body (`try: from X import L as NAME`), or a Python `NAME = L` or `NAME: <type> = L` whose type holds no `;` or `=`), at that line, a multi-line list's item at its own line; when the rename's relative or in-package `from` module, or the import that binds `L`, resolves to a file that declares `L`, that file must be the cited one or one that renames `L` in turn, by one of these forms or by a Python `L = M.L2` whose module `M` a column-0 `from X import M` binds, a chain classify-stale follows file by file until it reaches the cited declaration, so a rename of another file's same-named declaration stays a fabricated signature, as do a chain that leaves the files classify-stale resolves, loops or ends at another declaration, a generic alias whose parameters are constrained, defaulted, swapped or instantiated, a type assertion, any other expression (`NAME = a.b` included) and an import inside a function or a class), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c). Also a partial signature: a mismatch the signature judge tags `omitted-optional` (defined in signature-comparison-rules.md, Difference tags) on a form the skill labels partial (simplified, key parameters, a `...` that stands in for omitted members, not rest or variadic syntax), which the inventory marks `signature_form: "partial"`, as score-signatures.py checks: `Partial signature: {name}`, at the declaration, counted as matching in Signature Accuracy (coverage-check §2 and §2b). Also a return annotation the body contradicts: a mismatch the signature judge tags `annotation-contradicted` (defined in signature-comparison-rules.md, Difference tags) whose documented signature states a return type and writes the source's parameters, and whose `evidence_line`, the body line that returns the documented type, comes after the declaration and before the next one and, with the local source, starts with `return` (or is a one-line declaration's own expression body), as score-signatures.py checks: `Annotation contradicted: {name}`, at the declaration, counted as matching in Signature Accuracy, with no change to the skill (the annotation is reported upstream) (coverage-check §2 and §2b) Also a documentation error a Deep-tier run sees that no script checks (an import path whose module does not export the name, a type member that differs from its declaration): `Not checked by script: {name}`, at the documenting line, written by hand (coverage-check §2 and §5b) |
|
|
49
49
|
| Info | `provenance-unverified` | Provenance line not verified: the line-check rules found no definition line, or the recorded line is not a line number (`line-invalid`) (coverage-check §4c) |
|
|
50
50
|
| Info | `migration-section` | Historical migration content in Section 4b with no T2-future annotation (Case 2 of `migration-section-rules.md`) |
|
|
51
51
|
| Info | `multi-denominator` | Multi-denominator reporting: barrel and documented-surface clusters diverge by design (>10% cross-cluster) |
|
|
@@ -26,6 +26,8 @@ The names of Python's `*args` and `**kwargs` are never compared (`**kwargs` agai
|
|
|
26
26
|
|
|
27
27
|
The source's declaration decides: the documented form is compared with the parameters, the optionality and the annotations the signature declares, with one exception, a return annotation the body contradicts (Annotations).
|
|
28
28
|
|
|
29
|
+
A const annotated with a function type declares that type's parameters: `export const build: ViteBuilder['build'] = async ({ options }) => ...` takes the one parameter `ViteBuilder['build']` names, and the arrow's destructuring is that same parameter. A form may name it as the type does or destructure it as the arrow does, and a destructuring that names only some of the type's properties, in the arrow or in the form, does not read as a narrower type. A declaration whose parameter list is written only in a package outside the source tree (a binding destructured from a value cast to `typeof domTestingLibrary`, or `const spyOn: typeof vitestSpyOn = (...args) => ...`) has its parameters, their optionality and its return left uncompared and counts as matching, as a component whose props type lives there does (Components).
|
|
30
|
+
|
|
29
31
|
In the split-body cross-check both forms are documented: each is read by these rules, a type, return type or type arguments one of them leaves out are not compared, and a parameter's optionality is compared only when both forms state it. An ellipsis claims nothing about the members it stands for, required ones included. A call, a list of bare names or a form its own file labels partial (each side's label is read from that side's file, not from the export's one `signature_form`) may leave out a parameter the other form does not state as required, unless leaving it out shifts a caller's positions (a language without keyword arguments, as above); any other form claims its list is complete, so leaving out a parameter the other writes is a mismatch (inventory-contract.md's examples). Each export's form is read alone: what the rows of other exports write, in the same table or file, states nothing about it, so `(action, sink, tracking_record, child_invalidation)` matches `child_invalidation: X | None = None` even when every other row of its table writes its defaults.
|
|
30
32
|
|
|
31
33
|
A name the skill documents in several forms (a brief usage in SKILL.md, a full props list in a reference file) comes with `alternatives`: coverage-check.md §2's check lists them after its `documented` form, and the fallback per-file scan's `documentedAlternatives` holds them by name. Compare the declaration with each form the check offers, each by its own reading rules, and report a mismatch for each form that does not match, with its index as `form` (0 is `documented`, 1 the first alternative), its `documented_sig` written as that form and its `difference` tagged against it. With `candidates`, a form matches when it matches any candidate, and its mismatch is reported at the closest candidate.
|
|
@@ -38,7 +40,7 @@ A component is an export the source declares as one: a function that takes one p
|
|
|
38
40
|
|
|
39
41
|
A component's own props are the members written in its package: its interface or object type, object literals intersected into it, and members computed from its own code (`VariantProps<typeof buttonVariants>`). Members of a type from another package (`React.ComponentProps<"button">`, `HTMLAttributes<...>`, `ComponentProps<typeof Link>`) are inherited, as are the `ref` a `forwardRef` result takes and the optional `children` of `PropsWithChildren<P>`. A prop the form writes must exist on `P`, own or inherited. A component whose props type lives in a package outside the source tree (`const Tabs = TabsPrimitive.Root`) has its props left uncompared: only that it is a component is checked.
|
|
40
42
|
|
|
41
|
-
A prop the form writes that `P` lacks, or a type or an optionality it writes that the source contradicts, is a mismatch. So is a required own prop it leaves out, and a required inherited one (a `Link`'s `href`) left out of a full form. An optional own prop a typed props list or a table leaves out is an `omitted-optional` difference; one a list of bare names or a usage leaves out, and an optional inherited prop, is no difference. `| null` and `| undefined` on an optional prop are no difference (`VariantProps` adds both). A props type name the form writes that the source does not declare (`Button(props: ButtonProps)` against a source with no `ButtonProps`) is not compared by name, only the props the form lists. Values in a usage example (`variant="subtle"`) are not compared.
|
|
43
|
+
A prop the form writes that `P` lacks, or a type or an optionality it writes that the source contradicts, is a mismatch. So is a required own prop it leaves out, and a required inherited one (a `Link`'s `href`) left out of a full form. An optional own prop a typed props list or a table leaves out is an `omitted-optional` difference; one a list of bare names or a usage leaves out, and an optional inherited prop, is no difference. A rest element in a component's destructured props form (`({ className, variant = "default", ...props }: NativeButtonProps)`) stands for every prop of `P` the form does not name, own or inherited, required ones included, so the form leaves none of them out, and the rest's own name is not compared as a prop. `| null` and `| undefined` on an optional prop are no difference (`VariantProps` adds both). A props type name the form writes that the source does not declare (`Button(props: ButtonProps)` against a source with no `ButtonProps`) is not compared by name, only the props the form lists. Values in a usage example (`variant="subtle"`) are not compared.
|
|
42
44
|
|
|
43
45
|
A props list states optionality as Reading a form says: `({ of, meta, story })`, or a list of names, states none, and in an object type literal (`{ of?: X; meta: M }`) that states it for any member, each member written with no mark claims to be required. A table states optionality by a Required column, a `?` on a prop's name or a Default cell that gives a value; its Type column states types, never optionality. A Default cell with a value marks only its own row optional and an empty one states nothing, so Create Skill's `| Prop | Type | Default | Description |` table never reads a prop with an empty Default cell as required. A destructuring default (`({ of, meta = {} })`) likewise marks only its own member, and the others state no optionality.
|
|
44
46
|
|
|
@@ -46,6 +48,7 @@ A props list states optionality as Reading a form says: `({ of, meta, story })`,
|
|
|
46
48
|
|
|
47
49
|
- A parameter written without a type is not compared on type: its name and order are, and its optionality as Reading a form says. Bare `*args` matches `*args: P.args`, and `**kwargs` matches `**kwargs: P.kwargs`.
|
|
48
50
|
- A return type the form does not state is not compared. A form states one when its entry has a non-blank `return_type` or the form writes one: after `->`, `=>`, `→` or `):`, or in its language's own position (Go's result list after the parameters, the type before the name in C, C++ and Java). A blank `return_type` (`""`) states none, so an entry with `params: ""` and no return type matches `def f() -> None`.
|
|
51
|
+
- A return written as names of values rather than types, a parenthesized list of names such as `→ (nodes, edges)` (after the arrow or in `return_type`), states no return type, so no return is compared. A type name written alone stays a type, lowercase ones included: `-> list` and `→ str` state one.
|
|
49
52
|
- A type the source leaves unannotated, on a parameter or a return, is not compared, except that a TS/JS parameter with a literal default has that literal's type: `pageSize = 20` is a `number`, so `pageSize?: string` is a mismatch. An unannotated return stays uncompared.
|
|
50
53
|
- A documented parameter type that names some of the members of the source's union is no mismatch, since a caller passing it makes a valid call (`schema: VectorSchemaProvider` against `VectorSchemaProvider | MultiVectorSchemaProvider`); a documented return type must cover every member the source can return.
|
|
51
54
|
- Type arguments the form leaves out are not compared: `dict` matches `dict[str, Any]`, and `list[dict]` matches `list[dict[str, Any]]`. Type arguments it writes are: `dict[str, int]` against `dict[str, str]` is a mismatch.
|