bmad-module-skill-forge 6.0.0 → 7.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (36) hide show
  1. package/.claude-plugin/marketplace.json +1 -1
  2. package/docs/_data/pinned.yaml +1 -1
  3. package/docs/workflows.md +4 -4
  4. package/package.json +1 -1
  5. package/src/shared/references/output-contract-schema.md +1 -1
  6. package/src/shared/scripts/skf-quick-batch.py +16 -3
  7. package/src/shared/scripts/skf-verify-provenance-completeness.py +209 -33
  8. package/src/skf-analyze-source/customize.toml +3 -2
  9. package/src/skf-analyze-source/references/auto-docs-only.md +2 -2
  10. package/src/skf-analyze-source/references/generate-briefs.md +5 -3
  11. package/src/skf-analyze-source/references/headless-contract.md +2 -2
  12. package/src/skf-analyze-source/references/step-auto-scope.md +2 -2
  13. package/src/skf-campaign/references/campaign-contracts.md +1 -1
  14. package/src/skf-campaign/references/step-07-capstone.md +3 -3
  15. package/src/skf-campaign/references/step-08-verify.md +3 -3
  16. package/src/skf-campaign/references/step-09-refine.md +3 -3
  17. package/src/skf-campaign/scripts/campaign-state.py +22 -5
  18. package/src/skf-create-skill/SKILL.md +2 -2
  19. package/src/skf-create-skill/customize.toml +4 -4
  20. package/src/skf-create-skill/references/batch-mode.md +3 -1
  21. package/src/skf-create-skill/references/report.md +7 -2
  22. package/src/skf-create-skill/references/validate.md +1 -1
  23. package/src/skf-create-stack-skill/references/validate.md +12 -4
  24. package/src/skf-quick-skill/SKILL.md +1 -1
  25. package/src/skf-quick-skill/references/batch-mode.md +2 -0
  26. package/src/skf-quick-skill/references/write-and-validate.md +19 -9
  27. package/src/skf-test-skill/references/coverage-check.md +2 -2
  28. package/src/skf-test-skill/references/inventory-contract.md +1 -1
  29. package/src/skf-test-skill/references/scoring-rules.md +2 -2
  30. package/src/skf-test-skill/references/signature-comparison-rules.md +3 -1
  31. package/src/skf-test-skill/scripts/load-coverage-inputs.py +241 -11
  32. package/src/skf-test-skill/scripts/score-signatures.py +9 -13
  33. package/src/skf-test-skill/scripts/validate-inventory.py +91 -12
  34. package/src/skf-update-skill/customize.toml +1 -1
  35. package/src/skf-update-skill/references/report.md +10 -3
  36. package/src/skf-update-skill/references/write.md +1 -1
@@ -51,7 +51,7 @@ Every write to `_campaign-state.yaml` goes through `scripts/campaign-state.py`.
51
51
  | `set-skill` | step-05, step-10, step-resume | a skill's `status`, `quality_score`, `skill_path`, `brief_path` or `export` outcome (`--export`); `active` stamps `started_at` once, `completed` stamps `completed_at`, `pending` clears both and the `export` outcome |
52
52
  | `append-workarounds` | step-05 | entries of a skill's `workarounds_applied`, each once |
53
53
  | `apply-batch` | step-06 | the Tier B batch from its line-to-skill map: `--start`, `--results-file` or `--no-results` |
54
- | `set-campaign` | steps 07 to 09 | `architecture_doc_path`, and the capstone, verification or refinement summary read from the sub-skill's envelope line (`capstone.verified` with the verification) |
54
+ | `set-campaign` | steps 07 to 09 | `architecture_doc_path`, and the capstone, verification or refinement summary read from the sub-skill's envelope line (`capstone.verified` with the verification); null for a stage skipped on purpose (`--no-<field>`) or a sub-skill that printed no line (`--no-result <field>`, a `not_recorded` row with halt reason `no-result`) |
55
55
  | `set-stage` | steps 05, 10, 11 | `current_stage` alone |
56
56
  | `recover` | step-resume | `.bak` over the primary, when `.bak` validates (exit 9 otherwise) |
57
57
  | `archive` | SKILL.md (overwrite) | nothing: it moves the state, its `.bak` and the brief into a new folder under `archive/` and returns it |
@@ -50,7 +50,7 @@ Invoke `skf-create-stack-skill` with `--headless` and these inputs, so it asks n
50
50
  - `stack_name`: the `stack_name` that `uv run {manifestScript} --stack-name {stateFile}` prints, the campaign name as a skill name cut to leave room for the `-stack` SS appends; when it prints null, leave the input out and SS names the stack `{project_name}-stack`
51
51
  - `architecture_doc_path`: `campaign.architecture_doc_path`, only when state sets it
52
52
 
53
- Keep the `SKF_STACK_RESULT_JSON` line it prints for §4, which records it. On `status` `error`, or no envelope, log its `halt_reason` (or "no result envelope", type `event`): verification and the campaign report still run.
53
+ Keep the `SKF_STACK_RESULT_JSON` line it prints for §4, which records it: on stdout, or on stderr when SS halts. On `status` `error`, log its `halt_reason` (type `event`); a run with that line on neither stream is recorded in §4. Verification and the campaign report still run either way.
54
54
 
55
55
  ### §4: Record the Capstone and Complete the Stage
56
56
 
@@ -69,10 +69,10 @@ From a `success` envelope it sets `campaign.capstone`, the campaign-level summar
69
69
  - `campaign.capstone.verified`: `null` for now; set by the verify stage (step-08) once the stack is checked
70
70
  - `campaign.capstone.completed_at`: the time of the write, from the clock
71
71
 
72
- An `error` envelope sets `campaign.capstone` to `null` (the helper names it in `not_recorded`). When §2 found no skill to compose, or SS printed no envelope, record no capstone instead: `uv run {stateScript} set-campaign --state-file {stateFile} --no-capstone --stage 6`.
72
+ An `error` envelope sets `campaign.capstone` to `null` (the helper names it in `not_recorded`). When §2 found no skill to compose, record no capstone instead: `uv run {stateScript} set-campaign --state-file {stateFile} --no-capstone --stage 6`. When SS printed no `SKF_STACK_RESULT_JSON` line on stdout or stderr, there is no result to read, whatever its exit code: record it with `uv run {stateScript} set-campaign --state-file {stateFile} --no-result capstone --stage 6`, log "no SKF_STACK_RESULT_JSON line (exit <code>)" with SS's exit code (type `event`), and go on.
73
73
 
74
74
  The capstone is a derived artifact: it is **not** tracked as a skill entry in the `skills[]` array. The constituent skill list and any verbose detail are reported in the step output and are available to downstream steps (verify, refine).
75
75
 
76
76
  ## OUTPUT
77
77
 
78
- Display capstone summary: stack skill name, path, quality score, and the list of constituent skills. Chain to `{nextStepFile}`.
78
+ Display capstone summary: stack skill name, path, quality score, and the list of constituent skills, or "skipped" when §2 found no skill to compose, or "failed (<halt_reason>)" from the helper's `not_recorded` row. That row's `halt_reason` fills it (`failed (no-result)` when SS printed no line); when that is null, show `failed (exit <exit_code>)`, or `failed (no exit code)` when the exit code is null too. Chain to `{nextStepFile}`.
@@ -46,7 +46,7 @@ Once resolved (steps 2 and 3), persist the path at once, so the refine stage and
46
46
 
47
47
  Invoke `skf-verify-stack` with `--headless --architecture-doc <path>`, where `<path>` is the architecture doc §3 resolved.
48
48
 
49
- VS discovers skills from its own configured `{skills_output_folder}`; the campaign does NOT pass individual skill paths. Keep the `SKF_VERIFY_STACK_RESULT_JSON` line it prints on stdout for §5. Its fields are defined once, by `shared/scripts/schemas/skf-verify-stack-result-envelope.v1.json` (in the SKF module: `{project-root}/_bmad/skf/` when installed, `{project-root}/src/` during development); §5 reads `report_path`, `overall_verdict`, `coverage_percentage` and `recommendation_count` from it.
49
+ VS discovers skills from its own configured `{skills_output_folder}`; the campaign does NOT pass individual skill paths. Keep the `SKF_VERIFY_STACK_RESULT_JSON` line it prints for §5: on stdout, or on stderr when VS halts. Its fields are defined once, by `shared/scripts/schemas/skf-verify-stack-result-envelope.v1.json` (in the SKF module: `{project-root}/_bmad/skf/` when installed, `{project-root}/src/` during development); §5 reads `report_path`, `overall_verdict`, `coverage_percentage` and `recommendation_count` from it.
50
50
 
51
51
  ### §5: Record the VS Outcome and Complete the Stage
52
52
 
@@ -67,8 +67,8 @@ SKF_RESULT
67
67
 
68
68
  It also sets `campaign.capstone.verified` to `true` when `overall_verdict` is `FEASIBLE`, otherwise `false` (only if a `campaign.capstone` entry exists from step-07).
69
69
 
70
- **On VS failure** (a non-zero exit, an `error` envelope): the helper sets `campaign.verification` to `null` and names it in `not_recorded`; log the exit code and `halt_reason` (from the envelope or stderr, type `event`). Verification failure does NOT block the campaign: it produces diagnostic information for operator review. When VS printed no envelope, or §3 skipped it, record no verification instead: `uv run {stateScript} set-campaign --state-file {stateFile} --no-verification --stage 7`.
70
+ **On VS failure** (a line whose `status` is `error` or whose `exit_code` is non-zero): the helper sets `campaign.verification` to `null` and names it in `not_recorded`; log that line's `exit_code` and `halt_reason` (type `event`). Verification failure does NOT block the campaign: it produces diagnostic information for operator review. When §3 skipped VS, record no verification instead: `uv run {stateScript} set-campaign --state-file {stateFile} --no-verification --stage 7`. When VS printed no `SKF_VERIFY_STACK_RESULT_JSON` line on stdout or stderr, there is no result to read, whatever its exit code: record it with `uv run {stateScript} set-campaign --state-file {stateFile} --no-result verification --stage 7`, log "no SKF_VERIFY_STACK_RESULT_JSON line (exit <code>)" with VS's exit code (type `event`), and go on.
71
71
 
72
72
  ## OUTPUT
73
73
 
74
- Display verification summary: overall verdict (or "skipped" if architecture doc was not found), report path (if produced), and coverage percentage. Chain to `{nextStepFile}`.
74
+ Display verification summary: overall verdict (or "skipped" if architecture doc was not found, or "failed (<halt_reason>)" from the helper's `not_recorded` row), report path (if produced), and coverage percentage. That row's `halt_reason` fills it (`failed (no-result)` when VS printed no line); when that is null, show `failed (exit <exit_code>)`, or `failed (no exit code)` when the exit code is null too. Chain to `{nextStepFile}`.
@@ -56,7 +56,7 @@ skf-refine-architecture --headless --architecture-doc <arch_path> --vs-report-pa
56
56
  - `--vs-report-path`: `campaign.verification.report_path` from §3, or `none` when state holds no report path.
57
57
  - `--scope-skills`: comma-separated names of completed campaign skills (from `skills[]` where `status == "completed"`). Optional but improves focus by limiting refinement scope to campaign-relevant skills.
58
58
 
59
- Keep the `SKF_REFINE_ARCHITECTURE_RESULT_JSON` line it prints on stdout for §5. Its fields are defined once, by `shared/scripts/schemas/skf-refine-architecture-result-envelope.v1.json` (in the SKF module: `{project-root}/_bmad/skf/` when installed, `{project-root}/src/` during development); §5 reads `status`, `refined_path`, `gap_count`, `issue_count` and `improvement_count` from it.
59
+ Keep the `SKF_REFINE_ARCHITECTURE_RESULT_JSON` line it prints for §5: on stdout, or on stderr when RA halts. Its fields are defined once, by `shared/scripts/schemas/skf-refine-architecture-result-envelope.v1.json` (in the SKF module: `{project-root}/_bmad/skf/` when installed, `{project-root}/src/` during development); §5 reads `status`, `refined_path`, `gap_count`, `issue_count` and `improvement_count` from it.
60
60
 
61
61
  ### §5: Record the RA Outcome and Complete the Stage
62
62
 
@@ -75,8 +75,8 @@ SKF_RESULT
75
75
  - `campaign.refinement.issue_count`: from the envelope
76
76
  - `campaign.refinement.improvement_count`: from the envelope
77
77
 
78
- **On RA failure** (a non-zero exit, an `error` envelope): the helper sets `campaign.refinement` to `null` and names it in `not_recorded`; log the exit code and `halt_reason` (from the envelope or stderr, type `event`). Refinement failure does NOT block the campaign: the campaign continues to export with whatever state exists. When RA printed no envelope, or §3 skipped it, record no refinement instead: `uv run {stateScript} set-campaign --state-file {stateFile} --no-refinement --stage 8`.
78
+ **On RA failure** (a line whose `status` is `error` or whose `exit_code` is non-zero): the helper sets `campaign.refinement` to `null` and names it in `not_recorded`; log that line's `exit_code` and `halt_reason` (type `event`). Refinement failure does NOT block the campaign: the campaign continues to export with whatever state exists. When §3 skipped RA, record no refinement instead: `uv run {stateScript} set-campaign --state-file {stateFile} --no-refinement --stage 8`. When RA printed no `SKF_REFINE_ARCHITECTURE_RESULT_JSON` line on stdout or stderr, there is no result to read, whatever its exit code: record it with `uv run {stateScript} set-campaign --state-file {stateFile} --no-result refinement --stage 8`, log "no SKF_REFINE_ARCHITECTURE_RESULT_JSON line (exit <code>)" with RA's exit code (type `event`), and go on.
79
79
 
80
80
  ## OUTPUT
81
81
 
82
- Display refinement summary: refined architecture path (or "skipped" if architecture doc was not found), gap count, issue count, and improvement count. Chain to `{nextStepFile}`.
82
+ Display refinement summary: refined architecture path (or "skipped" if architecture doc was not found, or "failed (<halt_reason>)" from the helper's `not_recorded` row), gap count, issue count, and improvement count. That row's `halt_reason` fills it (`failed (no-result)` when RA printed no line); when that is null, show `failed (exit <exit_code>)`, or `failed (no exit code)` when the exit code is null too. Chain to `{nextStepFile}`.
@@ -80,8 +80,13 @@ Operations:
80
80
  summary read from the sub-skill's result envelope line
81
81
  (a file or `-`; the line's SKF_*_RESULT_JSON: prefix is
82
82
  optional). A success envelope is recorded; an error
83
- envelope, or --no-capstone and the like, records null.
84
- A recorded verification also sets capstone.verified.
83
+ envelope, or --no-capstone and the like (a stage
84
+ skipped on purpose), records null. --no-result <field>
85
+ (repeatable) records a sub-skill that printed no result
86
+ line: null, with halt reason no-result in its
87
+ not_recorded row; it does not go with that field's
88
+ --<field> or --no-<field>. A recorded verification also
89
+ sets capstone.verified.
85
90
  recover Copy a .bak that validates over the primary (step-resume).
86
91
  archive Move the state, its .bak and the brief, each one that
87
92
  exists, into archive/<campaign name>-<UTC stamp>/ beside
@@ -130,7 +135,8 @@ CLI:
130
135
  --entry <text> [--entry <text> ...] [--stage <N>]
131
136
  uv run campaign-state.py set-campaign --state-file <p> [--architecture-doc-path <p>] \\
132
137
  [--capstone <p|-> | --no-capstone] [--verification <p|-> | --no-verification] \\
133
- [--refinement <p|-> | --no-refinement] [--stage <N>]
138
+ [--refinement <p|-> | --no-refinement] \\
139
+ [--no-result capstone|verification|refinement ...] [--stage <N>]
134
140
  uv run campaign-state.py recover --state-file <p>
135
141
  uv run campaign-state.py archive --state-file <p> --brief-file <p>
136
142
  uv run campaign-state.py log --log-file <p> --type <decision|auto|event> --text <entry>
@@ -161,8 +167,8 @@ Exit codes (each is the campaign's HALT code for the same class):
161
167
  2 invalid input: a missing or unreadable input file, an unknown skill,
162
168
  output from another script that holds an error, a gate the gate script
163
169
  rejects, a state that init would overwrite, a brief template that is
164
- no YAML mapping, export flags that do not go together, or an archive
165
- folder that exists
170
+ no YAML mapping, export flags or set-campaign flags that do not go
171
+ together, or an archive folder that exists
166
172
  3 invalid state: the state is missing, malformed or fails the schema, or
167
173
  the change would make it fail (nothing written)
168
174
  4 apply-plan: a plan the stages cannot follow (circular-deps)
@@ -787,6 +793,11 @@ SUMMARIES = (("capstone", _capstone), ("verification", _verification), ("refinem
787
793
 
788
794
 
789
795
  def op_set_campaign(args: argparse.Namespace) -> dict:
796
+ no_result = set(args.no_result or ())
797
+ for field, _build in SUMMARIES:
798
+ if field in no_result and (getattr(args, field) is not None or getattr(args, f"no_{field}")):
799
+ raise StateError(EXIT_INPUT, "input-invalid",
800
+ f"--no-result {field} does not go with --{field} or --no-{field}")
790
801
  envelopes = {}
791
802
  for field, _build in SUMMARIES:
792
803
  source = getattr(args, field)
@@ -812,6 +823,9 @@ def op_set_campaign(args: argparse.Namespace) -> dict:
812
823
  elif getattr(args, f"no_{field}"):
813
824
  value = None
814
825
  not_recorded.append({"field": field, "status": None, "halt_reason": None, "exit_code": None})
826
+ elif field in no_result:
827
+ value = None
828
+ not_recorded.append({"field": field, "status": None, "halt_reason": "no-result", "exit_code": None})
815
829
  else:
816
830
  continue
817
831
  campaign[field] = value
@@ -1054,6 +1068,9 @@ def build_parser() -> argparse.ArgumentParser:
1054
1068
  group = p.add_mutually_exclusive_group()
1055
1069
  group.add_argument(f"--{field}", help=f"the {field} sub-skill's result envelope line, a file or -")
1056
1070
  group.add_argument(f"--no-{field}", action="store_true", help=f"record campaign.{field} as null")
1071
+ p.add_argument("--no-result", action="append", choices=[f for f, _ in SUMMARIES], metavar="FIELD",
1072
+ help="record the field as null with halt reason no-result: its sub-skill printed no "
1073
+ "result line (repeat for several; not with that field's --FIELD or --no-FIELD)")
1057
1074
 
1058
1075
  command("recover", "copy a valid .bak over the primary", stage=None)
1059
1076
 
@@ -65,8 +65,8 @@ These rules apply to every step in this workflow:
65
65
  |--------|--------|
66
66
  | **Inputs** | brief_path (a skill-brief.yaml path or a skill name) [optional: without it, step 1 offers the briefs in `{forge_data_folder}`, and a headless run halts unless there is one], --batch [optional: the briefs or folders of briefs after it, `{forge_data_folder}` when none is given, each validated before the first compiles] |
67
67
  | **Gates** | step 1: Brief Selection (no brief named and several in `{forge_data_folder}`; headless: HARD HALT `brief-missing`, exit 2; one brief: picked and recorded as gate `brief-selection`) | step 3: Tag Choice Gate (several tags match the brief's version; headless: the first in priority order) | step 3 §2a: Authoritative-File Gate [P]/[S]/[U] (an authoritative AI documentation file outside the brief's scope; headless: deferred-headless, one auto-decision per path) | step 3: Review Gate [C]/[R] (with the zero-export warning when a source brief without `doc_urls` extracted no export; [R] with it or the head-cap warning) | step 3c: Zero-Export Gate [C]/[R] (a source brief with no export whose every `doc_urls` fetch failed) | step 3d: Demo-Exclusion Gate [Y], then Registry Gate [Y] (candidate found) or [S] (none found), component-library scope only, each skipped once the brief holds `scope.demo_patterns` or `scope.registry_path`, which step 3d writes when a user confirms or gives the patterns or the registry path |
68
- | **Outputs** | SKILL.md, context-snippet.md, metadata.json, provenance-map.json, evidence-report.md, extraction-rules.yaml, references/; per brief, the result contract (`create-skill-result-{YYYYMMDD-HHmmss}.json` and its `-latest.json` copy in `{forge_version}`) and one `SKF_CREATE_SKILL_RESULT_JSON` line on stdout, on stderr at a HARD HALT |
69
- | **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true; each auto-decision reaches the envelope's and the result contract's `headless_decisions`. A caller reads the run's own `SKF_CREATE_SKILL_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and `create-skill-result-latest.json` may then describe an earlier run, so it is never read in its place. |
68
+ | **Outputs** | SKILL.md, context-snippet.md, metadata.json, provenance-map.json, evidence-report.md, extraction-rules.yaml, references/; per brief, the result contract (`create-skill-result-{YYYYMMDD-HHmmss}.json` and its `-latest.json` copy in `{forge_version}`) and one `SKF_CREATE_SKILL_RESULT_JSON` line on stdout, on stderr at a HARD HALT; and under `--batch` the batch summary (`create-skill-batch-{YYYYMMDD-HHmmss}.json` and its `-latest.json` copy in `{skills_output_folder}/_batch/`) |
69
+ | **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true; each auto-decision reaches the envelope's and the result contract's `headless_decisions`. A caller reads the run's own `SKF_CREATE_SKILL_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and `create-skill-result-latest.json` may then describe an earlier run, so it is never read in its place. Under `--batch`, a caller reads the batch's `batch_summary` event and the summary file its `summary_path` names, never `create-skill-batch-latest.json` in its place: that copy holds the last batch summary written, which may be another batch's. A printed `batch_summary` event stays the batch's result even if a halt line follows it. A batch that printed no `batch_summary` event has no batch result, whatever the process exit code: a halt line of its own (phase `batch-mode`) says why it stopped, and with no halt line it was cut off or could not print its halt line. A target it printed a `done` or `fail` event for ended as that event says, and a brief the plan listed as not compiled ended failed with its message; a target with a `start` event and no end event has no result, and any other target with no event did not run in this session. |
70
70
  | **Exit codes** | 0 for a finished brief; at a HARD HALT 2 (the brief is missing, malformed or invalid), 3 (no forge config, source, helper or documentation), 4 (a write failed), 5 (a refused or broken package), 6 (the user chose to refine the scope). A `--batch` run exits with the highest code among its briefs, 0 when every brief finished |
71
71
 
72
72
  ## On Activation
@@ -50,10 +50,10 @@ persistent_facts = [
50
50
  # run a skill such as test-skill or export-skill. It is called as:
51
51
  # <on_complete> --result-path=<{forge_version}/create-skill-result-latest.json>
52
52
  # Under --batch it runs once per brief, each time with that brief's result
53
- # file; a brief that halts, or whose result file was not written, gets no
54
- # call. Useful for a Slack or email notifier, dashboard ingest, a CI script
55
- # or a git-add wrapper. A failure is reported on its own line and never
56
- # fails the brief.
53
+ # file; a brief that halts, or whose result file or its -latest copy was
54
+ # not written, gets no call. Useful for a Slack or email notifier,
55
+ # dashboard ingest, a CI script or a git-add wrapper. A failure is reported
56
+ # on its own line and never fails the brief.
57
57
  #
58
58
  # Empty string = no-op (default).
59
59
 
@@ -78,4 +78,6 @@ The helper counts the briefs, sets the batch status and exit code, writes the su
78
78
 
79
79
  Then load, read in full and execute `{healthCheckStepFile}`: the health check runs once per batch, here. The batch ends with the event's `exit_code`: 0 when every brief finished, else the highest exit code among the briefs that failed.
80
80
 
81
- If `summarize` exits non-zero (a brief is still pending, or the batch file is gone or unreadable), the health check does not run: display "**The batch stopped:** {its `message`}" and **HARD HALT** (exit code 4, `write-failed`, phase `batch-mode`; emit with nothing to stage in, as §1 does, with the reason "The batch stopped: <its message>"). The summary files may be missing then, but each brief's own result contract is in its `{forge_version}`.
81
+ **Reading a batch.** Under `--batch`, a caller reads the batch's `batch_summary` event and the summary file its `summary_path` names, never `create-skill-batch-latest.json` in its place: that copy holds the last batch summary written, which may be another batch's. The event comes before the health check, so a caller reads it from the session's output stream (`claude -p --output-format stream-json`) or transcript, not from the final message. A printed `batch_summary` event stays the batch's result even if a halt line follows it. A batch that printed no `batch_summary` event has no batch result, whatever the process exit code: a halt line of its own (phase `batch-mode`) says why it stopped, and with no halt line it was cut off or could not print its halt line. A target it printed a `done` or `fail` event for ended as that event says, and a brief the plan listed as not compiled ended failed with its message; a target with a `start` event and no end event has no result, and any other target with no event did not run in this session. When the event's `summary_path` is null, the per-batch file was not written (its `summary_write_failed` warning says why): the event and `{batch_dir}/batch.jsonl`, which the helper then keeps, are the batch's result, never `create-skill-batch-latest.json`. A `summary_write_failed` warning beside a set `summary_path` means only the `-latest` copy failed: the file it names is still the batch's result. After a cut-off batch, run `--batch` again with the same briefs: while the batch has no summary, it finds the batch folder again and goes on with the brief it was working on (Execution, above). If `summarize` already ran but its event never reached the caller, `start` begins a new batch from the first brief, and that batch's summary is the per-batch file in `{skills_output_folder}/_batch/` whose `input_briefs` match.
82
+
83
+ If `summarize` exits non-zero (a brief is still pending, or the batch file is gone or unreadable), the health check does not run: display "**The batch stopped:** {its `message`}" and **HARD HALT** (exit code 4, `write-failed`, phase `batch-mode`; emit with nothing to stage in, as §1 does, with the reason "The batch stopped: <its message>"). The summary files may be missing then, but each brief's own result contract is in its `{forge_version}`, and `create-skill-batch-latest.json` may be another batch's, so it is never read in its place.
@@ -126,13 +126,18 @@ The evidence report in `outputs` carries the `## Auto-Decisions` audit table: a
126
126
  uv run {resolveCustomizationHelper} --skill {skill-root} --project-root {project-root} --key workflow.on_complete
127
127
  ```
128
128
 
129
- When `{onCompleteCommand}` is non-empty and the emitter wrote this brief's result contract (its `emit` exited 0), run it as a shell command from `{project-root}`, for this brief:
129
+ When `{onCompleteCommand}` is non-empty and the emitter wrote this brief's result contract (its `emit` exited 0, the line's `result_path` is not null and its `warnings` hold no `result_file_write_failed` entry naming `create-skill-result-latest.json`), run it as a shell command from `{project-root}`, for this brief:
130
130
 
131
131
  ```bash
132
132
  {onCompleteCommand} --result-path={forge_version}/create-skill-result-latest.json
133
133
  ```
134
134
 
135
- The hook runs after the contract, so a git-add, registry registration or notifier sees a complete package. A hook failure never fails the brief, since the skill is written and its result contract is final: display "on_complete failed for `{name}`: {its exit code and first stderr line}" and go on. When the emitter wrote no result contract for this brief (no path in `{emitEnvelopeProbeOrder}` exists, or `emit` failed twice), `--result-path` would name a missing file or an earlier compile's: skip the hook and display "on_complete skipped for `{name}`: no result contract was written ({the reason})". When `{onCompleteCommand}` is empty (bundled default), skip the invocation entirely.
135
+ The hook runs after the contract, so a git-add, registry registration or notifier sees a complete package. A hook failure never fails the brief, since the skill is written and its result contract is final: display "on_complete failed for `{name}`: {its exit code and first stderr line}" and go on. When `{onCompleteCommand}` is non-empty but the hook does not run, `--result-path` would name a missing file or an earlier compile's: skip the hook and display the line for its case:
136
+
137
+ - No `emit` succeeded (no path in `{emitEnvelopeProbeOrder}` exists, or `emit` failed twice): "on_complete skipped for `{name}`: no result contract was written ({the reason})".
138
+ - `emit` exited 0, but the line's `result_path` is null (the per-run record was not written) or a `result_file_write_failed` warning names `create-skill-result-latest.json` (the copy alone failed): "on_complete skipped for `{name}`: `create-skill-result-latest.json` was not written".
139
+
140
+ When `{onCompleteCommand}` is empty (bundled default), skip the invocation entirely.
136
141
 
137
142
  **Remove the run folder.** The brief finished, so outside `--batch` delete `{run_dir}` (`rm -rf "{run_dir}"`); under `--batch`, batch-mode.md §3 reads the staged `result-context.json` there and removes the folder itself. A halted brief keeps its folder, with the decisions it recorded and its staged payloads.
138
143
 
@@ -221,7 +221,7 @@ npx skill-check check <staging-skill-dir> --format json
221
221
 
222
222
  Record any security warnings in evidence-report. Security findings are advisory — they do not block artifact generation. If the full validation re-run produces a different quality score than section 2, update the evidence-report with the newer score.
223
223
 
224
- **If security scan fails due to missing SNYK_TOKEN:**
224
+ **If security scan fails due to missing SNYK_TOKEN** (skill-check exits non-zero and its stderr names `SNYK_TOKEN`):
225
225
 
226
226
  Display: "Security scan requires a Snyk Enterprise API token ([docs](https://docs.snyk.io/snyk-api/authentication-for-api)). Set `SNYK_TOKEN=your-token` in environment or `.env`, then re-run [SF] Setup Forge. Without Enterprise, use `--no-security-scan` to skip. Security scanning is optional and does not block skill compilation."
227
227
 
@@ -120,15 +120,23 @@ Record format violations as **WARNING** findings.
120
120
 
121
121
  ### 9. Security Scan (if skill-check available)
122
122
 
123
- Run: `npx skill-check check <skill-dir> --format json` (security scan enabled by default).
123
+ Run: `npx skill-check check <skill-dir> --format json > "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"; echo "skill-check exited $?"` (security scan enabled by default), so stdout, stderr and the exit code are read apart.
124
124
 
125
- Record security findings as advisory **WARNING** findings — they do not block the report.
125
+ **If `{run_dir}/skill-check.json` does not start with a JSON object**, skill-check stopped before it validated (exit 2 on a config, option or plugin error): record no finding, only the §10 note "security scan skipped: skill-check failed ({the stderr line that names the cause, or its last line})". Otherwise stdout can go on after the JSON (a `Wrote <path>` line when the project config sets a report path), so read only the JSON object at its start.
126
126
 
127
- **If unavailable:** Skip with note in validation results.
127
+ **Read the JSON, not the exit code.** skill-check exits 1 when `summary.errorCount` is above 0, when `--strict` or `--fail-on-warning` is passed and `summary.warningCount` is above 0 (this call passes neither), or when the scan exits non-zero, and 2 when no scan runner starts, so a non-zero exit is not a security finding. skill-check 1.2.0 puts no security key in the JSON. The scan exited non-zero, which without `--ci` means it did not finish, when skill-check exits 1 although `summary.errorCount` is 0: stderr is then the scan's last lines (its tail), even when blank. On exit 0, stderr is never a scan tail (npm and Node warnings can be there); when exit 1 has another cause, stderr is a scan tail only when it holds the scanner's own lines, such as its token message or a finding code. A finding code is `[`, a capital letter, three digits and `]` (such as `[E001]`), the codes skill-check's own scan summary counts. Read the scan from stderr and the exit code, the first that applies:
128
+
129
+ - **Could not run:** skill-check exits 2, the call stopped before it exited, or the scan exited non-zero and its tail carries no finding code. Record no finding, only a note in the §10 validation results: "security scan skipped: SNYK_TOKEN not configured" when the tail says to set the `SNYK_TOKEN` environment variable (the scanner prints "To use Agent Scan, set the SNYK_TOKEN environment variable."), otherwise "security scan skipped: scanner could not run ({the stderr line that names the cause, or its last line})", or "security scan skipped: scanner exited non-zero with no output" when stderr is blank.
130
+ - **Findings:** `security[]` holds findings, or the tail carries finding codes.
131
+ - **Neither:** record the §10 note "security scan exited 0; skill-check 1.2.0 does not pass the scan's findings on". Without `--ci`, agent-scan also exits 0 when its analysis fails (a 401 from a bad token, a 429, a network error), so this does not show that the scan found nothing (#892).
132
+
133
+ Record security findings as advisory **WARNING** findings: they do not block the report.
134
+
135
+ **If unavailable:** Skip, with the note "security scan skipped: skill-check unavailable" in the §10 validation results.
128
136
 
129
137
  ### 10. Display Validation Results
130
138
 
131
- Report the validation outcome. If all checks passed, state so and name what was verified: file presence (`{count}/{count}`), SKILL.md structure, metadata.json fields, the `{lib_count}` library + `{pair_count}` integration reference files, and complete confidence-tier coverage. If there were findings, report the `{warning_count}` finding(s) — each with severity, description, and file path — plus files present/expected and warning/error counts; when errors include missing files, note this may indicate a write failure in step 07.
139
+ Report the validation outcome. If all checks passed, state so and name what was verified: file presence (`{count}/{count}`), SKILL.md structure, metadata.json fields, the `{lib_count}` library + `{pair_count}` integration reference files, and complete confidence-tier coverage. If there were findings, report the `{warning_count}` finding(s), each with severity, description, and file path, plus files present/expected and warning/error counts; when errors include missing files, note this may indicate a write failure in step 07. In both cases, list each §9 security scan note.
132
140
 
133
141
  ### 11. Auto-Proceed to Next Step
134
142
 
@@ -53,7 +53,7 @@ These rules apply to every step in this workflow:
53
53
  | **Overrides** | `--language-hint`, `--scope-hint`, `--description`, `--exports`, `--skip-snippet`, `--no-active-pointer`, `--batch <file>`, `--fail-fast`: see On Activation step 4 |
54
54
  | **Gates** | step 1: target input, ambiguous package name [C/n/U/X] (if another registry also holds the name; headless keeps the first registry's pick), multi-language disambiguation [C/n/A] (pick any detected language); step 2: ecosystem match [P/I/A] (none until agentskills.io has a registry API); step 3: repo-shape [C/A] + zero-exports rescue [R/P/A]; step 4: review [C/E/S/Q]; step 5: overwrite [Y/N] |
55
55
  | **Outputs** | SKILL.md, context-snippet.md, metadata.json, active pointer, result contract (timestamped + `-latest` copy), one `SKF_QUICK_SKILL_RESULT_JSON` line (stdout when the run finishes, stderr at a HARD HALT), and under `--batch` the batch summary. Snippet and active pointer can be skipped per overrides. |
56
- | **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true; each auto-decision is recorded in the envelope's `headless_decisions`. A caller reads the run's own `SKF_QUICK_SKILL_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and the result contract's `-latest` copy may then describe an earlier run, so it is never read in its place. |
56
+ | **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true; each auto-decision is recorded in the envelope's `headless_decisions`. A caller reads the run's own `SKF_QUICK_SKILL_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and the result contract's `-latest` copy may then describe an earlier run, so it is never read in its place. Under `--batch`, a caller reads the batch's `batch_summary` event and the summary file its `summary_path` names, never the batch summary's `-latest` copy in its place: that copy holds the last batch summary written, which may be another batch's. A printed `batch_summary` event stays the batch's result even if a halt line follows it. A batch that printed no `batch_summary` event has no batch result, whatever the process exit code: a halt line of its own (phase `batch-mode`, or `on-activation`) says why it stopped, and with no halt line it was cut off or could not print its halt line. A target it printed a `done` or `fail` event for ended as that event says; a target with a `start` event and no end event has no result, and a target with no event did not run in this session. |
57
57
  | **Exit codes** | See `references/halt-contract.md`: the exit-code map and the envelope every HARD HALT emits, except On Activation step 1's check for `python3` and `uv`, which prints none |
58
58
 
59
59
  ## On Activation
@@ -93,6 +93,8 @@ uv run {quickBatchHelper} summarize --run-dir "{batch_dir}" --output-dir "{batch
93
93
 
94
94
  The helper counts the targets, sets the batch status and exit code, writes the summary files below, appends the final status to the batch file (and removes the batch run folder when no target failed), and prints the `batch_summary` event on stderr: display it verbatim. Then print step 7's `start` event and load, read in full and execute `{healthCheckStepFile}`: the health check runs once per batch, here. The batch ends with the event's `exit_code`.
95
95
 
96
+ **Reading a batch.** Under `--batch`, a caller reads the batch's `batch_summary` event and the summary file its `summary_path` names, never `quick-skill-batch-latest.json` in its place: that copy holds the last batch summary written, which may be another batch's. The event comes before the health check, so a caller reads it from the session's output stream (`claude -p --output-format stream-json`) or transcript, not from the final message. A printed `batch_summary` event stays the batch's result even if a halt line follows it. A batch that printed no `batch_summary` event has no batch result, whatever the process exit code: a halt line of its own (phase `batch-mode`, or `on-activation`) says why it stopped, and with no halt line it was cut off or could not print its halt line. A target it printed a `done` or `fail` event for ended as that event says; a target with a `start` event and no end event has no result, and a target with no event did not run in this session. When the event's `summary_path` is null, the per-batch file was not written (its `summary_write_failed` warning says why): the event and `{batch_dir}/batch.jsonl`, which the helper then keeps, are the batch's result, never `quick-skill-batch-latest.json`. A `summary_write_failed` warning beside a set `summary_path` means only the `-latest` copy failed: the file it names is still the batch's result. After a cut-off batch, run the batch again: each run takes a new run folder (SKILL.md On Activation step 1), so it starts over from the first target.
97
+
96
98
  ## Batch summary contract
97
99
 
98
100
  `summarize` writes the batch summary at:
@@ -112,23 +112,27 @@ Check:
112
112
  Run: `npx skill-check -h`
113
113
 
114
114
  - If succeeds (returns usage information): Continue to automated validation (section 4)
115
- - If fails (command not found or error): skip to the frontmatter validator in section 4, and log "security scan skipped: skill-check unavailable" in the validation results
115
+ - If fails (command not found or error): skip to the frontmatter validator in section 4, bind `{security_result}` ← `skipped`, and log "security scan skipped: skill-check unavailable" in the validation results
116
116
 
117
117
  ### 4. Validate SKILL.md via skill-check (if available)
118
118
 
119
- **If `npx skill-check` is available**, run validation and its security scan in one call against the skill package written in section 2. `--fix` can also rewrite the frontmatter `description` the user approved in step 4, which would leave SKILL.md out of step with `metadata.json`, so the call runs inside the description guard. Resolve `{descriptionGuardHelper}` ← first existing path in `{descriptionGuardProbeOrder}`, then run the three commands in order:
119
+ **If `npx skill-check` is available**, run validation and its security scan in one call against the skill package written in section 2. `--fix` can also rewrite the frontmatter `description` the user approved in step 4, which would leave SKILL.md out of step with `metadata.json`, so the call runs inside the description guard. Resolve `{descriptionGuardHelper}` ← first existing path in `{descriptionGuardProbeOrder}`, then run the three commands in order. `verify-restore` runs whatever skill-check exits; note that exit code:
120
120
 
121
121
  ```bash
122
122
  uv run {descriptionGuardHelper} capture "{skill_package}/SKILL.md" > "{run_dir}/description-guard.json"
123
- npx skill-check check "{skill_package}" --fix --format json
123
+ npx skill-check check "{skill_package}" --fix --format json > "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"; echo "skill-check exited $?"
124
124
  uv run {descriptionGuardHelper} verify-restore "{skill_package}/SKILL.md" --captured-file "{run_dir}/description-guard.json"
125
125
  ```
126
126
 
127
+ **If no "skill-check exited" line came back**, the shell tool stopped the block before skill-check finished (a cold `uvx mcp-scan` install can outlast its timeout): run the `verify-restore` command on its own before reading `{run_dir}/skill-check.json`, so a description `--fix` rewrote is restored.
128
+
127
129
  **Guard outputs.** `capture` stages the approved description in `{run_dir}/description-guard.json` and `verify-restore` reads it there, so it never passes through a shell string. Bind `{guard_restored}` ← `restored` and `{guard_diff_kind}` ← `diff_kind` from `verify-restore`. When `{guard_restored}` is true, the approved description is back in SKILL.md, matching `metadata.json`: record the validation note "description restored after `skill-check --fix` ({guard_diff_kind})", which §6 lists with the auto-fixed issues. `verify-restore` refuses an empty or whitespace-only captured description, and a file it cannot read (exit 1, SKILL.md untouched): re-run it with `--captured-file "{skill_package}/metadata.json"`, whose `description` step 4 rendered, never with an empty value, and if that exits 1 too, record a high-severity issue: the guard could not check the description. When it exits 2 with JSON on stdout (the JSON carries `restore_error`), the restore could not be written, so record a high-severity issue: SKILL.md's description no longer matches `metadata.json`.
128
130
 
129
- **If no `{descriptionGuardProbeOrder}` path exists**, never run `--fix` unguarded. Run `npx skill-check check "{skill_package}" --format json` instead, which validates and scans without fixing, and log the validation issue "description guard unavailable (`skf-description-guard.py` missing): skill-check ran without `--fix`".
131
+ **If no `{descriptionGuardProbeOrder}` path exists**, never run `--fix` unguarded. Run `npx skill-check check "{skill_package}" --format json` instead, which validates and scans without fixing, with the same capture (`> "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"`, its exit code noted), and log the validation issue "description guard unavailable (`skf-description-guard.py` missing): skill-check ran without `--fix`".
132
+
133
+ **If `{run_dir}/skill-check.json` does not start with a JSON object**, skill-check stopped before it validated (exit 2 on a config, option or plugin error): log "skill-check failed: {the stderr line that names the cause, or its last line}" and "security scan skipped: skill-check failed ({that line})", keep `{quality_score}` null, bind `{security_result}` ← `skipped`, and run the frontmatter validator below, as when skill-check is not available. Otherwise stdout can go on after the JSON (a `Wrote <path>` line when the project config sets a report path), so read only the JSON object at its start.
130
134
 
131
- **Parse JSON output** to extract:
135
+ **Parse the JSON in `{run_dir}/skill-check.json`** to extract:
132
136
  - `scores[].score` — overall score (0-100); match the entry by `relativePath`/`skillId`
133
137
  - `diagnostics[]` — remaining issues after auto-fix
134
138
  - `fixed[]` — issues automatically corrected
@@ -136,6 +140,12 @@ uv run {descriptionGuardHelper} verify-restore "{skill_package}/SKILL.md" --capt
136
140
 
137
141
  Bind `{quality_score}` ← that score, and record the remaining diagnostics and security findings as validation issues. `{quality_score}` is null when skill-check did not run.
138
142
 
143
+ **Read the JSON, not the exit code.** skill-check prints its JSON before the security scan runs, then exits 1 when `summary.errorCount` is above 0, when `--strict` or `--fail-on-warning` is passed and `summary.warningCount` is above 0 (this call passes neither), or when the scan exits non-zero, and 2 when no scan runner starts, so `summary.errorCount` alone is the validation verdict. skill-check 1.2.0 puts no security key in the JSON. The scan exited non-zero, which without `--ci` means it did not finish, when skill-check exits 1 although `summary.errorCount` is 0: stderr is then the scan's last lines (its tail), even when blank. On exit 0, stderr is never a scan tail (npm and Node warnings can be there); when exit 1 has another cause, stderr is a scan tail only when it holds the scanner's own lines, such as its token message or a finding code. A finding code is `[`, a capital letter, three digits and `]` (such as `[E001]`), the codes skill-check's own scan summary counts. Bind `{security_result}` from `{run_dir}/skill-check.stderr` and the exit code, the first that applies:
144
+
145
+ - `skipped`: §3 found no skill-check (§3's log is the note), skill-check exits 2, the call stopped before it exited, or the scan exited non-zero and its tail carries no finding code. Log in the validation results "security scan skipped: SNYK_TOKEN not configured" when the tail says to set the `SNYK_TOKEN` environment variable (the scanner prints "To use Agent Scan, set the SNYK_TOKEN environment variable."), otherwise "security scan skipped: scanner could not run ({the stderr line that names the cause, or its last line})", or "security scan skipped: scanner exited non-zero with no output" when stderr is blank.
146
+ - `warn`: `security[]` holds findings, or the tail carries finding codes. Record each finding as a validation issue.
147
+ - `pass`: otherwise. The scan exited 0 and skill-check reported no finding. Without `--ci`, agent-scan also exits 0 when its analysis fails (a 401 from a bad token, a 429, a network error), so this does not show that the scan found nothing (#892).
148
+
139
149
  **If skill-check is not available**, run the shared frontmatter validator. Resolve `{frontmatterValidator}` from `{frontmatterValidatorProbeOrder}`; first existing path wins. If no candidate exists, nothing can check the frontmatter: HARD HALT with **exit code 3 (resolution-failure)**, in interactive mode too: "**`{skill_package}` was written but not validated.** skill-check is unavailable and SKF's frontmatter validator (`skf-validate-frontmatter.py`) is missing from `{project-root}/_bmad/skf/shared/scripts/`, so re-install SKF, then re-run." Stage `{"phase": "write-and-validate", "halt_reason": "resolution-failure", "reason": "The skill was written but not validated: the frontmatter validator is missing.", "skill_package": "{skill_package}", "outputs": {<each deliverable §2 wrote, as "metadata", "skill_md" or "context_snippet" with its path>}, "details": {"cause": "frontmatter-validator-missing"}}` as `{run_dir}/halt.json` and run `uv run {emitEnvelopeHelper} emit-halt --workflow skf-quick-skill --run-dir "{run_dir}" --result-dir "{skill_package}" --target stderr < "{run_dir}/halt.json"`.
140
150
 
141
151
  ```bash
@@ -160,7 +170,7 @@ The validator emits JSON with `result` (PASS/FAIL), `validation.skill_md.body[]`
160
170
 
161
171
  "**Validation complete:**
162
172
 
163
- **SKILL.md:** {pass/issues found} (quality score: {score}/100 if skill-check was available)
173
+ **SKILL.md:** {pass/issues found}{ (quality score: {quality_score}/100), only when `{quality_score}` is not null}
164
174
  {list any issues}
165
175
  {list any auto-fixed issues, and the §4 description-guard note when the guard restored the description}
166
176
 
@@ -170,8 +180,8 @@ The validator emits JSON with `result` (PASS/FAIL), `validation.skill_md.body[]`
170
180
  **metadata.json:** {pass/issues found}
171
181
  {list any issues}
172
182
 
173
- **Security:** {pass/warn/skipped}
174
- {list any security findings}
183
+ **Security:** {security_result}
184
+ {list any security findings, or the §4 skipped note; for `pass`: "scan exited 0; skill-check 1.2.0 does not pass the scan's findings on"}
175
185
 
176
186
  **Overall:** {pass / N issues found}
177
187
 
@@ -180,7 +190,7 @@ These issues are advisory for community-tier skills: fix them in the written fil
180
190
 
181
191
  **Proceeding to finalize...**"
182
192
 
183
- Set `validation_result` with pass/fail status, quality score, and issues list, and `{validation_issues}` ← `{"skill_md": <n>, "context_snippet": <n>, "metadata": <n>, "security": <n>}`, each `<n>` the number of entries the validators returned for that heading: `skill_md` counts skill-check's `diagnostics[]` (without skill-check, the frontmatter validator's `issues[]`) plus `validation.skill_md.body[]`; `context_snippet` counts `validation.context_snippet.issues[]`; `metadata` counts `validation.metadata.issues[]`; `security` counts skill-check's `security[]`. A check that did not run counts 0, and `fixed[]` and the guard note count nothing. Step 6 §3's summary carries `{quality_score}` and `{validation_issues}`.
193
+ Set `validation_result` with pass/fail status, quality score, and issues list, and `{validation_issues}` ← `{"skill_md": <n>, "context_snippet": <n>, "metadata": <n>, "security": <n>}`, each `<n>` the number of entries the validators returned for that heading: `skill_md` counts skill-check's `diagnostics[]` (without skill-check, the frontmatter validator's `issues[]`) plus `validation.skill_md.body[]`; `context_snippet` counts `validation.context_snippet.issues[]`; `metadata` counts `validation.metadata.issues[]`; `security` counts skill-check's `security[]` (or, without one, the finding codes §4 read from the tail, each occurrence one finding, as skill-check's own scan summary counts them). A check that did not run counts 0, and `fixed[]` and the guard note count nothing. Step 6 §3's summary carries `{quality_score}` and `{validation_issues}`.
184
194
 
185
195
  ### 7. Auto-Proceed to Finalize
186
196
 
@@ -86,13 +86,13 @@ uv run {coverageInputsScript} floor --skill-dir "{resolved_skill_package}" --out
86
86
 
87
87
  Exit 1 printed its error on stderr: HALT with it (`halt_reason: "helper-failed"`, phase `coverage-check:inventory`).
88
88
 
89
- Delegate the reading of the skill under test to a subagent, given `{run_dir}` and the path to SKILL.md and to its `references/` directory when one exists: it reads SKILL.md, and every `references/*.md` file too when SKILL.md's `## Full` headings are absent or stubs. It is given `{run_dir}/inventory-floor.json` too, and lists each name there with an origin in a file it reads, choosing its kind from the name's `kindLabel`, the skill's own label, when it has one. For each export two or more of the files it reads document, it compares the SKILL.md body with each reference file, and two reference files with each other only when the body does not settle them (it does not document the export, or agrees with both), by the rules of `{signatureRulesFile}`, whose absolute path (under `{skill-root}`) it is given: parameters, the return type, and descriptions that contradict (brief against detailed is fine). That file alone says what a form may leave out. It writes only this JSON, with no prose and no commentary, to `{run_dir}/inventory-response.txt` with the Write tool, never a shell, and returns that path, or returns the JSON itself when it cannot write files. The JSON's shape and the contract the validator checks are in `{inventoryContractFile}`, whose absolute path (under `{skill-root}`) the subagent is given.
89
+ Delegate the reading of the skill under test to a subagent, given `{run_dir}` and the path to SKILL.md and to its `references/` directory when one exists: it reads SKILL.md, and every `references/*.md` file too when SKILL.md's `## Full` headings are absent or stubs. It is given `{run_dir}/inventory-floor.json` too, and lists each name there with an origin in a file it reads, choosing its kind from the name's `kindLabel`, the skill's own label, when it has one. An origin's `signature` is that place's entry, its non-blank `params` and `return_type` kept as given; a blank one is the reader's. For each export two or more of the files it reads document, it compares the SKILL.md body with each reference file, and two reference files with each other only when the body does not settle them (it does not document the export, or agrees with both), by the rules of `{signatureRulesFile}`, whose absolute path (under `{skill-root}`) it is given: parameters, the return type, and descriptions that contradict (brief against detailed is fine). That file alone says what a form may leave out. It writes only this JSON, with no prose and no commentary, to `{run_dir}/inventory-response.txt` with the Write tool, never a shell, and returns that path, or returns the JSON itself when it cannot write files. The JSON's shape and the contract the validator checks are in `{inventoryContractFile}`, whose absolute path (under `{skill-root}`) the subagent is given.
90
90
 
91
91
  When the subagent returned the JSON rather than the path, or the file is absent or empty afterwards, save its response, as it came back, to `{run_dir}/inventory-response.txt` with the Write tool. The documented inventory is the one §1a validates: do not load SKILL.md or the reference files into the parent context. Without subagents, the parent loads `{inventoryContractFile}` and builds the same JSON from the same files and `inventory-floor.json` and saves it to that file; §1a's checks still run on it.
92
92
 
93
93
  #### 1a. Parent-Side Schema Validation + Spot-Check
94
94
 
95
- Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped), spot-checks sampled names against SKILL.md and the subagent's listed reference files, and refuses it when it leaves out a floor name those files write. The look-up is the script's, never a grep: an export named `$state` or `a.b` is neither a shell variable nor a regex.
95
+ Before anything consumes the inventory, `{validateInventoryScript}` checks it against the §1 contract (a wrapping fence stripped), spot-checks sampled names against SKILL.md and the subagent's listed reference files, and refuses it when it leaves out a floor name those files write, or an origin's `signature`. The look-up is the script's, never a grep: an export named `$state` or `a.b` is neither a shell variable nor a regex.
96
96
 
97
97
  ```bash
98
98
  uv run {validateInventoryScript} --input "{run_dir}/inventory-response.txt" --skill-package "{resolved_skill_package}" --output "{run_dir}/inventory.json" --floor "{run_dir}/inventory-floor.json"
@@ -22,6 +22,6 @@ The JSON coverage-check.md §1's reader writes to `{run_dir}/inventory-response.
22
22
  }
23
23
  ```
24
24
 
25
- The contract `validate-inventory.py` checks (coverage-check.md §1a): each `kind` is one of `function`, `class`, `type`, `constant`, `hook`, `interface`, `method`, `struct`, `enum`, `trait`, `macro` or `adapter`; every entry carries the `description` the skill gives it (an empty string when it gives none); a function or method carries `params` (`""` or `[]` when it takes none) and `return_type` as the skill documents them, and leaves a key out when the skill documents nothing for it. A `method`'s `name` is the bare method name and its `owner` the class or object it is called on (`agents` for `cognee.agents.get`, where the skill documents `agents` as a class; the class for one reached through a property, `client.messages`); `owner` is not validated. A function the skill documents in a module is a `function` named bare, never a `method`, even when it is called through the module's name (`register_sql_connection` for `cognee.tools.register_sql_connection`, where the skill documents `tools` as a module). Each `methods` item starts with the method's name, optionally after `async` (`async next_uuid(dep=None)`). A `const` holding a component (a `forwardRef`, `memo` or other wrapper call, or an alias like `const Tabs = TabsPrimitive.Root`) is a `constant` with the declaration or usage form the skill writes as `usage_signature`; a data constant carries no `usage_signature`, `params` or `return_type`. A name documented in several places is listed once per place, each entry with that place's form (`Card` above), and keeps one `kind` in every place. Where the skill documents several names and states members they share once, in the same place (a table row, a paragraph) or over a table for all its rows (a sentence or caption just above it, or its column header), each name's entry carries that shared text in its `params` (in its `usage_signature`, for a usage form), never in a `methods` item, which still starts with the method's name: the shared text first, then `; `, then the name's own members as the skill writes them, without the name, never the name's own members alone (`Note` above, from a row that documents `Note` and `Task`, each with the base fields `id, scope=Scope()`, as `Note(content, title=None)` and `Task(title, done=False)`). An entry whose signature the skill labels partial (simplified, key parameters, a `...` standing in for omitted members, not rest or variadic syntax; a label in a table's column header, such as `Props (simplified)`, labels each of its rows) carries `signature_form: "partial"`, any other `"full"` or no key: copy the label, never judge the form.
25
+ The contract `validate-inventory.py` checks (coverage-check.md §1a): each `kind` is one of `function`, `class`, `type`, `constant`, `hook`, `interface`, `method`, `struct`, `enum`, `trait`, `macro` or `adapter`; every entry carries the `description` the skill gives it (an empty string when it gives none); a function or method carries `params` (`""` or `[]` when it takes none) and `return_type` as the skill documents them, and leaves a key out when the skill documents nothing for it. A name written with empty parentheses in running text or a description cell is a mention, not a form: its entry leaves `params` and `usage_signature` out (a cell reading "`instrument()` wraps functions"). A description cell is any table cell but an export table's first cell and a cell whose column writes the call or signature, by its role (Signature, Props, Params, Parameters, Arguments, Usage, Syntax, Example and the like); a heading or an export table's first cell that writes `name()` is a form, recorded as written. A `method`'s `name` is the bare method name and its `owner` the class or object it is called on (`agents` for `cognee.agents.get`, where the skill documents `agents` as a class; the class for one reached through a property, `client.messages`); `owner` is not validated. A function the skill documents in a module is a `function` named bare, never a `method`, even when it is called through the module's name (`register_sql_connection` for `cognee.tools.register_sql_connection`, where the skill documents `tools` as a module). Each `methods` item starts with the method's name, optionally after `async` (`async next_uuid(dep=None)`). A `const` holding a component (a `forwardRef`, `memo` or other wrapper call, or an alias like `const Tabs = TabsPrimitive.Root`) is a `constant` with the declaration or usage form the skill writes as `usage_signature`; a data constant carries no `usage_signature`, `params` or `return_type`. A name documented in several places is listed once per place, each entry with that place's form (`Card` above), and keeps one `kind` in every place. A place `inventory-floor.json` gives a `signature` (a cited function declaration, a Signature cell, or a heading or an export table's first cell that writes one call) has that form as its entry, whatever its kind, its non-blank `params` and `return_type` kept as given (a `params` may open with the shared text the next sentence gives, then `; `); a blank one is the reader's, and a form the skill writes only in prose is an entry of its own. Where the skill documents several names and states members they share once, in the same place (a table row, a paragraph) or over a table for all its rows (a sentence or caption just above it, or its column header), each name's entry carries that shared text in its `params` (in its `usage_signature`, for a usage form), never in a `methods` item, which still starts with the method's name: the shared text first, then `; `, then the name's own members as the skill writes them, without the name, never the name's own members alone (`Note` above, from a row that documents `Note` and `Task`, each with the base fields `id, scope=Scope()`, as `Note(content, title=None)` and `Task(title, done=False)`). An entry whose signature the skill labels partial (simplified, key parameters, a `...` standing in for omitted members, not rest or variadic syntax; a label in a table's column header, such as `Props (simplified)`, labels each of its rows) carries `signature_form: "partial"`, any other `"full"` or no key: copy the label, never judge the form.
26
26
 
27
27
  A `cross_check_mismatches` entry gives its `export`, `reference_file`, `reference_line` and an `issue` that names each form by its file's path, and the pair's other side: the SKILL.md body's line as `skill_md_line`, or, for two references, the one that sorts first in codepoint order, by path and then line, as `other_file` and `other_line`, with the one that sorts last as `reference_file` and `reference_line`. Each file is its path in the package, such as `references/api-reference.md`.
@@ -19,7 +19,7 @@ This table is the single source of truth for classifying a gap. Every stage reco
19
19
  | Severity | Category | Criteria |
20
20
  |----------|-----------------------------|----------|
21
21
  | Critical | `signature-mismatch` | Wrong signature: a documented signature that differs from the source in its parameters (name, type, order, optionality) or its return type, as signature-comparison-rules.md reads each form (a component by its props; an annotation either side leaves out is not compared), in any way but those the Low `signature-mismatch` row and the Info `observation` row grade, such as a renamed Python parameter (the script cannot see which parameters a caller must pass by position), an optional parameter left out of a form the skill does not label partial, or a mismatch with no `difference` (coverage-check §2 and §2b) |
22
- | Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name (coverage-check §2c) |
22
+ | Critical | `fabricated-signature` | Fabricated signature: a documented export absent at the source line the skill cites for it. Decided only where ast-grep enumerated the exports at the pinned commit: the name is outside that surface, and the cited file is missing or defines no such name, and no item of an export list (`export { L as NAME }`, `export type { ... }`) or of a Python `from X import` clause elsewhere renames to it a name that file declares; any other rename (`export { default as NAME }`, `const NAME = L`, `type NAME = L`, an import then an export, Python `NAME = L`) stays fabricated (coverage-check §2c) |
23
23
  | Critical | `broken-reference` | Broken reference (contextual mode): a reference whose target does not exist, such as a file not at the stated path, a skill not in the skills output folder, a type no module declares, or a `scripts/` or `assets/` file the skill names (coherence-check §4) |
24
24
  | High | `inaccurate-reference` | Inaccurate reference (contextual mode): the target exists but does not match, such as a type the cited module does not export, a signature that differs, or an integration pattern that does not match the implementation (coherence-check §4) |
25
25
  | High | `reference-escape` | A reference whose real path leaves the skill, its source and, for a stack, the skills output folder (coherence-check §4) |
@@ -45,7 +45,7 @@ This table is the single source of truth for classifying a gap. Every stage reco
45
45
  | Low | `metadata` | Missing optional metadata or examples |
46
46
  | Low | `description` | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps), a Tessl Review description suggestion included |
47
47
  | Low | `external-validator` | A skill-check diagnostic or a Tessl Review validation finding the run left unresolved (external-validators §5b) |
48
- | Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c). Also a partial signature: a mismatch the signature judge tags `omitted-optional` (defined in signature-comparison-rules.md, Difference tags) on a form the skill labels partial (simplified, key parameters, a `...` that stands in for omitted members, not rest or variadic syntax), which the inventory marks `signature_form: "partial"`, as score-signatures.py checks: `Partial signature: {name}`, at the declaration, counted as matching in Signature Accuracy (coverage-check §2 and §2b). Also a return annotation the body contradicts: a mismatch the signature judge tags `annotation-contradicted` (defined in signature-comparison-rules.md, Difference tags) whose documented signature states a return type and writes the source's parameters, and whose `evidence_line`, the body line that returns the documented type, comes after the declaration and before the next one and, with the local source, starts with `return` (or is a one-line declaration's own expression body), as score-signatures.py checks: `Annotation contradicted: {name}`, at the declaration, counted as matching in Signature Accuracy, with no change to the skill (the annotation is reported upstream) (coverage-check §2 and §2b) Also a documentation error a Deep-tier run sees that no script checks (an import path whose module does not export the name, a type member that differs from its declaration): `Not checked by script: {name}`, at the documenting line, written by hand (coverage-check §2 and §5b) |
48
+ | Info | `observation` | Style suggestions, non-blocking observations, and a documented extra: a documented name outside the enumerated surface that is not a fabricated signature and that the source still declares at column 0 (a module so named counts, unless the inventory gives the name a `function`, `hook`, `class`, `interface`, `enum` or `type` kind, no provenance map entry of the name types it a `module` or `package` and the module is not a TS/JS file that is itself the export, with an `export default` or a `module.exports =`; among several declaring files, the one the skill cites on a line naming it; an `aliased` name, which an export-list item or a Python `from X import` item renames from a name its cited file declares, at that item's line, a multi-line list's item at its own line), found only in a Python or TS/JS source where classify-stale ran (the Forge, Forge+ or Deep tier, `analysis_confidence` `full`, `workspaceDrift` not `overridden` and a bound provenance map) and `surface.json` has a complete `extraction`: `Documented extra: {name}`, at that line, which update-skill never routes (coverage-check §2c). Also a partial signature: a mismatch the signature judge tags `omitted-optional` (defined in signature-comparison-rules.md, Difference tags) on a form the skill labels partial (simplified, key parameters, a `...` that stands in for omitted members, not rest or variadic syntax), which the inventory marks `signature_form: "partial"`, as score-signatures.py checks: `Partial signature: {name}`, at the declaration, counted as matching in Signature Accuracy (coverage-check §2 and §2b). Also a return annotation the body contradicts: a mismatch the signature judge tags `annotation-contradicted` (defined in signature-comparison-rules.md, Difference tags) whose documented signature states a return type and writes the source's parameters, and whose `evidence_line`, the body line that returns the documented type, comes after the declaration and before the next one and, with the local source, starts with `return` (or is a one-line declaration's own expression body), as score-signatures.py checks: `Annotation contradicted: {name}`, at the declaration, counted as matching in Signature Accuracy, with no change to the skill (the annotation is reported upstream) (coverage-check §2 and §2b) Also a documentation error a Deep-tier run sees that no script checks (an import path whose module does not export the name, a type member that differs from its declaration): `Not checked by script: {name}`, at the documenting line, written by hand (coverage-check §2 and §5b) |
49
49
  | Info | `provenance-unverified` | Provenance line not verified: the line-check rules found no definition line, or the recorded line is not a line number (`line-invalid`) (coverage-check §4c) |
50
50
  | Info | `migration-section` | Historical migration content in Section 4b with no T2-future annotation (Case 2 of `migration-section-rules.md`) |
51
51
  | Info | `multi-denominator` | Multi-denominator reporting: barrel and documented-surface clusters diverge by design (>10% cross-cluster) |
@@ -6,9 +6,11 @@ Every signature judge compares by these rules: coverage-check.md §1's split-bod
6
6
 
7
7
  A form claims only what it writes, and only what it claims is compared. Its parameter list claims each parameter it lists, in that order, and no other: a parameter it leaves out is one it says the export does not take (Difference tags say when that is `omitted-optional`). A type or a return type it does not write claims nothing (Annotations).
8
8
 
9
+ A name written with empty parentheses in running text or a description cell is a mention, not a form: it claims nothing and is not compared. A description cell is any table cell but an export table's first cell and a cell whose column writes the call or signature, by its role (Signature, Props, Params, Parameters, Arguments, Usage, Syntax, Example and the like); a heading or an export table's first cell that writes `name()` is a form, an empty call (below).
10
+
9
11
  Three kinds of form do not claim a complete parameter list:
10
12
 
11
- - A call or usage example passes values (literals, variables, `name=value`): `RecursiveSplitter()`, `embed(text, model=m)`, `<Canvas of={X} />`. It is compared as a call: each argument it passes must exist, by position or by name, and every required parameter must be passed; an optional parameter it leaves out is no difference. An empty call `X()` is one too, so it matches a source whose parameters are all optional. A component usage `<X a={x} b="y" />` reads as a list of bare names of the props it sets, and a call `f(a, b)` as its arguments in order, each the parameter at its position (or, written `name=value`, the one it names). A JSX spread (`{...props}`) or children can pass a required prop, and a call's `...args` a required parameter, so neither leaves one out. An entry's `usage_signature` written as a usage is read this way, and each usage in a list of usages is compared as a form of its own, under its entry's `form` index.
13
+ - A call or usage example passes values (literals, variables, `name=value`): `RecursiveSplitter()`, `embed(text, model=m)`, `<Canvas of={X} />`. It is compared as a call: each argument it passes must exist, by position or by name, and every required parameter must be passed; an optional parameter it leaves out is no difference. An empty call `X()` written as a call example (in a heading, an export table's first cell, a call or signature column or a fenced example, never a mention) is one too, so it matches a source whose parameters are all optional. A component usage `<X a={x} b="y" />` reads as a list of bare names of the props it sets, and a call `f(a, b)` as its arguments in order, each the parameter at its position (or, written `name=value`, the one it names). A JSX spread (`{...props}`) or children can pass a required prop, and a call's `...args` a required parameter, so neither leaves one out. An entry's `usage_signature` written as a usage is read this way, and each usage in a list of usages is compared as a form of its own, under its entry's `form` index.
12
14
  - An ellipsis (`...` or `…`) in a list stands for members it does not show and claims nothing about them: `(...)` and `(tasks, data, dataset, ...)` leave out no parameter.
13
15
  - A list of bare names, with no type, default or `?` (`connect(host, port)`, `({ themes, defaultTheme })`), names parameters without claiming the list is complete: an optional member it leaves out is no difference, a required one is a mismatch.
14
16