bmad-module-skill-forge 6.0.0 → 8.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/README.md +1 -1
- package/docs/_data/pinned.yaml +9 -10
- package/docs/campaign.md +2 -2
- package/docs/skill-model.md +3 -3
- package/docs/verifying-a-skill.md +17 -18
- package/docs/workflows.md +10 -10
- package/package.json +1 -1
- package/src/shared/references/description-guard-protocol.md +12 -10
- package/src/shared/references/output-contract-schema.md +1 -1
- package/src/shared/scripts/schemas/skf-analyze-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-audit-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-campaign-result-envelope.v1.json +38 -0
- package/src/shared/scripts/schemas/skf-create-skill-result-envelope.v1.json +5 -0
- package/src/shared/scripts/schemas/skf-drop-skill-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-export-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-quick-skill-result-envelope.v1.json +8 -3
- package/src/shared/scripts/schemas/skf-refine-architecture-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-stack-result-envelope.v1.json +6 -1
- package/src/shared/scripts/schemas/skf-test-result-envelope.v1.json +1 -1
- package/src/shared/scripts/schemas/skf-update-result-envelope.v1.json +7 -2
- package/src/shared/scripts/schemas/skf-verify-stack-result-envelope.v1.json +1 -1
- package/src/shared/scripts/skf-parse-gaps.py +4 -1
- package/src/shared/scripts/skf-quick-batch.py +24 -7
- package/src/shared/scripts/skf-verify-provenance-completeness.py +809 -85
- package/src/skf-analyze-source/customize.toml +3 -2
- package/src/skf-analyze-source/references/auto-docs-only.md +2 -2
- package/src/skf-analyze-source/references/generate-briefs.md +5 -3
- package/src/skf-analyze-source/references/headless-contract.md +2 -2
- package/src/skf-analyze-source/references/step-auto-scope.md +2 -2
- package/src/skf-audit-skill/customize.toml +4 -3
- package/src/skf-audit-skill/references/report.md +1 -1
- package/src/skf-campaign/assets/campaign-state-schema.json +58 -0
- package/src/skf-campaign/references/campaign-contracts.md +1 -1
- package/src/skf-campaign/references/step-06-batch.md +2 -2
- package/src/skf-campaign/references/step-07-capstone.md +3 -3
- package/src/skf-campaign/references/step-08-verify.md +3 -3
- package/src/skf-campaign/references/step-09-refine.md +3 -3
- package/src/skf-campaign/references/step-11-maintenance.md +1 -1
- package/src/skf-campaign/scripts/campaign-render-batch.py +43 -10
- package/src/skf-campaign/scripts/campaign-report.py +60 -4
- package/src/skf-campaign/scripts/campaign-state.py +82 -17
- package/src/skf-campaign/templates/campaign-report-template.md +4 -0
- package/src/skf-create-skill/SKILL.md +2 -2
- package/src/skf-create-skill/assets/skill-sections.md +2 -2
- package/src/skf-create-skill/customize.toml +4 -4
- package/src/skf-create-skill/references/batch-mode.md +3 -1
- package/src/skf-create-skill/references/report.md +9 -4
- package/src/skf-create-skill/references/validate.md +49 -29
- package/src/skf-create-stack-skill/references/invocation-contract.md +1 -1
- package/src/skf-create-stack-skill/references/report.md +3 -3
- package/src/skf-create-stack-skill/references/validate.md +21 -8
- package/src/skf-drop-skill/references/invocation-contract.md +1 -1
- package/src/skf-export-skill/references/summary.md +1 -1
- package/src/skf-quick-skill/SKILL.md +1 -1
- package/src/skf-quick-skill/references/batch-mode.md +30 -2
- package/src/skf-quick-skill/references/compile.md +2 -0
- package/src/skf-quick-skill/references/finalize.md +3 -2
- package/src/skf-quick-skill/references/halt-contract.md +3 -3
- package/src/skf-quick-skill/references/quick-extract.md +2 -1
- package/src/skf-quick-skill/references/write-and-validate.md +27 -11
- package/src/skf-refine-architecture/customize.toml +1 -1
- package/src/skf-refine-architecture/references/report.md +2 -2
- package/src/skf-rename-skill/references/report.md +1 -1
- package/src/skf-test-skill/references/coverage-check-tiers.md +3 -3
- package/src/skf-test-skill/references/coverage-check.md +4 -4
- package/src/skf-test-skill/references/inventory-contract.md +1 -1
- package/src/skf-test-skill/references/invocation-contract.md +1 -1
- package/src/skf-test-skill/references/report.md +1 -1
- package/src/skf-test-skill/references/scoring-rules.md +2 -2
- package/src/skf-test-skill/references/signature-comparison-rules.md +7 -2
- package/src/skf-test-skill/scripts/gap-ledger.py +196 -46
- package/src/skf-test-skill/scripts/load-coverage-inputs.py +253 -17
- package/src/skf-test-skill/scripts/score-signatures.py +9 -13
- package/src/skf-test-skill/scripts/validate-inventory.py +91 -12
- package/src/skf-update-skill/customize.toml +1 -1
- package/src/skf-update-skill/references/gap-driven.md +1 -1
- package/src/skf-update-skill/references/invocation-contract.md +1 -1
- package/src/skf-update-skill/references/merge.md +1 -1
- package/src/skf-update-skill/references/report.md +27 -9
- package/src/skf-update-skill/references/write.md +24 -12
- package/src/skf-verify-stack/references/report.md +1 -1
|
@@ -80,8 +80,22 @@ Operations:
|
|
|
80
80
|
summary read from the sub-skill's result envelope line
|
|
81
81
|
(a file or `-`; the line's SKF_*_RESULT_JSON: prefix is
|
|
82
82
|
optional). A success envelope is recorded; an error
|
|
83
|
-
envelope, or --no-capstone and the like
|
|
84
|
-
|
|
83
|
+
envelope, or --no-capstone and the like (a stage
|
|
84
|
+
skipped on purpose), records null. A line that does
|
|
85
|
+
not parse, is not a JSON object, or has no string
|
|
86
|
+
`status` or integer `exit_code` records null with halt
|
|
87
|
+
reason result-unreadable in its not_recorded row (an
|
|
88
|
+
empty line, or a file it cannot read or decode, is
|
|
89
|
+
still exit 2). --no-result <field> (repeatable) records
|
|
90
|
+
a sub-skill that printed no result line: null, with
|
|
91
|
+
halt reason no-result in its not_recorded row; it does
|
|
92
|
+
not go with that field's --<field> or --no-<field>.
|
|
93
|
+
Each field it writes also sets
|
|
94
|
+
campaign.stage_outcomes.<field>: `completed` with the
|
|
95
|
+
envelope's exit_code, `skipped` for --no-<field>, or
|
|
96
|
+
`failed` with the not_recorded row's halt_reason and
|
|
97
|
+
exit_code. A recorded verification also sets
|
|
98
|
+
capstone.verified.
|
|
85
99
|
recover Copy a .bak that validates over the primary (step-resume).
|
|
86
100
|
archive Move the state, its .bak and the brief, each one that
|
|
87
101
|
exists, into archive/<campaign name>-<UTC stamp>/ beside
|
|
@@ -130,7 +144,8 @@ CLI:
|
|
|
130
144
|
--entry <text> [--entry <text> ...] [--stage <N>]
|
|
131
145
|
uv run campaign-state.py set-campaign --state-file <p> [--architecture-doc-path <p>] \\
|
|
132
146
|
[--capstone <p|-> | --no-capstone] [--verification <p|-> | --no-verification] \\
|
|
133
|
-
[--refinement <p|-> | --no-refinement]
|
|
147
|
+
[--refinement <p|-> | --no-refinement] \\
|
|
148
|
+
[--no-result capstone|verification|refinement ...] [--stage <N>]
|
|
134
149
|
uv run campaign-state.py recover --state-file <p>
|
|
135
150
|
uv run campaign-state.py archive --state-file <p> --brief-file <p>
|
|
136
151
|
uv run campaign-state.py log --log-file <p> --type <decision|auto|event> --text <entry>
|
|
@@ -144,7 +159,9 @@ Output (one JSON object on stdout):
|
|
|
144
159
|
"last_updated", "current_stage", "changed": [...]} plus the
|
|
145
160
|
operation's own keys (init: "quality_gate", "brief_file", null
|
|
146
161
|
without --brief-out; apply-batch: "activated", "skipped",
|
|
147
|
-
"completed", "failed"; set-campaign: "recorded",
|
|
162
|
+
"completed", "failed"; set-campaign: "recorded", and
|
|
163
|
+
"not_recorded", one {"field", "status", "halt_reason",
|
|
164
|
+
"exit_code"} row for each field it set to null)
|
|
148
165
|
recover: {"op", "state_file", "backup_file", "recovered", "last_updated",
|
|
149
166
|
"current_stage"}
|
|
150
167
|
archive: {"op", "archive_dir", "moved": [...]}
|
|
@@ -161,8 +178,8 @@ Exit codes (each is the campaign's HALT code for the same class):
|
|
|
161
178
|
2 invalid input: a missing or unreadable input file, an unknown skill,
|
|
162
179
|
output from another script that holds an error, a gate the gate script
|
|
163
180
|
rejects, a state that init would overwrite, a brief template that is
|
|
164
|
-
no YAML mapping, export flags that do not go
|
|
165
|
-
folder that exists
|
|
181
|
+
no YAML mapping, export flags or set-campaign flags that do not go
|
|
182
|
+
together, or an archive folder that exists
|
|
166
183
|
3 invalid state: the state is missing, malformed or fails the schema, or
|
|
167
184
|
the change would make it fail (nothing written)
|
|
168
185
|
4 apply-plan: a plan the stages cannot follow (circular-deps)
|
|
@@ -417,8 +434,16 @@ def read_json(source: str, label: str) -> dict:
|
|
|
417
434
|
return data
|
|
418
435
|
|
|
419
436
|
|
|
420
|
-
def read_envelope(source: str, label: str) -> dict:
|
|
421
|
-
"""A sub-skill's result envelope: the last SKF_*_RESULT_JSON line, or bare JSON.
|
|
437
|
+
def read_envelope(source: str, label: str) -> dict | None:
|
|
438
|
+
"""A sub-skill's result envelope: the last SKF_*_RESULT_JSON line, or bare JSON.
|
|
439
|
+
|
|
440
|
+
None when that payload does not parse or is not a JSON object, which
|
|
441
|
+
set-campaign records as result-unreadable. json.loads raises ValueError
|
|
442
|
+
on a payload it cannot parse (JSONDecodeError is one, an integer of more
|
|
443
|
+
than 4,300 digits another) and RecursionError on nesting too deep for
|
|
444
|
+
it; both read as a payload that does not parse. An empty payload, or
|
|
445
|
+
input it cannot read or decode, is still exit 2.
|
|
446
|
+
"""
|
|
422
447
|
text = read_text(source, label)
|
|
423
448
|
lines = [line for line in text.splitlines() if "_RESULT_JSON:" in line and "SKF_" in line]
|
|
424
449
|
payload = lines[-1].split("_RESULT_JSON:", 1)[1].strip() if lines else text.strip()
|
|
@@ -426,11 +451,9 @@ def read_envelope(source: str, label: str) -> dict:
|
|
|
426
451
|
raise StateError(EXIT_INPUT, "input-unreadable", f"{label} holds no result envelope")
|
|
427
452
|
try:
|
|
428
453
|
data = json.loads(payload)
|
|
429
|
-
except
|
|
430
|
-
|
|
431
|
-
if
|
|
432
|
-
raise StateError(EXIT_INPUT, "input-unreadable", f"{label} is not a JSON object")
|
|
433
|
-
return data
|
|
454
|
+
except (ValueError, RecursionError):
|
|
455
|
+
return None
|
|
456
|
+
return data if isinstance(data, dict) else None
|
|
434
457
|
|
|
435
458
|
|
|
436
459
|
def _skills_by_name(state: dict) -> dict[str, dict]:
|
|
@@ -786,7 +809,31 @@ def _refinement(envelope: dict, stamp: str) -> dict | None:
|
|
|
786
809
|
SUMMARIES = (("capstone", _capstone), ("verification", _verification), ("refinement", _refinement))
|
|
787
810
|
|
|
788
811
|
|
|
812
|
+
def _readable(envelope: dict | None) -> bool:
|
|
813
|
+
"""An envelope set-campaign can read: an object with a string `status` and an integer `exit_code`.
|
|
814
|
+
|
|
815
|
+
The SS, VS and RA envelope schemas all require both, so a line missing
|
|
816
|
+
either is recorded result-unreadable, as step-10 records an Export line.
|
|
817
|
+
"""
|
|
818
|
+
if not isinstance(envelope, dict):
|
|
819
|
+
return False
|
|
820
|
+
exit_code = envelope.get("exit_code")
|
|
821
|
+
return isinstance(envelope.get("status"), str) and isinstance(exit_code, int) and not isinstance(exit_code, bool)
|
|
822
|
+
|
|
823
|
+
|
|
824
|
+
def _failed_outcome(row: dict) -> dict:
|
|
825
|
+
"""The stage outcome of a field set-campaign did not record, from its not_recorded row."""
|
|
826
|
+
reason = row["halt_reason"]
|
|
827
|
+
return {"outcome": "failed", "halt_reason": reason if isinstance(reason, str) else None,
|
|
828
|
+
"exit_code": row["exit_code"]}
|
|
829
|
+
|
|
830
|
+
|
|
789
831
|
def op_set_campaign(args: argparse.Namespace) -> dict:
|
|
832
|
+
no_result = set(args.no_result or ())
|
|
833
|
+
for field, _build in SUMMARIES:
|
|
834
|
+
if field in no_result and (getattr(args, field) is not None or getattr(args, f"no_{field}")):
|
|
835
|
+
raise StateError(EXIT_INPUT, "input-invalid",
|
|
836
|
+
f"--no-result {field} does not go with --{field} or --no-{field}")
|
|
790
837
|
envelopes = {}
|
|
791
838
|
for field, _build in SUMMARIES:
|
|
792
839
|
source = getattr(args, field)
|
|
@@ -800,21 +847,36 @@ def op_set_campaign(args: argparse.Namespace) -> dict:
|
|
|
800
847
|
campaign["architecture_doc_path"] = args.architecture_doc_path
|
|
801
848
|
changed.append("campaign.architecture_doc_path")
|
|
802
849
|
for field, build in SUMMARIES:
|
|
803
|
-
|
|
850
|
+
# Each branch decides the stage outcome; a failed one takes its row's halt_reason and exit_code.
|
|
851
|
+
if field in envelopes and not _readable(envelopes[field]):
|
|
852
|
+
value = None
|
|
853
|
+
row = {"field": field, "status": None, "halt_reason": "result-unreadable", "exit_code": None}
|
|
854
|
+
not_recorded.append(row)
|
|
855
|
+
outcome = _failed_outcome(row)
|
|
856
|
+
elif field in envelopes:
|
|
804
857
|
envelope = envelopes[field]
|
|
805
858
|
value = build(envelope, stamp)
|
|
806
859
|
if value is None:
|
|
807
|
-
|
|
808
|
-
|
|
809
|
-
|
|
860
|
+
row = {"field": field, "status": envelope.get("status"),
|
|
861
|
+
"halt_reason": envelope.get("halt_reason"), "exit_code": envelope.get("exit_code")}
|
|
862
|
+
not_recorded.append(row)
|
|
863
|
+
outcome = _failed_outcome(row)
|
|
810
864
|
else:
|
|
811
865
|
recorded.append(field)
|
|
866
|
+
outcome = {"outcome": "completed", "halt_reason": None, "exit_code": envelope["exit_code"]}
|
|
812
867
|
elif getattr(args, f"no_{field}"):
|
|
813
868
|
value = None
|
|
814
869
|
not_recorded.append({"field": field, "status": None, "halt_reason": None, "exit_code": None})
|
|
870
|
+
outcome = {"outcome": "skipped", "halt_reason": None, "exit_code": None}
|
|
871
|
+
elif field in no_result:
|
|
872
|
+
value = None
|
|
873
|
+
row = {"field": field, "status": None, "halt_reason": "no-result", "exit_code": None}
|
|
874
|
+
not_recorded.append(row)
|
|
875
|
+
outcome = _failed_outcome(row)
|
|
815
876
|
else:
|
|
816
877
|
continue
|
|
817
878
|
campaign[field] = value
|
|
879
|
+
campaign.setdefault("stage_outcomes", {})[field] = outcome
|
|
818
880
|
changed.append(f"campaign.{field}")
|
|
819
881
|
if field == "verification" and isinstance(campaign.get("capstone"), dict):
|
|
820
882
|
verdict = (value or {}).get("overall_verdict")
|
|
@@ -1054,6 +1116,9 @@ def build_parser() -> argparse.ArgumentParser:
|
|
|
1054
1116
|
group = p.add_mutually_exclusive_group()
|
|
1055
1117
|
group.add_argument(f"--{field}", help=f"the {field} sub-skill's result envelope line, a file or -")
|
|
1056
1118
|
group.add_argument(f"--no-{field}", action="store_true", help=f"record campaign.{field} as null")
|
|
1119
|
+
p.add_argument("--no-result", action="append", choices=[f for f, _ in SUMMARIES], metavar="FIELD",
|
|
1120
|
+
help="record the field as null with halt reason no-result: its sub-skill printed no "
|
|
1121
|
+
"result line (repeat for several; not with that field's --FIELD or --no-FIELD)")
|
|
1057
1122
|
|
|
1058
1123
|
command("recover", "copy a valid .bak over the primary", stage=None)
|
|
1059
1124
|
|
|
@@ -65,8 +65,8 @@ These rules apply to every step in this workflow:
|
|
|
65
65
|
|--------|--------|
|
|
66
66
|
| **Inputs** | brief_path (a skill-brief.yaml path or a skill name) [optional: without it, step 1 offers the briefs in `{forge_data_folder}`, and a headless run halts unless there is one], --batch [optional: the briefs or folders of briefs after it, `{forge_data_folder}` when none is given, each validated before the first compiles] |
|
|
67
67
|
| **Gates** | step 1: Brief Selection (no brief named and several in `{forge_data_folder}`; headless: HARD HALT `brief-missing`, exit 2; one brief: picked and recorded as gate `brief-selection`) | step 3: Tag Choice Gate (several tags match the brief's version; headless: the first in priority order) | step 3 §2a: Authoritative-File Gate [P]/[S]/[U] (an authoritative AI documentation file outside the brief's scope; headless: deferred-headless, one auto-decision per path) | step 3: Review Gate [C]/[R] (with the zero-export warning when a source brief without `doc_urls` extracted no export; [R] with it or the head-cap warning) | step 3c: Zero-Export Gate [C]/[R] (a source brief with no export whose every `doc_urls` fetch failed) | step 3d: Demo-Exclusion Gate [Y], then Registry Gate [Y] (candidate found) or [S] (none found), component-library scope only, each skipped once the brief holds `scope.demo_patterns` or `scope.registry_path`, which step 3d writes when a user confirms or gives the patterns or the registry path |
|
|
68
|
-
| **Outputs** | SKILL.md, context-snippet.md, metadata.json, provenance-map.json, evidence-report.md, extraction-rules.yaml, references/; per brief, the result contract (`create-skill-result-{YYYYMMDD-HHmmss}.json` and its `-latest.json` copy in `{forge_version}`) and one `SKF_CREATE_SKILL_RESULT_JSON` line on stdout, on stderr at a HARD HALT |
|
|
69
|
-
| **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true; each auto-decision reaches the envelope's and the result contract's `headless_decisions`. A caller reads the run's own `SKF_CREATE_SKILL_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and `create-skill-result-latest.json` may then describe an earlier run, so it is never read in its place. |
|
|
68
|
+
| **Outputs** | SKILL.md, context-snippet.md, metadata.json, provenance-map.json, evidence-report.md, extraction-rules.yaml, references/; per brief, the result contract (`create-skill-result-{YYYYMMDD-HHmmss}.json` and its `-latest.json` copy in `{forge_version}`) and one `SKF_CREATE_SKILL_RESULT_JSON` line on stdout, on stderr at a HARD HALT; and under `--batch` the batch summary (`create-skill-batch-{YYYYMMDD-HHmmss}.json` and its `-latest.json` copy in `{skills_output_folder}/_batch/`) |
|
|
69
|
+
| **Headless** | All gates auto-resolve with default action when `{headless_mode}` is true; each auto-decision reaches the envelope's and the result contract's `headless_decisions`. A caller reads the run's own `SKF_CREATE_SKILL_RESULT_JSON` line. A run that printed no envelope line has no result to read, whatever the process exit code: it was cut off, or could not write its result, and `create-skill-result-latest.json` may then describe an earlier run, so it is never read in its place. Under `--batch`, a caller reads the batch's `batch_summary` event and the summary file its `summary_path` names, never `create-skill-batch-latest.json` in its place: that copy holds the last batch summary written, which may be another batch's. A printed `batch_summary` event stays the batch's result even if a halt line follows it. A batch that printed no `batch_summary` event has no batch result, whatever the process exit code: a halt line of its own (phase `batch-mode`) says why it stopped, and with no halt line it was cut off or could not print its halt line. A target it printed a `done` or `fail` event for ended as that event says, and a brief the plan listed as not compiled ended failed with its message; a target with a `start` event and no end event has no result, and any other target with no event did not run in this session. |
|
|
70
70
|
| **Exit codes** | 0 for a finished brief; at a HARD HALT 2 (the brief is missing, malformed or invalid), 3 (no forge config, source, helper or documentation), 4 (a write failed), 5 (a refused or broken package), 6 (the user chose to refine the scope). A `--batch` run exits with the highest code among its briefs, 0 when every brief finished |
|
|
71
71
|
|
|
72
72
|
## On Activation
|
|
@@ -283,7 +283,7 @@ t2_future_count: {N}
|
|
|
283
283
|
- Schema: {pass/fail} (quality score: {score}/100)
|
|
284
284
|
- Frontmatter: {pass/fail}
|
|
285
285
|
- Body: {pass/fail} {body split applied if applicable}
|
|
286
|
-
- Security: {pass/warn/skipped}
|
|
286
|
+
- Security: {pass/warn/skipped} ({the note §5's Security row gives})
|
|
287
287
|
- Description angle brackets: {none | re-sanitized ({count} substitutions) | not checked — no description}
|
|
288
288
|
- Tessl Review: {tessl_summary}
|
|
289
289
|
- Metadata: {pass/fail}
|
|
@@ -300,7 +300,7 @@ t2_future_count: {N}
|
|
|
300
300
|
- Notes: {one-sentence detail or —}
|
|
301
301
|
|
|
302
302
|
## Auto-Fixed Issues
|
|
303
|
-
- {
|
|
303
|
+
- {"skill-check --fix rewrote SKILL.md" when §2 recorded `skill_check_fix_changed: true`, "SKILL.md may have been rewritten (not hashed) by skill-check --fix" when it recorded `skill_check_fix_changed: unchecked`, otherwise "none"}
|
|
304
304
|
|
|
305
305
|
## Remaining Warnings
|
|
306
306
|
- {any warnings from extraction or validation}
|
|
@@ -50,10 +50,10 @@ persistent_facts = [
|
|
|
50
50
|
# run a skill such as test-skill or export-skill. It is called as:
|
|
51
51
|
# <on_complete> --result-path=<{forge_version}/create-skill-result-latest.json>
|
|
52
52
|
# Under --batch it runs once per brief, each time with that brief's result
|
|
53
|
-
# file; a brief that halts, or whose result file
|
|
54
|
-
# call. Useful for a Slack or email notifier,
|
|
55
|
-
# or a git-add wrapper. A failure is reported
|
|
56
|
-
# fails the brief.
|
|
53
|
+
# file; a brief that halts, or whose result file or its -latest copy was
|
|
54
|
+
# not written, gets no call. Useful for a Slack or email notifier,
|
|
55
|
+
# dashboard ingest, a CI script or a git-add wrapper. A failure is reported
|
|
56
|
+
# on its own line and never fails the brief.
|
|
57
57
|
#
|
|
58
58
|
# Empty string = no-op (default).
|
|
59
59
|
|
|
@@ -78,4 +78,6 @@ The helper counts the briefs, sets the batch status and exit code, writes the su
|
|
|
78
78
|
|
|
79
79
|
Then load, read in full and execute `{healthCheckStepFile}`: the health check runs once per batch, here. The batch ends with the event's `exit_code`: 0 when every brief finished, else the highest exit code among the briefs that failed.
|
|
80
80
|
|
|
81
|
-
|
|
81
|
+
**Reading a batch.** Under `--batch`, a caller reads the batch's `batch_summary` event and the summary file its `summary_path` names, never `create-skill-batch-latest.json` in its place: that copy holds the last batch summary written, which may be another batch's. The event comes before the health check, so a caller reads it from the session's output stream (`claude -p --output-format stream-json`) or transcript, not from the final message. A printed `batch_summary` event stays the batch's result even if a halt line follows it. A batch that printed no `batch_summary` event has no batch result, whatever the process exit code: a halt line of its own (phase `batch-mode`) says why it stopped, and with no halt line it was cut off or could not print its halt line. A target it printed a `done` or `fail` event for ended as that event says, and a brief the plan listed as not compiled ended failed with its message; a target with a `start` event and no end event has no result, and any other target with no event did not run in this session. When the event's `summary_path` is null, the per-batch file was not written (its `summary_write_failed` warning says why): the event and `{batch_dir}/batch.jsonl`, which the helper then keeps, are the batch's result, never `create-skill-batch-latest.json`. A `summary_write_failed` warning beside a set `summary_path` means only the `-latest` copy failed: the file it names is still the batch's result. After a cut-off batch, run `--batch` again with the same briefs: while the batch has no summary, it finds the batch folder again and goes on with the brief it was working on (Execution, above). If `summarize` already ran but its event never reached the caller, `start` begins a new batch from the first brief, and that batch's summary is the per-batch file in `{skills_output_folder}/_batch/` whose `input_briefs` match.
|
|
82
|
+
|
|
83
|
+
If `summarize` exits non-zero (a brief is still pending, or the batch file is gone or unreadable), the health check does not run: display "**The batch stopped:** {its `message`}" and **HARD HALT** (exit code 4, `write-failed`, phase `batch-mode`; emit with nothing to stage in, as §1 does, with the reason "The batch stopped: <its message>"). The summary files may be missing then, but each brief's own result contract is in its `{forge_version}`, and `create-skill-batch-latest.json` may be another batch's, so it is never read in its place.
|
|
@@ -107,11 +107,11 @@ Every brief that reaches this step gets its own result contract and `on_complete
|
|
|
107
107
|
grep -c . "{run_dir}/headless-decisions.jsonl" 2>/dev/null; grep -c '"gate":"zero-exports"' "{run_dir}/headless-decisions.jsonl" 2>/dev/null
|
|
108
108
|
```
|
|
109
109
|
|
|
110
|
-
Bind `{auto_decision_count}` ← the first count (0 when the file is missing), the rows of the evidence report's `## Auto-Decisions` table, so a consumer can tell from the result alone whether any gate auto-resolved. Bind `{run_status}` ← `partial` when the second count is above 0 (a headless run went on past zero exports at step 3), else `success`. Take `exports_documented` (`stats.exports_documented`) and `confidence_distribution` from the promoted `metadata.json`. Stage the payload with these values and `{auto_decision_count}` in place of the template's zeros, in both `summary` objects, adding `"warning": "zero-exports"` to both when `{run_status}` is `partial`, and run the emitter:
|
|
110
|
+
Bind `{auto_decision_count}` ← the first count (0 when the file is missing), the rows of the evidence report's `## Auto-Decisions` table, so a consumer can tell from the result alone whether any gate auto-resolved. Bind `{run_status}` ← `partial` when the second count is above 0 (a headless run went on past zero exports at step 3), else `success`. Take `exports_documented` (`stats.exports_documented`) and `confidence_distribution` from the promoted `metadata.json`. Bind `{security_result}` ← the first word of the `- Security:` line under `## Validation Results` in the promoted `{forge_version}/evidence-report.md` (`pass`, `warn` or `skipped`), the security scan's outcome step 6 recorded; when that line is missing or starts with another word, leave `security_result` out of both `summary` objects. Stage the payload with these values and `{auto_decision_count}` in place of the template's zeros, in both `summary` objects, adding `"warning": "zero-exports"` to both when `{run_status}` is `partial`, and run the emitter:
|
|
111
111
|
|
|
112
112
|
```bash
|
|
113
113
|
cat > "{run_dir}/result-context.json" <<'SKF_JSON'
|
|
114
|
-
{"status": "{run_status}", "phase": null, "halt_reason": null, "error": null, "skill_package": "{skill_package}", "outputs": {"skill_md": "{skill_package}/SKILL.md", "context_snippet": "{skill_package}/context-snippet.md", "metadata": "{skill_package}/metadata.json", "references": "{skill_package}/references", "provenance_map": "{forge_version}/provenance-map.json", "evidence_report": "{forge_version}/evidence-report.md", "extraction_rules": "{forge_version}/extraction-rules.yaml"}, "summary": {"skill_name": "{name}", "version": "{version}", "forge_tier": "{tier}", "exports_documented": 0, "confidence_distribution": {"t1": 0, "t1_low": 0, "t2": 0, "t3": 0}, "auto_decision_count": 0}, "result_contract": {"skill": "skf-create-skill", "status": "{run_status}", "outputs": [{"type": "skill", "path": "{skill_package}/SKILL.md"}, {"type": "skill", "path": "{skill_package}/context-snippet.md"}, {"type": "skill", "path": "{skill_package}/metadata.json"}, {"type": "report", "path": "{forge_version}/evidence-report.md"}], "summary": {"skill_name": "{name}", "version": "{version}", "exports_documented": 0, "confidence_distribution": {"t1": 0, "t1_low": 0, "t2": 0, "t3": 0}, "auto_decision_count": 0}}}
|
|
114
|
+
{"status": "{run_status}", "phase": null, "halt_reason": null, "error": null, "skill_package": "{skill_package}", "outputs": {"skill_md": "{skill_package}/SKILL.md", "context_snippet": "{skill_package}/context-snippet.md", "metadata": "{skill_package}/metadata.json", "references": "{skill_package}/references", "provenance_map": "{forge_version}/provenance-map.json", "evidence_report": "{forge_version}/evidence-report.md", "extraction_rules": "{forge_version}/extraction-rules.yaml"}, "summary": {"skill_name": "{name}", "version": "{version}", "forge_tier": "{tier}", "exports_documented": 0, "confidence_distribution": {"t1": 0, "t1_low": 0, "t2": 0, "t3": 0}, "auto_decision_count": 0, "security_result": "{security_result}"}, "result_contract": {"skill": "skf-create-skill", "status": "{run_status}", "outputs": [{"type": "skill", "path": "{skill_package}/SKILL.md"}, {"type": "skill", "path": "{skill_package}/context-snippet.md"}, {"type": "skill", "path": "{skill_package}/metadata.json"}, {"type": "report", "path": "{forge_version}/evidence-report.md"}], "summary": {"skill_name": "{name}", "version": "{version}", "exports_documented": 0, "confidence_distribution": {"t1": 0, "t1_low": 0, "t2": 0, "t3": 0}, "auto_decision_count": 0, "security_result": "{security_result}"}}}
|
|
115
115
|
SKF_JSON
|
|
116
116
|
uv run {emitEnvelopeHelper} emit --workflow skf-create-skill --run-dir "{run_dir}" --result-dir "{forge_version}" < "{run_dir}/result-context.json"
|
|
117
117
|
```
|
|
@@ -126,13 +126,18 @@ The evidence report in `outputs` carries the `## Auto-Decisions` audit table: a
|
|
|
126
126
|
uv run {resolveCustomizationHelper} --skill {skill-root} --project-root {project-root} --key workflow.on_complete
|
|
127
127
|
```
|
|
128
128
|
|
|
129
|
-
When `{onCompleteCommand}` is non-empty and the emitter wrote this brief's result contract (its `emit` exited 0), run it as a shell command from `{project-root}`, for this brief:
|
|
129
|
+
When `{onCompleteCommand}` is non-empty and the emitter wrote this brief's result contract (its `emit` exited 0, the line's `result_path` is not null and its `warnings` hold no `result_file_write_failed` entry naming `create-skill-result-latest.json`), run it as a shell command from `{project-root}`, for this brief:
|
|
130
130
|
|
|
131
131
|
```bash
|
|
132
132
|
{onCompleteCommand} --result-path={forge_version}/create-skill-result-latest.json
|
|
133
133
|
```
|
|
134
134
|
|
|
135
|
-
The hook runs after the contract, so a git-add, registry registration or notifier sees a complete package. A hook failure never fails the brief, since the skill is written and its result contract is final: display "on_complete failed for `{name}`: {its exit code and first stderr line}" and go on. When
|
|
135
|
+
The hook runs after the contract, so a git-add, registry registration or notifier sees a complete package. A hook failure never fails the brief, since the skill is written and its result contract is final: display "on_complete failed for `{name}`: {its exit code and first stderr line}" and go on. When `{onCompleteCommand}` is non-empty but the hook does not run, `--result-path` would name a missing file or an earlier compile's: skip the hook and display the line for its case:
|
|
136
|
+
|
|
137
|
+
- No `emit` succeeded (no path in `{emitEnvelopeProbeOrder}` exists, or `emit` failed twice): "on_complete skipped for `{name}`: no result contract was written ({the reason})".
|
|
138
|
+
- `emit` exited 0, but the line's `result_path` is null (the per-run record was not written) or a `result_file_write_failed` warning names `create-skill-result-latest.json` (the copy alone failed): "on_complete skipped for `{name}`: `create-skill-result-latest.json` was not written".
|
|
139
|
+
|
|
140
|
+
When `{onCompleteCommand}` is empty (bundled default), skip the invocation entirely.
|
|
136
141
|
|
|
137
142
|
**Remove the run folder.** The brief finished, so outside `--batch` delete `{run_dir}` (`rm -rf "{run_dir}"`); under `--batch`, batch-mode.md §3 reads the staged `result-context.json` there and removes the folder itself. A halted brief keeps its folder, with the decisions it recorded and its staged payloads.
|
|
138
143
|
|
|
@@ -15,6 +15,13 @@ atomicWriteProbeOrder:
|
|
|
15
15
|
descriptionGuardProbeOrder:
|
|
16
16
|
- '{project-root}/_bmad/skf/shared/scripts/skf-description-guard.py'
|
|
17
17
|
- '{project-root}/src/shared/scripts/skf-description-guard.py'
|
|
18
|
+
# Resolve `{hashContentHelper}` by probing `{hashContentProbeOrder}` in order
|
|
19
|
+
# (installed SKF module path first, src/ dev-checkout fallback); first existing
|
|
20
|
+
# path wins. Advisory: if neither resolves, §2 hashes nothing and reports that
|
|
21
|
+
# SKILL.md may have been rewritten (not hashed) by skill-check --fix.
|
|
22
|
+
hashContentProbeOrder:
|
|
23
|
+
- '{project-root}/_bmad/skf/shared/scripts/skf-hash-content.py'
|
|
24
|
+
- '{project-root}/src/shared/scripts/skf-hash-content.py'
|
|
18
25
|
# Resolve `{frontmatterValidator}` by probing `{frontmatterValidatorProbeOrder}`
|
|
19
26
|
# in order (installed SKF module path first, src/ dev-checkout fallback); first
|
|
20
27
|
# existing path wins.
|
|
@@ -90,7 +97,7 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
|
|
|
90
97
|
- Validation and auto-fix modify files in the staging directory
|
|
91
98
|
- `<staging-skill-dir>` resolves to `_bmad-output/.skf-stage/{skill-name}/` as created by step 5. Its last folder name must match the skill's frontmatter `name` field exactly — `skill-check`'s `frontmatter.name_matches_directory` rule rejects any suffix.
|
|
92
99
|
- If skill-check unavailable: skip validation, add warning to evidence report
|
|
93
|
-
-
|
|
100
|
+
- For the validation verdict, ignore non-zero exit codes from skill-check if JSON output shows 0 errors; the security scan's row follows its exit code and stderr tail (§5)
|
|
94
101
|
- Tessl Review (§6b) is optional and advisory: it never halts, never asks the user anything and never changes the staged files, and SKF applies none of its suggestions
|
|
95
102
|
- A HARD HALT, once step 3 §2b has bound `{source_tree}`, first runs `uv run {sourceTreeHelper} close --tree "{source_tree}"` from `{project-root}` (resolved from `{sourceTreeProbeOrder}`) and goes on whatever it prints, and emits through `{emitEnvelopeHelper}`, resolved from `{emitEnvelopeProbeOrder}` when it is not bound. After its envelope, under `--batch` it ends only this brief: return to `references/batch-mode.md` §3, even when the halt reads as the end of the run.
|
|
96
103
|
|
|
@@ -104,9 +111,9 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
|
|
|
104
111
|
|
|
105
112
|
The deterministic phases are executed via `{descriptionGuardHelper}`: the calling sections (§2 and §4) invoke the helper at the capture and verify-restore points.
|
|
106
113
|
|
|
107
|
-
**Guard outputs.**
|
|
114
|
+
**Guard outputs.** Each `capture` writes the description it reads from the staged `<staging-skill-dir>/SKILL.md`, just before the tool that may rewrite it, to `{run_dir}/description-guard.json`, and each `verify-restore` reads it there with `--captured-file`, so the description never passes through a shell string. Bind `{guarded_description}` ← that file's `description`. Bind `{guard_restored}` ← `restored` and `{guard_diff_kind}` ← `diff_kind` from each `verify-restore`. When `{guard_restored}` is true, set the in-context `description` to `{guarded_description}` so later sections do not work from the tool-mutated value, and record `description_guard_restored: true` with the tool name and `description_guard_diff_kind: {guard_diff_kind}` in workflow context for the evidence report (§8). A later `verify-restore` that exits 0 with `{guard_restored}` false leaves those records in place.
|
|
108
115
|
|
|
109
|
-
**Empty-snapshot rule.** `verify-restore` refuses an empty or whitespace-only
|
|
116
|
+
**Empty-snapshot rule.** `verify-restore` refuses an empty or whitespace-only description in `{run_dir}/description-guard.json`, and a file it cannot read (exit 1, file untouched). Never re-run it with the empty value: writing it back would blank the field the guard protects. If the description compile §2a sanitized is still in context (an in-context copy of the description, not of the file), re-run `verify-restore` with that value: write `{"description": <that value as a JSON string>}` to `{run_dir}/description-guard.json` with a file write, never `echo` (the value can hold quotes, `$` or backticks), and run the same `verify-restore` line again, once. Otherwise, or when that re-run exits 1 too, record `description_guard_restored: false` and `description_guard_refused: empty-capture` with the tool name; the evidence report (§8) renders that as a fired guard, not as a clean run.
|
|
110
117
|
|
|
111
118
|
**This skill's post-restore re-validation hook:** after `{descriptionGuardHelper}` reports `restored: true`, resolve `{frontmatterValidator}` from `{frontmatterValidatorProbeOrder}` (first existing path wins), run `uv run {frontmatterValidator} <staging-skill-dir>/SKILL.md` and capture `schema_revalidation_result` in context. If the validator exits non-zero OR reports failure for the `description` field, flip the Schema result back to `FAIL` in the evidence report (overriding any prior PASS/WARN from §2), record `description_guard_revalidation: FAIL` with the validator's diagnostic message, and continue — do not halt (step 9 health-check and result contract still need to run so the failure is surfaced through the normal artifact path).
|
|
112
119
|
|
|
@@ -133,32 +140,39 @@ npx skill-check check --help 2>/dev/null | grep -- --no-security-scan
|
|
|
133
140
|
- Else run a second probe — `npx skill-check check --help 2>/dev/null | grep -- --skip-security` — and if it matches, set `{security_scan_flag} = "--skip-security"`.
|
|
134
141
|
- If neither flag exists: set `{security_scan_flag} = ""` (empty) AND set `{skill_check_flag_fallback} = true`. Skip §2 and §4 automated flows entirely — fall through to §3 manual frontmatter validation. Record in evidence-report: `skill_check_flag_probe: neither --no-security-scan nor --skip-security supported by installed skill-check; validation performed manually`.
|
|
135
142
|
|
|
136
|
-
**If a security-scan-disable flag was resolved (probe succeeded):**
|
|
137
|
-
|
|
138
|
-
```bash
|
|
139
|
-
npx skill-check check <staging-skill-dir> --fix --format json {security_scan_flag}
|
|
140
|
-
```
|
|
143
|
+
**If a security-scan-disable flag was resolved (probe succeeded):** run the `skill-check check --fix` call where the Description Guard block below holds it (its Phase 2), never before that block's first `hash` and its `capture`.
|
|
141
144
|
|
|
142
145
|
This performs frontmatter validation, description quality checks, body limit enforcement, local link resolution, file formatting, auto-fix of deterministic issues, and quality scoring (0-100) across five weighted categories.
|
|
143
146
|
|
|
144
|
-
**Parse the JSON output** for: `scores[].score` (0-100
|
|
147
|
+
**Parse the JSON output** of the Phase 2 call below for: `scores[].score` (0-100; match the entry by `relativePath`/`skillId`; falls back to a top-level `qualityScore` on older skill-check builds) and `diagnostics[]` (remaining issues). skill-check's JSON lists no fix: the two `hash` calls below tell whether `--fix` changed SKILL.md.
|
|
145
148
|
|
|
146
|
-
**Description Guard Protocol:** This invocation may modify SKILL.md
|
|
149
|
+
**Description Guard Protocol:** This invocation may modify SKILL.md. Wrap the `skill-check check --fix` call in the four-phase guard defined in §0 by invoking `{descriptionGuardHelper}` at the capture and verify-restore points, and hash SKILL.md with `{hashContentHelper}` (resolve it ← first existing path in `{hashContentProbeOrder}`) before the capture and after the verify-restore:
|
|
147
150
|
|
|
148
151
|
```bash
|
|
149
|
-
#
|
|
150
|
-
uv run {
|
|
151
|
-
# stash the returned `
|
|
152
|
+
# Before the guard: hash SKILL.md
|
|
153
|
+
uv run {hashContentHelper} hash <staging-skill-dir>/SKILL.md
|
|
154
|
+
# stash the returned `content_hash` as `skill_md_hash_before`
|
|
155
|
+
|
|
156
|
+
# Phase 1: capture before the tool call
|
|
157
|
+
uv run {descriptionGuardHelper} capture <staging-skill-dir>/SKILL.md > "{run_dir}/description-guard.json"
|
|
158
|
+
# the snapshot lands in the run folder; read its `description` as `guarded_description`
|
|
152
159
|
|
|
153
|
-
# Phase 2
|
|
160
|
+
# Phase 2: run skill-check
|
|
161
|
+
npx skill-check check <staging-skill-dir> --fix --format json {security_scan_flag}
|
|
154
162
|
|
|
155
|
-
# Phases 3+4
|
|
163
|
+
# Phases 3+4: verify and restore after the tool call
|
|
156
164
|
uv run {descriptionGuardHelper} verify-restore <staging-skill-dir>/SKILL.md \
|
|
157
|
-
--captured-
|
|
165
|
+
--captured-file "{run_dir}/description-guard.json"
|
|
166
|
+
|
|
167
|
+
# After the guard: hash SKILL.md again
|
|
168
|
+
uv run {hashContentHelper} hash <staging-skill-dir>/SKILL.md
|
|
169
|
+
# stash the returned `content_hash` as `skill_md_hash_after`
|
|
158
170
|
```
|
|
159
171
|
|
|
160
172
|
If `restored: true` in the verify-restore output, apply §0's post-restore re-validation hook. Step 7 promotes the staged files as they are on disk, `--fix` changes included.
|
|
161
173
|
|
|
174
|
+
Run the second `hash` call after the last `verify-restore`, a §0 re-run included. When `skill_md_hash_before` and `skill_md_hash_after` differ, record `skill_check_fix_changed: true` for §8. When no `{hashContentProbeOrder}` path exists (skip both `hash` calls) or a `hash` call prints no `content_hash`, record `skill_check_fix_changed: unchecked` whatever the other call printed, so §8 says SKILL.md may have been rewritten.
|
|
175
|
+
|
|
162
176
|
**Note:** `skill-check` may return non-zero exit code even when `summary.errorCount` is 0. Always rely on parsed JSON, not the shell exit code.
|
|
163
177
|
|
|
164
178
|
- **Score ≥ 70:** Record "Schema: PASS (score: {score}/100)" in evidence-report
|
|
@@ -186,15 +200,15 @@ If fails: auto-fix (deterministic), re-validate once, record result. If passes:
|
|
|
186
200
|
|
|
187
201
|
```bash
|
|
188
202
|
# Phase 1: capture before the split
|
|
189
|
-
uv run {descriptionGuardHelper} capture <staging-skill-dir>/SKILL.md
|
|
190
|
-
#
|
|
203
|
+
uv run {descriptionGuardHelper} capture <staging-skill-dir>/SKILL.md > "{run_dir}/description-guard.json"
|
|
204
|
+
# the snapshot lands in the run folder; read its `description` as `guarded_description`
|
|
191
205
|
|
|
192
206
|
# Phase 2: the split
|
|
193
207
|
uv run {shardBodyHelper} <staging-skill-dir>/SKILL.md --budget 400
|
|
194
208
|
|
|
195
209
|
# Phases 3 and 4: verify and restore after the split
|
|
196
210
|
uv run {descriptionGuardHelper} verify-restore <staging-skill-dir>/SKILL.md \
|
|
197
|
-
--captured-
|
|
211
|
+
--captured-file "{run_dir}/description-guard.json"
|
|
198
212
|
```
|
|
199
213
|
|
|
200
214
|
If `restored: true` in the verify-restore output, apply §0's post-restore re-validation hook.
|
|
@@ -214,20 +228,26 @@ Then re-validate: `npx skill-check check <staging-skill-dir> --format json {secu
|
|
|
214
228
|
**If skill-check available:**
|
|
215
229
|
|
|
216
230
|
```bash
|
|
217
|
-
npx skill-check check <staging-skill-dir> --format json
|
|
231
|
+
npx skill-check check <staging-skill-dir> --format json > "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"; echo "skill-check exited $?"
|
|
218
232
|
```
|
|
219
233
|
|
|
220
|
-
(Security scan enabled by default when `--no-security-scan` omitted. The scan uses [Snyk](https://docs.snyk.io/) to check for prompt injection risks, sensitive data exposure, and unsafe tool permissions.)
|
|
234
|
+
(Security scan enabled by default when `--no-security-scan` omitted. The scan uses [Snyk](https://docs.snyk.io/) to check for prompt injection risks, sensitive data exposure, and unsafe tool permissions.) The call keeps stdout, stderr and the exit code apart. Give it the longest timeout your shell tool takes. When no "skill-check exited" line came back, the shell tool stopped the call before skill-check finished (a cold `uvx mcp-scan` install can outlast its timeout).
|
|
221
235
|
|
|
222
|
-
|
|
236
|
+
**Score refresh.** stdout can go on after the JSON (a `Wrote <path>` line when the project config sets a report path), so read only the JSON object at the start of `{run_dir}/skill-check.json`. If its quality score differs from section 2's, update the evidence-report with the newer score. When the file does not start with a JSON object (exit 2 on a config, option or plugin error, or a stopped call), keep section 2's score.
|
|
223
237
|
|
|
224
|
-
**
|
|
238
|
+
**Read the JSON, not the exit code.** skill-check prints its JSON before the security scan runs, then exits 1 when `summary.errorCount` is above 0, when `--strict` or `--fail-on-warning` is passed and `summary.warningCount` is above 0 (this call passes neither), or when the scan exits non-zero, and 2 when no scan runner starts, so a non-zero exit is not a security finding. skill-check 1.2.0 puts no security key in the JSON. The scan exited non-zero, which without `--ci` means it did not finish, when skill-check exits 1 although `summary.errorCount` is 0: stderr is then the scan's last lines (its tail), even when blank. On exit 0, stderr is never a scan tail (npm and Node warnings can be there); when exit 1 has another cause, stderr is a scan tail only when it holds the scanner's own lines, such as its token message or a finding code. A finding code is `[`, a capital letter, three digits and `]` (such as `[E001]`), the codes skill-check's own scan summary counts. Bind `{security_result}` from `{run_dir}/skill-check.stderr` and the exit code, the first that applies:
|
|
225
239
|
|
|
226
|
-
|
|
240
|
+
- `skipped`: §1 found no skill-check (**If skill-check unavailable** below), skill-check exits 2, the call stopped before it exited, or the scan exited non-zero and its tail carries no finding code. Log under the evidence report's Remaining Warnings "security scan skipped: SNYK_TOKEN not configured" when the tail says to set the `SNYK_TOKEN` environment variable (the scanner prints "To use Agent Scan, set the SNYK_TOKEN environment variable."), otherwise "security scan skipped: scanner could not run ({the stderr line that names the cause, or its last line})", or "security scan skipped: scanner exited non-zero with no output" when stderr is blank.
|
|
241
|
+
- `warn`: `security[]` holds findings, or the tail carries finding codes. Record each finding under the evidence report's Remaining Warnings as an advisory warning: security findings do not block artifact generation.
|
|
242
|
+
- `pass`: otherwise. The scan exited 0 and skill-check reported no finding. Without `--ci`, agent-scan also exits 0 when its analysis fails (a 401 from a bad token, a 429, a network error), so this does not show that the scan found nothing (#892).
|
|
227
243
|
|
|
228
|
-
|
|
244
|
+
**Security row.** §8 writes the `- Security:` row as `{security_result}`, then its note in parentheses: for `skipped` the note above less its "security scan skipped: " prefix, such as `skipped (SNYK_TOKEN not configured)`; for `warn` each finding; for `pass` the row reads `pass (scan exited 0; skill-check 1.2.0 does not pass the scan's findings on)`. Step 8 takes the row's first word as `security_result`.
|
|
245
|
+
|
|
246
|
+
**If the tail says to set `SNYK_TOKEN`** (the first `skipped` note):
|
|
247
|
+
|
|
248
|
+
Display: "Security scan requires a Snyk Enterprise API token ([docs](https://docs.snyk.io/snyk-api/authentication-for-api)). Set `SNYK_TOKEN=your-token` in environment or `.env`, then re-run [SF] Setup Forge. Without Enterprise, use `--no-security-scan` to skip. Security scanning is optional and does not block skill compilation."
|
|
229
249
|
|
|
230
|
-
**If skill-check unavailable:**
|
|
250
|
+
**If skill-check unavailable:** bind `{security_result}` ← `skipped`, keep the Security row §1 marked, and log "security scan skipped: skill-check unavailable" under the evidence report's Remaining Warnings.
|
|
231
251
|
|
|
232
252
|
### 6. Description Angle-Bracket Check
|
|
233
253
|
|
|
@@ -346,14 +366,14 @@ Record the result on the evidence report's `Provenance lines` line (§8): `pass`
|
|
|
346
366
|
|
|
347
367
|
### 8. Update Evidence Report
|
|
348
368
|
|
|
349
|
-
Write the validation results into the staged `<staging-skill-dir>/evidence-report.md`, in place of the `[PENDING: populated by step 6]` placeholders compile left, with `python3 {atomicWriteHelper} write --target <staging-skill-dir>/evidence-report.md`: step 7 promotes that file as it is. In the same write, add under `## Remaining Warnings` the notes steps 5a to 5c kept for the evidence report (the doc-sources note, `{temporal_feeder_notice}` when step 5c set it, because the Deep-tier temporal feeder held no files) and, once each, the skipped authoritative-files scan when the inventory's `authoritative_files_scan` records `not_scanned` (a remote source that was never cloned; the `skipped` count of candidates the user skipped is not a warning). When no split ran and the staged `references/` holds no file, the Body line says `Tier-2 kept inline (references/ empty)`, so an audit can tell inline by design from a split that failed.
|
|
369
|
+
Write the validation results into the staged `<staging-skill-dir>/evidence-report.md`, in place of the `[PENDING: populated by step 6]` placeholders compile left and of the line under `## Auto-Fixed Issues`, which compile wrote before §2 ran, with `python3 {atomicWriteHelper} write --target <staging-skill-dir>/evidence-report.md`: step 7 promotes that file as it is. In the same write, add under `## Remaining Warnings` the notes steps 5a to 5c kept for the evidence report (the doc-sources note, `{temporal_feeder_notice}` when step 5c set it, because the Deep-tier temporal feeder held no files) and, once each, the skipped authoritative-files scan when the inventory's `authoritative_files_scan` records `not_scanned` (a remote source that was never cloned; the `skipped` count of candidates the user skipped is not a warning). When no split ran and the staged `references/` holds no file, the Body line says `Tier-2 kept inline (references/ empty)`, so an audit can tell inline by design from a split that failed.
|
|
350
370
|
|
|
351
371
|
```markdown
|
|
352
372
|
## Validation Results
|
|
353
373
|
- Schema: {pass/fail} (quality score: {score}/100)
|
|
354
374
|
- Frontmatter: {pass/fail}
|
|
355
375
|
- Body: {pass/fail} {body split applied if applicable}
|
|
356
|
-
- Security: {pass/warn/skipped}
|
|
376
|
+
- Security: {pass/warn/skipped} ({the note §5's Security row gives})
|
|
357
377
|
- Description angle brackets: {none | re-sanitized ({count} substitutions) | not checked — no description}
|
|
358
378
|
- Tessl Review: {tessl_summary}
|
|
359
379
|
- Metadata: {pass/fail}
|
|
@@ -370,7 +390,7 @@ Write the validation results into the staged `<staging-skill-dir>/evidence-repor
|
|
|
370
390
|
- Notes: {one-sentence detail or —}
|
|
371
391
|
|
|
372
392
|
## Auto-Fixed Issues
|
|
373
|
-
- {
|
|
393
|
+
- {"skill-check --fix rewrote SKILL.md" when §2 recorded `skill_check_fix_changed: true`, "SKILL.md may have been rewritten (not hashed) by skill-check --fix" when it recorded `skill_check_fix_changed: unchecked`, otherwise "none"}
|
|
374
394
|
|
|
375
395
|
## Remaining Warnings
|
|
376
396
|
- {warnings and security results, then each entry of {tessl_warnings} — or "none"}
|
|
@@ -31,4 +31,4 @@ The shared emitter, `skf-emit-result-envelope.py`, builds every `SKF_STACK_RESUL
|
|
|
31
31
|
SKF_STACK_RESULT_JSON: {"status":"success|error","skill_package":"...|null","skill_name":"...","stack_libraries":["..."],"mode":"code|compose","quality_score":null,"exit_code":0,"halt_reason":null,"run_id":"...","result_path":"...|null","headless_decisions":[],"warnings":[]}
|
|
32
32
|
```
|
|
33
33
|
|
|
34
|
-
A halt adds `error`: its `phase`, its message as `reason`, and `path` when it names a file or folder. When the customization resolver could not run, `warnings` holds `[activation/warn] customization_resolver_unavailable: <reason>`, which step 1 records before any other warning: the run used the bundled `customize.toml` alone, without the `{project-root}/_bmad/custom/` overrides.
|
|
34
|
+
A halt adds `error`: its `phase`, its message as `reason`, and `path` when it names a file or folder. A success line adds `security_result`, the outcome of step 8 §9's security scan (`pass`, `warn` or `skipped`); a halt leaves it out. When the customization resolver could not run, `warnings` holds `[activation/warn] customization_resolver_unavailable: <reason>`, which step 1 records before any other warning: the run used the bundled `customize.toml` alone, without the `{project-root}/_bmad/custom/` overrides.
|
|
@@ -53,17 +53,17 @@ The shared emitter writes the result contract (`shared/references/output-contrac
|
|
|
53
53
|
|
|
54
54
|
```json
|
|
55
55
|
{"status": "success", "skill_package": "{skill_package}", "skill_name": "{stack_name}",
|
|
56
|
-
"stack_libraries": ["<library>"], "mode": "code|compose", "quality_score": null, "halt_reason": null,
|
|
56
|
+
"stack_libraries": ["<library>"], "mode": "code|compose", "quality_score": null, "security_result": "pass|warn|skipped", "halt_reason": null,
|
|
57
57
|
"result_contract": {"skill": "skf-create-stack-skill", "status": "success",
|
|
58
58
|
"outputs": [{"type": "skill", "path": "{skill_package}/SKILL.md"},
|
|
59
59
|
{"type": "skill", "path": "{skill_package}/context-snippet.md"},
|
|
60
60
|
{"type": "skill", "path": "{skill_package}/metadata.json"},
|
|
61
61
|
{"type": "report", "path": "{forge_version}/evidence-report.md"}],
|
|
62
62
|
"summary": {"lib_count": 0, "integration_count": 0, "forge_tier": "<tier>", "confidence_tier": "<tier>",
|
|
63
|
-
"confidence_distribution": {}, "quality_score": null}}}
|
|
63
|
+
"confidence_distribution": {}, "quality_score": null, "security_result": "pass|warn|skipped"}}}
|
|
64
64
|
```
|
|
65
65
|
|
|
66
|
-
`skill_package` is the absolute path to the committed package and `stack_libraries` the committed `metadata.json`'s `libraries`. Both `quality_score` fields are `{quality_score}`, the skill-check score step 8 §3 bound (`null` when skill-check did not run), never a test-skill score. The summary's counts, `confidence_tier` and `confidence_distribution` are the committed `metadata.json`'s `library_count`, `integration_count`, `confidence_tier` and `confidence_distribution`, and `forge_tier` its run tier. Then run:
|
|
66
|
+
`skill_package` is the absolute path to the committed package and `stack_libraries` the committed `metadata.json`'s `libraries`. Both `quality_score` fields are `{quality_score}`, the skill-check score step 8 §3 bound (`null` when skill-check did not run), never a test-skill score. Both `security_result` fields are `{security_result}`, the scan outcome step 8 §9 bound. The summary's counts, `confidence_tier` and `confidence_distribution` are the committed `metadata.json`'s `library_count`, `integration_count`, `confidence_tier` and `confidence_distribution`, and `forge_tier` its run tier. Then run:
|
|
67
67
|
|
|
68
68
|
```bash
|
|
69
69
|
uv run {emitEnvelopeHelper} emit --workflow skf-create-stack-skill --run-dir "{run_dir}" --result-dir "{forge_version}" < "{run_dir}/result-context.json"
|
|
@@ -3,6 +3,9 @@ nextStepFile: 'report.md'
|
|
|
3
3
|
outputValidatorProbeOrder:
|
|
4
4
|
- '{project-root}/_bmad/skf/shared/scripts/skf-validate-output.py'
|
|
5
5
|
- '{project-root}/src/shared/scripts/skf-validate-output.py'
|
|
6
|
+
hashContentProbeOrder:
|
|
7
|
+
- '{project-root}/_bmad/skf/shared/scripts/skf-hash-content.py'
|
|
8
|
+
- '{project-root}/src/shared/scripts/skf-hash-content.py'
|
|
6
9
|
---
|
|
7
10
|
|
|
8
11
|
<!-- Config: communicate in {communication_language}. Artifact text in {document_output_language}. -->
|
|
@@ -70,17 +73,17 @@ timeout 10s npx --no-install skill-check -h
|
|
|
70
73
|
|
|
71
74
|
### 3. Validate SKILL.md via skill-check (if available)
|
|
72
75
|
|
|
73
|
-
**If available**, run: `npx skill-check check <skill-dir> --fix --format json --no-security-scan`
|
|
76
|
+
**If available**, resolve `{hashContentHelper}` ← first existing path in `{hashContentProbeOrder}`, then run in order: `uv run {hashContentHelper} hash <skill-dir>/SKILL.md`, `npx skill-check check <skill-dir> --fix --format json --no-security-scan`, then `uv run {hashContentHelper} hash <skill-dir>/SKILL.md` again, keeping the `content_hash` each `hash` call prints.
|
|
74
77
|
|
|
75
|
-
This validates frontmatter, description, body limits, links
|
|
78
|
+
This validates frontmatter, description, body limits, links and formatting, and auto-fixes deterministic issues. Parse JSON for `scores[].score` (match the entry by `relativePath`/`skillId`; falls back to a top-level `qualityScore` on older skill-check builds) and `diagnostics[]`. skill-check's JSON lists no fix: SKILL.md changed across the `--fix` call when the two `content_hash` values differ, and is taken as changed, not hashed, when no `{hashContentProbeOrder}` path exists (run the skill-check call alone) or a `hash` call prints no `content_hash`.
|
|
76
79
|
|
|
77
80
|
Bind `{quality_score}` ← that score, from the latest skill-check run (§3 runs it again after a split); `{quality_score}` is null when skill-check is unavailable. It is the skill-check score, not a test-skill score, and step 9 puts it in the result envelope and the result contract.
|
|
78
81
|
|
|
79
|
-
**Post-fix provenance drift guard (S15):**
|
|
82
|
+
**Post-fix provenance drift guard (S15):** When SKILL.md changed across the `--fix` call, `skill-check --fix` modified it after step 7 wrote it, so the `metadata.json` hashes and provenance recorded against the earlier body may now be stale. Emit one **WARNING** finding, `"skill-check --fix modified SKILL.md after step 7 wrote it: metadata.json hashes and provenance may be out of date"` (`"SKILL.md may have been rewritten (not hashed) by skill-check --fix after step 7 wrote it: metadata.json hashes and provenance may be out of date"` when SKILL.md was not hashed), rather than silently accepting the fix, so the drift is surfaced. If the caller wants authoritative metadata, they should re-run the workflow.
|
|
80
83
|
|
|
81
84
|
**If `body.max_lines` reported**, prefer selective split: extract only the largest Tier 2 section(s) to `references/`, keeping Tier 1 content inline (inline passive context achieves 100% task accuracy vs 79% for on-demand retrieval). For a stack capstone the canonical split is the catalog (`Library Reference Index` + `Per-Library Summaries`) → `references/stack-catalog.md`, leaving an inline pointer (see `{stackSkillTemplatePath}` "Sizing Guidance"). The moved catalog's links resolve from `references/`, so rewrite each `[ref](references/{name}.md)` in it to `[ref]({name}.md)`. This is the **intended** large-stack layout, not a violation: §4 below accepts the pointer form, so clearing the skill-check body ERROR this way does not also trip the structure check. Fall back to `npx skill-check split-body <skill-dir> --write` if not feasible. After the split, verify that any in-SKILL.md anchor links (e.g. to the catalog/pointer or other moved sections) still resolve, and that every link inside a moved section resolves from its new file.
|
|
82
85
|
|
|
83
|
-
**Re-validate a changed package.** When `
|
|
86
|
+
**Re-validate a changed package.** When SKILL.md changed across the `--fix` call (not hashed included) or a split ran, run the §1 `{outputValidator}` command again (after a split, skill-check too), so §4 to §8 record the package as it now stands.
|
|
84
87
|
|
|
85
88
|
**If unavailable**, do not hand-walk the frontmatter: use `validation.skill_md.frontmatter` from the §1 output-validator run, which checks delimiters, `name` format + directory match (`{stack_name}`), `description` presence/length, and unknown fields against the agentskills.io allow-set. Record each reported issue at its severity as a **WARNING** finding. (If the output validator was *also* unavailable in §1, fall back to the manual checklist: `---` delimiters; `name` lowercase-alphanumeric-plus-hyphens 1-64 chars matching `{stack_name}`; `description` present and 1-1024 chars; only `name`/`description`/`license`/`compatibility`/`metadata`/`allowed-tools` permitted.) Invalid frontmatter will fail `npx skills add` and `npx skill-check check`.
|
|
86
89
|
|
|
@@ -120,15 +123,25 @@ Record format violations as **WARNING** findings.
|
|
|
120
123
|
|
|
121
124
|
### 9. Security Scan (if skill-check available)
|
|
122
125
|
|
|
123
|
-
Run: `npx skill-check check <skill-dir> --format json` (security scan enabled by default).
|
|
126
|
+
Run: `npx skill-check check <skill-dir> --format json > "{run_dir}/skill-check.json" 2> "{run_dir}/skill-check.stderr"; echo "skill-check exited $?"` (security scan enabled by default), so stdout, stderr and the exit code are read apart.
|
|
124
127
|
|
|
125
|
-
|
|
128
|
+
**If `{run_dir}/skill-check.json` does not start with a JSON object**, skill-check stopped before it validated (exit 2 on a config, option or plugin error): record no finding, only the §10 note "security scan skipped: skill-check failed ({the stderr line that names the cause, or its last line})". Otherwise stdout can go on after the JSON (a `Wrote <path>` line when the project config sets a report path), so read only the JSON object at its start.
|
|
126
129
|
|
|
127
|
-
**
|
|
130
|
+
**Read the JSON, not the exit code.** skill-check exits 1 when `summary.errorCount` is above 0, when `--strict` or `--fail-on-warning` is passed and `summary.warningCount` is above 0 (this call passes neither), or when the scan exits non-zero, and 2 when no scan runner starts, so a non-zero exit is not a security finding. skill-check 1.2.0 puts no security key in the JSON. The scan exited non-zero, which without `--ci` means it did not finish, when skill-check exits 1 although `summary.errorCount` is 0: stderr is then the scan's last lines (its tail), even when blank. On exit 0, stderr is never a scan tail (npm and Node warnings can be there); when exit 1 has another cause, stderr is a scan tail only when it holds the scanner's own lines, such as its token message or a finding code. A finding code is `[`, a capital letter, three digits and `]` (such as `[E001]`), the codes skill-check's own scan summary counts. Read the scan from stderr and the exit code, the first that applies:
|
|
131
|
+
|
|
132
|
+
- **Could not run:** skill-check exits 2, the call stopped before it exited, or the scan exited non-zero and its tail carries no finding code. Record no finding, only a note in the §10 validation results: "security scan skipped: SNYK_TOKEN not configured" when the tail says to set the `SNYK_TOKEN` environment variable (the scanner prints "To use Agent Scan, set the SNYK_TOKEN environment variable."), otherwise "security scan skipped: scanner could not run ({the stderr line that names the cause, or its last line})", or "security scan skipped: scanner exited non-zero with no output" when stderr is blank.
|
|
133
|
+
- **Findings:** `security[]` holds findings, or the tail carries finding codes.
|
|
134
|
+
- **Neither:** record the §10 note "security scan exited 0; skill-check 1.2.0 does not pass the scan's findings on". Without `--ci`, agent-scan also exits 0 when its analysis fails (a 401 from a bad token, a 429, a network error), so this does not show that the scan found nothing (#892).
|
|
135
|
+
|
|
136
|
+
Record security findings as advisory **WARNING** findings: they do not block the report.
|
|
137
|
+
|
|
138
|
+
**If unavailable:** Skip, with the note "security scan skipped: skill-check unavailable" in the §10 validation results.
|
|
139
|
+
|
|
140
|
+
Bind `{security_result}` ← `skipped` for **Could not run**, a run whose `{run_dir}/skill-check.json` does not start with a JSON object, or an unavailable skill-check; `warn` for **Findings**; `pass` for **Neither**. Step 9 puts it in the result envelope and the result contract.
|
|
128
141
|
|
|
129
142
|
### 10. Display Validation Results
|
|
130
143
|
|
|
131
|
-
Report the validation outcome. If all checks passed, state so and name what was verified: file presence (`{count}/{count}`), SKILL.md structure, metadata.json fields, the `{lib_count}` library + `{pair_count}` integration reference files, and complete confidence-tier coverage. If there were findings, report the `{warning_count}` finding(s)
|
|
144
|
+
Report the validation outcome. If all checks passed, state so and name what was verified: file presence (`{count}/{count}`), SKILL.md structure, metadata.json fields, the `{lib_count}` library + `{pair_count}` integration reference files, and complete confidence-tier coverage. If there were findings, report the `{warning_count}` finding(s), each with severity, description, and file path, plus files present/expected and warning/error counts; when errors include missing files, note this may indicate a write failure in step 07. In both cases, list each §9 security scan note.
|
|
132
145
|
|
|
133
146
|
### 11. Auto-Proceed to Next Step
|
|
134
147
|
|