bmad-module-skill-forge 5.0.0 → 6.0.0

package/src/skf-test-skill/scripts/validate-inventory.py

@@ -8,13 +8,29 @@ Structural validator for the SKF test-skill coverage step (coverage-check.md
 §1a "Schema validation" block). test-skill is a quality gate, so it must not
 trust subagent output blindly: before any downstream step consumes the
 documented inventory, the shape is validated. That validation is pure
-structural checking — strip wrapping fences, parse JSON, assert required keys
+structural checking (strip wrapping fences, parse JSON, assert required keys
 and types, assert each export entry is a dict with a non-empty `name` and a
-`kind` drawn from a fixed enum (and, when present, a `signature_form` of
-`partial` or `full`), and assert each cross-check mismatch carries its
-five fields — with exactly one correct pass/fail per input and no
-interpretation of meaning. It is therefore script work, not prompt work; the
-prompt reads this verdict and owns only the HALT decision.
+`kind` drawn from a fixed enum and, when present, a `signature_form` of
+`partial` or `full`, and assert each cross-check mismatch carries its fields
+and one other side, in order), with exactly one correct pass/fail per input
+and no interpretation of meaning. It is therefore script work, not prompt
+work; the prompt reads this verdict and owns only the HALT decision.
+
+A cross-check mismatch (coverage-check.md §1 compares the SKILL.md body
+with each reference file, and two reference files the body does not settle)
+carries `export`, `reference_file`, `reference_line` and `issue`, and the
+other side of its pair: `skill_md_line` for the SKILL.md body, or
+`other_file` and `other_line` for a second reference file, never both and
+never neither (a JSON null counts as absent). Each file is a
+`references/**/*.md` file of the package, normalized to its path relative
+to the package with forward slashes (an absolute path counts only inside
+--skill-package), and the inventory echoed back carries that path; each line
+is a positive integer. A pair of references is written in one order,
+(`other_file`, `other_line`) sorting before (`reference_file`,
+`reference_line`) in codepoint order, so a rerun gives the gap one title and
+the ledger records it once; one export documented twice in one reference is
+a pair of its two lines. A violation of the other side names both shapes,
+and one of the order names both places.
 
 With --skill-package it also runs the ground-truth spot-check (coverage-check.md
 §1a): it sorts the exports by name (a stable sort), samples the first, the
@@ -27,6 +43,18 @@ violation, and the prompt re-dispatches the subagent with them, as it does
 for a schema violation. No grep runs in the prompt, so an export named
 `$state` or `a.b` is never a shell variable or a regex.
 
+With --floor it also checks the inventory against the floor
+load-coverage-inputs.py floor wrote (#790): the names the skill documents in
+a fixed form. A floor name is checked when one of its origins is SKILL.md
+or a file the inventory's `references` lists (the spot-check's files; a
+listed absolute path inside the --skill-package folder counts as its path
+relative to it); a name only an unlisted reference writes is not. It is
+kept when any inventory entry has that name, whatever its kind: kinds stay
+the reader's. Each checked name not
+kept is a violation naming its first checked origin, and the inventory is
+invalid, as for a fabricated name, so the prompt re-dispatches the subagent
+with the names it left out.
+
 An entry's optional `signature_form` says how the skill presents its
 signature: `partial` only when the skill labels the form partial (a
 simplified form, a key-parameter summary, a `...` that stands in for
@@ -70,7 +98,7 @@ declaration the skill cites.
 
 CLI usage (mirrors reconcile-coverage.py):
   uv run validate-inventory.py --input <response file> [--output <inventory file>]
-      [--skill-package <package folder>]
+      [--skill-package <package folder>] [--floor <floor file>]
   uv run validate-inventory.py '<raw response>'                 # positional
   uv run validate-inventory.py --json-input '<raw response>'    # explicit flag
   echo '<raw response>' | uv run validate-inventory.py --stdin  # piped input
@@ -84,7 +112,8 @@ path, so a quote or an apostrophe in a description never meets a shell.
 JSON) when it is valid, for the steps that read it by path; an invalid
 inventory removes a file left there by an earlier attempt. --skill-package
 names the skill package under test (the folder that holds SKILL.md) and
-adds the spot-check.
+adds the spot-check. --floor names the floor file (UTF-8 JSON) and adds the
+floor check.
 
 Input: the subagent's RAW response text (JSON, optionally wrapped in a markdown
 code fence — a leading line of three backticks with an optional language tag and
@@ -107,15 +136,21 @@ Output (stdout, one object):
       "absent": [<name>, ...],   # the sampled names no file writes
       "files": [<path>, ...],    # the files read, relative to the package
       "unread": [<path>, ...]    # listed references that are missing,
-    }                            # unreadable or outside the package
+    },                           # unreadable or outside the package
+    "floor": {                   # with --floor, once the schema passed
+      "names": <int>,            # the floor's names
+      "checked": <int>,          # those with an origin in SKILL.md or a listed reference
+      "absent": [<name>, ...]    # the checked names no inventory entry lists
+    }
   }
 
 Exit codes:
-  0  inventory valid (and every sampled name found)
-  1  no input provided, the --input file cannot be read, or the
-     --skill-package folder holds no readable SKILL.md (usage error)
-  2  inventory invalid (parse failure, schema violation or a sampled name
-     absent; result JSON emitted)
+  0  inventory valid (every sampled name found, every checked floor name kept)
+  1  no input provided, the --input file cannot be read, the
+     --skill-package folder holds no readable SKILL.md, or the --floor file
+     cannot be read or is not a floor (usage error)
+  2  inventory invalid (parse failure, schema violation, a sampled name
+     absent or a floor name left out; result JSON emitted)
 """
 
 from __future__ import annotations
@@ -154,12 +189,20 @@ SIGNATURE_FORMS = ("partial", "full")
 
 REQUIRED_MISMATCH_FIELDS = (
     "export",
-    "skill_md_line",
     "reference_file",
     "reference_line",
     "issue",
 )
 
+# The other side of a cross-check pair, exactly one of these shapes: the
+# SKILL.md body's line, or a second reference file and its line.
+BODY_SIDE = ("skill_md_line",)
+REFERENCE_SIDE = ("other_file", "other_line")
+OTHER_SIDE_SHAPES = (
+    "`skill_md_line` (the SKILL.md body) or `other_file` and `other_line` "
+    "(a second reference file)"
+)
+
 # Kinds whose documented item also needs its parameters and return type to
 # be complete (the docs-only Export Coverage rule).
 SIGNATURE_KINDS = ("function", "method")
@@ -366,6 +409,77 @@ def spot_check(inventory, skill_package_path):
     }
 
 
+# --- Floor ------------------------------------------------------------------
+
+
+class FloorError(Exception):
+    """The --floor file cannot be read or is not a floor: exit 1."""
+
+
+def _is_origin(origin):
+    return (isinstance(origin, dict) and isinstance(origin.get("file"), str) and bool(origin["file"])
+            and isinstance(origin.get("line"), int) and not isinstance(origin["line"], bool)
+            and isinstance(origin.get("form"), str))
+
+
+def read_floor(path):
+    """The floor load-coverage-inputs.py floor wrote to `path`, its shape checked."""
+    try:
+        data = json.loads(Path(path).read_bytes().decode("utf-8-sig"))
+    except (OSError, UnicodeDecodeError) as exc:
+        raise FloorError(f"cannot read {path}: {exc}") from exc
+    except json.JSONDecodeError as exc:
+        raise FloorError(f"{path} is not JSON: {exc.msg}") from exc
+    names = data.get("names") if isinstance(data, dict) else None
+    if not isinstance(names, list) or not all(
+            isinstance(item, dict) and isinstance(item.get("name"), str) and item["name"]
+            and isinstance(item.get("origins"), list) and all(_is_origin(o) for o in item["origins"])
+            for item in names):
+        raise FloorError(f"{path} is not a floor: `names` must give each name's `name` and `origins` "
+                         "(each a file, a line and a form)")
+    return data
+
+
+def _listed_path(ref, skill_package_path):
+    """A listed reference as a floor origin names it: relative to the package, forward slashes. An absolute
+    path inside the package is made relative to it (it needs the package to be)."""
+    if skill_package_path is not None and os.path.isabs(ref):
+        target, root = os.path.realpath(ref), os.path.realpath(skill_package_path)
+        if _inside(target, root):
+            return Path(os.path.relpath(target, root)).as_posix()
+    return _cited_path(ref)
+
+
+def floor_check(inventory, floor, skill_package_path=None):
+    """The floor names the inventory leaves out: (the `floor` result, its violations).
+
+    A floor name is checked when one of its origins is SKILL.md or a file
+    the inventory's `references` lists (the spot-check's files, an absolute
+    path inside `skill_package_path` matched by its path relative to it),
+    and kept when any inventory entry has that name, whatever its kind. A
+    name left out is a violation naming its first checked origin.
+    """
+    listed = inventory.get("references")
+    files = {"SKILL.md"} | {_listed_path(ref, skill_package_path)
+                            for ref in (listed if isinstance(listed, list) else [])
+                            if isinstance(ref, str) and ref.strip()}
+    kept = {e.get("name") for e in inventory.get("exports") or [] if isinstance(e, dict)}
+    names = floor.get("names") or []
+    checked, absent, violations = 0, [], []
+    for item in names:
+        origins = [o for o in item["origins"] if _cited_path(o["file"]) in files]
+        if not origins:
+            continue
+        checked += 1
+        name = item["name"]
+        if name in kept:
+            continue
+        absent.append(name)
+        where = f"{origins[0]['file']}:{origins[0]['line']} ({origins[0]['form']})"
+        violations.append(f"floor: `{name}` is documented at {where}, but no inventory entry lists it")
+    return {"names": len(names), "checked": checked, "absent": absent}, violations
+
+
 # --- Completeness -------------------------------------------------------------
 
 
@@ -415,13 +529,100 @@ def completeness(exports):
 # --- Validation ---------------------------------------------------------------
 
 
-def validate_inventory(raw, skill_package_path=None):
-    """Structural validation of the subagent inventory response, then the spot-check.
+def _reference_path(value, skill_package_path=None):
+    """A pair's file as the package names it (`_listed_path`: relative, forward slashes, no `./`), or None when
+    it is no `references/**/*.md` file of the package: empty, not a string, `SKILL.md`, another file, or an
+    absolute path outside the package (or with no package to make it relative to)."""
+    if not isinstance(value, str) or not value.strip():
+        return None
+    path = _listed_path(value.strip(), skill_package_path)
+    parts = path.split("/")
+    if len(parts) < 2 or parts[0] != "references" or ".." in parts or not parts[-1].endswith(".md") \
+            or parts[-1] == ".md":
+        return None
+    return path
+
+
+def _is_line(value):
+    """A line number: a positive integer, never a bool or a string."""
+    return isinstance(value, int) and not isinstance(value, bool) and value > 0
+
+
+def mismatch_violations(idx, entry, skill_package_path=None):
+    """The violations of one cross_check_mismatches entry (an object).
+
+    A required field left out; a `reference_file` or `other_file` that is no
+    `references/**/*.md` path of the package; a line that is no positive
+    integer; an other side that is not exactly one shape, `skill_md_line` or
+    both of `other_file` and `other_line` (a JSON null counts as absent); or a
+    pair of references out of order: (`other_file`, `other_line`) must sort
+    before (`reference_file`, `reference_line`) in codepoint order, so the pair
+    has one title. Each path is checked, and left in the entry, as the package
+    names it, relative and with forward slashes.
+    """
+    where = f"cross_check_mismatches[{idx}]"
+    violations = []
+    missing = [f for f in REQUIRED_MISMATCH_FIELDS if f not in entry]
+    if missing:
+        violations.append(f"{where} missing field(s): {', '.join(missing)}")
+    ref = None
+    if "reference_file" in entry:
+        ref = _reference_path(entry["reference_file"], skill_package_path)
+        if ref is None:
+            violations.append(f"{where}: `reference_file` must be a `references/*.md` file of the skill package, "
+                              f"not {json.dumps(entry['reference_file'])}")
+        else:
+            entry["reference_file"] = ref
+    lines_ok = True
+    for field in ("reference_line",) + BODY_SIDE + REFERENCE_SIDE[1:]:
+        given = entry.get(field) is not None or (field == "reference_line" and field in entry)
+        if given and not _is_line(entry[field]):
+            lines_ok = False
+            violations.append(f"{where}: `{field}` must be a line number (a positive integer), "
+                              f"not {json.dumps(entry[field])}")
+    body = [f for f in BODY_SIDE if entry.get(f) is not None]
+    reference = [f for f in REFERENCE_SIDE if entry.get(f) is not None]
+    if body and reference:
+        violations.append(f"{where} gives two other sides: give one, {OTHER_SIDE_SHAPES}")
+        return violations
+    if not body and not reference:
+        violations.append(f"{where} missing its other side: give {OTHER_SIDE_SHAPES}")
+        return violations
+    if body:
+        return violations
+    absent = [f for f in REFERENCE_SIDE if f not in reference]
+    if absent:
+        violations.append(f"{where} missing field(s): {', '.join(absent)}; its other side is "
+                          f"{OTHER_SIDE_SHAPES}")
+        return violations
+    other = _reference_path(entry["other_file"], skill_package_path)
+    if other is None:
+        violations.append(f"{where}: `other_file` must be a `references/*.md` file of the skill package, "
+                          f"not {json.dumps(entry['other_file'])}")
+        return violations
+    entry["other_file"] = other
+    if ref is None or not lines_ok or "reference_line" not in entry:
+        return violations
+    first, last = (other, entry["other_line"]), (ref, entry["reference_line"])
+    if not first < last:
+        relation = "names the same line as" if first == last else "sorts after"
+        violations.append(
+            f"{where}: `other_file` {json.dumps(other)} at `other_line` {first[1]} {relation} "
+            f"`reference_file` {json.dumps(ref)} at `reference_line` {last[1]}: a pair of references gives the "
+            "one that sorts first in codepoint order, by path and then line, as `other_file` and `other_line`, "
+            "and the one that sorts last as `reference_file` and `reference_line`"
+        )
+    return violations
+
+
+def validate_inventory(raw, skill_package_path=None, floor=None):
+    """Structural validation of the subagent inventory response, then the spot-check and the floor check.
 
     `raw` is the raw response text. Returns the result dict documented in the
     module docstring. With `skill_package_path`, an inventory that passed the
-    schema is spot-checked against the package (spot_check). Deterministic:
-    identical input yields an identical verdict.
+    schema is spot-checked against the package (spot_check); with `floor`
+    (read_floor's object), it is checked against the floor (floor_check).
+    Deterministic: identical input yields an identical verdict.
     """
     violations = []
     inner = strip_fences(raw)
@@ -491,25 +692,27 @@ def validate_inventory(raw, skill_package_path=None):
                 f"not {json.dumps(entry['signature_form'])}"
             )
 
-    # Non-empty cross-check mismatch entries must carry all five fields.
+    # Each cross-check mismatch carries its fields and one other side of its
+    # pair, a pair of references in order.
     for idx, entry in enumerate(cross):
         if not isinstance(entry, dict):
             violations.append(f"cross_check_mismatches[{idx}] is not an object")
             continue
-        missing = [f for f in REQUIRED_MISMATCH_FIELDS if f not in entry]
-        if missing:
-            violations.append(
-                f"cross_check_mismatches[{idx}] missing field(s): {', '.join(missing)}"
-            )
+        violations.extend(mismatch_violations(idx, entry, skill_package_path))
 
+    schema_valid = not violations
     checked = None
-    if not violations and skill_package_path is not None:
+    if schema_valid and skill_package_path is not None:
         checked = spot_check(data, skill_package_path)
         for name in checked["absent"]:
             violations.append(
                 f"spot-check: `{name}` is listed as an export, but SKILL.md and the "
                 f"listed reference files never write it"
             )
+    floor_result = None
+    if schema_valid and floor is not None:
+        floor_result, floor_violations = floor_check(data, floor, skill_package_path)
+        violations.extend(floor_violations)
 
     valid = not violations
     result = {
@@ -523,6 +726,8 @@ def validate_inventory(raw, skill_package_path=None):
         result["completeness"] = completeness(exports)
     if checked is not None:
         result["spotCheck"] = checked
+    if floor_result is not None:
+        result["floor"] = floor_result
     return result
 
 
@@ -536,8 +741,9 @@ def _build_parser():
             "Deterministic schema validation of the §1 subagent inventory JSON "
             "(coverage-check.md §1a). Strips wrapping fences, parses, and asserts "
             "the required-keys/types/enum/mismatch-field contract, then with "
-            "--skill-package spot-checks sampled names against the package, returning "
-            "{valid, violations, rejectedCount, exportsCount, inventory, completeness, spotCheck}."
+            "--skill-package spot-checks sampled names against the package and --floor checks it "
+            "keeps the floor's names, returning "
+            "{valid, violations, rejectedCount, exportsCount, inventory, completeness, spotCheck, floor}."
         ),
         formatter_class=argparse.RawDescriptionHelpFormatter,
     )
@@ -573,6 +779,11 @@ def _build_parser():
         metavar="DIR",
         help="Spot-check sampled names against this package's SKILL.md and listed references.",
     )
+    parser.add_argument(
+        "--floor",
+        metavar="PATH",
+        help="Check the inventory keeps the names of this load-coverage-inputs.py floor file.",
+    )
     return parser
 
 
@@ -612,7 +823,12 @@ def main(argv=None):
         return 1
 
     try:
-        result = validate_inventory(raw, args.skill_package)
+        floor = read_floor(args.floor) if args.floor is not None else None
+    except FloorError as exc:
+        print(f"error: --floor: {exc}", file=sys.stderr)
+        return 1
+    try:
+        result = validate_inventory(raw, args.skill_package, floor)
     except PackageError as exc:
         print(f"error: --skill-package: {exc}", file=sys.stderr)
         return 1

package/src/skf-update-skill/SKILL.md

@@ -31,6 +31,7 @@ These rules apply to every step in this workflow:
 - Never hallucinate — every statement must have AST provenance
 - [MANUAL] sections survive regeneration with zero content loss
 - Only load one step file at a time — never preload future steps
+- Run every subagent in the foreground (parallel ones as several calls in one message, never with `run_in_background`) and wait for every long-running command to exit before the next instruction; in a headless run, never end a turn while either still runs, since under `claude -p` a run stops when its turn ends.
 - Always communicate in `{communication_language}`
 - If `{headless_mode}` is true, auto-proceed through confirmation gates with their default action and log each auto-decision
 - Every HALT, ABORT or other exit before step 7 runs the **Halt procedure** of the step file it fires in: one `{runStateHelper}` `halt` call that undoes this run's writes, removes the private source tree, releases the run lock and, headless, prints the halt's line. A halt never falls through to step 6.