bmad-module-skill-forge 8.1.0 → 9.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -54,14 +54,29 @@ the reader's. Each checked name not
54
54
  kept is a violation naming its first checked origin, and the inventory is
55
55
  invalid, as for a fabricated name, so the prompt re-dispatches the subagent
56
56
  with the names it left out. A checked origin that carries a `signature`
57
- (#888: the `params` and `return_type` the skill writes there in a fixed
57
+ (#888: the `params`, or for a usage form such as `new Client(opts)` the
58
+ `usage_signature`, and the `return_type` the skill writes there in a fixed
58
59
  form) is carried when an entry of that name, whatever its kind, reads as it
59
- does: each of `params` and `return_type` the signature pins non-blank gives
60
- the same field_text (whitespace runs as one space, a list's items joined
61
- with `, `, a null field as blank), and a `params` that opens with the shared
62
- text the contract has several names' entries carry, then `; `, carries the
63
- pinned one after it. A field the signature leaves blank, and the entry's
64
- other fields, a `usage_signature` among them, are not read.
60
+ does: each of those fields the signature pins non-blank gives the same
61
+ field_text (whitespace runs as one space, a list's items joined with `, `,
62
+ a null field as blank), and a `params` or `usage_signature` that opens with
63
+ the shared text the contract has several names' entries carry, then `; `,
64
+ carries the pinned one after it. A field the signature leaves blank, and
65
+ the entry's other fields (its `usage_signature` beside a pinned `params`,
66
+ its `params` beside a pinned `usage_signature`), are not read, but for one:
67
+ a signature read from a column under a partial label (`Signature
68
+ (simplified)`, `Props (simplified)`, load-coverage-inputs.py's
69
+ FLOOR_PARTIAL_LABELS) also gives `signature_form` "partial", and an entry
70
+ carries it only with that `signature_form` too. Such an origin is carried
71
+ only at its own place: by the entries of that name whose `at` is that
72
+ origin and that give its form, at least one, each with the label, while
73
+ each entry of that name with no `at` (none given, or a bad one dropped, as
74
+ below) that gives that form gives the label too: score-signatures.py would
75
+ keep an unlabelled twin at the origin, or an unlabelled copy that shows
76
+ no place, over the label. An unlabelled copy whose kept `at` is another
77
+ place (with --skill-package, a line that writes the name) stays valid. A
78
+ dropped label, such an unlabelled twin or copy, and a carrier elsewhere
79
+ or with no `at`, is then a violation, never the warning below.
65
80
  Each checked origin no entry carries is a violation naming the origin and
66
81
  its signature, for a name left out too, and the inventory is invalid: the
67
82
  form is never overwritten, so the re-dispatched subagent copies it as
@@ -76,18 +91,56 @@ is a violation naming the entry. score-signatures.py reads it: an optional
76
91
  member a partial form leaves out is an Info observation, not a wrong
77
92
  signature.
78
93
 
94
+ An entry's `at` (#918) is the place the skill writes its form,
95
+ "<path>:<line>": the file as the package names it, SKILL.md or a file the
96
+ inventory's `references` lists (normalized as a pair's file is, and left in
97
+ the entry that way), and the line the form starts on. Each entry whose
98
+ `params` or `usage_signature` score-signatures.py plan compares
99
+ (compared_form: no `method`, with a `params` or `usage_signature` list or
100
+ non-blank string, or a function's `params` string, `""` included) is asked
101
+ for one. A missing or bad `at` never makes the inventory invalid: a bad one
102
+ is dropped from its entry, so the inventory --output writes carries none,
103
+ and each is listed in `warnings`, naming the entry. Its form is then
104
+ compared as one whose citations name no declaration, as before #918. An
105
+ `at` (on any entry) is bad when it is no "<path>:<line>" with a line from
106
+ 1 or names another file. With --skill-package it is bad too when its file
107
+ cannot be read, when its line is past the end, and when that line does not
108
+ write the entry's name outside its citations (the documented-name match)
109
+ and, for a line inside a fenced block (the fences load-coverage-inputs.py
110
+ floor reads, locate-export-segments.py's), neither does the block. With
111
+ --floor, a checked origin with a `signature` (one with no partial label;
112
+ a labelled one is placed as above) is placed when an entry that
113
+ carries it gives that origin as its `at`; when none does, the `at` of each
114
+ entry that carries it is bad unless it is a checked origin whose signature
115
+ that entry carries, and the origin is a warning of its own when each entry
116
+ that carries it keeps its `at` at another such origin (the citations there
117
+ are not read). A warning says its form is left unbound (UNBOUND) only when
118
+ it is, and at most once for an entry: a method's dropped `at` and an
119
+ origin's own warning do not. score-signatures.py plan reads the citations
120
+ of each form's place (form_citations) to compare it with the declaration
121
+ they name. A form that writes only a `return_type` gives no `at`, so it is
122
+ never bound: plan compares it as it compares a form whose citations name no
123
+ declaration.
124
+
79
125
  Distinct from reconcile-coverage.py (coverage numerator arithmetic) and
80
126
  compute-score.py (weight tables): this script only validates the *shape* of the
81
127
  subagent inventory and, on success, echoes the fence-stripped parsed inventory
82
- back so the prompt consumes it directly instead of re-parsing the raw response.
128
+ back, with the `crossChecked` count added once the spot-check ran, so the
129
+ prompt consumes it directly instead of re-parsing the raw response.
130
+ `crossChecked` (#920) is `{"count": <int>, "names": [<name>, ...]}`: each
131
+ inventory name, once, that two or more of the spot-check's files write
132
+ (cross_checked), the Exports Cross-Checked count coherence-check reports.
133
+ Only the validator writes it: one the response carries is replaced with
134
+ --skill-package and dropped without it.
83
135
 
84
136
  It also reports each documented item's completeness, the docs-only Export
85
137
  Coverage rule: an item (one name and kind) is complete when it has a
86
138
  non-empty `description` and, for a function or a method, `params` (a string
87
- or a list, empty when the function takes none) and a non-empty
88
- `return_type`. A missing description makes an item incomplete; it never
89
- makes the inventory invalid. reconcile-coverage.py's docsOnly branch scores
90
- with the same rule.
139
+ or a list, empty when the function takes none) or a non-blank
140
+ `usage_signature` (#916: the contract records a call there whole; a list
141
+ with a non-blank usage), and a non-empty `return_type`. A missing
142
+ description makes an item incomplete; it never makes the inventory
143
+ invalid. reconcile-coverage.py's docsOnly branch scores with the same rule.
91
144
 
92
145
  Shared helpers. The other coverage scripts load these from this file rather
93
146
  than keep copies: `strip_fences` (score-signatures.py), `completeness`
@@ -109,7 +162,27 @@ files the skill's `[AST:]` and `[SRC:]` citations name on a line of SKILL.md
109
162
  or references/**/*.md that writes a name with the match above, or on the
110
163
  next non-blank line. gap-ledger.py reads it to pick the declaration of a
111
164
  stale homonym, score-signatures.py to plan a signature check at the
112
- declaration the skill cites.
165
+ declaration the skill cites. A form's own citations are shared the same way
166
+ (#918): `form_citations` gives the citations of the form an `at` places, on
167
+ its line; on the next non-blank line, unless that line writes the name (it
168
+ is another place of the name, such as the next table row) or, inside a
169
+ fenced block, is a comment (it heads the next declaration); for a line
170
+ inside a fenced block, on the comment lines just above it in the block;
171
+ and, when the block holds no other place the inventory gives the name, on
172
+ the block's first non-blank line when it is a comment and on the first
173
+ non-blank line after its closing fence unless that line writes the name.
174
+ Each comes with the lines it covers (a citation with no `:L<n>` covers its
175
+ whole file), and `cites_declaration` says whether they name a declaration:
176
+ its file, at a line or in a range that holds a line of its span, from its
177
+ line to its last one (a citation of a function's body names it as one of
178
+ its `def` line does; score-signatures.py ends a span before the next
179
+ declaration the surface records in its file). score-signatures.py plan
180
+ binds a form to the one declaration of its name they name (with
181
+ `parse_at`, `compared_form` and `skill_markdown_by_path`), but, for a
182
+ name declared more than once, never by the citations of a place where the
183
+ inventory gives the name two forms that differ (a row that quotes another
184
+ declaration's form as a contrast): there each form is bound only by its
185
+ other places, if any.
113
186
 
114
187
  CLI usage (mirrors reconcile-coverage.py):
115
188
  uv run validate-inventory.py --input <response file> [--output <inventory file>]
@@ -127,8 +200,8 @@ path, so a quote or an apostrophe in a description never meets a shell.
127
200
  JSON) when it is valid, for the steps that read it by path; an invalid
128
201
  inventory removes a file left there by an earlier attempt. --skill-package
129
202
  names the skill package under test (the folder that holds SKILL.md) and
130
- adds the spot-check. --floor names the floor file (UTF-8 JSON) and adds the
131
- floor check.
203
+ adds the spot-check and the inventory's `crossChecked` count. --floor names
204
+ the floor file (UTF-8 JSON) and adds the floor check.
132
205
 
133
206
  Input: the subagent's RAW response text (JSON, optionally wrapped in a markdown
134
207
  code fence — a leading line of three backticks with an optional language tag and
@@ -138,9 +211,12 @@ Output (stdout, one object):
138
211
  {
139
212
  "valid": <bool>, # true only when parse + every schema check pass
140
213
  "violations": [<str>, ...], # human-readable failures; empty when valid
214
+ "warnings": [<str>, ...], # each missing or bad `at` (dropped), never a failure
141
215
  "rejectedCount": <int>, # exports[] entries with no valid name or kind
142
216
  "exportsCount": <int>, # len(exports) when parsed; 0 otherwise
143
- "inventory": {...} | null, # fence-stripped parsed inventory when valid; null otherwise
217
+ "inventory": {...} | null, # fence-stripped parsed inventory when valid; null otherwise;
218
+ # with --skill-package it carries
219
+ # "crossChecked": {"count": <int>, "names": [<name>, ...]}
144
220
  "completeness": { # present when valid
145
221
  "total": <int>, # documented items, one per name and kind
146
222
  "complete": <int>,
@@ -161,24 +237,30 @@ Output (stdout, one object):
161
237
 
162
238
  Exit codes:
163
239
  0 inventory valid (every sampled name found, every checked floor name
164
- kept and every checked floor signature carried)
240
+ kept and every checked floor signature carried; a missing or bad `at`
241
+ is only a warning, but a labelled floor signature is carried only at
242
+ its own place, and never unlabelled there or with no `at`)
165
243
  1 no input provided, the --input file cannot be read, the
166
244
  --skill-package folder holds no readable SKILL.md, or the --floor file
167
245
  cannot be read or is not a floor, a malformed `signature` among them
168
246
  (usage error)
169
247
  2 inventory invalid (parse failure, schema violation, a sampled name
170
- absent, a floor name left out or a floor signature no entry carries;
248
+ absent, a floor name left out or a floor signature no entry carries,
249
+ or a labelled one no entry at its place carries with its label, or
250
+ that an entry there or with no `at` gives without it;
171
251
  result JSON emitted)
172
252
  """
173
253
 
174
254
  from __future__ import annotations
175
255
 
176
256
  import argparse
257
+ import importlib.util
177
258
  import json
178
259
  import os
179
260
  import posixpath
180
261
  import re
181
262
  import sys
263
+ from collections.abc import Iterable
182
264
  from pathlib import Path
183
265
 
184
266
  # Fixed enum for exports[].kind — the constructs SKF documents across languages
@@ -229,8 +311,15 @@ SIGNATURE_KINDS = ("function", "method")
229
311
  SIGNATURE_FIELDS = ("params", "return_type", "usage_signature")
230
312
 
231
313
  # The fields of a floor origin's `signature` (#888), each a string, and the
232
- # ones the floor check compares with an entry's.
233
- FLOOR_SIGNATURE_FIELDS = ("params", "return_type")
314
+ # ones the floor check compares with an entry's: `return_type` and either
315
+ # `params` (a complete form or a props or parameter cell) or `usage_signature`
316
+ # (a usage form, such as `new Client(opts)`).
317
+ FLOOR_SIGNATURE_FIELDS = ("params", "usage_signature", "return_type")
318
+
319
+ # The label a floor origin's `signature` may also give (#888), its key and its
320
+ # one value: the partial label of the Signature, Props, Params or Parameters
321
+ # column it was read from, which the entry at that origin copies.
322
+ FLOOR_LABEL_FIELD, FLOOR_LABEL = "signature_form", "partial"
234
323
 
235
324
  # Characters that continue an identifier in the languages SKF documents
236
325
  # (`$` for JS/TS names such as `$state`).
@@ -335,18 +424,26 @@ def skill_markdown_files(skill_dir: Path) -> list[Path]:
335
424
  return files
336
425
 
337
426
 
338
- def skill_markdown_lines(skill_dir: Path) -> list[list[str]]:
339
- """The lines of each file skill_markdown_files lists (an unreadable one is
340
- left out), read once for every name _skill_cited_files looks up."""
341
- texts: list[list[str]] = []
427
+ def skill_markdown_by_path(skill_dir: Path) -> dict[str, list[str]]:
428
+ """{the path of each file skill_markdown_files lists, relative to
429
+ `skill_dir` with forward slashes: its lines} (an unreadable one is left
430
+ out), read once for every name or form the citation scan looks up."""
431
+ texts: dict[str, list[str]] = {}
342
432
  for path in skill_markdown_files(skill_dir):
343
433
  try:
344
- texts.append(path.read_bytes().decode("utf-8-sig", errors="replace").split("\n"))
434
+ texts[path.relative_to(skill_dir).as_posix()] = (
435
+ path.read_bytes().decode("utf-8-sig", errors="replace").split("\n"))
345
436
  except OSError:
346
437
  continue
347
438
  return texts
348
439
 
349
440
 
441
+ def skill_markdown_lines(skill_dir: Path) -> list[list[str]]:
442
+ """The lines of each file skill_markdown_files lists (an unreadable one is
443
+ left out), read once for every name _skill_cited_files looks up."""
444
+ return list(skill_markdown_by_path(skill_dir).values())
445
+
446
+
350
447
  def _skill_cited_files(skill_dir: Path, name: str, texts: list[list[str]] | None = None) -> set[str]:
351
448
  """The files the skill's `[AST:]` and `[SRC:]` citations name on a line of
352
449
  SKILL.md or references/**/*.md that writes `name` outside its citations
@@ -365,6 +462,151 @@ def _skill_cited_files(skill_dir: Path, name: str, texts: list[list[str]] | None
365
462
  return cited
366
463
 
367
464
 
465
+ # --- A form's place and its own citations (#918) -------------------------------
466
+
467
+ # An entry's `at`: the place the skill writes its form, "<path>:<line>", the line from 1.
468
+ _AT_RE = re.compile(r"(.+):([1-9][0-9]*)")
469
+ # A source citation as a form's own citation reads it: one with no `:L<n>` part too, which names every declaration
470
+ # in its file. Groups as _SKILL_CITATION_RE's, the line ones absent for no line.
471
+ _FORM_CITATION_RE = re.compile(r"\[(AST|SRC):([^\[\]\n]+?)(?::L(\d+)(?:-(L?)(\d+))?)?\]")
472
+ # A comment's opening marker: `//`, `#`, `/*`, `--`, `<!--` or `;`. Keep identical to COMMENT_MARKER_RE in
473
+ # load-coverage-inputs.py, which reads a fenced block the same way (the floor's `declaration` form).
474
+ _COMMENT_MARKER_RE = re.compile(r"[ \t]*(?://+|#+|/\*+|--|<!--|;+)")
475
+
476
+ _SEGMENTS_MODULE = None
477
+
478
+
479
+ def _segments():
480
+ """locate-export-segments.py beside this script: the fence model the floor reads a skill with."""
481
+ global _SEGMENTS_MODULE
482
+ if _SEGMENTS_MODULE is None:
483
+ sibling = Path(__file__).resolve().parent / "locate-export-segments.py"
484
+ spec = importlib.util.spec_from_file_location("skf_validate_inventory_segments", sibling)
485
+ if spec is None or spec.loader is None or not sibling.is_file():
486
+ raise ImportError(f"locate-export-segments.py not found beside {Path(__file__).name}")
487
+ module = importlib.util.module_from_spec(spec)
488
+ spec.loader.exec_module(module)
489
+ _SEGMENTS_MODULE = module
490
+ return _SEGMENTS_MODULE
491
+
492
+
493
+ def compared_form(entry: dict) -> bool:
494
+ """Whether score-signatures.py plan compares a form the entry's `params`
495
+ or `usage_signature` writes, so the entry carries `at`: an entry that is
496
+ no `method` whose `params` or `usage_signature` is a list or a non-blank
497
+ string, or, for a function, whose `params` is a string (`""`, the form
498
+ of one that takes none). plan also compares a form that writes only a
499
+ `return_type`; that one carries no `at`, so it is never bound."""
500
+ if entry.get("kind") == "method":
501
+ return False
502
+ if entry.get("kind") in SIGNATURE_KINDS and isinstance(entry.get("params"), str):
503
+ return True
504
+ return any(isinstance(entry.get(f), list) or (isinstance(entry.get(f), str) and bool(entry[f].strip()))
505
+ for f in ("params", "usage_signature"))
506
+
507
+
508
+ def parse_at(value: object) -> tuple[str, int] | None:
509
+ """An entry's `at` as (its path, _cited_path, its line), None when it is
510
+ no "<path>:<line>" with a line from 1."""
511
+ m = _AT_RE.fullmatch(value.strip()) if isinstance(value, str) else None
512
+ if m is None or not m.group(1).strip():
513
+ return None
514
+ return _cited_path(m.group(1)), int(m.group(2))
515
+
516
+
517
+ def _fence_holding(lines: list[str], index: int) -> tuple[int, int] | None:
518
+ """(its opening fence, its closing fence) of the fenced block whose body
519
+ holds line `index` (from 0), as the floor reads fences
520
+ (locate-export-segments.py's _fenced_blocks past the frontmatter, an
521
+ unclosed block running to the end); None for a line outside every body,
522
+ a fence included."""
523
+ segments = _segments()
524
+ blocks, _ = segments._fenced_blocks(lines, segments._frontmatter_end(lines))
525
+ return next(((opening, closing) for opening, closing, _ in blocks if opening < index < closing), None)
526
+
527
+
528
+ def _first_filled(lines: list[str], start: int) -> str:
529
+ return next((ln for ln in lines[start:] if ln.strip()), "")
530
+
531
+
532
+ def _writes(pattern, line):
533
+ """Whether `line` writes the name `pattern` matches outside its citations (the documented-name match)."""
534
+ return bool(pattern.search(_ANY_CITATION_RE.sub(" ", line)))
535
+
536
+
537
+ def form_citations(lines: list[str], line: int, name: str,
538
+ others: Iterable[int] = ()) -> set[tuple[str, int | None, int | None]]:
539
+ """The own citations of the form of `name` a skill file writes at `line`
540
+ (from 1, its `at`), from that file's `lines`; `others` are the lines of
541
+ the other places the inventory gives `name` in that file. They are the
542
+ `[AST:]` and `[SRC:]` citations:
543
+
544
+ - on that line;
545
+ - on the next non-blank line, unless that line writes `name` outside
546
+ its citations (another place of the name, such as the next table
547
+ row, whose citations are its own) or, inside a fenced block, is a
548
+ comment (it heads the next declaration);
549
+ - for a line inside a fenced block (_fence_holding), on the comment
550
+ lines just above it in the block, its own heading;
551
+ - and, when no line of `others` is in that block's body (a block that
552
+ holds two places of the name gives neither its block-wide
553
+ citations), on the block's first non-blank line when it is a comment,
554
+ and on the first non-blank line after its closing fence unless that
555
+ line writes `name`.
556
+
557
+ Each is (its file, _cited_path; the first line it covers; the last):
558
+ `:L12` covers line 12, `:L12-34` and `:L12-L34` lines 12 to 34, and a
559
+ citation with no `:L<n>` part gives (its file, None, None), every
560
+ declaration in its file."""
561
+ index = line - 1
562
+ if not 0 <= index < len(lines):
563
+ return set()
564
+ pattern = name_pattern(name)
565
+ fence = _fence_holding(lines, index)
566
+ places = [lines[index]]
567
+ following = _first_filled(lines, index + 1)
568
+ if not _writes(pattern, following) and not (fence is not None and _COMMENT_MARKER_RE.match(following)):
569
+ places.append(following)
570
+ if fence is not None:
571
+ opening, closing = fence
572
+ above = index - 1
573
+ while above > opening and _COMMENT_MARKER_RE.match(lines[above]):
574
+ places.append(lines[above])
575
+ above -= 1
576
+ if not any(opening < other - 1 < closing for other in others if other != line):
577
+ heading = _first_filled(lines[opening + 1:closing], 0)
578
+ if _COMMENT_MARKER_RE.match(heading):
579
+ places.append(heading)
580
+ after = _first_filled(lines, closing + 1)
581
+ if not _writes(pattern, after):
582
+ places.append(after)
583
+ cited: set[tuple[str, int | None, int | None]] = set()
584
+ for text in places:
585
+ for m in _FORM_CITATION_RE.finditer(text):
586
+ if m.group(3) is None:
587
+ cited.add((_cited_path(m.group(2)), None, None))
588
+ else:
589
+ first, last = int(m.group(3)), int(m.group(5) or m.group(3))
590
+ cited.add((_cited_path(m.group(2)), min(first, last), max(first, last)))
591
+ return cited
592
+
593
+
594
+ def cites_declaration(citations: set[tuple[str, int | None, int | None]], file: str, line: object,
595
+ end: int | None = None) -> bool:
596
+ """Whether one of `citations` (form_citations) names the declaration at
597
+ `file` and `line`: it cites that file (both read by _cited_path) with no
598
+ line, or at a line or in a range that holds a line of its span, from
599
+ `line` to `end` (its last line; None for `line` alone). A citation of
600
+ the body names the function as its `def` line does, and a range over
601
+ two declarations names each one it holds a line of."""
602
+ path = _cited_path(file)
603
+ if not _is_line(line):
604
+ return any(cited == path and first is None for cited, first, _ in citations)
605
+ last_line = line if end is None or end < line else end
606
+ return any(cited == path and (first is None or (first <= last_line and line <= last))
607
+ for cited, first, last in citations)
608
+
609
+
368
610
  # --- Spot-check -------------------------------------------------------------
369
611
 
370
612
 
@@ -396,44 +638,80 @@ def _inside(path, root):
396
638
  return path == root or path.startswith(root.rstrip(os.sep) + os.sep)
397
639
 
398
640
 
399
- def spot_check(inventory, skill_package_path):
400
- """Look the sampled export names up in SKILL.md and the inventory's references.
641
+ def _package_texts(inventory, skill_package_path):
642
+ """Read the files the inventory reads, for spot_check and cross_checked.
401
643
 
402
644
  The file set is the package's SKILL.md plus each `references` entry,
403
645
  resolved against the package (a split-body skill documents some exports
404
- only in a reference file). A listed file that is missing, unreadable or
405
- resolves outside the package is not read and is named in `unread`.
646
+ only in a reference file), a listed spelling read once. A listed file
647
+ that is missing, unreadable or resolves outside the package is not read
648
+ and is named in `unread`. Returns `(read, unread)`: `read` holds
649
+ `(spelling, resolved path, text)` for each file read, SKILL.md first.
650
+ Raises PackageError when SKILL.md cannot be read.
406
651
  """
407
652
  root = Path(skill_package_path)
408
653
  real_root = os.path.realpath(root)
654
+ skill_md = root / "SKILL.md"
409
655
  try:
410
- texts = [(root / "SKILL.md").read_bytes().decode("utf-8-sig")]
656
+ read = [("SKILL.md", os.path.realpath(skill_md), skill_md.read_bytes().decode("utf-8-sig"))]
411
657
  except (OSError, UnicodeDecodeError) as exc:
412
- raise PackageError(f"cannot read {(root / 'SKILL.md').as_posix()}: {exc}") from exc
413
- files, unread = ["SKILL.md"], []
658
+ raise PackageError(f"cannot read {skill_md.as_posix()}: {exc}") from exc
659
+ unread = []
414
660
  listed = inventory.get("references")
415
661
  for ref in listed if isinstance(listed, list) else []:
416
- if not isinstance(ref, str) or not ref.strip() or ref in files or ref in unread:
662
+ if not isinstance(ref, str) or not ref.strip() or ref in unread or any(ref == r[0] for r in read):
417
663
  continue
418
664
  target = os.path.realpath(root / ref)
419
665
  try:
420
666
  if not _inside(target, real_root):
421
667
  raise OSError("outside the package")
422
- texts.append(Path(target).read_bytes().decode("utf-8-sig"))
423
- files.append(ref)
668
+ read.append((ref, target, Path(target).read_bytes().decode("utf-8-sig")))
424
669
  except (OSError, UnicodeDecodeError):
425
670
  unread.append(ref)
426
- doc_text = "\n".join(texts)
671
+ return read, unread
672
+
673
+
674
+ def spot_check(inventory, skill_package_path):
675
+ """Look the sampled export names up in SKILL.md and the inventory's references.
676
+
677
+ The files are _package_texts's: `files` names each one read as the
678
+ inventory spells it, `unread` each listed one that is missing,
679
+ unreadable or resolves outside the package.
680
+ """
681
+ read, unread = _package_texts(inventory, skill_package_path)
682
+ doc_text = "\n".join(text for _, _, text in read)
427
683
  sampled = sample_names(inventory.get("exports"))
428
684
  found = set(names_present(sampled, doc_text))
429
685
  return {
430
686
  "sampled": sampled,
431
687
  "absent": [name for name in sampled if name not in found],
432
- "files": files,
688
+ "files": [spelling for spelling, _, _ in read],
433
689
  "unread": unread,
434
690
  }
435
691
 
436
692
 
693
+ def cross_checked(exports, texts):
694
+ """The inventory's names that two or more of the files it reads write (#920).
695
+
696
+ `texts` is _package_texts's `read`, the spot-check's files. A file
697
+ counts once, by its resolved path, so a reference listed under two
698
+ spellings (`references/a.md`, `./references/a.md`), or SKILL.md listed
699
+ again, is one file. A file writes a name when names_present finds it
700
+ outside its citations, so a mention in prose counts, and a name counts
701
+ once however many entries give it (a homonym). Returns `{"count": <int>,
702
+ "names": [<name>, ...]}`, the names sorted; coherence-check reports the
703
+ count as Exports Cross-Checked.
704
+ """
705
+ names = {entry["name"] for entry in exports
706
+ if isinstance(entry, dict) and isinstance(entry.get("name"), str) and entry["name"]}
707
+ by_path = {}
708
+ for _, path, text in texts:
709
+ by_path.setdefault(os.path.normcase(path), text)
710
+ found = [set(names_present(names, _ANY_CITATION_RE.sub(" ", text))) for text in by_path.values()]
711
+ crossed = sorted(name for name in names if sum(name in written for written in found) >= 2)
712
+ return {"count": len(crossed), "names": crossed}
713
+
714
+
437
715
  # --- Signature forms ----------------------------------------------------------
438
716
 
439
717
 
@@ -461,9 +739,16 @@ class FloorError(Exception):
461
739
 
462
740
 
463
741
  def _is_signature(value):
464
- """A floor origin's `signature`: an object of exactly the strings `params` and `return_type`."""
465
- return (isinstance(value, dict) and sorted(value) == sorted(FLOOR_SIGNATURE_FIELDS)
466
- and all(isinstance(value[f], str) for f in FLOOR_SIGNATURE_FIELDS))
742
+ """A floor origin's `signature`: an object of the string `return_type` and exactly one of the string `params` and
743
+ a non-blank string `usage_signature` (a usage form, #888), and optionally `signature_form` "partial" (a column's
744
+ partial label, #888); any other key or value is not one."""
745
+ if not isinstance(value, dict):
746
+ return False
747
+ form = set(value) - {FLOOR_LABEL_FIELD, "return_type"}
748
+ return ("return_type" in value and len(form) == 1 and form <= set(FLOOR_SIGNATURE_FIELDS)
749
+ and all(isinstance(value[f], str) for f in form | {"return_type"})
750
+ and (form != {"usage_signature"} or bool(value["usage_signature"].strip()))
751
+ and value.get(FLOOR_LABEL_FIELD, FLOOR_LABEL) == FLOOR_LABEL)
467
752
 
468
753
 
469
754
  def _is_origin(origin):
@@ -487,8 +772,9 @@ def read_floor(path):
487
772
  and isinstance(item.get("origins"), list) and all(_is_origin(o) for o in item["origins"])
488
773
  for item in names):
489
774
  raise FloorError(f"{path} is not a floor: `names` must give each name's `name` and `origins` "
490
- "(each a file, a line and a form, and an optional `signature` of the strings "
491
- "`params` and `return_type`)")
775
+ "(each a file, a line and a form, and an optional `signature` of the string "
776
+ "`return_type` and either the string `params` or a non-blank string `usage_signature`, "
777
+ "and an optional `signature_form` \"partial\")")
492
778
  return data
493
779
 
494
780
 
@@ -503,18 +789,31 @@ def _listed_path(ref, skill_package_path):
503
789
 
504
790
 
505
791
  def _carries(entry, signature):
506
- """Whether an entry carries a floor origin's signature: each field it pins non-blank has the same field_text
507
- there, a `params` that opens with members the skill states once for several names (the contract's shared
508
- text, then `; `) included."""
792
+ """Whether an entry carries a floor origin's signature: each field the signature gives (`params` or
793
+ `usage_signature`, and `return_type`) and pins non-blank has the same field_text there, a `params` or
794
+ `usage_signature` that opens with members the skill states once for several names (the contract's shared text,
795
+ then `; `) included; and, when it gives the partial label (#888), the entry's `signature_form` is that label
796
+ too."""
797
+ if FLOOR_LABEL_FIELD in signature and entry.get(FLOOR_LABEL_FIELD) != signature[FLOOR_LABEL_FIELD]:
798
+ return False
509
799
  for field in FLOOR_SIGNATURE_FIELDS:
800
+ if field not in signature:
801
+ continue
510
802
  wanted, given = field_text(signature[field]), field_text(entry.get(field))
511
- if wanted and given != wanted and not (field == "params" and given.endswith("; " + wanted)):
803
+ if wanted and given != wanted and not (field != "return_type" and given.endswith("; " + wanted)):
512
804
  return False
513
805
  return True
514
806
 
515
807
 
808
+ def _origin_signature(origin):
809
+ """A floor origin's signature, the fields it gives (#888: `params` or `usage_signature`, and `return_type`) and
810
+ its label only when it gives one."""
811
+ return {f: origin["signature"][f] for f in FLOOR_SIGNATURE_FIELDS + (FLOOR_LABEL_FIELD,)
812
+ if f in origin["signature"]}
813
+
814
+
516
815
  def floor_check(inventory, floor, skill_package_path=None):
517
- """The floor names and signatures the inventory leaves out: (the `floor` result, its violations).
816
+ """The floor names and signatures the inventory leaves out: (the `floor` result, its violations, its warnings).
518
817
 
519
818
  A floor name is checked when one of its origins is SKILL.md or a file
520
819
  the inventory's `references` lists (the spot-check's files, an absolute
@@ -523,19 +822,37 @@ def floor_check(inventory, floor, skill_package_path=None):
523
822
  name left out is a violation naming its first checked origin. Then each
524
823
  checked origin with a `signature` is carried when an entry of that name,
525
824
  whatever its kind, gives the same field_text to each field the
526
- signature pins non-blank (a blank pinned `return_type` leaves the
527
- entry's free; a `params` may open with shared text and `; `, _carries);
528
- one no entry carries is a violation naming the origin and the
529
- signature, for a name left out too.
825
+ signature pins non-blank (its `params` or, for a usage form, its
826
+ `usage_signature`, and its `return_type`; a blank pinned `return_type`
827
+ leaves the entry's free; a `params` or `usage_signature` may open with
828
+ shared text and `; `, _carries); one no entry carries is a violation
829
+ naming the origin and the signature, for a name left out too. A carried
830
+ origin is placed when an entry that carries it gives that origin as its
831
+ `at` (parse_at; #918). When none does, the `at` of each entry that
832
+ carries it is dropped with a warning (_drop), unless it is a checked
833
+ origin whose signature that entry carries. When none is dropped and each
834
+ still has an `at`, a place at another such origin, the origin is a
835
+ warning of its own: the citations there are not read. An entry left with
836
+ no `at` has a warning already when its form is compared, so the origin
837
+ adds none. A misplaced form never makes the inventory invalid, except
838
+ one whose signature gives the partial label (#888): it is placed only by
839
+ the entries of that name whose `at` is that origin and that give its
840
+ form, at least one and each with its `signature_form`, while each entry
841
+ of that name with no `at` (none given, or a bad one at_warnings or
842
+ place_warnings dropped first) that gives that form gives the label too.
843
+ It is otherwise one violation, a carrier elsewhere or with no `at`, an
844
+ unlabelled twin there and an unlabelled copy with no `at` included; an
845
+ unlabelled copy whose `at` is another place (with the package, a line
846
+ that writes the name) stays valid.
530
847
  """
531
848
  listed = inventory.get("references")
532
849
  files = {"SKILL.md"} | {_listed_path(ref, skill_package_path)
533
850
  for ref in (listed if isinstance(listed, list) else [])
534
851
  if isinstance(ref, str) and ref.strip()}
535
- entries = [e for e in inventory.get("exports") or [] if isinstance(e, dict)]
536
- kept = {e.get("name") for e in entries}
852
+ entries = [(idx, e) for idx, e in enumerate(inventory.get("exports") or []) if isinstance(e, dict)]
853
+ kept = {e.get("name") for _, e in entries}
537
854
  names = floor.get("names") or []
538
- checked, absent, violations, signed = 0, [], [], []
855
+ checked, absent, violations, warnings, signed = 0, [], [], [], []
539
856
  for item in names:
540
857
  origins = [o for o in item["origins"] if _cited_path(o["file"]) in files]
541
858
  if not origins:
@@ -548,15 +865,48 @@ def floor_check(inventory, floor, skill_package_path=None):
548
865
  absent.append(name)
549
866
  where = f"{origins[0]['file']}:{origins[0]['line']} ({origins[0]['form']})"
550
867
  violations.append(f"floor: `{name}` is documented at {where}, but no inventory entry lists it")
868
+
869
+ def carried(entry):
870
+ """The checked origins of the entry's name whose signature it carries, (path, line) as parse_at gives."""
871
+ return {(_cited_path(o["file"]), o["line"]) for n, o in signed
872
+ if n == entry.get("name") and _carries(entry, _origin_signature(o))}
873
+
551
874
  for name, origin in signed:
552
- signature = {f: origin["signature"][f] for f in FLOOR_SIGNATURE_FIELDS}
553
- if any(_carries(e, signature) for e in entries if e.get("name") == name):
875
+ signature = _origin_signature(origin)
876
+ place = (_cited_path(origin["file"]), origin["line"])
877
+ pinned = (f"floor: `{name}` is documented at {origin['file']}:{origin['line']} ({origin['form']}) with the "
878
+ f"signature {json.dumps(signature, ensure_ascii=False)}")
879
+ at = json.dumps(f"{place[0]}:{place[1]}")
880
+ if FLOOR_LABEL_FIELD in signature:
881
+ # A labelled pin (#888) is placed only at its own `at`: the entries of the name there that give its form
882
+ # are at least one, and each of them and each one with no `at` (a bad one is dropped by now) gives its
883
+ # label, so no unlabelled twin, nor a copy that shows no place, is the form score-signatures.py keeps. A
884
+ # copy whose `at` is another place is compared there.
885
+ form = {f: signature[f] for f in FLOOR_SIGNATURE_FIELDS if f in signature}
886
+ copies = [e for _, e in entries if e.get("name") == name and _carries(e, form)]
887
+ here = [e for e in copies if parse_at(e.get("at")) == place]
888
+ if not here or not all(_carries(e, signature) for e in copies
889
+ if e.get("at") is None or parse_at(e["at"]) == place):
890
+ violations.append(f"{pinned}, but no inventory entry of that name at that place carries it, or one "
891
+ f"gives its form without its `signature_form` there or with no `at`; copy it as "
892
+ f"given, with `at` {at}")
893
+ continue
894
+ carriers = [(idx, e) for idx, e in entries if e.get("name") == name and _carries(e, signature)]
895
+ if any(parse_at(e.get("at")) == place for _, e in carriers):
554
896
  continue
555
- where = f"{origin['file']}:{origin['line']} ({origin['form']})"
556
- violations.append(f"floor: `{name}` is documented at {where} with the signature "
557
- f"{json.dumps(signature, ensure_ascii=False)}, but no inventory entry of that name "
558
- "carries it; copy it as given")
559
- return {"names": len(names), "checked": checked, "absent": absent}, violations
897
+ if not carriers:
898
+ violations.append(f"{pinned}, but no inventory entry of that name carries it; copy it as given, "
899
+ f"with `at` {at}")
900
+ continue
901
+ dropped = [_drop(e, f"{pinned}, but exports[{idx}], which carries it, gives `at` {json.dumps(e['at'])}")
902
+ for idx, e in carriers
903
+ if parse_at(e.get("at")) is not None and parse_at(e["at"]) not in carried(e)]
904
+ if dropped:
905
+ warnings += dropped
906
+ elif all(e.get("at") is not None for _, e in carriers):
907
+ warnings.append(f"{pinned}, but no inventory entry that carries it gives `at` {at}, so the citations "
908
+ "there are not read")
909
+ return {"names": len(names), "checked": checked, "absent": absent}, violations, warnings
560
910
 
561
911
 
562
912
  # --- Completeness -------------------------------------------------------------
@@ -572,7 +922,9 @@ def missing_fields(entry):
572
922
  if not _filled(entry.get("description")):
573
923
  missing.append("description")
574
924
  if entry.get("kind") in SIGNATURE_KINDS:
575
- if not isinstance(entry.get("params"), (str, list)):
925
+ usage = entry.get("usage_signature")
926
+ if not isinstance(entry.get("params"), (str, list)) and not any(
927
+ map(_filled, usage if isinstance(usage, list) else [usage])):
576
928
  missing.append("params")
577
929
  if not _filled(entry.get("return_type")):
578
930
  missing.append("return_type")
@@ -694,16 +1046,107 @@ def mismatch_violations(idx, entry, skill_package_path=None):
694
1046
  return violations
695
1047
 
696
1048
 
1049
+ def _read_files(inventory, skill_package_path=None):
1050
+ """The files the inventory reads, as an `at` names them: `SKILL.md` and
1051
+ each `references` entry, _listed_path (relative to the package, forward
1052
+ slashes)."""
1053
+ listed = inventory.get("references")
1054
+ return {"SKILL.md"} | {_listed_path(ref, skill_package_path)
1055
+ for ref in (listed if isinstance(listed, list) else [])
1056
+ if isinstance(ref, str) and ref.strip()}
1057
+
1058
+
1059
+ # What a missing or dropped `at` leaves its entry's form (#918): plan binds it to no declaration, as before #918.
1060
+ UNBOUND = "the form is compared as one whose citations name no declaration"
1061
+
1062
+
1063
+ def _drop(entry, why):
1064
+ """Drop the entry's bad `at` (#918) and return its warning: `why`, then what the drop leaves (UNBOUND, for an
1065
+ entry that carries a compared form)."""
1066
+ del entry["at"]
1067
+ return f"{why}; it is dropped and {UNBOUND}" if compared_form(entry) else f"{why}; it is dropped"
1068
+
1069
+
1070
+ def at_warnings(idx, entry, files, skill_package_path=None):
1071
+ """The warnings of one named entry's `at` (#918), never a violation: an
1072
+ entry that carries a compared form (compared_form) and no `at` (a JSON
1073
+ null removed), or an `at` (on any entry) that is no "<path>:<line>" with
1074
+ a line from 1, or whose path is no file of `files` (_read_files), which
1075
+ is dropped (_drop). A good `at` is left in the entry as the package
1076
+ names it, "<path>:<line>" (_listed_path)."""
1077
+ where = f"exports[{idx}] (`{entry['name']}`)"
1078
+ value = entry.get("at")
1079
+ if value is None:
1080
+ entry.pop("at", None)
1081
+ return [f"{where} carries a form but no `at`, so {UNBOUND}"] if compared_form(entry) else []
1082
+ m = _AT_RE.fullmatch(value.strip()) if isinstance(value, str) else None
1083
+ if m is None or not m.group(1).strip():
1084
+ return [_drop(entry, f"{where}: `at` must be \"<path>:<line>\", a file of the package and a line from 1 "
1085
+ f"(\"SKILL.md:42\"), not {json.dumps(value)}")]
1086
+ path = _listed_path(m.group(1).strip(), skill_package_path)
1087
+ if path not in files:
1088
+ return [_drop(entry, f"{where}: `at` names {json.dumps(path)}, no file the inventory reads: SKILL.md or a "
1089
+ "file `references` lists")]
1090
+ entry["at"] = f"{path}:{int(m.group(2))}"
1091
+ return []
1092
+
1093
+
1094
+ def place_warnings(inventory, skill_package_path):
1095
+ """With the package: each entry's `at` (one at_warnings kept) is a line
1096
+ of its file that writes the entry's name outside its citations, or, for
1097
+ a line inside a fenced block (_fence_holding), one of a block that
1098
+ writes it. A file that cannot be read (missing, unreadable or outside
1099
+ the package), a line past its end and a line that does not write the
1100
+ name each drop the `at` with a warning naming the entry (_drop)."""
1101
+ root = Path(skill_package_path)
1102
+ real_root = os.path.realpath(root)
1103
+ read: dict[str, list[str] | None] = {}
1104
+ warnings = []
1105
+ for idx, entry in enumerate(inventory.get("exports") or []):
1106
+ if not isinstance(entry, dict) or entry.get("at") is None:
1107
+ continue
1108
+ path, line = parse_at(entry["at"])
1109
+ if path not in read:
1110
+ target = os.path.realpath(root / path)
1111
+ try:
1112
+ if not _inside(target, real_root):
1113
+ raise OSError("outside the package")
1114
+ read[path] = _segments()._split_lines(Path(target).read_bytes().decode("utf-8-sig"))
1115
+ except (OSError, UnicodeDecodeError):
1116
+ read[path] = None
1117
+ lines, name = read[path], entry["name"]
1118
+ where = f"exports[{idx}] (`{name}`): `at` {json.dumps(entry['at'])}"
1119
+ if lines is None:
1120
+ warnings.append(_drop(entry, f"{where} names a file that cannot be read"))
1121
+ continue
1122
+ if line > len(lines):
1123
+ warnings.append(_drop(entry, f"{where} is past the end of {path} ({len(lines)} lines)"))
1124
+ continue
1125
+ pattern = name_pattern(name)
1126
+ if _writes(pattern, lines[line - 1]):
1127
+ continue
1128
+ fence = _fence_holding(lines, line - 1)
1129
+ if fence is not None and any(_writes(pattern, ln) for ln in lines[fence[0] + 1:fence[1]]):
1130
+ continue
1131
+ held = ", nor does the fenced block that holds it" if fence is not None else ""
1132
+ warnings.append(_drop(entry, f"{where} is a line that does not write `{name}`{held}"))
1133
+ return warnings
1134
+
1135
+
697
1136
  def validate_inventory(raw, skill_package_path=None, floor=None):
698
1137
  """Structural validation of the subagent inventory response, then the spot-check and the floor check.
699
1138
 
700
1139
  `raw` is the raw response text. Returns the result dict documented in the
701
1140
  module docstring. With `skill_package_path`, an inventory that passed the
702
- schema is spot-checked against the package (spot_check); with `floor`
703
- (read_floor's object), it is checked against the floor (floor_check).
1141
+ schema is spot-checked against the package (spot_check), given its
1142
+ `crossChecked` count (cross_checked) and each `at` is looked up there
1143
+ (place_warnings); a `crossChecked` the response carries never survives.
1144
+ With `floor` (read_floor's object),
1145
+ it is checked against the floor (floor_check). A missing or bad `at` is
1146
+ a warning, the bad one dropped, never a violation (#918).
704
1147
  Deterministic: identical input yields an identical verdict.
705
1148
  """
706
- violations = []
1149
+ violations, warnings = [], []
707
1150
  inner = strip_fences(raw)
708
1151
  try:
709
1152
  data = json.loads(inner)
@@ -711,6 +1154,7 @@ def validate_inventory(raw, skill_package_path=None, floor=None):
711
1154
  return {
712
1155
  "valid": False,
713
1156
  "violations": [f"subagent response not valid JSON: {exc.msg}"],
1157
+ "warnings": [],
714
1158
  "rejectedCount": 0,
715
1159
  "exportsCount": 0,
716
1160
  "inventory": None,
@@ -720,10 +1164,13 @@ def validate_inventory(raw, skill_package_path=None, floor=None):
720
1164
  return {
721
1165
  "valid": False,
722
1166
  "violations": ["subagent response is not a JSON object"],
1167
+ "warnings": [],
723
1168
  "rejectedCount": 0,
724
1169
  "exportsCount": 0,
725
1170
  "inventory": None,
726
1171
  }
1172
+ # The validator alone writes `crossChecked` (#920): a reader's never survives.
1173
+ data.pop("crossChecked", None)
727
1174
 
728
1175
  exports = data.get("exports")
729
1176
  if not isinstance(exports, list):
@@ -771,6 +1218,13 @@ def validate_inventory(raw, skill_package_path=None, floor=None):
771
1218
  f"not {json.dumps(entry['signature_form'])}"
772
1219
  )
773
1220
 
1221
+ # Each entry that carries a compared form is asked for its place, `at`, in a file the inventory reads (#918):
1222
+ # one missing or bad is a warning, the bad one dropped.
1223
+ files = _read_files(data, skill_package_path)
1224
+ for idx, entry in enumerate(exports):
1225
+ if isinstance(entry, dict) and isinstance(entry.get("name"), str) and entry["name"]:
1226
+ warnings.extend(at_warnings(idx, entry, files, skill_package_path))
1227
+
774
1228
  # Each cross-check mismatch carries its fields and one other side of its
775
1229
  # pair, a pair of references in order.
776
1230
  for idx, entry in enumerate(cross):
@@ -788,15 +1242,20 @@ def validate_inventory(raw, skill_package_path=None, floor=None):
788
1242
  f"spot-check: `{name}` is listed as an export, but SKILL.md and the "
789
1243
  f"listed reference files never write it"
790
1244
  )
1245
+ # In the inventory object, so --output writes it and stdout's `inventory` equals the file.
1246
+ data["crossChecked"] = cross_checked(exports, _package_texts(data, skill_package_path)[0])
1247
+ warnings.extend(place_warnings(data, skill_package_path))
791
1248
  floor_result = None
792
1249
  if schema_valid and floor is not None:
793
- floor_result, floor_violations = floor_check(data, floor, skill_package_path)
1250
+ floor_result, floor_violations, floor_warnings = floor_check(data, floor, skill_package_path)
794
1251
  violations.extend(floor_violations)
1252
+ warnings.extend(floor_warnings)
795
1253
 
796
1254
  valid = not violations
797
1255
  result = {
798
1256
  "valid": valid,
799
1257
  "violations": violations,
1258
+ "warnings": warnings,
800
1259
  "rejectedCount": rejected,
801
1260
  "exportsCount": len(exports),
802
1261
  "inventory": data if valid else None,
@@ -822,7 +1281,7 @@ def _build_parser():
822
1281
  "the required-keys/types/enum/mismatch-field contract, then with "
823
1282
  "--skill-package spot-checks sampled names against the package and --floor checks it "
824
1283
  "keeps the floor's names, returning "
825
- "{valid, violations, rejectedCount, exportsCount, inventory, completeness, spotCheck, floor}."
1284
+ "{valid, violations, warnings, rejectedCount, exportsCount, inventory, completeness, spotCheck, floor}."
826
1285
  ),
827
1286
  formatter_class=argparse.RawDescriptionHelpFormatter,
828
1287
  )
@@ -856,7 +1315,8 @@ def _build_parser():
856
1315
  parser.add_argument(
857
1316
  "--skill-package",
858
1317
  metavar="DIR",
859
- help="Spot-check sampled names against this package's SKILL.md and listed references.",
1318
+ help="Spot-check sampled names against this package's SKILL.md and listed references, "
1319
+ "and add their crossChecked count to the inventory.",
860
1320
  )
861
1321
  parser.add_argument(
862
1322
  "--floor",