bmad-module-skill-forge 3.0.0 → 4.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/docs/_data/pinned.yaml +1 -1
- package/docs/examples.md +1 -1
- package/docs/troubleshooting.md +8 -0
- package/docs/verifying-a-skill.md +7 -3
- package/docs/workflows.md +3 -3
- package/package.json +1 -1
- package/src/knowledge/qmd-registry.md +1 -1
- package/src/knowledge/version-paths.md +1 -1
- package/src/shared/health-check.md +2 -50
- package/src/shared/scripts/schemas/skf-audit-result-envelope.v1.json +2 -2
- package/src/shared/scripts/schemas/skf-update-result-envelope.v1.json +2 -2
- package/src/shared/scripts/skf-atomic-write.py +212 -59
- package/src/shared/scripts/skf-build-change-manifest.py +593 -47
- package/src/shared/scripts/skf-classify-changed-files.py +78 -0
- package/src/shared/scripts/skf-compare-file-hashes.py +266 -104
- package/src/shared/scripts/skf-derive-assembly-shape.py +18 -1
- package/src/shared/scripts/skf-detect-scripts-assets.py +196 -54
- package/src/shared/scripts/skf-extract-public-api.py +1309 -157
- package/src/shared/scripts/skf-extraction-inventory.py +580 -46
- package/src/shared/scripts/skf-extraction-snapshot.py +158 -33
- package/src/shared/scripts/skf-forge-tier-rw.py +26 -3
- package/src/shared/scripts/skf-github-probe.py +47 -8
- package/src/shared/scripts/skf-hash-content.py +14 -8
- package/src/shared/scripts/skf-load-provenance.py +3 -1
- package/src/shared/scripts/skf-merge-doc-urls.py +142 -19
- package/src/shared/scripts/skf-preflight.py +15 -1
- package/src/shared/scripts/skf-provenance-gap-dispatch.py +25 -1
- package/src/shared/scripts/skf-resolve-authoritative-files.py +29 -3
- package/src/shared/scripts/skf-structural-diff.py +133 -18
- package/src/shared/scripts/skf-update-active-symlink.py +115 -62
- package/src/shared/scripts/skf-validate-brief-schema.py +19 -1
- package/src/shared/scripts/skf-verify-provenance-completeness.py +977 -49
- package/src/shared/scripts/skf-write-skill-brief.py +41 -8
- package/src/skf-analyze-source/references/map-unit-exports.md +1 -1
- package/src/skf-audit-skill/assets/drift-report-template.md +1 -0
- package/src/skf-audit-skill/references/headless-contract.md +1 -1
- package/src/skf-audit-skill/references/health-check.md +2 -0
- package/src/skf-audit-skill/references/init.md +2 -1
- package/src/skf-audit-skill/references/re-index.md +14 -9
- package/src/skf-audit-skill/references/report.md +2 -2
- package/src/skf-audit-skill/references/semantic-diff.md +15 -15
- package/src/skf-audit-skill/references/structural-diff.md +10 -5
- package/src/skf-audit-skill/scripts/render-drift-tables.py +29 -2
- package/src/skf-brief-skill/SKILL.md +1 -1
- package/src/skf-brief-skill/assets/scope-templates.md +1 -1
- package/src/skf-brief-skill/assets/skill-brief-schema.md +2 -9
- package/src/skf-brief-skill/references/analyze-target.md +6 -6
- package/src/skf-brief-skill/references/confirm-brief.md +1 -1
- package/src/skf-brief-skill/references/scope-definition.md +4 -4
- package/src/skf-brief-skill/references/step-auto-brief.md +11 -29
- package/src/skf-brief-skill/references/version-resolution.md +5 -9
- package/src/skf-create-skill/assets/compile-assembly-rules.md +5 -3
- package/src/skf-create-skill/assets/skill-sections.md +6 -6
- package/src/skf-create-skill/references/authoritative-files-protocol.md +2 -23
- package/src/skf-create-skill/references/compile.md +31 -60
- package/src/skf-create-skill/references/component-extraction.md +3 -3
- package/src/skf-create-skill/references/enrich.md +2 -2
- package/src/skf-create-skill/references/entry-points-by-hand.md +24 -7
- package/src/skf-create-skill/references/extract.md +22 -19
- package/src/skf-create-skill/references/extraction-patterns-tracing.md +9 -9
- package/src/skf-create-skill/references/extraction-patterns.md +8 -9
- package/src/skf-create-skill/references/generate-artifacts.md +1 -1
- package/src/skf-create-skill/references/report.md +16 -5
- package/src/skf-create-skill/references/source-resolution-protocols.md +12 -17
- package/src/skf-create-skill/references/sub/fetch-docs.md +1 -1
- package/src/skf-drop-skill/references/execute.md +5 -1
- package/src/skf-drop-skill/references/health-check.md +2 -0
- package/src/skf-drop-skill/references/invocation-contract.md +1 -1
- package/src/skf-drop-skill/references/report.md +2 -2
- package/src/skf-drop-skill/references/select.md +7 -3
- package/src/skf-test-skill/references/coherence-check.md +1 -1
- package/src/skf-test-skill/references/coverage-check-tiers.md +5 -5
- package/src/skf-test-skill/references/coverage-check.md +39 -57
- package/src/skf-test-skill/references/init.md +3 -5
- package/src/skf-test-skill/references/report.md +10 -8
- package/src/skf-test-skill/references/scoring-rules.md +6 -5
- package/src/skf-test-skill/references/source-access-protocol.md +14 -17
- package/src/skf-test-skill/scripts/build-result-context.py +114 -21
- package/src/skf-test-skill/scripts/gap-ledger.py +315 -30
- package/src/skf-test-skill/scripts/load-coverage-inputs.py +534 -96
- package/src/skf-test-skill/scripts/reconcile-coverage.py +276 -2
- package/src/skf-test-skill/scripts/score-signatures.py +36 -16
- package/src/skf-test-skill/scripts/verify-declared-numerator.py +7 -5
- package/src/skf-update-skill/references/detect-changes.md +32 -13
- package/src/skf-update-skill/references/gap-driven.md +3 -3
- package/src/skf-update-skill/references/init.md +6 -2
- package/src/skf-update-skill/references/invocation-contract.md +1 -1
- package/src/skf-update-skill/references/merge.md +2 -2
- package/src/skf-update-skill/references/re-extract.md +2 -2
- package/src/skf-update-skill/references/report.md +3 -1
- package/src/skf-update-skill/references/write.md +43 -23
- package/src/skf-update-skill/scripts/skf-new-file-diff.py +157 -42
|
@@ -72,7 +72,8 @@ Context payload shape (consumed by `write`):
|
|
|
72
72
|
"doc_urls": [{"url": "...", "label": "...", "source": "..."}],
|
|
73
73
|
# `source` (optional per #432): language-registry | readme-detection |
|
|
74
74
|
# homepage | pages-api | docs-folder. Threaded through when present;
|
|
75
|
-
# absent → entry renders as {url, label} only (no false drift)
|
|
75
|
+
# absent → entry renders as {url, label} only (no false drift); any
|
|
76
|
+
# other value is refused before anything is written (exit 1).
|
|
76
77
|
"scripts_intent": "detect" | "none" | free-text,
|
|
77
78
|
"assets_intent": "detect" | "none" | free-text,
|
|
78
79
|
"source_authority": "official" | "community" | "internal",
|
|
@@ -87,7 +88,9 @@ Version precedence (resolved into the rendered YAML's `version` field):
|
|
|
87
88
|
3. Otherwise: detected_version when it is full X.Y.Z semver (SEMVER_RE);
|
|
88
89
|
a detected version of another shape (`0.1`, `2.0.0rc1`) is passed as
|
|
89
90
|
found and skipped here, with a warning.
|
|
90
|
-
4. Otherwise: "1.0.0"
|
|
91
|
+
4. Otherwise: "1.0.0", with a warning when no detected_version was given
|
|
92
|
+
(a docs-only brief aside), so a headless caller tells the default
|
|
93
|
+
from a detected 1.0.0.
|
|
91
94
|
|
|
92
95
|
When target_version is set, the rendered YAML includes a `target_version`
|
|
93
96
|
field whose value MUST match `version` (the script enforces this
|
|
@@ -235,6 +238,8 @@ VALID_SCOPE_TYPES = {
|
|
|
235
238
|
"reference-app",
|
|
236
239
|
"docs-only",
|
|
237
240
|
}
|
|
241
|
+
# skill-brief.v1.json's doc_urls[].source enum (a test pins it to the schema).
|
|
242
|
+
VALID_DOC_URL_SOURCES = {"language-registry", "readme-detection", "homepage", "pages-api", "docs-folder"}
|
|
238
243
|
|
|
239
244
|
|
|
240
245
|
def _die(message: str, field: str | None = None, code: int = 1) -> None:
|
|
@@ -321,6 +326,10 @@ def validate_context(ctx: dict[str, Any]) -> list[str]:
|
|
|
321
326
|
url = entry.get("url")
|
|
322
327
|
if not url or not isinstance(url, str):
|
|
323
328
|
_die(f"doc_urls[{i}].url is required and must be a non-empty string", field="doc_urls")
|
|
329
|
+
source = entry.get("source")
|
|
330
|
+
if source and (not isinstance(source, str) or source not in VALID_DOC_URL_SOURCES):
|
|
331
|
+
_die(f"doc_urls[{i}].source must be one of {sorted(VALID_DOC_URL_SOURCES)}. Got: {source!r}",
|
|
332
|
+
field="doc_urls")
|
|
324
333
|
|
|
325
334
|
# source_authority (default 'community') with docs-only force rule
|
|
326
335
|
source_authority = ctx.get("source_authority", "community")
|
|
@@ -437,6 +446,12 @@ def validate_context(ctx: dict[str, Any]) -> list[str]:
|
|
|
437
446
|
warnings.append(
|
|
438
447
|
f"detected_version {detected!r} is not full X.Y.Z semver — falling through to default 1.0.0"
|
|
439
448
|
)
|
|
449
|
+
elif detected is None and tv is None and ctx.get("version_resolved") is None and source_type != "docs-only":
|
|
450
|
+
# The default would otherwise reach a headless caller as if detected.
|
|
451
|
+
warnings.append(
|
|
452
|
+
"no source version detected: version defaults to 1.0.0; for a remote source skf-create-skill "
|
|
453
|
+
"tries tag 1.0.0 or v1.0.0; set target_version in the brief to pin it"
|
|
454
|
+
)
|
|
440
455
|
|
|
441
456
|
return warnings
|
|
442
457
|
|
|
@@ -594,10 +609,11 @@ def render_yaml(brief: dict[str, Any]) -> str:
|
|
|
594
609
|
|
|
595
610
|
The step 5 §3 template shows leading and trailing `---` markers, but those were
|
|
596
611
|
wrapping the example YAML for documentation purposes — actual on-disk YAML uses
|
|
597
|
-
only the leading `---` (or none). A trailing `---`
|
|
598
|
-
document
|
|
599
|
-
|
|
600
|
-
|
|
612
|
+
only the leading `---` (or none). A trailing `---` starts a second, empty
|
|
613
|
+
document: SKF's own readers skip it (they parse a brief with
|
|
614
|
+
skf-resolve-authoritative-files.py's parse_brief_yaml, which reads the SKF 1.x
|
|
615
|
+
briefs that ended so), but other YAML tools that expect a single document,
|
|
616
|
+
such as `yaml.safe_load`, refuse it, so the writer leaves it out.
|
|
601
617
|
"""
|
|
602
618
|
body = yaml.safe_dump(
|
|
603
619
|
brief,
|
|
@@ -755,6 +771,23 @@ def _read_json_object(path: Path, flag: str) -> dict[str, Any]:
|
|
|
755
771
|
return data
|
|
756
772
|
|
|
757
773
|
|
|
774
|
+
RESOLVER = Path(__file__).resolve().parent / "skf-resolve-authoritative-files.py"
|
|
775
|
+
|
|
776
|
+
|
|
777
|
+
def _parse_brief_yaml(text: str) -> Any:
|
|
778
|
+
"""A brief's YAML as skf-resolve-authoritative-files.py (from this folder)
|
|
779
|
+
reads it: an SKF 1.x brief's trailing `---` is no second document."""
|
|
780
|
+
spec = importlib.util.spec_from_file_location("skf_resolve_authoritative_files", RESOLVER)
|
|
781
|
+
if spec is None or spec.loader is None:
|
|
782
|
+
_die(f"cannot load {RESOLVER.name}", code=2)
|
|
783
|
+
resolver = importlib.util.module_from_spec(spec)
|
|
784
|
+
try:
|
|
785
|
+
spec.loader.exec_module(resolver)
|
|
786
|
+
except (OSError, ImportError, SyntaxError) as exc:
|
|
787
|
+
_die(f"cannot load {RESOLVER.name} beside {Path(__file__).name}: {exc}", code=2)
|
|
788
|
+
return resolver.parse_brief_yaml(text)
|
|
789
|
+
|
|
790
|
+
|
|
758
791
|
def base_context(base_brief: Path, doc_urls_file: Path | None, patch_file: Path | None) -> dict[str, Any]:
|
|
759
792
|
"""The write context of the base brief form (see the module docstring)."""
|
|
760
793
|
try:
|
|
@@ -762,7 +795,7 @@ def base_context(base_brief: Path, doc_urls_file: Path | None, patch_file: Path
|
|
|
762
795
|
except OSError as e:
|
|
763
796
|
_die(f"write: cannot read the base brief {base_brief}: {e}", code=2)
|
|
764
797
|
try:
|
|
765
|
-
brief =
|
|
798
|
+
brief = _parse_brief_yaml(original.decode("utf-8-sig"))
|
|
766
799
|
except (UnicodeDecodeError, yaml.YAMLError) as e:
|
|
767
800
|
_die(f"write: the base brief {base_brief} is not YAML: {e}")
|
|
768
801
|
if not isinstance(brief, dict):
|
|
@@ -895,7 +928,7 @@ def cmd_amend(target: Path) -> int:
|
|
|
895
928
|
except OSError as e:
|
|
896
929
|
_die(f"amend: cannot read {target}: {e}", code=2)
|
|
897
930
|
try:
|
|
898
|
-
brief =
|
|
931
|
+
brief = _parse_brief_yaml(original.decode("utf-8-sig"))
|
|
899
932
|
except (UnicodeDecodeError, yaml.YAMLError) as e:
|
|
900
933
|
_die(f"amend: {target} is not a YAML brief: {e}")
|
|
901
934
|
if not isinstance(brief, dict) or not isinstance(brief.get("scope"), dict):
|
|
@@ -27,7 +27,7 @@ uv run {extractPublicApiHelper} --mode full --source-root "{unit_root}" --scope-
|
|
|
27
27
|
|
|
28
28
|
Otherwise read the JSON by the exit code:
|
|
29
29
|
|
|
30
|
-
- **0 or 1:** `exports_count` ← `counts.exports_public_api`, `api_surface` ← the names in `entry_point_diff.public[]
|
|
30
|
+
- **0 or 1:** `exports_count` ← `counts.exports_public_api`, `api_surface` ← the names in `entry_point_diff.public[]` (a `namespace` item with a `members` list given as its members' names, the names its module passes on), `export_pattern` ← `entry_points.status` with `aggregates.by_type` (for example "barrel: 12 functions, 5 classes", marked "capped" when `truncated` is true), `strategy_used: "ast-grep"`, `confidence: "T1"`. Then read by eye what the recipes could not, and name each one in `warnings`: every `entry_point_diff.extraction_gaps[]` name (public, but no recipe found its definition: read it at its `file` and `line`) and every `file_issues[]` file (one the parser could not fully read: add the exports it shows to `api_surface`). Exit 1 (`status: "incomplete"`) means an ast-grep run failed on some files: keep what it returned and put its `errors[]` in `warnings`.
|
|
31
31
|
- **3** (no ast-grep the runner can run): the JSON still lists `entry_points.files`. Read the exports from those entry-point files by eye (from the unit's source files when it lists none), with `strategy_used: "source-read"` and `confidence: "T1-low"`.
|
|
32
32
|
- **2:** an input error, named on stderr (a wrong path or flag): fix the call and run it again.
|
|
33
33
|
|
|
@@ -60,6 +60,7 @@ previousWorkflow: 'create-skill'
|
|
|
60
60
|
<!-- Appended by structural-diff, or for a compose-mode stack by constituent-freshness -->
|
|
61
61
|
<!-- Constituent Freshness (compose-mode stacks): drifted, missing and not-compared constituents; each drifted (HIGH) or missing (MEDIUM) one is a finding -->
|
|
62
62
|
<!-- Provenance label differences (not drift): informational table, rendered only when label_changes[] is non-empty and excluded from Total Drift Items -->
|
|
63
|
+
<!-- Off the public surface (not drift): a note under the Summary naming a public-api skill's new exports off its public surface (not_public[]), rendered only when summary.not_public is above 0 and excluded from Total Drift Items -->
|
|
63
64
|
|
|
64
65
|
---
|
|
65
66
|
|
|
@@ -20,7 +20,7 @@ Every hard halt in this workflow exits with a stable code so headless automators
|
|
|
20
20
|
|
|
21
21
|
## Result Contract (Headless)
|
|
22
22
|
|
|
23
|
-
When `{headless_mode}` is true, step 6 prints one `SKF_AUDIT_RESULT_JSON: {...}` line on **stdout
|
|
23
|
+
When `{headless_mode}` is true, step 6 prints one `SKF_AUDIT_RESULT_JSON: {...}` line on **stdout**, which the health check (step 7) displays as the run's last line, and every hard halt one on **stderr** with `status: "error"`. The shared emitter builds each line from the payload the stage stages in `{run_dir}`, folds in the decisions and warnings the run recorded there, stamps `run_id` and `result_path`, and checks it against `{project-root}/_bmad/skf/shared/scripts/schemas/skf-audit-result-envelope.v1.json` (`{project-root}/src/shared/scripts/schemas/skf-audit-result-envelope.v1.json` in a dev checkout), whose descriptions say what each field holds. Never type the line yourself:
|
|
24
24
|
|
|
25
25
|
```
|
|
26
26
|
SKF_AUDIT_RESULT_JSON: {"status":"success|error","skill_name":"…","drift_score":"CLEAN|MINOR|SIGNIFICANT|CRITICAL|null","report_path":"…|null","next_workflow":"update-skill|null","audit_ref":"…|null","upstream_moved":true|false|null,"upstream_ref":"…|null","exit_code":0,"halt_reason":null,"run_id":"…","result_path":"…|null"}
|
|
@@ -13,6 +13,8 @@ nextStepFile: 'shared/health-check.md'
|
|
|
13
13
|
|
|
14
14
|
Chain to the shared workflow self-improvement health check at `{nextStepFile}`. This is the terminal step of audit-skill — after the shared health check completes, the workflow is fully done.
|
|
15
15
|
|
|
16
|
+
In a headless run report.md's Result Contract bound `{result_envelope_line}` to the `SKF_AUDIT_RESULT_JSON` line without displaying it: the shared health check displays that line verbatim as the run's last line wherever it stops, so a `claude -p` caller reads the envelope as the final message.
|
|
17
|
+
|
|
16
18
|
## Rules
|
|
17
19
|
|
|
18
20
|
- No user-facing reports, file writes, or result contracts in this step — those belong in step 6
|
|
@@ -156,6 +156,7 @@ Load the following from the skill directory:
|
|
|
156
156
|
- `name`, `version` and `generation_date`, and `{original_tier}`, the tier the skill was compiled at: `forge_tier` when it holds Quick, Forge, Forge+ or Deep (a stack records it there), else `confidence_tier` when it does (a single skill records it there), else null
|
|
157
157
|
- `source_root` — Resolved source code path used during extraction
|
|
158
158
|
- `source_repo`: the repository the skill was built from, which the [C] of `upstream-checkout.md` reads a newer ref of (`{source_repo}`, an empty string when the field is null or missing)
|
|
159
|
+
- `scope_type`: bind `{scope_type}` ← the `scope_type` metadata.json records, else the `scope.type` of `{forge_data_folder}/{skill_name}/skill-brief.yaml` when that file exists and names one (a skill whose metadata.json records no scope type is audited under its brief's), else null. When it is `public-api`, step 2 passes `--scope-type public-api` to the recipe runner and the snapshot build (`re-index.md` §3), so step 3 does not report as added an export off the public surface that create-skill and update-skill keep the skill's map to. The docs-only rule below reads the `scope_type` and `source_type` metadata.json itself records, never this brief fallback
|
|
159
160
|
|
|
160
161
|
**A docs-only skill** (its `scope_type`, or the `source_type` an older `metadata.json` records, is `docs-only`) has no source tree: create-skill recorded each document it was built from in `doc_sources`, with its content hash, and step 5a compares them. Set `{docs_only_skill}` to true. When `doc_sources` is missing or empty, or no entry of it has a `content_hash` (create-skill records null for a document it could not fetch), nothing records the documents' state: HALT with **exit 3**, `halt_reason: "no-baseline"`, phase `init:baseline`: "**`{skill_name}` records no documentation hashes, so there is no baseline to audit it against.** Re-create it with `[CS] Create Skill`, which records them." Otherwise skip §4, Stack Skill Detection and §5, take §5b's docs-only skip, and continue at §6. Every other skill sets `{docs_only_skill}` to false and continues at §4.
|
|
161
162
|
|
|
@@ -252,7 +253,7 @@ Show `n/a` for a value the run does not have (a docs-only skill reads no provena
|
|
|
252
253
|
- {Quick: text-diff comparison → T1-low for every export (read by eye)}
|
|
253
254
|
- {Forge: AST structural comparison → T1 for each export an ast-grep rule matches, T1-low for each export read by eye}
|
|
254
255
|
- {Forge+: AST structural comparison + CCC-assisted rename detection → the same labels as Forge}
|
|
255
|
-
- {Deep: AST structural +
|
|
256
|
+
- {Deep: AST structural + a semantic comparison → the same labels as Forge, plus `T2` from QMD or `T1-low-fallback` from the skill's files}
|
|
256
257
|
- {Compose-mode stack, at any tier: constituent freshness by metadata hash (step 1c), with no source re-index}
|
|
257
258
|
- {Docs-only skill, at any tier: each tracked document's content hash against the one recorded at compile time (step 5a), each changed document graded by step 5, with no source re-index}"
|
|
258
259
|
|
|
@@ -56,7 +56,7 @@ Based on forge tier detected in Step 01:
|
|
|
56
56
|
**Forge tier (ast-grep available):**
|
|
57
57
|
- The recipe runner extracts, as the **AST Extraction Protocol** in `{extractionPatternsData}` says: one `skf-extract-public-api.py --mode full` call over the bounded scan list from §2 (§3 gives the command) runs every recipe for the language in one process, reads only the files on the list, and keeps every match (`--head-cap 0`: a match past a cap would read as a removed export in step 3). While it can run, never run the recipes one at a time, batch them, or merge and dedupe their matches by hand
|
|
58
58
|
- Each export it returns is T1 as it stands: its name (`$NAME`), type, file, line (`$NAME`'s line, as create-skill records it, not the match's first line, which a decorator or an `export` line can hold), `ast_node_type` and `signature`, the declaration on one line even when a formatter split its parameters over several
|
|
59
|
-
- Read by eye only what the runner leaves (§3): the forms the recipes leave out (Known Limitation #11 in `{extractionPatternsData}`) when a file uses them, the names an entry point exports that no recipe found, and the files it could not parse or read. Label each export by the tool that produced it: T1 with `extraction_method: ast-grep` for a runner export, T1-low with `extraction_method: source-read` and `ast_node_type: null` for an export read by eye
|
|
59
|
+
- Read by eye only what the runner leaves (§3): the forms the recipes leave out (Known Limitation #11 in `{extractionPatternsData}`) when a file uses them, the names an entry point exports that no recipe found, the provenance map's entries the runner leaves out (an underscore name such as `__version__`, a module) that the source still declares, and the files it could not parse or read. Label each export by the tool that produced it: T1 with `extraction_method: ast-grep` for a runner export, T1-low with `extraction_method: source-read` and `ast_node_type: null` for an export read by eye
|
|
60
60
|
|
|
61
61
|
**Tier degradation handling (Forge/Forge+/Deep):** If the runner cannot run and ast-grep is unavailable, follow `{tierDegradationRulesData}` (AST Tool Unavailable, Per-File AST Failure) for the fallback and the user notification. A file ast-grep cannot parse falls back to reading that file only: its exports are T1-low with `extraction_method: source-read` and `ast_node_type: null`, and exports the runner matched in other files stay T1. Silent degradation is forbidden: the snapshot names each file's status, and the run names each file read after an ast-grep failure, and why (§3, §4); when ast-grep is unavailable for the whole run, every export is T1-low.
|
|
62
62
|
|
|
@@ -102,11 +102,12 @@ uv run {extractPublicApiHelper} --mode full \
|
|
|
102
102
|
--source-root "{source_root}" \
|
|
103
103
|
--files-from "{auditDataFolder}/scan-files.json" \
|
|
104
104
|
--head-cap 0 \
|
|
105
|
+
[--scope-type public-api] \
|
|
105
106
|
[--timeout "{runner_timeout}"] \
|
|
106
107
|
-o "{auditDataFolder}/extraction.json"
|
|
107
108
|
```
|
|
108
109
|
|
|
109
|
-
The runner stops its ast-grep runs after 540 seconds and still writes its JSON (status `incomplete`), so give the command your shell tool's longest timeout; when that is under 10 minutes, pass `--timeout` as `{runner_timeout}`, a little under it (`100` under a two-minute limit). Never open the JSON: item 2's helper reads it and prints what this step acts on, its `status` included (never the exit code). At Quick tier no runner runs.
|
|
110
|
+
The runner stops its ast-grep runs after 540 seconds and still writes its JSON (status `incomplete`), so give the command your shell tool's longest timeout; when that is under 10 minutes, pass `--timeout` as `{runner_timeout}`, a little under it (`100` under a two-minute limit). Pass `--scope-type public-api` only when `{scope_type}` (step 1 §3) is `public-api`, and never pass the brief (`--brief`): its globs would leave out files on the scan list, which step 3 would then report removed. Never open the JSON: item 2's helper reads it and prints what this step acts on, its `status` included (never the exit code). At Quick tier no runner runs.
|
|
110
111
|
|
|
111
112
|
**2. Build the snapshot.** Resolve `{extractionSnapshotHelper}` ← first existing path in `{extractionSnapshotProbeOrder}` (if none exists, HALT with **exit 3**, `halt_reason: "helper-missing"`, phase `re-index:snapshot`) and, from `{project-root}`, run:
|
|
112
113
|
|
|
@@ -119,27 +120,31 @@ uv run {extractionSnapshotHelper} build \
|
|
|
119
120
|
--provenance-map "{provenanceMap}" \
|
|
120
121
|
[--extraction "{auditDataFolder}/extraction.json"] \
|
|
121
122
|
[--details "{auditDataFolder}/export-details-{n}.json"]... \
|
|
123
|
+
[--scope-type public-api] \
|
|
122
124
|
-o "{forge_version}/extraction-snapshot.json"
|
|
123
125
|
```
|
|
124
126
|
|
|
125
|
-
Pass `--extraction` when the runner wrote its JSON, and one `--details` for each file item 3 wrote (none on the first build). The helper merges the runner's exports with the ones read by eye, gives every file on the list one status (`extracted`, `read-by-eye`, `hash-tracked` for a script, asset or doc only `file_entries[]` names, which step 3 compares by content hash, `missing`, `parse-failed` or `unread`) and prints one line. Act on its fields:
|
|
127
|
+
Pass `--extraction` when the runner wrote its JSON, `--scope-type public-api` when `{scope_type}` is `public-api` (at Quick tier too, where no runner runs), and one `--details` for each file item 3 wrote (none on the first build). The helper merges the runner's exports with the ones read by eye, gives every file on the list one status (`extracted`, `read-by-eye`, `hash-tracked` for a script, asset or doc only `file_entries[]` names, which step 3 compares by content hash, `missing`, `parse-failed` or `unread`) and prints one line. Act on its fields:
|
|
126
128
|
|
|
127
129
|
- **`runner_status` is `incomplete`** on the first build (an ast-grep run failed, or the time limit ran out): run item 1 once more, then this command again.
|
|
128
|
-
- **`runner_status` is `no-ast-grep`, or null at Forge tier and above** (no JSON at `-o` after the call, or no runner candidate): the runner cannot run. Follow **When the Runner Cannot Run** in `{extractionPatternsData}` over the files in `to_read`, with this audit's parameters in place of create-skill's: the files in scope are those `to_read` lists, and their count is the Decision Tree's input (not create-skill's step 3 count). Keep every match, in every directory batch and every retry: pass `find_code_by_rule` a `max_results` that no recipe can reach (100000), and a Known Limitation #4 `find_code` retry the same, and drop the `| head -{HEAD_CAP}` from the CLI streaming template, in each batch, since a match past a cap reads as a removed export in step 3. Record each export as item 3 does, T1 with `ast-grep` for one the recipes match.
|
|
129
|
-
- **`to_read` or `extraction_gaps` is not empty**, or a file uses the forms the recipes leave out: item 3 reads them, then run this command again with its details. When `complete` is still false, the files `to_read` lists could not be read: HALT with **exit 3**, `halt_reason: "source-unreadable"`, phase `re-index:snapshot`, naming them. Step 3 never reads an incomplete snapshot.
|
|
130
|
+
- **`runner_status` is `no-ast-grep`, or null at Forge tier and above** (no JSON at `-o` after the call, or no runner candidate): the runner cannot run. Follow **When the Runner Cannot Run** in `{extractionPatternsData}` over the files in `to_read`, with this audit's parameters in place of create-skill's: the files in scope are those `to_read` lists, and their count is the Decision Tree's input (not create-skill's step 3 count). Keep every match, in every directory batch and every retry: pass `find_code_by_rule` a `max_results` that no recipe can reach (100000), and a Known Limitation #4 `find_code` retry the same, and drop the `| head -{HEAD_CAP}` from the CLI streaming template, in each batch, since a match past a cap reads as a removed export in step 3. Record each export as item 3 does, T1 with `ast-grep` for one the recipes match. When item 1's call left no JSON and a stderr line that holds `error:` (an input error, such as a source root it cannot find, or a failure it did not foresee), show that section's **Recipe runner failed** warning with item 3's **AST fallback** warnings, and record `recipe_runner_failed: {that line}` for the envelope: write the text to `{run_dir}/warning.txt` with a file write (the line can hold quotes), then, from `{project-root}`, run `uv run {emitEnvelopeHelper} record --run-dir "{run_dir}" --warning "$(cat "{run_dir}/warning.txt")"`.
|
|
131
|
+
- **`to_read` or `extraction_gaps` is not empty**, or a file uses the forms the recipes leave out: item 3 reads them, then run this command again with its details. When `complete` is still false, the files `to_read` lists could not be read: HALT with **exit 3**, `halt_reason: "source-unreadable"`, phase `re-index:snapshot`, naming them. Step 3 never reads an incomplete snapshot. Item 3 reads the gaps once: when `complete` is true and `extraction_gaps` still lists a name after that read (one the reader could not place), warn "**Extraction gap:** {name} ({file}:{line}) was not found by eye; step 3 reports it as removed." for each and go on to §4 without reading again.
|
|
132
|
+
- **`public_surface.problem` is not null** on the build that goes on to §4 (with `--scope-type public-api`, the public-surface rule could read no surface: no runner JSON, as at Quick tier, a runner that could not run, an incomplete run, one with errors, or an entry point the runner could not trace): warn "**Public surface not applied:** {problem}; step 3 reports every export the provenance map does not hold as added." and record `public_surface_not_applied: {problem}` for the envelope, once: write the text to `{run_dir}/warning.txt` with a file write (the problem can hold quotes), then, from `{project-root}`, run `uv run {emitEnvelopeHelper} record --run-dir "{run_dir}" --warning "$(cat "{run_dir}/warning.txt")"`.
|
|
130
133
|
- **`complete` is true** and nothing is left to read: go on to §4.
|
|
131
|
-
- **`status: "error"`:** an `error` that starts `cannot write` is a failed write: HALT with **exit 4**, `halt_reason: "write-failed"`, phase `re-index:snapshot`, `"path": "{forge_version}/extraction-snapshot.json"`. Any other names a file this step wrote: fix it and run the command again.
|
|
134
|
+
- **`status: "error"`:** an `error` that starts `cannot write` is a failed write: HALT with **exit 4**, `halt_reason: "write-failed"`, phase `re-index:snapshot`, `"path": "{forge_version}/extraction-snapshot.json"`. An `error` that starts `cannot load` (a helper this build loads beside it is missing) HALTs with **exit 3**, `halt_reason: "helper-missing"`, phase `re-index:snapshot`, showing the `error`. Any other names a file this step wrote: fix it and run the command again.
|
|
132
135
|
- **The command fails otherwise, or prints no JSON:** HALT with **exit 3**, `halt_reason: "helper-missing"`, phase `re-index:snapshot`, showing the first stderr line.
|
|
133
136
|
|
|
134
137
|
**3. What the runner leaves.** Read by eye (T1-low, `extraction_method: source-read`, `ast_node_type: null`) only:
|
|
135
138
|
|
|
136
139
|
- each file in `to_read`: a `parse-failed` one (its `issue`: a syntax error, where a recipe can miss an export; not UTF-8; unreadable), with the warning "**AST fallback:** ast-grep could not parse {file} ({issue}); its exports were read from source and labeled T1-low.", and an `unread` one (an incomplete run, or no runner: then every file, by text pattern matching at Quick tier);
|
|
137
|
-
- each name in `extraction_gaps`, in its `file`
|
|
140
|
+
- each name in `extraction_gaps`, in its `file` at its `line`: a gap with an `entry` is a name that entry point exports and no recipe found; a gap whose `entry` is null is a provenance map entry the runner leaves out (an underscore name such as `__version__`, a module) that its file still declares;
|
|
138
141
|
- the forms the recipes leave out (Known Limitation #11 in `{extractionPatternsData}`) in a file that uses them.
|
|
139
142
|
|
|
140
|
-
For each file to read, launch a subprocess (Pattern 2) that loads it and returns ONLY `{"files": [...], "exports": [...]}`, with no prose and no fences: `files` lists the file when the worker read the whole of it (also when it holds no export), and each export gives `name`, `type`, `signature` (the declaration on one line), `file` (relative to `{source_root}`, with forward slashes), `line` (`$NAME`'s line), `confidence: "T1-low"`, `extraction_method: "source-read"` and `ast_node_type: null`. Save each worker's JSON as it returned it, as `{auditDataFolder}/export-details-{n}.json` (`{n}` counting from 1), never merged by hand; without subprocesses, read the files in the main thread, one by one, and save one such file. A stack's export needs no library: the snapshot helper gives each one its file's `source_library` from the provenance map (the map `normalize` returns as `source_library_by_file`).
|
|
143
|
+
For each file to read, launch a subprocess (Pattern 2) that loads it and returns ONLY `{"files": [...], "exports": [...]}`, with no prose and no fences: `files` lists the file when the worker read the whole of it (also when it holds no export), and each export gives `name`, `type`, `signature` (the declaration on one line), `file` (relative to `{source_root}`, with forward slashes), `line` (`$NAME`'s line), `confidence: "T1-low"`, `extraction_method: "source-read"` and `ast_node_type: null`. For a gap whose `entry` is null, record its `name`, `file` and `line`; as `type`, its `export_type` when the declaration at that line still has that kind, else the kind it has now (a `variable` now declared by a `def` is a `function`, which step 3 reports as a changed kind), and when the gap gives no `export_type`, the kind the declaration has; as `signature`, the declaration on one line, the import line for an alias that renames (`from cognee.api.v1.visualize import visualize_graph as visualize`), and for a module or package (a gap at line 1 of a file the name is the module or package of) `module` and the file's path from `{source_root}` without its extension or a final `__init__` or `index`, dotted for Python (`module cognee.modules.pipelines`). Save each worker's JSON as it returned it, as `{auditDataFolder}/export-details-{n}.json` (`{n}` counting from 1), never merged by hand; without subprocesses, read the files in the main thread, one by one, and save one such file. A stack's export needs no library: the snapshot helper gives each one its file's `source_library` from the provenance map (the map `normalize` returns as `source_library_by_file`).
|
|
141
144
|
|
|
142
|
-
**The
|
|
145
|
+
**The public surface.** With `--scope-type public-api` the helper marks each export of a language family whose entry points are a barrel `public: true` or `public: false`, by the rule create-skill and update-skill keep a public-api skill's provenance map to (`public_surface` in `skf-extraction-inventory.py`), and the line's `public_surface` gives the rule's `problem` (null when it applied) and `marked_not_public`. Every export stays in the snapshot, whatever its mark: a map an older SKF wrote can hold an export off the surface, and dropping it would read as removed. Step 3's diff lists an export marked `public: false` that pairs with no map entry under `not_public[]`, never under `added[]`. Without the flag, or with any other scope type, nothing is marked and `public_surface` is null.
|
|
146
|
+
|
|
147
|
+
**The snapshot.** The command writes `{forge_version}/extraction-snapshot.json`, which step 3 (`structural-diff.md`) reads directly: `exports[]` (for a stack, each with its file's `source_library`; for a public-api skill, each with its `public` mark when the surface marks it), `files[]` with each file's status, `files_by_status`, `counts` and `cap_hits` (§4) and `outside_scope` (step 6's Out-of-Scope New Public API table); `skf-extraction-snapshot.py --help` gives the full shape. Record the written path as `{extractionSnapshot}` in workflow context: step 3 passes it to the structural-diff helper.
|
|
143
148
|
|
|
144
149
|
### 4. Validate Extraction Completeness
|
|
145
150
|
|
|
@@ -139,7 +139,7 @@ Build the section from the frontmatter of {outputFile}, where step 1 §6, step 2
|
|
|
139
139
|
**Confidence Legend:**
|
|
140
140
|
- **T1:** an ast-grep match (`extraction_method: ast-grep`) at any tier: high reliability, structural truth
|
|
141
141
|
- **T1-low:** read by eye (`extraction_method: source-read`) at any tier: moderate reliability
|
|
142
|
-
- **T1-low-fallback:** Deep-tier semantic diff read directly from the skill's docs and the current source because
|
|
142
|
+
- **T1-low-fallback:** Deep-tier semantic diff read directly from the skill's docs and the current source because no QMD extraction collection was registered for the skill, or the one registered was empty: moderate reliability
|
|
143
143
|
- **T2:** Deep-tier semantic diff: a claim the skill's QMD collection holds, checked against the current source line it cites: evidence-backed semantic analysis
|
|
144
144
|
- **T3:** external documentation reference: variable reliability, secondary source
|
|
145
145
|
```
|
|
@@ -196,7 +196,7 @@ When On Activation step 3 fell back to the bundled `customize.toml`, add `"custo
|
|
|
196
196
|
uv run {emitEnvelopeHelper} emit --workflow skf-audit-skill --run-dir "{run_dir}" --result-dir "{forge_version}" < "{run_dir}/result-context.json"
|
|
197
197
|
```
|
|
198
198
|
|
|
199
|
-
It writes `{forge_version}/audit-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name) and its `audit-skill-result-latest.json` copy (stable path for pipeline consumers: a copy, not a symlink), stamps the timestamp, `run_id`, `headless_decisions` and `warnings` into both, and prints the `SKF_AUDIT_RESULT_JSON:` line on stdout. When `{headless_mode}` is true,
|
|
199
|
+
It writes `{forge_version}/audit-skill-result-{YYYYMMDD-HHmmss}.json` (UTC; it picks the name) and its `audit-skill-result-latest.json` copy (stable path for pipeline consumers: a copy, not a symlink), stamps the timestamp, `run_id`, `headless_decisions` and `warnings` into both, and prints the `SKF_AUDIT_RESULT_JSON:` line on stdout. When `{headless_mode}` is true, bind `{result_envelope_line}` ← the line and do not display it here: the shared health check displays it verbatim as the run's last line, so a `claude -p` caller reads `drift_score`, `report_path`, `next_workflow`, `upstream_moved` and `upstream_ref` from the final message. A result file that could not be written leaves `result_path` null and a `result_file_write_failed` warning in the line, and the run still finishes. If the emitter exits non-zero, correct `result-context.json` from the message on its stderr and run it once more; if it fails again, HALT with **exit 4**, `halt_reason: "write-failed"`, phase `report:result-contract`: the run's result contract could not be written.
|
|
200
200
|
|
|
201
201
|
**Hard-halt envelope (headless only).** Every hard halt prints its envelope through the emitter on **stderr** with `status: "error"` and the `exit_code` / `halt_reason` for its failure class, as each step's **Halt envelope** paragraph shows: it is the only failure signal a wrapping pipeline receives. `drift_score` carries its last known value (`null` if classification never ran); `report_path` is `null` when the report write failed.
|
|
202
202
|
|
|
@@ -11,7 +11,7 @@ auditDataFolder: '{forge_version}/.skf-audit/{timestamp}'
|
|
|
11
11
|
|
|
12
12
|
## STEP GOAL:
|
|
13
13
|
|
|
14
|
-
Compare what the skill documents about each export (retrieved from its QMD extraction collection) against the current source under `{source_root}` to detect meaning-level changes that structural diff cannot catch. This step executes ONLY at Deep tier: at Quick, Forge, and Forge+ tiers, it appends a skip notice and auto-proceeds.
|
|
14
|
+
Compare what the skill documents about each export (retrieved from its QMD extraction collection, or read from its own files when none is registered or the one registered is empty) against the current source under `{source_root}` to detect meaning-level changes that structural diff cannot catch. This step executes ONLY at Deep tier: at Quick, Forge, and Forge+ tiers, it appends a skip notice and auto-proceeds.
|
|
15
15
|
|
|
16
16
|
## Rules
|
|
17
17
|
|
|
@@ -48,16 +48,16 @@ Continue to section 2.
|
|
|
48
48
|
|
|
49
49
|
### 2. Query Original Knowledge Context
|
|
50
50
|
|
|
51
|
-
Run item 1 in the main thread, because only the main thread can end this step, then launch a subprocess (Pattern 3, data operations) for items 2 to 4:
|
|
51
|
+
Run item 1 in the main thread, because only the main thread can end this step or choose its route, then launch a subprocess (Pattern 3, data operations) for items 2 to 4:
|
|
52
52
|
1. Read the `qmd_collections` registry from `{sidecar_path}/forge-tier.yaml`. Find the entry where `skill_name` matches `{skill_name}` AND `type` is `"extraction"`. Three cases must be handled distinctly — collapsing them into "found vs. not found" silently degrades semantic diff when a collection is registered but never indexed.
|
|
53
53
|
|
|
54
|
-
- **Registry entry missing.**
|
|
54
|
+
- **Registry entry missing.** Whatever tier the skill was compiled at, a Deep-tier audit still checks its claims: log "No QMD extraction collection is registered for `{skill_name}`: semantic diff reads the skill's files instead.", bind `{collection_label}` ← `none (no QMD extraction collection is registered for {skill_name})` and take the **direct-content fallback** below. Never skip the step for it.
|
|
55
55
|
- **Registry entry present but collection empty.** Run a pre-query probe — `qmd ls {collection_name}` (CLI) or the equivalent MCP call. If it reports zero files (`Files: 0 (updated never)` or an empty listing), the collection is registered but has never been indexed. Do **not** proceed to querying — queries will return nothing and the step would silently degrade.
|
|
56
56
|
- Log: "QMD collection `{collection_name}` is registered but empty. Run `qmd update` to (re-)index `{collection.path}`, then re-audit for full Deep-tier semantic coverage."
|
|
57
|
-
-
|
|
58
|
-
- **Registry entry present and populated.** Use the `name` field from the registry entry as the collection to query
|
|
57
|
+
- Bind `{collection_label}` ← `{collection_name} (registered but empty)` and fall through to the **direct-content fallback** below instead of skipping outright.
|
|
58
|
+
- **Registry entry present and populated.** Use the `name` field from the registry entry as the collection to query, bind `{collection_label}` ← `{collection_name}`, `{claims_source}` ← `from QMD` and `{confidence}` ← `T2`, and proceed to bullet 2.
|
|
59
59
|
|
|
60
|
-
**Direct-content fallback** (used when
|
|
60
|
+
**Direct-content fallback** (used when no extraction collection is registered for the skill, or the one registered is empty): read what the skill documents about each export from `SKILL.md` and `references/*.md` of the audited skill in place of item 2's query, then run items 3 and 4 on those claims, checking the current source under `{source_root}` with the Deep-tier AST tooling this step already requires (ast_bridge; see step 2 §1 "Deep tier"). This fallback is reachable only from Deep tier: §1 short-circuits Quick/Forge/Forge+ before §2 runs, so AST tooling is guaranteed available here. Record findings with confidence label `T1-low-fallback` rather than T2: the claims come from reading the skill's files, not from its QMD collection. Bind `{claims_source}` ← `read from its SKILL.md and references/*.md in direct-content fallback mode` and `{confidence}` ← `T1-low-fallback`: §4's header then says the claims came from the skill's files and, in `{collection_label}`, why. The step's output schema is otherwise unchanged.
|
|
61
61
|
|
|
62
62
|
2. Query the collection for what the skill documents about each export (usage, conventions, dependencies, architecture): the original side
|
|
63
63
|
3. For each claim, read the export's current definition and call sites under `{source_root}` at the file and line `{forge_version}/extraction-snapshot.json` records for it: the current side (an export the snapshot no longer holds is a removed export step 3 already reports: skip it)
|
|
@@ -78,7 +78,7 @@ For each finding, record:
|
|
|
78
78
|
- What changed (description)
|
|
79
79
|
- Evidence (the current source file:line; a claim with no source line is not recorded)
|
|
80
80
|
- Affected exports
|
|
81
|
-
- Confidence: T2
|
|
81
|
+
- Confidence: `{confidence}` (`T2`, or `T1-low-fallback` in direct-content fallback mode)
|
|
82
82
|
|
|
83
83
|
### 4. Compile Semantic Drift Section
|
|
84
84
|
|
|
@@ -87,38 +87,38 @@ Append to {outputFile}:
|
|
|
87
87
|
```markdown
|
|
88
88
|
## Semantic Drift
|
|
89
89
|
|
|
90
|
-
**Method:** the skill's claims (
|
|
91
|
-
**QMD Collection:** {
|
|
90
|
+
**Method:** the skill's claims ({claims_source}) checked against the current source (Deep tier)
|
|
91
|
+
**QMD Collection:** {collection_label}
|
|
92
92
|
|
|
93
93
|
### New Patterns Detected ({count})
|
|
94
94
|
|
|
95
95
|
| Pattern | Description | Affected Exports | Evidence | Confidence |
|
|
96
96
|
|---------|------------|-----------------|----------|------------|
|
|
97
|
-
| {pattern} | {description} | {exports} | {evidence} |
|
|
97
|
+
| {pattern} | {description} | {exports} | {evidence} | {confidence} |
|
|
98
98
|
|
|
99
99
|
### Changed Conventions ({count})
|
|
100
100
|
|
|
101
101
|
| Convention | Before | After | Affected Exports | Evidence | Confidence |
|
|
102
102
|
|-----------|--------|-------|-----------------|----------|------------|
|
|
103
|
-
| {convention} | {old} | {new} | {exports} | {evidence} |
|
|
103
|
+
| {convention} | {old} | {new} | {exports} | {evidence} | {confidence} |
|
|
104
104
|
|
|
105
105
|
### Dependency Shifts ({count})
|
|
106
106
|
|
|
107
107
|
| Export | Original Dependencies | Current Dependencies | Change | Evidence | Confidence |
|
|
108
108
|
|--------|---------------------|---------------------|--------|----------|------------|
|
|
109
|
-
| {export} | {old_deps} | {new_deps} | {description} | {evidence} |
|
|
109
|
+
| {export} | {old_deps} | {new_deps} | {description} | {evidence} | {confidence} |
|
|
110
110
|
|
|
111
111
|
### Architectural Changes ({count})
|
|
112
112
|
|
|
113
113
|
| Change | Description | Affected Exports | Evidence | Confidence |
|
|
114
114
|
|--------|-------------|------------------|----------|------------|
|
|
115
|
-
| {change} | {description} | {exports} | {evidence} |
|
|
115
|
+
| {change} | {description} | {exports} | {evidence} | {confidence} |
|
|
116
116
|
|
|
117
117
|
### Deprecated Patterns ({count})
|
|
118
118
|
|
|
119
119
|
| Pattern | Documented In Skill | Current Status | Evidence | Confidence |
|
|
120
120
|
|---------|-------------------|----------------|----------|------------|
|
|
121
|
-
| {pattern} | {skill_reference} | {status} | {evidence} |
|
|
121
|
+
| {pattern} | {skill_reference} | {status} | {evidence} | {confidence} |
|
|
122
122
|
|
|
123
123
|
### Summary
|
|
124
124
|
|
|
@@ -132,7 +132,7 @@ Append to {outputFile}:
|
|
|
132
132
|
| **Total Semantic Items** | {total} |
|
|
133
133
|
```
|
|
134
134
|
|
|
135
|
-
Save the same rows to `{auditDataFolder}/semantic-findings.json`: step 5 classifies this file, not the tables. It is a JSON array with one object per row, `{"type": "semantic", "category", "name", "detail", "file", "line", "confidence"}`. `category` is its table's: New Patterns `pattern`, Changed Conventions `convention`, Dependency Shifts `dependency`, Architectural Changes `architecture` and Deprecated Patterns `deprecated_pattern`. `name` is the row's first cell, `detail` its description or change, `file` and `line` the current source line its evidence cites, and `confidence` the row's (T2
|
|
135
|
+
Save the same rows to `{auditDataFolder}/semantic-findings.json`: step 5 classifies this file, not the tables. It is a JSON array with one object per row, `{"type": "semantic", "category", "name", "detail", "file", "line", "confidence"}`. `category` is its table's: New Patterns `pattern`, Changed Conventions `convention`, Dependency Shifts `dependency`, Architectural Changes `architecture` and Deprecated Patterns `deprecated_pattern`. `name` is the row's first cell, `detail` its description or change, `file` and `line` the current source line its evidence cites, and `confidence` the row's `{confidence}` (`T2`, or `T1-low-fallback` in direct-content fallback mode).
|
|
136
136
|
|
|
137
137
|
### 5. Update Report and Auto-Proceed
|
|
138
138
|
|
|
@@ -69,7 +69,7 @@ With `-o` the helper saves the diff to the file and prints one line, `{"status":
|
|
|
69
69
|
- It starts `Cannot write output`: HALT with **exit 4**, `halt_reason: "write-failed"`, phase `structural-diff:diff`, `"path": "{auditDataFolder}/structural-diff.json"`.
|
|
70
70
|
- Otherwise it names `{provenanceMap}`, which step 1 §4 read: HALT with **exit 3**, `halt_reason: "provenance-invalid"`, phase `structural-diff:diff`, `"path": "{provenanceMap}"`, showing the `error`.
|
|
71
71
|
|
|
72
|
-
The helper canonicalizes both sides before it matches each export by its name and its file (the quote style of string defaults, stdlib module prefixes, and a renamed public re-export resolved through the re-export map it derives from the provenance map), so a cosmetic extractor difference never reads as drift. It records each transform it applied in `applied_transforms`, and takes `--reexport-map {file}` only to override the derived map.
|
|
72
|
+
The helper canonicalizes both sides before it matches each export by its name and its file (the quote style of string defaults, stdlib module prefixes, and a renamed public re-export resolved through the re-export map it derives from the provenance map), and, for a matched export, takes a semantic kind a by-eye read records as the runner's base kind when the runner's signature shows that form (`async_function` or `decorator` as `function`, `enum` as `class`), so a cosmetic extractor difference never reads as drift. It records each transform it applied in `applied_transforms`, and takes `--reexport-map {file}` only to override the derived map.
|
|
73
73
|
|
|
74
74
|
The saved diff holds every set §5 renders. This step reads three of its fields itself: §1b reads `summary.removed` and the `name` of each `removed[]` entry (an export in the provenance map but not in the snapshot), and §6 records `applied_transforms` (each transform the helper applied, with its count) in the drift report's frontmatter for step 6's Provenance table.
|
|
75
75
|
|
|
@@ -95,33 +95,38 @@ When `added` is above 0, run §1's command again with the same arguments and the
|
|
|
95
95
|
|
|
96
96
|
### 2. What the Diff Decides
|
|
97
97
|
|
|
98
|
-
Every set comes straight from the saved diff, never from set arithmetic of your own.
|
|
98
|
+
Every set comes straight from the saved diff, never from set arithmetic of your own. Four of its facts steer what later steps judge:
|
|
99
99
|
|
|
100
100
|
- **Moved** (`moved[]`): a matched export whose file changed. A move is **not** a removal.
|
|
101
101
|
- **Ambiguous names** (`ambiguous_names[]`): a name left on both sides that occurs more than once on a side (a `GET` handler in several route files, for example), so the helper paired none of its entries. They stay in `removed[]` and `added[]`, and step 5 judges whether a removed and an added entry of one name are one export that moved.
|
|
102
|
+
- **Off the public surface** (`not_public[]`): for a public-api skill, step 2 marked each export by the public surface create-skill and update-skill keep its map to, and an export marked `public: false` that pairs with no map entry is listed here, not in `added[]`, since neither workflow documents it. It is not drift: it is not counted in Total Drift Items or the exit code, and step 5 never classifies it (`--from-diff` reads only `added[]`, `removed[]`, `changed[]` and `moved[]`). §5's renderer counts it in an **Off the public surface:** note. A name in `ambiguous_names[]` stays in `added[]` for step 5 to pair, and a map entry is matched whatever its mark, so none reads as removed.
|
|
102
103
|
- **Not drift:** `summary.signature_unverified` counts matched exports whose signature sits in different fields on the two sides, so a change there cannot be seen, and `label_changes[]` lists matched exports whose `confidence` or `extraction_method` differs, which names the tool that read the export, not what the source says. Neither is counted in Total Drift Items, and step 5 classifies neither.
|
|
103
104
|
|
|
104
105
|
### 4b. Detect Script/Asset Drift
|
|
105
106
|
|
|
106
107
|
**Only execute if provenance-map.json contains `file_entries`.**
|
|
107
108
|
|
|
108
|
-
**Resolve `{compareFileHashesHelper}`** from `{compareFileHashesProbeOrder}`; first existing path wins. Run one comparison: it hashes the tracked `file_entries[]` and, the other way round, the
|
|
109
|
+
**Resolve `{compareFileHashesHelper}`** from `{compareFileHashesProbeOrder}`; first existing path wins. Run one comparison: it hashes the tracked `file_entries[]` and, the other way round, walks the source tree for the files the map does not track, with the script and asset detector create-skill and update-skill use, plus the documents under `docs/authoritative/`, keeping only those the skill brief's scope takes, by the scope test update-skill and create-skill use. A module of a Python package named like a script folder, a file the map's `entries[]` cite, a file under `scripts/[MANUAL]/` or `assets/[MANUAL]/`, and a script or an asset whose `scripts_intent` or `assets_intent` is `none` are never new files. Pass `--brief` only when that brief, `{forge_data_folder}/{skill_name}/skill-brief.yaml`, exists (a skill built without one, such as a quick skill, has none):
|
|
109
110
|
|
|
110
111
|
```bash
|
|
111
|
-
uv run {compareFileHashesHelper} compare "{provenanceMap}" "{source_root}"
|
|
112
|
+
uv run {compareFileHashesHelper} compare "{provenanceMap}" "{source_root}" \
|
|
113
|
+
[--brief "{forge_data_folder}/{skill_name}/skill-brief.yaml"] > "{auditDataFolder}/file-drift.json"
|
|
112
114
|
```
|
|
113
115
|
|
|
114
116
|
The saved JSON, which step 5 builds its Script/Asset findings from:
|
|
115
117
|
|
|
116
118
|
```
|
|
117
119
|
{
|
|
118
|
-
"added": ["<rel-path>", ...], //
|
|
120
|
+
"added": ["<rel-path>", ...], // a script or an asset to the detector, or a docs/authoritative/ file, on disk, in the brief's scope, NOT in file_entries
|
|
121
|
+
"added_not_checked": null, // or why no new file was looked for: "no skill brief", the brief's error, or a helper it cannot load
|
|
119
122
|
"removed": ["<rel-path>", ...], // in file_entries, missing on disk
|
|
120
123
|
"changed": [{"path": "...", "stored_hash": "sha256:...", "current_hash": "sha256:..."}],
|
|
121
124
|
"stats": {"added": N, "removed": N, "changed": N, "unchanged": N}
|
|
122
125
|
}
|
|
123
126
|
```
|
|
124
127
|
|
|
128
|
+
**When `added_not_checked` is not null** (`no skill brief` without `--brief`, the error of a brief the helper could not read, or `cannot load ... re-install SKF` when a helper beside it could not be loaded), no new file was looked for, so `added` is empty, while the tracked files were still compared: §5's renderer prints the reason under the Script/Asset Drift table. Record it for the envelope too: write `new_files_not_checked: {added_not_checked}` to `{run_dir}/warning.txt` with a file write (the reason can hold quotes), then, from `{project-root}`, run `uv run {emitEnvelopeHelper} record --run-dir "{run_dir}" --warning "$(cat "{run_dir}/warning.txt")"`.
|
|
129
|
+
|
|
125
130
|
**When no candidate resolves, or the command exits non-zero** (delete the file then), skip the check with a `### Script/Asset Drift: skipped ({the reason})` note: it is supplementary to the export diff, so it never halts the audit.
|
|
126
131
|
|
|
127
132
|
### Stack-Specific Structural Diff
|
|
@@ -31,6 +31,11 @@ structural DIFF [--file-drift FILE]
|
|
|
31
31
|
**Total Drift Items**, their sum
|
|
32
32
|
**Signatures not compared:** (only when summary.signature_unverified
|
|
33
33
|
is above 0)
|
|
34
|
+
**Off the public surface:** (only when summary.not_public is above 0:
|
|
35
|
+
the exports of a public-api skill the
|
|
36
|
+
diff left out of Added Exports, which are
|
|
37
|
+
not drift; the first ten of not_public[]
|
|
38
|
+
as `name` (`file`), then `and N more`)
|
|
34
39
|
### By Library a stack diff (--group-by source_library):
|
|
35
40
|
each groups[] entry's counts
|
|
36
41
|
### Script/Asset Drift (added N, removed N, changed N)
|
|
@@ -40,7 +45,11 @@ structural DIFF [--file-drift FILE]
|
|
|
40
45
|
given and exists; when FILE cannot be read
|
|
41
46
|
or is not such a result, the heading reads
|
|
42
47
|
`### Script/Asset Drift: skipped (<reason>)`
|
|
43
|
-
and the other tables still print)
|
|
48
|
+
and the other tables still print). When
|
|
49
|
+
FILE's `added_not_checked` names a reason
|
|
50
|
+
(no brief, or one the comparison could not
|
|
51
|
+
read), a **New files not checked:** line
|
|
52
|
+
follows the table
|
|
44
53
|
### Provenance label differences (not drift) (N)
|
|
45
54
|
Export | Baseline label | Current label
|
|
46
55
|
(only when label_changes[] is non-empty)
|
|
@@ -121,6 +130,8 @@ from pathlib import Path
|
|
|
121
130
|
ROLLUP_MIN = 10
|
|
122
131
|
# Representative names a rollup row shows.
|
|
123
132
|
REP_COUNT = 3
|
|
133
|
+
# The exports the Off the public surface note names, as update-skill's not_public warning does.
|
|
134
|
+
NOT_PUBLIC_NAMED = 10
|
|
124
135
|
SEVERITIES = ("CRITICAL", "HIGH", "MEDIUM", "LOW")
|
|
125
136
|
NONE = "(none)"
|
|
126
137
|
NA = "n/a"
|
|
@@ -400,7 +411,12 @@ def _file_drift_section(data) -> str:
|
|
|
400
411
|
text(details[0]) if len(details) == 1 else NA])
|
|
401
412
|
heading = (f"### Script/Asset Drift (added {_count(stats, 'added')}, removed {_count(stats, 'removed')}, "
|
|
402
413
|
f"changed {_count(stats, 'changed')})")
|
|
403
|
-
|
|
414
|
+
section = heading + "\n\n" + _table(("File", "Change", "Detail"), rows)
|
|
415
|
+
reason = data.get("added_not_checked")
|
|
416
|
+
if isinstance(reason, str) and reason.strip():
|
|
417
|
+
section += (f"\n\n**New files not checked:** {text(reason.strip())}. The tracked files were compared, but "
|
|
418
|
+
"no new script, asset or document was looked for, so `added` lists none.")
|
|
419
|
+
return section
|
|
404
420
|
|
|
405
421
|
|
|
406
422
|
def render_structural(diff, file_drift=None, file_drift_skipped=None) -> str:
|
|
@@ -455,6 +471,17 @@ def render_structural(diff, file_drift=None, file_drift_skipped=None) -> str:
|
|
|
455
471
|
if unverified:
|
|
456
472
|
block += (f"\n\n**Signatures not compared:** {unverified} matched exports hold their signature in "
|
|
457
473
|
"different fields on the two sides, so a change there cannot be seen.")
|
|
474
|
+
not_public = _count(summary, "not_public")
|
|
475
|
+
if not_public:
|
|
476
|
+
one = not_public == 1
|
|
477
|
+
named = [f"{export_cell(r)} ({where(r.get('file'))})"
|
|
478
|
+
for r in _records(diff, "not_public")[:NOT_PUBLIC_NAMED]]
|
|
479
|
+
more = f" and {not_public - len(named)} more" if named and not_public > len(named) else ""
|
|
480
|
+
listed = f": {', '.join(named)}{more}" if named else ""
|
|
481
|
+
block += (f"\n\n**Off the public surface:** {not_public} {'export' if one else 'exports'} the provenance "
|
|
482
|
+
f"map does not hold {'is' if one else 'are'} off this public-api skill's public surface "
|
|
483
|
+
f"(`not_public[]` in the saved diff), so {'it is' if one else 'they are'} not reported as added "
|
|
484
|
+
f"and {'is' if one else 'are'} excluded from Total Drift Items{listed}.")
|
|
458
485
|
parts.append(block)
|
|
459
486
|
|
|
460
487
|
groups = _records(diff, "groups")
|
|
@@ -32,7 +32,7 @@ You are a skill scoping architect collaborating with a developer who wants to cr
|
|
|
32
32
|
These rules apply to every step in this workflow:
|
|
33
33
|
|
|
34
34
|
- Only load one step file at a time — never preload future steps
|
|
35
|
-
- **Lazy-load references and assets:** `references/*.md` and `assets/*.md` files are loaded inside the section that needs them, not at step entry. If a section is skipped (e.g. `version-resolution.md` when
|
|
35
|
+
- **Lazy-load references and assets:** `references/*.md` and `assets/*.md` files are loaded inside the section that needs them, not at step entry. If a section is skipped (e.g. `version-resolution.md` when step 2's extractor ran, `scope-templates.md` for the `docs-only` branch that bypasses §2c), do not load that file. Each unnecessary load costs context (~5-10 KB per reference) and biases the LLM toward consulting material the current path does not need.
|
|
36
36
|
- Always communicate in `{communication_language}` (the language for user-facing prose). Written artifact text — the `description`, `notes`, and other free-form fields persisted into `skill-brief.yaml` — is in `{document_output_language}`; per-step rules call this out where it applies (see step 5). The two values may be the same.
|
|
37
37
|
- If `{headless_mode}` is true, auto-proceed through confirmation gates with their default action and log each auto-decision
|
|
38
38
|
- Keep one list of the run's warnings, `workflow_warnings[]`, in the order they are raised: each `warn:` line a step logs, each entry of a helper's `warnings[]` that a step surfaces, and each degraded signal the run goes on past (a truncated tree listing, a failed doc detection or QMD step), one line of text each. A helper warning that is an object, such as the `{field, message}` entries skf-validate-brief-inputs.py and skf-validate-brief-schema.py return, goes in as one line, `<field>: <message>`: the emitter takes only strings, and one object in `warnings` makes it exit non-zero and print no line. The run's `SKF_BRIEF_RESULT_JSON` envelope carries the list as `warnings`, at the end of a successful run and at a halt
|
|
@@ -133,4 +133,4 @@ Prompt: "Does this component library scope look right? Adjust before continuing.
|
|
|
133
133
|
|
|
134
134
|
## Scripts & Assets Detection (Optional Refinement)
|
|
135
135
|
|
|
136
|
-
When `scripts_intent` or `assets_intent` is `detect` (default), SKF auto-detects from source directories matching: `scripts/`, `bin/`, `tools/`, `cli/` (for scripts) and `assets/`, `templates/`, `schemas/`, `configs/`, `examples/` (for assets). Detection applies to all scope types except `docs-only`.
|
|
136
|
+
When `scripts_intent` or `assets_intent` is `detect` (default), SKF auto-detects from source directories matching: `scripts/`, `bin/`, `tools/`, `cli/` (for scripts; when one of these four folders is a Python package, holding `__init__.py`, only its `__main__.py` and the modules with a shebang or a `__main__` block count) and `assets/`, `templates/`, `schemas/`, `configs/`, `examples/` (for assets). Detection applies to all scope types except `docs-only`.
|
|
@@ -39,16 +39,9 @@ When `source_type: "docs-only"`:
|
|
|
39
39
|
|
|
40
40
|
## Version Detection
|
|
41
41
|
|
|
42
|
-
During brief creation,
|
|
42
|
+
During brief creation, step 2 auto-detects the source version (`references/version-resolution.md`, Detection Algorithm) before defaulting to `"1.0.0"`.
|
|
43
43
|
|
|
44
|
-
|
|
45
|
-
- **JavaScript/TypeScript:** root `package.json` (`"version"`) → if root has `"private": true` with a `"workspaces"` array or lacks a `"version"` field, fall back to a primary workspace package's `package.json` (e.g., `code/core/package.json`, or the first matching `packages/*/package.json`). For GitHub sources, prefer `gh api repos/{owner}/{repo}/releases/latest` → `tag_name` when a non-pre-release tag exists, over a default-branch pre-release. Treat a version containing `-alpha`, `-beta`, `-rc`, `-next`, or `-canary` as a pre-release.
|
|
46
|
-
- **Rust:** `Cargo.toml` `[package] version` (static) → if `version = { workspace = true }`, resolve from workspace root `Cargo.toml` → `git describe --tags --abbrev=0`
|
|
47
|
-
- **Go:** version tag from `go.mod` or `git describe --tags --abbrev=0`
|
|
48
|
-
|
|
49
|
-
If the source is a remote GitHub repo, read the version file step 2 fetched into the run folder with `skf-github-fetch.py`, which needs `gh` only for a private repository. If the source is local, read the file directly.
|
|
50
|
-
|
|
51
|
-
If detection succeeds, use the detected version. If it fails or returns a non-semver value, fall back to `"1.0.0"`.
|
|
44
|
+
If detection succeeds, use the detected version. If it fails or returns a non-semver value, fall back to `"1.0.0"`; the brief writer's result then carries a warning that says so.
|
|
52
45
|
|
|
53
46
|
The create-skill workflow (extract) also performs version reconciliation at extraction time — if the source version has changed since the brief was created, the extraction step warns and uses the source version.
|
|
54
47
|
|