bmad-module-skill-forge 1.9.0 → 2.0.0

package/src/shared/scripts/skf-validate-brief-schema.py

@@ -5,7 +5,7 @@
 """SKF Validate Brief Schema — schema check for a skill-brief.yaml on disk.
 
 Loads `skill-brief.yaml` from a path (or from stdin/inline-YAML), validates
-it against `src/shared/scripts/schemas/skill-brief.v1.json`, and applies the
+it against `schemas/skill-brief.v1.json` (sibling of this script), and applies the
 two conditional rules from `skf-create-skill/references/load-brief.md §3`
 that the JSON schema doesn't express today:
 
@@ -56,10 +56,7 @@ import yaml
 from jsonschema import Draft202012Validator
 
 
-REPO_ROOT = Path(__file__).resolve().parent.parent.parent.parent
-SCHEMA_PATH = (
-    REPO_ROOT / "src" / "shared" / "scripts" / "schemas" / "skill-brief.v1.json"
-)
+SCHEMA_PATH = Path(__file__).resolve().parent / "schemas" / "skill-brief.v1.json"
 
 
 # --------------------------------------------------------------------------

package/src/shared/scripts/skf-validate-pins.py

@@ -5,7 +5,7 @@
 """SKF Validate Pins — resolve and validate version pins for a GitHub repository.
 
 Shared pin validation script consumed by skf-analyze-source (AN auto-scope,
-deepwiki --pin) and campaign workflows.  Validates that a user-supplied --pin
+forge-auto --pin) and campaign workflows.  Validates that a user-supplied --pin
 resolves to an existing tag or branch, or auto-resolves the latest release tag
 when no --pin is provided.
 
@@ -17,7 +17,7 @@ Tag matching priority mirrors source-resolution-protocols.md:
   4. Crate prefix:  {name}/v{pin}, {name}/{pin}, {name}-v{pin}
 
 CLI:
-  uv run python src/shared/scripts/skf-validate-pins.py \\
+  uv run src/shared/scripts/skf-validate-pins.py \\
       --repo-url <url> [--pin <version>] [--format tag|branch|any]
 
 Input:

package/src/shared/scripts/skf-write-skill-brief.py

@@ -51,7 +51,10 @@ Context payload shape (consumed by `write`):
     },
 
     # Conditionally present:
-    "doc_urls":         [{"url": "...", "label": "..."}],
+    "doc_urls":         [{"url": "...", "label": "...", "source": "..."}],
+    #   `source` (optional per #432): language-registry | readme-detection |
+    #   homepage | pages-api | docs-folder. Threaded through when present;
+    #   absent → entry renders as {url, label} only (no false drift).
     "scripts_intent":   "detect" | "none" | free-text,
     "assets_intent":    "detect" | "none" | free-text,
     "source_authority": "official" | "community" | "internal",
@@ -95,7 +98,7 @@ Flat input form (`--from-flat`):
     "scope_notes":          "",
     "scope_tier_a_include": null | ["code/core/src/**"],
     "scope_amendments":     null | [{...}],
-    "doc_urls":             null | [...],
+    "doc_urls":             null | [{"url": "...", "label": "...", "source": "..."}],
     "scripts_intent":       null | "detect" | "none" | "...",
     "assets_intent":        null | "detect" | "none" | "...",
     "source_authority":     null | "official" | "community" | "internal",
@@ -433,11 +436,24 @@ def assemble_brief(ctx: dict[str, Any], resolved_version: str) -> dict[str, Any]
             brief[ref_field] = ref_val
 
     # Conditional: doc_urls (always emitted when present)
+    #
+    # Each entry carries the {url, label} contract plus an OPTIONAL `source`
+    # provenance field (issue #432: language-registry | readme-detection |
+    # homepage | pages-api | docs-folder). `source` is emitted ONLY when present
+    # on the input entry — mirroring the source_authority null-drop discipline
+    # below (lines ~448-454): a legacy/hand-authored entry without `source`
+    # round-trips to exactly {url, label}, so a ratify/re-write of an old brief
+    # stays byte-identical. (An explicit empty-string source is treated as
+    # absent here; the schema enum is the real gate and rejects "" upstream.)
     doc_urls = ctx.get("doc_urls")
     if doc_urls:
-        brief["doc_urls"] = [
-            {"url": e["url"], "label": e.get("label", "")} for e in doc_urls
-        ]
+        rendered_doc_urls: list[dict[str, Any]] = []
+        for e in doc_urls:
+            entry = {"url": e["url"], "label": e.get("label", "")}
+            if e.get("source"):
+                entry["source"] = e["source"]
+            rendered_doc_urls.append(entry)
+        brief["doc_urls"] = rendered_doc_urls
 
     # Conditional: scripts_intent / assets_intent — emit when explicitly non-detect
     for intent_field in ("scripts_intent", "assets_intent"):

package/src/skf-analyze-source/SKILL.md

@@ -43,7 +43,7 @@ These rules apply to every step in this workflow:
 | 6 | Generate Briefs | references/generate-briefs.md | Yes | Interactive mode only |
 | 7 | Workflow Health Check | references/health-check.md | Yes | Always |
 
-**Auto mode path:** When `[auto]` flag is present, init (step 1) routes directly to step 1a, which performs manifest scan → shape detection → scope generation → brief write → health check, bypassing steps 2–6. After URL type detection (§0), pin resolution (§0b) validates the `--pin` argument (or resolves the latest release tag) and stores the resolved ref for downstream brief writes. Coexistence detection (§0c) then checks for existing skills matching the target. If found, the user chooses alongside/merge/skip. Headless mode auto-selects alongside. Auto-scope may produce N > 1 confirmed units when decomposition thresholds are met (`export_count > 500` `[PENDING VALIDATION]` or `package_count > 3` `[PENDING VALIDATION]`), resulting in N briefs and N `brief_paths` in the envelope. When the target is a documentation URL (not a GitHub repo or local path), auto-scope detects the docs-only input at §0, validates URL reachability, writes a docs-only brief, and emits the envelope without performing source analysis.
+**Auto mode path:** When `[auto]` flag is present, init (step 1) routes directly to step 1a, which performs manifest scan → shape detection → scope generation → brief write → health check, bypassing steps 2–6. After URL type detection (§0), pin resolution (§0b) validates the `--pin` argument (or resolves the latest release tag) and stores the resolved ref for downstream brief writes. Coexistence detection (§0c) then checks for existing skills matching the target. If found, the user chooses alongside/merge/skip. Headless mode auto-selects alongside. Auto-scope may produce N > 1 confirmed units when a monorepo crosses the decomposition threshold (`package_count > 3`) **and** the §3b cohesion check decides to split, resulting in N briefs and N `brief_paths` in the envelope; a cohesive monorepo merges to one skill instead. A single large library is never decomposed — its size is handled by `skf-create-skill`'s auto-shard. When the target is a documentation URL (not a GitHub repo or local path), auto-scope detects the docs-only input at §0, validates URL reachability, writes a docs-only brief, and emits the envelope without performing source analysis.
 
 **Shape detection reference:** `references/step-shape-detect.md` — loaded by step 1a as a reference doc (not a chained step).
 

package/src/skf-analyze-source/assets/skill-brief-schema.md

@@ -23,7 +23,7 @@ Defines the output contract for skill-brief.yaml files generated by analyze-sour
 | Field              | Type   | Constraint                                       | Description                                                                                                                                                                                                                    |
 |--------------------|--------|--------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | source_type        | string | `source` or `docs-only`                          | Default `source`. When `docs-only`: `source_repo` optional, `doc_urls` required                                                                                                                                                |
-| doc_urls           | array  | `{url, label}` objects                           | Documentation URLs for T3 content. Required when `source_type: "docs-only"`                                                                                                                                                    |
+| doc_urls           | array  | `{url, label, source?}` objects                  | Documentation URLs for T3 content. Required when `source_type: "docs-only"`. Optional `source` provenance enum: `language-registry` (registry-guaranteed corpus) \| `readme-detection` \| `homepage` \| `pages-api` \| `docs-folder` |
 | `scripts_intent`   | string | `detect` / `none` / free-text                    | Describes whether scripts should be extracted. Values: `detect` (auto-detect from source — default when absent), `none` (skip scripts), or a free-text description of expected scripts (e.g., "CLI validation tools in bin/"). |
 | `assets_intent`    | string | `detect` / `none` / free-text                    | Describes whether assets should be extracted. Values: `detect` (auto-detect from source — default when absent), `none` (skip assets), or a free-text description of expected assets (e.g., "JSON schemas in schemas/").        |
 | `target_version`   | string | Semantic version (`X.Y.Z` or `X.Y.Z-prerelease`) | User-specified target version. When present, overrides auto-detection and becomes the skill's version. Recommended for docs-only skills where auto-detection is unavailable.                                                   |

package/src/skf-analyze-source/references/step-auto-scope.md

@@ -1,7 +1,27 @@
 ---
 nextStepFile: 'health-check.md'
 outputFile: '{forge_data_folder}/analyze-source-report-{project_name}.md'
-shapeDetectScript: 'src/shared/scripts/skf-shape-detect.py'
+shapeDetectProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-shape-detect.py'
+  - '{project-root}/src/shared/scripts/skf-shape-detect.py'
+validatePinsProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-validate-pins.py'
+  - '{project-root}/src/shared/scripts/skf-validate-pins.py'
+skillInventoryProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-skill-inventory.py'
+  - '{project-root}/src/shared/scripts/skf-skill-inventory.py'
+scanManifestsProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-scan-manifests.py'
+  - '{project-root}/src/shared/scripts/skf-scan-manifests.py'
+detectLanguageProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-detect-language.py'
+  - '{project-root}/src/shared/scripts/skf-detect-language.py'
+languageCorporaProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-language-corpora.py'
+  - '{project-root}/src/shared/scripts/skf-language-corpora.py'
+writeSkillBriefProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-write-skill-brief.py'
+  - '{project-root}/src/shared/scripts/skf-write-skill-brief.py'
 ---
 
 <!-- Config: communicate in {communication_language}. -->
@@ -48,8 +68,10 @@ This section validates and resolves version pins. It runs for repo URLs and loca
 
 **For repo URLs when `--pin` is provided:**
 
+**Resolve `{validatePinsHelper}`** from `{validatePinsProbeOrder}`; first existing path wins; HALT if neither resolves.
+
 ```bash
-uv run python src/shared/scripts/skf-validate-pins.py --repo-url {project_path} --pin {pin_value}
+uv run {validatePinsHelper} --repo-url {project_path} --pin {pin_value}
 ```
 
 Handle exit codes:
@@ -63,8 +85,10 @@ Handle exit codes:
 
 **For repo URLs when `--pin` is NOT provided (default):**
 
+Using the same `{validatePinsHelper}` resolved above:
+
 ```bash
-uv run python src/shared/scripts/skf-validate-pins.py --repo-url {project_path}
+uv run {validatePinsHelper} --repo-url {project_path}
 ```
 
 Handle exit codes:
@@ -79,8 +103,10 @@ This section checks for existing skills matching the target before proceeding. I
 
 **1. Load skill inventory:**
 
+**Resolve `{skillInventoryHelper}`** from `{skillInventoryProbeOrder}`; first existing path wins; HALT if neither resolves.
+
 ```bash
-uv run python src/shared/scripts/skf-skill-inventory.py {skills_output_folder}
+uv run {skillInventoryHelper} {skills_output_folder}
 ```
 
 Parse the JSON output. If the exit code is non-zero or the `skills` array is empty, skip coexistence detection silently (no existing skills to conflict with) and continue to the next section: §0a for documentation URLs, §1 for all other input types.
@@ -183,9 +209,11 @@ confirmed_units:
 
 **4. Write skill brief via canonical writer:**
 
+**Resolve `{writeSkillBriefHelper}`** from `{writeSkillBriefProbeOrder}`; first existing path wins; HALT if neither resolves.
+
 Create directory `{forge_data_folder}/{skill_name}/` if it does not exist.
 
-Write `{forge_data_folder}/{skill_name}/skill-brief.yaml` through the canonical writer (`skf-write-skill-brief.py`) with the following context:
+Pipe the flat context JSON below into the resolved writer with the `--from-flat` flag:
 
 ```json
 {
@@ -194,7 +222,7 @@ Write `{forge_data_folder}/{skill_name}/skill-brief.yaml` through the canonical
   "detected_version": null,
   "source_type":      "docs-only",
   "source_repo":      "{url}",
-  "language":         "",
+  "language":         "documentation",
   "description":      "Skill created from documentation at {url}",
   "forge_tier":       "{forge_tier}",
   "created":          "{current_date}",
@@ -216,6 +244,10 @@ Write `{forge_data_folder}/{skill_name}/skill-brief.yaml` through the canonical
 }
 ```
 
+```bash
+echo '<context-json>' | uv run {writeSkillBriefHelper} write --target {forge_data_folder}/{skill_name}/skill-brief.yaml --from-flat
+```
+
 **5. Emit success envelope:**
 
 ```
@@ -245,27 +277,79 @@ Load `references/step-shape-detect.md` as reference for shape detection invocati
 
 ### 2. Manifest Scan
 
-Perform a lightweight manifest scan — find standard package manifests in the project root and workspace paths. Do NOT crawl the full directory tree.
+Enumerate package manifests **deterministically** via `{scanManifestsHelper}` (the same helper the interactive `scan-project.md` uses) — do not hand-scan. Resolve `{scanManifestsHelper}` as the first path in `{scanManifestsProbeOrder}` that exists. The scanner reads a **local directory**, so how you point it at the target depends on the input form classified in §0:
 
 **For each path in `project_paths[]`:**
 
-1. Check the project root for: `package.json`, `pyproject.toml`, `Cargo.toml`
-2. If a workspace configuration exists (e.g., `pnpm-workspace.yaml`, Cargo.toml `[workspace].members`), scan workspace member paths for additional manifests
-3. Record each discovered manifest as `{path, type}` pairs
+- **Local filesystem path** (starts with `/`, `./`, `~/`, `~`, or is an existing directory) — scan it directly:
+
+  ```bash
+  uv run {scanManifestsHelper} scan {path}
+  ```
+
+- **Remote git URL** (e.g. `github.com/{owner}/{repo}`) — auto-scope has no working tree yet and the scanner cannot read a URL. Fetch **just the manifests** first (blobless + sparse + depth-1 — no source blobs, typically KB–MB even for large monorepos), then scan that tree:
 
-**IF no manifests found:**
-- Emit fallback message: "**Auto-scope could not find any package manifests — switching to interactive mode.**"
-- Load, read fully, then execute `references/scan-project.md`. **STOP HERE.**
+  ```bash
+  tmp="$(mktemp -d)"
+  git clone --filter=blob:none --no-checkout --depth 1 {pinned_branch_flag} {path} "$tmp"
+  git -C "$tmp" sparse-checkout set --no-cone '**/package.json' '**/Cargo.toml' '**/pyproject.toml' '**/go.mod' '**/pom.xml' '**/build.gradle' '**/build.gradle.kts' '**/Package.swift' 'pnpm-workspace.yaml' '**/pnpm-workspace.yaml'
+  git -C "$tmp" checkout
+  uv run {scanManifestsHelper} scan "$tmp"
+  ```
+
+  where `{pinned_branch_flag}` is `--branch {pinned_ref}` when a pin was resolved in §0b (so manifests match the target version), otherwise omitted. **Retain `"$tmp"` through §3** — shape detection reads the discovered manifest files from it — then it may be discarded.
+
+Parse the JSON envelope: `{manifests: [{path, ecosystem, ...}], total_unique, monorepo, warnings?}`. The scanner discovers the project root plus monorepo workspace members (npm/pnpm/yarn `workspaces`, Cargo `[workspace]`, and other ecosystems) and sets the `monorepo` flag — so members are found without hand-listing each workspace convention, for both local trees and remote fetches.
+
+From the envelope, record:
+
+1. **Supported manifest paths** — filter `manifests[].path` to the types `skf-shape-detect.py` accepts (`package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `pom.xml`, `build.gradle`, `build.gradle.kts`, `Package.swift`). Each `manifests[].path` is **relative to the scan root**, so resolve them against that root (`{path}` for a local scan, `"$tmp"` for a remote fetch) before use. This filtered, comma-joined list of resolved paths is fed to shape detection in §3. For a monorepo, it includes each workspace member's manifest, so the package surface is classified accurately rather than from a bare (and often export-less) repo root. The scanner may discover ecosystems shape detection does not yet classify; those are excluded here, so a repo with no supported manifest falls back to interactive at the next check rather than auto-scoping.
+2. **`monorepo` flag** and the count of discovered supported packages — carried forward as a signal for the decomposition decision in §3a.
+
+**Harvest tree-level language signals.** A whole-language repo may declare no parser-generator dependency (a hand-written compiler such as rustc, TypeScript, or the Go toolchain) or carry no supported manifest at all (CPython, Ruby). From the **same** fetched tree — no second clone and no blobs, since tree objects are already present in the blobless clone — collect two signals for shape detection. These are pure path listings (`git ls-tree` reads tree objects; no checkout, no blob download):
+
+- **Remote fetch** (`"$tmp"`), or a **local path** that is a git repo (`git -C {path}`):
+  ```bash
+  files="$(git -C "$tmp" ls-tree -r    --name-only HEAD)"   # every file path
+  dirs="$( git -C "$tmp" ls-tree -r -d --name-only HEAD)"   # every directory
+  # Grammar files (depth-capped to skip deep vendored fixtures, hard-capped):
+  grammar_matches="$(printf '%s\n' "$files" \
+    | grep -Ei '\.(g4|pest|lalrpop|y|gram|lark|ebnf|peg|ungram)$|/grammar\.(js|json)$' \
+    | awk -F/ 'NF<=4' | head -n 50 | paste -sd, -)"
+  # Directory signals (trailing /) + depth-capped file basenames, narrowed to
+  # compiler-relevant paths so the argument stays bounded on huge repos. The
+  # filter is a loose superset of shape detection's gates — the script does the
+  # precise matching; this only keeps the list small.
+  tree_paths="$({ printf '%s\n' "$dirs" | sed 's#$#/#'; \
+                  printf '%s\n' "$files" | awk -F/ 'NF<=5'; } \
+    | grep -Ei '(^|/)(compiler|compile|syntax|scanner|lexer|tokeniz|parser|parse|ast|binder|checker|codegen|ssagen|interpreter|vm|eval|rustc_[a-z]+)' \
+    | head -n 400 | paste -sd, -)"
+  ```
+- **Local path that is not a git repo** (`{path}`): list the tree with `find` instead, then derive `grammar_matches` / `tree_paths` the same way:
+  ```bash
+  files="$(cd {path} && find . -type f -not -path '*/.git/*' | sed 's#^\./##')"
+  dirs="$( cd {path} && find . -type d -not -path '*/.git/*' | sed 's#^\./##')"
+  ```
+
+Record `<grammar_matches>` and `<tree_paths>` (each a comma-joined list, possibly empty) for §3.
+
+**IF no supported manifests are found** (the filtered list is empty):
+- **AND** `<grammar_matches>` is empty **AND** `<tree_paths>` shows no compiler directory (none of `compiler/`, `src/compiler/`, `cmd/compile/`, `internal/syntax/`, or a `Parser/`): the repo carries no language signal — emit fallback message: "**Auto-scope could not find any supported package manifests — switching to interactive mode.**" Load, read fully, then execute `references/scan-project.md`. **STOP HERE.**
+- **Otherwise** (a grammar file or a compiler directory is present) the repo is a manifest-less language toolchain (CPython, Ruby): proceed to §3 with an **empty** `--manifests` and the harvested `--grammar-files` / `--tree-paths`.
 
 ### 3. Invoke Shape Detection
 
-Invoke the shape detection script with discovered manifests:
+**Resolve `{shapeDetectHelper}`** from `{shapeDetectProbeOrder}`; first existing path wins; HALT if neither resolves.
+
+Invoke the shape detection script with the discovered manifests and the harvested tree-level signals:
 
 ```
-uv run python {shapeDetectScript} --repo-url <project_path_or_url> --manifests <comma_separated_manifest_paths>
+uv run {shapeDetectHelper} --repo-url <project_path_or_url> \
+  --manifests <comma_separated_manifest_paths> \
+  --grammar-files <grammar_matches> --tree-paths <tree_paths>
 ```
 
-Parse the JSON output: `{shape, signals, confidence, export_count, package_count}`
+`<comma_separated_manifest_paths>` may be empty for a manifest-less language repo, provided `<grammar_matches>` or `<tree_paths>` carries the signal. Parse the JSON output: `{shape, signals, confidence, export_count, package_count}`
 
 **Handle exit codes:**
 
@@ -278,19 +362,41 @@ Parse the JSON output: `{shape, signals, confidence, export_count, package_count
 
 ### 3a. Check Decomposition Thresholds
 
-Evaluate the shape detection output to determine whether this repo should be decomposed into multiple skills.
+Evaluate the shape detection output to determine whether this **monorepo** should be decomposed into multiple skills.
 
-**Threshold conditions** (both constants are `[PENDING VALIDATION]` — subject to empirical tuning):
+**Threshold condition:**
 
 | Threshold | Condition | Signal |
 |-----------|-----------|--------|
-| Large export surface | `export_count > 500` `[PENDING VALIDATION]` | Single skill covering 500+ exports produces unwieldy output |
-| Monorepo / multi-package | `package_count > 3` `[PENDING VALIDATION]` | 4+ packages are almost always multi-concern |
+| Monorepo / multi-package | `package_count > 3` | 4+ packages — a decomposition **candidate** (empirically validated: fires on real 15-, 38-, and 442-package workspaces) |
+
+> A *single* package with a large API surface is **not** a decomposition trigger — it produces one cohesive skill that `skf-create-skill`'s auto-shard step splits into `references/` shards at the 400-line ceiling (a single library is one install / one import namespace; fragmenting it by source directory loses cohesion). Decomposition is for genuinely multi-package monorepos only.
 
 **Decision:**
 
-- **Neither threshold met** → Continue to §4 (single-scope flow, entirely unchanged).
-- **Either or both thresholds met** → Route to §4a (multi-scope decomposition flow). Log: "Auto-decomposition triggered: {reason} ({value} exceeds threshold {threshold} `[PENDING VALIDATION]`)" where reason is `export_threshold`, `package_threshold`, or `both`.
+- **Threshold not met** (`package_count ≤ 3`) → Continue to §4 (single-scope flow, entirely unchanged).
+- **Threshold met** (`package_count > 3`) → this repo is a **decomposition candidate**. A threshold firing means the repo *could* decompose, not that it *should* — continue to §3b to decide merge-vs-split. Log: "Auto-decomposition candidate: package_threshold ({value} packages exceeds 3)".
+
+### 3b. Cohesion Check — Merge to One Skill vs Split into N
+
+Reached only when §3a flagged a decomposition candidate. Most published monorepos are **cohesive** and produce a better single skill than a pile of fragments — empirically, 5/5 real monorepos (animato 15 crates, trpc, react 38 packages, aws-sdk-js-v3 442 packages, plus zod) were best served as one cohesive skill or a curated few, not one-skill-per-package. Decide deliberately:
+
+**Merge into ONE cohesive skill** (override the threshold → continue to §4 single-scope) when **any** of these hold:
+
+- **Umbrella facade** — one package re-exports the members: a root or named package whose dependencies include the other workspace members, or which `pub use` / `export *`s them. The facade *is* the public surface (e.g. animato's `crates/animato` re-exporting its 15 sub-crates).
+- **Shared runtime contract** — the members are consumed together through one entry point, and teaching the shared invariant covers them (e.g. tRPC's adapters around `@trpc/server`; aws-sdk's `new XClient(...) → client.send(new YCommand(...))` shared by every `@aws-sdk/client-*`).
+- **Internal building blocks** — the members are private/internal pieces of one product, not independently meaningful to a consumer.
+
+**Split into N skills** (→ §4a) when:
+
+- The members are **independently published with distinct public surfaces serving different concerns**, **and no umbrella re-exports them** — e.g. `react-dom` and `react-server-dom-*` are separate installs with separate jobs, or a federated SDK where a consumer only ever wants one service. Each genuinely-distinct facet earns its own skill.
+
+If genuinely unsure, **prefer merge** — a too-broad single skill is recoverable with `US`; N fragmented skills are not.
+
+**Facet-coverage guard (merged facet-diverse repos only).** When you merge a repo whose members have genuinely distinct surfaces and you scope to only some of them, record the decision explicitly — never drop a facet silently:
+
+- In `scope.notes`, name the in-scope facets **and** the excluded major facets, e.g. _"Scoped to react + react-dom core; excludes react-server-dom-\* (RSC), the specialized renderers (react-art/native/test), and the compiler — forge a separate skill for those."_
+- Surface the excluded facets in the analysis report (§7) so the operator can re-scope or forge a companion skill.
 
 ### 4. Map Shape to Scope
 
@@ -302,17 +408,20 @@ Apply the shape→scope.type mapping:
 | `library-API` | `public-api` | export_count > 200 |
 | `reference-app` | `reference-app` | — |
 | `language-reference` | `full-library` | — |
-| `stack-compose` | `full-library` | Multi-scope via §3a when `package_count > 3` `[PENDING VALIDATION]` |
+| `stack-compose` | `full-library` | Decomposition candidate when `package_count > 3` — cohesion-checked at §3b |
 
 ### 5. Generate Include/Exclude Patterns
 
 Generate `scope.include` and `scope.exclude` arrays from the detected language and project structure.
 
-**Detect primary language** from manifest type:
+**Detect primary language** from manifest type (the same set shape detection classifies):
 - `package.json` → TypeScript/JavaScript
 - `pyproject.toml` → Python
 - `Cargo.toml` → Rust
 - `go.mod` → Go
+- `pom.xml` → Java
+- `build.gradle` / `build.gradle.kts` → Java or Kotlin (Kotlin when a `src/main/kotlin/` tree exists)
+- `Package.swift` → Swift
 
 **Default patterns (adjust based on actual project structure):**
 
@@ -321,6 +430,10 @@ Generate `scope.include` and `scope.exclude` arrays from the detected language a
 | TypeScript/JavaScript | `['src/**/*.ts', 'src/**/*.tsx']` | `['**/*.test.ts', '**/*.spec.ts', '**/node_modules/**']` |
 | Python | `['src/**/*.py']` or `['{package_name}/**/*.py']` | `['**/*_test.py', '**/test_*.py', '**/tests/**']` |
 | Rust | `['src/**/*.rs']` | `['**/tests/**', '**/benches/**']` |
+| Go | `['**/*.go']` | `['**/*_test.go', '**/vendor/**']` |
+| Java | `['src/main/java/**/*.java']` | `['**/src/test/**']` |
+| Kotlin | `['src/main/kotlin/**/*.kt']` | `['**/src/test/**']` |
+| Swift | `['Sources/**/*.swift']` | `['**/Tests/**']` |
 
 **Adjust for actual layout:** If the project uses a non-standard layout (e.g., `lib/` instead of `src/`, or a named package directory for Python), detect and use the actual paths. Check for the existence of common source directories (`src/`, `lib/`, `pkg/`, the package name directory) and prefer the one that exists.
 
@@ -332,7 +445,7 @@ scope:
   type: '{mapped_scope_type}'
   include: ['{generated_include_patterns}']
   exclude: ['{generated_exclude_patterns}']
-  notes: 'Auto-scoped from shape detection (shape: {shape}, confidence: {confidence})'
+  notes: 'Auto-scoped from shape detection (shape: {shape}, confidence: {confidence}).{corpus_caveat}'
 ```
 
 Determine the skill name from the project name or package name (kebab-case, lowercase). Use the manifest `name` field if available, otherwise derive from the project directory name. If `{coexistence_suffix}` is non-empty, append it to the skill name.
@@ -341,32 +454,51 @@ Detect the primary language from the manifest ecosystem:
 - `npm` → `typescript` (or `javascript` if no `.ts` files in includes)
 - `python` → `python`
 - `rust` → `rust`
+- `go` → `go`
+- `maven` → `java`
+- `gradle` → `java` (or `kotlin` when a `src/main/kotlin/` tree exists)
+- `swift` → `swift`
+
+### 6b. Seed Companion Corpora (whole-language references only)
+
+Runs only when §3 classified the repo as `language-reference` **via a whole-language signal** — the `signals` array contains a `grammar_file:` or `tree_triad:` entry (a compiler / interpreter / grammar repo such as rust-lang/rust, TypeScript, CPython). **Skip** when `language-reference` fired only from `parser_producer:` / `parser_dep:` signals (a parser *library* such as pest or lalrpop): there the code **is** the product, so no companion prose is needed and the §6/§7 caveat below does not apply.
+
+A whole-language skill's value is in the language's **prose** — the guide/Book, the standard/library API docs, idioms — not the compiler internals. Seed those canonical corpora so the forged skill teaches the language rather than its implementation.
+
+**Resolve `{detectLanguageHelper}`** from `{detectLanguageProbeOrder}` and **`{languageCorporaHelper}`** from `{languageCorporaProbeOrder}` (first existing path wins).
+
+1. **Derive the corpus language key `{corpus_language}`.** Prefer the §6 manifest language when non-empty. Otherwise — a manifest-less toolchain such as CPython or Ruby — resolve it from the file paths harvested in §2:
+   ```bash
+   echo '{"tree": [<harvested §2 file paths>]}' | uv run {detectLanguageHelper}
+   ```
+   Use its `language` field. (`.c`/`.h`/`.y` are not in the detector's extension map, so a C-hosted language resolves by its real sources — Ruby via `.rb`.)
+2. **Look up canonical corpora:**
+   ```bash
+   uv run {languageCorporaHelper} --language {corpus_language}
+   ```
+   - exit 0 → parse the `[{url, label, source}]` array (each seed carries `source: language-registry`) → these are `{corpus_seeds}`.
+   - exit 1 → no registry entry (long-tail language) → `{corpus_seeds}` is empty (README detection in brief-skill remains the only source).
+   - exit 2 → log a warning and treat as empty (best-effort; never halt).
+3. Record `{N}` = number of seeds and `{corpus_labels}` = comma-joined labels, carried into the brief `doc_urls` (§8) and the honest caveat (§6/§7).
+4. Build `{corpus_caveat}` (appended to `scope.notes` in §6/§8 and surfaced in §7) so the operator knows a code-only whole-language skill is low-value:
+   - `{N}` ≥ 1: `" LANGUAGE-REFERENCE CAVEAT: this skill's value is the {corpus_language} prose (guide/Book + std/library docs), not compiler internals. Seeded {N} corpus URL(s): {corpus_labels}. create-skill foregrounds this registry prose as the skill's Language Guide and demotes compiler-internal signatures to a reference-only section — review the forged skill if compiler internals still dominate."`
+   - `{N}` == 0: `" LANGUAGE-REFERENCE CAVEAT: no canonical corpora were found for {corpus_language} (README detection and the registry both came up empty). This skill is LOW-VALUE as code-only — attach the {corpus_language} guide + std/library docs manually (re-run with a doc URL, or enrich via US) before forging."`
+
+   For a parser-library `language-reference` (skipped above) and every other shape, `{corpus_caveat}` is empty.
 
 ### 4a. Multi-Scope Decomposition
 
-This section is reached only from §3a when one or both decomposition thresholds are met. It replaces §4→§5→§6 for repos that will produce N > 1 skills.
-
-**Determine decomposition path:**
+This section is reached only from §3b when the cohesion check decided to **split** a monorepo (members are independently published with distinct surfaces and no umbrella re-exports them). It replaces §4→§5→§6 for repos that will produce N > 1 skills.
 
-- **Monorepo path** (`package_count > 3`): Use workspace package discovery from §2 manifest scan results. Each workspace package with its own manifest becomes a separate skill boundary. Name each skill as `{project_name}-{package_name}` (kebab-case); if `{coexistence_suffix}` is non-empty, append it. Trivial workspace members (no source files, no exports) are excluded.
-- **Large-export path** (`export_count > 500`, single package): Group by top-level source directory modules (e.g., `src/auth/`, `src/core/`, `src/api/`). Each directory subtree with a meaningful export surface becomes a separate skill boundary. Candidate boundaries with fewer than ~50 exports `[PENDING VALIDATION]` should be merged into an "other" catch-all skill rather than becoming standalone skills. Name each skill as `{project_name}-{module_name}` (kebab-case); if `{coexistence_suffix}` is non-empty, append it. If no clear module structure exists (flat `src/` with all files at root level), **do not force decomposition** — fall back to single-scope flow at §4.
-- **Combined path** (both thresholds met): Use the monorepo path. Package boundaries are explicit and take priority over export-count grouping (which is heuristic).
+**Decompose by workspace package:** Use workspace package discovery from §2 manifest scan results. Each workspace package with its own manifest becomes a separate skill boundary. Name each skill as `{project_name}-{package_name}` (kebab-case); if `{coexistence_suffix}` is non-empty, append it. Trivial workspace members (no source files, no exports) are excluded.
 
 **Per-boundary shape→scope mapping:**
 
-For each decomposed boundary, apply the shape→scope mapping from §4 independently — the boundary's local characteristics determine its scope.type:
-
-| Decomposition Type | Per-Boundary Shape Mapping |
-|--------------------|---------------------------|
-| Monorepo (`package_count > 3`) | Re-run the shape→scope heuristic ladder from `step-shape-detect.md` per package using each package's own manifest data. Packages may have different shapes (e.g., a `library-API` core + a `reference-app` CLI). |
-| Large-export (`export_count > 500`) | All boundaries inherit the parent shape. scope.type varies by per-boundary export count (e.g., ≤200 → `full-library`, >200 → `public-api`). |
+For each decomposed boundary, apply the shape→scope mapping from §4 independently — re-run the shape→scope heuristic ladder from `step-shape-detect.md` per package using each package's own manifest data. Packages may have different shapes (e.g., a `library-API` core + a `reference-app` CLI).
 
 ### 5a. Generate Multi-Scope Patterns
 
-For each decomposed boundary, generate include/exclude patterns using the same language-aware rules as §5, but scoped to the boundary's source paths.
-
-- Monorepo boundaries: patterns are rooted at the package path (e.g., `packages/auth/src/**/*.ts` instead of `src/**/*.ts`)
-- Large-export boundaries: patterns are rooted at the module directory (e.g., `src/auth/**/*.ts`)
+For each decomposed boundary, generate include/exclude patterns using the same language-aware rules as §5, but scoped to the boundary's source paths. Monorepo boundaries are rooted at the package path (e.g., `packages/auth/src/**/*.ts` instead of `src/**/*.ts`).
 
 ### 6a. Build Multi-Scope
 
@@ -404,11 +536,11 @@ Add `decomposition` to frontmatter:
 ```yaml
 decomposition:
   triggered: true
-  reason: 'export_threshold' | 'package_threshold' | 'both'
+  reason: 'package_threshold'
   boundary_count: N
 ```
 
-Each `confirmed_units` entry includes `boundary_path` — the relative path to the boundary's root (e.g., `packages/core` for monorepo, `src/auth` for large-export). Omit the `decomposition` key entirely when single-scope (N = 1).
+Each `confirmed_units` entry includes `boundary_path` — the relative path to the boundary's root (e.g., `packages/core`). Omit the `decomposition` key entirely when single-scope (N = 1).
 
 **When single-scope (N = 1):** No `decomposition` key. `confirmed_units` contains a single entry (existing behavior).
 
@@ -428,6 +560,17 @@ For single-scope (unchanged):
 **Exclude Patterns:** {exclude patterns}
 ```
 
+**When the shape is a whole-language `language-reference`** (§6b ran — a `grammar_file:`/`tree_triad:` signal), append a Companion Corpora subsection so the operator sees whether the skill has the prose that makes it useful. The status is computed from the **final** brief `doc_urls` (the entries that will actually be fetched), not the seed count alone:
+
+```markdown
+## Companion Corpora (language-reference)
+
+**Why:** A whole-language skill's value is its prose (guide/Book, std/library docs, idioms), not compiler internals.
+**Corpora in brief doc_urls:** {final_doc_urls_count}
+  - {label}: {url}   # one line per doc_urls entry
+**Status:** {ATTACHED — canonical corpora present | DEGRADED — code-only, no canonical corpora; attach the {corpus_language} guide + std/library docs before forging}
+```
+
 For multi-scope (N > 1):
 ```markdown
 ## Auto-Scope Analysis — Decomposition ({N} skills)
@@ -468,13 +611,22 @@ scope:
     - '{include_patterns}'
   exclude:
     - '{exclude_patterns}'
-  notes: 'Auto-scoped from shape detection (shape: {shape}, confidence: {confidence})'
+  notes: 'Auto-scoped from shape detection (shape: {shape}, confidence: {confidence}).{corpus_caveat}'
 description: '{1-3 sentence description based on shape, language, and manifest name}'
 forge_tier: '{forge_tier}'
 created: '{current_date}'
 created_by: '{user_name}'
 ```
 
+**Companion corpora (whole-language references).** When §6b produced `{corpus_seeds}` (`{N}` ≥ 1), add them as the brief's `doc_urls` so the language's prose is fetched and assembled alongside the code:
+
+```yaml
+doc_urls:
+  - { url: '{seed.url}', label: '{seed.label}', source: '{seed.source}' }   # one entry per §6b seed; source is 'language-registry'
+```
+
+These are the brief's *existing* `doc_urls`; brief-skill's README detection then merges additional discovered docs on top (existing entries win). **When `{N}` is 0, omit the `doc_urls` key entirely** — the schema requires at least one entry when the key is present.
+
 **When decomposition is active (N > 1 units):**
 
 Loop over all N boundaries. For each boundary:

package/src/skf-analyze-source/references/step-shape-detect.md

@@ -1,14 +1,22 @@
+---
+shapeDetectProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-shape-detect.py'
+  - '{project-root}/src/shared/scripts/skf-shape-detect.py'
+---
+
 # Shape Detection Reference
 
 Reference document for invoking `skf-shape-detect.py` — the shared shape classification module. Loaded by `step-auto-scope.md` for auto-scope analysis.
 
 ## Invocation Contract
 
-**Script:** `src/shared/scripts/skf-shape-detect.py`
+**Resolve `{shapeDetectHelper}`** from `{shapeDetectProbeOrder}`; first existing path wins; HALT if neither resolves.
 
 **Command:**
 ```
-uv run python src/shared/scripts/skf-shape-detect.py --repo-url <url> --manifests <path1,path2,...>
+uv run {shapeDetectHelper} --repo-url <url> \
+  --manifests <path1,path2,...> \
+  --grammar-files <g1,g2,...> --tree-paths <d1/,d2/,file,...>
 ```
 
 **Arguments:**
@@ -16,9 +24,11 @@ uv run python src/shared/scripts/skf-shape-detect.py --repo-url <url> --manifest
 | Arg | Required | Description |
 |-----|----------|-------------|
 | `--repo-url` | Yes | Repository URL (context only — no cloning performed) |
-| `--manifests` | Yes | Comma-separated local file paths to manifest files |
+| `--manifests` | Yes | Comma-separated local file paths to manifest files (may be empty when a tree-level signal carries the classification) |
+| `--grammar-files` | No | Comma-separated repo-relative grammar files (`*.y`, `*.g4`, `*.pest`, `Grammar/python.gram`, ...) — a whole-language signal |
+| `--tree-paths` | No | Comma-separated repo-relative directory (trailing `/`) and structural file signals harvested from the clone (a `compiler/` dir, a lexer+parser+ast triad) |
 
-**Supported manifests:** `package.json`, `pyproject.toml`, `Cargo.toml`
+**Supported manifests:** `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod`, `pom.xml`, `build.gradle`, `build.gradle.kts`, `Package.swift`
 
 ## Output Schema
 
@@ -49,22 +59,19 @@ On exit code 2, error details are written to stderr as JSON: `{"error": "message
 | `library-API` | `full-library` | export_count ≤ 200 |
 | `library-API` | `public-api` | export_count > 200 (surface too large for full coverage) |
 | `reference-app` | `reference-app` | Direct mapping — apps, CLIs, demos |
-| `language-reference` | `full-library` | Language tools/parsers are library-shaped from a skill perspective |
-| `stack-compose` | `full-library` | Multi-skill decomposition when `package_count > 3` `[PENDING VALIDATION]` — see step-auto-scope.md §3a |
+| `language-reference` | `full-library` | Language tools/parsers are library-shaped from a skill perspective. **Corpora-dependent** for a *whole-language* reference (a `grammar_file:`/`tree_triad:` signal — a compiler/interpreter): its value is the language's prose (guide/Book + std/library docs), not compiler internals, so step-auto-scope.md §6b seeds companion corpora and §6/§7 record an honest DEGRADED caveat when none are found (mirrors the §3b facet-coverage guard). A parser *library* (`parser_producer:`/`parser_dep:`) is exempt — its code is the product. |
+| `stack-compose` | `full-library` | Decomposition candidate when `package_count > 3` — cohesion-checked in step-auto-scope.md §3b |
 | `unknown` | N/A | Triggers fallback to interactive mode |
 
 ## Decomposition Thresholds
 
-When auto-scope detects a repo exceeding complexity thresholds, it recommends multi-skill decomposition instead of producing a single unwieldy skill. The thresholds are evaluated in step-auto-scope.md §3a.
+When auto-scope detects a multi-package monorepo, it may recommend multi-skill decomposition instead of producing one unwieldy skill. The threshold is evaluated in step-auto-scope.md §3a.
 
 | Threshold | Value | Signal | Decomposition Path |
 |-----------|-------|--------|-------------------|
-| Large export surface | `export_count > 500` `[PENDING VALIDATION]` | Shape detection `export_count` | Group by top-level source directory modules |
-| Multi-package / monorepo | `package_count > 3` `[PENDING VALIDATION]` | Shape detection `package_count` | One skill per workspace package |
-
-Both thresholds are marked `[PENDING VALIDATION]` — no empirical data exists yet for real-world repos. Expected tuning: after running deepwiki against 5–10 real repos, adjust thresholds based on whether decomposition produces useful skill boundaries or noise.
+| Multi-package / monorepo | `package_count > 3` | Shape detection `package_count` | Cohesion check (§3b): merge to one skill or split per package |
 
-When both thresholds are met simultaneously, the monorepo path takes priority (package boundaries are explicit; export-count grouping is heuristic).
+`package_count > 3` makes a **monorepo** a decomposition candidate; step-auto-scope.md §3b then decides merge-vs-split. It is empirically validated (fires on real 15-, 38-, and 442-package workspaces). A *single* package with a large API surface is **not** decomposed — it produces one cohesive skill that `skf-create-skill`'s auto-shard splits into `references/` shards at the 400-line ceiling.
 
 When neither threshold is met, the single-scope flow proceeds unchanged.
 
@@ -72,7 +79,7 @@ When neither threshold is met, the single-scope flow proceeds unchanged.
 
 The five-shape heuristic ladder applies in order (first match wins):
 
-1. **language-reference** — parser/grammar/language-toolchain project. Signals: parser-related deps (pest, antlr4, tree-sitter, lark, etc.)
+1. **language-reference** — parser/grammar/language-toolchain project. Signals, strongest first: a hand-written-compiler tree structure (a dedicated `compiler/` directory with a lexer+parser+ast triad plus a codegen/VM/type-checker member — catches rustc, TypeScript, Go); a declared grammar file (`Grammar/python.gram`, a root `parse.y`, a `*.g4` — catches CPython, Ruby); the repo's own name being a known parser/grammar tool (pest, lalrpop, lark — the producer); or a parser-generator dependency (a DSL built on antlr4/lalrpop — the consumer). Delegating consumers (formatters, linters, bundlers that depend on a parser) and markup/DSL parsers (CSS, markdown, GraphQL) are excluded.
 2. **stack-compose** — multi-ecosystem composite project. Signals: manifests from 2+ distinct ecosystems
 3. **reference-app** — application, CLI, or demo project. Signals: npm `bin` field, Rust `[[bin]]`, framework deps (next, fastapi, axum, etc.)
 4. **library-API** — library exposing a programmatic API. Signals: `main`/`module`/`exports` fields, `[lib]` target, export count

package/src/skf-audit-skill/references/init.md

@@ -252,7 +252,7 @@ When skipping, log the reason, then set the audit-ref context variables to basel
 - **[D]:** Read the prior report's findings_list (parse the Structural/Semantic/Severity sections, or the appended findings tables) and stash as `prior_findings` in workflow context. Run the new audit normally. In step 6 (report.md), after the Remediation Suggestions section, emit a `## Diff Against Prior Report` subsection summarizing added / removed / changed findings vs `prior_findings`.
 - **[R]:** Load the prior report's frontmatter (`stepsCompleted`, `drift_score`, any intermediate state). Set `{outputFile}` to the prior report path (do NOT create a new one). Determine the next un-completed step from `stepsCompleted` and skip forward to it; downstream steps append to the existing report.
 
-  > **Note (resumability):** the existing template frontmatter captures `stepsCompleted` and `drift_score` but does not currently persist intermediate findings_list between stages. If [R] is selected and stepsCompleted indicates the prior run halted after structural-diff or semantic-diff, the appended sections in the report body (`## Structural Drift`, `## Semantic Drift`, `## Severity Classification`) serve as the implicit intermediate state — re-parse them on resume rather than re-running completed stages. **TODO:** if resumability proves unreliable in practice, extend the report frontmatter to carry an explicit `intermediate_findings` block (out of scope for this PR).
+  > **Note (resumability):** the existing template frontmatter captures `stepsCompleted` and `drift_score` but does not currently persist intermediate findings_list between stages. If [R] is selected and stepsCompleted indicates the prior run halted after structural-diff or semantic-diff, the appended sections in the report body (`## Structural Drift`, `## Semantic Drift`, `## Severity Classification`) serve as the implicit intermediate state — re-parse them on resume rather than re-running completed stages. **Design note:** explicit mid-stage resume state is intentionally not persisted today — if resumability proves unreliable in practice, a future enhancement is to extend the report frontmatter to carry an explicit `intermediate_findings` block.
 
 - **Other input:** help user, redisplay the gate.