bmad-module-skill-forge 2.1.0 → 3.0.0

package/src/skf-analyze-source/references/auto-docs-only.md

@@ -4,32 +4,53 @@ outputFile: '{forge_data_folder}/analyze-source-report-{project_name}.md'
 writeSkillBriefProbeOrder:
   - '{project-root}/_bmad/skf/shared/scripts/skf-write-skill-brief.py'
   - '{project-root}/src/shared/scripts/skf-write-skill-brief.py'
+skillInventoryProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-skill-inventory.py'
+  - '{project-root}/src/shared/scripts/skf-skill-inventory.py'
 ---
 
 <!-- Config: communicate in {communication_language}. -->
 
 # Step 1a §0a: Docs-Only Short-Circuit
 
-Reached from `step-auto-scope.md` §0c when the target is a documentation URL (not a GitHub repo or local path). It validates the URL, writes a minimal brief and analysis report, emits the result envelope, and chains directly to health-check — the standard auto-scope body (§1 through §11 in `step-auto-scope.md`) never runs for a docs-only target. `{coexistence_suffix}`, `{forge_tier}`, `{user_name}`, `{current_date}`, and the classification set upstream in §0/§0c carry into this file.
+Reached from `step-auto-scope.md` §0c when the target is a documentation URL (not a repository URL or local path). It validates the URL, writes a minimal brief and analysis report, emits the result envelope, and chains directly to health-check: the standard auto-scope body (§1 through §9 in `step-auto-scope.md`) never runs for a docs-only target. `{coexistence_suffix}`, `{forge_tier}`, `{user_name}`, `{current_date}`, and the classification set upstream in §0/§0c carry into this file.
 
 ## MANDATORY SEQUENCE — §0a
 
+Every HARD HALT in this file names its exit code, `halt_reason` and phase. When `{headless_mode}` is true it first prints its envelope on stderr through the shared emitter (`{emitEnvelopeHelper}` and `{run_dir}` come from SKILL.md On Activation): stage `{run_dir}/halt.json` as `{"phase": "<phase>", "reason": "<the halt message>", "halt_reason": "<halt_reason>", "mode": "auto", "source_type": "docs-only"}`, plus `"path"` when the halt names one, then run
+
+```bash
+uv run {emitEnvelopeHelper} emit-halt --workflow skf-analyze-source --run-dir "{run_dir}" --target stderr < "{run_dir}/halt.json"
+```
+
+and display the line it prints verbatim. Write the payload as valid JSON: in the halt message and `path`, replace each backslash with / and each double quote with a backtick. If the emitter exits non-zero or prints no line, display the halt message alone. An interactive HALT displays its message and emits nothing.
+
 ### 1. Validate URL reachability
 
+When §0's `derive-name` call gave `git_probe` `no-git` (git is missing, so nothing asked whether `{url}` is a repository), run `uv run {emitEnvelopeHelper} record --run-dir "{run_dir}" --warning 'git_probe_unanswered: {url} was not checked for a repository: git is missing'`, a single quote in it written as a backtick (if it fails, go on). Then check that the URL answers:
+
 ```bash
 curl -sI --max-time 5 {url}
 ```
 
 - On **2xx/3xx** response: URL is reachable. Continue.
-- On **4xx/5xx**, DNS failure, or timeout: HARD HALT with exit code 3 (`resolution-failure`). Emit error message: `"Documentation URL unreachable: {url} — {status or error}"`, then the error envelope (shape in `references/headless-contract.md`) with `exit_code: 3`, `halt_reason: "resolution-failure"`, `mode: "auto"`, `source_type: "docs-only"`.
+- On **4xx/5xx**, DNS failure, or timeout: HARD HALT (exit code 3, `halt_reason: "resolution-failure"`, phase `auto-docs-only:1`, path `{url}`): "Documentation URL unreachable: {url}: {status or error}".
 
 ### 2. Derive skill name from URL domain
 
-Extract the hostname from the URL (e.g., `docs.example.com` from `https://docs.example.com/guide/intro`), convert to kebab-case (replace `.` with `-`), yielding e.g. `docs-example-com`. If `{coexistence_suffix}` is non-empty, append it to the skill name (e.g., `docs-example-com-wiki`).
+**Resolve `{skillInventoryHelper}`** from `{skillInventoryProbeOrder}`; first existing path wins. If neither resolves, HARD HALT (exit code 3, `halt_reason: "resolution-failure"`, phase `auto-docs-only:2`): "`skf-skill-inventory.py` is missing. Re-install SKF."
+
+Name the skill with the helper that names every brief. It names a documentation URL after its host (`https://docs.example.com/guide/intro` gives `docs-example-com`), the name §0c's coexistence check compared:
+
+```bash
+uv run {skillInventoryHelper} derive-name --target "{url}"
+```
+
+`{skill_name}` ← `.name`. If `{coexistence_suffix}` is non-empty, append it to the skill name (e.g., `docs-example-com-wiki`).
 
 ### 3. Write analysis report
 
-Update {outputFile} with docs-only results. If the write fails, HARD HALT with exit code 4 (`write-failed`) per `references/headless-contract.md` (applies equally to the brief write in §4 and the result contract in §6).
+Update {outputFile} with docs-only results. If the write fails, HARD HALT (exit code 4, `halt_reason: "write-failed"`, phase `auto-docs-only:3`, path `{outputFile}`): "The analysis report could not be written: {the error}."
 
 **Update frontmatter:**
 ```yaml
@@ -56,11 +77,9 @@ confirmed_units:
 
 ### 4. Write skill brief via canonical writer
 
-**Resolve `{writeSkillBriefHelper}`** from `{writeSkillBriefProbeOrder}`; first existing path wins; HALT if neither resolves.
+**Resolve `{writeSkillBriefHelper}`** from `{writeSkillBriefProbeOrder}`; first existing path wins. If neither resolves, HARD HALT (exit code 3, `halt_reason: "resolution-failure"`, phase `auto-docs-only:4`): "`skf-write-skill-brief.py` is missing. Re-install SKF."
 
-Create directory `{forge_data_folder}/{skill_name}/` if it does not exist.
-
-Pipe the flat context JSON below into the resolved writer with the `--from-flat` flag:
+Write the flat context below as `{run_dir}/brief-{skill_name}.json`, a file and never an echo'd string:
 
 ```json
 {
@@ -91,26 +110,45 @@ Pipe the flat context JSON below into the resolved writer with the `--from-flat`
 }
 ```
 
+Then hand it to the writer, which checks it, creates the folder and writes the brief atomically:
+
 ```bash
-echo '<context-json>' | uv run {writeSkillBriefHelper} write --target {forge_data_folder}/{skill_name}/skill-brief.yaml --from-flat
+uv run {writeSkillBriefHelper} write --target "{forge_data_folder}/{skill_name}/skill-brief.yaml" --from-flat < "{run_dir}/brief-{skill_name}.json"
 ```
 
-### 5. Emit success envelope
+Keep the `brief_path` it prints for §5. If it exits non-zero, the brief is rejected or unwritten: HARD HALT (exit code 4, `halt_reason: "write-failed"`, phase `auto-docs-only:4`, path `{forge_data_folder}/{skill_name}/skill-brief.yaml`): "The brief for {skill_name} was rejected or could not be written: {its `message`}."
 
-```
-SKF_ANALYZE_RESULT_JSON: {"status":"success","report_path":"{outputFile_path}","brief_paths":["{brief_path}"],"unit_counts":{"confirmed":1,"skipped":0,"maybe":0},"exit_code":0,"halt_reason":null,"mode":"auto","source_type":"docs-only"}
-```
+### 5. End the run
 
-If `{coexistence_suffix}` is non-empty (i.e., [A]longside was selected in §0c), include `"coexistence":"alongside"` in the envelope.
+The shared emitter writes the result files and prints the envelope; this section stages their content, in every mode. Write `{run_dir}/result-context.json`:
 
-The `source_type` field signals downstream consumers (BS) to skip repo-based enrichment.
+```json
+{
+  "status": "success",
+  "report_path": "{outputFile as an absolute path}",
+  "brief_paths": ["{the brief_path §4 printed}"],
+  "unit_counts": {"confirmed": 1, "skipped": 0, "maybe": 0},
+  "mode": "auto",
+  "source_type": "docs-only",
+  "result_contract": {
+    "skill": "skf-analyze-source",
+    "status": "success",
+    "outputs": [{"type": "report", "path": "{outputFile as an absolute path}"}, {"type": "brief", "path": "{the brief_path §4 printed}"}],
+    "summary": {"mode": "auto", "source_type": "docs-only", "brief_count": 1, "units": ["{skill_name}"]}
+  }
+}
+```
+
+Add `"coexistence": "alongside"` when `{coexistence_suffix}` is non-empty ([A]longside was selected in §0c). The `source_type` field signals downstream consumers (BS) to skip repo-based enrichment. Then run:
 
-### 6. Write result contract
+```bash
+uv run {emitEnvelopeHelper} emit --workflow skf-analyze-source --run-dir "{run_dir}" --result-dir "{forge_data_folder}" < "{run_dir}/result-context.json"
+```
 
-Write the result contract per `shared/references/output-contract-schema.md`: the per-run record and latest copy, same as `step-auto-scope.md` §10.
+The emitter stamps the UTC time and the run id into the record, writes `{forge_data_folder}/analyze-source-result-{YYYYMMDD-HHmmss}.json` and its `analyze-source-result-latest.json` copy, and prints the `SKF_ANALYZE_RESULT_JSON:` line on stdout. When `{headless_mode}` is true, display the line verbatim. A result file that could not be written leaves the line's `result_path` null and a `result_file_write_failed` warning in it, and the run still finishes. If the emitter exits non-zero, correct `result-context.json` from the message on its stderr and run it once more; if it fails again, HARD HALT (exit code 4, `halt_reason: "write-failed"`, phase `auto-docs-only:5`, path `{forge_data_folder}`): "The result contract could not be written: {its message}."
 
-If `{onCompleteCommand}` is non-empty, invoke it now with `--result-path={result_json_path}`.
+If `{onCompleteCommand}` is non-empty and the line's `result_path` is not null, invoke it now: `{onCompleteCommand} --result-path={forge_data_folder}/analyze-source-result-latest.json`. Display a hook failure; it never fails the run.
 
-### 7. Chain to health check
+### 6. Chain to health check
 
-Load, read fully, then execute {nextStepFile} to run the shared workflow health check.
+Delete the run folder, whose payloads the emitter has read: `rm -rf "{run_dir}"`. Then load, read fully, and execute {nextStepFile} to run the shared workflow health check.

package/src/skf-analyze-source/references/continue.md

@@ -1,13 +1,11 @@
 ---
 outputFile: '{forge_data_folder}/analyze-source-report-{project_name}.md'
 nextStepOptions:
-  step 1a: 'step-auto-scope.md'
   step 2: 'scan-project.md'
   step 3: 'identify-units.md'
   step 4: 'map-and-detect.md'
   step 5: 'recommend.md'
   step 6: 'generate-briefs.md'
-  step 7: 'health-check.md'
 ---
 
 <!-- Config: communicate in {communication_language}. -->
@@ -16,7 +14,7 @@ nextStepOptions:
 
 ## STEP GOAL:
 
-To resume the analyze-source workflow from where it was left off in a previous session by reading the analysis report's progress state and routing to the correct next step.
+To resume an unfinished analyze-source run from where a previous session left off, by reading the analysis report's progress state and routing to the correct next step. Init section 1 sends only an unfinished report of the step-by-step analysis here: it archives a finished report and one whose `mode` is `'auto'` and starts a fresh analysis instead.
 
 ## Rules
 
@@ -39,7 +37,6 @@ Load {outputFile} and read frontmatter:
 - `forge_tier`
 - `existing_skills`
 - `confirmed_units`
-- `mode` — `'auto'` when the report was produced by the auto-scope path; absent or any other value means interactive
 
 ### 3. Present Progress Summary
 
@@ -55,11 +52,7 @@ Load {outputFile} and read frontmatter:
 
 ### 4. Determine Next Step
 
-**IF the report's `mode` is `'auto'`** (produced by the auto-scope path; reached here because the resume invocation did not carry `[auto]`): an auto analysis is a single idempotent pass, not a resumable interactive chain. Do not use the interactive table below.
-- If `auto-scope` is in `stepsCompleted`, the auto analysis already completed — announce "**This auto analysis is already complete.** Would you like to start a new analysis?" and stop.
-- Otherwise (an auto run interrupted before auto-scope finished), re-enter the auto path: load, read fully, then execute `step-auto-scope.md` (it reads the existing report frontmatter and re-runs cleanly). **STOP HERE.**
-
-For interactive reports, map the last completed step to the next step file:
+Map the last completed step to the next step file:
 
 | Last Completed | Next Step |
 |----------------|-----------|
@@ -68,10 +61,6 @@ For interactive reports, map the last completed step to the next step file:
 | identify-units | map-and-detect |
 | map-and-detect | recommend |
 | recommend | generate-briefs |
-| generate-briefs | health-check |
-
-**IF `health-check` is in `stepsCompleted`:**
-"**This analysis appears to be complete.** All steps have been finished. Would you like to start a new analysis?"
 
 ### 5. Update and Route
 

package/src/skf-analyze-source/references/discover-additional-source.md

@@ -0,0 +1,147 @@
+---
+outputFile: '{forge_data_folder}/analyze-source-report-{project_name}.md'
+heuristicsFile: 'references/unit-detection-heuristics.md'
+scanRootFile: 'references/scan-root.md'
+unitExportsFile: 'references/map-unit-exports.md'
+scanManifestsProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-scan-manifests.py'
+  - '{project-root}/src/shared/scripts/skf-scan-manifests.py'
+disqualifyCandidatesProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-disqualify-candidates.py'
+  - '{project-root}/src/shared/scripts/skf-disqualify-candidates.py'
+skillInventoryProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-skill-inventory.py'
+  - '{project-root}/src/shared/scripts/skf-skill-inventory.py'
+detectLanguageProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-detect-language.py'
+  - '{project-root}/src/shared/scripts/skf-detect-language.py'
+extractPublicApiProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-extract-public-api.py'
+  - '{project-root}/src/shared/scripts/skf-extract-public-api.py'
+checkUnitRecordsProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-check-unit-records.py'
+  - '{project-root}/src/shared/scripts/skf-check-unit-records.py'
+countImportsProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-count-imports.py'
+  - '{project-root}/src/shared/scripts/skf-count-imports.py'
+findCyclesProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-find-cycles.py'
+  - '{project-root}/src/shared/scripts/skf-find-cycles.py'
+pairIntersectProbeOrder:
+  - '{project-root}/_bmad/skf/shared/scripts/skf-pair-intersect.py'
+  - '{project-root}/src/shared/scripts/skf-pair-intersect.py'
+---
+
+<!-- Config: communicate in {communication_language}. -->
+
+# Discover Additional Source
+
+## STEP GOAL:
+
+To add one more project path to a running analysis: scan it, classify and name its units, map their exports and imports, propose its composite merges, and merge all of it into the report, then return to the menu that loaded this file (the [D] option of map-and-detect and of recommend), which says what to do with the new units.
+
+## Rules
+
+- Interactive only: a headless run never takes [D], since both menus default to [C]
+- Work on the new path only: the units already in the report keep their classification, records and decisions
+- Every helper below reads the new path's scan root, never `{project-root}` (the forge workspace)
+
+## MANDATORY SEQUENCE
+
+Every HARD HALT in this file names its exit code, `halt_reason` and phase. When `{headless_mode}` is true it first prints its envelope on stderr through the shared emitter (`{emitEnvelopeHelper}` and `{run_dir}` come from SKILL.md On Activation): stage `{run_dir}/halt.json` as `{"phase": "<phase>", "reason": "<the halt message>", "halt_reason": "<halt_reason>", "mode": "interactive", "report_path": "{outputFile as an absolute path}"}`, plus `"path"` when the halt names one, then run
+
+```bash
+uv run {emitEnvelopeHelper} emit-halt --workflow skf-analyze-source --run-dir "{run_dir}" --target stderr < "{run_dir}/halt.json"
+```
+
+and display the line it prints verbatim. Write the payload as valid JSON: in the halt message and `path`, replace each backslash with / and each double quote with a backtick. If the emitter exits non-zero or prints no line, display the halt message alone. An interactive HALT displays its message and emits nothing.
+
+**Resolve every helper first**, each from its probe order in the frontmatter (first existing path wins): `{scanManifestsHelper}`, `{disqualifyCandidatesHelper}`, `{skillInventoryHelper}`, `{detectLanguageHelper}`, `{extractPublicApiHelper}`, `{checkUnitRecordsHelper}`, `{countImportsHelper}`, `{findCyclesHelper}` and `{pairIntersectHelper}`. If one has no candidate, HARD HALT (exit code 3, `halt_reason: "resolution-failure"`, phase `discover-additional-source:0`): "`{the missing script}` is missing. Re-install SKF."
+
+### 1. Take the New Path
+
+Take the path the user gave with [D], or ask for one: "**Which repository or folder should I add?** A local path or a URL." Check that it exists (local) or is accessible (remote); otherwise say so and ask again. Read {outputFile}'s frontmatter (`project_paths`, `scan_roots`, `refs`, `target_ref`, `forge_tier`, `intent_hint`, `existing_skills`, `existing_briefs`). The new path is entry `{i}` of `project_paths[]` (its current length plus one). It takes `target_ref` as its ref when the report sets one (no `--target-refs` entry can name it), else none. Load {heuristicsFile}.
+
+### 2. Scan It
+
+Make its **scan root** by {scanRootFile}, with the ref §1 gave it. If a command there fails, show its first stderr line and ask for another path (§1). Then list its manifests:
+
+```bash
+uv run {scanManifestsHelper} scan "{scan_root}" > "{run_dir}/manifests-{i}.json"
+```
+
+Its candidate boundaries are that envelope's `candidates[]` ({heuristicsFile}'s **Candidate Boundaries**).
+
+### 3. Classify and Name the New Units
+
+List each candidate's files and apply the deterministic disqualification rules in one call, one entry per candidate with its path relative to the scan root:
+
+```bash
+uv run {disqualifyCandidatesHelper} filter --boundaries - --source-root "{scan_root}" --tree-dir "{run_dir}/unit-trees/{i}" <<'SKF_BOUNDARIES'
+[{"name": "<candidate folder name>", "path": "<candidate path>"}, ...]
+SKF_BOUNDARIES
+```
+
+Each record of `kept[]` and `dropped[]` gives the evidence {heuristicsFile}'s Disqualification Rules judge: `files_count`, `loc_total`, `generated_files`, `generated_ratio`, `passes_with_generated`, `manifest`, `signals` and `tree_file` (the candidate's file list). Then judge what the script cannot: drop a mixed candidate only when its own code is no more than glue around its generated files; restore a candidate dropped over files you judge its own code when `passes_with_generated` is true; drop pure configuration and test-only candidates. Name every kept one in one call, by {heuristicsFile}'s Unit Names (`target` is its path, or the new project path itself for a candidate at the root):
+
+```bash
+uv run {skillInventoryHelper} derive-name --from - <<'SKF_UNIT_NAMES'
+[{"target": "<candidate path>", "manifest_name": <manifest.name>, "private": <manifest.private>}, ...]
+SKF_UNIT_NAMES
+```
+
+Give the entries `unnamed` and `duplicates` list a name of your own, from their folder, and never a name a unit of the report already has. A unit whose name is in `existing_skills` is already skilled: drop it (recommend update-skill); one in `existing_briefs` keeps the status `briefed`. Classify each unit's boundary type and scope type by {heuristicsFile}'s Boundary Classification, and detect its language from its file list:
+
+```bash
+uv run {detectLanguageHelper} --tree-file "{the unit's tree_file}"
+```
+
+When `detected_languages` has more than one entry, choose the language the unit documents from its own manifest, or `.source_language` without one. When `intent_hint` is not empty, mark a unit the stated goal clearly leaves out `deferred`, as the report's other deferred units are: it is listed and not analyzed, and the user can restore it.
+
+### 4. Map the New Units
+
+Load, read the entire file, then execute {unitExportsFile} for the new units that are not deferred, each with `{unit_root}` its path under the new scan root and its file count the `files_count` of its §3 record. It writes each unit's record to `{run_dir}/unit-records/{unit_name}.json`, checks the records, records their problems and warnings, and returns them.
+
+**Import graph.** Write the units JSON for the new units, one entry each: `{"name": "<unit name>", "path": "<its path, relative to the scan root>", "manifest_name": "<its own manifest's name>", "ecosystem": "<its own manifest's ecosystem>"}` (no `manifest_name` or `ecosystem` without a manifest; `"modules"` only for a folder without a manifest that code imports by name), then:
+
+```bash
+cat > "{run_dir}/discover-{i}-units.json" <<'SKF_UNITS'
+{"units": [<one entry per new unit that is not deferred>]}
+SKF_UNITS
+uv run {countImportsHelper} count "{scan_root}" --units "{run_dir}/discover-{i}-units.json" --deps "{run_dir}/manifests-{i}.json" > "{run_dir}/discover-{i}-imports.json"
+uv run {countImportsHelper} summary "{run_dir}/discover-{i}-imports.json" --manifests "{run_dir}/manifests-{i}.json"
+uv run {findCyclesHelper} find --edges "{run_dir}/discover-{i}-imports.json"
+uv run {countImportsHelper} count "{scan_root}" --units "{run_dir}/discover-{i}-units.json" --format libraries > "{run_dir}/discover-{i}-importers.json"
+uv run {pairIntersectHelper} intersect --libraries "{run_dir}/discover-{i}-importers.json"
+```
+
+Read each `units[]` entry's `imports_from`, `imported_by`, `file_count` and `external_dep_count` and each `edge_files[]` entry from the summary line (the full file lists stay in the imports file, for an edge that needs them), each `cycles[]` entry (units that import each other) and each `pairs[]` entry (units imported together in `intersection_count` files; when `truncated` is true, run the last call again with `--top-k` set to its `total_pairs`). A helper that exits non-zero names the problem on stderr: fix the units JSON and run the block again.
+
+### 5. Propose Composites and Stack Skill Candidates
+
+Judge the new units as one pass, so a group is never both a composite and a stack skill candidate. Apply the Composite Boundary triggers and the Cohesion Triggers of {heuristicsFile}: a mutual hard dependency from `cycles[]`, a shared integration surface from the summary's `edge_files[]`, and the cohesion triggers from its `umbrella_candidates[]` and each manifest's `name`, `private` and `internal_deps`. Name every proposal in one call (`target` is the new project path, `manifest_name` the facade's manifest name when an umbrella facade triggered it, else null, and `members` the constituents' manifest names):
+
+```bash
+uv run {skillInventoryHelper} derive-name --from - <<'SKF_COMPOSITE_NAMES'
+[{"target": "<project path>", "manifest_name": "<facade name or null>", "members": ["<constituent manifest name>", ...]}, ...]
+SKF_COMPOSITE_NAMES
+```
+
+Check the four indicators of {heuristicsFile}'s Stack Skill Candidate Detection against the new units no proposal groups (co-import frequency is a `pairs[]` entry with an `intersection_count` of 3 or more).
+
+### 6. Merge Into the Report and Return
+
+Add to {outputFile}: the new units, with their project path, to the Identified Units classification table (deferred units included), and the dropped candidates to its Disqualification table. Then:
+
+- **Loaded by map-and-detect:** write nothing more. Its Export Map and Integration Points still hold their placeholders, and its §7 writes both sections from the findings these units join.
+- **Loaded by recommend:** both sections are already written: add the new units' export rows to the Export Map, and their imports, composite proposals and stack skill candidates to Integration Points.
+
+Update its frontmatter:
+
+```yaml
+project_paths: [append the new path]
+scan_roots: {add the new path: its scan root}
+refs: {add the new path: its ref, when §1 gave it one}
+```
+
+Return to the menu that loaded this file with the new units, their records, their import graph, their composite proposals and their stack skill candidates: it presents them and takes the user's decisions.