bmad-module-skill-forge 0.2.0 → 0.3.0

package/src/workflows/create-skill/steps-c/step-03-extract.md

@@ -1,8 +1,9 @@
 ---
 name: 'step-03-extract'
 description: 'Tier-dependent source code extraction — AST or source reading for exports, signatures, and types'
-nextStepFile: './step-04-enrich.md'
+nextStepFile: './step-03b-fetch-temporal.md'
 extractionPatternsData: '../data/extraction-patterns.md'
+sourceResolutionData: '../data/source-resolution-protocols.md'
 ---
 
 # Step 3: Extract
@@ -64,9 +65,21 @@ From the brief, apply scope and pattern filters:
 - `include_patterns` — file globs to include (if specified)
 - `exclude_patterns` — file globs to exclude (if specified)
 
-Build the filtered file list from the source tree resolved in step-01.
+Build the filtered file list from the source tree resolved in step-01. Record the result: "**Filtered file count: {N} files in scope**" — this count is the input to the AST Extraction Protocol decision tree in the extraction patterns data file.
 
-### 3. Execute Tier-Dependent Extraction
+### 3. Check for Docs-Only Mode
+
+**If `source_type: "docs-only"` in the brief data:**
+
+"**Docs-only mode:** No source code to extract. Documentation content will be fetched from `doc_urls` in step-03c."
+
+Build an empty extraction inventory with zero exports. Set `extraction_mode: "docs-only"` in context. Auto-proceed through Gate 2 (section 5) — display the empty inventory and note that T3 content will be produced by the doc-fetcher step.
+
+**If `source_type: "source"` (default):** Continue with extraction below.
+
+### 4. Execute Tier-Dependent Extraction
+
+Load `{sourceResolutionData}` completely. Follow the **Remote Source Resolution** protocol for Forge/Deep tiers (ephemeral clone, sparse-checkout, cleanup) and the **Version Reconciliation** protocol for all tiers. Apply the protocols before proceeding with the tier-specific extraction strategy below.
 
 **Quick Tier (No AST tools):**
 
@@ -79,8 +92,13 @@ Build the filtered file list from the source tree resolved in step-01.
 
 **Forge/Deep Tier (AST available):**
 
+⚠️ **CRITICAL:** Before executing AST extraction, load the **AST Extraction Protocol** section from `{extractionPatternsData}`. Follow the decision tree based on the file count from step-01's file tree. This determines whether to use the MCP tool, scoped YAML rules, or CLI streaming. Never use `ast-grep --json` (without `=stream`) — it loads the entire result set into memory and will fail on large codebases.
+
 1. Detect language from brief or file extensions
-2. Use `ast_bridge.scan_definitions(path, language)` for each file in scope
+2. Follow the AST Extraction Protocol decision tree from `{extractionPatternsData}`:
+   - ≤100 files: use `find_code()` MCP tool with `max_results` and `output_format="text"`
+   - ≤500 files: use `find_code_by_rule()` MCP tool with scoped YAML rules
+   - >500 files: use CLI `--json=stream` with line-by-line streaming Python — **CRITICAL:** inject the brief's `scope.exclude` patterns into the Python filter's `EXCLUDES` list (use `[]` if absent) so excluded files are discarded before consuming `head -N` slots (see template in extraction patterns data)
 3. For each export: extract function name, full signature, parameter types, return type, line number
 4. Use `ast_bridge.detect_co_imports(path, libraries[])` to find integration points
 5. Build extraction rules YAML data for reproducibility
@@ -88,14 +106,35 @@ Build the filtered file list from the source tree resolved in step-01.
 
 **If AST tool is unavailable at Forge/Deep tier:**
 
-Degrade to Quick tier extraction silently. Note the degradation in context for the evidence report.
+⚠️ **Warn the user explicitly:** "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
+
+Degrade to Quick tier extraction. Note the degradation reason in context for the evidence report.
 
 **For each file — handle failures gracefully:**
 
 - If a file cannot be read: log warning, skip file, continue with remaining files
 - If AST parsing fails on a file: fall back to source reading for that file, continue
 
-### 4. Build Extraction Inventory
+**Re-export tracing (Forge/Deep only):**
+
+After the initial AST scan, check for public exports from the entry point (`__init__.py`, `index.ts`, `lib.rs`) that were NOT found by AST extraction. These are likely module-level re-exports. Follow the **Re-Export Tracing** protocol in `{extractionPatternsData}` to resolve them to their actual definition files. This ensures module re-export patterns (common in Python libraries with clean public APIs) do not create gaps in the extraction inventory.
+
+### 4b. Validate Exports Against Package Entry Point
+
+After extraction, validate the collected exports against the package's actual public API surface:
+
+- **Python:** Read `{source_root}/__init__.py` — extract all import statements to build the actual public export list. Compare against AST-extracted exports:
+  - In AST results but not in `__init__.py` → mark as internal (exclude from `metadata.json` exports, keep in provenance-map with a note)
+  - In `__init__.py` but not in AST results → flag as extraction gap (add to inventory, trace via re-export protocol)
+- **TypeScript/JavaScript:** Read `index.ts`/`index.js` — extract named exports. Same comparison logic.
+- **Rust:** Read `lib.rs` — extract `pub use` items. Same comparison logic.
+- **Go:** Scan package-level files for exported (capitalized) identifiers.
+
+Use the entry point as the authoritative source for `metadata.json`'s `exports[]` array. This prevents internal symbols from inflating the exports list and ensures all public API items are captured.
+
+**If entry point is missing or unreadable:** Skip validation with a warning. Use the AST-extracted list as-is.
+
+### 5. Build Extraction Inventory
 
 Compile all extracted data into a structured inventory:
 
@@ -113,12 +152,13 @@ Compile all extracted data into a structured inventory:
 - Total exports found
 - Exports by type (functions, types/interfaces, constants)
 - Confidence breakdown (T1 count, T1-low count)
+- `top_exports[]` — sorted list of the top 10-20 public API function names by prominence (import frequency or documentation position). This named field is consumed by step-03b for targeted temporal fetching and cache fingerprinting.
 
 **Co-import patterns (Forge/Deep only):**
 - Libraries commonly imported alongside extracted exports
 - Integration point suggestions
 
-### 5. Present Extraction Summary (Gate 2)
+### 6. Present Extraction Summary (Gate 2)
 
 Display the extraction findings for user confirmation:
 
@@ -137,7 +177,7 @@ Display the extraction findings for user confirmation:
 
 Review the extraction summary above. Select an option to continue."
 
-### 6. Present MENU OPTIONS
+### 7. Present MENU OPTIONS
 
 Display: "**Extraction Summary — Select an Option:** [C] Continue to compilation"
 
@@ -154,7 +194,7 @@ Display: "**Extraction Summary — Select an Option:** [C] Continue to compilati
 
 ## CRITICAL STEP COMPLETION NOTE
 
-ONLY WHEN the extraction inventory is built with provenance citations and the user has confirmed the extraction summary will you proceed to load `{nextStepFile}` for QMD enrichment.
+ONLY WHEN the extraction inventory is built with provenance citations and the user has confirmed the extraction summary will you proceed to load `{nextStepFile}` for temporal context fetching.
 
 ---
 

package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md

@@ -0,0 +1,238 @@
+---
+name: 'step-03b-fetch-temporal'
+description: 'Fetch temporal context (issues, PRs, changelogs) and index into QMD — Deep tier only'
+nextStepFile: './step-03c-fetch-docs.md'
+---
+
+# Step 3b: Fetch Temporal Context
+
+## STEP GOAL:
+
+To fetch temporal context (issues, PRs, changelogs, release notes) from the source repository and index it into a QMD collection for Deep tier enrichment. This ensures step-04 has historical data to search when annotating extracted functions with T2 provenance.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- 📖 CRITICAL: Read the complete step file before taking any action
+- 🎯 ALWAYS follow the exact instructions in the step file
+- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a tool you do not have access to, you MUST still achieve the outcome in your main context thread
+- ✅ YOU MUST ALWAYS SPEAK OUTPUT in your Agent communication style with the config `{communication_language}`
+
+### Role Reinforcement:
+
+- ✅ You are a skill compilation engine fetching temporal data for QMD enrichment
+- ✅ Temporal context is best-effort — failures never block the workflow
+- ✅ Graceful degradation is paramount — non-GitHub repos or missing tools are silent skips, not errors
+
+### Step-Specific Rules:
+
+- 🎯 Deep tier only — Quick and Forge tiers skip this step entirely and silently
+- 🎯 GitHub repositories only — other source types degrade gracefully
+- 🚫 FORBIDDEN to halt the workflow if fetching or indexing fails
+- 🚫 FORBIDDEN to modify extraction data from step-03 — this step only creates QMD collections
+- 💾 Write fetched content to a staging directory, index into QMD, then clean up
+
+## EXECUTION PROTOCOLS:
+
+- 🎯 Follow MANDATORY SEQUENCE exactly
+- 💾 Write fetched markdown files to a staging directory: `_bmad-output/{skill-name}-temporal/`
+- 📖 Index the staging directory into QMD and register in `forge-tier.yaml`
+- 🚫 Never leave staging files on disk after indexing — always clean up
+
+## CONTEXT BOUNDARIES:
+
+- Available: brief_data, tier, source_location from step-01; extraction_inventory from step-03
+- **Used from extraction_inventory:** `top_exports[]` — the list of top-level public API function names (typically 10-20). Used for targeted GitHub searches (section 3b).
+- Focus: Creating a QMD temporal collection for the source repository
+- Limits: Do NOT modify extraction data, begin enrichment, or compile content
+- Dependencies: Extraction must be complete from step-03
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+### 1. Check Eligibility
+
+Evaluate the following conditions sequentially. **If ANY condition fails, skip silently to section 5 (auto-proceed) with no output:**
+
+1. **Tier is Deep:** If tier is Quick or Forge, skip silently.
+2. **Source is GitHub:** Verify `source_repo` is a GitHub URL (`https://github.com/...`) or `owner/repo` format. If the source is a local path, a non-GitHub URL, or any other format, skip silently.
+3. **`gh` CLI is available:** Run `gh auth status` to verify the CLI is installed and authenticated. If it fails, skip silently.
+
+All three conditions must pass to proceed to section 2.
+
+### 2. Check Cache (Skip If Fresh)
+
+Read `forge-tier.yaml` from the sidecar path.
+
+- Look for a `qmd_collections` entry where `skill_name` matches the current brief AND `type` is `"temporal"`.
+- If found AND `created_at` is within the last **7 days**: the temporal collection is fresh. Display:
+
+"**Temporal context: cached.** Collection `{skill-name}-temporal` is fresh ({days} days old). Skipping re-fetch."
+
+Skip to section 5 (auto-proceed).
+
+- If not found OR `created_at` is older than 7 days: continue to section 3.
+
+### 3. Fetch Temporal Context
+
+Create a staging directory: `_bmad-output/{skill-name}-temporal/`
+
+Resolve the `owner` and `repo` from `source_repo` (e.g., `acme/toolkit` from `https://github.com/acme/toolkit`).
+
+Execute the following fetches, writing output as markdown files to the staging directory. **If any individual fetch fails, log a warning and continue with the others:**
+
+1. **Issues (last 100):**
+
+   ```bash
+   gh issue list -R {owner}/{repo} --state all --limit 100 --json number,title,state,labels,createdAt,closedAt,body | ...
+   ```
+
+   Write to `{staging}/issues.md` — format as a markdown document with one section per issue (number, title, state, labels, body summary).
+
+2. **Merged PRs (last 100):**
+
+   ```bash
+   gh pr list -R {owner}/{repo} --state merged --limit 100 --json number,title,mergedAt,labels,body | ...
+   ```
+
+   Write to `{staging}/prs.md` — format as a markdown document with one section per PR.
+
+3. **Releases (last 10):**
+
+   **Note:** `gh release list --json` does **not** support the `body` field. Use a two-step approach: list tags first, then fetch each release individually with `--json` (which IS supported on `gh release view`).
+
+   ```bash
+   # Step 1: Get release tags (body NOT available here)
+   gh release list -R {owner}/{repo} --limit 10 --json tagName,name,publishedAt
+   ```
+
+   If Step 1 returns an empty array (no releases), skip Step 2 and omit the releases section entirely.
+
+   ```bash
+   # Step 2: For EACH tagName from Step 1, fetch the full release
+   gh release view {tagName} -R {owner}/{repo} --json tagName,name,publishedAt,body
+   ```
+
+   Iterate over every `tagName` from Step 1's JSON array. If `gh release view` fails for a specific tag, log a warning and skip that release — continue with remaining tags. If a rate limit (HTTP 429) is hit, stop the release loop, keep results collected so far, and log: "Release fetch stopped at tag {N}/{total} due to rate limiting."
+
+   Write to `{staging}/releases.md` — format as a markdown document with one section per release (tag, name, date, body).
+
+4. **Changelog (if exists):**
+
+   Check if `CHANGELOG.md` or `RELEASES.md` exists in the repository root:
+
+   ```bash
+   gh api repos/{owner}/{repo}/contents/CHANGELOG.md --jq '.content' | base64 -d
+   ```
+
+   If found, write to `{staging}/changelog.md`. If not found (404), skip silently.
+
+#### 3b. Targeted Function Searches (Uses Extraction Inventory)
+
+After the generic fetches above, perform **targeted searches** using the top-level public API function names from `extraction_inventory.top_exports[]`. This produces high-signal results that generic list fetches miss.
+
+**Limit:** Search the top **10 function names** maximum to control API call volume and avoid `gh` rate limiting.
+
+For each function name in `top_exports[]` (up to 10):
+
+```bash
+gh search issues --repo {owner}/{repo} "{function_name}" --limit 5 --json number,title,state,body
+```
+
+Aggregate all targeted search results into a single file: `{staging}/targeted-issues.md`. Format as a markdown document with one section per function name, listing the matching issues/PRs found.
+
+**If `gh search` is unavailable** (older `gh` CLI versions): skip targeted searches silently. The generic fetches from section 3 still provide baseline temporal context.
+
+**If rate limiting occurs** (HTTP 429 or similar): stop targeted searches immediately, keep results collected so far. Log: "Targeted search stopped at function {N}/{total} due to rate limiting."
+
+**After all fetching,** verify at least one file was written to the staging directory. If the staging directory is empty (all fetches failed), log a warning and skip to section 5.
+
+### 4. Index Into QMD & Register
+
+**Index the staging directory:**
+
+If a `{skill-name}-temporal` collection already exists, remove and recreate for atomic replace:
+
+```bash
+qmd collection remove {skill-name}-temporal
+qmd collection add _bmad-output/{skill-name}-temporal/ --name {skill-name}-temporal --mask "*.md"
+qmd embed
+```
+
+**Note:** `qmd embed` generates vector embeddings required for `vector_search` and `deep_search`. Without it, only BM25 keyword `search` works. Run it after every `qmd collection add`.
+
+**Update the registry** in `forge-tier.yaml`:
+
+If an entry with `name: "{skill-name}-temporal"` already exists in `qmd_collections`, replace it. Otherwise, append:
+
+```yaml
+  - name: "{skill-name}-temporal"
+    type: "temporal"
+    source_workflow: "create-skill"
+    skill_name: "{skill-name}"
+    created_at: "{current ISO date}"
+```
+
+**Clean up** the staging directory after successful indexing:
+
+```bash
+rm -rf _bmad-output/{skill-name}-temporal/
+```
+
+**Error handling:**
+
+- If QMD indexing fails: log the error, note that temporal enrichment will be unavailable. Do NOT fail the workflow.
+- If registry update fails: log the error, continue. The collection may exist in QMD even if the registry entry failed.
+- If cleanup fails: log a warning and continue.
+
+Display brief confirmation:
+
+"**Temporal context indexed.** Collection `{skill-name}-temporal` created ({file_count} files: {list files}). Proceeding to enrichment..."
+
+### 5. Menu Handling Logic
+
+**Auto-proceed step — no user interaction.**
+
+After temporal context is fetched and indexed (or skipped for any reason), immediately load, read entire file, then execute `{nextStepFile}`.
+
+#### EXECUTION RULES:
+
+- This is an auto-proceed step with no user choices
+- Quick/Forge tiers skip directly to next step with no output
+- Non-GitHub sources skip directly to next step with no output
+- Cached collections (< 7 days old) skip with brief cache-hit message
+- Deep tier with fresh fetch displays brief confirmation then auto-proceeds
+- All failures degrade gracefully — skip and auto-proceed
+
+## CRITICAL STEP COMPLETION NOTE
+
+ONLY WHEN temporal context is indexed into QMD (or the step is skipped due to eligibility, cache, or failure) will you proceed to load `{nextStepFile}` for QMD enrichment.
+
+---
+
+## 🚨 SYSTEM SUCCESS/FAILURE METRICS
+
+### ✅ SUCCESS:
+
+- Non-eligible scenarios (Quick/Forge tier, non-GitHub source, no `gh` CLI) skipped silently
+- Cached collections (< 7 days old) detected and re-fetch skipped
+- Temporal data fetched via `gh` CLI into staging directory (generic + targeted)
+- Targeted searches performed for up to 10 top_exports function names
+- At least one temporal file written (issues, PRs, releases, changelog, or targeted-issues)
+- Collection `{skill-name}-temporal` indexed into QMD
+- Registry entry added/updated in forge-tier.yaml with type `"temporal"`
+- Staging directory cleaned up after indexing
+- Auto-proceeded to step-04
+
+### ❌ SYSTEM FAILURE:
+
+- Halting the workflow due to a `gh` CLI, QMD, or network failure
+- Leaving staging files on disk after indexing (must clean up)
+- Overwriting or modifying extraction data from step-03
+- Displaying skip messages for Quick/Forge tiers (should be silent)
+- Attempting to fetch temporal data from non-GitHub sources
+- Not registering the collection in forge-tier.yaml after successful indexing
+
+**Master Rule:** Temporal context is best-effort enrichment data. Fetch what you can, index it, clean up, and move on. Failures degrade gracefully — they never block the skill compilation pipeline.

package/src/workflows/create-skill/steps-c/step-03c-fetch-docs.md

@@ -0,0 +1,220 @@
+---
+name: 'step-03c-fetch-docs'
+description: 'Fetch remote documentation from URLs for T3-confidence content — all tiers, tool-agnostic'
+nextStepFile: './step-04-enrich.md'
+---
+
+# Step 3c: Fetch Remote Documentation
+
+## STEP GOAL:
+
+Fetch remote documentation from brief-specified URLs using whatever web fetching capability is available in the agent's environment, extract API information, and add T3-confidence content to the extraction inventory. Tool-agnostic — the agent uses Firecrawl, WebFetch, web-reader, curl, or any available web tool.
+
+## MANDATORY EXECUTION RULES (READ FIRST):
+
+### Universal Rules:
+
+- CRITICAL: Read the complete step file before taking any action
+- ALWAYS follow the exact instructions in the step file
+- TOOL/SUBPROCESS FALLBACK: If any instruction references a tool you do not have access to, you MUST still achieve the outcome in your main context thread
+- YOU MUST ALWAYS SPEAK OUTPUT in your Agent communication style with the config `{communication_language}`
+
+### Role Reinforcement:
+
+- You are a skill compilation engine fetching external documentation for T3 enrichment
+- Doc fetching is best-effort — individual URL failures never block the workflow
+- Graceful degradation is paramount — if no web fetching tools are available, skip with a warning and proceed
+
+### Step-Specific Rules:
+
+- No tier gate — this step runs at any tier when `doc_urls` are present in the brief
+- Tool-agnostic — use whatever web fetching capability is available (Firecrawl, WebFetch, web-reader, curl, browser tools, etc.)
+- FORBIDDEN to halt the workflow if web fetching is unavailable or fails
+- FORBIDDEN to override existing T1, T1-low, or T2 extraction data with T3 content
+- FORBIDDEN to begin compilation — that is step-05
+
+## EXECUTION PROTOCOLS:
+
+- Follow MANDATORY SEQUENCE exactly
+- Every extracted item receives a T3 confidence citation: `[EXT:{url}]`
+- Merge doc-fetch results into the extraction inventory per the conflict rule in section 5
+- Do NOT hallucinate content — if information is not in the fetched text, exclude it
+
+## CONTEXT BOUNDARIES:
+
+- Available: brief_data (including `doc_urls`, `source_type`), tier, extraction_inventory from step-03
+- Focus: Fetching external documentation and producing T3-confidence extraction items
+- Limits: Do NOT modify existing T1/T1-low/T2 items, begin enrichment, or compile content
+- Dependencies: Step-03 extraction must be complete (even if inventory is empty for docs-only briefs)
+
+## MANDATORY SEQUENCE
+
+**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise.
+
+### 1. Check Eligibility
+
+Evaluate the following conditions. **If the condition fails, skip silently to section 7 (auto-proceed) with no output:**
+
+1. **`doc_urls` is present in the brief data:** Check that `doc_urls` contains at least one URL entry from step-01 context. If `doc_urls` is absent or empty, skip silently.
+
+No tier gate — if `doc_urls` are present, this step runs at Quick, Forge, and Deep tiers alike.
+
+### 2. Security Notice
+
+Display an informational notice (not a gate — the user already approved these URLs in the brief):
+
+"**Documentation fetch:** The following external URLs will be fetched:
+{for each URL: `- {label}: {url}`}
+
+Content fetched from external URLs is classified as **T3** (external, untrusted) and cited as `[EXT:{url}]`."
+
+### 3. Fetch Documentation
+
+**Discover available web fetching capability.** Try tools in any order — use whatever is accessible in the current environment (e.g., Firecrawl scrape, WebFetch, web-reader, MCP fetch, curl, browser tools). If no web fetching capability can be found:
+
+- Log warning: "No web fetching capability available in this environment. Skipping documentation fetch."
+- Skip to section 7 (auto-proceed).
+
+**For each URL in `doc_urls`:**
+
+- Fetch the content at `{url}` as clean markdown using the discovered web tool.
+- **If fetch succeeds:** Store the markdown content with the URL as provenance source.
+- **If fetch fails:** Log warning: "Failed to fetch {url}: {reason}. Skipping." Continue with remaining URLs.
+
+**Subpage discovery (root URL detection):**
+
+After fetching a URL, apply the following heuristic to detect documentation root pages that contain no useful API content. This is common with modern documentation sites (Mintlify, Docusaurus, ReadTheDocs, GitBook) that render API content on subpages.
+
+**Root page detection heuristic — a page is a root if BOTH conditions are true:**
+
+1. **Zero API content indicators:** The fetched markdown contains none of: fenced code blocks (`` ``` ``), parameter tables (`|---|`), or function signature patterns (`def `, `function `, `fn `, `func `, `export `).
+2. **High link density:** More than 70% of non-empty lines are markdown links (matching `[text](url)` with no other substantive content on the line).
+
+If only one condition is true, treat the page as having partial content — keep it as-is and do NOT trigger subpage discovery.
+
+**If a root URL with minimal content is detected:**
+
+1. **Attempt sitemap/map discovery:** Use whatever discovery tool is available:
+   - Firecrawl: `firecrawl_map({url})` to discover all subpages
+   - Manual: try fetching `{url}/sitemap.xml` and parsing URLs from it
+   - Crawl: if a crawl tool is available, use it with depth=1 on the root URL
+   - If no discovery tool is available, keep the root page content as-is and continue
+
+2. **Filter discovered URLs by relevance:** From the discovered subpage list, select the most relevant pages by searching for API-related terms in the URL path or title (e.g., `api`, `reference`, `quickstart`, `setup`, `config`, `getting-started`, `guide`, `sdk`, `methods`, `functions`). Exclude pages that are clearly non-API content (e.g., `blog`, `changelog`, `pricing`, `about`, `careers`).
+
+3. **Fetch top subpages:** Fetch up to **10** of the most relevant subpages. For each:
+   - Use the same web fetching tool as the root URL
+   - Store with the subpage URL as provenance: `[EXT:{subpage-url}]`
+   - If a subpage fetch fails, skip it and continue
+
+4. **Rate limiting:** If rate limiting (HTTP 429) is encountered during subpage fetching, stop discovery for this root URL. Keep results collected so far. Log: "Subpage discovery stopped due to rate limiting."
+
+**If ALL URLs fail (including any subpage fetches):** Log warning: "No documentation could be fetched. Proceeding without T3 content." Skip to section 7 (auto-proceed).
+
+### 4. Extract API Information from Fetched Content
+
+Parse the successfully fetched markdown for:
+
+- **Function/method signatures** and their parameters
+- **Return types** and data structures
+- **Configuration options** and their defaults
+- **Usage examples** and code snippets
+
+**Citation rule:** Every extracted item gets a T3 confidence citation: `[EXT:{url}]` where `{url}` is the source URL the item was extracted from.
+
+**No hallucination:** If information cannot be found in the fetched content, exclude it. Do not infer or fabricate API details.
+
+### 5. Build Doc-Fetch Inventory
+
+**Mode determines merge behavior:**
+
+- **`source_type: "docs-only"`** — The doc-fetch inventory IS the extraction inventory. It replaces the empty inventory from step-03, since there was no source code to extract from.
+- **`source_type: "source"` (supplemental mode)** — Merge T3 items into the existing extraction inventory from step-03.
+
+**Conflict rule:** T3 items NEVER override existing T1, T1-low, or T2 items for the same export. When an export already has a higher-confidence entry, the T3 item is discarded. T3 has the lowest priority.
+
+**Aggregate totals for reporting:**
+- URLs fetched successfully vs. total
+- URLs that failed
+- T3 items extracted
+
+### 5b. Index into QMD (Deep Tier Only)
+
+**If tier is not Deep:** Skip this section silently.
+
+**If tier is Deep and at least one URL was fetched successfully:**
+
+1. Write fetched markdown files to a staging directory: `_bmad-output/{skill-name}-docs/`
+2. Index into QMD: `qmd collection add _bmad-output/{skill-name}-docs/ --name {skill-name}-docs --mask "*.md"`
+3. Generate embeddings: `qmd embed` (required for `vector_search` and `deep_search`)
+4. Register in forge-tier.yaml `qmd_collections` array:
+
+```yaml
+- name: "{skill-name}-docs"
+  type: "docs"
+  source_workflow: "create-skill"
+  skill_name: "{skill-name}"
+  created_at: "{current ISO date}"
+```
+
+5. Clean up staging directory after indexing: `rm -rf _bmad-output/{skill-name}-docs/`
+
+**If QMD indexing fails:** Warn: "QMD indexing of fetched docs failed. T3 items are still in the extraction inventory — enrichment will proceed without QMD-indexed docs." Continue.
+
+### 6. Report
+
+Display:
+
+"**Documentation fetch complete.**
+**URLs processed:** {fetched}/{total}
+**T3 items extracted:** {count}
+**Confidence:** All doc-fetched items are T3 — `[EXT:{url}]` citations applied.
+{If docs-only mode: '**Mode:** Docs-only — all skill content is T3. source_authority: community'}
+
+Proceeding to enrichment..."
+
+### 7. Menu Handling Logic
+
+**Auto-proceed step — no user interaction.**
+
+After documentation fetch is complete (or skipped for any reason), immediately load, read entire file, then execute `{nextStepFile}`.
+
+#### EXECUTION RULES:
+
+- This is an auto-proceed step with no user choices
+- No `doc_urls` in brief: skip directly to next step with no output
+- No web fetching available: skip with warning then auto-proceed
+- All URLs failed: skip with warning then auto-proceed
+- Successful fetch: display report then auto-proceed
+- All failures degrade gracefully — skip and auto-proceed
+
+## CRITICAL STEP COMPLETION NOTE
+
+ONLY WHEN documentation is fetched and T3 items are merged into the extraction inventory (or the step is skipped due to no `doc_urls`, no web tools, or fetch failures) will you proceed to load `{nextStepFile}` for enrichment.
+
+---
+
+## SYSTEM SUCCESS/FAILURE METRICS
+
+### SUCCESS:
+
+- No `doc_urls` in brief: skipped silently, auto-proceeded
+- `doc_urls` present: each URL fetched using whatever web tool is available
+- Root URLs with minimal content: subpage discovery attempted, relevant subpages fetched
+- Individual fetch failures handled gracefully (skip and continue)
+- All extracted content cited as T3 with `[EXT:{url}]` provenance
+- Existing T1/T1-low/T2 items never overridden by T3 data
+- Docs-only mode: doc-fetch inventory correctly replaces empty extraction inventory
+- Supplemental mode: T3 items merged into existing inventory respecting conflict rule
+- Auto-proceeded to step-04
+
+### SYSTEM FAILURE:
+
+- Halting the workflow because web fetching is unavailable or a URL fails
+- Including fetched content without `[EXT:{url}]` citations
+- Overriding existing higher-confidence extractions (T1, T1-low, T2) with T3 data
+- Hardcoding a specific fetching tool instead of being tool-agnostic
+- Hallucinating API details not found in the fetched content
+- Beginning compilation in this step (that is step-05)
+
+**Master Rule:** Documentation fetching is best-effort T3 enrichment. Fetch what you can, cite everything as `[EXT:{url}]`, never override higher-confidence data, and move on. Failures degrade gracefully — they never block the skill compilation pipeline.