bmad-module-skill-forge 0.3.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/README.md +46 -176
  2. package/docs/agents.md +1 -1
  3. package/docs/concepts.md +96 -0
  4. package/docs/examples.md +39 -0
  5. package/docs/getting-started.md +15 -1
  6. package/docs/{architecture.md → how-it-works.md} +106 -5
  7. package/docs/index.md +14 -41
  8. package/docs/workflows.md +1 -1
  9. package/package.json +2 -2
  10. package/src/knowledge/agentskills-spec.md +25 -0
  11. package/src/knowledge/manual-section-integrity.md +8 -0
  12. package/src/knowledge/overview.md +2 -1
  13. package/src/knowledge/provenance-tracking.md +13 -0
  14. package/src/knowledge/skf-knowledge-index.csv +3 -2
  15. package/src/knowledge/skill-lifecycle.md +2 -2
  16. package/src/knowledge/split-body-strategy.md +41 -0
  17. package/src/module.yaml +2 -2
  18. package/src/workflows/analyze-source/data/skill-brief-schema.md +2 -0
  19. package/src/workflows/analyze-source/data/unit-detection-heuristics.md +26 -0
  20. package/src/workflows/analyze-source/steps-c/step-02-scan-project.md +1 -1
  21. package/src/workflows/analyze-source/steps-c/step-03-identify-units.md +1 -1
  22. package/src/workflows/analyze-source/steps-c/step-04-map-and-detect.md +5 -4
  23. package/src/workflows/analyze-source/steps-c/step-05-recommend.md +1 -1
  24. package/src/workflows/analyze-source/steps-c/step-06-generate-briefs.md +1 -1
  25. package/src/workflows/audit-skill/steps-c/step-01-init.md +2 -0
  26. package/src/workflows/audit-skill/steps-c/step-03-structural-diff.md +14 -0
  27. package/src/workflows/brief-skill/data/scope-templates.md +4 -0
  28. package/src/workflows/brief-skill/data/skill-brief-schema.md +4 -0
  29. package/src/workflows/brief-skill/steps-c/step-03-scope-definition.md +12 -0
  30. package/src/workflows/create-skill/data/compile-assembly-rules.md +43 -2
  31. package/src/workflows/create-skill/data/extraction-patterns-tracing.md +110 -0
  32. package/src/workflows/create-skill/data/extraction-patterns.md +12 -81
  33. package/src/workflows/create-skill/data/skill-sections.md +31 -5
  34. package/src/workflows/create-skill/data/source-resolution-protocols.md +9 -1
  35. package/src/workflows/create-skill/data/tier-degradation-rules.md +46 -0
  36. package/src/workflows/create-skill/steps-c/step-03-extract.md +24 -2
  37. package/src/workflows/create-skill/steps-c/step-05-compile.md +8 -4
  38. package/src/workflows/create-skill/steps-c/step-06-validate.md +12 -10
  39. package/src/workflows/create-skill/steps-c/step-07-generate-artifacts.md +14 -1
  40. package/src/workflows/create-stack-skill/steps-c/step-06-compile-stack.md +2 -2
  41. package/src/workflows/create-stack-skill/steps-c/step-08-validate.md +1 -1
  42. package/src/workflows/export-skill/data/snippet-format.md +1 -1
  43. package/src/workflows/export-skill/steps-c/step-02-package.md +7 -3
  44. package/src/workflows/export-skill/steps-c/step-03-generate-snippet.md +3 -1
  45. package/src/workflows/export-skill/workflow.md +1 -1
  46. package/src/workflows/quick-skill/steps-c/step-04-compile.md +3 -2
  47. package/src/workflows/test-skill/data/scoring-rules.md +9 -0
  48. package/src/workflows/test-skill/data/source-access-protocol.md +8 -0
  49. package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +3 -1
  50. package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +8 -5
  51. package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +11 -1
  52. package/src/workflows/test-skill/steps-c/step-05-score.md +5 -4
  53. package/src/workflows/test-skill/steps-c/step-06-report.md +27 -0
  54. package/src/workflows/update-skill/data/tier-degradation-rules.md +46 -0
  55. package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +11 -0
  56. package/src/workflows/update-skill/steps-c/step-03-re-extract.md +1 -0
  57. package/src/workflows/update-skill/steps-c/step-04-merge.md +8 -0
  58. package/src/workflows/update-skill/steps-c/step-05-validate.md +1 -1
  59. package/src/workflows/update-skill/steps-c/step-06-write.md +6 -1
  60. package/tools/cli/lib/ui.js +1 -1
@@ -0,0 +1,110 @@
1
+ # Extraction Patterns: Tracing and File-Level Extraction
2
+
3
+ This file covers re-export tracing protocols and script/asset file-level extraction patterns. For core tier strategies and AST extraction protocol, see `extraction-patterns.md`.
4
+
5
+ ---
6
+
7
+ ## Re-Export Tracing
8
+
9
+ After initial AST extraction, some top-level exports may resolve to **module imports** rather than direct function definitions. This is common in Python libraries that use `__init__.py` re-exports for a clean public API.
10
+
11
+ **Detection heuristic:** For each top-level export from `__init__.py` (or equivalent entry point), check if the import path resolves to a directory (contains `__init__.py`) rather than a `.py` file with a matching `def` or `class`. If the initial AST scan found no function/class definition for a known public export, it is likely a module re-export.
12
+
13
+ **Tracing protocol:**
14
+
15
+ 1. Read the entry point file (e.g., `{package}/__init__.py`) and extract all `from .X import Y` statements
16
+ 2. For each import where Y was NOT found by the initial AST scan:
17
+ - Check if the import path resolves to a directory (e.g., `{package}/api/v1/delete/` exists with `__init__.py`)
18
+ - If directory: read its `__init__.py` to find the actual re-exported symbol
19
+ - **Handle aliases:** Check for `from .module import A as B` patterns in the intermediate `__init__.py`. If the parent imports `B`, trace through to `A` in `.module`. If the parent imports `A` but the `__init__.py` only exports it as `B` (via `from .module import A as B`), match by original name `A` and note the alias
20
+ - Trace the symbol to its definition file and run AST extraction on that file
21
+ 3. Cite the actual definition location: `[AST:{definition_file}:L{line}]`
22
+
23
+ **Examples:**
24
+
25
+ ```python
26
+ # Module re-export — follow required
27
+ from .api.v1.delete import delete # delete/ is a directory → read delete/__init__.py
28
+
29
+ # Direct function import — no follow needed
30
+ from .api.v1.add.add import add # add.py exists with def add()
31
+
32
+ # Aliased re-export — follow through alias
33
+ # In cognee/api/v1/visualize/__init__.py:
34
+ # from .start_visualization_server import visualization_server
35
+ # In cognee/__init__.py:
36
+ # from .api.v1.visualize import start_visualization_server
37
+ # → Match start_visualization_server against both definition names AND alias names
38
+ # in the intermediate __init__.py to resolve the chain
39
+ ```
40
+
41
+ **Unresolvable imports:** If the import statement is a star-import (`from .X import *`) or a conditional import (`try`/`except`), the symbol cannot be reliably traced via this protocol. Record it with `[SRC:{package}/__init__.py:L{line}]` (T1-low) and a note: "star/conditional import — manual trace required."
42
+
43
+ **Scope limit:** Only trace re-exports for symbols listed in the top-level entry point's public API. Do not recursively trace beyond one level of `__init__.py` indirection. If a re-export cannot be resolved after one level, record it with a `[SRC:{package}/__init__.py:L{line}]` citation (T1-low) from the import statement itself.
44
+
45
+ **Other languages:** JS/TS barrel files (`index.ts` with `export { X } from './module'`) follow the same principle — trace the re-export to the definition file. Rust `pub use` and Go package-level re-exports are less common but follow the same heuristic when encountered.
46
+
47
+ ---
48
+
49
+ ## Script/Asset Extraction Patterns
50
+
51
+ Scripts and assets are file-level artifacts, not code exports. They follow the **file-copy extraction method** — detected in source, copied with provenance citations.
52
+
53
+ ### Detection Heuristics
54
+
55
+ **Script directories:** `scripts/`, `bin/`, `tools/`, `cli/`
56
+ **Asset directories:** `assets/`, `templates/`, `schemas/`, `configs/`, `examples/`
57
+
58
+ **Script file signals:**
59
+
60
+ | Signal | Strength | Pattern |
61
+ |--------|----------|---------|
62
+ | Entry point declaration | Strong | `package.json` `bin` field, Cargo.toml `[[bin]]`, pyproject.toml `[project.scripts]` |
63
+ | Shebang + executable | Strong | `#!/bin/bash`, `#!/usr/bin/env python`, `#!/usr/bin/env node` |
64
+ | CLI argument parser | Moderate | `argparse`, `yargs`, `commander`, `cobra`, `clap` imports in file |
65
+ | Directory convention | Moderate | File in `scripts/`, `bin/`, `tools/` directory |
66
+ | CI/CD reference | Moderate | Script referenced in `.github/workflows/*.yml` |
67
+
68
+ **Asset file signals:**
69
+
70
+ | Signal | Strength | Pattern |
71
+ |--------|----------|---------|
72
+ | JSON Schema | Strong | `*.schema.json`, file contains `"$schema"` key |
73
+ | Config template | Strong | `*.example`, `*.template.*`, `*.sample` extension |
74
+ | Official example | Moderate | File in `examples/` directory, referenced in README |
75
+ | OpenAPI/GraphQL | Moderate | `openapi.json`, `*.graphql`, `swagger.yaml` |
76
+ | Design tokens | Weak | `tokens.json`, `theme.json` in `assets/` |
77
+
78
+ ### Inclusion Rules
79
+
80
+ - Only include files within brief's `scope.include` patterns (or auto-detected directories)
81
+ - Exclude binary files (check extension: `.so`, `.dll`, `.jar`, `.wasm`, `.exe`)
82
+ - Exclude generated files (`dist/`, `build/`, `.webpack/` output)
83
+ - Exclude vendored/third-party files
84
+ - Flag files >500 lines for user confirmation (may be too large for skill package)
85
+ - If `scripts_intent: "none"` or `assets_intent: "none"` in brief, skip that category
86
+
87
+ ### Provenance and Hashing
88
+
89
+ Each extracted file receives:
90
+ - Citation: `[SRC:{source_path}:L1]` (T1-low — file verified to exist but content not AST-analyzed)
91
+ - Content hash: SHA-256 of file content (for drift detection in audit-skill)
92
+ - Extraction method: `"file-copy"` (distinct from `"ast_bridge"` for code exports)
93
+
94
+ ### Inventory Structure
95
+
96
+ **Script inventory entry:**
97
+ - `name`: filename (e.g., `validate-config.sh`)
98
+ - `source_file`: path relative to source root
99
+ - `purpose`: extracted from file header comments or README reference (if none found, use filename)
100
+ - `language`: detected from extension or shebang
101
+ - `content_hash`: SHA-256
102
+ - `confidence`: T1-low
103
+
104
+ **Asset inventory entry:**
105
+ - `name`: filename (e.g., `config-schema.json`)
106
+ - `source_file`: path relative to source root
107
+ - `purpose`: extracted from file header or schema `title`/`description` field
108
+ - `type`: `template`, `schema`, `config`, `example`
109
+ - `content_hash`: SHA-256
110
+ - `confidence`: T1-low
@@ -26,13 +26,16 @@ Source reading via gh_bridge — infer exports from file structure and content.
26
26
 
27
27
  ## Forge Tier (AST Available)
28
28
 
29
- Structural extraction via ast_bridge — verified exports with line-level citations.
29
+ Structural extraction via ast-grep — verified exports with line-level citations.
30
+
31
+ > **Note:** `ast_bridge.*` and `qmd_bridge.*` references below are **conceptual interfaces**, not callable functions. They describe the operation to perform. Use ast-grep (MCP tool or CLI) for `ast_bridge.*` operations and QMD (MCP tool or CLI) for `qmd_bridge.*` operations. See the AST Extraction Protocol section below and the TOOL/SUBPROCESS FALLBACK rule for dispatch details.
30
32
 
31
33
  ### Strategy
34
+
32
35
  1. Detect language from brief or file extensions
33
- 2. `ast_bridge.scan_definitions(path, language)` extract all exports
36
+ 2. Use ast-grep to extract all exports from `path` for the given `language` (scan definitions)
34
37
  3. For each export: function name, full signature, parameter types, return type, line number
35
- 4. `ast_bridge.detect_co_imports(path, libraries[])` — find integration points
38
+ 4. Use ast-grep to detect co-imported symbols in `path` for the given `libraries[]`
36
39
  5. Build extraction rules YAML for reproducibility
37
40
 
38
41
  ### Confidence
@@ -130,7 +133,8 @@ When MCP tools are unavailable or the repo exceeds 500 files in scope, use `--js
130
133
  # Patterns are matched against the full file path as emitted by ast-grep.
131
134
  # Ensure paths are relative to the same root as the patterns (strip ./ prefix if needed).
132
135
  ast-grep -p '{pattern}' -l {language} --json=stream {path} | python3 -c "
133
- import sys, json, fnmatch
136
+ import sys, json, fnmatch, signal
137
+ signal.signal(signal.SIGPIPE, signal.SIG_DFL)
134
138
 
135
139
  EXCLUDES = {exclude_patterns}
136
140
 
@@ -230,82 +234,9 @@ constraints:
230
234
  regex: '^[A-Z]'
231
235
  ```
232
236
 
233
- ### Re-Export Tracing
234
-
235
- After initial AST extraction, some top-level exports may resolve to **module imports** rather than direct function definitions. This is common in Python libraries that use `__init__.py` re-exports for a clean public API.
236
-
237
- **Detection heuristic:** For each top-level export from `__init__.py` (or equivalent entry point), check if the import path resolves to a directory (contains `__init__.py`) rather than a `.py` file with a matching `def` or `class`. If the initial AST scan found no function/class definition for a known public export, it is likely a module re-export.
238
-
239
- **Tracing protocol:**
240
-
241
- 1. Read the entry point file (e.g., `{package}/__init__.py`) and extract all `from .X import Y` statements
242
- 2. For each import where Y was NOT found by the initial AST scan:
243
- - Check if the import path resolves to a directory (e.g., `{package}/api/v1/delete/` exists with `__init__.py`)
244
- - If directory: read its `__init__.py` to find the actual re-exported symbol
245
- - Trace the symbol to its definition file and run AST extraction on that file
246
- 3. Cite the actual definition location: `[AST:{definition_file}:L{line}]`
247
-
248
- **Examples:**
249
-
250
- ```python
251
- # Module re-export — follow required
252
- from .api.v1.delete import delete # delete/ is a directory → read delete/__init__.py
253
-
254
- # Direct function import — no follow needed
255
- from .api.v1.add.add import add # add.py exists with def add()
256
- ```
257
-
258
- **Unresolvable imports:** If the import statement is a star-import (`from .X import *`) or a conditional import (`try`/`except`), the symbol cannot be reliably traced via this protocol. Record it with `[SRC:{package}/__init__.py:L{line}]` (T1-low) and a note: "star/conditional import — manual trace required."
259
-
260
- **Scope limit:** Only trace re-exports for symbols listed in the top-level entry point's public API. Do not recursively trace beyond one level of `__init__.py` indirection. If a re-export cannot be resolved after one level, record it with a `[SRC:{package}/__init__.py:L{line}]` citation (T1-low) from the import statement itself.
261
-
262
- **Other languages:** JS/TS barrel files (`index.ts` with `export { X } from './module'`) follow the same principle — trace the re-export to the definition file. Rust `pub use` and Go package-level re-exports are less common but follow the same heuristic when encountered.
263
-
264
- ---
265
-
266
- ## Tier Degradation Rules
267
-
268
- ### Remote Source at Forge/Deep Tier
269
-
270
- When `source_repo` is a remote URL (GitHub URL or owner/repo format) and the tier is Forge or Deep:
271
-
272
- - **ast-grep requires local files** — it cannot operate on remote URLs
273
-
274
- **Ephemeral clone strategy (preferred):**
275
-
276
- 1. Check `git` availability (`git --version`). `git` is effectively guaranteed at Deep tier (via `gh` dependency) but NOT guaranteed at Forge tier.
277
- 2. If `git` is available: perform an ephemeral shallow clone to a system temp path (`{system_temp}/skf-ephemeral-{skill-name}-{timestamp}/`).
278
- 3. For create-skill: use `--depth 1 --single-branch --filter=blob:none`; if `include_patterns` are specified, apply mode selection: if `exclude_patterns` are absent, use cone mode (convert include_patterns to directory roots; use `--skip-checks` for individual file paths); if `exclude_patterns` are present, use `--no-cone` mode (pass gitignore-style patterns with `!`-prefixed excludes — includes first, then negations). See source-resolution-protocols.md for the full conversion rules.
279
- 4. For update-skill: use sparse-checkout with `--skip-checks` scoped to the changed files from the change manifest only (file paths require `--skip-checks`). No `--branch` flag — uses the remote default branch (must match the branch used during original create-skill run).
280
- 5. If clone succeeds: use the local clone path for AST extraction. All results are T1 with `[AST:...]` citations.
281
- 6. Cleanup: delete the temp directory after extraction inventory is built and all data is in context. The clone never persists beyond the extraction step.
282
-
283
- **Fallback (clone fails or `git` unavailable):**
284
-
285
- - The extraction step MUST warn the user explicitly before degrading
286
- - **create-skill:** Warning must include actionable guidance — clone locally and update `source_repo` in the brief to the local path
287
- - **update-skill:** Warning must include actionable guidance — clone locally, re-run [CS] Create Skill with the local path to regenerate provenance data, then re-run the update
288
- - Extraction proceeds using Quick tier strategy (source reading via gh_bridge)
289
- - All results labeled T1-low with `[SRC:...]` citations
290
- - The degradation reason is recorded in the evidence report
291
-
292
- Silent degradation is **forbidden**. The user must always know when AST extraction was skipped and why.
293
-
294
- ### AST Tool Unavailable at Forge/Deep Tier
295
-
296
- When the tier is Forge or Deep but ast-grep is not functional:
297
-
298
- - The extraction step MUST warn the user explicitly before degrading
299
- - Warning must include actionable guidance: run [SF] Setup Forge to detect tools
300
- - Extraction proceeds using Quick tier strategy
301
- - All results labeled T1-low
302
- - The degradation reason is recorded in the evidence report
303
-
304
- ### Per-File AST Failure
237
+ ### Re-Export Tracing and Script/Asset Extraction
305
238
 
306
- When ast-grep fails on an individual file (parse error, unsupported syntax):
239
+ See `extraction-patterns-tracing.md` for:
240
+ - **Re-export tracing protocol** — resolving module imports through `__init__.py`, barrel files, `pub use`
241
+ - **Script/asset extraction patterns** — detection heuristics, inclusion rules, provenance, inventory structure
307
242
 
308
- - Fall back to source reading for **that file only**
309
- - Other files continue with AST extraction
310
- - The affected file's results are labeled T1-low; unaffected files retain T1
311
- - Log a warning noting which file degraded and why
@@ -17,8 +17,8 @@ description: >
17
17
 
18
18
  **Frontmatter rules (agentskills.io specification):**
19
19
 
20
- - `name`: 1-64 characters, lowercase alphanumeric + hyphens only, must match parent directory name
21
- - `description`: 1-1024 characters, trigger-optimized for agent matching
20
+ - `name`: 1-64 characters, lowercase alphanumeric + hyphens only, must match parent directory name. Prefer gerund form (`processing-pdfs`, `analyzing-spreadsheets`) for clarity; noun phrases and action-oriented forms are acceptable alternatives.
21
+ - `description`: 1-1024 characters, trigger-optimized for agent matching. MUST use third-person voice ("Processes..." not "I can..." or "You can...").
22
22
  - Only 6 fields permitted: `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools`
23
23
  - `version` and `author` belong in metadata.json, NOT in frontmatter
24
24
 
@@ -38,6 +38,7 @@ SKILL.md uses a two-tier structure to ensure actionable content survives `split-
38
38
  | 5 | **Key Types** | ~20 lines | Most important enum/type values inline |
39
39
  | 6 | **Architecture at a Glance** | ~10 lines | Bullet list of subsystem categories |
40
40
  | 7 | **CLI** | ~10 lines | Basic CLI commands (skip if no CLI) |
41
+ | 7b | **Scripts & Assets** | ~10 lines | Manifest of included scripts and assets (skip if none detected) |
41
42
  | 8 | **Manual Sections** | ~5 lines | `<!-- [MANUAL] -->` markers for update-skill |
42
43
 
43
44
  #### Tier 2 — Reference-Eligible (extracted by split-body into references/)
@@ -56,6 +57,7 @@ SKILL.md uses a two-tier structure to ensure actionable content survives `split-
56
57
  - After split-body, SKILL.md retains all Tier 1 content — actionable without loading references
57
58
  - An agent loading only SKILL.md (no references) must get enough to act
58
59
  - **Section 4b (Migration & Deprecation Warnings)** is conditional: only emitted for Deep tier when T2-future annotations exist. Quick/Forge tiers and Deep tiers without T2-future annotations omit it entirely (no empty section). Parsers and validators must treat this section as optional.
60
+ - **Section 7b (Scripts & Assets)** is conditional: only emitted when `scripts_inventory` or `assets_inventory` is non-empty. Omitted entirely when no scripts or assets are detected. Parsers and validators must treat this section as optional.
59
61
 
60
62
  ### Provenance Citation Format
61
63
 
@@ -121,11 +123,14 @@ Indexed pipe-delimited format for CLAUDE.md managed section (~80-120 tokens per
121
123
  "confidence_tier": "{Quick|Forge|Deep}",
122
124
  "spec_version": "1.3",
123
125
  "generation_date": "{ISO-8601}",
126
+ "description": "{SKILL.md frontmatter description}",
127
+ "language": "{primary-source-language}",
128
+ "ast_node_count": "{number-or-omitted-if-no-ast}",
124
129
  "exports": [],
125
130
  "tool_versions": {
126
131
  "ast_grep": "{version-or-null}",
127
132
  "qmd": "{version-or-null}",
128
- "skf": "1.0.0"
133
+ "skf": "{skf_version}"
129
134
  },
130
135
  "stats": {
131
136
  "exports_documented": 0,
@@ -136,8 +141,16 @@ Indexed pipe-delimited format for CLAUDE.md managed section (~80-120 tokens per
136
141
  "total_coverage": 0.0,
137
142
  "confidence_t1": 0,
138
143
  "confidence_t2": 0,
139
- "confidence_t3": 0
144
+ "confidence_t3": 0,
145
+ "scripts_count": 0,
146
+ "assets_count": 0
140
147
  },
148
+ "scripts": [
149
+ { "file": "scripts/{name}", "purpose": "{description}", "source_file": "{source-path}", "confidence": "T1-low" }
150
+ ],
151
+ "assets": [
152
+ { "file": "assets/{name}", "purpose": "{description}", "source_file": "{source-path}", "confidence": "T1-low" }
153
+ ],
141
154
  "dependencies": [],
142
155
  "compatibility": "{semver-range}"
143
156
  }
@@ -163,6 +176,7 @@ Each reference file includes:
163
176
  - Complete usage examples
164
177
  - Related functions cross-references
165
178
  - Temporal annotations (Deep tier: T2-past, T2-future)
179
+ - **Table of contents** (required for files exceeding 100 lines) — a `## Contents` section at the top listing all sub-sections for agent discoverability
166
180
 
167
181
  ---
168
182
 
@@ -186,10 +200,22 @@ Each reference file includes:
186
200
  "extraction_method": "ast_bridge.scan_definitions",
187
201
  "ast_node_type": "export_function_declaration"
188
202
  }
203
+ ],
204
+ "file_entries": [
205
+ {
206
+ "file_name": "scripts/{name}",
207
+ "file_type": "script",
208
+ "source_file": "{source-path}",
209
+ "confidence": "T1-low",
210
+ "extraction_method": "file-copy",
211
+ "content_hash": "sha256:{hash}"
212
+ }
189
213
  ]
190
214
  }
191
215
  ```
192
216
 
217
+ `scripts` and `assets` arrays in metadata.json are optional — omit entirely (not empty arrays) when no scripts/assets exist. `file_entries` in provenance-map.json is optional — omit when no scripts/assets exist.
218
+
193
219
  ---
194
220
 
195
221
  ## evidence-report.md Structure
@@ -204,7 +230,7 @@ Each reference file includes:
204
230
  ## Tool Versions
205
231
  - ast-grep: {version}
206
232
  - QMD: {version}
207
- - SKF: 1.0.0
233
+ - SKF: {skf_version}
208
234
 
209
235
  ## Extraction Summary
210
236
  - Files scanned: {count}
@@ -90,9 +90,17 @@ If `source_repo` is a remote URL (GitHub URL or owner/repo format) AND tier is F
90
90
 
91
91
  **Note:** `--no-cone` mode is slower than cone mode for very large repositories but eliminates downloading excluded blobs entirely.
92
92
 
93
+ **Always-included root files:**
94
+
95
+ Regardless of `include_patterns`, always add these root-level version/manifest files to the sparse-checkout pattern list. These are needed for version reconciliation and must not require a fallback to `gh api`:
96
+
97
+ `pyproject.toml`, `package.json`, `Cargo.toml`, `go.mod`, `setup.py`, `setup.cfg`, `VERSION`
98
+
99
+ In cone mode, always use the `--skip-checks` command form when adding these files — even if `include_patterns` resolved to only directory roots (which would normally use the form without `--skip-checks`). The command becomes: `git -C {temp_path} sparse-checkout set --skip-checks {directory_roots} pyproject.toml package.json Cargo.toml go.mod setup.py setup.cfg VERSION`. In no-cone mode, list them as explicit include patterns before any negation patterns. Do not flag them as extraneous inclusions during post-checkout filtering.
100
+
93
101
  **Post-checkout filtering:**
94
102
 
95
- After checkout, apply the original glob `include_patterns` as file-level filters when building the extraction file list — sparse-checkout gets the right directories, glob filtering narrows to the exact files. When `--no-cone` mode was used, most exclude filtering is already done at the git level, but apply `exclude_patterns` as a final pass to catch any edge cases where gitignore pattern matching diverges from the brief's glob semantics.
103
+ After checkout, apply the original glob `include_patterns` as file-level filters when building the extraction file list — sparse-checkout gets the right directories, glob filtering narrows to the exact files. When `--no-cone` mode was used, most exclude filtering is already done at the git level, but apply `exclude_patterns` as a final pass to catch any edge cases where gitignore pattern matching diverges from the brief's glob semantics. Always-included root files (see above) are exempt from post-checkout filtering.
96
104
 
97
105
  3. **If clone succeeds:** Update the working source path to `{temp_path}` for all subsequent AST operations in this step. Proceed with the **Forge/Deep Tier** extraction strategy below. Mark `ephemeral_clone_active = true` for cleanup.
98
106
 
@@ -0,0 +1,46 @@
1
+ # Tier Degradation Rules
2
+
3
+ ## Remote Source at Forge/Deep Tier
4
+
5
+ When `source_repo` is a remote URL (GitHub URL or owner/repo format) and the tier is Forge or Deep:
6
+
7
+ - **ast-grep requires local files** — it cannot operate on remote URLs
8
+
9
+ **Ephemeral clone strategy (preferred):**
10
+
11
+ 1. Check `git` availability (`git --version`). `git` is effectively guaranteed at Deep tier (via `gh` dependency) but NOT guaranteed at Forge tier.
12
+ 2. If `git` is available: perform an ephemeral shallow clone to a system temp path (`{system_temp}/skf-ephemeral-{skill-name}-{timestamp}/`).
13
+ 3. For create-skill: use `--depth 1 --single-branch --filter=blob:none`; if `include_patterns` are specified, apply mode selection: if `exclude_patterns` are absent, use cone mode (convert include_patterns to directory roots; use `--skip-checks` for individual file paths); if `exclude_patterns` are present, use `--no-cone` mode (pass gitignore-style patterns with `!`-prefixed excludes — includes first, then negations). See source-resolution-protocols.md for the full conversion rules.
14
+ 4. For update-skill: use sparse-checkout with `--skip-checks` scoped to the changed files from the change manifest only (file paths require `--skip-checks`). No `--branch` flag — uses the remote default branch (must match the branch used during original create-skill run).
15
+ 5. If clone succeeds: use the local clone path for AST extraction. All results are T1 with `[AST:...]` citations.
16
+ 6. Cleanup: delete the temp directory after extraction inventory is built and all data is in context. The clone never persists beyond the extraction step.
17
+
18
+ **Fallback (clone fails or `git` unavailable):**
19
+
20
+ - The extraction step MUST warn the user explicitly before degrading
21
+ - **create-skill:** Warning must include actionable guidance — clone locally and update `source_repo` in the brief to the local path
22
+ - **update-skill:** Warning must include actionable guidance — clone locally, re-run [CS] Create Skill with the local path to regenerate provenance data, then re-run the update
23
+ - Extraction proceeds using Quick tier strategy (source reading via gh_bridge)
24
+ - All results labeled T1-low with `[SRC:...]` citations
25
+ - The degradation reason is recorded in the evidence report
26
+
27
+ Silent degradation is **forbidden**. The user must always know when AST extraction was skipped and why.
28
+
29
+ ## AST Tool Unavailable at Forge/Deep Tier
30
+
31
+ When the tier is Forge or Deep but ast-grep is not functional:
32
+
33
+ - The extraction step MUST warn the user explicitly before degrading
34
+ - Warning must include actionable guidance: run [SF] Setup Forge to detect tools
35
+ - Extraction proceeds using Quick tier strategy
36
+ - All results labeled T1-low
37
+ - The degradation reason is recorded in the evidence report
38
+
39
+ ## Per-File AST Failure
40
+
41
+ When ast-grep fails on an individual file (parse error, unsupported syntax):
42
+
43
+ - Fall back to source reading for **that file only**
44
+ - Other files continue with AST extraction
45
+ - The affected file's results are labeled T1-low; unaffected files retain T1
46
+ - Log a warning noting which file degraded and why
@@ -3,6 +3,8 @@ name: 'step-03-extract'
3
3
  description: 'Tier-dependent source code extraction — AST or source reading for exports, signatures, and types'
4
4
  nextStepFile: './step-03b-fetch-temporal.md'
5
5
  extractionPatternsData: '../data/extraction-patterns.md'
6
+ extractionPatternsTracingData: '../data/extraction-patterns-tracing.md'
7
+ tierDegradationRulesData: '../data/tier-degradation-rules.md'
6
8
  sourceResolutionData: '../data/source-resolution-protocols.md'
7
9
  ---
8
10
 
@@ -104,7 +106,7 @@ Load `{sourceResolutionData}` completely. Follow the **Remote Source Resolution*
104
106
  5. Build extraction rules YAML data for reproducibility
105
107
  6. Confidence: All results T1 — `[AST:{file}:L{line}]`
106
108
 
107
- **If AST tool is unavailable at Forge/Deep tier:**
109
+ **If AST tool is unavailable at Forge/Deep tier** (see `{tierDegradationRulesData}` for full rules):
108
110
 
109
111
  ⚠️ **Warn the user explicitly:** "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
110
112
 
@@ -117,7 +119,7 @@ Degrade to Quick tier extraction. Note the degradation reason in context for the
117
119
 
118
120
  **Re-export tracing (Forge/Deep only):**
119
121
 
120
- After the initial AST scan, check for public exports from the entry point (`__init__.py`, `index.ts`, `lib.rs`) that were NOT found by AST extraction. These are likely module-level re-exports. Follow the **Re-Export Tracing** protocol in `{extractionPatternsData}` to resolve them to their actual definition files. This ensures module re-export patterns (common in Python libraries with clean public APIs) do not create gaps in the extraction inventory.
122
+ After the initial AST scan, check for public exports from the entry point (`__init__.py`, `index.ts`, `lib.rs`) that were NOT found by AST extraction. These are likely module-level re-exports. Follow the **Re-Export Tracing** protocol in `{extractionPatternsTracingData}` to resolve them to their actual definition files. This ensures module re-export patterns (common in Python libraries with clean public APIs) do not create gaps in the extraction inventory.
121
123
 
122
124
  ### 4b. Validate Exports Against Package Entry Point
123
125
 
@@ -134,6 +136,20 @@ Use the entry point as the authoritative source for `metadata.json`'s `exports[]
134
136
 
135
137
  **If entry point is missing or unreadable:** Skip validation with a warning. Use the AST-extracted list as-is.
136
138
 
139
+ ### 4c. Detect and Inventory Scripts/Assets
140
+
141
+ **If `scripts_intent: "none"` AND `assets_intent: "none"` in brief:** Skip this section.
142
+
143
+ After export extraction, scan the source for scripts and assets using the detection patterns in `{extractionPatternsTracingData}`:
144
+
145
+ 1. Scan source tree for directories/files matching detection heuristics (scripts/, bin/, tools/, cli/ for scripts; assets/, templates/, schemas/, configs/, examples/ for assets)
146
+ 2. For each candidate: verify file exists, check size (flag if >500 lines for user confirmation), exclude binary files
147
+ 3. Compute SHA-256 content hash for each included file
148
+ 4. Extract purpose from file header comments, shebang, README references, or schema `title`/`description` fields. If no purpose found, use filename.
149
+ 5. Record: file_path, purpose, source_path, language (for scripts), type (for assets), content_hash, confidence (T1-low)
150
+
151
+ Add results to `scripts_inventory[]` and `assets_inventory[]` alongside the existing export inventory.
152
+
137
153
  ### 5. Build Extraction Inventory
138
154
 
139
155
  Compile all extracted data into a structured inventory:
@@ -154,6 +170,10 @@ Compile all extracted data into a structured inventory:
154
170
  - Confidence breakdown (T1 count, T1-low count)
155
171
  - `top_exports[]` — sorted list of the top 10-20 public API function names by prominence (import frequency or documentation position). This named field is consumed by step-03b for targeted temporal fetching and cache fingerprinting.
156
172
 
173
+ **Script/asset counts (when detected):**
174
+ - `scripts_found`: count of scripts detected
175
+ - `assets_found`: count of assets detected
176
+
157
177
  **Co-import patterns (Forge/Deep only):**
158
178
  - Libraries commonly imported alongside extracted exports
159
179
  - Integration point suggestions
@@ -169,6 +189,8 @@ Display the extraction findings for user confirmation:
169
189
  **Confidence:** {t1_count} T1 (AST-verified), {t1_low_count} T1-low (source reading)
170
190
  **Tier used:** {tier}
171
191
  **Co-import patterns:** {pattern_count} detected
192
+ {if scripts_found > 0: **Scripts detected:** {scripts_found}}
193
+ {if assets_found > 0: **Assets detected:** {assets_found}}
172
194
 
173
195
  **Top exports:**
174
196
  {list top 10 exports with signatures}
@@ -61,7 +61,7 @@ Load `{skillSectionsData}` and `{assemblyRulesData}` completely. These define th
61
61
 
62
62
  ### 2. Build SKILL.md Content
63
63
 
64
- Assemble each section in order using the assembly rules data file (`{assemblyRulesData}`). The data file specifies frontmatter format, Tier 1 section details (Sections 1-8), Tier 2 section details (Sections 9-11), and assembly ordering rules. Follow it exactly.
64
+ Assemble each section in order using the assembly rules data file (`{assemblyRulesData}`). The data file specifies frontmatter format, Tier 1 section details (Sections 1-8, including conditional Section 7b for scripts/assets), Tier 2 section details (Sections 9-11), and assembly ordering rules. Follow it exactly. Assemble Section 7b (Scripts & Assets) only if `scripts_inventory` or `assets_inventory` is non-empty.
65
65
 
66
66
  ### 3. Build context-snippet.md Content
67
67
 
@@ -97,7 +97,11 @@ Following the structure from the skill-sections data file:
97
97
  - `exports_total`: `exports_public_api` + `exports_internal`
98
98
  - `public_api_coverage`: `exports_documented / exports_public_api` (1.0 when all public API exports are documented; `null` if `exports_public_api` is 0)
99
99
  - `total_coverage`: `exports_documented / exports_total` (may be low for large codebases — this is expected; `null` if `exports_total` is 0)
100
- - Set `tool_versions` based on tier and available tools
100
+ - Set `description` from the SKILL.md frontmatter `description` field (already assembled in section 2)
101
+ - Set `language` from source analysis (e.g., `"typescript"`, `"python"`) — use the primary language of the entry point file
102
+ - Set `ast_node_count` from extraction stats if ast-grep was used (Forge/Deep tier), otherwise omit
103
+ - Set `tool_versions` based on tier and available tools. Resolve `{skf_version}` from the SKF module's `package.json` `version` field (run `node -p "require('./node_modules/bmad-module-skill-forge/package.json').version"` or read the installed module's `package.json`). If unresolvable, fall back to `git describe --tags --abbrev=0` in the SKF module root. Never hardcode the version.
104
+ - If `scripts_inventory` is non-empty, populate `scripts[]` array and set `stats.scripts_count`. If `assets_inventory` is non-empty, populate `assets[]` array and set `stats.assets_count`. Omit these fields entirely when inventories are empty.
101
105
 
102
106
  ### 5. Build references/ Content
103
107
 
@@ -111,11 +115,11 @@ Group functions logically by module, file, or functional area.
111
115
 
112
116
  ### 6. Build provenance-map.json Content
113
117
 
114
- One entry per extracted export: export_name, export_type, params[] (typed strings), return_type, source_file, source_line, confidence tier (T1/T1-low/T2), extraction_method, ast_node_type. See `{skillSectionsData}` for full schema.
118
+ One entry per extracted export: export_name, export_type, params[] (typed strings), return_type, source_file, source_line, confidence tier (T1/T1-low/T2), extraction_method, ast_node_type. If `scripts_inventory` or `assets_inventory` is non-empty, add `file_entries[]` with file_name, file_type, source_file, content_hash, confidence, extraction_method. See `{skillSectionsData}` for full schema.
115
119
 
116
120
  ### 7. Build evidence-report.md Content
117
121
 
118
- Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template.
122
+ Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template. Use the same `{skf_version}` value resolved in section 4 when populating the Tool Versions block.
119
123
 
120
124
  ### 8. Menu Handling Logic
121
125
 
@@ -40,6 +40,7 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
40
40
  - 💾 Validation results are added to evidence-report content in context
41
41
  - 📖 Auto-fix pattern: validate → fix → re-validate (once)
42
42
  - 🚫 Maximum one auto-fix attempt per validation failure
43
+ - ⏸️ **Conditional interaction:** If tessl returns suggestions (section 6b), halt for user input. Otherwise auto-proceed. This is a conditional gate step, not a pure auto-proceed step.
43
44
 
44
45
  ## CONTEXT BOUNDARIES:
45
46
 
@@ -73,6 +74,8 @@ This performs frontmatter validation, description quality checks, body limit enf
73
74
 
74
75
  **Parse the JSON output** for: `qualityScore` (0-100), `diagnostics[]` (remaining issues), `fixed[]` (auto-corrected issues).
75
76
 
77
+ **Context sync after --fix:** If `fixed[]` is non-empty (i.e., `--fix` modified files on disk), re-read the modified SKILL.md to update the in-context copy. Verify the re-read content matches expectations before proceeding. This prevents silent divergence between the in-context SKILL.md and the on-disk version that step-07 will use for artifact generation.
78
+
76
79
  **Note:** `skill-check` may return non-zero exit code even when `errorCount` is 0. Always rely on parsed JSON, not the shell exit code.
77
80
 
78
81
  - **Score ≥ 70:** Record "Schema: PASS (score: {score}/100)" in evidence-report
@@ -98,10 +101,16 @@ If fails: auto-fix (deterministic), re-validate once, record result. If passes:
98
101
 
99
102
  **If step 2 reported `body.max_lines` failure:**
100
103
 
104
+ **Preferred approach — selective split:** Rather than a full `split-body`, identify which Tier 2 section(s) are largest (usually `## Full API Reference`) and manually extract only those to `references/`. Keep all Tier 1 content and smaller Tier 2 sections inline. This preserves agent accuracy — inline passive context achieves 100% task accuracy vs 79% for on-demand retrieval (per Vercel research). See `knowledge/split-body-strategy.md` for the full rationale.
105
+
106
+ **If manual selective split is not feasible**, fall back to the automated full split:
107
+
101
108
  ```bash
102
109
  npx skill-check split-body <staging-skill-dir> --write
103
110
  ```
104
111
 
112
+ After any split (selective or full), verify that context snippet section anchors (`#quick-start`, `#key-types`) still resolve to headings in SKILL.md.
113
+
105
114
  Then re-validate: `npx skill-check check <staging-skill-dir> --format json --no-security-scan`
106
115
 
107
116
  **If skill-check unavailable or no body size issue:** Skip.
@@ -120,15 +129,7 @@ Record any security warnings in evidence-report. Security findings are advisory
120
129
 
121
130
  **If security scan fails due to missing SNYK_TOKEN:**
122
131
 
123
- Display actionable guidance:
124
-
125
- "Security scan requires a Snyk API token. **Important:** The Snyk API requires an Enterprise plan ([authentication docs](https://docs.snyk.io/snyk-api/authentication-for-api)). To enable:
126
- 1. Obtain a Snyk Enterprise account with API access
127
- 2. Get your API token from Account Settings → API Token
128
- 3. Set `SNYK_TOKEN=your-token` in your environment or `.env`
129
- 4. Re-run [SF] Setup Forge to update tool detection, then re-run [CS] Create Skill
130
-
131
- If you don't have an Enterprise account, use `--no-security-scan` flag in skill-check to skip, or `--security-scan-runner pipx|uvx|local` to control the execution method. Security scanning is optional and does not affect tier level or block skill compilation."
132
+ Display: "Security scan requires a Snyk Enterprise API token ([docs](https://docs.snyk.io/snyk-api/authentication-for-api)). Set `SNYK_TOKEN=your-token` in environment or `.env`, then re-run [SF] Setup Forge. Without Enterprise, use `--no-security-scan` to skip. Security scanning is optional and does not block skill compilation."
132
133
 
133
134
  Record: "Security scan skipped — SNYK_TOKEN not configured"
134
135
 
@@ -143,7 +144,7 @@ Parse output for: `description_score`, `content_score`, `average_score`, `valida
143
144
  - **Content score < 70%:** Record warning: "Content quality warning: tessl scored content at {score}%."
144
145
  - **Unavailable:** Skip with note: "Content quality review skipped — tessl tool unavailable"
145
146
 
146
- **Expected scoring for two-tier design:** The Skill Forge two-tier design (Tier 1 Key API Summary + Tier 2 Full API Reference) intentionally includes progressive disclosure. tessl's `conciseness` scorer may flag this as redundancy (typically scoring 2/3), which is expected behavior — not a defect. Content scores >= 60% are acceptable when both tiers are correctly applied with proper differentiation (see step-05 Tier 2 assembly rules). Do NOT consolidate Tier 1 and Tier 2 content to improve the score — the two-tier structure is a deliberate design choice for standalone usability.
147
+ > **EXPECTED BEHAVIOR Two-Tier Scoring:** The Skill Forge two-tier design (Tier 1 Key API Summary + Tier 2 Full API Reference) intentionally includes progressive disclosure. tessl's `conciseness` scorer will flag this as redundancy (typically scoring 2/3), which is **expected behavior — not a defect**. tessl may also suggest removing `[MANUAL]` markers, moving Full API Reference to a separate file, or consolidating duplicate parameter documentation — **all three suggestions conflict with SKF design principles and must be dismissed.** Acceptable threshold: content scores >= 60% are normal for two-tier skills. Do NOT consolidate Tier 1 and Tier 2 content to improve the score — the two-tier structure is a deliberate design choice for standalone usability.
147
148
 
148
149
  tessl installs automatically via `npx`. A missing tool is not an error — graceful skip.
149
150
 
@@ -182,6 +183,7 @@ Cross-check metadata.json against extraction inventory:
182
183
  - `stats.public_api_coverage` and `stats.total_coverage` are correctly computed (null when denominator is 0)
183
184
  - `confidence_t1`, `confidence_t2`, `confidence_t3` match actual counts
184
185
  - `spec_version` is "1.3"
186
+ - If `scripts[]` or `assets[]` arrays present: verify `stats.scripts_count`/`stats.assets_count` match array lengths; verify `file_entries` count in provenance-map.json matches
185
187
 
186
188
  Auto-fix any discrepancies (these are computed values).
187
189
 
@@ -62,6 +62,9 @@ skills/{name}/references/
62
62
  forge-data/{name}/
63
63
  ```
64
64
 
65
+ If `scripts_inventory` is non-empty, also create: `skills/{name}/scripts/`
66
+ If `assets_inventory` is non-empty, also create: `skills/{name}/assets/`
67
+
65
68
  Where `{name}` is the skill name from the brief (kebab-case).
66
69
 
67
70
  If directories already exist, do not error — proceed with file writing (overwrites existing files).
@@ -85,6 +88,14 @@ Write these 4 files from the compiled content:
85
88
  - One file per function group or type
86
89
  - Progressive disclosure detail files
87
90
 
91
+ **Files 4b (conditional):** `skills/{name}/scripts/*`
92
+ - One file per detected script, copied from source with content preserved
93
+ - Only created when `scripts_inventory` is non-empty
94
+
95
+ **Files 4c (conditional):** `skills/{name}/assets/*`
96
+ - One file per detected asset, copied from source with content preserved
97
+ - Only created when `assets_inventory` is non-empty
98
+
88
99
  ### 3. Write Workspace Artifacts to forge-data/{name}/
89
100
 
90
101
  Write these 3 files from the compiled content:
@@ -101,7 +112,7 @@ Write these 3 files from the compiled content:
101
112
  ### 4. Verify Write Completion
102
113
 
103
114
  After all files are written, verify:
104
- - All 7 files exist at their expected paths
115
+ - All 7 base files exist at their expected paths, plus scripts/ and assets/ files when inventories are non-empty
105
116
  - List each file with its path and size
106
117
 
107
118
  **If any write failed:**
@@ -116,6 +127,8 @@ Display brief confirmation:
116
127
  - SKILL.md
117
128
  - context-snippet.md
118
129
  - metadata.json
130
+ {if scripts: - scripts/ ({scripts_count} files)}
131
+ {if assets: - assets/ ({assets_count} files)}
119
132
  - references/ ({reference_count} files)
120
133
 
121
134
  **Workspace (forge-data/{name}/):**
@@ -72,8 +72,8 @@ description: >
72
72
 
73
73
  **Frontmatter rules:**
74
74
 
75
- - `name`: lowercase alphanumeric + hyphens only, must match skill output directory name
76
- - `description`: non-empty, max 1024 chars, trigger-optimized for agent discovery
75
+ - `name`: lowercase alphanumeric + hyphens only, must match skill output directory name. Prefer gerund form (`processing-pdfs`) for clarity.
76
+ - `description`: non-empty, max 1024 chars, trigger-optimized for agent discovery. MUST use third-person voice ("Processes..." not "I can..." or "You can...").
77
77
  - No other frontmatter fields — only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` are permitted by spec
78
78
 
79
79
  ### 3. Compile Integration Layer
@@ -83,7 +83,7 @@ Run: `npx skill-check -h`
83
83
 
84
84
  This validates frontmatter, description, body limits, links, formatting — and auto-fixes deterministic issues. Parse JSON for `qualityScore`, `diagnostics[]`, `fixed[]`.
85
85
 
86
- **If `body.max_lines` reported**, run: `npx skill-check split-body <skill-dir> --write`, then re-validate.
86
+ **If `body.max_lines` reported**, prefer selective split: extract only the largest Tier 2 section(s) to `references/`, keeping Tier 1 content inline (see `knowledge/split-body-strategy.md`). Fall back to `npx skill-check split-body <skill-dir> --write` if not feasible. Verify `#quick-start` and `#key-types` anchors still resolve after split. Then re-validate.
87
87
 
88
88
  **If unavailable**, perform manual frontmatter check:
89
89
  - [ ] Frontmatter present with `---` delimiters