bmad-module-skill-forge 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +46 -176
- package/docs/agents.md +1 -1
- package/docs/architecture.md +106 -5
- package/docs/concepts.md +96 -0
- package/docs/examples.md +39 -0
- package/docs/getting-started.md +15 -1
- package/docs/index.md +14 -41
- package/docs/workflows.md +1 -1
- package/package.json +2 -2
- package/src/module.yaml +2 -2
- package/src/workflows/create-skill/data/extraction-patterns.md +17 -52
- package/src/workflows/create-skill/data/skill-sections.md +2 -2
- package/src/workflows/create-skill/data/source-resolution-protocols.md +9 -1
- package/src/workflows/create-skill/data/tier-degradation-rules.md +46 -0
- package/src/workflows/create-skill/steps-c/step-03-extract.md +2 -1
- package/src/workflows/create-skill/steps-c/step-05-compile.md +2 -2
- package/src/workflows/create-skill/steps-c/step-06-validate.md +4 -1
- package/src/workflows/export-skill/workflow.md +1 -1
- package/src/workflows/test-skill/data/source-access-protocol.md +8 -0
- package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +4 -4
- package/src/workflows/test-skill/steps-c/step-04b-external-validators.md +6 -0
- package/src/workflows/test-skill/steps-c/step-05-score.md +5 -4
- package/src/workflows/update-skill/data/tier-degradation-rules.md +46 -0
- package/src/workflows/update-skill/steps-c/step-03-re-extract.md +1 -0
- package/tools/cli/lib/ui.js +1 -1
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
# Skill Forge (SKF)
|
|
6
6
|
|
|
7
|
-
**
|
|
7
|
+
**Turn code and docs into instructions AI agents can actually follow.**
|
|
8
8
|
|
|
9
9
|
[](https://github.com/armelhbobdad/bmad-module-skill-forge/actions/workflows/quality.yaml)
|
|
10
10
|
[](https://www.npmjs.com/package/bmad-module-skill-forge)
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
[](https://armelhbobdad.github.io/bmad-module-skill-forge/)
|
|
14
14
|
[](https://github.com/armelhbobdad/bmad-module-skill-forge/stargazers)
|
|
15
15
|
|
|
16
|
-
*
|
|
16
|
+
*Skill Forge analyzes your code repositories, documentation, and developer discourse to build verified instruction files for AI agents. Every instruction links back to where it came from — nothing is made up.*
|
|
17
17
|
|
|
18
18
|
**If SKF helps your agent stop hallucinating, give it a ⭐ — it helps others find this tool.**
|
|
19
19
|
|
|
@@ -21,208 +21,78 @@
|
|
|
21
21
|
|
|
22
22
|
---
|
|
23
23
|
|
|
24
|
-
|
|
24
|
+
## The Problem
|
|
25
25
|
|
|
26
|
-
|
|
26
|
+
You ask an AI agent to use a library. It invents function names that don't exist. It guesses parameter types. You paste documentation into the context — it still gets details wrong. You write instructions by hand — they go stale the moment the code changes.
|
|
27
27
|
|
|
28
|
-
|
|
29
|
-
- Progressive capability model that meets developers where they are
|
|
30
|
-
- Full lifecycle: discover, brief, compile, test, audit, update, export
|
|
31
|
-
- Version-pinned skills that track source changes and detect drift
|
|
32
|
-
- Zero hallucination tolerance — every instruction traces to code
|
|
28
|
+
This isn't an edge case. It's the default experience.
|
|
33
29
|
|
|
34
|
-
## How
|
|
30
|
+
## How Skill Forge Fixes This
|
|
35
31
|
|
|
36
|
-
|
|
32
|
+
1. **Analyzes your sources** — extracts real function signatures, types, and patterns from code repositories, documentation websites, and developer discourse
|
|
33
|
+
2. **Compiles verified instruction files** — every instruction links to the exact file and line it came from
|
|
34
|
+
3. **Follows an open standard** — skills comply with the [agentskills.io](https://agentskills.io) spec and work across Claude, Cursor, Copilot, and other AI agents
|
|
37
35
|
|
|
38
|
-
##
|
|
36
|
+
## Before vs After
|
|
39
37
|
|
|
40
|
-
|
|
38
|
+
**Without SKF** — your agent guesses:
|
|
41
39
|
|
|
42
|
-
|
|
40
|
+
```python
|
|
41
|
+
import cognee
|
|
43
42
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
### Building Blocks
|
|
47
|
-
|
|
48
|
-
Each workflow directory contains these files, and each has a specific job:
|
|
49
|
-
|
|
50
|
-
| File | What it does | When it loads |
|
|
51
|
-
|---------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
|
|
52
|
-
| `forger.agent.yaml` | Expert persona — identity, principles, critical actions, menu of triggers | First — always in context |
|
|
53
|
-
| `workflow.md` | Human-readable entry point — goals, mode menu (Create/Edit/Validate), routes to first step | Second — presents mode choice |
|
|
54
|
-
| `steps-c/*.md` | **Create** steps — primary execution, 4-9 sequential files | One at a time (just-in-time) |
|
|
55
|
-
| `data/*.md` | Workflow-specific reference data — schemas, heuristics, rules, patterns | Read by steps on demand |
|
|
56
|
-
| `templates/*.md` | Output skeletons with placeholder vars — steps fill these in to produce the final artifact | Read by steps when generating output |
|
|
57
|
-
| `skf-knowledge-index.csv` | Knowledge fragment index — id, name, tags, tier, file path | Read by steps to decide which fragments to load |
|
|
58
|
-
| `knowledge/*.md` | 10 reusable fragments — cross-cutting principles and patterns (e.g., `zero-hallucination.md`, `confidence-tiers.md`) | Selectively read into context when a step directs |
|
|
59
|
-
|
|
60
|
-
```mermaid
|
|
61
|
-
flowchart LR
|
|
62
|
-
U[User] --> A[Agent Persona]
|
|
63
|
-
A --> W[Workflow Entry: workflow.md]
|
|
64
|
-
W --> S[Step Files: steps-c/]
|
|
65
|
-
S --> K[Knowledge Fragments<br/>skf-knowledge-index.csv → knowledge/*.md]
|
|
66
|
-
S --> D[Data & Templates<br/>data/*.md, templates/*.md]
|
|
67
|
-
S --> O[Outputs: skills/reports<br/>when a step writes output]
|
|
43
|
+
# Agent hallucinates: sync call, wrong parameter name, missing await
|
|
44
|
+
results = cognee.search("What does Cognee do?", mode="graph")
|
|
68
45
|
```
|
|
69
46
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
1. **Trigger** — User types `@Ferris CS` (or fuzzy match like `create-skill`). The agent menu in `forger.agent.yaml` maps the trigger to the workflow path.
|
|
73
|
-
2. **Agent loads** — `forger.agent.yaml` injects the persona (identity, principles, critical actions) into the context window. Sidecar files (`forge-tier.yaml`, `preferences.yaml`) are loaded for persistent state.
|
|
74
|
-
3. **Workflow loads** — `workflow.md` presents the mode choice and routes to the first step file.
|
|
75
|
-
4. **Step-by-step execution** — Only the current step file is in context (just-in-time loading). Each step explicitly names the next one. The LLM reads, executes, saves output, then loads the next step. No future steps are ever preloaded.
|
|
76
|
-
5. **Knowledge injection** — Steps consult `skf-knowledge-index.csv` and selectively load fragments from `knowledge/` by tags and relevance. Cross-cutting principles (zero hallucination, confidence tiers, provenance) are loaded only when a step directs — not preloaded.
|
|
77
|
-
6. **Data injection** — Steps read `data/*.md` files as needed (schemas, heuristics, extraction patterns). This is deliberate context engineering: only the data relevant to the current step enters the context window.
|
|
78
|
-
7. **Templates** — When a step produces output (e.g., a skill brief or test report), it reads the template file and fills in placeholders with computed results. The template provides consistent structure; the step provides the content.
|
|
79
|
-
8. **Progress tracking** — Each step appends to an output file with state tracking. Resume mode reads this state and routes to the next incomplete step.
|
|
47
|
+
**With SKF** — your agent reads the verified skill:
|
|
80
48
|
|
|
81
|
-
|
|
49
|
+
```python
|
|
50
|
+
import cognee
|
|
82
51
|
|
|
83
|
-
|
|
52
|
+
# Agent follows the skill instruction:
|
|
53
|
+
# `search(query_text: str, query_type: SearchType = GRAPH_COMPLETION) -> List[SearchResult]`
|
|
54
|
+
# [AST:cognee/api/v1/search/search.py:L26]
|
|
55
|
+
results = await cognee.search(
|
|
56
|
+
query_text="What does Cognee do?",
|
|
57
|
+
query_type=cognee.SearchType.GRAPH_COMPLETION
|
|
58
|
+
)
|
|
59
|
+
```
|
|
84
60
|
|
|
85
|
-
|
|
86
|
-
|---------------|--------------------|-------------------------------------------------------------|
|
|
87
|
-
| **Architect** | SF, AN, BS, CS, QS, SS | Exploratory, assembling — discovers structure and scope |
|
|
88
|
-
| **Surgeon** | US | Precise, preserving — extracts and compiles with provenance |
|
|
89
|
-
| **Audit** | AS, TS | Judgmental, scoring — evaluates quality and detects drift |
|
|
90
|
-
| **Delivery** | EX | Packaging, ecosystem-ready — bundles for distribution |
|
|
61
|
+
The skill told the agent the real function name, the real parameters, and that the call requires `await` — all traced to the exact source line. This is from a [real generated skill](https://github.com/armelhbobdad/oh-my-skills).
|
|
91
62
|
|
|
92
63
|
## Install
|
|
93
64
|
|
|
94
65
|
Requires [Node.js](https://nodejs.org/) >= 22.
|
|
95
66
|
|
|
96
|
-
There are three ways to install SKF, depending on your setup.
|
|
97
|
-
|
|
98
|
-
### Method 1: Standalone (recommended for trying SKF)
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
npx bmad-module-skill-forge install
|
|
102
|
-
```
|
|
103
|
-
|
|
104
|
-
Installs SKF on its own. You'll be prompted for project name, output folders, and which IDEs to configure. The installer generates IDE-specific command files (e.g. `.claude/commands/`, `.cursor/commands/`) so workflows appear in your IDE's command palette.
|
|
105
|
-
|
|
106
|
-
### Method 2: As a custom module during BMad Method installation
|
|
107
|
-
|
|
108
|
-
```bash
|
|
109
|
-
npx bmad-method install
|
|
110
|
-
```
|
|
111
|
-
|
|
112
|
-
When prompted **"Add custom modules from your computer?"**, select Yes and provide the path to the SKF `src/` folder (clone this repo first):
|
|
113
|
-
|
|
114
|
-
```
|
|
115
|
-
Path to custom module folder: /path/to/bmad-module-skill-forge/src/
|
|
116
|
-
```
|
|
117
|
-
|
|
118
|
-
This installs BMad core + SKF together with full IDE integration, manifests, and help catalog. Best when you want the complete BMad development workflow.
|
|
119
|
-
|
|
120
|
-
### Method 3: Add SKF to an existing BMad project
|
|
121
|
-
|
|
122
|
-
If you already have BMad installed, you can add SKF afterward by running the standalone installer in the same directory:
|
|
123
|
-
|
|
124
67
|
```bash
|
|
125
68
|
npx bmad-module-skill-forge install
|
|
126
69
|
```
|
|
127
70
|
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
## Quickstart
|
|
131
|
-
|
|
132
|
-
1. **Setup your forge:** `@Ferris SF` — detects tools, sets your tier (Quick/Forge/Deep)
|
|
133
|
-
2. **Quick skill (fastest):** `@Ferris QS <package-name>` — fast skill from a package name
|
|
134
|
-
3. **Full skill:** `@Ferris BS` then `@Ferris CS` — brief then compile for maximum quality
|
|
135
|
-
4. **Stack skill:** `@Ferris SS` — consolidated project stack skill with integration patterns
|
|
136
|
-
5. **Export:** `@Ferris EX` — package for distribution, update CLAUDE.md
|
|
137
|
-
|
|
138
|
-
## Workflows
|
|
139
|
-
|
|
140
|
-
| Trigger | Command | Purpose |
|
|
141
|
-
| --- | --- | --- |
|
|
142
|
-
| SF | `skf_setup_forge` | Initialize forge environment, detect tools, set tier |
|
|
143
|
-
| AN | `skf_analyze_source` | Discover what to skill in a large repo |
|
|
144
|
-
| BS | `skf_brief_skill` | Design a skill scope through guided discovery |
|
|
145
|
-
| CS | `skf_create_skill` | Compile a skill from brief (supports --batch) |
|
|
146
|
-
| QS | `skf_quick_skill` | Fast skill from package name or GitHub URL |
|
|
147
|
-
| SS | `skf_create_stack_skill` | Consolidated project stack skill with integration patterns |
|
|
148
|
-
| US | `skf_update_skill` | Smart regeneration preserving \[MANUAL\] sections |
|
|
149
|
-
| AS | `skf_audit_skill` | Drift detection between skill and current source |
|
|
150
|
-
| TS | `skf_test_skill` | Cognitive completeness verification — quality gate before export |
|
|
151
|
-
| EX | `skf_export_skill` | Package for distribution, inject into CLAUDE.md/AGENTS.md |
|
|
152
|
-
|
|
153
|
-
## Progressive Capability Model
|
|
154
|
-
|
|
155
|
-
| Tier | Tools | Capability |
|
|
156
|
-
| --- | --- | --- |
|
|
157
|
-
| **Quick** | gh + skill-check + tessl | Source reading + spec validation + content quality review |
|
|
158
|
-
| **Forge** | + ast-grep | Structural truth, T1 confidence |
|
|
159
|
-
| **Deep** | + QMD | Knowledge search, temporal provenance |
|
|
160
|
-
|
|
161
|
-
The `setup-forge` workflow detects available tools and writes the tier to `forge-tier.yaml`. All subsequent workflows adapt their behavior to the detected tier.
|
|
162
|
-
|
|
163
|
-
> **Recommended:** If your IDE supports MCP servers, install the [ast-grep MCP server](https://github.com/ast-grep/ast-grep-mcp) alongside the CLI. SKF's extraction protocol prefers the MCP tool for compact, memory-efficient AST queries and falls back to CLI streaming for large codebases.
|
|
71
|
+
You'll be prompted for project name, output folders, and IDE configuration. See the [docs](https://armelhbobdad.github.io/bmad-module-skill-forge/getting-started/) for other install methods.
|
|
164
72
|
|
|
165
|
-
|
|
73
|
+
## Quick Start
|
|
166
74
|
|
|
167
|
-
|
|
75
|
+
1. **Set up your environment:** `@Ferris SF` — detects your tools and sets your capability tier
|
|
76
|
+
2. **Generate your first skill:** `@Ferris QS <package-name>` — creates a verified skill in under a minute
|
|
77
|
+
3. **Full quality path:** `@Ferris BS` then `@Ferris CS` — brief first, then compile for maximum accuracy
|
|
168
78
|
|
|
169
|
-
|
|
170
|
-
2. Copy your API token from Account Settings
|
|
171
|
-
3. Add to your environment: `export SNYK_TOKEN=your-token`
|
|
172
|
-
4. Re-run `@Ferris SF` to detect the token
|
|
79
|
+
See the [workflows docs](https://armelhbobdad.github.io/bmad-module-skill-forge/workflows/) for all 10 available workflows.
|
|
173
80
|
|
|
174
|
-
|
|
81
|
+
## Who Is This For?
|
|
175
82
|
|
|
176
|
-
|
|
83
|
+
- **You use AI agents to write code** and they keep getting API calls wrong — hallucinating function names, guessing parameter types, inventing methods that don't exist
|
|
84
|
+
- **You maintain a library** and want to ship official, verified instruction files so AI agents use your API correctly
|
|
85
|
+
- **You manage a codebase with many dependencies** and want a consolidated "stack skill" that teaches your agent how all the pieces fit together
|
|
86
|
+
- **You use a SaaS API or closed-source tool** with no public code — SKF can generate skills from documentation alone
|
|
177
87
|
|
|
178
|
-
##
|
|
88
|
+
## Learn More
|
|
179
89
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
-
|
|
183
|
-
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
## Configuration
|
|
188
|
-
|
|
189
|
-
SKF variables are defined in `src/module.yaml` and prompted during install:
|
|
190
|
-
|
|
191
|
-
| Variable | Purpose | Default |
|
|
192
|
-
|------------------------|------------------------------------------------------------------------------------------------------|-----------------------------|
|
|
193
|
-
| `skills_output_folder` | Where generated skills are saved | `{project-root}/skills` |
|
|
194
|
-
| `forge_data_folder` | Where workspace artifacts are stored | `{project-root}/forge-data` |
|
|
195
|
-
| `tier_override` | Force a specific tier for comparison or testing (in `_bmad/_memory/forger-sidecar/preferences.yaml`) | `~` (auto-detect) |
|
|
196
|
-
|
|
197
|
-
Runtime configuration (tool detection, tier, parallel settings) is managed by the `setup-forge` workflow in `forge-tier.yaml`.
|
|
198
|
-
|
|
199
|
-
## Module Structure
|
|
200
|
-
|
|
201
|
-
```
|
|
202
|
-
src/
|
|
203
|
-
├── module.yaml
|
|
204
|
-
├── module-help.csv
|
|
205
|
-
├── agents/
|
|
206
|
-
│ └── forger.agent.yaml
|
|
207
|
-
├── forger/
|
|
208
|
-
│ ├── forge-tier.yaml
|
|
209
|
-
│ ├── preferences.yaml
|
|
210
|
-
│ └── README.md
|
|
211
|
-
├── knowledge/
|
|
212
|
-
│ ├── skf-knowledge-index.csv
|
|
213
|
-
│ └── *.md (10 fragments)
|
|
214
|
-
└── workflows/
|
|
215
|
-
├── setup-forge/
|
|
216
|
-
├── analyze-source/
|
|
217
|
-
├── brief-skill/
|
|
218
|
-
├── create-skill/
|
|
219
|
-
├── quick-skill/
|
|
220
|
-
├── create-stack-skill/
|
|
221
|
-
├── update-skill/
|
|
222
|
-
├── audit-skill/
|
|
223
|
-
├── test-skill/
|
|
224
|
-
└── export-skill/
|
|
225
|
-
```
|
|
90
|
+
- **[Getting Started](https://armelhbobdad.github.io/bmad-module-skill-forge/getting-started/)** — Installation, prerequisites, and your first skill
|
|
91
|
+
- **[Concepts](https://armelhbobdad.github.io/bmad-module-skill-forge/concepts/)** — Plain-English definitions of all key terms
|
|
92
|
+
- **[How It Works](https://armelhbobdad.github.io/bmad-module-skill-forge/architecture/)** — Architecture, capability tiers, output format, and design decisions
|
|
93
|
+
- **[Workflows](https://armelhbobdad.github.io/bmad-module-skill-forge/workflows/)** — All 10 workflows with commands and connection diagrams
|
|
94
|
+
- **[Agents](https://armelhbobdad.github.io/bmad-module-skill-forge/agents/)** — Ferris: the AI agent that runs all SKF workflows
|
|
95
|
+
- **[Examples](https://armelhbobdad.github.io/bmad-module-skill-forge/examples/)** — Real-world scenarios, tips, and troubleshooting
|
|
226
96
|
|
|
227
97
|
## Acknowledgements
|
|
228
98
|
|
|
@@ -242,7 +112,7 @@ SKF builds on these excellent open-source tools:
|
|
|
242
112
|
|
|
243
113
|
## Contributing
|
|
244
114
|
|
|
245
|
-
See [
|
|
115
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
|
|
246
116
|
|
|
247
117
|
## License
|
|
248
118
|
|
package/docs/agents.md
CHANGED
|
@@ -5,7 +5,7 @@ description: Ferris agent reference — modes, capabilities, menu, and communica
|
|
|
5
5
|
|
|
6
6
|
# Agents Reference
|
|
7
7
|
|
|
8
|
-
SKF
|
|
8
|
+
Ferris is the AI agent that runs all SKF workflows. You always interact with Ferris — he switches modes based on which workflow you invoke.
|
|
9
9
|
|
|
10
10
|
---
|
|
11
11
|
|
package/docs/architecture.md
CHANGED
|
@@ -1,11 +1,73 @@
|
|
|
1
1
|
---
|
|
2
|
-
title:
|
|
3
|
-
description: How Skill Forge works — output format, confidence model, progressive tiers, tool ecosystem
|
|
2
|
+
title: How It Works
|
|
3
|
+
description: How Skill Forge works — the BMad framework, architecture, output format, confidence model, progressive tiers, and tool ecosystem
|
|
4
4
|
---
|
|
5
5
|
|
|
6
|
-
#
|
|
6
|
+
# How It Works
|
|
7
7
|
|
|
8
|
-
This page
|
|
8
|
+
This page is for people who want to understand how SKF works under the hood. It covers the BMad framework, workflow architecture, capability tiers, output format, tool ecosystem, and key design decisions. For plain-English definitions of key terms, see [Concepts](../concepts.md).
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## How BMad Works
|
|
13
|
+
|
|
14
|
+
BMad works because it turns big, fuzzy work into **repeatable workflows**. Each workflow is broken into small steps with clear instructions, so the AI follows the same path every time. It also uses a **shared knowledge base** (standards and patterns) so outputs are consistent, not random. In short: **structured steps + shared standards = reliable results**.
|
|
15
|
+
|
|
16
|
+
## How SKF Fits In
|
|
17
|
+
|
|
18
|
+
SKF plugs into BMad the same way a specialist plugs into a team. It uses the same step-by-step workflow engine and shared standards, but focuses exclusively on skill compilation and quality assurance. That means you get **evidence-based agent skills**, **AST-verified instructions**, and **drift detection** that align with the rest of the BMad process.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Architecture & Flow
|
|
23
|
+
|
|
24
|
+
BMad is a small **agent + workflow engine**. There is no external orchestrator — everything runs inside the LLM context window through structured instructions.
|
|
25
|
+
|
|
26
|
+
### Building Blocks
|
|
27
|
+
|
|
28
|
+
Each workflow directory contains these files, and each has a specific job:
|
|
29
|
+
|
|
30
|
+
| File | What it does | When it loads |
|
|
31
|
+
|---------------------------|---------------------------------------------------------------------------------------------------------------------|---------------------------------------------------|
|
|
32
|
+
| `forger.agent.yaml` | Expert persona — identity, principles, critical actions, menu of triggers | First — always in context |
|
|
33
|
+
| `workflow.md` | Human-readable entry point — goals, mode menu (Create/Edit/Validate), routes to first step | Second — presents mode choice |
|
|
34
|
+
| `steps-c/*.md` | **Create** steps — primary execution, 4-9 sequential files | One at a time (just-in-time) |
|
|
35
|
+
| `data/*.md` | Workflow-specific reference data — schemas, heuristics, rules, patterns | Read by steps on demand |
|
|
36
|
+
| `templates/*.md` | Output skeletons with placeholder vars — steps fill these in to produce the final artifact | Read by steps when generating output |
|
|
37
|
+
| `skf-knowledge-index.csv` | Knowledge fragment index — id, name, tags, tier, file path | Read by steps to decide which fragments to load |
|
|
38
|
+
| `knowledge/*.md` | 10 reusable fragments — cross-cutting principles and patterns (e.g., `zero-hallucination.md`, `confidence-tiers.md`) | Selectively read into context when a step directs |
|
|
39
|
+
|
|
40
|
+
```mermaid
|
|
41
|
+
flowchart LR
|
|
42
|
+
U[User] --> A[Agent Persona]
|
|
43
|
+
A --> W[Workflow Entry: workflow.md]
|
|
44
|
+
W --> S[Step Files: steps-c/]
|
|
45
|
+
S --> K[Knowledge Fragments<br/>skf-knowledge-index.csv → knowledge/*.md]
|
|
46
|
+
S --> D[Data & Templates<br/>data/*.md, templates/*.md]
|
|
47
|
+
S --> O[Outputs: skills/reports<br/>when a step writes output]
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### How It Works at Runtime
|
|
51
|
+
|
|
52
|
+
1. **Trigger** — User types `@Ferris CS` (or fuzzy match like `create-skill`). The agent menu in `forger.agent.yaml` maps the trigger to the workflow path.
|
|
53
|
+
2. **Agent loads** — `forger.agent.yaml` injects the persona (identity, principles, critical actions) into the context window. Sidecar files (`forge-tier.yaml`, `preferences.yaml`) are loaded for persistent state.
|
|
54
|
+
3. **Workflow loads** — `workflow.md` presents the mode choice and routes to the first step file.
|
|
55
|
+
4. **Step-by-step execution** — Only the current step file is in context (just-in-time loading). Each step explicitly names the next one. The LLM reads, executes, saves output, then loads the next step. No future steps are ever preloaded.
|
|
56
|
+
5. **Knowledge injection** — Steps consult `skf-knowledge-index.csv` and selectively load fragments from `knowledge/` by tags and relevance. Cross-cutting principles (zero hallucination, confidence tiers, provenance) are loaded only when a step directs — not preloaded.
|
|
57
|
+
6. **Data injection** — Steps read `data/*.md` files as needed (schemas, heuristics, extraction patterns). This is deliberate context engineering: only the data relevant to the current step enters the context window.
|
|
58
|
+
7. **Templates** — When a step produces output (e.g., a skill brief or test report), it reads the template file and fills in placeholders with computed results. The template provides consistent structure; the step provides the content.
|
|
59
|
+
8. **Progress tracking** — Each step appends to an output file with state tracking. Resume mode reads this state and routes to the next incomplete step.
|
|
60
|
+
|
|
61
|
+
### Ferris Operating Modes
|
|
62
|
+
|
|
63
|
+
Ferris operates in four workflow-driven modes (mode is determined by which workflow is running, not conversation state):
|
|
64
|
+
|
|
65
|
+
| Mode | Workflows | Behavior |
|
|
66
|
+
|---------------|--------------------|-------------------------------------------------------------|
|
|
67
|
+
| **Architect** | SF, AN, BS, CS, QS, SS | Exploratory, assembling — discovers structure and scope |
|
|
68
|
+
| **Surgeon** | US | Precise, preserving — extracts and compiles with provenance |
|
|
69
|
+
| **Audit** | AS, TS | Judgmental, scoring — evaluates quality and detects drift |
|
|
70
|
+
| **Delivery** | EX | Packaging, ecosystem-ready — bundles for distribution |
|
|
9
71
|
|
|
10
72
|
---
|
|
11
73
|
|
|
@@ -22,7 +84,7 @@ AI agents hallucinate APIs. Not sometimes — constantly. The table below shows
|
|
|
22
84
|
| Copilot/Cursor built-in | Convenient | Generic. Doesn't know YOUR integration patterns. |
|
|
23
85
|
| **Skill Forge** | **Structural truth + automation** | **Rigid. (Feature, not bug.)** |
|
|
24
86
|
|
|
25
|
-
SKF solves this by mechanically extracting function signatures, type definitions, and usage patterns from
|
|
87
|
+
SKF solves this by mechanically extracting function signatures, type definitions, and usage patterns from code repositories — and enriching them with documentation and developer discourse — then compiling everything into verifiable, version-pinned skills that comply with the [agentskills.io specification](https://agentskills.io/specification).
|
|
26
88
|
|
|
27
89
|
---
|
|
28
90
|
|
|
@@ -311,6 +373,45 @@ Provenance maps enable verification: an `official` skill's provenance must trace
|
|
|
311
373
|
|
|
312
374
|
---
|
|
313
375
|
|
|
376
|
+
## Knowledge Base
|
|
377
|
+
|
|
378
|
+
SKF relies on a curated skill compilation knowledge base:
|
|
379
|
+
|
|
380
|
+
- Index: `src/knowledge/skf-knowledge-index.csv`
|
|
381
|
+
- Fragments: `src/knowledge/`
|
|
382
|
+
|
|
383
|
+
Workflows load only the fragments required for the current task to stay focused and compliant.
|
|
384
|
+
|
|
385
|
+
## Module Structure
|
|
386
|
+
|
|
387
|
+
```
|
|
388
|
+
src/
|
|
389
|
+
├── module.yaml
|
|
390
|
+
├── module-help.csv
|
|
391
|
+
├── agents/
|
|
392
|
+
│ └── forger.agent.yaml
|
|
393
|
+
├── forger/
|
|
394
|
+
│ ├── forge-tier.yaml
|
|
395
|
+
│ ├── preferences.yaml
|
|
396
|
+
│ └── README.md
|
|
397
|
+
├── knowledge/
|
|
398
|
+
│ ├── skf-knowledge-index.csv
|
|
399
|
+
│ └── *.md (10 fragments)
|
|
400
|
+
└── workflows/
|
|
401
|
+
├── setup-forge/
|
|
402
|
+
├── analyze-source/
|
|
403
|
+
├── brief-skill/
|
|
404
|
+
├── create-skill/
|
|
405
|
+
├── quick-skill/
|
|
406
|
+
├── create-stack-skill/
|
|
407
|
+
├── update-skill/
|
|
408
|
+
├── audit-skill/
|
|
409
|
+
├── test-skill/
|
|
410
|
+
└── export-skill/
|
|
411
|
+
```
|
|
412
|
+
|
|
413
|
+
---
|
|
414
|
+
|
|
314
415
|
## Security
|
|
315
416
|
|
|
316
417
|
- All tool wrappers use array-style subprocess execution — no shell interpolation
|
package/docs/concepts.md
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Concepts
|
|
3
|
+
description: Plain-English definitions of key Skill Forge terms — agent skills, provenance, confidence tiers, drift, and more
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Concepts
|
|
7
|
+
|
|
8
|
+
This page defines the key terms you'll encounter in Skill Forge. Each one is explained in plain English with a concrete example.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## Agent Skills
|
|
13
|
+
|
|
14
|
+
An agent skill is an instruction file that tells an AI agent how to use your code. Instead of the agent guessing your API from its training data, it reads the skill and gets real function names, real parameter types, and real usage patterns.
|
|
15
|
+
|
|
16
|
+
Skills follow the [agentskills.io](https://agentskills.io) open standard, so they work across Claude, Cursor, Copilot, and other AI tools.
|
|
17
|
+
|
|
18
|
+
**Example:** A skill for `better-auth` tells your agent: "The function is `authClient.signIn.email()`, it takes `{ email: string, password: string }`, and it returns a `Promise<Session>`. Here's the source: `src/client.ts:L47`."
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Provenance
|
|
23
|
+
|
|
24
|
+
Provenance means every instruction in a skill traces back to where it came from. For code, that's a file and line number. For documentation, it's a URL. For developer discourse, it's an issue or PR reference. If SKF can't point to a source, it doesn't include the instruction.
|
|
25
|
+
|
|
26
|
+
**Examples** (from a [real generated skill](https://github.com/armelhbobdad/oh-my-skills)):
|
|
27
|
+
- `[AST:cognee/api/v1/search/search.py:L26]` — extracted from source code via AST parsing
|
|
28
|
+
- `[SRC:cognee/api/v1/session/__init__.py:L8]` — read from source code without AST verification
|
|
29
|
+
- `[EXT:docs.cognee.ai/getting-started/quickstart]` — sourced from external documentation
|
|
30
|
+
- `[QMD:cognee-temporal:issues.md]` — surfaced from indexed developer discourse
|
|
31
|
+
|
|
32
|
+
---
|
|
33
|
+
|
|
34
|
+
## Confidence Tiers (T1/T2/T3)
|
|
35
|
+
|
|
36
|
+
Each piece of information in a skill carries a confidence level based on where it came from:
|
|
37
|
+
|
|
38
|
+
- **T1 — AST extraction:** Pulled directly from source code via AST parsing. This is structural truth — the function signature actually exists in the code right now.
|
|
39
|
+
- **T2 — Evidence:** Found in issues, PRs, changelogs, or documentation within the repository. Reliable context, but not as definitive as code.
|
|
40
|
+
- **T3 — External:** Pulled from external documentation or websites. Treated with caution and clearly marked.
|
|
41
|
+
|
|
42
|
+
**Example:** A function signature is T1. A deprecation warning from a closed GitHub issue is T2. A usage example from a blog post is T3.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## Capability Tiers (Quick/Forge/Deep)
|
|
47
|
+
|
|
48
|
+
Your capability tier depends on which tools you have installed. Each tier builds on the previous one:
|
|
49
|
+
|
|
50
|
+
- **Quick** — GitHub CLI only. SKF reads source files and builds best-effort skills. Works in under a minute.
|
|
51
|
+
- **Forge** — Adds [ast-grep](https://ast-grep.github.io). SKF uses AST parsing for structural truth. Instructions are verified against the actual code structure.
|
|
52
|
+
- **Deep** — Adds [QMD](https://github.com/tobi/qmd). SKF indexes knowledge for semantic search. Skills get enriched with historical context, deprecation warnings, and cross-reference intelligence.
|
|
53
|
+
|
|
54
|
+
You don't need all tools to start. SKF detects what you have and sets your tier automatically. See [How It Works](../architecture.md) for the full technical treatment.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## Drift
|
|
59
|
+
|
|
60
|
+
Drift happens when the source code changes but the skill instructions haven't been updated to match. A skill might still reference a function that was renamed, removed, or had its signature changed.
|
|
61
|
+
|
|
62
|
+
SKF detects drift by comparing the skill's recorded provenance against the current code. The `audit-skill` workflow (`@Ferris AS`) scans for these mismatches.
|
|
63
|
+
|
|
64
|
+
**Example:** Your skill says `createUser(name: string)` but the function was renamed to `registerUser(name: string, email: string)` in the last release. That's drift.
|
|
65
|
+
|
|
66
|
+
---
|
|
67
|
+
|
|
68
|
+
## Version Pinning
|
|
69
|
+
|
|
70
|
+
Every skill records the exact version (or commit) of the source code it was built from. This means you always know which version of the library the instructions apply to.
|
|
71
|
+
|
|
72
|
+
When the source updates, you can re-run `@Ferris US` (update-skill) to regenerate the skill for the new version while preserving any manual additions you've made.
|
|
73
|
+
|
|
74
|
+
---
|
|
75
|
+
|
|
76
|
+
## BMAD Module
|
|
77
|
+
|
|
78
|
+
SKF is a plugin (called a "module") for the [BMad Method](https://github.com/bmad-code-org/BMAD-METHOD), a framework for running structured AI workflows. BMad provides the workflow engine — step-by-step execution, shared knowledge bases, and consistent outputs. SKF plugs into that engine and focuses specifically on skill compilation.
|
|
79
|
+
|
|
80
|
+
You don't need to know BMad to use SKF. The standalone installer sets everything up.
|
|
81
|
+
|
|
82
|
+
---
|
|
83
|
+
|
|
84
|
+
## Ferris
|
|
85
|
+
|
|
86
|
+
Ferris is the AI agent persona that runs all SKF workflows. When you type `@Ferris CS`, you're telling Ferris to run the create-skill workflow.
|
|
87
|
+
|
|
88
|
+
Ferris switches between four modes depending on which workflow is active: Architect (exploring and building), Surgeon (precise updates), Audit (quality checks), and Delivery (packaging for distribution).
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## Zero Hallucination
|
|
93
|
+
|
|
94
|
+
SKF's core principle: if an instruction can't be traced back to actual source code, it doesn't get included in the skill. This is the opposite of how most AI tools work — they generate plausible-sounding content from training data. SKF only includes what it can verify.
|
|
95
|
+
|
|
96
|
+
This doesn't mean skills are perfect. Quick-tier skills read source files without AST verification, so they rely on best-effort extraction. But even Quick skills cite their sources, and no tier includes invented information.
|
package/docs/examples.md
CHANGED
|
@@ -9,6 +9,45 @@ This section provides practical examples for using SKF: Skill Forge.
|
|
|
9
9
|
|
|
10
10
|
---
|
|
11
11
|
|
|
12
|
+
## What the Output Looks Like
|
|
13
|
+
|
|
14
|
+
When SKF generates a skill, you get a `SKILL.md` file with machine-readable frontmatter and provenance-backed instructions. Here's a trimmed example from a real skill generated for [cognee](https://github.com/topoteretes/cognee) (browse the full output at [oh-my-skills](https://github.com/armelhbobdad/oh-my-skills)):
|
|
15
|
+
|
|
16
|
+
**Frontmatter (tells AI agents when to load this skill):**
|
|
17
|
+
|
|
18
|
+
```yaml
|
|
19
|
+
name: cognee
|
|
20
|
+
description: Use when cognee is a Python AI memory engine that transforms
|
|
21
|
+
documents into knowledge graphs with vector and graph storage for semantic
|
|
22
|
+
search and reasoning. Use this skill when writing code that calls cognee's
|
|
23
|
+
Python API (add, cognify, search, memify, config, datasets, prune, session).
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
**Body (what your AI agent reads):**
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
## Key API Summary
|
|
30
|
+
|
|
31
|
+
| Function | Purpose | Key Params | Source |
|
|
32
|
+
|----------|---------|------------|--------|
|
|
33
|
+
| add() | Ingest text, files, binary data | data, dataset_name | [AST:cognee/api/v1/add/add.py:L22] |
|
|
34
|
+
| cognify() | Build knowledge graph | datasets, graph_model | [AST:cognee/api/v1/cognify/cognify.py:L47] |
|
|
35
|
+
| search() | Query knowledge graph | query_text, query_type | [AST:cognee/api/v1/search/search.py:L26] |
|
|
36
|
+
| memify() | Enrich graph with custom tasks | extraction_tasks, data | [AST:cognee/modules/memify/memify.py:L27] |
|
|
37
|
+
| session.* | Session history and feedback | get_session(), add_feedback() | [SRC:cognee/api/v1/session/__init__.py:L8] |
|
|
38
|
+
| DataPoint | Base class for custom graph nodes | inherit and add fields | [EXT:docs.cognee.ai/guides/custom-data-models] |
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Provenance tags trace each instruction to its source:
|
|
42
|
+
- `[AST:file:line]` — extracted from code via AST parsing (highest confidence)
|
|
43
|
+
- `[SRC:file:line]` — read from source code without AST verification
|
|
44
|
+
- `[EXT:url]` — sourced from external documentation
|
|
45
|
+
- `[QMD:collection:doc]` — surfaced from indexed developer discourse (issues, PRs, changelogs)
|
|
46
|
+
|
|
47
|
+
See [How It Works](../architecture.md) for the full output structure.
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
12
51
|
## Example Workflows
|
|
13
52
|
|
|
14
53
|
### Quick Skill — 47 Seconds
|
package/docs/getting-started.md
CHANGED
|
@@ -11,7 +11,7 @@ Welcome to Skill Forge! This guide will help you get up and running.
|
|
|
11
11
|
|
|
12
12
|
## What This Module Does
|
|
13
13
|
|
|
14
|
-
Skill Forge
|
|
14
|
+
Skill Forge analyzes code repositories, documentation websites, and developer discourse to build verified instruction files ("skills") for AI agents. Instead of your agent guessing API calls from training data, it follows instructions where every function, type, and pattern traces back to its source — a file and line for code, a URL for documentation, an issue or PR for discourse. Skills comply with the [agentskills.io](https://agentskills.io) open standard and work across Claude, Cursor, Copilot, and other AI tools. See the [Concepts](../concepts.md) page for definitions of key terms.
|
|
15
15
|
|
|
16
16
|
---
|
|
17
17
|
|
|
@@ -68,6 +68,20 @@ Don't worry if you don't have all tools — SKF detects what's available and set
|
|
|
68
68
|
|
|
69
69
|
---
|
|
70
70
|
|
|
71
|
+
## Configuration
|
|
72
|
+
|
|
73
|
+
SKF has two install-time variables (defined in `src/module.yaml`) and one runtime preference:
|
|
74
|
+
|
|
75
|
+
| Variable | Purpose | Default |
|
|
76
|
+
|------------------------|------------------------------------------------------------------------------------------------------|-----------------------------|
|
|
77
|
+
| `skills_output_folder` | Where generated skills are saved | `{project-root}/skills` |
|
|
78
|
+
| `forge_data_folder` | Where workspace artifacts are stored | `{project-root}/forge-data` |
|
|
79
|
+
| `tier_override` | Force a specific tier for comparison or testing (in `_bmad/_memory/forger-sidecar/preferences.yaml`) | `~` (auto-detect) |
|
|
80
|
+
|
|
81
|
+
Runtime configuration (tool detection, tier, parallel settings) is managed by the `setup-forge` workflow in `forge-tier.yaml`.
|
|
82
|
+
|
|
83
|
+
---
|
|
84
|
+
|
|
71
85
|
## First Steps
|
|
72
86
|
|
|
73
87
|
### 1. Setup Your Forge
|
package/docs/index.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
---
|
|
2
2
|
title: Skill Forge (SKF)
|
|
3
|
-
description:
|
|
3
|
+
description: Turn code and docs into instructions AI agents can actually follow
|
|
4
4
|
template: splash
|
|
5
5
|
hero:
|
|
6
6
|
title: Skill Forge (SKF)
|
|
7
|
-
tagline:
|
|
7
|
+
tagline: Turn code and docs into instructions AI agents can actually follow.
|
|
8
8
|
actions:
|
|
9
9
|
- text: Getting Started
|
|
10
10
|
link: ./getting-started/
|
|
@@ -15,62 +15,35 @@ hero:
|
|
|
15
15
|
icon: external
|
|
16
16
|
---
|
|
17
17
|
|
|
18
|
-
## What
|
|
18
|
+
## What does Skill Forge do?
|
|
19
19
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
- **AST-Verified**: Structural truth via ast-grep — no guessing, no hallucination.
|
|
23
|
-
- **Provenance-Backed**: Every claim traces to source code with file and line references.
|
|
24
|
-
- **Progressive Tiers**: Quick (no setup) → Forge (ast-grep) → Deep (QMD knowledge).
|
|
25
|
-
- **Ecosystem-First**: Checks for official skills before generating community ones.
|
|
20
|
+
AI agents hallucinate API calls. They invent function names, guess parameter types, and produce code that doesn't compile. Skill Forge fixes this by analyzing code repositories, documentation, and developer discourse — extracting real signatures and patterns and compiling them into verified instruction files that any AI agent can follow. Every instruction traces back to where it came from.
|
|
26
21
|
|
|
27
22
|
## Quick Install
|
|
28
23
|
|
|
29
24
|
Requires [Node.js](https://nodejs.org/) >= 22.
|
|
30
25
|
|
|
31
|
-
**Standalone:**
|
|
32
|
-
|
|
33
26
|
```bash
|
|
34
27
|
npx bmad-module-skill-forge install
|
|
35
28
|
```
|
|
36
29
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
```bash
|
|
40
|
-
npx bmad-method install # select "Add custom modules" and point to src/
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
Then interact with the forge agent:
|
|
30
|
+
Then set up your environment and generate your first skill:
|
|
44
31
|
|
|
45
32
|
```
|
|
46
|
-
@Ferris SF
|
|
47
|
-
@Ferris QS #
|
|
33
|
+
@Ferris SF # Set up your forge
|
|
34
|
+
@Ferris QS <package> # Generate a skill in under a minute
|
|
48
35
|
```
|
|
49
36
|
|
|
50
|
-
See the [Getting Started](./getting-started
|
|
51
|
-
|
|
52
|
-
## Core Workflows
|
|
53
|
-
|
|
54
|
-
| Workflow | Trigger | Purpose |
|
|
55
|
-
|----------|---------|---------|
|
|
56
|
-
| [Setup Forge](/workflows/#setup-forge-sf) | SF | Initialize forge, detect tools, set tier |
|
|
57
|
-
| [Brief Skill](/workflows/#brief-skill-bs) | BS | Scope and design a skill |
|
|
58
|
-
| [Create Skill](/workflows/#create-skill-cs) | CS | Compile a skill from a brief |
|
|
59
|
-
| [Quick Skill](/workflows/#quick-skill-qs) | QS | Fast skill, no brief needed |
|
|
60
|
-
| [Stack Skill](/workflows/#stack-skill-ss) | SS | Consolidated project stack skill |
|
|
61
|
-
| [Update Skill](/workflows/#update-skill-us) | US | Regenerate after source changes |
|
|
62
|
-
| [Audit Skill](/workflows/#audit-skill-as) | AS | Drift detection |
|
|
63
|
-
| [Test Skill](/workflows/#test-skill-ts) | TS | Verify completeness |
|
|
64
|
-
| [Export Skill](/workflows/#export-skill-ex) | EX | Package for distribution |
|
|
65
|
-
| [Analyze Source](/workflows/#analyze-source-an) | AN | Discover what to skill |
|
|
37
|
+
See the [Getting Started](./getting-started/) guide for full installation and usage instructions.
|
|
66
38
|
|
|
67
39
|
## Documentation
|
|
68
40
|
|
|
69
|
-
- **[Getting Started](./getting-started
|
|
70
|
-
- **[
|
|
71
|
-
- **[
|
|
72
|
-
- **[Workflows](./workflows
|
|
73
|
-
- **[
|
|
41
|
+
- **[Getting Started](./getting-started/)** — Installation, prerequisites, and your first skill
|
|
42
|
+
- **[Concepts](./concepts/)** — Plain-English definitions of key terms (provenance, tiers, drift, and more)
|
|
43
|
+
- **[How It Works](./architecture/)** — Architecture, capability model, output format, and design decisions
|
|
44
|
+
- **[Workflows](./workflows/)** — All 10 workflows with commands and connection diagrams
|
|
45
|
+
- **[Agents](./agents/)** — Ferris: the AI agent that runs all SKF workflows
|
|
46
|
+
- **[Examples](./examples/)** — Real-world scenarios, tips, and troubleshooting
|
|
74
47
|
|
|
75
48
|
## Support
|
|
76
49
|
|
package/docs/workflows.md
CHANGED
|
@@ -5,7 +5,7 @@ description: All 10 SKF workflows with commands, steps, and connection diagram
|
|
|
5
5
|
|
|
6
6
|
# Workflows Reference
|
|
7
7
|
|
|
8
|
-
SKF
|
|
8
|
+
SKF has 10 workflows. You trigger them by typing commands to [Ferris](../agents.md), the AI agent that runs everything. Each workflow handles a specific part of the skill lifecycle — from analyzing source code to packaging for distribution. If any terms are unfamiliar, see the [Concepts](../concepts.md) page for plain-English definitions.
|
|
9
9
|
|
|
10
10
|
---
|
|
11
11
|
|
package/package.json
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$schema": "https://json.schemastore.org/package.json",
|
|
3
3
|
"name": "bmad-module-skill-forge",
|
|
4
|
-
"version": "0.
|
|
5
|
-
"description": "BMAD module —
|
|
4
|
+
"version": "0.4.0",
|
|
5
|
+
"description": "BMAD module — Turn code and docs into instructions AI agents can actually follow. Progressive capability tiers (Quick/Forge/Deep).",
|
|
6
6
|
"keywords": [
|
|
7
7
|
"bmad",
|
|
8
8
|
"bmad-method",
|
package/src/module.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
code: skf
|
|
2
2
|
name: "SKF: Skill Forge — Evidence-Based Agent Skills Compiler"
|
|
3
|
-
header: "
|
|
4
|
-
subheader: "
|
|
3
|
+
header: "Turn code and docs into instructions AI agents can actually follow"
|
|
4
|
+
subheader: "Every instruction traces to source. Zero hallucination tolerance."
|
|
5
5
|
description: "Skill Forge transforms code repositories, documentation, and developer discourse into agentskills.io-compliant, version-pinned, provenance-backed agent skills. Uses ast-grep for structural truth, QMD for knowledge intelligence, and a progressive capability model (Quick/Forge/Deep). Standalone module — no other modules required. Recommended alongside BMM for full development workflow integration."
|
|
6
6
|
default_selected: false
|
|
7
7
|
|
|
@@ -26,13 +26,16 @@ Source reading via gh_bridge — infer exports from file structure and content.
|
|
|
26
26
|
|
|
27
27
|
## Forge Tier (AST Available)
|
|
28
28
|
|
|
29
|
-
Structural extraction via
|
|
29
|
+
Structural extraction via ast-grep — verified exports with line-level citations.
|
|
30
|
+
|
|
31
|
+
> **Note:** `ast_bridge.*` and `qmd_bridge.*` references below are **conceptual interfaces**, not callable functions. They describe the operation to perform. Use ast-grep (MCP tool or CLI) for `ast_bridge.*` operations and QMD (MCP tool or CLI) for `qmd_bridge.*` operations. See the AST Extraction Protocol section below and the TOOL/SUBPROCESS FALLBACK rule for dispatch details.
|
|
30
32
|
|
|
31
33
|
### Strategy
|
|
34
|
+
|
|
32
35
|
1. Detect language from brief or file extensions
|
|
33
|
-
2.
|
|
36
|
+
2. Use ast-grep to extract all exports from `path` for the given `language` (scan definitions)
|
|
34
37
|
3. For each export: function name, full signature, parameter types, return type, line number
|
|
35
|
-
4. `
|
|
38
|
+
4. Use ast-grep to detect co-imported symbols in `path` for the given `libraries[]`
|
|
36
39
|
5. Build extraction rules YAML for reproducibility
|
|
37
40
|
|
|
38
41
|
### Confidence
|
|
@@ -130,7 +133,8 @@ When MCP tools are unavailable or the repo exceeds 500 files in scope, use `--js
|
|
|
130
133
|
# Patterns are matched against the full file path as emitted by ast-grep.
|
|
131
134
|
# Ensure paths are relative to the same root as the patterns (strip ./ prefix if needed).
|
|
132
135
|
ast-grep -p '{pattern}' -l {language} --json=stream {path} | python3 -c "
|
|
133
|
-
import sys, json, fnmatch
|
|
136
|
+
import sys, json, fnmatch, signal
|
|
137
|
+
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
|
|
134
138
|
|
|
135
139
|
EXCLUDES = {exclude_patterns}
|
|
136
140
|
|
|
@@ -242,6 +246,7 @@ After initial AST extraction, some top-level exports may resolve to **module imp
|
|
|
242
246
|
2. For each import where Y was NOT found by the initial AST scan:
|
|
243
247
|
- Check if the import path resolves to a directory (e.g., `{package}/api/v1/delete/` exists with `__init__.py`)
|
|
244
248
|
- If directory: read its `__init__.py` to find the actual re-exported symbol
|
|
249
|
+
- **Handle aliases:** Check for `from .module import A as B` patterns in the intermediate `__init__.py`. If the parent imports `B`, trace through to `A` in `.module`. If the parent imports `A` but the `__init__.py` only exports it as `B` (via `from .module import A as B`), match by original name `A` and note the alias
|
|
245
250
|
- Trace the symbol to its definition file and run AST extraction on that file
|
|
246
251
|
3. Cite the actual definition location: `[AST:{definition_file}:L{line}]`
|
|
247
252
|
|
|
@@ -253,6 +258,14 @@ from .api.v1.delete import delete # delete/ is a directory → read delete/__
|
|
|
253
258
|
|
|
254
259
|
# Direct function import — no follow needed
|
|
255
260
|
from .api.v1.add.add import add # add.py exists with def add()
|
|
261
|
+
|
|
262
|
+
# Aliased re-export — follow through alias
|
|
263
|
+
# In cognee/api/v1/visualize/__init__.py:
|
|
264
|
+
# from .start_visualization_server import visualization_server
|
|
265
|
+
# In cognee/__init__.py:
|
|
266
|
+
# from .api.v1.visualize import start_visualization_server
|
|
267
|
+
# → Match start_visualization_server against both definition names AND alias names
|
|
268
|
+
# in the intermediate __init__.py to resolve the chain
|
|
256
269
|
```
|
|
257
270
|
|
|
258
271
|
**Unresolvable imports:** If the import statement is a star-import (`from .X import *`) or a conditional import (`try`/`except`), the symbol cannot be reliably traced via this protocol. Record it with `[SRC:{package}/__init__.py:L{line}]` (T1-low) and a note: "star/conditional import — manual trace required."
|
|
@@ -261,51 +274,3 @@ from .api.v1.add.add import add # add.py exists with def add()
|
|
|
261
274
|
|
|
262
275
|
**Other languages:** JS/TS barrel files (`index.ts` with `export { X } from './module'`) follow the same principle — trace the re-export to the definition file. Rust `pub use` and Go package-level re-exports are less common but follow the same heuristic when encountered.
|
|
263
276
|
|
|
264
|
-
---
|
|
265
|
-
|
|
266
|
-
## Tier Degradation Rules
|
|
267
|
-
|
|
268
|
-
### Remote Source at Forge/Deep Tier
|
|
269
|
-
|
|
270
|
-
When `source_repo` is a remote URL (GitHub URL or owner/repo format) and the tier is Forge or Deep:
|
|
271
|
-
|
|
272
|
-
- **ast-grep requires local files** — it cannot operate on remote URLs
|
|
273
|
-
|
|
274
|
-
**Ephemeral clone strategy (preferred):**
|
|
275
|
-
|
|
276
|
-
1. Check `git` availability (`git --version`). `git` is effectively guaranteed at Deep tier (via `gh` dependency) but NOT guaranteed at Forge tier.
|
|
277
|
-
2. If `git` is available: perform an ephemeral shallow clone to a system temp path (`{system_temp}/skf-ephemeral-{skill-name}-{timestamp}/`).
|
|
278
|
-
3. For create-skill: use `--depth 1 --single-branch --filter=blob:none`; if `include_patterns` are specified, apply mode selection: if `exclude_patterns` are absent, use cone mode (convert include_patterns to directory roots; use `--skip-checks` for individual file paths); if `exclude_patterns` are present, use `--no-cone` mode (pass gitignore-style patterns with `!`-prefixed excludes — includes first, then negations). See source-resolution-protocols.md for the full conversion rules.
|
|
279
|
-
4. For update-skill: use sparse-checkout with `--skip-checks` scoped to the changed files from the change manifest only (file paths require `--skip-checks`). No `--branch` flag — uses the remote default branch (must match the branch used during original create-skill run).
|
|
280
|
-
5. If clone succeeds: use the local clone path for AST extraction. All results are T1 with `[AST:...]` citations.
|
|
281
|
-
6. Cleanup: delete the temp directory after extraction inventory is built and all data is in context. The clone never persists beyond the extraction step.
|
|
282
|
-
|
|
283
|
-
**Fallback (clone fails or `git` unavailable):**
|
|
284
|
-
|
|
285
|
-
- The extraction step MUST warn the user explicitly before degrading
|
|
286
|
-
- **create-skill:** Warning must include actionable guidance — clone locally and update `source_repo` in the brief to the local path
|
|
287
|
-
- **update-skill:** Warning must include actionable guidance — clone locally, re-run [CS] Create Skill with the local path to regenerate provenance data, then re-run the update
|
|
288
|
-
- Extraction proceeds using Quick tier strategy (source reading via gh_bridge)
|
|
289
|
-
- All results labeled T1-low with `[SRC:...]` citations
|
|
290
|
-
- The degradation reason is recorded in the evidence report
|
|
291
|
-
|
|
292
|
-
Silent degradation is **forbidden**. The user must always know when AST extraction was skipped and why.
|
|
293
|
-
|
|
294
|
-
### AST Tool Unavailable at Forge/Deep Tier
|
|
295
|
-
|
|
296
|
-
When the tier is Forge or Deep but ast-grep is not functional:
|
|
297
|
-
|
|
298
|
-
- The extraction step MUST warn the user explicitly before degrading
|
|
299
|
-
- Warning must include actionable guidance: run [SF] Setup Forge to detect tools
|
|
300
|
-
- Extraction proceeds using Quick tier strategy
|
|
301
|
-
- All results labeled T1-low
|
|
302
|
-
- The degradation reason is recorded in the evidence report
|
|
303
|
-
|
|
304
|
-
### Per-File AST Failure
|
|
305
|
-
|
|
306
|
-
When ast-grep fails on an individual file (parse error, unsupported syntax):
|
|
307
|
-
|
|
308
|
-
- Fall back to source reading for **that file only**
|
|
309
|
-
- Other files continue with AST extraction
|
|
310
|
-
- The affected file's results are labeled T1-low; unaffected files retain T1
|
|
311
|
-
- Log a warning noting which file degraded and why
|
|
@@ -125,7 +125,7 @@ Indexed pipe-delimited format for CLAUDE.md managed section (~80-120 tokens per
|
|
|
125
125
|
"tool_versions": {
|
|
126
126
|
"ast_grep": "{version-or-null}",
|
|
127
127
|
"qmd": "{version-or-null}",
|
|
128
|
-
"skf": "
|
|
128
|
+
"skf": "{skf_version}"
|
|
129
129
|
},
|
|
130
130
|
"stats": {
|
|
131
131
|
"exports_documented": 0,
|
|
@@ -204,7 +204,7 @@ Each reference file includes:
|
|
|
204
204
|
## Tool Versions
|
|
205
205
|
- ast-grep: {version}
|
|
206
206
|
- QMD: {version}
|
|
207
|
-
- SKF:
|
|
207
|
+
- SKF: {skf_version}
|
|
208
208
|
|
|
209
209
|
## Extraction Summary
|
|
210
210
|
- Files scanned: {count}
|
|
@@ -90,9 +90,17 @@ If `source_repo` is a remote URL (GitHub URL or owner/repo format) AND tier is F
|
|
|
90
90
|
|
|
91
91
|
**Note:** `--no-cone` mode is slower than cone mode for very large repositories but eliminates downloading excluded blobs entirely.
|
|
92
92
|
|
|
93
|
+
**Always-included root files:**
|
|
94
|
+
|
|
95
|
+
Regardless of `include_patterns`, always add these root-level version/manifest files to the sparse-checkout pattern list. These are needed for version reconciliation and must not require a fallback to `gh api`:
|
|
96
|
+
|
|
97
|
+
`pyproject.toml`, `package.json`, `Cargo.toml`, `go.mod`, `setup.py`, `setup.cfg`, `VERSION`
|
|
98
|
+
|
|
99
|
+
In cone mode, always use the `--skip-checks` command form when adding these files — even if `include_patterns` resolved to only directory roots (which would normally use the form without `--skip-checks`). The command becomes: `git -C {temp_path} sparse-checkout set --skip-checks {directory_roots} pyproject.toml package.json Cargo.toml go.mod setup.py setup.cfg VERSION`. In no-cone mode, list them as explicit include patterns before any negation patterns. Do not flag them as extraneous inclusions during post-checkout filtering.
|
|
100
|
+
|
|
93
101
|
**Post-checkout filtering:**
|
|
94
102
|
|
|
95
|
-
After checkout, apply the original glob `include_patterns` as file-level filters when building the extraction file list — sparse-checkout gets the right directories, glob filtering narrows to the exact files. When `--no-cone` mode was used, most exclude filtering is already done at the git level, but apply `exclude_patterns` as a final pass to catch any edge cases where gitignore pattern matching diverges from the brief's glob semantics.
|
|
103
|
+
After checkout, apply the original glob `include_patterns` as file-level filters when building the extraction file list — sparse-checkout gets the right directories, glob filtering narrows to the exact files. When `--no-cone` mode was used, most exclude filtering is already done at the git level, but apply `exclude_patterns` as a final pass to catch any edge cases where gitignore pattern matching diverges from the brief's glob semantics. Always-included root files (see above) are exempt from post-checkout filtering.
|
|
96
104
|
|
|
97
105
|
3. **If clone succeeds:** Update the working source path to `{temp_path}` for all subsequent AST operations in this step. Proceed with the **Forge/Deep Tier** extraction strategy below. Mark `ephemeral_clone_active = true` for cleanup.
|
|
98
106
|
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Tier Degradation Rules
|
|
2
|
+
|
|
3
|
+
## Remote Source at Forge/Deep Tier
|
|
4
|
+
|
|
5
|
+
When `source_repo` is a remote URL (GitHub URL or owner/repo format) and the tier is Forge or Deep:
|
|
6
|
+
|
|
7
|
+
- **ast-grep requires local files** — it cannot operate on remote URLs
|
|
8
|
+
|
|
9
|
+
**Ephemeral clone strategy (preferred):**
|
|
10
|
+
|
|
11
|
+
1. Check `git` availability (`git --version`). `git` is effectively guaranteed at Deep tier (via `gh` dependency) but NOT guaranteed at Forge tier.
|
|
12
|
+
2. If `git` is available: perform an ephemeral shallow clone to a system temp path (`{system_temp}/skf-ephemeral-{skill-name}-{timestamp}/`).
|
|
13
|
+
3. For create-skill: use `--depth 1 --single-branch --filter=blob:none`; if `include_patterns` are specified, apply mode selection: if `exclude_patterns` are absent, use cone mode (convert include_patterns to directory roots; use `--skip-checks` for individual file paths); if `exclude_patterns` are present, use `--no-cone` mode (pass gitignore-style patterns with `!`-prefixed excludes — includes first, then negations). See source-resolution-protocols.md for the full conversion rules.
|
|
14
|
+
4. For update-skill: use sparse-checkout with `--skip-checks` scoped to the changed files from the change manifest only (file paths require `--skip-checks`). No `--branch` flag — uses the remote default branch (must match the branch used during original create-skill run).
|
|
15
|
+
5. If clone succeeds: use the local clone path for AST extraction. All results are T1 with `[AST:...]` citations.
|
|
16
|
+
6. Cleanup: delete the temp directory after extraction inventory is built and all data is in context. The clone never persists beyond the extraction step.
|
|
17
|
+
|
|
18
|
+
**Fallback (clone fails or `git` unavailable):**
|
|
19
|
+
|
|
20
|
+
- The extraction step MUST warn the user explicitly before degrading
|
|
21
|
+
- **create-skill:** Warning must include actionable guidance — clone locally and update `source_repo` in the brief to the local path
|
|
22
|
+
- **update-skill:** Warning must include actionable guidance — clone locally, re-run [CS] Create Skill with the local path to regenerate provenance data, then re-run the update
|
|
23
|
+
- Extraction proceeds using Quick tier strategy (source reading via gh_bridge)
|
|
24
|
+
- All results labeled T1-low with `[SRC:...]` citations
|
|
25
|
+
- The degradation reason is recorded in the evidence report
|
|
26
|
+
|
|
27
|
+
Silent degradation is **forbidden**. The user must always know when AST extraction was skipped and why.
|
|
28
|
+
|
|
29
|
+
## AST Tool Unavailable at Forge/Deep Tier
|
|
30
|
+
|
|
31
|
+
When the tier is Forge or Deep but ast-grep is not functional:
|
|
32
|
+
|
|
33
|
+
- The extraction step MUST warn the user explicitly before degrading
|
|
34
|
+
- Warning must include actionable guidance: run [SF] Setup Forge to detect tools
|
|
35
|
+
- Extraction proceeds using Quick tier strategy
|
|
36
|
+
- All results labeled T1-low
|
|
37
|
+
- The degradation reason is recorded in the evidence report
|
|
38
|
+
|
|
39
|
+
## Per-File AST Failure
|
|
40
|
+
|
|
41
|
+
When ast-grep fails on an individual file (parse error, unsupported syntax):
|
|
42
|
+
|
|
43
|
+
- Fall back to source reading for **that file only**
|
|
44
|
+
- Other files continue with AST extraction
|
|
45
|
+
- The affected file's results are labeled T1-low; unaffected files retain T1
|
|
46
|
+
- Log a warning noting which file degraded and why
|
|
@@ -3,6 +3,7 @@ name: 'step-03-extract'
|
|
|
3
3
|
description: 'Tier-dependent source code extraction — AST or source reading for exports, signatures, and types'
|
|
4
4
|
nextStepFile: './step-03b-fetch-temporal.md'
|
|
5
5
|
extractionPatternsData: '../data/extraction-patterns.md'
|
|
6
|
+
tierDegradationRulesData: '../data/tier-degradation-rules.md'
|
|
6
7
|
sourceResolutionData: '../data/source-resolution-protocols.md'
|
|
7
8
|
---
|
|
8
9
|
|
|
@@ -104,7 +105,7 @@ Load `{sourceResolutionData}` completely. Follow the **Remote Source Resolution*
|
|
|
104
105
|
5. Build extraction rules YAML data for reproducibility
|
|
105
106
|
6. Confidence: All results T1 — `[AST:{file}:L{line}]`
|
|
106
107
|
|
|
107
|
-
**If AST tool is unavailable at Forge/Deep tier
|
|
108
|
+
**If AST tool is unavailable at Forge/Deep tier** (see `{tierDegradationRulesData}` for full rules):
|
|
108
109
|
|
|
109
110
|
⚠️ **Warn the user explicitly:** "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
|
|
110
111
|
|
|
@@ -97,7 +97,7 @@ Following the structure from the skill-sections data file:
|
|
|
97
97
|
- `exports_total`: `exports_public_api` + `exports_internal`
|
|
98
98
|
- `public_api_coverage`: `exports_documented / exports_public_api` (1.0 when all public API exports are documented; `null` if `exports_public_api` is 0)
|
|
99
99
|
- `total_coverage`: `exports_documented / exports_total` (may be low for large codebases — this is expected; `null` if `exports_total` is 0)
|
|
100
|
-
- Set `tool_versions` based on tier and available tools
|
|
100
|
+
- Set `tool_versions` based on tier and available tools. Resolve `{skf_version}` from the SKF module's `package.json` `version` field (run `node -p "require('./node_modules/bmad-module-skill-forge/package.json').version"` or read the installed module's `package.json`). If unresolvable, fall back to `git describe --tags --abbrev=0` in the SKF module root. Never hardcode the version.
|
|
101
101
|
|
|
102
102
|
### 5. Build references/ Content
|
|
103
103
|
|
|
@@ -115,7 +115,7 @@ One entry per extracted export: export_name, export_type, params[] (typed string
|
|
|
115
115
|
|
|
116
116
|
### 7. Build evidence-report.md Content
|
|
117
117
|
|
|
118
|
-
Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template.
|
|
118
|
+
Compilation audit trail: generation date, forge tier, source info, tool versions, extraction summary (files/exports/confidence), validation results (populated in step-06), warnings. See `{skillSectionsData}` for full template. Use the same `{skf_version}` value resolved in section 4 when populating the Tool Versions block.
|
|
119
119
|
|
|
120
120
|
### 8. Menu Handling Logic
|
|
121
121
|
|
|
@@ -40,6 +40,7 @@ To validate the compiled SKILL.md content against the agentskills.io specificati
|
|
|
40
40
|
- 💾 Validation results are added to evidence-report content in context
|
|
41
41
|
- 📖 Auto-fix pattern: validate → fix → re-validate (once)
|
|
42
42
|
- 🚫 Maximum one auto-fix attempt per validation failure
|
|
43
|
+
- ⏸️ **Conditional interaction:** If tessl returns suggestions (section 6b), halt for user input. Otherwise auto-proceed. This is a conditional gate step, not a pure auto-proceed step.
|
|
43
44
|
|
|
44
45
|
## CONTEXT BOUNDARIES:
|
|
45
46
|
|
|
@@ -73,6 +74,8 @@ This performs frontmatter validation, description quality checks, body limit enf
|
|
|
73
74
|
|
|
74
75
|
**Parse the JSON output** for: `qualityScore` (0-100), `diagnostics[]` (remaining issues), `fixed[]` (auto-corrected issues).
|
|
75
76
|
|
|
77
|
+
**Context sync after --fix:** If `fixed[]` is non-empty (i.e., `--fix` modified files on disk), re-read the modified SKILL.md to update the in-context copy. Verify the re-read content matches expectations before proceeding. This prevents silent divergence between the in-context SKILL.md and the on-disk version that step-07 will use for artifact generation.
|
|
78
|
+
|
|
76
79
|
**Note:** `skill-check` may return non-zero exit code even when `errorCount` is 0. Always rely on parsed JSON, not the shell exit code.
|
|
77
80
|
|
|
78
81
|
- **Score ≥ 70:** Record "Schema: PASS (score: {score}/100)" in evidence-report
|
|
@@ -143,7 +146,7 @@ Parse output for: `description_score`, `content_score`, `average_score`, `valida
|
|
|
143
146
|
- **Content score < 70%:** Record warning: "Content quality warning: tessl scored content at {score}%."
|
|
144
147
|
- **Unavailable:** Skip with note: "Content quality review skipped — tessl tool unavailable"
|
|
145
148
|
|
|
146
|
-
**
|
|
149
|
+
> **EXPECTED BEHAVIOR — Two-Tier Scoring:** The Skill Forge two-tier design (Tier 1 Key API Summary + Tier 2 Full API Reference) intentionally includes progressive disclosure. tessl's `conciseness` scorer will flag this as redundancy (typically scoring 2/3), which is **expected behavior — not a defect**. tessl may also suggest removing `[MANUAL]` markers, moving Full API Reference to a separate file, or consolidating duplicate parameter documentation — **all three suggestions conflict with SKF design principles and must be dismissed.** Acceptable threshold: content scores >= 60% are normal for two-tier skills. Do NOT consolidate Tier 1 and Tier 2 content to improve the score — the two-tier structure is a deliberate design choice for standalone usability.
|
|
147
150
|
|
|
148
151
|
tessl installs automatically via `npx`. A missing tool is not an error — graceful skip.
|
|
149
152
|
|
|
@@ -49,7 +49,7 @@ description: Package for distribution + platform-aware context injection
|
|
|
49
49
|
Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
|
|
50
50
|
|
|
51
51
|
- `project_name`, `output_folder`, `user_name`, `communication_language`, `document_output_language`
|
|
52
|
-
- `skills_output_folder`, `forge_data_folder`
|
|
52
|
+
- `skills_output_folder`, `forge_data_folder`, `sidecar_path`
|
|
53
53
|
|
|
54
54
|
### 2. First Step Execution
|
|
55
55
|
|
|
@@ -23,6 +23,8 @@ Check if `{source_path}` (from metadata.json `source_root`) exists on disk. If y
|
|
|
23
23
|
**State 2 — Local absent, provenance-map exists:**
|
|
24
24
|
Check `{forge_data_folder}/{skill_name}/provenance-map.json`. If present, use it as the baseline export inventory — each entry contains structured fields: `export_name`, `export_type`, `params[]`, `return_type`, `source_file`, `source_line`, `confidence`, and `ast_node_type`. Cross-reference against SKILL.md documented exports for name-matching and param-by-param coverage. Signature verification compares SKILL.md's documented params/return types against provenance-map entries directly. If remote reading tools are available (zread, deepwiki, gh API, or similar), supplement by reading the entry point file for live signature verification. Set `analysis_confidence: provenance-map`.
|
|
25
25
|
|
|
26
|
+
**State 2 limitations:** Signature verification at State 2 is **string comparison only**, not semantic. Provenance-map stores parameters as flat string arrays (e.g., `["data: Union[BinaryIO, list, str]"]`), so `str` vs `String` or `list` vs `List[Any]` would be treated as mismatches even when semantically equivalent. For full type-aware verification (handling type aliases, generic equivalence), State 1 (local source) with AST re-parsing is required. When the SKILL.md was compiled from the same provenance-map (typical for create-then-test flows), strings match exactly and this limitation has no practical effect.
|
|
27
|
+
|
|
26
28
|
**State 3 — No provenance-map, metadata exports exist (quick-skill path):**
|
|
27
29
|
If no provenance-map.json exists (typical for quick-skill output), fall back to `metadata.json`'s `exports[]` array for the export name list. Coverage check becomes a self-consistency comparison: are all names in `exports[]` documented in SKILL.md with description, parameters, and return type? Signatures cannot be verified. If remote reading tools are available, supplement by reading the entry point for live export comparison. Set `analysis_confidence: metadata-only`.
|
|
28
30
|
|
|
@@ -35,3 +37,9 @@ If none of the above succeed, fall through to docs-only mode (section 0 already
|
|
|
35
37
|
Set `analysis_confidence` in context for use in Section 2 analysis depth, step-05 output, and step-05 scoring.
|
|
36
38
|
|
|
37
39
|
**Confidence tier mapping:** `full` = T1, `provenance-map` = T1, `metadata-only` = T1-low, `remote-only` = T1-low, `docs-only` = T3. This aligns with the T1/T1-low/T2/T3 scale used across all SKF workflows.
|
|
40
|
+
|
|
41
|
+
**Degradation notice rules:** When `analysis_confidence` is `provenance-map`, check the `confidence` field of provenance-map entries before emitting a degradation recommendation:
|
|
42
|
+
|
|
43
|
+
- **All/most entries T1 (AST-verified):** The provenance-map data is already at highest confidence. Do NOT recommend re-running with a local clone — it would produce identical results. Use: "Resolved via: provenance-map (T1 AST-verified at compilation time). Local clone not required — provenance data is already at highest confidence."
|
|
44
|
+
- **Mixed T1/T1-low entries:** Report the breakdown. Recommend local clone only for the T1-low entries: "Resolved via: provenance-map ({n} T1, {m} T1-low). Re-run with local clone to upgrade T1-low entries to full AST verification."
|
|
45
|
+
- **All/most entries T1-low or lower:** Keep the standard recommendation: "Re-run with local clone for full AST-backed verification."
|
|
@@ -168,7 +168,7 @@ Basic structural validation of **{skill_name}**:
|
|
|
168
168
|
- {N} structural issues found
|
|
169
169
|
- Coherence category not scored (weight redistributed to coverage)
|
|
170
170
|
|
|
171
|
-
**Proceeding to
|
|
171
|
+
**Proceeding to external validation...**"
|
|
172
172
|
|
|
173
173
|
**For Contextual Mode:**
|
|
174
174
|
"**Coherence check complete (contextual mode).**
|
|
@@ -180,11 +180,11 @@ Reference validation of **{skill_name}**:
|
|
|
180
180
|
|
|
181
181
|
**{N} issues found** — details in Coherence Analysis section.
|
|
182
182
|
|
|
183
|
-
**Proceeding to
|
|
183
|
+
**Proceeding to external validation...**"
|
|
184
184
|
|
|
185
185
|
### 8. Auto-Proceed
|
|
186
186
|
|
|
187
|
-
Display: "**Proceeding to
|
|
187
|
+
Display: "**Proceeding to external validation...**"
|
|
188
188
|
|
|
189
189
|
#### Menu Handling Logic:
|
|
190
190
|
|
|
@@ -197,7 +197,7 @@ Display: "**Proceeding to scoring...**"
|
|
|
197
197
|
|
|
198
198
|
## CRITICAL STEP COMPLETION NOTE
|
|
199
199
|
|
|
200
|
-
ONLY WHEN coherence analysis is complete (naive structural or contextual full validation), the Coherence Analysis section has been appended to {outputFile}, and coherence scores (if contextual) have been calculated, will you then load and read fully `{nextStepFile}` to execute
|
|
200
|
+
ONLY WHEN coherence analysis is complete (naive structural or contextual full validation), the Coherence Analysis section has been appended to {outputFile}, and coherence scores (if contextual) have been calculated, will you then load and read fully `{nextStepFile}` to execute external validation.
|
|
201
201
|
|
|
202
202
|
---
|
|
203
203
|
|
|
@@ -56,6 +56,12 @@ Run external validation tools (`skill-check` and `tessl`) against the skill dire
|
|
|
56
56
|
|
|
57
57
|
Read {outputFile} frontmatter to get the skill directory path (`skillDir`).
|
|
58
58
|
|
|
59
|
+
### 1b. Check for Recent Validation Results (Auto-Reuse)
|
|
60
|
+
|
|
61
|
+
Before running external validators, check if `{forge_data_folder}/{skill_name}/evidence-report.md` contains validation results (a `## Validation Results` section with quality scores). If recent results exist (from a create-skill run that just completed), auto-reuse them — skip re-running validators and use the existing scores from the evidence report. Record: "External validation: reused from create-skill evidence report." Skip to section 5 (append results).
|
|
62
|
+
|
|
63
|
+
If no evidence report exists or it contains no validation section, proceed to section 2 (fresh run).
|
|
64
|
+
|
|
59
65
|
### 2. Run skill-check
|
|
60
66
|
|
|
61
67
|
**Check availability:**
|
|
@@ -5,6 +5,7 @@ description: 'Calculate completeness score from coverage and coherence findings'
|
|
|
5
5
|
nextStepFile: './step-06-report.md'
|
|
6
6
|
outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
|
|
7
7
|
scoringRulesFile: '../data/scoring-rules.md'
|
|
8
|
+
sourceAccessProtocol: '../data/{sourceAccessProtocol}'
|
|
8
9
|
---
|
|
9
10
|
|
|
10
11
|
# Step 5: Score
|
|
@@ -136,14 +137,14 @@ Append the **Completeness Score** section to `{outputFile}`:
|
|
|
136
137
|
**Analysis Confidence:** {full | provenance-map | metadata-only | remote-only | docs-only}
|
|
137
138
|
```
|
|
138
139
|
|
|
139
|
-
If `analysis_confidence` is not `full`, append a degradation notice
|
|
140
|
+
If `analysis_confidence` is not `full`, append a degradation notice. **The notice must be confidence-aware** — see the degradation notice rules in `{sourceAccessProtocol}`:
|
|
140
141
|
|
|
141
142
|
```markdown
|
|
142
143
|
### Access Degradation Notice
|
|
143
144
|
|
|
144
|
-
**Resolved via:** {analysis_confidence}
|
|
145
|
-
**Impact:** {describe limitation — e.g., "Signature checks limited to name-matching. Source file:line citations from provenance-map, not live AST."}
|
|
146
|
-
**Recommendation:**
|
|
145
|
+
**Resolved via:** {analysis_confidence} {confidence breakdown if provenance-map, e.g., "(T1 AST-verified at compilation time)" or "(12 T1, 3 T1-low)"}
|
|
146
|
+
**Impact:** {describe limitation — e.g., "Signature checks limited to name-matching. Source file:line citations from provenance-map, not live AST." — or "Provenance data is at highest confidence; no limitation." for all-T1 provenance-map}
|
|
147
|
+
**Recommendation:** {confidence-dependent — see {sourceAccessProtocol} degradation notice rules. Do NOT recommend local clone when provenance-map entries are already T1.}
|
|
147
148
|
```
|
|
148
149
|
|
|
149
150
|
### 7. Update Output Frontmatter
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Tier Degradation Rules
|
|
2
|
+
|
|
3
|
+
## Remote Source at Forge/Deep Tier
|
|
4
|
+
|
|
5
|
+
When `source_repo` is a remote URL (GitHub URL or owner/repo format) and the tier is Forge or Deep:
|
|
6
|
+
|
|
7
|
+
- **ast-grep requires local files** — it cannot operate on remote URLs
|
|
8
|
+
|
|
9
|
+
**Ephemeral clone strategy (preferred):**
|
|
10
|
+
|
|
11
|
+
1. Check `git` availability (`git --version`). `git` is effectively guaranteed at Deep tier (via `gh` dependency) but NOT guaranteed at Forge tier.
|
|
12
|
+
2. If `git` is available: perform an ephemeral shallow clone to a system temp path (`{system_temp}/skf-ephemeral-{skill-name}-{timestamp}/`).
|
|
13
|
+
3. For create-skill: use `--depth 1 --single-branch --filter=blob:none`; if `include_patterns` are specified, apply mode selection: if `exclude_patterns` are absent, use cone mode (convert include_patterns to directory roots; use `--skip-checks` for individual file paths); if `exclude_patterns` are present, use `--no-cone` mode (pass gitignore-style patterns with `!`-prefixed excludes — includes first, then negations). See source-resolution-protocols.md for the full conversion rules.
|
|
14
|
+
4. For update-skill: use sparse-checkout with `--skip-checks` scoped to the changed files from the change manifest only (file paths require `--skip-checks`). No `--branch` flag — uses the remote default branch (must match the branch used during original create-skill run).
|
|
15
|
+
5. If clone succeeds: use the local clone path for AST extraction. All results are T1 with `[AST:...]` citations.
|
|
16
|
+
6. Cleanup: delete the temp directory after extraction inventory is built and all data is in context. The clone never persists beyond the extraction step.
|
|
17
|
+
|
|
18
|
+
**Fallback (clone fails or `git` unavailable):**
|
|
19
|
+
|
|
20
|
+
- The extraction step MUST warn the user explicitly before degrading
|
|
21
|
+
- **create-skill:** Warning must include actionable guidance — clone locally and update `source_repo` in the brief to the local path
|
|
22
|
+
- **update-skill:** Warning must include actionable guidance — clone locally, re-run [CS] Create Skill with the local path to regenerate provenance data, then re-run the update
|
|
23
|
+
- Extraction proceeds using Quick tier strategy (source reading via gh_bridge)
|
|
24
|
+
- All results labeled T1-low with `[SRC:...]` citations
|
|
25
|
+
- The degradation reason is recorded in the evidence report
|
|
26
|
+
|
|
27
|
+
Silent degradation is **forbidden**. The user must always know when AST extraction was skipped and why.
|
|
28
|
+
|
|
29
|
+
## AST Tool Unavailable at Forge/Deep Tier
|
|
30
|
+
|
|
31
|
+
When the tier is Forge or Deep but ast-grep is not functional:
|
|
32
|
+
|
|
33
|
+
- The extraction step MUST warn the user explicitly before degrading
|
|
34
|
+
- Warning must include actionable guidance: run [SF] Setup Forge to detect tools
|
|
35
|
+
- Extraction proceeds using Quick tier strategy
|
|
36
|
+
- All results labeled T1-low
|
|
37
|
+
- The degradation reason is recorded in the evidence report
|
|
38
|
+
|
|
39
|
+
## Per-File AST Failure
|
|
40
|
+
|
|
41
|
+
When ast-grep fails on an individual file (parse error, unsupported syntax):
|
|
42
|
+
|
|
43
|
+
- Fall back to source reading for **that file only**
|
|
44
|
+
- Other files continue with AST extraction
|
|
45
|
+
- The affected file's results are labeled T1-low; unaffected files retain T1
|
|
46
|
+
- Log a warning noting which file degraded and why
|
|
@@ -4,6 +4,7 @@ description: 'Tier-aware AST extraction on changed files only, producing fresh e
|
|
|
4
4
|
|
|
5
5
|
nextStepFile: './step-04-merge.md'
|
|
6
6
|
extractionPatternsData: '../../create-skill/data/extraction-patterns.md'
|
|
7
|
+
tierDegradationRulesData: '../data/tier-degradation-rules.md'
|
|
7
8
|
remoteSourceResolutionData: '../data/remote-source-resolution.md'
|
|
8
9
|
---
|
|
9
10
|
|
package/tools/cli/lib/ui.js
CHANGED
|
@@ -61,7 +61,7 @@ class UI {
|
|
|
61
61
|
console.log(row(chalk.white.bold('Skill Forge') + chalk.dim(` v${version}`)));
|
|
62
62
|
console.log(row(chalk.dim('Agent Skill Compiler') + ' '.repeat(15) + brand.spark('⚒')));
|
|
63
63
|
console.log(mid);
|
|
64
|
-
console.log(row(chalk.dim('
|
|
64
|
+
console.log(row(chalk.dim('Code · Docs · Discourse → Verified agent skills')));
|
|
65
65
|
console.log(bottom);
|
|
66
66
|
console.log();
|
|
67
67
|
|