bmad-module-skill-forge 0.10.0 → 0.10.1-alpha.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +39 -0
- package/.gitattributes +16 -0
- package/README.md +105 -38
- package/docs/404.md +13 -10
- package/docs/RELEASING.md +185 -0
- package/docs/STABILITY.md +123 -0
- package/docs/_data/pinned.yaml +98 -0
- package/docs/agents.md +16 -11
- package/docs/architecture.md +221 -0
- package/docs/bmad-synergy.md +11 -13
- package/docs/concepts.md +18 -108
- package/docs/examples.md +121 -80
- package/docs/getting-started.md +85 -147
- package/docs/how-it-works.md +25 -536
- package/docs/index.md +44 -27
- package/docs/skill-model.md +328 -0
- package/docs/troubleshooting.md +39 -0
- package/docs/verifying-a-skill.md +232 -0
- package/docs/why-skf.md +93 -0
- package/docs/workflows.md +119 -13
- package/package.json +25 -4
- package/src/README.md +23 -25
- package/src/forger/forge-tier.yaml +4 -2
- package/src/forger/preferences.yaml +7 -1
- package/src/knowledge/agentskills-spec.md +1 -1
- package/src/knowledge/architecture-verification.md +1 -1
- package/src/knowledge/ccc-bridge.md +16 -15
- package/src/knowledge/overview.md +11 -11
- package/src/knowledge/progressive-capability.md +3 -3
- package/src/knowledge/provenance-tracking.md +9 -3
- package/src/knowledge/qmd-registry.md +8 -8
- package/src/knowledge/skf-knowledge-index.csv +2 -2
- package/src/knowledge/skill-lifecycle.md +16 -1
- package/src/knowledge/split-body-strategy.md +1 -1
- package/src/knowledge/version-paths.md +17 -12
- package/src/module-help.csv +18 -17
- package/src/shared/health-check.md +372 -0
- package/src/shared/references/feasibility-report-schema.md +86 -0
- package/src/shared/references/headless-gate-convention.md +62 -0
- package/src/shared/references/output-contract-schema.md +35 -0
- package/src/shared/references/pipeline-contracts.md +102 -0
- package/src/shared/scripts/skf-atomic-write.py +370 -0
- package/src/shared/scripts/skf-manifest-ops.py +236 -0
- package/src/shared/scripts/skf-preflight.py +164 -0
- package/src/shared/scripts/skf-rebuild-managed-sections.py +201 -0
- package/src/shared/scripts/skf-severity-classify.py +163 -0
- package/src/shared/scripts/skf-skill-inventory.py +210 -0
- package/src/shared/scripts/skf-structural-diff.py +257 -0
- package/src/shared/scripts/skf-validate-frontmatter.py +318 -0
- package/src/shared/scripts/skf-validate-output.py +247 -0
- package/src/skf-analyze-source/SKILL.md +57 -0
- package/src/skf-analyze-source/assets/skill-brief-schema.md +125 -0
- package/src/skf-analyze-source/references/unit-detection-heuristics.md +124 -0
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-01-init.md +6 -66
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-01b-continue.md +4 -59
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-02-scan-project.md +7 -68
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-03-identify-units.md +6 -66
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-04-map-and-detect.md +7 -68
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-05-recommend.md +6 -66
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-06-generate-briefs.md +14 -71
- package/src/skf-analyze-source/steps-c/step-07-health-check.md +22 -0
- package/src/skf-audit-skill/SKILL.md +59 -0
- package/src/{workflows/audit-skill/data → skf-audit-skill/assets}/drift-report-template.md +7 -7
- package/src/{workflows/audit-skill/data → skf-audit-skill/references}/severity-rules.md +11 -11
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-01-init.md +24 -68
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-02-re-index.md +36 -71
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-03-structural-diff.md +21 -63
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-04-semantic-diff.md +6 -65
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-05-severity-classify.md +5 -63
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-06-report.md +10 -64
- package/src/skf-audit-skill/steps-c/step-07-health-check.md +22 -0
- package/src/skf-brief-skill/SKILL.md +53 -0
- package/src/{workflows/brief-skill/data → skf-brief-skill/assets}/scope-templates.md +23 -0
- package/src/skf-brief-skill/assets/skill-brief-schema.md +206 -0
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-01-gather-intent.md +6 -67
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-02-analyze-target.md +6 -66
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-03-scope-definition.md +12 -68
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-04-confirm-brief.md +5 -63
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-05-write-brief.md +10 -70
- package/src/skf-brief-skill/steps-c/step-06-health-check.md +22 -0
- package/src/skf-create-skill/SKILL.md +63 -0
- package/src/{workflows/create-skill/data → skf-create-skill/assets}/compile-assembly-rules.md +60 -0
- package/src/{workflows/create-skill/data → skf-create-skill/assets}/skill-sections.md +92 -23
- package/src/skf-create-skill/assets/tessl-dismissal-rules.md +102 -0
- package/src/{workflows/create-skill/data → skf-create-skill/references}/extraction-patterns-tracing.md +13 -13
- package/src/{workflows/create-skill/data → skf-create-skill/references}/extraction-patterns.md +8 -3
- package/src/skf-create-skill/references/source-resolution-protocols.md +239 -0
- package/src/{workflows/create-skill/data → skf-create-skill/references}/tier-degradation-rules.md +8 -7
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-01-load-brief.md +25 -63
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-02-ecosystem-check.md +8 -64
- package/src/skf-create-skill/steps-c/step-03-extract.md +379 -0
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-03d-component-extraction.md +18 -35
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-04-enrich.md +12 -69
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-05-compile.md +71 -75
- package/src/skf-create-skill/steps-c/step-06-validate.md +267 -0
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-07-generate-artifacts.md +30 -74
- package/src/skf-create-skill/steps-c/step-08-report.md +145 -0
- package/src/skf-create-skill/steps-c/step-09-health-check.md +23 -0
- package/src/{workflows/create-skill/steps-c → skf-create-skill/steps-c/sub}/step-02b-ccc-discover.md +8 -66
- package/src/skf-create-skill/steps-c/sub/step-03b-fetch-temporal.md +229 -0
- package/src/{workflows/create-skill/steps-c → skf-create-skill/steps-c/sub}/step-03c-fetch-docs.md +18 -42
- package/src/skf-create-stack-skill/SKILL.md +64 -0
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/assets}/stack-skill-template.md +2 -1
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/references}/compose-mode-rules.md +19 -5
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/references}/integration-patterns.md +1 -1
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/references}/manifest-patterns.md +10 -10
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-01-init.md +14 -55
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-02-detect-manifests.md +29 -63
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-03-rank-and-confirm.md +9 -59
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-04-parallel-extract.md +29 -64
- package/src/skf-create-stack-skill/steps-c/step-05-detect-integrations.md +164 -0
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-06-compile-stack.md +7 -60
- package/src/skf-create-stack-skill/steps-c/step-07-generate-output.md +328 -0
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-08-validate.md +17 -63
- package/src/skf-create-stack-skill/steps-c/step-09-report.md +128 -0
- package/src/skf-create-stack-skill/steps-c/step-10-health-check.md +22 -0
- package/src/skf-drop-skill/SKILL.md +57 -0
- package/src/{workflows/drop-skill → skf-drop-skill}/steps-c/step-01-select.md +9 -69
- package/src/{workflows/drop-skill → skf-drop-skill}/steps-c/step-02-execute.md +30 -106
- package/src/skf-drop-skill/steps-c/step-03-report.md +83 -0
- package/src/skf-drop-skill/steps-c/step-04-health-check.md +22 -0
- package/src/skf-export-skill/SKILL.md +57 -0
- package/src/skf-export-skill/assets/managed-section-format.md +138 -0
- package/src/{workflows/export-skill/data → skf-export-skill/assets}/snippet-format.md +8 -12
- package/src/skf-export-skill/steps-c/step-01-load-skill.md +247 -0
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-02-package.md +5 -58
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-03-generate-snippet.md +15 -77
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-04-update-context.md +59 -133
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-05-token-report.md +4 -58
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-06-summary.md +16 -68
- package/src/skf-export-skill/steps-c/step-07-health-check.md +22 -0
- package/src/skf-forger/SKILL.md +110 -0
- package/src/skf-forger/bmad-skill-manifest.yaml +11 -0
- package/src/skf-quick-skill/SKILL.md +56 -0
- package/src/{workflows/quick-skill/data → skf-quick-skill/references}/registry-resolution.md +6 -6
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-01-resolve-target.md +9 -62
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-02-ecosystem-check.md +5 -59
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-03-quick-extract.md +18 -61
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-04-compile.md +6 -60
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-05-validate.md +58 -78
- package/src/skf-quick-skill/steps-c/step-06-write.md +73 -0
- package/src/skf-quick-skill/steps-c/step-07-health-check.md +22 -0
- package/src/skf-refine-architecture/SKILL.md +57 -0
- package/src/{workflows/refine-architecture/data → skf-refine-architecture/references}/refinement-rules.md +16 -16
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-01-init.md +6 -63
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-02-gap-analysis.md +29 -71
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-03-issue-detection.md +5 -65
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-04-improvements.md +5 -64
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-05-compile.md +5 -56
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-06-report.md +11 -64
- package/src/skf-refine-architecture/steps-c/step-07-health-check.md +22 -0
- package/src/skf-rename-skill/SKILL.md +58 -0
- package/src/{workflows/rename-skill → skf-rename-skill}/steps-c/step-01-select.md +12 -73
- package/src/{workflows/rename-skill → skf-rename-skill}/steps-c/step-02-execute.md +32 -113
- package/src/skf-rename-skill/steps-c/step-03-report.md +83 -0
- package/src/skf-rename-skill/steps-c/step-04-health-check.md +22 -0
- package/src/skf-setup/SKILL.md +55 -0
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-01-detect-and-tier.md +6 -62
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-01b-ccc-index.md +9 -65
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-02-write-config.md +15 -60
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-03-auto-index.md +7 -66
- package/src/skf-setup/steps-c/step-04-report.md +92 -0
- package/src/skf-setup/steps-c/step-05-health-check.md +22 -0
- package/src/skf-test-skill/SKILL.md +59 -0
- package/src/skf-test-skill/references/migration-section-rules.md +114 -0
- package/src/skf-test-skill/references/scoring-rules.md +207 -0
- package/src/skf-test-skill/references/source-access-protocol.md +109 -0
- package/src/skf-test-skill/scripts/compute-score.py +310 -0
- package/src/skf-test-skill/steps-c/step-01-init.md +238 -0
- package/src/{workflows/test-skill → skf-test-skill}/steps-c/step-02-detect-mode.md +4 -62
- package/src/skf-test-skill/steps-c/step-03-coverage-check.md +347 -0
- package/src/skf-test-skill/steps-c/step-04-coherence-check.md +239 -0
- package/src/{workflows/test-skill → skf-test-skill}/steps-c/step-04b-external-validators.md +42 -77
- package/src/skf-test-skill/steps-c/step-05-score.md +266 -0
- package/src/skf-test-skill/steps-c/step-06-report.md +290 -0
- package/src/skf-test-skill/steps-c/step-07-health-check.md +25 -0
- package/src/skf-test-skill/templates/test-report-template.md +58 -0
- package/src/skf-update-skill/SKILL.md +58 -0
- package/src/{workflows/update-skill/data → skf-update-skill/references}/manual-section-rules.md +6 -6
- package/src/{workflows/update-skill/data → skf-update-skill/references}/merge-conflict-rules.md +5 -5
- package/src/skf-update-skill/references/remote-source-resolution.md +94 -0
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-01-init.md +22 -69
- package/src/skf-update-skill/steps-c/step-02-detect-changes.md +255 -0
- package/src/skf-update-skill/steps-c/step-03-re-extract.md +336 -0
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-04-merge.md +60 -72
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-05-validate.md +5 -62
- package/src/skf-update-skill/steps-c/step-06-write.md +278 -0
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-07-report.md +11 -69
- package/src/skf-update-skill/steps-c/step-08-health-check.md +22 -0
- package/src/skf-verify-stack/SKILL.md +60 -0
- package/src/skf-verify-stack/assets/feasibility-report-template.md +76 -0
- package/src/{workflows/verify-stack/data → skf-verify-stack/references}/coverage-patterns.md +12 -4
- package/src/skf-verify-stack/references/integration-verification-rules.md +78 -0
- package/src/skf-verify-stack/steps-c/step-01-init.md +178 -0
- package/src/{workflows/verify-stack → skf-verify-stack}/steps-c/step-02-coverage.md +27 -72
- package/src/skf-verify-stack/steps-c/step-03-integrations.md +179 -0
- package/src/{workflows/verify-stack → skf-verify-stack}/steps-c/step-04-requirements.md +14 -67
- package/src/skf-verify-stack/steps-c/step-05-synthesize.md +147 -0
- package/src/skf-verify-stack/steps-c/step-06-report.md +156 -0
- package/src/skf-verify-stack/steps-c/step-07-health-check.md +25 -0
- package/tools/cli/commands/install.js +1 -1
- package/tools/cli/commands/status.js +21 -23
- package/tools/cli/commands/uninstall.js +11 -19
- package/tools/cli/commands/update.js +2 -2
- package/tools/cli/lib/ide-skills.js +216 -0
- package/tools/cli/lib/installer.js +29 -52
- package/tools/cli/lib/manifest.js +11 -5
- package/tools/cli/lib/platform-codes.yaml +223 -0
- package/tools/cli/lib/ui.js +76 -47
- package/tools/validate-docs-drift.js +193 -0
- package/src/agents/forger.agent.yaml +0 -114
- package/src/workflows/README.md +0 -174
- package/src/workflows/analyze-source/data/skill-brief-schema.md +0 -125
- package/src/workflows/analyze-source/data/unit-detection-heuristics.md +0 -124
- package/src/workflows/analyze-source/workflow.md +0 -61
- package/src/workflows/audit-skill/workflow.md +0 -64
- package/src/workflows/brief-skill/data/skill-brief-schema.md +0 -159
- package/src/workflows/brief-skill/workflow.md +0 -58
- package/src/workflows/create-skill/data/source-resolution-protocols.md +0 -200
- package/src/workflows/create-skill/steps-c/step-03-extract.md +0 -301
- package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +0 -253
- package/src/workflows/create-skill/steps-c/step-06-validate.md +0 -254
- package/src/workflows/create-skill/steps-c/step-08-report.md +0 -179
- package/src/workflows/create-skill/workflow.md +0 -53
- package/src/workflows/create-stack-skill/steps-c/step-05-detect-integrations.md +0 -196
- package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +0 -270
- package/src/workflows/create-stack-skill/steps-c/step-09-report.md +0 -150
- package/src/workflows/create-stack-skill/workflow.md +0 -59
- package/src/workflows/drop-skill/steps-c/step-03-report.md +0 -136
- package/src/workflows/drop-skill/workflow.md +0 -63
- package/src/workflows/export-skill/data/managed-section-format.md +0 -103
- package/src/workflows/export-skill/steps-c/step-01-load-skill.md +0 -237
- package/src/workflows/export-skill/workflow.md +0 -58
- package/src/workflows/quick-skill/steps-c/step-06-write.md +0 -178
- package/src/workflows/quick-skill/workflow.md +0 -58
- package/src/workflows/refine-architecture/workflow.md +0 -61
- package/src/workflows/rename-skill/steps-c/step-03-report.md +0 -133
- package/src/workflows/rename-skill/workflow.md +0 -64
- package/src/workflows/setup-forge/steps-c/step-04-report.md +0 -159
- package/src/workflows/setup-forge/workflow.md +0 -51
- package/src/workflows/shared/health-check.md +0 -231
- package/src/workflows/test-skill/data/scoring-rules.md +0 -123
- package/src/workflows/test-skill/data/source-access-protocol.md +0 -51
- package/src/workflows/test-skill/steps-c/step-01-init.md +0 -217
- package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +0 -280
- package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +0 -271
- package/src/workflows/test-skill/steps-c/step-05-score.md +0 -222
- package/src/workflows/test-skill/steps-c/step-06-report.md +0 -208
- package/src/workflows/test-skill/templates/test-report-template.md +0 -28
- package/src/workflows/test-skill/workflow.md +0 -57
- package/src/workflows/update-skill/data/remote-source-resolution.md +0 -57
- package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +0 -221
- package/src/workflows/update-skill/steps-c/step-03-re-extract.md +0 -256
- package/src/workflows/update-skill/steps-c/step-06-write.md +0 -265
- package/src/workflows/update-skill/workflow.md +0 -56
- package/src/workflows/verify-stack/data/feasibility-report-template.md +0 -63
- package/src/workflows/verify-stack/data/integration-verification-rules.md +0 -73
- package/src/workflows/verify-stack/steps-c/step-01-init.md +0 -179
- package/src/workflows/verify-stack/steps-c/step-03-integrations.md +0 -195
- package/src/workflows/verify-stack/steps-c/step-05-synthesize.md +0 -191
- package/src/workflows/verify-stack/steps-c/step-06-report.md +0 -200
- package/src/workflows/verify-stack/workflow.md +0 -61
- package/tools/cli/lib/compiler.js +0 -306
- package/tools/cli/lib/ide-commands.js +0 -244
- /package/src/{workflows/analyze-source → skf-analyze-source}/templates/analysis-report-template.md +0 -0
- /package/src/{workflows/quick-skill/data → skf-quick-skill/assets}/skill-template.md +0 -0
- /package/src/{workflows/setup-forge/data → skf-setup/references}/tier-rules.md +0 -0
- /package/src/{workflows/test-skill/data → skf-test-skill/assets}/output-section-formats.md +0 -0
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: setup-forge
|
|
3
|
-
description: Initialize forge environment, detect tools, set tier, verify QMD collection hygiene
|
|
4
|
-
web_bundle: true
|
|
5
|
-
installed_path: '{project-root}/_bmad/skf/workflows/skillforge/setup-forge'
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Setup Forge
|
|
9
|
-
|
|
10
|
-
**Goal:** Initialize the forge environment by detecting available tools, determining the capability tier (Quick/Forge/Forge+/Deep), writing persistent configuration, and optionally indexing the project for deep search.
|
|
11
|
-
|
|
12
|
-
**Your Role:** You are a system executor performing environment resolution. This is a fully autonomous workflow — no user interaction is required during execution. Run each step in sequence, write configuration files, and report results at completion.
|
|
13
|
-
|
|
14
|
-
## WORKFLOW ARCHITECTURE
|
|
15
|
-
|
|
16
|
-
### Core Principles
|
|
17
|
-
|
|
18
|
-
- **Micro-file Design**: Each step of the overall goal is a self contained instruction file that you will adhere to 1 file as directed at a time
|
|
19
|
-
- **Just-In-Time Loading**: Only 1 current step file will be loaded, read, and executed to completion - never load future step files until told to do so
|
|
20
|
-
- **Sequential Enforcement**: Sequence within the step files must be completed in order, no skipping or optimization allowed
|
|
21
|
-
- **Fully Autonomous**: All steps auto-proceed with no user interaction until the final report
|
|
22
|
-
|
|
23
|
-
### Step Processing Rules
|
|
24
|
-
|
|
25
|
-
1. **READ COMPLETELY**: Always read the entire step file before taking any action
|
|
26
|
-
2. **FOLLOW SEQUENCE**: Execute all numbered sections in order, never deviate
|
|
27
|
-
3. **AUTO-PROCEED**: Each step proceeds to the next automatically after completion
|
|
28
|
-
4. **LOAD NEXT**: When directed, load, read entire file, then execute the next step file
|
|
29
|
-
|
|
30
|
-
### Critical Rules (NO EXCEPTIONS)
|
|
31
|
-
|
|
32
|
-
- 🛑 **NEVER** load multiple step files simultaneously
|
|
33
|
-
- 📖 **ALWAYS** read entire step file before execution
|
|
34
|
-
- 🚫 **NEVER** skip steps or optimize the sequence
|
|
35
|
-
- 🎯 **ALWAYS** follow the exact instructions in the step file
|
|
36
|
-
- 📋 **NEVER** create mental todo lists from future steps
|
|
37
|
-
|
|
38
|
-
---
|
|
39
|
-
|
|
40
|
-
## INITIALIZATION SEQUENCE
|
|
41
|
-
|
|
42
|
-
### 1. Module Configuration Loading
|
|
43
|
-
|
|
44
|
-
Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
|
|
45
|
-
|
|
46
|
-
- `project_name`, `output_folder`, `user_name`, `communication_language`, `document_output_language`
|
|
47
|
-
- `skills_output_folder`, `forge_data_folder`, `sidecar_path`
|
|
48
|
-
|
|
49
|
-
### 2. First Step Execution
|
|
50
|
-
|
|
51
|
-
Load, read the full file and then execute ./steps-c/step-01-detect-and-tier.md to begin the workflow.
|
|
@@ -1,231 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: 'health-check'
|
|
3
|
-
description: 'Workflow self-improvement health check — captures real friction as GitHub issues'
|
|
4
|
-
# No nextStepFile — this is always the terminal step
|
|
5
|
-
healthCheckRepo: '{health_check_repo}'
|
|
6
|
-
localFallbackFolder: '{output_folder}/improvement-queue'
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
# Health Check: Workflow Self-Improvement
|
|
10
|
-
|
|
11
|
-
## STEP GOAL:
|
|
12
|
-
|
|
13
|
-
Reflect on the workflow that just completed. If real friction, bugs, or gaps were encountered in the SKF workflow instructions, capture them as structured findings for the user to review and optionally submit as GitHub issues.
|
|
14
|
-
|
|
15
|
-
**Zero overhead for clean runs.** If nothing went wrong, say so and exit immediately.
|
|
16
|
-
|
|
17
|
-
## MANDATORY EXECUTION RULES:
|
|
18
|
-
|
|
19
|
-
### Universal Rules:
|
|
20
|
-
- Read the complete step file before taking any action
|
|
21
|
-
- Speak in `{communication_language}`
|
|
22
|
-
|
|
23
|
-
### Role Reinforcement:
|
|
24
|
-
- You are a **self-improvement auditor** — honest, precise, evidence-based
|
|
25
|
-
- You report ONLY what you actually experienced during THIS session
|
|
26
|
-
- You are NOT a creative writer looking for things to say
|
|
27
|
-
|
|
28
|
-
### Anti-Hallucination Rules:
|
|
29
|
-
- **DO NOT FABRICATE ISSUES.** If the workflow ran smoothly, say so and exit. Inventing issues to appear thorough is a SYSTEM FAILURE.
|
|
30
|
-
- Only report issues you **ACTUALLY encountered** during THIS workflow execution
|
|
31
|
-
- Every finding MUST cite the **specific step file path and section** where the issue occurred
|
|
32
|
-
- If you are unsure whether something was a real issue or your own confusion, DO NOT report it
|
|
33
|
-
- Reporting zero issues is the EXPECTED outcome for a well-designed workflow
|
|
34
|
-
|
|
35
|
-
---
|
|
36
|
-
|
|
37
|
-
## MANDATORY SEQUENCE
|
|
38
|
-
|
|
39
|
-
### 1. Read Workflow Context
|
|
40
|
-
|
|
41
|
-
From the current session context, identify:
|
|
42
|
-
- **Workflow name** — which workflow just completed
|
|
43
|
-
- **Steps executed** — which step files were loaded and followed
|
|
44
|
-
- **Any friction points** — moments where instructions were unclear, wrong, contradictory, or missing
|
|
45
|
-
|
|
46
|
-
### 2. Reflect on Execution
|
|
47
|
-
|
|
48
|
-
Silently review the workflow execution. Ask yourself:
|
|
49
|
-
|
|
50
|
-
- Did any step instruction lead me astray or cause unnecessary back-and-forth with the user?
|
|
51
|
-
- Was any step ambiguous, causing me to guess rather than follow clear guidance?
|
|
52
|
-
- Did I encounter a scenario the workflow didn't account for?
|
|
53
|
-
- Were any step instructions wrong or contradictory?
|
|
54
|
-
|
|
55
|
-
**If the answer to ALL of these is "no":**
|
|
56
|
-
|
|
57
|
-
Display:
|
|
58
|
-
|
|
59
|
-
"**Health Check: Clean run.** No workflow issues to report.
|
|
60
|
-
|
|
61
|
-
Workflow complete."
|
|
62
|
-
|
|
63
|
-
**STOP HERE. Do not proceed further. The workflow is done.**
|
|
64
|
-
|
|
65
|
-
### 3. Present Findings (Only If Issues Exist)
|
|
66
|
-
|
|
67
|
-
For each genuine finding, present it in this format:
|
|
68
|
-
|
|
69
|
-
"**Workflow Health Check — {N} finding(s)**
|
|
70
|
-
|
|
71
|
-
---
|
|
72
|
-
|
|
73
|
-
**Finding {i}:**
|
|
74
|
-
|
|
75
|
-
| Field | Value |
|
|
76
|
-
|-------|-------|
|
|
77
|
-
| **Severity** | `bug` / `friction` / `gap` |
|
|
78
|
-
| **Workflow** | {workflow name} |
|
|
79
|
-
| **Step File** | `src/workflows/{workflow}/{step-file-path}` |
|
|
80
|
-
| **Section** | {the specific section or instruction number} |
|
|
81
|
-
|
|
82
|
-
**What happened:**
|
|
83
|
-
{Description of the actual issue encountered during execution}
|
|
84
|
-
|
|
85
|
-
**Evidence:**
|
|
86
|
-
{What specifically went wrong — error, confusion, user friction, missing guidance}
|
|
87
|
-
|
|
88
|
-
**Suggested improvement:**
|
|
89
|
-
{Concrete, actionable change to the step file or data}
|
|
90
|
-
|
|
91
|
-
---"
|
|
92
|
-
|
|
93
|
-
**Severity definitions:**
|
|
94
|
-
- **`bug`** — Step instructions were wrong or contradictory
|
|
95
|
-
- **`friction`** — Step worked but was unclear, ambiguous, or caused unnecessary back-and-forth
|
|
96
|
-
- **`gap`** — A scenario arose that the workflow didn't account for at all
|
|
97
|
-
|
|
98
|
-
### 4. User Review Gate
|
|
99
|
-
|
|
100
|
-
After presenting all findings, ask:
|
|
101
|
-
|
|
102
|
-
"**Submit these findings?**
|
|
103
|
-
|
|
104
|
-
- **[Y]** Yes — submit all findings
|
|
105
|
-
- **[N]** No — discard all findings
|
|
106
|
-
- **[E]** Edit — let me revise before submitting
|
|
107
|
-
|
|
108
|
-
_You are the final filter. Reject any finding that doesn't reflect a real issue you observed._"
|
|
109
|
-
|
|
110
|
-
**HALT and wait for user input.**
|
|
111
|
-
|
|
112
|
-
#### Menu Handling:
|
|
113
|
-
|
|
114
|
-
- **IF Y:** Proceed to step 5
|
|
115
|
-
- **IF N:** Display "Findings discarded. Workflow complete." — STOP
|
|
116
|
-
- **IF E:** Let user specify which findings to keep, modify, or remove. Then re-present the revised list and ask again.
|
|
117
|
-
|
|
118
|
-
### 5. Submit Findings
|
|
119
|
-
|
|
120
|
-
**Check GitHub CLI availability:**
|
|
121
|
-
|
|
122
|
-
Run `gh auth status` to determine if the `gh` CLI is authenticated.
|
|
123
|
-
|
|
124
|
-
#### IF `gh` is available:
|
|
125
|
-
|
|
126
|
-
For each confirmed finding, create a GitHub issue:
|
|
127
|
-
|
|
128
|
-
**Command:**
|
|
129
|
-
```
|
|
130
|
-
gh issue create \
|
|
131
|
-
--repo {healthCheckRepo} \
|
|
132
|
-
--title "[health-check] [{severity}] {workflow}: {short description}" \
|
|
133
|
-
--label "health-check,workflow-improvement,{severity}" \
|
|
134
|
-
--body "{formatted body using issue template structure}"
|
|
135
|
-
```
|
|
136
|
-
|
|
137
|
-
**Issue body format:**
|
|
138
|
-
```markdown
|
|
139
|
-
## Workflow
|
|
140
|
-
{workflow name}
|
|
141
|
-
|
|
142
|
-
## Step File
|
|
143
|
-
`{exact step file path relative to src/}`
|
|
144
|
-
|
|
145
|
-
## Severity
|
|
146
|
-
{bug | friction | gap}
|
|
147
|
-
|
|
148
|
-
## Finding
|
|
149
|
-
{description of what went wrong}
|
|
150
|
-
|
|
151
|
-
## Evidence
|
|
152
|
-
{what actually happened during execution}
|
|
153
|
-
|
|
154
|
-
## Suggested Improvement
|
|
155
|
-
{concrete change to improve the workflow}
|
|
156
|
-
|
|
157
|
-
## Environment
|
|
158
|
-
- **Date:** {date}
|
|
159
|
-
- **OS:** {detected OS, e.g. macOS 15.2, Ubuntu 24.04, Windows 11}
|
|
160
|
-
- **AI Editor:** {the AI editor or CLI being used, e.g. Claude Code, Cursor, Windsurf}
|
|
161
|
-
- **Model:** {the model executing this workflow, e.g. Claude Opus 4.6, Claude Sonnet 4.6}
|
|
162
|
-
- **Forge Tier:** {if available from sidecar, otherwise "N/A"}
|
|
163
|
-
- **SKF Version:** {read from {project-root}/_bmad/skf/VERSION, otherwise "N/A"}
|
|
164
|
-
```
|
|
165
|
-
|
|
166
|
-
After creating all issues, display:
|
|
167
|
-
|
|
168
|
-
"**{N} issue(s) created on {healthCheckRepo}:**
|
|
169
|
-
{list each issue URL}
|
|
170
|
-
|
|
171
|
-
Workflow complete."
|
|
172
|
-
|
|
173
|
-
#### IF `gh` is NOT available:
|
|
174
|
-
|
|
175
|
-
For each confirmed finding, write a local file to `{localFallbackFolder}/`:
|
|
176
|
-
|
|
177
|
-
**Filename:** `hc-{workflow}-{timestamp}.md` (one file per finding, timestamp as YYYYMMDD-HHmmss)
|
|
178
|
-
|
|
179
|
-
**File content:** Same structured format as the issue body above, with YAML frontmatter:
|
|
180
|
-
|
|
181
|
-
```yaml
|
|
182
|
-
---
|
|
183
|
-
type: workflow-health-finding
|
|
184
|
-
workflow: {workflow name}
|
|
185
|
-
step_file: {step file path}
|
|
186
|
-
severity: {bug | friction | gap}
|
|
187
|
-
date: {ISO date}
|
|
188
|
-
---
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
After writing all files, display:
|
|
192
|
-
|
|
193
|
-
"**{N} finding(s) saved locally:**
|
|
194
|
-
{list each file path}
|
|
195
|
-
|
|
196
|
-
GitHub CLI is not available. To submit these as issues, run:
|
|
197
|
-
`gh issue create --repo {healthCheckRepo} --title \"[title]\" --body-file {file-path}`
|
|
198
|
-
|
|
199
|
-
Or open them manually at: <https://github.com/{healthCheckRepo}/issues/new/choose>
|
|
200
|
-
|
|
201
|
-
Workflow complete."
|
|
202
|
-
|
|
203
|
-
---
|
|
204
|
-
|
|
205
|
-
## CRITICAL STEP COMPLETION NOTE
|
|
206
|
-
|
|
207
|
-
This is the TERMINAL step — shared across all SKF workflows. After the health check completes (clean run or findings submitted/discarded), the workflow is fully done. No further steps to load.
|
|
208
|
-
|
|
209
|
-
---
|
|
210
|
-
|
|
211
|
-
## SYSTEM SUCCESS/FAILURE METRICS
|
|
212
|
-
|
|
213
|
-
### SUCCESS:
|
|
214
|
-
|
|
215
|
-
- Clean runs exit immediately with no findings (most common outcome)
|
|
216
|
-
- Findings cite specific step files and sections with real evidence
|
|
217
|
-
- User review gate presented before any submission
|
|
218
|
-
- GitHub issues created with correct labels and structured body (when `gh` available)
|
|
219
|
-
- Local fallback files written with clear manual submission instructions (when `gh` unavailable)
|
|
220
|
-
- Workflow ends cleanly
|
|
221
|
-
|
|
222
|
-
### SYSTEM FAILURE:
|
|
223
|
-
|
|
224
|
-
- Fabricating issues that were not actually encountered during the session
|
|
225
|
-
- Reporting vague issues without step file citations ("the workflow was confusing")
|
|
226
|
-
- Skipping the user review gate
|
|
227
|
-
- Creating issues without user confirmation
|
|
228
|
-
- Not providing the local fallback when `gh` is unavailable
|
|
229
|
-
- Continuing to load steps after this one (this is terminal)
|
|
230
|
-
|
|
231
|
-
**Master Rule:** Honesty is the only policy. Zero findings is the expected, healthy outcome. Fabricating issues to appear thorough undermines the entire self-improvement system and constitutes SYSTEM FAILURE.
|
|
@@ -1,123 +0,0 @@
|
|
|
1
|
-
# Scoring Rules
|
|
2
|
-
|
|
3
|
-
## Default Threshold
|
|
4
|
-
|
|
5
|
-
**Pass threshold:** 80%
|
|
6
|
-
|
|
7
|
-
## Category Weights
|
|
8
|
-
|
|
9
|
-
| Category | Weight | Description |
|
|
10
|
-
|----------|--------|-------------|
|
|
11
|
-
| Export Coverage | 36% | Percentage of source exports documented in SKILL.md |
|
|
12
|
-
| Signature Accuracy | 22% | Documented signatures match actual source signatures |
|
|
13
|
-
| Type Coverage | 14% | Types and interfaces referenced are complete |
|
|
14
|
-
| Coherence (contextual) | 18% | Cross-references valid, integration patterns complete |
|
|
15
|
-
| Coherence (naive) | 0% | Not applicable — weight redistributed to other categories |
|
|
16
|
-
| External Validation | 10% | Average of skill-check quality score + tessl average score (redistributed if unavailable) |
|
|
17
|
-
|
|
18
|
-
## Naive Mode Weight Redistribution
|
|
19
|
-
|
|
20
|
-
The following weights replace the default table for naive mode. The 18% coherence weight from the default table has been proportionally redistributed into these values. Do not apply a second redistribution.
|
|
21
|
-
|
|
22
|
-
When running in naive mode (no coherence category):
|
|
23
|
-
- Export Coverage: 45%
|
|
24
|
-
- Signature Accuracy: 25%
|
|
25
|
-
- Type Coverage: 20%
|
|
26
|
-
- External Validation: 10%
|
|
27
|
-
|
|
28
|
-
## External Validation Unavailable
|
|
29
|
-
|
|
30
|
-
When neither skill-check nor tessl is available, redistribute the 10% external validation weight proportionally to the other active categories. When only one tool is available, use that tool's score as the external validation score.
|
|
31
|
-
|
|
32
|
-
## tessl and Split-Body Interaction
|
|
33
|
-
|
|
34
|
-
tessl evaluates SKILL.md body content only — it does not read `references/*.md` files. After split-body extraction, the tessl content score will drop significantly (e.g., 65% to 38%) because Tier 2 content is no longer inline. This is expected behavior and does not reflect actual content quality. When reporting scores for a split-body skill, note: "tessl content score reflects post-split inline content only. Use the pre-split tessl score as the content quality baseline."
|
|
35
|
-
|
|
36
|
-
## Tier-Dependent Scoring
|
|
37
|
-
|
|
38
|
-
### Quick Tier (no tools)
|
|
39
|
-
- Export Coverage: file/structure existence check only
|
|
40
|
-
- Signature Accuracy: skipped (no AST)
|
|
41
|
-
- Type Coverage: skipped (no AST)
|
|
42
|
-
- Score based on: structural completeness only
|
|
43
|
-
- Weight redistribution: skipped categories' weights (Signature Accuracy 22% + Type Coverage 14%) redistributed proportionally to remaining active categories
|
|
44
|
-
|
|
45
|
-
### Docs-Only Mode (Quick tier, all [EXT:...] citations)
|
|
46
|
-
|
|
47
|
-
When `docs_only_mode: true` is set by step-03 (indicating a Quick tier skill where all SKILL.md citations are `[EXT:...]` format with no local source code):
|
|
48
|
-
|
|
49
|
-
- **Signature Accuracy:** Not scored (no source to compare against)
|
|
50
|
-
- **Type Coverage:** Not scored (no source to compare against)
|
|
51
|
-
- **Weight redistribution:** Same as Quick tier — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories
|
|
52
|
-
- **Export Coverage basis:** Documentation completeness rather than source coverage. Score = (documented_items_with_complete_descriptions / total_documented_items) * 100. A "complete" item has: description, parameters (if function/method), and return type (if function/method).
|
|
53
|
-
- **Coherence:** Standard rules for the detected mode (naive or contextual) apply unchanged
|
|
54
|
-
|
|
55
|
-
This is functionally identical to Quick tier weight redistribution but with a different coverage denominator (self-consistency instead of source comparison).
|
|
56
|
-
|
|
57
|
-
### State 2 Source Access (Any Tier, Provenance-Map Only)
|
|
58
|
-
|
|
59
|
-
When source is not locally available and analysis resolves to State 2 (provenance-map baseline per source-access-protocol.md):
|
|
60
|
-
|
|
61
|
-
- **Signature Accuracy:** N/A — provenance-map stores parameters as flat string arrays; verification is string comparison only, not semantic AST verification. Type aliases (`str` vs `String`, `list` vs `List[Any]`) cannot be resolved without live source.
|
|
62
|
-
- **Type Coverage:** N/A — cannot verify type completeness without local source access for AST re-parsing.
|
|
63
|
-
- **Weight redistribution:** Same as Quick tier — Signature Accuracy (22%) and Type Coverage (14%) weights redistributed proportionally to remaining active categories (Export Coverage, Coherence, External Validation).
|
|
64
|
-
- **Applies regardless of detected tier** (including Forge, Forge+, Deep) whenever `analysis_confidence` is `provenance-map` and local source is unavailable.
|
|
65
|
-
- **Export Coverage denominator:** Uses the union of provenance-map entry names and metadata.json `exports[]` names (per source-access-protocol.md State 2 rules).
|
|
66
|
-
|
|
67
|
-
Note: When provenance-map entries are predominantly T1 (AST-verified at compilation time), the coverage and name-matching data is already at highest confidence. The N/A categories reflect the inability to re-verify at test time, not low-quality extraction data.
|
|
68
|
-
|
|
69
|
-
### Forge Tier (ast-grep)
|
|
70
|
-
- Export Coverage: AST-backed export comparison
|
|
71
|
-
- Signature Accuracy: AST-verified signature matching
|
|
72
|
-
- Type Coverage: AST-verified type completeness
|
|
73
|
-
- Full scoring formula applied
|
|
74
|
-
|
|
75
|
-
### Forge+ Tier (ast-grep + ccc)
|
|
76
|
-
- Same scoring as Forge tier — ccc provides pre-ranking but does not change scoring weights
|
|
77
|
-
- Improved extraction coverage (from ccc pre-discovery) may increase T1 count, but scoring formula is identical to Forge
|
|
78
|
-
- Full scoring formula applied
|
|
79
|
-
|
|
80
|
-
### Deep Tier (ast-grep + gh + QMD)
|
|
81
|
-
- All Forge tier checks plus:
|
|
82
|
-
- Cross-repository reference verification
|
|
83
|
-
- QMD knowledge enrichment for coherence
|
|
84
|
-
- Full scoring formula with maximum depth
|
|
85
|
-
- **Migration & Deprecation Warnings section:** If T2-future annotations exist in the enrichment data, verify that Section 4b is present in SKILL.md Tier 1 and that each warning traces to a T2 provenance citation. If no T2-future annotations exist, Section 4b should be absent (not empty). Presence/absence mismatch is a Medium severity gap.
|
|
86
|
-
|
|
87
|
-
## Score Calculation
|
|
88
|
-
|
|
89
|
-
```
|
|
90
|
-
score = sum(category_weight * category_score) for each category
|
|
91
|
-
category_score = (items_passing / items_total) * 100
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
## Coherence Score Aggregation (Contextual Mode)
|
|
95
|
-
|
|
96
|
-
```
|
|
97
|
-
reference_validity = (valid_references / total_references) * 100
|
|
98
|
-
integration_completeness = (complete_patterns / total_patterns) * 100
|
|
99
|
-
combined_coherence = (reference_validity * 0.6) + (integration_completeness * 0.4)
|
|
100
|
-
```
|
|
101
|
-
|
|
102
|
-
If no integration patterns exist, combined coherence equals reference validity.
|
|
103
|
-
|
|
104
|
-
## Result Determination
|
|
105
|
-
|
|
106
|
-
- score >= threshold → PASS
|
|
107
|
-
- score < threshold → FAIL
|
|
108
|
-
|
|
109
|
-
## Gap Severity
|
|
110
|
-
|
|
111
|
-
| Severity | Criteria |
|
|
112
|
-
|----------|----------|
|
|
113
|
-
| Critical | Missing exported function/class documentation |
|
|
114
|
-
| High | Signature mismatch between source and SKILL.md |
|
|
115
|
-
| Medium | Missing type or interface documentation |
|
|
116
|
-
| Medium | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
|
|
117
|
-
| Medium | Script/asset directory exists but no Scripts & Assets section in SKILL.md |
|
|
118
|
-
| Medium | Scripts & Assets section references file not found in scripts/ or assets/ directory |
|
|
119
|
-
| Low | Script/asset file present without provenance entry in provenance-map.json file_entries |
|
|
120
|
-
| Low | Missing optional metadata or examples |
|
|
121
|
-
| Low | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps) |
|
|
122
|
-
| Info | Style suggestions, non-blocking observations |
|
|
123
|
-
| Info | Discovery testing not performed — realistic prompt testing recommended before export |
|
|
@@ -1,51 +0,0 @@
|
|
|
1
|
-
# Source Access Protocol
|
|
2
|
-
|
|
3
|
-
## Source API Surface Definition
|
|
4
|
-
|
|
5
|
-
**Source API surface** = the package's top-level public exports. These are the symbols reachable from the primary entry point without importing internal modules:
|
|
6
|
-
|
|
7
|
-
- **Python:** symbols exported in `__init__.py` (including re-exports) — exclude private (`_prefixed`) names
|
|
8
|
-
- **TypeScript/JavaScript:** named exports from `index.ts` / `index.js` — exclude unexported locals
|
|
9
|
-
- **Go:** exported identifiers (capitalized) from the package's public-facing files
|
|
10
|
-
- **Rust:** items in `pub use` from `lib.rs` or `mod.rs`
|
|
11
|
-
|
|
12
|
-
Internal module symbols are **excluded** from the coverage denominator unless they are explicitly documented in SKILL.md (in which case they count as documented extras, not missing coverage).
|
|
13
|
-
|
|
14
|
-
This matches the extraction-patterns.md convention used during skill creation: coverage measures how well SKILL.md documents what users actually import, not the entire internal codebase.
|
|
15
|
-
|
|
16
|
-
## Source Access Resolution
|
|
17
|
-
|
|
18
|
-
Before analysis, determine source access level. Walk through these states in order — use the first that succeeds:
|
|
19
|
-
|
|
20
|
-
**State 1 — Local source available:**
|
|
21
|
-
Check if `{source_path}` (from metadata.json `source_root`) exists on disk. If yes → full analysis at detected tier (AST + signatures). Set `analysis_confidence: full`.
|
|
22
|
-
|
|
23
|
-
**State 2 — Local absent, provenance-map exists:**
|
|
24
|
-
Check `{forge_data_folder}/{skill_name}/provenance-map.json`. If present AND contains at least 1 entry, use it as the baseline export inventory — each entry contains structured fields: `export_name`, `export_type`, `params[]`, `return_type`, `source_file`, `source_line`, `confidence`, and `ast_node_type`. Cross-reference against SKILL.md documented exports for name-matching and param-by-param coverage. Signature verification compares SKILL.md's documented params/return types against provenance-map entries directly.
|
|
25
|
-
|
|
26
|
-
**Cross-reference with metadata.json:** After loading provenance-map entries, compare the entry count against `metadata.json`'s `exports[]` array length and `stats.exports_public_api` count. If metadata reports more exports than provenance-map entries:
|
|
27
|
-
- Compute `gap = metadata.exports.length - provenance_map.entries.length`
|
|
28
|
-
- Report: "Provenance-map contains {pmap_count} entries but metadata.json lists {meta_count} exports ({gap} gap). Coverage denominator uses the union."
|
|
29
|
-
- Build the coverage denominator from the **union** of provenance-map entry names and metadata.json `exports[]` names. Exports present in metadata but absent from provenance-map are counted as "missing documentation" in the coverage calculation.
|
|
30
|
-
- If metadata.json is unavailable or has no `exports[]` array, use provenance-map count alone with a note: "Coverage denominator is provenance-map only — may undercount if extraction was incomplete." If remote reading tools are available (zread, deepwiki, gh API, or similar), supplement by reading the entry point file for live signature verification. Set `analysis_confidence: provenance-map`.
|
|
31
|
-
|
|
32
|
-
**State 2 limitations:** Signature verification at State 2 is **string comparison only**, not semantic. Provenance-map stores parameters as flat string arrays (e.g., `["data: Union[BinaryIO, list, str]"]`), so `str` vs `String` or `list` vs `List[Any]` would be treated as mismatches even when semantically equivalent. For full type-aware verification (handling type aliases, generic equivalence), State 1 (local source) with AST re-parsing is required. When the SKILL.md was compiled from the same provenance-map (typical for create-then-test flows), most strings will match. However, enrichment (step-04) and doc-fetching (step-03c) during compilation may alter parameter descriptions, add type annotations, or normalize signatures, causing mismatches even in create-then-test flows. Expect some string-level mismatches and treat them as compilation artifacts, not source drift signals, until signature fidelity is enforced by step-05's Signature Fidelity Rule (see `signature_source` field in provenance-map entries).
|
|
33
|
-
|
|
34
|
-
**State 3 — No provenance-map, metadata exports exist (quick-skill path):**
|
|
35
|
-
If no provenance-map.json exists (typical for quick-skill output), fall back to `metadata.json`'s `exports[]` array for the export name list. Coverage check becomes a self-consistency comparison: are all names in `exports[]` documented in SKILL.md with description, parameters, and return type? Signatures cannot be verified. If remote reading tools are available, supplement by reading the entry point for live export comparison. Set `analysis_confidence: metadata-only`.
|
|
36
|
-
|
|
37
|
-
**State 4 — No local source, no forge-data, remote tools available:**
|
|
38
|
-
If neither provenance-map nor metadata exports provide a usable baseline, but remote reading tools (zread, deepwiki, gh API, or similar) are available and `source_repo` is set in metadata.json, read the entry point remotely to build the export inventory from scratch. Name-matching only — no AST. Set `analysis_confidence: remote-only`.
|
|
39
|
-
|
|
40
|
-
**State 5 — No source access at all:**
|
|
41
|
-
If none of the above succeed, fall through to docs-only mode (as defined in step-01-init.md Section 0: pre-analysis source type detection). Set `analysis_confidence: docs-only`. Warn: "**No source access available.** Coverage check evaluates documentation self-consistency only. Re-run with local clone or remote access for source-backed verification."
|
|
42
|
-
|
|
43
|
-
Set `analysis_confidence` in context for use in Section 2 analysis depth, step-05 output, and step-05 scoring.
|
|
44
|
-
|
|
45
|
-
**Confidence tier mapping:** `full` = T1, `provenance-map` = T1, `metadata-only` = T1-low, `remote-only` = T1-low, `docs-only` = T3. This aligns with the T1/T1-low/T2/T3 scale used across all SKF workflows.
|
|
46
|
-
|
|
47
|
-
**Degradation notice rules:** When `analysis_confidence` is `provenance-map`, check the `confidence` field of provenance-map entries before emitting a degradation recommendation:
|
|
48
|
-
|
|
49
|
-
- **All/most entries T1 (AST-verified):** The provenance-map data is already at highest confidence. Do NOT recommend re-running with a local clone — it would produce identical results. Use: "Resolved via: provenance-map (T1 AST-verified at compilation time). Local clone not required — provenance data is already at highest confidence."
|
|
50
|
-
- **Mixed T1/T1-low entries:** Report the breakdown. Recommend local clone only for the T1-low entries: "Resolved via: provenance-map ({n} T1, {m} T1-low). Re-run with local clone to upgrade T1-low entries to full AST verification."
|
|
51
|
-
- **All/most entries T1-low or lower:** Keep the standard recommendation: "Re-run with local clone for full AST-backed verification."
|
|
@@ -1,217 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: 'step-01-init'
|
|
3
|
-
description: 'Discover skill, load forge tier state, validate inputs, create test report'
|
|
4
|
-
|
|
5
|
-
nextStepFile: './step-02-detect-mode.md'
|
|
6
|
-
outputFile: '{forge_version}/test-report-{skill_name}.md'
|
|
7
|
-
templateFile: '../templates/test-report-template.md'
|
|
8
|
-
sidecarFile: '{sidecar_path}/forge-tier.yaml'
|
|
9
|
-
skillsOutputFolder: '{skills_output_folder}'
|
|
10
|
-
---
|
|
11
|
-
|
|
12
|
-
# Step 1: Initialize Test
|
|
13
|
-
|
|
14
|
-
## STEP GOAL:
|
|
15
|
-
|
|
16
|
-
Discover and validate the target skill, load forge tier state to determine analysis depth, and create the test report document from template.
|
|
17
|
-
|
|
18
|
-
## MANDATORY EXECUTION RULES (READ FIRST):
|
|
19
|
-
|
|
20
|
-
### Universal Rules:
|
|
21
|
-
|
|
22
|
-
- 🛑 NEVER proceed if required inputs are missing — halt with clear error
|
|
23
|
-
- 📖 CRITICAL: Read the complete step file before taking any action
|
|
24
|
-
- 🔄 CRITICAL: When loading next step, ensure entire file is read
|
|
25
|
-
- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
|
|
26
|
-
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
|
27
|
-
|
|
28
|
-
### Role Reinforcement:
|
|
29
|
-
|
|
30
|
-
- ✅ You are a skill auditor operating in Ferris's Audit mode
|
|
31
|
-
- ✅ If you already have been given a name, communication_style and identity, continue to use those while playing this new role
|
|
32
|
-
- ✅ Methodical, precise, evidence-based — zero hallucination
|
|
33
|
-
- ✅ Every finding must trace to actual artifacts
|
|
34
|
-
|
|
35
|
-
### Step-Specific Rules:
|
|
36
|
-
|
|
37
|
-
- 🎯 Focus only on input discovery and validation — do NOT begin analysis
|
|
38
|
-
- 🚫 FORBIDDEN to start coverage or coherence checks in this step
|
|
39
|
-
- 💬 Report what was found clearly and concisely
|
|
40
|
-
- 📋 Validate all required files exist before proceeding
|
|
41
|
-
|
|
42
|
-
## EXECUTION PROTOCOLS:
|
|
43
|
-
|
|
44
|
-
- 🎯 Discover skill path from user input or batch list
|
|
45
|
-
- 💾 Create output document from {templateFile}
|
|
46
|
-
- 📖 Load forge tier state from {sidecarFile}
|
|
47
|
-
- 🚫 FORBIDDEN to proceed if SKILL.md is missing
|
|
48
|
-
|
|
49
|
-
## CONTEXT BOUNDARIES:
|
|
50
|
-
|
|
51
|
-
- Available: User-provided skill path, SKF module config
|
|
52
|
-
- Focus: Input validation and tier detection only
|
|
53
|
-
- Limits: Do not analyze skill content — just confirm it exists and is loadable
|
|
54
|
-
- Dependencies: setup-forge must have run (forge-tier.yaml must exist)
|
|
55
|
-
|
|
56
|
-
## MANDATORY SEQUENCE
|
|
57
|
-
|
|
58
|
-
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
59
|
-
|
|
60
|
-
### 1. Receive Skill Path
|
|
61
|
-
|
|
62
|
-
If skill path was provided as workflow argument, use it directly.
|
|
63
|
-
|
|
64
|
-
If no path provided, ask:
|
|
65
|
-
|
|
66
|
-
"**Which skill would you like to test?**
|
|
67
|
-
|
|
68
|
-
Provide the skill path or name. I'll search in `{skillsOutputFolder}`.
|
|
69
|
-
|
|
70
|
-
**Path or name:**"
|
|
71
|
-
|
|
72
|
-
### 2. Validate Skill Exists (version-aware)
|
|
73
|
-
|
|
74
|
-
Resolve the skill path using version-aware resolution (see [knowledge/version-paths.md](../../../knowledge/version-paths.md)):
|
|
75
|
-
|
|
76
|
-
1. Read `{skillsOutputFolder}/.export-manifest.json` and look up the skill name in `exports` to get `active_version`
|
|
77
|
-
2. If found: resolve to `{skill_package}` = `{skillsOutputFolder}/{skill_name}/{active_version}/{skill_name}/`
|
|
78
|
-
3. If not in manifest: check for `active` symlink at `{skillsOutputFolder}/{skill_name}/active` — resolve to `{skill_group}/active/{skill_name}/`
|
|
79
|
-
4. If neither: fall back to flat path `{skillsOutputFolder}/{skill_name}/`. If SKILL.md exists at the flat path, auto-migrate per `knowledge/version-paths.md` migration rules
|
|
80
|
-
5. Store the resolved path as `{resolved_skill_package}`
|
|
81
|
-
|
|
82
|
-
Check that the skill package contains required files:
|
|
83
|
-
|
|
84
|
-
**Required files:**
|
|
85
|
-
- `{resolved_skill_package}/SKILL.md` — the skill documentation
|
|
86
|
-
- `{resolved_skill_package}/metadata.json` — skill metadata
|
|
87
|
-
|
|
88
|
-
**If SKILL.md missing:**
|
|
89
|
-
"**Error: SKILL.md not found at `{resolved_skill_package}/SKILL.md`**
|
|
90
|
-
|
|
91
|
-
This skill has not been created yet. Run the **create-skill** workflow first."
|
|
92
|
-
|
|
93
|
-
HALT — do not proceed.
|
|
94
|
-
|
|
95
|
-
**If metadata.json missing:**
|
|
96
|
-
"**Warning:** metadata.json not found. Proceeding with limited metadata. Some checks may be skipped."
|
|
97
|
-
|
|
98
|
-
### 3. Validate Frontmatter Compliance
|
|
99
|
-
|
|
100
|
-
Before proceeding to content analysis, verify that SKILL.md has valid agentskills.io frontmatter. This catches compliance issues early, regardless of which workflow generated the skill.
|
|
101
|
-
|
|
102
|
-
**Check the following:**
|
|
103
|
-
- [ ] SKILL.md begins with `---` and has a closing `---` (frontmatter present)
|
|
104
|
-
- [ ] `name` field exists, non-empty, lowercase alphanumeric + hyphens, 1-64 chars
|
|
105
|
-
- [ ] `name` matches the skill directory name
|
|
106
|
-
- [ ] `description` field exists, non-empty, 1-1024 chars
|
|
107
|
-
- [ ] No unknown frontmatter fields (only `name`, `description`, `license`, `compatibility`, `metadata`, `allowed-tools` are permitted)
|
|
108
|
-
|
|
109
|
-
**If frontmatter is missing or invalid:**
|
|
110
|
-
"**Warning: SKILL.md frontmatter is non-compliant with agentskills.io specification.**
|
|
111
|
-
|
|
112
|
-
{list specific issues}
|
|
113
|
-
|
|
114
|
-
This skill will fail `npx skills add` and `npx skill-check check`. Consider fixing frontmatter before proceeding (run `npx skill-check check <skill-dir> --fix` to auto-fix deterministic issues)."
|
|
115
|
-
|
|
116
|
-
Log each issue as a pre-check finding. Continue with testing — frontmatter issues will be reported in the gap report alongside coverage/coherence findings.
|
|
117
|
-
|
|
118
|
-
### 4. Load Forge Tier State
|
|
119
|
-
|
|
120
|
-
Read `{sidecarFile}` to determine available analysis depth.
|
|
121
|
-
|
|
122
|
-
**If forge-tier.yaml exists:**
|
|
123
|
-
- Read `tier` value (Quick, Forge, Forge+, or Deep)
|
|
124
|
-
- Read tool availability flags (ast_grep, gh_cli, qmd)
|
|
125
|
-
|
|
126
|
-
**If forge-tier.yaml missing:**
|
|
127
|
-
"**Cannot proceed.** forge-tier.yaml not found at `{sidecarFile}`. Please run the **setup-forge** workflow first to configure your forge tier (Quick/Forge/Forge+/Deep)."
|
|
128
|
-
|
|
129
|
-
HALT — do not proceed.
|
|
130
|
-
|
|
131
|
-
### 4b. Apply Tier Override (if set)
|
|
132
|
-
|
|
133
|
-
Read `{sidecar_path}/preferences.yaml`. If `tier_override` is set and is a valid tier value (Quick, Forge, Forge+, or Deep), update `detected_tier` to the override value for use in subsequent steps and output documents.
|
|
134
|
-
|
|
135
|
-
### 5. Load Skill Metadata
|
|
136
|
-
|
|
137
|
-
Read `metadata.json` to extract:
|
|
138
|
-
- `name` — display name
|
|
139
|
-
- `skill_type` — single or stack (needed for mode detection)
|
|
140
|
-
- `source_path` — path to source code (if present)
|
|
141
|
-
- `generation_date` — when skill was generated
|
|
142
|
-
- `confidence_tier` — tier used during creation
|
|
143
|
-
|
|
144
|
-
If source path override was provided as optional input, use that instead.
|
|
145
|
-
|
|
146
|
-
### 6. Create Output Document
|
|
147
|
-
|
|
148
|
-
Create `{outputFile}` from `{templateFile}` with initial frontmatter:
|
|
149
|
-
|
|
150
|
-
```yaml
|
|
151
|
-
---
|
|
152
|
-
workflowType: 'test-skill'
|
|
153
|
-
skillName: '{skill_name}'
|
|
154
|
-
skillDir: '{skill_path}'
|
|
155
|
-
testMode: ''
|
|
156
|
-
forgeTier: '{detected_tier}'
|
|
157
|
-
testResult: ''
|
|
158
|
-
score: ''
|
|
159
|
-
threshold: ''
|
|
160
|
-
analysisConfidence: ''
|
|
161
|
-
testDate: '{current_date}'
|
|
162
|
-
stepsCompleted: ['step-01-init']
|
|
163
|
-
nextWorkflow: ''
|
|
164
|
-
---
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
### 7. Report Initialization Status
|
|
168
|
-
|
|
169
|
-
"**Test initialization complete.**
|
|
170
|
-
|
|
171
|
-
**Skill:** {skill_name}
|
|
172
|
-
**Path:** {skill_path}
|
|
173
|
-
**Type:** {skill_type}
|
|
174
|
-
**Forge Tier:** {detected_tier}
|
|
175
|
-
**Source:** {source_path}
|
|
176
|
-
|
|
177
|
-
**Proceeding to mode detection...**"
|
|
178
|
-
|
|
179
|
-
### 8. Auto-Proceed
|
|
180
|
-
|
|
181
|
-
Display: "**Proceeding to mode detection...**"
|
|
182
|
-
|
|
183
|
-
#### Menu Handling Logic:
|
|
184
|
-
|
|
185
|
-
- After initialization is complete, update {outputFile} frontmatter stepsCompleted, then immediately load, read entire file, then execute {nextStepFile}
|
|
186
|
-
|
|
187
|
-
#### EXECUTION RULES:
|
|
188
|
-
|
|
189
|
-
- This is an auto-proceed initialization step with no user choices
|
|
190
|
-
- Proceed directly to next step after setup
|
|
191
|
-
|
|
192
|
-
## CRITICAL STEP COMPLETION NOTE
|
|
193
|
-
|
|
194
|
-
ONLY WHEN all required files are validated, forge tier is loaded, output document is created, and initialization status is reported, will you then load and read fully `{nextStepFile}` to execute mode detection.
|
|
195
|
-
|
|
196
|
-
---
|
|
197
|
-
|
|
198
|
-
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
|
|
199
|
-
|
|
200
|
-
### ✅ SUCCESS:
|
|
201
|
-
|
|
202
|
-
- Skill path validated (SKILL.md exists)
|
|
203
|
-
- Forge tier state loaded (HALT if missing), override applied if set in preferences.yaml
|
|
204
|
-
- Skill metadata extracted
|
|
205
|
-
- Output document created from template with initial frontmatter
|
|
206
|
-
- Initialization status reported clearly
|
|
207
|
-
- Auto-proceeded to step 02
|
|
208
|
-
|
|
209
|
-
### ❌ SYSTEM FAILURE:
|
|
210
|
-
|
|
211
|
-
- Proceeding without validating SKILL.md exists
|
|
212
|
-
- Not loading forge tier state
|
|
213
|
-
- Not creating output document before proceeding
|
|
214
|
-
- Starting analysis in this step (coverage/coherence belongs to later steps)
|
|
215
|
-
- Hardcoding paths instead of using frontmatter variables
|
|
216
|
-
|
|
217
|
-
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE.
|