bmad-module-skill-forge 0.9.0 → 0.10.1-alpha.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +39 -0
- package/.gitattributes +16 -0
- package/README.md +105 -38
- package/docs/404.md +13 -10
- package/docs/RELEASING.md +185 -0
- package/docs/STABILITY.md +123 -0
- package/docs/_data/pinned.yaml +98 -0
- package/docs/agents.md +16 -11
- package/docs/architecture.md +221 -0
- package/docs/bmad-synergy.md +11 -13
- package/docs/concepts.md +18 -108
- package/docs/examples.md +121 -80
- package/docs/getting-started.md +85 -147
- package/docs/how-it-works.md +25 -536
- package/docs/index.md +44 -27
- package/docs/skill-model.md +328 -0
- package/docs/troubleshooting.md +39 -0
- package/docs/verifying-a-skill.md +232 -0
- package/docs/why-skf.md +93 -0
- package/docs/workflows.md +119 -13
- package/package.json +25 -4
- package/src/README.md +23 -25
- package/src/forger/forge-tier.yaml +4 -2
- package/src/forger/preferences.yaml +7 -1
- package/src/knowledge/agentskills-spec.md +1 -1
- package/src/knowledge/architecture-verification.md +1 -1
- package/src/knowledge/ccc-bridge.md +16 -15
- package/src/knowledge/overview.md +11 -11
- package/src/knowledge/progressive-capability.md +3 -3
- package/src/knowledge/provenance-tracking.md +9 -3
- package/src/knowledge/qmd-registry.md +8 -8
- package/src/knowledge/skf-knowledge-index.csv +2 -2
- package/src/knowledge/skill-lifecycle.md +16 -1
- package/src/knowledge/split-body-strategy.md +1 -1
- package/src/knowledge/version-paths.md +17 -12
- package/src/module-help.csv +18 -17
- package/src/module.yaml +4 -0
- package/src/shared/health-check.md +372 -0
- package/src/shared/references/feasibility-report-schema.md +86 -0
- package/src/shared/references/headless-gate-convention.md +62 -0
- package/src/shared/references/output-contract-schema.md +35 -0
- package/src/shared/references/pipeline-contracts.md +102 -0
- package/src/shared/scripts/skf-atomic-write.py +370 -0
- package/src/shared/scripts/skf-manifest-ops.py +236 -0
- package/src/shared/scripts/skf-preflight.py +164 -0
- package/src/shared/scripts/skf-rebuild-managed-sections.py +201 -0
- package/src/shared/scripts/skf-severity-classify.py +163 -0
- package/src/shared/scripts/skf-skill-inventory.py +210 -0
- package/src/shared/scripts/skf-structural-diff.py +257 -0
- package/src/shared/scripts/skf-validate-frontmatter.py +318 -0
- package/src/shared/scripts/skf-validate-output.py +247 -0
- package/src/skf-analyze-source/SKILL.md +57 -0
- package/src/skf-analyze-source/assets/skill-brief-schema.md +125 -0
- package/src/skf-analyze-source/references/unit-detection-heuristics.md +124 -0
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-01-init.md +6 -66
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-01b-continue.md +4 -59
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-02-scan-project.md +7 -68
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-03-identify-units.md +6 -66
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-04-map-and-detect.md +7 -68
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-05-recommend.md +6 -66
- package/src/{workflows/analyze-source → skf-analyze-source}/steps-c/step-06-generate-briefs.md +14 -66
- package/src/skf-analyze-source/steps-c/step-07-health-check.md +22 -0
- package/src/skf-audit-skill/SKILL.md +59 -0
- package/src/{workflows/audit-skill/data → skf-audit-skill/assets}/drift-report-template.md +7 -7
- package/src/{workflows/audit-skill/data → skf-audit-skill/references}/severity-rules.md +11 -11
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-01-init.md +24 -68
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-02-re-index.md +36 -71
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-03-structural-diff.md +21 -63
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-04-semantic-diff.md +6 -65
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-05-severity-classify.md +5 -63
- package/src/{workflows/audit-skill → skf-audit-skill}/steps-c/step-06-report.md +10 -59
- package/src/skf-audit-skill/steps-c/step-07-health-check.md +22 -0
- package/src/skf-brief-skill/SKILL.md +53 -0
- package/src/{workflows/brief-skill/data → skf-brief-skill/assets}/scope-templates.md +23 -0
- package/src/skf-brief-skill/assets/skill-brief-schema.md +206 -0
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-01-gather-intent.md +6 -67
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-02-analyze-target.md +6 -66
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-03-scope-definition.md +12 -68
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-04-confirm-brief.md +5 -63
- package/src/{workflows/brief-skill → skf-brief-skill}/steps-c/step-05-write-brief.md +10 -69
- package/src/skf-brief-skill/steps-c/step-06-health-check.md +22 -0
- package/src/skf-create-skill/SKILL.md +63 -0
- package/src/{workflows/create-skill/data → skf-create-skill/assets}/compile-assembly-rules.md +60 -0
- package/src/{workflows/create-skill/data → skf-create-skill/assets}/skill-sections.md +92 -23
- package/src/skf-create-skill/assets/tessl-dismissal-rules.md +102 -0
- package/src/{workflows/create-skill/data → skf-create-skill/references}/extraction-patterns-tracing.md +13 -13
- package/src/{workflows/create-skill/data → skf-create-skill/references}/extraction-patterns.md +8 -3
- package/src/skf-create-skill/references/source-resolution-protocols.md +239 -0
- package/src/{workflows/create-skill/data → skf-create-skill/references}/tier-degradation-rules.md +8 -7
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-01-load-brief.md +25 -63
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-02-ecosystem-check.md +8 -64
- package/src/skf-create-skill/steps-c/step-03-extract.md +379 -0
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-03d-component-extraction.md +18 -35
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-04-enrich.md +12 -69
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-05-compile.md +71 -75
- package/src/skf-create-skill/steps-c/step-06-validate.md +267 -0
- package/src/{workflows/create-skill → skf-create-skill}/steps-c/step-07-generate-artifacts.md +30 -74
- package/src/skf-create-skill/steps-c/step-08-report.md +145 -0
- package/src/skf-create-skill/steps-c/step-09-health-check.md +23 -0
- package/src/{workflows/create-skill/steps-c → skf-create-skill/steps-c/sub}/step-02b-ccc-discover.md +8 -66
- package/src/skf-create-skill/steps-c/sub/step-03b-fetch-temporal.md +229 -0
- package/src/{workflows/create-skill/steps-c → skf-create-skill/steps-c/sub}/step-03c-fetch-docs.md +18 -42
- package/src/skf-create-stack-skill/SKILL.md +64 -0
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/assets}/stack-skill-template.md +2 -1
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/references}/compose-mode-rules.md +19 -5
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/references}/integration-patterns.md +1 -1
- package/src/{workflows/create-stack-skill/data → skf-create-stack-skill/references}/manifest-patterns.md +10 -10
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-01-init.md +14 -55
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-02-detect-manifests.md +29 -63
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-03-rank-and-confirm.md +9 -59
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-04-parallel-extract.md +29 -64
- package/src/skf-create-stack-skill/steps-c/step-05-detect-integrations.md +164 -0
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-06-compile-stack.md +7 -60
- package/src/skf-create-stack-skill/steps-c/step-07-generate-output.md +328 -0
- package/src/{workflows/create-stack-skill → skf-create-stack-skill}/steps-c/step-08-validate.md +17 -63
- package/src/skf-create-stack-skill/steps-c/step-09-report.md +128 -0
- package/src/skf-create-stack-skill/steps-c/step-10-health-check.md +22 -0
- package/src/skf-drop-skill/SKILL.md +57 -0
- package/src/{workflows/drop-skill → skf-drop-skill}/steps-c/step-01-select.md +9 -69
- package/src/{workflows/drop-skill → skf-drop-skill}/steps-c/step-02-execute.md +30 -106
- package/src/skf-drop-skill/steps-c/step-03-report.md +83 -0
- package/src/skf-drop-skill/steps-c/step-04-health-check.md +22 -0
- package/src/skf-export-skill/SKILL.md +57 -0
- package/src/skf-export-skill/assets/managed-section-format.md +138 -0
- package/src/{workflows/export-skill/data → skf-export-skill/assets}/snippet-format.md +8 -12
- package/src/skf-export-skill/steps-c/step-01-load-skill.md +247 -0
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-02-package.md +5 -58
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-03-generate-snippet.md +15 -77
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-04-update-context.md +59 -133
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-05-token-report.md +4 -58
- package/src/{workflows/export-skill → skf-export-skill}/steps-c/step-06-summary.md +20 -64
- package/src/skf-export-skill/steps-c/step-07-health-check.md +22 -0
- package/src/skf-forger/SKILL.md +110 -0
- package/src/skf-forger/bmad-skill-manifest.yaml +11 -0
- package/src/skf-quick-skill/SKILL.md +56 -0
- package/src/{workflows/quick-skill/data → skf-quick-skill/references}/registry-resolution.md +6 -6
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-01-resolve-target.md +9 -62
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-02-ecosystem-check.md +5 -59
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-03-quick-extract.md +18 -61
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-04-compile.md +6 -60
- package/src/{workflows/quick-skill → skf-quick-skill}/steps-c/step-05-validate.md +58 -78
- package/src/skf-quick-skill/steps-c/step-06-write.md +73 -0
- package/src/skf-quick-skill/steps-c/step-07-health-check.md +22 -0
- package/src/skf-refine-architecture/SKILL.md +57 -0
- package/src/{workflows/refine-architecture/data → skf-refine-architecture/references}/refinement-rules.md +16 -16
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-01-init.md +6 -63
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-02-gap-analysis.md +29 -71
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-03-issue-detection.md +5 -65
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-04-improvements.md +5 -64
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-05-compile.md +5 -56
- package/src/{workflows/refine-architecture → skf-refine-architecture}/steps-c/step-06-report.md +14 -64
- package/src/skf-refine-architecture/steps-c/step-07-health-check.md +22 -0
- package/src/skf-rename-skill/SKILL.md +58 -0
- package/src/{workflows/rename-skill → skf-rename-skill}/steps-c/step-01-select.md +12 -73
- package/src/{workflows/rename-skill → skf-rename-skill}/steps-c/step-02-execute.md +32 -113
- package/src/skf-rename-skill/steps-c/step-03-report.md +83 -0
- package/src/skf-rename-skill/steps-c/step-04-health-check.md +22 -0
- package/src/skf-setup/SKILL.md +55 -0
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-01-detect-and-tier.md +6 -62
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-01b-ccc-index.md +9 -65
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-02-write-config.md +15 -60
- package/src/{workflows/setup-forge → skf-setup}/steps-c/step-03-auto-index.md +7 -66
- package/src/skf-setup/steps-c/step-04-report.md +92 -0
- package/src/skf-setup/steps-c/step-05-health-check.md +22 -0
- package/src/skf-test-skill/SKILL.md +59 -0
- package/src/skf-test-skill/references/migration-section-rules.md +114 -0
- package/src/skf-test-skill/references/scoring-rules.md +207 -0
- package/src/skf-test-skill/references/source-access-protocol.md +109 -0
- package/src/skf-test-skill/scripts/compute-score.py +310 -0
- package/src/skf-test-skill/steps-c/step-01-init.md +238 -0
- package/src/{workflows/test-skill → skf-test-skill}/steps-c/step-02-detect-mode.md +4 -62
- package/src/skf-test-skill/steps-c/step-03-coverage-check.md +347 -0
- package/src/skf-test-skill/steps-c/step-04-coherence-check.md +239 -0
- package/src/{workflows/test-skill → skf-test-skill}/steps-c/step-04b-external-validators.md +42 -77
- package/src/skf-test-skill/steps-c/step-05-score.md +266 -0
- package/src/skf-test-skill/steps-c/step-06-report.md +290 -0
- package/src/skf-test-skill/steps-c/step-07-health-check.md +25 -0
- package/src/skf-test-skill/templates/test-report-template.md +58 -0
- package/src/skf-update-skill/SKILL.md +58 -0
- package/src/{workflows/update-skill/data → skf-update-skill/references}/manual-section-rules.md +6 -6
- package/src/{workflows/update-skill/data → skf-update-skill/references}/merge-conflict-rules.md +5 -5
- package/src/skf-update-skill/references/remote-source-resolution.md +94 -0
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-01-init.md +22 -69
- package/src/skf-update-skill/steps-c/step-02-detect-changes.md +255 -0
- package/src/skf-update-skill/steps-c/step-03-re-extract.md +336 -0
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-04-merge.md +60 -72
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-05-validate.md +5 -62
- package/src/skf-update-skill/steps-c/step-06-write.md +278 -0
- package/src/{workflows/update-skill → skf-update-skill}/steps-c/step-07-report.md +11 -66
- package/src/skf-update-skill/steps-c/step-08-health-check.md +22 -0
- package/src/skf-verify-stack/SKILL.md +60 -0
- package/src/skf-verify-stack/assets/feasibility-report-template.md +76 -0
- package/src/{workflows/verify-stack/data → skf-verify-stack/references}/coverage-patterns.md +12 -4
- package/src/skf-verify-stack/references/integration-verification-rules.md +78 -0
- package/src/skf-verify-stack/steps-c/step-01-init.md +178 -0
- package/src/{workflows/verify-stack → skf-verify-stack}/steps-c/step-02-coverage.md +27 -72
- package/src/skf-verify-stack/steps-c/step-03-integrations.md +179 -0
- package/src/{workflows/verify-stack → skf-verify-stack}/steps-c/step-04-requirements.md +14 -67
- package/src/skf-verify-stack/steps-c/step-05-synthesize.md +147 -0
- package/src/skf-verify-stack/steps-c/step-06-report.md +156 -0
- package/src/skf-verify-stack/steps-c/step-07-health-check.md +25 -0
- package/tools/cli/commands/install.js +1 -1
- package/tools/cli/commands/status.js +21 -23
- package/tools/cli/commands/uninstall.js +11 -19
- package/tools/cli/commands/update.js +2 -2
- package/tools/cli/lib/ide-skills.js +216 -0
- package/tools/cli/lib/installer.js +29 -52
- package/tools/cli/lib/manifest.js +11 -5
- package/tools/cli/lib/platform-codes.yaml +223 -0
- package/tools/cli/lib/ui.js +76 -47
- package/tools/validate-docs-drift.js +193 -0
- package/src/agents/forger.agent.yaml +0 -114
- package/src/workflows/README.md +0 -174
- package/src/workflows/analyze-source/data/skill-brief-schema.md +0 -125
- package/src/workflows/analyze-source/data/unit-detection-heuristics.md +0 -124
- package/src/workflows/analyze-source/workflow.md +0 -61
- package/src/workflows/audit-skill/workflow.md +0 -64
- package/src/workflows/brief-skill/data/skill-brief-schema.md +0 -159
- package/src/workflows/brief-skill/workflow.md +0 -58
- package/src/workflows/create-skill/data/source-resolution-protocols.md +0 -200
- package/src/workflows/create-skill/steps-c/step-03-extract.md +0 -301
- package/src/workflows/create-skill/steps-c/step-03b-fetch-temporal.md +0 -253
- package/src/workflows/create-skill/steps-c/step-06-validate.md +0 -254
- package/src/workflows/create-skill/steps-c/step-08-report.md +0 -176
- package/src/workflows/create-skill/workflow.md +0 -53
- package/src/workflows/create-stack-skill/steps-c/step-05-detect-integrations.md +0 -196
- package/src/workflows/create-stack-skill/steps-c/step-07-generate-output.md +0 -270
- package/src/workflows/create-stack-skill/steps-c/step-09-report.md +0 -145
- package/src/workflows/create-stack-skill/workflow.md +0 -59
- package/src/workflows/drop-skill/steps-c/step-03-report.md +0 -135
- package/src/workflows/drop-skill/workflow.md +0 -63
- package/src/workflows/export-skill/data/managed-section-format.md +0 -103
- package/src/workflows/export-skill/steps-c/step-01-load-skill.md +0 -237
- package/src/workflows/export-skill/workflow.md +0 -58
- package/src/workflows/quick-skill/steps-c/step-06-write.md +0 -179
- package/src/workflows/quick-skill/workflow.md +0 -58
- package/src/workflows/refine-architecture/workflow.md +0 -61
- package/src/workflows/rename-skill/steps-c/step-03-report.md +0 -132
- package/src/workflows/rename-skill/workflow.md +0 -64
- package/src/workflows/setup-forge/steps-c/step-04-report.md +0 -156
- package/src/workflows/setup-forge/workflow.md +0 -51
- package/src/workflows/test-skill/data/scoring-rules.md +0 -123
- package/src/workflows/test-skill/data/source-access-protocol.md +0 -51
- package/src/workflows/test-skill/steps-c/step-01-init.md +0 -217
- package/src/workflows/test-skill/steps-c/step-03-coverage-check.md +0 -280
- package/src/workflows/test-skill/steps-c/step-04-coherence-check.md +0 -271
- package/src/workflows/test-skill/steps-c/step-05-score.md +0 -222
- package/src/workflows/test-skill/steps-c/step-06-report.md +0 -207
- package/src/workflows/test-skill/templates/test-report-template.md +0 -28
- package/src/workflows/test-skill/workflow.md +0 -57
- package/src/workflows/update-skill/data/remote-source-resolution.md +0 -57
- package/src/workflows/update-skill/steps-c/step-02-detect-changes.md +0 -221
- package/src/workflows/update-skill/steps-c/step-03-re-extract.md +0 -256
- package/src/workflows/update-skill/steps-c/step-06-write.md +0 -265
- package/src/workflows/update-skill/workflow.md +0 -56
- package/src/workflows/verify-stack/data/feasibility-report-template.md +0 -63
- package/src/workflows/verify-stack/data/integration-verification-rules.md +0 -73
- package/src/workflows/verify-stack/steps-c/step-01-init.md +0 -179
- package/src/workflows/verify-stack/steps-c/step-03-integrations.md +0 -195
- package/src/workflows/verify-stack/steps-c/step-05-synthesize.md +0 -191
- package/src/workflows/verify-stack/steps-c/step-06-report.md +0 -197
- package/src/workflows/verify-stack/workflow.md +0 -61
- package/tools/cli/lib/compiler.js +0 -306
- package/tools/cli/lib/ide-commands.js +0 -244
- /package/src/{workflows/analyze-source → skf-analyze-source}/templates/analysis-report-template.md +0 -0
- /package/src/{workflows/quick-skill/data → skf-quick-skill/assets}/skill-template.md +0 -0
- /package/src/{workflows/setup-forge/data → skf-setup/references}/tier-rules.md +0 -0
- /package/src/{workflows/test-skill/data → skf-test-skill/assets}/output-section-formats.md +0 -0
|
@@ -1,222 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: 'step-05-score'
|
|
3
|
-
description: 'Calculate completeness score from coverage and coherence findings'
|
|
4
|
-
|
|
5
|
-
nextStepFile: './step-06-report.md'
|
|
6
|
-
outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
|
|
7
|
-
scoringRulesFile: '../data/scoring-rules.md'
|
|
8
|
-
sourceAccessProtocol: '../data/source-access-protocol.md'
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
# Step 5: Score
|
|
12
|
-
|
|
13
|
-
## STEP GOAL:
|
|
14
|
-
|
|
15
|
-
Calculate the overall completeness score by aggregating coverage, coherence, and external validation category scores with the appropriate weight distribution (naive or contextual), apply the pass/fail threshold, and determine the test result.
|
|
16
|
-
|
|
17
|
-
## MANDATORY EXECUTION RULES (READ FIRST):
|
|
18
|
-
|
|
19
|
-
### Universal Rules:
|
|
20
|
-
|
|
21
|
-
- 🛑 NEVER invent scores — all values must derive from step 03 and step 04 findings
|
|
22
|
-
- 📖 CRITICAL: Read the complete step file before taking any action
|
|
23
|
-
- 🔄 CRITICAL: When loading next step, ensure entire file is read
|
|
24
|
-
- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
|
|
25
|
-
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
|
26
|
-
|
|
27
|
-
### Role Reinforcement:
|
|
28
|
-
|
|
29
|
-
- ✅ You are a skill auditor in Ferris's Audit mode — zero hallucination
|
|
30
|
-
- ✅ If you already have been given a name, communication_style and identity, continue to use those while playing this new role
|
|
31
|
-
- ✅ Scoring is mechanical — apply the formula, report the math
|
|
32
|
-
- ✅ Do not editorialize on the score — facts and numbers only
|
|
33
|
-
|
|
34
|
-
### Step-Specific Rules:
|
|
35
|
-
|
|
36
|
-
- 🎯 Focus only on score calculation — do NOT generate remediation suggestions (that's step 06)
|
|
37
|
-
- 🚫 FORBIDDEN to adjust scores subjectively — the formula determines the result
|
|
38
|
-
- 💬 Show the math: category scores, weights, weighted contributions, total
|
|
39
|
-
- 📋 Score must be reproducible — same inputs always produce same output
|
|
40
|
-
|
|
41
|
-
## EXECUTION PROTOCOLS:
|
|
42
|
-
|
|
43
|
-
- 🎯 Load scoring rules and read findings from {outputFile}
|
|
44
|
-
- 💾 Append Completeness Score section to {outputFile}
|
|
45
|
-
- 📖 Update {outputFile} frontmatter with testResult, score, and stepsCompleted
|
|
46
|
-
- 🚫 FORBIDDEN to proceed without setting testResult to pass or fail
|
|
47
|
-
|
|
48
|
-
## CONTEXT BOUNDARIES:
|
|
49
|
-
|
|
50
|
-
- Available: Coverage Analysis (step 03), Coherence Analysis (step 04), and External Validation (step 04b) in {outputFile}
|
|
51
|
-
- Focus: Score calculation and pass/fail determination only
|
|
52
|
-
- Limits: Do NOT generate gap remediation — that's step 06
|
|
53
|
-
- Dependencies: steps 03, 04, and 04b must have appended their analysis sections
|
|
54
|
-
|
|
55
|
-
## MANDATORY SEQUENCE
|
|
56
|
-
|
|
57
|
-
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
58
|
-
|
|
59
|
-
### 1. Load Scoring Rules
|
|
60
|
-
|
|
61
|
-
Load `{scoringRulesFile}` to get:
|
|
62
|
-
- Category weights (naive vs contextual distribution)
|
|
63
|
-
- Default threshold (80%)
|
|
64
|
-
- Tier-dependent scoring adjustments
|
|
65
|
-
- Any custom threshold override from workflow input
|
|
66
|
-
|
|
67
|
-
**Docs-only mode check:** If the Coverage Analysis section in `{outputFile}` notes docs-only mode (set by step-03 for skills with all `[EXT:...]` citations and no local source), apply Quick-tier weight redistribution: Signature Accuracy and Type Coverage are not scored, their weights (22% + 14%) are redistributed proportionally to remaining active categories. Coverage score is based on documentation completeness rather than source coverage (as calculated by step-03).
|
|
68
|
-
|
|
69
|
-
### 2. Read Category Scores from Output
|
|
70
|
-
|
|
71
|
-
Read `{outputFile}` and extract the category scores calculated in previous steps:
|
|
72
|
-
|
|
73
|
-
**From Coverage Analysis (step 03):**
|
|
74
|
-
- Export Coverage: {percentage}%
|
|
75
|
-
- Signature Accuracy: {percentage}% or N/A (Quick tier)
|
|
76
|
-
- Type Coverage: {percentage}% or N/A (Quick tier)
|
|
77
|
-
|
|
78
|
-
**From Coherence Analysis (step 04):**
|
|
79
|
-
- Combined Coherence: {percentage}% (contextual mode only)
|
|
80
|
-
- Or: not scored (naive mode — weight redistributed)
|
|
81
|
-
|
|
82
|
-
**From External Validation (step 04b):**
|
|
83
|
-
- External Validation Score: {percentage}% (combined skill-check + tessl average)
|
|
84
|
-
- Or: N/A (if neither tool was available — weight redistributed to other categories)
|
|
85
|
-
|
|
86
|
-
### 3. Apply Weight Distribution
|
|
87
|
-
|
|
88
|
-
**Read testMode from {outputFile} frontmatter.**
|
|
89
|
-
|
|
90
|
-
Apply the weight distribution from `{scoringRulesFile}` for the detected mode (naive or contextual). The scoring rules define category weights, external validation redistribution when unavailable, and Quick tier adjustments. Calculate the weighted score for each category and sum for the total.
|
|
91
|
-
|
|
92
|
-
### 4. Determine Pass/Fail
|
|
93
|
-
|
|
94
|
-
```
|
|
95
|
-
threshold = custom_threshold OR default_threshold (80%)
|
|
96
|
-
score = sum of weighted category scores
|
|
97
|
-
|
|
98
|
-
IF score >= threshold → PASS
|
|
99
|
-
IF score < threshold → FAIL
|
|
100
|
-
```
|
|
101
|
-
|
|
102
|
-
### 5. Determine Next Workflow Recommendation
|
|
103
|
-
|
|
104
|
-
Based on test result:
|
|
105
|
-
|
|
106
|
-
**IF PASS:**
|
|
107
|
-
- `nextWorkflow: 'export-skill'` — skill is ready for export
|
|
108
|
-
|
|
109
|
-
**IF FAIL:**
|
|
110
|
-
- `nextWorkflow: 'update-skill'` — skill needs remediation before export
|
|
111
|
-
|
|
112
|
-
### 6. Append Completeness Score to Output
|
|
113
|
-
|
|
114
|
-
Append the **Completeness Score** section to `{outputFile}`:
|
|
115
|
-
|
|
116
|
-
```markdown
|
|
117
|
-
## Completeness Score
|
|
118
|
-
|
|
119
|
-
### Score Breakdown
|
|
120
|
-
|
|
121
|
-
| Category | Score | Weight | Weighted |
|
|
122
|
-
|----------|-------|--------|----------|
|
|
123
|
-
| Export Coverage | {N}% | {W}% | {WS}% |
|
|
124
|
-
| Signature Accuracy | {N}% | {W}% | {WS}% |
|
|
125
|
-
| Type Coverage | {N}% | {W}% | {WS}% |
|
|
126
|
-
| Coherence | {N}% | {W}% | {WS}% |
|
|
127
|
-
| External Validation | {N}% | {W}% | {WS}% |
|
|
128
|
-
| **Total** | | **100%** | **{total}%** |
|
|
129
|
-
|
|
130
|
-
### Result
|
|
131
|
-
|
|
132
|
-
**Score:** {total}%
|
|
133
|
-
**Threshold:** {threshold}%
|
|
134
|
-
**Result:** **{PASS|FAIL}**
|
|
135
|
-
|
|
136
|
-
**Weight Distribution:** {naive (redistributed) | contextual (full)}
|
|
137
|
-
**Tier Adjustment:** {none | Quick tier — signature and type coverage not scored}
|
|
138
|
-
**External Validators:** {both available | skill-check only | tessl only | none — weight redistributed}
|
|
139
|
-
**Analysis Confidence:** {full | provenance-map | metadata-only | remote-only | docs-only}
|
|
140
|
-
```
|
|
141
|
-
|
|
142
|
-
If `analysis_confidence` is not `full`, append a degradation notice. **The notice must be confidence-aware** — see the degradation notice rules in `{sourceAccessProtocol}`:
|
|
143
|
-
|
|
144
|
-
```markdown
|
|
145
|
-
### Access Degradation Notice
|
|
146
|
-
|
|
147
|
-
**Resolved via:** {analysis_confidence} {confidence breakdown if provenance-map, e.g., "(T1 AST-verified at compilation time)" or "(12 T1, 3 T1-low)"}
|
|
148
|
-
**Impact:** {describe limitation — e.g., "Signature checks limited to name-matching. Source file:line citations from provenance-map, not live AST." — or "Provenance data is at highest confidence; no limitation." for all-T1 provenance-map}
|
|
149
|
-
**Recommendation:** {confidence-dependent — see {sourceAccessProtocol} degradation notice rules. Do NOT recommend local clone when provenance-map entries are already T1.}
|
|
150
|
-
```
|
|
151
|
-
|
|
152
|
-
### 7. Update Output Frontmatter
|
|
153
|
-
|
|
154
|
-
Update `{outputFile}` frontmatter:
|
|
155
|
-
- `testResult: '{pass|fail}'`
|
|
156
|
-
- `score: '{total}%'`
|
|
157
|
-
- `threshold: '{threshold}%'`
|
|
158
|
-
- `analysisConfidence: '{analysis_confidence}'`
|
|
159
|
-
- `nextWorkflow: '{export-skill|update-skill}'`
|
|
160
|
-
- Append `'step-05-score'` to `stepsCompleted`
|
|
161
|
-
|
|
162
|
-
### 8. Report Score
|
|
163
|
-
|
|
164
|
-
"**Completeness score calculated.**
|
|
165
|
-
|
|
166
|
-
**{skill_name}:** **{total}%** — **{PASS|FAIL}**
|
|
167
|
-
|
|
168
|
-
| Category | Score | Weighted |
|
|
169
|
-
|----------|-------|----------|
|
|
170
|
-
| Export Coverage | {N}% | {WS}% |
|
|
171
|
-
| Signature Accuracy | {N}% | {WS}% |
|
|
172
|
-
| Type Coverage | {N}% | {WS}% |
|
|
173
|
-
| Coherence | {N}% | {WS}% |
|
|
174
|
-
| External Validation | {N}% | {WS}% |
|
|
175
|
-
|
|
176
|
-
**Threshold:** {threshold}%
|
|
177
|
-
**Recommendation:** {export-skill if pass | update-skill if fail}
|
|
178
|
-
|
|
179
|
-
**Proceeding to gap report...**"
|
|
180
|
-
|
|
181
|
-
### 9. Auto-Proceed
|
|
182
|
-
|
|
183
|
-
Display: "**Proceeding to gap report...**"
|
|
184
|
-
|
|
185
|
-
#### Menu Handling Logic:
|
|
186
|
-
|
|
187
|
-
- After score is calculated and frontmatter updated, immediately load, read entire file, then execute {nextStepFile}
|
|
188
|
-
|
|
189
|
-
#### EXECUTION RULES:
|
|
190
|
-
|
|
191
|
-
- This is an auto-proceed scoring step with no user choices
|
|
192
|
-
- Proceed directly to next step after score is determined
|
|
193
|
-
|
|
194
|
-
## CRITICAL STEP COMPLETION NOTE
|
|
195
|
-
|
|
196
|
-
ONLY WHEN the score is calculated, pass/fail is determined, the Completeness Score section is appended to {outputFile}, and frontmatter is updated with testResult and score, will you then load and read fully `{nextStepFile}` to execute gap report generation.
|
|
197
|
-
|
|
198
|
-
---
|
|
199
|
-
|
|
200
|
-
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
|
|
201
|
-
|
|
202
|
-
### ✅ SUCCESS:
|
|
203
|
-
|
|
204
|
-
- All category scores read from previous steps (not invented)
|
|
205
|
-
- Correct weight distribution applied (naive vs contextual)
|
|
206
|
-
- Tier adjustment applied if Quick tier
|
|
207
|
-
- Score calculated with visible math (show the formula)
|
|
208
|
-
- Pass/fail determined against threshold
|
|
209
|
-
- Completeness Score section appended to output
|
|
210
|
-
- Frontmatter updated with testResult, score, threshold, nextWorkflow
|
|
211
|
-
- Auto-proceeded to step 06
|
|
212
|
-
|
|
213
|
-
### ❌ SYSTEM FAILURE:
|
|
214
|
-
|
|
215
|
-
- Inventing category scores not calculated in steps 03/04
|
|
216
|
-
- Using wrong weight distribution for the detected mode
|
|
217
|
-
- Not showing the math (score must be reproducible)
|
|
218
|
-
- Subjectively adjusting the score
|
|
219
|
-
- Not setting testResult in frontmatter
|
|
220
|
-
- Generating remediation suggestions (that's step 06)
|
|
221
|
-
|
|
222
|
-
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE. Scoring is mechanical — apply the formula, report the math.
|
|
@@ -1,207 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: 'step-06-report'
|
|
3
|
-
description: 'Generate gap report with remediation suggestions and finalize test report'
|
|
4
|
-
|
|
5
|
-
outputFile: '{forge_data_folder}/{skill_name}/test-report-{skill_name}.md'
|
|
6
|
-
scoringRulesFile: '../data/scoring-rules.md'
|
|
7
|
-
outputFormatsFile: '../data/output-section-formats.md'
|
|
8
|
-
---
|
|
9
|
-
|
|
10
|
-
# Step 6: Gap Report
|
|
11
|
-
|
|
12
|
-
## STEP GOAL:
|
|
13
|
-
|
|
14
|
-
Generate a detailed gap report listing every issue found during coverage and coherence analysis, assign severity to each gap, provide specific actionable remediation suggestions, and finalize the test report document. This is the final step — no next step file.
|
|
15
|
-
|
|
16
|
-
## MANDATORY EXECUTION RULES (READ FIRST):
|
|
17
|
-
|
|
18
|
-
### Universal Rules:
|
|
19
|
-
|
|
20
|
-
- 🛑 NEVER fabricate gaps — every item must trace to findings from steps 03, 04, and 04b
|
|
21
|
-
- 📖 CRITICAL: Read the complete step file before taking any action
|
|
22
|
-
- ⚙️ TOOL/SUBPROCESS FALLBACK: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
|
|
23
|
-
- ✅ YOU MUST ALWAYS SPEAK OUTPUT In your Agent communication style with the config `{communication_language}`
|
|
24
|
-
|
|
25
|
-
### Role Reinforcement:
|
|
26
|
-
|
|
27
|
-
- ✅ You are a skill auditor in Ferris's Audit mode — zero hallucination
|
|
28
|
-
- ✅ If you already have been given a name, communication_style and identity, continue to use those while playing this new role
|
|
29
|
-
- ✅ Remediation suggestions must be specific and actionable — not vague advice
|
|
30
|
-
- ✅ Each gap tells the user exactly what to fix, where, and how
|
|
31
|
-
|
|
32
|
-
### Step-Specific Rules:
|
|
33
|
-
|
|
34
|
-
- 🎯 Focus on gap enumeration, severity classification, and remediation
|
|
35
|
-
- 🚫 FORBIDDEN to recalculate scores — use the score from step 05
|
|
36
|
-
- 💬 Remediation suggestions reference specific files, exports, and line numbers
|
|
37
|
-
- 📋 Gaps are ordered by severity (Critical → High → Medium → Low → Info)
|
|
38
|
-
|
|
39
|
-
## EXECUTION PROTOCOLS:
|
|
40
|
-
|
|
41
|
-
- 🎯 Read all findings from {outputFile} coverage and coherence sections
|
|
42
|
-
- 💾 Append Gap Report section to {outputFile}
|
|
43
|
-
- 📖 Update {outputFile} frontmatter: mark stepsCompleted with all steps, finalize document
|
|
44
|
-
- 🚫 This is the final step — no next step to load
|
|
45
|
-
|
|
46
|
-
## CONTEXT BOUNDARIES:
|
|
47
|
-
|
|
48
|
-
- Available: Complete {outputFile} with Test Summary, Coverage Analysis, Coherence Analysis, Completeness Score
|
|
49
|
-
- Focus: Gap enumeration and remediation only
|
|
50
|
-
- Limits: Do NOT re-analyze source code — work from existing findings
|
|
51
|
-
- Dependencies: steps 03, 04, and 05 must have completed their sections
|
|
52
|
-
|
|
53
|
-
## MANDATORY SEQUENCE
|
|
54
|
-
|
|
55
|
-
**CRITICAL:** Follow this sequence exactly. Do not skip, reorder, or improvise unless user explicitly requests a change.
|
|
56
|
-
|
|
57
|
-
### 1. Collect All Issues
|
|
58
|
-
|
|
59
|
-
Read `{outputFile}` and extract every issue found across all analysis sections:
|
|
60
|
-
|
|
61
|
-
**From Coverage Analysis (step 03):**
|
|
62
|
-
- Missing documentation (exports in source but not in SKILL.md)
|
|
63
|
-
- Signature mismatches (documented signature differs from source)
|
|
64
|
-
- Stale documentation (documented but no longer in source)
|
|
65
|
-
- Type coverage gaps (undocumented types/interfaces)
|
|
66
|
-
|
|
67
|
-
**From Coherence Analysis (step 04):**
|
|
68
|
-
- Broken references (file paths, skill references, type imports that don't resolve)
|
|
69
|
-
- Incomplete integration patterns (contextual mode)
|
|
70
|
-
- Structural issues (naive mode — missing sections, broken examples)
|
|
71
|
-
|
|
72
|
-
**From External Validation (step 04b):**
|
|
73
|
-
- skill-check diagnostics (unresolved errors and warnings)
|
|
74
|
-
- tessl judge suggestions (content quality and actionability improvements)
|
|
75
|
-
|
|
76
|
-
### 2. Load Severity Rules
|
|
77
|
-
|
|
78
|
-
Load `{scoringRulesFile}` for gap severity classification:
|
|
79
|
-
|
|
80
|
-
| Severity | Criteria |
|
|
81
|
-
|----------|----------|
|
|
82
|
-
| **Critical** | Missing exported function/class documentation |
|
|
83
|
-
| **High** | Signature mismatch between source and SKILL.md |
|
|
84
|
-
| **Medium** | Missing type or interface documentation |
|
|
85
|
-
| **Medium** | Migration section present/absent mismatch with T2-future annotation data (Deep tier) |
|
|
86
|
-
| **Medium** | Script/asset directory exists but no Scripts & Assets section in SKILL.md |
|
|
87
|
-
| **Medium** | Scripts & Assets section references file not found in scripts/ or assets/ directory |
|
|
88
|
-
| **Low** | Script/asset file present without provenance entry in provenance-map.json file_entries |
|
|
89
|
-
| **Low** | Missing optional metadata or examples |
|
|
90
|
-
| **Low** | Description trigger optimization recommended (third-person voice, negative triggers, or keyword coverage gaps) |
|
|
91
|
-
| **Info** | Style suggestions, non-blocking observations |
|
|
92
|
-
| **Info** | Discovery testing not performed — realistic prompt testing recommended before export |
|
|
93
|
-
|
|
94
|
-
### 3. Classify and Order Gaps
|
|
95
|
-
|
|
96
|
-
Load `{outputFormatsFile}` for gap entry format and remediation quality rules.
|
|
97
|
-
|
|
98
|
-
For each issue, assign severity from `{scoringRulesFile}` and generate a specific remediation following the quality rules in `{outputFormatsFile}`. Order gaps by severity: Critical → High → Medium → Low → Info.
|
|
99
|
-
|
|
100
|
-
### 4. Generate Remediation Summary and Append Gap Report
|
|
101
|
-
|
|
102
|
-
Load the Gap Report section format from `{outputFormatsFile}`. Count gaps by severity, estimate effort per the guidelines in `{outputFormatsFile}`, and append the complete **Gap Report** section to `{outputFile}`.
|
|
103
|
-
|
|
104
|
-
If no gaps found, append a clean pass message recommending **export-skill** workflow.
|
|
105
|
-
|
|
106
|
-
### 4b. Discovery and Description Quality Recommendations
|
|
107
|
-
|
|
108
|
-
After gap enumeration, append a **Discovery Quality** subsection to the gap report. Use the `Gap Entry Format` from `{outputFormatsFile}` for any Low/Info entries. The prose recommendations below are appended after the gap entries:
|
|
109
|
-
|
|
110
|
-
**Description optimization:** If tessl `description_score` (from step 04b) is below 90%, or if skill-check flagged description issues, recommend description improvements:
|
|
111
|
-
- Check that the description uses third-person voice consistently
|
|
112
|
-
- Check for specific trigger keywords that match how users would phrase requests
|
|
113
|
-
- Check for negative triggers ("NOT for: ...") to prevent false matches
|
|
114
|
-
- Check for alternative skill references for excluded use cases
|
|
115
|
-
|
|
116
|
-
**Discovery testing recommendation:** Regardless of pass/fail, always append:
|
|
117
|
-
|
|
118
|
-
"**Discovery testing recommended.** Before export, test the skill with 3-5 realistic prompts phrased the way real users actually talk — with casual language, typos, incomplete context, and implicit references. A skill tested only with clean prompts may fail to trigger in production. Example realistic prompt patterns:
|
|
119
|
-
- Vague: 'can you help me with this csv file my boss sent'
|
|
120
|
-
- Implicit: 'why did revenue drop last quarter'
|
|
121
|
-
- Abbreviated: 'run the {skill-name} thing on this data'"
|
|
122
|
-
|
|
123
|
-
Record discovery testing status as Info-level in the gap table. This is advisory — it does not affect the score.
|
|
124
|
-
|
|
125
|
-
### 5. Finalize Output Document
|
|
126
|
-
|
|
127
|
-
Update `{outputFile}` frontmatter:
|
|
128
|
-
- Set `stepsCompleted` to `['step-01-init', 'step-02-detect-mode', 'step-03-coverage-check', 'step-04-coherence-check', 'step-04b-external-validators', 'step-05-score', 'step-06-report']`
|
|
129
|
-
|
|
130
|
-
### 6. Present Final Report
|
|
131
|
-
|
|
132
|
-
"**Test complete for {skill_name}.**
|
|
133
|
-
|
|
134
|
-
---
|
|
135
|
-
|
|
136
|
-
**Result:** **{PASS|FAIL}** — **{score}%** (threshold: {threshold}%)
|
|
137
|
-
|
|
138
|
-
**Gaps Found:** {total_gaps}
|
|
139
|
-
- Critical: {N}
|
|
140
|
-
- High: {N}
|
|
141
|
-
- Medium: {N}
|
|
142
|
-
- Low: {N}
|
|
143
|
-
- Info: {N}
|
|
144
|
-
|
|
145
|
-
**Report saved to:** `{outputFile}`
|
|
146
|
-
|
|
147
|
-
---
|
|
148
|
-
|
|
149
|
-
**Recommended next step:**
|
|
150
|
-
|
|
151
|
-
{IF PASS:}
|
|
152
|
-
**export-skill** — This skill is ready for export. Run the export-skill workflow to package it for distribution.
|
|
153
|
-
|
|
154
|
-
{IF FAIL:}
|
|
155
|
-
**update-skill** — This skill needs remediation. Review the gap report above and run the update-skill workflow to address the {N} blocking issues (Critical + High).
|
|
156
|
-
|
|
157
|
-
---
|
|
158
|
-
|
|
159
|
-
**See Discovery Quality section in the report for description optimization and realistic prompt testing recommendations.**
|
|
160
|
-
|
|
161
|
-
**Test report finalized.**"
|
|
162
|
-
|
|
163
|
-
### 7. Present MENU OPTIONS
|
|
164
|
-
|
|
165
|
-
Display: "**Test complete.** [C] Finish"
|
|
166
|
-
|
|
167
|
-
#### Menu Handling Logic:
|
|
168
|
-
|
|
169
|
-
- IF C: Workflow is complete. No further steps to load.
|
|
170
|
-
- IF Any other: help user respond, then redisplay menu
|
|
171
|
-
|
|
172
|
-
#### EXECUTION RULES:
|
|
173
|
-
|
|
174
|
-
- ALWAYS halt and wait for user input after presenting menu
|
|
175
|
-
- This is the final step — C ends the workflow
|
|
176
|
-
- User may ask questions about the report before finishing
|
|
177
|
-
|
|
178
|
-
## CRITICAL STEP COMPLETION NOTE
|
|
179
|
-
|
|
180
|
-
This is the final step of the test-skill workflow. When the user selects C, the workflow is complete. The test report document at `{outputFile}` contains the full analysis: Test Summary, Coverage Analysis, Coherence Analysis, Completeness Score, and Gap Report.
|
|
181
|
-
|
|
182
|
-
---
|
|
183
|
-
|
|
184
|
-
## 🚨 SYSTEM SUCCESS/FAILURE METRICS
|
|
185
|
-
|
|
186
|
-
### ✅ SUCCESS:
|
|
187
|
-
|
|
188
|
-
- Every gap traces to a finding from steps 03, 04, or 04b (zero fabrication)
|
|
189
|
-
- Gaps classified by severity using scoring rules
|
|
190
|
-
- Gaps ordered by severity (Critical first)
|
|
191
|
-
- Every gap has a specific, actionable remediation suggestion
|
|
192
|
-
- Discovery quality section appended with description optimization and testing recommendations
|
|
193
|
-
- Remediation summary with counts and effort estimates
|
|
194
|
-
- Output document finalized with all stepsCompleted
|
|
195
|
-
- Correct next workflow recommended (export-skill or update-skill)
|
|
196
|
-
- Report presented clearly to user
|
|
197
|
-
|
|
198
|
-
### ❌ SYSTEM FAILURE:
|
|
199
|
-
|
|
200
|
-
- Fabricating gaps not found in analysis steps
|
|
201
|
-
- Vague remediation suggestions ("fix the issue", "add documentation")
|
|
202
|
-
- Not classifying severity
|
|
203
|
-
- Recalculating or modifying the score from step 05
|
|
204
|
-
- Not finalizing the output document frontmatter
|
|
205
|
-
- Missing the next workflow recommendation
|
|
206
|
-
|
|
207
|
-
**Master Rule:** Skipping steps, optimizing sequences, or not following exact instructions is FORBIDDEN and constitutes SYSTEM FAILURE. Every gap is specific, every remediation is actionable.
|
|
@@ -1,28 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
workflowType: 'test-skill'
|
|
3
|
-
skillName: ''
|
|
4
|
-
skillDir: ''
|
|
5
|
-
testMode: ''
|
|
6
|
-
forgeTier: ''
|
|
7
|
-
testResult: ''
|
|
8
|
-
score: ''
|
|
9
|
-
threshold: ''
|
|
10
|
-
analysisConfidence: ''
|
|
11
|
-
testDate: ''
|
|
12
|
-
stepsCompleted: []
|
|
13
|
-
nextWorkflow: ''
|
|
14
|
-
---
|
|
15
|
-
|
|
16
|
-
# Test Report: {{skillName}}
|
|
17
|
-
|
|
18
|
-
## Test Summary
|
|
19
|
-
|
|
20
|
-
## Coverage Analysis
|
|
21
|
-
|
|
22
|
-
## Coherence Analysis
|
|
23
|
-
|
|
24
|
-
## External Validation
|
|
25
|
-
|
|
26
|
-
## Completeness Score
|
|
27
|
-
|
|
28
|
-
## Gap Report
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: test-skill
|
|
3
|
-
description: Cognitive completeness verification. Naive and contextual modes.
|
|
4
|
-
web_bundle: true
|
|
5
|
-
installed_path: '{project-root}/_bmad/skf/workflows/skillforge/test-skill'
|
|
6
|
-
---
|
|
7
|
-
|
|
8
|
-
# Test Skill
|
|
9
|
-
|
|
10
|
-
**Goal:** Verify that a skill is complete enough to be useful to an AI agent by checking coverage of the public API surface (naive mode) or validating SKILL.md + references coherence (contextual mode), producing a completeness score and gap report as a quality gate before export.
|
|
11
|
-
|
|
12
|
-
**Your Role:** In addition to your name, communication_style, and persona, you are also a skill auditor and completeness analyst operating in Ferris's Audit mode. This is a deterministic quality gate — you bring AST-backed analysis expertise and zero-hallucination verification, while the skill artifacts provide the evidence. Every finding must trace to actual code with file:line citations.
|
|
13
|
-
|
|
14
|
-
## WORKFLOW ARCHITECTURE
|
|
15
|
-
|
|
16
|
-
### Core Principles
|
|
17
|
-
|
|
18
|
-
- **Micro-file Design**: Each step of the overall goal is a self contained instruction file that you will adhere to 1 file as directed at a time
|
|
19
|
-
- **Just-In-Time Loading**: Only 1 current step file will be loaded, read, and executed to completion - never load future step files until told to do so
|
|
20
|
-
- **Sequential Enforcement**: Sequence within the step files must be completed in order, no skipping or optimization allowed
|
|
21
|
-
- **State Tracking**: Document progress in output file frontmatter using `stepsCompleted` array
|
|
22
|
-
- **Append-Only Building**: Build the test report by appending content as directed to the output file
|
|
23
|
-
|
|
24
|
-
### Step Processing Rules
|
|
25
|
-
|
|
26
|
-
1. **READ COMPLETELY**: Always read the entire step file before taking any action
|
|
27
|
-
2. **FOLLOW SEQUENCE**: Execute all numbered sections in order, never deviate
|
|
28
|
-
3. **WAIT FOR INPUT**: If a menu is presented, halt and wait for user selection
|
|
29
|
-
4. **CHECK CONTINUATION**: If the step has a menu with Continue as an option, only proceed to next step when user selects 'C' (Continue)
|
|
30
|
-
5. **SAVE STATE**: Update `stepsCompleted` in frontmatter before loading next step
|
|
31
|
-
6. **LOAD NEXT**: When directed, load, read entire file, then execute the next step file
|
|
32
|
-
|
|
33
|
-
### Critical Rules (NO EXCEPTIONS)
|
|
34
|
-
|
|
35
|
-
- 🛑 **NEVER** load multiple step files simultaneously
|
|
36
|
-
- 📖 **ALWAYS** read entire step file before execution
|
|
37
|
-
- 🚫 **NEVER** skip steps or optimize the sequence
|
|
38
|
-
- 💾 **ALWAYS** update frontmatter of output files when writing the final output for a specific step
|
|
39
|
-
- 🎯 **ALWAYS** follow the exact instructions in the step file
|
|
40
|
-
- ⏸️ **ALWAYS** halt at menus and wait for user input
|
|
41
|
-
- 📋 **NEVER** create mental todo lists from future steps
|
|
42
|
-
- ⚙️ **TOOL/SUBPROCESS FALLBACK**: If any instruction references a subprocess, subagent, or tool you do not have access to, you MUST still achieve the outcome in your main context thread
|
|
43
|
-
|
|
44
|
-
---
|
|
45
|
-
|
|
46
|
-
## INITIALIZATION SEQUENCE
|
|
47
|
-
|
|
48
|
-
### 1. Module Configuration Loading
|
|
49
|
-
|
|
50
|
-
Load and read full config from {project-root}/_bmad/skf/config.yaml and resolve:
|
|
51
|
-
|
|
52
|
-
- `project_name`, `user_name`, `communication_language`, `document_output_language`
|
|
53
|
-
- `skills_output_folder`, `forge_data_folder`, `sidecar_path`
|
|
54
|
-
|
|
55
|
-
### 2. First Step Execution
|
|
56
|
-
|
|
57
|
-
Load, read the full file and then execute `./steps-c/step-01-init.md` to begin the workflow.
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
# Remote Source Resolution (Forge/Deep Tier)
|
|
2
|
-
|
|
3
|
-
If `source_root` is a local path: proceed with the tier-appropriate strategy as normal.
|
|
4
|
-
|
|
5
|
-
If `source_root` (from metadata.json) is a remote URL (GitHub URL or owner/repo format) AND tier is Forge or Deep:
|
|
6
|
-
|
|
7
|
-
1. **Check `git` availability:** Verify `git` is functional (`git --version`). If `git` is not available, skip to the fallback warning below.
|
|
8
|
-
|
|
9
|
-
2. **Resolve source ref:** Read `source_ref` from the existing `metadata.json`. If the user provided a new `target_version`, resolve its tag first (using the Tag Resolution algorithm in `create-skill/data/source-resolution-protocols.md`).
|
|
10
|
-
|
|
11
|
-
3. **Ephemeral sparse clone:** Clone only the changed files from the change manifest to a system temp path. Note: at this point in the flow, `{source_root}` is known to be a remote URL (the local-path case was already handled above).
|
|
12
|
-
|
|
13
|
-
```
|
|
14
|
-
temp_path = {system_temp}/skf-ephemeral-{skill-name}-{timestamp}/
|
|
15
|
-
```
|
|
16
|
-
|
|
17
|
-
If `source_ref` exists and is not `HEAD`/null/`"local"`:
|
|
18
|
-
```
|
|
19
|
-
git clone --depth 1 --branch {source_ref} --single-branch --filter=blob:none --sparse {source_root} {temp_path}
|
|
20
|
-
```
|
|
21
|
-
|
|
22
|
-
If `source_ref` is `HEAD`/null/`"local"` (or absent):
|
|
23
|
-
```
|
|
24
|
-
git clone --depth 1 --single-branch --filter=blob:none --sparse {source_root} {temp_path}
|
|
25
|
-
```
|
|
26
|
-
|
|
27
|
-
```
|
|
28
|
-
git -C {temp_path} sparse-checkout set --skip-checks {changed_files_from_manifest}
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
**Note:** `--skip-checks` is required because `changed_files_from_manifest` contains individual file paths (e.g., `src/core/parser.py`), not directories. Without this flag, `git sparse-checkout set` rejects non-directory entries.
|
|
32
|
-
|
|
33
|
-
When `source_ref` is a tag (e.g., `v0.5.0`), the clone targets that specific tag to ensure update-skill extracts from the same source version as the original create-skill run. When `source_ref` is `HEAD`, null, or `"local"`, no `--branch` flag is used — the clone targets the remote's default branch. This scopes the clone to only the files identified in step-02's change manifest, avoiding a full repository download.
|
|
34
|
-
|
|
35
|
-
4. **If clone succeeds:** Update the working source path to `{temp_path}` for all subsequent AST operations in this step. Proceed with the **Forge tier** extraction strategy below. Mark `ephemeral_clone_active = true` for cleanup.
|
|
36
|
-
|
|
37
|
-
5. **If clone fails (network error, auth failure, timeout):**
|
|
38
|
-
|
|
39
|
-
Warning message: "Ephemeral clone of `{source_root}` failed: {error}. Degrading to source reading (T1-low) for this run. For T1 (AST-verified) confidence, clone the repository locally and re-run [CS] Create Skill with the local path, then re-run this update."
|
|
40
|
-
|
|
41
|
-
Override the extraction strategy to Quick tier for this run. Note the degradation reason in context for the evidence report.
|
|
42
|
-
|
|
43
|
-
## Ephemeral Clone Cleanup
|
|
44
|
-
|
|
45
|
-
After extraction is complete for all files in scope (whether successful or partially failed), before presenting the extraction summary, if `ephemeral_clone_active`, delete the `{temp_path}` directory. Log: "Ephemeral source clone cleaned up." This ensures cleanup runs even if some extractions failed, as long as the step itself is still executing.
|
|
46
|
-
|
|
47
|
-
## Version Reconciliation
|
|
48
|
-
|
|
49
|
-
After the source path is accessible, check whether the source version has changed since the original skill was created. Look for the version file matching the detected language (e.g., `pyproject.toml`, `package.json`, `Cargo.toml`). If the source version differs from the current `metadata.json` version, record `source_version_detected` in context for step-06 to use when updating `metadata.json`. No warning needed here — step-06 handles the version update.
|
|
50
|
-
|
|
51
|
-
## AST Tool Unavailability (Local Source)
|
|
52
|
-
|
|
53
|
-
If AST tool is unavailable at Forge/Deep tier with local source:
|
|
54
|
-
|
|
55
|
-
Warning message: "AST tools are unavailable — extraction will use source reading (T1-low). Run [SF] Setup Forge to detect and configure AST tools for T1 confidence."
|
|
56
|
-
|
|
57
|
-
Degrade to Quick tier extraction. Note the degradation reason in context for the evidence report.
|