@skill-graph/cli 0.5.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +247 -0
- package/LICENSE +200 -0
- package/NOTICE +62 -0
- package/README.md +398 -0
- package/SKILL_GRAPH.md +443 -0
- package/bin/skill-graph.js +374 -0
- package/docs/ADOPTION.md +117 -0
- package/docs/CONFORMANCE.md +66 -0
- package/docs/PRIMER.md +384 -0
- package/docs/QUICKSTART-30MIN.md +333 -0
- package/docs/ROUTING-METRICS.md +120 -0
- package/docs/SKILL-MD-FORMAT-COMPATIBILITY.md +127 -0
- package/docs/SKILL_AUDIT_CHECKLIST.md +199 -0
- package/docs/SKILL_AUDIT_LOOP.md +195 -0
- package/docs/SKILL_METADATA_PROTOCOL.md +609 -0
- package/docs/_archived/marketplace-publication-priority-2026-05-18.md +239 -0
- package/docs/adr/0001-predicate-set.md +69 -0
- package/docs/adr/0002-json-ld-context.md +82 -0
- package/docs/adr/0003-ontoclean-rigidity-tags.md +65 -0
- package/docs/adr/0004-persistent-identifiers.md +74 -0
- package/docs/adr/0005-freshness-consolidation.md +70 -0
- package/docs/adr/0006-revise-predicate-rename.md +105 -0
- package/docs/adr/0007-audit-loop-cadence.md +99 -0
- package/docs/adr/0008-skill-surface-split-and-curation-policy.md +93 -0
- package/docs/category-consumers.md +168 -0
- package/docs/concept-map.md +194 -0
- package/docs/diagrams/drift-states.mmd +21 -0
- package/docs/diagrams/manifest-pipeline.mmd +25 -0
- package/docs/diagrams/routing-harness.mmd +41 -0
- package/docs/diagrams/starter-graph.mmd +53 -0
- package/docs/field-decision-guide.md +315 -0
- package/docs/field-rationale.md +211 -0
- package/docs/field-reference.generated.md +624 -0
- package/docs/field-reference.md +1426 -0
- package/docs/glossary.md +190 -0
- package/docs/head-noun-glossary.md +63 -0
- package/docs/images/audit-phases.png +0 -0
- package/docs/images/drift-states.png +0 -0
- package/docs/images/graded-mode.png +0 -0
- package/docs/images/manifest-pipeline.png +0 -0
- package/docs/images/routing-harness.png +0 -0
- package/docs/images/skill-anatomy.png +0 -0
- package/docs/images/starter-graph.png +0 -0
- package/docs/images/system-model.png +0 -0
- package/docs/integrations/github-actions.md +155 -0
- package/docs/manifest-field-mapping.md +443 -0
- package/docs/marketplace-publication-queue.generated.md +240 -0
- package/docs/marketplace-release-agent-prompt.md +82 -0
- package/docs/marketplace-skill-candidate-list.md +272 -0
- package/docs/marketplace-syndication.md +222 -0
- package/docs/migration-sample-review.md +155 -0
- package/docs/migrations/v4-to-v5.md +168 -0
- package/docs/migrations/v5-to-v6.md +221 -0
- package/docs/name-exceptions.yaml +37 -0
- package/docs/plans/marketplace-p1-public-migration-plan.md +41 -0
- package/docs/plans/multi-root-workspace.md +148 -0
- package/docs/plans/scripts-roadmap.md +107 -0
- package/docs/plans/v4-schema-bump.md +160 -0
- package/docs/plans/wave-2-extraction.md +122 -0
- package/docs/positioning-vs-marketplaces.md +175 -0
- package/docs/proposals/skill-audit-loop-positioning.md +160 -0
- package/docs/quality-doctrine.md +138 -0
- package/docs/recommended-skills.md +150 -0
- package/docs/research/skill-comprehension-eval-research.md +1830 -0
- package/docs/research/skill-retrieval-evidence.md +66 -0
- package/docs/skill-metadata-protocol.md +471 -0
- package/docs/skills-sh-maintainer-cleanup-request.md +80 -0
- package/examples/audits/a11y/findings.md +52 -0
- package/examples/audits/a11y/scorecard.md +21 -0
- package/examples/audits/a11y/verdict.md +44 -0
- package/examples/audits/debugging/findings.md +59 -0
- package/examples/audits/debugging/scorecard.md +22 -0
- package/examples/audits/debugging/verdict.md +33 -0
- package/examples/audits/documentation/findings.md +59 -0
- package/examples/audits/documentation/scorecard.md +22 -0
- package/examples/audits/documentation/verdict.md +33 -0
- package/examples/evals/a11y.json +140 -0
- package/examples/evals/api-design.json +52 -0
- package/examples/evals/code-review.json +52 -0
- package/examples/evals/data-modeling.json +52 -0
- package/examples/evals/database-migration.json +52 -0
- package/examples/evals/debugging.json +118 -0
- package/examples/evals/dependency-architecture.json +52 -0
- package/examples/evals/design-system-architecture.json +52 -0
- package/examples/evals/error-tracking.json +52 -0
- package/examples/evals/event-contract-design.json +52 -0
- package/examples/evals/form-ux-architecture.json +52 -0
- package/examples/evals/framework-fit-analysis.json +52 -0
- package/examples/evals/graph-audit.json +139 -0
- package/examples/evals/information-architecture.json +52 -0
- package/examples/evals/interaction-feedback.json +52 -0
- package/examples/evals/interaction-patterns.json +52 -0
- package/examples/evals/layout-composition.json +52 -0
- package/examples/evals/lint-overlay.json +117 -0
- package/examples/evals/microcopy.json +52 -0
- package/examples/evals/observability-modeling.json +52 -0
- package/examples/evals/pattern-recognition.json +96 -0
- package/examples/evals/performance-engineering.json +52 -0
- package/examples/evals/refactor.json +128 -0
- package/examples/evals/semiotics.json +52 -0
- package/examples/evals/skill-infrastructure.json +96 -0
- package/examples/evals/skill-router.json +140 -0
- package/examples/evals/skill-router.routing.json +113 -0
- package/examples/evals/system-interface-contracts.json +52 -0
- package/examples/evals/task-analysis.json +52 -0
- package/examples/evals/testing-strategy.json +118 -0
- package/examples/evals/type-safety.json +249 -0
- package/examples/evals/visual-design-foundations.json +52 -0
- package/examples/evals/webhook-integration.json +52 -0
- package/examples/exports/a11y.skill-md.md +80 -0
- package/examples/exports/debugging.skill-md.md +80 -0
- package/examples/exports/refactor.skill-md.md +78 -0
- package/examples/exports/testing-strategy.skill-md.md +81 -0
- package/examples/projects/markdown-static-site/README.md +115 -0
- package/examples/projects/markdown-static-site/skills/content-source-router/SKILL.md +131 -0
- package/examples/projects/markdown-static-site/skills/image-optimization-pipeline-config/SKILL.md +132 -0
- package/examples/projects/markdown-static-site/skills/link-rot-detection/SKILL.md +103 -0
- package/examples/projects/markdown-static-site/skills/markdown-post-frontmatter-validation/SKILL.md +133 -0
- package/examples/projects/markdown-static-site/skills/migrate-posts-to-v2-frontmatter/SKILL.md +140 -0
- package/examples/projects/saas-stripe-postgres/README.md +208 -0
- package/examples/projects/saas-stripe-postgres/db/migrations/0004_canonicalize_orders.sql +37 -0
- package/examples/projects/saas-stripe-postgres/db/schema.sql +112 -0
- package/examples/projects/saas-stripe-postgres/skills/migrate-orders-to-canonical-schema/SKILL.md +149 -0
- package/examples/projects/saas-stripe-postgres/skills/nextjs-server-action-validation/SKILL.md +154 -0
- package/examples/projects/saas-stripe-postgres/skills/payment-provider-router/SKILL.md +153 -0
- package/examples/projects/saas-stripe-postgres/skills/postgres-rls-pattern/SKILL.md +163 -0
- package/examples/projects/saas-stripe-postgres/skills/stripe-webhook-signature-verification/SKILL.md +137 -0
- package/examples/protocol/skill-metadata-template.md +301 -0
- package/examples/protocol/skills.manifest.sample.json +13245 -0
- package/examples/skill-metadata-template.md +317 -0
- package/examples/skills.manifest.sample.json +13519 -0
- package/examples/tests/v3-1-skos-fixture/SKILL.md +93 -0
- package/marketplace/README.md +17 -0
- package/marketplace/skills/a11y/SKILL.md +66 -0
- package/marketplace/skills/acid-fundamentals/SKILL.md +106 -0
- package/marketplace/skills/agent-engineering/SKILL.md +386 -0
- package/marketplace/skills/agent-eval-design/SKILL.md +55 -0
- package/marketplace/skills/ai-native-development/SKILL.md +294 -0
- package/marketplace/skills/api-design/SKILL.md +60 -0
- package/marketplace/skills/architecture-decision-records/SKILL.md +55 -0
- package/marketplace/skills/background-jobs/SKILL.md +265 -0
- package/marketplace/skills/bounded-context-mapping/SKILL.md +55 -0
- package/marketplace/skills/cap-theorem-tradeoffs/SKILL.md +127 -0
- package/marketplace/skills/client-server-boundary/SKILL.md +187 -0
- package/marketplace/skills/code-review/SKILL.md +120 -0
- package/marketplace/skills/color-system-design/SKILL.md +43 -0
- package/marketplace/skills/component-architecture/SKILL.md +126 -0
- package/marketplace/skills/compression/SKILL.md +112 -0
- package/marketplace/skills/conceptual-modeling/SKILL.md +181 -0
- package/marketplace/skills/connection-pooling/SKILL.md +105 -0
- package/marketplace/skills/constraint-awareness/SKILL.md +287 -0
- package/marketplace/skills/content-monitor/SKILL.md +209 -0
- package/marketplace/skills/context-engineering/SKILL.md +320 -0
- package/marketplace/skills/context-graph/SKILL.md +174 -0
- package/marketplace/skills/context-management/SKILL.md +174 -0
- package/marketplace/skills/context-window/SKILL.md +239 -0
- package/marketplace/skills/contract-testing/SKILL.md +120 -0
- package/marketplace/skills/cron-scheduling/SKILL.md +223 -0
- package/marketplace/skills/dark-mode-implementation/SKILL.md +47 -0
- package/marketplace/skills/data-modeling/SKILL.md +59 -0
- package/marketplace/skills/data-modeling-fundamentals/SKILL.md +117 -0
- package/marketplace/skills/database-migration/SKILL.md +429 -0
- package/marketplace/skills/debugging/SKILL.md +67 -0
- package/marketplace/skills/dependency-architecture/SKILL.md +58 -0
- package/marketplace/skills/design-module-composition/SKILL.md +43 -0
- package/marketplace/skills/design-system-architecture/SKILL.md +61 -0
- package/marketplace/skills/design-thinking/SKILL.md +44 -0
- package/marketplace/skills/diagnosis/SKILL.md +296 -0
- package/marketplace/skills/diff-analysis/SKILL.md +188 -0
- package/marketplace/skills/e2e-test-design/SKILL.md +113 -0
- package/marketplace/skills/entity-relationship-modeling/SKILL.md +218 -0
- package/marketplace/skills/epistemic-grounding/SKILL.md +112 -0
- package/marketplace/skills/error-boundary/SKILL.md +235 -0
- package/marketplace/skills/error-tracking/SKILL.md +261 -0
- package/marketplace/skills/eval-driven-development/SKILL.md +147 -0
- package/marketplace/skills/evaluation/SKILL.md +113 -0
- package/marketplace/skills/event-contract-design/SKILL.md +60 -0
- package/marketplace/skills/event-storming/SKILL.md +56 -0
- package/marketplace/skills/form-ux-architecture/SKILL.md +60 -0
- package/marketplace/skills/framework-fit-analysis/SKILL.md +59 -0
- package/marketplace/skills/frontend-architecture/SKILL.md +43 -0
- package/marketplace/skills/generative-ui/SKILL.md +118 -0
- package/marketplace/skills/graph-audit/SKILL.md +81 -0
- package/marketplace/skills/guardrails/SKILL.md +118 -0
- package/marketplace/skills/hooks-patterns/SKILL.md +185 -0
- package/marketplace/skills/http-semantics/SKILL.md +136 -0
- package/marketplace/skills/ideation/SKILL.md +41 -0
- package/marketplace/skills/indexing-strategy/SKILL.md +108 -0
- package/marketplace/skills/information-architecture/SKILL.md +59 -0
- package/marketplace/skills/integration-test-design/SKILL.md +111 -0
- package/marketplace/skills/intent-recognition/SKILL.md +136 -0
- package/marketplace/skills/interaction-feedback/SKILL.md +59 -0
- package/marketplace/skills/interaction-patterns/SKILL.md +59 -0
- package/marketplace/skills/journey-mapping/SKILL.md +41 -0
- package/marketplace/skills/keywords/SKILL.md +213 -0
- package/marketplace/skills/knowledge-modeling/SKILL.md +232 -0
- package/marketplace/skills/layout-composition/SKILL.md +59 -0
- package/marketplace/skills/linguistics/SKILL.md +429 -0
- package/marketplace/skills/lint-overlay/SKILL.md +76 -0
- package/marketplace/skills/mental-models/SKILL.md +126 -0
- package/marketplace/skills/merge-queue/SKILL.md +94 -0
- package/marketplace/skills/methodology/SKILL.md +317 -0
- package/marketplace/skills/microcopy/SKILL.md +232 -0
- package/marketplace/skills/middleware-patterns/SKILL.md +363 -0
- package/marketplace/skills/mobile-responsive-ux/SKILL.md +287 -0
- package/marketplace/skills/mutation-testing/SKILL.md +112 -0
- package/marketplace/skills/naming-conventions/SKILL.md +112 -0
- package/marketplace/skills/observability-modeling/SKILL.md +59 -0
- package/marketplace/skills/ontology-modeling/SKILL.md +67 -0
- package/marketplace/skills/owasp-security/SKILL.md +153 -0
- package/marketplace/skills/pattern-recognition/SKILL.md +472 -0
- package/marketplace/skills/performance-budgets/SKILL.md +185 -0
- package/marketplace/skills/performance-engineering/SKILL.md +58 -0
- package/marketplace/skills/performance-testing/SKILL.md +125 -0
- package/marketplace/skills/printify/SKILL.md +42 -0
- package/marketplace/skills/prioritization/SKILL.md +118 -0
- package/marketplace/skills/problem-framing/SKILL.md +41 -0
- package/marketplace/skills/problem-locating-solving/SKILL.md +203 -0
- package/marketplace/skills/project-knowledge-extraction/SKILL.md +54 -0
- package/marketplace/skills/prompt-craft/SKILL.md +134 -0
- package/marketplace/skills/prompt-injection-defense/SKILL.md +132 -0
- package/marketplace/skills/property-based-testing/SKILL.md +100 -0
- package/marketplace/skills/prototyping/SKILL.md +43 -0
- package/marketplace/skills/query-optimization/SKILL.md +144 -0
- package/marketplace/skills/real-time-updates/SKILL.md +324 -0
- package/marketplace/skills/ref-patterns/SKILL.md +284 -0
- package/marketplace/skills/refactor/SKILL.md +65 -0
- package/marketplace/skills/rendering-models/SKILL.md +142 -0
- package/marketplace/skills/replication-patterns/SKILL.md +110 -0
- package/marketplace/skills/research-synthesis/SKILL.md +41 -0
- package/marketplace/skills/route-handler-design/SKILL.md +347 -0
- package/marketplace/skills/schema-evolution/SKILL.md +140 -0
- package/marketplace/skills/security-fundamentals/SKILL.md +139 -0
- package/marketplace/skills/semantic-center/SKILL.md +194 -0
- package/marketplace/skills/semantic-relations/SKILL.md +250 -0
- package/marketplace/skills/semantics/SKILL.md +366 -0
- package/marketplace/skills/semiotics/SKILL.md +230 -0
- package/marketplace/skills/seo-strategy/SKILL.md +260 -0
- package/marketplace/skills/server-actions-design/SKILL.md +243 -0
- package/marketplace/skills/server-components-design/SKILL.md +190 -0
- package/marketplace/skills/sharding-strategy/SKILL.md +123 -0
- package/marketplace/skills/shopify/SKILL.md +42 -0
- package/marketplace/skills/skill-infrastructure/SKILL.md +320 -0
- package/marketplace/skills/skill-router/SKILL.md +71 -0
- package/marketplace/skills/skill-scaffold/SKILL.md +105 -0
- package/marketplace/skills/snapshot-testing/SKILL.md +120 -0
- package/marketplace/skills/spec-driven-development/SKILL.md +148 -0
- package/marketplace/skills/state-machine-modeling/SKILL.md +56 -0
- package/marketplace/skills/state-management/SKILL.md +134 -0
- package/marketplace/skills/streaming-architecture/SKILL.md +194 -0
- package/marketplace/skills/summarization/SKILL.md +156 -0
- package/marketplace/skills/suspense-patterns/SKILL.md +265 -0
- package/marketplace/skills/system-interface-contracts/SKILL.md +59 -0
- package/marketplace/skills/task-analysis/SKILL.md +201 -0
- package/marketplace/skills/taxonomy-design/SKILL.md +66 -0
- package/marketplace/skills/test-coverage-strategy/SKILL.md +108 -0
- package/marketplace/skills/test-doubles-design/SKILL.md +98 -0
- package/marketplace/skills/test-driven-development/SKILL.md +96 -0
- package/marketplace/skills/testing-strategy/SKILL.md +67 -0
- package/marketplace/skills/theme-system-design/SKILL.md +43 -0
- package/marketplace/skills/tool-call-flow/SKILL.md +229 -0
- package/marketplace/skills/tool-call-strategy/SKILL.md +292 -0
- package/marketplace/skills/transaction-isolation/SKILL.md +98 -0
- package/marketplace/skills/type-safety/SKILL.md +177 -0
- package/marketplace/skills/typography-system/SKILL.md +43 -0
- package/marketplace/skills/usability-testing/SKILL.md +43 -0
- package/marketplace/skills/user-research/SKILL.md +43 -0
- package/marketplace/skills/vercel-composition-patterns/SKILL.md +157 -0
- package/marketplace/skills/version-control/SKILL.md +233 -0
- package/marketplace/skills/visual-design-foundations/SKILL.md +59 -0
- package/marketplace/skills/visual-hierarchy/SKILL.md +43 -0
- package/marketplace/skills/webhook-integration/SKILL.md +331 -0
- package/marketplace/skills/writing-humanizer/SKILL.md +380 -0
- package/package.json +67 -0
- package/schemas/manifest.schema.json +811 -0
- package/schemas/manifest.v2.schema.json +164 -0
- package/schemas/manifest.v3.schema.json +758 -0
- package/schemas/manifest.v4.schema.json +755 -0
- package/schemas/manifest.v5.schema.json +755 -0
- package/schemas/manifest.v6.schema.json +811 -0
- package/schemas/skill.context.jsonld +279 -0
- package/schemas/skill.schema.json +919 -0
- package/schemas/skill.v2.schema.json +201 -0
- package/schemas/skill.v3.schema.json +827 -0
- package/schemas/skill.v4.schema.json +822 -0
- package/schemas/skill.v5.schema.json +830 -0
- package/schemas/skill.v6.schema.json +946 -0
- package/schemas/vocabulary/keywords.json +180 -0
- package/schemas/vocabulary/workspace_tags.json +23 -0
- package/scripts/__tests__/migrate-skill-v2-to-v3.test.js +161 -0
- package/scripts/__tests__/migrate-skill-v3-to-v4.test.js +158 -0
- package/scripts/__tests__/test-export-parser-drift.js +149 -0
- package/scripts/__tests__/test-marketplace-export.js +114 -0
- package/scripts/__tests__/test-router-paths.js +82 -0
- package/scripts/__tests__/test-stability-promotion.js +244 -0
- package/scripts/__tests__/test-v3-1-alias-contract.js +109 -0
- package/scripts/__tests__/test-v3-1-skos-runtime.js +116 -0
- package/scripts/backfill-schema-version.js +198 -0
- package/scripts/build-field-reference.js +160 -0
- package/scripts/build-retrieval-baseline.js +511 -0
- package/scripts/check-markdown-links.js +211 -0
- package/scripts/check-protocol-consistency.js +979 -0
- package/scripts/export-marketplace-skills.js +610 -0
- package/scripts/export-skill.js +374 -0
- package/scripts/generate-manifest.js +787 -0
- package/scripts/lib/alias-contract.js +83 -0
- package/scripts/lib/audit-prompt-builder.js +771 -0
- package/scripts/lib/mock-grader.js +134 -0
- package/scripts/lib/parse-frontmatter.js +429 -0
- package/scripts/lib/roots.js +119 -0
- package/scripts/lint/check-archetype-sections.js +185 -0
- package/scripts/lint/check-category-enum.js +83 -0
- package/scripts/lint/check-routing-eval.js +146 -0
- package/scripts/lint/check-routing-quality.js +211 -0
- package/scripts/lint/check-stability-promotion.js +220 -0
- package/scripts/lint/format-code-frame.js +206 -0
- package/scripts/marketplace-install.js +125 -0
- package/scripts/migrate-category-to-enum.js +169 -0
- package/scripts/migrate-skill-v2-to-v3.js +424 -0
- package/scripts/migrate-skill-v3-to-v4.js +200 -0
- package/scripts/migrate-skill-v5-to-v6.js +304 -0
- package/scripts/restructure-by-category.js +85 -0
- package/scripts/seed-publication-classification.js +282 -0
- package/scripts/skill-audit.js +893 -0
- package/scripts/skill-graph-drift.js +483 -0
- package/scripts/skill-graph-route.js +766 -0
- package/scripts/skill-graph-routing-eval.js +393 -0
- package/scripts/skill-lint.js +1317 -0
- package/scripts/skill-overlap.js +213 -0
- package/scripts/verify-skill-md-export.js +201 -0
|
@@ -0,0 +1,320 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: skill-infrastructure
|
|
3
|
+
description: "Use when designing the deterministic health-tooling layer for a skill library, diagnosing why an existing library is decaying invisibly, deciding which categories of automated check to add (inventory, protocol consistency, conflict detection, routing health, drift sentinel), debugging eval-threshold violations across many skills at once, or auditing whether a skill-system has the safety nets a production library needs. Covers the five categories of skill-health tooling, the library-as-database mental model, eval quality patterns (minimum thresholds, contradiction-check pattern, negative-expectation requirement), maintenance workflows triggered by batch skill changes, and the anti-patterns that cause skill libraries to decay until agents loading them get worse over time. Do NOT use for authoring an individual SKILL.md (use `skill-scaffold`), for running the conformance audit on the Skill Graph repo itself (use `graph-audit`), or for general lint rule selection across a codebase (use `lint-overlay`)."
|
|
4
|
+
license: MIT
|
|
5
|
+
compatibility: "Library- and harness-agnostic. Patterns apply to any skill-style library (Skill Graph, Claude skills, Cursor rules, custom in-house skill systems). Specific tool names in this skill (skill-lint, generate-manifest, routing-eval, drift-sentinel) are concrete examples from the Skill Graph reference implementation -- substitute your library's equivalents."
|
|
6
|
+
allowed-tools: Read Grep Bash Edit Write
|
|
7
|
+
metadata:
|
|
8
|
+
metadata: "{\"schema_version\":6,\"version\":\"1.0.0\",\"type\":\"capability\",\"category\":\"agent\",\"domain\":\"agent/skill-system\",\"scope\":\"portable\",\"owner\":\"skill-graph-maintainer\",\"freshness\":\"2026-05-13\",\"drift_check\":\"{\\\\\\\"last_verified\\\\\\\":\\\\\\\"2026-05-13\\\\\\\",\\\\\\\"truth_source_hashes\\\\\\\":{\\\\\\\"package.json\\\\\\\":\\\\\\\"7a3410a004aea78a2065092e289c0f3cf3c082298804dda6c5829eff22c14b62\\\\\\\",\\\\\\\"bin/skill-graph.js\\\\\\\":\\\\\\\"113a1e01ac7276ac1b5d77a1c32e35a73113da93fc33cfd0caf6db842d2d679f\\\\\\\",\\\\\\\"scripts/skill-lint.js\\\\\\\":\\\\\\\"3a78f75f8921542b91dc619cd41bde29bf379de3c16bdcf3653c854ecbe9fa29\\\\\\\",\\\\\\\"scripts/lib/roots.js\\\\\\\":\\\\\\\"e742efa57b6c33ff1c87034b16a689d1499f6d53c1e6b740f3e9783db7fd557f\\\\\\\",\\\\\\\"scripts/check-protocol-consistency.js\\\\\\\":\\\\\\\"0ff39406d36e7a9e51c176f657f4f426d8bd5a3fe6411d28b9e9a93dc7d89f29\\\\\\\",\\\\\\\"scripts/generate-manifest.js\\\\\\\":\\\\\\\"9d7bbbdae440fdb1763d61ffa7bda10c9efae92359d1c2139d0e971582d59e0e\\\\\\\",\\\\\\\"scripts/skill-graph-drift.js\\\\\\\":\\\\\\\"6b69c25b59c16b477a377e5ab40adb6ff30f72d5a12947772053a6cd16b1f409\\\\\\\",\\\\\\\"scripts/skill-overlap.js\\\\\\\":\\\\\\\"ed642cbc677cc76ec1321300b37d6752337b6b5541c7a9f558fd315d6f934e4b\\\\\\\",\\\\\\\"scripts/skill-graph-routing-eval.js\\\\\\\":\\\\\\\"fffac2858863662bde6bc54c56bb77a219ae93f626e0c8d5886566f998181deb\\\\\\\",\\\\\\\"docs/manifest-field-mapping.md\\\\\\\":\\\\\\\"aca0b7f2d4631be24a3e7daed1a1d207b488f253164a7d514b9db7af21c6177f\\\\\\\"}}\",\"eval_artifacts\":\"planned\",\"eval_state\":\"unverified\",\"routing_eval\":\"absent\",\"stability\":\"experimental\",\"keywords\":\"[\\\\\\\"skill library health\\\\\\\",\\\\\\\"skill system tooling\\\\\\\",\\\\\\\"skill library decay\\\\\\\",\\\\\\\"skill library maintenance\\\\\\\",\\\\\\\"skill census\\\\\\\",\\\\\\\"skill inventory\\\\\\\",\\\\\\\"frontmatter validation\\\\\\\",\\\\\\\"imperative conflict\\\\\\\",\\\\\\\"skill overlap detection\\\\\\\",\\\\\\\"skill conflict\\\\\\\",\\\\\\\"routing gap\\\\\\\",\\\\\\\"routing miss\\\\\\\",\\\\\\\"routing health\\\\\\\",\\\\\\\"eval threshold\\\\\\\",\\\\\\\"eval minimum\\\\\\\",\\\\\\\"contradiction check\\\\\\\",\\\\\\\"negative expectation\\\\\\\",\\\\\\\"drift sentinel\\\\\\\",\\\\\\\"truth source hash\\\\\\\",\\\\\\\"mirror parity\\\\\\\",\\\\\\\"skill graph health\\\\\\\",\\\\\\\"production skill library\\\\\\\",\\\\\\\"skill linter\\\\\\\",\\\\\\\"skill quality gate\\\\\\\",\\\\\\\"phantom ref\\\\\\\"]\",\"examples\":\"[\\\\\\\"our skill library is growing and we're getting silent decay — eval counts dropping, conflicts emerging — what tooling should we add?\\\\\\\",\\\\\\\"two of our skills give opposite instructions for the same function — how do we detect this automatically?\\\\\\\",\\\\\\\"we keep getting skill-router misses on real user queries — how do we surface and close routing gaps?\\\\\\\",\\\\\\\"design a health-check pipeline for a 200-skill library that runs in CI\\\\\\\",\\\\\\\"what's a reasonable minimum eval count per skill, and how do we enforce it?\\\\\\\",\\\\\\\"our skill mirror in `.claude/skills` keeps drifting from the source — what's the parity check?\\\\\\\",\\\\\\\"we want to add a contradiction-check eval pattern — what does it look like and when do we use it?\\\\\\\",\\\\\\\"skill-overlap-detector flagged 12 imperative conflicts — how do we triage which to fix vs suppress?\\\\\\\"]\",\"anti_examples\":\"[\\\\\\\"scaffold a new SKILL.md for our team's deploy procedure\\\\\\\",\\\\\\\"audit this Skill Graph repo for schema conformance and dangling relation targets\\\\\\\",\\\\\\\"the manifest sample drifted from the generator — find the mismatch\\\\\\\",\\\\\\\"improve this prompt's wording to get better outputs\\\\\\\",\\\\\\\"review this AI-generated PR for correctness\\\\\\\",\\\\\\\"set up ESLint for our TypeScript repo\\\\\\\",\\\\\\\"draft an architecture note explaining why we chose Postgres\\\\\\\"]\",\"relations\":\"{\\\\\\\"boundary\\\\\\\":[{\\\\\\\"skill\\\\\\\":\\\\\\\"skill-scaffold\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"skill-scaffold owns authoring methodology for one new SKILL.md; skill-infrastructure owns the deterministic health-tooling layer that watches the entire library after authoring\\\\\\\"},{\\\\\\\"skill\\\\\\\":\\\\\\\"graph-audit\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"graph-audit is the operational audit of one specific library (Skill Graph), scope: codebase; skill-infrastructure is the portable discipline of designing health tooling for any skill library\\\\\\\"},{\\\\\\\"skill\\\\\\\":\\\\\\\"lint-overlay\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"lint-overlay covers lint-rule selection and gate placement for general codebases; skill-infrastructure covers the skill-system-specific tooling category that includes lint but extends to overlap, routing-gap, drift, and mirror-parity\\\\\\\"}],\\\\\\\"related\\\\\\\":[\\\\\\\"skill-scaffold\\\\\\\",\\\\\\\"graph-audit\\\\\\\",\\\\\\\"testing-strategy\\\\\\\"],\\\\\\\"verify_with\\\\\\\":[\\\\\\\"testing-strategy\\\\\\\",\\\\\\\"code-review\\\\\\\"]}\",\"grounding\":\"{\\\\\\\"domain_object\\\\\\\":\\\\\\\"Deterministic health tooling for Skill Graph libraries\\\\\\\",\\\\\\\"grounding_mode\\\\\\\":\\\\\\\"hybrid\\\\\\\",\\\\\\\"truth_sources\\\\\\\":[\\\\\\\"package.json\\\\\\\",\\\\\\\"bin/skill-graph.js\\\\\\\",\\\\\\\"scripts/skill-lint.js\\\\\\\",\\\\\\\"scripts/lib/roots.js\\\\\\\",\\\\\\\"scripts/check-protocol-consistency.js\\\\\\\",\\\\\\\"scripts/generate-manifest.js\\\\\\\",\\\\\\\"scripts/skill-graph-drift.js\\\\\\\",\\\\\\\"scripts/skill-overlap.js\\\\\\\",\\\\\\\"scripts/skill-graph-routing-eval.js\\\\\\\",\\\\\\\"docs/manifest-field-mapping.md\\\\\\\"],\\\\\\\"failure_modes\\\\\\\":[\\\\\\\"health_tooling_categories_missing_from_ci\\\\\\\",\\\\\\\"protocol_mapping_drift\\\\\\\",\\\\\\\"eval_thresholds_become_self_attested\\\\\\\",\\\\\\\"overlap_or_drift_checks_not_run_after_batch_changes\\\\\\\"],\\\\\\\"evidence_priority\\\\\\\":\\\\\\\"repo_code_first\\\\\\\"}\",\"portability\":\"{\\\\\\\"readiness\\\\\\\":\\\\\\\"scripted\\\\\\\",\\\\\\\"targets\\\\\\\":[\\\\\\\"skill-md\\\\\\\"]}\",\"lifecycle\":\"{\\\\\\\"stale_after_days\\\\\\\":90,\\\\\\\"review_cadence\\\\\\\":\\\\\\\"quarterly\\\\\\\"}\",\"skill_graph_source_repo\":\"https://github.com/jacob-balslev/skill-graph\",\"skill_graph_protocol\":\"Skill Metadata Protocol v5\",\"skill_graph_project\":\"Skill Graph\",\"skill_graph_canonical_skill\":\"skills/skill-infrastructure/SKILL.md\"}"
|
|
9
|
+
skill_graph_source_repo: "https://github.com/jacob-balslev/skill-graph"
|
|
10
|
+
skill_graph_protocol: Skill Metadata Protocol v4
|
|
11
|
+
skill_graph_project: Skill Graph
|
|
12
|
+
skill_graph_canonical_skill: skills/skill-infrastructure/SKILL.md
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Skill Infrastructure
|
|
16
|
+
|
|
17
|
+
## Coverage
|
|
18
|
+
|
|
19
|
+
- The library-as-database mental model: skill-infrastructure as the linter, integrity checker, and query planner for a SKILL.md library
|
|
20
|
+
- Why deterministic, zero-LLM tooling is mandatory for skill libraries (trustworthy enough for CI gates; LLM-based health checks are circular)
|
|
21
|
+
- The five categories of skill-health tooling: (1) inventory and frontmatter validation, (2) protocol consistency, (3) conflict detection (overlap, imperative, code duplication, heading overlap), (4) routing health (gap analysis, miss tracking), (5) drift detection (truth-source hashing, mirror parity)
|
|
22
|
+
- Eval quality patterns: minimum eval threshold per skill, the contradiction-check eval type, the negative-expectation requirement, valid eval-type taxonomy
|
|
23
|
+
- Imperative conflict detection: the same-target-opposite-polarity rule, three-check false-positive suppression, when conflicts indicate scope-tightening vs merging
|
|
24
|
+
- Routing gap analysis: how to read a "routing-misses" log, how to distinguish keyword gaps from skill content gaps, signal-hygiene rules to suppress noise
|
|
25
|
+
- Maintenance workflows: when to run a full health check, the order in which to run the categories, what to fix before what
|
|
26
|
+
- Anti-patterns that cause invisible decay: dirty-tree manifest writes, deletion-as-conflict-resolution, eval-renumbering during cleanup, scope misuse to mask threshold violations
|
|
27
|
+
- The verification gate before any batch skill commit: every category clean, every new skill meets eval minimums, every routing change is reflected in the manifest
|
|
28
|
+
- Package and workspace-root integrity: the npm CLI entrypoint must dispatch to the same scripts as local development while resolving schemas from the package and skills/manifests from the caller workspace
|
|
29
|
+
|
|
30
|
+
## Philosophy
|
|
31
|
+
|
|
32
|
+
A skill library is only as useful as its worst skill. When agents load stale, conflicting, poorly-routed, or mirror-drifted skills, they get *worse* at tasks — not better. A skill library at scale (50+, certainly 200+) decays invisibly: eval counts drift below minimums, keyword maps miss whole product areas, mirror copies fall out of sync, and two skills start giving opposite instructions for the same function.
|
|
33
|
+
|
|
34
|
+
The infrastructure exists to catch that decay deterministically, with **zero LLM calls**. Every health check reads files, parses structure, and computes — it does not reason, infer, or hallucinate. That makes the output trustworthy enough to use in CI-style gates. An LLM-based health check is circular: the same probabilistic component that produces the skill library cannot reliably grade its own output.
|
|
35
|
+
|
|
36
|
+
> If the skill library is a codebase, skill-infrastructure is its combined linter, type-checker, and dead-code detector — always-on, zero-model, CI-safe.
|
|
37
|
+
|
|
38
|
+
The maintenance commitment is: run a full health check after any batch skill work, fix threshold violations before they reach the gate, and document conflicts before the agents loading those skills are affected.
|
|
39
|
+
|
|
40
|
+
## The Library-as-Database Mental Model
|
|
41
|
+
|
|
42
|
+
Treat the skill library as a database and skill-infrastructure as its query planner, integrity checker, and migration-safety net combined.
|
|
43
|
+
|
|
44
|
+
| Database concept | Skill-library equivalent |
|
|
45
|
+
|---|---|
|
|
46
|
+
| Schema | SKILL.md frontmatter schema (JSON Schema or equivalent) |
|
|
47
|
+
| Constraints | Required fields, eval thresholds, valid enum values |
|
|
48
|
+
| Foreign keys | `relations.boundary[]`, `relations.related[]` targets — must point at real skills |
|
|
49
|
+
| Indexes | Manifest, routing-keyword map, path-glob inverse index |
|
|
50
|
+
| Integrity check | Protocol-consistency tooling (cross-schema parity, sample correctness, generated-field parity) |
|
|
51
|
+
| Query planner | Skill router (matches user prompt → skill activation) |
|
|
52
|
+
| Replication lag | Mirror parity (`.claude/skills`, `.agents/skills`, harness-specific copies) |
|
|
53
|
+
| Dead code | Skills with broken relation targets, phantom routing entries |
|
|
54
|
+
|
|
55
|
+
Every script in this domain reads files and computes — it does not reason. Output is deterministic, reproducible, and safe to embed in CI gates.
|
|
56
|
+
|
|
57
|
+
## The Five Categories of Skill-Health Tooling
|
|
58
|
+
|
|
59
|
+
A production skill library needs all five. Missing any one allows a class of decay through.
|
|
60
|
+
|
|
61
|
+
### 1. Inventory and Frontmatter Validation
|
|
62
|
+
|
|
63
|
+
Walks the skill tree, parses every SKILL.md's frontmatter, and validates against the schema.
|
|
64
|
+
|
|
65
|
+
| Check | Why it matters |
|
|
66
|
+
|---|---|
|
|
67
|
+
| Required fields present (`name`, `description`, `version`, `type`, `scope`, `owner`, `freshness`) | Missing fields break the manifest generator and the router |
|
|
68
|
+
| Field-value enums valid (`type`, `scope`, `eval_state`) | Drift in valid values leaks invalid skills into routing |
|
|
69
|
+
| Description-length and quality (≥ 100 chars, contains "Use when X / Do NOT use for Y") | Short or vague descriptions degrade router precision |
|
|
70
|
+
| Eval-count per skill (warn at < 7, error at < 3 for non-reference skills) | Under-evaluated skills regress quality undetected |
|
|
71
|
+
| Reference-path resolution (every `references/X.md` cited in frontmatter actually exists) | Dangling references mislead the agent at activation time |
|
|
72
|
+
| Scope-grounding consistency (`scope: codebase` requires populated `grounding` block) | Codebase claims without grounding can hallucinate file paths |
|
|
73
|
+
|
|
74
|
+
**Reference implementation:** `scripts/skill-lint.js` in this repo runs T5 per-skill lint covering most of these.
|
|
75
|
+
|
|
76
|
+
### 2. Protocol Consistency
|
|
77
|
+
|
|
78
|
+
Cross-checks that the schema, generator, sample manifest, and field-reference docs all agree.
|
|
79
|
+
|
|
80
|
+
| Check | Failure mode if missing |
|
|
81
|
+
|---|---|
|
|
82
|
+
| Cross-schema parity (versioned schemas describe the same fields) | Schema bumps silently drop fields |
|
|
83
|
+
| Authored-to-generated parity (frontmatter fields match what the generator emits) | Generator drift breaks downstream consumers |
|
|
84
|
+
| Sample manifest matches generator output | Sample stales as skills are added |
|
|
85
|
+
| Generated field-reference matches authored field-reference | Documentation drift hides field changes |
|
|
86
|
+
| Truth invariants on example evals (e.g. truth-source line ranges still resolve) | Eval expectations point at non-existent code |
|
|
87
|
+
|
|
88
|
+
**Reference implementation:** `scripts/check-protocol-consistency.js` runs C1–C7 protocol checks.
|
|
89
|
+
|
|
90
|
+
### 3. Conflict Detection
|
|
91
|
+
|
|
92
|
+
Pairwise comparison across all active skills to surface duplication, contradiction, and structural overlap.
|
|
93
|
+
|
|
94
|
+
| Dimension | Method | Signal |
|
|
95
|
+
|---|---|---|
|
|
96
|
+
| Description similarity | Jaccard on word bigrams | > 0.4 = potential duplicate |
|
|
97
|
+
| Heading overlap | Shared H2/H3 headings excluding boilerplate | High overlap = structural duplication |
|
|
98
|
+
| Code duplication | Identical fenced code blocks > 30 chars | Copy-paste anti-pattern |
|
|
99
|
+
| Imperative conflicts | Same target, opposite polarity (ALWAYS vs NEVER) | Agents get contradictory instructions |
|
|
100
|
+
|
|
101
|
+
**Imperative extraction patterns:**
|
|
102
|
+
|
|
103
|
+
- Positive: `\b(?:always|must|required|mandatory)\b\s+(.{10,80})/gi`, `\buse\s+(.{5,60})\s+(?:for|when|instead)/gi`
|
|
104
|
+
- Negative: `\b(?:never|do\s+not|don't|must\s+not|prohibited)\b\s+(.{10,80})/gi`, `\bdo\s+NOT\s+use\s+(.{5,60})/gi`
|
|
105
|
+
|
|
106
|
+
**Three-check false-positive suppression** for the most common phantom-conflict sources (e.g. `Do NOT use X` matching as positive `use X`):
|
|
107
|
+
|
|
108
|
+
1. **Lookbehind check** — if `not`, `never`, `don't`, or `cannot` appears in the 20 characters before a positive match, suppress it
|
|
109
|
+
2. **Within-match check** — if `not` appears between the start of the match and the target identifier (extracted via backticks), suppress it
|
|
110
|
+
3. **Same-line dedup** — if the target was already extracted as a negative on the current line, suppress any positive extraction for that target
|
|
111
|
+
|
|
112
|
+
**When a conflict is real vs spurious:**
|
|
113
|
+
|
|
114
|
+
| Conflict type | Resolution |
|
|
115
|
+
|---|---|
|
|
116
|
+
| Phantom ref | Routing config maps to a non-existent skill — delete the stale entry |
|
|
117
|
+
| Code duplication | One skill owns the example; the other links to it |
|
|
118
|
+
| Heading overlap | Acceptable for structural templates; review only the differentiating content |
|
|
119
|
+
| Imperative conflict (real) | Scope the instructions more precisely (`always use X` → `always use X in app code`); rarely merge skills |
|
|
120
|
+
| Imperative conflict (spurious) | Un-backticking an identifier hides a real boundary ambiguity — fix the skill wording instead |
|
|
121
|
+
|
|
122
|
+
### 4. Routing Health
|
|
123
|
+
|
|
124
|
+
Reads the routing-miss log and the routing config to identify where the keyword map fails.
|
|
125
|
+
|
|
126
|
+
| Input | Purpose |
|
|
127
|
+
|---|---|
|
|
128
|
+
| Routing-misses log (queries that matched zero skills) | Surface keyword gaps — words users typed that never reached a skill |
|
|
129
|
+
| Eval-history log with `failure_category` | Surface skills failing for reasons routing can fix (`skill_not_activated`, `wrong_answer`) |
|
|
130
|
+
| Active routing config (`keywordMap`, `labelMap`) | Authoritative truth for what currently routes |
|
|
131
|
+
|
|
132
|
+
**Output sections:**
|
|
133
|
+
|
|
134
|
+
1. **Keyword gaps** — words appearing in routing-misses but absent from the active keyword map, sorted by frequency
|
|
135
|
+
2. **Eval failure breakdown** — skills with 2+ failures, grouped by failure category
|
|
136
|
+
3. **Suggested actions** — `add_keyword` for frequent gaps; `improve_skill` for skills failing for content reasons
|
|
137
|
+
|
|
138
|
+
**Signal hygiene rules** (suppress these from the suggestion list):
|
|
139
|
+
|
|
140
|
+
- Low-signal tokens (`v`, `v4`, `skill`, `start`, `events`, `daily`, `log`, `status`, `health`)
|
|
141
|
+
- Single-word misses already covered by an existing multi-word phrase (e.g. `error` should not surface if `error recovery` is already mapped)
|
|
142
|
+
- Stale historical misses where the full query already routes under the current config
|
|
143
|
+
- Synthetic no-match probes used to verify the router's miss path
|
|
144
|
+
- Bundle entries (only direct `keywordMap` entries participate in free-text matching)
|
|
145
|
+
|
|
146
|
+
**Reference implementation:** the routing harness (`scripts/skill-graph-routing-eval.js` in this repo) provides the negative case — verifying that a skill's authored `examples` actually route to it. A real routing-gap report is the symmetric positive case (verifying that real user prompts reach a skill).
|
|
147
|
+
|
|
148
|
+
### 5. Drift Detection
|
|
149
|
+
|
|
150
|
+
Detects when the *world the skill claims* has changed without the skill being updated.
|
|
151
|
+
|
|
152
|
+
| Drift type | Detection |
|
|
153
|
+
|---|---|
|
|
154
|
+
| Truth-source drift | `drift_check.truth_source_hashes` records SHA-256 of cited files; rerun hashes; mismatch = drift |
|
|
155
|
+
| Mirror parity | Hash every `<library>/skills/<name>/SKILL.md` and every `<harness-mirror>/skills/<name>/SKILL.md`; mismatch = drift |
|
|
156
|
+
| Stale `freshness` date | Skill has not been touched in > `lifecycle.stale_after_days`; surface for review |
|
|
157
|
+
| Tracker → skill drift | Issue tracker references a skill that no longer exists; or skill references a closed issue |
|
|
158
|
+
|
|
159
|
+
**Reference implementation:** `scripts/skill-graph-drift.js` in this repo records and verifies truth-source hashes against the live files.
|
|
160
|
+
|
|
161
|
+
## Eval Quality Patterns
|
|
162
|
+
|
|
163
|
+
### The minimum-threshold rule
|
|
164
|
+
|
|
165
|
+
Every active skill should have at least 7 evals. Most healthy skills carry 9–15 covering happy paths, edge cases, anti-patterns, and contradiction checks. The threshold is a *floor*, not a goal.
|
|
166
|
+
|
|
167
|
+
**Recommended enforcement:**
|
|
168
|
+
|
|
169
|
+
- **Error** if `eval_count < 3` (or **warn** for `scope: reference` skills, where the file is documentation more than testable behaviour)
|
|
170
|
+
- **Warn** if `eval_count < 7`
|
|
171
|
+
|
|
172
|
+
Below 7, the skill is statistically under-tested. Below 3, it is effectively un-evaluated.
|
|
173
|
+
|
|
174
|
+
### The contradiction-check eval type
|
|
175
|
+
|
|
176
|
+
A `contradiction-check` eval tests that the agent correctly handles a documented exception or boundary condition that a simpler reading of the skill would mishandle. Format:
|
|
177
|
+
|
|
178
|
+
```json
|
|
179
|
+
{
|
|
180
|
+
"id": 5,
|
|
181
|
+
"type": "contradiction-check",
|
|
182
|
+
"grounding": "repo-specific",
|
|
183
|
+
"difficulty": "adversarial",
|
|
184
|
+
"prompt": "Skill A says always use the scoped fetcher. One service uses the unscoped fetcher with an inline comment. Is this wrong?",
|
|
185
|
+
"expected_output": "Not wrong — the unscoped fetcher with an inline justification comment is the documented exception for system-level reads.",
|
|
186
|
+
"expectations": [
|
|
187
|
+
"Correctly identifies the documented exception from the skill's anti-patterns table",
|
|
188
|
+
"Does NOT flag the usage as a bug without reading the inline comment",
|
|
189
|
+
"Distinguishes a system-level exception from a regular violation in application code"
|
|
190
|
+
]
|
|
191
|
+
}
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
Use a contradiction-check when:
|
|
195
|
+
|
|
196
|
+
- A skill has a documented exception that overrides the general rule
|
|
197
|
+
- Two adjacent skills appear to contradict each other but actually operate in different scopes
|
|
198
|
+
- A historical false positive or conflict was resolved and the resolution is non-obvious
|
|
199
|
+
|
|
200
|
+
### The negative-expectation requirement
|
|
201
|
+
|
|
202
|
+
Every eval case with an `expectations` array must include at least one expectation containing `does not`, `never`, `must not`, `should not`, or `do not`. Without this, evals become pure happy-path tests and miss the failure modes that motivated the skill.
|
|
203
|
+
|
|
204
|
+
**Recommended enforcement:** the inventory tool flags any eval missing this pattern in a `missingNegativeEvalIds` field of its report.
|
|
205
|
+
|
|
206
|
+
### Valid eval types
|
|
207
|
+
|
|
208
|
+
| Type | When to use |
|
|
209
|
+
|---|---|
|
|
210
|
+
| `knowledge` | Tests a factual claim or pattern from the skill |
|
|
211
|
+
| `contradiction-check` | Tests documented exceptions and boundary conditions |
|
|
212
|
+
| `browser` | Tests a browser-executable interaction (requires running server) |
|
|
213
|
+
| `edge-case` | Tests unusual inputs or rare conditions |
|
|
214
|
+
| `business-model` | Tests domain-specific logic (e.g. SaaS billing rules, e-commerce fulfilment) |
|
|
215
|
+
| `negative` | Tests refusal or correct non-action |
|
|
216
|
+
|
|
217
|
+
## Maintenance Workflows
|
|
218
|
+
|
|
219
|
+
### The full health check sequence
|
|
220
|
+
|
|
221
|
+
Run after any batch skill work — creating ≥ 3 skills, changing routing config, or modifying skill content across multiple files. Order matters:
|
|
222
|
+
|
|
223
|
+
```bash
|
|
224
|
+
# 1. Inventory snapshot — discover invalid frontmatter, missing fields, broken refs
|
|
225
|
+
<inventory tool>
|
|
226
|
+
|
|
227
|
+
# 2. Protocol consistency — cross-schema, manifest, generated-doc parity
|
|
228
|
+
<protocol-consistency tool>
|
|
229
|
+
|
|
230
|
+
# 3. Conflict detection — overlap, imperatives, code duplication
|
|
231
|
+
<conflict tool> --conflicts
|
|
232
|
+
|
|
233
|
+
# 4. Routing gaps (last 7 days)
|
|
234
|
+
<routing-gap tool> --since 7d
|
|
235
|
+
|
|
236
|
+
# 5. Drift sentinel — truth-source hashes, mirror parity
|
|
237
|
+
<drift tool>
|
|
238
|
+
|
|
239
|
+
# 6. Regenerate manifest + index from the now-clean tree
|
|
240
|
+
<manifest generator> --write
|
|
241
|
+
```
|
|
242
|
+
|
|
243
|
+
Review the output of steps 1–5 before running step 6. The manifest and index should reflect a clean state — never write a manifest from a dirty tree.
|
|
244
|
+
|
|
245
|
+
### Adding evals to an existing skill
|
|
246
|
+
|
|
247
|
+
1. Read the skill's existing eval set
|
|
248
|
+
2. Identify gaps: are all eval types represented? Are there contradiction-checks for documented exceptions?
|
|
249
|
+
3. Draft new cases with `id`, `prompt`, `expected_output`, `expectations`, `type`, `grounding`, `difficulty`
|
|
250
|
+
4. Ensure every new case has at least one `does not / never / must not` expectation
|
|
251
|
+
5. **Append** to the eval array — never renumber existing IDs (eval-history logs reference numeric IDs)
|
|
252
|
+
6. Re-run the inventory tool and verify the skill no longer appears in `belowMinimum` or `missingNegativeEvalIds`
|
|
253
|
+
|
|
254
|
+
### Triaging imperative conflicts
|
|
255
|
+
|
|
256
|
+
1. Run the conflict tool with `--conflicts --json` to get the structured list
|
|
257
|
+
2. For each conflict, identify which skill's instruction is more precise or more scoped
|
|
258
|
+
3. Resolve by *narrowing scope*, not by deleting the instruction:
|
|
259
|
+
- "Use X" → "Use X in app code" (narrower)
|
|
260
|
+
- "Always use X" → "Always use X for tenant-scoped data" (specific)
|
|
261
|
+
- Move the negative instruction to an anti-patterns table row that does not start with "Use"
|
|
262
|
+
- Add an inline qualifier (e.g. `// system: <reason>`) that the detector can distinguish
|
|
263
|
+
4. Re-run `--conflicts` to confirm the conflict is gone
|
|
264
|
+
5. Document the resolution in a conflict log so future audits know it was deliberate
|
|
265
|
+
|
|
266
|
+
### Fixing invalid frontmatter
|
|
267
|
+
|
|
268
|
+
| Problem | Fix |
|
|
269
|
+
|---|---|
|
|
270
|
+
| `scope` field absent | Add the correct value (`portable` / `reference` / `codebase`) |
|
|
271
|
+
| `drift_check.last_verified` absent or stale | Add or update to today's ISO date |
|
|
272
|
+
| `eval_artifacts` absent | Set to `present` if eval files exist, `planned` if intended, `none` otherwise |
|
|
273
|
+
| `keywords` empty or short | Add 5–15 natural-language phrases users would actually type |
|
|
274
|
+
| `description` too short | Quote it; require ≥ 100 chars; include trigger phrases and a "Do NOT use for X (use Y)" exclusion |
|
|
275
|
+
| `version` absent | Set to `1.0.0` for new skills; bump per semver on substantive change |
|
|
276
|
+
|
|
277
|
+
After editing, re-run the inventory tool and confirm the skill no longer appears in the invalid-frontmatter list.
|
|
278
|
+
|
|
279
|
+
## Anti-Patterns
|
|
280
|
+
|
|
281
|
+
| Anti-pattern | Why it fails | What to do instead |
|
|
282
|
+
|---|---|---|
|
|
283
|
+
| Running manifest-write on a dirty skill tree | Writes a broken index that downstream consumers trust | Fix all inventory errors first, then regenerate |
|
|
284
|
+
| Resolving conflicts by deleting one instruction | Removes useful guidance the agent needs in the right scope | Narrow the scope of the instruction instead |
|
|
285
|
+
| Adding evals without negative expectations | Census flags them; they test only happy paths | Every eval must have at least one `does not / never / must not` expectation |
|
|
286
|
+
| Renumbering eval IDs during cleanup | Breaks eval-history references that use numeric IDs | Always append; never renumber |
|
|
287
|
+
| Un-backticking an identifier to suppress a false positive when the conflict is real | Hides a real boundary ambiguity | Fix the skill wording to accurately reflect the scope |
|
|
288
|
+
| Adding routing keywords without a real skill to route to | Creates more broken mappings | Only add keywords that map to an existing skill |
|
|
289
|
+
| Treating heading overlap as always wrong | Structural-template skills (model profiles, integration patterns) legitimately share structure | Review the differentiating content instead of restructuring |
|
|
290
|
+
| Using `scope: reference` to mask threshold violations | Reference scope downgrades eval-error to warning, hiding real gaps | Use `reference` only for genuine reference docs (contracts, schemas), never to dodge threshold enforcement |
|
|
291
|
+
| Producing a thin audit summary after a multi-hour session | A two-hour audit that outputs "5 entities missing evals" has performed a *census*, not an *audit* — 95% of the invested tokens are wasted | Census counts things; audits verify claims against evidence. Every audited skill needs per-claim verdicts (verified / drift) referencing specific file:line evidence |
|
|
292
|
+
| Running a "skill loop" without a minimum-output specification | Agents read methodology sections but skip output-format sections, then produce free-form summaries | Before any audit/eval/improvement session, define the output format up front. Templates exist — use them |
|
|
293
|
+
| LLM-based health checks instead of deterministic ones | Probabilistic grading of probabilistic content is circular and unreliable | Health-tooling layer is zero-LLM by design. LLMs grade *task output*, not *skill metadata* |
|
|
294
|
+
|
|
295
|
+
## Verification
|
|
296
|
+
|
|
297
|
+
Before any batch skill commit, verify:
|
|
298
|
+
|
|
299
|
+
- [ ] Inventory tool exits with zero critical errors
|
|
300
|
+
- [ ] Zero invalid-frontmatter entries
|
|
301
|
+
- [ ] Zero broken mappings in routing config
|
|
302
|
+
- [ ] All new skills meet the eval minimum (≥ 7, or ≥ 3 with explicit warning acceptance)
|
|
303
|
+
- [ ] All new evals include at least one negative expectation
|
|
304
|
+
- [ ] Conflict tool shows no NEW conflicts vs baseline
|
|
305
|
+
- [ ] Routing-gap report shows no new keyword gaps caused by your changes
|
|
306
|
+
- [ ] Manifest generator completed without errors
|
|
307
|
+
- [ ] Mirror parity verified if the library replicates to multiple harness directories
|
|
308
|
+
- [ ] Skill index header count matches actual skill count
|
|
309
|
+
|
|
310
|
+
## Do NOT Use When
|
|
311
|
+
|
|
312
|
+
| Use instead | When |
|
|
313
|
+
|---|---|
|
|
314
|
+
| `skill-scaffold` | Authoring or restructuring a single new SKILL.md (the contract for one file, not the system around the library) |
|
|
315
|
+
| `graph-audit` | Running the conformance audit on this Skill Graph repo specifically (operational), not designing the discipline |
|
|
316
|
+
| `lint-overlay` | General-purpose lint-rule selection and gate placement for any codebase, not skill-system-specific tooling |
|
|
317
|
+
| `documentation` | Writing prose for a human reader explaining how the skill system works |
|
|
318
|
+
| `code-review` | Reviewing a code change to the health-tooling scripts themselves |
|
|
319
|
+
| `testing-strategy` | Designing the test pyramid / trophy / honeycomb shape for a non-skill-library codebase |
|
|
320
|
+
| `prompt-craft` | Improving the wording of a single skill's prompt or eval prompt |
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: skill-router
|
|
3
|
+
description: "Use when routing an agent request across multiple skills, building or auditing a routing table, detecting routing coverage gaps, or answering questions like 'which skill handles this?', 'who routes X?', or 'why did skill A activate instead of B?'. Covers trigger-label matching, file-path matching, keyword matching, description-based semantic matching, scope/type tiebreakers, and coverage-gap detection. Do NOT use when the target skill is already known (load it directly), when authoring a new skill (use `skill-metadata-template` instead), or when evaluating a SINGLE skill's quality (use `graph-audit`)."
|
|
4
|
+
license: MIT
|
|
5
|
+
compatibility: "Markdown, YAML, any agent runtime"
|
|
6
|
+
allowed-tools: Read Grep
|
|
7
|
+
metadata:
|
|
8
|
+
metadata: "{\"schema_version\":6,\"version\":\"1.0.0\",\"type\":\"router\",\"category\":\"agent\",\"domain\":\"agent/skill-system\",\"scope\":\"portable\",\"owner\":\"skill-graph-maintainer\",\"freshness\":\"2026-04-18\",\"drift_check\":\"{\\\\\\\"last_verified\\\\\\\":\\\\\\\"2026-05-13\\\\\\\",\\\\\\\"truth_source_hashes\\\\\\\":{\\\\\\\"scripts/skill-graph-route.js\\\\\\\":\\\\\\\"b9a7b51d0e8b845b11473f479c4593754768c3775508049db7339892b2f08cc2\\\\\\\",\\\\\\\"scripts/skill-graph-routing-eval.js\\\\\\\":\\\\\\\"fffac2858863662bde6bc54c56bb77a219ae93f626e0c8d5886566f998181deb\\\\\\\",\\\\\\\"examples/evals/skill-router.json\\\\\\\":\\\\\\\"fccabcbc5f9d8057536f397fb0fc71a567371f75fb9a21afda343b197af30293\\\\\\\",\\\\\\\"examples/evals/skill-router.routing.json\\\\\\\":\\\\\\\"c4dc88db1e746bea78a7cf96b50c6c84532a07eda42da2e35d790c8928d4da8c\\\\\\\"}}\",\"eval_artifacts\":\"present\",\"eval_state\":\"passing\",\"routing_eval\":\"present\",\"stability\":\"experimental\",\"keywords\":\"[\\\\\\\"skill routing\\\\\\\",\\\\\\\"skill dispatch\\\\\\\",\\\\\\\"keyword routing\\\\\\\",\\\\\\\"route skill\\\\\\\",\\\\\\\"which skill to use\\\\\\\",\\\\\\\"skill selector\\\\\\\",\\\\\\\"routing table\\\\\\\",\\\\\\\"coverage gap\\\\\\\",\\\\\\\"ambiguous skill activation\\\\\\\",\\\\\\\"skill activate\\\\\\\",\\\\\\\"skill activates\\\\\\\",\\\\\\\"activate skill\\\\\\\",\\\\\\\"skill should activate\\\\\\\",\\\\\\\"which skill activates\\\\\\\",\\\\\\\"why did skill activate\\\\\\\",\\\\\\\"why skill activated\\\\\\\",\\\\\\\"routing decision\\\\\\\",\\\\\\\"dispatch request\\\\\\\",\\\\\\\"dispatch agent request\\\\\\\",\\\\\\\"agent request routing\\\\\\\",\\\\\\\"route this request\\\\\\\",\\\\\\\"route the request\\\\\\\",\\\\\\\"find the right skill\\\\\\\",\\\\\\\"choose the right skill\\\\\\\",\\\\\\\"which skill handles\\\\\\\"]\",\"triggers\":\"[\\\\\\\"skill-router\\\\\\\"]\",\"examples\":\"[\\\\\\\"activate the right skill for this agent request: 'my tests are failing in CI'\\\\\\\",\\\\\\\"build a routing table that covers every agent request type we see\\\\\\\",\\\\\\\"why did the documentation skill activate when the user asked about a11y?\\\\\\\",\\\\\\\"find the coverage gaps — which agent requests match no skill at all?\\\\\\\"]\",\"anti_examples\":\"[\\\\\\\"audit the graph-audit skill for schema conformance\\\\\\\",\\\\\\\"write a guide explaining how our routing works\\\\\\\",\\\\\\\"reproduce this routing mis-dispatch from production logs\\\\\\\"]\",\"relations\":\"{\\\\\\\"boundary\\\\\\\":[{\\\\\\\"skill\\\\\\\":\\\\\\\"graph-audit\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"graph-audit verifies ONE skill's metadata; skill-router chooses BETWEEN skills at request time\\\\\\\"},{\\\\\\\"skill\\\\\\\":\\\\\\\"debugging\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"debugging reproduces a specific routing mis-dispatch from evidence; skill-router designs the routing table itself\\\\\\\"},{\\\\\\\"skill\\\\\\\":\\\\\\\"skill-infrastructure\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"skill-infrastructure analyses routing-miss patterns across the whole library to find systemic gaps; skill-router authors the routing logic for one library at a time\\\\\\\"}],\\\\\\\"verify_with\\\\\\\":[\\\\\\\"graph-audit\\\\\\\"]}\",\"grounding\":\"{\\\\\\\"domain_object\\\\\\\":\\\\\\\"Skill Graph reference routing behavior\\\\\\\",\\\\\\\"grounding_mode\\\\\\\":\\\\\\\"repo_specific\\\\\\\",\\\\\\\"truth_sources\\\\\\\":[\\\\\\\"scripts/skill-graph-route.js\\\\\\\",\\\\\\\"scripts/skill-graph-routing-eval.js\\\\\\\",\\\\\\\"examples/evals/skill-router.json\\\\\\\",\\\\\\\"examples/evals/skill-router.routing.json\\\\\\\"],\\\\\\\"failure_modes\\\\\\\":[\\\\\\\"negation_paths_score_as_positive_matches\\\\\\\",\\\\\\\"routing_eval_claim_without_harness_pass\\\\\\\",\\\\\\\"boundary_exclusion_removes_stronger_match\\\\\\\",\\\\\\\"coverage_gap_silently_falls_back\\\\\\\"],\\\\\\\"evidence_priority\\\\\\\":\\\\\\\"repo_code_first\\\\\\\"}\",\"portability\":\"{\\\\\\\"readiness\\\\\\\":\\\\\\\"scripted\\\\\\\",\\\\\\\"targets\\\\\\\":[\\\\\\\"skill-md\\\\\\\"]}\",\"skill_graph_source_repo\":\"https://github.com/jacob-balslev/skill-graph\",\"skill_graph_protocol\":\"Skill Metadata Protocol v5\",\"skill_graph_project\":\"Skill Graph\",\"skill_graph_canonical_skill\":\"skills/skill-router/SKILL.md\"}"
|
|
9
|
+
skill_graph_source_repo: "https://github.com/jacob-balslev/skill-graph"
|
|
10
|
+
skill_graph_protocol: Skill Metadata Protocol v4
|
|
11
|
+
skill_graph_project: Skill Graph
|
|
12
|
+
skill_graph_canonical_skill: skills/skill-router/SKILL.md
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Skill Router
|
|
16
|
+
|
|
17
|
+
## Coverage
|
|
18
|
+
|
|
19
|
+
- Routing by keyword pattern: matching inbound query terms to skill `keywords` arrays to identify the best candidate
|
|
20
|
+
- Routing by trigger label: matching explicit skill-router labels (`triggers` field) to eliminate ambiguity when the intent is declared
|
|
21
|
+
- Routing by file path: matching touched or mentioned file paths against skill `paths` arrays for file-activated skills
|
|
22
|
+
- Fallback ordering: how to rank skills when multiple candidates score equally, including `scope` and `type` tiebreakers
|
|
23
|
+
- Coverage gaps: detecting when no skill matches a request and how to surface that gap as an authoring signal
|
|
24
|
+
|
|
25
|
+
## Philosophy
|
|
26
|
+
|
|
27
|
+
Routing is adversarial against convenience. The tempting move — "if nothing matches exactly, just pick the closest skill and activate it" — is the one that silently degrades every agent that depends on the router. A wrong skill that activates confidently is worse than a coverage gap that surfaces loudly, because silent wrongness has no signal for anyone to fix. The router's job is to produce either a certain winner or an explicit non-answer, never a confident guess.
|
|
28
|
+
|
|
29
|
+
Four principles follow from that stance:
|
|
30
|
+
|
|
31
|
+
- **First unique winner stops the chain.** A match on `triggers` is stronger than a match on `paths`, which is stronger than a match on `keywords`. Evaluate surfaces in priority order and stop at the first one that yields a unique winner — do not second-guess a trigger-label hit with keyword analysis.
|
|
32
|
+
- **Tiebreakers favor specificity, not popularity.** When scores tie, a skill specific to *this* codebase (`scope: codebase`) wins over a portable skill, and a procedural skill (`type: workflow`) wins over a reference (`type: capability`) because the inbound query is more likely to need action than lookup. Never rank by skill age, usage count, or author preference.
|
|
33
|
+
- **Explicit coverage gaps beat silent wrong fallback.** If no surface produces a winner, surface the gap to the caller — recommend authoring a new skill or broadening a keyword list. Silent fallback to a default skill is a bug that no test catches, because the misrouted query looks successful to the router but nonsensical to the downstream agent.
|
|
34
|
+
- **The router is a mapping, not a judge.** It decides which skill owns a query; it does not decide whether the query is well-formed, worth handling, or strategically important. Those are the activated skill's concerns. Overloading the router with domain judgment makes it harder to audit and harder to change.
|
|
35
|
+
|
|
36
|
+
## Routing Rules
|
|
37
|
+
|
|
38
|
+
The router evaluates three matching surfaces in priority order. The first surface that produces a unique winner stops the evaluation chain.
|
|
39
|
+
|
|
40
|
+
| Priority | Surface | Field consulted | Match rule |
|
|
41
|
+
|---|---|---|---|
|
|
42
|
+
| 1 | Trigger label | `triggers` | Exact match against the declared label. Winner is unambiguous. |
|
|
43
|
+
| 2 | File path | `paths` | Glob match against the touched/mentioned path. Prefer the most specific match. |
|
|
44
|
+
| 3 | Keyword pattern | `keywords` | Token overlap between query terms and skill keyword list. Rank by match count; break ties with `scope` then `type`. |
|
|
45
|
+
|
|
46
|
+
### Scope tiebreaker
|
|
47
|
+
|
|
48
|
+
When keyword scores are equal, prefer skills in this `scope` order: `codebase` > `reference` > `portable`. A codebase-scoped skill is specific to *this* repository and wins over a portable one when both match the query equally.
|
|
49
|
+
|
|
50
|
+
> **Schema version note.** The v1 enum values `operational` and `generic` were renamed to `codebase` and `portable` in `schema_version: 2`. Always use the v2 names; the current schema rejects the v1 names as hard errors. A full rename map ships with the Skill Graph schema migration notes — consult those when porting a v1 skill.
|
|
51
|
+
|
|
52
|
+
### Type tiebreaker
|
|
53
|
+
|
|
54
|
+
After scope, prefer `workflow` over `capability` over `router` over `overlay`. A workflow skill ships procedural decision logic that is usually more actionable than a pure capability reference when the query is ambiguous.
|
|
55
|
+
|
|
56
|
+
### Fallback behavior
|
|
57
|
+
|
|
58
|
+
If no skill matches any surface, the router does not fall back to a default skill. It surfaces a coverage gap and recommends authoring a new skill or broadening an existing skill's `keywords` array. Silent fallback to a wrong skill is worse than an explicit coverage-gap signal.
|
|
59
|
+
|
|
60
|
+
## Evals
|
|
61
|
+
|
|
62
|
+
This skill ships a comprehension-eval artifact as `skill-router.json` in the Skill Graph `examples/evals/` directory. The eval prompts specifically test the priority-ordered match surfaces, the scope/type tiebreakers, and the explicit refusal to fall back to a default. The eval file is how this skill is graded by the Skill Graph audit runner — consumers in other agent runtimes can treat the eval cases as conformance tests translated to their own grading harness.
|
|
63
|
+
|
|
64
|
+
## Do NOT Use When
|
|
65
|
+
|
|
66
|
+
| Use instead | When |
|
|
67
|
+
|---|---|
|
|
68
|
+
| The target skill directly | The correct skill is already known — skip the router and load it |
|
|
69
|
+
| `documentation` | The task is writing or structuring doc prose, not routing |
|
|
70
|
+
| `graph-audit` | The task is auditing whether routing metadata is consistent, not dispatching a query |
|
|
71
|
+
| `examples/skill-metadata-template.md` | The task is authoring a new skill from scratch, not dispatching to an existing one (the template is a reference artifact, not a routable skill) |
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: skill-scaffold
|
|
3
|
+
description: "Use when creating a new SKILL.md from scratch, adapting an existing skill to a different archetype, or teaching another author the canonical Skill Metadata Protocol frontmatter and body structure. Covers schema-conformant frontmatter, archetype-aware body layout, semantic-layer discipline (description vs Coverage), teaching-layer mechanics (TEMPLATE NOTE blockquotes), the lint-first authoring gate, and the routing-eval honesty rule. Do NOT use when modifying an already-written skill (edit it directly), when writing general technical documentation (use `documentation`), or when fixing a malformed skill detected by lint (use `graph-audit` for systematic library health, not authoring scaffold help)."
|
|
4
|
+
license: MIT
|
|
5
|
+
compatibility: "Markdown, YAML, JSON Schema"
|
|
6
|
+
allowed-tools: Read Grep Bash Write Edit
|
|
7
|
+
metadata:
|
|
8
|
+
metadata: "{\"schema_version\":6,\"version\":\"1.0.0\",\"type\":\"capability\",\"category\":\"agent\",\"domain\":\"agent/skill-system\",\"scope\":\"reference\",\"owner\":\"skill-graph-maintainer\",\"freshness\":\"2026-05-04\",\"drift_check\":\"{\\\\\\\"last_verified\\\\\\\":\\\\\\\"2026-05-14\\\\\\\",\\\\\\\"truth_source_hashes\\\\\\\":{\\\\\\\"examples/skill-metadata-template.md\\\\\\\":\\\\\\\"42b3185ebf53f9efc6a32977ee9408efce0957c0a7ed62cabca97cb83c33600a\\\\\\\",\\\\\\\"schemas/skill.v4.schema.json\\\\\\\":\\\\\\\"e83d6be8b1314488b39b8c7bec2784d6459980d3f9965be68ad1c9a53865622d\\\\\\\",\\\\\\\"docs/skill-metadata-protocol.md\\\\\\\":\\\\\\\"08ad662c5f470fac337aa559dacd0b9e882be7df8d7918f20a5d4e3aaaaa2ed7\\\\\\\"}}\",\"eval_artifacts\":\"planned\",\"eval_state\":\"unverified\",\"routing_eval\":\"absent\",\"stability\":\"experimental\",\"keywords\":\"[\\\\\\\"skill authoring\\\\\\\",\\\\\\\"new skill\\\\\\\",\\\\\\\"writing a skill\\\\\\\",\\\\\\\"skill scaffold\\\\\\\",\\\\\\\"skill template\\\\\\\",\\\\\\\"skill frontmatter\\\\\\\",\\\\\\\"skill graph contract\\\\\\\",\\\\\\\"capability or workflow\\\\\\\",\\\\\\\"description vs coverage\\\\\\\",\\\\\\\"archetype selection\\\\\\\",\\\\\\\"skill body layout\\\\\\\",\\\\\\\"teaching layer\\\\\\\",\\\\\\\"placeholder sludge\\\\\\\",\\\\\\\"cargo cult meta sections\\\\\\\",\\\\\\\"lint first authoring\\\\\\\",\\\\\\\"routing eval honesty\\\\\\\"]\",\"examples\":\"[\\\\\\\"I'm writing a new skill from scratch — where do I start?\\\\\\\",\\\\\\\"how do I pick between capability and workflow for my skill type?\\\\\\\",\\\\\\\"what's the difference between description and the ## Coverage section?\\\\\\\",\\\\\\\"scaffold a new skill that teaches react component composition patterns\\\\\\\",\\\\\\\"I copied skill-metadata-template.md but my new skill won't pass lint — help\\\\\\\",\\\\\\\"draft frontmatter for a workflow skill that owns deployment rollback\\\\\\\",\\\\\\\"how do I strip teaching annotations from the template before commit?\\\\\\\",\\\\\\\"should I flip routing_eval to present on my new skill?\\\\\\\"]\",\"anti_examples\":\"[\\\\\\\"refactor my existing skill to be more concise\\\\\\\",\\\\\\\"my skill's routing isn't activating — why?\\\\\\\",\\\\\\\"audit my skill library for stale frontmatter\\\\\\\",\\\\\\\"write a developer guide for the contributor docs\\\\\\\",\\\\\\\"review this skill's content for correctness\\\\\\\"]\",\"relations\":\"{\\\\\\\"boundary\\\\\\\":[{\\\\\\\"skill\\\\\\\":\\\\\\\"refactor\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"refactor is behaviour-preserving modification of existing code or skills; skill-scaffold creates a new skill from scratch\\\\\\\"},{\\\\\\\"skill\\\\\\\":\\\\\\\"skill-router\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"skill-router dispatches between existing skills at request time; skill-scaffold is the authoring-time guide for a NEW skill\\\\\\\"},{\\\\\\\"skill\\\\\\\":\\\\\\\"graph-audit\\\\\\\",\\\\\\\"reason\\\\\\\":\\\\\\\"graph-audit verifies the authored metadata of an existing skill; skill-scaffold is the authoring-time guide before a skill exists\\\\\\\"}],\\\\\\\"related\\\\\\\":[\\\\\\\"naming-conventions\\\\\\\"],\\\\\\\"verify_with\\\\\\\":[]}\",\"grounding\":\"{\\\\\\\"domain_object\\\\\\\":\\\\\\\"Authoring a new SKILL.md against Skill Metadata Protocol v5\\\\\\\",\\\\\\\"grounding_mode\\\\\\\":\\\\\\\"repo_specific\\\\\\\",\\\\\\\"truth_sources\\\\\\\":[\\\\\\\"examples/skill-metadata-template.md\\\\\\\",\\\\\\\"schemas/skill.v4.schema.json\\\\\\\",\\\\\\\"docs/skill-metadata-protocol.md\\\\\\\"],\\\\\\\"failure_modes\\\\\\\":[\\\\\\\"placeholder_sludge\\\\\\\",\\\\\\\"cargo_cult_meta_sections\\\\\\\",\\\\\\\"description_coverage_collapse\\\\\\\",\\\\\\\"authoring_gate_skipped\\\\\\\",\\\\\\\"inflated_routing_eval\\\\\\\"],\\\\\\\"evidence_priority\\\\\\\":\\\\\\\"repo_code_first\\\\\\\"}\",\"portability\":\"{\\\\\\\"readiness\\\\\\\":\\\\\\\"scripted\\\\\\\",\\\\\\\"targets\\\\\\\":[\\\\\\\"skill-md\\\\\\\"]}\",\"lifecycle\":\"{\\\\\\\"stale_after_days\\\\\\\":180,\\\\\\\"review_cadence\\\\\\\":\\\\\\\"quarterly\\\\\\\"}\",\"skill_graph_source_repo\":\"https://github.com/jacob-balslev/skill-graph\",\"skill_graph_protocol\":\"Skill Metadata Protocol v5\",\"skill_graph_project\":\"Skill Graph\",\"skill_graph_canonical_skill\":\"skills/skill-scaffold/SKILL.md\"}"
|
|
9
|
+
skill_graph_source_repo: "https://github.com/jacob-balslev/skill-graph"
|
|
10
|
+
skill_graph_protocol: Skill Metadata Protocol v4
|
|
11
|
+
skill_graph_project: Skill Graph
|
|
12
|
+
skill_graph_canonical_skill: skills/skill-scaffold/SKILL.md
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Skill Scaffold
|
|
16
|
+
|
|
17
|
+
## Coverage
|
|
18
|
+
|
|
19
|
+
- Authoring flow: copy → rename → adapt → strip teaching annotations → verify → commit
|
|
20
|
+
- Frontmatter identity: `name`, `description`, `version`, `type`, `category`, `scope`, `owner`, plus the eval-health triple and `drift_check` required by every Skill Metadata Protocol skill
|
|
21
|
+
- Archetype selection: how to pick between `capability`, `workflow`, `router`, and `overlay` and which `## H2` body sections each archetype requires
|
|
22
|
+
- Semantic-layer discipline: how `description:` (≤ 3 sentences, pushy, boundary-aware routing contract) differs from `## Coverage` (bulleted scope map of distinct topics) and why each must stay in its own layer
|
|
23
|
+
- Teaching-layer mechanics: how to use `> **TEMPLATE NOTE:**` blockquotes and `# TEMPLATE NOTE:` YAML comments to teach without cargo-culting meta sections into derived skills
|
|
24
|
+
- Lint-first authoring gate: passing `node scripts/skill-lint.js --strict` against your new skill before commit, every time, no exceptions
|
|
25
|
+
- Routing-eval honesty: defaulting to `routing_eval: absent` and only flipping to `present` after `node scripts/skill-graph-routing-eval.js --skill <name>` exits 0
|
|
26
|
+
- Grounding declarations: when to populate `grounding.truth_sources` vs leaving the block off, and how the drift sentinel consumes recorded `truth_source_hashes`
|
|
27
|
+
|
|
28
|
+
## Philosophy
|
|
29
|
+
|
|
30
|
+
A scaffold teaches by example, not by placeholder. A concrete, internally consistent specimen of a finished skill is a more reliable authoring reference than any amount of abstract instruction. The teaching layer — meta-commentary about how to read and adapt the scaffold — must live in structurally distinct slots that disappear when the author tightens a new skill, never in the `## H2` section slots that AI agents copy verbatim. Authoring is also a *lint-first* discipline: the lint output is the primary debugging surface for new skills, deliberately verbose, and the canonical answer to "did I do this right?" — not a senior reviewer's pattern-match.
|
|
31
|
+
|
|
32
|
+
## Authoring Flow
|
|
33
|
+
|
|
34
|
+
The five steps are non-negotiable; skipping any step produces a skill that lints in your editor but breaks on someone else's machine.
|
|
35
|
+
|
|
36
|
+
**Step 0 (precondition):** Before authoring, check the redundancy registry at [`_meta/REDUNDANT-SKILLS.md`](https://github.com/jacob-balslev/skills/blob/main/_meta/REDUNDANT-SKILLS.md) (in the sibling broader library). It lists skills that have been authored, evaluated via comprehension grading, and judged redundant or harmful by the grader. If your intended subject matches an entry in that registry, either skip authoring (the model already knows the topic) or document the new grounding evidence that justifies a fresh attempt — re-authoring a tombstoned skill without contradicting evidence wastes context and risks regressing on measured quality.
|
|
37
|
+
|
|
38
|
+
1. **Copy** `examples/skill-metadata-template.md` to `skills/<your-skill-name>/SKILL.md`. Do not rename in-place; the template stays as the canonical specimen.
|
|
39
|
+
2. **Rename** identity fields: `name`, `description`, `category` (if used), `keywords`, `examples`, `anti_examples`, `paths` (if applicable), and the body title. Every reference to "skill-metadata-template" should be gone.
|
|
40
|
+
3. **Adapt** body sections to your skill's subject. Match the `## H2` layout to your declared archetype per `docs/skill-metadata-protocol.md § Archetype section map`. Remove sections that do not apply — do not keep them with placeholder content.
|
|
41
|
+
4. **Strip** every `> **TEMPLATE NOTE:**` body blockquote and every `# TEMPLATE NOTE:` YAML comment. They are authoring scaffolding; shipping them in a derived skill is the most common authoring mistake. Run `grep -n "TEMPLATE NOTE" skills/<your-skill>/SKILL.md` to confirm zero hits.
|
|
42
|
+
5. **Verify** by running the gate sequence: `node scripts/skill-lint.js --strict` (must show 0 errors), `node scripts/check-protocol-consistency.js` (C1-C7 must all pass), and (if you populated `examples` and `anti_examples`) `node scripts/skill-graph-routing-eval.js --skill <your-skill>` (verdict PASS before flipping `routing_eval` to `present`).
|
|
43
|
+
|
|
44
|
+
## Archetype Selection
|
|
45
|
+
|
|
46
|
+
Pick `type:` once, deliberately, before writing the body — the archetype determines which body sections are required.
|
|
47
|
+
|
|
48
|
+
| Archetype | When to pick it | Required body sections |
|
|
49
|
+
|---|---|---|
|
|
50
|
+
| `capability` | The skill teaches *how to do something* with no fixed sequence (e.g., a11y, naming-conventions, prompt-craft) | `## Coverage`, `## Philosophy`, `## Verification` |
|
|
51
|
+
| `workflow` | The skill orchestrates a *sequence of steps* the agent follows in order (e.g., debugging, refactor, skill-audit-loop) | `## Coverage`, `## Workflow` (numbered steps), `## Verification` |
|
|
52
|
+
| `router` | The skill dispatches between *other skills* (rare; only one starter — `skill-router`) | `## Coverage`, `## Routing Rules`, `## Verification` |
|
|
53
|
+
| `overlay` | The skill *specialises an `extends:` parent* with project-specific or stack-specific overrides (e.g., `lint-overlay`) | `## Overlay Rules` plus everything inherited from the parent's archetype |
|
|
54
|
+
|
|
55
|
+
The fastest way to pick wrong: choose `workflow` because your skill happens to list ordered steps in the body. If the *user* could reasonably consume the skill in any order based on the section they need, it is a `capability`, not a `workflow`. Workflows are skills the agent literally walks through start-to-end every time.
|
|
56
|
+
|
|
57
|
+
## Semantic-Layer Discipline
|
|
58
|
+
|
|
59
|
+
`description` and `## Coverage` look like duplicates and are not. They live in different layers and serve different consumers.
|
|
60
|
+
|
|
61
|
+
- `description` (≤ 3 sentences, frontmatter): the **routing contract**. Pushy, specific, boundary-aware. Tells a router whether to activate this skill for a given query. Should include an explicit "Do NOT use for…" boundary clause so the router doesn't over-activate.
|
|
62
|
+
- `## Coverage` (bulleted list, body): the **scope map**. Enumerates the distinct topics this skill teaches. The reader scans `## Coverage` to decide whether to read further. Should NOT restate `description` as a single line; should NOT be a checklist of placeholder commitments.
|
|
63
|
+
|
|
64
|
+
If you can drop your `description` into `## Coverage` without changing meaning, both layers are wrong. Rewrite `description` until it is too pushy for the body, then rewrite `## Coverage` until it is too enumerative for the description.
|
|
65
|
+
|
|
66
|
+
## Common Authoring Mistakes
|
|
67
|
+
|
|
68
|
+
| Mistake | Symptom | Fix |
|
|
69
|
+
|---|---|---|
|
|
70
|
+
| Placeholder sludge | `description: "Use when... for..."`, paths like `your/file.md`, `todo` markers | Search the file for `your-`, `path/to/`, `todo`, `lorem`. Zero hits before commit. |
|
|
71
|
+
| Cargo-culted teaching layer | `> **TEMPLATE NOTE:**` blocks survive into derived skill | `grep -n "TEMPLATE NOTE" skills/<name>/SKILL.md` returns nothing |
|
|
72
|
+
| Description-Coverage collapse | `description` and `## Coverage` say the same thing in different shapes | Rewrite description as routing contract, Coverage as scope map (see § Semantic-Layer Discipline) |
|
|
73
|
+
| Inflated routing_eval | `routing_eval: present` set without running the harness | Default to `absent`. Flip to `present` only after `node scripts/skill-graph-routing-eval.js --skill <name>` returns PASS |
|
|
74
|
+
| Wrong archetype | Skill body sections don't match `type:` | Re-pick archetype per § Archetype Selection; rewrite the body to match |
|
|
75
|
+
| Anti-examples for skills that don't exist | `anti_examples` references skills not in the library | Either author those skills first or use existing starter names |
|
|
76
|
+
| Lint-skipped commit | New skill is committed without `--strict` lint pass | Run `node scripts/skill-lint.js --strict` before every commit; fix all errors |
|
|
77
|
+
|
|
78
|
+
## Verification
|
|
79
|
+
|
|
80
|
+
Use this checklist as the authoring gate before committing a skill. Every item must pass.
|
|
81
|
+
|
|
82
|
+
- [ ] Every retained field has a real reason to exist in the new skill
|
|
83
|
+
- [ ] Every removed field was removed because of archetype or grounding mismatch, not laziness
|
|
84
|
+
- [ ] Body sections match the declared archetype per `docs/skill-metadata-protocol.md § Archetype section map`
|
|
85
|
+
- [ ] `description:` is ≤ 3 sentences, contains pushy trigger phrases, and names an explicit negative boundary
|
|
86
|
+
- [ ] `## Coverage` is a scope map of distinct topics, not a one-line restate of the description
|
|
87
|
+
- [ ] `drift_check` is an object with `last_verified`; `truth_source_hashes` recorded when truth sources exist
|
|
88
|
+
- [ ] `compatibility` is an object (not a free-text string) when present
|
|
89
|
+
- [ ] `eval_artifacts`, `eval_state`, `routing_eval` reflect the actual skill state — no inflation
|
|
90
|
+
- [ ] All `relations` entries point to skills that exist in the target repo; `boundary` entries with non-obvious rationale use the `{skill, reason}` form
|
|
91
|
+
- [ ] No placeholder sludge (`your-skill-name`, `path/to/file`, `todo`) remains
|
|
92
|
+
- [ ] No `> **TEMPLATE NOTE:**` blockquotes or `# TEMPLATE NOTE:` YAML comments remain
|
|
93
|
+
- [ ] `node scripts/skill-lint.js --strict` returns 0 errors against the new skill
|
|
94
|
+
- [ ] `node scripts/check-protocol-consistency.js` passes C1-C7
|
|
95
|
+
- [ ] If `routing_eval: present`, `node scripts/skill-graph-routing-eval.js --skill <name>` returns verdict PASS
|
|
96
|
+
|
|
97
|
+
## Do NOT Use When
|
|
98
|
+
|
|
99
|
+
| Use instead | When |
|
|
100
|
+
|---|---|
|
|
101
|
+
| `documentation` | Writing general technical documentation (guides, tutorials, references) — not a skill specifically |
|
|
102
|
+
| `refactor` | Modifying an existing skill's content while preserving its identity |
|
|
103
|
+
| `graph-audit` | Auditing the metadata health of skills already in the library |
|
|
104
|
+
| `skill-router` | Diagnosing why a router doesn't activate an existing skill on a given query |
|
|
105
|
+
| `code-review` | Reviewing the technical content of a skill (correctness, clarity) once authored |
|