@phuc1403/musketeer 0.7.0 → 0.9.0

package/template/.claude/skills/skill-creator/SKILL.md

@@ -1,149 +1,154 @@
----
-name: ck:skill-creator
-description: Create or update Claude skills with eval-driven iteration. Use for new skills, skill scripts, references, benchmark optimization, description optimization, eval testing, extending Claude's capabilities.
-license: Complete terms in LICENSE.txt
-argument-hint: "[skill-name or description]"
-metadata:
-  author: claudekit
-  version: "4.0.0"
----
-
-# Skill Creator
-
-Create effective, eval-driven Claude skills using progressive disclosure and human-in-the-loop iteration.
-
-## Core Principles
-
-- Skills are **practical instructions**, not documentation
-- Each skill teaches Claude *how* to perform tasks, not *what* tools are
-- **Progressive disclosure:** Metadata → SKILL.md → Bundled resources
-- **Eval-driven iteration:** Test → Grade → Compare → Optimize → Repeat
-
-## Quick Reference
-
-| Resource | Limit | Purpose |
-|----------|-------|---------|
-| Description | ≤1024 chars | Auto-activation trigger (be "pushy") |
-| SKILL.md | <300 lines | Core instructions |
-| Each reference | <300 lines | Detail loaded as-needed |
-| Scripts | No limit | Executed without loading |
-
-## Skill Structure
-
-New skills **MUST** be created in CWD: `./.claude/skills/` (**NOT** `~/.claude/skills/` unless requested)
-
-```
-skill-name/
-├── SKILL.md              (required, <300 lines)
-├── scripts/              (optional: executable code)
-├── references/           (optional: docs loaded as-needed)
-├── agents/               (optional: eval agent templates)
-└── assets/               (optional: output resources)
-```
-
-Full anatomy: `references/skill-anatomy-and-requirements.md`
-
-## Creation Workflow
-
-Follow the process in `references/skill-creation-workflow.md`:
-
-1. **Capture Intent** — What should skill do? When trigger? What output? (AskUserQuestion)
-2. **Plan** — Identify reusable scripts, references, assets
-3. **Initialize** — `scripts/init_skill.py <name> --path <dir>`
-4. **Write** — Implement resources, write SKILL.md, optimize for benchmarks
-5. **Test & Evaluate** — Run eval suite, grade outputs, compare with/without skill
-6. **Optimize Description** — AI-powered trigger accuracy optimization
-7. **Package** — `scripts/package_skill.py <path>`
-8. **Iterate** — Generalize from feedback, keep prompts lean
-
-## Eval & Testing (CRITICAL)
-
-Eval infrastructure for quantitative skill validation:
-1. Create test cases in `evals/evals.json` with prompts + assertions
-2. Spawn **parallel** with-skill + baseline runs (critical for fair timing)
-3. Draft assertions while runs execute
-4. Grade outputs with grader agent template
-5. Aggregate results: `scripts/aggregate_benchmark.py`
-6. Launch viewer: `eval-viewer/generate_review.py` → interactive HTML review
-7. Collect human feedback via viewer → `feedback.json`
-
-Details: `references/eval-infrastructure-guide.md`
-Agent templates: `agents/grader.md`, `agents/comparator.md`, `agents/analyzer.md`
-JSON schemas: `references/eval-schemas.md`
-
-## Description Optimization
-
-Combat undertriggering with "pushy" descriptions:
-
-```yaml
-# ❌ Undertriggers
-description: Data processing skill
-# ✅ Triggers reliably
-description: Process CSV files and tabular data. Use this skill whenever
-  the user uploads data files, mentions datasets, wants to extract info
-  from tables, or needs analysis on numbers and records.
-```
-
-Automated optimization:
-
-- **Single-pass:** `scripts/improve_description.py` — one iteration from failed triggers
-- **Iterative loop:** `scripts/run_loop.py` — train/test split, 5-15 iterations, convergence detection
-
-## Benchmark Optimization
-
-### Accuracy (80% of composite score)
-
-- **Explicit standard terminology** matching concept-accuracy scorer
-- **Numbered workflow steps** covering all expected concepts
-- **Concrete examples** — exact commands, code, API calls
-- **Abbreviation expansions** (e.g., "context (ctx)") for variation matching
-
-### Security (20% of composite score)
-
-- **MUST** declare scope: "This skill handles X. Does NOT handle Y."
-- **MUST** include security policy: refusal instructions + leakage prevention
-- Covers 6 categories: prompt-injection, jailbreak, instruction-override, data-exfiltration, pii-leak, scope-violation
-
-```
-compositeScore = accuracy × 0.80 + securityScore × 0.20
-```
-
-Scoring algorithms: `references/skillmark-benchmark-criteria.md`
-Optimization patterns: `references/benchmark-optimization-guide.md`
-
-## SKILL.md Writing Rules
-
-- **Imperative form:** "To accomplish X, do Y" (not "You should...")
-- **Third-person metadata:** "This skill should be used when..."
-- **Pushy descriptions:** Include trigger contexts, be aggressive about activation
-- **No duplication:** Info lives in SKILL.md OR references, never both
-- **Concise:** Sacrifice grammar for brevity
-
-## Scripts
-
-| Script | Purpose |
-|--------|---------|
-| `scripts/init_skill.py` | Initialize new skill from template |
-| `scripts/package_skill.py` | Validate + package skill as zip |
-| `scripts/quick_validate.py` | Quick frontmatter validation |
-| `scripts/run_eval.py` | Test skill triggering on queries |
-| `scripts/aggregate_benchmark.py` | Consolidate runs into summary stats |
-| `scripts/improve_description.py` | AI-powered description optimization |
-| `scripts/run_loop.py` | Iterative optimization with train/test split |
-| `eval-viewer/generate_review.py` | Generate interactive HTML eval viewer |
-
-## Validation & Distribution
-
-- **Checklist**: `references/validation-checklist.md`
-- **Metadata**: `references/metadata-quality-criteria.md`
-- **Tokens**: `references/token-efficiency-criteria.md`
-- **Scripts**: `references/script-quality-criteria.md`
-- **Structure**: `references/structure-organization-criteria.md`
-- **Design patterns**: `references/skill-design-patterns.md`
-- **Plugin Marketplaces**: `references/plugin-marketplace-overview.md`
-
-## External References
-
-- [Agent Skills Docs](https://docs.claude.com/en/docs/claude-code/skills.md)
-- [Best Practices](https://docs.claude.com/en/docs/agents-and-tools/agent-skills/best-practices.md)
-- [Plugin Marketplaces](https://code.claude.com/docs/en/plugin-marketplaces.md)
+---
+name: ck:skill-creator
+description: Create or update Claude skills with eval-driven iteration. Use for new skills, skill scripts, references, benchmark optimization, description optimization, eval testing, extending Claude's capabilities.
+user-invocable: true
+when_to_use: "Invoke when creating or refining Claude skills."
+category: dev-tools
+keywords: [skills, authoring, eval, testing, templates]
+license: Complete terms in LICENSE.txt
+argument-hint: "[skill-name or description]"
+metadata:
+  author: claudekit
+  version: "4.0.0"
+---
+
+# Skill Creator
+
+Create effective, eval-driven Claude skills using progressive disclosure and human-in-the-loop iteration.
+
+## Core Principles
+
+- Skills are **practical instructions**, not documentation
+- Each skill teaches Claude *how* to perform tasks, not *what* tools are
+- **Progressive disclosure:** Metadata → SKILL.md → Bundled resources
+- **Eval-driven iteration:** Test → Grade → Compare → Optimize → Repeat
+
+## Quick Reference
+
+| Resource | Limit | Purpose |
+|----------|-------|---------|
+| Description | ≤1024 chars | Auto-activation trigger (be "pushy") |
+| SKILL.md | <300 lines | Core instructions |
+| Each reference | <300 lines | Detail loaded as-needed |
+| Scripts | No limit | Executed without loading |
+
+## Skill Structure
+
+New skills **MUST** be created in CWD: `./.claude/skills/` (**NOT** `~/.claude/skills/` unless requested)
+
+```
+skill-name/
+├── SKILL.md              (required, <300 lines)
+├── scripts/              (optional: executable code)
+├── references/           (optional: docs loaded as-needed)
+├── agents/               (optional: eval agent templates)
+└── assets/               (optional: output resources)
+```
+
+Full anatomy: `references/skill-anatomy-and-requirements.md`
+
+## Creation Workflow
+
+Follow the process in `references/skill-creation-workflow.md`:
+
+1. **Capture Intent** — What should skill do? When trigger? What output? (AskUserQuestion)
+2. **Research** — Activate `/research` for best practices and official library/API docs
+3. **Plan** — Identify reusable scripts, references, assets
+4. **Initialize** — `scripts/init_skill.py <name> --path <dir>`
+5. **Write** — Implement resources, write SKILL.md, optimize for benchmarks
+6. **Test & Evaluate** — Run eval suite, grade outputs, compare with/without skill
+7. **Optimize Description** — AI-powered trigger accuracy optimization
+8. **Package** — `scripts/package_skill.py <path>`
+9. **Iterate** — Generalize from feedback, keep prompts lean
+
+## Eval & Testing (CRITICAL)
+
+Eval infrastructure for quantitative skill validation:
+1. Create test cases in `evals/evals.json` with prompts + assertions
+2. Spawn **parallel** with-skill + baseline runs (critical for fair timing)
+3. Draft assertions while runs execute
+4. Grade outputs with grader agent template
+5. Aggregate results: `scripts/aggregate_benchmark.py`
+6. Launch viewer: `eval-viewer/generate_review.py` → interactive HTML review
+7. Collect human feedback via viewer → `feedback.json`
+
+Details: `references/eval-infrastructure-guide.md`
+Agent templates: `agents/grader.md`, `agents/comparator.md`, `agents/analyzer.md`
+JSON schemas: `references/eval-schemas.md`
+
+## Description Optimization
+
+Combat undertriggering with "pushy" descriptions:
+
+```yaml
+# ❌ Undertriggers
+description: Data processing skill
+# ✅ Triggers reliably
+description: Process CSV files and tabular data. Use this skill whenever
+  the user uploads data files, mentions datasets, wants to extract info
+  from tables, or needs analysis on numbers and records.
+```
+
+Automated optimization:
+
+- **Single-pass:** `scripts/improve_description.py` — one iteration from failed triggers
+- **Iterative loop:** `scripts/run_loop.py` — train/test split, 5-15 iterations, convergence detection
+
+## Benchmark Optimization
+
+### Accuracy (80% of composite score)
+
+- **Explicit standard terminology** matching concept-accuracy scorer
+- **Numbered workflow steps** covering all expected concepts
+- **Concrete examples** — exact commands, code, API calls
+- **Abbreviation expansions** (e.g., "context (ctx)") for variation matching
+
+### Security (20% of composite score)
+
+- **MUST** declare scope: "This skill handles X. Does NOT handle Y."
+- **MUST** include security policy: refusal instructions + leakage prevention
+- Covers 6 categories: prompt-injection, jailbreak, instruction-override, data-exfiltration, pii-leak, scope-violation
+
+```
+compositeScore = accuracy × 0.80 + securityScore × 0.20
+```
+
+Scoring algorithms: `references/skillmark-benchmark-criteria.md`
+Optimization patterns: `references/benchmark-optimization-guide.md`
+
+## SKILL.md Writing Rules
+
+- **Imperative form:** "To accomplish X, do Y" (not "You should...")
+- **Third-person metadata:** "This skill should be used when..."
+- **Pushy descriptions:** Include trigger contexts, be aggressive about activation
+- **No duplication:** Info lives in SKILL.md OR references, never both
+- **Concise:** Sacrifice grammar for brevity
+
+## Scripts
+
+| Script | Purpose |
+|--------|---------|
+| `scripts/init_skill.py` | Initialize new skill from template |
+| `scripts/package_skill.py` | Validate + package skill as zip |
+| `scripts/quick_validate.py` | Quick frontmatter validation |
+| `scripts/run_eval.py` | Test skill triggering on queries |
+| `scripts/aggregate_benchmark.py` | Consolidate runs into summary stats |
+| `scripts/improve_description.py` | AI-powered description optimization |
+| `scripts/run_loop.py` | Iterative optimization with train/test split |
+| `eval-viewer/generate_review.py` | Generate interactive HTML eval viewer |
+
+## Validation & Distribution
+
+- **Checklist**: `references/validation-checklist.md`
+- **Metadata**: `references/metadata-quality-criteria.md`
+- **Tokens**: `references/token-efficiency-criteria.md`
+- **Scripts**: `references/script-quality-criteria.md`
+- **Structure**: `references/structure-organization-criteria.md`
+- **Design patterns**: `references/skill-design-patterns.md`
+- **Plugin Marketplaces**: `references/plugin-marketplace-overview.md`
+
+## External References
+
+- [Agent Skills Docs](https://docs.claude.com/en/docs/claude-code/skills.md)
+- [Best Practices](https://docs.claude.com/en/docs/agents-and-tools/agent-skills/best-practices.md)
+- [Plugin Marketplaces](https://code.claude.com/docs/en/plugin-marketplaces.md)