@soleur/opencode 0.0.0-stage → 1.0.0

package/skills/skill-creator/references/iteration-and-testing.md

@@ -0,0 +1,509 @@
+<overview>
+Skills improve through iteration and testing. This reference covers evaluation-driven development, Claude A/B testing patterns, and heading-structure validation during testing.
+</overview>
+
+<evaluation_driven_development>
+<principle>
+Create evaluations BEFORE writing extensive documentation. This ensures your skill solves real problems rather than documenting imagined ones.
+</principle>
+
+<workflow>
+<step_1>
+**Identify gaps**: Run Claude on representative tasks without a skill. Document specific failures or missing context.
+</step_1>
+
+<step_2>
+**Create evaluations**: Build three scenarios that test these gaps.
+</step_2>
+
+<step_3>
+**Establish baseline**: Measure Claude's performance without the skill.
+</step_3>
+
+<step_4>
+**Write minimal instructions**: Create just enough content to address the gaps and pass evaluations.
+</step_4>
+
+<step_5>
+**Iterate**: Execute evaluations, compare against baseline, and refine.
+</step_5>
+</workflow>
+
+<evaluation_structure>
+
+```json
+{
+  "skills": ["pdf-processing"],
+  "query": "Extract all text from this PDF file and save it to output.txt",
+  "files": ["test-files/document.pdf"],
+  "expected_behavior": [
+    "Successfully reads the PDF file using appropriate library",
+    "Extracts text content from all pages without missing any",
+    "Saves extracted text to output.txt in clear, readable format"
+  ]
+}
+```
+
+</evaluation_structure>
+
+<why_evaluations_first>
+
+- Prevents documenting imagined problems
+- Forces clarity about what success looks like
+- Provides objective measurement of skill effectiveness
+- Keeps skill focused on actual needs
+- Enables quantitative improvement tracking
+</why_evaluations_first>
+</evaluation_driven_development>
+
+<iterative_development_with_claude>
+<principle>
+The most effective skill development uses Claude itself. Work with "Claude A" (expert who helps refine) to create skills used by "Claude B" (agent executing tasks).
+</principle>
+
+<creating_skills>
+<workflow>
+<step_1>
+**Complete task without skill**: Work through problem with Claude A, noting what context you repeatedly provide.
+</step_1>
+
+<step_2>
+**Ask Claude A to create skill**: "Create a skill that captures this pattern we just used"
+</step_2>
+
+<step_3>
+**Review for conciseness**: Remove unnecessary explanations.
+</step_3>
+
+<step_4>
+**Improve architecture**: Organize content with progressive disclosure.
+</step_4>
+
+<step_5>
+**Test with Claude B**: Use fresh instance to test on real tasks.
+</step_5>
+
+<step_6>
+**Iterate based on observation**: Return to Claude A with specific issues observed.
+</step_6>
+</workflow>
+
+<insight>
+Claude models understand skill format natively. Simply ask Claude to create a skill and it will generate properly structured SKILL.md content.
+</insight>
+</creating_skills>
+
+<improving_skills>
+<workflow>
+<step_1>
+**Use skill in real workflows**: Give Claude B actual tasks.
+</step_1>
+
+<step_2>
+**Observe behavior**: Where does it struggle, succeed, or make unexpected choices?
+</step_2>
+
+<step_3>
+**Return to Claude A**: Share observations and current SKILL.md.
+</step_3>
+
+<step_4>
+**Review suggestions**: Claude A might suggest reorganization, stronger language, or workflow restructuring.
+</step_4>
+
+<step_5>
+**Apply and test**: Update skill and test again.
+</step_5>
+
+<step_6>
+**Repeat**: Continue based on real usage, not assumptions.
+</step_6>
+</workflow>
+
+<what_to_watch_for>
+
+- **Unexpected exploration paths**: Structure might not be intuitive
+- **Missed connections**: Links might need to be more explicit
+- **Overreliance on sections**: Consider moving frequently-read content to main SKILL.md
+- **Ignored content**: Poorly signaled or unnecessary files
+- **Critical metadata**: The name and description in your skill's metadata are critical for discovery
+</what_to_watch_for>
+</improving_skills>
+</iterative_development_with_claude>
+
+<model_testing>
+<principle>
+Test with all models you plan to use. Different models have different strengths and need different levels of detail.
+</principle>
+
+<haiku_testing>
+**Claude Haiku** (fast, economical)
+
+Questions to ask:
+
+- Does the skill provide enough guidance?
+- Are examples clear and complete?
+- Do implicit assumptions become explicit?
+- Does Haiku need more structure?
+
+Haiku benefits from:
+
+- More explicit instructions
+- Complete examples (no partial code)
+- Clear success criteria
+- Step-by-step workflows
+</haiku_testing>
+
+<sonnet_testing>
+**Claude Sonnet** (balanced)
+
+Questions to ask:
+
+- Is the skill clear and efficient?
+- Does it avoid over-explanation?
+- Are workflows well-structured?
+- Does progressive disclosure work?
+
+Sonnet benefits from:
+
+- Balanced detail level
+- Clear heading structure
+- Progressive disclosure
+- Concise but complete guidance
+</sonnet_testing>
+
+<opus_testing>
+**Claude Opus** (powerful reasoning)
+
+Questions to ask:
+
+- Does the skill avoid over-explaining?
+- Can Opus infer obvious steps?
+- Are constraints clear?
+- Is context minimal but sufficient?
+
+Opus benefits from:
+
+- Concise instructions
+- Principles over procedures
+- High degrees of freedom
+- Trust in reasoning capabilities
+</opus_testing>
+
+<balancing_across_models>
+What works for Opus might need more detail for Haiku. Aim for instructions that work well across all target models. Find the balance that serves your target audience.
+
+See [core-principles.md](core-principles.md) for model testing examples.
+</balancing_across_models>
+</model_testing>
+
+<heading_structure_validation>
+<principle>
+During testing, validate that your skill's heading structure is correct and complete. Markdown headings structure a skill body; XML tags are optional semantic wrappers inside a section and never replace headings.
+</principle>
+
+<validation_checklist>
+After updating a skill, verify:
+
+<required_sections_present>
+
+- ✅ A `#` title followed by what the skill does
+- ✅ A `## Quick start` (or first numbered phase) with immediate guidance
+- ✅ A completion criterion (`## Success criteria` or a per-phase exit condition)
+</required_sections_present>
+
+<heading_hierarchy>
+
+- ✅ Body sections are `#` / `##` / `###` headings, in a sensible hierarchy
+- ✅ No top-level tag stands in for a section
+- ✅ Markdown formatting within sections is preserved (bold, italic, lists, code blocks)
+</heading_hierarchy>
+
+<optional_wrappers>
+
+- ✅ Any XML wrapper sits inside a section and bounds one block
+- ✅ Every wrapper is closed
+</optional_wrappers>
+
+<conditional_sections_appropriate>
+
+- ✅ Conditional sections match skill complexity
+- ✅ Simple skills use required sections only
+- ✅ Complex skills add appropriate conditional sections
+- ✅ No over-engineering or under-specifying
+</conditional_sections_appropriate>
+
+<reference_files_check>
+
+- ✅ Reference files are also structured with markdown headings
+- ✅ Links to reference files are correct
+- ✅ References are one level deep from SKILL.md
+</reference_files_check>
+</validation_checklist>
+
+<testing_structure_during_iteration>
+When iterating on a skill:
+
+1. Make changes to the section structure
+2. **Validate heading structure** (hierarchy, required sections, closed wrappers)
+3. Test with Claude on representative tasks
+4. Observe whether the structure aids or hinders Claude's understanding
+5. Iterate structure based on actual performance
+</testing_structure_during_iteration>
+</heading_structure_validation>
+
+<observation_based_iteration>
+<principle>
+Iterate based on what you observe, not what you assume. Real usage reveals issues assumptions miss.
+</principle>
+
+<observation_categories>
+<what_claude_reads>
+Which sections does Claude actually read? Which are ignored? This reveals:
+
+- Relevance of content
+- Effectiveness of progressive disclosure
+- Whether section names are clear
+</what_claude_reads>
+
+<where_claude_struggles>
+Which tasks cause confusion or errors? This reveals:
+
+- Missing context
+- Unclear instructions
+- Insufficient examples
+- Ambiguous requirements
+</where_claude_struggles>
+
+<where_claude_succeeds>
+Which tasks go smoothly? This reveals:
+
+- Effective patterns
+- Good examples
+- Clear instructions
+- Appropriate detail level
+</where_claude_succeeds>
+
+<unexpected_behaviors>
+What does Claude do that surprises you? This reveals:
+
+- Unstated assumptions
+- Ambiguous phrasing
+- Missing constraints
+- Alternative interpretations
+</unexpected_behaviors>
+</observation_categories>
+
+<iteration_pattern>
+
+1. **Observe**: Run Claude on real tasks with current skill
+2. **Document**: Note specific issues, not general feelings
+3. **Hypothesize**: Why did this issue occur?
+4. **Fix**: Make targeted changes to address specific issues
+5. **Test**: Verify fix works on same scenario
+6. **Validate**: Ensure fix doesn't break other scenarios
+7. **Repeat**: Continue with next observed issue
+</iteration_pattern>
+</observation_based_iteration>
+
+<progressive_refinement>
+<principle>
+Skills don't need to be perfect initially. Start minimal, observe usage, add what's missing.
+</principle>
+
+<initial_version>
+Start with:
+
+- Valid YAML frontmatter
+- Required sections: title + purpose, quick start, completion criterion
+- Minimal working example
+- Basic success criteria
+
+Skip initially:
+
+- Extensive examples
+- Edge case documentation
+- Advanced features
+- Detailed reference files
+</initial_version>
+
+<iteration_additions>
+Add through iteration:
+
+- Examples when patterns aren't clear from description
+- Edge cases when observed in real usage
+- Advanced features when users need them
+- Reference files when SKILL.md approaches 500 lines
+- Validation scripts when errors are common
+</iteration_additions>
+
+<benefits>
+- Faster to initial working version
+- Additions solve real needs, not imagined ones
+- Keeps skills focused and concise
+- Progressive disclosure emerges naturally
+- Documentation stays aligned with actual usage
+</benefits>
+</progressive_refinement>
+
+<testing_discovery>
+<principle>
+Test that Claude can discover and use your skill when appropriate.
+</principle>
+
+<discovery_testing>
+<test_description>
+Test if Claude loads your skill when it should:
+
+1. Start fresh conversation (Claude B)
+2. Ask question that should trigger skill
+3. Check if skill was loaded
+4. Verify skill was used appropriately
+</test_description>
+
+<description_quality>
+If skill isn't discovered:
+
+- Check description includes trigger keywords
+- Verify description is specific, not vague
+- Ensure description explains when to use skill
+- Test with different phrasings of the same request
+
+The description is Claude's primary discovery mechanism.
+</description_quality>
+</discovery_testing>
+</testing_discovery>
+
+<common_iteration_patterns>
+<pattern name="too_verbose">
+**Observation**: Skill works but uses lots of tokens
+
+**Fix**:
+
+- Remove obvious explanations
+- Assume Claude knows common concepts
+- Use examples instead of lengthy descriptions
+- Move advanced content to reference files
+</pattern>
+
+<pattern name="too_minimal">
+**Observation**: Claude makes incorrect assumptions or misses steps
+
+**Fix**:
+
+- Add explicit instructions where assumptions fail
+- Provide complete working examples
+- Define edge cases
+- Add validation steps
+</pattern>
+
+<pattern name="poor_discovery">
+**Observation**: Skill exists but Claude doesn't load it when needed
+
+**Fix**:
+
+- Improve description with specific triggers
+- Add relevant keywords
+- Test description against actual user queries
+- Make description more specific about use cases
+</pattern>
+
+<pattern name="unclear_structure">
+**Observation**: Claude reads wrong sections or misses relevant content
+
+**Fix**:
+
+- Use clearer section headings
+- Reorganize content hierarchy
+- Move frequently-needed content earlier
+- Add explicit links to relevant sections
+</pattern>
+
+<pattern name="incomplete_examples">
+**Observation**: Claude produces outputs that don't match expected pattern
+
+**Fix**:
+
+- Add more examples showing pattern
+- Make examples more complete
+- Show edge cases in examples
+- Add anti-pattern examples (what not to do)
+</pattern>
+
+</common_iteration_patterns>
+
+<iteration_velocity>
+<principle>
+Small, frequent iterations beat large, infrequent rewrites.
+</principle>
+
+<fast_iteration>
+**Good approach**:
+
+1. Make one targeted change
+2. Test on specific scenario
+3. Verify improvement
+4. Commit change
+5. Move to next issue
+
+Total time: Minutes per iteration
+Iterations per day: 10-20
+Learning rate: High
+</fast_iteration>
+
+<slow_iteration>
+**Problematic approach**:
+
+1. Accumulate many issues
+2. Make large refactor
+3. Test everything at once
+4. Debug multiple issues simultaneously
+5. Hard to know what fixed what
+
+Total time: Hours per iteration
+Iterations per day: 1-2
+Learning rate: Low
+</slow_iteration>
+
+<benefits_of_fast_iteration>
+
+- Isolate cause and effect
+- Build pattern recognition faster
+- Less wasted work from wrong directions
+- Easier to revert if needed
+- Maintains momentum
+</benefits_of_fast_iteration>
+</iteration_velocity>
+
+<success_metrics>
+<principle>
+Define how you'll measure if the skill is working. Quantify success.
+</principle>
+
+<objective_metrics>
+
+- **Success rate**: Percentage of tasks completed correctly
+- **Token usage**: Average tokens consumed per task
+- **Iteration count**: How many tries to get correct output
+- **Error rate**: Percentage of tasks with errors
+- **Discovery rate**: How often skill loads when it should
+</objective_metrics>
+
+<subjective_metrics>
+
+- **Output quality**: Does output meet requirements?
+- **Appropriate detail**: Too verbose or too minimal?
+- **Claude confidence**: Does Claude seem uncertain?
+- **User satisfaction**: Does skill solve the actual problem?
+</subjective_metrics>
+
+<tracking_improvement>
+Compare metrics before and after changes:
+
+- Baseline: Measure without skill
+- Initial: Measure with first version
+- Iteration N: Measure after each change
+
+Track which changes improve which metrics. Double down on effective patterns.
+</tracking_improvement>
+</success_metrics>

package/skills/skill-creator/references/official-spec.md

@@ -0,0 +1,193 @@
+# Anthropic Official Skill Specification
+
+Source: [code.claude.com/docs/en/skills](https://code.claude.com/docs/en/skills)
+
+## SKILL.md File Structure
+
+Every Skill requires a `SKILL.md` file with YAML frontmatter followed by Markdown instructions.
+
+### Basic Format
+
+```markdown
+---
+name: your-skill-name
+description: Brief description of what this Skill does and when to use it
+---
+
+# Your Skill Name
+
+## Instructions
+Provide clear, step-by-step guidance for Claude.
+
+## Examples
+Show concrete examples of using this Skill.
+```
+
+## Required Frontmatter Fields
+
+| Field | Required | Description |
+|-------|----------|-------------|
+| `name` | Yes | Skill name using lowercase letters, numbers, and hyphens only (max 64 characters). Should match the directory name. |
+| `description` | Yes | What the Skill does and when to use it (max 1024 characters). Claude uses this to decide when to apply the Skill. |
+| `allowed-tools` | No | Tools Claude can use without asking permission when this Skill is active. Example: `Read, Grep, Glob` |
+| `disallowed-tools` | No | Tools REMOVED from the pool while the skill is active. Restriction clears on the operator's next message (per-turn only). Space/comma-separated string or YAML list. |
+| `model` | No | Specific model to use when this Skill is active (e.g., `claude-sonnet-5-5`). Defaults to the conversation's model. |
+| `disable-model-invocation` | No | Set to `true` to prevent Claude from loading the Skill automatically; it runs only when the user types `/name`. The description is then not in context. Also prevents preloading into subagents and, as of v2.1.196, running when a scheduled task fires with the Skill as its prompt. Default: `false`. Source: [code.claude.com/docs/en/skills](https://code.claude.com/docs/en/skills). Soleur's decision test for setting it is in [authoring-levers.md](authoring-levers.md). |
+| `user-invocable` | No | Set to `false` when only Claude should invoke the Skill: Claude Code hides it from the `/` menu and does not run it when the user types `/name`. The description stays in context. For background knowledge users should not invoke directly. Default: `true`. Source: [code.claude.com/docs/en/skills](https://code.claude.com/docs/en/skills). |
+
+> **Security note (do not claim `allowed-tools` is a sandbox).** `allowed-tools` is **pre-approval only** — the official spec states it "does not restrict which tools are available: every tool remains callable." A skill listing only MCP tools does NOT prevent `Bash`/`Read`/`Write`; it only suppresses the prompt for the listed tools. Never write a plan/ADR/skill that claims "the skill physically cannot read X because X isn't in `allowed-tools`." Durable per-skill tool restriction requires a subagent boundary (`context: fork` + `tools:`) or a committed `.claude/settings.json` `permissions.deny` rule; `disallowed-tools` only covers the intra-turn window. See `knowledge-base/project/learnings/security-issues/2026-07-10-skill-allowed-tools-is-pre-approval-not-a-sandbox.md` (ADR-107 / #6260).
+
+## Skill Locations & Priority
+
+```
+Enterprise (highest priority) → Personal → Project → Plugin (lowest priority)
+```
+
+| Type | Path | Applies to |
+|------|------|-----------|
+| **Enterprise** | See managed settings | All users in organization |
+| **Personal** | `~/.claude/skills/` | You, across all projects |
+| **Project** | `.claude/skills/` | Anyone working in repository |
+| **Plugin** | Bundled with plugins | Anyone with plugin installed |
+
+## How Skills Work
+
+1. **Discovery**: Claude loads only name and description at startup
+2. **Activation**: When your request matches a Skill's description, Claude asks for confirmation
+3. **Execution**: Claude follows the Skill's instructions and loads referenced files
+
+**Key Principle**: Skills are **model-invoked** — Claude automatically decides which Skills to use based on your request.
+
+## Progressive Disclosure Pattern
+
+Keep `SKILL.md` under 500 lines by linking to supporting files:
+
+```
+my-skill/
+├── SKILL.md (required - overview and navigation)
+├── reference.md (detailed API docs - loaded when needed)
+├── examples.md (usage examples - loaded when needed)
+└── scripts/
+    └── helper.py (utility script - executed, not loaded)
+```
+
+### Example SKILL.md with References
+
+```markdown
+---
+name: pdf-processing
+description: Extract text, fill forms, merge PDFs. Use when working with PDF files, forms, or document extraction. Requires pypdf and pdfplumber packages.
+allowed-tools: Read, Bash(python:*)
+---
+
+# PDF Processing
+
+## Quick start
+
+Extract text:
+```python
+import pdfplumber
+with pdfplumber.open("doc.pdf") as pdf:
+    text = pdf.pages[0].extract_text()
+```
+
+For form filling, see [FORMS.md](FORMS.md).
+For detailed API reference, see [REFERENCE.md](REFERENCE.md).
+
+## Requirements
+
+Packages must be installed:
+
+```bash
+pip install pypdf pdfplumber
+```
+
+```
+
+## Restricting Tool Access
+
+```yaml
+---
+name: reading-files-safely
+description: Read files without making changes. Use when you need read-only file access.
+allowed-tools: Read, Grep, Glob
+---
+```
+
+Benefits:
+
+- Read-only Skills that shouldn't modify files
+- Limited scope for specific tasks
+- Security-sensitive workflows
+
+## Writing Effective Descriptions
+
+The `description` field enables Skill discovery and should include both what the Skill does and when to use it.
+
+**Always write in third person.** The description is injected into the system prompt.
+
+- **Good:** "Processes Excel files and generates reports"
+- **Avoid:** "I can help you process Excel files"
+- **Avoid:** "You can use this to process Excel files"
+
+**Be specific and include key terms:**
+
+```yaml
+description: Extract text and tables from PDF files, fill forms, merge documents. Use when working with PDF files or when the user mentions PDFs, forms, or document extraction.
+```
+
+**Avoid vague descriptions:**
+
+```yaml
+description: Helps with documents  # Too vague!
+```
+
+## Complete Example: Commit Message Generator
+
+```markdown
+---
+name: generating-commit-messages
+description: Generates clear commit messages from git diffs. Use when writing commit messages or reviewing staged changes.
+---
+
+# Generating Commit Messages
+
+## Instructions
+
+1. Run `git diff --staged` to see changes
+2. I'll suggest a commit message with:
+   - Summary under 50 characters
+   - Detailed description
+   - Affected components
+
+## Best practices
+
+- Use present tense
+- Explain what and why, not how
+```
+
+## Complete Example: Code Explanation Skill
+
+```markdown
+---
+name: explaining-code
+description: Explains code with visual diagrams and analogies. Use when explaining how code works, teaching about a codebase, or when the user asks "how does this work?"
+---
+
+# Explaining Code
+
+When explaining code, always include:
+
+1. **Start with an analogy**: Compare the code to something from everyday life
+2. **Draw a diagram**: Use ASCII art to show the flow, structure, or relationships
+3. **Walk through the code**: Explain step-by-step what happens
+4. **Highlight a gotcha**: What's a common misconception?
+
+Keep explanations conversational. For complex concepts, use multiple analogies.
+```
+
+## Distribution
+
+- **Project Skills**: Commit `.claude/skills/` to version control
+- **Plugins**: Add `skills/` directory to plugin with Skill folders
+- **Enterprise**: Deploy organization-wide through managed settings