@dzhechkov/skills-bto 1.0.0

package/templates/.claude/skills/bto/modules/optimize.md

@@ -0,0 +1,201 @@
+# OPTIMIZE Module — Evolutionary Prompt Optimization Protocol
+
+## Purpose
+
+Improve Claude Code artifacts through evolutionary prompt optimization: generate variants, evaluate, select, mutate, repeat.
+
+## Input
+
+- **Path:** Path to artifact to optimize
+- **Rounds:** Number of optimization rounds (default: 3, max: 5)
+- **Budget:** max evaluations (default: 15)
+- **Focus:** Optional dimension to prioritize (METHODOLOGY / DEPTH / CORRECTNESS / USABILITY / ROBUSTNESS)
+
+## Prerequisites
+
+- Artifact must pass Layer 0 checks (run TEST first)
+- Baseline score established (Layer 2 evaluation)
+- Only optimize if baseline < 8.0 (otherwise artifact is already good)
+
+---
+
+## Protocol
+
+### Step 1: Baseline Evaluation
+
+1. Run TEST module with level=layer2 on current artifact
+2. Record:
+   - Per-dimension scores
+   - Overall score
+   - Key weaknesses identified by judges
+3. If overall ≥ 8.0 → report "Artifact already high quality" and suggest minor tweaks only
+4. Identify **target dimensions**: dimensions scoring < 7.0
+
+### Step 2: Variant Generation (Round 1)
+
+Generate N=5 variants of the artifact. Each variant applies ONE mutation strategy.
+
+**Mutation Assignment:**
+- Variant 1: **Rephrase** — reword unclear instructions for precision
+- Variant 2: **Restructure** — reorganize sections for better flow
+- Variant 3: **Add Constraints** — add guardrails, edge cases, boundary conditions
+- Variant 4: **Simplify** — remove redundancy, tighten language, reduce verbosity
+- Variant 5: **Specialize** — add domain-specific context and examples
+
+**Strategy-to-Weakness Mapping:**
+
+| Weak Dimension | Primary Strategy | Secondary Strategy |
+|---------------|-----------------|-------------------|
+| METHODOLOGY | Restructure | Add Constraints |
+| DEPTH | Specialize | Add Constraints |
+| CORRECTNESS | Add Constraints | Rephrase |
+| USABILITY | Rephrase | Restructure |
+| ROBUSTNESS | Add Constraints | Specialize |
+
+If a focus dimension is specified, generate 3 variants targeting that dimension's strategies and 2 variants with other strategies.
+
+### Variant Generation Prompt
+
+```
+You are optimizing a Claude Code {artifact_type}.
+
+## Current Artifact
+{content}
+
+## Baseline Evaluation
+Overall: {score}/10
+Weaknesses: {weaknesses}
+Target dimensions: {target_dimensions}
+
+## Mutation Strategy: {strategy_name}
+{strategy_description}
+
+## Task
+Apply the {strategy_name} mutation to improve this artifact.
+Focus on addressing these specific weaknesses: {target_weaknesses}
+
+Rules:
+- Preserve the original intent and scope
+- Maintain all existing sections
+- Do not change the artifact type or structure fundamentally
+- Changes should be targeted and purposeful
+- Output the COMPLETE modified artifact (not a diff)
+```
+
+### Step 3: Evaluate Variants (Round 1)
+
+Run TEST module with level=layer1 (haiku — fast and cheap) on each variant.
+
+**Parallel execution:** Spawn 5 agents (model: haiku), one per variant.
+
+Record scores for all 5 variants.
+
+### Step 4: Selection + Crossover
+
+1. **Select:** Top 2 variants by overall score
+2. **Crossover:** Combine best elements:
+   - Take sections where Variant A scored higher from A
+   - Take sections where Variant B scored higher from B
+   - Generate 3 new variants from the crossover
+
+**Crossover Prompt:**
+```
+You are creating an improved Claude Code artifact by combining the best
+elements of two high-scoring variants.
+
+## Variant A (Score: {score_a})
+{variant_a}
+Strengths: {strengths_a}
+
+## Variant B (Score: {score_b})
+{variant_b}
+Strengths: {strengths_b}
+
+## Task
+Create a new variant that combines the strengths of both:
+- From A, take: {specific_sections_a}
+- From B, take: {specific_sections_b}
+- Ensure coherence and consistency
+- Output the COMPLETE artifact
+```
+
+### Step 5: Evaluate + Select (Rounds 2-3)
+
+**Round 2:**
+- Evaluate 3 crossover variants with Layer 1
+- Select top 2
+- Generate 3 new crossover variants
+
+**Round 3 (Final):**
+- Evaluate 3 final variants with **Layer 2** (full judge panel — thorough)
+- Select the single best variant
+
+### Step 6: Output
+
+1. **Best variant** — the optimized artifact
+2. **Before/After comparison:**
+   ```
+   ═══════════════════════════════════════════════════════
+   🔧 BTO OPTIMIZATION REPORT
+   Artifact: <path>
+   Rounds: 3
+   Total evaluations: 15
+
+   BEFORE → AFTER:
+     METHODOLOGY:  6.2 → 8.1  (+1.9) ⬆️
+     DEPTH:        5.8 → 7.5  (+1.7) ⬆️
+     CORRECTNESS:  7.0 → 8.3  (+1.3) ⬆️
+     USABILITY:    6.5 → 8.0  (+1.5) ⬆️
+     ROBUSTNESS:   5.5 → 7.8  (+2.3) ⬆️
+
+     OVERALL:      6.2 → 7.9  (+1.7) ⬆️
+
+   Winning Strategy: Restructure + Add Constraints (crossover)
+
+   CHANGELOG:
+   - Restructured protocol into clearer numbered steps
+   - Added edge case handling for empty inputs
+   - Expanded anti-patterns with 3 new entries
+   - Simplified module loading instructions
+   - Added concrete examples to each section
+   ═══════════════════════════════════════════════════════
+   ```
+
+3. **Recommendation:**
+   - If improvement > 1.0: "Apply changes"
+   - If improvement 0.5-1.0: "Review changes, consider applying"
+   - If improvement < 0.5: "Minimal improvement — original may be preferred"
+
+---
+
+## Cost Summary
+
+| Operation | Count | Model | Est. Tokens |
+|-----------|-------|-------|-------------|
+| Baseline eval | 1 | sonnet ×3 | ~15K |
+| Variant generation | 5 | opus | ~25K |
+| Round 1 eval | 5 | haiku | ~10K |
+| Crossover generation | 3 | opus | ~15K |
+| Round 2 eval | 3 | haiku | ~6K |
+| Crossover generation | 3 | opus | ~15K |
+| Round 3 eval | 3 | sonnet ×3 | ~45K |
+| **Total** | | | **~131K tokens** |
+
+## Anti-Patterns
+
+| Anti-Pattern | Detection | Fix |
+|-------------|-----------|-----|
+| Overfitting to one metric | One dimension +3, others flat or down | Balance mutations across dimensions |
+| Losing generality | Specialized variant breaks other use cases | Test with multiple example inputs |
+| Infinite loop | > 5 rounds without improvement | Hard cap at configured rounds |
+| Semantic drift | Optimized version changes intent | Compare purpose statement before/after |
+| Premature optimization | Baseline ≥ 8.0 | Skip optimization, suggest minor tweaks |
+| Score inflation | Haiku gives high scores to everything | Calibrate with sonnet baseline |
+
+## Abort Conditions
+
+Stop optimization immediately if:
+1. Any round shows overall regression > 0.5 from baseline
+2. Critical structural checks (Layer 0) fail on any variant
+3. Artifact semantics change fundamentally
+4. User requests stop

package/templates/.claude/skills/bto/modules/test.md

@@ -0,0 +1,348 @@
+# TEST Module — Multi-Agent Evaluation Protocol
+
+## Purpose
+
+Evaluate any Claude Code artifact (skill, command, rule, agent template) using layered evaluation: deterministic pre-checks, single-judge quick eval, and full 3-judge panel.
+
+## Input
+
+- **Path:** Path to artifact file or directory
+- **Level:** layer0 | layer1 | layer2 | full (default: full)
+- **Artifact type:** auto-detected from path/content
+
+## Type Detection
+
+| Path Pattern | Detected Type |
+|-------------|--------------|
+| `.claude/skills/*/SKILL.md` | skill |
+| `.claude/skills/*/` (directory) | skill |
+| `.claude/commands/*.md` | command |
+| `.claude/rules/*.md` | rule |
+| `.claude/agents/*.md` | agent |
+| `researches/**/*.md` | research artifact |
+
+---
+
+## Layer 0: Deterministic Pre-checks
+
+**Cost:** Zero (no LLM calls)
+**Speed:** Instant
+**Purpose:** Catch structural issues before expensive LLM evaluation
+
+### Universal Checks (all types)
+
+```
+CHECK-U1: File exists and is non-empty
+CHECK-U2: File is valid UTF-8 text
+CHECK-U3: Has at least one markdown heading (#)
+CHECK-U4: No consecutive empty lines (> 2)
+CHECK-U5: File size within bounds (see per-type limits)
+```
+
+### Skill Checks
+
+```
+CHECK-S1: SKILL.md exists in skill directory
+CHECK-S2: Has "# Title" as first heading
+CHECK-S3: Has "## Overview" or "## Purpose" section
+CHECK-S4: Has "## Anti-Patterns" section
+CHECK-S5: All files in modules/ referenced in SKILL.md
+CHECK-S6: All files in references/ referenced in SKILL.md
+CHECK-S7: No empty sections (heading → next heading with no content)
+CHECK-S8: Size: 1KB < SKILL.md < 50KB
+CHECK-S9: Total directory size < 200KB
+CHECK-S10: At least one file in references/ OR examples/
+```
+
+### Command Checks
+
+```
+CHECK-C1: Contains "$ARGUMENTS" or parameter reference
+CHECK-C2: Has checkpoint banner or protocol
+CHECK-C3: Has skill loading instruction (Read *.SKILL.md)
+CHECK-C4: Size: 500B < file < 20KB
+CHECK-C5: Has "## Usage" or "## Protocol" section
+```
+
+### Rule Checks
+
+```
+CHECK-R1: Has table or structured list of patterns
+CHECK-R2: Each pattern has detection signal and fix
+CHECK-R3: Size: 200B < file < 10KB
+CHECK-R4: Has "Auto-Detection" or similar section
+```
+
+### Agent Template Checks
+
+```
+CHECK-A1: Specifies model (haiku/sonnet/opus)
+CHECK-A2: Specifies isolation scope
+CHECK-A3: Has prompt template or instructions
+CHECK-A4: Size: 200B < file < 10KB
+```
+
+### Layer 0 Scoring
+
+- Each check: PASS (1) or FAIL (0)
+- Score = passed / total
+- **Gate: score ≥ 0.80 to proceed to Layer 1+**
+- If score < 0.80: return report with specific failures, skip LLM evaluation
+
+### Layer 0 Output
+
+```
+═══════════════════════════════════════════════════════
+📋 LAYER 0: Deterministic Pre-checks
+Artifact: <path>
+Type: <detected type>
+
+Results: X/Y passed (Z%)
+
+✅ CHECK-S1: SKILL.md exists
+✅ CHECK-S2: Has title heading
+❌ CHECK-S7: Empty section found at line 45
+...
+
+Gate: PASS ✅ / FAIL ❌
+═══════════════════════════════════════════════════════
+```
+
+---
+
+## Layer 1: Single LLM Judge (Quick)
+
+**Cost:** Low (1 haiku call)
+**Speed:** ~10 seconds
+**Purpose:** Fast quality signal for iteration or batch evaluation
+
+### Model Selection
+
+- Default: **haiku** (cost-optimized)
+- For critical artifacts: **sonnet**
+
+### Judge Prompt
+
+```
+You are evaluating a Claude Code {artifact_type}.
+
+## Artifact Content
+{content}
+
+## Evaluation Dimensions
+
+Rate each dimension 1-10:
+
+1. CLARITY (Are instructions unambiguous? Can an LLM follow them precisely?)
+2. COMPLETENESS (Are all necessary sections present? No missing pieces?)
+3. ACTIONABILITY (Can Claude produce concrete output from these instructions?)
+4. QUALITY (Well-structured? Professional? Good formatting?)
+5. ANTI-PATTERNS (Avoids known pitfalls? Has failure mode coverage?)
+
+## Required Output Format
+
+SCORES:
+- CLARITY: X/10 — [one-line justification]
+- COMPLETENESS: X/10 — [one-line justification]
+- ACTIONABILITY: X/10 — [one-line justification]
+- QUALITY: X/10 — [one-line justification]
+- ANTI-PATTERNS: X/10 — [one-line justification]
+
+AVERAGE: X.X/10
+
+TOP 3 IMPROVEMENTS:
+1. [specific, actionable suggestion]
+2. [specific, actionable suggestion]
+3. [specific, actionable suggestion]
+
+VERDICT: PASS (≥7.0) / NEEDS WORK (5.0-6.9) / FAIL (<5.0)
+```
+
+### Layer 1 Gate
+
+- Average ≥ 7.0: PASS
+- Average 5.0-6.9: NEEDS WORK (can proceed to Layer 2 for detailed feedback)
+- Average < 5.0: FAIL (fix before proceeding)
+
+---
+
+## Layer 2: Full Judge Panel (3 Agents)
+
+**Cost:** Moderate (3 sonnet calls)
+**Speed:** ~30 seconds (parallel)
+**Purpose:** Comprehensive, multi-perspective evaluation
+
+### Architecture
+
+Spawn 3 agents in parallel using Agent tool:
+
+```
+Agent 1: "BTO Judge — Domain Expert"    model: sonnet
+Agent 2: "BTO Judge — Critic"            model: sonnet
+Agent 3: "BTO Judge — Completeness"      model: sonnet
+```
+
+**Isolation:** Each agent reads the same artifact independently. No cross-communication.
+
+### Judge 1: Domain Expert
+
+Focus: Is the content technically sound and appropriate for the domain?
+
+```
+You are a Domain Expert evaluator for Claude Code artifacts.
+
+Evaluate this {artifact_type} for technical quality:
+
+{content}
+
+Score 1-10 on each dimension:
+1. METHODOLOGY — Is the approach well-designed? Sound structure?
+2. DEPTH — Sufficient detail for the task? Not too shallow?
+3. CORRECTNESS — Are all claims and instructions valid?
+4. USABILITY — Can a user/agent effectively use this?
+5. ROBUSTNESS — Handles edge cases and failure modes?
+
+For each dimension provide:
+- Score (1-10)
+- 2-3 sentence justification
+- One specific improvement suggestion
+
+OVERALL_SCORE: weighted average
+CONFIDENCE: HIGH/MEDIUM/LOW
+KEY_STRENGTHS: [top 2]
+KEY_WEAKNESSES: [top 2]
+```
+
+### Judge 2: Critic
+
+Focus: Find weaknesses, gaps, and anti-patterns.
+
+```
+You are a Critical Evaluator. Your job is to find problems.
+
+Evaluate this {artifact_type} adversarially:
+
+{content}
+
+Score 1-10 on each dimension (be strict — average should be ~5-6):
+1. METHODOLOGY — Any logical flaws or unjustified assumptions?
+2. DEPTH — What's missing? What's under-specified?
+3. CORRECTNESS — Any instructions that could mislead or produce wrong output?
+4. USABILITY — What would confuse a user? Where would someone get stuck?
+5. ROBUSTNESS — What breaks this? What edge cases are unhandled?
+
+For each dimension provide:
+- Score (1-10) — err on the side of strict
+- 2-3 sentence justification focusing on PROBLEMS
+- One specific failure scenario
+
+OVERALL_SCORE: weighted average
+CRITICAL_ISSUES: [list of blocking problems]
+IMPROVEMENT_PRIORITY: [ordered list of what to fix first]
+```
+
+### Judge 3: Completeness Auditor
+
+Focus: Structural completeness and cross-reference integrity.
+
+```
+You are a Completeness Auditor for Claude Code artifacts.
+
+Audit this {artifact_type} for structural completeness:
+
+{content}
+
+Score 1-10 on each dimension:
+1. METHODOLOGY — Does the structure follow established patterns?
+2. DEPTH — Is every section adequately populated?
+3. CORRECTNESS — Do all cross-references resolve? Are all claims supported?
+4. USABILITY — Is the information well-organized and findable?
+5. ROBUSTNESS — Are failure modes documented? Are anti-patterns listed?
+
+For each dimension provide:
+- Score (1-10)
+- 2-3 sentence justification
+- Missing element (if any)
+
+OVERALL_SCORE: weighted average
+MISSING_SECTIONS: [list]
+BROKEN_REFERENCES: [list]
+COVERAGE_GAPS: [list]
+```
+
+### Score Aggregation
+
+After all 3 judges return:
+
+**Weights:**
+- Domain Expert: 0.40
+- Critic: 0.30
+- Completeness Auditor: 0.30
+
+**Per-dimension:**
+```
+dimension_score = expert[dim] * 0.4 + critic[dim] * 0.3 + auditor[dim] * 0.3
+```
+
+**Overall:**
+```
+overall = mean(all dimension_scores)
+```
+
+**Disagreement Detection:**
+For each dimension, if `max(scores) - min(scores) > 3` → FLAG for meta-judge.
+
+---
+
+## Meta-Judge: Disagreement Resolution
+
+**Trigger:** Any flagged dimension from Layer 2.
+
+**Model:** default (opus)
+
+**Prompt:**
+```
+Three judges evaluated a Claude Code artifact and disagreed significantly
+on the following dimension(s): {flagged_dimensions}
+
+Judge 1 (Domain Expert): {score} — {justification}
+Judge 2 (Critic): {score} — {justification}
+Judge 3 (Completeness): {score} — {justification}
+
+Reconcile these scores. Provide:
+1. Your reconciled score (1-10) with reasoning
+2. Which judge(s) had the most valid perspective and why
+3. Whether this needs human review (YES/NO)
+```
+
+---
+
+## Output: Evaluation Report
+
+Format defined in `examples/sample-eval-report.md`.
+
+Summary structure:
+```
+═══════════════════════════════════════════════════════
+📊 BTO EVALUATION REPORT
+Artifact: <path>
+Type: <type>
+Level: Layer 0 + Layer 1 + Layer 2
+
+OVERALL SCORE: X.X / 10  [PASS / NEEDS WORK / FAIL]
+
+Per-Dimension:
+  METHODOLOGY:  X.X  ██████████░░
+  DEPTH:        X.X  ████████░░░░
+  CORRECTNESS:  X.X  █████████░░░
+  USABILITY:    X.X  ██████████░░
+  ROBUSTNESS:   X.X  ███████░░░░░
+
+Flagged: [dimensions with >3 disagreement]
+
+Top Improvements:
+1. ...
+2. ...
+3. ...
+═══════════════════════════════════════════════════════
+```