@dzhechkov/skills-bto 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,192 @@
1
+ # Agent Template: BTO Judge Panel
2
+
3
+ ## Purpose
4
+ Evaluates a single artifact through a 3-judge panel (extendable to 5).
5
+ Each judge runs in isolation and scores the artifact on a shared rubric.
6
+ Reusable for any artifact type: skills, prompts, presentations, architectures.
7
+
8
+ ## Spawning Pattern
9
+ ```
10
+ Agent(
11
+ subagent_type="general-purpose",
12
+ description="BTO Judge Panel — [ARTIFACT_TYPE]",
13
+ prompt="""
14
+ Spawn 3 parallel judge agents for artifact: [ARTIFACT_PATH]
15
+
16
+ Each agent:
17
+ 1. Reads the artifact at [ARTIFACT_PATH]
18
+ 2. Reads the rubric at [RUBRIC_PATH]
19
+ 3. Scores independently (NO inter-judge communication)
20
+ 4. Writes evaluation to [EVAL_DIR]/judge-[N].md
21
+
22
+ After all 3 complete:
23
+ - Compute weighted average score
24
+ - Check disagreement threshold (max - min > 3 → escalate)
25
+ - Write aggregated result to [EVAL_DIR]/panel-verdict.md
26
+ """
27
+ )
28
+ ```
29
+
30
+ ## Agents
31
+
32
+ ### Agent 1: Domain Expert
33
+ ```
34
+ Agent(
35
+ subagent_type="general-purpose",
36
+ model="sonnet",
37
+ description="BTO Judge 1 — Domain Expert",
38
+ prompt="""
39
+ Role: Domain Expert evaluator. Weight: 0.4.
40
+
41
+ Read artifact: [ARTIFACT_PATH]
42
+ Read rubric: [RUBRIC_PATH]
43
+
44
+ Evaluate on:
45
+ - Domain accuracy: are claims correct for this field?
46
+ - Technical depth: does the artifact address non-obvious aspects?
47
+ - Practical applicability: can a practitioner use this as-is?
48
+
49
+ Scoring: integer 1–10 per criterion. No half-points.
50
+ Calibration: reserve 9-10 for genuinely exceptional work.
51
+
52
+ Output format (write to [EVAL_DIR]/judge-1.md):
53
+ ## Judge 1: Domain Expert
54
+ | Criterion | Score | Justification |
55
+ |-----------|-------|---------------|
56
+ | Domain accuracy | X | ... |
57
+ | Technical depth | X | ... |
58
+ | Practical applicability | X | ... |
59
+ **Weighted subtotal:** [score * 0.4]
60
+ **Strengths:** [2-3 bullet points]
61
+ **Critical gaps:** [2-3 bullet points]
62
+ """
63
+ )
64
+ ```
65
+
66
+ ### Agent 2: Critic
67
+ ```
68
+ Agent(
69
+ subagent_type="general-purpose",
70
+ model="sonnet",
71
+ description="BTO Judge 2 — Critic",
72
+ prompt="""
73
+ Role: Calibrated Critic evaluator. Weight: 0.3.
74
+ Calibration: you are instructed to be strict. If in doubt, score lower.
75
+
76
+ Read artifact: [ARTIFACT_PATH]
77
+ Read rubric: [RUBRIC_PATH]
78
+
79
+ Evaluate on:
80
+ - Logical consistency: no internal contradictions
81
+ - Verifiability: claims are traceable to sources or marked [ANALYSIS]
82
+ - Anti-pattern absence: none of the forbidden patterns from bto-quality-gates.md
83
+
84
+ Scoring: integer 1–10 per criterion. Penalize vague claims heavily.
85
+ If an anti-pattern is detected, cap the criterion score at 5.
86
+
87
+ Output format (write to [EVAL_DIR]/judge-2.md):
88
+ ## Judge 2: Critic
89
+ | Criterion | Score | Justification |
90
+ |-----------|-------|---------------|
91
+ | Logical consistency | X | ... |
92
+ | Verifiability | X | ... |
93
+ | Anti-pattern absence | X | ... |
94
+ **Weighted subtotal:** [score * 0.3]
95
+ **Anti-patterns detected:** [list or "none"]
96
+ **Blocking issues:** [list or "none"]
97
+ """
98
+ )
99
+ ```
100
+
101
+ ### Agent 3: Completeness Auditor
102
+ ```
103
+ Agent(
104
+ subagent_type="general-purpose",
105
+ model="sonnet",
106
+ description="BTO Judge 3 — Completeness Auditor",
107
+ prompt="""
108
+ Role: Completeness Auditor. Weight: 0.3.
109
+
110
+ Read artifact: [ARTIFACT_PATH]
111
+ Read rubric: [RUBRIC_PATH]
112
+ Read required structure spec: [STRUCTURE_SPEC_PATH]
113
+
114
+ Evaluate on:
115
+ - Section coverage: all required sections present and non-empty
116
+ - Depth per section: each section meets minimum depth (not a stub)
117
+ - Edge cases addressed: boundary conditions and failure modes mentioned
118
+
119
+ Scoring: integer 1–10 per criterion.
120
+ Deduct 2 points for each missing required section.
121
+ Deduct 1 point for each stub section (< 3 substantive sentences).
122
+
123
+ Output format (write to [EVAL_DIR]/judge-3.md):
124
+ ## Judge 3: Completeness Auditor
125
+ | Criterion | Score | Justification |
126
+ |-----------|-------|---------------|
127
+ | Section coverage | X | ... |
128
+ | Depth per section | X | ... |
129
+ | Edge cases addressed | X | ... |
130
+ **Weighted subtotal:** [score * 0.3]
131
+ **Missing sections:** [list or "none"]
132
+ **Stub sections:** [list or "none"]
133
+ """
134
+ )
135
+ ```
136
+
137
+ ## Aggregation Protocol
138
+ After all 3 judge agents complete, the orchestrator runs aggregation:
139
+
140
+ ```
141
+ 1. Read [EVAL_DIR]/judge-1.md, judge-2.md, judge-3.md
142
+ 2. Extract weighted subtotals: S1, S2, S3
143
+ 3. Compute panel_score = S1 + S2 + S3 (already weighted)
144
+ 4. Compute raw scores: r1, r2, r3 (average of each judge's criteria)
145
+ 5. Check disagreement: if max(r1,r2,r3) - min(r1,r2,r3) > 3 → escalate to meta-judge
146
+ 6. Write panel-verdict.md
147
+ ```
148
+
149
+ ### panel-verdict.md Format
150
+ ```markdown
151
+ ## Panel Verdict — [ARTIFACT_NAME]
152
+ **Panel score:** [X.X / 10]
153
+ **Judge scores:** Expert=[r1], Critic=[r2], Auditor=[r3]
154
+ **Disagreement flag:** [YES/NO]
155
+ **Decision:** [PASS / FAIL / ESCALATE]
156
+ **Pass threshold:** [defined in rubric, default 7.0]
157
+ **Top improvement areas:**
158
+ - [from Critic blocking issues]
159
+ - [from Auditor missing sections]
160
+ ```
161
+
162
+ ## Disagreement Escalation (Meta-Judge)
163
+ When disagreement > 3 points:
164
+ ```
165
+ Agent(
166
+ subagent_type="general-purpose",
167
+ model="sonnet",
168
+ description="BTO Meta-Judge",
169
+ prompt="""
170
+ Read all 3 judge evaluations: [EVAL_DIR]/judge-*.md
171
+ Read artifact: [ARTIFACT_PATH]
172
+ Read rubric: [RUBRIC_PATH]
173
+
174
+ Identify the source of disagreement.
175
+ Provide a final binding score with explicit reasoning.
176
+ Write to [EVAL_DIR]/meta-judge.md
177
+ """
178
+ )
179
+ ```
180
+
181
+ ## Configuration Variables
182
+ | Variable | Description | Example |
183
+ |----------|-------------|---------|
184
+ | ARTIFACT_PATH | Path to artifact being evaluated | researches/slug/03_solution_strategy.md |
185
+ | RUBRIC_PATH | Path to scoring rubric | .claude/rubrics/skill-rubric.md |
186
+ | STRUCTURE_SPEC_PATH | Path to required structure | .claude/specs/skill-structure.md |
187
+ | EVAL_DIR | Directory for evaluation outputs | researches/slug/evals/round-2/ |
188
+ | ARTIFACT_TYPE | Human-readable artifact type label | "Solution Strategy" |
189
+
190
+ ## Reusability Note
191
+ Swap ARTIFACT_PATH and RUBRIC_PATH to evaluate any artifact type:
192
+ skills, prompts, presentations, architecture documents, research findings.
@@ -0,0 +1,181 @@
1
+ # Agent Template: BTO Optimizer Worker
2
+
3
+ ## Purpose
4
+ Generates 1-2 prompt or skill variants using an assigned mutation strategy.
5
+ Used as one of N parallel workers in an optimization round.
6
+ Reusable for any prompt optimization task, not Keysarium-specific.
7
+
8
+ ## Spawning Pattern
9
+ ```
10
+ Agent(
11
+ subagent_type="general-purpose",
12
+ description="BTO Optimizer — [MUTATION_STRATEGY]",
13
+ prompt="""
14
+ Read the base artifact: [BASE_ARTIFACT_PATH]
15
+ Read the last evaluation: [LAST_EVAL_PATH]
16
+ Read the rubric: [RUBRIC_PATH]
17
+
18
+ Mutation strategy assigned: [MUTATION_STRATEGY]
19
+ Worker ID: [WORKER_ID]
20
+
21
+ Generate [1 or 2] variants using your assigned strategy.
22
+ Write each variant to [OUTPUT_DIR]/variant-[WORKER_ID]-[A|B].md
23
+
24
+ Then run a Layer 0 self-check on each variant before saving.
25
+ If Layer 0 fails, fix and retry once. If it fails again, skip that variant.
26
+
27
+ Write a brief mutation log to [OUTPUT_DIR]/mutation-log-[WORKER_ID].md
28
+ """
29
+ )
30
+ ```
31
+
32
+ ## Mutation Strategies
33
+
34
+ Assign one strategy per worker. Rotate strategies across rounds.
35
+
36
+ | Strategy | Description | Good For |
37
+ |----------|-------------|----------|
38
+ | `expand-depth` | Add concrete examples, edge cases, non-obvious details | Thin sections |
39
+ | `compress-clarity` | Remove redundancy, sharpen wording, improve signal/noise | Verbose artifacts |
40
+ | `reframe-domain` | Re-express the same content through a different domain lens | Generic claims |
41
+ | `add-metrics` | Replace vague claims with quantified statements | Abstract recommendations |
42
+ | `invert-critic` | Address top blocking issues from last evaluation | Low Critic score |
43
+ | `fill-gaps` | Expand missing or stub sections identified by Auditor | Low Auditor score |
44
+ | `crossover` | Blend two highest-scoring variants from previous round | Late-stage refinement |
45
+
46
+ ## Worker Configuration
47
+
48
+ ### Standard Round: 3 Workers
49
+ ```
50
+ Worker 1: mutation_strategy="expand-depth", variants=2, model="sonnet"
51
+ Worker 2: mutation_strategy="add-metrics", variants=1, model="sonnet"
52
+ Worker 3: mutation_strategy="invert-critic", variants=2, model="sonnet"
53
+ ```
54
+
55
+ ### Crossover Round: 2 Workers (after round 3+)
56
+ ```
57
+ Worker 1: mutation_strategy="crossover", source_A=[best_variant_path], source_B=[second_best_path], model="opus"
58
+ Worker 2: mutation_strategy="compress-clarity", variants=1, model="sonnet"
59
+ ```
60
+
61
+ ## Layer 0 Self-Check (inline, before saving)
62
+ Each worker performs this check on every variant it generates:
63
+
64
+ ```
65
+ - [ ] All required sections present
66
+ - [ ] No placeholders: [TODO], [TBD], <INSERT>, ???
67
+ - [ ] Length in bounds: min=[MIN_TOKENS] max=[MAX_TOKENS]
68
+ - [ ] No self-citation (variant does not reference itself)
69
+ - [ ] Mutation is substantive (diff from base > 10% of content)
70
+ ```
71
+
72
+ If any check fails → log reason to mutation-log-[WORKER_ID].md, skip variant.
73
+
74
+ ## Evaluation of Variants (Parallel Haiku Agents)
75
+ After all workers complete, launch lightweight haiku agents for fast scoring:
76
+
77
+ ```
78
+ For each variant file in [OUTPUT_DIR]/variant-*.md:
79
+ Agent(
80
+ subagent_type="general-purpose",
81
+ model="haiku",
82
+ description="BTO Layer 1 Fast Eval — [variant_file]",
83
+ prompt="""
84
+ Read variant: [variant_file]
85
+ Read rubric: [RUBRIC_PATH]
86
+
87
+ Score on 3 quick criteria (integer 1-10 each):
88
+ 1. Relevance: does it address the rubric goals?
89
+ 2. Coherence: is it internally consistent?
90
+ 3. Improvement signal: does it improve on the base artifact?
91
+
92
+ Output one line: [variant_id] [r1] [r2] [r3] [average]
93
+ Write to [SCORES_DIR]/score-[variant_id].txt
94
+ """
95
+ )
96
+ ```
97
+
98
+ ## Results Collection and Ranking
99
+ After all haiku evaluations complete, the orchestrator:
100
+
101
+ ```
102
+ 1. Read all [SCORES_DIR]/score-*.txt
103
+ 2. Sort variants by average score descending
104
+ 3. Select top-K variants (default K=2) for full judge panel
105
+ 4. Log ranking to [OUTPUT_DIR]/round-ranking.md
106
+ 5. Pass top-K to bto-judge-panel agent template
107
+ ```
108
+
109
+ ### round-ranking.md Format
110
+ ```markdown
111
+ ## Optimization Round [N] — Ranking
112
+ | Rank | Variant | R1 | R2 | R3 | Avg | Strategy |
113
+ |------|---------|----|----|----|----|---------|
114
+ | 1 | variant-2-A | 8 | 7 | 8 | 7.7 | invert-critic |
115
+ | 2 | variant-1-B | 7 | 8 | 7 | 7.3 | expand-depth |
116
+ ...
117
+ **Selected for full evaluation:** variant-2-A, variant-1-B
118
+ **Discarded:** [list with reason]
119
+ ```
120
+
121
+ ## Crossover Protocol
122
+ When top 2 variants exist from a previous round:
123
+
124
+ ```
125
+ Agent(
126
+ subagent_type="general-purpose",
127
+ model="opus",
128
+ description="BTO Crossover",
129
+ prompt="""
130
+ Read Variant A (higher domain score): [VARIANT_A_PATH]
131
+ Read Variant B (higher completeness score): [VARIANT_B_PATH]
132
+ Read evaluation of each: [EVAL_A_PATH], [EVAL_B_PATH]
133
+
134
+ Produce one crossover variant that:
135
+ - Takes structure and domain depth from Variant A
136
+ - Takes completeness and edge cases from Variant B
137
+ - Resolves any contradictions explicitly
138
+
139
+ Write to [OUTPUT_DIR]/variant-crossover.md
140
+ """
141
+ )
142
+ ```
143
+
144
+ ## Cost Bounds and Abort Conditions
145
+
146
+ | Condition | Action |
147
+ |-----------|--------|
148
+ | Round count > 10 | Abort optimization, deliver best-so-far |
149
+ | Delta <= 0.5 for 3 consecutive rounds | Declare convergence, stop |
150
+ | Score regression > 1.0 | Rollback to previous best, log regression |
151
+ | Layer 0 fail rate > 50% in one round | Halt, human review required |
152
+ | Total haiku evals > 50 per session | Warn, continue only with human approval |
153
+ | Crossover score < both parents | Discard crossover, keep best parent |
154
+
155
+ ## Configuration Variables
156
+ | Variable | Description | Example |
157
+ |----------|-------------|---------|
158
+ | BASE_ARTIFACT_PATH | Starting artifact to optimize | researches/slug/03_solution_strategy.md |
159
+ | LAST_EVAL_PATH | Most recent panel-verdict.md | researches/slug/evals/round-1/panel-verdict.md |
160
+ | RUBRIC_PATH | Scoring rubric | .claude/rubrics/skill-rubric.md |
161
+ | OUTPUT_DIR | Where variants are written | researches/slug/evals/round-2/ |
162
+ | SCORES_DIR | Where haiku scores are written | researches/slug/evals/round-2/scores/ |
163
+ | MIN_TOKENS | Minimum variant length | 300 |
164
+ | MAX_TOKENS | Maximum variant length | 2000 |
165
+ | K | Top variants to promote to full panel | 2 |
166
+
167
+ ## Mutation Log Format
168
+ Each worker writes a log regardless of success:
169
+ ```markdown
170
+ ## Mutation Log — Worker [WORKER_ID] — Round [N]
171
+ **Strategy:** [MUTATION_STRATEGY]
172
+ **Variants attempted:** [N]
173
+ **Variants passed Layer 0:** [N]
174
+ **Layer 0 failures:** [list with reasons or "none"]
175
+ **Substantive changes made:**
176
+ - [brief description of what was changed and why]
177
+ ```
178
+
179
+ ## Reusability Note
180
+ This template is artifact-type agnostic. Replace BASE_ARTIFACT_PATH and RUBRIC_PATH
181
+ to optimize any text artifact: prompts, skills, presentations, research sections, code docstrings.
@@ -0,0 +1,169 @@
1
+ # /bto-build — Generate Skill or Command from Description
2
+
3
+ ## Usage
4
+ ```
5
+ /bto-build [natural language description of what to generate]
6
+ ```
7
+
8
+ ## Parameters
9
+ - $ARGUMENTS — Natural language description of the skill, command, rule, or agent template to generate. Optionally include "deep" to activate DEEP mode with interactive clarification.
10
+
11
+ ## Protocol
12
+
13
+ ### Step 1: Load Skill and Module
14
+
15
+ Read `.claude/skills/bto/SKILL.md`
16
+ Read `.claude/skills/bto/modules/build.md`
17
+
18
+ ### Step 2: Validate Input
19
+
20
+ If $ARGUMENTS is empty:
21
+ - Ask: "Describe the skill or command you want to build. Include: what it does, who uses it, expected inputs and outputs."
22
+ - Stop and wait.
23
+
24
+ ### Step 3: Detect Mode
25
+
26
+ Scan $ARGUMENTS for "deep" or "углубленный":
27
+ - If found → **DEEP mode**: load `explore` skill, run interactive clarification
28
+ - Otherwise → **QUICK mode**: proceed directly to generation
29
+
30
+ ### Step 4: Detect Artifact Type
31
+
32
+ Auto-detect from $ARGUMENTS using these signals:
33
+
34
+ | Signal Words | Detected Type |
35
+ |-------------|--------------|
36
+ | "skill", "module", "capability", "protocol", "скилл" | skill |
37
+ | "command", "slash command", "/something", "pipeline", "команда" | command |
38
+ | "rule", "constraint", "convention", "anti-pattern", "правило" | rule |
39
+ | "agent", "worker", "parallel", "swarm", "агент" | agent |
40
+
41
+ If ambiguous — default to `skill` and note the assumption.
42
+
43
+ ### Step 5: DEEP Mode (if activated)
44
+
45
+ Read `.claude/skills/explore/SKILL.md`
46
+
47
+ Follow the explore protocol to clarify:
48
+ 1. Exact scope and boundaries — what is in scope, what is out of scope?
49
+ 2. Target consumers — which agent/user/command will load this?
50
+ 3. Input format — what $ARGUMENTS or parameters does it accept?
51
+ 4. Expected output — what files, text, or actions does it produce?
52
+ 5. Quality criteria — how will success be measured?
53
+ 6. Edge cases — what inputs could break it?
54
+ 7. Reference examples — any existing artifacts to model after?
55
+
56
+ Produce a requirements brief and confirm with user before proceeding to generation.
57
+
58
+ ### Step 6: QUICK Mode (if activated)
59
+
60
+ Extract directly from $ARGUMENTS:
61
+ 1. Parse artifact name → kebab-case slug
62
+ 2. Extract key capabilities and responsibilities
63
+ 3. Identify domain context (banking, retail, enterprise, etc.)
64
+ 4. Identify output artifacts (files, reports, diagrams)
65
+ 5. Proceed immediately to generation
66
+
67
+ ### Step 7: Generate Artifact
68
+
69
+ Following the template from build.md for the detected type:
70
+
71
+ #### For Skills:
72
+ Create directory `.claude/skills/<name>/` with:
73
+ - `SKILL.md` — Main orchestrator with Overview, Protocol, Output Format, Anti-Patterns, Dependencies
74
+ - `modules/<module-name>.md` — Detailed per-module protocols (if multi-module)
75
+ - `references/<ref-name>.md` — Supporting material, rubrics, checklists (at least one)
76
+ - `examples/<example-name>.md` — Few-shot examples showing expected output (at least one)
77
+
78
+ #### For Commands:
79
+ Create file `.claude/commands/<name>.md` with:
80
+ - `# /command-name — Short Description`
81
+ - `## Usage` with invocation syntax
82
+ - `## Parameters` documenting $ARGUMENTS
83
+ - `## Protocol` with numbered steps
84
+ - Skill loading instructions (Read `.claude/skills/.../SKILL.md`)
85
+ - Agent tool usage for parallel work where applicable
86
+ - Checkpoint banner at end
87
+
88
+ #### For Rules:
89
+ Create file `.claude/rules/<name>.md` with:
90
+ - Title heading
91
+ - Pattern/Detection Signal/Fix table
92
+ - Auto-Detection section
93
+
94
+ #### For Agent Templates:
95
+ Create file `.claude/agents/<name>.md` with:
96
+ - Purpose statement
97
+ - Model selection with justification
98
+ - Isolation scope (reads X, writes Y)
99
+ - Prompt template
100
+
101
+ ### Step 8: Self-Review (Layer 0)
102
+
103
+ Before finalizing, validate against quality checklist:
104
+
105
+ **Structure check:**
106
+ - Required sections present for artifact type?
107
+ - No empty sections (heading immediately followed by another heading)?
108
+ - Proper markdown formatting?
109
+
110
+ **Content check:**
111
+ - No generic placeholder content left?
112
+ - Anti-patterns section populated with real failure modes?
113
+ - At least one concrete example or reference included?
114
+
115
+ **Convention check:**
116
+ - File naming: kebab-case?
117
+ - Heading hierarchy: properly nested?
118
+ - All cross-references point to files that will exist?
119
+
120
+ **Size check:**
121
+ - SKILL.md: 2KB-30KB
122
+ - Module files: 1KB-15KB
123
+ - Reference files: 500B-10KB
124
+ - Command files: 500B-20KB
125
+
126
+ If any check fails — fix before outputting. Flag with [FIXED: reason].
127
+
128
+ ### Step 9: Create Files
129
+
130
+ Write all generated files to disk. Display creation summary.
131
+
132
+ **Checkpoint:**
133
+ ```
134
+ ═══════════════════════════════════════════════════════
135
+ CHECKPOINT: BUILD Complete
136
+ Artifact type: [skill / command / rule / agent]
137
+ Mode used: [QUICK / DEEP]
138
+
139
+ Files created:
140
+ [path/to/SKILL.md] (X KB)
141
+ [path/to/modules/module.md] (X KB)
142
+ [path/to/references/ref.md] (X KB)
143
+ [path/to/examples/example.md] (X KB)
144
+
145
+ Self-review: X/Y checks passed
146
+ [List any issues found and fixed]
147
+
148
+ Next steps:
149
+ /bto-test [path] — evaluate quality
150
+ /bto [path] — test + optimize in one pipeline
151
+ • "ок" — done
152
+ • "переделай [aspect]" — adjust and regenerate
153
+ • "углуби [section]" — expand a specific section
154
+ • "добавь пример" — add another example
155
+ ═══════════════════════════════════════════════════════
156
+ ```
157
+
158
+ ## Anti-Patterns
159
+
160
+ | Anti-Pattern | Detection | Fix |
161
+ |-------------|-----------|-----|
162
+ | Generic artifact | No domain-specific terms in output | Add domain context from $ARGUMENTS |
163
+ | Missing references | references/ empty or not created | Always include at least one reference file |
164
+ | No examples | examples/ empty or not created | Always include at least one example |
165
+ | Placeholder content | "[TODO]" or "[INSERT]" left in output | Generate actual content, never leave placeholders |
166
+ | Over-scoped SKILL.md | SKILL.md > 30KB | Split into modules, keep SKILL.md as orchestrator |
167
+ | Under-specified | SKILL.md < 2KB | Expand — artifact too thin to be useful |
168
+ | Wrong artifact type | Description says "command" but skill generated | Re-detect and regenerate |
169
+ | Skipping self-review | Files written before quality check | Always run Layer 0 before finalizing |