@dzhechkov/skills-bto 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md
ADDED
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
# @dzhechkov/skills-bto
|
|
2
|
+
|
|
3
|
+
**Build-Test-Optimize skill pack for Claude Code**
|
|
4
|
+
|
|
5
|
+
Multi-agent evaluation and iterative optimization pipeline for Claude Code skills, commands, and prompts. Part of the [Keysarium](https://www.npmjs.com/package/@dzhechkov/keysarium) ecosystem.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Quick Start
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
# One-command install via npx
|
|
13
|
+
npx @dzhechkov/skills-bto
|
|
14
|
+
|
|
15
|
+
# Or install globally
|
|
16
|
+
npm install -g @dzhechkov/skills-bto
|
|
17
|
+
skills-bto init
|
|
18
|
+
|
|
19
|
+
# Install into a project that already has @dzhechkov/keysarium
|
|
20
|
+
npx @dzhechkov/skills-bto init
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
After installation, open Claude Code in your project directory and start using BTO commands.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## What You Get
|
|
28
|
+
|
|
29
|
+
| Component | Count | Description |
|
|
30
|
+
|-----------|-------|-------------|
|
|
31
|
+
| **Skill** | 1 | `bto` — core Build-Test-Optimize skill with modules (build, test, optimize) |
|
|
32
|
+
| **Commands** | 4 | `/bto`, `/bto-build`, `/bto-test`, `/bto-optimize` |
|
|
33
|
+
| **Rules** | 1 | `bto-quality-gates` — quality gate enforcement |
|
|
34
|
+
| **Shards** | 1 | `bto-evaluation` — context shard for BTO evaluation pipeline |
|
|
35
|
+
| **Agent Templates** | 2 | `bto-judge-panel`, `bto-optimizer-worker` |
|
|
36
|
+
| **References** | 4 | Eval patterns, judge rubrics, optimization methods, quality checklist |
|
|
37
|
+
| **Examples** | 1 | Sample evaluation report |
|
|
38
|
+
|
|
39
|
+
Everything is installed into your project's `.claude/` directory and works natively with Claude Code.
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## Commands
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
npx @dzhechkov/skills-bto # Full install (interactive, same as init)
|
|
47
|
+
npx @dzhechkov/skills-bto init # Install all components
|
|
48
|
+
npx @dzhechkov/skills-bto init --force # Overwrite existing files
|
|
49
|
+
npx @dzhechkov/skills-bto init --dry-run # Preview without making changes
|
|
50
|
+
npx @dzhechkov/skills-bto update # Update to latest version
|
|
51
|
+
npx @dzhechkov/skills-bto remove # Clean uninstall
|
|
52
|
+
npx @dzhechkov/skills-bto list # Show installed components
|
|
53
|
+
npx @dzhechkov/skills-bto doctor # Health check
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## BTO Pipeline
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
BUILD ──→ TEST ──→ OPTIMIZE
|
|
62
|
+
│ │ │
|
|
63
|
+
│ │ └── Evolutionary mutation + re-evaluation (3 rounds)
|
|
64
|
+
│ └── Multi-layer evaluation: Layer 0 → Layer 1 → Layer 2
|
|
65
|
+
└── Generate skill/command from description
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
### Usage in Claude Code
|
|
69
|
+
|
|
70
|
+
```bash
|
|
71
|
+
# Full BTO cycle: build → test → optimize
|
|
72
|
+
/bto Create a skill for code review automation
|
|
73
|
+
|
|
74
|
+
# Build only — generate a new skill or command
|
|
75
|
+
/bto-build Create a skill that analyzes git commit patterns
|
|
76
|
+
|
|
77
|
+
# Test only — evaluate an existing artifact
|
|
78
|
+
/bto-test .claude/skills/my-skill/SKILL.md
|
|
79
|
+
|
|
80
|
+
# Optimize only — iteratively improve an artifact
|
|
81
|
+
/bto-optimize .claude/skills/my-skill/SKILL.md
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## Evaluation Architecture
|
|
87
|
+
|
|
88
|
+
### Layer Model
|
|
89
|
+
|
|
90
|
+
| Layer | Agents | Model | Purpose |
|
|
91
|
+
|-------|--------|-------|---------|
|
|
92
|
+
| **Layer 0** | 0 | — | Deterministic pre-checks (structure, completeness, encoding) |
|
|
93
|
+
| **Layer 1** | 1 | haiku | Fast semantic evaluation across 5 dimensions |
|
|
94
|
+
| **Layer 2** | 3 | sonnet | Full judge panel: Domain Expert + Critic + Completeness Auditor |
|
|
95
|
+
| **Meta** | 1 | opus | Disagreement resolution (triggered when score delta > 3) |
|
|
96
|
+
|
|
97
|
+
### Judge Panel
|
|
98
|
+
|
|
99
|
+
- **3 independent judges** evaluate each artifact in isolation
|
|
100
|
+
- Judges never see each other's scores before submitting
|
|
101
|
+
- Standard weights: Domain Expert (0.4) / Critic (0.3) / Completeness Auditor (0.3)
|
|
102
|
+
- If `max_score - min_score > 3` → meta-judge escalation
|
|
103
|
+
|
|
104
|
+
### Quality Gates
|
|
105
|
+
|
|
106
|
+
- Layer 0 must pass before Layer 1
|
|
107
|
+
- Layer 1 must pass before Layer 2
|
|
108
|
+
- Optimization accepted only if `new_score - prev_score > 0.5`
|
|
109
|
+
- 3 consecutive iterations with delta ≤ 0.5 → convergence declared
|
|
110
|
+
- Score decrease > 1.0 → automatic rollback to previous best
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## Optimization Process
|
|
115
|
+
|
|
116
|
+
The optimizer runs up to 3 rounds of evolutionary improvement:
|
|
117
|
+
|
|
118
|
+
1. **Round 1** — 5 parallel haiku agents generate mutations, fast-rank variants
|
|
119
|
+
2. **Round 2** — Top variants evaluated by sonnet judge panel
|
|
120
|
+
3. **Round 3** — 3×3 parallel sonnet agents for full Layer 2 evaluation of finalists
|
|
121
|
+
|
|
122
|
+
Each round selects the best-performing variant and uses it as the base for the next iteration.
|
|
123
|
+
|
|
124
|
+
---
|
|
125
|
+
|
|
126
|
+
## Integration with Keysarium
|
|
127
|
+
|
|
128
|
+
BTO works standalone but integrates seamlessly with `@dzhechkov/keysarium`:
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
# Install Keysarium first (optional)
|
|
132
|
+
npx @dzhechkov/keysarium init
|
|
133
|
+
|
|
134
|
+
# Then add BTO — it detects Keysarium automatically
|
|
135
|
+
npx @dzhechkov/skills-bto init
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
When installed alongside Keysarium, BTO can evaluate and optimize any skill or command in the Keysarium toolkit.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Requirements
|
|
143
|
+
|
|
144
|
+
- **Claude Code CLI** — installed and configured ([installation guide](https://docs.anthropic.com/en/docs/claude-code))
|
|
145
|
+
- **Node.js >= 16.0.0** — required for the npm install method
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## License
|
|
150
|
+
|
|
151
|
+
[MIT](https://opensource.org/licenses/MIT)
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## Links
|
|
156
|
+
|
|
157
|
+
- **GitHub:** [https://github.com/dzhechko/product-keysarium-2026](https://github.com/dzhechko/product-keysarium-2026)
|
|
158
|
+
- **Issues:** [https://github.com/dzhechko/product-keysarium-2026/issues](https://github.com/dzhechko/product-keysarium-2026/issues)
|
|
159
|
+
- **npm:** [https://www.npmjs.com/package/@dzhechkov/skills-bto](https://www.npmjs.com/package/@dzhechkov/skills-bto)
|
|
160
|
+
- **Keysarium:** [https://www.npmjs.com/package/@dzhechkov/keysarium](https://www.npmjs.com/package/@dzhechkov/keysarium)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@dzhechkov/skills-bto",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Build-Test-Optimize skill pack for Claude Code — structured BTO pipeline with quality gates, testing strategies, and optimization workflows",
|
|
5
5
|
"main": "src/cli.js",
|
|
6
6
|
"bin": {
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# BTO Evaluation — Governance Shard
|
|
2
|
+
|
|
3
|
+
## Skills
|
|
4
|
+
Load: `.claude/skills/bto/SKILL.md` + relevant module
|
|
5
|
+
|
|
6
|
+
## Layer Architecture
|
|
7
|
+
|
|
8
|
+
| Layer | Model | Gate | Action on Fail |
|
|
9
|
+
|-------|-------|------|----------------|
|
|
10
|
+
| Layer 0 | — (deterministic) | ≥ 80% checks pass | STOP, auto-retry up to 3x |
|
|
11
|
+
| Layer 1 | haiku | avg ≥ 7.0 | NEEDS WORK (flag) |
|
|
12
|
+
| Layer 2 | sonnet × 3 judges | weighted avg ≥ 7.0 | FAIL |
|
|
13
|
+
| Meta | sonnet | disagreement > 3 | Arbitrate |
|
|
14
|
+
|
|
15
|
+
## Judge Isolation — INVARIANT
|
|
16
|
+
- Each judge reads the SAME artifact
|
|
17
|
+
- Each judge writes to a SEPARATE evaluation
|
|
18
|
+
- Judges do NOT see each other's scores before submitting
|
|
19
|
+
- Generator and judge models MUST differ
|
|
20
|
+
|
|
21
|
+
## Weights
|
|
22
|
+
- Domain Expert: 0.40
|
|
23
|
+
- Critic: 0.30
|
|
24
|
+
- Completeness Auditor: 0.30
|
|
25
|
+
|
|
26
|
+
## Optimization Delta Gate
|
|
27
|
+
- Accepted if: new_score - prev_score > 0.5
|
|
28
|
+
- 3 consecutive iterations ≤ 0.5 delta → convergence, stop
|
|
29
|
+
- Score decrease > 1.0 → rollback to previous best
|
|
30
|
+
|
|
31
|
+
## Model Routing
|
|
32
|
+
- Layer 0: deterministic (no LLM)
|
|
33
|
+
- Layer 1: haiku
|
|
34
|
+
- Layer 2 judges: sonnet
|
|
35
|
+
- Meta-judge: sonnet
|
|
36
|
+
- Crossover synthesis: opus
|
|
37
|
+
- Variant fast-eval: haiku
|
|
38
|
+
|
|
39
|
+
## Promises
|
|
40
|
+
- `<promise>BTO_LAYER0_PASSED</promise>` — after Layer 0
|
|
41
|
+
- `<promise>BTO_LAYER2_SCORED</promise>` — after Layer 2
|
|
42
|
+
- `<promise>BTO_OPTIMIZED</promise>` — after optimization converges
|
|
43
|
+
|
|
44
|
+
## Anti-Patterns
|
|
45
|
+
- Score inflation (all > 8.5 first attempt) → add calibration
|
|
46
|
+
- Conformity collapse (identical scores) → enforce isolation
|
|
47
|
+
- Runaway optimization (> 10 iterations) → abort
|
|
48
|
+
- Judge-generator collusion (same model) → BLOCK
|