@dzhechkov/keysarium 1.5.0 → 1.5.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -4
- package/README.ru.md +541 -0
- package/bin/cli.js +0 -0
- package/package.json +8 -8
- package/src/cli.js +10 -1
- package/src/commands/init.js +13 -5
- package/src/commands/update.js +5 -1
- package/src/utils.js +34 -0
- package/templates/.claude/commands/bto-benchmark.md +0 -302
package/README.md
CHANGED
|
@@ -28,10 +28,10 @@ After installation, open Claude Code in your project directory and start using s
|
|
|
28
28
|
|
|
29
29
|
| Component | Count | Description |
|
|
30
30
|
|-----------|-------|-------------|
|
|
31
|
-
| **Skills** |
|
|
32
|
-
| **Commands** |
|
|
33
|
-
| **Rules** |
|
|
34
|
-
| **Shards** |
|
|
31
|
+
| **Skills** | 12 | `explore`, `frontend-design`, `goap-research-ed25519`, `presentation-storyteller`, `problem-solver-enhanced`, `reverse-engineering-unicorn`, `knowledge-extractor`, `ai-factory-mapper`, `analyst-manual-full`, `edu-site-generator`, `transcript-site-generator`, `feature-adr` |
|
|
32
|
+
| **Commands** | 20 | `/casarium`, `/new-research`, `/parallel-research`, `/discovery`, `/explore-case`, `/research`, `/cjm-prototype`, `/solve`, `/architecture-phase`, `/presentation`, `/harvest`, `/brain-export`, `/brain-import`, `/bto-benchmark`, `/feature-adr`, `/init-platform`, `/learning-stats`, `/dream`, `/workers`, `/analyst-manual` |
|
|
33
|
+
| **Rules** | 14 | `research-quality`, `checkpoint-protocol`, `agent-swarm`, `domain-specific`, `anti-patterns`, `file-conventions`, `modular-reuse`, `feedback-loops`, `model-routing`, `trust-tiers`, `background-workers`, `dream-cycles`, `reward-learning`, `feature-adr-conventions` |
|
|
34
|
+
| **Shards** | 9 | Context shards for each phase: `phase-0-discovery`, `phase-1-explore`, `phase-2-research`, `phase-25-cjm`, `phase-3-solve`, `phase-4-architecture`, `phase-5-presentation`, `phase-ai-factory`, `feature-adr` |
|
|
35
35
|
| **Agent Templates** | 5 | `discovery-worker`, `research-worker`, `cjm-variant-worker`, `presentation-worker`, `parallel-research-orchestrator` |
|
|
36
36
|
| **Library** | 4 | `phase-utils`, `agent-patterns`, `skill-loader`, `domain-templates` |
|
|
37
37
|
| **Documentation** | 12 | Deployment, admin, user guide, infrastructure, architecture, flows (for Keysarium and BTO) |
|
package/README.ru.md
ADDED
|
@@ -0,0 +1,541 @@
|
|
|
1
|
+
# Keysarium — AI-исследовательский тулкит
|
|
2
|
+
|
|
3
|
+
**7-фазный pipeline** для проведения AI-исследований на кейсариумах, хакатонах и в продуктовой аналитике. Содержит 12 скиллов, 20 slash-команд, 14 правил, 9 governance shards и систему самообучения.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
npx @dzhechkov/keysarium init
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
## Содержание
|
|
10
|
+
|
|
11
|
+
- [Быстрый старт](#быстрый-старт)
|
|
12
|
+
- [7-фазный pipeline](#7-фазный-pipeline)
|
|
13
|
+
- [Скиллы — внутреннее устройство](#скиллы--внутреннее-устройство)
|
|
14
|
+
- [Reverse Engineering Unicorn — подробно](#reverse-engineering-unicorn--подробно)
|
|
15
|
+
- [Slash-команды](#slash-команды)
|
|
16
|
+
- [Правила (Rules)](#правила-rules)
|
|
17
|
+
- [Governance Shards](#governance-shards)
|
|
18
|
+
- [Система самообучения](#система-самообучения)
|
|
19
|
+
- [Фоновые воркеры и Dream Cycles](#фоновые-воркеры-и-dream-cycles)
|
|
20
|
+
- [Портативный мозг (Brain)](#портативный-мозг-brain)
|
|
21
|
+
- [Мульти-платформенная поддержка](#мульти-платформенная-поддержка)
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Быстрый старт
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
# Установить в проект
|
|
29
|
+
npx @dzhechkov/keysarium init
|
|
30
|
+
|
|
31
|
+
# Запустить полный pipeline по кейсу
|
|
32
|
+
/casarium "Автоматизация кредитного скоринга для банка"
|
|
33
|
+
|
|
34
|
+
# С маппингом на Cloud.ru AI Factory
|
|
35
|
+
/casarium --ai-factory "Чат-бот поддержки для ритейла"
|
|
36
|
+
|
|
37
|
+
# Параллельно несколько кейсов
|
|
38
|
+
/parallel-research "Кейс 1" | "Кейс 2" | "Кейс 3"
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## 7-фазный pipeline
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
Phase 0 (15%) → Phase 1 (5%) → Phase 2 (15%) → Phase 2.5 ⭐ (10%) → Phase 3 (15%) → Phase 4 (15%) → Phase 5 (20%)
|
|
45
|
+
DISCOVERY EXPLORE RESEARCH CJM ПРОТОТИП SOLVE ARCHITECTURE PRESENTATION
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
| Фаза | Команда | Скилл | Что делает | Артефакт |
|
|
49
|
+
|------|---------|-------|-----------|----------|
|
|
50
|
+
| **0** | `/discovery` | reverse-engineering-unicorn | JTBD + конкуренты + ROI + бизнес-кейс | `00_product_discovery.md` |
|
|
51
|
+
| **1** | `/explore-case` | explore | Глубокое понимание кейса, 5 Whys, критерии успеха | `01_case_brief.md` |
|
|
52
|
+
| **2** | `/research` | goap-research-ed25519 | PARANOID research: аналоги, технологии, регуляторика (3 параллельных агента) | `02_research_findings.md` |
|
|
53
|
+
| **2.5** ⭐ | `/cjm-prototype` | reverse-eng + frontend-design | 3 варианта CJM → кликабельный React прототип → выбор | `prototype/cjm-prototype.jsx` |
|
|
54
|
+
| **3** | `/solve` | problem-solver-enhanced | Стратегия решения: SCQA, Process Design, AI Pipeline, HITL | `03_solution_strategy.md` |
|
|
55
|
+
| **4** | `/architecture-phase` | (встроенный) | C4 диаграммы, компоненты, безопасность, MVP scope | `04_architecture.md` |
|
|
56
|
+
| **5** | `/presentation` | presentation-storyteller | 10-12 слайдов + speaker script + Q&A + Executive Summary | `05-08_*.md` |
|
|
57
|
+
|
|
58
|
+
**Ключевые правила:**
|
|
59
|
+
- Phase 2.5 (CJM) — **ОБЯЗАТЕЛЬНА, нельзя пропустить**
|
|
60
|
+
- `08_executive_summary.md` — **ОБЯЗАТЕЛЕН**
|
|
61
|
+
- Checkpoint после каждой фазы — ждёт `"ок"` от пользователя
|
|
62
|
+
- Research в PARANOID mode — ноль непроверенных утверждений
|
|
63
|
+
- `{CHOSEN_CJM}` передаётся из Phase 2.5 в Phase 3-5
|
|
64
|
+
|
|
65
|
+
---
|
|
66
|
+
|
|
67
|
+
## Скиллы — внутреннее устройство
|
|
68
|
+
|
|
69
|
+
### Граф зависимостей
|
|
70
|
+
|
|
71
|
+
```
|
|
72
|
+
reverse-engineering-unicorn (оркестратор pipeline)
|
|
73
|
+
├── explore (Socratic кларификация)
|
|
74
|
+
├── goap-research-ed25519 (GOAP + Ed25519 верификация)
|
|
75
|
+
├── problem-solver-enhanced (TRIZ + Game Theory + 5 Whys)
|
|
76
|
+
├── frontend-design (UI для CJM прототипа)
|
|
77
|
+
└── presentation-storyteller (слайды + speaker script)
|
|
78
|
+
├── explore
|
|
79
|
+
└── goap-research-ed25519
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
### 1. explore — адаптивная кларификация
|
|
83
|
+
|
|
84
|
+
**Что делает:** Превращает расплывчатые запросы в чёткие спецификации через Socratic questioning.
|
|
85
|
+
|
|
86
|
+
**Протокол:**
|
|
87
|
+
1. Классифицирует задачу: Product / Problem / Decision / Creative / Research / Process
|
|
88
|
+
2. Выбирает 3-5 измерений для исследования (цель, ограничения, ресурсы, timeline, критерии успеха)
|
|
89
|
+
3. Задаёт таргетированные вопросы (не шаблонные — адаптивные)
|
|
90
|
+
4. Формирует **Task Brief** с objective, constraints, success criteria, out-of-scope
|
|
91
|
+
|
|
92
|
+
**Когда вызывается:** Phase 1, а также как зависимость в reverse-engineering-unicorn, presentation-storyteller, analyst-manual, feature-adr, ai-factory-mapper.
|
|
93
|
+
|
|
94
|
+
### 2. goap-research-ed25519 — верифицированный research
|
|
95
|
+
|
|
96
|
+
**Что делает:** GOAP (Goal-Oriented Action Planning) + OODA loop + Ed25519 криптографические подписи для антигаллюцинации.
|
|
97
|
+
|
|
98
|
+
**Протокол:**
|
|
99
|
+
1. Оценивает текущее состояние (что знаем / не знаем)
|
|
100
|
+
2. Строит план исследования через A* алгоритм с учётом стоимости верификации
|
|
101
|
+
3. Выполняет OODA цикл (Observe → Orient → Decide → Act) с checkpoint'ами подписей
|
|
102
|
+
4. Каждый факт подписывается Ed25519 → цепочка доверия (citation chain)
|
|
103
|
+
5. Генерирует подписанный research report
|
|
104
|
+
|
|
105
|
+
**3 уровня строгости:** moderate (0.85), strict (0.95), paranoid (0.99)
|
|
106
|
+
|
|
107
|
+
**Когда вызывается:** Phase 2 (основной), а также все DEEP/VERIFIED режимы в других скиллах.
|
|
108
|
+
|
|
109
|
+
### 3. problem-solver-enhanced — решатель проблем
|
|
110
|
+
|
|
111
|
+
**Что делает:** 9-модульная система решения сложных проблем.
|
|
112
|
+
|
|
113
|
+
**9 модулей:**
|
|
114
|
+
1. **First Principles** — декомпозиция до фундаментальных истин
|
|
115
|
+
2. **5 Whys** — корневой анализ причин
|
|
116
|
+
3. **SCQA** — Situation, Complication, Question, Answer
|
|
117
|
+
4. **Game Theory** — анализ стратегических взаимодействий
|
|
118
|
+
5. **Second-Order Thinking** — последствия 2-го и 3-го порядка
|
|
119
|
+
6. **TRIZ** — разрешение противоречий без компромисса + Идеальный Конечный Результат
|
|
120
|
+
7. **Design Thinking** — Empathize → Define → Ideate → Prototype → Test
|
|
121
|
+
8. **OODA Loop** — Observe → Orient → Decide → Act (для динамичных ситуаций)
|
|
122
|
+
9. **Solution Synthesis** — объединение находок в actionable решение
|
|
123
|
+
|
|
124
|
+
**Когда вызывается:** Phase 3, а также при разрешении сложных trade-off'ов в feature-adr, ai-factory-mapper.
|
|
125
|
+
|
|
126
|
+
### 4. frontend-design — дизайн интерфейсов
|
|
127
|
+
|
|
128
|
+
**Что делает:** Создаёт production-grade UI с bold aesthetic direction. Антитеза "generic AI slop".
|
|
129
|
+
|
|
130
|
+
**Протокол:**
|
|
131
|
+
1. Выбирает контекст и обязывается к эстетическому направлению (minimal / maximalist / retro-futuristic / etc.)
|
|
132
|
+
2. Применяет: distinctive typography, committed color palette, micro-animations, unexpected spatial composition
|
|
133
|
+
3. Генерирует рабочий HTML/CSS/JS или React код
|
|
134
|
+
|
|
135
|
+
**Когда вызывается:** Phase 2.5 (CJM прототип), а также в feature-adr Step 7 для UI-фич.
|
|
136
|
+
|
|
137
|
+
### 5. presentation-storyteller — презентации со storytelling
|
|
138
|
+
|
|
139
|
+
**Что делает:** 3 файла за один запуск: outline → full presentation → speaker script.
|
|
140
|
+
|
|
141
|
+
**Фреймворки:** AIDA, Hero's Journey, Problem-Solution.
|
|
142
|
+
|
|
143
|
+
**Протокол:**
|
|
144
|
+
1. Кларификация (explore) → цель, аудитория, ограничения
|
|
145
|
+
2. Research (goap-research-ed25519) → верифицированные данные и источники
|
|
146
|
+
3. Структура (storytelling arc + outline с тезисами)
|
|
147
|
+
4. Полная презентация в Markdown со ссылками на источники
|
|
148
|
+
5. Speaker script «Как рассказывать?» — связная история для каждого слайда
|
|
149
|
+
|
|
150
|
+
**Когда вызывается:** Phase 5.
|
|
151
|
+
|
|
152
|
+
### 6. knowledge-extractor — извлечение знаний (Trust Tier 2)
|
|
153
|
+
|
|
154
|
+
**Что делает:** 5 параллельных агентов-экстракторов → 7 категорий → 8 blocking quality gates.
|
|
155
|
+
|
|
156
|
+
**4 модуля:**
|
|
157
|
+
1. **Extract** — 5 агентов сканируют проект через разные линзы (patterns / commands / rules / templates / snippets)
|
|
158
|
+
2. **Classify** — 7-категорная классификация + cross-dedup
|
|
159
|
+
3. **Gate** — 8 quality gates в 2 прохода (deterministic G1-G2/G5-G7 + semantic G3-G4/G8 через haiku)
|
|
160
|
+
4. **Integrate** — auto-placement в `.claude/` + обновление TOOLKIT_HARVEST.md
|
|
161
|
+
|
|
162
|
+
**Когда вызывается:** `/harvest` после завершения любого проекта.
|
|
163
|
+
|
|
164
|
+
### 7. ai-factory-mapper — маппинг на Cloud.ru AI Factory
|
|
165
|
+
|
|
166
|
+
**Что делает:** Декомпозирует ЛЮБУЮ бизнес-задачу на pipeline из сервисов Cloud.ru Evolution AI Factory.
|
|
167
|
+
|
|
168
|
+
**Протокол:**
|
|
169
|
+
1. Clarity gate → уточнение через explore
|
|
170
|
+
2. Каталог sync (catalog.md + web search для обновлений)
|
|
171
|
+
3. Декомпозиция через problem-solver-enhanced (7-12 шагов)
|
|
172
|
+
4. Маппинг: шаг → сервис → ✅/⚠️/❌
|
|
173
|
+
5. Gap identification + Coverage scoring
|
|
174
|
+
6. Synthesis: report + Mermaid + DOCX
|
|
175
|
+
|
|
176
|
+
**Когда вызывается:** `/casarium --ai-factory` (аддитивно на всех фазах).
|
|
177
|
+
|
|
178
|
+
### 8. analyst-manual-full — стратегический аналитик
|
|
179
|
+
|
|
180
|
+
**Что делает:** Композитный оркестратор: explore → goap-research → problem-solver с ручными checkpoint'ами.
|
|
181
|
+
|
|
182
|
+
**3 фазы с подтверждением:**
|
|
183
|
+
1. Explore → Task Brief → **CHECKPOINT** (выбор verification mode: moderate/strict/paranoid)
|
|
184
|
+
2. Verified Research (Ed25519) → **CHECKPOINT**
|
|
185
|
+
3. Solve (9-module framework) → **CHECKPOINT**
|
|
186
|
+
4. Синтез: 4 документа × 2 формата (md + docx)
|
|
187
|
+
|
|
188
|
+
**Когда вызывается:** `/analyst-manual` для стратегических задач (ценообразование, GTM, конкурентный анализ).
|
|
189
|
+
|
|
190
|
+
### 9-10. edu-site-generator + transcript-site-generator
|
|
191
|
+
|
|
192
|
+
**edu-site-generator:** Генерирует gamified educational SPA из документации. 6 типов упражнений, Vite + TailwindCSS, GitHub Pages deploy.
|
|
193
|
+
|
|
194
|
+
**transcript-site-generator (Trust Tier 2):** Превращает транскрипт / YouTube ссылку в интерактивный static site. Full-text search, YouTube timestamp sync, dark/light mode, speaker labels.
|
|
195
|
+
|
|
196
|
+
### 11-12. feature-adr + bto
|
|
197
|
+
|
|
198
|
+
**feature-adr:** 11-шаговый pipeline разработки фич с Complexity Router (S/M/L/XL). Независим от casarium pipeline — артефакты в `features/<slug>/`.
|
|
199
|
+
|
|
200
|
+
**bto:** Build-Benchmark-Test-Optimize. Мульти-агентная оценка и эволюционная оптимизация скиллов. 4 модуля: BUILD → BENCHMARK → TEST → OPTIMIZE.
|
|
201
|
+
|
|
202
|
+
---
|
|
203
|
+
|
|
204
|
+
## Reverse Engineering Unicorn — подробно
|
|
205
|
+
|
|
206
|
+
Главный оркестратор pipeline. Декомпозирует любую компанию в actionable launch playbook за 7 модулей.
|
|
207
|
+
|
|
208
|
+
### 3 режима
|
|
209
|
+
|
|
210
|
+
| Режим | Research | Анализ | Время |
|
|
211
|
+
|-------|---------|--------|-------|
|
|
212
|
+
| 🟢 **QUICK** | Статические запросы | Шаблоны | ~70 мин |
|
|
213
|
+
| 🔵 **DEEP** | GOAP A* + OODA | + Game Theory, TRIZ, CJM прототип, BS-check | ~140 мин |
|
|
214
|
+
| 🟣 **VERIFIED** | GOAP + Ed25519 | + Криптоподписи, audit trail, trusted issuers | ~170 мин |
|
|
215
|
+
|
|
216
|
+
### 7 модулей
|
|
217
|
+
|
|
218
|
+
#### Module 0: PRE-FLIGHT
|
|
219
|
+
Если входные данные неясны → вызывает `explore` для кларификации.
|
|
220
|
+
Определяет: `{COMPANY}`, `{URL}`, `{INDUSTRY}`, `{GEOGRAPHY}`, `{MODE}`.
|
|
221
|
+
|
|
222
|
+
#### Module 1: INTELLIGENCE → Verified Fact Sheet
|
|
223
|
+
- Год основания, штаб-квартира, команда, финансирование
|
|
224
|
+
- Ключевые метрики: выручка, MAU, NPS
|
|
225
|
+
- Каждый факт верифицирован (DEEP → GOAP, VERIFIED → Ed25519)
|
|
226
|
+
- **Артефакт:** Fact Sheet с confidence scores
|
|
227
|
+
|
|
228
|
+
#### Module 2: PRODUCT & CUSTOMERS → JTBD + Segments + Voice of Customer
|
|
229
|
+
- [JTBD Canvas](references/jtbd-canvas.md) для каждого сегмента
|
|
230
|
+
- Сегментация (STP): демография, поведение, потребности
|
|
231
|
+
- Voice of Customer: реальные цитаты из отзывов, App Store, Product Hunt
|
|
232
|
+
- **Артефакт:** JTBD карты + VoC матрица
|
|
233
|
+
|
|
234
|
+
#### Module 2.5: CJM PROTOTYPE → Кликабельный React прототип ⭐
|
|
235
|
+
Это **ключевой** модуль — обязателен, нельзя пропустить.
|
|
236
|
+
|
|
237
|
+
1. Генерирует **3 варианта CJM** (Variant A, B, C) — каждый с разным фокусом
|
|
238
|
+
2. Параллельно запускает **Trend Research** → Variant D (на базе трендов)
|
|
239
|
+
3. Вызывает `frontend-design` → генерирует **кликабельный React JSX** прототип
|
|
240
|
+
4. Пользователь **интерактивно** сравнивает варианты:
|
|
241
|
+
- ⚖️ Сравнение бок о бок
|
|
242
|
+
- 🔧 Кастомный микс (шаги из разных вариантов)
|
|
243
|
+
- 👁 Предпросмотр с данными
|
|
244
|
+
5. Пользователь **фиксирует** выбранный CJM → `{CHOSEN_CJM}`
|
|
245
|
+
|
|
246
|
+
**{CHOSEN_CJM} передаётся во все последующие фазы:**
|
|
247
|
+
- Phase 3: user flow строится на базе выбранного CJM
|
|
248
|
+
- Phase 4: MVP scope = экраны из CJM
|
|
249
|
+
- Phase 5: слайд с CJM в презентации
|
|
250
|
+
|
|
251
|
+
#### Module 3: MARKET & COMPETITION → TAM/SAM + Game Theory + TRIZ
|
|
252
|
+
- TAM/SAM/SOM оценка рынка
|
|
253
|
+
- Game Theory: матрица стратегий конкурентов, Nash equilibrium
|
|
254
|
+
- TRIZ: противоречия в рынке → изобретательские решения
|
|
255
|
+
- Competitive Aha: сравнение aha-момента конкурента vs `{CHOSEN_CJM}`
|
|
256
|
+
- **Артефакт:** Market analysis + competition matrix
|
|
257
|
+
|
|
258
|
+
#### Module 4: BUSINESS & FINANCE → Unit Economics + P&L
|
|
259
|
+
- Unit economics: LTV, CAC, LTV/CAC ratio (с бенчмарками из [industry-benchmarks.md](references/industry-benchmarks.md))
|
|
260
|
+
- P&L модель на 3 года
|
|
261
|
+
- Sensitivity analysis: что если CAC ×2? что если retention −20%?
|
|
262
|
+
- Revenue model привязан к paywall timing из `{CHOSEN_CJM}`
|
|
263
|
+
- **Артефакт:** Financial model + sensitivity table
|
|
264
|
+
|
|
265
|
+
#### Module 5: GROWTH ENGINE → Loop + Channels + Moats
|
|
266
|
+
- Growth Loop: вирусный цикл из CJM
|
|
267
|
+
- Channel scoring: organic vs paid vs partnership
|
|
268
|
+
- Moats: network effects, data, brand, switching costs
|
|
269
|
+
- Second-Order Thinking: "что произойдёт через 3 года?"
|
|
270
|
+
- **Артефакт:** Growth strategy + moat analysis
|
|
271
|
+
|
|
272
|
+
#### Module 6: PLAYBOOK SYNTHESIS → 90-Day Plan + BS-check
|
|
273
|
+
- 90-Day Launch Plan: Week 1-4 → 5-8 → 9-12 с конкретными задачами
|
|
274
|
+
- MVP scope = экраны из `{CHOSEN_CJM}`
|
|
275
|
+
- BS-check (DEEP mode): вызывает `brutal-honesty-review` — безжалостная проверка на bullsh*t
|
|
276
|
+
- **Артефакт:** Launch playbook + BS-check report
|
|
277
|
+
|
|
278
|
+
### Параллелизм внутри модулей
|
|
279
|
+
|
|
280
|
+
```
|
|
281
|
+
Module 2.5: Agent 1 (opus) → Variant A
|
|
282
|
+
Agent 2 (opus) → Variant B + C
|
|
283
|
+
Agent 3 (sonnet) → Trend Research → Variant D
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
### Пример использования
|
|
287
|
+
|
|
288
|
+
```
|
|
289
|
+
"Проанализируй Notion в режиме DEEP"
|
|
290
|
+
→ Module 0: Pre-flight (Notion, notion.so, Productivity, Global, DEEP)
|
|
291
|
+
→ Module 1: Fact Sheet (2016, SF, $10B valuation, 100M users)
|
|
292
|
+
→ Module 2: JTBD (3 сегмента: students, teams, enterprise) + VoC
|
|
293
|
+
→ Module 2.5: 3 CJM варианта → React прототип → пользователь выбирает Variant B
|
|
294
|
+
→ Module 3: TAM $40B, конкуренты (Obsidian, Coda, ClickUp), TRIZ
|
|
295
|
+
→ Module 4: LTV/CAC=4.2, P&L breakeven Year 2
|
|
296
|
+
→ Module 5: Growth via templates (viral loop), moat = integrations + community
|
|
297
|
+
→ Module 6: 90-Day Plan + BS-check ✅
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
---
|
|
301
|
+
|
|
302
|
+
## Slash-команды
|
|
303
|
+
|
|
304
|
+
### Pipeline-команды (7 фаз)
|
|
305
|
+
|
|
306
|
+
| Команда | Фаза | Что делает |
|
|
307
|
+
|---------|------|-----------|
|
|
308
|
+
| `/casarium [текст]` | Все | Полный 7-фазный pipeline |
|
|
309
|
+
| `/casarium --ai-factory [текст]` | Все | + маппинг на Cloud.ru AI Factory |
|
|
310
|
+
| `/discovery` | 0 | Product Discovery |
|
|
311
|
+
| `/explore-case` | 1 | Глубокое понимание кейса |
|
|
312
|
+
| `/research` | 2 | PARANOID research (3 агента) |
|
|
313
|
+
| `/cjm-prototype` | 2.5 | CJM → React прототип |
|
|
314
|
+
| `/solve` | 3 | Стратегия решения |
|
|
315
|
+
| `/architecture-phase` | 4 | Техническая архитектура |
|
|
316
|
+
| `/presentation` | 5 | Презентация + speaker script |
|
|
317
|
+
|
|
318
|
+
### Управление исследованиями
|
|
319
|
+
|
|
320
|
+
| Команда | Что делает |
|
|
321
|
+
|---------|-----------|
|
|
322
|
+
| `/new-research [название]` | Создать новую директорию исследования |
|
|
323
|
+
| `/parallel-research кейс1 \| кейс2` | Параллельный запуск Phase 0 для нескольких кейсов |
|
|
324
|
+
| `/harvest [путь]` | Извлечение знаний (5 агентов → 7 категорий → 8 gates) |
|
|
325
|
+
| `/brain-export` | Экспорт знаний в portable JSON (v1.1, SHA-256 manifest) |
|
|
326
|
+
| `/brain-import [файл]` | Импорт знаний из другого проекта |
|
|
327
|
+
|
|
328
|
+
### Инструменты разработки
|
|
329
|
+
|
|
330
|
+
| Команда | Что делает |
|
|
331
|
+
|---------|-----------|
|
|
332
|
+
| `/feature-adr [описание]` | 11-шаговый pipeline разработки фич (S/M/L/XL) |
|
|
333
|
+
| `/analyst-manual [тема]` | Стратегический анализ с checkpoint'ами |
|
|
334
|
+
| `/bto [путь]` | Build-Benchmark-Test-Optimize |
|
|
335
|
+
| `/init-platform --platform X` | Генерация конфигов для Cursor/OpenCode/Copilot |
|
|
336
|
+
|
|
337
|
+
### Самообучение
|
|
338
|
+
|
|
339
|
+
| Команда | Что делает |
|
|
340
|
+
|---------|-----------|
|
|
341
|
+
| `/learning-stats` | Dashboard аналитики reward learning |
|
|
342
|
+
| `/dream run` | Запуск dream cycle (анализ паттернов) |
|
|
343
|
+
| `/dream insights` | Показать последние инсайты |
|
|
344
|
+
| `/workers start <тип>` | Запуск фонового воркера |
|
|
345
|
+
| `/workers status` | Статус воркеров |
|
|
346
|
+
|
|
347
|
+
---
|
|
348
|
+
|
|
349
|
+
## Правила (Rules)
|
|
350
|
+
|
|
351
|
+
14 правил в `.claude/rules/` — всегда активны, применяются автоматически.
|
|
352
|
+
|
|
353
|
+
| Правило | Назначение |
|
|
354
|
+
|---------|-----------|
|
|
355
|
+
| `research-quality` | PARANOID mode: каждый факт должен иметь источник, confidence ≥ 0.99 |
|
|
356
|
+
| `checkpoint-protocol` | Формат checkpoint'ов, promise tags, запрет авто-продвижения |
|
|
357
|
+
| `agent-swarm` | Когда и как запускать параллельных агентов, модели, изоляция |
|
|
358
|
+
| `domain-specific` | Авто-детекция домена → доменные правила (Banking, Retail, Enterprise, Healthcare) |
|
|
359
|
+
| `anti-patterns` | 9 запрещённых паттернов с авто-детекцией и fix'ами |
|
|
360
|
+
| `file-conventions` | Всё в `researches/<slug>/`, нумерация, timing |
|
|
361
|
+
| `modular-reuse` | Скиллы domain-agnostic, команды pipeline-specific |
|
|
362
|
+
| `feedback-loops` | 7 именованных cross-phase feedback loops с контрактами |
|
|
363
|
+
| `model-routing` | 3-tier маршрутизация: haiku (1x) / sonnet (15x) / opus (75x) |
|
|
364
|
+
| `trust-tiers` | 4-tier классификация скиллов (Advisory → Structured → Validated → Verified) |
|
|
365
|
+
| `background-workers` | Авто-запуск, изоляция, concurrency limit (max 3) |
|
|
366
|
+
| `dream-cycles` | Триггеры dream cycles, интеграция с memory, retention |
|
|
367
|
+
| `reward-learning` | Когда вызывать memory_query/store, reward scores (1.0/0.7/0.3/0.0) |
|
|
368
|
+
| `feature-adr-conventions` | Когда /feature-adr vs /casarium, output directory, slug naming |
|
|
369
|
+
|
|
370
|
+
---
|
|
371
|
+
|
|
372
|
+
## Governance Shards
|
|
373
|
+
|
|
374
|
+
9 shards в `.claude/shards/` — загружаются перед каждой фазой для предотвращения context drift.
|
|
375
|
+
|
|
376
|
+
Каждый shard содержит:
|
|
377
|
+
- **Time budget** — % от общего времени
|
|
378
|
+
- **Skill to load** — какой скилл загрузить
|
|
379
|
+
- **Prerequisites** — upstream promise tags
|
|
380
|
+
- **Quality gates** — что проверить перед выходом
|
|
381
|
+
- **Agent swarm config** — параллельные агенты
|
|
382
|
+
- **Promise tag** — что эмитировать в checkpoint
|
|
383
|
+
|
|
384
|
+
| Shard | Фаза | Обязательные upstream |
|
|
385
|
+
|-------|------|---------------------|
|
|
386
|
+
| `phase-0-discovery` | Discovery | — |
|
|
387
|
+
| `phase-1-explore` | Explore | `DISCOVERY_COMPLETE` |
|
|
388
|
+
| `phase-2-research` | Research | `CASE_EXPLORED` |
|
|
389
|
+
| `phase-25-cjm` | CJM Prototype ⭐ | `RESEARCH_PARANOID_PASSED` |
|
|
390
|
+
| `phase-3-solve` | Solve | `CJM_VALIDATED` + `{CHOSEN_CJM}` |
|
|
391
|
+
| `phase-4-architecture` | Architecture | `SOLUTION_DESIGNED` |
|
|
392
|
+
| `phase-5-presentation` | Presentation | `ARCHITECTURE_DEFINED` |
|
|
393
|
+
| `phase-ai-factory` | AI Factory (все фазы) | Только при `--ai-factory` |
|
|
394
|
+
| `feature-adr` | Feature Development | Независим от casarium |
|
|
395
|
+
|
|
396
|
+
---
|
|
397
|
+
|
|
398
|
+
## Система самообучения
|
|
399
|
+
|
|
400
|
+
### Reward-Calibrated Learning
|
|
401
|
+
|
|
402
|
+
На каждом checkpoint:
|
|
403
|
+
1. `memory_query()` — загружает исторические паттерны для текущего домена/фазы
|
|
404
|
+
2. Пользователь отвечает → reward score:
|
|
405
|
+
|
|
406
|
+
| Ответ | Reward | Метка |
|
|
407
|
+
|-------|--------|-------|
|
|
408
|
+
| "ок" | 1.0 | excellent |
|
|
409
|
+
| "углуби X" | 0.7 | good |
|
|
410
|
+
| "переделай" (3+ замечаний) | 0.3 | needs_work |
|
|
411
|
+
| "заново" | 0.0 | failed |
|
|
412
|
+
|
|
413
|
+
3. `memory_store()` — сохраняет результат с promise tag и контекстом
|
|
414
|
+
4. Данные хранятся в `.keysarium/memory/` (90 дней retention)
|
|
415
|
+
|
|
416
|
+
### Аналитика
|
|
417
|
+
|
|
418
|
+
```
|
|
419
|
+
/learning-stats # общий dashboard
|
|
420
|
+
/learning-stats --domain banking # по домену
|
|
421
|
+
/learning-stats --phase 2 # по фазе
|
|
422
|
+
```
|
|
423
|
+
|
|
424
|
+
---
|
|
425
|
+
|
|
426
|
+
## Фоновые воркеры и Dream Cycles
|
|
427
|
+
|
|
428
|
+
### 4 типа воркеров (max 3 одновременно)
|
|
429
|
+
|
|
430
|
+
| Тип | Модель | Что делает |
|
|
431
|
+
|-----|--------|-----------|
|
|
432
|
+
| `consolidate` | sonnet | Анализирует завершённые исследования, обновляет TOOLKIT_HARVEST.md |
|
|
433
|
+
| `export-brain` | haiku | Фоновый экспорт brain в JSON |
|
|
434
|
+
| `health-check` | haiku | Проверка trust tiers, поиск устаревших данных |
|
|
435
|
+
| `pattern-analysis` | sonnet | Анализ трендов в reward data |
|
|
436
|
+
|
|
437
|
+
### Dream Cycles
|
|
438
|
+
|
|
439
|
+
Background sonnet-агент анализирует накопленные reward данные → генерирует инсайты.
|
|
440
|
+
|
|
441
|
+
**Триггеры:** 20+ новых записей, 7+ дней с последнего dream, quality gate failure, завершение кейса.
|
|
442
|
+
|
|
443
|
+
**Инсайты применяются в:**
|
|
444
|
+
- Phase 0: фильтрация по домену, предупреждения
|
|
445
|
+
- Phase 2: warning о проблемных фазах для текущего домена
|
|
446
|
+
|
|
447
|
+
```
|
|
448
|
+
/dream run # Запустить dream cycle
|
|
449
|
+
/dream insights # Показать инсайты
|
|
450
|
+
/dream status # Состояние триггеров
|
|
451
|
+
/dream clear # Очистить старые (оставить 10)
|
|
452
|
+
```
|
|
453
|
+
|
|
454
|
+
---
|
|
455
|
+
|
|
456
|
+
## Портативный мозг (Brain)
|
|
457
|
+
|
|
458
|
+
Brain v1.1 — JSON контейнер с SHA-256 manifest для переноса знаний между проектами.
|
|
459
|
+
|
|
460
|
+
```bash
|
|
461
|
+
/brain-export # Полный экспорт
|
|
462
|
+
/brain-export --delta <parent-brain.json> # Дельта (COW, JSON Patch RFC 6902)
|
|
463
|
+
/brain-import <brain.json> # Импорт с merge-not-overwrite
|
|
464
|
+
```
|
|
465
|
+
|
|
466
|
+
**Содержит:** skill metadata, domain patterns, research summaries, harvest patterns, pipeline metrics, reward data.
|
|
467
|
+
|
|
468
|
+
**Lifecycle записей:** HOT → WARM → COLD → PURGE.
|
|
469
|
+
|
|
470
|
+
---
|
|
471
|
+
|
|
472
|
+
## Мульти-платформенная поддержка
|
|
473
|
+
|
|
474
|
+
```bash
|
|
475
|
+
/init-platform --platform cursor # → .cursorrules
|
|
476
|
+
/init-platform --platform opencode # → .opencode/
|
|
477
|
+
/init-platform --platform copilot # → .github/copilot-instructions.md
|
|
478
|
+
```
|
|
479
|
+
|
|
480
|
+
Транслирует содержимое `.claude/` в формат целевой платформы.
|
|
481
|
+
|
|
482
|
+
---
|
|
483
|
+
|
|
484
|
+
## Доменная поддержка
|
|
485
|
+
|
|
486
|
+
Домен авто-детектируется из описания кейса и активирует специфичные правила:
|
|
487
|
+
|
|
488
|
+
| Домен | Ключевые ограничения | Палитра |
|
|
489
|
+
|-------|---------------------|---------|
|
|
490
|
+
| **Banking/FinTech** | On-premise LLM (GigaChat/YandexGPT), ФЗ-152, ЦБ, ФСТЭК, HITL обязателен | Blue/Navy/Silver |
|
|
491
|
+
| **Retail/E-commerce** | Latency < 200ms, A/B testing, GDPR/ФЗ-152, персонализация vs privacy | Amber/Orange |
|
|
492
|
+
| **Enterprise/B2B** | Change Management, Legacy integration, SLA, ROI в FTE/часах | Teal/Indigo |
|
|
493
|
+
| **Healthcare** | HITL для ВСЕХ клинических решений, ФЗ-323, explainability, изоляция данных | — |
|
|
494
|
+
|
|
495
|
+
---
|
|
496
|
+
|
|
497
|
+
## Антипаттерны
|
|
498
|
+
|
|
499
|
+
| Паттерн | Исправление |
|
|
500
|
+
|---------|------------|
|
|
501
|
+
| «Засунем GPT и всё заработает» | Конкретные модели + pipeline |
|
|
502
|
+
| Игнорировать ограничения | Явно адресовать каждое |
|
|
503
|
+
| Слишком сложная архитектура | MVP-first |
|
|
504
|
+
| Нет метрик | Конкретные KPI + baseline |
|
|
505
|
+
| Читать со слайдов | Storytelling + визуалы |
|
|
506
|
+
| Нет Human-in-the-Loop | Чёткая escalation policy |
|
|
507
|
+
| «Улучшим эффективность» без цифр | «Сократим с 4 часов до 15 минут» |
|
|
508
|
+
| Пропустить Phase 2.5 (CJM) | БЛОКИРОВКА — CJM обязательна |
|
|
509
|
+
| Нет Executive Summary | БЛОКИРОВКА — `08_executive_summary.md` обязателен |
|
|
510
|
+
|
|
511
|
+
---
|
|
512
|
+
|
|
513
|
+
## Структура проекта
|
|
514
|
+
|
|
515
|
+
```
|
|
516
|
+
.claude/
|
|
517
|
+
├── commands/ ← 20 slash-команд (/casarium, /discovery, /harvest, ...)
|
|
518
|
+
├── shards/ ← 9 governance shards (per-phase правила)
|
|
519
|
+
├── skills/ ← 12 скиллов (building blocks)
|
|
520
|
+
└── rules/ ← 14 правил (всегда активны)
|
|
521
|
+
lib/ ← Библиотеки (memory, dream, workers, platform adapters)
|
|
522
|
+
researches/ ← Результаты исследований (по кейсам)
|
|
523
|
+
features/ ← Результаты feature-adr (по фичам)
|
|
524
|
+
.keysarium/ ← Runtime данные (memory, insights, exports, workers)
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
---
|
|
528
|
+
|
|
529
|
+
## Связанные пакеты
|
|
530
|
+
|
|
531
|
+
| Пакет | Что делает |
|
|
532
|
+
|-------|-----------|
|
|
533
|
+
| [@dzhechkov/keysarium](https://www.npmjs.com/package/@dzhechkov/keysarium) | Этот тулкит (research pipeline) |
|
|
534
|
+
| [@dzhechkov/keysarium-core](https://www.npmjs.com/package/@dzhechkov/keysarium-core) | Shared framework (governance, memory, orchestration) |
|
|
535
|
+
| [@dzhechkov/skills-bto](https://www.npmjs.com/package/@dzhechkov/skills-bto) | Build-Benchmark-Test-Optimize |
|
|
536
|
+
| [@dzhechkov/skills-feature-adr](https://www.npmjs.com/package/@dzhechkov/skills-feature-adr) | 11-step feature pipeline |
|
|
537
|
+
| [@dzhechkov/harness-cli](https://www.npmjs.com/package/@dzhechkov/harness-cli) | `dz` CLI — единая точка входа |
|
|
538
|
+
|
|
539
|
+
---
|
|
540
|
+
|
|
541
|
+
*Версия 1.5.0 · MIT License · [GitHub](https://github.com/djd1m/dz-harness-hub/tree/main/packages/@dzhechkov/keysarium)*
|
package/bin/cli.js
CHANGED
|
File without changes
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@dzhechkov/keysarium",
|
|
3
|
-
"version": "1.5.
|
|
3
|
+
"version": "1.5.2",
|
|
4
4
|
"description": "AI Case Research Toolkit for Claude Code — full 7-phase pipeline with feature-adr, dream cycles, background workers, multi-platform support, reward learning, and Cloud.ru AI Factory mapping (--ai-factory flag)",
|
|
5
5
|
"main": "src/cli.js",
|
|
6
6
|
"bin": {
|
|
@@ -11,10 +11,6 @@
|
|
|
11
11
|
"src/",
|
|
12
12
|
"templates/"
|
|
13
13
|
],
|
|
14
|
-
"scripts": {
|
|
15
|
-
"test": "node bin/cli.js doctor",
|
|
16
|
-
"prepublishOnly": "node scripts/sync-templates.js"
|
|
17
|
-
},
|
|
18
14
|
"keywords": [
|
|
19
15
|
"claude",
|
|
20
16
|
"claude-code",
|
|
@@ -36,9 +32,10 @@
|
|
|
36
32
|
},
|
|
37
33
|
"repository": {
|
|
38
34
|
"type": "git",
|
|
39
|
-
"url": "https://github.com/
|
|
35
|
+
"url": "https://github.com/djd1m/dz-harness-hub.git",
|
|
36
|
+
"directory": "packages/@dzhechkov/keysarium"
|
|
40
37
|
},
|
|
41
|
-
"homepage": "https://github.com/
|
|
38
|
+
"homepage": "https://github.com/djd1m/dz-harness-hub/tree/main/packages/@dzhechkov/keysarium#readme",
|
|
42
39
|
"bugs": {
|
|
43
40
|
"url": "https://github.com/dzhechko/product-keysarium-2026/issues"
|
|
44
41
|
},
|
|
@@ -49,5 +46,8 @@
|
|
|
49
46
|
"@dzhechkov/keysarium-core": {
|
|
50
47
|
"optional": true
|
|
51
48
|
}
|
|
49
|
+
},
|
|
50
|
+
"publishConfig": {
|
|
51
|
+
"access": "public"
|
|
52
52
|
}
|
|
53
|
-
}
|
|
53
|
+
}
|
package/src/cli.js
CHANGED
|
@@ -91,8 +91,17 @@ function parseArgs(argv) {
|
|
|
91
91
|
command = 'version';
|
|
92
92
|
break;
|
|
93
93
|
default:
|
|
94
|
-
if (
|
|
94
|
+
if (arg.startsWith('-')) {
|
|
95
|
+
// Reject unknown options loudly — a typo like --dry-rum or --froce must
|
|
96
|
+
// never silently downgrade behavior (real install instead of dry-run, or
|
|
97
|
+
// an unguarded destructive command).
|
|
98
|
+
error(`Unknown option: ${arg}`);
|
|
99
|
+
process.exit(1);
|
|
100
|
+
} else if (command === null) {
|
|
95
101
|
command = arg;
|
|
102
|
+
} else {
|
|
103
|
+
error(`Unexpected argument: ${arg}`);
|
|
104
|
+
process.exit(1);
|
|
96
105
|
}
|
|
97
106
|
break;
|
|
98
107
|
}
|
package/src/commands/init.js
CHANGED
|
@@ -5,7 +5,7 @@ const path = require('path');
|
|
|
5
5
|
const {
|
|
6
6
|
green, yellow, cyan, bold, dim,
|
|
7
7
|
info, success, warn, error: logError, step,
|
|
8
|
-
copyDirRecursive, fileExists, readJSON,
|
|
8
|
+
copyDirRecursive, copyDirTracked, fileExists, readJSON,
|
|
9
9
|
ensureDir, getRelativePaths,
|
|
10
10
|
createManifest, writeManifest, getTemplatesDir,
|
|
11
11
|
COMPONENTS, MANIFEST_FILE,
|
|
@@ -41,7 +41,7 @@ function resolveComponents(options) {
|
|
|
41
41
|
* Copy a single component from templates to the target directory.
|
|
42
42
|
* Returns array of relative file paths that were installed.
|
|
43
43
|
*/
|
|
44
|
-
function installComponent(comp, templatesDir, targetDir) {
|
|
44
|
+
function installComponent(comp, templatesDir, targetDir, force) {
|
|
45
45
|
const src = path.join(templatesDir, comp.src);
|
|
46
46
|
const dest = path.join(targetDir, comp.src);
|
|
47
47
|
|
|
@@ -51,13 +51,21 @@ function installComponent(comp, templatesDir, targetDir) {
|
|
|
51
51
|
}
|
|
52
52
|
|
|
53
53
|
if (comp.isFile) {
|
|
54
|
+
if (fileExists(dest) && !force) {
|
|
55
|
+
// Preserve a user-modified file (e.g. their own CLAUDE.md / settings.json)
|
|
56
|
+
// when re-installing without --force. It is still manifest-tracked because
|
|
57
|
+
// it originates from the template.
|
|
58
|
+
return [comp.src];
|
|
59
|
+
}
|
|
54
60
|
ensureDir(path.dirname(dest));
|
|
55
61
|
fs.copyFileSync(src, dest);
|
|
56
62
|
return [comp.src];
|
|
57
63
|
}
|
|
58
64
|
|
|
59
|
-
|
|
60
|
-
|
|
65
|
+
// Track files via the TEMPLATE source — never a scan of the destination —
|
|
66
|
+
// so user-created files under dest are neither recorded nor (later) removed.
|
|
67
|
+
const { copied, skipped } = copyDirTracked(src, dest, force);
|
|
68
|
+
return [...copied, ...skipped].map((rel) => path.join(comp.src, rel));
|
|
61
69
|
}
|
|
62
70
|
|
|
63
71
|
// ---------------------------------------------------------------------------
|
|
@@ -118,7 +126,7 @@ async function run(options) {
|
|
|
118
126
|
const comp = COMPONENTS[key];
|
|
119
127
|
step(i + 1, totalComponents, `Installing ${comp.label}...`);
|
|
120
128
|
|
|
121
|
-
const files = installComponent(comp, templatesDir, targetDir);
|
|
129
|
+
const files = installComponent(comp, templatesDir, targetDir, force);
|
|
122
130
|
installedFiles.push(...files);
|
|
123
131
|
}
|
|
124
132
|
|
package/src/commands/update.js
CHANGED
|
@@ -159,12 +159,16 @@ async function run(options) {
|
|
|
159
159
|
const comp = COMPONENTS[key];
|
|
160
160
|
if (!comp) continue;
|
|
161
161
|
const destPath = path.join(targetDir, comp.src);
|
|
162
|
+
const srcPath = path.join(templatesDir, comp.src);
|
|
162
163
|
if (comp.isFile) {
|
|
163
164
|
if (fileExists(destPath)) {
|
|
164
165
|
allFiles.push(comp.src);
|
|
165
166
|
}
|
|
166
167
|
} else if (fileExists(destPath)) {
|
|
167
|
-
|
|
168
|
+
// Record files from the TEMPLATE source, not a scan of the destination —
|
|
169
|
+
// otherwise user-created files in dest get adopted into the manifest and
|
|
170
|
+
// a later `remove` would delete them.
|
|
171
|
+
const paths = getRelativePaths(fileExists(srcPath) ? srcPath : destPath);
|
|
168
172
|
allFiles.push(...paths.map((rel) => path.join(comp.src, rel)));
|
|
169
173
|
}
|
|
170
174
|
}
|
package/src/utils.js
CHANGED
|
@@ -64,6 +64,39 @@ function copyDirRecursive(src, dest) {
|
|
|
64
64
|
}
|
|
65
65
|
}
|
|
66
66
|
|
|
67
|
+
/**
|
|
68
|
+
* Copy from a TEMPLATE source into dest, tracking which files were written.
|
|
69
|
+
* Walks the SOURCE tree (never the destination), so only template-originated
|
|
70
|
+
* files are ever recorded — a user-created file in dest with no template
|
|
71
|
+
* counterpart is never touched and never returned (prevents `remove` from
|
|
72
|
+
* deleting user data). When `force` is false, a pre-existing destination file
|
|
73
|
+
* is PRESERVED (skipped) rather than clobbered (protects user-modified files
|
|
74
|
+
* on a re-install where the manifest was lost).
|
|
75
|
+
*
|
|
76
|
+
* @returns {{copied: string[], skipped: string[]}} relative paths (to dest root)
|
|
77
|
+
*/
|
|
78
|
+
function copyDirTracked(src, dest, force, rel = '', acc = { copied: [], skipped: [] }) {
|
|
79
|
+
const stat = fs.statSync(src);
|
|
80
|
+
if (stat.isFile()) {
|
|
81
|
+
if (fileExists(dest) && !force) {
|
|
82
|
+
acc.skipped.push(rel);
|
|
83
|
+
} else {
|
|
84
|
+
ensureDir(path.dirname(dest));
|
|
85
|
+
fs.copyFileSync(src, dest);
|
|
86
|
+
acc.copied.push(rel);
|
|
87
|
+
}
|
|
88
|
+
return acc;
|
|
89
|
+
}
|
|
90
|
+
if (stat.isDirectory()) {
|
|
91
|
+
ensureDir(dest);
|
|
92
|
+
for (const entry of fs.readdirSync(src)) {
|
|
93
|
+
copyDirTracked(path.join(src, entry), path.join(dest, entry), force,
|
|
94
|
+
rel ? path.join(rel, entry) : entry, acc);
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
return acc;
|
|
98
|
+
}
|
|
99
|
+
|
|
67
100
|
/**
|
|
68
101
|
* Returns true if the path exists (file or directory).
|
|
69
102
|
*/
|
|
@@ -254,6 +287,7 @@ module.exports = {
|
|
|
254
287
|
|
|
255
288
|
// File operations
|
|
256
289
|
copyDirRecursive,
|
|
290
|
+
copyDirTracked,
|
|
257
291
|
fileExists,
|
|
258
292
|
readJSON,
|
|
259
293
|
writeJSON,
|
|
@@ -1,302 +0,0 @@
|
|
|
1
|
-
# /bto-benchmark — Deterministic Benchmarking of a Skill or Command
|
|
2
|
-
|
|
3
|
-
## Usage
|
|
4
|
-
```
|
|
5
|
-
/bto-benchmark [path to skill directory or file]
|
|
6
|
-
```
|
|
7
|
-
|
|
8
|
-
## Parameters
|
|
9
|
-
- $ARGUMENTS — Path to a skill directory (`.claude/skills/<name>/`), a command file (`.claude/commands/<name>.md`), a rule file (`.claude/rules/<name>.md`), or any structured artifact. Optionally include "verbose" to show expanded per-section diagnostics for all layers.
|
|
10
|
-
|
|
11
|
-
## Protocol
|
|
12
|
-
|
|
13
|
-
### Step 1: Load Skill and Module
|
|
14
|
-
|
|
15
|
-
Read `.claude/skills/bto/SKILL.md`
|
|
16
|
-
Read `.claude/skills/bto/modules/benchmark.md`
|
|
17
|
-
Read `.claude/skills/bto/references/golden-samples.md`
|
|
18
|
-
|
|
19
|
-
### Step 2: Validate Input
|
|
20
|
-
|
|
21
|
-
If $ARGUMENTS is empty:
|
|
22
|
-
- Ask: "Provide a path to the skill directory or artifact file you want to benchmark."
|
|
23
|
-
- Stop and wait.
|
|
24
|
-
|
|
25
|
-
Resolve the artifact path from $ARGUMENTS:
|
|
26
|
-
- Strip any trailing slash
|
|
27
|
-
- If the path points to a directory → look for `SKILL.md` inside it as the primary file, but include all files in the directory for context
|
|
28
|
-
- If the path points to a file → use that file directly
|
|
29
|
-
- If the path does not exist → report "Path not found: [path]" and stop
|
|
30
|
-
|
|
31
|
-
### Step 3: Detect Artifact Type
|
|
32
|
-
|
|
33
|
-
Auto-detect from path pattern:
|
|
34
|
-
|
|
35
|
-
| Path Pattern | Detected Type |
|
|
36
|
-
|-------------|--------------|
|
|
37
|
-
| `.claude/skills/*/SKILL.md` or `.claude/skills/*/` | skill |
|
|
38
|
-
| `.claude/commands/*.md` | command |
|
|
39
|
-
| `.claude/rules/*.md` | rule |
|
|
40
|
-
| `.claude/agents/*.md` | agent |
|
|
41
|
-
| `researches/**/*.md` | research artifact |
|
|
42
|
-
|
|
43
|
-
If type cannot be determined from path → infer from file content structure.
|
|
44
|
-
|
|
45
|
-
### Step 4: Layer B0 — Golden Sample Comparison (Deterministic)
|
|
46
|
-
|
|
47
|
-
**Purpose:** Compare artifact structure against the canonical golden sample for its type. Zero LLM cost.
|
|
48
|
-
|
|
49
|
-
Load golden samples for the detected type from `references/golden-samples.md`.
|
|
50
|
-
|
|
51
|
-
**Procedure:**
|
|
52
|
-
|
|
53
|
-
1. Parse the artifact into a section tree (heading hierarchy, section lengths, ordering)
|
|
54
|
-
2. Parse the matching golden sample into the same section tree format
|
|
55
|
-
3. Compare across three structural axes:
|
|
56
|
-
|
|
57
|
-
**Axis 1 — Section Coverage:**
|
|
58
|
-
For each required section in the golden sample, check if the artifact contains a matching section (by heading text or semantic equivalent).
|
|
59
|
-
- `section_coverage = matched_sections / total_golden_sections`
|
|
60
|
-
|
|
61
|
-
**Axis 2 — Ordering Score:**
|
|
62
|
-
Compare the ordering of matched sections against the golden sample ordering.
|
|
63
|
-
- Use normalized Kendall tau distance: `ordering_score = 1 - (inversions / max_inversions)`
|
|
64
|
-
- If fewer than 3 matched sections → ordering_score = 1.0 (too few to meaningfully compare)
|
|
65
|
-
|
|
66
|
-
**Axis 3 — Proportion Score:**
|
|
67
|
-
For each matched section, compute the ratio of artifact section length to golden sample section length.
|
|
68
|
-
- `proportion_score = 1 - mean(|log(artifact_len / golden_len)|)` clamped to [0, 1]
|
|
69
|
-
- Measures whether sections are proportionally sized (not too bloated, not too thin)
|
|
70
|
-
|
|
71
|
-
**Display per-section MATCH/MISS table:**
|
|
72
|
-
|
|
73
|
-
```
|
|
74
|
-
Golden Section Status Artifact Section Size Ratio
|
|
75
|
-
─────────────────────────── ──────── ──────────────────────────── ──────────
|
|
76
|
-
## Overview MATCH ## Overview 1.2x
|
|
77
|
-
## Protocol MATCH ## Protocol 0.8x
|
|
78
|
-
## Anti-Patterns MATCH ## Anti-Patterns 1.0x
|
|
79
|
-
## Dependencies MISS — —
|
|
80
|
-
## Quick Start MATCH ## Quick Start 1.5x
|
|
81
|
-
```
|
|
82
|
-
|
|
83
|
-
**Golden Similarity Score:**
|
|
84
|
-
```
|
|
85
|
-
B0 = section_coverage * 0.50 + ordering_score * 0.25 + proportion_score * 0.25
|
|
86
|
-
```
|
|
87
|
-
|
|
88
|
-
**Gate:** If B0 < 0.30 → BLOCK immediately. The artifact's structure is too far from the expected form to benchmark meaningfully.
|
|
89
|
-
|
|
90
|
-
### Step 5: Layer B1 — Deterministic Test Suite
|
|
91
|
-
|
|
92
|
-
**Purpose:** Run all applicable deterministic tests for the detected artifact type. Zero LLM cost.
|
|
93
|
-
|
|
94
|
-
Execute all applicable checks from `benchmark.md` for the detected artifact type. These overlap with but extend the Layer 0 checks from `/bto-test`:
|
|
95
|
-
|
|
96
|
-
**Universal Checks (U1-U12):**
|
|
97
|
-
- U-01: File exists and is non-empty
|
|
98
|
-
- U-02: UTF-8 encoding valid
|
|
99
|
-
- U-03: Starts with level-1 heading
|
|
100
|
-
- U-04: No placeholder text (`[TODO]`, `[TBD]`, `<INSERT>`, `[PLACEHOLDER]`)
|
|
101
|
-
- U-05: No empty sections (heading with no content before next heading)
|
|
102
|
-
- U-06: Consistent heading hierarchy (no h1→h3 jumps)
|
|
103
|
-
- U-07: No broken internal cross-references
|
|
104
|
-
- U-08: File size within bounds (200B – 100KB per file)
|
|
105
|
-
- U-09: No trailing whitespace on lines
|
|
106
|
-
- U-10: Standard Markdown only (no HTML unless in code blocks)
|
|
107
|
-
- U-11: All code blocks properly closed
|
|
108
|
-
- U-12: No duplicate top-level sections
|
|
109
|
-
|
|
110
|
-
**Type-Specific Checks:**
|
|
111
|
-
- Skill: SK-01 through SK-16 (SKILL.md exists, required sections, modules/, references/, examples/, etc.)
|
|
112
|
-
- Command: CM-01 through CM-08 (Usage section, Parameters, Protocol steps, Checkpoint, Critical Rules, etc.)
|
|
113
|
-
- Rule: RL-01 through RL-06 (Table format, detection signals, fix actions, no contradictions, etc.)
|
|
114
|
-
- Agent: AG-01 through AG-04 (Model specified, isolation rules, naming convention, output format)
|
|
115
|
-
|
|
116
|
-
**Display per-test PASS/FAIL table:**
|
|
117
|
-
|
|
118
|
-
```
|
|
119
|
-
ID Check Result Note
|
|
120
|
-
─────── ────────────────────────────────── ──────── ────────────────────────────
|
|
121
|
-
U-01 File exists and is non-empty PASS 8.4 KB
|
|
122
|
-
U-02 UTF-8 encoding PASS
|
|
123
|
-
...
|
|
124
|
-
SK-01 SKILL.md exists at skill root PASS
|
|
125
|
-
SK-02 Has ## Overview section PASS
|
|
126
|
-
SK-03 Has ## Anti-Patterns section FAIL Section missing
|
|
127
|
-
...
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
**Test Pass Rate:**
|
|
131
|
-
```
|
|
132
|
-
B1 = passed_tests / total_applicable_tests
|
|
133
|
-
```
|
|
134
|
-
|
|
135
|
-
**Gate:** If B1 < 0.60 → BLOCK. Too many structural failures to proceed to consistency probing.
|
|
136
|
-
|
|
137
|
-
### Step 6: Layer B2 — Consistency Probe
|
|
138
|
-
|
|
139
|
-
**Purpose:** Measure evaluation stability by running identical prompts across multiple agents. Detects artifacts that produce inconsistent interpretations.
|
|
140
|
-
|
|
141
|
-
**Model:** haiku (cost-optimized)
|
|
142
|
-
|
|
143
|
-
**Spawn 3 parallel haiku agents with the identical evaluation prompt:**
|
|
144
|
-
|
|
145
|
-
Each agent receives the exact same prompt:
|
|
146
|
-
```
|
|
147
|
-
You are evaluating a Claude Code artifact for structural quality.
|
|
148
|
-
Rate these 4 dimensions (0.0 to 1.0 each, two decimal places):
|
|
149
|
-
1. STRUCTURE — Does the artifact follow a clear, logical organization?
|
|
150
|
-
2. CLARITY — Are instructions unambiguous and actionable?
|
|
151
|
-
3. COVERAGE — Does it address all aspects implied by its title/scope?
|
|
152
|
-
4. CONSISTENCY — Is the internal terminology and style uniform?
|
|
153
|
-
Output as JSON: {"structure": X.XX, "clarity": X.XX, "coverage": X.XX, "consistency": X.XX}
|
|
154
|
-
```
|
|
155
|
-
|
|
156
|
-
**Isolation:** Each agent evaluates independently. No cross-communication.
|
|
157
|
-
|
|
158
|
-
**After all 3 return, compute structural agreement:**
|
|
159
|
-
|
|
160
|
-
For each dimension:
|
|
161
|
-
```
|
|
162
|
-
agreement[dim] = 1 - (max(scores[dim]) - min(scores[dim]))
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
**Consistency Score:**
|
|
166
|
-
```
|
|
167
|
-
B2 = mean(agreement across all 4 dimensions)
|
|
168
|
-
```
|
|
169
|
-
|
|
170
|
-
**Interpretation:**
|
|
171
|
-
- B2 > 0.85 → HIGH consistency — the artifact is unambiguously structured
|
|
172
|
-
- B2 0.60-0.85 → MODERATE consistency — some sections are open to interpretation
|
|
173
|
-
- B2 < 0.60 → LOW consistency — the artifact is structurally ambiguous
|
|
174
|
-
|
|
175
|
-
**Display per-probe comparison:**
|
|
176
|
-
|
|
177
|
-
```
|
|
178
|
-
Dimension Probe 1 Probe 2 Probe 3 Range Agreement
|
|
179
|
-
──────────── ────────── ────────── ────────── ──────── ──────────
|
|
180
|
-
STRUCTURE 0.80 0.85 0.82 0.05 0.95
|
|
181
|
-
CLARITY 0.70 0.75 0.65 0.10 0.90
|
|
182
|
-
COVERAGE 0.60 0.70 0.55 0.15 0.85
|
|
183
|
-
CONSISTENCY 0.75 0.78 0.73 0.05 0.95
|
|
184
|
-
Mean: 0.91
|
|
185
|
-
```
|
|
186
|
-
|
|
187
|
-
No gate on B2 — consistency is informational and feeds into the aggregate score.
|
|
188
|
-
|
|
189
|
-
### Step 7: Layer B3 — Performance Metrics
|
|
190
|
-
|
|
191
|
-
**Purpose:** Quantify token efficiency and information density. Zero LLM cost.
|
|
192
|
-
|
|
193
|
-
**Metric 1 — Token Efficiency:**
|
|
194
|
-
```
|
|
195
|
-
token_efficiency = meaningful_content_tokens / total_tokens
|
|
196
|
-
```
|
|
197
|
-
Where `meaningful_content_tokens` excludes: blank lines, decorative separators, repeated boilerplate headers, and excessive whitespace. Target: > 0.70.
|
|
198
|
-
|
|
199
|
-
**Metric 2 — Information Density:**
|
|
200
|
-
```
|
|
201
|
-
information_density = unique_concepts / total_sections
|
|
202
|
-
```
|
|
203
|
-
Where `unique_concepts` = count of distinct topics, terms, or instructions introduced. Measures whether each section contributes new information vs. repeating prior content. Target: > 2.0 concepts per section.
|
|
204
|
-
|
|
205
|
-
**Metric 3 — Bloat Detection:**
|
|
206
|
-
Identify sections where:
|
|
207
|
-
- Section length > 3x the golden sample equivalent → BLOATED
|
|
208
|
-
- Section length < 0.2x the golden sample equivalent → THIN
|
|
209
|
-
- Section repeats > 30% of content from another section → REDUNDANT
|
|
210
|
-
|
|
211
|
-
```
|
|
212
|
-
bloat_score = 1 - (bloated_sections + thin_sections + redundant_sections) / total_sections
|
|
213
|
-
```
|
|
214
|
-
Clamped to [0, 1].
|
|
215
|
-
|
|
216
|
-
**Performance Score:**
|
|
217
|
-
```
|
|
218
|
-
B3 = token_efficiency * 0.35 + normalize(information_density, 0, 4) * 0.35 + bloat_score * 0.30
|
|
219
|
-
```
|
|
220
|
-
Where `normalize(x, min, max)` clamps and scales x to [0, 1].
|
|
221
|
-
|
|
222
|
-
**Display per-section metrics:**
|
|
223
|
-
|
|
224
|
-
```
|
|
225
|
-
Section Tokens Concepts Golden Ratio Flag
|
|
226
|
-
───────────────────────── ──────── ────────── ────────────── ──────────
|
|
227
|
-
## Overview 320 4 1.2x OK
|
|
228
|
-
## Protocol 1840 12 0.8x OK
|
|
229
|
-
## Anti-Patterns 580 6 2.8x BLOATED
|
|
230
|
-
## Quick Start 95 2 0.3x THIN
|
|
231
|
-
## Dependencies 210 3 1.0x OK
|
|
232
|
-
```
|
|
233
|
-
|
|
234
|
-
### Step 8: Aggregate & Gate
|
|
235
|
-
|
|
236
|
-
**BENCHMARK_SCORE formula:**
|
|
237
|
-
```
|
|
238
|
-
BENCHMARK_SCORE = B0 * 0.30 + B1 * 0.35 + B2 * 0.15 + B3 * 0.20
|
|
239
|
-
```
|
|
240
|
-
|
|
241
|
-
**Weight rationale:**
|
|
242
|
-
- B0 (Golden Comparison) at 0.30 — structural conformance is the primary benchmark signal
|
|
243
|
-
- B1 (Test Suite) at 0.35 — deterministic correctness is the most reliable signal
|
|
244
|
-
- B2 (Consistency) at 0.15 — stability matters but is a secondary signal
|
|
245
|
-
- B3 (Performance) at 0.20 — efficiency matters for production artifacts
|
|
246
|
-
|
|
247
|
-
**Gate:**
|
|
248
|
-
- BENCHMARK_SCORE < 0.50 → **BLOCK** — artifact requires significant rework before evaluation
|
|
249
|
-
- BENCHMARK_SCORE 0.50-0.70 → **WARN** — artifact has structural issues; proceed to TEST with caution
|
|
250
|
-
- BENCHMARK_SCORE > 0.70 → **PASS** — artifact meets benchmark standards; ready for TEST
|
|
251
|
-
|
|
252
|
-
Record `BENCHMARK_SCORE` for downstream consumption by `/bto-test` and `/bto`.
|
|
253
|
-
|
|
254
|
-
---
|
|
255
|
-
|
|
256
|
-
## Checkpoint
|
|
257
|
-
|
|
258
|
-
```
|
|
259
|
-
═══════════════════════════════════════════════════════
|
|
260
|
-
CHECKPOINT: BENCHMARK Complete
|
|
261
|
-
<promise>BTO_BENCHMARKED</promise>
|
|
262
|
-
Artifact: [path]
|
|
263
|
-
Type: [detected type]
|
|
264
|
-
|
|
265
|
-
Layer B0 (Golden): X.XX — section coverage, ordering, proportions
|
|
266
|
-
Layer B1 (Test Suite): X.XX — Y/Z tests passed
|
|
267
|
-
Layer B2 (Consistency): X.XX — agreement across 3 probes
|
|
268
|
-
Layer B3 (Performance): X.XX — token efficiency, density, bloat
|
|
269
|
-
|
|
270
|
-
BENCHMARK SCORE: X.XX [PASS / WARN / BLOCK]
|
|
271
|
-
|
|
272
|
-
• "ок" — proceed to TEST
|
|
273
|
-
• "покажи детали [B0/B1/B2/B3]" — expand layer details
|
|
274
|
-
• "пропусти тест" — skip TEST (if PASS)
|
|
275
|
-
═══════════════════════════════════════════════════════
|
|
276
|
-
```
|
|
277
|
-
|
|
278
|
-
Wait for user confirmation.
|
|
279
|
-
|
|
280
|
-
---
|
|
281
|
-
|
|
282
|
-
## Modular Usage
|
|
283
|
-
|
|
284
|
-
This command is also invoked internally by:
|
|
285
|
-
- `/bto` — as a pre-flight step before TEST phase (if benchmark module is loaded)
|
|
286
|
-
- `/bto-test` — optionally as a prerequisite gate (if user requests `benchmark` flag)
|
|
287
|
-
|
|
288
|
-
The `BENCHMARK_SCORE` is available to downstream commands:
|
|
289
|
-
- `/bto-test` may use it to calibrate Layer 1 expectations
|
|
290
|
-
- `/bto-optimize` may use it to set a structural baseline floor
|
|
291
|
-
|
|
292
|
-
## Critical Rules
|
|
293
|
-
|
|
294
|
-
- Layer B0 and B1 are fully deterministic — no LLM calls, zero cost
|
|
295
|
-
- Layer B2 uses haiku only — NEVER use sonnet or opus for consistency probes
|
|
296
|
-
- Agent tool is REQUIRED for Layer B2 — do not run probes sequentially
|
|
297
|
-
- Golden samples from `references/golden-samples.md` are the authoritative structural reference
|
|
298
|
-
- If golden sample does not exist for the detected type → skip B0, set B0 = 0.50 (neutral), and log a warning
|
|
299
|
-
- Report `BENCHMARK_SCORE` explicitly so downstream commands can consume it
|
|
300
|
-
- BLOCK verdict at B0 or B1 halts the benchmark — do NOT proceed to subsequent layers
|
|
301
|
-
- Layer B3 bloat detection uses golden sample proportions — without a golden sample, use absolute thresholds only
|
|
302
|
-
- If "verbose" is in $ARGUMENTS, show expanded per-section diagnostics for all layers without prompting
|