@dzhechkov/keysarium 1.2.0 → 1.4.0

package/templates/docs/skills-bto/01-deployment-guide.md

@@ -98,7 +98,7 @@ echo 'export ANTHROPIC_API_KEY="sk-ant-..."' >> ~/.bashrc
 source ~/.bashrc
 ```
 
-> **Токены и стоимость:** Полный BTO-цикл (BUILD + TEST + OPTIMIZE) потребляет примерно 100–200K токенов. Детальная разбивка по операциям: Baseline Layer 2 ~15K (sonnet×3), генерация 5 вариантов ~25K (opus), оценка Round 1 ~10K (haiku×5), финальный Round 3 ~45K (sonnet×9). Итого около 131K токенов за полный прогон. Отдельный `/bto-test` с Layer 2 потребляет около 15–20K токенов.
+> **Токены и стоимость:** Полный BTO-цикл (BUILD + BENCHMARK + TEST + OPTIMIZE) потребляет примерно 100–200K токенов. BENCHMARK (B0-B3) добавляет ~3.5K токенов (B0/B1/B3 детерминистические, B2 consistency probe — 3× haiku). Детальная разбивка: BUILD ~22K, BENCHMARK ~3.5K, TEST Layer 2 ~19K (sonnet×3), OPTIMIZE 3 rounds ~131K. Итого ~175K токенов за полный прогон. Отдельный `/bto-benchmark` потребляет ~3.5K токенов, `/bto-test` с Layer 2 — около 15–20K токенов.
 
 ### 1.4. Git (опционально, но рекомендуется)
 

package/templates/docs/skills-bto/02-admin-guide.md

@@ -27,16 +27,19 @@
 | Компонент | Путь | Назначение |
 |-----------|------|------------|
 | BTO Skill | `.claude/skills/bto/` | Основной оркестратор: загружается командами |
-| SKILL.md | `.claude/skills/bto/SKILL.md` | Точка входа, документирует все три модуля |
+| SKILL.md | `.claude/skills/bto/SKILL.md` | Точка входа, документирует все четыре модуля |
 | Модуль BUILD | `.claude/skills/bto/modules/build.md` | Протокол генерации артефактов |
+| Модуль BENCHMARK | `.claude/skills/bto/modules/benchmark.md` | Детерминистический бенчмаркинг (B0-B3) |
 | Модуль TEST | `.claude/skills/bto/modules/test.md` | Протокол мульти-агентной оценки |
 | Модуль OPTIMIZE | `.claude/skills/bto/modules/optimize.md` | Протокол эволюционной оптимизации |
 | Рубрики судей | `.claude/skills/bto/references/judge-rubrics.md` | Якорные точки для scoring |
 | Паттерны оценки | `.claude/skills/bto/references/eval-patterns.md` | Паттерны мульти-агентной оценки |
 | Методы оптимизации | `.claude/skills/bto/references/optimization-methods.md` | Методология оптимизации |
 | Чеклист качества | `.claude/skills/bto/references/quality-checklist.md` | Детерминистические проверки Layer 0 |
+| Golden samples | `.claude/skills/bto/references/golden-samples.md` | Эталонные структуры для бенчмаркинга |
 | Команда /bto | `.claude/commands/bto.md` | Полный пайплайн |
 | Команда /bto-build | `.claude/commands/bto-build.md` | BUILD-фаза отдельно |
+| Команда /bto-benchmark | `.claude/commands/bto-benchmark.md` | BENCHMARK-фаза отдельно |
 | Команда /bto-test | `.claude/commands/bto-test.md` | TEST-фаза отдельно |
 | Команда /bto-optimize | `.claude/commands/bto-optimize.md` | OPTIMIZE-фаза отдельно |
 | Правила quality gates | `.claude/rules/bto-quality-gates.md` | Автоматически применяемые правила |

package/templates/docs/skills-bto/03-user-guide.md

@@ -20,13 +20,14 @@ This guide covers everything you need to use the BTO (Build-Test-Optimize) skill
 
 ## 1. Quick Start (5 minutes)
 
-BTO gives you four commands. Each can be used independently or together as a pipeline.
+BTO gives you five commands. Each can be used independently or together as a pipeline.
 
 ```
 /bto-build [description]       — Generate a new skill, command, rule, or agent template
+/bto-benchmark [path]          — Deterministic benchmarking against golden samples
 /bto-test [path]               — Evaluate an existing artifact through layered quality gates
 /bto-optimize [path]           — Improve an artifact through evolutionary optimization
-/bto [path or description]     — Full pipeline: BUILD → TEST → OPTIMIZE
+/bto [path or description]     — Full pipeline: BUILD → BENCHMARK → TEST → OPTIMIZE
 ```
 
 ### Your first BTO run
@@ -577,7 +578,7 @@ BTO does not automatically overwrite your artifact. It presents the optimized ve
 
 ## 5. Full BTO Pipeline with /bto
 
-The `/bto` command runs BUILD, TEST, and OPTIMIZE in sequence for a complete quality lifecycle. Use it when starting a new artifact from scratch and wanting production-quality output with minimal manual intervention.
+The `/bto` command runs BUILD, BENCHMARK, TEST, and OPTIMIZE in sequence for a complete quality lifecycle. Use it when starting a new artifact from scratch and wanting production-quality output with minimal manual intervention.
 
 ### Usage
 

package/templates/docs/skills-bto/04-infrastructure-requirements.md

@@ -251,9 +251,10 @@ Note: if convergence is detected after Round 2, Round 3 evaluation is skipped. T
 | Stage | Est. tokens |
 |-------|------------:|
 | BUILD (QUICK mode, typical skill) | ~22,000 |
+| BENCHMARK (B0-B3, incl. 3× haiku consistency probe) | ~3,500 |
 | TEST (full, no meta-judge) | ~18,800 |
 | OPTIMIZE (3 rounds) | ~131,000 |
-| **Full /bto pipeline** | **~172,000** |
+| **Full /bto pipeline** | **~175,500** |
 
 For a description-triggered build where DEEP mode is invoked: add approximately 10,000 tokens to the BUILD phase.
 

package/templates/docs/skills-bto/05-architecture.md

@@ -19,23 +19,23 @@
 
 ## Обзор системы и философия проектирования
 
-`@dzhechkov/skills-bto` — это **Build-Test-Optimize (BTO)** скилл-пак для Claude Code, реализующий полный цикл создания, оценки и итеративного улучшения Claude Code артефактов. Система работает как standalone-инструмент и как интегрированный компонент пайплайна `@dzhechkov/keysarium`.
+`@dzhechkov/skills-bto` — это **Build-Benchmark-Test-Optimize (BTO)** скилл-пак для Claude Code, реализующий полный цикл создания, бенчмаркинга, оценки и итеративного улучшения Claude Code артефактов. Система работает как standalone-инструмент и как интегрированный компонент пайплайна `@dzhechkov/keysarium`.
 
 ### Фундаментальная идея
 
-Большинство промпт-инжиниринговых задач имеют три отдельных фазы с принципиально разными требованиями:
+Промпт-инжиниринговые задачи имеют четыре отдельных фазы с принципиально разными требованиями:
 
 ```
-BUILD               TEST                OPTIMIZE
-──────              ────                ────────
-Генерация           Оценка              Улучшение
-Требует:            Требует:            Требует:
-• Творчество        • Объективность     • Управляемые мутации
-• Структуру         • Разные точки зр.  • Быструю итерацию
-• Domain знание     • Детерминизм       • Отбор лучших вариантов
+BUILD               BENCHMARK           TEST                OPTIMIZE
+──────              ─────────           ────                ────────
+Генерация           Бенчмаркинг         Оценка              Улучшение
+Требует:            Требует:            Требует:            Требует:
+• Творчество        • Golden samples    • Объективность     • Управляемые мутации
+• Структуру         • Детерм. тесты     • Разные точки зр.  • Быструю итерацию
+• Domain знание     • Consistency probe  • Экспертные судьи  • Отбор лучших вариантов
 ```
 
-BTO изолирует эти три режима в отдельные модули с чёткими контрактами на входе и выходе. Модули можно запускать независимо или последовательно как пайплайн.
+BTO изолирует эти четыре режима в отдельные модули с чёткими контрактами на входе и выходе. Модули можно запускать независимо или последовательно как пайплайн.
 
 ### Ключевые принципы проектирования
 
@@ -717,7 +717,7 @@ Variant 3-B ──→ haiku eval ──→ score-3-B.txt  ┘
 
 ## Диаграммы потоков данных
 
-### Полный BTO пайплайн (BUILD → TEST → OPTIMIZE)
+### Полный BTO пайплайн (BUILD → BENCHMARK → TEST → OPTIMIZE)
 
 ```mermaid
 flowchart TD

package/templates/docs/skills-bto/06-user-flows.md

@@ -947,8 +947,9 @@ flowchart LR
 
 | Команда | Назначение | Минимальный вход | Стоимость |
 |---------|-----------|-----------------|-----------|
-| `/bto [path]` | Полный пайплайн BUILD→TEST→OPTIMIZE | Путь или описание | ~$0.15 |
+| `/bto [path]` | Полный пайплайн BUILD→BENCHMARK→TEST→OPTIMIZE | Путь или описание | ~$0.16 |
 | `/bto-build [description]` | Только генерация | Текстовое описание | ~$0.01 |
+| `/bto-benchmark [path]` | Детерминистический бенчмаркинг | Путь к артефакту | ~$0.001 |
 | `/bto-test [path]` | Только оценка (full) | Путь к артефакту | ~$0.01 |
 | `/bto-test [path] level=layer0` | Только Layer 0 | Путь к артефакту | $0 |
 | `/bto-test [path] level=layer1` | Layer 0 + haiku | Путь к артефакту | ~$0.001 |

package/templates/lib/dream-engine.md

@@ -118,7 +118,7 @@ Location: `.keysarium/insights/trigger-state.json`
 4. Scan all `*.json` files recursively under `.keysarium/memory/` (excluding `config.json`, `reward-summary.json`, `domain-patterns.json`).
 5. Parse each as a RewardRecord (schema defined in `lib/memory-protocol.md`).
 6. Exclude records where `expires_at < current_date`.
-7. Sort by `reward` DESC, then `timestamp` DESC.
+7. Sort by `reward` DESC, then `usage_count` DESC, then `timestamp` DESC. Records with higher usage_count carry more weight in concept graph construction.
 8. Take top 200 records.
 9. If fewer than 5 valid records, exit with status `insufficient_data`.
 

package/templates/lib/memory-protocol.md

@@ -2,6 +2,8 @@
 
 Core protocol for persistent memory in the Keysarium pipeline. Provides `memory_query()` before tasks and `memory_store()` after tasks.
 
+**Protocol version: 1.1** — adds 2-tier index, record lifecycle (HOT/WARM/COLD/PURGE), and brain container manifest.
+
 ## Memory Namespace
 
 All memory files live in `.keysarium/memory/`:
@@ -9,13 +11,18 @@ All memory files live in `.keysarium/memory/`:
 ```
 .keysarium/memory/
 ├── config.json                                    ← Global configuration
+├── index.json                                     ← 2-tier index (v1.1)
 ├── _patterns/
 │   └── domain-patterns.json                       ← Detected domain patterns
 ├── _stats/
-│   └── reward-summary.json                        ← Aggregate statistics
+│   ├── reward-summary.json                        ← Aggregate statistics
+│   └── imported-baselines.json                    ← Imported reward baselines
 ├── {domain}/                                      ← banking | retail | enterprise | healthcare
-│   └── {case-slug}/
-│       └── {phase}_{timestamp}.json               ← Individual reward records
+│   ├── {case-slug}/
+│   │   └── {phase}_{timestamp}.json               ← HOT/WARM reward records
+│   └── _archive/                                  ← COLD compressed records (v1.1)
+│       └── {case-slug}/
+│           └── {phase}_{timestamp}.json            ← Archived records
 ```
 
 ## Configuration
@@ -24,7 +31,7 @@ All memory files live in `.keysarium/memory/`:
 
 ```json
 {
-  "version": "1.0",
+  "version": "1.1",
   "retention_days": 90,
   "max_results_per_query": 10,
   "enabled": true,
@@ -34,10 +41,63 @@ All memory files live in `.keysarium/memory/`:
     "good": 0.7,
     "needs_work": 0.3,
     "failed": 0.0
+  },
+  "tier_hot_days": 30,
+  "tier_warm_days": 90,
+  "tier_cold_days": 180
+}
+```
+
+**Tier fields (v1.1):** If `tier_hot_days` is absent, fall back to v1.0 behavior (flat `retention_days` with no tiering). When present, `retention_days` is ignored in favor of the tier thresholds.
+
+## Index File (v1.1)
+
+The index provides a compact catalog of all records, enabling targeted reads instead of full directory scans.
+
+### index.json Schema
+
+```json
+{
+  "version": "1.0",
+  "last_updated": "2026-03-02T14:00:00Z",
+  "record_count": 47,
+  "by_domain": {
+    "banking": {
+      "count": 23,
+      "slugs": ["case-a", "case-b"],
+      "avg_reward": 0.82
+    }
+  },
+  "by_phase": {
+    "phase-0": { "count": 8, "avg_reward": 0.91 },
+    "phase-2": { "count": 12, "avg_reward": 0.65 }
+  },
+  "by_skill": {
+    "goap-research-ed25519": { "count": 10, "avg_reward": 0.78 }
+  },
+  "records": {
+    "bank_kc_automation_phase-2_2026-03-01T12-30-00": {
+      "domain": "banking",
+      "phase": "phase-2",
+      "slug": "bank_kc_automation",
+      "reward": 0.7,
+      "skill": "goap-research-ed25519",
+      "tier": "hot",
+      "usage_count": 0,
+      "file": "banking/bank_kc_automation/phase-2_2026-03-01T12-30-00.json",
+      "expires_at": "2026-08-28T12:30:00Z"
+    }
   }
 }
 ```
 
+### Index Behavior
+
+- **Created:** On first `memory_store()` call when tier config is present (or `index.json` already exists)
+- **Updated:** On every `memory_store()`, archive, and purge operation
+- **Fallback:** If missing, `memory_query()` uses v1.0 full-scan algorithm
+- **Rebuild:** If corrupted, delete `index.json` — it will be recreated on next `memory_store()`
+
 ## Protocol: memory_query(context)
 
 Call at the **start** of each pipeline phase to load relevant historical patterns.
@@ -56,12 +116,22 @@ Call at the **start** of each pipeline phase to load relevant historical pattern
 ### Algorithm
 
 1. **Check enabled:** Read `.keysarium/memory/config.json`. If `enabled` is false or directory does not exist, return empty list.
-2. **Resolve path:** Construct `.keysarium/memory/{domain}/` (scan all slugs, not just current).
-3. **Scan records:** Read all `{phase}_*.json` files matching the requested phase across all slugs in the domain.
+2. **Read index (v1.1):** If `.keysarium/memory/index.json` exists, read it and filter `records` entries by `domain` + `phase`. Go to step 2a. If index does not exist, fall back to step 3 (v1.0 full scan).
+   - 2a. **Index hit:** If filtered record count ≤ `max_results_per_query`, read only those files by `file` path (targeted read). Go to step 4.
+   - 2b. **Index overflow:** If filtered count > limit, rank by `reward` from index entries, select top-N `file` paths, read those files. Go to step 4.
+3. **Full scan (fallback):** Construct `.keysarium/memory/{domain}/` (scan all slugs). Read all `{phase}_*.json` files matching the requested phase.
 4. **Filter expired:** Exclude records where `expires_at < current_date`.
-5. **Sort:** By `reward` DESC, then `timestamp` DESC.
-6. **Limit:** Return top `max_results_per_query` records (default 10).
-7. **Enrich:** Also load `.keysarium/memory/_patterns/domain-patterns.json` for matching domain patterns.
+5. **Archive old records (v1.1):** If tier config is present, check record ages. For records older than `tier_warm_days`:
+   - Read full record
+   - Generate `summary` from `reward_reason` + `outcome.checkpoint_response` (first 200 chars)
+   - Write compressed record to `.keysarium/memory/{domain}/_archive/{slug}/{filename}`
+   - Delete original file from active directory
+   - Update `index.json`: set `tier` to `"cold"`, update `file` path
+   - Log: "Archived {count} records to cold storage in {domain}"
+6. **Purge ancient records (v1.1):** Delete records from `_archive/` older than `tier_cold_days`. In v1.0 mode, delete records where `expires_at < current_date`. Update `index.json`. Log: "Purged {count} expired records from {domain}"
+7. **Sort:** By `reward` DESC, then `usage_count` DESC, then `timestamp` DESC. HOT and WARM records first; COLD records included only if result count < `max_results_per_query`. Records with higher usage_count are more battle-tested and should be preferred when rewards are equal.
+8. **Limit:** Return top `max_results_per_query` records (default 10).
+9. **Enrich:** Also load `.keysarium/memory/_patterns/domain-patterns.json` for matching domain patterns.
 
 ### Output
 
@@ -98,6 +168,7 @@ After calling `memory_query()`, the agent should:
 2. If `records` is non-empty, review the top 3 records for relevant approaches
 3. If `patterns` is non-empty, apply actionable advice to current phase execution
 4. If both are empty (first run), proceed normally
+5. **Increment usage_count:** For each record actually applied (top 3), increment `usage_count` in both the record file and the index entry. Records with higher usage_count are more proven — prioritize them in future queries.
 
 ## Protocol: memory_store(result, reward)
 
@@ -136,17 +207,25 @@ Call at each **checkpoint** after the user responds, to persist the phase outcom
 
 1. **Ensure directory exists:** Create `.keysarium/memory/{domain}/{slug}/` if it does not exist.
 2. **Build record:** Construct full RewardRecord JSON from input.
-3. **Compute expires_at:** `current_date + retention_days` (from config, default 90 days).
+3. **Compute expires_at:** If tier config is present, use `tier_cold_days` (default 180 days). Otherwise, use `retention_days` (default 90 days). Formula: `current_date + threshold`.
 4. **Generate filename:** `{phase}_{ISO-timestamp}.json` (e.g., `phase-2_2026-03-01T12-30-00.json`).
 5. **Write file:** Write JSON to `.keysarium/memory/{domain}/{slug}/{filename}`.
-6. **Log:** "Stored reward {reward} ({reward_label}) for {phase} of {slug}"
+6. **Update index (v1.1):** Read `.keysarium/memory/index.json` (or create if absent and tier config is present):
+   - Add record entry to `records` map: key = record id, value = `{domain, phase, slug, reward, skill, tier: "hot", usage_count: 0, file, expires_at}`
+   - Increment `record_count`
+   - Update `by_domain` aggregate: increment count, add slug if new, recalculate `avg_reward`
+   - Update `by_phase` aggregate: increment count, recalculate `avg_reward`
+   - Update `by_skill` aggregate: increment count, recalculate `avg_reward`
+   - Set `last_updated` to current timestamp
+   - Write updated `index.json`
+7. **Log:** "Stored reward {reward} ({reward_label}) for {phase} of {slug}"
 
 ### RewardRecord JSON Schema
 
 ```json
 {
   "id": "bank_kc_automation_phase-2_2026-03-01T12-30-00",
-  "version": "1.0",
+  "version": "1.1",
   "timestamp": "2026-03-01T12:30:00Z",
   "case_slug": "bank_kc_automation",
   "domain": "banking",
@@ -171,10 +250,47 @@ Call at each **checkpoint** after the user responds, to persist the phase outcom
     "promise_emitted": "RESEARCH_PARANOID_PASSED"
   },
   "promise_tag": "RESEARCH_PARANOID_PASSED",
-  "expires_at": "2026-05-30T12:30:00Z"
+  "usage_count": 0,
+  "expires_at": "2026-08-28T12:30:00Z"
 }
 ```
 
+## Record Lifecycle (v1.1)
+
+Records transition through tiers based on age:
+
+| Tier | Age Range | Location | Fields |
+|------|-----------|----------|--------|
+| **HOT** | 0 – 30 days | `{domain}/{slug}/` | All fields |
+| **WARM** | 30 – 90 days | `{domain}/{slug}/` | All fields (deprioritized in search) |
+| **COLD** | 90 – 180 days | `{domain}/_archive/{slug}/` | Compressed (see below) |
+| **PURGE** | > 180 days | Deleted | — |
+
+Tier thresholds are configurable via `config.json` fields: `tier_hot_days`, `tier_warm_days`, `tier_cold_days`.
+
+### COLD Record Schema (compressed)
+
+```json
+{
+  "id": "bank_kc_automation_phase-2_2026-01-01T12-00-00",
+  "phase": "phase-2",
+  "domain": "banking",
+  "slug": "bank_kc_automation",
+  "reward": 0.7,
+  "reward_label": "good",
+  "skill_used": "goap-research-ed25519",
+  "promise_tag": "RESEARCH_PARANOID_PASSED",
+  "usage_count": 3,
+  "summary": "User requested minor edits to competitor analysis section. ok, but expand the FinTech competitor section",
+  "archived_at": "2026-03-02T14:00:00Z",
+  "original_timestamp": "2026-01-01T12:00:00Z"
+}
+```
+
+### Backward Compatibility
+
+If `tier_hot_days` is NOT present in `config.json`, the lifecycle system is disabled. Records use the v1.0 flat `retention_days` for expiration. No archiving occurs.
+
 ## Reward Assignment Rules
 
 The reward score is determined by the user's response at the checkpoint:
@@ -193,15 +309,6 @@ The reward score is determined by the user's response at the checkpoint:
 - **Positive feedback with minor note** ("great, just fix the typo in section 3") = 1.0
 - **No explicit response** (session ends) = do not store (no record created)
 
-## Purge Protocol
-
-To prevent unbounded growth, expired records should be cleaned up:
-
-1. **Trigger:** At each `memory_query()` call, check for expired records
-2. **Scan:** Within the queried domain directory, find records where `expires_at < current_date`
-3. **Delete:** Remove expired JSON files
-4. **Log:** "Purged {count} expired records from {domain}"
-
 ## Error Handling
 
 | Error | Behavior |
@@ -211,6 +318,8 @@ To prevent unbounded growth, expired records should be cleaned up:
 | Malformed JSON file | Skip that record, log warning, continue |
 | Write fails (permissions, disk) | Log error, continue pipeline without storing |
 | Domain not in `known_domains` | Use "unknown" as domain directory |
+| `index.json` corrupted | Delete and rebuild on next `memory_store()` call |
+| Archive directory write fails | Log warning, keep record in active directory |
 
 ## Integration with Brain Export
 

package/templates/lib/reward-tracker.md

@@ -86,12 +86,14 @@ Group records by `skill_used` and cross-reference with domain:
   "goap-research-ed25519": {
     "overall_avg": 0.74,
     "total_runs": 15,
+    "total_usage_count": 42,
     "by_domain": {
-      "banking": { "avg": 0.65, "runs": 8 },
-      "retail": { "avg": 0.85, "runs": 7 }
+      "banking": { "avg": 0.65, "runs": 8, "usage_count": 18 },
+      "retail": { "avg": 0.85, "runs": 7, "usage_count": 24 }
     },
     "best_domain": "retail",
-    "worst_domain": "banking"
+    "worst_domain": "banking",
+    "most_reused_domain": "retail"
   }
 }
 ```
@@ -110,6 +112,7 @@ Analyze accumulated data to detect actionable patterns. Each pattern has a confi
 | Improving Trend | Trend = "improving" for phase in domain | "{phase} quality is improving in {domain} domain" |
 | Degrading Trend | Trend = "degrading" for phase in domain | "{phase} quality is degrading in {domain} -- investigate" |
 | Time Overhead | Phase avg iterations > 2.0 in domain | "{domain} cases require more iterations in {phase}" |
+| High Reuse | Record usage_count >= 5 | "Pattern from {slug}/{phase} is highly reused ({count} times) — consider promoting to a rule" |
 
 **Confidence Calculation:**
 ```