@dzhechkov/skills-bto 1.2.0 → 1.3.0

package/README.md

@@ -1,8 +1,8 @@
 # @dzhechkov/skills-bto
 
-**Build-Test-Optimize skill pack for Claude Code**
+**Build-Benchmark-Test-Optimize skill pack for Claude Code**
 
-Multi-agent evaluation and iterative optimization pipeline for Claude Code skills, commands, and prompts. Part of the [Keysarium](https://www.npmjs.com/package/@dzhechkov/keysarium) ecosystem.
+Multi-agent evaluation and iterative optimization pipeline for Claude Code skills, commands, and prompts. Includes deterministic benchmarking with golden sample comparison, test suites, consistency probes, and performance metrics. Part of the [Keysarium](https://www.npmjs.com/package/@dzhechkov/keysarium) ecosystem.
 
 ---
 
@@ -28,13 +28,13 @@ After installation, open Claude Code in your project directory and start using B
 
 | Component | Count | Description |
 |-----------|-------|-------------|
-| **Skill** | 1 | `bto` — core Build-Test-Optimize skill with modules (build, test, optimize) |
-| **Commands** | 4 | `/bto`, `/bto-build`, `/bto-test`, `/bto-optimize` |
-| **Rules** | 1 | `bto-quality-gates` — quality gate enforcement |
+| **Skill** | 1 | `bto` — core Build-Benchmark-Test-Optimize skill with 4 modules |
+| **Commands** | 5 | `/bto`, `/bto-build`, `/bto-benchmark`, `/bto-test`, `/bto-optimize` |
+| **Rules** | 1 | `bto-quality-gates` — quality gate enforcement (incl. benchmark gates) |
 | **Shards** | 1 | `bto-evaluation` — context shard for BTO evaluation pipeline |
 | **Agent Templates** | 2 | `bto-judge-panel`, `bto-optimizer-worker` |
-| **References** | 4 | Eval patterns, judge rubrics, optimization methods, quality checklist |
-| **Examples** | 1 | Sample evaluation report |
+| **References** | 5 | Eval patterns, judge rubrics, optimization methods, quality checklist, golden samples |
+| **Examples** | 2 | Sample evaluation report, sample benchmark report |
 
 Everything is installed into your project's `.claude/` directory and works natively with Claude Code.
 
@@ -58,22 +58,26 @@ npx @dzhechkov/skills-bto doctor             # Health check
 ## BTO Pipeline
 
 ```
-BUILD ──→ TEST ──→ OPTIMIZE
-  │         │         │
-  │         │         └── Evolutionary mutation + re-evaluation (3 rounds)
-  │         └── Multi-layer evaluation: Layer 0 → Layer 1 → Layer 2
+BUILD ──→ BENCHMARK ──→ TEST ──→ OPTIMIZE
+  │           │           │         │
+  │           │           │         └── Evolutionary mutation + re-evaluation (3 rounds)
+  │           │           └── Multi-layer evaluation: Layer 0 → Layer 1 → Layer 2
+  │           └── Deterministic benchmarking: golden samples, test suite, consistency, metrics
   └── Generate skill/command from description
 ```
 
 ### Usage in Claude Code
 
 ```bash
-# Full BTO cycle: build → test → optimize
+# Full BTO cycle: build → benchmark → test → optimize
 /bto Create a skill for code review automation
 
 # Build only — generate a new skill or command
 /bto-build Create a skill that analyzes git commit patterns
 
+# Benchmark only — deterministic benchmarking against golden samples
+/bto-benchmark .claude/skills/my-skill/SKILL.md
+
 # Test only — evaluate an existing artifact
 /bto-test .claude/skills/my-skill/SKILL.md
 
@@ -85,7 +89,20 @@ BUILD ──→ TEST ──→ OPTIMIZE
 
 ## Evaluation Architecture
 
-### Layer Model
+### Benchmark Layers (deterministic, pre-TEST)
+
+| Layer | Cost | Purpose |
+|-------|------|---------|
+| **B0** | Zero (deterministic) | Golden sample comparison — section coverage, ordering, proportions |
+| **B1** | Zero (deterministic) | Deterministic test suite — 5 tests per artifact type, PASS/FAIL |
+| **B2** | Minimal (3× haiku) | Consistency probe — 3 parallel agents, agreement measurement |
+| **B3** | Zero (deterministic) | Performance metrics — token efficiency, bloat detection, redundancy |
+
+**Scoring:** `BENCHMARK = B0×0.30 + B1×0.35 + B2×0.15 + B3×0.20`
+
+**Gate:** < 0.50 BLOCK | 0.50–0.70 WARN | > 0.70 PASS → proceed to TEST
+
+### TEST Layer Model
 
 | Layer | Agents | Model | Purpose |
 |-------|--------|-------|---------|
@@ -103,6 +120,8 @@ BUILD ──→ TEST ──→ OPTIMIZE
 
 ### Quality Gates
 
+- **BENCHMARK** must pass (score ≥ 0.50) before TEST begins
+- BENCHMARK score < 0.50 → BLOCK (artifact needs rework)
 - Layer 0 must pass before Layer 1
 - Layer 1 must pass before Layer 2
 - Optimization accepted only if `new_score - prev_score > 0.5`

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@dzhechkov/skills-bto",
-  "version": "1.2.0",
-  "description": "Build-Test-Optimize skill pack for Claude Code — structured BTO pipeline with quality gates, witness chain verification, judge attestation, and optimization workflows",
+  "version": "1.3.0",
+  "description": "Build-Benchmark-Test-Optimize skill pack for Claude Code — deterministic benchmarking, quality gates, witness chain, judge attestation, and optimization",
   "main": "src/cli.js",
   "bin": {
     "skills-bto": "./bin/cli.js"
@@ -20,6 +20,7 @@
     "claude-code",
     "ai",
     "bto",
+    "benchmark",
     "build-test-optimize",
     "skills",
     "quality-gates",

package/src/cli.js

@@ -8,10 +8,10 @@ const { bold, cyan, dim, green, yellow, red, info, error } = require('./utils');
 // ---------------------------------------------------------------------------
 function showBanner() {
   console.log('');
-  console.log(cyan('\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557'));
-  console.log(cyan('\u2551') + bold('   DZ-SKILLS-BTO \u2014 Build\u00B7Test\u00B7Optimize for Claude Code  ') + cyan('\u2551'));
-  console.log(cyan('\u2551') + '   Structured BTO workflow with quality gates          ' + cyan('\u2551'));
-  console.log(cyan('\u255A\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255D'));
+  console.log(cyan('\u2554\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2557'));
+  console.log(cyan('\u2551') + bold('   DZ-SKILLS-BTO \u2014 Build\u00B7Benchmark\u00B7Test\u00B7Optimize          ') + cyan('\u2551'));
+  console.log(cyan('\u2551') + '   Deterministic benchmarking + multi-agent evaluation   ' + cyan('\u2551'));
+  console.log(cyan('\u255A\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255D'));
   console.log('');
 }
 

package/src/utils.js

@@ -299,12 +299,12 @@ function getTemplatesDir() {
 const COMPONENTS = {
   skill: {
     src: '.claude/skills/bto',
-    label: 'BTO Skill Pack (3 modules)',
+    label: 'BTO Skill Pack (4 modules)',
     group: 'core',
   },
   commands: {
     src: '.claude/commands',
-    label: 'BTO Commands (4 commands)',
+    label: 'BTO Commands (5 commands)',
     group: 'core',
     filter: 'bto',
   },

package/templates/.claude/commands/bto-benchmark.md

@@ -0,0 +1,302 @@
+# /bto-benchmark — Deterministic Benchmarking of a Skill or Command
+
+## Usage
+```
+/bto-benchmark [path to skill directory or file]
+```
+
+## Parameters
+- $ARGUMENTS — Path to a skill directory (`.claude/skills/<name>/`), a command file (`.claude/commands/<name>.md`), a rule file (`.claude/rules/<name>.md`), or any structured artifact. Optionally include "verbose" to show expanded per-section diagnostics for all layers.
+
+## Protocol
+
+### Step 1: Load Skill and Module
+
+Read `.claude/skills/bto/SKILL.md`
+Read `.claude/skills/bto/modules/benchmark.md`
+Read `.claude/skills/bto/references/golden-samples.md`
+
+### Step 2: Validate Input
+
+If $ARGUMENTS is empty:
+- Ask: "Provide a path to the skill directory or artifact file you want to benchmark."
+- Stop and wait.
+
+Resolve the artifact path from $ARGUMENTS:
+- Strip any trailing slash
+- If the path points to a directory → look for `SKILL.md` inside it as the primary file, but include all files in the directory for context
+- If the path points to a file → use that file directly
+- If the path does not exist → report "Path not found: [path]" and stop
+
+### Step 3: Detect Artifact Type
+
+Auto-detect from path pattern:
+
+| Path Pattern | Detected Type |
+|-------------|--------------|
+| `.claude/skills/*/SKILL.md` or `.claude/skills/*/` | skill |
+| `.claude/commands/*.md` | command |
+| `.claude/rules/*.md` | rule |
+| `.claude/agents/*.md` | agent |
+| `researches/**/*.md` | research artifact |
+
+If type cannot be determined from path → infer from file content structure.
+
+### Step 4: Layer B0 — Golden Sample Comparison (Deterministic)
+
+**Purpose:** Compare artifact structure against the canonical golden sample for its type. Zero LLM cost.
+
+Load golden samples for the detected type from `references/golden-samples.md`.
+
+**Procedure:**
+
+1. Parse the artifact into a section tree (heading hierarchy, section lengths, ordering)
+2. Parse the matching golden sample into the same section tree format
+3. Compare across three structural axes:
+
+**Axis 1 — Section Coverage:**
+For each required section in the golden sample, check if the artifact contains a matching section (by heading text or semantic equivalent).
+- `section_coverage = matched_sections / total_golden_sections`
+
+**Axis 2 — Ordering Score:**
+Compare the ordering of matched sections against the golden sample ordering.
+- Use normalized Kendall tau distance: `ordering_score = 1 - (inversions / max_inversions)`
+- If fewer than 3 matched sections → ordering_score = 1.0 (too few to meaningfully compare)
+
+**Axis 3 — Proportion Score:**
+For each matched section, compute the ratio of artifact section length to golden sample section length.
+- `proportion_score = 1 - mean(|log(artifact_len / golden_len)|)` clamped to [0, 1]
+- Measures whether sections are proportionally sized (not too bloated, not too thin)
+
+**Display per-section MATCH/MISS table:**
+
+```
+Golden Section              Status    Artifact Section             Size Ratio
+─────────────────────────── ──────── ──────────────────────────── ──────────
+## Overview                 MATCH    ## Overview                  1.2x
+## Protocol                 MATCH    ## Protocol                  0.8x
+## Anti-Patterns            MATCH    ## Anti-Patterns             1.0x
+## Dependencies             MISS     —                            —
+## Quick Start              MATCH    ## Quick Start               1.5x
+```
+
+**Golden Similarity Score:**
+```
+B0 = section_coverage * 0.50 + ordering_score * 0.25 + proportion_score * 0.25
+```
+
+**Gate:** If B0 < 0.30 → BLOCK immediately. The artifact's structure is too far from the expected form to benchmark meaningfully.
+
+### Step 5: Layer B1 — Deterministic Test Suite
+
+**Purpose:** Run all applicable deterministic tests for the detected artifact type. Zero LLM cost.
+
+Execute all applicable checks from `benchmark.md` for the detected artifact type. These overlap with but extend the Layer 0 checks from `/bto-test`:
+
+**Universal Checks (U1-U12):**
+- U-01: File exists and is non-empty
+- U-02: UTF-8 encoding valid
+- U-03: Starts with level-1 heading
+- U-04: No placeholder text (`[TODO]`, `[TBD]`, `<INSERT>`, `[PLACEHOLDER]`)
+- U-05: No empty sections (heading with no content before next heading)
+- U-06: Consistent heading hierarchy (no h1→h3 jumps)
+- U-07: No broken internal cross-references
+- U-08: File size within bounds (200B – 100KB per file)
+- U-09: No trailing whitespace on lines
+- U-10: Standard Markdown only (no HTML unless in code blocks)
+- U-11: All code blocks properly closed
+- U-12: No duplicate top-level sections
+
+**Type-Specific Checks:**
+- Skill: SK-01 through SK-16 (SKILL.md exists, required sections, modules/, references/, examples/, etc.)
+- Command: CM-01 through CM-08 (Usage section, Parameters, Protocol steps, Checkpoint, Critical Rules, etc.)
+- Rule: RL-01 through RL-06 (Table format, detection signals, fix actions, no contradictions, etc.)
+- Agent: AG-01 through AG-04 (Model specified, isolation rules, naming convention, output format)
+
+**Display per-test PASS/FAIL table:**
+
+```
+ID      Check                              Result   Note
+─────── ──────────────────────────────────  ──────── ────────────────────────────
+U-01    File exists and is non-empty       PASS     8.4 KB
+U-02    UTF-8 encoding                     PASS
+...
+SK-01   SKILL.md exists at skill root      PASS
+SK-02   Has ## Overview section            PASS
+SK-03   Has ## Anti-Patterns section       FAIL     Section missing
+...
+```
+
+**Test Pass Rate:**
+```
+B1 = passed_tests / total_applicable_tests
+```
+
+**Gate:** If B1 < 0.60 → BLOCK. Too many structural failures to proceed to consistency probing.
+
+### Step 6: Layer B2 — Consistency Probe
+
+**Purpose:** Measure evaluation stability by running identical prompts across multiple agents. Detects artifacts that produce inconsistent interpretations.
+
+**Model:** haiku (cost-optimized)
+
+**Spawn 3 parallel haiku agents with the identical evaluation prompt:**
+
+Each agent receives the exact same prompt:
+```
+You are evaluating a Claude Code artifact for structural quality.
+Rate these 4 dimensions (0.0 to 1.0 each, two decimal places):
+1. STRUCTURE — Does the artifact follow a clear, logical organization?
+2. CLARITY — Are instructions unambiguous and actionable?
+3. COVERAGE — Does it address all aspects implied by its title/scope?
+4. CONSISTENCY — Is the internal terminology and style uniform?
+Output as JSON: {"structure": X.XX, "clarity": X.XX, "coverage": X.XX, "consistency": X.XX}
+```
+
+**Isolation:** Each agent evaluates independently. No cross-communication.
+
+**After all 3 return, compute structural agreement:**
+
+For each dimension:
+```
+agreement[dim] = 1 - (max(scores[dim]) - min(scores[dim]))
+```
+
+**Consistency Score:**
+```
+B2 = mean(agreement across all 4 dimensions)
+```
+
+**Interpretation:**
+- B2 > 0.85 → HIGH consistency — the artifact is unambiguously structured
+- B2 0.60-0.85 → MODERATE consistency — some sections are open to interpretation
+- B2 < 0.60 → LOW consistency — the artifact is structurally ambiguous
+
+**Display per-probe comparison:**
+
+```
+Dimension      Probe 1   Probe 2   Probe 3   Range    Agreement
+──────────── ────────── ────────── ────────── ──────── ──────────
+STRUCTURE      0.80       0.85       0.82      0.05     0.95
+CLARITY        0.70       0.75       0.65      0.10     0.90
+COVERAGE       0.60       0.70       0.55      0.15     0.85
+CONSISTENCY    0.75       0.78       0.73      0.05     0.95
+                                                Mean:    0.91
+```
+
+No gate on B2 — consistency is informational and feeds into the aggregate score.
+
+### Step 7: Layer B3 — Performance Metrics
+
+**Purpose:** Quantify token efficiency and information density. Zero LLM cost.
+
+**Metric 1 — Token Efficiency:**
+```
+token_efficiency = meaningful_content_tokens / total_tokens
+```
+Where `meaningful_content_tokens` excludes: blank lines, decorative separators, repeated boilerplate headers, and excessive whitespace. Target: > 0.70.
+
+**Metric 2 — Information Density:**
+```
+information_density = unique_concepts / total_sections
+```
+Where `unique_concepts` = count of distinct topics, terms, or instructions introduced. Measures whether each section contributes new information vs. repeating prior content. Target: > 2.0 concepts per section.
+
+**Metric 3 — Bloat Detection:**
+Identify sections where:
+- Section length > 3x the golden sample equivalent → BLOATED
+- Section length < 0.2x the golden sample equivalent → THIN
+- Section repeats > 30% of content from another section → REDUNDANT
+
+```
+bloat_score = 1 - (bloated_sections + thin_sections + redundant_sections) / total_sections
+```
+Clamped to [0, 1].
+
+**Performance Score:**
+```
+B3 = token_efficiency * 0.35 + normalize(information_density, 0, 4) * 0.35 + bloat_score * 0.30
+```
+Where `normalize(x, min, max)` clamps and scales x to [0, 1].
+
+**Display per-section metrics:**
+
+```
+Section                    Tokens   Concepts   Golden Ratio   Flag
+─────────────────────────  ──────── ────────── ──────────────  ──────────
+## Overview                 320      4          1.2x           OK
+## Protocol                 1840     12         0.8x           OK
+## Anti-Patterns            580      6          2.8x           BLOATED
+## Quick Start              95       2          0.3x           THIN
+## Dependencies             210      3          1.0x           OK
+```
+
+### Step 8: Aggregate & Gate
+
+**BENCHMARK_SCORE formula:**
+```
+BENCHMARK_SCORE = B0 * 0.30 + B1 * 0.35 + B2 * 0.15 + B3 * 0.20
+```
+
+**Weight rationale:**
+- B0 (Golden Comparison) at 0.30 — structural conformance is the primary benchmark signal
+- B1 (Test Suite) at 0.35 — deterministic correctness is the most reliable signal
+- B2 (Consistency) at 0.15 — stability matters but is a secondary signal
+- B3 (Performance) at 0.20 — efficiency matters for production artifacts
+
+**Gate:**
+- BENCHMARK_SCORE < 0.50 → **BLOCK** — artifact requires significant rework before evaluation
+- BENCHMARK_SCORE 0.50-0.70 → **WARN** — artifact has structural issues; proceed to TEST with caution
+- BENCHMARK_SCORE > 0.70 → **PASS** — artifact meets benchmark standards; ready for TEST
+
+Record `BENCHMARK_SCORE` for downstream consumption by `/bto-test` and `/bto`.
+
+---
+
+## Checkpoint
+
+```
+═══════════════════════════════════════════════════════
+CHECKPOINT: BENCHMARK Complete
+<promise>BTO_BENCHMARKED</promise>
+Artifact: [path]
+Type: [detected type]
+
+Layer B0 (Golden):      X.XX — section coverage, ordering, proportions
+Layer B1 (Test Suite):  X.XX — Y/Z tests passed
+Layer B2 (Consistency): X.XX — agreement across 3 probes
+Layer B3 (Performance): X.XX — token efficiency, density, bloat
+
+BENCHMARK SCORE: X.XX [PASS / WARN / BLOCK]
+
+• "ок" — proceed to TEST
+• "покажи детали [B0/B1/B2/B3]" — expand layer details
+• "пропусти тест" — skip TEST (if PASS)
+═══════════════════════════════════════════════════════
+```
+
+Wait for user confirmation.
+
+---
+
+## Modular Usage
+
+This command is also invoked internally by:
+- `/bto` — as a pre-flight step before TEST phase (if benchmark module is loaded)
+- `/bto-test` — optionally as a prerequisite gate (if user requests `benchmark` flag)
+
+The `BENCHMARK_SCORE` is available to downstream commands:
+- `/bto-test` may use it to calibrate Layer 1 expectations
+- `/bto-optimize` may use it to set a structural baseline floor
+
+## Critical Rules
+
+- Layer B0 and B1 are fully deterministic — no LLM calls, zero cost
+- Layer B2 uses haiku only — NEVER use sonnet or opus for consistency probes
+- Agent tool is REQUIRED for Layer B2 — do not run probes sequentially
+- Golden samples from `references/golden-samples.md` are the authoritative structural reference
+- If golden sample does not exist for the detected type → skip B0, set B0 = 0.50 (neutral), and log a warning
+- Report `BENCHMARK_SCORE` explicitly so downstream commands can consume it
+- BLOCK verdict at B0 or B1 halts the benchmark — do NOT proceed to subsequent layers
+- Layer B3 bloat detection uses golden sample proportions — without a golden sample, use absolute thresholds only
+- If "verbose" is in $ARGUMENTS, show expanded per-section diagnostics for all layers without prompting

package/templates/.claude/commands/bto.md

@@ -19,12 +19,12 @@ Read `.claude/skills/bto/SKILL.md`
 Inspect $ARGUMENTS to determine mode:
 
 **If $ARGUMENTS is a path that exists on disk:**
-- Mode: TEST → OPTIMIZE
+- Mode: BENCHMARK → TEST → OPTIMIZE
 - Skip BUILD
-- Proceed to Step 4 (TEST)
+- Proceed to Step 4 (BENCHMARK)
 
 **If $ARGUMENTS is a natural language description (not a path):**
-- Mode: BUILD → TEST → OPTIMIZE
+- Mode: BUILD → BENCHMARK → TEST → OPTIMIZE
 - Proceed to Step 3 (BUILD)
 
 **If $ARGUMENTS is empty:**
@@ -62,7 +62,51 @@ Wait for user confirmation before proceeding.
 
 ---
 
-### Step 4: TEST
+### Step 4: BENCHMARK
+
+Read `.claude/skills/bto/modules/benchmark.md`
+Read `.claude/skills/bto/references/golden-samples.md`
+
+**Resolve artifact path:**
+- If BUILD was run → use `BUILD_OUTPUT_PATH`
+- If path was provided in $ARGUMENTS → use that path
+
+Execute BENCHMARK protocol:
+1. **B0 — Golden Sample Comparison:** Diff artifact structure against golden samples
+2. **B1 — Deterministic Test Suite:** Run rule-based checks (cross-references, terminology, specificity)
+3. **B2 — Consistency Probe:** Spawn 3 parallel haiku agents to interpret the artifact; measure agreement
+4. **B3 — Performance Metrics:** Compute token efficiency, section balance, reference/example coverage
+
+Compute: `BENCHMARK_SCORE = B0 * 0.30 + B1 * 0.35 + B2 * 0.15 + B3 * 0.20`
+
+**Gate decision:**
+- If `BENCHMARK_SCORE < 0.50` → BLOCK. Report per-layer failures. Do NOT proceed to TEST. Stop here.
+- If `BENCHMARK_SCORE 0.50-0.70` → WARN. Proceed to TEST with advisory context for judges.
+- If `BENCHMARK_SCORE > 0.70` → PASS. Proceed to TEST.
+
+Record `BENCHMARK_SCORE` for downstream use (TEST judges receive it as quantitative context).
+
+**Checkpoint BENCHMARK:**
+```
+═══════════════════════════════════════════════════════
+CHECKPOINT 2: BENCHMARK Complete
+Artifact: [path]
+B0 Golden Sample:    X.XX
+B1 Test Suite:       X.XX
+B2 Consistency:      X.XX
+B3 Performance:      X.XX
+BENCHMARK_SCORE:     X.XX — [PASS / WARN / BLOCK]
+
+• "ок" — run TEST
+• "покажи детали [layer]" — expand layer results
+• "исправь" — fix issues and re-benchmark
+═══════════════════════════════════════════════════════
+```
+Wait for user confirmation before proceeding.
+
+---
+
+### Step 5: TEST
 
 Read `.claude/skills/bto/modules/test.md`
 Read `.claude/skills/bto/references/judge-rubrics.md`
@@ -71,6 +115,8 @@ Read `.claude/skills/bto/references/judge-rubrics.md`
 - If BUILD was run → use `BUILD_OUTPUT_PATH`
 - If path was provided in $ARGUMENTS → use that path
 
+**BENCHMARK_SCORE is available for judge context.** If BENCHMARK produced a WARN, include the benchmark details in the judge prompt so they can factor quantitative weaknesses into their evaluation.
+
 Execute TEST module in layers:
 
 **Layer 0 — Deterministic pre-checks (always run first):**
@@ -98,7 +144,7 @@ Record `TEST_SCORE` (overall weighted average) for use in OPTIMIZE step.
 **Checkpoint TEST:**
 ```
 ═══════════════════════════════════════════════════════
-CHECKPOINT 2: TEST Complete
+CHECKPOINT 3: TEST Complete
 Artifact: [path]
 Layer 0: X/Y checks passed
 Layer 1: X.X/10 — [PASS / NEEDS WORK / FAIL]
@@ -114,7 +160,7 @@ Wait for user confirmation before proceeding.
 
 ---
 
-### Step 5: OPTIMIZE
+### Step 6: OPTIMIZE
 
 Read `.claude/skills/bto/modules/optimize.md`
 
@@ -130,7 +176,7 @@ Execute OPTIMIZE module:
 **Checkpoint OPTIMIZE:**
 ```
 ═══════════════════════════════════════════════════════
-CHECKPOINT 3: OPTIMIZE Complete
+CHECKPOINT 4: OPTIMIZE Complete
 Artifact: [path]
 Rounds run: 3
 Total evaluations: 15
@@ -158,14 +204,17 @@ Recommendation: [Apply / Review / Original preferred]
 
 Each module can be run independently:
 - `/bto-build [description]` — BUILD only
+- `/bto-benchmark [path]` — BENCHMARK only
 - `/bto-test [path]` — TEST only
 - `/bto-optimize [path]` — OPTIMIZE only
 
 ## Critical Rules
 
 - Always run Layer 0 before any LLM evaluation — it is free and fast
+- Always run BENCHMARK before TEST — it is cheap and provides quantitative context
+- BENCHMARK gate: if BENCHMARK_SCORE < 0.50, do NOT proceed to TEST
 - Never skip checkpoints — wait for user "ок" between modules
 - Only run OPTIMIZE if TEST score < 8.0
 - Agent tool is REQUIRED for Layer 2 parallel judge panel
 - BUILD mode: QUICK by default, DEEP only if user explicitly requests it
-- Record artifact path from BUILD and pass it through TEST → OPTIMIZE
+- Record artifact path from BUILD and pass it through BENCHMARK → TEST → OPTIMIZE

package/templates/.claude/rules/bto-quality-gates.md

@@ -10,7 +10,8 @@ These rules apply to ANY evaluation system, not just Keysarium.
 | Layer | Role | Model | Trigger |
 |-------|------|-------|---------|
 | Layer 0 | Structural pre-check (format, completeness) | haiku | Always |
-| Layer 1 | Shallow semantic check (relevance, coherence) | haiku | After Layer 0 passes |
+| Benchmark | Golden sample + test suite + consistency + perf | haiku (B2 only) | After Layer 0 passes |
+| Layer 1 | Shallow semantic check (relevance, coherence) | haiku | After Benchmark passes |
 | Layer 2 | Deep evaluation (quality, domain fit) | sonnet (judge panel) | After Layer 1 passes |
 | Layer 3 | Creative synthesis / optimization crossover | opus | On top-N candidates only |
 
@@ -29,6 +30,8 @@ Never promote an artifact to a higher layer if the lower layer gate fails.
 | Crossover / creative synthesis | opus | Novel combination of best candidates |
 | Mutation workers (standard) | sonnet | Requires reasoning about improvement direction |
 | Variant fast-eval (ranking pass) | haiku | Volume scoring before full panel |
+| Benchmark B0-B1,B3 | -- (deterministic) | Zero LLM cost |
+| Benchmark B2 consistency probe | haiku x 3 | Parallel consistency check |
 
 Escalate to a higher-cost model only when the lower-cost model has failed or is insufficient.
 
@@ -44,6 +47,46 @@ Every generated skill or artifact MUST pass ALL of these before entering judge p
 If any check fails → reject immediately, log reason, do NOT send to judges.
 Layer 0 may auto-retry up to 3 times before escalating to human review.
 
+## Benchmark Gate Rules
+
+The BENCHMARK layer sits between Layer 0 and Layer 1 and provides objective, quantitative evaluation before subjective expert judging begins.
+
+### Gate Thresholds
+
+| BENCHMARK_SCORE | Decision | Action |
+|-----------------|----------|--------|
+| < 0.50 | BLOCK | Do NOT send to judge panel. Fix structural issues first. Log all failing layers. |
+| 0.50 - 0.70 | WARN | Proceed to TEST with advisory. Include benchmark details in judge context. |
+| > 0.70 | PASS | Proceed to TEST. Benchmark data available as optional judge context. |
+
+### Scoring Formula
+
+```
+BENCHMARK_SCORE = B0 * 0.30 + B1 * 0.35 + B2 * 0.15 + B3 * 0.20
+```
+
+Where:
+- **B0 (Golden Sample Comparison):** Structural diff against known-good artifacts (deterministic, zero LLM cost)
+- **B1 (Deterministic Test Suite):** Cross-reference integrity, terminology consistency, instruction specificity (deterministic, zero LLM cost)
+- **B2 (Consistency Probe):** 3 parallel haiku agents interpret the artifact; measure agreement (LLM cost: haiku x 3)
+- **B3 (Performance Metrics):** Token efficiency, section balance, reference/example coverage (deterministic, zero LLM cost)
+
+### BENCHMARK-to-TEST Integration
+
+- If BENCHMARK produces a BLOCK, do NOT invoke Layer 1 or Layer 2 judges. Fix issues and re-benchmark.
+- If BENCHMARK produces a WARN, include the per-layer benchmark scores and failure details in the judge prompt so judges can factor quantitative weaknesses into their subjective evaluation.
+- If BENCHMARK produces a PASS, benchmark data is available but not mandatory for judges.
+- BENCHMARK_SCORE is recorded alongside TEST scores for longitudinal tracking.
+
+### BENCHMARK Anti-Patterns
+
+| Anti-Pattern | Detection Signal | Required Fix |
+|-------------|-----------------|--------------|
+| Skipping BENCHMARK for "simple" artifacts | No benchmark record in evaluation output | BENCHMARK is cheap -- always run it |
+| Golden samples outdated | B0 scores systematically low across all artifacts | Review and refresh golden samples quarterly |
+| Consistency probe too lenient | B2 always scores > 0.95 | Vary probe prompts to stress-test edge interpretations |
+| Ignoring WARN and proceeding blindly | No benchmark context in judge prompts after WARN | Always surface WARN details to TEST judges |
+
 ## Judge Panel Rules
 
 - Panel MUST have an odd number of judges: 3 (standard) or 5 (high-stakes)

package/templates/.claude/shards/bto-evaluation.shard.md

@@ -8,6 +8,7 @@ Load: `.claude/skills/bto/SKILL.md` + relevant module
 | Layer | Model | Gate | Action on Fail |
 |-------|-------|------|----------------|
 | Layer 0 | — (deterministic) | ≥ 80% checks pass | STOP, auto-retry up to 3x |
+| Benchmark | haiku × 3 (B2 only) | score ≥ 0.50 | BLOCK, fix structure |
 | Layer 1 | haiku | avg ≥ 7.0 | NEEDS WORK (flag) |
 | Layer 2 | sonnet × 3 judges | weighted avg ≥ 7.0 | FAIL |
 | Meta | sonnet | disagreement > 3 | Arbitrate |
@@ -30,6 +31,8 @@ Load: `.claude/skills/bto/SKILL.md` + relevant module
 
 ## Model Routing
 - Layer 0: deterministic (no LLM)
+- Benchmark B0, B1, B3: deterministic (no LLM)
+- Benchmark B2: haiku × 3 (consistency probe)
 - Layer 1: haiku
 - Layer 2 judges: sonnet
 - Meta-judge: sonnet
@@ -38,6 +41,7 @@ Load: `.claude/skills/bto/SKILL.md` + relevant module
 
 ## Promises
 - `<promise>BTO_LAYER0_PASSED</promise>` — after Layer 0
+- `<promise>BTO_BENCHMARKED</promise>` — after benchmark
 - `<promise>BTO_LAYER2_SCORED</promise>` — after Layer 2
 - `<promise>BTO_OPTIMIZED</promise>` — after optimization converges