@dzhechkov/keysarium 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -11
- package/package.json +2 -2
- package/src/utils.js +1 -1
- package/templates/.claude/commands/brain-export.md +99 -20
- package/templates/.claude/commands/brain-import.md +75 -4
- package/templates/.claude/commands/bto-benchmark.md +302 -0
- package/templates/.claude/commands/harvest.md +28 -46
- package/templates/.claude/rules/trust-tiers.md +1 -0
- package/templates/.claude/skills/knowledge-extractor/SKILL.md +254 -0
- package/templates/.claude/skills/knowledge-extractor/modules/01-extract.md +188 -0
- package/templates/.claude/skills/knowledge-extractor/modules/02-classify.md +99 -0
- package/templates/.claude/skills/knowledge-extractor/modules/03-gate.md +150 -0
- package/templates/.claude/skills/knowledge-extractor/modules/04-integrate.md +159 -0
- package/templates/.claude/skills/knowledge-extractor/references/artifact-categories.md +100 -0
- package/templates/.claude/skills/knowledge-extractor/references/maturity-model.md +94 -0
- package/templates/.claude/skills/knowledge-extractor/references/quality-gates.md +98 -0
- package/templates/.claude/skills/knowledge-extractor/templates/artifact-card.md +63 -0
- package/templates/.claude/skills/knowledge-extractor/templates/harvest-report.md +102 -0
- package/templates/CLAUDE.md +42 -22
- package/templates/docs/continuous-learning-guide.md +622 -0
- package/templates/docs/feature-adr/01-architecture.md +750 -0
- package/templates/docs/feature-adr/02-functionality.md +712 -0
- package/templates/docs/feature-adr/03-deployment-guide.md +614 -0
- package/templates/docs/feature-adr/04-admin-guide.md +564 -0
- package/templates/docs/feature-adr/05-user-guide.md +508 -0
- package/templates/docs/feature-adr/06-user-flows.md +1101 -0
- package/templates/docs/improvement-directions.md +8 -4
- package/templates/docs/keysarium-core/01-architecture.md +559 -0
- package/templates/docs/keysarium-core/02-functionality.md +955 -0
- package/templates/docs/keysarium-core/03-deployment-guide.md +625 -0
- package/templates/docs/keysarium-core/04-admin-guide.md +529 -0
- package/templates/docs/keysarium-core/05-user-guide.md +599 -0
- package/templates/docs/keysarium-core/06-user-flows.md +804 -0
- package/templates/docs/skills-bto/01-deployment-guide.md +1 -1
- package/templates/docs/skills-bto/02-admin-guide.md +4 -1
- package/templates/docs/skills-bto/03-user-guide.md +4 -3
- package/templates/docs/skills-bto/04-infrastructure-requirements.md +2 -1
- package/templates/docs/skills-bto/05-architecture.md +11 -11
- package/templates/docs/skills-bto/06-user-flows.md +2 -1
- package/templates/lib/dream-engine.md +1 -1
- package/templates/lib/memory-protocol.md +131 -22
- package/templates/lib/reward-tracker.md +6 -3
|
@@ -0,0 +1,302 @@
|
|
|
1
|
+
# /bto-benchmark — Deterministic Benchmarking of a Skill or Command
|
|
2
|
+
|
|
3
|
+
## Usage
|
|
4
|
+
```
|
|
5
|
+
/bto-benchmark [path to skill directory or file]
|
|
6
|
+
```
|
|
7
|
+
|
|
8
|
+
## Parameters
|
|
9
|
+
- $ARGUMENTS — Path to a skill directory (`.claude/skills/<name>/`), a command file (`.claude/commands/<name>.md`), a rule file (`.claude/rules/<name>.md`), or any structured artifact. Optionally include "verbose" to show expanded per-section diagnostics for all layers.
|
|
10
|
+
|
|
11
|
+
## Protocol
|
|
12
|
+
|
|
13
|
+
### Step 1: Load Skill and Module
|
|
14
|
+
|
|
15
|
+
Read `.claude/skills/bto/SKILL.md`
|
|
16
|
+
Read `.claude/skills/bto/modules/benchmark.md`
|
|
17
|
+
Read `.claude/skills/bto/references/golden-samples.md`
|
|
18
|
+
|
|
19
|
+
### Step 2: Validate Input
|
|
20
|
+
|
|
21
|
+
If $ARGUMENTS is empty:
|
|
22
|
+
- Ask: "Provide a path to the skill directory or artifact file you want to benchmark."
|
|
23
|
+
- Stop and wait.
|
|
24
|
+
|
|
25
|
+
Resolve the artifact path from $ARGUMENTS:
|
|
26
|
+
- Strip any trailing slash
|
|
27
|
+
- If the path points to a directory → look for `SKILL.md` inside it as the primary file, but include all files in the directory for context
|
|
28
|
+
- If the path points to a file → use that file directly
|
|
29
|
+
- If the path does not exist → report "Path not found: [path]" and stop
|
|
30
|
+
|
|
31
|
+
### Step 3: Detect Artifact Type
|
|
32
|
+
|
|
33
|
+
Auto-detect from path pattern:
|
|
34
|
+
|
|
35
|
+
| Path Pattern | Detected Type |
|
|
36
|
+
|-------------|--------------|
|
|
37
|
+
| `.claude/skills/*/SKILL.md` or `.claude/skills/*/` | skill |
|
|
38
|
+
| `.claude/commands/*.md` | command |
|
|
39
|
+
| `.claude/rules/*.md` | rule |
|
|
40
|
+
| `.claude/agents/*.md` | agent |
|
|
41
|
+
| `researches/**/*.md` | research artifact |
|
|
42
|
+
|
|
43
|
+
If type cannot be determined from path → infer from file content structure.
|
|
44
|
+
|
|
45
|
+
### Step 4: Layer B0 — Golden Sample Comparison (Deterministic)
|
|
46
|
+
|
|
47
|
+
**Purpose:** Compare artifact structure against the canonical golden sample for its type. Zero LLM cost.
|
|
48
|
+
|
|
49
|
+
Load golden samples for the detected type from `references/golden-samples.md`.
|
|
50
|
+
|
|
51
|
+
**Procedure:**
|
|
52
|
+
|
|
53
|
+
1. Parse the artifact into a section tree (heading hierarchy, section lengths, ordering)
|
|
54
|
+
2. Parse the matching golden sample into the same section tree format
|
|
55
|
+
3. Compare across three structural axes:
|
|
56
|
+
|
|
57
|
+
**Axis 1 — Section Coverage:**
|
|
58
|
+
For each required section in the golden sample, check if the artifact contains a matching section (by heading text or semantic equivalent).
|
|
59
|
+
- `section_coverage = matched_sections / total_golden_sections`
|
|
60
|
+
|
|
61
|
+
**Axis 2 — Ordering Score:**
|
|
62
|
+
Compare the ordering of matched sections against the golden sample ordering.
|
|
63
|
+
- Use normalized Kendall tau distance: `ordering_score = 1 - (inversions / max_inversions)`
|
|
64
|
+
- If fewer than 3 matched sections → ordering_score = 1.0 (too few to meaningfully compare)
|
|
65
|
+
|
|
66
|
+
**Axis 3 — Proportion Score:**
|
|
67
|
+
For each matched section, compute the ratio of artifact section length to golden sample section length.
|
|
68
|
+
- `proportion_score = 1 - mean(|log(artifact_len / golden_len)|)` clamped to [0, 1]
|
|
69
|
+
- Measures whether sections are proportionally sized (not too bloated, not too thin)
|
|
70
|
+
|
|
71
|
+
**Display per-section MATCH/MISS table:**
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
Golden Section Status Artifact Section Size Ratio
|
|
75
|
+
─────────────────────────── ──────── ──────────────────────────── ──────────
|
|
76
|
+
## Overview MATCH ## Overview 1.2x
|
|
77
|
+
## Protocol MATCH ## Protocol 0.8x
|
|
78
|
+
## Anti-Patterns MATCH ## Anti-Patterns 1.0x
|
|
79
|
+
## Dependencies MISS — —
|
|
80
|
+
## Quick Start MATCH ## Quick Start 1.5x
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
**Golden Similarity Score:**
|
|
84
|
+
```
|
|
85
|
+
B0 = section_coverage * 0.50 + ordering_score * 0.25 + proportion_score * 0.25
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
**Gate:** If B0 < 0.30 → BLOCK immediately. The artifact's structure is too far from the expected form to benchmark meaningfully.
|
|
89
|
+
|
|
90
|
+
### Step 5: Layer B1 — Deterministic Test Suite
|
|
91
|
+
|
|
92
|
+
**Purpose:** Run all applicable deterministic tests for the detected artifact type. Zero LLM cost.
|
|
93
|
+
|
|
94
|
+
Execute all applicable checks from `benchmark.md` for the detected artifact type. These overlap with but extend the Layer 0 checks from `/bto-test`:
|
|
95
|
+
|
|
96
|
+
**Universal Checks (U1-U12):**
|
|
97
|
+
- U-01: File exists and is non-empty
|
|
98
|
+
- U-02: UTF-8 encoding valid
|
|
99
|
+
- U-03: Starts with level-1 heading
|
|
100
|
+
- U-04: No placeholder text (`[TODO]`, `[TBD]`, `<INSERT>`, `[PLACEHOLDER]`)
|
|
101
|
+
- U-05: No empty sections (heading with no content before next heading)
|
|
102
|
+
- U-06: Consistent heading hierarchy (no h1→h3 jumps)
|
|
103
|
+
- U-07: No broken internal cross-references
|
|
104
|
+
- U-08: File size within bounds (200B – 100KB per file)
|
|
105
|
+
- U-09: No trailing whitespace on lines
|
|
106
|
+
- U-10: Standard Markdown only (no HTML unless in code blocks)
|
|
107
|
+
- U-11: All code blocks properly closed
|
|
108
|
+
- U-12: No duplicate top-level sections
|
|
109
|
+
|
|
110
|
+
**Type-Specific Checks:**
|
|
111
|
+
- Skill: SK-01 through SK-16 (SKILL.md exists, required sections, modules/, references/, examples/, etc.)
|
|
112
|
+
- Command: CM-01 through CM-08 (Usage section, Parameters, Protocol steps, Checkpoint, Critical Rules, etc.)
|
|
113
|
+
- Rule: RL-01 through RL-06 (Table format, detection signals, fix actions, no contradictions, etc.)
|
|
114
|
+
- Agent: AG-01 through AG-04 (Model specified, isolation rules, naming convention, output format)
|
|
115
|
+
|
|
116
|
+
**Display per-test PASS/FAIL table:**
|
|
117
|
+
|
|
118
|
+
```
|
|
119
|
+
ID Check Result Note
|
|
120
|
+
─────── ────────────────────────────────── ──────── ────────────────────────────
|
|
121
|
+
U-01 File exists and is non-empty PASS 8.4 KB
|
|
122
|
+
U-02 UTF-8 encoding PASS
|
|
123
|
+
...
|
|
124
|
+
SK-01 SKILL.md exists at skill root PASS
|
|
125
|
+
SK-02 Has ## Overview section PASS
|
|
126
|
+
SK-03 Has ## Anti-Patterns section FAIL Section missing
|
|
127
|
+
...
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
**Test Pass Rate:**
|
|
131
|
+
```
|
|
132
|
+
B1 = passed_tests / total_applicable_tests
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
**Gate:** If B1 < 0.60 → BLOCK. Too many structural failures to proceed to consistency probing.
|
|
136
|
+
|
|
137
|
+
### Step 6: Layer B2 — Consistency Probe
|
|
138
|
+
|
|
139
|
+
**Purpose:** Measure evaluation stability by running identical prompts across multiple agents. Detects artifacts that produce inconsistent interpretations.
|
|
140
|
+
|
|
141
|
+
**Model:** haiku (cost-optimized)
|
|
142
|
+
|
|
143
|
+
**Spawn 3 parallel haiku agents with the identical evaluation prompt:**
|
|
144
|
+
|
|
145
|
+
Each agent receives the exact same prompt:
|
|
146
|
+
```
|
|
147
|
+
You are evaluating a Claude Code artifact for structural quality.
|
|
148
|
+
Rate these 4 dimensions (0.0 to 1.0 each, two decimal places):
|
|
149
|
+
1. STRUCTURE — Does the artifact follow a clear, logical organization?
|
|
150
|
+
2. CLARITY — Are instructions unambiguous and actionable?
|
|
151
|
+
3. COVERAGE — Does it address all aspects implied by its title/scope?
|
|
152
|
+
4. CONSISTENCY — Is the internal terminology and style uniform?
|
|
153
|
+
Output as JSON: {"structure": X.XX, "clarity": X.XX, "coverage": X.XX, "consistency": X.XX}
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
**Isolation:** Each agent evaluates independently. No cross-communication.
|
|
157
|
+
|
|
158
|
+
**After all 3 return, compute structural agreement:**
|
|
159
|
+
|
|
160
|
+
For each dimension:
|
|
161
|
+
```
|
|
162
|
+
agreement[dim] = 1 - (max(scores[dim]) - min(scores[dim]))
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
**Consistency Score:**
|
|
166
|
+
```
|
|
167
|
+
B2 = mean(agreement across all 4 dimensions)
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
**Interpretation:**
|
|
171
|
+
- B2 > 0.85 → HIGH consistency — the artifact is unambiguously structured
|
|
172
|
+
- B2 0.60-0.85 → MODERATE consistency — some sections are open to interpretation
|
|
173
|
+
- B2 < 0.60 → LOW consistency — the artifact is structurally ambiguous
|
|
174
|
+
|
|
175
|
+
**Display per-probe comparison:**
|
|
176
|
+
|
|
177
|
+
```
|
|
178
|
+
Dimension Probe 1 Probe 2 Probe 3 Range Agreement
|
|
179
|
+
──────────── ────────── ────────── ────────── ──────── ──────────
|
|
180
|
+
STRUCTURE 0.80 0.85 0.82 0.05 0.95
|
|
181
|
+
CLARITY 0.70 0.75 0.65 0.10 0.90
|
|
182
|
+
COVERAGE 0.60 0.70 0.55 0.15 0.85
|
|
183
|
+
CONSISTENCY 0.75 0.78 0.73 0.05 0.95
|
|
184
|
+
Mean: 0.91
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
No gate on B2 — consistency is informational and feeds into the aggregate score.
|
|
188
|
+
|
|
189
|
+
### Step 7: Layer B3 — Performance Metrics
|
|
190
|
+
|
|
191
|
+
**Purpose:** Quantify token efficiency and information density. Zero LLM cost.
|
|
192
|
+
|
|
193
|
+
**Metric 1 — Token Efficiency:**
|
|
194
|
+
```
|
|
195
|
+
token_efficiency = meaningful_content_tokens / total_tokens
|
|
196
|
+
```
|
|
197
|
+
Where `meaningful_content_tokens` excludes: blank lines, decorative separators, repeated boilerplate headers, and excessive whitespace. Target: > 0.70.
|
|
198
|
+
|
|
199
|
+
**Metric 2 — Information Density:**
|
|
200
|
+
```
|
|
201
|
+
information_density = unique_concepts / total_sections
|
|
202
|
+
```
|
|
203
|
+
Where `unique_concepts` = count of distinct topics, terms, or instructions introduced. Measures whether each section contributes new information vs. repeating prior content. Target: > 2.0 concepts per section.
|
|
204
|
+
|
|
205
|
+
**Metric 3 — Bloat Detection:**
|
|
206
|
+
Identify sections where:
|
|
207
|
+
- Section length > 3x the golden sample equivalent → BLOATED
|
|
208
|
+
- Section length < 0.2x the golden sample equivalent → THIN
|
|
209
|
+
- Section repeats > 30% of content from another section → REDUNDANT
|
|
210
|
+
|
|
211
|
+
```
|
|
212
|
+
bloat_score = 1 - (bloated_sections + thin_sections + redundant_sections) / total_sections
|
|
213
|
+
```
|
|
214
|
+
Clamped to [0, 1].
|
|
215
|
+
|
|
216
|
+
**Performance Score:**
|
|
217
|
+
```
|
|
218
|
+
B3 = token_efficiency * 0.35 + normalize(information_density, 0, 4) * 0.35 + bloat_score * 0.30
|
|
219
|
+
```
|
|
220
|
+
Where `normalize(x, min, max)` clamps and scales x to [0, 1].
|
|
221
|
+
|
|
222
|
+
**Display per-section metrics:**
|
|
223
|
+
|
|
224
|
+
```
|
|
225
|
+
Section Tokens Concepts Golden Ratio Flag
|
|
226
|
+
───────────────────────── ──────── ────────── ────────────── ──────────
|
|
227
|
+
## Overview 320 4 1.2x OK
|
|
228
|
+
## Protocol 1840 12 0.8x OK
|
|
229
|
+
## Anti-Patterns 580 6 2.8x BLOATED
|
|
230
|
+
## Quick Start 95 2 0.3x THIN
|
|
231
|
+
## Dependencies 210 3 1.0x OK
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
### Step 8: Aggregate & Gate
|
|
235
|
+
|
|
236
|
+
**BENCHMARK_SCORE formula:**
|
|
237
|
+
```
|
|
238
|
+
BENCHMARK_SCORE = B0 * 0.30 + B1 * 0.35 + B2 * 0.15 + B3 * 0.20
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
**Weight rationale:**
|
|
242
|
+
- B0 (Golden Comparison) at 0.30 — structural conformance is the primary benchmark signal
|
|
243
|
+
- B1 (Test Suite) at 0.35 — deterministic correctness is the most reliable signal
|
|
244
|
+
- B2 (Consistency) at 0.15 — stability matters but is a secondary signal
|
|
245
|
+
- B3 (Performance) at 0.20 — efficiency matters for production artifacts
|
|
246
|
+
|
|
247
|
+
**Gate:**
|
|
248
|
+
- BENCHMARK_SCORE < 0.50 → **BLOCK** — artifact requires significant rework before evaluation
|
|
249
|
+
- BENCHMARK_SCORE 0.50-0.70 → **WARN** — artifact has structural issues; proceed to TEST with caution
|
|
250
|
+
- BENCHMARK_SCORE > 0.70 → **PASS** — artifact meets benchmark standards; ready for TEST
|
|
251
|
+
|
|
252
|
+
Record `BENCHMARK_SCORE` for downstream consumption by `/bto-test` and `/bto`.
|
|
253
|
+
|
|
254
|
+
---
|
|
255
|
+
|
|
256
|
+
## Checkpoint
|
|
257
|
+
|
|
258
|
+
```
|
|
259
|
+
═══════════════════════════════════════════════════════
|
|
260
|
+
CHECKPOINT: BENCHMARK Complete
|
|
261
|
+
<promise>BTO_BENCHMARKED</promise>
|
|
262
|
+
Artifact: [path]
|
|
263
|
+
Type: [detected type]
|
|
264
|
+
|
|
265
|
+
Layer B0 (Golden): X.XX — section coverage, ordering, proportions
|
|
266
|
+
Layer B1 (Test Suite): X.XX — Y/Z tests passed
|
|
267
|
+
Layer B2 (Consistency): X.XX — agreement across 3 probes
|
|
268
|
+
Layer B3 (Performance): X.XX — token efficiency, density, bloat
|
|
269
|
+
|
|
270
|
+
BENCHMARK SCORE: X.XX [PASS / WARN / BLOCK]
|
|
271
|
+
|
|
272
|
+
• "ок" — proceed to TEST
|
|
273
|
+
• "покажи детали [B0/B1/B2/B3]" — expand layer details
|
|
274
|
+
• "пропусти тест" — skip TEST (if PASS)
|
|
275
|
+
═══════════════════════════════════════════════════════
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Wait for user confirmation.
|
|
279
|
+
|
|
280
|
+
---
|
|
281
|
+
|
|
282
|
+
## Modular Usage
|
|
283
|
+
|
|
284
|
+
This command is also invoked internally by:
|
|
285
|
+
- `/bto` — as a pre-flight step before TEST phase (if benchmark module is loaded)
|
|
286
|
+
- `/bto-test` — optionally as a prerequisite gate (if user requests `benchmark` flag)
|
|
287
|
+
|
|
288
|
+
The `BENCHMARK_SCORE` is available to downstream commands:
|
|
289
|
+
- `/bto-test` may use it to calibrate Layer 1 expectations
|
|
290
|
+
- `/bto-optimize` may use it to set a structural baseline floor
|
|
291
|
+
|
|
292
|
+
## Critical Rules
|
|
293
|
+
|
|
294
|
+
- Layer B0 and B1 are fully deterministic — no LLM calls, zero cost
|
|
295
|
+
- Layer B2 uses haiku only — NEVER use sonnet or opus for consistency probes
|
|
296
|
+
- Agent tool is REQUIRED for Layer B2 — do not run probes sequentially
|
|
297
|
+
- Golden samples from `references/golden-samples.md` are the authoritative structural reference
|
|
298
|
+
- If golden sample does not exist for the detected type → skip B0, set B0 = 0.50 (neutral), and log a warning
|
|
299
|
+
- Report `BENCHMARK_SCORE` explicitly so downstream commands can consume it
|
|
300
|
+
- BLOCK verdict at B0 or B1 halts the benchmark — do NOT proceed to subsequent layers
|
|
301
|
+
- Layer B3 bloat detection uses golden sample proportions — without a golden sample, use absolute thresholds only
|
|
302
|
+
- If "verbose" is in $ARGUMENTS, show expanded per-section diagnostics for all layers without prompting
|
|
@@ -2,66 +2,48 @@
|
|
|
2
2
|
|
|
3
3
|
## Использование
|
|
4
4
|
```
|
|
5
|
-
/harvest [путь к директории
|
|
5
|
+
/harvest [путь к директории или "all"] [only категория1,категория2]
|
|
6
6
|
```
|
|
7
7
|
|
|
8
8
|
## Аргумент
|
|
9
9
|
$ARGUMENTS
|
|
10
10
|
|
|
11
|
-
##
|
|
12
|
-
|
|
13
|
-
Систематический процесс извлечения и организации полезных знаний после завершения исследования. Не героическое усилие, а лёгкий процесс после каждого проекта.
|
|
14
|
-
|
|
15
|
-
## Шаг 1 — Ревью с агентом
|
|
11
|
+
## Скилл
|
|
16
12
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
3. Ошибки и workaround-ы, которые стоит запомнить
|
|
21
|
-
4. Удачные формулировки и шаблоны
|
|
22
|
-
5. Инсайты о домене
|
|
23
|
-
|
|
24
|
-
## Шаг 2 — Классификация
|
|
13
|
+
```
|
|
14
|
+
Read: .claude/skills/knowledge-extractor/SKILL.md
|
|
15
|
+
```
|
|
25
16
|
|
|
26
|
-
|
|
17
|
+
Загрузи knowledge-extractor skill и следуй его Pipeline Protocol.
|
|
27
18
|
|
|
28
|
-
|
|
29
|
-
|-----------|--------------|--------|
|
|
30
|
-
| skills/ | Новые или улучшенные SKILL.md | Лучший подход к CJM |
|
|
31
|
-
| commands/ | Slash-команды для Claude Code | Новая фаза пайплайна |
|
|
32
|
-
| rules/ | Правила и ограничения | "Не используй X потому что Y" |
|
|
33
|
-
| templates/ | Шаблоны документов | Обновлённый шаблон архитектуры |
|
|
34
|
-
| patterns/ | Архитектурные паттерны | Event sourcing для AI pipeline |
|
|
35
|
-
| snippets/ | Готовые куски кода | Универсальный CJM renderer |
|
|
19
|
+
## Парсинг аргументов
|
|
36
20
|
|
|
37
|
-
|
|
21
|
+
1. **Путь:** Первый аргумент — путь к директории исследования или `"all"` для всех директорий в `researches/`
|
|
22
|
+
2. **Scope filter:** Если после пути есть `only X,Y` — передай как `{SCOPE_FILTER}` в skill
|
|
23
|
+
- Допустимые значения: `skills`, `commands`, `hooks`, `rules`, `templates`, `patterns`, `snippets`
|
|
24
|
+
- Множественные через запятую: `only rules,templates`
|
|
25
|
+
3. Если аргументов нет — запроси путь у пользователя
|
|
38
26
|
|
|
39
|
-
|
|
40
|
-
1. **Деконтекстуализировать** — убрать привязки к конкретному проекту
|
|
41
|
-
2. **Документировать** — когда использовать, когда нет, предусловия
|
|
42
|
-
3. **Версионировать** — пометить что изменилось и почему
|
|
27
|
+
## Примеры
|
|
43
28
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
- v1: начальная версия из [проект]
|
|
29
|
+
```
|
|
30
|
+
/harvest researches/bank-kc-automation/ ← полный harvest одной директории
|
|
31
|
+
/harvest researches/bank-kc-automation/ only patterns ← только паттерны
|
|
32
|
+
/harvest all ← все исследования
|
|
33
|
+
/harvest all only rules,templates ← все исследования, только правила и шаблоны
|
|
34
|
+
/harvest features/add-user-auth/ ← harvest фичи (не только исследования)
|
|
51
35
|
```
|
|
52
36
|
|
|
53
|
-
##
|
|
54
|
-
|
|
55
|
-
Обнови корневой `TOOLKIT_HARVEST.md` с новыми находками.
|
|
37
|
+
## Параллелизация
|
|
56
38
|
|
|
57
|
-
|
|
39
|
+
Если путь = `"all"`:
|
|
40
|
+
1. Получи список всех директорий в `researches/`
|
|
41
|
+
2. Для каждой директории запусти отдельную harvest сессию
|
|
42
|
+
3. Каждая сессия создаёт свой findings JSON и проходит полный pipeline
|
|
43
|
+
4. По завершении всех сессий — объедини отчёты
|
|
58
44
|
|
|
59
|
-
|
|
60
|
-
- Паттерны, использованные один раз без уверенности в качестве
|
|
61
|
-
- Workaround-ы для конкретных багов библиотек (устаревают)
|
|
45
|
+
## Суть
|
|
62
46
|
|
|
63
|
-
|
|
47
|
+
Систематический процесс извлечения и организации полезных знаний после завершения проекта. Использует 5 параллельных агентов-экстракторов, 7-категорийную классификацию, 8 блокирующих quality gates, и автоматическое размещение артефактов.
|
|
64
48
|
|
|
65
|
-
|
|
66
|
-
- Каждый агент ревьюит свою директорию
|
|
67
|
-
- Результаты синтезируются в единый TOOLKIT_HARVEST.md
|
|
49
|
+
Pipeline: Extract (5 agents) → Classify (7 categories) → User Checkpoint → Gate (8 checks) → Integrate (auto-place)
|
|
@@ -20,6 +20,7 @@
|
|
|
20
20
|
| presentation-storyteller | 1 | Structured | SKILL.md + references/ |
|
|
21
21
|
| reverse-engineering-unicorn | 1 | Structured | SKILL.md + modules/ + references/ + examples/ |
|
|
22
22
|
| bto | 1 | Structured | SKILL.md + modules/ + references/ + examples/ |
|
|
23
|
+
| knowledge-extractor | 2 | Validated | /bto-test Layer 2 score 7.5, optimized 2026-03-03 |
|
|
23
24
|
|
|
24
25
|
## Tier Enforcement
|
|
25
26
|
|
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: knowledge-extractor
|
|
3
|
+
description: >
|
|
4
|
+
Multi-agent knowledge harvesting system that extracts reusable patterns, commands,
|
|
5
|
+
rules, templates, and snippets from any project directory. Uses 5 parallel extractor
|
|
6
|
+
agents, 7-category classification, 8 blocking quality gates, and auto-placement.
|
|
7
|
+
Triggers on "harvest", "extract knowledge", "извлеки знания", "/harvest".
|
|
8
|
+
trust_tier: 2
|
|
9
|
+
trust_tier_label: "Validated"
|
|
10
|
+
bto_score: 7.5
|
|
11
|
+
bto_date: "2026-03-03"
|
|
12
|
+
trust_tier_path: "Run /bto-test, score >= 8.5 + eval tests for Tier 3"
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Knowledge Extractor
|
|
16
|
+
|
|
17
|
+
> Domain-agnostic multi-agent system for extracting, classifying, gating, and placing
|
|
18
|
+
> reusable knowledge from completed projects into the toolkit.
|
|
19
|
+
|
|
20
|
+
## When To Activate
|
|
21
|
+
|
|
22
|
+
Trigger on:
|
|
23
|
+
- `/harvest [path]` or `/harvest all`
|
|
24
|
+
- "extract knowledge from [path]"
|
|
25
|
+
- "извлеки знания из [path]"
|
|
26
|
+
- "harvest patterns from [project]"
|
|
27
|
+
- "что можно переиспользовать из [project]"
|
|
28
|
+
|
|
29
|
+
## What You Get
|
|
30
|
+
|
|
31
|
+
After a successful harvest session:
|
|
32
|
+
- **Findings JSON** — structured extraction results at `.keysarium/harvest/findings-{SESSION_ID}.json`
|
|
33
|
+
- **Placed artifacts** — skills, commands, and rules placed in `.claude/skills/`, `.claude/commands/`, `.claude/rules/`
|
|
34
|
+
- **TOOLKIT_HARVEST.md** — updated with new patterns, templates, snippets, and hooks
|
|
35
|
+
- **Harvest report** — session summary with metrics (extracted → approved → placed counts, gate pass rates)
|
|
36
|
+
- **Memory record** — reward data stored via `memory_store()` for future session optimization
|
|
37
|
+
|
|
38
|
+
## Architecture
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
.claude/skills/knowledge-extractor/
|
|
42
|
+
├── SKILL.md ← This file (orchestrator)
|
|
43
|
+
├── modules/
|
|
44
|
+
│ ├── 01-extract.md ← 5 parallel extractor agents
|
|
45
|
+
│ ├── 02-classify.md ← 7-category classification + dedup
|
|
46
|
+
│ ├── 03-gate.md ← 8 quality gates (2-pass)
|
|
47
|
+
│ └── 04-integrate.md ← Auto-placement + harvest report
|
|
48
|
+
├── references/
|
|
49
|
+
│ ├── quality-gates.md ← Gate definitions (G1-G8)
|
|
50
|
+
│ ├── artifact-categories.md ← 7 categories with destinations
|
|
51
|
+
│ └── maturity-model.md ← Alpha → Beta → Stable
|
|
52
|
+
└── templates/
|
|
53
|
+
├── artifact-card.md ← Per-finding output template
|
|
54
|
+
└── harvest-report.md ← Session summary template
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Modules
|
|
58
|
+
|
|
59
|
+
| Module | Purpose | Agents | Model |
|
|
60
|
+
|--------|---------|--------|-------|
|
|
61
|
+
| 01-extract | Scan project via 5 focused lenses | 5 parallel | sonnet |
|
|
62
|
+
| 02-classify | Assign categories, deduplicate, cross-ref | 0 (inline) | — |
|
|
63
|
+
| 03-gate | Evaluate 8 quality gates in 2 passes | 1 (semantic) | haiku |
|
|
64
|
+
| 04-integrate | Place artifacts, generate report | 0 (inline) | — |
|
|
65
|
+
|
|
66
|
+
## Pipeline Protocol
|
|
67
|
+
|
|
68
|
+
### Step 0: Initialize Session
|
|
69
|
+
|
|
70
|
+
1. Parse arguments: `{TARGET_PATH}` (directory or "all"), `{SCOPE_FILTER}` (optional `only X,Y`)
|
|
71
|
+
2. **Resolve target path:**
|
|
72
|
+
- If `{TARGET_PATH}` is a specific directory → validate it exists, set `{slug}` = directory basename
|
|
73
|
+
- If `{TARGET_PATH}` is `"all"` → list all subdirectories in `researches/`. For each directory, run a separate harvest session (each gets its own findings JSON, session ID, and pipeline run). Merge reports at the end.
|
|
74
|
+
- If `{TARGET_PATH}` does not exist → report error and stop
|
|
75
|
+
3. **Resolve domain:** If running inside a `/casarium` pipeline, inherit `{domain}` from Phase 0 discovery. If standalone, detect domain from file content keywords (banking/retail/enterprise/healthcare). If unable to detect, set `{domain}` = `"unknown"`.
|
|
76
|
+
4. Generate session ID: `{slug}-{YYYYMMDDHHmmss}`
|
|
77
|
+
5. If `TOOLKIT_HARVEST.md` does not exist at repo root → create it with a standard skeleton (title + empty section headers for each category).
|
|
78
|
+
6. Call `memory_query({phase: "harvest", domain, slug, skill: "knowledge-extractor"})`
|
|
79
|
+
7. Log loaded patterns count
|
|
80
|
+
|
|
81
|
+
### Step 1: Extract (Module 01)
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
Read: modules/01-extract.md
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Spawn 5 parallel extractor agents (sonnet), each with a focused lens:
|
|
88
|
+
|
|
89
|
+
| Agent | Lens | Seeks |
|
|
90
|
+
|-------|------|-------|
|
|
91
|
+
| extractor-patterns | patterns | Architecture patterns, design patterns, agent topologies |
|
|
92
|
+
| extractor-commands | commands | Scripts, CLI workflows, pipeline stages, automation |
|
|
93
|
+
| extractor-rules | rules | Constraints, anti-patterns, lessons learned, domain rules |
|
|
94
|
+
| extractor-templates | templates | Document structures, config templates, diagram templates |
|
|
95
|
+
| extractor-snippets | snippets | Reusable code fragments, hooks, utility functions |
|
|
96
|
+
|
|
97
|
+
Each agent returns numbered findings with reusability confidence (0.0-1.0).
|
|
98
|
+
Orchestrator merges all findings into a single numbered list and writes to JSON:
|
|
99
|
+
`.keysarium/harvest/findings-{SESSION_ID}.json`
|
|
100
|
+
|
|
101
|
+
**Scope filtering:** If `{SCOPE_FILTER}` is set, only spawn matching agents. See mapping in `references/artifact-categories.md`.
|
|
102
|
+
|
|
103
|
+
### Step 2: Classify (Module 02)
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
Read: modules/02-classify.md
|
|
107
|
+
Read: references/artifact-categories.md
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
1. Read findings from JSON file
|
|
111
|
+
2. Assign each finding to one of 7 categories
|
|
112
|
+
3. Deduplicate: check content similarity across findings and against TOOLKIT_HARVEST.md
|
|
113
|
+
4. Cross-reference: link findings that reference existing skills/commands
|
|
114
|
+
5. Write classified findings back to JSON
|
|
115
|
+
|
|
116
|
+
### Step 3: User Checkpoint (Interactive)
|
|
117
|
+
|
|
118
|
+
Display all findings grouped by category as a numbered list:
|
|
119
|
+
|
|
120
|
+
```
|
|
121
|
+
═══════════════════════════════════════════════════════
|
|
122
|
+
📋 HARVEST FINDINGS: {N} items extracted
|
|
123
|
+
|
|
124
|
+
## patterns (K items)
|
|
125
|
+
#1 [0.85] Agent Swarm Topology — parallel agents with merge pattern
|
|
126
|
+
#4 [0.72] Event-Driven Pipeline — phase-to-phase data flow via files
|
|
127
|
+
|
|
128
|
+
## rules (M items)
|
|
129
|
+
#2 [0.91] Domain Detection Rule — auto-detect domain from keywords
|
|
130
|
+
#7 [0.65] Regulatory Constraint — ФЗ-152 data isolation requirement
|
|
131
|
+
|
|
132
|
+
## snippets (P items)
|
|
133
|
+
#3 [0.78] CJM Renderer — universal React component for CJM display
|
|
134
|
+
...
|
|
135
|
+
|
|
136
|
+
Commands:
|
|
137
|
+
• "убери #N" — remove finding
|
|
138
|
+
• "переклассифицируй #N в X" — reclassify to category X
|
|
139
|
+
• "доработай #N" — expand/improve finding
|
|
140
|
+
• "объедини #N и #M" — merge two findings
|
|
141
|
+
• "покажи только X" — filter by category
|
|
142
|
+
• "ок" — proceed to quality gates
|
|
143
|
+
═══════════════════════════════════════════════════════
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Apply user mutations to JSON file. Numbers are stable (removed items leave gaps).
|
|
147
|
+
Loop until user says "ок".
|
|
148
|
+
|
|
149
|
+
### Step 4: Gate (Module 03)
|
|
150
|
+
|
|
151
|
+
```
|
|
152
|
+
Read: modules/03-gate.md
|
|
153
|
+
Read: references/quality-gates.md
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Evaluate each active finding against 8 quality gates in 2 passes:
|
|
157
|
+
|
|
158
|
+
**Pass 1 — Deterministic (zero LLM cost):**
|
|
159
|
+
- G1: Has "When to use" section
|
|
160
|
+
- G2: Has "When NOT to use" section
|
|
161
|
+
- G5: Reusability confidence >= 0.5
|
|
162
|
+
- G6: Not a duplicate of existing toolkit entry
|
|
163
|
+
- G7: Has maturity label (alpha/beta/stable)
|
|
164
|
+
|
|
165
|
+
**Pass 2 — Semantic (haiku agent):**
|
|
166
|
+
- G3: Properly decontextualized (no project-specific refs)
|
|
167
|
+
- G4: Has at least one concrete example
|
|
168
|
+
- G8: Passes brutal-honesty review (self-critical assessment)
|
|
169
|
+
|
|
170
|
+
Findings passing all gates → status `approved`.
|
|
171
|
+
Findings failing any gate → status `blocked` with gate IDs.
|
|
172
|
+
|
|
173
|
+
Display blocked findings and allow overrides:
|
|
174
|
+
```
|
|
175
|
+
⚠️ BLOCKED: #7 — failed G3 (project-specific references remain)
|
|
176
|
+
• "пропусти G3 для #7" — override gate
|
|
177
|
+
• "ок" — accept blocks, proceed with approved only
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
### Step 5: Integrate (Module 04)
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
Read: modules/04-integrate.md
|
|
184
|
+
Read: templates/artifact-card.md
|
|
185
|
+
Read: templates/harvest-report.md
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
1. Render artifact cards for each approved finding
|
|
189
|
+
2. Auto-place by category:
|
|
190
|
+
- `skills` → create `.claude/skills/<name>/SKILL.md` (skeleton)
|
|
191
|
+
- `commands` → create `.claude/commands/<name>.md`
|
|
192
|
+
- `hooks` → document in TOOLKIT_HARVEST.md (hooks section)
|
|
193
|
+
- `rules` → create `.claude/rules/<name>.md`
|
|
194
|
+
- `templates` → document in TOOLKIT_HARVEST.md (templates section)
|
|
195
|
+
- `patterns` → document in TOOLKIT_HARVEST.md (patterns section)
|
|
196
|
+
- `snippets` → document in TOOLKIT_HARVEST.md (snippets section)
|
|
197
|
+
3. Update TOOLKIT_HARVEST.md "Обработанные проекты" table
|
|
198
|
+
4. Generate harvest report
|
|
199
|
+
5. Call `memory_store()` with harvest results and user reward
|
|
200
|
+
6. Update `.keysarium/insights/trigger-state.json` (case_completion event)
|
|
201
|
+
7. Clean up findings JSON file
|
|
202
|
+
|
|
203
|
+
## Scope Filtering
|
|
204
|
+
|
|
205
|
+
The `only` keyword filters which extractor agents are spawned:
|
|
206
|
+
|
|
207
|
+
```
|
|
208
|
+
/harvest path/ → all 5 agents
|
|
209
|
+
/harvest path/ only patterns → extractor-patterns only
|
|
210
|
+
/harvest path/ only rules,templates → extractor-rules + extractor-templates
|
|
211
|
+
/harvest all only snippets → extractor-snippets for each directory
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Category-to-agent mapping (see `references/artifact-categories.md`):
|
|
215
|
+
- patterns → extractor-patterns
|
|
216
|
+
- commands → extractor-commands
|
|
217
|
+
- hooks → extractor-commands (shared lens)
|
|
218
|
+
- rules → extractor-rules
|
|
219
|
+
- templates → extractor-templates
|
|
220
|
+
- snippets → extractor-snippets
|
|
221
|
+
- skills → all agents (skills are cross-cutting)
|
|
222
|
+
|
|
223
|
+
## Integration Points
|
|
224
|
+
|
|
225
|
+
| System | When | Operation |
|
|
226
|
+
|--------|------|-----------|
|
|
227
|
+
| memory_query() | Pipeline start | Load historical harvest patterns |
|
|
228
|
+
| memory_store() | After user checkpoint | Store harvest reward data |
|
|
229
|
+
| brain-export | N/A (passive) | Findings JSON is brain-compatible schema |
|
|
230
|
+
| BTO | Post-harvest | Placed skills can be evaluated via `/bto-test` |
|
|
231
|
+
| dream cycles | Pipeline end | Update trigger-state.json |
|
|
232
|
+
| /harvest command | Entry point | Thin wrapper loads this skill |
|
|
233
|
+
|
|
234
|
+
## Anti-Patterns
|
|
235
|
+
|
|
236
|
+
| Anti-Pattern | Detection | Fix |
|
|
237
|
+
|-------------|-----------|-----|
|
|
238
|
+
| Extracting project-specific details | Finding contains project names, slugs, dates | Decontextualize: replace specifics with generic placeholders |
|
|
239
|
+
| Duplicate extraction | content_hash matches existing TOOLKIT_HARVEST.md entry | Skip or merge; gate G6 blocks duplicates |
|
|
240
|
+
| Extracting from incomplete phases | Artifact lacks promise tag or checkpoint | Only harvest from completed projects |
|
|
241
|
+
| Over-classifying snippets as patterns | Single-use code without abstraction | Require reuse evidence before promoting to pattern |
|
|
242
|
+
| Under-specifying maturity | All findings marked "stable" on first run | First extraction is always "alpha" or "beta" |
|
|
243
|
+
| Skipping user checkpoint | Proceeding to gates without user review | BLOCK — user checkpoint is mandatory |
|
|
244
|
+
| Running all agents when scope is filtered | `only patterns` but 5 agents spawned | Check SCOPE_FILTER before spawning |
|
|
245
|
+
|
|
246
|
+
## Dependencies
|
|
247
|
+
|
|
248
|
+
| Resource | Path | Purpose |
|
|
249
|
+
|----------|------|---------|
|
|
250
|
+
| quality-gates.md | references/quality-gates.md | Gate definitions for Module 03 |
|
|
251
|
+
| artifact-categories.md | references/artifact-categories.md | 7-category taxonomy |
|
|
252
|
+
| maturity-model.md | references/maturity-model.md | Alpha/Beta/Stable criteria |
|
|
253
|
+
| artifact-card.md | templates/artifact-card.md | Per-finding output format |
|
|
254
|
+
| harvest-report.md | templates/harvest-report.md | Session report format |
|