@dzhechkov/keysarium 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -9
- package/package.json +1 -1
- package/src/utils.js +1 -1
- package/templates/.claude/commands/brain-export.md +2 -0
- package/templates/.claude/commands/bto-benchmark.md +302 -0
- package/templates/.claude/commands/harvest.md +28 -46
- package/templates/.claude/rules/trust-tiers.md +1 -0
- package/templates/.claude/skills/knowledge-extractor/SKILL.md +254 -0
- package/templates/.claude/skills/knowledge-extractor/modules/01-extract.md +188 -0
- package/templates/.claude/skills/knowledge-extractor/modules/02-classify.md +99 -0
- package/templates/.claude/skills/knowledge-extractor/modules/03-gate.md +150 -0
- package/templates/.claude/skills/knowledge-extractor/modules/04-integrate.md +159 -0
- package/templates/.claude/skills/knowledge-extractor/references/artifact-categories.md +100 -0
- package/templates/.claude/skills/knowledge-extractor/references/maturity-model.md +94 -0
- package/templates/.claude/skills/knowledge-extractor/references/quality-gates.md +98 -0
- package/templates/.claude/skills/knowledge-extractor/templates/artifact-card.md +63 -0
- package/templates/.claude/skills/knowledge-extractor/templates/harvest-report.md +102 -0
- package/templates/CLAUDE.md +22 -8
- package/templates/docs/continuous-learning-guide.md +622 -0
- package/templates/docs/improvement-directions.md +8 -4
- package/templates/docs/keysarium-core/02-functionality.md +3 -1
- package/templates/docs/skills-bto/01-deployment-guide.md +1 -1
- package/templates/docs/skills-bto/02-admin-guide.md +4 -1
- package/templates/docs/skills-bto/03-user-guide.md +4 -3
- package/templates/docs/skills-bto/04-infrastructure-requirements.md +2 -1
- package/templates/docs/skills-bto/05-architecture.md +11 -11
- package/templates/docs/skills-bto/06-user-flows.md +2 -1
- package/templates/lib/dream-engine.md +1 -1
- package/templates/lib/memory-protocol.md +6 -2
- package/templates/lib/reward-tracker.md +6 -3
|
@@ -0,0 +1,254 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: knowledge-extractor
|
|
3
|
+
description: >
|
|
4
|
+
Multi-agent knowledge harvesting system that extracts reusable patterns, commands,
|
|
5
|
+
rules, templates, and snippets from any project directory. Uses 5 parallel extractor
|
|
6
|
+
agents, 7-category classification, 8 blocking quality gates, and auto-placement.
|
|
7
|
+
Triggers on "harvest", "extract knowledge", "извлеки знания", "/harvest".
|
|
8
|
+
trust_tier: 2
|
|
9
|
+
trust_tier_label: "Validated"
|
|
10
|
+
bto_score: 7.5
|
|
11
|
+
bto_date: "2026-03-03"
|
|
12
|
+
trust_tier_path: "Run /bto-test, score >= 8.5 + eval tests for Tier 3"
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
# Knowledge Extractor
|
|
16
|
+
|
|
17
|
+
> Domain-agnostic multi-agent system for extracting, classifying, gating, and placing
|
|
18
|
+
> reusable knowledge from completed projects into the toolkit.
|
|
19
|
+
|
|
20
|
+
## When To Activate
|
|
21
|
+
|
|
22
|
+
Trigger on:
|
|
23
|
+
- `/harvest [path]` or `/harvest all`
|
|
24
|
+
- "extract knowledge from [path]"
|
|
25
|
+
- "извлеки знания из [path]"
|
|
26
|
+
- "harvest patterns from [project]"
|
|
27
|
+
- "что можно переиспользовать из [project]"
|
|
28
|
+
|
|
29
|
+
## What You Get
|
|
30
|
+
|
|
31
|
+
After a successful harvest session:
|
|
32
|
+
- **Findings JSON** — structured extraction results at `.keysarium/harvest/findings-{SESSION_ID}.json`
|
|
33
|
+
- **Placed artifacts** — skills, commands, and rules placed in `.claude/skills/`, `.claude/commands/`, `.claude/rules/`
|
|
34
|
+
- **TOOLKIT_HARVEST.md** — updated with new patterns, templates, snippets, and hooks
|
|
35
|
+
- **Harvest report** — session summary with metrics (extracted → approved → placed counts, gate pass rates)
|
|
36
|
+
- **Memory record** — reward data stored via `memory_store()` for future session optimization
|
|
37
|
+
|
|
38
|
+
## Architecture
|
|
39
|
+
|
|
40
|
+
```
|
|
41
|
+
.claude/skills/knowledge-extractor/
|
|
42
|
+
├── SKILL.md ← This file (orchestrator)
|
|
43
|
+
├── modules/
|
|
44
|
+
│ ├── 01-extract.md ← 5 parallel extractor agents
|
|
45
|
+
│ ├── 02-classify.md ← 7-category classification + dedup
|
|
46
|
+
│ ├── 03-gate.md ← 8 quality gates (2-pass)
|
|
47
|
+
│ └── 04-integrate.md ← Auto-placement + harvest report
|
|
48
|
+
├── references/
|
|
49
|
+
│ ├── quality-gates.md ← Gate definitions (G1-G8)
|
|
50
|
+
│ ├── artifact-categories.md ← 7 categories with destinations
|
|
51
|
+
│ └── maturity-model.md ← Alpha → Beta → Stable
|
|
52
|
+
└── templates/
|
|
53
|
+
├── artifact-card.md ← Per-finding output template
|
|
54
|
+
└── harvest-report.md ← Session summary template
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Modules
|
|
58
|
+
|
|
59
|
+
| Module | Purpose | Agents | Model |
|
|
60
|
+
|--------|---------|--------|-------|
|
|
61
|
+
| 01-extract | Scan project via 5 focused lenses | 5 parallel | sonnet |
|
|
62
|
+
| 02-classify | Assign categories, deduplicate, cross-ref | 0 (inline) | — |
|
|
63
|
+
| 03-gate | Evaluate 8 quality gates in 2 passes | 1 (semantic) | haiku |
|
|
64
|
+
| 04-integrate | Place artifacts, generate report | 0 (inline) | — |
|
|
65
|
+
|
|
66
|
+
## Pipeline Protocol
|
|
67
|
+
|
|
68
|
+
### Step 0: Initialize Session
|
|
69
|
+
|
|
70
|
+
1. Parse arguments: `{TARGET_PATH}` (directory or "all"), `{SCOPE_FILTER}` (optional `only X,Y`)
|
|
71
|
+
2. **Resolve target path:**
|
|
72
|
+
- If `{TARGET_PATH}` is a specific directory → validate it exists, set `{slug}` = directory basename
|
|
73
|
+
- If `{TARGET_PATH}` is `"all"` → list all subdirectories in `researches/`. For each directory, run a separate harvest session (each gets its own findings JSON, session ID, and pipeline run). Merge reports at the end.
|
|
74
|
+
- If `{TARGET_PATH}` does not exist → report error and stop
|
|
75
|
+
3. **Resolve domain:** If running inside a `/casarium` pipeline, inherit `{domain}` from Phase 0 discovery. If standalone, detect domain from file content keywords (banking/retail/enterprise/healthcare). If unable to detect, set `{domain}` = `"unknown"`.
|
|
76
|
+
4. Generate session ID: `{slug}-{YYYYMMDDHHmmss}`
|
|
77
|
+
5. If `TOOLKIT_HARVEST.md` does not exist at repo root → create it with a standard skeleton (title + empty section headers for each category).
|
|
78
|
+
6. Call `memory_query({phase: "harvest", domain, slug, skill: "knowledge-extractor"})`
|
|
79
|
+
7. Log loaded patterns count
|
|
80
|
+
|
|
81
|
+
### Step 1: Extract (Module 01)
|
|
82
|
+
|
|
83
|
+
```
|
|
84
|
+
Read: modules/01-extract.md
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
Spawn 5 parallel extractor agents (sonnet), each with a focused lens:
|
|
88
|
+
|
|
89
|
+
| Agent | Lens | Seeks |
|
|
90
|
+
|-------|------|-------|
|
|
91
|
+
| extractor-patterns | patterns | Architecture patterns, design patterns, agent topologies |
|
|
92
|
+
| extractor-commands | commands | Scripts, CLI workflows, pipeline stages, automation |
|
|
93
|
+
| extractor-rules | rules | Constraints, anti-patterns, lessons learned, domain rules |
|
|
94
|
+
| extractor-templates | templates | Document structures, config templates, diagram templates |
|
|
95
|
+
| extractor-snippets | snippets | Reusable code fragments, hooks, utility functions |
|
|
96
|
+
|
|
97
|
+
Each agent returns numbered findings with reusability confidence (0.0-1.0).
|
|
98
|
+
Orchestrator merges all findings into a single numbered list and writes to JSON:
|
|
99
|
+
`.keysarium/harvest/findings-{SESSION_ID}.json`
|
|
100
|
+
|
|
101
|
+
**Scope filtering:** If `{SCOPE_FILTER}` is set, only spawn matching agents. See mapping in `references/artifact-categories.md`.
|
|
102
|
+
|
|
103
|
+
### Step 2: Classify (Module 02)
|
|
104
|
+
|
|
105
|
+
```
|
|
106
|
+
Read: modules/02-classify.md
|
|
107
|
+
Read: references/artifact-categories.md
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
1. Read findings from JSON file
|
|
111
|
+
2. Assign each finding to one of 7 categories
|
|
112
|
+
3. Deduplicate: check content similarity across findings and against TOOLKIT_HARVEST.md
|
|
113
|
+
4. Cross-reference: link findings that reference existing skills/commands
|
|
114
|
+
5. Write classified findings back to JSON
|
|
115
|
+
|
|
116
|
+
### Step 3: User Checkpoint (Interactive)
|
|
117
|
+
|
|
118
|
+
Display all findings grouped by category as a numbered list:
|
|
119
|
+
|
|
120
|
+
```
|
|
121
|
+
═══════════════════════════════════════════════════════
|
|
122
|
+
📋 HARVEST FINDINGS: {N} items extracted
|
|
123
|
+
|
|
124
|
+
## patterns (K items)
|
|
125
|
+
#1 [0.85] Agent Swarm Topology — parallel agents with merge pattern
|
|
126
|
+
#4 [0.72] Event-Driven Pipeline — phase-to-phase data flow via files
|
|
127
|
+
|
|
128
|
+
## rules (M items)
|
|
129
|
+
#2 [0.91] Domain Detection Rule — auto-detect domain from keywords
|
|
130
|
+
#7 [0.65] Regulatory Constraint — ФЗ-152 data isolation requirement
|
|
131
|
+
|
|
132
|
+
## snippets (P items)
|
|
133
|
+
#3 [0.78] CJM Renderer — universal React component for CJM display
|
|
134
|
+
...
|
|
135
|
+
|
|
136
|
+
Commands:
|
|
137
|
+
• "убери #N" — remove finding
|
|
138
|
+
• "переклассифицируй #N в X" — reclassify to category X
|
|
139
|
+
• "доработай #N" — expand/improve finding
|
|
140
|
+
• "объедини #N и #M" — merge two findings
|
|
141
|
+
• "покажи только X" — filter by category
|
|
142
|
+
• "ок" — proceed to quality gates
|
|
143
|
+
═══════════════════════════════════════════════════════
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Apply user mutations to JSON file. Numbers are stable (removed items leave gaps).
|
|
147
|
+
Loop until user says "ок".
|
|
148
|
+
|
|
149
|
+
### Step 4: Gate (Module 03)
|
|
150
|
+
|
|
151
|
+
```
|
|
152
|
+
Read: modules/03-gate.md
|
|
153
|
+
Read: references/quality-gates.md
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Evaluate each active finding against 8 quality gates in 2 passes:
|
|
157
|
+
|
|
158
|
+
**Pass 1 — Deterministic (zero LLM cost):**
|
|
159
|
+
- G1: Has "When to use" section
|
|
160
|
+
- G2: Has "When NOT to use" section
|
|
161
|
+
- G5: Reusability confidence >= 0.5
|
|
162
|
+
- G6: Not a duplicate of existing toolkit entry
|
|
163
|
+
- G7: Has maturity label (alpha/beta/stable)
|
|
164
|
+
|
|
165
|
+
**Pass 2 — Semantic (haiku agent):**
|
|
166
|
+
- G3: Properly decontextualized (no project-specific refs)
|
|
167
|
+
- G4: Has at least one concrete example
|
|
168
|
+
- G8: Passes brutal-honesty review (self-critical assessment)
|
|
169
|
+
|
|
170
|
+
Findings passing all gates → status `approved`.
|
|
171
|
+
Findings failing any gate → status `blocked` with gate IDs.
|
|
172
|
+
|
|
173
|
+
Display blocked findings and allow overrides:
|
|
174
|
+
```
|
|
175
|
+
⚠️ BLOCKED: #7 — failed G3 (project-specific references remain)
|
|
176
|
+
• "пропусти G3 для #7" — override gate
|
|
177
|
+
• "ок" — accept blocks, proceed with approved only
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
### Step 5: Integrate (Module 04)
|
|
181
|
+
|
|
182
|
+
```
|
|
183
|
+
Read: modules/04-integrate.md
|
|
184
|
+
Read: templates/artifact-card.md
|
|
185
|
+
Read: templates/harvest-report.md
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
1. Render artifact cards for each approved finding
|
|
189
|
+
2. Auto-place by category:
|
|
190
|
+
- `skills` → create `.claude/skills/<name>/SKILL.md` (skeleton)
|
|
191
|
+
- `commands` → create `.claude/commands/<name>.md`
|
|
192
|
+
- `hooks` → document in TOOLKIT_HARVEST.md (hooks section)
|
|
193
|
+
- `rules` → create `.claude/rules/<name>.md`
|
|
194
|
+
- `templates` → document in TOOLKIT_HARVEST.md (templates section)
|
|
195
|
+
- `patterns` → document in TOOLKIT_HARVEST.md (patterns section)
|
|
196
|
+
- `snippets` → document in TOOLKIT_HARVEST.md (snippets section)
|
|
197
|
+
3. Update TOOLKIT_HARVEST.md "Обработанные проекты" table
|
|
198
|
+
4. Generate harvest report
|
|
199
|
+
5. Call `memory_store()` with harvest results and user reward
|
|
200
|
+
6. Update `.keysarium/insights/trigger-state.json` (case_completion event)
|
|
201
|
+
7. Clean up findings JSON file
|
|
202
|
+
|
|
203
|
+
## Scope Filtering
|
|
204
|
+
|
|
205
|
+
The `only` keyword filters which extractor agents are spawned:
|
|
206
|
+
|
|
207
|
+
```
|
|
208
|
+
/harvest path/ → all 5 agents
|
|
209
|
+
/harvest path/ only patterns → extractor-patterns only
|
|
210
|
+
/harvest path/ only rules,templates → extractor-rules + extractor-templates
|
|
211
|
+
/harvest all only snippets → extractor-snippets for each directory
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Category-to-agent mapping (see `references/artifact-categories.md`):
|
|
215
|
+
- patterns → extractor-patterns
|
|
216
|
+
- commands → extractor-commands
|
|
217
|
+
- hooks → extractor-commands (shared lens)
|
|
218
|
+
- rules → extractor-rules
|
|
219
|
+
- templates → extractor-templates
|
|
220
|
+
- snippets → extractor-snippets
|
|
221
|
+
- skills → all agents (skills are cross-cutting)
|
|
222
|
+
|
|
223
|
+
## Integration Points
|
|
224
|
+
|
|
225
|
+
| System | When | Operation |
|
|
226
|
+
|--------|------|-----------|
|
|
227
|
+
| memory_query() | Pipeline start | Load historical harvest patterns |
|
|
228
|
+
| memory_store() | After user checkpoint | Store harvest reward data |
|
|
229
|
+
| brain-export | N/A (passive) | Findings JSON is brain-compatible schema |
|
|
230
|
+
| BTO | Post-harvest | Placed skills can be evaluated via `/bto-test` |
|
|
231
|
+
| dream cycles | Pipeline end | Update trigger-state.json |
|
|
232
|
+
| /harvest command | Entry point | Thin wrapper loads this skill |
|
|
233
|
+
|
|
234
|
+
## Anti-Patterns
|
|
235
|
+
|
|
236
|
+
| Anti-Pattern | Detection | Fix |
|
|
237
|
+
|-------------|-----------|-----|
|
|
238
|
+
| Extracting project-specific details | Finding contains project names, slugs, dates | Decontextualize: replace specifics with generic placeholders |
|
|
239
|
+
| Duplicate extraction | content_hash matches existing TOOLKIT_HARVEST.md entry | Skip or merge; gate G6 blocks duplicates |
|
|
240
|
+
| Extracting from incomplete phases | Artifact lacks promise tag or checkpoint | Only harvest from completed projects |
|
|
241
|
+
| Over-classifying snippets as patterns | Single-use code without abstraction | Require reuse evidence before promoting to pattern |
|
|
242
|
+
| Under-specifying maturity | All findings marked "stable" on first run | First extraction is always "alpha" or "beta" |
|
|
243
|
+
| Skipping user checkpoint | Proceeding to gates without user review | BLOCK — user checkpoint is mandatory |
|
|
244
|
+
| Running all agents when scope is filtered | `only patterns` but 5 agents spawned | Check SCOPE_FILTER before spawning |
|
|
245
|
+
|
|
246
|
+
## Dependencies
|
|
247
|
+
|
|
248
|
+
| Resource | Path | Purpose |
|
|
249
|
+
|----------|------|---------|
|
|
250
|
+
| quality-gates.md | references/quality-gates.md | Gate definitions for Module 03 |
|
|
251
|
+
| artifact-categories.md | references/artifact-categories.md | 7-category taxonomy |
|
|
252
|
+
| maturity-model.md | references/maturity-model.md | Alpha/Beta/Stable criteria |
|
|
253
|
+
| artifact-card.md | templates/artifact-card.md | Per-finding output format |
|
|
254
|
+
| harvest-report.md | templates/harvest-report.md | Session report format |
|
|
@@ -0,0 +1,188 @@
|
|
|
1
|
+
# Extract Module — Parallel Knowledge Extraction
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Spawn 5 focused extractor agents to scan a project directory through different lenses, collecting reusable knowledge findings with confidence scores.
|
|
6
|
+
|
|
7
|
+
## Input
|
|
8
|
+
|
|
9
|
+
- `{TARGET_PATH}` — directory to scan (e.g., `researches/bank-kc-automation/`)
|
|
10
|
+
- `{SCOPE_FILTER}` — optional list of lenses to activate (null = all 5)
|
|
11
|
+
- `{MEMORY_PATTERNS}` — historical patterns from memory_query() (may be empty)
|
|
12
|
+
- `{SESSION_ID}` — harvest session identifier
|
|
13
|
+
|
|
14
|
+
## Protocol
|
|
15
|
+
|
|
16
|
+
### Step 1: Inventory Target Directory
|
|
17
|
+
|
|
18
|
+
List all files in `{TARGET_PATH}`. Build a file manifest with paths and sizes.
|
|
19
|
+
Skip binary files, images, and files > 100KB.
|
|
20
|
+
|
|
21
|
+
### Step 2: Determine Active Agents
|
|
22
|
+
|
|
23
|
+
If `{SCOPE_FILTER}` is set, map categories to lenses:
|
|
24
|
+
|
|
25
|
+
| Requested Category | Agent Lens to Spawn |
|
|
26
|
+
|-------------------|---------------------|
|
|
27
|
+
| patterns | extractor-patterns |
|
|
28
|
+
| commands | extractor-commands |
|
|
29
|
+
| hooks | extractor-commands (shared) |
|
|
30
|
+
| rules | extractor-rules |
|
|
31
|
+
| templates | extractor-templates |
|
|
32
|
+
| snippets | extractor-snippets |
|
|
33
|
+
| skills | ALL agents (cross-cutting) |
|
|
34
|
+
|
|
35
|
+
If `{SCOPE_FILTER}` is null, spawn all 5 agents.
|
|
36
|
+
|
|
37
|
+
### Step 3: Spawn Extractor Agents
|
|
38
|
+
|
|
39
|
+
Spawn selected agents in parallel using the Agent tool:
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
Agent(
|
|
43
|
+
subagent_type="general-purpose",
|
|
44
|
+
model="sonnet",
|
|
45
|
+
description="Harvest Extractor — {LENS_NAME}",
|
|
46
|
+
prompt="""
|
|
47
|
+
You are a knowledge extraction agent with a focused lens: {LENS_DESCRIPTION}.
|
|
48
|
+
|
|
49
|
+
## Your Task
|
|
50
|
+
Scan the following project files and extract reusable knowledge through your lens.
|
|
51
|
+
For each finding, provide:
|
|
52
|
+
1. A short descriptive title (3-8 words)
|
|
53
|
+
2. Content: the reusable knowledge (decontextualized where possible)
|
|
54
|
+
3. "When to use" guidance (1-2 sentences)
|
|
55
|
+
4. "When NOT to use" guidance (1-2 sentences)
|
|
56
|
+
5. A concrete example of usage
|
|
57
|
+
6. Reusability confidence (0.0 to 1.0):
|
|
58
|
+
- 0.9-1.0: Universal, works in any project
|
|
59
|
+
- 0.7-0.89: Works in most similar projects
|
|
60
|
+
- 0.5-0.69: Works in some contexts
|
|
61
|
+
- < 0.5: Probably too project-specific
|
|
62
|
+
7. Suggested maturity: alpha (first time seen) or beta (if evidence of reuse)
|
|
63
|
+
|
|
64
|
+
## Project Files
|
|
65
|
+
Read the following directory: {TARGET_PATH}
|
|
66
|
+
|
|
67
|
+
## Historical Patterns (from previous harvests)
|
|
68
|
+
{MEMORY_PATTERNS_OR_NONE}
|
|
69
|
+
|
|
70
|
+
Avoid extracting findings that overlap with these historical patterns.
|
|
71
|
+
|
|
72
|
+
## Extraction Rules
|
|
73
|
+
- Extract ONLY genuinely reusable knowledge
|
|
74
|
+
- DO NOT extract project-specific implementation details
|
|
75
|
+
- DO NOT extract trivial or obvious patterns
|
|
76
|
+
- Each finding must be independently useful outside this project
|
|
77
|
+
- Prefer actionable knowledge over abstract observations
|
|
78
|
+
- If in doubt about reusability, include it with a lower confidence score
|
|
79
|
+
|
|
80
|
+
## Output Format
|
|
81
|
+
Return a numbered list of findings in this exact format:
|
|
82
|
+
|
|
83
|
+
### Finding 1
|
|
84
|
+
**Title:** [short title]
|
|
85
|
+
**Confidence:** [0.0-1.0]
|
|
86
|
+
**Maturity:** [alpha|beta]
|
|
87
|
+
**Content:** [the reusable knowledge]
|
|
88
|
+
**When to use:** [guidance]
|
|
89
|
+
**When NOT to use:** [anti-guidance]
|
|
90
|
+
**Example:** [concrete example]
|
|
91
|
+
|
|
92
|
+
### Finding 2
|
|
93
|
+
...
|
|
94
|
+
"""
|
|
95
|
+
)
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
### Step 4: Handle Agent Failures
|
|
99
|
+
|
|
100
|
+
If any extractor agent fails, times out, or returns malformed output:
|
|
101
|
+
|
|
102
|
+
1. **Partial success:** Proceed with results from successful agents. Do NOT block the entire pipeline.
|
|
103
|
+
2. **Log failures:** Record which lenses failed and why:
|
|
104
|
+
```
|
|
105
|
+
⚠️ Extractor agent failed: extractor-snippets (timeout after 120s)
|
|
106
|
+
Proceeding with 4/5 lenses. Snippets will not be harvested.
|
|
107
|
+
```
|
|
108
|
+
3. **Surface in report:** Add failed lenses to the harvest report's "Coverage Gaps" section.
|
|
109
|
+
4. **Retry option:** Offer the user: `"повтори snippets"` — retry the failed lens only.
|
|
110
|
+
5. **Malformed output:** If an agent returns output that cannot be parsed into the Finding format, discard that agent's results and log: `"Extractor {lens} returned unparseable output — discarded."`
|
|
111
|
+
6. **Zero findings:** If an agent returns no findings, this is normal (not all lenses find relevant content). Log: `"Extractor {lens}: 0 findings (no relevant content detected)."`
|
|
112
|
+
|
|
113
|
+
### Step 5: Report Skipped Files
|
|
114
|
+
|
|
115
|
+
Before merging, surface any files that were skipped during inventory:
|
|
116
|
+
|
|
117
|
+
```
|
|
118
|
+
📋 File inventory: {total_files} files scanned, {skipped_count} skipped
|
|
119
|
+
Skipped: {file1} (112KB, exceeds 100KB limit), {file2} (binary)
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
Include this in the harvest report under "Coverage Gaps" so the user knows what was NOT examined.
|
|
123
|
+
|
|
124
|
+
### Step 6: Merge Results
|
|
125
|
+
|
|
126
|
+
After all agents complete (or after handling failures):
|
|
127
|
+
|
|
128
|
+
1. Collect all findings from all agents
|
|
129
|
+
2. Assign sequential numbers starting from 1 (global numbering across all agents)
|
|
130
|
+
3. Tag each finding with its source lens
|
|
131
|
+
4. Set initial status to `raw`
|
|
132
|
+
5. Write to findings JSON file at `.keysarium/harvest/findings-{SESSION_ID}.json`
|
|
133
|
+
|
|
134
|
+
### 5 Agent Lens Descriptions
|
|
135
|
+
|
|
136
|
+
| Agent | Lens Description |
|
|
137
|
+
|-------|-----------------|
|
|
138
|
+
| extractor-patterns | Architecture patterns, design patterns, agent topologies, data flow patterns, orchestration strategies, parallelism approaches. Look for structural solutions that could be applied to other systems. |
|
|
139
|
+
| extractor-commands | CLI commands, scripts, pipeline stages, automation workflows, slash commands, build processes. Look for reusable command patterns and workflow automations. |
|
|
140
|
+
| extractor-rules | Constraints, quality gates, anti-patterns, domain rules, naming conventions, regulatory requirements, lessons learned. Look for guardrails and restrictions that prevent mistakes. |
|
|
141
|
+
| extractor-templates | Document structures, config file formats, diagram templates, report formats, checklist templates. Look for reusable document and configuration scaffolding. |
|
|
142
|
+
| extractor-snippets | Reusable code fragments, utility functions, React components, bash one-liners, prompt templates, hook implementations. Look for copy-paste-ready code blocks. |
|
|
143
|
+
|
|
144
|
+
## Output Format
|
|
145
|
+
|
|
146
|
+
The findings JSON file structure:
|
|
147
|
+
|
|
148
|
+
```json
|
|
149
|
+
{
|
|
150
|
+
"session_id": "{SESSION_ID}",
|
|
151
|
+
"status": "extracting",
|
|
152
|
+
"target_paths": ["{TARGET_PATH}"],
|
|
153
|
+
"scope_filter": null,
|
|
154
|
+
"created_at": "ISO-8601",
|
|
155
|
+
"next_number": 16,
|
|
156
|
+
"extractors_spawned": ["patterns", "commands", "rules", "templates", "snippets"],
|
|
157
|
+
"findings": {
|
|
158
|
+
"1": {
|
|
159
|
+
"number": 1,
|
|
160
|
+
"title": "Agent Swarm Topology",
|
|
161
|
+
"content": "...",
|
|
162
|
+
"when_to_use": "...",
|
|
163
|
+
"when_not_to_use": "...",
|
|
164
|
+
"example": "...",
|
|
165
|
+
"confidence": 0.85,
|
|
166
|
+
"maturity": "alpha",
|
|
167
|
+
"source_lens": "patterns",
|
|
168
|
+
"category": null,
|
|
169
|
+
"status": "raw",
|
|
170
|
+
"gate_results": {},
|
|
171
|
+
"gate_overrides": [],
|
|
172
|
+
"merged_from": null
|
|
173
|
+
}
|
|
174
|
+
}
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Anti-Patterns
|
|
179
|
+
|
|
180
|
+
| Anti-Pattern | Detection | Fix |
|
|
181
|
+
|-------------|-----------|-----|
|
|
182
|
+
| Agent extracts trivial findings | Findings like "use git for version control" | Prompt emphasizes non-obvious, actionable knowledge |
|
|
183
|
+
| Agent reads files outside target | File paths outside {TARGET_PATH} | Prompt constrains to target directory only |
|
|
184
|
+
| Duplicate findings across agents | Two agents extract the same pattern | Handled by Module 02 (dedup), not here |
|
|
185
|
+
| Agent returns unstructured text | Missing required fields in output | Strict output format in prompt; orchestrator validates |
|
|
186
|
+
| All findings have confidence 0.9+ | Agent is not being self-critical | Prompt calibration: explain the confidence scale |
|
|
187
|
+
| Agent timeout with no fallback | Pipeline stalls waiting for failed agent | Use Step 4 failure protocol: proceed with partial results |
|
|
188
|
+
| Large files silently skipped | User unaware of coverage gaps | Step 5 surfaces skipped files; harvest report includes coverage gaps |
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
# Classify Module — 7-Category Classification and Deduplication
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Assign each raw finding to one of 7 categories, deduplicate across agents, and cross-reference against existing toolkit entries.
|
|
6
|
+
|
|
7
|
+
## Input
|
|
8
|
+
|
|
9
|
+
- Findings JSON file (from Module 01, status: `extracting` → set to `classifying`)
|
|
10
|
+
- `TOOLKIT_HARVEST.md` — existing entries to check for duplicates
|
|
11
|
+
- `references/artifact-categories.md` — category definitions and destination mapping
|
|
12
|
+
|
|
13
|
+
## Protocol
|
|
14
|
+
|
|
15
|
+
### Step 1: Load and Validate
|
|
16
|
+
|
|
17
|
+
1. Read findings JSON file
|
|
18
|
+
2. Verify all findings have status `raw`
|
|
19
|
+
3. Set session status to `classifying`
|
|
20
|
+
4. Read `references/artifact-categories.md` for category definitions
|
|
21
|
+
|
|
22
|
+
### Step 2: Classify Each Finding
|
|
23
|
+
|
|
24
|
+
For each finding, assign a category based on content analysis:
|
|
25
|
+
|
|
26
|
+
| Category | Classification Signals |
|
|
27
|
+
|----------|----------------------|
|
|
28
|
+
| skills | Multi-step methodology, complete workflow, orchestrator + modules pattern |
|
|
29
|
+
| commands | Starts with `/` or describes a CLI invocation, has arguments/parameters |
|
|
30
|
+
| hooks | Triggered by an event, pre/post pattern, automation on lifecycle events |
|
|
31
|
+
| rules | Contains "must", "never", "always", constraints, anti-pattern tables |
|
|
32
|
+
| templates | Document skeleton, config file format, placeholder structure `{VAR}` |
|
|
33
|
+
| patterns | Architectural approach, design strategy, topology, reusable structure |
|
|
34
|
+
| snippets | Code block, function, component, < 50 lines, copy-paste ready |
|
|
35
|
+
|
|
36
|
+
**Tie-breaking rules:**
|
|
37
|
+
- If a finding could be both `patterns` and `skills` → prefer `patterns` unless it has modules/sub-components
|
|
38
|
+
- If a finding could be both `rules` and `patterns` → prefer `rules` if it's primarily a constraint
|
|
39
|
+
- If a finding could be both `templates` and `snippets` → prefer `templates` if it has placeholders, `snippets` if it's code
|
|
40
|
+
|
|
41
|
+
### Step 3: Deduplicate
|
|
42
|
+
|
|
43
|
+
**Cross-agent dedup:** Compare all findings pairwise within the session using this algorithm:
|
|
44
|
+
|
|
45
|
+
1. **Title similarity:** Normalize titles (lowercase, strip punctuation, remove stop words). Compute token overlap ratio: `shared_tokens / max(tokens_A, tokens_B)`.
|
|
46
|
+
2. **Content similarity:** Extract the first 3 non-empty lines of each finding's `content` field. Compute 3-gram set overlap (Jaccard index): `|A ∩ B| / |A ∪ B|`.
|
|
47
|
+
3. **Combined score:** `similarity = title_weight * title_sim + content_weight * content_sim` where `title_weight = 0.4`, `content_weight = 0.6`.
|
|
48
|
+
4. **Threshold:** If `similarity >= 0.80` → duplicate. Keep the finding with higher confidence. Set the other's status to `removed`.
|
|
49
|
+
5. Log: `"Dedup: #N removed (duplicate of #M, similarity: {score})"`
|
|
50
|
+
|
|
51
|
+
Note: Since an LLM is performing this comparison (not a deterministic function), the algorithm above provides guidance for consistent judgment. The orchestrator should evaluate each pair and apply the threshold consistently.
|
|
52
|
+
|
|
53
|
+
**Cross-toolkit dedup:** Compare findings against existing `TOOLKIT_HARVEST.md` entries:
|
|
54
|
+
1. For each finding, scan TOOLKIT_HARVEST.md section headings and first paragraphs.
|
|
55
|
+
2. If a finding's title matches an existing entry name (exact or near-match after normalization):
|
|
56
|
+
- Set finding's `duplicate_of` field to the existing entry name
|
|
57
|
+
- Flag for gate G6 (will be evaluated in Module 03)
|
|
58
|
+
3. Near-duplicate range (0.70-0.85 similarity): propose a MERGE instead of flagging as duplicate.
|
|
59
|
+
|
|
60
|
+
### Step 4: Cross-Reference
|
|
61
|
+
|
|
62
|
+
For each classified finding, check if it references or depends on existing toolkit items:
|
|
63
|
+
- If finding mentions a skill name → add cross-reference link
|
|
64
|
+
- If finding extends an existing pattern → note the relationship
|
|
65
|
+
- Record cross-references in the finding's metadata
|
|
66
|
+
|
|
67
|
+
### Step 5: Write Results
|
|
68
|
+
|
|
69
|
+
1. Update each finding in the JSON file with: `category`, `status: "classified"`
|
|
70
|
+
2. Update session status
|
|
71
|
+
3. Return classification summary for display
|
|
72
|
+
|
|
73
|
+
## Output Format
|
|
74
|
+
|
|
75
|
+
The orchestrator displays classified findings grouped by category:
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
## patterns ({N} items)
|
|
79
|
+
#1 [0.85] Agent Swarm Topology — parallel agents with merge pattern
|
|
80
|
+
#4 [0.72] Event-Driven Pipeline — phase-to-phase data flow via files
|
|
81
|
+
|
|
82
|
+
## rules ({N} items)
|
|
83
|
+
#2 [0.91] Domain Detection Rule — auto-detect domain from keywords
|
|
84
|
+
#7 [0.65] Regulatory Constraint — ФЗ-152 data isolation requirement
|
|
85
|
+
|
|
86
|
+
## commands ({N} items)
|
|
87
|
+
...
|
|
88
|
+
|
|
89
|
+
## Removed by dedup: #5 (duplicate of #1), #9 (duplicate of #3)
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## Anti-Patterns
|
|
93
|
+
|
|
94
|
+
| Anti-Pattern | Detection | Fix |
|
|
95
|
+
|-------------|-----------|-----|
|
|
96
|
+
| Everything classified as "patterns" | > 50% of findings in one category | Re-evaluate with stricter classification signals |
|
|
97
|
+
| Aggressive dedup removes valid findings | High similarity but different use cases | Compare "When to use" sections, not just content |
|
|
98
|
+
| Missing cross-references | Finding extends existing tool but no link | Check TOOLKIT_HARVEST.md entry names explicitly |
|
|
99
|
+
| Category mismatch with source lens | extractor-rules finding classified as pattern | Lens is a hint, not a binding — classify on content |
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# Gate Module — 8 Quality Gates with 2-Pass Evaluation
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Evaluate each active finding against 8 quality gates using a two-pass approach: deterministic checks first (zero LLM cost), then semantic checks via haiku agent. Set finding status to `approved` or `blocked`.
|
|
6
|
+
|
|
7
|
+
## Input
|
|
8
|
+
|
|
9
|
+
- Findings JSON file (from user checkpoint, active findings only)
|
|
10
|
+
- `references/quality-gates.md` — gate definitions, pass criteria, failure messages
|
|
11
|
+
|
|
12
|
+
## Protocol
|
|
13
|
+
|
|
14
|
+
### Step 1: Load Gate Definitions
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
Read: references/quality-gates.md
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Load all 8 gate definitions with their type (deterministic/semantic) and pass criteria.
|
|
21
|
+
|
|
22
|
+
### Step 2: Filter Active Findings
|
|
23
|
+
|
|
24
|
+
From the findings JSON, select only findings with status `classified` (not `removed` or `merged`).
|
|
25
|
+
These are the findings that survived the user checkpoint.
|
|
26
|
+
|
|
27
|
+
### Step 3: Pass 1 — Deterministic Gates
|
|
28
|
+
|
|
29
|
+
For each active finding, evaluate deterministic gates. These are simple checks that require no LLM:
|
|
30
|
+
|
|
31
|
+
| Gate | Check Logic |
|
|
32
|
+
|------|------------|
|
|
33
|
+
| G1 | `finding.when_to_use` exists AND length > 10 characters |
|
|
34
|
+
| G2 | `finding.when_not_to_use` exists AND length > 10 characters |
|
|
35
|
+
| G5 | `finding.confidence >= 0.5` |
|
|
36
|
+
| G6 | `finding.duplicate_of` is null OR empty (set by Module 02) |
|
|
37
|
+
| G7 | `finding.maturity` is one of: `alpha`, `beta`, `stable` |
|
|
38
|
+
|
|
39
|
+
For each gate, record the result:
|
|
40
|
+
```json
|
|
41
|
+
{
|
|
42
|
+
"gate_id": "G1",
|
|
43
|
+
"passed": true,
|
|
44
|
+
"evaluation_pass": "deterministic",
|
|
45
|
+
"reason": null
|
|
46
|
+
}
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
If any deterministic gate fails, record the failure reason from `quality-gates.md`.
|
|
50
|
+
|
|
51
|
+
### Step 4: Pass 2 — Semantic Gates
|
|
52
|
+
|
|
53
|
+
For findings that passed all deterministic gates (or for all findings if you want complete gate data), evaluate semantic gates using a haiku agent:
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
Agent(
|
|
57
|
+
subagent_type="general-purpose",
|
|
58
|
+
model="haiku",
|
|
59
|
+
description="Harvest Gate Evaluator — semantic gates",
|
|
60
|
+
prompt="""
|
|
61
|
+
You are a quality gate evaluator. For each finding below, evaluate 3 semantic gates.
|
|
62
|
+
Be strict but fair. Err on the side of blocking rather than passing questionable findings.
|
|
63
|
+
|
|
64
|
+
## Gates to Evaluate
|
|
65
|
+
|
|
66
|
+
**G3 — Decontextualized:**
|
|
67
|
+
Check if the finding contains project-specific references: variable names, file paths,
|
|
68
|
+
company names, dates, slugs, or any hardcoded values tied to the source project.
|
|
69
|
+
PASS if the finding is generic and reusable as-is.
|
|
70
|
+
FAIL if any project-specific detail remains.
|
|
71
|
+
|
|
72
|
+
**G4 — Concrete Example:**
|
|
73
|
+
Check if the finding includes at least one concrete, actionable example.
|
|
74
|
+
The example must show actual usage, not just describe it abstractly.
|
|
75
|
+
PASS if there is a real example with specifics.
|
|
76
|
+
FAIL if the example is vague, abstract, or missing.
|
|
77
|
+
|
|
78
|
+
**G8 — Brutal Honesty Review:**
|
|
79
|
+
Ask yourself: "Would a senior engineer actually find this useful in a different project?"
|
|
80
|
+
PASS only if the answer is genuinely yes — the finding adds real value.
|
|
81
|
+
FAIL if the finding is obvious, trivial, or too niche to be broadly useful.
|
|
82
|
+
|
|
83
|
+
## Findings to Evaluate
|
|
84
|
+
|
|
85
|
+
{FINDINGS_LIST}
|
|
86
|
+
|
|
87
|
+
## Required Output Format
|
|
88
|
+
|
|
89
|
+
For each finding, return:
|
|
90
|
+
|
|
91
|
+
Finding #N:
|
|
92
|
+
- G3: PASS/FAIL — [reason]
|
|
93
|
+
- G4: PASS/FAIL — [reason]
|
|
94
|
+
- G8: PASS/FAIL — [reason]
|
|
95
|
+
"""
|
|
96
|
+
)
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
### Step 5: Set Approval Status
|
|
100
|
+
|
|
101
|
+
For each finding:
|
|
102
|
+
- If ALL 8 gates passed → status = `approved`
|
|
103
|
+
- If ANY gate failed → status = `blocked`, record failing gate IDs
|
|
104
|
+
|
|
105
|
+
Write results to findings JSON.
|
|
106
|
+
|
|
107
|
+
### Step 6: Report Blocked Findings
|
|
108
|
+
|
|
109
|
+
The orchestrator displays blocked findings to the user:
|
|
110
|
+
|
|
111
|
+
```
|
|
112
|
+
✅ APPROVED: {N} findings passed all gates
|
|
113
|
+
|
|
114
|
+
⚠️ BLOCKED: {M} findings failed quality gates:
|
|
115
|
+
#7 — G3 (project-specific references remain: mentions "bank-kc-automation")
|
|
116
|
+
#12 — G5 (confidence 0.35 < 0.5 threshold)
|
|
117
|
+
#14 — G4, G8 (no concrete example; fails brutal-honesty review)
|
|
118
|
+
|
|
119
|
+
Commands:
|
|
120
|
+
• "пропусти GN для #X" — override specific gate for specific finding
|
|
121
|
+
• "ок" — accept blocks, proceed with approved only
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
### Step 7: Apply Overrides (if any)
|
|
125
|
+
|
|
126
|
+
If user overrides a gate:
|
|
127
|
+
1. Add gate ID to finding's `gate_overrides` array
|
|
128
|
+
2. Recalculate: if all non-overridden gates pass → status = `approved`
|
|
129
|
+
3. Log override for audit trail
|
|
130
|
+
|
|
131
|
+
## Override Protocol
|
|
132
|
+
|
|
133
|
+
User command: `"пропусти G3 для #7"` (override G3 for finding #7)
|
|
134
|
+
|
|
135
|
+
Validation:
|
|
136
|
+
- Finding #7 must exist and be `blocked`
|
|
137
|
+
- G3 must be one of the failing gates for finding #7
|
|
138
|
+
- **G6 (Not Duplicate) and G8 (Brutal Honesty) CANNOT be overridden** — reject override attempts for these gates
|
|
139
|
+
- Record the override with timestamp
|
|
140
|
+
|
|
141
|
+
Multiple overrides: `"пропусти G3,G4 для #14"` — override multiple gates at once.
|
|
142
|
+
|
|
143
|
+
## Anti-Patterns
|
|
144
|
+
|
|
145
|
+
| Anti-Pattern | Detection | Fix |
|
|
146
|
+
|-------------|-----------|-----|
|
|
147
|
+
| All findings pass all gates | 100% pass rate | Haiku semantic evaluator may be too lenient — add calibration examples |
|
|
148
|
+
| Gate G8 blocks everything | > 80% fail on G8 | Review extraction quality — if findings are genuinely weak, that's correct |
|
|
149
|
+
| Overriding all blocked gates | User overrides every block | Advisory: suggest improving extraction rather than overriding all gates |
|
|
150
|
+
| Semantic evaluator contradicts itself | G3 passes but content has project refs | Log discrepancy, flag for human review |
|