@dzhechkov/keysarium 1.3.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -9
- package/package.json +1 -1
- package/src/utils.js +1 -1
- package/templates/.claude/commands/brain-export.md +2 -0
- package/templates/.claude/commands/bto-benchmark.md +302 -0
- package/templates/.claude/commands/harvest.md +28 -46
- package/templates/.claude/rules/trust-tiers.md +1 -0
- package/templates/.claude/skills/knowledge-extractor/SKILL.md +254 -0
- package/templates/.claude/skills/knowledge-extractor/modules/01-extract.md +188 -0
- package/templates/.claude/skills/knowledge-extractor/modules/02-classify.md +99 -0
- package/templates/.claude/skills/knowledge-extractor/modules/03-gate.md +150 -0
- package/templates/.claude/skills/knowledge-extractor/modules/04-integrate.md +159 -0
- package/templates/.claude/skills/knowledge-extractor/references/artifact-categories.md +100 -0
- package/templates/.claude/skills/knowledge-extractor/references/maturity-model.md +94 -0
- package/templates/.claude/skills/knowledge-extractor/references/quality-gates.md +98 -0
- package/templates/.claude/skills/knowledge-extractor/templates/artifact-card.md +63 -0
- package/templates/.claude/skills/knowledge-extractor/templates/harvest-report.md +102 -0
- package/templates/CLAUDE.md +22 -8
- package/templates/docs/continuous-learning-guide.md +622 -0
- package/templates/docs/improvement-directions.md +8 -4
- package/templates/docs/keysarium-core/02-functionality.md +3 -1
- package/templates/docs/skills-bto/01-deployment-guide.md +1 -1
- package/templates/docs/skills-bto/02-admin-guide.md +4 -1
- package/templates/docs/skills-bto/03-user-guide.md +4 -3
- package/templates/docs/skills-bto/04-infrastructure-requirements.md +2 -1
- package/templates/docs/skills-bto/05-architecture.md +11 -11
- package/templates/docs/skills-bto/06-user-flows.md +2 -1
- package/templates/lib/dream-engine.md +1 -1
- package/templates/lib/memory-protocol.md +6 -2
- package/templates/lib/reward-tracker.md +6 -3
|
@@ -0,0 +1,159 @@
|
|
|
1
|
+
# Integrate Module — Auto-Placement and Harvest Report
|
|
2
|
+
|
|
3
|
+
## Purpose
|
|
4
|
+
|
|
5
|
+
Place approved findings into their target directories as artifact cards, update TOOLKIT_HARVEST.md, generate the harvest report, and trigger post-harvest integrations (memory_store, dream cycles).
|
|
6
|
+
|
|
7
|
+
## Input
|
|
8
|
+
|
|
9
|
+
- Findings JSON file (from Module 03, approved findings)
|
|
10
|
+
- `templates/artifact-card.md` — per-finding output template
|
|
11
|
+
- `templates/harvest-report.md` — session report template
|
|
12
|
+
- `TOOLKIT_HARVEST.md` — existing toolkit tracker
|
|
13
|
+
|
|
14
|
+
## Protocol
|
|
15
|
+
|
|
16
|
+
### Step 1: Load Approved Findings
|
|
17
|
+
|
|
18
|
+
1. Read findings JSON file
|
|
19
|
+
2. Filter findings with status `approved`
|
|
20
|
+
3. Set session status to `integrating`
|
|
21
|
+
4. Load artifact card template and harvest report template
|
|
22
|
+
|
|
23
|
+
### Step 2: Render Artifact Cards
|
|
24
|
+
|
|
25
|
+
For each approved finding, render the artifact card by filling template placeholders:
|
|
26
|
+
|
|
27
|
+
| Placeholder | Source |
|
|
28
|
+
|-------------|--------|
|
|
29
|
+
| `{NAME}` | finding.title |
|
|
30
|
+
| `{CATEGORY}` | finding.category |
|
|
31
|
+
| `{MATURITY_BADGE}` | Lookup from maturity-model.md: alpha=🔴, beta=🟡, stable=🟢 |
|
|
32
|
+
| `{MATURITY_LEVEL}` | finding.maturity (capitalized) |
|
|
33
|
+
| `{CONFIDENCE}` | finding.confidence |
|
|
34
|
+
| `{SOURCE_PROJECT}` | session.slug |
|
|
35
|
+
| `{DATE}` | current date (YYYY-MM-DD) |
|
|
36
|
+
| `{DESCRIPTION}` | finding.content |
|
|
37
|
+
| `{WHEN_TO_USE}` | finding.when_to_use |
|
|
38
|
+
| `{WHEN_NOT_TO_USE}` | finding.when_not_to_use |
|
|
39
|
+
| `{EXAMPLE}` | finding.example |
|
|
40
|
+
|
|
41
|
+
### Step 3: Auto-Place by Category
|
|
42
|
+
|
|
43
|
+
Place each rendered artifact card according to its category:
|
|
44
|
+
|
|
45
|
+
| Category | Placement Action |
|
|
46
|
+
|----------|-----------------|
|
|
47
|
+
| **skills** | Create `.claude/skills/{name}/SKILL.md` with the artifact card as initial skeleton. Add frontmatter (trust_tier: 0, trust_tier_label: "Advisory"). |
|
|
48
|
+
| **commands** | Create `.claude/commands/{name}.md` with `$ARGUMENTS` parameter reference and the artifact card content as the command body. |
|
|
49
|
+
| **hooks** | Append artifact card to TOOLKIT_HARVEST.md under `## Хуки` section. Create the section if it doesn't exist. |
|
|
50
|
+
| **rules** | Create `.claude/rules/{name}.md` with the artifact card content formatted as a rule table (anti-pattern detection format). |
|
|
51
|
+
| **templates** | Append artifact card to TOOLKIT_HARVEST.md under `## Шаблоны` section. |
|
|
52
|
+
| **patterns** | Append artifact card to TOOLKIT_HARVEST.md under `## Паттерны` section. |
|
|
53
|
+
| **snippets** | Append artifact card to TOOLKIT_HARVEST.md under `## Сниппеты` section. |
|
|
54
|
+
|
|
55
|
+
**Naming convention:** Convert finding title to kebab-case for file/directory names. Max 40 characters.
|
|
56
|
+
|
|
57
|
+
**Collision handling:** If a file already exists at the target path:
|
|
58
|
+
- Do NOT overwrite
|
|
59
|
+
- Log: "Skipped placement for #{N} — target already exists: {path}"
|
|
60
|
+
- Set finding status to `approved` (not `integrated`) and flag for manual review
|
|
61
|
+
|
|
62
|
+
### Step 4: Update TOOLKIT_HARVEST.md
|
|
63
|
+
|
|
64
|
+
After placing all artifacts:
|
|
65
|
+
|
|
66
|
+
1. Update the "Обработанные проекты" table:
|
|
67
|
+
```markdown
|
|
68
|
+
| Проект | Дата harvest | Извлечено артефактов |
|
|
69
|
+
|--------|-------------|---------------------|
|
|
70
|
+
| {slug} | {date} | {approved_count} approved / {total_count} total |
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
2. Mark checked items in the relevant sections (skills, patterns, commands, etc.)
|
|
74
|
+
|
|
75
|
+
### Step 5: Generate Harvest Report
|
|
76
|
+
|
|
77
|
+
Fill the harvest report template with computed metrics:
|
|
78
|
+
|
|
79
|
+
| Metric | Computation |
|
|
80
|
+
|--------|-------------|
|
|
81
|
+
| TOTAL_EXTRACTED | Count of all findings at creation |
|
|
82
|
+
| AFTER_DEDUP | Count after Module 02 dedup |
|
|
83
|
+
| USER_REMOVALS | Count where status = `removed` AND not from dedup |
|
|
84
|
+
| USER_MERGES | Count where merged_from is not null |
|
|
85
|
+
| APPROVED_COUNT | Count where status = `approved` or `integrated` |
|
|
86
|
+
| BLOCKED_COUNT | Count where status = `blocked` |
|
|
87
|
+
| OVERRIDE_COUNT | Sum of all gate_overrides arrays |
|
|
88
|
+
| PLACED_COUNT | Count where status = `integrated` |
|
|
89
|
+
|
|
90
|
+
Display the report to the user.
|
|
91
|
+
|
|
92
|
+
### Step 6: Post-Harvest Integration
|
|
93
|
+
|
|
94
|
+
1. **memory_store()** — Store harvest results:
|
|
95
|
+
```json
|
|
96
|
+
{
|
|
97
|
+
"case_slug": "{slug}",
|
|
98
|
+
"domain": "{domain}",
|
|
99
|
+
"phase": "harvest",
|
|
100
|
+
"skill_used": "knowledge-extractor",
|
|
101
|
+
"reward": "{user_reward}",
|
|
102
|
+
"reward_label": "{label}",
|
|
103
|
+
"context": {
|
|
104
|
+
"patterns_loaded": "{count}",
|
|
105
|
+
"extractors_spawned": "{lens_list}",
|
|
106
|
+
"scope_filter": "{filter_or_null}"
|
|
107
|
+
},
|
|
108
|
+
"outcome": {
|
|
109
|
+
"artifacts_created": "{placed_files_list}",
|
|
110
|
+
"total_extracted": "{N}",
|
|
111
|
+
"total_approved": "{N}",
|
|
112
|
+
"total_placed": "{N}",
|
|
113
|
+
"gate_pass_rate": "{rate}"
|
|
114
|
+
}
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
2. **Dream trigger** — Update `.keysarium/insights/trigger-state.json`:
|
|
119
|
+
```json
|
|
120
|
+
{
|
|
121
|
+
"event_type": "case_completion",
|
|
122
|
+
"timestamp": "ISO-8601",
|
|
123
|
+
"details": "Harvest completed for {slug}: {placed_count} artifacts placed"
|
|
124
|
+
}
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
3. **Clean up** — Delete the findings JSON file ONLY after confirming all placements succeeded and the harvest report was written. If any placement failed or the report could not be generated, keep the findings JSON as a recovery artifact and log: `"Findings JSON retained at {path} — manual review needed."`
|
|
128
|
+
|
|
129
|
+
### Step 7: Final Display
|
|
130
|
+
|
|
131
|
+
Show the user a completion summary:
|
|
132
|
+
|
|
133
|
+
```
|
|
134
|
+
═══════════════════════════════════════════════════════
|
|
135
|
+
✅ HARVEST COMPLETE: {SESSION_ID}
|
|
136
|
+
|
|
137
|
+
Extracted: {N} → Approved: {M} → Placed: {P}
|
|
138
|
+
|
|
139
|
+
Placed artifacts:
|
|
140
|
+
📁 .claude/skills/agent-swarm-topology/SKILL.md
|
|
141
|
+
📁 .claude/rules/domain-detection.md
|
|
142
|
+
📄 TOOLKIT_HARVEST.md (3 entries added)
|
|
143
|
+
|
|
144
|
+
Gate pass rate: {rate}%
|
|
145
|
+
Memory reward stored: {score} ({label})
|
|
146
|
+
|
|
147
|
+
Run /bto-test on placed skills for promotion to Tier 2.
|
|
148
|
+
═══════════════════════════════════════════════════════
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
## Anti-Patterns
|
|
152
|
+
|
|
153
|
+
| Anti-Pattern | Detection | Fix |
|
|
154
|
+
|-------------|-----------|-----|
|
|
155
|
+
| Overwriting existing files | Target file already exists | NEVER overwrite; skip and flag for manual review |
|
|
156
|
+
| Placing blocked findings | Finding status is `blocked` | Only place findings with status `approved` |
|
|
157
|
+
| Missing TOOLKIT_HARVEST.md sections | Section header doesn't exist yet | Create the section before appending |
|
|
158
|
+
| Forgetting to update "Обработанные проекты" | Table not updated after placement | Always update as the last TOOLKIT_HARVEST.md operation |
|
|
159
|
+
| Leaving findings JSON behind | Temp file not cleaned up | Delete after successful report generation |
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
# Artifact Categories Reference
|
|
2
|
+
|
|
3
|
+
Seven mutually exclusive categories for classifying extracted findings.
|
|
4
|
+
Every finding must be assigned exactly one category before passing through the quality gates.
|
|
5
|
+
If a finding spans multiple categories, split it or assign the primary category.
|
|
6
|
+
|
|
7
|
+
## Category Taxonomy
|
|
8
|
+
|
|
9
|
+
| Category | Destination | Description | Examples | Agent Lens |
|
|
10
|
+
|----------|-------------|-------------|----------|------------|
|
|
11
|
+
| skills | .claude/skills/<name>/ | Complete skill bundles (SKILL.md + modules) | New analysis methodology, reusable research approach | All lenses (cross-cutting) |
|
|
12
|
+
| commands | .claude/commands/<name>.md | Slash commands for Claude Code | Pipeline phases, utility commands | extractor-commands |
|
|
13
|
+
| hooks | .claude/hooks/ or TOOLKIT_HARVEST.md | Event-driven automations, pre/post hooks | Pre-commit checks, post-phase triggers | extractor-commands |
|
|
14
|
+
| rules | .claude/rules/<name>.md | Constraints, quality gates, anti-patterns | Domain restrictions, naming conventions | extractor-rules |
|
|
15
|
+
| templates | TOOLKIT_HARVEST.md or lib/ | Document/config structure templates | ADR template, mermaid diagram template | extractor-templates |
|
|
16
|
+
| patterns | TOOLKIT_HARVEST.md or lib/ | Architecture and design patterns | Agent swarm topology, event-driven pipeline | extractor-patterns |
|
|
17
|
+
| snippets | TOOLKIT_HARVEST.md | Reusable code fragments | React components, utility functions, bash one-liners | extractor-snippets |
|
|
18
|
+
|
|
19
|
+
## Category Descriptions
|
|
20
|
+
|
|
21
|
+
### skills
|
|
22
|
+
A skill is a self-contained capability bundle that can be loaded by a Claude Code session.
|
|
23
|
+
It requires at minimum a SKILL.md with trigger conditions, usage instructions, and output specification.
|
|
24
|
+
Extract as a skill when the finding represents a repeatable methodology that warrants its own invocation pattern.
|
|
25
|
+
Destination: create a new directory under `.claude/skills/<name>/` with SKILL.md as the entry point.
|
|
26
|
+
|
|
27
|
+
### commands
|
|
28
|
+
A command is a slash command definition — a markdown file in `.claude/commands/` that Claude Code exposes as `/command-name`.
|
|
29
|
+
Extract as a command when the finding defines a distinct user-facing workflow step or utility.
|
|
30
|
+
Destination: `.claude/commands/<name>.md`
|
|
31
|
+
|
|
32
|
+
### hooks
|
|
33
|
+
A hook is an event-driven automation that fires on a specific lifecycle event (session start, phase completion, file save).
|
|
34
|
+
Extract as a hook when the finding describes behavior that should trigger automatically, not on user request.
|
|
35
|
+
Destination: `.claude/hooks/` directory if the project uses the hooks directory convention; otherwise document in TOOLKIT_HARVEST.md under the Hooks section.
|
|
36
|
+
|
|
37
|
+
### rules
|
|
38
|
+
A rule is a constraint, convention, or quality gate that governs agent behavior.
|
|
39
|
+
Extract as a rule when the finding specifies what agents must or must not do, regardless of context.
|
|
40
|
+
Destination: `.claude/rules/<name>.md`
|
|
41
|
+
|
|
42
|
+
### templates
|
|
43
|
+
A template is a document or configuration scaffold with placeholders that gets filled in for each use.
|
|
44
|
+
Extract as a template when the finding defines a repeatable document structure (not the content itself).
|
|
45
|
+
Destination: `lib/` if it will be referenced by multiple commands; otherwise TOOLKIT_HARVEST.md under Templates section.
|
|
46
|
+
|
|
47
|
+
### patterns
|
|
48
|
+
A pattern is an architectural or design solution to a recurring structural problem.
|
|
49
|
+
Extract as a pattern when the finding describes how to compose components or agents, not what they contain.
|
|
50
|
+
Patterns are higher-level than snippets and lower-level than skills.
|
|
51
|
+
Destination: `lib/` if it has widespread applicability; otherwise TOOLKIT_HARVEST.md under Patterns section.
|
|
52
|
+
|
|
53
|
+
### snippets
|
|
54
|
+
A snippet is a concrete, copy-paste-ready code or configuration fragment.
|
|
55
|
+
Extract as a snippet when the finding is language/framework-specific code that solves one small problem.
|
|
56
|
+
Destination: TOOLKIT_HARVEST.md under Snippets section, with language and framework tags.
|
|
57
|
+
|
|
58
|
+
## Scope Filtering Mapping (category to agent lens for /harvest only X)
|
|
59
|
+
|
|
60
|
+
When the user invokes `/harvest only <scope>`, only the corresponding extractor agent lens runs.
|
|
61
|
+
All other lenses are suppressed to reduce cost and noise.
|
|
62
|
+
|
|
63
|
+
| User Scope Argument | Active Agent Lenses | Categories Produced |
|
|
64
|
+
|--------------------|---------------------|---------------------|
|
|
65
|
+
| `skills` | ALL 5 lenses (cross-cutting) | skills |
|
|
66
|
+
| `commands` | extractor-commands | commands, hooks |
|
|
67
|
+
| `rules` | extractor-rules | rules |
|
|
68
|
+
| `templates` | extractor-templates | templates |
|
|
69
|
+
| `patterns` | extractor-patterns | patterns |
|
|
70
|
+
| `snippets` | extractor-snippets | snippets |
|
|
71
|
+
| `code` | extractor-snippets | snippets |
|
|
72
|
+
| `all` (default) | All 5 lenses | All 7 categories |
|
|
73
|
+
|
|
74
|
+
Note: hooks are produced by extractor-commands because hooks are closely related to command lifecycle automation.
|
|
75
|
+
Note: skills scope spawns ALL 5 lenses because skills are cross-cutting — any lens may discover patterns worthy of promotion to a full skill bundle.
|
|
76
|
+
|
|
77
|
+
## Deduplication Rules (per-category)
|
|
78
|
+
|
|
79
|
+
Deduplication is applied during gate G6 (Not Duplicate). The similarity threshold is 0.85.
|
|
80
|
+
Similarity is assessed by the orchestrator using structural comparison (not LLM) unless specified.
|
|
81
|
+
|
|
82
|
+
| Category | Deduplication Key | Comparison Method |
|
|
83
|
+
|----------|------------------|-------------------|
|
|
84
|
+
| skills | Skill name + trigger conditions | Exact name match OR semantic trigger overlap |
|
|
85
|
+
| commands | Command name (slug) | Exact slash-command name match |
|
|
86
|
+
| hooks | Event type + trigger condition | Exact event type + condition string match |
|
|
87
|
+
| rules | Rule title + first sentence of constraint | Normalized string similarity |
|
|
88
|
+
| templates | Template name + section headings list | Section heading set overlap |
|
|
89
|
+
| patterns | Pattern name + problem statement | Normalized string similarity on problem statement |
|
|
90
|
+
| snippets | Language + first 5 non-whitespace lines | Normalized code similarity |
|
|
91
|
+
|
|
92
|
+
When a near-duplicate is found (similarity 0.70–0.85), the extractor must decide:
|
|
93
|
+
- If the new finding improves the existing entry (adds a section, fixes an error): propose a MERGE, not a new entry.
|
|
94
|
+
- If the new finding is a different context of the same idea: add as a "variant" note under the existing entry.
|
|
95
|
+
- If similarity >= 0.85: treat as duplicate, reject via G6.
|
|
96
|
+
|
|
97
|
+
Merge proposals are surfaced in the harvest output as:
|
|
98
|
+
```
|
|
99
|
+
MERGE PROPOSAL: Finding #{X} improves existing entry "{existing_name}" — {reason}
|
|
100
|
+
```
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
# Maturity Model Reference
|
|
2
|
+
|
|
3
|
+
Three maturity levels are used to communicate how well-tested and battle-proven an extracted finding is.
|
|
4
|
+
Every finding must be assigned a maturity level at extraction time. The level is surfaced in the artifact card
|
|
5
|
+
and in TOOLKIT_HARVEST.md entries so consumers know how much trust to place in the finding.
|
|
6
|
+
|
|
7
|
+
## Maturity Levels
|
|
8
|
+
|
|
9
|
+
| Level | Label | Criteria | Badge |
|
|
10
|
+
|-------|-------|----------|-------|
|
|
11
|
+
| Alpha | Alpha | First extraction, untested. Used in exactly 1 project. No validation. | 🔴 |
|
|
12
|
+
| Beta | Beta | Used in 2+ projects. Basic validation. May have rough edges. | 🟡 |
|
|
13
|
+
| Stable | Stable | Used in 3+ projects. Proven track record. Passed /bto-test or equivalent. | 🟢 |
|
|
14
|
+
|
|
15
|
+
## Promotion Protocol
|
|
16
|
+
|
|
17
|
+
Promotion moves a finding from one maturity level to the next. Promotion requires evidence — it is not
|
|
18
|
+
self-declared. The `/harvest` command can suggest promotion but not enforce it; the user confirms.
|
|
19
|
+
|
|
20
|
+
### Alpha to Beta
|
|
21
|
+
|
|
22
|
+
Required evidence (any one of the following):
|
|
23
|
+
- The same pattern, rule, template, or snippet was independently observed in a second project's harvest.
|
|
24
|
+
- A team member explicitly applied the finding in a second project and confirmed it worked.
|
|
25
|
+
- The finding was referenced in a command or skill that was used in a second research session.
|
|
26
|
+
|
|
27
|
+
Promotion action:
|
|
28
|
+
1. Update the `maturity` field in the finding's TOOLKIT_HARVEST.md entry from `alpha` to `beta`.
|
|
29
|
+
2. Add a `used_in` list to the entry noting the two (or more) projects.
|
|
30
|
+
3. Log the promotion: `PROMOTED: "{name}" alpha -> beta — evidence: {source}`.
|
|
31
|
+
|
|
32
|
+
### Beta to Stable
|
|
33
|
+
|
|
34
|
+
Required evidence (ALL of the following):
|
|
35
|
+
- Used in 3 or more distinct projects (verified by `used_in` list).
|
|
36
|
+
- At least one of the following validation signals exists:
|
|
37
|
+
- Passed `/bto-test` Layer 2 evaluation with score >= 7.0.
|
|
38
|
+
- Has a documented test case in a `references/` or `examples/` directory.
|
|
39
|
+
- Was reviewed and approved by a second human reviewer in a harvest session.
|
|
40
|
+
|
|
41
|
+
Promotion action:
|
|
42
|
+
1. Update the `maturity` field from `beta` to `stable`.
|
|
43
|
+
2. Record the validation signal: BTO score, test case path, or reviewer name.
|
|
44
|
+
3. Add the finding to the summary table in TOOLKIT_HARVEST.md under the Stable Findings section.
|
|
45
|
+
4. Log the promotion: `PROMOTED: "{name}" beta -> stable — validation: {signal}`.
|
|
46
|
+
|
|
47
|
+
### Demotion Protocol
|
|
48
|
+
|
|
49
|
+
A stable or beta finding may be demoted if evidence of unreliability emerges:
|
|
50
|
+
- A project reports the finding caused errors or required significant modification.
|
|
51
|
+
- A `/bto-test` run scores the finding below 5.0.
|
|
52
|
+
|
|
53
|
+
Demotion action:
|
|
54
|
+
1. Downgrade maturity by one level (stable -> beta, beta -> alpha).
|
|
55
|
+
2. Add a `demotion_reason` note to the TOOLKIT_HARVEST.md entry.
|
|
56
|
+
3. Log the demotion: `DEMOTED: "{name}" {from} -> {to} — reason: {reason}`.
|
|
57
|
+
|
|
58
|
+
## Initial Assignment Rules
|
|
59
|
+
|
|
60
|
+
1. The first extraction of any finding is ALWAYS assigned `alpha` maturity.
|
|
61
|
+
2. Exception: if the finding describes a pattern that was already in use across multiple projects
|
|
62
|
+
prior to being formally extracted, it may be assigned `beta` at extraction time — but only if
|
|
63
|
+
the extractor can cite at least 2 distinct prior uses in the finding's `used_in` field.
|
|
64
|
+
3. Exception: if the extracted finding is a verbatim copy of content that already exists in the
|
|
65
|
+
toolkit at beta or stable maturity (e.g., an updated version of a known pattern), inherit the
|
|
66
|
+
existing maturity level and propose a MERGE rather than a new entry.
|
|
67
|
+
4. No finding may be assigned `stable` at first extraction, regardless of prior use.
|
|
68
|
+
|
|
69
|
+
## Maturity in Artifact Card
|
|
70
|
+
|
|
71
|
+
Every finding written to TOOLKIT_HARVEST.md includes a maturity badge in its header card.
|
|
72
|
+
|
|
73
|
+
### Format
|
|
74
|
+
|
|
75
|
+
```
|
|
76
|
+
## [Category] — {finding_name}
|
|
77
|
+
**Maturity:** [ALPHA | BETA | STABLE]
|
|
78
|
+
**Confidence:** {confidence_score}
|
|
79
|
+
**Used in:** {project_slug_1}, {project_slug_2}, ...
|
|
80
|
+
**Validated by:** {bto_score or test_path or reviewer or "not yet validated"}
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Display Rules
|
|
84
|
+
|
|
85
|
+
- Display maturity label in uppercase.
|
|
86
|
+
- If `used_in` has only one entry, display it as `{slug} (source project)`.
|
|
87
|
+
- If `validated_by` is empty, display `not yet validated`.
|
|
88
|
+
- Include the maturity badge on the same line as the finding name when embedding in summary tables:
|
|
89
|
+
|
|
90
|
+
```
|
|
91
|
+
| {finding_name} [ALPHA] | {category} | {one-line description} |
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
- In summary tables, badge order is: STABLE first, then BETA, then ALPHA (most trustworthy at top).
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Quality Gates Reference
|
|
2
|
+
|
|
3
|
+
Gates applied to every candidate finding during the harvest process.
|
|
4
|
+
All 8 gates must be evaluated before a finding can be accepted into the toolkit.
|
|
5
|
+
A gate failure is a hard block unless overridden by the user (see Override Protocol).
|
|
6
|
+
|
|
7
|
+
## Gate Definitions
|
|
8
|
+
|
|
9
|
+
| Gate | Name | Check | Type | Pass Criteria | Failure Message |
|
|
10
|
+
|------|------|-------|------|---------------|-----------------|
|
|
11
|
+
| G1 | When-To-Use | Finding has a "When to use" section or equivalent guidance | deterministic | Section exists and is non-empty | Missing "When to use" guidance |
|
|
12
|
+
| G2 | When-Not-To-Use | Finding has a "When NOT to use" section or equivalent anti-guidance | deterministic | Section exists and is non-empty | Missing "When NOT to use" guidance |
|
|
13
|
+
| G3 | Decontextualized | No project-specific names, paths, slugs, dates, or hardcoded values remain | semantic | Haiku confirms no project-specific references | Project-specific references remain: {details} |
|
|
14
|
+
| G4 | Concrete Example | At least one concrete usage example is provided | semantic | Haiku confirms example is concrete and actionable | No concrete example provided |
|
|
15
|
+
| G5 | Reusability Threshold | Reusability confidence score meets minimum | deterministic | confidence >= 0.5 | Confidence {score} < 0.5 threshold |
|
|
16
|
+
| G6 | Not Duplicate | Finding is not a duplicate of an existing toolkit entry | deterministic | No existing entry with content similarity > 0.85 | Duplicate of existing entry: {existing_name} |
|
|
17
|
+
| G7 | Maturity Label | Finding has a maturity level assigned | deterministic | maturity field is one of: alpha, beta, stable | Missing maturity label |
|
|
18
|
+
| G8 | Brutal Honesty | Finding survives self-critical review — is it genuinely useful or just noise? | semantic | Haiku confirms finding has genuine reuse value beyond the source project | Fails brutal-honesty review: {reason} |
|
|
19
|
+
|
|
20
|
+
## Gate Types
|
|
21
|
+
|
|
22
|
+
**Deterministic gates** (G1, G2, G5, G6, G7) are checked by the orchestrator without LLM calls.
|
|
23
|
+
They operate on the structured finding JSON: presence of fields, numeric thresholds, set membership.
|
|
24
|
+
|
|
25
|
+
**Semantic gates** (G3, G4, G8) require a haiku LLM call with a focused single-question prompt.
|
|
26
|
+
The prompt must ask for a binary YES/NO answer plus a short explanation (1-2 sentences max).
|
|
27
|
+
Use haiku — these are high-frequency, pattern-matching evaluations that do not require deep reasoning.
|
|
28
|
+
|
|
29
|
+
## Pass Assignment (which pass each gate belongs to)
|
|
30
|
+
|
|
31
|
+
Gates are evaluated in two passes to minimize unnecessary LLM cost.
|
|
32
|
+
|
|
33
|
+
### Pass 1 — Deterministic (zero LLM cost)
|
|
34
|
+
|
|
35
|
+
Run all deterministic gates first. If any deterministic gate fails, reject immediately.
|
|
36
|
+
Do NOT proceed to Pass 2 for a rejected finding.
|
|
37
|
+
|
|
38
|
+
Gates in Pass 1 (in evaluation order):
|
|
39
|
+
1. G7 — Maturity Label (cheapest check: field presence)
|
|
40
|
+
2. G5 — Reusability Threshold (numeric comparison)
|
|
41
|
+
3. G1 — When-To-Use (section presence)
|
|
42
|
+
4. G2 — When-Not-To-Use (section presence)
|
|
43
|
+
5. G6 — Not Duplicate (similarity comparison against existing entries)
|
|
44
|
+
|
|
45
|
+
### Pass 2 — Semantic (haiku LLM calls, parallelizable)
|
|
46
|
+
|
|
47
|
+
Run only after all Pass 1 gates pass.
|
|
48
|
+
Gates G3, G4, G8 may be evaluated in parallel (3 concurrent haiku calls).
|
|
49
|
+
|
|
50
|
+
Gates in Pass 2:
|
|
51
|
+
- G3 — Decontextualized
|
|
52
|
+
- G4 — Concrete Example
|
|
53
|
+
- G8 — Brutal Honesty
|
|
54
|
+
|
|
55
|
+
## Override Protocol
|
|
56
|
+
|
|
57
|
+
A user may override a gate for a specific finding using the syntax:
|
|
58
|
+
|
|
59
|
+
```
|
|
60
|
+
пропусти G3 для #X
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
Where `#X` is the finding number (1-indexed, matching the extraction output list).
|
|
64
|
+
|
|
65
|
+
### Override Rules
|
|
66
|
+
|
|
67
|
+
- Multiple gates may be overridden in a single command using comma-separated syntax: `"пропусти G3,G4 для #14"`.
|
|
68
|
+
- G8 (Brutal Honesty) cannot be overridden. If a finding fails G8, it must be revised or dropped.
|
|
69
|
+
- G6 (Not Duplicate) cannot be overridden. If a finding is a duplicate, either update the existing entry or drop the new one.
|
|
70
|
+
- G5 (Reusability Threshold) can be overridden only if the user provides a rationale explaining why low-confidence is acceptable for this finding.
|
|
71
|
+
- All overrides are logged in the harvest output with the gate ID, finding number, and user rationale.
|
|
72
|
+
|
|
73
|
+
### Override Log Format
|
|
74
|
+
|
|
75
|
+
When an override is applied, append to the harvest session log:
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
OVERRIDE: G{N} skipped for finding #{X} — user rationale: "{rationale}"
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## Scoring
|
|
82
|
+
|
|
83
|
+
After all gates are evaluated for a finding:
|
|
84
|
+
|
|
85
|
+
```
|
|
86
|
+
gate_pass_rate = gates_passed / 8
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
This score is recorded in the finding's metadata alongside the per-gate pass/fail results.
|
|
90
|
+
|
|
91
|
+
| gate_pass_rate | Interpretation |
|
|
92
|
+
|----------------|----------------|
|
|
93
|
+
| 1.0 | Clean pass — add to toolkit |
|
|
94
|
+
| 0.875 (7/8) | One gate overridden — add with override note |
|
|
95
|
+
| < 0.875 | Rejected — do not add to toolkit |
|
|
96
|
+
|
|
97
|
+
The gate_pass_rate is surfaced in the harvest summary table for each finding.
|
|
98
|
+
It is not used as a quality score for ranking — a finding either passes all required gates or it does not.
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Artifact Card Template
|
|
2
|
+
|
|
3
|
+
## Usage
|
|
4
|
+
This template is rendered once per approved finding during Module 04 (Integrate).
|
|
5
|
+
Replace all {PLACEHOLDER} values with finding data.
|
|
6
|
+
|
|
7
|
+
## Template
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
### {NAME}
|
|
12
|
+
|
|
13
|
+
**Category:** {CATEGORY}
|
|
14
|
+
**Maturity:** {MATURITY_BADGE} {MATURITY_LEVEL}
|
|
15
|
+
**Reusability Confidence:** {CONFIDENCE}
|
|
16
|
+
**Extracted from:** {SOURCE_PROJECT}
|
|
17
|
+
**Last updated:** {DATE}
|
|
18
|
+
|
|
19
|
+
#### Description
|
|
20
|
+
|
|
21
|
+
{DESCRIPTION}
|
|
22
|
+
|
|
23
|
+
#### When to use
|
|
24
|
+
|
|
25
|
+
{WHEN_TO_USE}
|
|
26
|
+
|
|
27
|
+
#### When NOT to use
|
|
28
|
+
|
|
29
|
+
{WHEN_NOT_TO_USE}
|
|
30
|
+
|
|
31
|
+
#### Example
|
|
32
|
+
|
|
33
|
+
{EXAMPLE}
|
|
34
|
+
|
|
35
|
+
#### Changelog
|
|
36
|
+
|
|
37
|
+
- v1: Initial extraction from {SOURCE_PROJECT} ({DATE})
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## Placeholder Reference
|
|
42
|
+
|
|
43
|
+
| Placeholder | Source | Example |
|
|
44
|
+
|-------------|--------|---------|
|
|
45
|
+
| {NAME} | finding.title | "Agent Swarm Topology" |
|
|
46
|
+
| {CATEGORY} | finding.category | "patterns" |
|
|
47
|
+
| {MATURITY_BADGE} | maturity-model.md lookup | "🔴" |
|
|
48
|
+
| {MATURITY_LEVEL} | finding.maturity | "Alpha" |
|
|
49
|
+
| {CONFIDENCE} | finding.confidence | "0.85" |
|
|
50
|
+
| {SOURCE_PROJECT} | session.slug | "bank-kc-automation" |
|
|
51
|
+
| {DATE} | current date | "2026-03-03" |
|
|
52
|
+
| {DESCRIPTION} | finding.content (first paragraph) | "..." |
|
|
53
|
+
| {WHEN_TO_USE} | finding.when_to_use | "..." |
|
|
54
|
+
| {WHEN_NOT_TO_USE} | finding.when_not_to_use | "..." |
|
|
55
|
+
| {EXAMPLE} | finding.example | "..." |
|
|
56
|
+
|
|
57
|
+
## Placement Rules
|
|
58
|
+
|
|
59
|
+
For **skills** category: This template becomes the initial SKILL.md skeleton
|
|
60
|
+
For **commands** category: This template becomes the command .md file body
|
|
61
|
+
For **rules** category: Convert to rule table format (see .claude/rules/ convention)
|
|
62
|
+
For **templates/patterns/snippets** category: This card is appended to TOOLKIT_HARVEST.md under the relevant section
|
|
63
|
+
For **hooks** category: This card is appended to TOOLKIT_HARVEST.md hooks section (or created inline)
|
|
@@ -0,0 +1,102 @@
|
|
|
1
|
+
# Harvest Report Template
|
|
2
|
+
|
|
3
|
+
## Usage
|
|
4
|
+
Generated at the end of Module 04 (Integrate) to summarize the harvest session.
|
|
5
|
+
All {PLACEHOLDER} values are computed from the findings registry.
|
|
6
|
+
|
|
7
|
+
## Template
|
|
8
|
+
|
|
9
|
+
---
|
|
10
|
+
|
|
11
|
+
# Harvest Report: {SESSION_ID}
|
|
12
|
+
|
|
13
|
+
**Date:** {DATE}
|
|
14
|
+
**Target:** {TARGET_PATHS}
|
|
15
|
+
**Scope:** {SCOPE_FILTER_OR_ALL}
|
|
16
|
+
**Steps completed:** {STEPS_COMPLETED} (Extract → Classify → Checkpoint → Gate → Integrate)
|
|
17
|
+
|
|
18
|
+
## Summary
|
|
19
|
+
|
|
20
|
+
| Metric | Value |
|
|
21
|
+
|--------|-------|
|
|
22
|
+
| Total findings extracted | {TOTAL_EXTRACTED} |
|
|
23
|
+
| After deduplication | {AFTER_DEDUP} |
|
|
24
|
+
| User removals | {USER_REMOVALS} |
|
|
25
|
+
| User merges | {USER_MERGES} |
|
|
26
|
+
| Approved (passed gates) | {APPROVED_COUNT} |
|
|
27
|
+
| Blocked (failed gates) | {BLOCKED_COUNT} |
|
|
28
|
+
| Gate overrides | {OVERRIDE_COUNT} |
|
|
29
|
+
| Placed artifacts | {PLACED_COUNT} |
|
|
30
|
+
|
|
31
|
+
## Findings by Category
|
|
32
|
+
|
|
33
|
+
| Category | Extracted | Approved | Placed | Destination |
|
|
34
|
+
|----------|-----------|----------|--------|-------------|
|
|
35
|
+
| skills | {N} | {N} | {N} | .claude/skills/ |
|
|
36
|
+
| commands | {N} | {N} | {N} | .claude/commands/ |
|
|
37
|
+
| hooks | {N} | {N} | {N} | TOOLKIT_HARVEST.md |
|
|
38
|
+
| rules | {N} | {N} | {N} | .claude/rules/ |
|
|
39
|
+
| templates | {N} | {N} | {N} | TOOLKIT_HARVEST.md |
|
|
40
|
+
| patterns | {N} | {N} | {N} | TOOLKIT_HARVEST.md |
|
|
41
|
+
| snippets | {N} | {N} | {N} | TOOLKIT_HARVEST.md |
|
|
42
|
+
|
|
43
|
+
## Quality Gate Results
|
|
44
|
+
|
|
45
|
+
| Gate | Pass | Fail | Override | Pass Rate |
|
|
46
|
+
|------|------|------|----------|-----------|
|
|
47
|
+
| G1 When-To-Use | {N} | {N} | {N} | {%} |
|
|
48
|
+
| G2 When-Not-To-Use | {N} | {N} | {N} | {%} |
|
|
49
|
+
| G3 Decontextualized | {N} | {N} | {N} | {%} |
|
|
50
|
+
| G4 Concrete Example | {N} | {N} | {N} | {%} |
|
|
51
|
+
| G5 Reusability | {N} | {N} | {N} | {%} |
|
|
52
|
+
| G6 Not Duplicate | {N} | {N} | {N} | {%} |
|
|
53
|
+
| G7 Maturity Label | {N} | {N} | {N} | {%} |
|
|
54
|
+
| G8 Brutal Honesty | {N} | {N} | {N} | {%} |
|
|
55
|
+
|
|
56
|
+
## Coverage Gaps
|
|
57
|
+
|
|
58
|
+
| Item | Reason |
|
|
59
|
+
|------|--------|
|
|
60
|
+
| {SKIPPED_FILES_TABLE_OR_NONE} | |
|
|
61
|
+
|
|
62
|
+
**Failed extractors:** {FAILED_EXTRACTORS_OR_NONE}
|
|
63
|
+
|
|
64
|
+
## Confidence Distribution
|
|
65
|
+
|
|
66
|
+
| Range | Count | Percentage |
|
|
67
|
+
|-------|-------|------------|
|
|
68
|
+
| 0.9 - 1.0 (Excellent) | {N} | {%} |
|
|
69
|
+
| 0.7 - 0.89 (Good) | {N} | {%} |
|
|
70
|
+
| 0.5 - 0.69 (Acceptable) | {N} | {%} |
|
|
71
|
+
| < 0.5 (Below threshold) | {N} | {%} |
|
|
72
|
+
|
|
73
|
+
## Memory Integration
|
|
74
|
+
|
|
75
|
+
- Patterns loaded from memory: {MEMORY_PATTERNS_COUNT}
|
|
76
|
+
- Reward stored: {REWARD_SCORE} ({REWARD_LABEL})
|
|
77
|
+
- Dream trigger updated: {YES_NO}
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## Placeholder Reference
|
|
82
|
+
|
|
83
|
+
| Placeholder | Computation |
|
|
84
|
+
|-------------|-------------|
|
|
85
|
+
| {SESSION_ID} | HarvestSession.session_id |
|
|
86
|
+
| {DATE} | Current date (YYYY-MM-DD) |
|
|
87
|
+
| {TARGET_PATHS} | Directory path(s) scanned |
|
|
88
|
+
| {SCOPE_FILTER_OR_ALL} | Scope filter value or "all categories" |
|
|
89
|
+
| {STEPS_COMPLETED} | Count of pipeline steps that ran successfully |
|
|
90
|
+
| {TOTAL_EXTRACTED} | count(all findings at extraction) |
|
|
91
|
+
| {AFTER_DEDUP} | count(findings after Module 02 dedup) |
|
|
92
|
+
| {USER_REMOVALS} | count(findings removed by user at checkpoint) |
|
|
93
|
+
| {USER_MERGES} | count(findings with non-null merged_from) |
|
|
94
|
+
| {APPROVED_COUNT} | count(findings where status = approved) |
|
|
95
|
+
| {BLOCKED_COUNT} | count(findings where status = blocked) |
|
|
96
|
+
| {OVERRIDE_COUNT} | sum(length of all gate_overrides arrays) |
|
|
97
|
+
| {PLACED_COUNT} | count(findings where status = integrated) |
|
|
98
|
+
| {SKIPPED_FILES_TABLE_OR_NONE} | Table of files skipped (>100KB or binary), or "None" |
|
|
99
|
+
| {FAILED_EXTRACTORS_OR_NONE} | List of failed agent lenses, or "None" |
|
|
100
|
+
| {MEMORY_PATTERNS_COUNT} | count(patterns loaded from memory_query) |
|
|
101
|
+
| {REWARD_SCORE} | User checkpoint response mapped to 0.0-1.0 |
|
|
102
|
+
| {REWARD_LABEL} | excellent/good/needs_work/failed (see reward-learning.md) |
|