@automagik/genie-brain 0.260404.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/dist/bench/cli.js +146 -0
- package/dist/bench-ui/css/brain-lab.css +623 -0
- package/dist/bench-ui/index.html +147 -0
- package/dist/bench-ui/js/charts.js +409 -0
- package/dist/bench-ui/js/common.js +264 -0
- package/dist/bench-ui/js/compare.js +239 -0
- package/dist/bench-ui/js/detail.js +219 -0
- package/dist/bench-ui/js/leaderboard.js +222 -0
- package/dist/bench-ui/js/progress.js +158 -0
- package/dist/bench-ui/ui/css/brain-lab.css +623 -0
- package/dist/bench-ui/ui/index.html +147 -0
- package/dist/bench-ui/ui/js/charts.js +409 -0
- package/dist/bench-ui/ui/js/common.js +264 -0
- package/dist/bench-ui/ui/js/compare.js +239 -0
- package/dist/bench-ui/ui/js/detail.js +219 -0
- package/dist/bench-ui/ui/js/leaderboard.js +222 -0
- package/dist/bench-ui/ui/js/progress.js +158 -0
- package/dist/cli.js +1047 -0
- package/dist/index.js +1069 -0
- package/package.json +61 -0
- package/skills/brain/SKILL.md +115 -0
- package/skills/brain-analyze/SKILL.md +86 -0
- package/skills/brain-build/SKILL.md +100 -0
- package/skills/brain-health/SKILL.md +101 -0
- package/skills/brain-ingest/SKILL.md +107 -0
- package/skills/brain-init/SKILL.md +81 -0
- package/skills/brain-learn/SKILL.md +91 -0
- package/skills/brain-observe/SKILL.md +77 -0
- package/skills/brain-search/SKILL.md +82 -0
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: brain-learn
|
|
3
|
+
description: Manage agent learnings -- ingest, search, classify, and forget knowledge memories
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# /brain-learn — Agent Learning Management
|
|
7
|
+
|
|
8
|
+
Manage the agent's learning memory -- ingest feedback, discover patterns, store preferences, and recall relevant learnings. Supports classification by type, enforcement flags for mandatory rules, and soft-delete for forgetting without permanent loss.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- Agent receives a correction or feedback from a human
|
|
13
|
+
- Agent discovers a recurring pattern worth remembering
|
|
14
|
+
- Agent needs to store a user preference or communication style
|
|
15
|
+
- Agent needs to recall previous learnings before taking action
|
|
16
|
+
- Cleaning up outdated or incorrect learnings
|
|
17
|
+
|
|
18
|
+
## Prerequisites
|
|
19
|
+
|
|
20
|
+
- Brain must exist (`genie brain status`)
|
|
21
|
+
- Brain should have a `memory/` directory (created automatically on first learn ingest)
|
|
22
|
+
|
|
23
|
+
## Flow
|
|
24
|
+
|
|
25
|
+
1. **Classify** -- determine the learning type (feedback, project, user, reference)
|
|
26
|
+
2. **Ingest** -- store the learning with metadata and optional enforcement
|
|
27
|
+
3. **Index** -- learning is indexed for search (automatic after ingest)
|
|
28
|
+
4. **Recall** -- search learnings when context is needed for a task
|
|
29
|
+
|
|
30
|
+
## Commands
|
|
31
|
+
|
|
32
|
+
| Command | Description |
|
|
33
|
+
|---------|-------------|
|
|
34
|
+
| `genie brain learn ingest "content" --brain <id>` | Ingest a learning -- auto-classifies type |
|
|
35
|
+
| `genie brain learn ingest "content" --brain <id> --learning-type feedback` | Ingest with explicit type |
|
|
36
|
+
| `genie brain learn ingest "content" --brain <id> --name "name"` | Ingest with a human-readable name |
|
|
37
|
+
| `genie brain learn ingest "content" --brain <id> --reason "why"` | Ingest with rationale explaining why this matters |
|
|
38
|
+
| `genie brain learn ingest "content" --brain <id> --enforce` | Mark as enforced rule -- mandatory compliance |
|
|
39
|
+
| `genie brain learn search "query" --brain <id>` | Search across all learnings |
|
|
40
|
+
| `genie brain learn search "query" --brain <id> --enforce` | Search only enforced (mandatory) learnings |
|
|
41
|
+
| `genie brain learn search "query" --brain <id> --learning-type feedback` | Search filtered by learning type |
|
|
42
|
+
| `genie brain learn list --brain <id>` | List all active learnings |
|
|
43
|
+
| `genie brain learn list --brain <id> --learning-type user` | List learnings filtered by type |
|
|
44
|
+
| `genie brain learn list --brain <id> --all` | List all learnings including forgotten ones |
|
|
45
|
+
| `genie brain learn forget <path\|#docid> --brain <id>` | Soft-delete a learning (recoverable) |
|
|
46
|
+
| `genie brain learn classify --brain <id>` | Auto-classify unclassified learnings |
|
|
47
|
+
|
|
48
|
+
## Learning Types
|
|
49
|
+
|
|
50
|
+
| Type | When to Use | Examples |
|
|
51
|
+
|------|-------------|---------|
|
|
52
|
+
| `feedback` | Corrections, directives, behavioral rules | "Never use pip, always use uv", "Always create PRs to dev" |
|
|
53
|
+
| `project` | Architecture, design decisions, how systems work | "The brain uses RRF for search fusion", "Embeddings are in Postgres" |
|
|
54
|
+
| `user` | Preferences, communication style, personal patterns | "User prefers concise responses", "Use Brazilian Portuguese for chat" |
|
|
55
|
+
| `reference` | General knowledge, facts, documentation | "API rate limit is 100/min", "Deploy process uses GitHub Actions" |
|
|
56
|
+
|
|
57
|
+
## Enforce Flag
|
|
58
|
+
|
|
59
|
+
The `--enforce` flag marks a learning as a mandatory rule:
|
|
60
|
+
|
|
61
|
+
- Enforced learnings appear in `--enforce` filtered searches
|
|
62
|
+
- Future hook support will automatically check enforced learnings before actions
|
|
63
|
+
- Use sparingly -- only for strong corrections and non-negotiable rules
|
|
64
|
+
- Enforced learnings are highlighted differently in list output
|
|
65
|
+
|
|
66
|
+
**When to enforce:**
|
|
67
|
+
- Human says "never do X" or "always do Y"
|
|
68
|
+
- A mistake was made and the correction is absolute
|
|
69
|
+
- A policy or rule must be followed without exception
|
|
70
|
+
|
|
71
|
+
**When NOT to enforce:**
|
|
72
|
+
- Soft preferences ("I usually prefer...")
|
|
73
|
+
- Context-dependent patterns ("In this project, we tend to...")
|
|
74
|
+
- Reference information (facts, docs, architecture notes)
|
|
75
|
+
|
|
76
|
+
## Rules
|
|
77
|
+
|
|
78
|
+
- Let the auto-classifier choose the type unless you are certain of the correct classification
|
|
79
|
+
- Use `--enforce` only for strong corrections and non-negotiable directives
|
|
80
|
+
- Include `--reason` to explain why a learning matters -- aids future recall and context
|
|
81
|
+
- Include `--name` for important learnings to make them easy to find in lists
|
|
82
|
+
- `forget` is a soft-delete -- the learning is hidden but recoverable with `--all`
|
|
83
|
+
- Search learnings before starting tasks that might be affected by past feedback
|
|
84
|
+
- Do not duplicate learnings -- search first to check if a similar learning already exists
|
|
85
|
+
- Periodically review learnings with `list` and `forget` outdated ones
|
|
86
|
+
|
|
87
|
+
## Composition
|
|
88
|
+
|
|
89
|
+
- Chains with **/brain-search** -- recall learnings alongside factual knowledge base searches
|
|
90
|
+
- Chains with **/brain-analyze** -- synthesize patterns across multiple learnings
|
|
91
|
+
- Chains with **/brain-health** -- learning memory health is part of overall brain quality
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: brain-observe
|
|
3
|
+
description: Monitor brain performance — gap analysis, strategy tuning, cost analysis, and trace review
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# /brain-observe — Brain Observability
|
|
7
|
+
|
|
8
|
+
Monitor and tune brain search performance. Review query traces, identify knowledge gaps, tune strategy routing, and analyze costs. Essential for maintaining retrieval quality in production.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- Search quality has degraded or users report irrelevant results
|
|
13
|
+
- Need to identify knowledge gaps (queries that return poor results)
|
|
14
|
+
- Tuning strategy routing for specific query patterns
|
|
15
|
+
- Analyzing costs for CAG/embedding operations
|
|
16
|
+
- Periodic brain maintenance and optimization
|
|
17
|
+
|
|
18
|
+
## Prerequisites
|
|
19
|
+
|
|
20
|
+
- Brain must exist and have query traces (`genie brain traces`)
|
|
21
|
+
- Brain should have been used for searches (traces are recorded automatically)
|
|
22
|
+
|
|
23
|
+
## Flow
|
|
24
|
+
|
|
25
|
+
### Gap Analysis
|
|
26
|
+
1. `genie brain traces --brain <id> --failed` — find queries with low confidence
|
|
27
|
+
2. Analyze patterns — which topics have gaps?
|
|
28
|
+
3. Use **/brain-ingest** to fill identified gaps
|
|
29
|
+
4. Re-test gap queries with `genie brain search`
|
|
30
|
+
|
|
31
|
+
### Strategy Tuning
|
|
32
|
+
1. `genie brain traces --brain <id> --limit 50` — review recent traces
|
|
33
|
+
2. Identify query patterns that underperform with default strategy
|
|
34
|
+
3. `genie brain strategy set "<pattern>" cag --brain <id>` — route pattern to CAG
|
|
35
|
+
4. Re-test to verify improvement
|
|
36
|
+
|
|
37
|
+
### Cost Analysis
|
|
38
|
+
1. `genie brain cache --estimate --brain <id>` — estimate CAG cache costs
|
|
39
|
+
2. Review trace latencies for strategy optimization decisions
|
|
40
|
+
3. Consider switching high-cost patterns from CAG to RAG if quality is sufficient
|
|
41
|
+
|
|
42
|
+
### Conflict Resolution
|
|
43
|
+
1. `genie brain conflicts --brain <id>` — detect contradictory documents
|
|
44
|
+
2. `genie brain link --brain <id> --detect-conflicts` — link + conflict detection
|
|
45
|
+
3. Resolve conflicts by updating source documents or forgetting outdated ones
|
|
46
|
+
|
|
47
|
+
## Commands
|
|
48
|
+
|
|
49
|
+
| Command | Description |
|
|
50
|
+
|---------|-------------|
|
|
51
|
+
| `genie brain traces --brain <id>` | List recent query traces |
|
|
52
|
+
| `genie brain traces --brain <id> --limit 50` | More traces |
|
|
53
|
+
| `genie brain traces --brain <id> --failed` | Only failed/low-confidence queries |
|
|
54
|
+
| `genie brain traces --brain <id> --strategy cag` | Filter by strategy |
|
|
55
|
+
| `genie brain traces --brain <id> --purge --older-than 90` | Clean old traces |
|
|
56
|
+
| `genie brain strategy --brain <id>` | List strategy configs |
|
|
57
|
+
| `genie brain strategy set "<pattern>" <strategy> --brain <id>` | Set routing rule |
|
|
58
|
+
| `genie brain strategy set "<pattern>" <strategy> --brain <id> --reason "why"` | With rationale |
|
|
59
|
+
| `genie brain strategy rm "<pattern>" --brain <id>` | Remove routing rule |
|
|
60
|
+
| `genie brain cache --estimate --brain <id>` | Estimate CAG costs |
|
|
61
|
+
| `genie brain conflicts --brain <id>` | Detect contradictions |
|
|
62
|
+
| `genie brain status` | Brain dashboard with stats |
|
|
63
|
+
|
|
64
|
+
## Rules
|
|
65
|
+
|
|
66
|
+
- Review traces weekly for production brains
|
|
67
|
+
- Purge traces older than 90 days to keep the database lean
|
|
68
|
+
- Always include `--reason` when setting strategy configs — future you will thank you
|
|
69
|
+
- Fix knowledge gaps before tuning strategies — missing content causes more issues than wrong strategies
|
|
70
|
+
- Use `--failed` filter to focus on actual problems, not noise
|
|
71
|
+
|
|
72
|
+
## Composition
|
|
73
|
+
|
|
74
|
+
- Chains with **/brain-search** for re-testing after tuning
|
|
75
|
+
- Chains with **/brain-ingest** to fill knowledge gaps identified by traces
|
|
76
|
+
- Chains with **/brain-health** for comprehensive quality assessment
|
|
77
|
+
- Follow-up to **/brain-build** Phase 5 (ongoing tuning)
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: brain-search
|
|
3
|
+
description: Search a brain knowledge base with confidence scoring and strategy selection
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# /brain-search — Knowledge Base Search
|
|
7
|
+
|
|
8
|
+
Search a brain knowledge base using hybrid retrieval strategies, evaluate result confidence, and cite sources with appropriate authority levels. Supports both quick RAG lookups and deep CAG reasoning for complex queries.
|
|
9
|
+
|
|
10
|
+
## When to Use
|
|
11
|
+
|
|
12
|
+
- Agent needs to find information from the knowledge base
|
|
13
|
+
- Agent needs to verify facts or claims against stored knowledge
|
|
14
|
+
- Agent needs to locate specific documents, entities, or references
|
|
15
|
+
- Agent needs to answer a question using brain content as ground truth
|
|
16
|
+
|
|
17
|
+
## Prerequisites
|
|
18
|
+
|
|
19
|
+
- Brain must exist (`genie brain status`)
|
|
20
|
+
- Brain must be indexed (`genie brain update`) so chunks and embeddings are available
|
|
21
|
+
- For CAG strategy, rlmx must be reachable (uses LLM reasoning over full documents)
|
|
22
|
+
|
|
23
|
+
## Flow
|
|
24
|
+
|
|
25
|
+
1. **Intent detection** -- determine what the user is really asking
|
|
26
|
+
2. **Strategy selection** -- pick RAG (general) or CAG (complex reasoning) based on query type
|
|
27
|
+
3. **Search execution** -- run the query against the indexed brain
|
|
28
|
+
4. **Confidence evaluation** -- score results and decide citation authority
|
|
29
|
+
5. **Citation** -- present findings with the appropriate confidence framing
|
|
30
|
+
|
|
31
|
+
## Commands
|
|
32
|
+
|
|
33
|
+
| Command | Description |
|
|
34
|
+
|---------|-------------|
|
|
35
|
+
| `genie brain search "<query>" --brain <id>` | Main search -- auto-selects strategy |
|
|
36
|
+
| `genie brain search "<query>" --brain <id> --strategy rag` | Force RAG strategy (BM25 + Trigram + Vector via RRF) |
|
|
37
|
+
| `genie brain search "<query>" --brain <id> --strategy cag` | Force CAG strategy (RAG then full doc retrieval then rlmx reasoning) |
|
|
38
|
+
| `genie brain search "<query>" --brain <id> --limit 20` | Return more results (default varies by strategy) |
|
|
39
|
+
| `genie brain search "<query>" --brain <id> --min-confidence 0.7` | Filter out results below confidence threshold |
|
|
40
|
+
| `genie brain analyze "<query>" --brain <id>` | Deep analysis via rlmx -- synthesizes across multiple documents |
|
|
41
|
+
| `genie brain analyze "<query>" --brain <id> --mode synthesize` | Synthesis mode -- combines information from multiple sources |
|
|
42
|
+
| `genie brain get <path\|#docid> --brain <id>` | Retrieve the full content of a specific document by path or doc ID |
|
|
43
|
+
|
|
44
|
+
## Strategy Guide
|
|
45
|
+
|
|
46
|
+
| Strategy | Method | Best For |
|
|
47
|
+
|----------|--------|----------|
|
|
48
|
+
| `rag` (default) | Fuses BM25 + Trigram + Vector search via Reciprocal Rank Fusion | General queries, factual lookups, keyword-heavy searches |
|
|
49
|
+
| `cag` | RAG results then full document retrieval then rlmx reasoning | Complex questions needing synthesis, "why" and "how" questions, multi-hop reasoning |
|
|
50
|
+
| Auto (no flag) | Auto-selects based on query type using strategy config or default RAG | Most cases -- let the system decide |
|
|
51
|
+
|
|
52
|
+
**When to force a strategy:**
|
|
53
|
+
|
|
54
|
+
- Force `--strategy rag` when you need fast, precise lookups and the query is straightforward
|
|
55
|
+
- Force `--strategy cag` when the answer requires reasoning across multiple documents or the question is analytical
|
|
56
|
+
- Leave strategy unset for most queries and let the system auto-select
|
|
57
|
+
|
|
58
|
+
## Confidence Decision Tree
|
|
59
|
+
|
|
60
|
+
| Level | Threshold | How to Cite |
|
|
61
|
+
|-------|-----------|-------------|
|
|
62
|
+
| **FULL** | >= 0.80 | Cite with authority. "According to [doc]..." |
|
|
63
|
+
| **HIGH** | >= 0.70 | Cite with confidence. "The brain indicates..." |
|
|
64
|
+
| **PARTIAL** | >= 0.50 | Cite with caveat. "Based on available information..." |
|
|
65
|
+
| **LOW** | >= 0.30 | Mention uncertainty. "I found limited information suggesting..." |
|
|
66
|
+
| **NONE** | < 0.30 | Do not fabricate. "No relevant information found in the brain." |
|
|
67
|
+
|
|
68
|
+
## Rules
|
|
69
|
+
|
|
70
|
+
- NEVER fabricate information that is not present in search results
|
|
71
|
+
- ALWAYS include the confidence level when citing brain content
|
|
72
|
+
- Prefer CAG for "why" and "how" questions that require reasoning
|
|
73
|
+
- Use `--min-confidence` to filter noise in brains with lots of low-relevance content
|
|
74
|
+
- When confidence is NONE, say so explicitly -- do not guess or hallucinate an answer
|
|
75
|
+
- Use `genie brain get` to retrieve the full document when a chunk looks relevant but incomplete
|
|
76
|
+
- For ambiguous queries, run a broad search first then refine with tighter terms
|
|
77
|
+
|
|
78
|
+
## Composition
|
|
79
|
+
|
|
80
|
+
- Chains with **/brain-analyze** -- escalate to deep analysis when RAG results are insufficient
|
|
81
|
+
- Chains with **/brain-ingest** -- if search reveals knowledge gaps, ingest new content to fill them
|
|
82
|
+
- Chains with **/brain-learn** -- recall agent learnings and preferences alongside factual search
|