nano-brain 2026.6.26 → 2026.6.102

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (95) hide show
  1. package/README.md +347 -886
  2. package/npm/postinstall.js +161 -0
  3. package/npm/run.js +24 -0
  4. package/package.json +22 -47
  5. package/.opencode/command/nano-brain-init.md +0 -47
  6. package/.opencode/command/nano-brain-reindex.md +0 -56
  7. package/.opencode/command/nano-brain-status.md +0 -49
  8. package/.opencode/command/opsx-apply.md +0 -149
  9. package/.opencode/command/opsx-archive.md +0 -154
  10. package/.opencode/command/opsx-explore.md +0 -170
  11. package/.opencode/command/opsx-propose.md +0 -103
  12. package/.opencode/skills/openspec-apply-change/SKILL.md +0 -156
  13. package/.opencode/skills/openspec-archive-change/SKILL.md +0 -114
  14. package/.opencode/skills/openspec-explore/SKILL.md +0 -288
  15. package/.opencode/skills/openspec-propose/SKILL.md +0 -110
  16. package/AGENTS.md +0 -95
  17. package/AGENTS_SNIPPET.md +0 -51
  18. package/SKILL.md +0 -145
  19. package/bin/cli.js +0 -29
  20. package/dist/web/assets/index-BTxK9Q9T.js +0 -509
  21. package/dist/web/assets/index-FNtOdy8n.css +0 -1
  22. package/dist/web/index.html +0 -13
  23. package/opencode-mcp.json +0 -9
  24. package/src/bandits.ts +0 -144
  25. package/src/bench.ts +0 -1133
  26. package/src/cache.ts +0 -59
  27. package/src/categorizer.ts +0 -61
  28. package/src/chunker.ts +0 -553
  29. package/src/codebase.ts +0 -876
  30. package/src/collections.ts +0 -274
  31. package/src/connection-graph.ts +0 -50
  32. package/src/consolidation-worker.ts +0 -109
  33. package/src/consolidation.ts +0 -436
  34. package/src/db/corruption-recovery.ts +0 -285
  35. package/src/embeddings.ts +0 -433
  36. package/src/entity-extraction.ts +0 -124
  37. package/src/entity-merger.ts +0 -267
  38. package/src/event-store.ts +0 -75
  39. package/src/expansion.ts +0 -61
  40. package/src/extraction.ts +0 -234
  41. package/src/flow-detection.ts +0 -250
  42. package/src/graph.ts +0 -710
  43. package/src/harvester.ts +0 -854
  44. package/src/host.ts +0 -31
  45. package/src/importance.ts +0 -101
  46. package/src/index.ts +0 -3975
  47. package/src/intent-classifier.ts +0 -56
  48. package/src/llm-categorizer.ts +0 -99
  49. package/src/llm-provider.ts +0 -140
  50. package/src/logger.ts +0 -122
  51. package/src/memory-graph.ts +0 -192
  52. package/src/metrics.ts +0 -98
  53. package/src/preference-model.ts +0 -142
  54. package/src/providers/qdrant.ts +0 -259
  55. package/src/providers/sqlite-vec.ts +0 -227
  56. package/src/pruning.ts +0 -83
  57. package/src/reranker.ts +0 -102
  58. package/src/search.ts +0 -736
  59. package/src/sequence-analyzer.ts +0 -422
  60. package/src/server.ts +0 -3867
  61. package/src/service-installer.ts +0 -260
  62. package/src/storage.ts +0 -269
  63. package/src/store.ts +0 -3058
  64. package/src/symbol-graph.ts +0 -691
  65. package/src/symbols.ts +0 -556
  66. package/src/telemetry.ts +0 -105
  67. package/src/treesitter.ts +0 -860
  68. package/src/types.ts +0 -885
  69. package/src/vector-store.ts +0 -84
  70. package/src/watcher.ts +0 -852
  71. package/src/web/index.html +0 -12
  72. package/src/web/package-lock.json +0 -3088
  73. package/src/web/package.json +0 -38
  74. package/src/web/src/App.tsx +0 -28
  75. package/src/web/src/api/client.ts +0 -232
  76. package/src/web/src/components/GraphCanvas.tsx +0 -102
  77. package/src/web/src/components/Layout.tsx +0 -88
  78. package/src/web/src/components/NodeDetail.tsx +0 -27
  79. package/src/web/src/components/SearchResult.tsx +0 -44
  80. package/src/web/src/index.css +0 -65
  81. package/src/web/src/lib/colors.ts +0 -60
  82. package/src/web/src/lib/graph-adapter.ts +0 -239
  83. package/src/web/src/main.tsx +0 -28
  84. package/src/web/src/store/app.ts +0 -11
  85. package/src/web/src/views/CodeGraph.tsx +0 -90
  86. package/src/web/src/views/ConnectionsView.tsx +0 -198
  87. package/src/web/src/views/Dashboard.tsx +0 -137
  88. package/src/web/src/views/FlowsView.tsx +0 -117
  89. package/src/web/src/views/GraphExplorer.tsx +0 -94
  90. package/src/web/src/views/InfrastructureView.tsx +0 -182
  91. package/src/web/src/views/Search.tsx +0 -63
  92. package/src/web/src/views/SymbolGraph.tsx +0 -110
  93. package/src/web/tsconfig.json +0 -20
  94. package/src/web/vite.config.ts +0 -18
  95. package/src/workspace-profile.ts +0 -64
package/README.md CHANGED
@@ -1,998 +1,459 @@
1
1
  # nano-brain
2
2
 
3
- Persistent memory and code intelligence for AI coding agents.
3
+ **Persistent memory and code intelligence for AI coding agents.**
4
4
 
5
- ## What It Does
5
+ [![Go 1.23](https://img.shields.io/badge/Go-1.23-00ADD8?logo=go)](https://go.dev/)
6
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
7
+ [![GitHub](https://img.shields.io/badge/GitHub-nano--step%2Fnano--brain-181717?logo=github)](https://github.com/nano-step/nano-brain)
6
8
 
7
- A persistent memory server for AI coding agents. It solves the #1 problem with AI assistants: **they forget everything between sessions.**
9
+ ## What It Does
8
10
 
9
- nano-brain automatically ingests your AI sessions, notes, and codebase, indexes everything with full-text search + vector embeddings + knowledge graph, serves memories via 22 MCP tools, and learns which memories matter most to you over time.
11
+ nano-brain is a persistent memory server for AI coding agents that solves session amnesia. It automatically ingests AI sessions, notes, and codebase files, indexes everything with hybrid search (BM25 + pgvector), and serves memories via MCP tools and REST API. Built in Go with PostgreSQL single static binary, zero CGO dependencies.
10
12
 
11
13
  ## Key Features
12
14
 
13
- - **Hybrid search pipeline** — BM25 + vector + RRF fusion + VoyageAI neural reranking with 6 ranking signals
14
- - **Code intelligence** — symbol graph, call flow detection, impact analysis, change detection via Tree-sitter AST
15
- - **Automatic data ingestion** — session harvesting (2min poll), file watching (chokidar), codebase indexing
16
- - **Multi-workspace isolation** — per-workspace SQLite databases, cross-workspace search with `--scope=all`
17
- - **Flexible embedding providers** — VoyageAI, Ollama, OpenAI-compatible
18
- - **Dual vector stores** — Qdrant (production) or sqlite-vec (embedded)
19
- - **Privacy-first** 100% local processing option, your code never leaves your machine
20
- - **MCP + CLI** — stdio/HTTP/SSE transports for local or containerized environments
21
- - **Automatic corruption recovery** — detects & recovers from database corruption on startup with zero user intervention
22
- - **Self-learning system** — Thompson Sampling tunes search parameters, preference learning personalizes results
23
- - **Knowledge graph** — LLM-extracted entities and relationships, graph traversal, temporal queries
24
- - **Memory intelligence** — LLM categorization, entity pruning, proactive suggestions
25
-
26
- Inspired by [QMD](https://github.com/tobi/qmd) and [OpenClaw](https://github.com/openclaw/openclaw).
15
+ - **Hybrid search** — BM25 full-text + pgvector HNSW cosine similarity + RRF fusion + recency decay
16
+ - **9 MCP tools** — query, search, vsearch, get, write, tags, status, update, wake_up
17
+ - **Session harvesting** — auto-ingest OpenCode and Claude Code sessions
18
+ - **File watcher** — fsnotify-based directory monitoring with debounce
19
+ - **Content-addressed storage** — SHA-256 deduplication
20
+ - **Heading-aware markdown chunking**
21
+ - **Multi-workspace isolation** with per-workspace data
22
+ - **Config hot-reload** — `POST /api/reload-config`
23
+ - **V1 migration** — import from SQLite (pure Go, no CGO)
24
+ - **Benchmarking suite** — generate, run, compare, stress
25
+ - **Search telemetry** — local-only, 90-day retention, non-blocking
27
26
 
28
- ## Architecture
27
+ ## Prerequisites
29
28
 
30
- ```
31
- User Query
32
-
33
-
34
- ┌─────────────────┐
35
- │ Query Expansion │ ← (currently stubbed, planned)
36
- │ (optional) │ generates 2-3 query variants
37
- └────────┬────────┘
38
-
39
- ┌────┴────┐
40
- ▼ ▼
41
- ┌────────┐ ┌──────────┐
42
- │ BM25 │ │ Vector │
43
- │ (FTS5) │ │ (Qdrant │
44
- │ │ │ or │
45
- │ │ │ sqlite- │
46
- │ │ │ vec) │
47
- └───┬────┘ └────┬─────┘
48
- │ │
49
- ▼ ▼
50
- ┌─────────────────┐
51
- │ RRF Fusion │ ← k=60, original query 2× weight
52
- │ │
53
- └────────┬────────┘
54
-
55
-
56
- ┌─────────────────┐
57
- │ PageRank Boost │ ← Centrality from file dependency graph
58
- │ │ weight: 0.1 (default)
59
- └────────┬────────┘
60
-
61
-
62
- ┌─────────────────┐
63
- │ Supersede │ ← 0.3× demotion for replaced documents
64
- │ Demotion │
65
- └────────┬────────┘
66
-
67
-
68
- ┌─────────────────┐
69
- │ Neural Reranking│ ← VoyageAI rerank-2.5-lite
70
- │ (optional) │
71
- └────────┬────────┘
72
-
73
-
74
- ┌─────────────────┐
75
- │ Position-Aware │ ← top 3: 75/25, 4-10: 60/40, 11+: 40/60
76
- │ Blending │ (RRF weight / rerank weight)
77
- └────────┬────────┘
78
-
79
-
80
- Final Results
81
- ```
29
+ - **Go 1.23+** (building from source) OR pre-built binary
30
+ - **PostgreSQL 17** with **pgvector 0.8.2** extension
31
+ - **Embedding provider:** Ollama (default, local) or Voyage AI
82
32
 
83
- ### Write Pipeline
33
+ ## Quick Start
84
34
 
85
- ```
86
- Memory Write
87
-
88
-
89
- ┌─────────────────┐
90
- │ Save to File │ → ~/.nano-brain/memory/
91
- │ Hash + DB Insert │ → documents + content tables
92
- │ FTS5 Index │ → documents_fts (auto-trigger)
93
- └────────┬────────┘
94
-
95
- ┌────┴────┐
96
- ▼ ▼
97
- ┌────────┐ ┌──────────┐
98
- │Keyword │ │ Async │ (fire-and-forget)
99
- │Categorize│ │ Processes│
100
- │auto:* │ │ │
101
- └────────┘ ├──────────┤
102
- │ LLM │ → llm:* tags
103
- │ Categorize│
104
- ├──────────┤
105
- │ Entity │ → knowledge graph
106
- │ Extract │
107
- └──────────┘
108
- ```
35
+ ### Option A: Via npx (no Go required)
109
36
 
110
- ## Search Pipeline (3 Tiers)
111
-
112
- **`memory_search`** — BM25 only (fast, exact keyword matching)
113
-
114
- **`memory_vsearch`** — Vector only (semantic similarity via embeddings)
115
-
116
- **`memory_query`** — Full hybrid pipeline with 6 ranking signals:
117
-
118
- 1. **BM25 full-text scoring** — SQLite FTS5 with porter stemming
119
- 2. **Vector cosine similarity** — Qdrant or sqlite-vec embeddings
120
- 3. **RRF fusion** — k=60, original query weighted 2×
121
- 4. **PageRank centrality boost** — from file dependency graph (weight: 0.1)
122
- 5. **Supersede demotion** — 0.3× penalty for replaced documents
123
- 6. **VoyageAI neural reranking** — rerank-2.5-lite with position-aware blending:
124
- - Top 3 results: 75% RRF / 25% rerank
125
- - Ranks 4-10: 60% RRF / 40% rerank
126
- - Ranks 11+: 40% RRF / 60% rerank
127
-
128
- Query expansion generates 2-3 query variants before search. The pipeline supports it, but no expansion provider is currently active.
129
-
130
- ## Code Intelligence
131
-
132
- Built on Tree-sitter AST parsing for TypeScript, JavaScript, and Python:
133
-
134
- **`code_context`** — 360° view of a code symbol:
135
- - Direct callers and callees
136
- - Transitive call flows (upstream/downstream)
137
- - File location, definition, and references
138
- - Centrality score (PageRank) and cluster membership
139
-
140
- **`code_impact`** — Change impact analysis:
141
- - Upstream dependencies (what calls this?)
142
- - Downstream dependencies (what does this call?)
143
- - BFS traversal with configurable depth
144
- - Risk assessment for refactoring
145
-
146
- **`code_detect_changes`** — Map git diff to affected symbols:
147
- - Parses `git diff` output
148
- - Identifies modified symbols via Tree-sitter
149
- - Returns symbol names, types, and file locations
150
- - Scope: `staged`, `unstaged`, or `all`
151
-
152
- **`memory_focus`** — File dependency context:
153
- - Import/export graph for a file
154
- - Centrality score (PageRank)
155
- - Cluster membership (Louvain algorithm)
156
- - Direct dependencies and dependents
157
-
158
- **`memory_graph_stats`** — Dependency graph overview:
159
- - Total files, symbols, edges
160
- - Cycle detection
161
- - Clustering coefficient
162
- - Top central files
163
-
164
- **Symbol tracking** — Cross-repo symbol queries:
165
- - Redis keys, PubSub channels
166
- - MySQL tables, columns
167
- - API endpoints (Express, FastAPI)
168
- - Bull/BullMQ queues
169
- - GraphQL types, queries, mutations
170
-
171
- ## Data Ingestion
172
-
173
- All data sources are indexed automatically:
174
-
175
- **Session harvesting** — Converts OpenCode JSON sessions into searchable markdown:
176
- - Polls `~/.opencode/sessions/` every 2 minutes
177
- - Extracts user queries, assistant responses, tool calls
178
- - Incremental append (hash-based deduplication)
179
-
180
- **File watching** — Monitors collections for changes:
181
- - Chokidar watches configured directories
182
- - Dirty-flag tracking for incremental updates
183
- - Reindexes every 5 minutes if changes detected
184
-
185
- **Codebase indexing** — Tree-sitter AST → symbol graph:
186
- - Parses TS/JS/Python files
187
- - Extracts functions, classes, methods, variables
188
- - Builds call graph (caller → callee edges)
189
- - Computes PageRank centrality
190
- - Detects clusters via Louvain algorithm
191
- - Identifies call flows (entry points → leaf functions)
192
-
193
- **Incremental behavior**:
194
- - Hash-based file skipping (SHA-256 content addressing)
195
- - Adaptive embedding backoff (exponential retry)
196
- - Batch processing for large codebases
197
-
198
- ## Background Jobs
199
-
200
- nano-brain runs 9 background jobs to keep your memory fresh and intelligent:
201
-
202
- | Job | Interval | What It Does |
203
- |-----|----------|-------------|
204
- | File reindex | 5 min | Watch collections, reindex changed files |
205
- | Session harvest | 2 min | Convert OpenCode sessions → searchable markdown |
206
- | Embedding | 60s (adaptive) | Generate vector embeddings for new docs |
207
- | Learning cycle | 10 min | Thompson Sampling + preference weight updates |
208
- | Consolidation | 1 hour | LLM summarizes related memories |
209
- | Importance | 30 min | Rescore document importance from usage |
210
- | Sequence analysis | 30 min | Detect query patterns for proactive suggestions |
211
- | Pruning (soft) | 6 hours | Soft-delete contradicted/orphan entities |
212
- | Pruning (hard) | 7 days | Permanently delete old soft-deleted entities |
213
-
214
- ## Chunking Strategy
215
-
216
- Heading-aware markdown chunking that respects document structure:
217
-
218
- - **Target size:** 900 tokens (~3600 characters)
219
- - **Overlap:** 15% between chunks (~540 characters)
220
- - **Respects boundaries:** Code fences, headings, paragraphs
221
- - **Break point scoring:** h1=100, h2=90, h3=80, code-fence=80, hr=60, blank-line=40
222
- - **Content-addressed storage:** SHA-256 hash deduplication
223
-
224
- ## Storage & Infrastructure
225
-
226
- **SQLite** (via better-sqlite3):
227
- - `documents` — metadata, content, embeddings
228
- - `chunks` — heading-aware markdown chunks (900 tokens, 15% overlap)
229
- - `fts_index` — FTS5 virtual table with porter stemming
230
- - `vec_index` — sqlite-vec extension (cosine distance)
231
- - `symbols` — code symbols (functions, classes, variables)
232
- - `call_edges` — caller → callee relationships
233
- - `file_deps` — import/export graph
234
- - `clusters` — Louvain clustering results
235
- - `flows` — detected call flows (entry → leaf)
236
-
237
- **Qdrant** (optional, production vector store):
238
- - Managed via `qdrant up/down/status/migrate/verify/activate/cleanup` commands
239
- - Docker-based deployment
240
- - Automatic migration from sqlite-vec
241
- - Verification and cleanup tools
242
-
243
- **Embedding providers**:
244
- - **VoyageAI** — voyage-code-3 (1024 dims, code-optimized)
245
- - **Ollama** — local models (nomic-embed-text, etc.)
246
- - **OpenAI-compatible** — Azure, LM Studio, custom endpoints
247
-
248
- **Reranking**:
249
- - **VoyageAI** — rerank-2.5-lite (neural reranking)
250
-
251
- **Storage management**:
252
- - Per-workspace SQLite databases (isolated)
253
- - Content-addressed storage (SHA-256 deduplication)
254
- - Retention policies (maxSize budget, auto-cleanup)
255
- - Disk space checks before indexing
256
-
257
- ## Database Schema
258
-
259
- nano-brain uses 18 SQLite tables organized into 5 functional groups:
260
-
261
- | Table | Purpose |
262
- |-------|---------|
263
- | **Core Documents** | |
264
- | `documents` | Document metadata, content, embeddings |
265
- | `chunks` | Heading-aware markdown chunks (900 tokens, 15% overlap) |
266
- | `content` | Raw content storage (content-addressed) |
267
- | **Search Indexes** | |
268
- | `documents_fts` | FTS5 full-text search index (porter stemming) |
269
- | `vec_index` | sqlite-vec vector index (cosine distance) |
270
- | **Code Intelligence** | |
271
- | `symbols` | Code symbols (functions, classes, variables) |
272
- | `call_edges` | Caller → callee relationships |
273
- | `file_deps` | Import/export graph |
274
- | `clusters` | Louvain clustering results |
275
- | `flows` | Detected call flows (entry → leaf) |
276
- | **Knowledge Graph** | |
277
- | `entities` | LLM-extracted entities (people, concepts, tools) |
278
- | `relationships` | Entity-to-entity connections |
279
- | **Learning & Intelligence** | |
280
- | `telemetry` | Search queries, results, expand feedback |
281
- | `bandit_variants` | Thompson Sampling search parameter tuning |
282
- | `config_versions` | Search config version history |
283
- | `consolidations` | LLM-generated memory summaries |
284
- | `query_sequences` | Query pattern detection for proactive suggestions |
285
- | `category_preferences` | Per-workspace category weights from expand patterns |
286
-
287
- ## Database Reliability & Corruption Recovery
288
-
289
- **Why corruption happens**:
290
- SQLite databases can become corrupted due to:
291
- - Unexpected process termination during write operations
292
- - Filesystem crashes or power loss during WAL (Write-Ahead Log) checkpoint
293
- - Disk I/O errors or hardware faults
294
- - Rare race conditions in concurrent access (even with better-sqlite3 serialization)
295
-
296
- **Automatic corruption detection**:
297
- nano-brain automatically detects database corruption on startup via `PRAGMA integrity_check`:
298
- - Runs before any database operations in `createStore()`
299
- - Checks database file integrity without modifying data
300
- - Takes 50-500ms depending on database size
301
-
302
- **Automatic recovery**:
303
- When corruption is detected:
304
- 1. **Backup corrupted file** — Renamed to `.corrupted.{ISO-timestamp}` for forensics/recovery
305
- 2. **Clear WAL/SHM files** — Removes Write-Ahead Log and shared memory files
306
- 3. **Initialize fresh database** — Creates clean SQLite database from scratch
307
- 4. **Verify fresh database** — Runs integrity check to confirm recovery succeeded
308
- 5. **Emit metric** — `database_corruption_detected` counter for monitoring/alerting
309
-
310
- **Why this works**:
311
- The database is a **cache/index** — all data is re-derivable from source files. Recovery involves:
312
- - Session harvesting (re-ingests from session logs)
313
- - Codebase reindexing (rescan source files)
314
- - Memory re-embedding (regenerates vectors)
315
- - Call graph rebuilding (reparses symbols)
316
-
317
- **Automatic restart with launchd**:
318
- On macOS, nano-brain runs as a launchd service (`com.tamlh.nano-brain`):
319
- - If corruption causes a fatal error, process exits
320
- - launchd automatically restarts it after 10-second throttle
321
- - On restart, `checkAndRecoverDB()` detects corruption and recovers
322
- - Service comes back online automatically with fresh database
323
-
324
- **Installation (macOS)**:
325
37
  ```bash
326
- # Copy plist to launchd directory
327
- cp ~/.config/nano-brain/launchd/com.tamlh.nano-brain.plist ~/Library/LaunchAgents/
328
-
329
- # Load the service
330
- launchctl load ~/Library/LaunchAgents/com.tamlh.nano-brain.plist
331
-
332
- # Check status
333
- launchctl list | grep nano-brain
334
- ```
38
+ # Start PostgreSQL + pgvector
39
+ docker run -d --name nanobrain-pg -p 5432:5432 \
40
+ -e POSTGRES_USER=nanobrain -e POSTGRES_PASSWORD=nanobrain -e POSTGRES_DB=nanobrain_dev \
41
+ pgvector/pgvector:pg17
335
42
 
336
- **Monitoring**:
337
- Check for corruption metrics in your monitoring/alerting system:
338
- - Counter: `database_corruption_detected`
339
- - Alert threshold: > 3 events per 24 hours (indicates underlying hardware/filesystem issue)
43
+ # Start Ollama + pull embedding model
44
+ ollama pull nomic-embed-text
340
45
 
341
- **Troubleshooting**:
342
- If corruption happens frequently:
343
- 1. Check system logs for disk I/O errors: `log stream --predicate 'eventMessage contains[c] "I/O error"'`
344
- 2. Verify filesystem health: `diskutil verifyVolume /` (macOS)
345
- 3. Check disk space: `df -h ~/.nano-brain/`
346
- 4. Review database size: `ls -lh ~/.nano-brain/index.db`
347
- 5. Consider moving database to a different drive if corruption persists
348
-
349
- **Forensics**:
350
- Corrupted backups are kept for analysis:
351
- - Located at: `~/.nano-brain/index.db.corrupted.{ISO-timestamp}`
352
- - Last 5 backups are kept by default; older ones are auto-cleaned
353
- - File size indicates when corruption occurred (if truncated vs intact)
354
-
355
- ## MCP Tools (22+ Total)
356
-
357
- ### Search & Retrieval
358
-
359
- | Tool | Description |
360
- |------|-------------|
361
- | `memory_search` | BM25 keyword search (fast, exact matching) |
362
- | `memory_vsearch` | Semantic vector search (embeddings) |
363
- | `memory_query` | Full hybrid search (BM25 + vector + RRF + reranking) |
364
- | `memory_get` | Retrieve document by path or docid (#abc123) |
365
- | `memory_multi_get` | Batch retrieve by glob pattern |
366
-
367
- ### Memory Management
368
-
369
- | Tool | Description |
370
- |------|-------------|
371
- | `memory_write` | Write to daily log (supports tags, supersedes) |
372
- | `memory_tags` | List all tags with document counts |
373
- | `memory_status` | Index health, collections, model status, graph stats |
374
- | `memory_update` | Trigger reindex of all collections |
375
- | `memory_consolidate` | Trigger LLM memory consolidation |
376
- | `memory_suggestions` | Proactive next-query predictions based on patterns |
46
+ # Check prerequisites
47
+ npx @nano-step/nano-brain@beta doctor
377
48
 
378
- ### Code Intelligence
379
-
380
- | Tool | Description |
381
- |------|-------------|
382
- | `code_context` | 360° view of a code symbol (callers, callees, flows, centrality) |
383
- | `code_impact` | Change impact analysis (upstream/downstream BFS) |
384
- | `code_detect_changes` | Map git diff to affected symbols (staged/unstaged/all) |
385
- | `memory_index_codebase` | Index codebase files in current workspace (Tree-sitter AST) |
386
-
387
- ### Dependency Graph
49
+ # Start server
50
+ npx @nano-step/nano-brain@beta
51
+ ```
388
52
 
389
- | Tool | Description |
390
- |------|-------------|
391
- | `memory_focus` | File dependency context (imports/exports, centrality, cluster) |
392
- | `memory_graph_stats` | Dependency graph overview (files, symbols, edges, cycles) |
393
- | `memory_symbols` | Cross-repo symbol query (Redis, MySQL, API endpoints, queues) |
394
- | `memory_impact` | Cross-repo impact analysis (writers vs readers) |
395
- | `memory_graph_query` | BFS traversal from entity through knowledge graph |
396
- | `memory_related` | Find related memories via entity graph connections |
397
- | `memory_timeline` | Temporal view of entity changes over time |
53
+ > **Also available as:** `npx nano-brain@beta` (unscoped alias)
54
+ >
55
+ > **Note:** Do NOT run `npx nano-brain` from the nano-brain source directory npm will resolve the local package instead of the registry. Run from any other directory.
398
56
 
399
- ## Installation & Quick Start
57
+ ### Option B: Build from source
400
58
 
401
59
  ```bash
402
- # Install globally
403
- npm install -g nano-brain
60
+ # Build
61
+ CGO_ENABLED=0 go build -o nano-brain ./cmd/nano-brain
404
62
 
405
- # Initialize (creates config, indexes codebase, generates embeddings)
406
- npx nano-brain init --root=/path/to/your/project
63
+ # Start PostgreSQL + pgvector (example with Docker)
64
+ docker run -d --name nanobrain-pg -p 5432:5432 \
65
+ -e POSTGRES_USER=nanobrain -e POSTGRES_PASSWORD=nanobrain -e POSTGRES_DB=nanobrain_dev \
66
+ pgvector/pgvector:pg17
407
67
 
408
- # Check everything is working
409
- npx nano-brain status
410
- ```
411
-
412
- ### MCP Configuration
68
+ # Start server
69
+ DATABASE_URL="postgres://nanobrain:nanobrain@localhost:5432/nanobrain_dev" ./nano-brain
413
70
 
414
- Add to your AI agent's MCP config (e.g., `~/.config/opencode/opencode.json`):
415
-
416
- **Local mode (stdio):**
417
- ```json
418
- {
419
- "mcp": {
420
- "nano-brain": {
421
- "type": "local",
422
- "command": ["npx", "nano-brain", "mcp"],
423
- "enabled": true
424
- }
425
- }
426
- }
427
- ```
71
+ # Register workspace
72
+ curl -X POST http://localhost:3100/api/v1/init \
73
+ -H "Content-Type: application/json" \
74
+ -d '{"root_path":"/path/to/project","name":"my-project"}'
428
75
 
429
- **Remote mode (HTTP/SSE, for Docker/containers):**
430
- ```json
431
- {
432
- "mcp": {
433
- "nano-brain": {
434
- "type": "remote",
435
- "url": "http://host.docker.internal:3100/mcp",
436
- "enabled": true
437
- }
438
- }
439
- }
440
- ```
76
+ # Write a document
77
+ curl -X POST http://localhost:3100/api/v1/write \
78
+ -H "Content-Type: application/json" \
79
+ -d '{"workspace":"<hash>","source_path":"notes/decision.md","content":"# Decision\nUse PostgreSQL.","tags":["decision"]}'
441
80
 
442
- Start the remote server:
443
- ```bash
444
- npx nano-brain serve # Background daemon (port 3100)
445
- npx nano-brain serve --foreground # Foreground (for debugging)
446
- npx nano-brain serve status # Check if running
447
- npx nano-brain serve stop # Stop daemon
81
+ # Search
82
+ curl -X POST http://localhost:3100/api/v1/query \
83
+ -H "Content-Type: application/json" \
84
+ -d '{"workspace":"<hash>","query":"database decision"}'
448
85
  ```
449
86
 
450
87
  ## Configuration
451
88
 
452
- Create `~/.nano-brain/config.yml` (auto-generated by `init`):
89
+ Config file: `~/.nano-brain/config.yml`
453
90
 
454
91
  ```yaml
455
- # Collections (directories to index)
456
- collections:
457
- memory:
458
- path: ~/.nano-brain/memory
459
- pattern: "**/*.md"
460
- update: auto
461
- sessions:
462
- path: ~/.nano-brain/sessions
463
- pattern: "**/*.md"
464
- update: auto
465
-
466
- # Vector store (qdrant or sqlite-vec)
467
- vector:
468
- provider: qdrant
469
- url: http://localhost:6333
470
- collection: nano_brain
471
- # OR: provider: sqlite-vec (embedded, no external service)
472
-
473
- # Embedding provider
474
- embedding:
475
- provider: openai # 'ollama' or 'openai' (OpenAI-compatible)
476
- url: https://api.voyageai.com # VoyageAI, Azure, LM Studio, etc.
477
- model: voyage-code-3
478
- apiKey: ${VOYAGE_API_KEY}
479
- # OR: provider: ollama, url: http://localhost:11434, model: nomic-embed-text
480
-
481
- # Reranker (uses embedding.apiKey if not set separately)
482
- reranker:
483
- model: rerank-2.5-lite
484
- # apiKey: ${VOYAGE_API_KEY} # optional, falls back to embedding.apiKey
485
-
486
- # Codebase indexing
487
- codebase:
488
- enabled: true
489
- languages: [typescript, javascript, python]
490
- exclude: [node_modules, dist, build, .git]
491
- maxFileSize: 1048576 # 1MB
92
+ server:
93
+ host: localhost
94
+ port: 3100
492
95
 
493
- # File watcher
494
- watcher:
495
- enabled: true
496
- debounce: 300 # ms
497
- reindexInterval: 300 # seconds (5 minutes)
96
+ database:
97
+ url: postgres://nanobrain:nanobrain@localhost:5432/nanobrain_dev
98
+
99
+ embedding:
100
+ provider: ollama # ollama or voyage
101
+ url: http://localhost:11434
102
+ model: nomic-embed-text
103
+ dimension: 0 # auto-detect from provider
104
+ concurrency: 3
498
105
 
499
- # Search configuration
500
106
  search:
501
107
  rrf_k: 60
502
- top_k: 30
503
- expansion:
504
- enabled: true
505
- weight: 1
506
- reranking:
507
- enabled: true
508
- blending:
509
- top3:
510
- rrf: 0.75
511
- rerank: 0.25
512
- mid:
513
- rrf: 0.60
514
- rerank: 0.40
515
- tail:
516
- rrf: 0.40
517
- rerank: 0.60
518
- centrality_weight: 0.1
519
- supersede_demotion: 0.3
520
-
521
- # Polling intervals
522
- intervals:
523
- sessionHarvest: 120 # seconds (2 minutes)
524
- healthCheck: 60 # seconds
525
-
526
- # Storage management
527
- storage:
528
- maxSize: 10737418240 # 10GB
529
- retention:
530
- sessions: 90 # days
531
- logs: 30 # days
532
-
533
- # Workspaces
534
- workspaces:
535
- isolation: true # Per-workspace SQLite databases
536
- defaultScope: current # or 'all' for cross-workspace search
537
-
538
- # Entity pruning (Memory Intelligence v2)
539
- pruning:
540
- enabled: true
541
- interval_ms: 21600000 # 6 hours
542
- contradicted_ttl_days: 30
543
- orphan_ttl_days: 90
544
- batch_size: 100
545
- hard_delete_after_days: 30
546
-
547
- # LLM categorization (Memory Intelligence v2)
548
- categorization:
549
- llm_enabled: true
550
- confidence_threshold: 0.6
551
- max_content_length: 2000
552
-
553
- # Preference learning (Memory Intelligence v2)
554
- preferences:
555
- enabled: true
556
- min_queries: 20
557
- weight_min: 0.5
558
- weight_max: 2.0
559
- baseline_expand_rate: 0.1
560
-
561
- # Logging
562
- logging:
563
- level: info # debug, info, warn, error
564
- file: ~/.nano-brain/logs/nano-brain.log
565
- maxSize: 10485760 # 10MB
566
- maxFiles: 5
567
- ```
568
-
569
- **Data directory layout (`~/.nano-brain/`):**
570
- ```
571
- ~/.nano-brain/
572
- ├── config.yml # Configuration
573
- ├── data/ # SQLite databases (per-workspace)
574
- ├── memory/ # Curated notes
575
- ├── sessions/ # Harvested sessions
576
- └── logs/ # Application logs
577
- ```
578
-
579
- ## CLI Commands (24 Total)
108
+ recency_weight: 0.3
109
+ recency_half_life_days: 180
110
+ limit: 20
111
+
112
+ harvester:
113
+ opencode:
114
+ db_root: "" # e.g., ~/.ai-sandbox/opencode-dbs (multi-DB, highest priority)
115
+ db_path: "" # e.g., ~/.local/share/opencode/opencode.db (single DB)
116
+ session_dir: "" # e.g., ~/.local/share/opencode/storage (legacy JSON)
117
+ claudecode:
118
+ enabled: false
119
+ session_dir: ""
580
120
 
581
- ### Setup & Initialization
582
-
583
- ```bash
584
- nano-brain init # Full initialization (config, index, embed, AGENTS.md)
585
- nano-brain init --root=/path # Initialize for specific project
586
- nano-brain status # Show index health, collections, model status
587
- ```
121
+ watcher:
122
+ debounce_ms: 2000
123
+ reindex_interval: 300
124
+ # Per-collection exclude_patterns and allowed_extensions are also supported
125
+ # via the workspaces map. See "Global ignore patterns" section below for
126
+ # the cross-collection .nano-brainignore file.
588
127
 
589
- ### MCP Server
128
+ storage:
129
+ max_file_size: 314572800 # 300MB
130
+ max_size: 10737418240 # 10GB
590
131
 
591
- ```bash
592
- nano-brain mcp # Start MCP server (stdio)
593
- nano-brain mcp --http --port=3100 --host=0.0.0.0 # Start MCP server (HTTP/SSE)
594
- ```
132
+ telemetry:
133
+ retention_days: 90
595
134
 
596
- ### Remote Server (Daemon)
135
+ logging:
136
+ level: info
137
+ file: "" # empty = stdout only
597
138
 
598
- ```bash
599
- nano-brain serve # Start SSE server as background daemon (port 3100)
600
- nano-brain serve status # Check if server is running
601
- nano-brain serve stop # Stop the daemon
602
- nano-brain serve --foreground # Run in foreground (for debugging)
603
- nano-brain serve --port=8080 # Custom port
139
+ summarization:
140
+ enabled: false # set to true to generate LLM summaries of harvested sessions
141
+ provider_url: "" # OpenAI-compatible endpoint, e.g. https://ai-proxy.example.com/v1
142
+ api_key: "" # or set NANO_BRAIN_SUMMARIZE_API_KEY env var
143
+ model: "nano-brain" # model name passed to the provider
144
+ max_tokens: 8000 # max tokens per LLM completion
145
+ concurrency: 3 # parallel map-phase LLM calls
604
146
  ```
605
147
 
606
- ### Search
607
-
608
- ```bash
609
- nano-brain search "query" # BM25 keyword search
610
- nano-brain vsearch "query" # Vector semantic search
611
- nano-brain query "query" # Hybrid search (BM25 + vector + reranking)
612
- nano-brain query "query" --tags=bug,fix # Filter by tags
613
- nano-brain query "query" --scope=all # Cross-workspace search
614
- ```
148
+ ### Authentication (VPS / remote deployment)
615
149
 
616
- ### Memory Management
150
+ When binding to a non-loopback address, enable auth to protect your memory:
617
151
 
618
- ```bash
619
- nano-brain write "content" # Write to daily log
620
- nano-brain write "content" --tags=decision,architecture
621
- nano-brain write "content" --supersedes=abc123 # Mark as replacement
622
- nano-brain get <path> # Retrieve document by path
623
- nano-brain get "#abc123" # Retrieve by docid
624
- nano-brain tags # List all tags with counts
152
+ ```yaml
153
+ server:
154
+ host: 0.0.0.0
155
+ port: 3100
156
+ auth:
157
+ enabled: true
158
+ realm: nano-brain
159
+ users:
160
+ - username: admin
161
+ password_hash: "$2a$10$..." # from: nano-brain auth hash <password>
162
+ tokens:
163
+ - "nbt_..." # from: nano-brain auth token
164
+ bypass_paths:
165
+ - /health
625
166
  ```
626
167
 
627
- ### Index Management
168
+ Generate credentials:
628
169
 
629
170
  ```bash
630
- nano-brain update # Reindex all collections
631
- nano-brain index-codebase # Index codebase in current workspace
632
- nano-brain reset --confirm # Reset all data (requires confirmation)
633
- nano-brain reset --dry-run # Preview what would be deleted
634
- ```
635
-
636
- ### Collections
171
+ # Generate bcrypt hash for Basic Auth
172
+ nano-brain auth hash mypassword
637
173
 
638
- ```bash
639
- nano-brain collection add <name> <path> # Add collection
640
- nano-brain collection remove <name> # Remove collection
641
- nano-brain collection list # List collections
174
+ # Generate bearer token
175
+ nano-brain auth token
642
176
  ```
643
177
 
644
- ### Workspace Management
178
+ Usage examples:
645
179
 
646
180
  ```bash
647
- nano-brain rm --list # List all workspaces
648
- nano-brain rm <workspace> --dry-run # Preview what would be deleted
649
- nano-brain rm <workspace> # Remove workspace and all its data
650
- # <workspace> can be: absolute path, hash prefix, or workspace name
651
- ```
181
+ # Basic Auth
182
+ curl -u admin:mypassword http://host:3100/api/v1/query -d '{"query":"test"}'
652
183
 
653
- ### Qdrant Management
184
+ # Bearer token
185
+ curl -H "Authorization: Bearer nbt_..." http://host:3100/api/v1/query -d '{"query":"test"}'
654
186
 
655
- ```bash
656
- nano-brain qdrant up # Start Qdrant Docker container
657
- nano-brain qdrant down # Stop Qdrant container
658
- nano-brain qdrant status # Check Qdrant status
659
- nano-brain qdrant migrate # Migrate from sqlite-vec to Qdrant
660
- nano-brain qdrant verify # Verify Qdrant data integrity
661
- nano-brain qdrant activate # Switch to Qdrant (update config)
662
- nano-brain qdrant cleanup # Remove orphaned vectors
187
+ # MCP client with URL-embedded credentials
188
+ # url: http://admin:mypassword@host:3100/mcp
663
189
  ```
664
190
 
665
- ### Cache Management
191
+ ### Global ignore patterns (`~/.nano-brain/.nano-brainignore`)
666
192
 
667
- ```bash
668
- nano-brain cache clear # Clear all caches
669
- nano-brain cache clear --type=embeddings # Clear specific cache type
670
- nano-brain cache stats # Show cache statistics
671
- ```
672
-
673
- ### Benchmarking
193
+ The watcher loads a global gitignore-style file at `~/.nano-brain/.nano-brainignore`
194
+ on startup. Patterns in this file apply to **all** registered collections, removing
195
+ the need to repeat the same exclusions in each `watcher.workspaces` block.
674
196
 
675
- ```bash
676
- nano-brain bench # Run default benchmark suite
677
- nano-brain bench --suite=search # Run specific suite
678
- nano-brain bench --iterations=100 --json --save
679
- nano-brain bench --compare=baseline.json # Compare with baseline
680
- ```
197
+ **Format**: standard `.gitignore` syntax (one pattern per line, supports `**`, `!negation`, blank lines, `#` comments).
681
198
 
682
- ### Logging
199
+ **Example** `~/.nano-brain/.nano-brainignore`:
683
200
 
684
- ```bash
685
- nano-brain logs # Show recent logs (last 50 lines)
686
- nano-brain logs -f # Tail logs in real-time
687
- nano-brain logs -n 100 # Show last 100 lines
688
- nano-brain logs --date=2026-03-01 # Show log for specific date
689
- nano-brain logs --clear # Delete all log files
690
- nano-brain logs path # Print log directory path
691
201
  ```
202
+ # Skip generated files everywhere
203
+ *.png
204
+ *.jpg
205
+ *.pdf
206
+ build/
207
+ dist/
208
+ node_modules/
692
209
 
693
- ## Project Structure
694
-
695
- ```
696
- src/
697
- ├── index.ts # CLI entry point
698
- ├── server.ts # MCP server (22+ tools, stdio/HTTP/SSE)
699
- ├── store.ts # SQLite storage (FTS5 + sqlite-vec)
700
- ├── storage.ts # Storage management (retention, disk space)
701
- ├── vector-store.ts # Vector store abstraction (Qdrant + sqlite-vec)
702
- ├── search.ts # Hybrid search pipeline (RRF, reranking, blending)
703
- ├── chunker.ts # Heading-aware markdown chunking
704
- ├── collections.ts # YAML config, collection scanning
705
- ├── embeddings.ts # Embedding providers (VoyageAI, Ollama, OpenAI-compatible)
706
- ├── reranker.ts # VoyageAI reranker
707
- ├── expansion.ts # Query expansion (interface only, no active provider)
708
- ├── harvester.ts # OpenCode session → markdown converter
709
- ├── watcher.ts # File watcher (chokidar, dirty flags)
710
- ├── codebase.ts # Codebase indexing orchestrator
711
- ├── treesitter.ts # Tree-sitter AST parsing
712
- ├── symbols.ts # Symbol extraction (functions, classes, variables)
713
- ├── graph.ts # File dependency graph (imports/exports)
714
- ├── symbol-graph.ts # Symbol call graph (caller → callee)
715
- ├── flow-detection.ts # Call flow detection (entry → leaf)
716
- ├── types.ts # TypeScript interfaces
717
- └── providers/ # Vector store implementations
718
- ├── qdrant.ts # Qdrant vector store
719
- └── sqlite-vec.ts # sqlite-vec vector store
720
- bin/
721
- └── cli.js # CLI wrapper
722
-
723
- test/
724
- └── *.test.ts # 760+ tests (vitest)
725
- SKILL.md # AI agent routing instructions (auto-loaded by OpenCode)
726
- AGENTS_SNIPPET.md # Optional project-level AGENTS.md managed block
210
+ # But keep this one icon
211
+ !icons/important.png
727
212
  ```
728
213
 
729
- ## Tech Stack
730
-
731
- - **TypeScript + Node.js** (via tsx)
732
- - **better-sqlite3** + **sqlite-vec** for embedded storage
733
- - **Qdrant** for production vector store (optional)
734
- - **Tree-sitter** for AST parsing (TS, JS, Python)
735
- - **@modelcontextprotocol/sdk** for MCP server (stdio/HTTP/SSE transports)
736
- - **chokidar** for file watching
737
- - **vitest** for testing (760+ tests)
214
+ **Order of evaluation** (most aggressive first):
738
215
 
739
- ## Embedding & Reranking Providers
216
+ 1. Hardcoded default exclude dirs (`node_modules`, `.git`, `dist`, `build`, `target`, etc.)
217
+ 2. **Global `.nano-brainignore`** (this feature)
218
+ 3. Per-collection `.gitignore` (in collection root)
219
+ 4. Per-collection `exclude_patterns` (config-level)
220
+ 5. Per-collection `allowed_extensions` (whitelist)
740
221
 
741
- **Embeddings:**
742
- - **VoyageAI** — voyage-code-3 (1024 dims, code-optimized, recommended)
743
- - **Ollama** — nomic-embed-text, mxbai-embed-large, etc. (local, free)
744
- - **OpenAI-compatible** — Azure OpenAI, LM Studio, custom endpoints
222
+ **Restart required**: changes to `.nano-brainignore` take effect on next server start (hot-reload deferred to a follow-up). Issue #263.
745
223
 
746
- **Reranking:**
747
- - **VoyageAI** — rerank-2.5-lite (neural reranking, recommended)
224
+ ### Session Summarization
748
225
 
749
- **Query Expansion:**
750
- - Pipeline support exists but no active provider. The interface is ready for future integration.
226
+ When `summarization.enabled: true`, nano-brain automatically generates structured markdown summaries of each harvested session using an OpenAI-compatible LLM provider. Summaries are:
751
227
 
752
- ## How nano-brain Compares
228
+ - Stored in PostgreSQL under collection `session-summary` for semantic search via the standard query/vsearch API (PG is the source of truth)
229
+ - Optionally written to disk as Markdown files for Obsidian-compatible access (see [Disk persistence](#disk-persistence-obsidian-compatible) below)
230
+ - Idempotent — unchanged sessions are skipped; re-harvested sessions overwrite old summaries
753
231
 
754
- | | nano-brain | Mem0 / OpenMemory | Zep / Graphiti | OMEGA | Letta (MemGPT) | Claude Native |
755
- |---|---|---|---|---|---|---|
756
- | **Search** | Hybrid (BM25 + vector + 6 ranking signals) | Vector only | Graph traversal + vector | Semantic + BM25 | Agent-managed | Text file read |
757
- | **Storage** | SQLite + Qdrant (optional) | PostgreSQL + Qdrant | Neo4j | SQLite | PostgreSQL / SQLite | Flat text files |
758
- | **MCP Tools** | 22+ | 4-9 | 9-10 | 12 | 7 | 0 |
759
- | **Code Intelligence** | Yes (Tree-sitter AST, symbol graph, impact analysis) | No | No | No | No | No |
760
- | **Codebase Indexing** | Yes (AST → symbols → call graph → flows) | No | No | No | No | No |
761
- | **Session Recall** | Yes (auto-harvests past sessions) | No | No | No | No | Limited (CLAUDE.md) |
762
- | **Query Expansion** | Pipeline ready (no active provider) | No | No | No | No | No |
763
- | **Neural Reranking** | Yes (VoyageAI rerank-2.5-lite) | No | No | No | No | No |
764
- | **Local-First** | Yes (Ollama + sqlite-vec) | Requires OpenAI API key | Requires Docker + Neo4j | Yes | Yes | Yes |
765
- | **Cloud Option** | Yes (VoyageAI, OpenAI-compatible) | Cloud API (OpenAI) | Cloud API | Local ONNX | Cloud API | None |
766
- | **Privacy** | 100% local option available | Cloud API calls | Cloud or self-host | 100% local | Self-host or cloud | Local files |
767
- | **Dependencies** | SQLite + embedding API (+ optional Qdrant) | Docker + PostgreSQL + Qdrant + OpenAI key | Docker + Neo4j | SQLite + ONNX | PostgreSQL | None |
768
- | **Pricing** | Free (open source, MIT) | Free tier / Pro $249/mo | Free self-host / Cloud $25-475/mo | Free (Apache-2.0) | Free (Apache-2.0) | Free (with Claude) |
769
- | **GitHub Stars** | New | ~47K | ~23K | ~25 | ~21K | N/A |
232
+ #### Disk persistence (Obsidian-compatible)
770
233
 
771
- ### Where nano-brain shines
234
+ By default, summaries are written to disk as Markdown files at the path configured in
235
+ `summarization.output_dir` (default: `~/.nano-brain/summaries`). The file layout is:
772
236
 
773
- - **6-signal hybrid search** — BM25 + vector + RRF + PageRank + supersede + neural reranking in a single pipeline
774
- - **Code intelligence** — Tree-sitter AST parsing, symbol graph, call flow detection, impact analysis
775
- - **Codebase indexing** — index your source files with structural boundary detection, not just conversations
776
- - **Session recall** — automatically harvests and indexes past AI coding sessions
777
- - **Flexible deployment** — 100% local (Ollama + sqlite-vec) or cloud (VoyageAI + Qdrant)
778
- - **Privacy-first** — local processing option, your code never leaves your machine
779
-
780
- ### Consider alternatives if
781
-
782
- - You need a knowledge graph with temporal reasoning (Zep/Graphiti)
783
- - You want a full agent framework, not just memory (Letta)
784
- - You need cloud-hosted memory shared across teams (Mem0 Cloud)
785
- - You only need basic session notes (Claude native memory)
786
-
787
- ## AI Agent Integration
788
-
789
- nano-brain ships with a SKILL.md that teaches AI agents when and how to use memory tools. When loaded as an OpenCode skill, agents automatically:
790
-
791
- - **Check memory before starting work** — recall past decisions, patterns, and context
792
- - **Save context after completing work** — persist key decisions and debugging insights
793
- - **Route queries to the right search tool** — BM25 for exact terms, vector for concepts, hybrid for best quality
794
- - **Use code intelligence** — understand symbol relationships, assess change impact, detect affected code
795
-
796
- ### SKILL.md (Auto-loaded)
797
-
798
- The skill file at `SKILL.md` provides routing rules, trigger phrases, tool selection guides, and integration patterns. It's automatically loaded when any agent references the `nano-brain` skill.
799
-
800
- ### AGENTS_SNIPPET.md (Optional, project-level)
801
-
802
- For project-level integration, `AGENTS_SNIPPET.md` provides a managed block that can be injected into a project's `AGENTS.md`:
803
-
804
- ```bash
805
- npx nano-brain init --root=/path/to/project
237
+ ```
238
+ <output_dir>/<workspace_name>/<source>_<slugified-title>_<YYYY-MM-DD>.md
806
239
  ```
807
240
 
808
- This adds a managed block to your project's `AGENTS.md` with quick reference tables for CLI commands and MCP tools (if available).
809
-
810
- See [SKILL.md](./SKILL.md) for full routing rules and [AGENTS_SNIPPET.md](./AGENTS_SNIPPET.md) for the project-level snippet.
811
-
812
- ## Self-Learning Configuration
813
-
814
- nano-brain includes an adaptive self-learning system that improves search quality over time.
815
-
816
- ### Quick Start
241
+ Files are byte-identical to the `documents.content` field in PostgreSQL disk is a
242
+ derivative view, DB is source of truth. Disk write failures (permission denied, disk
243
+ full) log a WARN but do not roll back the DB transaction.
817
244
 
818
- Add to your `config.yml`:
245
+ To opt out (DB-only persistence):
819
246
 
820
247
  ```yaml
821
- telemetry:
822
- enabled: true # Log search queries and feedback (default: true)
823
- retention_days: 90 # How long to keep telemetry data
824
-
825
- learning:
826
- enabled: true # Enable adaptive search tuning
827
- update_interval_ms: 600000 # Cold-path update interval (10 min)
828
-
829
- consolidation:
830
- enabled: true # Enable memory consolidation
831
- interval_ms: 3600000 # Consolidation interval (60 min)
832
- model: gpt-4o-mini # LLM model for consolidation
833
- endpoint: https://api.openai.com/v1 # OpenAI-compatible endpoint
834
- apiKey: sk-... # API key (not required for Ollama)
835
- provider: openai # 'openai' (default) or 'ollama'
836
-
837
- extraction:
838
- enabled: true # Enable fact extraction from sessions
839
- model: gpt-4o-mini # LLM model for extraction
840
- endpoint: https://api.openai.com/v1
841
- apiKey: sk-...
842
- maxFactsPerSession: 20 # Max facts to extract per session
843
-
844
- importance:
845
- enabled: true # Enable importance scoring
846
- weight: 0.1 # Importance boost weight
847
- decay_half_life_days: 30
848
-
849
- intents:
850
- enabled: true # Enable query intent classification
248
+ summarization:
249
+ write_to_disk: false
851
250
  ```
852
251
 
853
- ### How It Works
854
-
855
- 1. **Telemetry**: Every search query is logged with results and timing. When you expand a result, it's recorded as positive feedback.
856
- 2. **Thompson Sampling**: Search parameters (rrf_k, centrality_weight) are automatically tuned using multi-armed bandits based on expand feedback.
857
- 3. **Consolidation**: Periodically, an LLM reviews recent memories and finds connections, generating consolidated insights.
858
- 4. **Importance Scoring**: Documents accessed frequently get boosted in search results. Unused documents decay over time.
859
- 5. **Intent Classification**: Queries are classified by intent (lookup, explanation, architecture, recall) and routed to optimized search configs.
860
-
861
- ### CLI Commands
252
+ To backfill historical summaries already in the DB:
862
253
 
863
- - `nano-brain learning rollback [version_id]` — View or rollback to a previous config version
864
- - `nano-brain consolidate` — Trigger a manual consolidation cycle
865
-
866
- ### MCP Tools
867
-
868
- - `memory_consolidate` — Trigger consolidation manually
869
- - `memory_consolidation_status` — View consolidation queue stats and recent logs
870
- - `memory_importance` — View document importance scores
871
- - `memory_status` — View learning system status (telemetry records, bandit variants, config version)
872
-
873
- ## Memory Intelligence v2
874
-
875
- nano-brain includes three advanced intelligence features that make your memory smarter over time.
876
-
877
- ### Entity Pruning
878
-
879
- Automatically cleans up outdated or contradicted information from the knowledge graph.
254
+ ```
255
+ nano-brain backfill-summaries
256
+ ```
880
257
 
881
- **How it works:**
882
- - Background job runs every 6 hours
883
- - Soft-deletes contradicted entities after 30 days (when newer information supersedes them)
884
- - Soft-deletes orphan entities after 90 days (entities with no relationships)
885
- - Hard-deletes soft-deleted entities after 30-day retention period
886
- - Processes in batches of 100 to avoid SQLite lock contention
258
+ **Quick setup with ai-proxy:**
887
259
 
888
- **Configuration:**
889
260
  ```yaml
890
- pruning:
261
+ summarization:
891
262
  enabled: true
892
- interval_ms: 21600000 # 6 hours
893
- contradicted_ttl_days: 30 # How long before contradicted entities are soft-deleted
894
- orphan_ttl_days: 90 # How long before orphan entities are soft-deleted
895
- batch_size: 100 # Max entities to process per cycle
896
- hard_delete_after_days: 30 # Retention period for soft-deleted entities
263
+ provider_url: "https://ai-proxy.example.com/v1"
264
+ api_key: "" # set NANO_BRAIN_SUMMARIZE_API_KEY instead
265
+ model: "claude-sonnet-4-5"
266
+ max_tokens: 8000
267
+ concurrency: 3
897
268
  ```
898
269
 
899
- **Why this matters:** Prevents your knowledge graph from accumulating stale information. When you update a decision or pattern, the old version is automatically pruned after the TTL expires.
270
+ Or via environment variable:
900
271
 
901
- ### LLM Categorization
272
+ ```bash
273
+ export NANO_BRAIN_SUMMARIZE_API_KEY="sk-..."
274
+ ```
902
275
 
903
- Adds semantic categorization to your memories using an LLM, going beyond simple keyword matching.
276
+ Large sessions (100K+ tokens) are handled via map-reduce chunking no session is too large.
904
277
 
905
- **How it works:**
906
- - Runs asynchronously after the keyword categorizer (fire-and-forget)
907
- - Uses the same LLM provider configured for consolidation (same endpoint/model/apiKey)
908
- - Adds tags prefixed with `llm:` (e.g., `llm:architecture-decision`, `llm:debugging-insight`)
909
- - Only processes documents under 2000 characters (configurable)
910
- - Requires confidence threshold of 0.6 or higher (configurable)
278
+ ### Environment Variables
911
279
 
912
- **7 semantic categories:**
913
- 1. `llm:architecture-decision` — System design choices, trade-offs, patterns
914
- 2. `llm:debugging-insight` Root cause analysis, fix patterns, gotchas
915
- 3. `llm:tool-config` Setup instructions, configuration patterns
916
- 4. `llm:pattern` Reusable code patterns, best practices
917
- 5. `llm:preference` User preferences, workflow choices
918
- 6. `llm:context` Background information, explanations
919
- 7. `llm:workflow` Process documentation, how-to guides
280
+ | Variable | Description |
281
+ |----------|-------------|
282
+ | `NANO_BRAIN_CONFIG` | Path to YAML config file (12-factor; useful in Docker/k8s). Precedence: `--config` flag > `NANO_BRAIN_CONFIG` > `~/.nano-brain/config.yml`. Leading/trailing whitespace is stripped. If the env-pointed file does not exist, a `WARNING:` is printed to stderr and defaults are used (operator can spot typos). |
283
+ | `DATABASE_URL` | PostgreSQL connection string |
284
+ | `VOYAGE_API_KEY` | Voyage AI API key |
285
+ | `OPENCODE_DB_ROOT` | OpenCode per-project DB root directory (multi-DB mode) |
286
+ | `OPENCODE_DB_PATH` | OpenCode single SQLite database path |
287
+ | `OPENCODE_STORAGE_DIR` | OpenCode session directory (legacy) |
288
+ | `NANO_BRAIN_SUMMARIZE_API_KEY` | API key for the summarization LLM provider |
289
+ | `NANO_BRAIN_AUTH_ENABLED` | Enable Basic Auth + Bearer Token (`true`/`false`) |
290
+ | `NANO_BRAIN_AUTH_TOKENS` | Comma-separated bearer tokens |
291
+ | `NANO_BRAIN_*` | Override any config field (e.g., `NANO_BRAIN_SERVER_PORT=3100`) |
920
292
 
921
- **Configuration:**
922
- ```yaml
923
- categorization:
924
- llm_enabled: true
925
- confidence_threshold: 0.6 # Minimum confidence to apply a category
926
- max_content_length: 2000 # Skip documents longer than this
927
- ```
293
+ **Docker example** — run the server in a container against a host PostgreSQL:
928
294
 
929
- **Why this matters:** Enables semantic filtering like `--tags=llm:architecture-decision` to find all design decisions, even if they don't use the word "architecture."
295
+ ```bash
296
+ # /path/to/container-config.yml uses host.docker.internal for DB/Ollama
297
+ docker run -d \
298
+ -e NANO_BRAIN_CONFIG=/etc/nano-brain/config.yml \
299
+ -v /path/to/container-config.yml:/etc/nano-brain/config.yml:ro \
300
+ -p 3100:3100 \
301
+ nano-brain:latest
302
+ ```
303
+
304
+ ## REST API
305
+
306
+ ### Public Endpoints
307
+
308
+ | Method | Path | Description |
309
+ |--------|------|-------------|
310
+ | GET | `/health` | Health check |
311
+ | GET | `/api/status` | Server status with version, uptime, workspace stats |
312
+ | POST | `/api/v1/init` | Register workspace |
313
+ | GET | `/api/v1/workspaces` | List all workspaces (with doc counts) |
314
+ | DELETE | `/api/v1/workspaces/:hash` | Permanently delete a workspace + cascade docs/chunks/embeddings |
315
+ | GET | `/api/v1/wake-up` | Workspace briefing |
316
+ | POST | `/api/harvest` | Trigger session harvesting |
317
+ | POST | `/api/reload-config` | Hot-reload configuration |
318
+
319
+ ### Workspace-Scoped Endpoints
320
+
321
+ Workspace is passed in the JSON body for POST, query param for GET.
322
+
323
+ | Method | Path | Description |
324
+ |--------|------|-------------|
325
+ | POST | `/api/v1/write` | Write/update document |
326
+ | POST | `/api/v1/embed` | Trigger embedding |
327
+ | POST | `/api/v1/search` | BM25 keyword search |
328
+ | POST | `/api/v1/vsearch` | Vector similarity search |
329
+ | POST | `/api/v1/query` | Hybrid search (BM25 + vector + RRF + recency) |
330
+ | POST | `/api/v1/collections` | Add collection |
331
+ | GET | `/api/v1/collections` | List collections |
332
+ | PUT | `/api/v1/collections/:name` | Rename collection |
333
+ | DELETE | `/api/v1/collections/:name` | Remove collection |
334
+ | GET | `/api/v1/tags` | List tags with counts |
335
+ | POST | `/api/v1/get` | Get single document by source_path or id |
336
+ | POST | `/api/v1/multi-get` | Batch fetch documents by paths or ids |
337
+ | POST | `/api/v1/reindex` | Queue reindex (202) |
338
+ | POST | `/api/v1/update` | Queue update (202) |
339
+ | POST | `/api/v1/summarize` | Trigger LLM summarization of harvested sessions |
340
+ | POST | `/api/v1/wake-up` | Workspace briefing with session_dir |
341
+
342
+ ### MCP Endpoints
343
+
344
+ | Method | Path | Description |
345
+ |--------|------|-------------|
346
+ | GET/POST | `/mcp` | Streamable HTTP (MCP 2025-03-26) |
347
+ | GET/POST | `/sse` | SSE transport (legacy) |
348
+
349
+ ## CLI Commands
350
+
351
+ | Command | Description |
352
+ |---------|-------------|
353
+ | `nano-brain` (no args) | Start HTTP server (default: port 3100) |
354
+ | `nano-brain init --root=<path>` | Register workspace |
355
+ | `nano-brain workspaces list` | List registered workspaces with doc counts |
356
+ | `nano-brain workspaces remove --workspace=<hash> [--dry-run\|--force]` | Permanently delete a workspace + all its documents/chunks/embeddings |
357
+ | `nano-brain write` | Write document via CLI |
358
+ | `nano-brain query [--scope=all] [--tags=t1,t2]` | Hybrid search (BM25 + vector + RRF + recency) |
359
+ | `nano-brain search [--scope=all] [--tags=t1,t2]` | BM25 keyword search |
360
+ | `nano-brain vsearch [--scope=all] [--tags=t1,t2]` | Vector similarity search |
361
+ | `nano-brain wake-up --workspace=<hash>` | Workspace briefing (collections, stats, recent memories) |
362
+ | `nano-brain get <source_path\|uuid> --workspace=<hash>` | Fetch a single document by source_path or UUID |
363
+ | `nano-brain tags --workspace=<hash>` | List all tags with document counts |
364
+ | `nano-brain multi-get --workspace=<hash> --paths=p1,p2` | Fetch multiple documents in one round-trip |
365
+ | `nano-brain collection add\|remove\|list` | Manage collections |
366
+ | `nano-brain harvest` | Trigger session harvesting |
367
+ | `nano-brain backfill-summaries [--dry-run] [--workspace=] [--since=]` | Export existing DB summaries to disk (.md files for Obsidian etc.) |
368
+ | `nano-brain cleanup-stale-raw [--dry-run]` | Delete pre-#192 raw OpenCode session docs superseded by summaries |
369
+ | `nano-brain cleanup-orphan-workspaces [--dry-run]` | Delete documents/chunks under workspace_hash values not registered in `workspaces`. Run BEFORE migration 00011 (issue #238). |
370
+ | `nano-brain bench generate\|run\|compare\|stress` | Benchmarking suite |
371
+ | `nano-brain db:migrate` | Run pending goose migrations |
372
+ | `nano-brain db:migrate --from-v1 <path>` | Import V1 SQLite data |
373
+ | `nano-brain logs [-n 50] [-f]` | Tail log file |
374
+ | `nano-brain docker start\|stop\|status` | Docker compose management |
375
+ | `nano-brain status [--json]` | Server status |
376
+ | `nano-brain auth hash <password>` | Generate bcrypt password hash for config |
377
+ | `nano-brain auth token` | Generate random bearer token (`nbt_`-prefixed) |
378
+ | `nano-brain doctor [--json]` | Check prerequisites (config, PostgreSQL, pgvector, Ollama, model) |
379
+
380
+ ## MCP Tools
381
+
382
+ nano-brain exposes 13 tools via MCP (Model Context Protocol):
930
383
 
931
- ### Preference Learning
384
+ | Tool | Description |
385
+ |------|-------------|
386
+ | `memory_query` | Hybrid search (BM25 + vector + RRF + recency) |
387
+ | `memory_search` | BM25 keyword search |
388
+ | `memory_vsearch` | Vector similarity search |
389
+ | `memory_get` | Get document by path |
390
+ | `memory_write` | Write/update document |
391
+ | `memory_tags` | List tags with counts |
392
+ | `memory_status` | Server and embedding status |
393
+ | `memory_update` | Trigger re-embedding |
394
+ | `memory_wake_up` | Workspace briefing |
395
+ | `memory_graph` | Knowledge graph view (module → function → dep) |
396
+ | `memory_trace` | Call chain trace from entry point |
397
+ | `memory_impact` | Cross-file change impact analysis |
398
+ | `memory_symbols` | Symbol search (functions, types, constants) |
932
399
 
933
- Learns which types of memories you expand most often and boosts them in future searches.
400
+ ### MCP Configuration
934
401
 
935
- **How it works:**
936
- - Tracks expand events per category per workspace
937
- - Calculates category weights based on expand rate vs baseline (10% default)
938
- - Applies weights as multipliers in search scoring (0.5× to 2.0× range)
939
- - Cold start: uses neutral weights until 20 queries collected
940
- - Updates every 10 minutes via learning cycle background job
402
+ ```json
403
+ {
404
+ "mcp": {
405
+ "nano-brain": {
406
+ "type": "remote",
407
+ "url": "http://localhost:3100/mcp"
408
+ }
409
+ }
410
+ }
411
+ ```
941
412
 
942
- **Example:** If you expand `llm:debugging-insight` memories 30% of the time (vs 10% baseline), those memories get a 1.5× boost in future searches.
413
+ ## Search Pipeline
943
414
 
944
- **Configuration:**
945
- ```yaml
946
- preferences:
947
- enabled: true
948
- min_queries: 20 # Minimum queries before personalization kicks in
949
- weight_min: 0.5 # Minimum category weight (demotion)
950
- weight_max: 2.0 # Maximum category weight (boost)
951
- baseline_expand_rate: 0.1 # Expected expand rate (10%)
415
+ ```
416
+ Query --> BM25 (ts_rank_cd) ---+
417
+ +--> RRF Fusion (k=60) --> Recency Decay --> Results
418
+ Query --> Vector (HNSW cos) ---+
952
419
  ```
953
420
 
954
- **Why this matters:** Your memory system adapts to your workflow. If you're debugging, debugging insights automatically surface higher. If you're designing, architecture decisions get prioritized.
955
-
956
- ## Proactive Intelligence
421
+ - **BM25:** `websearch_to_tsquery` + `ts_rank_cd` on PostgreSQL tsvector
422
+ - **Vector:** pgvector HNSW index with cosine distance
423
+ - **RRF:** Reciprocal Rank Fusion (k=60), scores normalized to [0,1]
424
+ - **Recency:** exponential half-life decay (default 180 days, weight 0.3)
957
425
 
958
- nano-brain can predict what you'll need next based on your query patterns.
426
+ ## Architecture
959
427
 
960
- ### How It Works
428
+ - 15 internal packages: config, server, handlers, storage, sqlc, embed, search, watcher, harvest, mcp, migrate, telemetry, health, bench
429
+ - 7 goose SQL migrations (embedded)
430
+ - Constructor injection (no DI framework)
431
+ - errgroup + context for goroutine lifecycle
432
+ - Echo v4 middleware: workspace extraction, content-type enforcement, version header
961
433
 
962
- 1. **Query chains**: Groups of related queries within 5-minute windows are detected
963
- 2. **Clustering**: Similar queries are grouped into semantic clusters using embeddings
964
- 3. **Transition learning**: The system learns which clusters follow which (e.g., "auth questions" → "token refresh questions")
965
- 4. **Predictions**: When you ask about topic A, nano-brain predicts you'll need topic B next
434
+ ## Migration from V1
966
435
 
967
- ### Configuration
436
+ ```bash
437
+ # Import V1 SQLite data to PostgreSQL
438
+ nano-brain db:migrate --from-v1 /path/to/old/index.db
968
439
 
969
- ```yaml
970
- proactive:
971
- enabled: true
972
- chain_timeout_ms: 300000 # 5 min window for grouping queries
973
- min_queries_for_prediction: 50 # Minimum queries before predictions activate
974
- max_suggestions: 5
975
- confidence_threshold: 0.3
976
- cluster_count: 50
977
- analysis_interval_ms: 1800000 # Rebuild every 30 min
440
+ # Idempotent — safe to run multiple times
441
+ # Uses content-addressed SHA-256 hashing
442
+ # Pure Go SQLite reader (modernc.org/sqlite, no CGO)
978
443
  ```
979
444
 
980
- ### MCP Tool
981
-
982
- - `memory_suggestions` — Get predicted next queries based on current context
983
- - `context` (optional): Current query or topic
984
- - `workspace` (optional): Workspace path
985
- - `limit` (optional): Max suggestions (default 3)
986
-
987
- ## Troubleshooting
988
-
989
- **LLM provider unreachable**: Check endpoint URL and network connectivity. For Ollama, ensure `ollama serve` is running.
990
-
991
- **Invalid API key**: Verify `apiKey` in config or `CONSOLIDATION_API_KEY` env var. Ollama doesn't require an API key.
992
-
993
- **Empty LLM responses**: Check model name is correct. For Ollama, run `ollama list` to see available models.
445
+ ## Tech Stack
994
446
 
995
- **Consolidation not running**: Ensure `consolidation.enabled: true` and check `memory_consolidation_status` for queue state.
447
+ - **Go 1.23** compiled to single static binary (`CGO_ENABLED=0`)
448
+ - **PostgreSQL 17** — relational storage + full-text search (tsvector/tsquery)
449
+ - **pgvector 0.8.2** — HNSW vector indexing
450
+ - **Echo v4** — HTTP framework
451
+ - **sqlc** — type-safe SQL code generation
452
+ - **goose v3** — database migrations
453
+ - **zerolog** — structured JSON logging
454
+ - **koanf** — YAML + env configuration
455
+ - **fsnotify** — file system watching
456
+ - **modernc.org/sqlite** — V1 migration reader (pure Go)
996
457
 
997
458
  ## License
998
459