@naveenadi/mnemonic 0.1.0 → 0.2.0

package/README.md

@@ -2,6 +2,12 @@
 
 On-device hybrid search for markdown knowledge bases. BM25 + vector + LLM reranking with link graphs, time decay, and HyDE. Designed for [pi](https://github.com/earendil-works/pi) coding agent.
 
+## Quick Start
+
+### Global mode (default)
+
+All collections share one index at `~/.cache/mnemonic/index.sqlite`. Search everything at once.
+
 ```bash
 npm install -g @naveenadi/mnemonic
 mne init
@@ -11,6 +17,18 @@ mne embed
 mne query "what was the Q4 planning discussion"
 ```
 
+### Project-local mode
+
+Use `--db` for a per-repo index. Keeps project docs separate.
+
+```bash
+mne --db .mnemonic/index.sqlite init
+mne --db .mnemonic/index.sqlite collection add . --name myproject
+mne --db .mnemonic/index.sqlite index
+mne --db .mnemonic/index.sqlite embed
+mne --db .mnemonic/index.sqlite query "deploy steps"
+```
+
 ## Features
 
 - **Hybrid search** — BM25 (FTS5) + Vector embeddings + RRF fusion
@@ -27,17 +45,31 @@ mne query "what was the Q4 planning discussion"
 
 ## Pi Integration
 
-Three layers:
+Three layers plus an interactive setup command, each installable **globally** (all projects) or **per project**.
+
+| Layer | What | Global path | Per-project path |
+|---|---|---|---|
+| **Interactive** | `/mne init` walks through setup | — | — |
+| **MCP server** | Typed tools: `query`, `get`, `multi_get`, `status` | `~/.pi/agent/mcp.json` | `.pi/mcp.json` |
+| **Pi skill** | Bash commands via `SKILL.md` | `~/.pi/agent/skills/mnemonic/` | `.pi/skills/mnemonic/` |
+| **Pi extension** | 4 tools + `/mne` command via `pi.registerTool/Command` | `~/.pi/agent/extensions/mnemonic/` | `.pi/extensions/mnemonic/` |
 
-| Layer | What | How |
-|---|---|---|
-| **Pi Skill** | `SKILL.md` | Bash commands via `mne search`, `mne query`, `mne get` |
-| **MCP Server** | `mne mcp` | Stdio + HTTP, typed tools: `query`, `get`, `multi_get`, `status` |
-| **Pi Extension** | `src/pi-extension/index.ts` | 4 custom tools registered with `pi.registerTool()` |
+### Interactive setup (extension required)
 
-Configure MCP in `~/.pi/agent/mcp.json`:
+After installing the extension and running `/reload`, type:
+
+```
+/mne init
+```
+
+This walks through: scope (global vs project) → add directories → index → embed → configure MCP → install skill.
+
+Other commands: `/mne add <path>`, `/mne status`, `/mne help`.
+
+### MCP — global
 
 ```json
+// ~/.pi/agent/mcp.json
 {
   "mcpServers": {
     "mnemonic": {
@@ -49,6 +81,38 @@ Configure MCP in `~/.pi/agent/mcp.json`:
 }
 ```
 
+### MCP — per project
+
+Same config in `.pi/mcp.json` (project root).
+
+### Skill — global
+
+```bash
+mkdir -p ~/.pi/agent/skills/mnemonic
+cp SKILL.md ~/.pi/agent/skills/mnemonic/
+```
+
+### Skill — per project
+
+```bash
+mkdir -p .pi/skills/mnemonic
+cp SKILL.md .pi/skills/mnemonic/
+```
+
+### Extension — global
+
+```bash
+mkdir -p ~/.pi/agent/extensions/mnemonic
+cp src/pi-extension/index.ts ~/.pi/agent/extensions/mnemonic/
+```
+
+### Extension — per project
+
+```bash
+mkdir -p .pi/extensions/mnemonic
+cp src/pi-extension/index.ts .pi/extensions/mnemonic/
+```
+
 ## Architecture
 
 ```
@@ -88,6 +152,14 @@ mne orphans                  Find orphan documents
 mne mcp                      Start MCP server
 ```
 
+## References
+
+- `SKILL.md` — Pi skill for agentic workflows (dig loop, cross-reference, setup)
+- `references/setup.md` — Detailed CLI setup and diagnostics
+- `references/pi-integration.md` — Pi integration: MCP, skill, extension (both modes)
+- `references/link-graph.md` — Cross-reference commands and usage
+- `src/pi-extension/` — Pi extension source + standalone package.json
+
 ## License
 
 MIT

package/SKILL.md

@@ -1,158 +1,118 @@
 ---
 name: mnemonic
-description: Search local markdown knowledge bases, notes, docs, and wikis with mnemonic. Hybrid BM25 + vector + LLM reranking. Use when users ask to find notes, retrieve documents, or answer from indexed markdown.
+description: Search local indexed markdown knowledge bases. Use when the user asks to find notes, dig up a concept from personal docs, cross-reference ideas across wikis, or answer from indexed local files. Triggers on "look up in notes", "search my docs", "what did I write about", "find in my vault", "check my index", "retrieve from mne".
 license: MIT
-compatibility: Requires mnemonic CLI. Install via `npm install -g @naveenadi/mnemonic`. Initialize with `mne init`.
+compatibility: Requires `@naveenadi/mnemonic` CLI. Install via `npm install -g @naveenadi/mnemonic`.
 metadata:
   version: "0.1.0"
 allowed-tools: Bash(mne:*)
 ---
 
-# mnemonic — On-Device Hybrid Search
+# mnemonic — dig your local index
 
-## How search works
+mnemonic runs searches against a local SQLite index of markdown files — notes, docs, wikis, vaults. Three branches: **dig** (the loop), **setup** (add collections), **cross-reference** (follow links between documents). Most runs only need the dig loop.
 
-mnemonic searches local markdown collections: notes, docs, wikis, and project
-knowledge bases. Use it before web search when the answer may already be in
-indexed local files.
+## Dig loop
 
-The workflow is always:
-
-1. Check what's indexed with `mne status`.
-2. Search for candidate documents.
-3. Retrieve the full source with `mne get` or `mne multi-get`.
-4. Answer from retrieved text, citing paths or docids.
-
-Do not answer from snippets alone when the user needs facts, decisions, quotes,
-or nuance. Snippets are only leads.
-
-Typical loop:
+Run through every time. Every **dig** completes when the answer cites the documents it came from — **docid** and **line numbers** on every claim.
 
 ```bash
+# 1. Check what's indexed
 mne status
-mne query "merchant reality support interviews" -n 5
-# leads: #abc123 concepts/customer-proximity.md; #def432 sources/merchant-call.md
-mne get "#abc123" --no-line-numbers
 ```
 
-## Pick the right search mode
-
-Use **BM25 lexical search** when you know exact words, titles, names, code
-symbols, or rare phrases:
-
 ```bash
-mne search "cockpit OKR Goodhart" -n 10
-mne search '"AI Before Headcount"' -c concepts -n 5
-```
+# 2. Find candidates
+# Exact terms? Use BM25:
+mne search "cockpit OKR Goodhart" -n 5
 
-Use **`mne query` with structured fields** when the user describes an idea
-indirectly, uses different wording than the source, or needs conceptual recall.
-Write the fields yourself rather than leaning on query expansion. Combine exact
-anchors with semantic recall:
-
-```bash
-mne query $'intent: Find the concept note about metrics as instruments without letting OKRs replace judgment.\nlex: cockpit instruments OKR Goodhart metrics judgment\nvec: data informed not metric driven product judgment\nhyde: A concept note says metrics are useful like cockpit instruments, but leaders should remain data-informed rather than metric-driven because OKRs and dashboards can Goodhart product judgment.'
+# Concept, vague wording, or the user paraphrases? Use hybrid with structured fields:
+mne query $'intent: Find the concept note about metrics as instruments without replacing judgment.\nlex: cockpit instruments OKR Goodhart metrics\nvec: data informed not metric driven product judgment\nhyde: A concept note explains metrics are cockpit instruments that should inform, not drive, product judgment.'
 ```
 
-Structured query fields (you author each one):
+Structured query fields:
+- `intent:` what you're trying to find **and what to avoid**
+- `lex:` exact terms, titles, code symbols, rare words
+- `vec:` paraphrase of the idea in natural language
+- `hyde:` a hypothetical document that would answer the request
 
-- `intent:` states what you are trying to find **and what to avoid**. Always
-  supply this.
-- `lex:` exact terms, aliases, titles, code symbols, and rare words.
-- `vec:` paraphrases the idea in natural language.
-- `hyde:` a hypothetical document that would answer the request.
-
-You do not need all four every time, but always supply at least `intent:` plus
-one of `lex:`/`vec:`.
-
-## Retrieve sources
-
-Search results include docids like `#abc123` and `mne://` paths. Fetch them:
+Always write `intent:` plus at least one of `lex:`/`vec:`. You know the domain and what to dodge — do not delegate this to the expansion model.
 
 ```bash
+# 3. Retrieve matched documents
+# Results carry a docid (#abc123) and mne:// path
 mne get "#abc123"
-mne get mne://concepts/ai-before-headcount.md
+mne get "#abc123:120:40"       # 40 lines from line 120
 mne multi-get "#abc123,#def432" --json
-mne get "#abc123:120:40"           # 40 lines starting at line 120
-mne get mne://concepts/note.md -l 80 --from 200
-```
-
-Output is line-numbered by default and carries the docid:
-
-```text
-mne://concepts/note.md  #abc123
----
 
-1: # Metrics as instruments
-2:
-3: Treat dashboards like cockpit instruments...
+# Output is line-numbered by default; cite line numbers with every claim
 ```
 
-Cite the docid and exact line numbers in your answer. Pass `--no-line-numbers`
-only when you need raw content to copy verbatim.
-
-## Discover what is indexed
+**Completion criterion**: every claim backed by a docid and line numbers. Do not answer from snippets alone — fetch the source.
 
 ```bash
-mne collection list
-mne ls
-mne status
-mne ls concepts               # List files in a collection
+# 4. Scope collections when results drift
+mne query "headcount autonomous agents" -c concepts -n 10
+mne ls concepts                 # List files in a collection
 ```
 
-Add collection filters to scope searches:
+## Cross-reference
+
+After digging, follow links between documents to find connected context.
 
 ```bash
-mne search "headcount autonomous agents" -c concepts -n 10
-mne query "merchant support product reality" -c concepts -c sources -n 10
+mne links #abc123               # Outgoing wikilinks
+mne backlinks #abc123           # Incoming (what links here)
+mne orphans                     # Documents with no links at all
+mne query "deploy" --boost-links  # Prefer well-linked docs
 ```
 
-Omit `-c` to search all default-included collections.
+See [references/link-graph.md](references/link-graph.md) for full cross-reference commands.
 
-## Link graph
+## Setup
 
-mnemonic tracks wikilinks (`[[target]]`) and markdown links between documents:
+Never mutate the index unprompted. Only do this when the user asks to add a collection, index a new directory, or run diagnostics.
 
-```bash
-mne links #abc123             # Outgoing links
-mne backlinks #abc123         # Incoming links (what links to this)
-mne orphans                   # Documents with no links at all
-```
+### Quick setup via pi
 
-Use the link graph to discover related context:
+If the pi extension is loaded, type `/mne init` — it asks questions interactively:
 
-```bash
-mne query "deployment" --boost-links    # Boost results with many backlinks
+```
+/mne init
+  → Choose global or project-local scope
+  → Enter directories to index
+  → Index + embed (if Ollama available)
+  → Optionally configure MCP, copy skill
 ```
 
-## Setup and maintenance
-
-Only mutate indexes when the user asked for setup or maintenance.
+### Manual setup
 
 ```bash
 npm install -g @naveenadi/mnemonic
 mne init
 mne collection add ~/notes --name notes
-mne collection add ~/Documents --name docs --mask "**/*.md"
-mne index
-mne embed
+mne index                        # Scan files, build FTS5
+mne embed                        # Generate vector embeddings
 ```
 
-Health and diagnostics:
+See [references/setup.md](references/setup.md) for project-local `--db` mode, diagnostics, and maintenance.
 
-```bash
-mne doctor
-mne status
-```
+### Pi integration
+
+mnemonic integrates at three pi layers — MCP, skill, and extension. Each can be installed globally (all projects) or project-local (per repo).
+
+| Layer | Global | Per project |
+|---|---|---|
+| MCP | `~/.pi/agent/mcp.json` | `.pi/mcp.json` |
+| Skill | `~/.pi/agent/skills/mnemonic/` | `.pi/skills/mnemonic/` |
+| Extension | `~/.pi/agent/extensions/mnemonic/` | `.pi/extensions/mnemonic/` |
+
+Full instructions at [references/pi-integration.md](references/pi-integration.md).
 
 ## Pitfalls
 
-- **Do not stop at snippets.** Fetch documents before making claims.
-- **Do not slice files with `sed`/`head`/`tail`.** Use the `path:from:count`
-  suffix or `--from`/`-l` flags.
-- **Do not lean on auto query expansion.** Write `intent:`/`lex:`/`vec:`/`hyde:`
-  yourself when you know the domain context.
-- **Do not overuse semantic search.** If you know exact titles or terms,
-  `mne search` (BM25) is faster and often better.
-- **Do not mutate indexes casually.** `mne index` and `mne embed` change local
-  state and can be expensive.
+- **Fetch before claiming.** Snippets are leads, not sources.
+- **Do not shell-slice files.** Use `:from:count` suffix (e.g. `#abc123:120:40`) — never `sed`, `head`, `tail`.
+- **Do not lean on auto-expansion.** Write `intent:`/`lex:`/`vec:`/`hyde:` yourself. A bare `mne query "user sentence"` discards context only you have.
+- **Prefer BM25 for exact terms.** Semantic search is slower and drifts. If you know the title, `mne search "title"` is better.
+- **Do not mutate indexes casually.** `mne index` and `mne embed` are expensive.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@naveenadi/mnemonic",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "On-device hybrid search for markdown knowledge bases. BM25 + vector + LLM reranking with link graphs, time decay, and HyDE. Designed for pi coding agent.",
   "type": "module",
   "main": "dist/index.js",
@@ -12,7 +12,9 @@
     "bin/",
     "dist/",
     "SKILL.md",
-    "pi-extension/"
+    "pi-extension/",
+    "references/",
+    "src/pi-extension/package.json"
   ],
   "scripts": {
     "build": "tsc -p tsconfig.build.json",

package/references/link-graph.md

@@ -0,0 +1,25 @@
+# Cross-Reference Reference
+
+mnemonic tracks wikilinks (`[[target]]`) and markdown links (`[text](target.md)`) between documents in the `links` table.
+
+## Commands
+
+```bash
+mne links #abc123               # Outgoing — what this doc links to
+mne backlinks #abc123           # Incoming — what links to this doc
+mne orphans                     # Documents with zero links (no incoming or outgoing)
+```
+
+## Link boosting in search
+
+Pass `--boost-links` to `mne query` to boost results proportional to their backlink count. Documents with more incoming links rank higher:
+
+```bash
+mne query "deployment strategy" --boost-links
+```
+
+## Use cases
+
+- **Discover connected context**: after `mne get`, run `mne backlinks` to find related docs.
+- **Surface hubs**: `mne query "..." --boost-links` promotes widely cited documents.
+- **Find dead ends**: `mne orphans` surfaces disconnected notes that may need linking.

package/references/pi-integration.md

@@ -0,0 +1,117 @@
+# Pi Integration Setup
+
+mnemonic integrates with pi at three layers. Each can be installed **globally** (all projects) or **project-local** (per-repo).
+
+## Interactive setup — /mne init
+
+If the extension is loaded, type `/mne init` in pi to walk through everything:
+
+```
+/mne init
+  → Choose global or project-local scope
+  → Enter directories to index
+  → Index files, embed (if Ollama available)
+  → Optionally configure MCP in mcp.json
+  → Optionally install skill (SKILL.md)
+```
+
+Other commands:
+- `/mne add <path>` — quick-add a collection
+- `/mne status` — index health summary
+
+## MCP Server
+
+Exposes typed tools (`query`, `get`, `multi_get`, `status`) over stdio.
+
+### Global (all sessions)
+
+Add to `~/.pi/agent/mcp.json`:
+
+```json
+{
+  "mcpServers": {
+    "mnemonic": {
+      "command": "mne",
+      "args": ["mcp"],
+      "lifecycle": "keep-alive"
+    }
+  }
+}
+```
+
+### Project-local (per repo)
+
+Add to `.pi/mcp.json` in the project root (same format). Only active when pi is in that directory.
+
+## Pi Skill
+
+Lets the agent run `mne search`, `mne query`, `mne get` etc. via bash.
+
+### Global (all sessions)
+
+Copy the skill directory:
+
+```bash
+cp -r /path/to/mnemonic/skills/mnemonic ~/.pi/agent/skills/mnemonic
+```
+
+Or link the npm package's skill for auto-updates:
+
+```bash
+ln -s $(npm root -g)/@naveenadi/mnemonic/SKILL.md ~/.pi/agent/skills/mnemonic/SKILL.md
+```
+
+### Project-local (per repo)
+
+```bash
+mkdir -p .pi/skills
+cp -r /path/to/mnemonic/skills/mnemonic .pi/skills/mnemonic
+```
+
+Or via `settings.json`:
+
+```json
+{
+  "skills": ["../.pi/skills"]
+}
+```
+
+### Via npm package
+
+The skill ships in the npm tarball (`SKILL.md`). If you install `@naveenadi/mnemonic` as a project dependency, reference it in `.pi/settings.json`:
+
+```json
+{
+  "skills": ["node_modules/@naveenadi/mnemonic"]
+}
+```
+
+## Pi Extension
+
+Registers 4 custom tools (`mnemonic_search`, `mnemonic_query`, `mnemonic_get`, `mnemonic_status`) with `pi.registerTool()`.
+
+### Global (all sessions)
+
+```bash
+mkdir -p ~/.pi/agent/extensions/mnemonic
+cp src/pi-extension/index.ts ~/.pi/agent/extensions/mnemonic/index.ts
+```
+
+### Project-local (per repo)
+
+```bash
+mkdir -p .pi/extensions/mnemonic
+cp src/pi-extension/index.ts .pi/extensions/mnemonic/index.ts
+```
+
+### Via npm package (as pi package)
+
+Can be installed as a pi package for auto-discovery. See [pi packages docs](https://pi.dev/docs/packages).
+
+## Quick reference
+
+| Layer | Global path | Project-local path |
+|---|---|---|
+| MCP config | `~/.pi/agent/mcp.json` | `.pi/mcp.json` |
+| Skill | `~/.pi/agent/skills/mnemonic/` | `.pi/skills/mnemonic/` |
+| Extension | `~/.pi/agent/extensions/mnemonic/` | `.pi/extensions/mnemonic/` |

package/references/setup.md

@@ -0,0 +1,86 @@
+# Setup Reference
+
+## Global mode (default)
+
+All collections share one index at `~/.cache/mnemonic/index.sqlite`. Search everything at once.
+
+```bash
+npm install -g @naveenadi/mnemonic
+mne init
+mne collection add ~/notes --name notes
+mne collection add ~/Documents --name docs --mask "**/*.md"
+mne index
+mne embed
+```
+
+## Project-local mode
+
+Use `--db` for a per-repo index. Index and data stay in the repo.
+
+```bash
+mne --db .mnemonic/index.sqlite init
+mne --db .mnemonic/index.sqlite collection add . --name myproject
+mne --db .mnemonic/index.sqlite index
+mne --db .mnemonic/index.sqlite query "something in this repo"
+```
+
+## Pi integration (all three layers)
+
+Each layer — MCP server, skill, extension — can be installed globally or per project.
+
+### MCP server
+
+```bash
+# Global: add to ~/.pi/agent/mcp.json
+{
+  "mcpServers": {
+    "mnemonic": {
+      "command": "mne",
+      "args": ["mcp"],
+      "lifecycle": "keep-alive"
+    }
+  }
+}
+
+# Per project: same format in .pi/mcp.json
+```
+
+### Skill
+
+```bash
+# Global
+mkdir -p ~/.pi/agent/skills/mnemonic
+cp SKILL.md ~/.pi/agent/skills/mnemonic/
+
+# Per project (auto-discovered after trust)
+mkdir -p .pi/skills/mnemonic
+cp SKILL.md .pi/skills/mnemonic/
+```
+
+### Extension
+
+```bash
+# Global
+mkdir -p ~/.pi/agent/extensions/mnemonic
+cp src/pi-extension/index.ts ~/.pi/agent/extensions/mnemonic/
+
+# Per project
+mkdir -p .pi/extensions/mnemonic
+cp src/pi-extension/index.ts .pi/extensions/mnemonic/
+```
+
+Full reference: [pi-integration.md](pi-integration.md)
+
+## Diagnostics
+
+```bash
+mne doctor    # Check models, Ollama, DB health
+mne status    # Index health, collection stats, vector status
+```
+
+## Maintenance
+
+```bash
+mne index     # Re-scan all files, update FTS5 (safe to re-run)
+mne embed     # Generate vectors for unembedded documents (-f to force redo)
+```

package/src/pi-extension/package.json

@@ -0,0 +1,11 @@
+{
+  "name": "mnemonic-pi-extension",
+  "version": "0.1.0",
+  "description": "Pi extension for mnemonic — 4 custom tools for local markdown search",
+  "dependencies": {
+    "@naveenadi/mnemonic": "^0.1.0"
+  },
+  "pi": {
+    "extensions": ["./index.ts"]
+  }
+}