pi-vault-mind 0.7.0

package/skills/vault-mind-setup/references/obsidian-vault-structure.md

@@ -0,0 +1,106 @@
+# Obsidian Vault Structure
+
+Recommended layout for a vault used with pi-vault-mind. Combines **PARA method** + **Johnny Decimal organization** + **LLM Wiki structure** for a workspace that works for both humans and agents.
+
+## The Folder Tree
+
+```
+Vault/
+├── 00 - System/              ← Templates, inbox, daily notes, meta-configuration
+├── 10 - Projects/            ← Active projects (PARA)
+├── 20 - Areas/               ← Ongoing domains (PARA)
+├── 30 - Resources/           ← Reference material (PARA)
+│   ├── Inbox/                ← 📥 Drop files here for passive agent ingestion
+│   ├── Sources/              ← any2md-processed documents (immutable sources)
+│   └── Articles/             ← External references
+├── 40 - Archive/             ← Completed/archived items (PARA)
+│
+├── Agent/                    ← 🤖 Agent-Collaborative Workspace
+│   ├── Inbox/                ← Human drops tasks/requests for agents (@agent-*)
+│   ├── Wiki/                 ← Agent-curated structured knowledge
+│   │   ├── Concepts/         ← Agent-written concept pages with typed edges
+│   │   ├── Entities/         ← People/orgs/systems with `agent:derived-from` links
+│   │   ├── Syntheses/        ← Cross-topic analysis pages
+│   │   └── Summaries/        ← Source document summaries
+│   ├── Journal/              ← Agent activity log (timestamped, append-only)
+│   ├── Tasks/                ← Agent task boards (Kanban markdown or Bases)
+│   └── Presentations/        ← NotebookLM-generated podcasts, slides, quizzes
+│
+├── index.md                  ← Master routing table (agent + human navigable)
+└── CLAUDE.md                 ← Agent instructions + domain constraints
+```
+
+## Why this structure
+
+### PARA for humans
+- **Projects** (10-) have a deadline
+- **Areas** (20-) are ongoing responsibilities
+- **Resources** (30-) are topics of interest
+- **Archive** (40-) is everything completed
+
+### Agent/ for subagents
+This is the shared workspace where agents read, write, and produce. Each subfolder has a specific role:
+- `Inbox/` — human drops `@agent-XXX` markers here. Watcher detects them.
+- `Wiki/Concepts/` — Miner writes concept pages here with frontmatter
+- `Wiki/Entities/` — entity pages (people, systems) with `agent:derived-from` typed edges
+- `Wiki/Syntheses/` — cross-topic analysis from the Broadcaster
+- `Journal/` — append-only activity log, useful for `/wiki journal` queries
+- `Tasks/` — agent-maintained task boards (Kanban markdown or `.base` files)
+- `Presentations/` — NotebookLM outputs (audio, video, slides)
+
+### Johnny Decimal for navigation
+- `00-09` — System/meta
+- `10-19` — Active projects
+- `20-29` — Ongoing areas
+- `30-39` — Reference resources
+- `40-49` — Archive
+
+Each folder has an `index.md` for routing.
+
+## Required frontmatter
+
+All agent-written pages should have strict YAML frontmatter:
+
+```yaml
+---
+title: "RLHF"
+domain: "alignment"           # or "research", "decision", "experiment"
+source: "arxiv.org/abs/..."   # or "user-prompt", "notebook"
+status: "draft"               # or "needs-review", "approved"
+created: 2026-06-06
+tags: ["reinforcement-learning", "alignment", "ppo"]
+agent:related-to: "[[Reward_Modeling]]"
+agent:derived-from: "[[Constitutional_AI]]"
+---
+```
+
+The `agent:related-to`, `agent:derived-from`, etc. are **typed edges** that Breadcrumbs parses.
+
+## Sync strategy
+
+### Obsidian Sync (primary)
+For real-time sync across the user's devices. Handles 99% of human interaction.
+
+### Git (long-term + agent work)
+- **`obsidian-git` plugin** — auto-commits every X minutes
+- **Git worktrees** for Heavy-Lifter — large refactors in isolated branches, then merged
+
+## Recommended Obsidian settings
+
+In `Settings → Files & Links`:
+- **New link format**: Shortest possible (when possible)
+- **Use [[Wikilinks]]**: ON
+- **Always update internal links**: ON
+
+In `Settings → Core Plugins`:
+- ✅ Backlinks
+- ✅ Outline
+- ✅ Word count
+- ✅ File recovery
+
+Community plugins to install:
+- `obsidian-git` — auto-commit
+- `Breadcrumbs` — typed-edge graph
+- `Graph Analysis` — co-citation discovery
+- `Actions URI` — cross-app automation
+- `obsidian-shellcommands` — bridge to pi

package/skills/vault-mind-setup/references/pi-extension-wiring.md

@@ -0,0 +1,236 @@
+# pi Extension Wiring
+
+This document describes every pi extension and external tool that **pi-vault-mind** integrates with.
+
+## Required Extensions
+
+### pi-subagents
+
+**What it provides:** The `subagent()` tool — used by the Manager agent to fork isolated worker sessions for the Miner, Broadcaster, and Heavy-Lifter.
+
+```bash
+pi install npm:pi-subagents
+```
+
+**How we use it:**
+- Vault Watcher detects `@agent-miner` → sends task to Manager
+- Manager calls `subagent({ agent: "vault-mind-miner", context: "fork", async: true })`
+- Miner runs in isolated session, writes results to LanceDB + Vault, terminates
+- No chat pollution, clean context per task
+
+**Configuration:** None required. Works out of the box.
+
+### pi-context
+
+**What it provides:** Context management tools (`context_tag`, `context_log`, `context_checkout`) for clean session hygiene during long orchestration sessions.
+
+```bash
+pi install npm:pi-context
+```
+
+**How we use it:**
+- Manager tags milestones (`watcher-build-start`, `research-phase`) to create rollback points
+- Before risky operations, checks `context_log` to understand current session state
+- After noisy research: squashes history with `context_checkout` to keep context window clean
+
+**Configuration in `pi-vault-mind.config.json`:**
+```json
+{
+  "extensionCompatibility": {
+    "pi-context": {
+      "enabled": true,
+      "tagPatterns": [],
+      "enhanceInjectors": false,
+      "autoEnableAcm": true,
+      "indexContextEvents": true
+    }
+  }
+}
+```
+
+Enable/disable via:
+```
+/wiki context enable
+/wiki context disable
+/wiki context status
+```
+
+## Optional Extensions
+
+### NotebookLM CLI (`nlm`)
+
+**What it provides:** Command-line interface to Google NotebookLM for programmatic podcast generation, study guides, quizzes, and audio overviews.
+
+```bash
+# Install globally (not a pi extension):
+npm install -g notebooklm-cli
+
+# Or use the MCP server (preferred):
+# Add to ~/.pi/agent/mcp.json:
+# { "notebooklm-mcp": { "command": "npx", "args": ["-y", "notebooklm-mcp-cli"] } }
+```
+
+**How we use it (Broadcaster agent):**
+- User drops `@agent-broadcaster` on meeting notes
+- Broadcaster creates NotebookLM notebook from vault markdown files
+- Generates "Deep Dive" audio overview
+- Downloads `.wav` to `Vault/Agent/Presentations/`
+
+**Required skill:** `nlm-skill` — auto-loaded by pi when `nlm` or `notebooklm` keywords are detected.
+
+**Authentication:** Requires Google account login. Run once:
+```bash
+nlm login
+```
+
+### any2md
+
+**What it provides:** Converts URLs, PDFs, and other formats to clean markdown. Used by the Miner for passive document ingestion.
+
+```bash
+# Available via npx (no global install needed):
+npx any2md https://arxiv.org/abs/... > paper.md
+
+# Or install globally:
+npm install -g any2md
+```
+
+**How we use it (Miner agent):**
+- User drops a PDF URL or file reference with `@agent-miner ingest`
+- Miner calls `npx any2md <source>` to convert to markdown
+- Extracts entities and appends to LanceDB
+- Writes converted markdown to `Vault/Agent/Inbox/`
+
+### kepano/obsidian-skills
+
+**What it provides:** Five pi skills that teach agents Obsidian-specific formats.
+
+Install via:
+```bash
+# Clone or copy to ~/.pi/agent/skills/
+git clone https://github.com/kepano/obsidian-skills ~/.pi/agent/skills/obsidian-skills
+
+# Or install individual skills as pi packages
+pi install git:https://github.com/kepano/obsidian-skills
+```
+
+The 5 skills:
+- `obsidian-markdown` — Obsidian-flavored markdown syntax
+- `obsidian-bases` — `.base` file format (database views)
+- `json-canvas` — Canvas JSON format
+- `obsidian-cli` — official Obsidian CLI
+- `defuddle` — cleaner markdown extraction from web pages
+
+**Why required:** Without these skills, agents generate technically-valid markdown that Obsidian doesn't render correctly. The skills teach them wikilinks, embeds, callouts, Bases, and Canvas.
+
+## Extension Dependency Graph
+
+```
+pi-vault-mind
+├── REQUIRED: pi-subagents        → subagent() tool for forked dispatch
+├── REQUIRED: pi-context          → context_tag/log/checkout for session hygiene
+├── OPTIONAL: notebooklm-mcp-cli  → Broadcaster audio generation (nlm-skill)
+├── OPTIONAL: any2md              → Miner passive document ingestion
+├── OPTIONAL: pi-intercom         → inter-agent coordination channel
+└── OPTIONAL: kepano/obsidian-skills  → agents learn Obsidian formats
+```
+
+## Quick Install (all at once)
+
+```bash
+# Core (required)
+pi install npm:pi-vault-mind
+pi install npm:pi-subagents
+pi install npm:pi-context
+
+# Optional quality-of-life
+npm install -g notebooklm-cli    # or configure MCP server
+npm install -g any2md             # or use npx any2md
+
+# Obsidian skills (agents learn Obsidian format)
+git clone https://github.com/kepano/obsidian-skills ~/.pi/agent/skills/obsidian-skills
+
+# Verify
+pi -e npm:pi-vault-mind
+/wiki validate
+/wiki watcher status
+```
+
+## Global Config Template
+
+After installing all extensions, create `~/.pi/agent/pi-vault-mind.config.json`:
+
+```json
+{
+  "version": 2,
+  "wiki": {
+    "dataDir": ".lancedb",
+    "embedding": {
+      "provider": "transformers",
+      "ollamaModel": "embeddinggemma",
+      "ollamaHost": "http://127.0.0.1:11434"
+    },
+    "graph": {
+      "enabled": true,
+      "canvasSync": true
+    },
+    "vaults": {
+      "default": {
+        "path": "/home/YOU/Obsidian/YOUR_VAULT",
+        "autoSync": true,
+        "autoSyncTags": ["decision", "insight", "requirement"],
+        "autoSyncMinLength": 200
+      }
+    }
+  },
+  "extensionCompatibility": {
+    "pi-context": {
+      "enabled": true,
+      "tagPatterns": [],
+      "autoEnableAcm": true
+    }
+  },
+  "collections": {},
+  "injectors": []
+}
+```
+
+## How Extensions Wire Together at Runtime
+
+```
+1. pi starts with pi-vault-mind loaded
+   ├── Discovers skills/ directory → Manager, Miner, Broadcaster, Heavy-Lifter prompts
+   ├── Registers 12 wiki tools (wiki_search, append_wiki, wiki_sync, etc.)
+   ├── Registers /wiki commands (init, validate, watcher, etc.)
+   └── Auto-starts Vault Watcher if wiki.vaults is configured
+
+2. User saves @agent-miner in Obsidian
+   ├── Vault Watcher (src/watcher.ts) detects marker via fs.watch
+   ├── Groups markers, debounces 1s
+   └── Calls pi.sendUserMessage() with dispatch instruction
+
+3. Manager agent receives message
+   ├── Recognizes dispatch request
+   ├── Calls subagent({ agent: "vault-mind-miner", context: "fork", async: true })
+   │   └── pi-subagents extension handles forking, session isolation
+   └── pi-context tools available for session hygiene (context_tag, etc.)
+
+4. Miner subagent executes in isolation
+   ├── Reads the vault file (instruction + context)
+   ├── If URL/PDF detected: calls npx any2md to convert
+   ├── Extracts entities via regex (src/graph.ts)
+   ├── Calls append_wiki() → JSONL WAL + LanceDB
+   ├── Calls wiki_sync() → markdown in Vault/Agent/Inbox/
+   └── Terminates — context doesn't pollute Manager session
+```
+
+## Troubleshooting
+
+| Symptom | Check |
+|---|---|
+| `subagent is not defined` | Install pi-subagents: `pi install npm:pi-subagents` |
+| `context_tag is not defined` | Install pi-context: `pi install npm:pi-context` |
+| Watcher doesn't start | Vaults not configured: add `wiki.vaults` to config |
+| Subagent doesn't run | Check `/wiki watcher status` — is it RUNNING? Check vault path |
+| NotebookLM fails | `nlm login` to authenticate Google account |
+| any2md fails | Use `npx any2md` or install globally: `npm install -g any2md` |

package/skills/vault-mind-setup/references/troubleshooting-tree.md

@@ -0,0 +1,147 @@
+# Troubleshooting Tree
+
+When something is wrong, follow this decision tree. **Read the error message first — it usually tells you which branch to take.**
+
+## Step 1: Identify the layer
+
+| Symptom category | Likely layer | First check |
+|---|---|---|
+| "command not found", "is not defined" | pi extension not installed | `pi list` |
+| "connection refused", "ECONNREFUSED" | External service down (Ollama, LanceDB) | `ollama list`, `ls $dataDir` |
+| "permission denied" | Filesystem permissions | `ls -la $path` |
+| Watcher doesn't fire | File watcher / config | `/wiki watcher status` |
+| Subagent hangs/never returns | Subagent context / model | check `~/.pi/agent/sessions/` |
+| Embedding dimension mismatch | Config schema change | `/wiki reindex --reembed` |
+| Search returns nothing | Empty index or wrong collection | `wiki_status` |
+
+## Step 2: Decision tree per category
+
+### "command not found" / "is not defined"
+
+```
+Is the extension listed in `pi list`?
+├── NO → install it
+│   ├── `pi install npm:pi-vault-mind`
+│   ├── `pi install npm:pi-subagents`
+│   └── `pi install npm:pi-context`
+└── YES → is it enabled in this session?
+    ├── Check settings.json packages array
+    └── Try `pi -e npm:pi-vault-mind` to force-load
+```
+
+### "connection refused"
+
+```
+What service?
+├── Ollama → `ollama list` 
+│   ├── works → check `wiki.embedding.ollamaHost` in config (default http://127.0.0.1:11434)
+│   └── fails → `ollama serve` or install Ollama
+└── LanceDB → `ls $wiki.dataDir`
+    ├── exists but unreadable → check permissions
+    └── doesn't exist → run `/wiki init` to scaffold
+```
+
+### "permission denied"
+
+```
+Path is to a file or directory?
+├── file → `ls -la $file` — does the user own it?
+│   └── chown or chmod as appropriate
+└── directory → `ls -ld $dir` — does the user have +x (traverse)?
+    └── `chmod +x $dir` or fix ownership
+```
+
+### Watcher doesn't fire on save
+
+```
+Is the watcher running?
+├── `/wiki watcher status` says STOPPED
+│   └── `/wiki watcher start`
+└── RUNNING but not detecting
+    ├── Is the vault path correct?
+    │   └── check `wiki.vaults` in config
+    ├── Is the file inside a watched dir?
+    │   └── .obsidian/ and .git/ are auto-ignored
+    ├── Did the file have a recognized marker?
+    │   └── format: `@agent-Role[:id] instruction`
+    └── Check debounce — 1s after save before processing
+```
+
+### Subagent hangs or never returns
+
+```
+Check the child session file
+├── ls ~/.pi/agent/sessions/ | tail -5
+├── Find the most recent forked session
+├── Open the .jsonl — look for "needs_attention" or "interrupted"
+└── If needs_attention:
+    ├── The child is waiting for input
+    ├── Resume: subagent({ action: "resume", id: "..." })
+    └── Or interrupt: subagent({ action: "interrupt", id: "..." })
+```
+
+### Embedding dimension mismatch
+
+```
+Did you switch embedding provider?
+├── transformers (384d) → ollama embeddinggemma (768d)
+│   └── `/wiki reindex --reembed` (drops and recreates tables)
+└── ollama (768d) → transformers (384d)
+    └── same fix: `/wiki reindex --reembed`
+```
+
+### Search returns no results
+
+```
+Is the index populated?
+├── wiki_status shows 0 rows
+│   └── Append a test fact first: `append_wiki(...)`
+│       └── Tables are created lazily on first write
+└── wiki_status shows N rows but search is empty
+    ├── Wrong collection? `wiki_status` shows per-collection counts
+    ├── Wrong query? Try broader keywords
+    └── Try wiki_fts_search (Tantivy BM25) for exact keyword
+```
+
+## Step 3: Nuclear options
+
+When nothing else works, in order of destructiveness:
+
+1. **`/wiki validate`** — health check, lists everything that's wrong
+2. **`/wiki reindex --reembed`** — rebuild indexes (slow but safe)
+3. **`/wiki setup` (reconfigure)** — overwrites config (keeps data)
+4. **Delete `$wiki.dataDir`** — DESTROYS all indexed data. JSONL WAL is preserved.
+5. **Reinstall the extension** — `pi remove npm:pi-vault-mind && pi install npm:pi-vault-mind`
+
+Always check the JSONL WAL first — that's the source of truth and can be re-indexed from scratch.
+
+## Common error messages decoded
+
+| Error | What it means | Fix |
+|---|---|---|
+| `ENOSPC: no space left on device` | Disk full | Free space, then `/wiki reindex --all` |
+| `EMFILE: too many open files` | `fs.watch` hitting OS limit | `ulimit -n 1024` and restart |
+| `EACCES: permission denied` | Filesystem permissions | `chown` or `chmod` |
+| `ENOTDIR: not a directory` | Path points to a file | Fix the path in config |
+| `Cannot find module '@lancedb/lancedb'` | npm deps not installed | `pnpm install` in extension dir |
+| `EADDRINUSE` (Ollama) | Port 11434 already in use | Kill the other process or change `ollamaHost` |
+| `index out of range` | LanceDB table corruption | `/wiki reindex --all` |
+| `LanceDB version mismatch` | Old data on disk | Delete `$wiki.dataDir` and reindex from JSONL |
+
+## Where to find logs
+
+| Component | Log location |
+|---|---|
+| pi-vault-mind | `console.log` from extension (visible in pi session) |
+| Watcher | same |
+| LanceDB | `~/.lancedb/` or `$wiki.dataDir/` |
+| Subagent | `~/.pi/agent/sessions/<session-id>.jsonl` |
+| Ollama | `journalctl -u ollama` or `~/.ollama/logs/` |
+| Obsidian | Obsidian console: Ctrl+Shift+I |
+
+## When all else fails
+
+1. Read the error message slowly
+2. Check `TROUBLESHOOTING.md` in the repo
+3. Run `/wiki validate` and read its output
+4. File an issue: https://github.com/kylebrodeur/pi-vault-mind/issues