wicked-brain 0.10.0 → 0.12.0

package/README.md

@@ -45,28 +45,29 @@ wicked-brain is a set of **skills** — markdown instruction files that teach yo
 A lightweight background server handles the one thing that needs a database: full-text search via SQLite FTS5.
 
 ```
-┌─────────────────────────────────────────┐
-│     Your AI CLI (Claude / Gemini / ...) │
-│                                         │
-│  Skills:                                │
-│    wicked-brain:ingest   → add sources  │
-│    wicked-brain:search   → find content │
-│    wicked-brain:query    → ask questions│
-│    wicked-brain:compile  → build wiki   │
-│    wicked-brain:lint     → check quality│
+┌─────────────────────────────────────────┐       ┌──────────────────────┐
+│     Your AI CLI (Claude / Gemini / ...) │       │ Browser (you)        │
+│                                         │       │ http://localhost/…   │
+│  Skills:                                │       │ Search · Wiki viewer │
+│    wicked-brain:ingest   → add sources  │       └──────────┬───────────┘
+│    wicked-brain:search   → find content │                  │ GET /
+│    wicked-brain:query    → ask questions│                  ▼
+│    wicked-brain:compile  → build wiki   │       ┌──────────────────────┐
+│    wicked-brain:lint     → check quality│       │  Viewer HTML         │
+│    wicked-brain:ui       → open viewer  │       └──────────────────────┘
 │                                         │
 │  Your agent uses its own tools:         │
 │  Read, Write, Grep — no special APIs    │
 └────────────┬────────────────────────────┘
-             │  curl localhost (search only)
+             │  POST /api (search + index + …)
              ▼
 ┌─────────────────────────────────────────┐
 │  SQLite FTS5 server (auto-starts)       │
-│  ~300 lines, one dependency             │
+│  Optional: --read-only for shared mode  │
 └─────────────────────────────────────────┘
 ```
 
-**The server is invisible.** It auto-starts when needed and auto-reindexes when files change. You never think about it.
+**The server is invisible.** It auto-starts when needed and auto-reindexes when files change. You never think about it — unless you want to: `open http://localhost:4242/` drops you into a browser viewer that lets you search and browse the wiki without going through an agent.
 
 ## Install
 
@@ -109,6 +110,19 @@ Once installed, just talk to your agent:
 
 Every operation uses **progressive loading** — the agent never pulls more than it needs. Search returns one-line summaries first. You drill down only when something looks relevant.
 
+## Browser Viewer
+
+Every running brain server serves a read-only HTML viewer at `GET /`. Open `http://localhost:4242/` (or whatever port the brain bound to) and you get:
+
+- **Search tab** — type-to-refine with live results and source-type filter chips. Empty query shows every doc most-recent-first so you can browse even when you don't know what to search for.
+- **Wiki tab** — grid of cards for each wiki article with tags and word counts.
+- **Header actions** — refresh (re-detect mode + re-index from disk) and delete (purge content; typed confirmation required).
+- **Read-only mode** — start the server with `--read-only` and destructive actions return 403; the viewer greys out the buttons and tells you why.
+
+Or let an agent do it: say *"open the brain viewer"* and the `wicked-brain:ui` skill resolves the brain, health-checks the server, and launches your default browser.
+
+The viewer has no auth. It's localhost-only, same-machine trust.
+
 ## What Makes It Different
 
 | | Vector DB / RAG | wicked-brain |
@@ -145,6 +159,7 @@ Every operation uses **progressive loading** — the agent never pulls more than
 | `wicked-brain:retag` | Backfill synonym-expanded tags across all chunks for better search recall |
 | `wicked-brain:update` | Check npm for updates and reinstall skills across all detected CLIs |
 | `wicked-brain:lsp` | Universal code intelligence via LSP — hover, go-to-definition, diagnostics, completions |
+| `wicked-brain:ui` | Open the read-only browser viewer — Material-styled Search + Wiki tabs over `http://localhost:<port>/` |
 
 ## Multi-Brain Federation
 
@@ -185,6 +200,8 @@ If you really want one brain for everything, you can pass a custom path to
 
 ## What's on Disk
 
+Two places. The **brain** lives in your home directory:
+
 ```
 ~/.wicked-brain/projects/{project-name}/
   brain.json              # Identity and brain links
@@ -199,7 +216,18 @@ If you really want one brain for everything, you can pass a custom path to
   .brain.db               # SQLite search index (auto-managed)
 ```
 
-Everything is markdown. Everything is git-committable. Everything is human-readable. The SQLite file is a rebuildable cache — delete it and the server recreates it from your markdown files.
+And a small per-repo marker lives **in the project itself**:
+
+```
+<your-repo>/
+  .wicked-brain/
+    mode.json             # Detected repo mode + wiki location (gitignore this)
+  CLAUDE.md               # Gets a `Contributor wiki: <path>` pointer stamped in
+```
+
+`mode.json` records whether the repo is `code` / `content` / `mixed` / `unknown` and where the contributor wiki lives (`wiki/`, `docs/wiki/`, etc.). Agents read it first when they need to find the wiki — it's the canonical discovery hint.
+
+Everything is markdown. Everything is git-committable (except the brain's `.brain.db` and the per-repo `.wicked-brain/`). Everything is human-readable. The SQLite file is a rebuildable cache — delete it and the server recreates it from your markdown files.
 
 ## The Agent is the Parser
 
@@ -209,22 +237,25 @@ Modern LLMs read PDF, DOCX, PPTX, and XLSX natively. When you ingest a binary do
 
 ## Architecture
 
-**~300 lines of server JavaScript** (SQLite FTS5 + file watcher) + **~1,400 lines of skill markdown** (agent instructions).
+Plain Node.js server (SQLite FTS5 + file watcher + optional LSP client + HTML viewer) plus markdown skill instructions your AI CLI consumes. **One runtime dependency** (`better-sqlite3`); LSP layer is hand-rolled JSON-RPC; the viewer is vanilla JS with no build.
 
-That's the entire system. Compare that to a typical RAG stack:
+Compare that to a typical RAG stack:
 
 ```
-Typical RAG:                          wicked-brain:
-- Embedding model API                - SQLite (one file)
+Typical RAG:                           wicked-brain:
+- Embedding model API                 - SQLite (one file)
 - Vector database (Pinecone/Weaviate) - Markdown files
 - Chunking pipeline                   - Agent's native tools
-- Retrieval service                   - curl localhost
+- Retrieval service                   - curl localhost (POST /api)
 - Re-ranking model                    - LLM reasoning
 - Orchestration layer                 - Skills (markdown)
+- Admin UI                            - GET / (vanilla HTML, read-only)
 ─────────────────                     ─────────────────
-~5,000+ lines, 10+ deps              ~1,700 lines, 1 dep
+10+ deps, opaque vectors              1 runtime dep, plain markdown
 ```
 
+See [ARCHITECTURE.md](ARCHITECTURE.md) for component diagrams and the schema layout.
+
 ## Supported CLIs
 
 | CLI | Status |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wicked-brain",
-  "version": "0.10.0",
+  "version": "0.12.0",
   "type": "module",
   "description": "Digital brain as skills for AI coding CLIs — no vector DB, no embeddings, no infrastructure",
   "keywords": [

package/server/bin/onboard-wiki.mjs

@@ -0,0 +1,36 @@
+#!/usr/bin/env node
+/**
+ * wicked-brain-onboard-wiki
+ *
+ * Detect repo mode, write `.wicked-brain/mode.json`, and stamp the
+ * `Contributor wiki: <path>` pointer into CLAUDE.md / AGENTS.md.
+ *
+ * Usage:
+ *   wicked-brain-onboard-wiki                      # runs against cwd
+ *   wicked-brain-onboard-wiki --repo-root <path>   # runs against <path>
+ *   wicked-brain-onboard-wiki --force              # overrides override:true
+ */
+
+import process from "node:process";
+import { runOnboardWiki, formatOnboardResult } from "../lib/onboard-wiki.mjs";
+
+const args = process.argv.slice(2);
+
+function getFlag(name) {
+  const idx = args.indexOf(`--${name}`);
+  return idx !== -1 && args[idx + 1] && !args[idx + 1].startsWith("--")
+    ? args[idx + 1]
+    : null;
+}
+
+const repoRoot = getFlag("repo-root") ?? process.cwd();
+const force = args.includes("--force");
+
+try {
+  const result = await runOnboardWiki(repoRoot, { force });
+  console.log(formatOnboardResult(result));
+  process.exit(0);
+} catch (err) {
+  console.error(`onboard-wiki failed: ${err.message}`);
+  process.exit(1);
+}

package/server/bin/wicked-brain-server.mjs

@@ -8,6 +8,10 @@ import { SqliteSearch } from "../lib/sqlite-search.mjs";
 import { LspClient } from "../lib/lsp-client.mjs";
 import { emitEvent, waitForBus } from "../lib/bus.mjs";
 import { startMemorySubscriber } from "../lib/memory-subscriber.mjs";
+import { renderViewerHtml } from "../lib/viewer-page.mjs";
+import { walkBrainContent, purgeBrainContent } from "../lib/brain-walker.mjs";
+import { runOnboardWiki } from "../lib/onboard-wiki.mjs";
+import { readFile as readFileAsync } from "node:fs/promises";
 
 // Parse args
 const args = argv.slice(2);
@@ -29,6 +33,10 @@ const brainPath = resolve(getArg("brain") || ".");
 const preferredPort = parseInt(getArg("port") || "4242", 10);
 // Explicit --port means "bind this exact port or fail" — no probe.
 const portExplicit = args.includes("--port");
+// Read-only mode disables write + destructive actions at the API layer.
+// Intended for shared / exposed brains where only search and read should
+// be reachable. Default is off to preserve the existing ingest story.
+const readOnly = args.includes("--read-only");
 const configPath = join(brainPath, "brain.json");
 // Source path for LSP workspace root — prefer --source flag, fall back to config, then brainPath
 const sourceArgRaw = getArg("source");
@@ -119,7 +127,7 @@ process.on("SIGINT", () => shutdown());
 
 // Action dispatch
 const actions = {
-  health: () => db.health(),
+  health: () => ({ ...db.health(), read_only: readOnly }),
   search: (p) => {
     const result = db.search(p);
     emitEvent("wicked.search.executed", "brain.search", {
@@ -154,6 +162,10 @@ const actions = {
   },
   backlinks: (p) => ({ links: db.backlinks(p.id) }),
   forward_links: (p) => ({ links: db.forwardLinks(p.id) }),
+  get_document: (p) => ({
+    document: p.id ? db.getDocument(p.id) : p.path ? db.getDocumentByPath(p.path) : null,
+  }),
+  list_docs: (p = {}) => db.listDocuments(p),
   stats: () => db.stats(),
   memory_stats: () => db.memoryStats(),
   candidates: (p) => ({ candidates: db.candidates(p) }),
@@ -195,6 +207,17 @@ const actions = {
   tag_frequency: () => ({ tags: db.tagFrequency() }),
   search_misses: (p) => ({ misses: db.searchMisses(p) }),
   wiki_list: (p) => db.wikiList(p),
+  verify_wiki: (p = {}) => {
+    const result = db.verifyWiki(p);
+    emitEvent("wicked.wiki.verified", "brain.wiki", {
+      brain_id: brainId,
+      total: result.summary.total,
+      stale: result.summary.stale,
+      orphaned: result.summary.orphaned,
+      unverifiable: result.summary.unverifiable,
+    });
+    return result;
+  },
   // LSP actions
   "lsp-health": () => lsp.health(),
   "lsp-symbols": (p) => lsp.symbols(p),
@@ -206,13 +229,76 @@ const actions = {
   "lsp-call-hierarchy-in": (p) => lsp.callHierarchyIn(p),
   "lsp-call-hierarchy-out": (p) => lsp.callHierarchyOut(p),
   "lsp-diagnostics": (p) => lsp.diagnostics(p),
+  reonboard: async () => {
+    // Detect mode + stamp the CLAUDE.md/AGENTS.md pointer, then rebuild the
+    // search index from whatever content is on disk in this brain. Does NOT
+    // re-write chunks or wiki — authored content is preserved.
+    const onboardTarget = sourceArg ?? brainPath;
+    const onboard = await runOnboardWiki(onboardTarget).catch(() => null);
+    const entries = await walkBrainContent(brainPath);
+    const docs = [];
+    for (const entry of entries) {
+      const content = await readFileAsync(entry.abs, "utf8");
+      docs.push({ id: entry.rel, path: entry.rel, content });
+    }
+    db.reindex(docs);
+    emitEvent("wicked.brain.reonboarded", "brain", {
+      brain_id: brainId,
+      indexed: docs.length,
+      mode: onboard?.detection?.mode ?? null,
+    });
+    return {
+      indexed: docs.length,
+      onboard: onboard
+        ? {
+            mode: onboard.detection.mode,
+            wiki_root: onboard.wiki_root,
+            mode_write: onboard.mode_write.action,
+            stamps: onboard.stamps,
+          }
+        : { skipped: true },
+    };
+  },
+  purge_brain: async (p = {}) => {
+    // Destructive. Wipes chunks/, wiki/, and memory/ content and clears the
+    // SQLite index. Requires p.confirm === "DELETE" to execute — typed
+    // confirmation from the UI keeps accidental clicks from being catastrophic.
+    if (p.confirm !== "DELETE") {
+      return { error: 'confirmation missing: pass {"confirm":"DELETE"}' };
+    }
+    const removed = await purgeBrainContent(brainPath);
+    db.reindex([]);
+    emitEvent("wicked.brain.purged", "brain", { brain_id: brainId, removed });
+    return { removed };
+  },
 };
 
+// Actions that mutate state. Blocked when the server was started with
+// `--read-only`, plus the two destructive newcomers. The list is explicit
+// rather than heuristic so additions are deliberate.
+const WRITE_ACTIONS = new Set([
+  "index",
+  "remove",
+  "reindex",
+  "confirm_link",
+  "reonboard",
+  "purge_brain",
+]);
+
 // HTTP server
 const server = createServer((req, res) => {
+  // Read-only viewer at GET /
+  if (req.method === "GET" && (req.url === "/" || req.url.startsWith("/?") || req.url.startsWith("/#"))) {
+    res.writeHead(200, {
+      "Content-Type": "text/html; charset=utf-8",
+      "Cache-Control": "no-store",
+    });
+    res.end(renderViewerHtml({ brainId }));
+    return;
+  }
   if (req.method !== "POST" || req.url !== "/api") {
     res.writeHead(404, { "Content-Type": "application/json" });
-    res.end(JSON.stringify({ error: "Not found. Use POST /api" }));
+    res.end(JSON.stringify({ error: "Not found. Use POST /api or GET /" }));
     return;
   }
   let body = "";
@@ -226,6 +312,11 @@ const server = createServer((req, res) => {
         res.end(JSON.stringify({ error: `Unknown action: ${action}` }));
         return;
       }
+      if (readOnly && WRITE_ACTIONS.has(action)) {
+        res.writeHead(403, { "Content-Type": "application/json" });
+        res.end(JSON.stringify({ error: `Action blocked in --read-only mode: ${action}` }));
+        return;
+      }
       // Handle both sync and async results
       const result = handler(params);
       Promise.resolve(result)

package/server/lib/brain-walker.mjs

@@ -0,0 +1,78 @@
+import fs from "node:fs/promises";
+import path from "node:path";
+
+/**
+ * Walk a brain path and surface every authored `.md` file under the content
+ * subdirectories (chunks/, wiki/, memory/). Deliberately excludes `_meta/`,
+ * `raw/`, `.brain.db`, and any dotfile/dotdir. Paths returned are relative to
+ * the brain path and use forward slashes per INV-PATHS-FORWARD.
+ */
+const CONTENT_DIRS = ["chunks", "wiki", "memory"];
+
+export async function walkBrainContent(brainPath) {
+  const out = [];
+  for (const rel of CONTENT_DIRS) {
+    const abs = path.join(brainPath, rel);
+    await walk(abs, rel, out);
+  }
+  out.sort((a, b) => a.rel.localeCompare(b.rel));
+  return out;
+}
+
+/**
+ * Remove everything under chunks/, wiki/, memory/ in the brain path. Leaves
+ * the directories themselves (with empty .gitkeep placeholders) so the shape
+ * of the brain survives a purge. Returns a per-dir file count.
+ */
+export async function purgeBrainContent(brainPath) {
+  const counts = {};
+  for (const rel of CONTENT_DIRS) {
+    const abs = path.join(brainPath, rel);
+    counts[rel] = await removeDirContents(abs);
+    await fs.mkdir(abs, { recursive: true });
+    await fs.writeFile(path.join(abs, ".gitkeep"), "", "utf8").catch(() => {});
+  }
+  return counts;
+}
+
+// --- internals ---
+
+async function walk(absDir, relDir, out) {
+  let entries;
+  try {
+    entries = await fs.readdir(absDir, { withFileTypes: true });
+  } catch {
+    return;
+  }
+  for (const e of entries) {
+    if (e.name.startsWith(".")) continue;
+    const absChild = path.join(absDir, e.name);
+    const relChild = path.posix.join(relDir, e.name);
+    if (e.isDirectory()) {
+      await walk(absChild, relChild, out);
+    } else if (e.isFile() && e.name.endsWith(".md")) {
+      out.push({ abs: absChild, rel: relChild.replace(/\\/g, "/") });
+    }
+  }
+}
+
+async function removeDirContents(absDir) {
+  let count = 0;
+  let entries;
+  try {
+    entries = await fs.readdir(absDir, { withFileTypes: true });
+  } catch {
+    return 0;
+  }
+  for (const e of entries) {
+    const abs = path.join(absDir, e.name);
+    if (e.isDirectory()) {
+      count += await removeDirContents(abs);
+      await fs.rm(abs, { recursive: true, force: true });
+    } else if (e.isFile()) {
+      await fs.rm(abs, { force: true });
+      if (!e.name.startsWith(".")) count += 1;
+    }
+  }
+  return count;
+}

package/server/lib/canonical-registry.mjs

@@ -0,0 +1,128 @@
+import fs from "node:fs/promises";
+import path from "node:path";
+import { parseFrontmatter } from "./frontmatter.mjs";
+
+/**
+ * Canonical registry: maps canonical IDs (e.g. "INV-PATHS-FORWARD") to the
+ * single page that owns them. Detects violations of the "one page per ID"
+ * rule and broken references.
+ *
+ * The registry is built once from a list of pages (each with frontmatter
+ * data) and is cheap to query. It does not touch the DB — it is a pure
+ * map-building function. Persistence and search integration live in
+ * sqlite-search.
+ */
+
+/**
+ * Build a registry from an array of { path, data } entries where `data`
+ * comes from parseFrontmatter.
+ *
+ * Returns:
+ *   {
+ *     byId: Map<id, path>          canonical pages (first claimant wins)
+ *     duplicates: Array<{ id, paths: string[] }>
+ *     pages: Array<{ path, canonical_for: string[], references: string[] }>
+ *   }
+ */
+export function buildRegistry(entries) {
+  const byId = new Map();
+  const duplicateHits = new Map(); // id -> Set of paths
+  const pages = [];
+
+  for (const { path: p, data } of entries) {
+    const claimed = normalizeList(data?.canonical_for);
+    const refs = normalizeList(data?.references);
+    pages.push({ path: p, canonical_for: claimed, references: refs });
+
+    for (const id of claimed) {
+      if (byId.has(id)) {
+        if (!duplicateHits.has(id)) {
+          duplicateHits.set(id, new Set([byId.get(id)]));
+        }
+        duplicateHits.get(id).add(p);
+      } else {
+        byId.set(id, p);
+      }
+    }
+  }
+
+  const duplicates = [];
+  for (const [id, pathSet] of duplicateHits) {
+    duplicates.push({ id, paths: [...pathSet].sort() });
+  }
+  duplicates.sort((a, b) => a.id.localeCompare(b.id));
+
+  return { byId, duplicates, pages };
+}
+
+/**
+ * Find references that don't resolve.
+ *
+ * A reference is resolvable if:
+ *   - it matches a canonical ID in the registry
+ *   - OR it matches a known path (present in `knownPaths`)
+ *   - OR it is an anchor-style link to a canonical ID (e.g. "wiki/invariants.md#INV-A")
+ *
+ * Returns Array<{ page, ref, reason }>.
+ */
+export function findBrokenReferences(registry, knownPaths = new Set()) {
+  const broken = [];
+  const canonicalIds = new Set(registry.byId.keys());
+  for (const page of registry.pages) {
+    for (const ref of page.references) {
+      if (isResolvable(ref, canonicalIds, knownPaths)) continue;
+      broken.push({ page: page.path, ref, reason: "unresolved reference" });
+    }
+  }
+  return broken;
+}
+
+/**
+ * Walk a wiki root and load every .md file's frontmatter.
+ * Returns entries ready for buildRegistry.
+ */
+export async function loadWikiEntries(wikiRoot) {
+  const entries = [];
+  await walkMarkdown(wikiRoot, wikiRoot, entries);
+  return entries;
+}
+
+async function walkMarkdown(absRoot, absDir, out) {
+  let items;
+  try {
+    items = await fs.readdir(absDir, { withFileTypes: true });
+  } catch {
+    return;
+  }
+  for (const item of items) {
+    const abs = path.join(absDir, item.name);
+    if (item.isDirectory()) {
+      await walkMarkdown(absRoot, abs, out);
+    } else if (item.isFile() && item.name.endsWith(".md")) {
+      const content = await fs.readFile(abs, "utf8");
+      const { data } = parseFrontmatter(content);
+      const rel = path.relative(absRoot, abs).replace(/\\/g, "/");
+      out.push({ path: rel, data });
+    }
+  }
+}
+
+function normalizeList(value) {
+  if (value == null) return [];
+  if (Array.isArray(value)) return value.map(String);
+  if (typeof value === "string") return [value];
+  return [];
+}
+
+function isResolvable(ref, canonicalIds, knownPaths) {
+  if (canonicalIds.has(ref)) return true;
+  const hashIdx = ref.indexOf("#");
+  if (hashIdx >= 0) {
+    const anchorId = ref.slice(hashIdx + 1);
+    if (canonicalIds.has(anchorId)) return true;
+    const pathPart = ref.slice(0, hashIdx);
+    if (pathPart.length > 0 && knownPaths.has(pathPart)) return true;
+  }
+  if (knownPaths.has(ref)) return true;
+  return false;
+}