wicked-brain 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Mike Parcewski
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,208 @@
+# wicked-brain
+
+**Your AI agent's memory. No vector DB. No embeddings. No infrastructure.**
+
+wicked-brain gives your AI coding CLI a persistent, searchable knowledge base built on markdown and SQLite. Drop in files, let the agent organize them, and query your accumulated knowledge across sessions — all without leaving your terminal.
+
+Works with **Claude Code**, **Gemini CLI**, **Copilot CLI**, **Cursor**, and **Codex**.
+
+---
+
+## The Problem
+
+Every time you start a new AI session, your agent wakes up blank. The context you spent tokens building yesterday? Gone. The architecture decisions, the research, the tribal knowledge — all lost to the session boundary.
+
+The industry's answer has been RAG pipelines: chunk your docs, generate embeddings, spin up a vector database, build a retrieval layer, tune your similarity thresholds, and pray the cosine distance actually surfaces what you need.
+
+**That's a lot of infrastructure for "remember what I told you."**
+
+## A Different Approach
+
+wicked-brain follows the [Karpathy pattern](https://x.com/karpathy): treat the LLM as a research librarian that actively maintains a structured markdown knowledge base. No embeddings. No vector math. Just files, full-text search, and an agent that reasons about connections.
+
+```
+Your documents  ──>  Structured chunks  ──>  Synthesized wiki  ──>  Answers
+                     (evidence layer)        (knowledge layer)
+```
+
+- **Chunks** are source-faithful extractions with rich metadata (entities, themes, tags)
+- **Wiki articles** are LLM-synthesized concepts with `[[backlinks]]` to source chunks
+- **Every claim traces back** to a specific file you can read, edit, or delete
+
+The brain is plain markdown on your filesystem. Open it in Obsidian, VS Code, or `cat`. No black box.
+
+## How It Works
+
+wicked-brain is a set of **skills** — markdown instruction files that teach your AI agent how to manage a knowledge base using its native tools (read, write, search, grep).
+
+A lightweight background server handles the one thing that needs a database: full-text search via SQLite FTS5.
+
+```
+┌─────────────────────────────────────────┐
+│     Your AI CLI (Claude / Gemini / ...) │
+│                                         │
+│  Skills:                                │
+│    wicked-brain:ingest   → add sources  │
+│    wicked-brain:search   → find content │
+│    wicked-brain:query    → ask questions│
+│    wicked-brain:compile  → build wiki   │
+│    wicked-brain:lint     → check quality│
+│                                         │
+│  Your agent uses its own tools:         │
+│  Read, Write, Grep — no special APIs    │
+└────────────┬────────────────────────────┘
+             │  curl localhost (search only)
+             ▼
+┌─────────────────────────────────────────┐
+│  SQLite FTS5 server (auto-starts)       │
+│  ~300 lines, one dependency             │
+└─────────────────────────────────────────┘
+```
+
+**The server is invisible.** It auto-starts when needed and auto-reindexes when files change. You never think about it.
+
+## Install
+
+```bash
+npx wicked-brain
+```
+
+That's it. The installer detects your AI CLIs and drops in the skills. First time you use any skill, it walks you through setup.
+
+Or install via [agent-skills-cli](https://github.com/Karanjot786/agent-skills-cli):
+
+```bash
+skills install wicked-brain
+```
+
+## Usage
+
+Once installed, just talk to your agent:
+
+**Ingest a document:**
+> "Ingest this research paper" (works with PDF, DOCX, PPTX, XLSX, images — the LLM reads them natively)
+
+**Search your brain:**
+> "Search my brain for knowledge graph construction methods"
+
+**Ask a question:**
+> "What does my brain say about our SLA enforcement approach?"
+
+**Build wiki articles:**
+> "Compile wiki articles from the chunks we've ingested"
+
+**Check health:**
+> "What's in my brain?"
+
+Every operation uses **progressive loading** — the agent never pulls more than it needs. Search returns one-line summaries first. You drill down only when something looks relevant.
+
+## What Makes It Different
+
+| | Vector DB / RAG | wicked-brain |
+|---|---|---|
+| **Data format** | Opaque embeddings | Human-readable markdown |
+| **Search** | Cosine similarity (nearest neighbor) | Full-text search + LLM reasoning |
+| **Connections** | None (just similarity scores) | Explicit `[[backlinks]]` between concepts |
+| **Auditability** | Low (why did it retrieve this?) | High (every claim links to a source file) |
+| **Infrastructure** | Vector DB + embedding pipeline + retrieval service | One SQLite file + markdown |
+| **Maintenance** | Re-embed on changes, tune thresholds | Agent self-heals via lint and enhance |
+| **Cost to start** | Embedding API calls for entire corpus | Zero (deterministic chunking is free) |
+| **Ideal scale** | Millions of documents | 100 - 10,000 high-signal documents |
+
+## The Skills
+
+| Skill | What it does |
+|---|---|
+| `wicked-brain:init` | Set up a new brain (auto-triggers on first use) |
+| `wicked-brain:ingest` | Add source files — text extracted deterministically, binary docs read via LLM vision |
+| `wicked-brain:search` | Parallel search across your brain and linked brains |
+| `wicked-brain:read` | Progressive loading: depth 0 (stats), depth 1 (summary), depth 2 (full content) |
+| `wicked-brain:query` | Answer questions with source citations |
+| `wicked-brain:compile` | Synthesize wiki articles from chunks |
+| `wicked-brain:lint` | Find broken links, orphan chunks, inconsistencies |
+| `wicked-brain:enhance` | Identify and fill knowledge gaps |
+| `wicked-brain:status` | Brain health, stats, orientation |
+| `wicked-brain:server` | Manage the background search server (auto-triggered) |
+
+## Multi-Brain Federation
+
+Brains can link to other brains. A personal research brain can reference a team standards brain. A client brain can inherit from a company knowledge base.
+
+```json
+{
+  "id": "client-x",
+  "parents": ["../company-standards"],
+  "links": ["../shared-research"]
+}
+```
+
+When you search, wicked-brain dispatches parallel search agents across all accessible brains and merges the results. Access control is filesystem permissions — if you can read the directory, you can search it.
+
+## What's on Disk
+
+```
+~/.wicked-brain/
+  brain.json              # Identity and brain links
+  raw/                    # Your source files
+  chunks/
+    extracted/            # Source-faithful extractions with metadata
+    inferred/             # LLM-generated content (clearly separated)
+  wiki/                   # Synthesized articles with [[backlinks]]
+  _meta/
+    log.jsonl             # Append-only event log
+  .brain.db               # SQLite search index (auto-managed)
+```
+
+Everything is markdown. Everything is git-committable. Everything is human-readable. The SQLite file is a rebuildable cache — delete it and the server recreates it from your markdown files.
+
+## The Agent is the Parser
+
+No `pdf-parse`. No `mammoth`. No `pptx-parser`.
+
+Modern LLMs read PDF, DOCX, PPTX, and XLSX natively. When you ingest a binary document, the agent reads it with its vision capabilities and writes structured markdown chunks. Better extraction than any library, with semantic understanding built in.
+
+## Architecture
+
+**~500 lines of server JavaScript** (SQLite FTS5 + file watcher) + **~900 lines of skill markdown** (agent instructions).
+
+That's the entire system. Compare that to a typical RAG stack:
+
+```
+Typical RAG:                          wicked-brain:
+- Embedding model API                - SQLite (one file)
+- Vector database (Pinecone/Weaviate) - Markdown files
+- Chunking pipeline                   - Agent's native tools
+- Retrieval service                   - curl localhost
+- Re-ranking model                    - LLM reasoning
+- Orchestration layer                 - Skills (markdown)
+─────────────────                     ─────────────────
+~5,000+ lines, 10+ deps              ~1,400 lines, 1 dep
+```
+
+## Supported CLIs
+
+| CLI | Status |
+|---|---|
+| Claude Code | Supported |
+| Gemini CLI | Supported |
+| GitHub Copilot CLI | Supported |
+| Cursor | Supported |
+| Codex | Supported |
+
+Skills use only universally available operations (read files, write files, run shell commands, grep). No CLI-specific features.
+
+## Philosophy
+
+> "You rarely ever write or edit the wiki manually; it's the domain of the LLM." — Andrej Karpathy
+
+wicked-brain is built on three beliefs:
+
+1. **Files over databases.** Markdown is the most LLM-friendly, human-readable, future-proof format. Your knowledge shouldn't be locked in embeddings you can't read.
+
+2. **Reasoning over retrieval.** An LLM that reads summaries, follows links, and thinks about connections beats a nearest-neighbor lookup every time — at least for the scale most teams actually operate at.
+
+3. **Skills over infrastructure.** The agent already knows how to read, write, and search files. Teach it a workflow and it becomes a knowledge manager. No new services to deploy.
+
+## License
+
+MIT

package/install.mjs

@@ -0,0 +1,64 @@
+#!/usr/bin/env node
+// wicked-brain installer — detects CLIs and installs skills
+
+import { existsSync, mkdirSync, cpSync, readdirSync } from "node:fs";
+import { join, resolve } from "node:path";
+import { homedir } from "node:os";
+import { argv } from "node:process";
+import { fileURLToPath } from "node:url";
+
+const __dirname = fileURLToPath(new URL(".", import.meta.url));
+const skillsSource = join(__dirname, "skills");
+const home = homedir();
+
+const CLI_TARGETS = [
+  { name: "claude", dir: join(home, ".claude", "skills") },
+  { name: "gemini", dir: join(home, ".gemini", "skills") },
+  { name: "copilot", dir: join(home, ".github", "skills") },
+  { name: "codex", dir: join(home, ".codex", "skills") },
+  { name: "cursor", dir: join(home, ".cursor", "skills") },
+];
+
+// Detect which CLIs are installed by checking if parent dir exists
+const detected = CLI_TARGETS.filter((t) => {
+  const parentDir = resolve(t.dir, "..");
+  return existsSync(parentDir);
+});
+
+console.log("wicked-brain installer\n");
+
+if (detected.length === 0) {
+  console.log("No supported AI CLIs detected. Supported: claude, gemini, copilot, codex, cursor");
+  console.log("Install skills manually by copying the skills/ directory.");
+  process.exit(1);
+}
+
+console.log(`Detected CLIs: ${detected.map((d) => d.name).join(", ")}\n`);
+
+// Allow filtering via --cli flag
+const cliArg = argv.find((a) => a.startsWith("--cli="));
+const cliFilter = cliArg ? cliArg.split("=")[1].split(",") : null;
+const targets = cliFilter
+  ? detected.filter((d) => cliFilter.includes(d.name))
+  : detected;
+
+// Copy skills to each target CLI
+const skillDirs = readdirSync(skillsSource).filter((d) => !d.startsWith("."));
+
+for (const target of targets) {
+  console.log(`Installing to ${target.name} (${target.dir})...`);
+  mkdirSync(target.dir, { recursive: true });
+
+  for (const skill of skillDirs) {
+    const src = join(skillsSource, skill);
+    const dest = join(target.dir, skill);
+    cpSync(src, dest, { recursive: true });
+  }
+
+  console.log(`  ${skillDirs.length} skills installed`);
+}
+
+// Server binary is bundled — npx wicked-brain-server works automatically
+// Skills reference it as: npx wicked-brain-server --brain {path} --port {port}
+console.log("\nServer: bundled (use 'npx wicked-brain-server' to start)");
+console.log(`\nwicked-brain installed! Open your AI CLI and say "wicked-brain:init" to get started.`);

package/package.json

@@ -0,0 +1,53 @@
+{
+  "name": "wicked-brain",
+  "version": "0.1.0",
+  "type": "module",
+  "description": "Digital brain as skills for AI coding CLIs — no vector DB, no embeddings, no infrastructure",
+  "keywords": [
+    "ai",
+    "brain",
+    "knowledge-base",
+    "skills",
+    "claude-code",
+    "gemini-cli",
+    "copilot",
+    "cursor",
+    "codex",
+    "markdown",
+    "sqlite",
+    "fts5",
+    "rag-alternative"
+  ],
+  "author": "Mike Parcewski",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/mikeparcewski/wicked-brain.git"
+  },
+  "homepage": "https://github.com/mikeparcewski/wicked-brain",
+  "bugs": "https://github.com/mikeparcewski/wicked-brain/issues",
+  "bin": {
+    "wicked-brain": "install.mjs",
+    "wicked-brain-server": "server/bin/wicked-brain-server.mjs"
+  },
+  "dependencies": {
+    "better-sqlite3": "^12.0.0"
+  },
+  "files": [
+    "install.mjs",
+    "skills/",
+    "server/bin/",
+    "server/lib/",
+    "server/package.json",
+    "README.md",
+    "LICENSE"
+  ],
+  "scripts": {
+    "test": "cd server && node --test",
+    "prepublishOnly": "npm test",
+    "release": "npm version patch && git push && git push --tags"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  }
+}

package/server/bin/wicked-brain-server.mjs

@@ -0,0 +1,95 @@
+#!/usr/bin/env node
+import { createServer } from "node:http";
+import { readFileSync, writeFileSync, unlinkSync, existsSync, mkdirSync } from "node:fs";
+import { join, resolve } from "node:path";
+import { argv, pid, exit } from "node:process";
+import { FileWatcher } from "../lib/file-watcher.mjs";
+import { SqliteSearch } from "../lib/sqlite-search.mjs";
+
+// Parse args
+const args = argv.slice(2);
+function getArg(name) {
+  const idx = args.indexOf(`--${name}`);
+  return idx !== -1 && args[idx + 1] ? args[idx + 1] : null;
+}
+
+const brainPath = resolve(getArg("brain") || ".");
+const port = parseInt(getArg("port") || "4242", 10);
+const configPath = join(brainPath, "brain.json");
+
+// Read brain config
+let brainId = "unknown";
+try {
+  const config = JSON.parse(readFileSync(configPath, "utf-8"));
+  brainId = config.id || "unknown";
+} catch {
+  console.error(`Warning: Could not read brain.json at ${configPath}`);
+}
+
+// Initialize SQLite
+const dbPath = join(brainPath, ".brain.db");
+mkdirSync(join(brainPath, "_meta"), { recursive: true });
+const db = new SqliteSearch(dbPath, brainId);
+
+// PID file
+const pidPath = join(brainPath, "_meta", "server.pid");
+writeFileSync(pidPath, String(pid));
+
+// Graceful shutdown
+function shutdown() {
+  console.log("Shutting down...");
+  try { unlinkSync(pidPath); } catch {}
+  watcher.stop();
+  db.close();
+  exit(0);
+}
+process.on("SIGTERM", shutdown);
+process.on("SIGINT", shutdown);
+
+// Action dispatch
+const actions = {
+  health: () => db.health(),
+  search: (p) => db.search(p),
+  federated_search: (p) => db.federatedSearch(p),
+  index: (p) => db.index(p),
+  remove: (p) => db.remove(p.id),
+  reindex: (p) => db.reindex(p.docs),
+  backlinks: (p) => ({ links: db.backlinks(p.id) }),
+  forward_links: (p) => ({ links: db.forwardLinks(p.id) }),
+  stats: () => db.stats(),
+};
+
+// HTTP server
+const server = createServer((req, res) => {
+  if (req.method !== "POST" || req.url !== "/api") {
+    res.writeHead(404, { "Content-Type": "application/json" });
+    res.end(JSON.stringify({ error: "Not found. Use POST /api" }));
+    return;
+  }
+  let body = "";
+  req.on("data", (chunk) => { body += chunk; });
+  req.on("end", () => {
+    try {
+      const { action, params = {} } = JSON.parse(body);
+      const handler = actions[action];
+      if (!handler) {
+        res.writeHead(400, { "Content-Type": "application/json" });
+        res.end(JSON.stringify({ error: `Unknown action: ${action}` }));
+        return;
+      }
+      const result = handler(params);
+      res.writeHead(200, { "Content-Type": "application/json" });
+      res.end(JSON.stringify(result ?? { ok: true }));
+    } catch (err) {
+      res.writeHead(500, { "Content-Type": "application/json" });
+      res.end(JSON.stringify({ error: err.message }));
+    }
+  });
+});
+
+const watcher = new FileWatcher(brainPath, db, brainId);
+
+server.listen(port, () => {
+  console.log(`wicked-brain-server running on port ${port} (brain: ${brainId}, pid: ${pid})`);
+  watcher.start();
+});

package/server/lib/file-watcher.mjs

@@ -0,0 +1,159 @@
+import { watch, readFileSync, existsSync, readdirSync } from "node:fs";
+import { join, relative } from "node:path";
+import { createHash } from "node:crypto";
+
+function normalizePath(p) {
+  return p.replace(/\\/g, "/");
+}
+
+export class FileWatcher {
+  #brainPath;
+  #db;
+  #brainId;
+  #hashes = new Map(); // path -> content hash
+  #watchers = [];
+  #debounceTimers = new Map();
+  #pollInterval = null;
+
+  constructor(brainPath, db, brainId) {
+    this.#brainPath = brainPath;
+    this.#db = db;
+    this.#brainId = brainId;
+  }
+
+  start() {
+    // Build initial hash map
+    this.#scanAndHash("chunks");
+    this.#scanAndHash("wiki");
+
+    // Watch directories
+    for (const dir of ["chunks", "wiki"]) {
+      const absDir = join(this.#brainPath, dir);
+      if (!existsSync(absDir)) continue;
+
+      try {
+        const watcher = watch(absDir, { recursive: true }, (eventType, filename) => {
+          if (!filename || !filename.endsWith(".md")) return;
+          const relPath = normalizePath(`${dir}/${filename}`);
+          this.#debounce(relPath, () => this.#handleChange(relPath));
+        });
+        this.#watchers.push(watcher);
+      } catch {
+        // recursive watch not supported on this platform (Linux) — fall back to polling
+      }
+    }
+
+    // If no watchers were set up (Linux), use polling fallback
+    if (this.#watchers.length === 0) {
+      this.#startPolling();
+    } else {
+      console.log(`File watcher active on chunks/ and wiki/`);
+    }
+  }
+
+  stop() {
+    for (const w of this.#watchers) w.close();
+    this.#watchers = [];
+    for (const t of this.#debounceTimers.values()) clearTimeout(t);
+    this.#debounceTimers.clear();
+    if (this.#pollInterval) { clearInterval(this.#pollInterval); this.#pollInterval = null; }
+  }
+
+  // Scan a directory, hash all .md files, index any not yet in the DB
+  #scanAndHash(dir) {
+    const absDir = join(this.#brainPath, dir);
+    if (!existsSync(absDir)) return;
+    this.#walkDir(absDir, (absPath) => {
+      if (!absPath.endsWith(".md")) return;
+      const relPath = normalizePath(relative(this.#brainPath, absPath));
+      const content = readFileSync(absPath, "utf-8");
+      const hash = this.#hash(content);
+      this.#hashes.set(relPath, hash);
+    });
+  }
+
+  #walkDir(dir, callback) {
+    for (const entry of readdirSync(dir, { withFileTypes: true })) {
+      const full = join(dir, entry.name);
+      if (entry.isDirectory()) this.#walkDir(full, callback);
+      else if (entry.isFile()) callback(full);
+    }
+  }
+
+  #handleChange(relPath) {
+    const absPath = join(this.#brainPath, relPath);
+
+    if (!existsSync(absPath)) {
+      // File deleted
+      if (this.#hashes.has(relPath)) {
+        this.#hashes.delete(relPath);
+        this.#db.remove(relPath);
+        console.log(`[watcher] Removed from index: ${relPath}`);
+      }
+      return;
+    }
+
+    try {
+      const content = readFileSync(absPath, "utf-8");
+      const newHash = this.#hash(content);
+      const oldHash = this.#hashes.get(relPath);
+
+      if (newHash === oldHash) return; // No change
+
+      this.#hashes.set(relPath, newHash);
+      this.#db.index({
+        id: relPath,
+        path: relPath,
+        content: content,
+        brain_id: this.#brainId,
+      });
+      console.log(`[watcher] Reindexed: ${relPath}`);
+    } catch {
+      // File might be mid-write, ignore
+    }
+  }
+
+  #startPolling() {
+    console.log("File watcher using polling mode (recursive watch not available)");
+    this.#pollInterval = setInterval(() => {
+      for (const dir of ["chunks", "wiki"]) {
+        const absDir = join(this.#brainPath, dir);
+        if (!existsSync(absDir)) continue;
+        this.#walkDir(absDir, (absPath) => {
+          if (!absPath.endsWith(".md")) return;
+          const relPath = normalizePath(relative(this.#brainPath, absPath));
+          try {
+            const content = readFileSync(absPath, "utf-8");
+            const newHash = this.#hash(content);
+            const oldHash = this.#hashes.get(relPath);
+            if (newHash !== oldHash) {
+              this.#hashes.set(relPath, newHash);
+              this.#db.index({ id: relPath, path: relPath, content, brain_id: this.#brainId });
+              console.log(`[watcher] Reindexed: ${relPath}`);
+            }
+          } catch {}
+        });
+      }
+      // Check for deletions
+      for (const [relPath] of this.#hashes) {
+        if (!existsSync(join(this.#brainPath, relPath))) {
+          this.#hashes.delete(relPath);
+          this.#db.remove(relPath);
+          console.log(`[watcher] Removed from index: ${relPath}`);
+        }
+      }
+    }, 3000);
+  }
+
+  #debounce(key, fn) {
+    if (this.#debounceTimers.has(key)) clearTimeout(this.#debounceTimers.get(key));
+    this.#debounceTimers.set(key, setTimeout(() => {
+      this.#debounceTimers.delete(key);
+      fn();
+    }, 500)); // 500ms debounce
+  }
+
+  #hash(content) {
+    return createHash("sha256").update(content).digest("hex").slice(0, 16);
+  }
+}