wicked-brain 0.4.7 → 0.4.8

package/README.md

@@ -127,6 +127,7 @@ Every operation uses **progressive loading** — the agent never pulls more than
 | Skill | What it does |
 |---|---|
 | `wicked-brain:init` | Set up a new brain — creates structure, starts the server, and ingests your project in one shot |
+| `wicked-brain:migrate` | Migrate a legacy flat brain at `~/.wicked-brain/` into the per-project layout |
 | `wicked-brain:ingest` | Add source files — text extracted deterministically, binary docs read via LLM vision |
 | `wicked-brain:search` | Parallel search across your brain and linked brains |
 | `wicked-brain:read` | Progressive loading: depth 0 (stats), depth 1 (summary), depth 2 (full content) |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wicked-brain",
-  "version": "0.4.7",
+  "version": "0.4.8",
   "type": "module",
   "description": "Digital brain as skills for AI coding CLIs — no vector DB, no embeddings, no infrastructure",
   "keywords": [

package/server/package.json

@@ -1,6 +1,6 @@
 {
   "name": "wicked-brain-server",
-  "version": "0.4.7",
+  "version": "0.4.8",
   "type": "module",
   "description": "SQLite FTS5 search server for wicked-brain digital knowledge bases",
   "keywords": [

package/skills/wicked-brain-init/SKILL.md

@@ -69,9 +69,28 @@ directory, not a project subdirectory), push back: explain the per-project
 convention and suggest `~/.wicked-brain/projects/{project_name}` instead.
 Only accept the flat path if the user explicitly insists.
 
-### Step 2: Check for existing brain
+### Step 2: Check for existing brains
 
-If `{brain_path}/_meta/config.json` already exists, tell the user:
+#### 2a: Detect a flat brain at the parent path
+
+If `~/.wicked-brain/brain.json` exists (note: `brain.json` at the flat parent
+path, NOT inside a `projects/` subdirectory), this is a legacy flat brain from
+before v0.4.7. Stop and tell the user:
+
+"I found an existing flat brain at `~/.wicked-brain/`. The current layout puts
+each project under `~/.wicked-brain/projects/{name}/`. I can migrate the flat
+brain with `wicked-brain:migrate` before creating the new one. Migrate now?"
+
+If yes, invoke `wicked-brain:migrate` with `flat_path=~/.wicked-brain` and
+wait for it to complete before continuing.
+
+If no, confirm the user wants to keep the flat brain and proceed (accept the
+tradeoff: the new project brain will live under `projects/` but the old one
+stays at the flat path).
+
+#### 2b: Check target path
+
+If `{brain_path}/_meta/config.json` already exists at the chosen target, tell the user:
 "A brain already exists at `{brain_path}`. Do you want to re-initialize it (keeps existing chunks) or pick a different path?"
 
 Stop and wait for their answer before continuing.
@@ -123,9 +142,12 @@ Write to `{brain_path}/_meta/config.json`:
 }
 ```
 
-`server_port: 4242` is the *preferred* port. The server will find a free port starting
-from this value on startup and write the actual port back to this file. You do not
-need to find a free port manually.
+`server_port: 4242` is the *preferred* starting port, not the guaranteed port.
+When the server starts in Step 7, it probes from this value upward until it
+finds a free port, then writes the **actual** port back to this same file.
+If multiple project brains run at once, each gets a distinct port (4242, 4243,
+4244, ...). Always re-read `_meta/config.json` after the server starts to get
+the real port — never hardcode `4242` in downstream calls.
 
 ### Step 6: Initialize the event log
 
@@ -140,7 +162,22 @@ The server will pick a free port and write it back to `_meta/config.json`.
 npx wicked-brain-server --brain {brain_path} &
 ```
 
-Wait for the health check to confirm it's up before continuing.
+Do NOT pass `--port` unless the user specifies one — let the server pick a
+free port. After the process starts, **re-read `{brain_path}/_meta/config.json`**
+to get the actual `server_port` the server bound to. Use that port for the
+health check and all subsequent API calls.
+
+Then health-check to confirm it's up before continuing:
+
+```bash
+curl -s -X POST http://localhost:{actual_port}/api \
+  -H "Content-Type: application/json" \
+  -d '{"action":"health"}'
+```
+
+Verify the response includes `"brain_id"` matching this brain's id — this
+confirms you're talking to the right server (not an unrelated brain on the
+same machine).
 
 ### Step 8: Ingest the project
 

package/skills/wicked-brain-lsp/SKILL.md

@@ -53,12 +53,14 @@ Read `{brain_path}/_meta/config.json`. Look for a `source_path` key:
 
 ```json
 {
-  "brain_path": "/Users/me/.wicked-brain",
-  "server_port": 4243,
+  "brain_path": "/Users/me/.wicked-brain/projects/my-project",
+  "server_port": 4242,
   "source_path": "/Users/me/Projects/my-project"
 }
 ```
 
+(`server_port` is the preferred starting port — the server writes the actual bound port back to this field on startup.)
+
 If `source_path` is **present** — LSP is configured. Proceed with calls.
 
 If `source_path` is **missing** — LSP will fail. Fix it before continuing.
@@ -108,6 +110,78 @@ Symptoms that indicate missing or wrong `source_path`:
 
 Check `source_path` in config. If it points at the brain directory (e.g., `~/.wicked-brain/...`) instead of the source project, that is wrong — the brain dir has no `tsconfig.json` or language config. Set it to the project root and restart.
 
+## Warming LSP (important — read this before calling lsp-workspace-symbols)
+
+**Language servers start lazily.** The brain server does NOT spawn language
+servers on startup — they only spawn when a file-specific LSP action runs
+against a file in `source_path` (`lsp-symbols`, `lsp-definition`, `lsp-hover`,
+`lsp-references`, `lsp-implementation`, `lsp-call-hierarchy-in`,
+`lsp-call-hierarchy-out`).
+
+`lsp-workspace-symbols` and `lsp-health` do NOT trigger server spawn. They
+only report on servers that are already running. If you call
+`lsp-workspace-symbols` on a fresh brain server, you will get:
+
+```json
+{"symbols":[],"error":"no_running_server"}
+```
+
+This does NOT mean "no symbols exist in the project." It means "no language
+server has been spawned yet." Do not report empty results to the user as
+authoritative when you see this error code.
+
+### Warm-up procedure
+
+Before calling `lsp-workspace-symbols` for the first time on any brain server:
+
+1. Pick a real source file in `source_path`. Prefer a main entry file — e.g.,
+   for TypeScript: `src/index.ts`, `src/main.ts`, or whichever file exists.
+   Use Glob against `{source_path}` to find one if unsure. Do NOT pass a file
+   that does not exist — the warm-up will silently fail.
+
+2. Call a file-specific action to trigger `ensureReady`:
+   ```bash
+   curl -s -X POST http://localhost:{port}/api \
+     -H "Content-Type: application/json" \
+     -d '{"action":"lsp-symbols","params":{"file":"{absolute_path_to_real_file}"}}'
+   ```
+
+3. Poll `lsp-health` until a server is `ready` (not `starting`):
+   ```bash
+   curl -s -X POST http://localhost:{port}/api \
+     -H "Content-Type: application/json" \
+     -d '{"action":"lsp-health"}'
+   ```
+   Expected ready response:
+   ```json
+   {"servers":{"typescript":{"status":"ready","uptime":1234}}}
+   ```
+   Retry up to 10 times with 500ms between attempts (5 seconds total). Large
+   projects can take several seconds to finish indexing.
+
+   **If 10 retries are exhausted without a `ready` state**, do NOT treat this
+   as success-with-empty-results. Check:
+   - `lsp-health` response — is the server in `starting` state? Extend the
+     poll to 30 retries (15 seconds) for very large projects.
+   - Is the server in `error` state? Read the `message` field and surface it
+     to the user. Common causes: missing `tsconfig.json` (wrong `source_path`),
+     language server binary not installed, or a crash loop (see
+     `language_server_crashed` in Step 4 below).
+   - Is the `servers` object still empty? The warm-up file action likely
+     failed silently — verify the file path exists and is inside `source_path`.
+
+4. Now `lsp-workspace-symbols` will return real results.
+
+### Interpreting `lsp-health` states
+
+| State | Meaning | Action |
+|-------|---------|--------|
+| empty `servers` object | No language server spawned yet | Follow warm-up procedure above |
+| `status: "starting"` | Server process spawned, still indexing | Wait and retry |
+| `status: "ready"` | Ready to serve queries | Proceed |
+| `status: "crashed"` | Crashed ≤3 times | Check diagnostics for the language server's stderr |
+| `status: "error"` with `No Project` message | Wrong `source_path` | See source_path prerequisites above |
+
 ## When to Use
 
 | You want to... | Action | Example |

package/skills/wicked-brain-migrate/SKILL.md

@@ -0,0 +1,268 @@
+---
+name: wicked-brain:migrate
+description: |
+  Migrate a flat brain at ~/.wicked-brain/ into the per-project layout at
+  ~/.wicked-brain/projects/{name}/. Safe to run on already-migrated brains
+  (no-op). Auto-invoked by wicked-brain:init when a flat brain is detected.
+
+  Use when: "migrate my brain", "move brain to per-project layout",
+  "I have an old ~/.wicked-brain brain", or when init detects a flat brain.
+---
+
+# wicked-brain:migrate
+
+You migrate a flat brain (all data directly under `~/.wicked-brain/`) into the
+per-project layout (each brain under `~/.wicked-brain/projects/{project-name}/`).
+This was the layout change introduced in v0.4.7.
+
+## Cross-Platform Notes
+
+Commands in this skill work on macOS, Linux, and Windows. Prefer your native
+Read/Write/Glob tools over shell commands when possible.
+
+- macOS/Linux home: `~`
+- Windows home: `%USERPROFILE%`
+
+## When to use
+
+- User explicitly asks to migrate
+- `wicked-brain:init` detected a flat brain and invoked this skill
+- Another skill hit errors caused by the flat layout (e.g. port collisions
+  with a second project, meta-brain federation not working)
+
+## Parameters
+
+- **flat_path** (optional): path to the existing flat brain. Default: `~/.wicked-brain`
+- **project_name** (optional): target project name. Default: asked interactively
+
+## Multiple flat brains
+
+This skill migrates **one** flat brain per invocation. If the user has several
+flat brains in different locations (e.g. `~/.wicked-brain`, `~/work-brain`,
+`~/.config/wicked-brain`), run `wicked-brain:migrate` once per source, passing
+a different `flat_path` each time. Each becomes its own project brain — either
+under the same `~/.wicked-brain/projects/` umbrella (recommended — pick
+distinct project names) or under separate containers if the user prefers
+isolation.
+
+If the user asks "migrate all my brains," enumerate the flat brains you can
+find first (`find ~ -maxdepth 3 -name brain.json 2>/dev/null` on macOS/Linux)
+and confirm each with the user before running migration on it.
+
+## Process
+
+### Step 1: Detect what we're dealing with
+
+Read `{flat_path}/brain.json` — if it does not exist, there is no flat brain
+to migrate. Tell the user and stop.
+
+Read `{flat_path}/brain.json` to get the brain's `id` and `name`. These become
+the defaults for the target project.
+
+Check whether `{flat_path}/projects/` already exists:
+
+- **If `{flat_path}` looks like a pure container** (contains only a `projects/`
+  subdirectory and optionally `_meta/`) — this brain is already migrated.
+  Report "Already using per-project layout" and stop.
+
+- **If `{flat_path}/brain.json` exists at the root AND `{flat_path}/projects/`
+  also exists** — mixed state. This is a real migration case. Continue.
+
+- **If `{flat_path}/brain.json` exists and no `projects/` subdir** — standard
+  flat layout. Continue.
+
+### Step 2: Confirm target with user
+
+Ask the user:
+
+1. "What should this project brain be called?" — Default: the `id` from
+   `{flat_path}/brain.json`, or basename of the current working directory.
+2. "Target path?" — Default: `{flat_path}/projects/{project_name}`
+
+If the target directory already exists AND is non-empty, stop and tell the
+user: "A brain already exists at the target path. Pick a different name or
+remove the existing brain manually."
+
+### Step 3: Stop any running server on the flat brain
+
+**Critical on Windows** — SQLite's `.brain.db` may be locked by a running server
+process. Moving a locked file silently corrupts the migration.
+
+1. Read `{flat_path}/_meta/server.pid` if it exists.
+2. Check if the process is alive:
+   - macOS/Linux: `kill -0 {pid} 2>/dev/null`
+   - Windows: `tasklist /FI "PID eq {pid}" 2>nul | findstr {pid}`
+3. If alive, stop it:
+   - macOS/Linux: `kill {pid}`
+   - Windows PowerShell: `Stop-Process -Id {pid}`
+4. Wait 2 seconds for file handles to release.
+5. Delete the stale PID file.
+
+Also look for any other wicked-brain-server processes targeting this flat path
+and stop them. On macOS/Linux: `pgrep -f "wicked-brain-server.*{flat_path}"`.
+
+### Step 4: Create the target directory structure
+
+```bash
+# macOS/Linux
+mkdir -p {target_path}/_meta
+```
+```powershell
+# Windows
+New-Item -ItemType Directory -Force -Path "{target_path}\_meta"
+```
+
+Do NOT pre-create `raw/`, `chunks/`, `wiki/`, `memory/` — they'll be moved
+over in Step 5.
+
+### Step 5: Move data from flat to target
+
+Move each of these directories/files from `{flat_path}` to `{target_path}`
+(skip any that don't exist in the source):
+
+- `brain.json`
+- `raw/`
+- `chunks/`
+- `wiki/`
+- `memory/`
+- `.brain.db`
+- `.brain.db-shm` (SQLite WAL shared memory, may not exist)
+- `.brain.db-wal` (SQLite WAL log, may not exist)
+
+macOS/Linux:
+```bash
+for item in brain.json raw chunks wiki memory .brain.db .brain.db-shm .brain.db-wal; do
+  if [ -e "{flat_path}/$item" ]; then
+    mv "{flat_path}/$item" "{target_path}/$item"
+    echo "moved $item"
+  fi
+done
+```
+
+Windows PowerShell:
+```powershell
+$items = "brain.json","raw","chunks","wiki","memory",".brain.db",".brain.db-shm",".brain.db-wal"
+foreach ($item in $items) {
+  $src = Join-Path "{flat_path}" $item
+  if (Test-Path $src) {
+    Move-Item $src (Join-Path "{target_path}" $item)
+    Write-Host "moved $item"
+  }
+}
+```
+
+**Do NOT move `_meta/`.** The flat brain's `_meta/config.json` references the
+flat path and is about to be replaced. We'll write a fresh config in Step 6.
+
+If the move fails partway through, stop immediately. Partial migrations leave
+the brain in an unrecoverable state — tell the user what moved and what didn't,
+and ask them to resolve manually before retrying.
+
+### Step 6: Write the target's `_meta/config.json`
+
+```json
+{
+  "brain_path": "{target_path}",
+  "server_port": 4242,
+  "installed_clis": []
+}
+```
+
+`server_port: 4242` is the *preferred starting port*, not the guaranteed port.
+When the server starts in Step 7 it probes from this value upward and writes
+the actual bound port back to this same file. After Step 7, always re-read
+`_meta/config.json` to get the real port — never hardcode `4242` in downstream
+calls. This matters especially when migrating while another brain is already
+running on 4242.
+
+If the flat brain's `_meta/config.json` had a `source_path` field, copy it over
+to the new config — the LSP workspace root must follow the brain.
+
+Initialize the event log:
+- macOS/Linux: `touch {target_path}/_meta/log.jsonl`
+- Windows: `New-Item -ItemType File -Force -Path "{target_path}\_meta\log.jsonl"`
+
+### Step 7: Start the server against the new path (verification before cleanup)
+
+**Ordering matters.** Do not delete the flat `_meta/` yet — if the new server
+fails to start, you need to be able to restore. Start and verify the new
+server first; only clean up the flat path after verification succeeds.
+
+Do NOT pass `--port` — let the server pick a free port. It will write the
+actual port back to `{target_path}/_meta/config.json`.
+
+```bash
+npx wicked-brain-server --brain "{target_path}" &
+```
+
+Wait for the server to start, then re-read `{target_path}/_meta/config.json`
+to get the bound port.
+
+Health-check:
+```bash
+curl -s -X POST http://localhost:{port}/api \
+  -H "Content-Type: application/json" \
+  -d '{"action":"health"}'
+```
+
+Verify the response includes `"brain_id"` matching the id from Step 1. If the
+brain_id is different, the wrong server is responding — likely a stale process
+on that port. Stop and diagnose before proceeding.
+
+### Step 8: Verify the index still works
+
+Run a stats call against the new server and confirm the document count is
+non-zero (assuming the flat brain had any documents):
+
+```bash
+curl -s -X POST http://localhost:{port}/api \
+  -H "Content-Type: application/json" \
+  -d '{"action":"stats"}'
+```
+
+If document counts are zero but `.brain.db` was moved, the SQLite file may have
+been truncated during the move or the server is looking at the wrong path.
+**Do NOT proceed to Step 9.** Stop the new server and investigate — the flat
+`_meta/` is still intact at this point, so you can roll back by moving files
+back to `{flat_path}`.
+
+### Step 9: Clean up the flat path (only after Steps 7 and 8 pass)
+
+Do NOT run this step unless the health check AND stats check both succeeded.
+This step is irreversible.
+
+What's left at `{flat_path}` should now be:
+
+- `_meta/` directory (old config, log, stale PID)
+- Maybe a `projects/` subdirectory (if it existed before)
+
+Verify this is the case before cleanup. If other files remain at the flat
+path, stop and report them — they were not part of a standard flat brain and
+the user needs to decide what to do with them.
+
+Delete the flat `_meta/` directory:
+- macOS/Linux: `rm -rf {flat_path}/_meta`
+- Windows: `Remove-Item -Recurse -Force "{flat_path}\_meta"`
+
+The flat path is now a pure container with only `projects/` beneath it.
+
+### Step 10: Report
+
+Tell the user:
+
+"Migrated brain `{name}` from `{flat_path}` to `{target_path}`.
+- {N} documents preserved
+- Server running on port {port}
+- `source_path`: {source_path or 'not set'}
+
+The flat path `{flat_path}` is now a container for per-project brains. You can
+create additional project brains under `{flat_path}/projects/` with `wicked-brain:init`."
+
+## Rollback
+
+If anything goes wrong before Step 9 (cleanup), the migration can be rolled
+back by moving items back from `{target_path}` to `{flat_path}`. The flat
+`_meta/` is still intact through Steps 1-8.
+
+After Step 9 there is no clean rollback — the flat `_meta/` has been deleted.
+Never run Step 9 until Steps 7 and 8 both pass.

package/skills/wicked-brain-update/SKILL.md

@@ -30,18 +30,40 @@ For the brain path default:
 
 ## Process
 
-### Step 1: Check current version
+### Step 1: Check current installed version
 
-Read the installed server version:
-```bash
-wicked-brain-server --version 2>/dev/null || npx wicked-brain-server --version 2>/dev/null || echo "not installed"
-```
+The `wicked-brain-server` binary lives inside the globally installed `wicked-brain` npm package. Read its version directly from the installed package:
 
-If that doesn't work, check the package directly:
 ```bash
-npm list -g wicked-brain-server --json 2>/dev/null | grep version
+npm list -g wicked-brain --json 2>/dev/null | python3 -c "
+import json, sys
+try:
+    d = json.load(sys.stdin)
+    deps = d.get('dependencies', {})
+    v = deps.get('wicked-brain', {}).get('version', 'not installed')
+    print(v)
+except Exception:
+    print('not installed')
+" 2>/dev/null || npm list -g wicked-brain --json 2>/dev/null | python -c "
+import json, sys
+try:
+    d = json.load(sys.stdin)
+    deps = d.get('dependencies', {})
+    v = deps.get('wicked-brain', {}).get('version', 'not installed')
+    print(v)
+except Exception:
+    print('not installed')
+"
 ```
 
+If the result is `not installed`, the package was never globally installed (the
+user may have been running via `npx` only). Treat this as "needs install" and
+proceed to Step 4.
+
+**Do NOT use `npx wicked-brain-server --version`** — it may return the version
+of a stale cached npx copy, not the globally installed one that actually runs
+when brain servers start.
+
 ### Step 2: Check latest version on npm
 
 ```bash
@@ -58,15 +80,45 @@ If an update is available, ask the user:
 
 ### Step 4: Update (if user approves)
 
-#### Update everything (skills + server)
+**Critical:** `npx wicked-brain@latest` only runs the *installer* — it refreshes
+the skill markdown files in your CLI's skills directory, but it does NOT update
+the globally installed `wicked-brain-server` binary. The skills will then expect
+features that the old server doesn't have, producing confusing errors.
 
-The server is bundled in the main package — one command updates both:
+Use `npm install -g` to update the actual binary:
 
 ```bash
-npx wicked-brain@latest
+npm install -g wicked-brain@latest 2>&1
 ```
 
-This re-runs the installer with the latest version, updating all skills across detected CLIs. The server binary updates automatically since it's part of the same package.
+On Windows PowerShell (no change needed):
+```powershell
+npm install -g wicked-brain@latest
+```
+
+If this fails with `EACCES` / permission denied:
+- macOS/Linux: `sudo npm install -g wicked-brain@latest`
+- Windows: re-run the shell as Administrator, or fix npm's global prefix per
+  npm docs. Do NOT silently skip — report the failure to the user and stop.
+
+After a successful `npm install -g`, also run the installer to refresh skill
+files in all detected CLIs (skills are copied from the installed package, not
+downloaded separately):
+
+```bash
+npx wicked-brain
+```
+
+### Step 4a: Verify the update landed
+
+Re-run the Step 1 version check. The version reported MUST match the latest
+version from Step 2. If it still shows the old version:
+
+1. Check `which wicked-brain-server` (macOS/Linux) or `where wicked-brain-server` (Windows) — the shell may have cached a path to a different installation.
+2. Clear npm's global cache: `npm cache clean --force`
+3. Check if a different Node.js version (nvm, fnm, volta) is pinning a stale copy.
+
+Do NOT proceed to Step 5 until version verification succeeds. Reporting a successful update while the binary is stale is the top failure mode of this skill.
 
 ### Step 5: Restart server if running