opencode-mempalace-persistence 2.4.0 → 2.5.1

package/README.md

@@ -65,32 +65,29 @@ mempalace mcp
 
 The `mempalace mcp` command gives you the exact MCP setup string for your configuration.
 
-### 4. Memory injection
+### 4. Plugin config (all optional)
 
-The plugin automatically injects your identity + relevant memories from MemPalace into every prompt. No model discipline required.
-
-Create `~/.mempalace/plugin-config.json`:
+No config file is needed to start: every setting has a default, and **not
+creating anything means all defaults**. When you want to change one,
+create `~/.mempalace/plugin-config.json`:
 
 ```json
 {
-  "autoInjectContext": true
+  "autoInjectContext": false,
+  "saveInterval": 15,
+  "toasts": true
 }
 ```
 
-Optional tuning:
-
-```json
-{
-  "autoInjectContext": true,
-  "saveInterval": 15
-}
-```
-
-- `saveInterval` (default `15`, min `5`): human messages between AI checkpoints — same cadence as the official MemPalace save hook.
+| Key | Default | What it does |
+|---|---|---|
+| `autoInjectContext` | `false` | Inject identity + `mempalace search` results into every prompt. Needs no model discipline, but adds context (and noise) to every turn. Off by default: recall happens via the skill instead |
+| `saveInterval` | `15` (min `5`) | Human messages between AI checkpoints — same cadence as the official MemPalace save hook |
+| `toasts` | `true` | TUI toasts for mines, checkpoints and MemPalace calls. Set `false` to silence them |
 
 **Do NOT put this in `opencode.json`** — OpenCode's schema validation rejects unknown keys. The plugin reads its config from `~/.mempalace/plugin-config.json` instead.
 
-When enabled:
+When `autoInjectContext` is enabled:
 - **First message**: Injects your identity from `~/.mempalace/identity.txt`
 - **Every message**: Runs `mempalace search` and injects relevant results
 
@@ -101,12 +98,14 @@ Create `~/.config/opencode/AGENTS.md`:
 ```markdown
 # Memory & Knowledge instructions
 
-## Recall (usually already covered)
+## Recall (via skill, unless auto-inject is on)
 
-The plugin auto-injects identity + relevant memories into every prompt.
-Only search MemPalace yourself (`mempalace_mempalace_search`) when the
-question is about past work, decisions, people, or projects AND the
-injected context has nothing — quote results verbatim, never paraphrase.
+Recall follows the bundled `mempalace-recall` skill (question-driven
+search). If you enabled `autoInjectContext`, identity + relevant memories
+are additionally injected into every prompt — then only search MemPalace
+yourself when the question is about past work, decisions, people, or
+projects AND the injected context has nothing. Either way, quote results
+verbatim, never paraphrase.
 
 ## Record facts (after responding, only when something new emerged)
 
@@ -183,7 +182,16 @@ Session goes idle / process exits
 
 Every MemPalace call — plugin searches, model MCP calls (search, diary,
 KG) — also raises a short TUI toast with what was asked and a result
-preview, so background memory activity is always visible.
+preview, so background memory activity is always visible. A startup toast
+shows the loaded plugin version (`plugin v2.x loaded`), so you always know
+whether you're running the npm release or a local build.
+
+Multiple opencode instances are supported: mines coordinate through the
+palace lock with backoff-and-retry (up to ~10min per wing), so concurrent
+instances interleave wing by wing instead of starving each other — backfills
+complete even with two sessions open. Routine contention shows one info
+toast every 5 minutes max. To reduce contention during huge backfills, a
+single instance is still fastest.
 
 Compaction starts
   → [MemPalace Pre-Compact Emergency Save]: model files everything first
@@ -260,7 +268,7 @@ The plugin exports everything in the opencode database on the next sync, then re
 |---|---|
 | `~/.config/opencode/opencode.json` | OpenCode config with plugin + MCP |
 | `~/.config/opencode/AGENTS.md` | Tells the model to manage KG facts |
-| `~/.mempalace/plugin-config.json` | Plugin config (`autoInjectContext`, `saveInterval`) |
+| `~/.mempalace/plugin-config.json` | Plugin config (`autoInjectContext`, `saveInterval`, `toasts` — all optional, see §4) |
 | `~/.config/opencode/skills/mempalace-recall/SKILL.md` | Bundled recall skill (copy from `skills/` in this repo) |
 | `~/.mempalace/identity.txt` | Your identity (injected by plugin) |
 | `~/.mempalace/hook_state/opencode_counters.json` | Per-session message counters (checkpoint cadence) |
@@ -299,6 +307,28 @@ When set, the plugin writes a debug log to `/tmp/opencode-mempalace.log`.
 
 ---
 
+## Observability: toasts and commands
+
+Background memory activity is visible three ways — ephemeral first,
+history on demand, never polluting session context:
+
+- **TUI toasts** (on by default, `"toasts": false` to disable): mine
+  results and errors, armed checkpoints, and every MemPalace call
+  (plugin searches and model MCP calls) with what was asked plus a
+  short answer preview. A startup toast shows the loaded build
+  (`opencode-mempalace-persistence v2.x loaded`), so npm-cache vs
+  local build is never a mystery.
+- **`/memory-status`** — palace health in the transcript: drawers,
+  KG stats, last sync, pending backlog, recent activity, errors with
+  explanations, active config. Read-only.
+- **`/memory-log [N] [filter]`** — the interaction history: every
+  search (query → result count), tool call (asked → answered preview),
+  mine (outcome per wing) and checkpoint, newest last. Backed by
+  `~/.mempalace/hook_state/interactions.log` (JSON lines, auto-rotated).
+  Read-only.
+
+---
+
 ## License
 
 MIT

package/dist/index.js

@@ -1,8 +1,9 @@
 import { execSync, execFileSync, execFile, spawnSync } from "child_process";
-import { existsSync, readFileSync, writeFileSync, mkdirSync, rmdirSync, unlinkSync, appendFileSync } from "fs";
+import { existsSync, readFileSync, writeFileSync, mkdirSync, rmdirSync, unlinkSync, appendFileSync, statSync, readdirSync } from "fs";
 import { homedir } from "os";
-import { join } from "path";
+import { join, dirname } from "path";
 import { createHash } from "crypto";
+import { fileURLToPath } from "url";
 const HOME = homedir();
 const MEMPALACE_BIN = join(HOME, ".local/bin/mempalace");
 const OPENCODE_DB = join(HOME, ".local/share/opencode/opencode.db");
@@ -12,6 +13,9 @@ const IDENTITY_FILE = join(HOME, ".mempalace/identity.txt");
 const HOOK_STATE_DIR = join(HOME, ".mempalace/hook_state");
 const COUNTERS_FILE = join(HOOK_STATE_DIR, "opencode_counters.json");
 const HOOK_LOG = join(HOOK_STATE_DIR, "hook.log");
+const INTERACTIONS_LOG = join(HOOK_STATE_DIR, "interactions.log");
+// Cap the interactions log so it never grows unbounded (approx lines).
+const INTERACTIONS_MAX_LINES = 2000;
 // Private sync workspace (0700): transcripts contain conversation text,
 // so they must never sit world-readable in /tmp (see PR #1524 review).
 const SYNC_DIR = join(HOME, ".mempalace/oc-sessions");
@@ -46,6 +50,28 @@ function errLog(msg) {
     log("ERROR: " + msg);
     hookLog("ERROR: " + msg);
 }
+// Structured interaction log (JSON lines): every MemPalace question and
+// answer, readable by /memory-log. Ephemeral TUI toasts show the moment;
+// this file keeps the history — without polluting session context.
+function ilog(kind, data) {
+    try {
+        mkdirSync(HOOK_STATE_DIR, { recursive: true });
+        appendFileSync(INTERACTIONS_LOG, JSON.stringify({ ts: new Date().toISOString(), kind, ...data }) + "\n");
+        // Cheap rotation: count lines only when the file looks big.
+        let size = 0;
+        try {
+            size = statSync(INTERACTIONS_LOG).size;
+        }
+        catch { }
+        if (size > 600 * 1024) {
+            const lines = readFileSync(INTERACTIONS_LOG, "utf-8").split("\n");
+            if (lines.length > INTERACTIONS_MAX_LINES) {
+                writeFileSync(INTERACTIONS_LOG, lines.slice(-INTERACTIONS_MAX_LINES).join("\n"));
+            }
+        }
+    }
+    catch { }
+}
 function toastsEnabled() {
     try {
         const raw = readFileSync(PLUGIN_CONFIG, "utf-8");
@@ -56,9 +82,78 @@ function toastsEnabled() {
     catch { }
     return true;
 }
+// Messages arrived after the sync cursor: still waiting for the next run.
+function countPendingMessages(sinceMs) {
+    try {
+        const out = runPython(`
+import sqlite3
+db = sqlite3.connect(${JSON.stringify(OPENCODE_DB)})
+n = db.execute("SELECT COUNT(*) FROM message WHERE time_created > ${sinceMs}").fetchone()[0]
+db.close()
+print(n)
+`);
+        const n = parseInt(out.trim(), 10);
+        return isNaN(n) ? 0 : n;
+    }
+    catch {
+        return 0;
+    }
+}
+let cachedName = undefined;
+let cachedVersion = undefined;
+function pluginName() {
+    if (cachedName !== undefined)
+        return cachedName;
+    try {
+        const here = dirname(fileURLToPath(import.meta.url));
+        const pkg = JSON.parse(readFileSync(join(here, "..", "package.json"), "utf-8"));
+        cachedName = typeof pkg?.name === "string" ? pkg.name : "opencode-mempalace-persistence";
+    }
+    catch {
+        cachedName = "opencode-mempalace-persistence";
+    }
+    return cachedName ?? "opencode-mempalace-persistence";
+}
+function pluginVersion() {
+    if (cachedVersion !== undefined)
+        return cachedVersion;
+    try {
+        const here = dirname(fileURLToPath(import.meta.url));
+        const pkg = JSON.parse(readFileSync(join(here, "..", "package.json"), "utf-8"));
+        cachedVersion = typeof pkg?.version === "string" ? pkg.version : "unknown";
+    }
+    catch {
+        cachedVersion = "unknown";
+    }
+    return cachedVersion ?? "unknown";
+}
+// Pending export files (backlog). Pure fs walk, no shell.
+function countPendingFiles() {
+    let n = 0;
+    try {
+        for (const w of readdirSync(SYNC_DIR, { withFileTypes: true })) {
+            if (!w.isDirectory())
+                continue;
+            try {
+                n += readdirSync(join(SYNC_DIR, w.name)).length;
+            }
+            catch { }
+        }
+    }
+    catch { }
+    return n;
+}
+function countPendingSuffix() {
+    const pending = countPendingFiles();
+    return pending > 0 ? `, ${pending} file(s) waiting to mine` : ", queue empty";
+}
 // TUI toast client (set by the factory). Fire-and-forget: headless runs
 // (`opencode run`, no TUI attached) must never break on this.
 let tuiClient = null;
+// Throttle for routine skip notices (busy palace): at most one toast
+// per window, otherwise active sessions get spammed every turn.
+let lastBusyToastTs = 0;
+const BUSY_TOAST_WINDOW_MS = 5 * 60 * 1000;
 function toast(variant, title, message) {
     if (!toastsEnabled() || !tuiClient?.tui?.showToast)
         return;
@@ -244,6 +339,7 @@ function mempalaceSearch(query) {
     const bin = resolveBin();
     if (!bin)
         return "";
+    const started = Date.now();
     try {
         // argv array, no shell (see PR #2): the query is raw user message
         // text, so it must never pass through /bin/sh. No manual escaping needed.
@@ -253,10 +349,12 @@ function mempalaceSearch(query) {
         }).trim();
         if (!out || out.includes("No results")) {
             toast("info", "MemPalace", `search "${query.slice(0, 50)}" → no results`);
+            ilog("search", { via: "cli", query: query.slice(0, 200), results: 0, ms: Date.now() - started });
             return "";
         }
         const n = (out.match(/\n\s*\[\d+\]/g) || []).length || 1;
         toast("info", "MemPalace", `search "${query.slice(0, 50)}" → ${n} result(s)`);
+        ilog("search", { via: "cli", query: query.slice(0, 200), results: n, ms: Date.now() - started });
         return out.slice(0, MAX_INJECT_CHARS);
     }
     catch {
@@ -470,7 +568,19 @@ function doDbSync() {
     miningLock = true;
     log(`mining ${wingCount(wings)} sessions across ${wings.size} wings`);
     const entries = [...wings.entries()];
-    const mineNext = (i) => {
+    // Per-wing drawers tally for the final toast (parsed from mine stdout).
+    const wingDrawers = new Map();
+    const parseDrawers = (stdout) => {
+        const m = String(stdout || "").match(/Drawers filed:\s*(\d+)/i);
+        return m ? parseInt(m[1], 10) : 0;
+    };
+    // Retry schedule for lock contention: two instances (or an MCP write)
+    // interleave wing by wing instead of starving each other. Total ~10min
+    // of retries per wing, then give up until the next trigger (idle/exit).
+    // miningLock stays held during backoff so one process never piles up.
+    const RETRY_DELAYS_MS = [15000, 30000, 60000, 120000, 180000, 300000];
+    const jitter = (ms) => ms + Math.floor(Math.random() * 10000);
+    const mineNext = (i, attempt = 0) => {
         if (i >= entries.length) {
             miningLock = false;
             // Advance state only on full success: on failure the same
@@ -480,7 +590,13 @@ function doDbSync() {
             cleanupExport(wings);
             log("mine done");
             const names = [...wings.keys()].join(", ");
-            toast("success", "MemPalace", `mined ${wingCount(wings)} session(s) → ${names}`);
+            const totalDrawers = [...wingDrawers.values()].reduce((a, b) => a + b, 0);
+            const detail = totalDrawers > 0 ? ` (${totalDrawers} drawers)` : "";
+            // Anything that arrived while this mine was running stays pending.
+            const remaining = countPendingMessages(now);
+            const tail = remaining > 0 ? `, ${remaining} message(s) still waiting` : ", queue empty";
+            toast("success", "MemPalace", `mined ${wingCount(wings)} session(s) → ${names}${detail}${tail}`);
+            ilog("mine", { outcome: "ok", sessions: wingCount(wings), wings: [...wings.keys()], drawers: totalDrawers, remaining });
             return;
         }
         const [wing, files] = entries[i];
@@ -496,14 +612,42 @@ function doDbSync() {
         }
         execFile(bin, mineArgs(join(OUT_DIR, wing), wing), {
             encoding: "utf-8",
-        }, (err) => {
+        }, (err, stdout) => {
             if (err) {
+                const msg = err.message || String(err);
+                // Lock contention (second opencode instance mining, or an MCP
+                // write in flight) is routine, not a failure: back off and retry
+                // the same wing — the holder releases between its own wings, so
+                // concurrent instances interleave instead of starving. Anything
+                // else is a real error.
+                if (/is held by/i.test(msg) && attempt < RETRY_DELAYS_MS.length) {
+                    const wait = jitter(RETRY_DELAYS_MS[attempt]);
+                    log(`palace busy (${wing}), retry ${attempt + 1}/${RETRY_DELAYS_MS.length} in ${Math.round(wait / 1000)}s`);
+                    const nowTs = Date.now();
+                    if (nowTs - lastBusyToastTs > BUSY_TOAST_WINDOW_MS) {
+                        lastBusyToastTs = nowTs;
+                        toast("info", "MemPalace", "palace busy (another instance mining?) — backing off, will retry");
+                    }
+                    setTimeout(() => mineNext(i, attempt + 1), wait);
+                    return;
+                }
                 miningLock = false;
-                errLog(`mine err (${wing}): ${err.message}`);
-                toast("error", "MemPalace", `mine failed (${wing}): ${err.message.slice(0, 120)}`);
+                if (/is held by/i.test(msg)) {
+                    log(`mine skipped, palace busy (${wing}) after ${attempt} retries — next trigger will retry`);
+                    ilog("mine", { outcome: "busy", wing });
+                    return;
+                }
+                errLog(`mine err (${wing}): ${msg}`);
+                toast("error", "MemPalace", `mine failed (${wing}): ${msg.slice(0, 120)}`);
+                ilog("mine", { outcome: "error", wing, error: msg.slice(0, 200) });
                 return;
             }
             log(`mined wing ${wing} (${files.length} sessions)`);
+            wingDrawers.set(wing, parseDrawers(stdout));
+            // Truthful progress: one toast per completed wing (an exact % is
+            // impossible — the mine CLI is a black box with ~4s startup cost
+            // per invocation, so per-file mines would only add overhead).
+            toast("info", "MemPalace", `wing ${wing} done (${i + 1}/${entries.length})`);
             mineNext(i + 1);
         });
     };
@@ -560,8 +704,17 @@ export default (async ({ client }) => {
     // Catch anything missed by a previous run (e.g. content skipped when
     // the exit budget ran out). Fires once per server lifetime.
     setTimeout(() => dbSync(), 10000);
+    // Startup toast (delayed so the TUI is attached): shows exactly which
+    // plugin build is loaded — no more guessing npm-cache vs local build —
+    // plus pending backlog so a restarted-into-backlog state is visible.
+    setTimeout(() => {
+        toast("info", "MemPalace", `${pluginName()} v${pluginVersion()} loaded${countPendingSuffix()}`);
+        log(`startup toast fired (${pluginName()} v${pluginVersion()})`);
+    }, 15000);
     // Crash safety: best-effort synchronous save on hard exit.
     // Mirrors the official emergency-save intent (nothing async allowed here).
+    // SIGHUP included: closing the terminal / dropping SSH kills the process
+    // group, otherwise mines would die mid-run with no save attempt.
     let exitHandled = false;
     const onExit = () => {
         if (exitHandled)
@@ -571,6 +724,7 @@ export default (async ({ client }) => {
     };
     process.once("SIGINT", onExit);
     process.once("SIGTERM", onExit);
+    process.once("SIGHUP", onExit);
     process.once("exit", onExit);
     return {
         "chat.message": async (input, output) => {
@@ -596,6 +750,7 @@ export default (async ({ client }) => {
                 c.lastCheckpoint = boundary;
                 pendingCheckpoint = { sessionID, count: c.humanMsgs };
                 hookLog(`session ${sessionID}: ${c.humanMsgs} human msgs — checkpoint armed`);
+                ilog("checkpoint", { sessionID, count: c.humanMsgs });
                 toast("info", "MemPalace", `checkpoint armed (~${c.humanMsgs} msgs): the model will file memories now`);
             }
             counters[sessionID] = c;
@@ -682,6 +837,16 @@ export default (async ({ client }) => {
                 const summary = summarizeToolCall(name, input?.args, output?.output || "");
                 log(`tool: ${summary}`);
                 toast("info", "MemPalace", summary);
+                ilog("tool", {
+                    tool: String(name).replace(/^mcp_+/, "").replace(/^mempalace_mempalace_/, "").replace(/^mempalace_/, ""),
+                    asked: (() => { try {
+                        return JSON.stringify(input?.args || {}).replace(/\s+/g, " ").slice(0, 200);
+                    }
+                    catch {
+                        return "";
+                    } })(),
+                    answered: String(output?.output || "").replace(/\s+/g, " ").slice(0, 300),
+                });
             }
             catch { }
         },

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-mempalace-persistence",
-  "version": "2.4.0",
+  "version": "2.5.1",
   "description": "OpenCode plugin — auto-sync conversations to MemPalace memory in real-time. No forced wings, KG extraction via MCP tools.",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",