prism-mcp-server 15.7.4 → 16.0.0

package/README.md

@@ -157,8 +157,45 @@ Categories: abstention, adversarial traps, cascade, disambiguation, edge cases,
 ### 🔍 L3 Grounding Verifier
 When `prism_infer` receives an `evidence` payload, the grounding verifier automatically checks the model's response against the provided evidence before returning to the caller. Unverified or hallucinated claims are flagged. This is the third layer (L3) of the cascade — after tool routing (L1) and confidence gating (L2).
 
-### ⚡ Zero-search retrieval
-Holographic Reduced Representations (HRR) for instant similarity lookups without an index. ~5ms over 100K memories.
+### ⚡ Zero-search retrieval *(new in v15.8)*
+Holographic Reduced Representations (HRR) via Rust WASM for instant memory retrieval without a database query.
+
+**Three adaptive strategies:**
+- **GloVe embeddings** (offline, 50K words) — 87% Top-1 accuracy, stable at 200+ concepts
+- **API embeddings** (Gemini/Voyage) — 90%+ accuracy when online
+- **NeurIPS 2021 projection** — unit-modulus normalization for numerical stability
+
+**Retrieval cascade:** HRR (~0.2ms) → FTS5 (~50ms) → Supabase (~200ms)
+
+| Metric | HRR (WASM) | FTS5 | Supabase Vector |
+|--------|-----------|------|-----------------|
+| Latency | **0.2ms** | 50ms | 200ms |
+| Speedup | **1x** | 250x slower | 1000x slower |
+| Offline | **Yes** | Yes | No |
+| Accuracy (GloVe) | **87% Top-1** | 95%+ | 95%+ |
+| Hologram size | **8KB** | Index varies | Cloud |
+
+HRR acts as Tier 0 — if confidence is high, FTS5 is skipped entirely. Falls through gracefully when HRR has no match. 97 dedicated tests (72 system + 25 API/client). Built with Rust + `rustfft` + `wasm-bindgen` (229KB binary).
+
+**HRR AAC prediction benchmark** — real-world impact on Prism AAC word prediction (10 scenarios, 54 integration tests):
+
+| Scenario | Baseline Top-1 | +HRR Top-1 | Top-1 Lift | MRR Lift |
+|----------|---------------|------------|-----------|----------|
+| Core AAC phrases | 36.7% | 46.7% | **+27.3%** | +6.0% |
+| Personal vocabulary | 70.4% | 81.5% | **+15.8%** | +9.2% |
+| Mixed (all phrases) | 47.2% | 56.9% | **+20.6%** | +5.7% |
+| Cross-session recall | 80.0% | 80.0% | +0.0% | +0.0% |
+
+Top-1 = correct word is tile #1. MRR = Mean Reciprocal Rank. Zero Top-5 regressions in any scenario. HRR encodes bigrams + trigrams from every spoken phrase; probes take ~0.2ms — safe on every keystroke. All Synalux apps (clinical, AAC, PrismCoach) share HRR via the portal `/api/v1/hrr` endpoint.
+
+**Competitive comparison:**
+
+| System | Retrieval | Offline | Cost | Latency |
+|--------|-----------|---------|------|---------|
+| **Prism Coder** | **HRR + FTS5 + Supabase cascade** | **Yes** | **$0** | **0.2ms** |
+| Mem0 | Vector DB (Qdrant/Pinecone) | No | $249/mo | ~100ms |
+| Zep | Vector DB + temporal graph | No | $99/mo | ~80ms |
+| Hermes (NousResearch) | HRR + SQLite | Yes | Free | ~5ms |
 
 ### 🌐 Multi-agent Hivemind
 Multiple AI agents share the same Mind Palace. Each agent has a role (dev / qa / pm / etc.) and sees scoped context. Heartbeat + roster for coordination.
@@ -436,7 +473,7 @@ prism register-models         # Alias dcostenco/prism-coder:* → prism-coder:*
 ## Testing
 
 ```bash
-npm test                           # 1,815 test cases across 71 files (vitest)
+npm test                           # 2,418 test cases across 81 files (vitest)
 npm test -- --coverage             # coverage report
 python3 tests/benchmarks/prism-routing-100/benchmark.py --models 1b7 14b 32b
 ```
@@ -444,12 +481,16 @@ python3 tests/benchmarks/prism-routing-100/benchmark.py --models 1b7 14b 32b
 **Pinned in CI** — 327 tests enforce every constant: ACT-R decay `d=0.25`, spreading-activation hybrid score `0.7/0.3`, experience bias `MIN_SAMPLES=5` / `MAX_BIAS_CAP=0.15`, graph-metrics warning ratios `0.20 / 0.30 / 0.40`, compaction's 25KB prompt-budget. CI catches divergence automatically.
 
 **Coverage areas**:
-- HRR (Holographic Reduced Representations) edge cases + performance
-- Encrypted sync corruption recovery
+- HRR zero-search retrieval (97 tests: 3 embedding strategies, edge cases, persistence, adaptive cascade, API client, chat integration)
+- Knowledge ingestion (32 tests: chunker, Q&A gen, webhook, security, storage round-trip)
+- Prism infer cascade (110 tests: tier selection, cloud fallback, grounding verifier)
+- Compaction handler (rollup creation, concurrency guard, LLM failure)
+- Model picker (20 tests: 14b default ceiling, 4b verifier, RAM gating)
+- Storage round-trip (12 architectural guard tests preventing bypass)
 - BCBA skill integration
 - Deep storage tier
 - Dashboard rendering
-- Routing benchmarks (102-case Prism eval) — see `tests/benchmarks/prism-routing-100/`
+- Routing benchmarks (eval_300: 300 cases, 17 tools)
 
 ## Migration
 

package/dist/aba-protocol.js

@@ -70,7 +70,7 @@ export const RULE7_VSCODE = [
 ].join('\n');
 // ─── Assemblers ─────────────────────────────────────────────────
 /** Assemble the full ABA protocol for Cloud Portal */
-export function buildCloudPrompt(toolsSection) {
+function _unused_buildCloudPrompt(toolsSection) {
     return [
         toolsSection,
         '',
@@ -106,7 +106,7 @@ export function sanitizeUserInput(text) {
     return sanitizeMcpOutput(text);
 }
 /** Wrap user input in <user_input> tags after sanitization */
-export function wrapUserInput(text) {
+function _unused_wrapUserInput(text) {
     const safe = sanitizeUserInput(text);
     return `<user_input>\n${safe}\n</user_input>`;
 }

package/dist/hivemindWatchdog.js

@@ -66,7 +66,7 @@ export function drainAlerts(project) {
 /**
  * Get count of pending alerts (for testing/debugging).
  */
-export function getPendingAlertCount() {
+function _unused_getPendingAlertCount() {
     return pendingAlerts.size;
 }
 // ─── Watchdog Lifecycle ──────────────────────────────────────

package/dist/utils/analytics.js

@@ -33,7 +33,7 @@ function estimateTokens(text) {
  * Call this from server.ts after each tool handler completes.
  * Uses a write buffer to avoid per-call SQLite overhead.
  */
-export function recordInvocation(tool, project, args, response, durationMs, success, errorMessage) {
+function _unused_recordInvocation(tool, project, args, response, durationMs, success, errorMessage) {
     const invocation = {
         id: `${Date.now()}-${Math.random().toString(36).slice(2, 8)}`,
         tool,

package/dist/utils/llm/adapters/gemini.js

@@ -77,17 +77,67 @@ export class GeminiAdapter {
         return result.response.text();
     }
     // ─── Embedding Generation ────────────────────────────────────────────────
+    static _embeddingCache = new Map();
+    static _inflight = new Map();
+    static EMBED_CACHE_MAX = 256;
+    static EMBED_CACHE_TTL_MS = 5 * 60 * 1000;
+    getCachedEmbedding(key) {
+        const entry = GeminiAdapter._embeddingCache.get(key);
+        if (!entry)
+            return null;
+        if (Date.now() - entry.ts > GeminiAdapter.EMBED_CACHE_TTL_MS) {
+            GeminiAdapter._embeddingCache.delete(key);
+            return null;
+        }
+        // Move to tail for LRU on read
+        GeminiAdapter._embeddingCache.delete(key);
+        GeminiAdapter._embeddingCache.set(key, entry);
+        return entry.embedding;
+    }
+    setCachedEmbedding(key, embedding) {
+        // Delete-then-set moves the key to tail for correct LRU eviction
+        GeminiAdapter._embeddingCache.delete(key);
+        if (GeminiAdapter._embeddingCache.size >= GeminiAdapter.EMBED_CACHE_MAX) {
+            const oldest = GeminiAdapter._embeddingCache.keys().next().value;
+            if (oldest !== undefined)
+                GeminiAdapter._embeddingCache.delete(oldest);
+        }
+        GeminiAdapter._embeddingCache.set(key, { embedding, ts: Date.now() });
+    }
     async generateEmbedding(text) {
         // Guard: empty string would produce a useless/degenerate embedding.
         // Better to fail loudly here than store a zero-vector in the DB.
         if (!text || !text.trim()) {
             throw new Error("Cannot generate embedding for empty text.");
         }
+        const trimmedText = text.trim();
+        const cacheKey = `${trimmedText.substring(0, 500)}|L${trimmedText.length}`;
+        const cached = this.getCachedEmbedding(cacheKey);
+        if (cached) {
+            debugLog(`[GeminiAdapter] Embedding cache HIT`);
+            return cached;
+        }
+        // In-flight dedup: if another call is already generating this embedding, await it
+        const inflight = GeminiAdapter._inflight.get(cacheKey);
+        if (inflight) {
+            debugLog(`[GeminiAdapter] Embedding in-flight dedup HIT`);
+            return inflight;
+        }
+        const promise = this._generateEmbeddingImpl(trimmedText, cacheKey);
+        GeminiAdapter._inflight.set(cacheKey, promise);
+        try {
+            return await promise;
+        }
+        finally {
+            GeminiAdapter._inflight.delete(cacheKey);
+        }
+    }
+    async _generateEmbeddingImpl(inputTextRaw, cacheKey) {
         // ── Truncation Guard ───────────────────────────────────────────────────
         // gemini-embedding-001 has a ~2048 token context window.
         // Long session summaries (esp. code-heavy ones) can easily exceed this.
         // We truncate proactively rather than let the API return a 400 error.
-        let inputText = text;
+        let inputText = inputTextRaw;
         if (inputText.length > MAX_EMBEDDING_CHARS) {
             debugLog(`[GeminiAdapter] Embedding input truncated from ${inputText.length}` +
                 ` to ~${MAX_EMBEDDING_CHARS} chars (word-safe)`);
@@ -130,6 +180,7 @@ export class GeminiAdapter {
             throw new Error(`Embedding dimension mismatch: expected ${EMBEDDING_DIMS},` +
                 ` got ${values?.length ?? "unknown"}`);
         }
+        this.setCachedEmbedding(cacheKey, values);
         return values;
     }
     // ─── Image Description (VLM) ─────────────────────────────────────────────

package/dist/utils/llm/adapters/openai.js

@@ -102,18 +102,47 @@ export class OpenAIAdapter {
         return response.choices[0]?.message?.content ?? "";
     }
     // ─── Embedding Generation ────────────────────────────────────────────────
+    static _embeddingCache = new Map();
+    static _inflight = new Map();
+    static EMBED_CACHE_MAX = 256;
+    static EMBED_CACHE_TTL_MS = 5 * 60 * 1000;
     async generateEmbedding(text) {
         // Guard: empty input produces a degenerate embedding — fail loudly.
         if (!text || !text.trim()) {
             throw new Error("Cannot generate embedding for empty text.");
         }
-        // Read embedding model at call time for hot-swap support.
+        const trimmedText = text.trim();
+        const cacheKey = `${trimmedText.substring(0, 500)}|L${trimmedText.length}`;
+        const entry = OpenAIAdapter._embeddingCache.get(cacheKey);
+        if (entry && Date.now() - entry.ts < OpenAIAdapter.EMBED_CACHE_TTL_MS) {
+            debugLog(`[OpenAIAdapter] Embedding cache HIT`);
+            // Move to tail for LRU on read
+            OpenAIAdapter._embeddingCache.delete(cacheKey);
+            OpenAIAdapter._embeddingCache.set(cacheKey, entry);
+            return entry.embedding;
+        }
+        // In-flight dedup
+        const inflight = OpenAIAdapter._inflight.get(cacheKey);
+        if (inflight) {
+            debugLog(`[OpenAIAdapter] Embedding in-flight dedup HIT`);
+            return inflight;
+        }
+        const promise = this._generateEmbeddingImpl(trimmedText, cacheKey);
+        OpenAIAdapter._inflight.set(cacheKey, promise);
+        try {
+            return await promise;
+        }
+        finally {
+            OpenAIAdapter._inflight.delete(cacheKey);
+        }
+    }
+    async _generateEmbeddingImpl(inputTextRaw, cacheKey) {
         const model = getSettingSync("openai_embedding_model", "text-embedding-3-small");
         // ── Truncation Guard ───────────────────────────────────────────────────
         // text-embedding-3-small accepts up to 8191 tokens.
         // We apply the same preventive truncation as GeminiAdapter so behavior
         // is consistent regardless of which provider is active.
-        let inputText = text;
+        let inputText = inputTextRaw;
         if (inputText.length > MAX_EMBEDDING_CHARS) {
             debugLog(`[OpenAIAdapter] Embedding input truncated from ${inputText.length}` +
                 ` to ~${MAX_EMBEDDING_CHARS} chars (word-safe)`);
@@ -148,6 +177,13 @@ export class OpenAIAdapter {
                 `If using a local model, use one that natively outputs ${EMBEDDING_DIMS} dims ` +
                 `(e.g. nomic-embed-text) or supports the Matryoshka 'dimensions' parameter.`);
         }
+        OpenAIAdapter._embeddingCache.delete(cacheKey);
+        if (OpenAIAdapter._embeddingCache.size >= OpenAIAdapter.EMBED_CACHE_MAX) {
+            const oldest = OpenAIAdapter._embeddingCache.keys().next().value;
+            if (oldest !== undefined)
+                OpenAIAdapter._embeddingCache.delete(oldest);
+        }
+        OpenAIAdapter._embeddingCache.set(cacheKey, { embedding, ts: Date.now() });
         return embedding;
     }
     // ─── Image Description (VLM) ─────────────────────────────────────────────

package/dist/utils/localLlm.js

@@ -201,7 +201,7 @@ export async function callLocalLlm(userPrompt, model = PRISM_LOCAL_LLM_MODEL, sy
  *
  * @returns true if Ollama responds to /api/tags within 3 seconds.
  */
-export async function isLocalLlmAvailable() {
+async function _unused_isLocalLlmAvailable() {
     if (!PRISM_LOCAL_LLM_ENABLED)
         return false;
     try {

package/dist/utils/universalImporter.js

@@ -36,6 +36,7 @@
  *   For ambiguous files, --format= is mandatory.
  * ═══════════════════════════════════════════════════════════════════
  */
+import { debugLog } from "./logger.js";
 import { getStorage } from "../storage/index.js";
 import { claudeAdapter } from "./migration/claudeAdapter.js";
 import { geminiAdapter } from "./migration/geminiAdapter.js";
@@ -128,16 +129,16 @@ export async function universalImporter(options) {
         if (sniffed) {
             adapter = adapters.find((a) => a.id === sniffed);
             if (adapter) {
-                console.log(`🔍 Auto-detected format: ${sniffed} (via content sniffing)`);
+                debugLog(`🔍 Auto-detected format: ${sniffed} (via content sniffing)`);
             }
         }
     }
     if (!adapter) {
         throw new Error(`Could not determine adapter for file: ${filePathArg}. Use --format to specify.`);
     }
-    console.log(`🚀 Starting migration from ${adapter.id} to Prism...`);
+    debugLog(`🚀 Starting migration from ${adapter.id} to Prism...`);
     if (dryRun)
-        console.log("⚠️ DRY RUN MODE - storage writes disabled.");
+        debugLog("⚠️ DRY RUN MODE - storage writes disabled.");
     // ── Storage + Concurrency ──────────────────────────────────────
     const storage = await getStorage();
     const limit = pLimit(5);
@@ -169,7 +170,7 @@ export async function universalImporter(options) {
         conversationCount++;
         if (verbose) {
             const turnCount = turns.length;
-            console.log(`📦 Conversation #${conversationCount}: ${turnCount} turns (${sessionDate}) → ${conversationId}`);
+            debugLog(`📦 Conversation #${conversationCount}: ${turnCount} turns (${sessionDate}) → ${conversationId}`);
         }
         if (dryRun) {
             successCount += turns.length;
@@ -188,7 +189,7 @@ export async function universalImporter(options) {
             if (existing.length > 0) {
                 skipCount += turns.length;
                 if (verbose) {
-                    console.log(`⏭️  Skipping duplicate: ${conversationId}`);
+                    debugLog(`⏭️  Skipping duplicate: ${conversationId}`);
                 }
                 return;
             }
@@ -229,13 +230,13 @@ export async function universalImporter(options) {
         // ── Final Flush ──────────────────────────────────────────────
         // Flush the last conversation (no trailing time gap to trigger it)
         await flushConversation();
-        console.log("\n✅ Migration complete!");
-        console.log(`   Conversations: ${conversationCount}`);
-        console.log(`   Turns processed: ${successCount}`);
+        debugLog("\n✅ Migration complete!");
+        debugLog(`   Conversations: ${conversationCount}`);
+        debugLog(`   Turns processed: ${successCount}`);
         if (skipCount > 0)
-            console.log(`   Skipped (dup): ${skipCount}`);
+            debugLog(`   Skipped (dup): ${skipCount}`);
         if (failCount > 0)
-            console.log(`   Failed:         ${failCount}`);
+            debugLog(`   Failed:         ${failCount}`);
         return { successCount, failCount, skipCount, conversationCount };
     }
     catch (err) {
@@ -261,7 +262,7 @@ async function runCLI() {
     const dryRun = args.includes("--dry-run") || args.includes("-d");
     const verbose = args.includes("--verbose") || args.includes("-v");
     if (!filePathArg) {
-        console.log(`
+        debugLog(`
 Prism Universal History Importer
 Usage: node universalImporter.js <file> [options]
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "prism-mcp-server",
-  "version": "15.7.4",
+  "version": "16.0.0",
   "mcpName": "io.github.dcostenco/prism-coder",
   "description": "Prism Coder — Cognitive memory + tool-calling intelligence for AI agents. Mind Palace persistent memory (BFCL Gold Certified, 100% Tool-Call Accuracy, 54 Agent Skills, Zero-Search HDC/HRR retrieval, HIPAA-hardened local-first storage, SLERP-optimized GRPO alignment) plus the prism-coder:7b / 14b open-weights LLM fleet.",
   "module": "index.ts",