prism-mcp-server 9.2.7 → 9.3.0

package/README.md

@@ -826,8 +826,12 @@ The Generator strips the `console.log`, resubmits, and the next `EVALUATE` retur
 
 ## 🆕 What's New
 
-> **Current release: v9.2.4 — Cross-Backend Reconciliation**
+> **Current release: v9.3.0 — TurboQuant ResidualNorm Tiebreaker**
 
+- 🎯 **v9.3.0 — TurboQuant ResidualNorm Tiebreaker:** Configurable ranking optimization for Tier-2 search. When compressed cosine scores are within ε of each other, prefers the candidate with lower `residualNorm` (more trustworthy compressed representation). `PRISM_TURBOQUANT_TIEBREAKER_EPSILON=0.005` gives +2pp R@1, +1pp R@5. Empirically validated at N=5K with A/B test. 1066 tests, 0 regressions. Inspired by [@m13v's suggestion](https://github.com/xiaowu0162/LongMemEval/issues/31).
+- 🔒 **v9.2.7 — Security Hardening:** Typed `PrototypePollutionError` class (replaces generic `Error` in `sanitizeForMerge()` — enables catch-site discrimination and forensic logging with `offendingKey`), explicit null-byte path injection guard in `SafetyController.validateActionsInScope()` (C-string truncation attack vector), and corrected CRDT merge semantics documentation (Remove-Wins-from-Either, not Add-Wins). 1055 tests, 0 regressions.
+- 🪟 **v9.2.6 — Windows CI Timeout Fix:** CLI integration tests timed out on Windows + Node 22.x GitHub Actions runners. Added `{ timeout: 30_000 }` to the describe block. 6 new residual distribution tests validating TurboQuant's QJL correction stability (zero R@5 delta between P50 and P95 residual vectors at d=128, 2K corpus).
+- 🔧 **v9.2.5 — Reconciliation Credential Probe Fix:** `supabaseReady` guard only resolved credentials when `requestedBackend === "supabase"`, causing reconciliation to silently skip. Added second credential probe for local + reconciliation path. Fixed Supabase schema mismatch on `key_context` column.
 - 🔄 **v9.2.4 — Cross-Backend Reconciliation:** Automatic two-layer sync from Supabase → SQLite on startup. When Claude Desktop writes handoffs and ledger entries to Supabase, Antigravity (local SQLite) now automatically detects stale data and pulls newer handoffs + the 20 most recent ledger entries. 5-second timeout prevents startup freeze. Targeted ID lookups (not full table scans) keep it safe for large databases. 13 tests including malformed JSON resilience, multi-role dedup, and timeout handling.
 - 🔧 **v9.2.3 — Code Review Hardening:** 10x faster split-brain detection (lightweight direct queries replace full `StorageBackend` construction), variable shadowing fix in CLI, resource leak fix in SQLite alternate client.
 - 🚨 **v9.2.2 — Critical: Split-Brain Detection & Prevention:** When multiple MCP clients use different storage backends (e.g., Claude Desktop → Supabase, Antigravity → SQLite), session state could silently diverge, causing agents to act on stale TODOs and outdated context. **New: `--storage` flag** on `prism load` CLI lets callers explicitly select which backend to read from. **New: Split-Brain Drift Detection** in `session_load_context` — compares active and alternate backend versions at load time and warns prominently when they diverge. Session loader script updated to respect `PRISM_STORAGE` environment variable.
@@ -854,7 +858,7 @@ Standard memory servers (like Mem0, Zep, or the baseline Anthropic MCP) act as p
 | **Storage Engine** | **BYO SQLite or Supabase** | Managed Cloud / VectorDBs | Managed Cloud / Postgres | Local SQLite only |
 | **Context Assembly** | **Progressive (Quick/Std/Deep)** | Top-K Semantic Search | Top-K + Temporal Summaries | Basic Entity Search |
 | **Memory Mechanics** | **ACT-R Activation, Spreading Activation, Hebbian Consolidation, Rejection Gate** | Basic Vector + Entity | Fading Temporal Graph | None (Infinite growth) |
-| **Multi-Agent Sync** | **CRDT (Add-Wins / LWW)** | Cloud locks | Postgres locks | ❌ None (Data races) |
+| **Multi-Agent Sync** | **CRDT (Remove-Wins / LWW)** | Cloud locks | Postgres locks | ❌ None (Data races) |
 | **Data Compression** | **TurboQuant (7x smaller vectors)** | ❌ Standard F32 Vectors | ❌ Standard Vectors | ❌ No Vectors |
 | **Observability** | **OTel Traces + Built-in PWA UI** | Cloud Dashboard | Cloud Dashboard | ❌ None |
 | **Maintenance** | **Autonomous Background Scheduler** | Manual/API driven | Automated (Cloud) | ❌ Manual |
@@ -1225,8 +1229,16 @@ Prism has evolved from smart session logging into a **cognitive memory architect
 | **v7.8** | Multi-Hop Causal Reasoning — spreading activation traverses `caused_by`/`led_to` edges with damped fan effect (`1/ln(fan+e)`) and lateral inhibition | ACT-R spreading activation (Anderson), Collins & Loftus (1975) | ✅ Shipped |
 | **v7.8** | Uncertainty-Aware Rejection Gate — dual-signal (similarity floor + gap distance) safety layer prevents hallucination from low-confidence retrievals | Metacognition research, uncertainty quantification | ✅ Shipped |
 | **v7.8** | Dynamic Fast Weight Decay — `is_rollup` semantic nodes decay 50% slower (`ageModifier = 0.5`) than episodic entries, creating Long-Term Context anchors | ACT-R base-level activation with differential decay rates | ✅ Shipped |
-| **v7.x** | Affect-Tagged Memory — sentiment shapes what gets recalled | Affect-modulated retrieval (neuroscience) | 🔭 Horizon |
-| **v8+** | Zero-Search Retrieval — no index, no ANN, just ask the vector | Holographic Reduced Representations | 🔭 Horizon |
+| **v9.0** | Affect-Tagged Memory — valence-scored retrieval where `\|valence\|` boosts ranking; UX warnings surface historically negative topics | Affect-modulated retrieval (neuroscience), somatic marker hypothesis | ✅ Shipped |
+| **v9.0** | Surprisal Gate — vector-based novelty pricing: high-surprisal saves cost 0.5× tokens, low-surprisal 2.0×; forces LLM data compression | Information-theoretic surprisal (Shannon), predictive coding | ✅ Shipped |
+| **v9.0** | Cognitive Budget — per-project token economy with passive UBI recovery (+100 tokens/hr); agents that over-save enter Cognitive Debt | Resource-bounded rationality (Simon, 1955) | ✅ Shipped |
+| **v9.1** | Task Router v2 — 6-signal weighted heuristic engine routing tasks between cloud host and local LLM based on file-type complexity, scope, and multi-step detection | Heuristic classification, cognitive load theory | ✅ Shipped |
+| **v9.2** | Cross-Backend Reconciliation — automatic Supabase → SQLite sync with idempotent dedup and 5s timeout | Eventual consistency, crdt-style reconciliation | ✅ Shipped |
+| **v9.2** | Split-Brain Drift Detection — dual-backend version comparison with prominent divergence warnings at load time | Byzantine fault detection, split-brain resolution | ✅ Shipped |
+| **v9.2** | TurboQuant QJL Validation — zero R@5 delta between P50 and P95 residual vectors (d=128, N=2K); CV=0.038 at d=768 proves no long tail | QJL estimator (ICLR 2026), Householder orthogonal rotation | ✅ Shipped |
+| **v9.2** | Typed Security Errors — `PrototypePollutionError` with `offendingKey` for forensic logging; null-byte path injection guard in SafetyController | Defense-in-depth (NIST), C-string truncation attack mitigation | ✅ Shipped |
+| **v9.3** | ResidualNorm Tiebreaker — within-ε candidates ranked by compression fidelity (`PRISM_TURBOQUANT_TIEBREAKER_EPSILON`); +2pp R@1, +1pp R@5 at ε=0.005 | Quantization confidence scoring, compression-aware retrieval | ✅ Shipped |
+| **v10+** | Zero-Search Retrieval — no index, no ANN, just ask the vector | Holographic Reduced Representations | 🔭 Horizon |
 
 > Informed by Anderson's ACT-R (Adaptive Control of Thought—Rational), Collins & Loftus spreading activation networks (1975), Kanerva's SDM (1988), Hebb's learning rule, and LeCun's "Why AI Systems Don't Learn" (Dupoux, LeCun, Malik).
 
@@ -1258,7 +1270,7 @@ Prism MCP is open-source and free for individual developers. For teams and enter
 
 ## 📦 Milestones & Roadmap
 
-> **Current: v9.2.4** — Cross-Backend Reconciliation ([CHANGELOG](CHANGELOG.md))
+> **Current: v9.3.0** — TurboQuant ResidualNorm Tiebreaker ([CHANGELOG](CHANGELOG.md))
 
 | Release | Headline |
 |---------|----------|

package/dist/config.js

@@ -268,3 +268,17 @@ export const PRISM_DARK_FACTORY_ENABLED = process.env.PRISM_DARK_FACTORY_ENABLED
 export const PRISM_DARK_FACTORY_POLL_MS = parseInt(process.env.PRISM_DARK_FACTORY_POLL_MS || "30000", 10);
 /** Default max wall-clock time per pipeline (ms). Default: 15 minutes. */
 export const PRISM_DARK_FACTORY_MAX_RUNTIME_MS = parseInt(process.env.PRISM_DARK_FACTORY_MAX_RUNTIME_MS || "900000", 10);
+// ─── v9.3: TurboQuant ResidualNorm Tiebreaker ─────────────────
+// When two compressed cosine scores are within ε of each other,
+// prefer the candidate with lower residualNorm (its compressed
+// representation captured more signal energy, making its score
+// more trustworthy). Empirically validated: ε=0.005 gives +2pp
+// R@1, +1pp R@5 on random d=128 vectors. Set to 0 to disable.
+//
+// Only affects Tier-2 TurboQuant JS-side search (both SQLite and
+// Supabase backends). Tier-1 native vector search is unaffected.
+/** Tiebreaker threshold for TurboQuant Tier-2 ranking. 0 = disabled (default). */
+const rawTiebreakerEpsilon = parseFloat(process.env.PRISM_TURBOQUANT_TIEBREAKER_EPSILON || "0");
+export const PRISM_TURBOQUANT_TIEBREAKER_EPSILON = Number.isFinite(rawTiebreakerEpsilon) && rawTiebreakerEpsilon >= 0
+    ? rawTiebreakerEpsilon
+    : 0;

package/dist/storage/sqlite.js

@@ -1594,6 +1594,9 @@ export class SqliteStorage {
         `;
                 const fallbackResult = await this.db.execute({ sql: fallbackSql, args: fallbackArgs });
                 // Score each entry using asymmetric cosine similarity
+                // Track residualNorm for optional tiebreaker (v9.3)
+                const { PRISM_TURBOQUANT_TIEBREAKER_EPSILON } = await import("../config.js");
+                const eps = PRISM_TURBOQUANT_TIEBREAKER_EPSILON;
                 const scored = [];
                 for (const row of fallbackResult.rows) {
                     try {
@@ -1613,6 +1616,7 @@ export class SqliteStorage {
                                 is_rollup: Boolean(row.is_rollup),
                                 importance: row.importance ?? 0,
                                 last_accessed_at: row.last_accessed_at || null,
+                                _residualNorm: eps > 0 ? compressed.residualNorm : undefined,
                             });
                         }
                     }
@@ -1620,9 +1624,20 @@ export class SqliteStorage {
                         // Skip entries with corrupt compressed data
                     }
                 }
-                // Sort by similarity descending and limit
-                scored.sort((a, b) => b.similarity - a.similarity);
+                // Sort by similarity descending, with optional residualNorm tiebreaker
+                // When ε > 0: candidates within ε of each other are ranked by lower
+                // residualNorm (its compressed representation is more trustworthy).
+                scored.sort((a, b) => {
+                    const diff = b.similarity - a.similarity;
+                    if (eps > 0 && Math.abs(diff) < eps && a._residualNorm != null && b._residualNorm != null) {
+                        return a._residualNorm - b._residualNorm;
+                    }
+                    return diff;
+                });
                 const baseResults = scored.slice(0, params.limit);
+                // Strip internal tiebreaker field before returning
+                for (const r of baseResults)
+                    delete r._residualNorm;
                 debugLog(`[SqliteStorage] Tier-2 TurboQuant fallback: scored ${fallbackResult.rows.length} entries, ` +
                     `${scored.length} above threshold`);
                 if (params.activation?.enabled) {

package/dist/storage/supabase.js

@@ -280,6 +280,9 @@ export class SupabaseStorage {
                     queryParams.role = `eq.${params.role}`;
                 const rows = await supabaseGet("session_ledger", queryParams);
                 const scored = [];
+                // v9.3: Import tiebreaker config for optional residualNorm ranking
+                const { PRISM_TURBOQUANT_TIEBREAKER_EPSILON } = await import("../config.js");
+                const eps = PRISM_TURBOQUANT_TIEBREAKER_EPSILON;
                 for (const row of (Array.isArray(rows) ? rows : [])) {
                     try {
                         const compressedBase64 = row.embedding_compressed;
@@ -295,6 +298,7 @@ export class SupabaseStorage {
                                 session_date: (row.session_date || row.created_at),
                                 decisions: Array.isArray(row.decisions) ? row.decisions : [],
                                 files_changed: Array.isArray(row.files_changed) ? row.files_changed : [],
+                                _residualNorm: eps > 0 ? compressed.residualNorm : undefined,
                             });
                         }
                     }
@@ -302,10 +306,21 @@ export class SupabaseStorage {
                         // Skip entries with corrupt compressed data
                     }
                 }
-                scored.sort((a, b) => b.similarity - a.similarity);
+                // Sort by similarity descending, with optional residualNorm tiebreaker
+                scored.sort((a, b) => {
+                    const diff = b.similarity - a.similarity;
+                    if (eps > 0 && Math.abs(diff) < eps && a._residualNorm != null && b._residualNorm != null) {
+                        return a._residualNorm - b._residualNorm;
+                    }
+                    return diff;
+                });
                 debugLog(`[SupabaseStorage] Tier-2 TurboQuant fallback: scored ${rows.length} entries, ` +
                     `${scored.length} above threshold`);
-                return scored.slice(0, params.limit);
+                const results = scored.slice(0, params.limit);
+                // Strip internal tiebreaker field before returning
+                for (const r of results)
+                    delete r._residualNorm;
+                return results;
             }
             catch (tier2Err) {
                 // Both tiers failed — return empty; caller falls through to FTS5

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "prism-mcp-server",
-  "version": "9.2.7",
+  "version": "9.3.0",
   "mcpName": "io.github.dcostenco/prism-mcp",
   "description": "The Mind Palace for AI Agents — a true Cognitive Architecture with Hebbian learning (episodic→semantic consolidation), ACT-R spreading activation (multi-hop causal reasoning), uncertainty-aware rejection gates (agents that know when they don't know), adversarial evaluation (anti-sycophancy), fail-closed Dark Factory pipelines, persistent memory (SQLite/Supabase), multi-agent Hivemind, time travel & visual dashboard. Zero-config local mode.",
   "module": "index.ts",