pi-mega-compact 0.6.2 → 0.6.3

package/README.md

@@ -6,28 +6,39 @@ sessions into a **local SQLite store** and offers **deduped inline recall** —
 running **locally inside the extension**, with **no remote MCP server** and
 **zero network calls at runtime** (PREVENT-PI-004).
 
-> **Current version:** `v0.6.1` — storage backend is **`node:sqlite`**
+> **Current version:** `v0.6.2` — storage backend is **`node:sqlite`**
 > (`DatabaseSync`, a Node ≥22.13 built-in), replacing the old `better-sqlite3`
 > native addon and the per-session gzipped JSON checkpoint files. **Zero native
 > build step, fully local, zero network at runtime.** Legacy
 > `.checkpoints.json.gz` snapshots are retained as disaster-recovery fallbacks
-> and auto-imported on first run. Cross-repo recall, durable memory, and a
-> localhost dashboard round out the continuity story.
+> and auto-imported on first run. The S24 line ties auto-compact, the tier
+> label, trim depth, and durable-memory review to one **unified pressure
+> signal**, adds a **cross-repo memory-RAG index**, and relieves context
+> **during team runs** (not just at the end).
 
 ---
 
 ## What this is (the 30-second version)
 
-pi's context window is finite. When a session gets long, pi-mega-compact:
-
-1. **Watches** context usage and, past a threshold, **compacts** the older part of
-   the conversation into a short structured summary + key facts ("a checkpoint").
-2. **Stores** each checkpoint in a **local vector database** (SQLite) with an
-   embedding, so similar regions can be found later — and so **duplicate
-   work is never stored twice**.
-3. **Recalls** the right checkpoints automatically when you resume a session or
-   invoke a recall command, re-injecting only what's relevant (deduped against
-   what's already in view).
+pi's context window is finite. When a session gets long — especially a team run
+with sub-agents — pi-mega-compact keeps it going without overflowing:
+
+1. **Watches one signal.** A single live `pressure = currentTokens / thresholdTokens`
+   drives everything — the tier label, how aggressively the live trim drops
+   context, and how often durable memory is reviewed. As context fills, the whole
+   system reacts together; as it's relieved, it backs off.
+2. **Compacts in two layers.** On every LLM call it returns a **live, compacted
+   view** (the model sees a summary + recent anchor, non-destructively). And it
+   persists a durable **checkpoint** — and, at each agent settle during a team
+   run, fires pi's **native durable trim** so the on-disk transcript is actually
+   truncated (context relieves mid-run, and resume reloads the trimmed transcript
+   instead of a 150k window).
+3. **Stores** each checkpoint in a **local vector database** (SQLite) with an
+   embedding, so similar regions are found later and **duplicate work is never
+   stored twice**.
+4. **Recalls** the right context automatically — same-repo checkpoints on resume,
+   plus **cross-repo memory-RAG**: decisions you saved in one repo are inlined as
+   context when you start a session in another.
 
 Everything lives on **your disk**. No telemetry, no API, no MCP server, no cloud.
 The only optional network surface is a **user-triggered localhost dashboard** you
@@ -35,11 +46,13 @@ open yourself.
 
 ### Why "mega"?
 
-The compaction pipeline is a **Trident** — three deterministic stages that run
-over your conversation before anything is persisted. The checkpoint it produces
-is small (a summary + key decisions + next steps + files touched), so the same
-session that would otherwise overflow its window keeps going on a fraction of the
-tokens.
+The compaction pipeline is a **Trident** — three deterministic stages
+(supersede → collapse → cluster) that run over your conversation before anything
+is persisted. The checkpoint it produces is small (a summary + key decisions +
+next steps + files touched), so the same session that would otherwise overflow
+its window keeps going on a fraction of the tokens. On top of the Trident, a
+single pressure signal orchestrates the live trim, the durable trim, and memory
+review as one coherent system rather than four independent triggers.
 
 ---
 
@@ -52,9 +65,12 @@ Layer 3  Cluster (vectorize)  local vector index → semantic dedup + recall
 Layer 2  Collapse (summarize)  summarizeMessages() heuristic + agent summary on /mega-compact
 Layer 1  Supersede (prune)     drop obsolete file-reads / superseded turns (zero cost)
 ─────────────────────────────────────────────────────────────────────────
-Trigger   context/turn_end → % gate → auto_compact_check → fire
+Trigger   context → token fast-gate → autoCompactCheck → live trim (per call)
+Durable    agent_end (idle + over threshold) → ctx.compact() → session_before_compact
+          supplies the summary; pi truncates the transcript (relieves context)
+Live      context handler returns { messages:[summary, …recent] } — model sees a
+          compacted window every LLM call; the on-disk transcript is untouched
 Marker    insert compact-marker; dedupe so repeated triggers cost ~0 tokens
-Cancel    session_before_compact → { cancel:true } once persisted (no double-compact)
 ```
 
 **One store, three ways to read it back — one dedup engine:**
@@ -168,7 +184,7 @@ building.
 pi-mega-compact uses a dual local backend — **zero network, no native build step**:
 
 - **`node:sqlite`** (`DatabaseSync`, Node ≥22.13 built-in) — the synchronous source of truth for checkpoints, session state, and the dedup index. No dependency, no install script, survives pi's `install-scripts` block.
-- **PGlite + `@electric-sql/pglite-pgvector`** (WASM Postgres + HNSW `vector_cosine_ops`) — an optional, best-effort async vector index for **cross-repo recall** at `~/.pi/mega-compact-vector`. The sync store stays authoritative; the index degrades to the sync per-session scan on any failure.
+- **PGlite + `@electric-sql/pglite-pgvector`** (WASM Postgres + HNSW `vector_cosine_ops`) — an optional, best-effort async vector index for **cross-repo recall** at `~/.pi/mega-compact-vector`. It holds both checkpoint embeddings and durable-memory embeddings, so decisions saved in one repo are findable from another. The sync store stays authoritative; the index degrades to the sync per-session scan on any failure.
 
 Kill-switch: `MEGACOMPACT_PGLITE_DISABLED=1` fully disables the PGlite index (falls back to sync scan). Requires Node ≥22.13 (`engines.node`).
 
@@ -178,7 +194,7 @@ On resume, recall augments from other repos' checkpoints when this repo's store
 
 ### Memory
 
-pi-mega-compact auto-reviews the conversation every 10 turns and writes durable `decision`/`fact`/`preference` memories to SQLite (local, hallucination-guarded). Relevant memories are injected as RAG context on recall (capped, deduped). Manual: `/mega-memory save|list|forget`.
+pi-mega-compact auto-reviews the conversation every 10 turns (the cadence shortens as pressure climbs) and writes durable `decision`/`fact`/`preference` memories to SQLite (local, hallucination-guarded). Relevant memories are injected as RAG context on recall (capped, deduped). **Cross-repo memory-RAG (S24):** every memory write is mirrored into the PGlite/HNSW index, so when same-repo recall is thin the system augments with the nearest memories from *other* repos (stricter `MEGACOMPACT_CROSSREPO_COSINE` floor, deduped against what's already in view). Manual: `/mega-memory save|list|forget` (or `/m`).
 
 ### Uninstall
 
@@ -352,7 +368,7 @@ The extension entry adapts between the engine and pi's runtime types.
 
 ```bash
 npm run build      # tsc
-npm test           # build + node --test on dist/**/*.test.js (346 tests)
+npm test           # build + node --test on dist/**/*.test.js (353 tests)
 npm run lint       # tsc --noEmit + guardrails-scan
 npm run guardrails # regression_check + guardrails-scan
 ```

package/dist/extensions/conflict-scan.test.js

@@ -0,0 +1,115 @@
+/**
+ * conflict-scan.test.ts — unit tests for the extension-conflict scanner.
+ *
+ * Fixture trees are written under a temp dir and scanned via
+ * MEGACOMPACT_EXT_SCAN_DIR (which makes collectScanRoots() return that
+ * single root). This covers the S24 follow-up fix:
+ *
+ *   1. node_modules-style code extensions (package.json + pi.extensions) are
+ *      still detected by source-marker grep (regression).
+ *   2. USER-LEVEL extensions installed outside npm (e.g. pi-hermes-memory)
+ *      now get scanned too — previously only `node_modules` was walked, so a
+ *      data-only memory store (MEMORY.md + sessions.db, no package.json)
+ *      was never flagged (the 5000-char file-buffer error slipped through).
+ *   3. The data-only memory-store signature is detected even with no source.
+ *   4. pi-mega-compact (selfName) is always skipped.
+ */
+import { test, after } from "node:test";
+import assert from "node:assert/strict";
+import { mkdtempSync, rmSync, mkdirSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { detectConflicts, collectScanRoots } from "./conflict-scan.js";
+const base = mkdtempSync(join(tmpdir(), "mc-scan-"));
+let n = 0;
+/** Make a fixture root containing one or more fake extensions, return its path. */
+function fixture(build) {
+    const root = join(base, `case-${n++}`);
+    mkdirSync(root, { recursive: true });
+    build(root);
+    return root;
+}
+after(() => {
+    rmSync(base, { recursive: true, force: true });
+});
+test("scans a user-level, data-only memory store (no package.json)", () => {
+    const root = fixture((r) => {
+        const ext = join(r, "pi-hermes-memory");
+        mkdirSync(ext, { recursive: true });
+        // No package.json, no source — just pi's memory-store signature.
+        writeFileSync(join(ext, "MEMORY.md"), "# memory\n");
+        writeFileSync(join(ext, "sessions.db"), "");
+    });
+    process.env.MEGACOMPACT_EXT_SCAN_DIR = root;
+    try {
+        const { conflicts } = detectConflicts();
+        assert.ok(conflicts.length >= 1, "expected a memory conflict");
+        const hit = conflicts.find((c) => c.kind === "memory");
+        assert.ok(hit, "expected a memory-kind conflict");
+        assert.equal(hit.severity, "high");
+        assert.ok(hit.evidence.includes("MEMORY.md") ||
+            hit.evidence.includes("sessions.db"), "evidence should name the on-disk memory signature");
+    }
+    finally {
+        delete process.env.MEGACOMPACT_EXT_SCAN_DIR;
+    }
+});
+test("still detects a code extension by source marker (regression)", () => {
+    const root = fixture((r) => {
+        // A code extension is a DIRECT child of the scan root (mirrors the
+        // node_modules layout: packages live one level under the root).
+        const ext = join(r, "some-memory-ext");
+        mkdirSync(ext, { recursive: true });
+        writeFileSync(join(ext, "package.json"), JSON.stringify({ name: "some-memory-ext", pi: { extensions: ["x.ts"] } }));
+        writeFileSync(join(ext, "index.ts"), "export const MEMORY_TOOL = true;");
+    });
+    process.env.MEGACOMPACT_EXT_SCAN_DIR = root;
+    try {
+        const { conflicts } = detectConflicts();
+        const hit = conflicts.find((c) => c.package === "some-memory-ext");
+        assert.ok(hit, "expected some-memory-ext to be flagged");
+        assert.equal(hit.kind, "memory");
+    }
+    finally {
+        delete process.env.MEGACOMPACT_EXT_SCAN_DIR;
+    }
+});
+test("skips pi-mega-compact (selfName) and non-extension dirs", () => {
+    const root = fixture((r) => {
+        // selfName dir with a memory signature — must be ignored.
+        const me = join(r, "node_modules", "pi-mega-compact");
+        mkdirSync(me, { recursive: true });
+        writeFileSync(join(me, "sessions.db"), "");
+        // unrelated dir with no pi.extensions and no memory signature.
+        mkdirSync(join(r, "node_modules", "totally-fine"), { recursive: true });
+    });
+    process.env.MEGACOMPACT_EXT_SCAN_DIR = root;
+    try {
+        const { scanned, conflicts } = detectConflicts();
+        assert.equal(conflicts.length, 0, "no conflicts expected");
+        assert.ok(!scanned.some((s) => s.includes("pi-mega-compact")), "selfName should not appear in scanned");
+    }
+    finally {
+        delete process.env.MEGACOMPACT_EXT_SCAN_DIR;
+    }
+});
+test("collectScanRoots honors MEGACOMPACT_EXT_SCAN_DIR override", () => {
+    const root = fixture(() => { });
+    process.env.MEGACOMPACT_EXT_SCAN_DIR = root;
+    try {
+        const roots = collectScanRoots();
+        assert.deepEqual(roots, [root], "override replaces the whole root list");
+    }
+    finally {
+        delete process.env.MEGACOMPACT_EXT_SCAN_DIR;
+    }
+});
+test("collectScanRoots falls back to node_modules + user dir when no override", () => {
+    delete process.env.MEGACOMPACT_EXT_SCAN_DIR;
+    delete process.env.MEGACOMPACT_EXT_USER_DIR;
+    // No override set and this test file lives under extensions/, so node_modules
+    // resolution walks up from here; the user dir (~/.pi/agent) may or may
+    // not exist in CI. We only assert the call returns a non-throwing array.
+    const roots = collectScanRoots();
+    assert.ok(Array.isArray(roots), "collectScanRoots must return an array");
+});

package/dist/src/store/memoryIndex.js

@@ -22,17 +22,15 @@
 import { homedir } from "node:os";
 import { join } from "node:path";
 import { mkdirSync, rmSync, existsSync } from "node:fs";
-// PGlite + pgvector are script-free WASM (no native build) → survive pi's
-// install-script block. Imported lazily so a missing/broken package degrades
-// gracefully instead of crashing module load.
-import { PGlite } from "@electric-sql/pglite";
-import { vector } from "@electric-sql/pglite-pgvector";
 /** Vector dimension produced by the default TrigramEmbedder (src/embedder.ts). */
 export const MEMORY_INDEX_DIM = 512;
 let db;
 let initPromise;
 let disabled = false;
 let warned = false;
+/** Lazily-loaded PGlite module + pgvector extension (see loadPgLite). */
+let pgliteMod;
+let pgliteLoadFailed = false;
 function indexDir() {
     const override = process.env.MEGACOMPACT_INDEX_DIR;
     if (override && override.trim() !== "")
@@ -77,17 +75,44 @@ export function initMemoryIndex() {
     initPromise = openPgLite(/* retryOnCorrupt */ true);
     return initPromise;
 }
+/**
+ * Lazily load the PGlite module + pgvector extension via dynamic import. Caches
+ * success and permanent failure. Returns undefined (once, then forever) when the
+ * package is missing/broken so callers fall back to the same-repo scan. Never throws.
+ */
+async function loadPgLite() {
+    if (pgliteMod)
+        return pgliteMod;
+    if (pgliteLoadFailed)
+        return undefined;
+    try {
+        const [pglitePkg, pgvectorPkg] = await Promise.all([
+            import("@electric-sql/pglite"),
+            import("@electric-sql/pglite-pgvector"),
+        ]);
+        pgliteMod = { PGlite: pglitePkg.PGlite, vector: pgvectorPkg.vector };
+        return pgliteMod;
+    }
+    catch (err) {
+        pgliteLoadFailed = true;
+        logWarn(`package unavailable: ${err instanceof Error ? err.message : String(err)}`);
+        return undefined;
+    }
+}
 /**
  * Open + schema-init PGlite. When `retryOnCorrupt` is true, a WASM-level abort
  * (typically from a corrupted/torn data dir) triggers a delete + one retry.
  */
 async function openPgLite(retryOnCorrupt) {
     try {
+        const mod = await loadPgLite();
+        if (!mod)
+            return undefined;
         const dir = indexDir();
         mkdirSync(dir, { recursive: true });
-        const pg = await new PGlite({
+        const pg = await new mod.PGlite({
             dataDir: dir,
-            extensions: { vector },
+            extensions: { vector: mod.vector },
         });
         await pg.exec("CREATE EXTENSION IF NOT EXISTS vector;");
         await pg.exec(`

package/dist/src/store/vectorIndex.js

@@ -18,17 +18,15 @@
 import { homedir } from "node:os";
 import { join } from "node:path";
 import { mkdirSync, rmSync, existsSync } from "node:fs";
-// PGlite + pgvector are script-free WASM (no native build) → survive pi's
-// install-script block. Imported lazily so a missing/broken package degrades
-// gracefully instead of crashing module load.
-import { PGlite } from "@electric-sql/pglite";
-import { vector } from "@electric-sql/pglite-pgvector";
 /** Vector dimension produced by the default TrigramEmbedder (src/embedder.ts). */
 export const EMBEDDING_DIM = 512;
 let db;
 let initPromise;
 let disabled = false;
 let warned = false;
+/** Lazily-loaded PGlite module + pgvector extension (see loadPgLite). */
+let pgliteMod;
+let pgliteLoadFailed = false;
 function indexDir() {
     const override = process.env.MEGACOMPACT_VECTOR_INDEX_DIR;
     if (override && override.trim() !== "")
@@ -73,6 +71,30 @@ export function initVectorIndex() {
     initPromise = openPgLite(/* retryOnCorrupt */ true);
     return initPromise;
 }
+/**
+ * Lazily load the PGlite module + pgvector extension via dynamic import. Caches
+ * success and permanent failure. Returns undefined (once, then forever) when the
+ * package is missing/broken so callers fall back to the sync scan. Never throws.
+ */
+async function loadPgLite() {
+    if (pgliteMod)
+        return pgliteMod;
+    if (pgliteLoadFailed)
+        return undefined;
+    try {
+        const [pglitePkg, pgvectorPkg] = await Promise.all([
+            import("@electric-sql/pglite"),
+            import("@electric-sql/pglite-pgvector"),
+        ]);
+        pgliteMod = { PGlite: pglitePkg.PGlite, vector: pgvectorPkg.vector };
+        return pgliteMod;
+    }
+    catch (err) {
+        pgliteLoadFailed = true;
+        logWarn(`package unavailable: ${err instanceof Error ? err.message : String(err)}`);
+        return undefined;
+    }
+}
 /**
  * Open + schema-init PGlite. When `retryOnCorrupt` is true, a WASM-level
  * abort (typically from a corrupted/torn data dir) triggers a delete + one
@@ -80,11 +102,14 @@ export function initVectorIndex() {
  */
 async function openPgLite(retryOnCorrupt) {
     try {
+        const mod = await loadPgLite();
+        if (!mod)
+            return undefined;
         const dir = indexDir();
         mkdirSync(dir, { recursive: true });
-        const pg = await new PGlite({
+        const pg = await new mod.PGlite({
             dataDir: dir,
-            extensions: { vector },
+            extensions: { vector: mod.vector },
         });
         await pg.exec("CREATE EXTENSION IF NOT EXISTS vector;");
         await pg.exec(`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-mega-compact",
-  "version": "0.6.2",
+  "version": "0.6.3",
   "description": "Layered, local, vector-backed context compressor for pi — supersede/collapse/cluster compaction with deduped inline recall.",
   "type": "module",
   "license": "BSD-2-Clause",

package/src/store/memoryIndex.ts

@@ -25,10 +25,12 @@ import { join } from "node:path";
 import { mkdirSync, rmSync, existsSync } from "node:fs";
 
 // PGlite + pgvector are script-free WASM (no native build) → survive pi's
-// install-script block. Imported lazily so a missing/broken package degrades
-// gracefully instead of crashing module load.
-import { PGlite, type PGlite as PGliteInstance } from "@electric-sql/pglite";
-import { vector } from "@electric-sql/pglite-pgvector";
+// install-script block. The VALUE import is LAZY (dynamic import inside
+// openPgLite) so a missing/broken package degrades to the same-repo scan instead
+// of crashing module load. A static top-level `import { PGlite }` would throw
+// "Cannot find module" at pi startup and take down the whole extension. The
+// `import type` below is erased at compile time and emits NO runtime load.
+import type { PGlite as PGliteInstance, Extension } from "@electric-sql/pglite";
 
 /** Vector dimension produced by the default TrigramEmbedder (src/embedder.ts). */
 export const MEMORY_INDEX_DIM = 512;
@@ -47,6 +49,12 @@ let db: PGliteInstance | undefined;
 let initPromise: Promise<PGliteInstance | undefined> | undefined;
 let disabled = false;
 let warned = false;
+/** Lazily-loaded PGlite module + pgvector extension (see loadPgLite). */
+let pgliteMod: {
+  PGlite: typeof import("@electric-sql/pglite")["PGlite"];
+  vector: Extension;
+} | undefined;
+let pgliteLoadFailed = false;
 
 function indexDir(): string {
   const override = process.env.MEGACOMPACT_INDEX_DIR;
@@ -91,6 +99,30 @@ export function initMemoryIndex(): Promise<PGliteInstance | undefined> {
   return initPromise;
 }
 
+/**
+ * Lazily load the PGlite module + pgvector extension via dynamic import. Caches
+ * success and permanent failure. Returns undefined (once, then forever) when the
+ * package is missing/broken so callers fall back to the same-repo scan. Never throws.
+ */
+async function loadPgLite(): Promise<
+  { PGlite: typeof import("@electric-sql/pglite")["PGlite"]; vector: Extension } | undefined
+> {
+  if (pgliteMod) return pgliteMod;
+  if (pgliteLoadFailed) return undefined;
+  try {
+    const [pglitePkg, pgvectorPkg] = await Promise.all([
+      import("@electric-sql/pglite"),
+      import("@electric-sql/pglite-pgvector"),
+    ]);
+    pgliteMod = { PGlite: pglitePkg.PGlite, vector: pgvectorPkg.vector };
+    return pgliteMod;
+  } catch (err) {
+    pgliteLoadFailed = true;
+    logWarn(`package unavailable: ${err instanceof Error ? err.message : String(err)}`);
+    return undefined;
+  }
+}
+
 /**
  * Open + schema-init PGlite. When `retryOnCorrupt` is true, a WASM-level abort
  * (typically from a corrupted/torn data dir) triggers a delete + one retry.
@@ -99,11 +131,13 @@ async function openPgLite(
   retryOnCorrupt: boolean,
 ): Promise<PGliteInstance | undefined> {
   try {
+    const mod = await loadPgLite();
+    if (!mod) return undefined;
     const dir = indexDir();
     mkdirSync(dir, { recursive: true });
-    const pg = await new PGlite({
+    const pg = await new mod.PGlite({
       dataDir: dir,
-      extensions: { vector },
+      extensions: { vector: mod.vector },
     });
     await pg.exec("CREATE EXTENSION IF NOT EXISTS vector;");
     await pg.exec(`

package/src/store/vectorIndex.ts

@@ -21,10 +21,12 @@ import { join } from "node:path";
 import { mkdirSync, rmSync, existsSync } from "node:fs";
 
 // PGlite + pgvector are script-free WASM (no native build) → survive pi's
-// install-script block. Imported lazily so a missing/broken package degrades
-// gracefully instead of crashing module load.
-import { PGlite, type PGlite as PGliteInstance } from "@electric-sql/pglite";
-import { vector } from "@electric-sql/pglite-pgvector";
+// install-script block. The VALUE import is LAZY (dynamic import inside
+// openPgLite) so a missing/broken package degrades to the sync scan instead of
+// crashing module load. A static top-level `import { PGlite }` would throw
+// "Cannot find module" at pi startup and take down the whole extension. The
+// `import type` below is erased at compile time and emits NO runtime load.
+import type { PGlite as PGliteInstance, Extension } from "@electric-sql/pglite";
 
 /** Vector dimension produced by the default TrigramEmbedder (src/embedder.ts). */
 export const EMBEDDING_DIM = 512;
@@ -42,6 +44,12 @@ let db: PGliteInstance | undefined;
 let initPromise: Promise<PGliteInstance | undefined> | undefined;
 let disabled = false;
 let warned = false;
+/** Lazily-loaded PGlite module + pgvector extension (see loadPgLite). */
+let pgliteMod: {
+  PGlite: typeof import("@electric-sql/pglite")["PGlite"];
+  vector: Extension;
+} | undefined;
+let pgliteLoadFailed = false;
 
 function indexDir(): string {
   const override = process.env.MEGACOMPACT_VECTOR_INDEX_DIR;
@@ -86,6 +94,30 @@ export function initVectorIndex(): Promise<PGliteInstance | undefined> {
   return initPromise;
 }
 
+/**
+ * Lazily load the PGlite module + pgvector extension via dynamic import. Caches
+ * success and permanent failure. Returns undefined (once, then forever) when the
+ * package is missing/broken so callers fall back to the sync scan. Never throws.
+ */
+async function loadPgLite(): Promise<
+  { PGlite: typeof import("@electric-sql/pglite")["PGlite"]; vector: Extension } | undefined
+> {
+  if (pgliteMod) return pgliteMod;
+  if (pgliteLoadFailed) return undefined;
+  try {
+    const [pglitePkg, pgvectorPkg] = await Promise.all([
+      import("@electric-sql/pglite"),
+      import("@electric-sql/pglite-pgvector"),
+    ]);
+    pgliteMod = { PGlite: pglitePkg.PGlite, vector: pgvectorPkg.vector };
+    return pgliteMod;
+  } catch (err) {
+    pgliteLoadFailed = true;
+    logWarn(`package unavailable: ${err instanceof Error ? err.message : String(err)}`);
+    return undefined;
+  }
+}
+
 /**
  * Open + schema-init PGlite. When `retryOnCorrupt` is true, a WASM-level
  * abort (typically from a corrupted/torn data dir) triggers a delete + one
@@ -95,11 +127,13 @@ async function openPgLite(
   retryOnCorrupt: boolean,
 ): Promise<PGliteInstance | undefined> {
   try {
+    const mod = await loadPgLite();
+    if (!mod) return undefined;
     const dir = indexDir();
     mkdirSync(dir, { recursive: true });
-    const pg = await new PGlite({
+    const pg = await new mod.PGlite({
       dataDir: dir,
-      extensions: { vector },
+      extensions: { vector: mod.vector },
     });
     await pg.exec("CREATE EXTENSION IF NOT EXISTS vector;");
     await pg.exec(`

package/dist/extensions/openclaw-mega-compact.js

@@ -1,291 +0,0 @@
-/**
- * openclaw-mega-compact — OpenClaw plugin adapter for the pi-mega-compact engine.
- *
- * Wires the pi-agnostic Trident engine (src/) into OpenClaw's plugin lifecycle:
- *  - Registers a CompactionProvider that replaces the built-in summarizeInStages.
- *  - Exposes `mega_status` and `mega_recall` tools for on-demand inspection.
- *  - Hooks into `before_compaction` / `after_compaction` for diagnostics.
- *
- * Design constraints:
- *  - NO imports from `@earendil-works/pi-coding-agent` or pi-agent-core.
- *  - The engine core (src/) is pi-agnostic; this file is the sole OpenClaw boundary.
- *  - No network at runtime — everything is local (stores + extractive summarizer).
- */
-import { definePluginEntry } from "openclaw/plugin-sdk/plugin-entry";
-import { compactSession, setDefaultStore, } from "../src/engine.js";
-import { recallAndInline } from "../src/recall.js";
-import { VectorStore } from "../src/vectorStore.js";
-// ---------------------------------------------------------------------------
-// Constants
-// ---------------------------------------------------------------------------
-const PLUGIN_ID = "mega-compact";
-const PLUGIN_LABEL = "Mega Compact (Trident)";
-/** Default state directory for vector store persistence. */
-const STATE_DIR = process.env.MEGA_COMPACT_STATE_DIR ?? undefined;
-/** Minimum messages before we bother compacting. */
-const MIN_MESSAGES_FOR_COMPACT = 6;
-// ---------------------------------------------------------------------------
-// Message conversion — OpenClaw unknown[] → EngineMessage[]
-// ---------------------------------------------------------------------------
-/**
- * Best-effort conversion from OpenClaw's opaque message array to our
- * EngineMessage shape. OpenClaw messages are typed as `unknown[]` so we
- * handle whatever shape comes through gracefully.
- */
-function toEngineMessages(messages) {
-    return messages.map((msg) => {
-        if (!msg || typeof msg !== "object") {
-            // Primitive fallback — treat as custom text.
-            return {
-                role: "custom",
-                text: String(msg ?? ""),
-            };
-        }
-        const m = msg;
-        const role = typeof m.role === "string" ? m.role : "custom";
-        // Normalize role to one of our four engine roles.
-        let engineRole;
-        switch (role) {
-            case "user":
-                engineRole = "user";
-                break;
-            case "assistant":
-                engineRole = "assistant";
-                break;
-            case "tool":
-            case "function":
-                engineRole = "tool";
-                break;
-            default:
-                engineRole = "custom";
-                break;
-        }
-        // Extract text content from common message shapes.
-        const text = typeof m.content === "string"
-            ? m.content
-            : typeof m.text === "string"
-                ? m.text
-                : Array.isArray(m.content)
-                    ? m.content
-                        .filter((part) => part.type === "text" && typeof part.text === "string")
-                        .map((part) => part.text)
-                        .join("\n")
-                    : "";
-        // Preserve tool metadata when present.
-        const toolName = typeof m.name === "string"
-            ? m.name
-            : typeof m.toolName === "string"
-                ? m.toolName
-                : undefined;
-        const input = typeof m.input === "string"
-            ? m.input
-            : typeof m.arguments === "string"
-                ? m.arguments
-                : m.arguments !== undefined
-                    ? JSON.stringify(m.arguments)
-                    : undefined;
-        const output = typeof m.output === "string"
-            ? m.output
-            : engineRole === "tool" && typeof m.content === "string"
-                ? m.content
-                : undefined;
-        return { role: engineRole, text, toolName, input, output };
-    });
-}
-// ---------------------------------------------------------------------------
-// Compaction provider
-// ---------------------------------------------------------------------------
-function createCompactionProvider(store) {
-    return {
-        id: PLUGIN_ID,
-        label: PLUGIN_LABEL,
-        async summarize({ messages, signal, compressionRatio, }) {
-            // Abort check — bail early if the caller cancelled.
-            if (signal?.aborted) {
-                throw new DOMException("Aborted", "AbortError");
-            }
-            const engineMessages = toEngineMessages(messages);
-            // Nothing meaningful to compact.
-            if (engineMessages.length < MIN_MESSAGES_FOR_COMPACT) {
-                return "";
-            }
-            // Map compression ratio → keepFrom boundary.
-            // compressionRatio=0.5 means "compact the oldest 50%".
-            // Default to compacting the oldest half if not specified.
-            const ratio = compressionRatio ?? 0.5;
-            const keepFrom = Math.max(MIN_MESSAGES_FOR_COMPACT, Math.floor(engineMessages.length * (1 - ratio)));
-            // Abort check after conversion (conversion is cheap but check anyway).
-            if (signal?.aborted) {
-                throw new DOMException("Aborted", "AbortError");
-            }
-            const sessionId = `openclaw-${Date.now()}`;
-            const input = {
-                sessionId,
-                messages: engineMessages,
-                keepFrom,
-            };
-            const result = compactSession(input, store);
-            if (result.skipped) {
-                return "";
-            }
-            return result.summary;
-        },
-    };
-}
-// ---------------------------------------------------------------------------
-// Plugin entry
-// ---------------------------------------------------------------------------
-export default definePluginEntry({
-    id: PLUGIN_ID,
-    name: "Mega Compact",
-    description: "Layered, local, vector-backed context compressor (Trident engine) for OpenClaw compaction.",
-    register(api) {
-        const logger = api.logger;
-        // Resolve state directory — prefer plugin config override.
-        const pluginCfg = (api.pluginConfig ?? {});
-        const stateDir = typeof pluginCfg.stateDir === "string" && pluginCfg.stateDir.length > 0
-            ? pluginCfg.stateDir
-            : STATE_DIR;
-        // Initialize vector store.
-        let store;
-        try {
-            store = new VectorStore({ stateDir });
-            setDefaultStore(store);
-            logger.info?.(`${PLUGIN_ID}: vector store initialized (stateDir=${stateDir ?? "default"})`);
-        }
-        catch (err) {
-            logger.error?.(`${PLUGIN_ID}: failed to init vector store:`, err);
-            return; // Hard bail — no point registering if store is broken.
-        }
-        // -----------------------------------------------------------------------
-        // Register compaction provider
-        // -----------------------------------------------------------------------
-        const provider = createCompactionProvider(store);
-        api.registerCompactionProvider(provider);
-        logger.info?.(`${PLUGIN_ID}: registered compaction provider "${provider.id}"`);
-        // -----------------------------------------------------------------------
-        // Hooks — before / after compaction diagnostics
-        // -----------------------------------------------------------------------
-        api.registerHook({
-            event: "before_compaction",
-            handler: async (ctx) => {
-                const msgCount = Array.isArray(ctx?.messages) ? ctx.messages.length : 0;
-                logger.info?.(`${PLUGIN_ID}: before_compaction — ${msgCount} messages in scope`);
-            },
-        });
-        api.registerHook({
-            event: "after_compaction",
-            handler: async (ctx) => {
-                const summaryLen = typeof ctx?.summary === "string" ? ctx.summary.length : 0;
-                logger.info?.(`${PLUGIN_ID}: after_compaction — summary ${summaryLen} chars`);
-            },
-        });
-        // -----------------------------------------------------------------------
-        // Tool: mega_status
-        // -----------------------------------------------------------------------
-        api.registerTool({
-            name: "mega_status",
-            description: "Show the current status of the mega-compact engine: vector store stats, checkpoint count, and recent compaction activity.",
-            parameters: {
-                type: "object",
-                properties: {
-                    sessionId: {
-                        type: "string",
-                        description: "Optional session ID to scope stats to.",
-                    },
-                },
-                additionalProperties: false,
-            },
-            handler: async (args) => {
-                const sessionId = args?.sessionId ?? "global";
-                try {
-                    const stats = store.stats(sessionId);
-                    const parts = [
-                        `**Mega Compact Status**`,
-                        `Session: ${sessionId}`,
-                        `Checkpoints: ${stats.checkpointCount}`,
-                        `Total tokens saved: ${stats.totalTokenEstimate}`,
-                        `Last checkpoint: ${stats.lastCheckpointId ?? "—"}`,
-                        `Injected count: ${stats.injectedCount}`,
-                        `Dedup hit rate: ${(stats.dedupHitRate * 100).toFixed(0)}%`,
-                    ];
-                    if (stats.lastSummary) {
-                        parts.push(`\nLast summary (truncated):\n  ${stats.lastSummary.slice(0, 120).replace(/\n/g, " ")}…`);
-                    }
-                    return { content: [{ type: "text", text: parts.join("\n") }] };
-                }
-                catch (err) {
-                    return {
-                        content: [{ type: "text", text: `Error reading mega-compact status: ${err}` }],
-                        isError: true,
-                    };
-                }
-            },
-        });
-        // -----------------------------------------------------------------------
-        // Tool: mega_recall
-        // -----------------------------------------------------------------------
-        api.registerTool({
-            name: "mega_recall",
-            description: "Recall and inline relevant context from the mega-compact vector store for the current session.",
-            parameters: {
-                type: "object",
-                properties: {
-                    sessionId: {
-                        type: "string",
-                        description: "Session ID to recall context for.",
-                    },
-                    query: {
-                        type: "string",
-                        description: "Natural language query for relevant context.",
-                    },
-                    limit: {
-                        type: "number",
-                        description: "Max checkpoints to recall (default 3).",
-                    },
-                },
-                required: ["sessionId", "query"],
-                additionalProperties: false,
-            },
-            handler: async (args) => {
-                const { sessionId, query, limit } = args;
-                if (!sessionId || !query) {
-                    return {
-                        content: [{ type: "text", text: "Both `sessionId` and `query` are required." }],
-                        isError: true,
-                    };
-                }
-                try {
-                    const result = recallAndInline({ sessionId, query, limit: limit ?? 3, source: "command", skipInjected: false }, store);
-                    if (result.toInject.length === 0) {
-                        return {
-                            content: [{ type: "text", text: "No relevant context found in the mega-compact store." }],
-                        };
-                    }
-                    const parts = [
-                        `**Recalled ${result.toInject.length} checkpoint(s):**`,
-                        ...result.report,
-                        "",
-                        "---",
-                        result.block,
-                    ];
-                    return { content: [{ type: "text", text: parts.join("\n") }] };
-                }
-                catch (err) {
-                    return {
-                        content: [{ type: "text", text: `Error during mega-recall: ${err}` }],
-                        isError: true,
-                    };
-                }
-            },
-        });
-        // -----------------------------------------------------------------------
-        // Cleanup on shutdown
-        // -----------------------------------------------------------------------
-        api.on("shutdown", () => {
-            logger.info?.(`${PLUGIN_ID}: shutting down — clearing default store`);
-            setDefaultStore(undefined);
-        });
-        logger.info?.(`${PLUGIN_ID}: plugin registered (tools: mega_status, mega_recall)`);
-    },
-});

package/dist/src/minilm.js

@@ -1,92 +0,0 @@
-/**
- * minilm.ts — local MiniLM (all-MiniLM-L6-v2) sentence embedder (Sprint 12).
- *
- * Implements the `Embedder` interface so it drops into the existing VectorStore
- * dedup cascade and search with no call-site changes. Inference is 100% local:
- * the ONNX model + WordPiece vocab are on-disk artifacts fetched once by
- * scripts/setup-minilm.mjs. There is NO network call at runtime (PREVENT-PI-004).
- *
- * Inputs (dynamic): input_ids, attention_mask, token_type_ids (int64).
- * Output: last_hidden_state (batch, seq, 384). We mean-pool over non-padded
- * tokens (attention_mask == 1) and L2-normalize → 384-dim unit vector.
- *
- * The ONNX session + tokenizer are loaded LAZILY on first embed() so the default
- * TrigramEmbedder path (and its zero native-init cost) is untouched unless
- * MEGACOMPACT_EMBEDDER=minilm is selected.
- */
-import { join } from "node:path";
-import { homedir } from "node:os";
-import { existsSync } from "node:fs";
-import { l2Normalize, awaitSync } from "./embedder.js";
-import { WordPieceTokenizer } from "./wordpiece.js";
-export const MINILM_DIM = 384;
-export const MINILM_MAX_LEN = 256;
-/** Resolve the model directory: MEGACOMPACT_MINILM_DIR > ./models/minilm > ~/.pi … */
-function resolveModelDir() {
-    if (process.env.MEGACOMPACT_MINILM_DIR)
-        return process.env.MEGACOMPACT_MINILM_DIR;
-    // Repo-local vendored path (gitignored).
-    const local = join(process.cwd(), "models", "minilm");
-    if (existsSync(local))
-        return local;
-    return join(homedir(), ".pi", "agent", "extensions", "mega-compact", "models", "minilm");
-}
-export class MiniLMEmbedder {
-    dim = MINILM_DIM;
-    session = null;
-    tokenizer = null;
-    modelDir;
-    loadPromise = null;
-    constructor(modelDir = resolveModelDir()) {
-        this.modelDir = modelDir;
-    }
-    async ensureLoaded() {
-        if (this.session && this.tokenizer)
-            return;
-        if (this.loadPromise)
-            return this.loadPromise;
-        this.loadPromise = (async () => {
-            const ort = await import("onnxruntime-node");
-            const modelPath = join(this.modelDir, "model_quantized.onnx");
-            const vocabPath = join(this.modelDir, "vocab.txt");
-            if (!existsSync(modelPath) || !existsSync(vocabPath)) {
-                throw new Error(`MiniLM artifacts missing in ${this.modelDir}. Run: node scripts/setup-minilm.mjs`);
-            }
-            // 1 thread is plenty for a single short-region embed and bounds CPU.
-            this.session = await ort.InferenceSession.create(modelPath, {
-                executionProviders: ["cpu"],
-                graphOptimizationLevel: "all",
-            });
-            this.tokenizer = WordPieceTokenizer.fromVocabFile(vocabPath);
-        })();
-        return this.loadPromise;
-    }
-    embed(text) {
-        awaitSync(this.ensureLoaded());
-        const enc = this.tokenizer.encode(text, MINILM_MAX_LEN);
-        const n = enc.inputIds.length;
-        const BigInt64 = (arr) => arr.map((x) => BigInt(x));
-        const ort = awaitSync(import("onnxruntime-node"));
-        const tensors = {
-            input_ids: new ort.Tensor("int64", BigInt64(enc.inputIds), [1, n]),
-            attention_mask: new ort.Tensor("int64", BigInt64(enc.attentionMask), [1, n]),
-            token_type_ids: new ort.Tensor("int64", BigInt64(enc.tokenTypeIds), [1, n]),
-        };
-        const out = awaitSync(this.session.run(tensors));
-        const hidden = out.last_hidden_state.data;
-        // hidden shape: [1, n, 384]. Mean-pool over non-padded positions.
-        const pooled = new Array(MINILM_DIM).fill(0);
-        let count = 0;
-        for (let i = 0; i < n; i++) {
-            if (enc.attentionMask[i] === 0)
-                continue;
-            const base = i * MINILM_DIM;
-            for (let d = 0; d < MINILM_DIM; d++)
-                pooled[d] += hidden[base + d];
-            count++;
-        }
-        if (count === 0)
-            return l2Normalize(new Array(MINILM_DIM).fill(0));
-        return l2Normalize(pooled.map((x) => x / count));
-    }
-}

package/dist/src/wordpiece.js

@@ -1,129 +0,0 @@
-/**
- * wordpiece.ts — a self-contained WordPiece tokenizer for BERT/MiniLM.
- *
- * Loads the canonical `vocab.txt` (bert-base-uncased, ~30K tokens) from disk and
- * implements the standard uncased BERT preprocessing + greedy longest-match
- * WordPiece segmentation. No native dependency, no network — the vocab file is a
- * local artifact fetched once by scripts/setup-minilm.mjs (PREVENT-PI-004).
- *
- * This mirrors HuggingFace `BertTokenizer` closely enough for sentence-embedding
- * use: lowercase, strip accents, split on whitespace + punctuation, then
- * WordPiece each token with the `##` continuation convention. Special tokens
- * [CLS]/[SEP] are added by the caller's encode().
- */
-import { readFileSync, existsSync } from "node:fs";
-const UNK = "[UNK]";
-const CLS = "[CLS]";
-const SEP = "[SEP]";
-const PAD = "[PAD]";
-const MAX_INPUT_CHARS_PER_WORD = 200;
-export class WordPieceTokenizer {
-    vocab;
-    clsId;
-    sepId;
-    padId;
-    unkId;
-    constructor(vocab) {
-        this.vocab = vocab;
-        this.clsId = vocab.get(CLS) ?? 101;
-        this.sepId = vocab.get(SEP) ?? 102;
-        this.padId = vocab.get(PAD) ?? 0;
-        this.unkId = vocab.get(UNK) ?? 100;
-    }
-    /** Build a tokenizer from a vocab.txt file (one token per line, index = line). */
-    static fromVocabFile(path) {
-        if (!existsSync(path)) {
-            throw new Error(`WordPiece vocab not found at ${path}. Run: node scripts/setup-minilm.mjs`);
-        }
-        const lines = readFileSync(path, "utf-8").split("\n");
-        const vocab = new Map();
-        for (let i = 0; i < lines.length; i++) {
-            const tok = lines[i].replace(/\r$/, "");
-            if (tok.length > 0 || i < lines.length - 1)
-                vocab.set(tok, i);
-        }
-        return new WordPieceTokenizer(vocab);
-    }
-    /** Uncased BERT basic tokenization: lowercase, strip accents, split on ws+punct. */
-    basicTokenize(text) {
-        // NFD + strip combining marks (accent removal), then lowercase.
-        const cleaned = text
-            .normalize("NFD")
-            .replace(/[̀-ͯ]/g, "")
-            .toLowerCase();
-        const tokens = [];
-        let buf = "";
-        const flush = () => {
-            if (buf.length > 0) {
-                tokens.push(buf);
-                buf = "";
-            }
-        };
-        for (const ch of cleaned) {
-            if (/\s/.test(ch)) {
-                flush();
-            }
-            else if (/[!-/:-@[-`{-~¡-¿]/.test(ch)) {
-                // Punctuation becomes its own token.
-                flush();
-                tokens.push(ch);
-            }
-            else {
-                buf += ch;
-            }
-        }
-        flush();
-        return tokens;
-    }
-    /** Greedy longest-match WordPiece for a single word. */
-    wordpiece(word) {
-        if (word.length > MAX_INPUT_CHARS_PER_WORD)
-            return [UNK];
-        const pieces = [];
-        let start = 0;
-        while (start < word.length) {
-            let end = word.length;
-            let cur = null;
-            while (start < end) {
-                let sub = word.slice(start, end);
-                if (start > 0)
-                    sub = "##" + sub;
-                if (this.vocab.has(sub)) {
-                    cur = sub;
-                    break;
-                }
-                end--;
-            }
-            if (cur === null)
-                return [UNK]; // any unmatchable piece → whole word is UNK
-            pieces.push(cur);
-            start = end;
-        }
-        return pieces;
-    }
-    /** Tokenize text into WordPiece token strings (no special tokens). */
-    tokenize(text) {
-        const out = [];
-        for (const word of this.basicTokenize(text)) {
-            for (const piece of this.wordpiece(word))
-                out.push(piece);
-        }
-        return out;
-    }
-    /**
-     * Encode text into model inputs with [CLS]…[SEP], truncated to `maxLen`.
-     * attention_mask is all 1s (no padding for single-sequence inference).
-     */
-    encode(text, maxLen = 256) {
-        const pieces = this.tokenize(text).slice(0, Math.max(0, maxLen - 2));
-        const inputIds = [this.clsId];
-        for (const p of pieces)
-            inputIds.push(this.vocab.get(p) ?? this.unkId);
-        inputIds.push(this.sepId);
-        return {
-            inputIds,
-            attentionMask: inputIds.map(() => 1),
-            tokenTypeIds: inputIds.map(() => 0),
-        };
-    }
-}