pi-mega-compact 0.13.6 → 0.14.0

package/README.md

@@ -13,13 +13,14 @@ A local-first context compressor for the [pi coding agent](https://github.com/ea
 - **Durable memory** — on a cadence the store auto-reviews and safe-keeps decisions, facts, and preferences as first-class RAG memories, so long-running projects remember what mattered.
 - **Prompt-cache optimization** — message separation + cache striping (default OFF, opt-in). Targets 82-90% cache hit rate by structuring context around provider cache boundaries. Enables with `MEGACOMPACT_MESSAGE_SEPARATION` and `MEGACOMPACT_CACHE_STRIPING`.
 - **Context health + KV cache poison validation** (v0.12) — a real-time composite 0-1 health score per turn from five sub-scores (drift, output quality, error rate, cache health, cache poison). Catches garbled/hallucinated output and provider-side KV-cache corruption *before* they waste tokens — the failure mode where a large-context model (e.g. DeepSeek V4 Flash, 1M window) degrades at <1% usage. Tri-layer cache poison validation: prefix hash (L1 FNV-1a), output-quality-by-cache-hit (L2 semantic), error-rate correlation (L3 behavioral). The dashboard **Health tab** shows a gauge, sparkline, sub-score bars, alerts, and per-model breakdown. Auto-mitigation (force compaction on degraded context, prefix break on poisoned cache) is default OFF — toggle it in the Maintenance tab.
-- **RAG suite** — query reformulation (TF-IDF + RRF), tiered routing (L0 cache -> L1 FTS5 -> L2 PGlite), recall-quality metrics (CRAG), memory graph, and HyDE (hypothetical document embeddings). All default ON with graceful fallback — opt out via `MEGACOMPACT_<NAME>_DISABLED=true`. HyDE auto-activates only when an LLM embedder (Ollama/HTTP) is configured — it's a no-op with the default TrigramEmbedder. Toggle any flag from the dashboard Setup tab.
-- **Stacked memory graph** — the dashboard shows memory composition over time from 3 content sources (turns, durable memories, wiki) with a 9-gate validation system and a graph-health indicator. Per-model provider cache breakdown in the Cache tab.
+- **RAG suite** — query reformulation (TF-IDF + RRF), tiered routing (L0 cache -> L1 FTS5 -> L2 PGlite), recall-quality metrics (CRAG), memory graph, and HyDE (hypothetical document embeddings). All default ON with graceful fallback — opt out via `MEGACOMPACT_<NAME>_DISABLED=true`. HyDE auto-activates only when an LLM embedder (Ollama/HTTP) is configured — it's a no-op with the default TrigramEmbedder. The dashboard **Metrics tab** (v0.13) shows RAG recall-quality metrics (pass rate, avg lift, telemetry turns, HyDE runs) and the Overview tab has a RAG health card. Toggle any flag from the dashboard Setup tab.
 - **Debug bundle** — Maintenance tab in the dashboard has a **Gather Debug Logs** button that collects events, config, and store state into a shareable archive for bug reports.
-- **Local-first by default** — node:sqlite + trigram embeddings by default, zero calls off your machine. Bring your own localhost embedder (Ollama, ONNX, TEI) for better semantic matches — the embedder endpoint is loopback-only by default (`MEGACOMPACT_ALLOW_REMOTE_EMBEDDER=1` opts in to a remote/third-party endpoint). The optional dashboard is localhost-only. No cloud, no API calls, no telemetry.
+- **Local-first by default** — node:sqlite + trigram embeddings by default, zero calls off your machine. Bring your own localhost embedder (Ollama, ONNX, TEI) for better semantic matches — the embedder endpoint is loopback-only by default (`MEGACOMPACT_ALLOW_REMOTE_EMBEDDER=1` opts in to a remote/third-party endpoint). The optional dashboard is localhost-only. No cloud, no API calls, no telemetry. The optional Cost API lookup (`MEGACOMPACT_COST_API_ENABLED`, default OFF) is the one network exception — it fetches model pricing from a user-configured endpoint to show real $/token rates in the dashboard.
 - **Team-run aware** — fine-grained durable trim fires at agent settle during sub-agent runs, so long multi-agent work doesn't just collapse at the end.
-- **Multi-pi dashboard** — one dashboard tab per active pi process with the context stack, per-repo stats, and a live SSE feed across all of them. The React SPA has 15 lazy-loaded tabs — **Overview**, **Cache**, **Sessions**, **Turns** (per-turn memory + recall + rewind), and **Health** (context-health gauge + cache-poison alerts) are primary; **Repos**, **Events**, **Config**, **Setup** (embedding wizard), **Metrics** (perf latency/TPS/CPU), **Game**, **Achievements**, **Topics** (wiki), **Memory Map** (D3 graph), and **Maintenance** (debug bundle) are advanced.
-- **Auto-categorizing wiki.** Every 3 compactions (seeds from turns before that), the store clusters your real memory embeddings (k-means) and labels each cluster with its most discriminative terms (TF-IDF) — no LLM, no Ollama, fully local. The dashboard **Wiki tab** browses topics, searches by label or term, and drills down into the member memories of each cluster.
+- **Multi-pi dashboard** — one dashboard tab per active pi process with the context stack, per-repo stats, and a live SSE feed across all of them. The React SPA (Tailwind v3 + shadcn/ui, v0.13) has 12 lazy-loaded tabs — **Overview**, **Cache**, **Sessions**, and **Turns** (per-turn memory + recall + rewind) are primary; **Repos**, **Events**, **Setup** (embedding wizard + comprehensive settings panel), **Metrics** (perf latency/TPS/CPU + RAG recall quality), **Wiki** (auto-categorizing topic browser + evolution graph), **Memory Map** (D3 graph), **Maintenance** (debug bundle), and **Health** (context-health gauge + cache-poison alerts) are advanced. The Overview tab dynamically resolves the most recently active repo's snapshot so it always shows live data.
+- **Auto-categorizing wiki.** Every 3 compactions (seeds from turns before that), the store clusters your real memory embeddings (k-means) and labels each cluster with its most discriminative terms (TF-IDF) — no LLM, no Ollama, fully local. The dashboard **Wiki tab** (v0.13) browses topics, searches by label or term, drills down into the member memories of each cluster, and shows a topic evolution graph over time. Curation supports rename/merge/split operations with durable overrides that survive full rebuilds.
+- **Comprehensive settings panel** (v0.13.6) — every adjustable `MEGACOMPACT_*` env var is surfaced in the dashboard Setup tab, grouped by category (RAG Pipeline, Wiki/Turns, Dedup Tiers, Dedup Thresholds, RAPTOR Tuning). Toggle booleans, edit numerics with units, change strings — all write to `.mega-compact.env` via POST. The regression gate enforces that no new config flag ships without a dashboard entry.
+- **Stacked memory graph** — the dashboard shows memory composition over time from 3 content sources (turns, durable memories, wiki) with a 9-gate validation system and a graph-health indicator. Per-model provider cache breakdown in the Cache tab.
 
 ## Install
 
@@ -85,6 +86,12 @@ Set env vars before starting pi. Defaults are in `src/config/dedup.ts`.
 | `MEGACOMPACT_CONTEXT_HEALTH_OUTPUT_QUALITY` | `true` | Sub-score: repetition, coherence, token-salad detection |
 | `MEGACOMPACT_CONTEXT_HEALTH_CACHE_POISON` | `true` | Sub-score: tri-layer KV cache poison validation |
 | `MEGACOMPACT_CONTEXT_HEALTH_MITIGATE` | `false` | Opt-in: auto-compaction on degraded context, prefix break on poisoned cache |
+| `MEGACOMPACT_WIKI_ENHANCED` | `true` | Enhanced wiki with topic overrides + evolution tracking |
+| `MEGACOMPACT_WIKI_INCREMENTAL` | `true` | Incremental wiki updates (no full rebuild) |
+| `MEGACOMPACT_AUTO_WIKI` | `true` | Auto-generate wiki from turns |
+| `MEGACOMPACT_NEW_UI` | `true` | Use the new Tailwind/shadcn dashboard shell |
+| `MEGACOMPACT_COST_API_ENABLED` | `false` | Opt-in: fetch model pricing from an external API (PREVENT-PI-004 network exception). Enriches dashboard cost data for models not in the local pricing table |
+| `MEGACOMPACT_COST_API_URL` | _(unset)_ | OpenRouter-compatible model pricing endpoint (e.g. `https://openrouter.ai/api/v1/models`). Only contacted when `MEGACOMPACT_COST_API_ENABLED=true` |
 
 Full config reference: [`docs/CONFIGURATION.md`](docs/CONFIGURATION.md)
 
@@ -107,7 +114,7 @@ Detailed architecture: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md)
 
 ```bash
 npm run build     # TypeScript compile
-npm test          # Build + 1145 tests
+npm test          # Build + 1197 tests
 npm run lint      # Type check + guardrails scan
 ```
 

package/dist/extensions/dashboard-server/api-contracts/endpoints/registry.js

@@ -259,6 +259,12 @@ export const ENDPOINTS = {
         path: "/api/raptor-tree",
         description: "Hierarchical RAPTOR tree (summary nodes by level) for a session; defaults to the latest session with nodes.",
     },
+    /** GET /api/raptor-build-history — Build history (coherence, depth, timeout) for a session. */
+    raptorBuildHistory: {
+        method: "GET",
+        path: "/api/raptor-build-history",
+        description: "RAPTOR build history rows (node/leaf count, depth, coherence score, timed_out) for a session; defaults to the latest session with builds.",
+    },
     /** GET /api/embedder-health — Round-trip a test embed through the active embedder. */
     embedderHealth: {
         method: "GET",

package/dist/extensions/dashboard-server/index-reader.js

@@ -13,6 +13,8 @@ import { homedir } from "node:os";
 import { join } from "node:path";
 import { DatabaseSync } from "node:sqlite";
 import { readProviderCacheForRepo, } from "../../src/store/sqlite/perf-samples.js";
+import { lookupModelInputRate } from "../../src/pricing.js";
+import { costApiConfig, lookupCostApiPricing } from "../../src/costApi.js";
 export function getIndexDir() {
     const override = process.env.MEGACOMPACT_INDEX_DIR;
     if (override && override.trim() !== "")
@@ -37,31 +39,54 @@ export function readIndex() {
         const rows = db
             .prepare("SELECT * FROM repo_registry ORDER BY last_seen DESC")
             .all();
-        const mapped = rows.map((r) => ({
-            repoRoot: String(r.repo_root ?? ""),
-            displayName: String(r.display_name ?? ""),
-            stateDir: String(r.state_dir ?? ""),
-            checkpointCount: Number(r.checkpoint_count ?? 0),
-            tokensSaved: Number(r.tokens_saved ?? 0),
-            compressedOriginalBytes: Number(r.compressed_original_bytes ?? 0),
-            lastCompactedAt: r.last_compacted_at ?? null,
-            provider: r.provider ?? null,
-            providerName: r.provider_name ?? null,
-            modelName: r.model_name ?? null,
-            inputRate: r.input_rate ?? null,
-            outputRate: r.output_rate ?? null,
-            lastSeen: Number(r.last_seen ?? 0),
-            // Defaults — enriched below from each repo's own store.
-            tokensKept: 0,
-            tokensDropped: 0,
-            sessions: 0,
-            contextWindow: null,
-            maxTokens: null,
-            reasoning: null,
-            providerCachePct: null,
-            providerCacheRead: null,
-            providerCacheWrite: null,
-        }));
+        // Lazily fetch cost-API pricing once per readIndex call (cached for 1 hour
+        // inside costApi.ts). Avoids per-row network calls.
+        const costApi = costApiConfig();
+        const mapped = rows.map((r) => {
+            const modelName = String(r.model_name ?? "");
+            const localRate = r.input_rate ||
+                lookupModelInputRate(modelName) ||
+                null;
+            let inputRate = localRate;
+            let outputRate = r.output_rate ?? null;
+            // Fall back to the cost API when the local table has no data.
+            if (costApi.enabled && (inputRate == null || outputRate == null) && modelName) {
+                const apiPricing = lookupCostApiPricing(modelName);
+                if (apiPricing) {
+                    if (inputRate == null && apiPricing.inputRate != null) {
+                        inputRate = apiPricing.inputRate;
+                    }
+                    if (outputRate == null && apiPricing.outputRate != null) {
+                        outputRate = apiPricing.outputRate;
+                    }
+                }
+            }
+            return {
+                repoRoot: String(r.repo_root ?? ""),
+                displayName: String(r.display_name ?? ""),
+                stateDir: String(r.state_dir ?? ""),
+                checkpointCount: Number(r.checkpoint_count ?? 0),
+                tokensSaved: Number(r.tokens_saved ?? 0),
+                compressedOriginalBytes: Number(r.compressed_original_bytes ?? 0),
+                lastCompactedAt: r.last_compacted_at ?? null,
+                provider: r.provider ?? null,
+                providerName: r.provider_name ?? null,
+                modelName: r.model_name ?? null,
+                inputRate,
+                outputRate,
+                lastSeen: Number(r.last_seen ?? 0),
+                // Defaults — enriched below from each repo's own store.
+                tokensKept: 0,
+                tokensDropped: 0,
+                sessions: 0,
+                contextWindow: null,
+                maxTokens: null,
+                reasoning: null,
+                providerCachePct: null,
+                providerCacheRead: null,
+                providerCacheWrite: null,
+            };
+        });
         // Enrich each repo with per-store token + model detail read directly via
         // node:sqlite (same zero-dependency invariant as readIndex; no store graph
         // import). Best-effort: a missing/corrupt store degrades to the defaults

package/dist/extensions/dashboard-server/resolve-snapshot.js

@@ -0,0 +1,81 @@
+/**
+ * dashboard-server/resolve-snapshot.ts — dynamic per-repo snapshot resolver.
+ *
+ * The dashboard server is launched with a single static stateDir, but the
+ * active pi session may be running in a different repo. This module queries
+ * the repo registry to find the most recently active repo's dashboard.json,
+ * so the Overview always shows live data from the repo pi is actually running
+ * in — not the stale launcher dir.
+ *
+ * Resolution order:
+ * 1. Most recently seen repo (by last_seen in repo_registry) whose
+ *    dashboard.json exists and has a non-null updatedAt
+ * 2. The static launch stateDir's dashboard.json (original behavior)
+ */
+import { existsSync } from "node:fs";
+import { join } from "node:path";
+import { createRequire } from "node:module";
+import { readSnapshot } from "./snapshot.js";
+import { getIndexDir } from "./index-reader.js";
+const require = createRequire(import.meta.url);
+/**
+ * Find the most recently active repo's stateDir by scanning the repo registry
+ * for repos whose dashboard.json exists and was written recently. Returns null
+ * if no suitable repo is found.
+ */
+function mostRecentRepoStateDir(staticStateDir) {
+    const indexPath = join(getIndexDir(), "index.sqlite");
+    if (!existsSync(indexPath))
+        return null;
+    let db;
+    try {
+        const { DatabaseSync } = require("node:sqlite");
+        db = new DatabaseSync(indexPath, { readOnly: true });
+        const rows = db
+            .prepare(`SELECT state_dir, last_seen FROM repo_registry
+				 WHERE state_dir IS NOT NULL AND state_dir != ?
+				 ORDER BY last_seen DESC`)
+            .all(staticStateDir);
+        for (const r of rows) {
+            const dashPath = join(r.state_dir, "dashboard.json");
+            if (!existsSync(dashPath))
+                continue;
+            const snap = readSnapshot(dashPath);
+            if (snap.updatedAt) {
+                return r.state_dir;
+            }
+        }
+        return null;
+    }
+    catch {
+        return null;
+    }
+    finally {
+        try {
+            db?.close();
+        }
+        catch {
+            /* ignore */
+        }
+    }
+}
+/**
+ * Resolve the best snapshot to serve. Tries the most recently active repo's
+ * dashboard.json first, then falls back to the static launch path.
+ */
+export function resolveSnapshot(staticSnapshotPath, staticStateDir) {
+    // Try the most recently active repo first.
+    const activeDir = mostRecentRepoStateDir(staticStateDir);
+    if (activeDir) {
+        const activePath = join(activeDir, "dashboard.json");
+        const snap = readSnapshot(activePath);
+        if (snap.updatedAt) {
+            return { snapshot: snap, stateDir: activeDir };
+        }
+    }
+    // Fall back to the static launch path.
+    return {
+        snapshot: readSnapshot(staticSnapshotPath),
+        stateDir: staticStateDir,
+    };
+}

package/dist/extensions/dashboard-server/routes-core.js

@@ -5,7 +5,7 @@
  * builds it. Route handler files import from here; the barrel (routes.ts)
  * re-exports everything for server.ts.
  */
-import { readSnapshot } from "./snapshot.js";
+import { resolveSnapshot } from "./resolve-snapshot.js";
 // ---------------------------------------------------------------------------
 // Helper: overlayCurrentRepo
 // ---------------------------------------------------------------------------
@@ -13,16 +13,21 @@ function makeOverlayCurrentRepo(snapshotPath, stateDir) {
     return (idx) => {
         if (!idx || !idx.repos.length)
             return;
+        // Dynamically resolve the most recently active repo's snapshot so the
+        // overlay matches what /api/snapshot serves (not the stale launch dir).
         let snap = null;
+        let activeStateDir = stateDir;
         try {
-            snap = readSnapshot(snapshotPath);
+            const resolved = resolveSnapshot(snapshotPath, stateDir);
+            snap = resolved.snapshot;
+            activeStateDir = resolved.stateDir;
         }
         catch {
             return;
         }
         if (!snap || !snap.repo)
             return;
-        const cur = idx.repos.find((r) => r.stateDir === stateDir);
+        const cur = idx.repos.find((r) => r.stateDir === activeStateDir);
         if (!cur)
             return;
         const prevSaved = cur.tokensSaved;

package/dist/extensions/dashboard-server/routes-rag-settings-helpers.js

@@ -129,6 +129,13 @@ export const SETTINGS = [
             num("MEGACOMPACT_EMBED_CACHE", "Embed Cache Size", "Embedding cache entries (0 = disabled)", 256, 0, 10000),
         ],
     },
+    {
+        name: "Cost API",
+        settings: [
+            boolDirect("MEGACOMPACT_COST_API_ENABLED", "Cost API Lookup", "Fetch model pricing from an external API (PREVENT-PI-004 network exception — opt-in). Enriches the dashboard Cache/Repos tabs with real $/token rates for models not in the local pricing table.", false),
+            str("MEGACOMPACT_COST_API_URL", "Cost API URL", "OpenRouter-compatible model pricing endpoint (e.g. https://openrouter.ai/api/v1/models). Only contacted when Cost API Lookup is ON.", ""),
+        ],
+    },
 ];
 /** Flat index of every setting by key for O(1) lookup + validation. */
 export const SETTING_BY_KEY = new Map(SETTINGS.flatMap((c) => c.settings).map((s) => [s.key, s]));

package/dist/extensions/dashboard-server/routes-raptor.js

@@ -86,3 +86,63 @@ export function handleRaptorTree(req, res, ctx) {
         return true;
     }
 }
+/** Find the most recent session that has build history rows (else null). */
+function latestBuildSession(stateDir) {
+    try {
+        const db = openStore(stateDir);
+        const row = db
+            .prepare("SELECT DISTINCT session_id FROM raptor_build_history ORDER BY completed_at DESC LIMIT 1")
+            .get();
+        return row?.session_id ?? null;
+    }
+    catch {
+        return null;
+    }
+}
+export function handleRaptorBuildHistory(req, res, ctx) {
+    if (req.method !== "GET")
+        return false;
+    const parsed = parseUrl(req.url ?? "", true);
+    if (parsed.pathname !== "/api/raptor-build-history")
+        return false;
+    try {
+        const { listBuildHistory } = _req("../../src/dedup/raptor/buildHistory.js");
+        let sessionId = typeof parsed.query.sessionId === "string" ? parsed.query.sessionId : "";
+        if (!sessionId) {
+            sessionId = latestBuildSession(ctx.stateDir) ?? "";
+            if (!sessionId) {
+                const body = { builds: [], empty: true };
+                res.writeHead(200, { "Content-Type": "application/json" });
+                res.end(JSON.stringify(body));
+                return true;
+            }
+        }
+        const rows = listBuildHistory(sessionId, ctx.stateDir);
+        if (rows.length === 0) {
+            const body = { builds: [], empty: true };
+            res.writeHead(200, { "Content-Type": "application/json" });
+            res.end(JSON.stringify(body));
+            return true;
+        }
+        const dtos = rows.map((r) => ({
+            buildId: r.buildId,
+            sessionId: r.sessionId,
+            startedAt: r.startedAt,
+            completedAt: r.completedAt,
+            nodeCount: r.nodeCount,
+            leafCount: r.leafCount,
+            depth: r.depth,
+            coherenceScore: r.coherenceScore,
+            timedOut: r.timedOut,
+        }));
+        const body = { builds: dtos, empty: false };
+        res.writeHead(200, { "Content-Type": "application/json" });
+        res.end(JSON.stringify(body));
+        return true;
+    }
+    catch (e) {
+        res.writeHead(500, { "Content-Type": "application/json" });
+        res.end(JSON.stringify({ error: String(e) }));
+        return true;
+    }
+}

package/dist/extensions/dashboard-server/routes-repo.js

@@ -4,6 +4,7 @@
 import { existsSync, readFileSync } from "node:fs";
 import { join } from "node:path";
 import { readSnapshot } from "./snapshot.js";
+import { resolveSnapshot } from "./resolve-snapshot.js";
 import { dashboardHtml } from "./html.js";
 import { readIndex, getIndexDir } from "./index-reader.js";
 import { ACTIVE_WINDOW_SEC } from "./types.js";
@@ -26,7 +27,10 @@ export function handleIndex(req, res, ctx) {
         return true;
     }
     if (req.url === "/api/snapshot") {
-        const snap = readSnapshot(snapshotPath);
+        // Dynamically resolve the most recently active session's dashboard.json
+        // so the Overview shows live data from the repo pi is running in — not
+        // the stale launcher dir the server was started with.
+        const { snapshot: snap } = resolveSnapshot(snapshotPath, ctx.stateDir);
         res.writeHead(200, { "Content-Type": "application/json" });
         res.end(JSON.stringify(snap));
         return true;

package/dist/extensions/dashboard-server/routes-sessions.js

@@ -16,10 +16,13 @@ export function handleEvents(req, res, ctx) {
         "Cache-Control": "no-cache",
         Connection: "keep-alive",
     });
-    // Drain existing events so the client starts with history
+    // Drain existing events so the client starts with history.
+    // Only stream events with a "type" field — events.log also contains
+    // internal monitoring rows (e.g. captureModel:recorded) that use "event"
+    // instead of "type" and aren't part of the SseEvent contract.
     const { data: existing, offset: initialOffset } = readFrom(eventsPath, 0);
     eventOffsetRef.value = initialOffset;
-    const lines = existing.split("\n").filter((l) => l.trim());
+    const lines = existing.split("\n").filter((l) => l.trim() && l.includes('"type"'));
     for (const line of lines) {
         res.write(`data: ${line}\n\n`);
     }
@@ -32,7 +35,7 @@ export function handleEvents(req, res, ctx) {
             watchTimer = null;
             const { data, offset } = readFrom(eventsPath, eventOffsetRef.value);
             eventOffsetRef.value = offset;
-            const newLines = data.split("\n").filter((l) => l.trim());
+            const newLines = data.split("\n").filter((l) => l.trim() && l.includes('"type"'));
             for (const line of newLines) {
                 res.write(`data: ${line}\n\n`);
             }

package/dist/extensions/dashboard-server/routes.js

@@ -15,7 +15,7 @@ export { handleProviderCache } from "./routes-cache.js";
 export { handleMemoryStatus } from "./routes-memory.js";
 export { handleSetupStatus, handleSetupDetect, handleSetupConfigure } from "./routes-setup.js";
 export { handleMemoryMap } from "./routes-memory-map.js";
-export { handleRaptorTree } from "./routes-raptor.js";
+export { handleRaptorTree, handleRaptorBuildHistory } from "./routes-raptor.js";
 export { handleCacheStripes } from "./routes-cache.js";
 export { handleContextHealth, handleCachePoison, handleHealthSettings } from "./routes-health.js";
 export { handleEmbedderHealth } from "./routes-embedder-health.js";

package/dist/extensions/dashboard-server/server.js

@@ -13,7 +13,7 @@ import { fileURLToPath } from "node:url";
 import { createRequire } from "node:module";
 import { NEW_UI } from "../../src/config.js";
 import { log, setLogPath, setDashboardServerVersion } from "./state.js";
-import { buildRouteContext, handleIndex, handleRepoIndex, handleEvents, handleGameState, handleGameScores, handlePerf, handleAchievements, handleSessions, handleTopics, handleTurns, handleMaintenance, handleProviderCache, handleCacheStripes, handleMemoryStatus, handleSetupStatus, handleSetupDetect, handleSetupConfigure, handleMemoryMap, handleRaptorTree, handleContextHealth, handleCachePoison, handleHealthSettings, handleEmbedderHealth, handleRagSettings, handleRagMetrics, handleWiki, handleStatic, } from "./routes.js";
+import { buildRouteContext, handleIndex, handleRepoIndex, handleEvents, handleGameState, handleGameScores, handlePerf, handleAchievements, handleSessions, handleTopics, handleTurns, handleMaintenance, handleProviderCache, handleCacheStripes, handleMemoryStatus, handleSetupStatus, handleSetupDetect, handleSetupConfigure, handleMemoryMap, handleRaptorTree, handleRaptorBuildHistory, handleContextHealth, handleCachePoison, handleHealthSettings, handleEmbedderHealth, handleRagSettings, handleRagMetrics, handleWiki, handleStatic, } from "./routes.js";
 export async function launchDashboardServer(stateDir) {
     // Our own package version — exposed at /api/version so the launcher can
     // detect a stale server (started by an older build) and replace it on
@@ -219,6 +219,8 @@ export async function launchDashboardServer(stateDir) {
             return;
         if (handleRaptorTree(req, res, ctx))
             return;
+        if (handleRaptorBuildHistory(req, res, ctx))
+            return;
         if (handleContextHealth(req, res, ctx))
             return;
         if (handleCachePoison(req, res, ctx))

package/dist/extensions/mega-pipeline/recall.js

@@ -79,17 +79,23 @@ export function doRecall(runtime, config, ctx, query, source) {
         runtime.pushTicker(`${C.amber}↩${C.reset} recalled ${top.checkpoint.checkpointId} · ${scorePct}% · ${label}`);
         runtime.lastWhy = `why: recalled@${scorePct}% (${result.toInject.length} chkpt)`;
     }
+    let sumTokens = 0;
+    for (const h of result.toInject)
+        sumTokens += h.checkpoint.tokenEstimate;
     if (result.toInject.length > 0) {
-        let sumTokens = 0;
-        for (const h of result.toInject)
-            sumTokens += h.checkpoint.tokenEstimate;
         runtime.rt.recallInjections += result.toInject.length;
         runtime.rt.cacheHitTokens += sumTokens;
         incRecallInjected(result.toInject.length, runtime.currentStateDir);
         incCacheHitTokens(sumTokens, runtime.currentStateDir);
-        // S43: record recall provenance — which checkpoints/summaries served this
-        // turn, their score + source path. Linked to the turn row written at
-        // turn_end via the conversation+turnIndex. Best-effort + non-fatal.
+    }
+    // S43: record recall provenance — which checkpoints/summaries served this
+    // turn, their score + source path. Linked to the turn row written at
+    // turn_end via the conversation+turnIndex. Best-effort + non-fatal.
+    // Persists telemetry (HyDE + recall metrics) even when recall returned
+    // no hits, so empty-recall HyDE invocations are still visible in the
+    // dashboard Turns/Metrics tabs.
+    const hasTelemetry = result.hydeInfo != null || result.recallMetrics != null;
+    if (result.toInject.length > 0 || hasTelemetry) {
         try {
             const convId = ensureConversationIdFor(config, sid, runtime.currentStateDir);
             const turnId = recordTurnWrite(config, {
@@ -101,16 +107,18 @@ export function doRecall(runtime, config, ctx, query, source) {
                 hyde: result.hydeInfo ?? undefined,
                 recallMetrics: result.recallMetrics ?? undefined,
             }, runtime.currentStateDir);
-            recordRecallWrite(config, turnId, result.toInject.map((h) => ({
-                checkpointId: h.checkpoint.checkpointId,
-                score: h.score,
-                source: h.raptorLevel !== undefined
-                    ? "raptor"
-                    : h.repoId
-                        ? "cross-repo"
-                        : "flat",
-                raptorLevel: h.raptorLevel,
-            })), runtime.currentStateDir);
+            if (result.toInject.length > 0) {
+                recordRecallWrite(config, turnId, result.toInject.map((h) => ({
+                    checkpointId: h.checkpoint.checkpointId,
+                    score: h.score,
+                    source: h.raptorLevel !== undefined
+                        ? "raptor"
+                        : h.repoId
+                            ? "cross-repo"
+                            : "flat",
+                    raptorLevel: h.raptorLevel,
+                })), runtime.currentStateDir);
+            }
         }
         catch {
             /* non-fatal: recall provenance never breaks the recall path */

package/dist/extensions/mega-runtime/capture-model.js

@@ -7,6 +7,7 @@
  */
 import { resolveRepoRoot } from "../mega-config.js";
 import { recordModelSnapshot, recordRepoModel } from "../../src/store/sqlite.js";
+import { lookupModelInputRate } from "../../src/pricing.js";
 // -------------------------------------------------------------- captureModel
 export function captureModelImpl(ctx, ectx) {
     const m = ectx.model;
@@ -26,12 +27,14 @@ export function captureModelImpl(ctx, ectx) {
     catch {
         /* optional */
     }
+    const modelId = m.id;
+    const fallbackInput = lookupModelInputRate(modelId);
     const snap = {
         provider: m.provider,
         providerName,
-        modelId: m.id,
+        modelId,
         modelName: m.name ?? null,
-        inputRate: m.cost?.input ?? 0,
+        inputRate: m.cost?.input || fallbackInput || 0,
         outputRate: m.cost?.output ?? 0,
         contextWindow: m.contextWindow ?? 0,
         maxTokens: m.maxTokens ?? 0,

package/dist/src/config/dedup.js

@@ -43,7 +43,8 @@ export function loadDedupConfig() {
         L2_ENABLED: envBool("MEGACOMPACT_L2_ENABLED", true),
         // Fix D: RAPTOR promoted to live recall. Default ON; canary.ts sequences it
         // last (L0→L1→L2→RAPTOR) and auto-disables on p95 breach, so promotion is
-        // safe. `RAPTOR_SHADOW_MODE=false` still gates serving during transition.
+        // safe. Shadow mode (RAPTOR_SHADOW_MODE=true) opts out of serving while
+        // still building + persisting the tree (transition/eval use).
         RAPTOR_ENABLED: envBool("MEGACOMPACT_RAPTOR_ENABLED", true),
         MARK_ONLY_L0: envBool("MEGACOMPACT_MARK_ONLY_L0", false),
         MARK_ONLY_L1: envBool("MEGACOMPACT_MARK_ONLY_L1", false),

package/dist/src/costApi.js

@@ -0,0 +1,113 @@
+/**
+ * costApi.ts — optional model pricing lookup from an external API.
+ *
+ * PREVENT-PI-004 EXEMPTION: This module makes network calls to fetch model
+ * pricing data. It is disabled by default and must be explicitly opted in via
+ * MEGACOMPACT_COST_API_ENABLED=true + MEGACOMPACT_COST_API_URL=<url>.
+ *
+ * The dashboard server uses this to enrich the inputRate/outputRate fields
+ * when the local pricing table (pricing.ts) has no data for a model.
+ *
+ * Results are cached for 1 hour to minimize network calls.
+ */
+import { spawnSync } from "node:child_process"; // guardrails-allow PREVENT-PI-004: user-opted-in cost API — fetches model pricing from a user-configured endpoint, never called unless MEGACOMPACT_COST_API_ENABLED=true
+let _cache = null;
+const CACHE_TTL_MS = 60 * 60 * 1000; // 1 hour
+export function costApiConfig() {
+    const enabled = process.env.MEGACOMPACT_COST_API_ENABLED === "true" ||
+        process.env.MEGACOMPACT_COST_API_ENABLED === "1";
+    const url = process.env.MEGACOMPACT_COST_API_URL ?? "";
+    return { enabled, url };
+}
+/**
+ * Fetch model pricing from the configured API endpoint.
+ *
+ * Expects a JSON response of shape:
+ *   { "data": [{ "id": "model-name", "pricing": { "prompt": "0.000003", "completion": "0.000015" } }] }
+ * (OpenRouter-compatible format). Values are USD per token as strings.
+ *
+ * Returns a map keyed by lowercase model id → { inputRate, outputRate }.
+ * Returns an empty map on any error. Never throws.
+ */
+export function fetchModelPricing() {
+    const cfg = costApiConfig();
+    if (!cfg.enabled || !cfg.url)
+        return new Map();
+    // Return cached data if fresh.
+    if (_cache && Date.now() - _cache.timestamp < CACHE_TTL_MS) {
+        return _cache.data;
+    }
+    const result = _doFetch(cfg.url);
+    _cache = { timestamp: Date.now(), data: result };
+    return result;
+}
+function _doFetch(url) {
+    // Synchronous bridge: spawnSync an inline worker so the call blocks without
+    // deadlocking fetch (mirrors HttpEmbedder / RAPTOR summarizer pattern).
+    const WORKER = String.raw `
+		try {
+			const r = await fetch(process.env.COST_API_URL, { // guardrails-allow PREVENT-PI-004: user-opted-in cost API — fetches model pricing from a user-configured endpoint
+				method: "GET",
+				headers: { "accept": "application/json" },
+			});
+			const j = await r.json();
+			process.stdout.write(JSON.stringify({ ok: r.ok, data: j }));
+		} catch (e) {
+			process.stdout.write(JSON.stringify({ ok: false, error: String(e && e.message ? e.message : e) }));
+		}
+	`;
+    const res = spawnSync(process.execPath, ["-e", WORKER], {
+        encoding: "utf8",
+        timeout: 15_000,
+        env: { ...process.env, COST_API_URL: url },
+    });
+    const out = new Map();
+    let parsed;
+    try {
+        parsed = JSON.parse(res.stdout);
+    }
+    catch {
+        return out;
+    }
+    if (!parsed.ok || !parsed.data)
+        return out;
+    // OpenRouter-compatible: { data: [{ id, pricing: { prompt, completion } }] }
+    const models = parsed.data?.data;
+    if (!Array.isArray(models))
+        return out;
+    for (const m of models) {
+        if (!m.id)
+            continue;
+        const prompt = m.pricing?.prompt;
+        const completion = m.pricing?.completion;
+        const inputRate = prompt != null ? parseFloat(prompt) : null;
+        const outputRate = completion != null ? parseFloat(completion) : null;
+        if (inputRate != null || outputRate != null) {
+            out.set(m.id.toLowerCase(), { inputRate, outputRate });
+        }
+    }
+    return out;
+}
+/**
+ * Look up pricing for a model name from the cost API cache.
+ * Tries exact match, then prefix match (same pattern as lookupModelInputRate).
+ */
+export function lookupCostApiPricing(model) {
+    const data = fetchModelPricing();
+    if (data.size === 0)
+        return null;
+    const lower = model.toLowerCase();
+    // Exact match.
+    if (data.has(lower))
+        return data.get(lower) ?? null;
+    // Prefix match.
+    for (const [key, pricing] of data) {
+        if (lower.startsWith(key))
+            return pricing;
+    }
+    return null;
+}
+/** Clear the pricing cache (useful for testing). */
+export function clearCostApiCache() {
+    _cache = null;
+}