@echomem/mcp 1.4.33 → 1.4.35

package/README.md

@@ -37,38 +37,26 @@ legacy EchoMem-synthesized recall answer.
 
 ## Quick start (recommended)
 
-```bash
-# One command, everything: download the bridge + HUD, configure every coding agent you have
-# installed (Codex, Claude Code, Claude Desktop, Cursor, Windsurf…), log in, and launch the HUD.
-npm i -g @echomem/mcp@latest && echomem-mcp init
-```
+Install Echo Desktop, sign in once, then open **Connect MCP** and choose **Connect Echo**. The app
+ships and installs the MCP runtime, configures the detected Codex and Claude hosts, mirrors the same
+device credential used by the desktop app, and verifies the MCP handshake. No terminal login is
+required.
+
+The standalone HUD has been retired; Echo Desktop is the persistent setup and status surface. The
+bridge still reads local agent logs for on-demand context-health tools, but it no longer publishes or
+auto-launches an Electron overlay.
 
-A global install (not `npx`) is recommended because it wires each editor to a **stable** path: a bare
-`npx` run resolves the bridge into a throwaway `_npx/<hash>` cache dir that npm later garbage-collects,
-which would break the MCP server after the fact (`setup` now refuses to pin such a path and falls back
-to the global install, but installing globally avoids the issue entirely). The context HUD's
-launch-at-login also runs from the installed path. `init` is the flagship one-liner — it wraps `setup --all --with-hud`: it writes each
-installed agent's MCP config (with **no secret** in it — credentials live in
-`~/.echomem/credentials.json`, mode 0600), adds the EchoMem memory guidance to their global
-`AGENTS.md` / `CLAUDE.md`, installs first-party `echomem-search`, `echomem-save`,
-`echomem-forget`, and `echomem-login` skills for Codex, opens the browser to approve the device
-(and unlock the vault for encrypted accounts), then launches the context HUD. EchoMem updates only
-its own skill folders; other memory-provider skills are detected and reported but never modified.
-Reload your editors and you're done.
-
-Prefer to keep it minimal? `echomem-mcp setup` configures only the auto-detected editor and skips the
-HUD; the granular commands below still work.
+For headless systems and development, the granular CLI commands remain available:
 
 | Command | What it does |
 |---|---|
-| `npm i -g @echomem/mcp@latest && echomem-mcp init` | **Everything in one command** — all installed agents + HUD + login |
+| `npm i -g @echomem/mcp@latest && echomem-mcp init` | Legacy/headless setup for installed agents + login |
 | `npm i -g @echomem/mcp@latest && echomem-mcp setup` | Install the CLI globally and configure just the detected editor |
 | `npx -y @echomem/mcp@latest setup` | One-off setup without keeping a global CLI command |
 | `echomem-mcp setup [--client cursor\|windsurf\|claude-desktop\|claude-code\|codex]` | Write client config + log in |
 | `echomem-mcp setup --skip-login [--client cursor\|windsurf\|claude-desktop\|claude-code\|codex]` | Write client config without opening the browser or changing credentials |
 | `npx -y @echomem/mcp@latest update --all` | One-shot update: install the latest bridge durably and repoint detected client configs, with no browser login |
 | `npx -y @echomem/mcp@latest update --client codex` | Update one client only |
-| `echomem-mcp setup --with-hud [--client codex]` | Write client config + log in + launch the EchoMem context HUD |
 | `echomem-mcp login` | Approve device in browser (or use `--token` / `--passphrase`) |
 | `echomem-mcp unlock` | Privately unlock the vault on this trusted device |
 | `echomem-mcp lock` | Remove the local vault key while keeping the device login |
@@ -82,29 +70,8 @@ checks npm for a newer published bridge using a cached, non-blocking check. Agen
 `npx -y @echomem/mcp@latest update --all` if the user agrees. The bridge does not auto-update on
 every MCP startup.
 
-## EchoMem Context HUD
-
-The package also ships `echomem-hud`, a local context-health sidecar for Codex and Claude. It reads
-local agent logs, computes a tracked lower-bound "clean vs dirty" context score, and renders a small
-HUD without adding tokens to the agent conversation.
-
-```bash
-echomem-hud app                 # Electron floating HUD
-echomem-hud serve               # Browser fallback at http://127.0.0.1:17377
-echomem-hud summary --json      # Machine-readable current score
-echomem-hud status              # Show detected Codex/Claude sources
-```
-
-Supported local sources:
-- Codex: `~/.codex/sessions/**/rollout-*.jsonl`
-- Claude Code: `~/.claude/echo-ctx/*.json` first, then local transcripts when present
-- Claude desktop agent/Cowork: local agent-mode transcripts under Application Support
-
-The v1 metric counts `range_redundant` reads only and reports pollution as `tracked dead-weight ≥`
-because exact context composition and provider eviction are not observable.
-
-MCP fallback: agents can call `echo_context_health` to get the same score as markdown in-chat. This
-is on-demand; the passive HUD remains a separate local process.
+Agents can still call `echo_context_health` for an on-demand local context-health report. It reads
+the local Codex/Claude logs and does not require a separate process or desktop overlay.
 
 ## Agent Doctor — local workspace forensics (new in 1.4.9)
 

package/dist/context-analysis/claude-native-canonical.js

@@ -7,6 +7,14 @@ import { walk } from "../report.js";
 import { isStrongPositiveFeedback, } from "./workspace-report.js";
 const OUTPUT_TOKEN_CAP = 12_000;
 const IMAGE_TOKENS = 4_000;
+// Request-by-request retained-context scoring grows faster than the raw JSONL. Keep onboarding
+// predictable by analyzing a deterministic head/middle/tail sample from oversized sessions. The
+// source transcript is untouched and remains available to the separate import pipeline.
+const CLAUDE_MAX_TURNS_PER_SESSION = 1_000;
+const CLAUDE_HEAD_TURNS = 200;
+const CLAUDE_TAIL_TURNS = 500;
+const CLAUDE_MIDDLE_WINDOWS = 3;
+const CLAUDE_MIDDLE_WINDOW_TURNS = 100;
 const PROBLEM_META = {
     P01: { label: "Outdated Images & Screenshots", bucket: "dead", category: "Runtime Bug", confidence: "high" },
     P02: { label: "Ignored User Instructions", bucket: "refind", category: "Model Behavior", confidence: "medium" },
@@ -350,26 +358,31 @@ function resultItem(args) {
 }
 function scoreClaudeNativeSession(parsed) {
     classifyItems(parsed.items);
-    const requestRows = [];
+    const terminalRequestByTurn = new Map();
     for (const request of parsed.requests) {
         const rawBuckets = { keep_oh: 0, keep_prod: 0, opt_dup: 0, opt_refind: 0, opt_dead: 0 };
         const buckets = { duplicate: 0, refind: 0, dead: 0, unattributed: 0 };
         const problemStats = new Map();
-        const occurrences = [];
-        const priorItems = parsed.items
-            .filter((item) => item.requestSeq < request.seq)
-            .flatMap((item) => {
+        // Episode rendering only needs totals by episode/problem/bucket. Aggregating here avoids
+        // retaining one occurrence for every prior item on every request (quadratic heap growth).
+        const occurrenceMap = new Map();
+        const priorItems = [];
+        let usefulItemTokens = 0;
+        let wasteItemTokens = 0;
+        for (const item of parsed.items) {
+            if (item.requestSeq >= request.seq)
+                continue;
             const classification = classForRequest(item, request);
-            return classification ? [{ item, classification }] : [];
-        });
+            if (!classification)
+                continue;
+            priorItems.push({ item, classification });
+            if (classification.kind === "useful")
+                usefulItemTokens += item.tokens;
+            else
+                wasteItemTokens += item.tokens;
+        }
         const overhead = Math.min(request.inputTokens, parsed.overheadTokens);
         const productBudget = Math.max(0, request.inputTokens - overhead);
-        const usefulItemTokens = priorItems
-            .filter(({ classification }) => classification.kind === "useful")
-            .reduce((sum, { item }) => sum + item.tokens, 0);
-        const wasteItemTokens = priorItems
-            .filter(({ classification }) => classification.kind === "waste")
-            .reduce((sum, { item }) => sum + item.tokens, 0);
         const totalObserved = usefulItemTokens + wasteItemTokens;
         const scale = totalObserved > productBudget && totalObserved > 0 ? productBudget / totalObserved : 1;
         const keepProd = usefulItemTokens * scale;
@@ -413,7 +426,7 @@ function scoreClaudeNativeSession(parsed) {
                 });
             }
             problemStats.set(classification.problemId, stat);
-            occurrences.push({
+            recordNativeOccurrence(occurrenceMap, {
                 episode: request.episode,
                 problemId: classification.problemId,
                 bucket: classification.bucket,
@@ -437,7 +450,7 @@ function scoreClaudeNativeSession(parsed) {
             problemStats,
             buckets,
             rawBuckets,
-            occurrences,
+            occurrences: occurrenceMap,
             requestUsefulTokens: rawUseful,
             requestWasteTokens: rawResidue,
         });
@@ -466,7 +479,7 @@ function scoreClaudeNativeSession(parsed) {
                     turnWasteTokens: Math.round(rawResidue),
                 }];
         }
-        requestRows.push({
+        terminalRequestByTurn.set(request.turn, {
             request,
             officialInputTokens: request.inputTokens,
             usefulTokens: rawUseful,
@@ -477,10 +490,11 @@ function scoreClaudeNativeSession(parsed) {
             buckets,
             rawBuckets,
             problems: problemStats,
-            occurrences,
+            occurrences: [...occurrenceMap.values()],
         });
     }
-    const selectedRows = selectTerminalRequestPerTurn(requestRows);
+    const selectedRows = [...terminalRequestByTurn.values()]
+        .sort((left, right) => left.request.turn - right.request.turn || left.request.seq - right.request.seq);
     const selected = mergeRequestAccountingRows(selectedRows);
     const zeroBuckets = { duplicate: 0, refind: 0, dead: 0, unattributed: 0 };
     const zeroRawBuckets = { keep_oh: 0, keep_prod: 0, opt_dup: 0, opt_refind: 0, opt_dead: 0 };
@@ -537,7 +551,7 @@ function allocateResidualToProblemSignals(args) {
             });
         }
         args.problemStats.set(problemId, stat);
-        args.occurrences.push({
+        recordNativeOccurrence(args.occurrences, {
             episode: args.request.episode,
             problemId,
             bucket,
@@ -548,15 +562,15 @@ function allocateResidualToProblemSignals(args) {
     }
     return allocated;
 }
-function selectTerminalRequestPerTurn(rows) {
-    const byTurn = new Map();
-    for (const row of rows) {
-        const current = byTurn.get(row.request.turn);
-        if (!current || row.request.seq > current.request.seq) {
-            byTurn.set(row.request.turn, row);
-        }
+function recordNativeOccurrence(occurrences, occurrence) {
+    const key = `${occurrence.episode}:${occurrence.problemId}:${occurrence.bucket}`;
+    const existing = occurrences.get(key);
+    if (existing) {
+        existing.tokens += occurrence.tokens;
+        existing.turn = Math.max(existing.turn, occurrence.turn);
+        return;
     }
-    return [...byTurn.values()].sort((left, right) => left.request.turn - right.request.turn || left.request.seq - right.request.seq);
+    occurrences.set(key, { ...occurrence });
 }
 function mergeRequestAccountingRows(rows) {
     const buckets = { duplicate: 0, refind: 0, dead: 0, unattributed: 0 };
@@ -1023,7 +1037,52 @@ function internalBucket(bucket) {
     return "opt_dead";
 }
 function readRows(file) {
+    const totalTurns = countClaudeNativeTurns(file);
+    const ranges = claudeTurnSampleRanges(totalTurns);
     const rows = [];
+    let turn = 0;
+    forEachJsonRow(file, (row) => {
+        if (stringValue(row.type) === "user" && !isClaudeToolResultUser(row))
+            turn += 1;
+        if (turn === 0 || turnInClaudeRanges(turn, ranges))
+            rows.push(row);
+    });
+    return rows;
+}
+function countClaudeNativeTurns(file) {
+    let turns = 0;
+    forEachJsonRow(file, (row) => {
+        if (stringValue(row.type) === "user" && !isClaudeToolResultUser(row))
+            turns += 1;
+    });
+    return turns;
+}
+export function claudeTurnSampleRanges(totalTurns) {
+    if (totalTurns <= CLAUDE_MAX_TURNS_PER_SESSION)
+        return totalTurns > 0 ? [[1, totalTurns]] : [];
+    const tailStart = totalTurns - CLAUDE_TAIL_TURNS + 1;
+    const ranges = [
+        [1, CLAUDE_HEAD_TURNS],
+        [tailStart, totalTurns],
+    ];
+    const middleStart = CLAUDE_HEAD_TURNS + 1;
+    const middleEnd = tailStart - 1;
+    const middleSpan = Math.max(0, middleEnd - middleStart + 1);
+    for (let index = 0; index < CLAUDE_MIDDLE_WINDOWS; index += 1) {
+        const segmentStart = middleStart + Math.floor((middleSpan * index) / CLAUDE_MIDDLE_WINDOWS);
+        const segmentEnd = middleStart + Math.floor((middleSpan * (index + 1)) / CLAUDE_MIDDLE_WINDOWS) - 1;
+        const segmentTurns = Math.max(0, segmentEnd - segmentStart + 1);
+        const windowTurns = Math.min(CLAUDE_MIDDLE_WINDOW_TURNS, segmentTurns);
+        const start = segmentStart + Math.floor((segmentTurns - windowTurns) / 2);
+        if (windowTurns > 0)
+            ranges.push([start, start + windowTurns - 1]);
+    }
+    return ranges.sort((left, right) => left[0] - right[0]);
+}
+function turnInClaudeRanges(turn, ranges) {
+    return ranges.some(([start, end]) => turn >= start && turn <= end);
+}
+function forEachJsonRow(file, visit) {
     const fd = fs.openSync(file, "r");
     const decoder = new StringDecoder("utf8");
     const buffer = Buffer.allocUnsafe(1 << 20);
@@ -1034,7 +1093,7 @@ function readRows(file) {
         try {
             const parsed = JSON.parse(line);
             if (isRecord(parsed))
-                rows.push(parsed);
+                visit(parsed);
         }
         catch {
             // Claude Code may leave an active final JSONL line partial.
@@ -1064,7 +1123,6 @@ function readRows(file) {
     finally {
         fs.closeSync(fd);
     }
-    return rows;
 }
 function isClaudeToolResultUser(record) {
     const content = recordValue(record.message).content;

package/dist/forensics.js

@@ -1158,9 +1158,9 @@ function invalidSetupReport(code, message) {
 /**
  * Fail-closed boundary between the local scanner and setup UI.
  *
- * The setup page must never infer that a fixture, partial scan, or internally inconsistent token
- * ledger is the user's report. This validator checks the provenance and the cross-ledger identities
- * the UI relies on before any absolute token number is rendered.
+ * The setup page must never infer that a fixture, unreconciled scan, or internally inconsistent
+ * token ledger is the user's report. This validator checks provenance and the cross-ledger
+ * identities the UI relies on before any absolute token number is rendered.
  */
 export function validateForensicReportForSetup(value) {
     const report = recordValue(value);
@@ -1268,6 +1268,7 @@ export function validateForensicReportForSetup(value) {
         return invalidSetupReport("REPORT_COST_INVALID", "Per-workspace costs do not reconcile with total spend.");
     }
     const diagnostics = canonical.diagnostics === undefined ? null : recordValue(canonical.diagnostics);
+    let skippedCanonicalSessions = 0;
     if (canonical.diagnostics !== undefined && !diagnostics) {
         return invalidSetupReport("REPORT_MALFORMED", "Canonical scan diagnostics are malformed.");
     }
@@ -1277,12 +1278,15 @@ export function validateForensicReportForSetup(value) {
         if (skippedSessions === null || !Array.isArray(errors)) {
             return invalidSetupReport("REPORT_MALFORMED", "Canonical scan diagnostics are malformed.");
         }
-        if (skippedSessions > 0 || errors.length > 0) {
-            return invalidSetupReport("REPORT_PARTIAL", "Some local sessions could not be analyzed; no partial report was shown.");
+        if (skippedSessions !== errors.length) {
+            return invalidSetupReport("REPORT_MALFORMED", "Canonical scan diagnostics do not reconcile.");
         }
+        skippedCanonicalSessions = skippedSessions;
     }
-    if (sessionCount !== canonicalSessionCount) {
-        return invalidSetupReport("REPORT_COHORT_MISMATCH", "Provider and canonical ledgers cover different session cohorts.");
+    if (canonicalSessionCount > sessionCount ||
+        canonicalSessionCount + skippedCanonicalSessions !== sessionCount ||
+        (sessionCount > 0 && canonicalSessionCount === 0)) {
+        return invalidSetupReport("REPORT_COHORT_MISMATCH", "Canonical analysis did not retain a usable local-session cohort.");
     }
     if (canonicalUseful + canonicalWaste !== canonicalInput) {
         return invalidSetupReport("REPORT_CANONICAL_INVALID", "Canonical useful and waste totals do not reconcile.");
@@ -1517,8 +1521,7 @@ export async function buildForensicReport(opts) {
     }
     if (report.canonicalGoldenStandard &&
         !("error" in report.canonicalGoldenStandard) &&
-        (report.canonicalGoldenStandard.diagnostics?.skippedSessions ?? 0) === 0 &&
-        report.scale.sessionCount === report.canonicalGoldenStandard.summary.sessionsAnalyzed) {
+        report.canonicalGoldenStandard.summary.sessionsAnalyzed > 0) {
         const projection = projectCanonicalWasteToBilledInput(Number(report.scale.totalInputTokens || 0), report.canonicalGoldenStandard.summary);
         if (projection)
             report.billedWasteProjection = projection;