ei-tui 1.0.1 → 1.1.0

package/README.md

@@ -193,7 +193,9 @@ Personas can use tools. Not just read-from-memory tools — *actual* tools. Web
 
 | Tool | What it does |
 |------|-------------|
-| `read_memory` | Semantic search of your personal memory — facts, traits, topics, people, quotes. Personas call this automatically when the conversation touches something they might know about you. Supports the `persona` filter to scope results to what a specific persona has learned. |
+| `find_memory` | Semantic search of your personal memory — facts, traits, topics, people, quotes. Personas call this automatically when the conversation touches something they might know about you. Supports the `persona` filter to scope results to what a specific persona has learned. |
+| `fetch_memory` | Full-record lookup for a specific human entity (Fact, Topic, Person, or Quote) by ID. Use after `find_memory` to retrieve complete details. |
+| `fetch_message` | Retrieve a specific message by ID with optional surrounding context. Searches persona conversations and room messages. |
 | `file_read` | Read a file from your local filesystem *(TUI only)* |
 | `list_directory` | Explore folder structure *(TUI only)* |
 | `directory_tree` | Recursive directory tree *(TUI only)* |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ei-tui",
-  "version": "1.0.1",
+  "version": "1.1.0",
   "author": "Flare576",
   "repository": {
     "type": "git",
@@ -63,6 +63,8 @@
     "test:evals:rewrite-real-data": "vite-node tests/evals/rewrite-real-data.eval.ts",
     "test:evals:topic-validate": "vite-node tests/evals/topic-validate.eval.ts",
     "test:evals:person-scan": "vite-node tests/evals/person-scan.eval.ts",
+    "test:evals:person-scan-confidence": "vite-node tests/evals/person-scan-confidence.eval.ts",
+
     "test:evals:person-update": "vite-node tests/evals/person-update.eval.ts",
     "test:evals:persona-trait": "vite-node tests/evals/persona-trait-extraction.eval.ts",
     "test:evals:dedup": "vite-node tests/evals/dedup-tool-calls.eval.ts",

package/src/cli/README.md

@@ -37,13 +37,29 @@ ei "memory leak" | jq '.[0].id' | ei --id
 ei --install
 ```
 
-This registers Ei with every supported agent environment it detects:
-
-- **OpenCode**: writes `~/.config/opencode/tools/ei.ts`
-- **Claude Code**: runs `claude mcp add` (or writes `~/.claude.json` as fallback)
-- **Cursor**: writes `~/.cursor/mcp.json`
+This registers Ei with Claude Code and Cursor via MCP:
+
+- **Claude Code**: writes `~/.claude.json` with an MCP server entry
+- **Cursor**: writes `~/.cursor/mcp.json` with an MCP server entry
+
+**OpenCode**: add the MCP server manually to `~/.config/opencode/opencode.jsonc`:
+
+```json
+{
+  "mcp": {
+    "ei": {
+      "type": "local",
+      "command": ["bunx", "ei-tui", "mcp"],
+      "enabled": true,
+      "environment": {
+        "EI_DATA_PATH": "/path/to/your/ei/data"
+      }
+    }
+  }
+}
+```
 
-Restart your agent tool after running to activate.
+Restart your agent tool after changes to activate.
 
 ### MCP Server
 
@@ -123,23 +139,35 @@ conversations (facts, people, topics, quotes, personas).
 
 **How to use:**
 1. Call `ei_search` (server `user-ei`) with a natural-language query (or omit query and use `recent: true` to browse); optionally filter by `type` (facts, people, topics, quotes, personas) or `persona` display_name.
-2. If you need full detail for a result, call `ei_lookup` with the entity `id` from step 1.
+2. If you need full detail for a human entity (fact, topic, person, quote), call `ei_fetch_memory` with the entity `id`.
+3. If you need full detail for a result including personas, call `ei_lookup` with the entity `id` from step 1.
+4. To fetch a specific message with surrounding context, call `ei_fetch_message` with the message `id` and optional `before`/`after` counts.
 
 Prefer querying Ei before asking the user for context they may have already shared.
 ```
 
-## What the Tool Provides
+## MCP Tools Reference
+
+The MCP server exposes these tools to Claude Code, Cursor, and OpenCode:
+
+| Tool | Description |
+|------|-------------|
+| `ei_search` | Balanced search across all five data types (facts, topics, people, quotes, personas). Supports `type`, `persona`, `source`, `recent`, `limit` filters. |
+| `ei_lookup` | Full-record lookup for any entity by ID (facts, topics, people, quotes, personas). |
+| `ei_find_memory` | Grouped human-data search — facts, topics, people, quotes. Returns results grouped by type. Mirrors the persona `find_memory` tool interface. |
+| `ei_fetch_memory` | Full-record lookup for a human entity (Fact, Topic, Person, or Quote) by ID. Returns the complete record including all fields. |
+| `ei_fetch_message` | Retrieve a specific message by ID with optional `before`/`after` context window. Searches persona conversations and room messages. |
 
-The installed tool gives OpenCode agents access to all five data types with proper Zod-validated args:
+### `ei_search` / `ei_find_memory` arguments
 
 | Arg | Type | Description |
 |-----|------|-------------|
-| `query` | string (optional) | Search text, or entity ID when `lookup=true`. Omit to browse by recency. |
-| `persona` | string (optional) | Persona display_name to filter results — only returns entities that persona has extracted |
-| `type` | enum (optional) | `facts` \| `people` \| `topics` \| `quotes` \| `personas` — omit for balanced results |
+| `query` | string (optional) | Search text. Omit to browse by recency. |
+| `persona` | string (optional) | Persona display_name to scope results to what that persona has learned |
+| `type` | enum (optional, `ei_search` only) | `facts` \| `people` \| `topics` \| `quotes` \| `personas` — omit for balanced results |
+| `types` | array (optional, `ei_find_memory` only) | `["facts", "topics", "people", "quotes"]` — omit for all human types |
 | `limit` | number (optional) | Max results, default 10 |
-| `lookup` | boolean (optional) | If true, fetch single entity by ID |
-| `recent` | boolean (optional) | If true, sort by most recently mentioned. Can be combined with `persona` or `query`. |
+| `recent` | boolean (optional) | Sort by most recently mentioned instead of relevance |
 
 ## Output Shapes
 

package/src/cli/mcp.ts

@@ -3,6 +3,8 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"
 import { z } from "zod";
 import { retrieveBalanced, lookupById, loadLatestState, type BalancedResult } from "./retrieval.js";
 import type { StorageState } from "../core/types.js";
+import type { Message } from "../core/types.js";
+import type { RoomMessage } from "../core/types/rooms.js";
 import { resolvePersonaId, filterByPersona, filterTypeSpecificByPersona, filterBySource, filterTypeSpecificBySource } from "./persona-filter.js";
 
 // Exported so tests can inject their own transport
@@ -128,6 +130,241 @@ export function createMcpServer(): McpServer {
     }
   );
 
+  server.registerTool(
+    "ei_find_memory",
+    {
+      description:
+        "Search Ei's persistent knowledge base — facts, topics, people, and quotes learned across ALL conversations over time. Use when you need context about the user, their life, relationships, or interests that may not be visible in the current exchange. Returns results grouped by type. Use `recent: true` to retrieve what's been discussed recently.",
+      inputSchema: {
+        query: z
+          .string()
+          .optional()
+          .describe(
+            "What to search for — a person, topic, fact, or anything Ei has learned about the user. Omit with recent: true to browse recent items."
+          ),
+        types: z
+          .array(z.enum(["facts", "topics", "people", "quotes"]))
+          .optional()
+          .describe("Limit search to specific memory types (default: all types)"),
+        limit: z
+          .number()
+          .optional()
+          .default(10)
+          .describe("Max results per type to return (default: 10, max: 20)"),
+        recent: z
+          .boolean()
+          .optional()
+          .describe(
+            "If true, return recently-mentioned results sorted by last_mentioned date instead of relevance. Combine with a query to filter recent results by topic."
+          ),
+        persona: z
+          .string()
+          .optional()
+          .describe(
+            "Filter results to what a specific persona has learned. Use the persona display name."
+          ),
+      },
+    },
+    async ({ query: rawQuery, types, limit, recent, persona }) => {
+      const query = rawQuery ?? "";
+      const effectiveLimit = Math.min(limit ?? 10, 20);
+      const options = { recent: recent ?? false };
+
+      const humanTypes = ["facts", "topics", "people", "quotes"] as const;
+      type HumanType = (typeof humanTypes)[number];
+      const requestedTypes: HumanType[] =
+        types && types.length > 0
+          ? (types as HumanType[])
+          : [...humanTypes];
+
+      let state: StorageState | null = null;
+      let personaId: string | undefined;
+      if (persona) {
+        state = await loadLatestState();
+        if (state) {
+          personaId = resolvePersonaId(state, persona) ?? undefined;
+          if (!personaId) {
+            return {
+              content: [{ type: "text" as const, text: `Persona "${persona}" not found.` }],
+            };
+          }
+        }
+      }
+
+      const grouped: Record<string, unknown[]> = {};
+      for (const t of requestedTypes) {
+        const module = await import(`./commands/${t}.js`);
+        let results = await (
+          module.execute as (
+            q: string,
+            l: number,
+            o: { recent: boolean }
+          ) => Promise<{ id: string }[]>
+        )(query, effectiveLimit, options);
+        if (personaId && state) {
+          results = filterTypeSpecificByPersona(results, state, personaId, t);
+        }
+        if (results.length > 0) {
+          grouped[t] = results;
+        }
+      }
+
+      if (Object.keys(grouped).length === 0) {
+        return {
+          content: [
+            {
+              type: "text" as const,
+              text: JSON.stringify({ result: "No relevant memories found for this query." }),
+            },
+          ],
+        };
+      }
+
+      return {
+        content: [{ type: "text" as const, text: JSON.stringify(grouped, null, 2) }],
+      };
+    }
+  );
+
+  server.registerTool(
+    "ei_fetch_memory",
+    {
+      description:
+        "Retrieve the full record for a specific memory by its ID. Use when ei_find_memory or ei_search returns an item and you need its complete details. Returns the full Fact, Topic, Person, or Quote record.",
+      inputSchema: {
+        id: z.string().describe("The ID of the memory record to retrieve"),
+      },
+    },
+    async ({ id }) => {
+      const result = await lookupById(id);
+
+      if (result === null || result.type === "persona") {
+        return {
+          content: [{ type: "text" as const, text: `No memory record found with ID: ${id}` }],
+        };
+      }
+
+      return {
+        content: [{ type: "text" as const, text: JSON.stringify(result, null, 2) }],
+      };
+    }
+  );
+
+  server.registerTool(
+    "ei_fetch_message",
+    {
+      description:
+        "Retrieve a specific message by its ID, with optional surrounding context. Use when ei_find_memory returns a quote with a message_id and you want to read the original conversation. The 'before' and 'after' parameters return that many additional messages for context (default 0).",
+      inputSchema: {
+        id: z.string().describe("The ID of the message to retrieve"),
+        before: z
+          .number()
+          .optional()
+          .default(0)
+          .describe("Number of preceding messages to include (default 0)"),
+        after: z
+          .number()
+          .optional()
+          .default(0)
+          .describe("Number of following messages to include (default 0)"),
+      },
+    },
+    async ({ id, before: beforeCount, after: afterCount }) => {
+      const state = await loadLatestState();
+      if (!state) {
+        return {
+          content: [{ type: "text" as const, text: "No saved state found. Is EI_DATA_PATH set correctly?" }],
+        };
+      }
+
+      const beforeN = Math.max(0, Math.floor(beforeCount ?? 0));
+      const afterN = Math.max(0, Math.floor(afterCount ?? 0));
+
+      const stripPersonaMessage = (m: Message) => ({
+        id: m.id,
+        role: m.role,
+        ...(m.content !== undefined ? { content: m.content } : {}),
+        ...(m.silence_reason !== undefined ? { silence_reason: m.silence_reason } : {}),
+        timestamp: m.timestamp,
+        ...(m.speaker_name !== undefined ? { speaker_name: m.speaker_name } : {}),
+      });
+
+      for (const { entity: persona, messages } of Object.values(state.personas)) {
+        const idx = messages.findIndex((m) => m.id === id);
+        if (idx === -1) continue;
+
+        const msg = messages[idx];
+        const beforeMsgs = messages.slice(Math.max(0, idx - beforeN), idx).map(stripPersonaMessage);
+        const afterMsgs = messages.slice(idx + 1, idx + 1 + afterN).map(stripPersonaMessage);
+
+        return {
+          content: [
+            {
+              type: "text" as const,
+              text: JSON.stringify(
+                { message: stripPersonaMessage(msg), before: beforeMsgs, after: afterMsgs, source: persona.display_name },
+                null,
+                2
+              ),
+            },
+          ],
+        };
+      }
+
+      const resolveRoomPersonaName = (
+        m: RoomMessage,
+        personaMap: Record<string, { entity: { display_name: string }; messages: Message[] }>
+      ): string | undefined => {
+        if (m.role !== "persona" || !m.persona_id) return undefined;
+        return personaMap[m.persona_id]?.entity.display_name;
+      };
+
+      const stripRoomMessage = (
+        m: RoomMessage,
+        personaMap: Record<string, { entity: { display_name: string }; messages: Message[] }>
+      ) => ({
+        id: m.id,
+        role: m.role,
+        ...(m.content !== undefined ? { content: m.content } : {}),
+        ...(m.silence_reason !== undefined ? { silence_reason: m.silence_reason } : {}),
+        timestamp: m.timestamp,
+        ...(resolveRoomPersonaName(m, personaMap) !== undefined
+          ? { speaker_name: resolveRoomPersonaName(m, personaMap) }
+          : {}),
+      });
+
+      for (const room of Object.values(state.rooms ?? {})) {
+        const idx = room.messages.findIndex((m) => m.id === id);
+        if (idx === -1) continue;
+
+        const msg = room.messages[idx];
+        const beforeMsgs = room.messages
+          .slice(Math.max(0, idx - beforeN), idx)
+          .map((m) => stripRoomMessage(m, state.personas));
+        const afterMsgs = room.messages
+          .slice(idx + 1, idx + 1 + afterN)
+          .map((m) => stripRoomMessage(m, state.personas));
+
+        return {
+          content: [
+            {
+              type: "text" as const,
+              text: JSON.stringify(
+                { message: stripRoomMessage(msg, state.personas), before: beforeMsgs, after: afterMsgs, source: room.display_name },
+                null,
+                2
+              ),
+            },
+          ],
+        };
+      }
+
+      return {
+        content: [{ type: "text" as const, text: `Message not found: ${id}` }],
+      };
+    }
+  );
+
   return server;
 }
 

package/src/cli.ts

@@ -68,7 +68,7 @@ Options:
   --persona, -p    Filter to entities a specific persona has learned about
   --source, -s     Filter to entities from a specific source (prefix match, e.g. "cursor", "opencode:my-machine", "opencode:my-machine:ses_abc123")
   --id             Look up entity by ID (accepts value or stdin)
-  --install        Register Ei with OpenCode, Claude Code, and Cursor
+  --install        Register Ei with Claude Code and Cursor via MCP
   --help, -h       Show this help message
 
 Examples:
@@ -85,50 +85,9 @@ Examples:
 }
 
 
-async function installOpenCodeMcp(): Promise<void> {
-  const home = process.env.HOME || "~";
-  const opencodeDir = join(home, ".config", "opencode");
-  const opencodeJsoncPath = join(opencodeDir, "opencode.jsonc");
-
-  const eiDataPath = process.env.EI_DATA_PATH ?? (() => {
-    const xdgData = process.env.XDG_DATA_HOME || join(home, ".local", "share");
-    return join(xdgData, "ei");
-  })();
-
-  const mcpEntry = {
-    type: "local",
-    command: ["bunx", "ei-tui", "mcp"],
-    enabled: true,
-    environment: {
-      EI_DATA_PATH: eiDataPath,
-    },
-  };
-
-  let config: Record<string, unknown> = {};
-  try {
-    const rawText = await Bun.file(opencodeJsoncPath).text();
-    // Strip // line comments before parsing — opencode.jsonc uses line comments only
-    const stripped = rawText
-      .split("\n")
-      .map(line => line.replace(/\/\/.*$/, ""))
-      .join("\n");
-    config = JSON.parse(stripped) as Record<string, unknown>;
-  } catch {
-    // File doesn't exist or isn't valid — start fresh
-  }
-
-  const mcp = (config.mcp ?? {}) as Record<string, unknown>;
-  mcp["ei"] = mcpEntry;
-  config.mcp = mcp;
-
-  await Bun.$`mkdir -p ${opencodeDir}`;
-  const tmpPath = `${opencodeJsoncPath}.ei-install.tmp`;
-  await Bun.write(tmpPath, JSON.stringify(config, null, 2) + "\n");
-  const { rename } = await import(/* @vite-ignore */ "fs/promises");
-  await rename(tmpPath, opencodeJsoncPath);
-
-  console.log(`✓ Installed Ei MCP server to ~/.config/opencode/opencode.jsonc`);
-  console.log(`  Restart OpenCode to activate.`);
+async function installMcpClients(): Promise<void> {
+  await installClaudeCode();
+  await installCursor();
 }
 
 async function installClaudeCode(): Promise<void> {
@@ -202,11 +161,6 @@ async function installCursor(): Promise<void> {
   console.log(`  Restart Cursor to activate.`);
 }
 
-async function installMcpClients(): Promise<void> {
-  await installClaudeCode();
-  await installCursor();
-}
-
 async function main(): Promise<void> {
   const args = process.argv.slice(2);
 
@@ -228,9 +182,21 @@ async function main(): Promise<void> {
   }
 
   if (args[0] === "--install") {
-    await installOpenCodeMcp();
     await installMcpClients();
     console.log(`
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  OpenCode: add to ~/.config/opencode/opencode.jsonc
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+  "mcp": {
+    "ei": {
+      "type": "local",
+      "command": ["bunx", "ei-tui", "mcp"],
+      "enabled": true,
+      "environment": { "EI_DATA_PATH": "${process.env.EI_DATA_PATH ?? "~/.local/share/ei"}" }
+    }
+  }
+
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
   Add this to ~/.config/opencode/AGENTS.md
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

package/src/core/handlers/human-extraction.ts

@@ -310,6 +310,12 @@ export async function handleHumanPersonScan(response: LLMResponse, state: StateM
       }
     }
 
+    const confidence = typeof candidate.confidence === 'number' ? candidate.confidence : null;
+    if (confidence !== null && confidence <= 2 && !matchedPerson) {
+      console.debug(`[handleHumanPersonScan] Skipping low-confidence new person "${candidate.name}" (confidence=${confidence}, relationship_type=${candidate.relationship_type ?? 'none'})`);
+      continue;
+    }
+
     const matchResult: ItemMatchResult = { matched_guid: matchedPerson?.id ?? null };
     queuePersonUpdate(matchResult, {
       ...context,

package/src/core/handlers/human-matching.ts

@@ -85,6 +85,10 @@ export async function handleTopicUpdate(response: LLMResponse, state: StateManag
   const primaryId = personaIds[0] ?? personaId;
 
   const now = new Date().toISOString();
+  const { messages_analyze } = resolveMessageWindow(response, state);
+  const earliestMessageTimestamp = messages_analyze.length > 0
+    ? messages_analyze.reduce((a, b) => a.timestamp < b.timestamp ? a : b).timestamp
+    : now;
   const human = state.getHuman();
 
   const resolveItemId = (): string => {
@@ -144,7 +148,7 @@ export async function handleTopicUpdate(response: LLMResponse, state: StateManag
     exposure_current: calculateExposureCurrent(exposureImpact, existingTopic?.exposure_current ?? 0),
     exposure_desired: result.exposure_desired ?? 0.5,
     last_updated: now,
-    learned_on: isNewItem ? now : existingTopic?.learned_on,
+    learned_on: isNewItem ? earliestMessageTimestamp : existingTopic?.learned_on,
     last_mentioned: now,
     learned_by: isNewItem ? primaryId : existingTopic?.learned_by,
     last_changed_by: primaryId,
@@ -168,6 +172,26 @@ export async function handleTopicUpdate(response: LLMResponse, state: StateManag
   console.log(`[handleTopicUpdate] ${isNewItem ? "Created" : "Updated"} topic "${resolvedName}"`);
 }
 
+function ensureEiPersonaHasNickname(identifiers: PersonIdentifier[], state: StateManager): PersonIdentifier[] {
+  const eiPersonaId = identifiers.find(i => i.type === 'Ei Persona')?.value;
+  if (!eiPersonaId) return identifiers;
+
+  const persona = state.persona_getById(eiPersonaId);
+  if (!persona) return identifiers;
+
+  const hasNickname = identifiers.some(i => i.type === 'Nickname' && i.value === persona.display_name);
+  if (hasNickname) return identifiers;
+
+  const withoutPrimary = identifiers.map(i =>
+    i.type === 'Ei Persona' ? { ...i, is_primary: undefined } : i
+  ).map(({ is_primary, ...rest }) => is_primary ? { ...rest, is_primary } : rest);
+
+  return [
+    { type: 'Nickname', value: persona.display_name, is_primary: true as const },
+    ...withoutPrimary.map(i => i.type === 'Ei Persona' ? { type: i.type, value: i.value } : i),
+  ];
+}
+
 export async function handlePersonUpdate(response: LLMResponse, state: StateManager): Promise<void> {
   const result = response.parsed as (PersonUpdateResult & {
     identifiers?: PersonIdentifier[];
@@ -194,6 +218,10 @@ export async function handlePersonUpdate(response: LLMResponse, state: StateMana
   const primaryId = personaIds[0] ?? personaId;
 
   const now = new Date().toISOString();
+  const { messages_analyze } = resolveMessageWindow(response, state);
+  const earliestMessageTimestamp = messages_analyze.length > 0
+    ? messages_analyze.reduce((a, b) => a.timestamp < b.timestamp ? a : b).timestamp
+    : now;
   const human = state.getHuman();
 
   const resolveItemId = (): string => {
@@ -264,7 +292,7 @@ export async function handlePersonUpdate(response: LLMResponse, state: StateMana
         deduped.push(id);
       }
     }
-    resolvedIdentifiers = deduped;
+    resolvedIdentifiers = ensureEiPersonaHasNickname(deduped, state);
   } else {
     const base = [...(existingPerson?.identifiers ?? [])];
     const sanitizedToAdd = sanitizeEiPersonaIdentifiers(
@@ -279,12 +307,16 @@ export async function handlePersonUpdate(response: LLMResponse, state: StateMana
         base.push({ type: id.type, value: id.value, ...(id.is_primary ? { is_primary: id.is_primary } : {}) });
       }
     }
-    resolvedIdentifiers = base;
+    resolvedIdentifiers = ensureEiPersonaHasNickname(base, state);
   }
 
+  const personName = resolvedIdentifiers.find(i => i.is_primary && i.type !== 'Ei Persona')?.value
+    ?? resolvedIdentifiers.find(i => i.type !== 'Ei Persona')?.value
+    ?? candidateName;
+
   const person: Person = {
     id: itemId,
-    name: candidateName,
+    name: personName,
     description: resolvedDescription,
     sentiment: resolvedSentiment,
     relationship: result.relationship ?? candidateRelationship ?? existingPerson?.relationship ?? "Unknown",
@@ -293,7 +325,7 @@ export async function handlePersonUpdate(response: LLMResponse, state: StateMana
     identifiers: resolvedIdentifiers,
     validated_date: isNewItem ? '' : (existingPerson?.validated_date ?? ''),
     last_updated: now,
-    learned_on: isNewItem ? now : existingPerson?.learned_on,
+    learned_on: isNewItem ? earliestMessageTimestamp : existingPerson?.learned_on,
     last_mentioned: now,
     learned_by: isNewItem ? primaryId : existingPerson?.learned_by,
     last_changed_by: primaryId,
@@ -323,14 +355,13 @@ function normalizeText(text: string): string {
     .replace(/[\u2018\u2019\u0060\u00B4]/g, "'")  // curly single, backtick, acute accent
     .replace(/[\u2014\u2013\u2012]/g, '-')         // em-dash, en-dash, figure dash
     .replace(/\u00A0/g, ' ')                       // non-breaking space
-    .replace(/[\u2000-\u200F]/g, ' ');              // unicode space variants
+    .replace(/[\u2000-\u200F]/g, ' ')              // unicode space variants
+    .replace(/[*_`~]/g, '');                       // Markdown emphasis/code chars
 }
 
 function stripPunctuation(text: string): string {
-  // Remove characters LLMs commonly mangle, keep spaces and alphanumeric
-  // Strip: punctuation, unicode punctuation variants, curly quotes, dashes, etc.
-  // Keep: letters, digits, spaces
   return text
+    .replace(/[*_`~]/g, ' ')     // Markdown chars (kept by \w, must strip explicitly)
     .replace(/[^\w\s]/gu, ' ')   // replace non-word, non-space with space
     .replace(/\s+/g, ' ')        // collapse multiple spaces
     .trim()
@@ -406,6 +437,10 @@ async function validateAndStoreQuotes(
   if (!candidates || candidates.length === 0) return;
   
   for (const candidate of candidates) {
+    if (!candidate.text) {
+      console.warn('[extraction] Skipping quote candidate with missing text field');
+      continue;
+    }
     let found = false;
     for (const message of messages) {
       const msgText = getMessageText(message);
@@ -511,7 +546,7 @@ async function validateAndStoreQuotes(
       break;
     }
     if (!found) {
-      console.warn(`[extraction] Quote not found in messages (both levels), skipping: "${candidate.text?.slice(0, 50)}..."`);
+      console.warn(`[extraction] Quote not found in messages (both levels), skipping: "${candidate.text}"`);
     }
   }
 }

package/src/core/llm-client.ts

@@ -450,6 +450,7 @@ const JSON_REPAIR_PATTERNS: Array<{ pattern: RegExp; replacement: string }> = [
   { pattern: /:\s*(\d{4}-\d{2}-\d{2}T[^"}\],\n]+)/g, replacement: ': "$1"' },
   { pattern: /:\s*0([1-9][0-9]*)([,\s\n\r\]}])/g, replacement: ": 0.$1$2" },
   { pattern: /,(\s*[\]}])/g, replacement: "$1" },
+  { pattern: /"(\s*\n[ \t]+"[a-zA-Z_][a-zA-Z0-9_]*"\s*:)/g, replacement: '",$1' },
 ];
 
 export function repairJSON(jsonStr: string): string {
@@ -529,6 +530,41 @@ export function rescueGemmaToolCalls(content: string): unknown[] {
   return rescued;
 }
 
+function findOutermostObject(str: string): string | null {
+  const start = str.indexOf('{');
+  if (start === -1) return null;
+
+  let depth = 0;
+  let inString = false;
+  let escaped = false;
+
+  for (let i = start; i < str.length; i++) {
+    const ch = str[i];
+
+    if (escaped) {
+      escaped = false;
+      continue;
+    }
+    if (ch === '\\' && inString) {
+      escaped = true;
+      continue;
+    }
+    if (ch === '"') {
+      inString = !inString;
+      continue;
+    }
+    if (inString) continue;
+
+    if (ch === '{') depth++;
+    else if (ch === '}') {
+      depth--;
+      if (depth === 0) return str.slice(start, i + 1);
+    }
+  }
+
+  return null;
+}
+
 export function parseJSONResponse(content: string): unknown {
   const jsonMatch = content.match(/```(?:json)?\s*([\s\S]*?)```/);
   const jsonStr = jsonMatch ? jsonMatch[1].trim() : content.trim();
@@ -541,10 +577,10 @@ export function parseJSONResponse(content: string): unknown {
       return JSON.parse(repaired);
     } catch {
       // Last resort: extract the outermost {...} block from mixed prose/JSON content.
-      // Handles 'thinking prose...\n{...json...}' responses from extended-thinking models.
-      const outerMatch = jsonStr.match(/\{[\s\S]*\}/);
-      if (outerMatch) {
-        const extracted = outerMatch[0];
+      // Bracket-depth scan (not greedy regex) stops at the first valid close so extra
+      // trailing braces from models like Gemma are excluded from the extracted slice.
+      const extracted = findOutermostObject(jsonStr);
+      if (extracted) {
         try {
           return JSON.parse(extracted);
         } catch {