ei-tui 0.9.4 → 1.0.0

package/README.md

@@ -83,7 +83,7 @@ Ei can operate with three types of input, and three types of output.
                        ^
                     Sessions
                        |
-                   [OpenCode]
+          [OpenCode / Claude Code / Cursor]
 ```
 
 ```
@@ -169,6 +169,22 @@ All sessions map to a single "Cursor" persona.
 
 Sessions are processed oldest-first, one per queue cycle, so Ei won't overwhelm your LLM provider on first run. See [TUI Readme](tui/README.md)
 
+## Document Import
+
+Got notes, journals, markdown files? You can feed them directly to Ei.
+
+**Web**: Open **☰ menu** → **My Data** → **Documents** tab. Drop a `.txt`, `.md`, or `.markdown` file and Ei gets to work.
+
+**TUI**:
+```bash
+/import ~/notes/my-journal.md
+/import /path/to/report.pdf
+```
+
+Ei splits the document into segments, runs them through the extraction pipeline, and pulls out facts, topics, people, and quotes — exactly like it does with your conversations. The extracted knowledge is attributed to a reserved persona called **Emmett** so it doesn't pollute your chat history.
+
+Both surfaces show you which documents have been imported and let you remove their extracted knowledge (web: Delete button in the Documents tab; TUI: `/unsource <source_tag>`).
+
 ## Built-in Tool Integrations
 
 Personas can use tools. Not just read-from-memory tools — *actual* tools. Web search. Your music. Your filesystem. Here's what ships with Ei out of the box:
@@ -184,6 +200,7 @@ Personas can use tools. Not just read-from-memory tools — *actual* tools. Web
 | `search_files` | Find files by name pattern *(TUI only)* |
 | `grep` | Search file contents by regex *(TUI only)* |
 | `get_file_info` | File/directory metadata *(TUI only)* |
+| `web_fetch` | Fetch a URL and return its text content *(TUI only — blocked by CORS in browsers)* |
 
 The filesystem tools make Ei a legitimate coding assistant in the TUI. Ask a persona to review a file, understand a project structure, or track down where something is defined — it can actually look.
 
@@ -264,13 +281,15 @@ Tag a version to publish automatically:
 
 ```bash
 # bump version in package.json
-git commit -am "chore: bump to v0.1.4"
-git tag v0.1.4
+git commit -am "chore: bump to v1.0.0"
+git tag v1.0.0
 git push && git push --tags
 ```
 
 GitHub Actions picks up the tag and publishes to npm with provenance via OIDC. No stored secrets.
 
+> **Note**: Run the pre-flight checklist in `AGENTS.md` (or use the `release` skill in OpenCode) before tagging. The v0.1.9 incident is a cautionary tale.
+
 ## Project Structure
 
 See `AGENTS.md` for detailed architecture and contribution guidelines.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ei-tui",
-  "version": "0.9.4",
+  "version": "1.0.0",
   "author": "Flare576",
   "repository": {
     "type": "git",
@@ -57,6 +57,10 @@
     "test:evals:topic-scan": "vite-node tests/evals/topic-scan.eval.ts",
     "test:evals:topic-match": "vite-node tests/evals/topic-match.eval.ts",
     "test:evals:topic-update": "vite-node tests/evals/topic-update.eval.ts",
+    "test:evals:topic-technical": "vite-node tests/evals/topic-technical.eval.ts",
+    "test:evals:rewrite-scan": "vite-node tests/evals/rewrite-scan.eval.ts",
+    "test:evals:rewrite-rewrite": "vite-node tests/evals/rewrite-rewrite.eval.ts",
+    "test:evals:rewrite-real-data": "vite-node tests/evals/rewrite-real-data.eval.ts",
     "test:evals:topic-validate": "vite-node tests/evals/topic-validate.eval.ts",
     "test:evals:person-scan": "vite-node tests/evals/person-scan.eval.ts",
     "test:evals:person-update": "vite-node tests/evals/person-update.eval.ts",

package/src/README.md

@@ -57,39 +57,23 @@ Each Topic will have an "exposure" rating similar to those on Human Data points.
 
 # Ceremony Intent
 
-Every 24 hours, we want to freshen up the system. We do this in 4 parts: Exposure, Decay, Expire, Explore
+Every 24 hours, the system runs a ceremony to keep knowledge fresh and healthy. Phases run sequentially via `ceremony_progress`:
 
-## Exposure
+**Phase 1 → Dedup**: User-triggered only (not automated). Merges confirmed duplicate records.
 
-I also frequently refer to this as "Extract," but this is the first step where we determine what the human and that Persona talked about that day. It serves two purposes:
+**Phase 2 → Expose**: Human extraction catch-up (facts, topics, people) + persona topic rating for any messages that didn't hit the per-send threshold during the day.
 
-### Detail Extraction
+**Phase 3 → EventSummary**: Summarizes significant events from recent conversations.
 
-Since we also pull out details during normal discourse (see above), this is the less-important step at this point, but still vital for catching up with the last few messages, or Personas that only received a few messages during the day and may not have hit the current limit for natural extraction.
+**Decay** (synchronous after Phase 3): Applies exposure decay to persona topics + prunes old messages. Human ceremony (decay for human topics/people) runs here too.
 
-### Exposure Adjustment
+**Phase 4 → Person Rewrite**: Scans bloated Person records (>750 chars) and extracts non-relationship content into Topics. Gated so Topic Rewrite can snapshot the updated Topic list afterward.
 
-Exposure is calculated by two metrics - `desired` and `current`. If an entity REALLY likes talking about a subject, their `desired` will be very high (1.0 max), ranging down to 0.0 for subjects which that entity does NOT wish to discuss. You may have guessed already, but `current` is how much they've recently talked about a topic.
+**Topic Rewrite** (fire-and-forget after Phase 4): Scans bloated Topic records (>750 chars) and splits them into focused sub-topics. Topics created by Person Rewrite are included.
 
-Adjusting the values is different for Human Topics/People than Persona Topics. The Human subjects are actually adjusted during the previous step, while extracting details.
+**Reflection** (fire-and-forget alongside Phase 4): Persona-side critic pass on person records.
 
-The Persona Topic update only happens during the Ceremony, and really this step only increases exposure IF the subject was discussed, bumping the last_updated field accordingly.
-
-## Decay
-
-After we determine if topics were discussed (increasing exposure), we adjust exposure the _other_ way. Based on some heuristics (like current level, desired level, and time-since-discussion), we decrease the current exposure levels down.
-
-## Expire
-
-This and the following step (Explore) are exclusive to Persona Topics right now. In Expire, we analyze the Person Topics to determine if any of them have
-- Lost their meaning to the Persona
-- Been ignored or dismissed by the user
-
-This is largely tracked by exposure, but expiration is dictated by an Agent.
-
-## Explore
-
-After we've removed irrelevant topics, this is the Agent's opportunity to add NEW topics that might be of interest to the Persona (and the user). Again, it's a prompt to an agent if the Persona doesn't have its full capacity of Topics.
+> **Note**: Expire and Explore phases were removed in the Persona Ceremony Simplification (2026-04-05). Persona topics now only update `exposure_current` during ceremony. See CONTRACTS.md changelog for details.
 
 # Opencode Importer
 

package/src/core/handlers/document-segmentation.ts

@@ -0,0 +1,113 @@
+import { ContextStatus } from "../types.js";
+import type { LLMResponse, Message } from "../types.js";
+import type { StateManager } from "../state-manager.js";
+import {
+  queueAllScans,
+  type ExtractionContext,
+} from "../orchestrators/human-extraction.js";
+
+function parseSegmentArray(content: string): string[] | null {
+  const jsonMatch = content.match(/```json\s*([\s\S]*?)```/) ?? content.match(/```\s*([\s\S]*?)```/);
+  const jsonText = jsonMatch ? jsonMatch[1].trim() : content.trim();
+
+  const arrayMatch = jsonText.match(/\[[\s\S]*\]/);
+  if (!arrayMatch) return null;
+
+  try {
+    const parsed = JSON.parse(arrayMatch[0]);
+    if (!Array.isArray(parsed)) return null;
+    return parsed.filter((item): item is string => typeof item === "string" && item.trim().length > 0);
+  } catch {
+    return null;
+  }
+}
+
+export function handleDocumentSegmentation(response: LLMResponse, state: StateManager): void {
+  const { batchId, filename, originalContent } = response.request.data as {
+    batchId: string;
+    filename: string;
+    originalContent: string;
+  };
+
+  if (!batchId || !filename) {
+    console.error("[handleDocumentSegmentation] Missing batchId or filename in request data");
+    return;
+  }
+
+  let segments: string[];
+  if (response.content) {
+    const parsed = parseSegmentArray(response.content);
+    segments = (parsed && parsed.length > 0) ? parsed : [originalContent];
+  } else {
+    segments = [originalContent];
+  }
+
+  const emmett = state.persona_getById("emmet");
+  if (!emmett) {
+    console.warn("[handleDocumentSegmentation] Emmett persona not found — skipping segment write");
+    return;
+  }
+
+  const now = new Date().toISOString();
+  const sourceTag = `import:document:${filename}`;
+
+  for (const segment of segments) {
+    const message: Message = {
+      id: crypto.randomUUID(),
+      role: "system",
+      content: segment,
+      timestamp: now,
+      read: true,
+      context_status: ContextStatus.Always,
+      external: true,
+      source_tag: sourceTag,
+    };
+    state.messages_append("emmet", message);
+  }
+
+  console.log(`[handleDocumentSegmentation] Wrote ${segments.length} segment(s) for batch ${batchId} (${filename})`);
+}
+
+export function finishDocumentBatch(batchId: string, filename: string, state: StateManager): void {
+  const sourceTag = `import:document:${filename}`;
+
+  const emmettMessages = state.messages_get("emmet");
+  const docMessages = emmettMessages.filter(m => m.external === true && m.source_tag === sourceTag);
+
+  if (docMessages.length === 0) {
+    console.warn(`[finishDocumentBatch] No messages found for ${sourceTag} — skipping extraction`);
+  } else {
+    const extractionContext: ExtractionContext = {
+      personaId: "emmet",
+      channelDisplayName: "Document",
+      messages_context: [],
+      messages_analyze: docMessages,
+      sources: [sourceTag],
+    };
+
+    const docSettings = state.getHuman().settings?.document;
+    queueAllScans(extractionContext, state, {
+      extraction_model: docSettings?.extraction_model,
+      external_filter: "only",
+    });
+
+    console.log(`[finishDocumentBatch] Queued extraction for ${docMessages.length} message(s) from ${filename}`);
+  }
+
+  const updatedHuman = state.getHuman();
+  state.setHuman({
+    ...updatedHuman,
+    settings: {
+      ...updatedHuman.settings,
+      document: {
+        ...updatedHuman.settings?.document,
+        processed_documents: {
+          ...(updatedHuman.settings?.document?.processed_documents ?? {}),
+          [filename]: new Date().toISOString(),
+        },
+      },
+    },
+  });
+
+  console.log(`[finishDocumentBatch] Batch ${batchId} complete, ${filename} marked processed`);
+}

package/src/core/handlers/index.ts

@@ -15,6 +15,7 @@ import { handleRewriteScan, handleRewriteRewrite } from "./rewrite.js";
 import { handleDedupCurate } from "./dedup.js";
 import { handleRoomResponse, handleRoomJudge } from "./rooms.js";
 import { handlePersonaPreview } from "./persona-preview.js";
+import { handleDocumentSegmentation } from "./document-segmentation.js";
 
 export const handlers: Record<LLMNextStep, ResponseHandler> = {
   handlePersonaResponse,
@@ -41,4 +42,5 @@ export const handlers: Record<LLMNextStep, ResponseHandler> = {
   handlePersonaPreview,
   [LLMNextStep.HandleTopicValidate]: handleDedupCurate,
   [LLMNextStep.HandleReflectionCritic]: handleReflectionCritic,
+  [LLMNextStep.HandleDocumentSegmentation]: handleDocumentSegmentation,
 };

package/src/core/handlers/rewrite.ts

@@ -14,7 +14,8 @@ import type {
   RewriteResult,
   RewriteSubjectMatch,
 } from "../../prompts/ceremony/types.js";
-import { buildRewritePrompt } from "../../prompts/ceremony/rewrite.js";
+import { buildPersonRewriteSplitPrompt } from "../../prompts/ceremony/people-rewrite.js";
+import { buildTopicRewriteSplitPrompt } from "../../prompts/ceremony/topic-rewrite.js";
 import { getEmbeddingService, getItemEmbeddingText } from "../embedding-service.js";
 
 import { searchHumanData } from "../human-data-manager.js";
@@ -79,12 +80,10 @@ export async function handleRewriteScan(response: LLMResponse, state: StateManag
     }
   }
 
-  // Build Phase 2 prompt and queue it
-  const prompt = buildRewritePrompt({
-    item: currentItem,
-    itemType,
-    subjects: subjectMatches,
-  });
+  const splitData = { item: currentItem, itemType, subjects: subjectMatches };
+  const prompt = itemType === "person"
+    ? buildPersonRewriteSplitPrompt(splitData)
+    : buildTopicRewriteSplitPrompt(splitData);
 
   state.queue_enqueue({
     type: LLMRequestType.JSON,
@@ -125,6 +124,11 @@ export async function handleRewriteRewrite(response: LLMResponse, state: StateMa
   const human = state.getHuman();
   const now = new Date().toISOString();
 
+  const originalItem = itemType === "topic"
+    ? human.topics.find(t => t.id === itemId)
+    : human.people.find(p => p.id === itemId);
+  const originalCategory = itemType === "topic" ? (originalItem as Topic | undefined)?.category : undefined;
+
   const allItems: DataItemBase[] = [
     ...human.topics, ...human.people,
   ];
@@ -228,11 +232,11 @@ export async function handleRewriteRewrite(response: LLMResponse, state: StateMa
     switch (item.type) {
       case "topic": {
         if (!item.category) {
-          console.warn(`[handleRewriteRewrite] New topic "${item.name}" missing category — defaulting to "Interest"`);
+          console.warn(`[handleRewriteRewrite] New topic "${item.name}" missing category — inheriting from original (${originalCategory ?? "Interest"})`);
         }
         const topic: Topic = {
           ...baseFields,
-          category: item.category ?? "Interest",
+          category: item.category ?? originalCategory ?? "Interest",
           exposure_current: 0.5,
           exposure_desired: 0.5,
         };

package/src/core/heartbeat-manager.ts

@@ -143,7 +143,7 @@ export async function queueEiHeartbeat(
   }
 
   const activePersonas = personas
-    .filter((p) => !p.is_archived && !p.is_paused && p.id !== "ei")
+    .filter((p) => !p.is_archived && !p.is_paused && !p.is_static && p.id !== "ei")
     .map((p) => {
       const msgs = sm.messages_get(p.id);
       const lastHuman = [...msgs].reverse().find((m) => m.role === "human");
@@ -169,7 +169,7 @@ export async function queueEiHeartbeat(
   }
 
   const personasWithPendingUpdate = personas.filter(
-    (p) => !p.is_archived && !p.is_paused && p.id !== "ei" && p.pending_update?.critique
+    (p) => !p.is_archived && !p.is_paused && !p.is_static && p.id !== "ei" && p.pending_update?.critique
   );
   for (const p of personasWithPendingUpdate) {
     items.push({

package/src/core/llm-client.ts

@@ -274,7 +274,17 @@ export async function callLLMRaw(
   };
 
   if (modelConfig?.thinking_budget !== undefined) {
-    requestBody.think = { budget_tokens: modelConfig.thinking_budget };
+    if (modelConfig.thinking_budget === 0) {
+      // Universal kill switch — works on Ollama, LM Studio, and all OpenAI-compat providers.
+      requestBody.reasoning_effort = "none";
+    } else {
+      // Pass both signals: providers that honor the token budget get it (Qwen3 via Ollama,
+      // Anthropic), providers that reduce thinking to on/off use reasoning_effort as the
+      // on-signal (Gemma4 via Ollama/LM Studio). Non-conflicting — each provider reads
+      // whichever field it understands.
+      requestBody.reasoning_effort = "high";
+      requestBody.think = { budget_tokens: modelConfig.thinking_budget };
+    }
   }
 
   if (options.tools && options.tools.length > 0) {

package/src/core/message-manager.ts

@@ -177,25 +177,27 @@ export async function sendMessage(
 
   const history = sm.messages_get(persona.id);
 
-  const traitExtractionData: PersonaTraitExtractionPromptData = {
-    persona_name: persona.display_name,
-    current_traits: persona.traits,
-    messages_context: history.slice(-11, -1),
-    messages_analyze: [message],
-  };
-  const traitPrompt = buildPersonaTraitExtractionPrompt(traitExtractionData);
-
-  sm.queue_enqueue({
-    type: LLMRequestType.JSON,
-    priority: LLMPriority.Low,
-    system: traitPrompt.system,
-    user: traitPrompt.user,
-    next_step: LLMNextStep.HandlePersonaTraitExtraction,
-    model: getModelForPersona(persona.id),
-    data: { personaId: persona.id, personaDisplayName: persona.display_name },
-  });
+  if (!persona.is_static) {
+    const traitExtractionData: PersonaTraitExtractionPromptData = {
+      persona_name: persona.display_name,
+      current_traits: persona.traits,
+      messages_context: history.slice(-11, -1),
+      messages_analyze: [message],
+    };
+    const traitPrompt = buildPersonaTraitExtractionPrompt(traitExtractionData);
+
+    sm.queue_enqueue({
+      type: LLMRequestType.JSON,
+      priority: LLMPriority.Low,
+      system: traitPrompt.system,
+      user: traitPrompt.user,
+      next_step: LLMNextStep.HandlePersonaTraitExtraction,
+      model: getModelForPersona(persona.id),
+      data: { personaId: persona.id, personaDisplayName: persona.display_name },
+    });
 
-  checkAndQueueHumanExtraction(sm, persona.id, persona.display_name, history);
+    checkAndQueueHumanExtraction(sm, persona.id, persona.display_name, history);
+  }
 }
 
 // =============================================================================

package/src/core/orchestrators/ceremony.ts

@@ -1,4 +1,4 @@
-import { LLMRequestType, LLMPriority, LLMNextStep, RoomMode, ContextStatus, type CeremonyConfig, type PersonaTopic, type Topic, type DataItemBase } from "../types.js";
+import { LLMRequestType, LLMPriority, LLMNextStep, RoomMode, ContextStatus, type CeremonyConfig, type PersonaTopic, type Topic } from "../types.js";
 import type { StateManager } from "../state-manager.js";
 import { normalizeRoomMessages } from "../handlers/utils.js";
 import { applyDecayToValue } from "../utils/index.js";
@@ -12,7 +12,9 @@ import {
 } from "./human-extraction.js";
 import { queuePersonaTopicRating, type PersonaTopicContext, type PersonaTopicOptions } from "./persona-topics.js";
 import { getRoomVisibleMessages, queueRoomHumanExtraction } from "./room-extraction.js";
-import { buildRewriteScanPrompt, type RewriteItemType } from "../../prompts/ceremony/index.js";
+import { type RewriteItemType } from "../../prompts/ceremony/index.js";
+import { buildPersonRewriteScanPrompt } from "../../prompts/ceremony/people-rewrite.js";
+import { buildTopicRewriteScanPrompt } from "../../prompts/ceremony/topic-rewrite.js";
 import { buildReflectionCriticPrompt } from "../../prompts/reflection/index.js";
 import { getModelForPersona } from "../heartbeat-manager.js";
 
@@ -51,7 +53,7 @@ export function shouldStartCeremony(config: CeremonyConfig, state: StateManager,
  * Start the ceremony by queuing Exposure scans for all active personas with recent activity.
  * 
  * IMPORTANT: Sets last_ceremony FIRST to prevent re-triggering from the processor loop.
- * The actual Decay → Prune → Expire → Explore phases happen later via handleCeremonyProgress
+ * The actual Decay → Person Rewrite → Topic Rewrite phases happen later via handleCeremonyProgress
  * once all exposure scans have completed.
  */
 export function startCeremony(state: StateManager): void {
@@ -167,7 +169,7 @@ function queueExposurePhase(personaId: string, state: StateManager, options?: Ex
  * AND at the end of startCeremony (for the zero-messages edge case).
  * 
  * If any ceremony_progress items remain in the queue, does nothing — more work pending.
- * Phase 1: Dedup → Phase 2: Expose → Phase 3: EventSummary → Decay → Expire
+ * Phase 1: Dedup → Phase 2: Expose → Phase 3: EventSummary → Decay → Phase 4: Person Rewrite → Topic Rewrite (fire-and-forget)
  */
 export function handleCeremonyProgress(state: StateManager, lastPhase: number): void {
   if (state.queue_hasPendingCeremonies()) {
@@ -236,6 +238,12 @@ export function handleCeremonyProgress(state: StateManager, lastPhase: number):
     return;
   }
 
+  if (lastPhase === 4) {
+    console.log("[ceremony:progress] Person Rewrite complete, starting Topic Rewrite");
+    queueTopicRewritePhase(state);
+    return;
+  }
+
   if (lastPhase === 2) {
     console.log("[ceremony:progress] Expose complete, starting EventSummary phase");
     const options: ExtractionOptions = { ceremony_progress: 3 };
@@ -249,7 +257,7 @@ export function handleCeremonyProgress(state: StateManager, lastPhase: number):
     return;
   }
   
-  // Phase 3 (EventSummary) complete → advance to Decay/Prune/Expire/Explore
+  // Phase 3 (EventSummary) complete → advance to Decay/Prune then Person Rewrite (phase 4)
   console.log("[ceremony:progress] EventSummary complete, advancing to Decay");
   
   const personas = state.persona_getAll();
@@ -276,8 +284,16 @@ export function handleCeremonyProgress(state: StateManager, lastPhase: number):
   // Human ceremony: decay topics + people
   runHumanCeremony(state);
 
-  // Rewrite phase: fire-and-forget scans for bloated human data items
-  queueRewritePhase(state);
+  // Person Rewrite phase (phase 4): scan bloated Person records, extract Topics from them.
+  // Gated via ceremony_progress so Topic Rewrite can run after — Topics created here
+  // need to be visible before Topic Rewrite snapshots the threshold.
+  queuePersonRewritePhase(state);
+
+  // Zero-work guard: if no person rewrites queued, advance to topic rewrite immediately
+  if (!state.queue_hasPendingCeremonies()) {
+    console.log("[ceremony:progress] No person rewrite work, advancing to Topic Rewrite");
+    handleCeremonyProgress(state, 4);
+  }
 
   // Reflection phase: fire-and-forget critic calls for persona person records above threshold
   queueReflectionPhase(state);
@@ -441,15 +457,6 @@ export function runHumanCeremony(state: StateManager): void {
 
 const REWRITE_DESCRIPTION_THRESHOLD = 750;
 
-/**
- * Queue Phase 1 "scan" for every human data item whose description exceeds the
- * threshold. Gated on rewrite_model being set in HumanSettings.
- * 
- * Fire-and-forget: no ceremony_progress, no blocking. Expire/Explore proceed
- * immediately since they only touch persona topics (zero overlap with human data).
- * Phase 2 items enqueue at Normal priority, naturally processing before more
- * Low-priority Phase 1 scans.
- */
 /**
  * Forces an unconditional, threshold-bypassing Person scan on Apply/Dismiss.
  * Cannot be replaced by checkAndQueueHumanExtraction — that function gates on
@@ -479,41 +486,77 @@ export function queueReflectionDrain(personaId: string, state: StateManager): vo
   console.log(`[reflection:drain] Queued Person scan for ${persona.display_name} (${unextractedPeople.length} messages) — clears on completion`);
 }
 
-export function queueRewritePhase(state: StateManager): void {
-  const human = state.getHuman();
-  const rewriteModel = human.settings?.rewrite_model;
+function getRewriteModel(state: StateManager): string | undefined {
+  return state.getHuman().settings?.rewrite_model;
+}
 
+export function queuePersonRewritePhase(state: StateManager): void {
+  const rewriteModel = getRewriteModel(state);
   if (!rewriteModel) {
-    console.log("[ceremony:rewrite] rewrite_model not set — skipping rewrite phase");
+    console.log("[ceremony:rewrite] rewrite_model not set — skipping person rewrite phase");
     return;
   }
 
-  const itemsToScan: Array<{ item: DataItemBase; type: RewriteItemType }> = [];
-
-  for (const topic of human.topics) {
-    if ((topic.description?.length ?? 0) > REWRITE_DESCRIPTION_THRESHOLD && !topic.rewrite_checked) {
-      itemsToScan.push({ item: topic, type: "topic" });
-    }
-  }
-  for (const person of human.people) {
+  const human = state.getHuman();
+  const personsToScan = human.people.filter(person => {
     const isPersonaLinked = (person.identifiers ?? []).some(
       i => i.type.toLowerCase() === 'ei persona'
     );
-    if (!isPersonaLinked && (person.description?.length ?? 0) > REWRITE_DESCRIPTION_THRESHOLD && !person.rewrite_checked) {
-      itemsToScan.push({ item: person, type: "person" });
-    }
+    return !isPersonaLinked
+      && (person.description?.length ?? 0) > REWRITE_DESCRIPTION_THRESHOLD
+      && !person.rewrite_checked;
+  });
+
+  if (personsToScan.length === 0) {
+    console.log("[ceremony:rewrite] No persons above threshold — skipping person rewrite phase");
+    return;
+  }
+
+  console.log(`[ceremony:rewrite] Found ${personsToScan.length} person(s) above ${REWRITE_DESCRIPTION_THRESHOLD} chars — queueing person rewrite scans`);
+
+  for (const person of personsToScan) {
+    const prompt = buildPersonRewriteScanPrompt({ item: person, itemType: "person" });
+    state.queue_enqueue({
+      type: LLMRequestType.JSON,
+      priority: LLMPriority.Low,
+      system: prompt.system,
+      user: prompt.user,
+      next_step: LLMNextStep.HandleRewriteScan,
+      model: rewriteModel,
+      data: {
+        itemId: person.id,
+        itemType: "person" as RewriteItemType,
+        rewriteModel,
+        ceremony_progress: 4,
+      },
+    });
   }
 
-  if (itemsToScan.length === 0) {
-    console.log("[ceremony:rewrite] No items above threshold — nothing to rewrite");
+  console.log(`[ceremony:rewrite] Queued ${personsToScan.length} person rewrite scan(s)`);
+}
+
+export function queueTopicRewritePhase(state: StateManager): void {
+  const rewriteModel = getRewriteModel(state);
+  if (!rewriteModel) {
+    console.log("[ceremony:rewrite] rewrite_model not set — skipping topic rewrite phase");
     return;
   }
 
-  console.log(`[ceremony:rewrite] Found ${itemsToScan.length} item(s) above ${REWRITE_DESCRIPTION_THRESHOLD} chars — queueing Phase 1 scans`);
+  const human = state.getHuman();
+  const topicsToScan = human.topics.filter(topic =>
+    (topic.description?.length ?? 0) > REWRITE_DESCRIPTION_THRESHOLD
+    && !topic.rewrite_checked
+  );
+
+  if (topicsToScan.length === 0) {
+    console.log("[ceremony:rewrite] No topics above threshold — skipping topic rewrite phase");
+    return;
+  }
 
-  for (const { item, type } of itemsToScan) {
-    const prompt = buildRewriteScanPrompt({ item, itemType: type });
+  console.log(`[ceremony:rewrite] Found ${topicsToScan.length} topic(s) above ${REWRITE_DESCRIPTION_THRESHOLD} chars — queueing topic rewrite scans`);
 
+  for (const topic of topicsToScan) {
+    const prompt = buildTopicRewriteScanPrompt({ item: topic, itemType: "topic" });
     state.queue_enqueue({
       type: LLMRequestType.JSON,
       priority: LLMPriority.Low,
@@ -522,14 +565,14 @@ export function queueRewritePhase(state: StateManager): void {
       next_step: LLMNextStep.HandleRewriteScan,
       model: rewriteModel,
       data: {
-        itemId: item.id,
-        itemType: type,
-        rewriteModel,  // pass through so Phase 1 handler can queue Phase 2 with the same model
+        itemId: topic.id,
+        itemType: "topic" as RewriteItemType,
+        rewriteModel,
       },
     });
   }
 
-  console.log(`[ceremony:rewrite] Queued ${itemsToScan.length} Phase 1 scan(s) at Low priority`);
+  console.log(`[ceremony:rewrite] Queued ${topicsToScan.length} topic rewrite scan(s)`);
 }
 
 function queueEventSummaryForAll(state: StateManager, options?: ExtractionOptions): void {

package/src/core/orchestrators/human-extraction.ts

@@ -164,6 +164,7 @@ export function queueTopicScan(context: ExtractionContext, state: StateManager,
       messages_context: chunk.messages_context,
       messages_analyze: chunk.messages_analyze,
       participant_context: buildParticipantContext(context.personaId, state),
+      technical_context: (context.sources?.length ?? 0) > 0,
     });
 
     state.queue_enqueue({
@@ -275,6 +276,7 @@ export function queueDirectTopicUpdate(
       messages_analyze: chunk.messages_analyze,
       persona_name: chunk.channelDisplayName,
       participant_context: buildParticipantContext(context.personaId, state),
+      technical_context: (context.sources?.length ?? 0) > 0,
     });
 
     state.queue_enqueue({
@@ -291,6 +293,7 @@ export function queueDirectTopicUpdate(
         existingItemId: topic.id,
         analyze_from_timestamp: getAnalyzeFromTimestamp(chunk),
         extraction_model: extractionModel,
+        sources: context.sources,
       },
     });
   }
@@ -306,7 +309,7 @@ const EMBEDDING_MIN_SIMILARITY = 0.3;
  * Higher than EMBEDDING_MIN_SIMILARITY (0.3) because we need near-duplicates,
  * not just vague thematic overlap.
  */
-export const VALIDATE_MIN_SIMILARITY = 0.85;
+export const VALIDATE_MIN_SIMILARITY = 0.92;
 
 /**
  * Queue a topic match request using embedding-based similarity (topics only).
@@ -425,6 +428,7 @@ export function queueTopicUpdate(
       messages_analyze: chunk.messages_analyze,
       persona_name: chunk.channelDisplayName,
       participant_context: buildParticipantContext(primaryPersonaId, state),
+      technical_context: (context.sources?.length ?? 0) > 0,
     });
 
     state.queue_enqueue({