pi-observational-memory-extension 0.1.3 → 0.2.0

package/CHANGELOG.md

@@ -5,6 +5,49 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+---
+
+### 📖 How to Work with This Changelog
+
+When preparing a new release:
+1. **Always use Semantic Versioning** (`[Major.Minor.Patch]`).
+2. **Group changes under standard subheadings**:
+   - `Added` for new features.
+   - `Changed` for changes in existing functionality.
+   - `Deprecated` for soon-to-be removed features.
+   - `Removed` for now removed features.
+   - `Fixed` for any bug fixes.
+   - `Security` in case of vulnerabilities or security updates.
+3. **Keep descriptions concise, technical, and objective** (lowcase, founder-builder voice).
+4. **Link references or commit highlights** if appropriate.
+
+---
+
+## [0.2.0] - 2026-06-23
+
+### Added
+- **retrieval**: Added vector retrieval for observations before context injection.
+- **retrieval**: Added local hash/BOW retrieval as the default offline provider.
+- **retrieval**: Added configurable Gemini embedding retrieval with `gemini-embedding-001`.
+- **scope**: Added `session` and `project` memory scopes with separate durable storage paths.
+- **attachments**: Added `auto`, `on`, and `off` attachment observation modes with image and size safeguards.
+- **caveman**: Replaced the simplified terse mode with Mastra-style caveman compression instructions.
+- **dedupe**: Added exact-hash and similarity-based duplicate suppression before appending observations.
+
+### Changed
+- **state**: Migrated OM state schema to v3 for retrieval, vectors, attachment modes, and scoped storage.
+- **settings**: Extended `/om set` with `scope`, `retrieval`, `retrieval-model`, `retrieval-top-k`, and `retrieval-threshold`.
+
+### Fixed
+- **state**: Resolved JSON state file corruption caused by running regex redactions after serialization.
+- **redaction**: Refined token/secret sanitization to avoid over-redacting non-sensitive metrics (e.g., `observationTokens`).
+- **scope**: Added durable session pointer resolution, ensuring project scope selection persists across separate run processes.
+- **durability**: Added automatic `.corrupted` rename recovery for unparseable state files instead of crashing.
+
+### Tested
+- **regression**: Added tests for scope settings, attachment gates, duplicate suppression, and local retrieval.
+- **provider**: Smoke-tested Gemini embeddings against `gemini-embedding-001` with valid 3072-dimensional normalized vectors.
+
 ## [0.1.3] - 2026-06-23
 
 ### Added

package/README.md

@@ -25,9 +25,13 @@ Legacy compaction compresses raw history into a single monolithic block of text,
 - **Async Buffering:** Extracts observations in the background at regular intervals (every 20% of the threshold) so that when the main threshold is hit, the accumulated memory is swapped/activated instantly with zero LLM-latency overhead.
 - **Adaptive Thresholds:** The active message-observation threshold expands dynamically based on remaining memory capacity in the reflection pool, maximizing raw context usage safely.
 - **Recall-driven Retrieval:** Exposes an `om_recall` tool to the Actor so it can retrieve full, raw message payloads from observed history when exact code, quotes, or numbers are needed.
+- **Vector Retrieval:** Ranks relevant observations before context injection. Local hash/BOW retrieval is enabled by default; Gemini embeddings (`gemini-embedding-001`) are available through `/om set retrieval gemini`.
+- **Session or Project Scope:** Stores memory either per session head or shared project memory via `/om set scope session|project`.
+- **Attachment Observation Modes:** Supports `auto`, `on`, and `off` with image-only auto mode and size limits to avoid unsafe prompt bloat.
+- **Duplicate Suppression:** Skips exact and near-duplicate observations before appending new memory.
 - **Custom Compaction Hook:** Plugs directly into Pi's `session_before_compact` lifecycle event. Typing `/compact` or triggering auto-compaction launches the Observer/Reflector memory consolidation flow instead of Pi's legacy summary compaction.
 - **TUI Overlay Panel:** Fully custom, responsive, multi-tab overlay (`/om-memory`) in interactive mode for inspecting memory. Features tabbed navigation, smooth scroll, and precise border framing.
-- **Durable Persistence:** Serializes and loads state files safely under `.pi/om/<session-id>.json` using atomic writes and `.bak` recovery. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
+- **Durable Persistence:** Serializes and loads state files safely under `.pi/om/sessions/<session-id>.json` or `.pi/om/projects/project.json` using atomic writes and `.bak` recovery. Outputs JSON-formatted diagnostic logs to `.pi/om/debug/` for every operation.
 - **Secret Redaction:** Redacts common API keys, npm/GitHub tokens, bearer tokens, passwords, and secret fields before writing state/debug artifacts or observer text.
 - **Stale Lock Recovery:** Recovers old interrupted OM operation locks automatically so a crashed or killed process does not block future memory runs.
 - **Bounded Observer Context:** Sends only a safe tail of previous observations to the Observer, preventing recursive prompt bloat while preserving full memory in the main runtime.
@@ -36,7 +40,7 @@ Legacy compaction compresses raw history into a single monolithic block of text,
 
 ## 📋 Configuration & Settings
 
-Use `/om` in Pi to inspect and update runtime settings. Settings are persisted in the session OM state file under `.pi/om/<session-id>.json`.
+Use `/om` in Pi to inspect and update runtime settings. Settings are persisted in the OM state file under `.pi/om/sessions/<session-id>.json` or `.pi/om/projects/project.json`.
 
 ```bash
 /om
@@ -48,7 +52,13 @@ Use `/om` in Pi to inspect and update runtime settings. Settings are persisted i
 /om set observation-model google/gemini-2.5-flash
 /om set reflection-model google/gemini-2.5-flash
 /om set caveman on
-/om set attachments off
+/om set attachments auto
+/om set scope project
+/om set retrieval local
+/om set retrieval gemini
+/om set retrieval-model gemini-embedding-001
+/om set retrieval-top-k 6
+/om set retrieval-threshold 12%
 /om reset
 ```
 
@@ -56,7 +66,9 @@ Use `/om` in Pi to inspect and update runtime settings. Settings are persisted i
 - **Reflection Trigger Threshold:** default `40,000` observation tokens.
 - **Default Models:** `google/gemini-2.5-flash` with 0.3 temperature for Observer, and 0.0 temperature for Reflector.
 - **Caveman Mode:** optional terse compression style for denser memory.
-- **Attachment Observation Toggle:** controls whether image/attachment placeholders are exposed to observation text.
+- **Attachment Observation Mode:** `auto` observes small images only, `on` allows supported attachments, and `off` omits them.
+- **Scope:** `session` keeps each session isolated; `project` shares one memory file for the current project.
+- **Retrieval:** `local` is the default and works offline; `gemini` uses Gemini embeddings when `GEMINI_API_KEY` or `GOOGLE_GENERATIVE_AI_API_KEY` is available; `off` disables vector retrieval.
 
 ---
 

package/extensions/index.ts

@@ -7,12 +7,14 @@ import {
 } from "@earendil-works/pi-coding-agent";
 import { Key, matchesKey, truncateToWidth, wrapTextWithAnsi } from "@earendil-works/pi-tui";
 import { Type } from "typebox";
-import { copyFile, mkdir, readFile, rename, unlink, writeFile } from "node:fs/promises";
+import { copyFile, mkdir, readFile, readdir, rename, unlink, writeFile } from "node:fs/promises";
 import { existsSync } from "node:fs";
+import { createHash } from "node:crypto";
 import { basename, dirname, join } from "node:path";
+import { fileURLToPath } from "node:url";
 
 const EXTENSION_ID = "pi-observational-memory";
-const STATE_VERSION = 2;
+const STATE_VERSION = 3;
 const DEFAULT_OBSERVATION_MODEL = "google/gemini-2.5-flash";
 const DEFAULT_REFLECTION_MODEL = "google/gemini-2.5-flash";
 const OBSERVATION_THRESHOLD = 30_000;
@@ -28,6 +30,11 @@ const MAX_RESTART_ERRORS = 3;
 const STALE_OPERATION_LOCK_MS = 15 * 60 * 1000;
 const PREVIOUS_OBSERVATIONS_MAX_TOKENS = 2_000;
 const ATOMIC_BACKUP_SUFFIX = ".bak";
+const MAX_ATTACHMENT_OBSERVE_BYTES = 2_000_000;
+const VECTOR_TOP_K_DEFAULT = 6;
+const VECTOR_THRESHOLD_DEFAULT = 0.12;
+const PROJECT_REFERENCES_MAX_TOKENS = 2_000;
+const VECTOR_MAX_INDEX_CHUNKS = 2_000;
 
 const OBSERVATION_CONTEXT_PROMPT = `The following observations block contains your memory of past conversations with this user.`;
 const OBSERVATION_CONTEXT_INSTRUCTIONS = `IMPORTANT: When responding, reference specific details from these observations. Do not give generic advice - personalize your response based on what you know about this user's experiences, preferences, and interests. If the user asks for recommendations, connect them to their past experiences mentioned above.
@@ -52,6 +59,29 @@ Your memory is comprised of observations which may mention raw message IDs. The
 
 Use om_recall when the user asks to repeat/show/reproduce exact past content, code, quotes, error messages, URLs, file paths, or specific numbers; when observations mention something but lack detail; or when you need to verify a past event before answering. Default to recall for exact historical content. For high-level summaries, preferences, and facts already covered by observations, recall is not needed.`;
 
+const CAVEMAN_OM_INSTRUCTION = `Respond terse like smart caveman. All technical substance stay. Only fluff die.
+
+Use full caveman compression style.
+
+Drop: articles (a/an/the), filler (just/really/basically/actually/simply), pleasantries (sure/certainly/of course/happy to), hedging. Fragments OK. Short synonyms (big not extensive, fix not "implement a solution for"). Technical terms exact. Code blocks unchanged. Errors quoted exact. Leave out the words "agent" and "assistant" at the start of each observation line, it is assumed each line is referring to the assistant unless it specifically says it was about the user. Leave out parenthesis and other text characters like * that would not contribute to understanding the observations.
+
+Pattern: \`[thing] [action] [reason]. [next step]\`
+
+Not: "Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by..."
+Yes: "Bug in auth middleware. Token expiry check use < not <=. Fix:"
+
+Example 1
+🔴 14:31 user asks why React component rerenders
+🟡 14:32 saw inline object prop create new ref each render, cause rerender
+✅ 14:34 fixed render issue by wrap object in useMemo
+
+Example 2
+🟡 15:10 explained pool reuse DB connections, skip repeat handshake overhead
+
+Don't say "Agent did x", say "did x". It will be assumed the agent did what was observed. The who should only be specified for the user or other third parties: "user asked x"
+
+Drop caveman for: security warnings, irreversible action confirmations, multi-step sequences where fragment order risks misread, user asks to clarify or repeats question, and anything that requires remembering verbatim content. Resume caveman after clear part done`;
+
 const OBSERVER_EXTRACTION_INSTRUCTIONS = `CRITICAL: DISTINGUISH USER ASSERTIONS FROM QUESTIONS
 
 When the user TELLS you something about themselves, mark it as an assertion:
@@ -169,7 +199,7 @@ const OBSERVER_GUIDELINES = `- Be specific enough for the assistant to act on
 - If the user provides detailed messages or code snippets, observe all important details`;
 
 function buildObserverSystemPrompt(caveman = false): string {
-  const cavemanInstruction = caveman ? `\n\nCAVEMAN MODE: Write brutally short, dense observations. Remove filler. Preserve facts, decisions, dates, paths, and errors only.` : "";
+  const cavemanInstruction = caveman ? `\n\n${CAVEMAN_OM_INSTRUCTION}` : "";
   return `You are the memory consciousness of an AI assistant. Your observations will be the ONLY information the assistant has about past interactions with this user.
 
 Extract observations that will help the assistant remember:
@@ -217,7 +247,7 @@ function buildObserverTaskPrompt(existingObservations: string | undefined, opts:
 }
 
 function buildReflectorSystemPrompt(caveman = false): string {
-  const cavemanInstruction = caveman ? `\n\nCAVEMAN MODE: Compress aggressively. Prefer terse facts over prose. Preserve only actionable, durable memory.` : "";
+  const cavemanInstruction = caveman ? `\n\n${CAVEMAN_OM_INSTRUCTION}` : "";
   return `You are the memory consciousness of an AI assistant. Your memory observation reflections will be the ONLY information the assistant has about past interactions with this user.
 
 The following instructions were given to another part of your psyche (the observer) to create memories.
@@ -299,11 +329,20 @@ type BufferedChunk = {
   createdAt: string;
 };
 
+type ObserveAttachmentsMode = "auto" | "on" | "off";
+type RetrievalProvider = "off" | "local" | "gemini";
+
 type PiOMSettings = {
   observationModel: string;
   reflectionModel: string;
   caveman: boolean;
-  observeAttachments: boolean;
+  observeAttachments: ObserveAttachmentsMode;
+  retrieval: {
+    provider: RetrievalProvider;
+    model: string;
+    topK: number;
+    threshold: number;
+  };
 };
 
 type PiOMRecord = {
@@ -328,7 +367,25 @@ type PiOMRecord = {
   lastError?: string;
   lastProvider?: string;
   lastModel?: string;
+  vectorIndex?: VectorIndexState;
+  updatedAt: string;
+};
+
+type VectorIndexEntry = {
+  id: string;
+  hash: string;
+  text: string;
+  embedding: number[];
+  tokens: number;
+  createdAt: string;
+};
+
+type VectorIndexState = {
+  provider: RetrievalProvider;
+  model: string;
+  scopeKey: string;
   updatedAt: string;
+  entries: VectorIndexEntry[];
 };
 
 import type { SessionEntry } from "@earendil-works/pi-coding-agent";
@@ -350,6 +407,61 @@ type Runtime = {
 
 const runtime: Runtime = { failureCount: 0 };
 
+let checkedForUpdate = false;
+let updateNotification: string | undefined = undefined;
+
+function isNewerVersion(local: string, latest: string): boolean {
+  const localParts = local.split(".").map((x) => parseInt(x, 10) || 0);
+  const latestParts = latest.split(".").map((x) => parseInt(x, 10) || 0);
+  for (let i = 0; i < 3; i++) {
+    const localPart = localParts[i] ?? 0;
+    const latestPart = latestParts[i] ?? 0;
+    if (latestPart > localPart) return true;
+    if (latestPart < localPart) return false;
+  }
+  return false;
+}
+
+async function checkForUpdates(ctx: any) {
+  try {
+    const extensionDir = dirname(fileURLToPath(import.meta.url));
+    const pkgPath = join(extensionDir, "..", "package.json");
+    if (!existsSync(pkgPath)) return;
+    const pkg = JSON.parse(await readFile(pkgPath, "utf8"));
+    const localVersion = pkg.version;
+    if (!localVersion) return;
+
+    if (typeof fetch === "undefined") return;
+
+    const controller = new AbortController();
+    const timeoutId = setTimeout(() => controller.abort(), 5000);
+
+    try {
+      const res = await fetch("https://registry.npmjs.org/pi-observational-memory-extension/latest", {
+        signal: controller.signal,
+        headers: {
+          "User-Agent": `pi-observational-memory-update-checker/${localVersion}`
+        }
+      });
+      clearTimeout(timeoutId);
+
+      if (!res.ok) return;
+      const data = (await res.json()) as { version?: string };
+      const latestVersion = data.version;
+      if (!latestVersion) return;
+
+      if (isNewerVersion(localVersion, latestVersion)) {
+        updateNotification = `A new version of pi-observational-memory-extension is available: v${latestVersion} (installed: v${localVersion}). Please update using npm to get the latest features!`;
+        ctx?.ui?.notify?.(updateNotification, "warning");
+      }
+    } catch {
+      clearTimeout(timeoutId);
+    }
+  } catch {
+    // Fail silently
+  }
+}
+
 export default function (pi: ExtensionAPI) {
   pi.registerTool({
     name: "om_recall",
@@ -388,6 +500,13 @@ export default function (pi: ExtensionAPI) {
     await refreshCounts(ctx);
     updateStatus(ctx);
     runtime.statusTimer = setInterval(() => updateStatus(ctx), 1000);
+
+    if (!checkedForUpdate) {
+      checkedForUpdate = true;
+      checkForUpdates(ctx).catch(() => {});
+    } else if (updateNotification) {
+      ctx?.ui?.notify?.(updateNotification, "warning");
+    }
   });
 
   pi.on("session_shutdown", async (_event, ctx) => {
@@ -451,7 +570,10 @@ export default function (pi: ExtensionAPI) {
       .filter(isMessageLikeEntry)
       .map(entryToAgentMessage)
       .filter(Boolean) as AgentMessage[];
-    const omMessage = buildOMContextMessage(state);
+    const queryText = unobserved.map(m => formatAgentMessage(m)).join("\n\n");
+    const retrieved = await retrieveRelevantObservations(ctx, state, queryText);
+    const projectReferences = await buildProjectReferences(ctx, state);
+    const omMessage = buildOMContextMessage(state, retrieved, projectReferences);
     const finalMessages = [omMessage, ...unobserved];
     if (finalMessages.length <= 1) return { messages: [omMessage, ...event.messages.slice(-8)] };
     return { messages: finalMessages };
@@ -624,6 +746,27 @@ export default function (pi: ExtensionAPI) {
         ctx.ui.notify(formatMemoryText(state), "info");
         return;
       }
+      if (cmd === "vector") {
+        const sub = rest[0] || "status";
+        if (sub === "status") {
+          ctx.ui.notify(formatVectorStatus(state), "info");
+          return;
+        }
+        if (sub === "rebuild") {
+          await rebuildVectorIndex(ctx, state);
+          await saveState(state);
+          ctx.ui.notify(formatVectorStatus(state), "info");
+          return;
+        }
+        if (sub === "search") {
+          const query = rest.slice(1).join(" ").trim();
+          if (!query) throw new Error("Usage: /om vector search <query>");
+          const result = await retrieveRelevantObservations(ctx, state, query);
+          ctx.ui.notify(result || "No matching OM vector results.", "info");
+          return;
+        }
+        throw new Error("Unknown /om vector command. Use /om vector status|rebuild|search <query>");
+      }
       if (cmd === "set") {
         const key = rest[0];
         const value = rest.slice(1).join(" ");
@@ -724,38 +867,98 @@ async function ensureState(ctx: any): Promise<PiOMRecord> {
   const dir = join(ctx.cwd, CONFIG_DIR_NAME, "om");
   const debugDir = join(dir, "debug");
   await mkdir(debugDir, { recursive: true });
-  const statePath = join(dir, `${sanitizeFileName(sessionId)}.json`);
-  runtime.statePath = statePath;
+  const legacyStatePath = join(dir, `${sanitizeFileName(sessionId)}.json`);
   runtime.debugDir = debugDir;
   let state: PiOMRecord | undefined;
-  if (existsSync(statePath)) {
+  let statePath = legacyStatePath;
+  if (existsSync(legacyStatePath)) {
     try {
-      state = normalizeState(JSON.parse(await readFile(statePath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
+      state = normalizeState(JSON.parse(await readFile(legacyStatePath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
     } catch (error) {
-      const backupPath = `${statePath}${ATOMIC_BACKUP_SUFFIX}`;
+      const backupPath = `${legacyStatePath}${ATOMIC_BACKUP_SUFFIX}`;
       if (existsSync(backupPath)) {
         try {
           state = normalizeState(JSON.parse(await readFile(backupPath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
         } catch {
-          throw new Error(`Failed to load OM state ${statePath} and backup ${backupPath}: ${errorMessage(error)}`);
+          console.error(`[OM] Error: Failed to parse state and backup for ${legacyStatePath}. Falling back to default empty state.`);
+          try {
+            await rename(legacyStatePath, `${legacyStatePath}.corrupted`);
+            await rename(backupPath, `${backupPath}.corrupted`);
+          } catch {}
         }
       } else {
-        throw new Error(`Failed to load OM state ${statePath}: ${errorMessage(error)}`);
+        console.error(`[OM] Error: Failed to parse state for ${legacyStatePath} (no backup). Falling back to default empty state.`);
+        try {
+          await rename(legacyStatePath, `${legacyStatePath}.corrupted`);
+        } catch {}
       }
     }
   }
   if (!state) {
     state = createDefaultState({ sessionId, sessionFile, cwd: ctx.cwd });
-    await saveState(state);
   }
+  statePath = statePathFor(ctx.cwd, state.scope, sessionId);
+  if (statePath !== legacyStatePath && existsSync(statePath)) {
+    try {
+      state = normalizeState(JSON.parse(await readFile(statePath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
+    } catch {
+      const backupPath = `${statePath}${ATOMIC_BACKUP_SUFFIX}`;
+      if (existsSync(backupPath)) {
+        try {
+          state = normalizeState(JSON.parse(await readFile(backupPath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
+        } catch {
+          try {
+            await rename(statePath, `${statePath}.corrupted`);
+            await rename(backupPath, `${backupPath}.corrupted`);
+          } catch {}
+        }
+      } else {
+        try {
+          await rename(statePath, `${statePath}.corrupted`);
+        } catch {}
+      }
+    }
+  }
+  const resolvedStatePath = statePathFor(ctx.cwd, state.scope, sessionId);
+  if (resolvedStatePath !== statePath && existsSync(resolvedStatePath)) {
+    statePath = resolvedStatePath;
+    try {
+      state = normalizeState(JSON.parse(await readFile(statePath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
+    } catch {
+      const backupPath = `${statePath}${ATOMIC_BACKUP_SUFFIX}`;
+      if (existsSync(backupPath)) {
+        try {
+          state = normalizeState(JSON.parse(await readFile(backupPath, "utf8")), { sessionId, sessionFile, cwd: ctx.cwd });
+        } catch {
+          try {
+            await rename(statePath, `${statePath}.corrupted`);
+            await rename(backupPath, `${backupPath}.corrupted`);
+          } catch {}
+        }
+      } else {
+        try {
+          await rename(statePath, `${statePath}.corrupted`);
+        } catch {}
+      }
+    }
+  } else {
+    statePath = resolvedStatePath;
+  }
+  runtime.statePath = statePath;
   const recovered = recoverStaleOperationLock(state);
   runtime.state = state;
-  if (recovered || state.version !== STATE_VERSION) await saveState(state);
+  if (recovered || state.version !== STATE_VERSION || !existsSync(statePath)) await saveState(state);
   return state;
 }
 
 function defaultSettings(): PiOMSettings {
-  return { observationModel: DEFAULT_OBSERVATION_MODEL, reflectionModel: DEFAULT_REFLECTION_MODEL, caveman: false, observeAttachments: true };
+  return {
+    observationModel: DEFAULT_OBSERVATION_MODEL,
+    reflectionModel: DEFAULT_REFLECTION_MODEL,
+    caveman: false,
+    observeAttachments: "auto",
+    retrieval: { provider: "off", model: "local/hash-bow-v1", topK: VECTOR_TOP_K_DEFAULT, threshold: VECTOR_THRESHOLD_DEFAULT },
+  };
 }
 
 function defaultThresholds(): PiOMRecord["thresholds"] {
@@ -797,17 +1000,74 @@ function normalizeState(raw: any, identity: { sessionId: string; sessionFile?: s
     sessionFile: raw?.sessionFile || identity.sessionFile,
     cwd: raw?.cwd || identity.cwd,
     thresholds: { ...defaults.thresholds, ...(raw?.thresholds ?? {}) },
-    settings: { ...defaults.settings, ...(raw?.settings ?? {}) },
+    settings: normalizeSettings(raw?.settings, defaults.settings),
     buffered: { observations: [], ...(raw?.buffered ?? {}) },
   } as PiOMRecord;
   state.observations = redactSecrets(String(state.observations ?? ""));
   state.currentTask = state.currentTask ? redactSecrets(state.currentTask) : undefined;
   state.suggestedResponse = state.suggestedResponse ? redactSecrets(state.suggestedResponse) : undefined;
   state.lastError = state.lastError ? redactSecrets(state.lastError) : undefined;
+  state.scope = state.scope === "project" ? "project" : "session";
   state.observationTokens = estimateTokens(state.observations);
+  state.vectorIndex = normalizeVectorIndex(state.vectorIndex, state);
   return state;
 }
 
+function normalizeSettings(raw: any, defaults: PiOMSettings): PiOMSettings {
+  const legacyAttachments = raw?.observeAttachments;
+  let observeAttachments: ObserveAttachmentsMode = defaults.observeAttachments;
+  if (legacyAttachments === true) observeAttachments = "on";
+  else if (legacyAttachments === false) observeAttachments = "off";
+  else if (["auto", "on", "off"].includes(String(legacyAttachments))) observeAttachments = legacyAttachments;
+  const rawRetrieval = raw?.retrieval ?? {};
+  const provider = ["off", "local", "gemini"].includes(String(rawRetrieval.provider)) ? rawRetrieval.provider as RetrievalProvider : defaults.retrieval.provider;
+  return {
+    ...defaults,
+    ...raw,
+    observeAttachments,
+    retrieval: {
+      provider,
+      model: String(rawRetrieval.model || (provider === "gemini" ? "gemini-embedding-001" : defaults.retrieval.model)),
+      topK: clampInt(Number(rawRetrieval.topK ?? defaults.retrieval.topK), 1, 30),
+      threshold: clampNumber(Number(rawRetrieval.threshold ?? defaults.retrieval.threshold), 0, 1),
+    },
+  };
+}
+
+function normalizeVectorIndex(raw: any, state: PiOMRecord): VectorIndexState | undefined {
+  if (!raw || typeof raw !== "object" || !Array.isArray(raw.entries)) return undefined;
+  return {
+    provider: ["off", "local", "gemini"].includes(String(raw.provider)) ? raw.provider : state.settings.retrieval.provider,
+    model: String(raw.model || state.settings.retrieval.model),
+    scopeKey: String(raw.scopeKey || scopeKeyForState(state)),
+    updatedAt: String(raw.updatedAt || new Date().toISOString()),
+    entries: raw.entries.slice(-VECTOR_MAX_INDEX_CHUNKS).filter((e: any) => e && typeof e.text === "string" && Array.isArray(e.embedding)),
+  };
+}
+
+function clampInt(value: number, min: number, max: number): number {
+  if (!Number.isFinite(value)) return min;
+  return Math.max(min, Math.min(max, Math.round(value)));
+}
+
+function clampNumber(value: number, min: number, max: number): number {
+  if (!Number.isFinite(value)) return min;
+  return Math.max(min, Math.min(max, value));
+}
+
+function statePathFor(cwd: string, scope: PiOMRecord["scope"], sessionId: string): string {
+  const base = join(cwd, CONFIG_DIR_NAME, "om", scope === "project" ? "projects" : "sessions");
+  return join(base, `${scope === "project" ? projectHashForCwd(cwd) : sanitizeFileName(sessionId)}.json`);
+}
+
+function projectHashForCwd(cwd: string): string {
+  return createHash("sha256").update(cwd).digest("hex").slice(0, 16);
+}
+
+function scopeKeyForState(state: PiOMRecord): string {
+  return state.scope === "project" ? `project:${projectHashForCwd(state.cwd)}` : `session:${sanitizeFileName(state.sessionId)}`;
+}
+
 function isStaleOperationLock(lock: PiOMRecord["operationLock"], now = Date.now()): boolean {
   if (!lock) return false;
   const started = Date.parse(lock.startedAt);
@@ -825,21 +1085,55 @@ function recoverStaleOperationLock(state: PiOMRecord, now = Date.now()): boolean
 }
 
 async function saveState(state: PiOMRecord): Promise<void> {
+  runtime.statePath = statePathFor(state.cwd, state.scope, state.sessionId);
   if (!runtime.statePath) return;
   state.updatedAt = new Date().toISOString();
   state.observations = redactSecrets(state.observations);
   if (state.currentTask) state.currentTask = redactSecrets(state.currentTask);
   if (state.suggestedResponse) state.suggestedResponse = redactSecrets(state.suggestedResponse);
   if (state.lastError) state.lastError = redactSecrets(state.lastError);
+  if (state.settings.retrieval.provider === "local") state.vectorIndex = buildLocalVectorIndex(state);
+  await mergeExistingScopeState(runtime.statePath, state);
   await atomicWriteJson(runtime.statePath, state);
+  if (state.scope === "project") {
+    const sessionPointerPath = statePathFor(state.cwd, "session", state.sessionId);
+    const sessionPointer = { ...state, scope: "project" as const, observations: "", currentTask: undefined, suggestedResponse: undefined, vectorIndex: undefined };
+    await atomicWriteJson(sessionPointerPath, sessionPointer);
+  }
+  await saveVectorIndex(state);
   runtime.overlayHandle?.requestRender();
 }
 
+
+async function mergeExistingScopeState(filePath: string, state: PiOMRecord): Promise<void> {
+  if (!existsSync(filePath)) return;
+  try {
+    const existing = normalizeState(JSON.parse(await readFile(filePath, "utf8")), { sessionId: state.sessionId, sessionFile: state.sessionFile, cwd: state.cwd });
+    if (existing.updatedAt === state.updatedAt) return;
+    const merged = suppressDuplicateObservations(state.observations, existing.observations);
+    if (merged) state.observations = [state.observations.trim(), merged].filter(Boolean).join("\n\n");
+    state.currentTask = state.currentTask ?? existing.currentTask;
+    state.suggestedResponse = state.suggestedResponse ?? existing.suggestedResponse;
+    state.observationTokens = estimateTokens(state.observations);
+  } catch {
+    // Corrupt existing state is handled by .bak recovery during load; do not block saving current valid state.
+  }
+}
+
+async function saveVectorIndex(state: PiOMRecord): Promise<void> {
+  if (!state.vectorIndex || !runtime.statePath) return;
+  const dir = join(dirname(dirname(runtime.statePath)), "vectors");
+  await mkdir(dir, { recursive: true });
+  const file = join(dir, `${sanitizeFileName(state.vectorIndex.scopeKey)}.jsonl`);
+  const lines = state.vectorIndex.entries.map(entry => JSON.stringify(redactDeep(entry))).join("\n") + (state.vectorIndex.entries.length ? "\n" : "");
+  await writeFile(file, lines, "utf8");
+}
+
 async function atomicWriteJson(filePath: string, value: unknown): Promise<void> {
   await mkdir(dirname(filePath), { recursive: true });
   const tmpPath = `${filePath}.${process.pid}.${Date.now()}.tmp`;
   const backupPath = `${filePath}${ATOMIC_BACKUP_SUFFIX}`;
-  const json = redactSecrets(JSON.stringify(value, null, 2)) + "\n";
+  const json = JSON.stringify(redactDeep(value), null, 2) + "\n";
   await writeFile(tmpPath, json, "utf8");
   if (existsSync(filePath)) {
     try {
@@ -900,7 +1194,10 @@ async function observeNow(ctx: any, opts: { force: boolean; reason: string; sign
   if (candidates.length === 0 && !opts.manualText) return;
   const selected = opts.force ? candidates : selectEntriesForObservation(candidates, state);
   if (selected.length === 0 && !opts.manualText) return;
-  const inputText = opts.manualText ?? formatEntriesForObserver(selected);
+  const observerInput = opts.manualText
+    ? { text: opts.manualText, attachmentParts: [] as any[], omissions: [] as string[] }
+    : buildObserverInput(ctx, selected, state);
+  const inputText = observerInput.text;
   const startedAt = new Date().toISOString();
   state.operationLock = { type: "observation", startedAt };
   state.status = "observing";
@@ -910,7 +1207,7 @@ async function observeNow(ctx: any, opts: { force: boolean; reason: string; sign
   runtime.currentOperation = controller;
   const signal = mergeAbortSignals(opts.signal, controller.signal);
   try {
-    const result = await runObserver(ctx, inputText, signal, { source: opts.reason, existingObservations: state.observations });
+    const result = await runObserver(ctx, inputText, signal, { source: opts.reason, existingObservations: state.observations, attachmentParts: observerInput.attachmentParts });
     appendObservations(state, result, estimateTokens(inputText));
     const last = selected.at(-1);
     if (last?.id) state.lastObservedEntryId = last.id;
@@ -918,7 +1215,7 @@ async function observeNow(ctx: any, opts: { force: boolean; reason: string; sign
     state.operationLock = undefined;
     state.status = "idle";
     state.lastError = undefined;
-    await writeDebug(ctx, "observation", { startedAt, reason: opts.reason, inputText, rawOutput: result.rawOutput, parsed: result });
+    await writeDebug(ctx, "observation", { startedAt, reason: opts.reason, inputText, attachmentOmissions: observerInput.omissions, attachmentCount: observerInput.attachmentParts.length, rawOutput: result.rawOutput, parsed: result });
     await saveState(state);
   } catch (error) {
     state.operationLock = undefined;
@@ -943,7 +1240,8 @@ async function bufferObservation(ctx: any, reason: string): Promise<void> {
   if (tokens < state.thresholds.bufferTokens) return;
   const selected = takeEntriesUpTo(candidates, Math.min(tokens, OBSERVER_MAX_BATCH_TOKENS));
   if (selected.length === 0) return;
-  const inputText = formatEntriesForObserver(selected);
+  const observerInput = buildObserverInput(ctx, selected, state);
+  const inputText = observerInput.text;
   const startedAt = new Date().toISOString();
   state.operationLock = { type: "buffer", startedAt };
   state.status = "buffering";
@@ -952,7 +1250,7 @@ async function bufferObservation(ctx: any, reason: string): Promise<void> {
   const controller = new AbortController();
   runtime.currentOperation = controller;
   try {
-    const result = await runObserver(ctx, inputText, controller.signal, { source: reason, existingObservations: state.observations });
+    const result = await runObserver(ctx, inputText, controller.signal, { source: reason, existingObservations: state.observations, attachmentParts: observerInput.attachmentParts });
     const observations = result.observations.trim();
     if (!observations) throw new Error("Observer returned empty buffered observations");
     state.buffered.observations.push({
@@ -969,7 +1267,7 @@ async function bufferObservation(ctx: any, reason: string): Promise<void> {
     state.operationLock = undefined;
     state.status = "idle";
     state.lastOperation = { type: "buffer", startedAt, endedAt: new Date().toISOString(), inputTokens: estimateTokens(inputText), outputTokens: estimateTokens(observations), model: state.settings.observationModel || DEFAULT_OBSERVATION_MODEL };
-    await writeDebug(ctx, "buffer", { startedAt, reason, inputText, rawOutput: result.rawOutput, parsed: result });
+    await writeDebug(ctx, "buffer", { startedAt, reason, inputText, attachmentOmissions: observerInput.omissions, attachmentCount: observerInput.attachmentParts.length, rawOutput: result.rawOutput, parsed: result });
     await saveState(state);
   } catch (error) {
     state.operationLock = undefined;
@@ -1019,6 +1317,7 @@ async function activateBuffered(ctx: any, reason: string): Promise<void> {
   state.buffered.observations = chunks.slice(activated.length);
   state.pendingMessageTokens = Math.max(0, state.pendingMessageTokens - activatedTokens);
   state.observationTokens = estimateTokens(state.observations);
+  state.vectorIndex = buildLocalVectorIndex(state);
   state.lastOperation = { type: "activation", startedAt, endedAt: new Date().toISOString(), inputTokens: activatedTokens, outputTokens: estimateTokens(observations) };
   state.operationLock = undefined;
   state.status = "idle";
@@ -1065,6 +1364,7 @@ async function reflectNow(ctx: any, opts: { reason: string; signal?: AbortSignal
         state.currentTask = result.currentTask ?? state.currentTask;
         state.suggestedResponse = result.suggestedContinuation ?? state.suggestedResponse;
         state.observationTokens = reflectedTokens;
+        state.vectorIndex = buildLocalVectorIndex(state);
         state.operationLock = undefined;
         state.status = "idle";
         state.lastError = undefined;
@@ -1087,13 +1387,16 @@ async function reflectNow(ctx: any, opts: { reason: string; signal?: AbortSignal
   }
 }
 
-async function runObserver(ctx: any, historyText: string, signal: AbortSignal | undefined, opts: { source: string; existingObservations?: string }): Promise<ObserverResult> {
+async function runObserver(ctx: any, historyText: string, signal: AbortSignal | undefined, opts: { source: string; existingObservations?: string; attachmentParts?: any[] }): Promise<ObserverResult> {
   const state = await ensureState(ctx);
   const modelId = state.settings.observationModel || DEFAULT_OBSERVATION_MODEL;
   const model = resolveModel(ctx, modelId);
   const response = await runModel(ctx, model, [
     { role: "user", content: [{ type: "text", text: buildObserverSystemPrompt(state.settings.caveman) }] },
-    { role: "user", content: [{ type: "text", text: `## New Message History to Observe\n\n${historyText}\n\n---\n\n${buildObserverTaskPrompt(opts.existingObservations, { priorCurrentTask: runtime.state?.currentTask, priorSuggestedResponse: runtime.state?.suggestedResponse })}` }] },
+    { role: "user", content: [
+      { type: "text", text: `## New Message History to Observe\n\n${historyText}\n\n---\n\n${buildObserverTaskPrompt(opts.existingObservations, { priorCurrentTask: runtime.state?.currentTask, priorSuggestedResponse: runtime.state?.suggestedResponse })}` },
+      ...(opts.attachmentParts ?? []),
+    ] },
   ], { temperature: 0.3, maxTokens: 100_000, signal });
   const text = responseText(response);
   const parsed = parseObserverOutput(text);
@@ -1125,7 +1428,11 @@ function resolveModel(ctx: any, modelId: string): any {
   const slash = modelId.indexOf("/");
   const provider = slash >= 0 ? modelId.slice(0, slash) : ctx.model?.provider;
   const id = slash >= 0 ? modelId.slice(slash + 1) : modelId;
-  const model = ctx.modelRegistry.find(provider, id);
+  const currentModel = ctx.model;
+  if (currentModel && String(currentModel.provider) === String(provider) && String(currentModel.id ?? currentModel.model) === String(id)) {
+    return currentModel;
+  }
+  const model = typeof ctx.modelRegistry?.find === "function" ? ctx.modelRegistry.find(provider, id) : undefined;
   if (!model) throw new Error(`Could not find OM model ${provider}/${id}`);
   return model;
 }
@@ -1177,19 +1484,207 @@ function detectDegenerateRepetition(text: string): boolean {
 }
 
 function appendObservations(state: PiOMRecord, result: ObserverResult, inputTokens: number): void {
-  const observations = result.observations.trim();
-  if (!observations) throw new Error("No observations to append");
+  const observations = suppressDuplicateObservations(state.observations, result.observations.trim());
+  if (!observations) throw new Error("No new non-duplicate observations to append");
   state.observations = [state.observations.trim(), observations].filter(Boolean).join("\n\n");
   state.currentTask = result.currentTask ?? state.currentTask;
   state.suggestedResponse = result.suggestedContinuation ?? state.suggestedResponse;
   state.observationTokens = estimateTokens(state.observations);
+  state.vectorIndex = buildLocalVectorIndex(state);
   state.lastOperation = { type: "observation", startedAt: state.operationLock?.startedAt ?? new Date().toISOString(), endedAt: new Date().toISOString(), inputTokens, outputTokens: estimateTokens(observations), model: state.settings.observationModel || DEFAULT_OBSERVATION_MODEL };
 }
 
-function buildOMContextMessage(state: PiOMRecord): AgentMessage {
+function observationChunks(text: string): string[] {
+  return text.split(/\n{2,}/).flatMap(block => {
+    const lines = block.split("\n").map(l => l.trim()).filter(Boolean);
+    if (lines.length <= 1) return lines;
+    return lines.some(l => /^[-*]\s|^[🔴🟡🟢✅]/.test(l)) ? lines : [lines.join("\n")];
+  }).map(redactSecrets).filter(Boolean);
+}
+
+function normalizeForSimilarity(text: string): string {
+  return text.toLowerCase().replace(/[🔴🟡🟢✅]/g, "").replace(/\([^)]*\)/g, "").replace(/[^a-z0-9а-яё\s._/-]/gi, " ").replace(/\s+/g, " ").trim();
+}
+
+function textHash(text: string): string {
+  return createHash("sha256").update(normalizeForSimilarity(text)).digest("hex");
+}
+
+function localEmbedding(text: string): number[] {
+  const dims = 128;
+  const vec = new Array(dims).fill(0);
+  const words = normalizeForSimilarity(text).split(" ").filter(w => w.length > 2);
+  for (const word of words) {
+    const h = createHash("sha256").update(word).digest();
+    const idx = h[0] % dims;
+    vec[idx] += 1 + Math.min(3, word.length / 8);
+  }
+  const norm = Math.sqrt(vec.reduce((s, v) => s + v * v, 0)) || 1;
+  return vec.map(v => Number((v / norm).toFixed(6)));
+}
+
+function cosine(a: number[], b: number[]): number {
+  const n = Math.min(a.length, b.length);
+  let sum = 0;
+  for (let i = 0; i < n; i++) sum += (a[i] ?? 0) * (b[i] ?? 0);
+  return sum;
+}
+
+function suppressDuplicateObservations(existing: string, incoming: string): string {
+  const existingChunks = observationChunks(existing);
+  const existingHashes = new Set(existingChunks.map(textHash));
+  const existingEmbeddings = existingChunks.map(localEmbedding);
+  const kept: string[] = [];
+  for (const chunk of observationChunks(incoming)) {
+    const hash = textHash(chunk);
+    if (existingHashes.has(hash)) continue;
+    const emb = localEmbedding(chunk);
+    if (existingEmbeddings.some(e => cosine(e, emb) >= 0.94)) continue;
+    existingHashes.add(hash);
+    existingEmbeddings.push(emb);
+    kept.push(chunk);
+  }
+  return kept.join("\n").trim();
+}
+
+function buildLocalVectorIndex(state: PiOMRecord): VectorIndexState | undefined {
+  if (state.settings.retrieval.provider === "off") return undefined;
+  const provider = state.settings.retrieval.provider;
+  if (provider !== "local") return state.vectorIndex;
+  const now = new Date().toISOString();
+  const entries = observationChunks(state.observations).slice(-VECTOR_MAX_INDEX_CHUNKS).map((text, i) => ({
+    id: `obs-${i}-${textHash(text).slice(0, 10)}`,
+    hash: textHash(text),
+    text,
+    embedding: localEmbedding(text),
+    tokens: estimateTokens(text),
+    createdAt: now,
+  }));
+  return { provider: "local", model: state.settings.retrieval.model, scopeKey: scopeKeyForState(state), updatedAt: now, entries };
+}
+
+async function retrieveRelevantObservations(ctx: any, state: PiOMRecord, query: string): Promise<string> {
+  const cfg = state.settings.retrieval;
+  if (cfg.provider === "off" || !query.trim()) return "";
+  if (cfg.provider === "gemini") return retrieveGeminiObservations(ctx, state, query);
+  const index = state.vectorIndex?.provider === "local" ? state.vectorIndex : buildLocalVectorIndex(state);
+  if (!index || index.entries.length === 0) return "";
+  state.vectorIndex = index;
+  const queryVec = localEmbedding(query);
+  return index.entries
+    .map(entry => ({ entry, score: cosine(queryVec, entry.embedding) }))
+    .filter(x => x.score >= cfg.threshold)
+    .sort((a, b) => b.score - a.score)
+    .slice(0, cfg.topK)
+    .map(x => `- (${x.score.toFixed(2)}) ${x.entry.text}`)
+    .join("\n");
+}
+
+async function retrieveGeminiObservations(ctx: any, state: PiOMRecord, query: string): Promise<string> {
+  const apiKey = process.env.GOOGLE_GENERATIVE_AI_API_KEY || process.env.GEMINI_API_KEY;
+  if (!apiKey) throw new Error("OM Gemini retrieval requires GOOGLE_GENERATIVE_AI_API_KEY or GEMINI_API_KEY");
+  const chunks = observationChunks(state.observations).slice(-VECTOR_MAX_INDEX_CHUNKS);
+  if (chunks.length === 0) return "";
+  const queryVec = await geminiEmbedding(apiKey, state.settings.retrieval.model, query);
+  const entries = await Promise.all(chunks.map(async (text, i) => ({
+    id: `gem-${i}-${textHash(text).slice(0, 10)}`,
+    hash: textHash(text),
+    text,
+    embedding: await geminiEmbedding(apiKey, state.settings.retrieval.model, text),
+    tokens: estimateTokens(text),
+    createdAt: new Date().toISOString(),
+  })));
+  state.vectorIndex = { provider: "gemini", model: state.settings.retrieval.model, scopeKey: scopeKeyForState(state), updatedAt: new Date().toISOString(), entries };
+  return entries
+    .map(entry => ({ entry, score: cosine(queryVec, entry.embedding) }))
+    .filter(x => x.score >= state.settings.retrieval.threshold)
+    .sort((a, b) => b.score - a.score)
+    .slice(0, state.settings.retrieval.topK)
+    .map(x => `- (${x.score.toFixed(2)}) ${x.entry.text}`)
+    .join("\n");
+}
+
+async function geminiEmbedding(apiKey: string, model: string, text: string): Promise<number[]> {
+  const url = `https://generativelanguage.googleapis.com/v1beta/models/${encodeURIComponent(model)}:embedContent?key=${encodeURIComponent(apiKey)}`;
+  const res = await fetch(url, { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify({ content: { parts: [{ text: truncateText(redactSecrets(text), 20_000) }] } }) });
+  if (!res.ok) throw new Error(`Gemini embedding failed: HTTP ${res.status} ${truncateText(await res.text(), 300)}`);
+  const json = await res.json() as any;
+  const values = json?.embedding?.values;
+  if (!Array.isArray(values)) throw new Error("Gemini embedding response did not include embedding.values");
+  const norm = Math.sqrt(values.reduce((s: number, v: number) => s + v * v, 0)) || 1;
+  return values.map((v: number) => v / norm);
+}
+
+
+async function buildProjectReferences(ctx: any, state: PiOMRecord): Promise<string> {
+  if (state.scope !== "project") return "";
+  const sessionsDir = join(ctx.cwd, CONFIG_DIR_NAME, "om", "sessions");
+  if (!existsSync(sessionsDir)) return "";
+  let files: string[] = [];
+  try {
+    files = (await readdir(sessionsDir)).filter(f => f.endsWith(".json") && f !== `${sanitizeFileName(state.sessionId)}.json`);
+  } catch {
+    return "";
+  }
+  const records: PiOMRecord[] = [];
+  for (const file of files.slice(-20)) {
+    try {
+      const raw = JSON.parse(await readFile(join(sessionsDir, file), "utf8"));
+      const normalized = normalizeState(raw, { sessionId: raw.sessionId || file.replace(/\.json$/, ""), sessionFile: raw.sessionFile, cwd: ctx.cwd });
+      if (normalized.observations.trim()) records.push(normalized);
+    } catch {
+      // Skip unreadable references; main state remains authoritative.
+    }
+  }
+  records.sort((a, b) => Date.parse(b.updatedAt) - Date.parse(a.updatedAt));
+  const blocks: string[] = [];
+  let tokens = 0;
+  for (const rec of records.slice(0, 5)) {
+    const obs = limitTextByTokens(rec.observations, 500) || "";
+    const block = `<other-session id="${sanitizeFileName(rec.sessionId)}" updated="${rec.updatedAt}">\n${obs}\n</other-session>`;
+    tokens += estimateTokens(block);
+    if (tokens > PROJECT_REFERENCES_MAX_TOKENS) break;
+    blocks.push(block);
+  }
+  return blocks.join("\n\n");
+}
+
+async function rebuildVectorIndex(ctx: any, state: PiOMRecord): Promise<void> {
+  if (state.settings.retrieval.provider === "off") {
+    state.vectorIndex = undefined;
+    return;
+  }
+  if (state.settings.retrieval.provider === "local") {
+    state.vectorIndex = buildLocalVectorIndex(state);
+    return;
+  }
+  if (state.settings.retrieval.provider === "gemini") {
+    await retrieveGeminiObservations(ctx, state, state.observations.slice(0, 2_000) || "build index");
+    return;
+  }
+}
+
+function formatVectorStatus(state: PiOMRecord): string {
+  const idx = state.vectorIndex;
+  return [
+    "OM vector retrieval",
+    `provider: ${state.settings.retrieval.provider}`,
+    `model: ${state.settings.retrieval.model}`,
+    `scopeKey: ${scopeKeyForState(state)}`,
+    `topK: ${state.settings.retrieval.topK}`,
+    `threshold: ${state.settings.retrieval.threshold}`,
+    `index: ${idx ? `${idx.entries.length} entries, updated ${idx.updatedAt}` : "not built"}`,
+  ].join("\n");
+}
+
+function buildOMContextMessage(state: PiOMRecord, retrievedObservations = "", projectReferences = ""): AgentMessage {
+  const retrievalBlock = retrievedObservations ? `<relevant-observations>\n${retrievedObservations}\n</relevant-observations>` : "";
+  const projectBlock = projectReferences ? `<other-sessions>\n${projectReferences}\n</other-sessions>` : "";
   const sections = [
     OBSERVATION_CONTEXT_PROMPT,
     `<observations>\n${state.observations.trim()}\n</observations>`,
+    retrievalBlock,
+    projectBlock,
     OBSERVATION_CONTEXT_INSTRUCTIONS,
     state.currentTask ? `<current-task>\n${state.currentTask}\n</current-task>` : "",
     state.suggestedResponse ? `<suggested-response>\n${state.suggestedResponse}\n</suggested-response>` : "",
@@ -1281,6 +1776,99 @@ function formatEntriesForObserver(entries: SessionEntry[]): string {
   return entries.map(formatEntryForObserver).filter(Boolean).join("\n\n");
 }
 
+function buildObserverInput(ctx: any, entries: SessionEntry[], state: PiOMRecord): { text: string; attachmentParts: any[]; omissions: string[] } {
+  const model = resolveModel(ctx, state.settings.observationModel || DEFAULT_OBSERVATION_MODEL);
+  const attachmentParts: any[] = [];
+  const omissions: string[] = [];
+  const text = entries.map(entry => formatEntryForObserverWithAttachments(entry, state, model, attachmentParts, omissions)).filter(Boolean).join("\n\n");
+  return { text, attachmentParts, omissions };
+}
+
+function formatEntryForObserverWithAttachments(entry: SessionEntry, state: PiOMRecord, model: any, attachmentParts: any[], omissions: string[]): string {
+  const msg = entryToAgentMessage(entry);
+  const createdAt = new Date(entry.timestamp || msg?.timestamp || Date.now());
+  const date = createdAt.toLocaleDateString("en-US", { month: "short", day: "numeric", year: "numeric" });
+  const time = createdAt.toLocaleTimeString("en-US", { hour: "2-digit", minute: "2-digit", hour12: false });
+  const body = formatAgentMessageForObserverWithAttachments(msg, state, model, attachmentParts, omissions);
+  return body ? `${date}:\n[entry ${entry.id}] ${time} ${body}` : "";
+}
+
+function formatAgentMessageForObserverWithAttachments(msg: any, state: PiOMRecord, model: any, attachmentParts: any[], omissions: string[]): string {
+  if (!msg) return "";
+  const cap = MESSAGE_PART_MAX_CHARS;
+  switch (msg.role) {
+    case "user": return `User: ${formatContentForObserverWithAttachments(msg.content, cap, state, model, attachmentParts, omissions)}`;
+    case "assistant": return `Assistant: ${formatContentForObserverWithAttachments(msg.content, cap, state, model, attachmentParts, omissions)}`;
+    case "toolResult": return `Tool Result ${msg.toolName}: ${truncateText(formatContentForObserverWithAttachments(msg.content, cap, state, model, attachmentParts, omissions), TOOL_RESULT_MAX_CHARS)}`;
+    case "bashExecution": return `User Bash: ${msg.command}\nOutput: ${truncateText(msg.output ?? "", cap)}${msg.truncated ? "\n[output truncated by Pi]" : ""}`;
+    case "custom": return msg.customType === EXTENSION_ID ? "" : `Custom ${msg.customType}: ${formatContentForObserverWithAttachments(msg.content, cap, state, model, attachmentParts, omissions)}`;
+    case "compactionSummary": return `Previous Compaction Summary: ${truncateText(msg.summary ?? "", cap)}`;
+    case "branchSummary": return `Branch Summary: ${truncateText(msg.summary ?? "", cap)}`;
+    default: return `${msg.role ?? "Message"}: ${truncateText(JSON.stringify(redactDeep(msg)), cap)}`;
+  }
+}
+
+function formatContentForObserverWithAttachments(content: any, cap: number, state: PiOMRecord, model: any, attachmentParts: any[], omissions: string[]): string {
+  if (typeof content === "string") return truncateText(redactSecrets(content), cap);
+  if (!Array.isArray(content)) return truncateText(redactSecrets(JSON.stringify(redactDeep(content))), cap);
+  return content.map(part => {
+    if (part.type === "text") return redactSecrets(part.text);
+    if (part.type === "thinking") return `[Thinking]: ${redactSecrets(part.thinking ?? "")}`;
+    if (isAttachmentPart(part)) return handleObserverAttachmentPart(part, state, model, attachmentParts, omissions);
+    if (part.type === "toolCall") return `[Tool Call ${part.name}: ${truncateText(redactSecrets(JSON.stringify(redactDeep(part.arguments ?? {}))), 2_000)}]`;
+    return `[${part.type}: ${truncateText(redactSecrets(JSON.stringify(redactDeep(part))), 2_000)}]`;
+  }).join("\n").slice(0, cap);
+}
+
+function isAttachmentPart(part: any): boolean {
+  return part?.type === "image" || part?.type === "file" || Boolean(part?.mimeType || part?.mediaType || part?.source?.mediaType);
+}
+
+function modelSupportsAttachments(model: any): boolean {
+  const provider = String(model?.provider || "").toLowerCase();
+  const id = String(model?.id || model?.model || "").toLowerCase();
+  if (provider.includes("google") || provider.includes("gemini")) return id.includes("gemini");
+  if (provider.includes("openai")) return /gpt-4o|gpt-4\.1|o3|o4/.test(id);
+  if (provider.includes("anthropic")) return /claude-3|claude-sonnet-4|claude-opus-4/.test(id);
+  return false;
+}
+
+function handleObserverAttachmentPart(part: any, state: PiOMRecord, model: any, attachmentParts: any[], omissions: string[]): string {
+  const mode = state.settings.observeAttachments;
+  const mime = String(part.mimeType || part.mediaType || part.source?.mediaType || "unknown").toLowerCase();
+  const raw = part.data || part.image || part.source?.data || "";
+  const approxBytes = typeof raw === "string" ? Math.ceil(raw.length * 0.75) : 0;
+  const label = `${mime}${approxBytes ? `, ~${approxBytes} bytes` : ""}`;
+  if (mode === "off") return `[Attachment omitted: observation disabled; ${label}]`;
+  if (!mime.startsWith("image/")) {
+    const reason = `unsupported mime ${mime}`;
+    omissions.push(reason);
+    return `[Attachment omitted: ${reason}; ${label}]`;
+  }
+  if (approxBytes > MAX_ATTACHMENT_OBSERVE_BYTES) {
+    const reason = `attachment too large ${approxBytes} > ${MAX_ATTACHMENT_OBSERVE_BYTES} bytes`;
+    if (mode === "on") throw new Error(`OM attachment observation failed: ${reason}`);
+    omissions.push(reason);
+    return `[Attachment omitted: ${reason}; ${label}]`;
+  }
+  if (!modelSupportsAttachments(model)) {
+    const reason = `unsupported observer model ${model?.provider ?? "unknown"}/${model?.id ?? "unknown"}`;
+    if (mode === "on") throw new Error(`OM attachment observation failed: ${reason}`);
+    omissions.push(reason);
+    return `[Attachment omitted: ${reason}; ${label}]`;
+  }
+  const normalized = normalizeAttachmentPartForModel(part);
+  attachmentParts.push(normalized);
+  return `[Attachment included for Observer: ${label}]`;
+}
+
+function normalizeAttachmentPartForModel(part: any): any {
+  const mimeType = part.mimeType || part.mediaType || part.source?.mediaType;
+  const data = part.data || part.image || part.source?.data;
+  if (part.type === "image") return { ...part, mimeType, data };
+  return { type: "image", mimeType, data };
+}
+
 function formatEntryForObserver(entry: SessionEntry): string {
   const msg = entryToAgentMessage(entry);
   const createdAt = new Date(entry.timestamp || msg?.timestamp || Date.now());
@@ -1327,26 +1915,48 @@ function redactSecrets(input: string): string {
     .replace(/\bBearer\s+[A-Za-z0-9._~+/=-]{16,}/gi, "Bearer [REDACTED_TOKEN]");
 }
 
+function isSensitiveObjectKey(key: string): boolean {
+  const normalized = key.replace(/([a-z0-9])([A-Z])/g, "$1-$2").toLowerCase();
+  return /^(api-?key|authorization|auth-?token|access-?token|refresh-?token|secret|password|bearer)$/i.test(normalized);
+}
+
 function redactDeep<T>(value: T): T {
   if (typeof value === "string") return redactSecrets(value) as T;
   if (Array.isArray(value)) return value.map(item => redactDeep(item)) as T;
   if (value && typeof value === "object") {
     const out: Record<string, unknown> = {};
     for (const [key, child] of Object.entries(value as Record<string, unknown>)) {
-      out[key] = /apiKey|authorization|token|secret|password/i.test(key) ? "[REDACTED_SECRET]" : redactDeep(child);
+      out[key] = isSensitiveObjectKey(key) ? "[REDACTED_SECRET]" : redactDeep(child);
     }
     return out as T;
   }
   return value;
 }
 
+function shouldObserveAttachment(part: any, state?: PiOMRecord): boolean {
+  const mode = state?.settings.observeAttachments ?? "auto";
+  if (mode === "off") return false;
+  const mime = String(part.mimeType || part.mediaType || "").toLowerCase();
+  if (mode === "auto" && mime && !mime.startsWith("image/")) return false;
+  const raw = part.data || part.image || part.source?.data || "";
+  const approxBytes = typeof raw === "string" ? Math.ceil(raw.length * 0.75) : 0;
+  return approxBytes <= MAX_ATTACHMENT_OBSERVE_BYTES;
+}
+
+function formatAttachmentPart(part: any): string {
+  const mime = part.mimeType || part.mediaType || "unknown";
+  const raw = part.data || part.image || part.source?.data || "";
+  const approxBytes = typeof raw === "string" ? Math.ceil(raw.length * 0.75) : 0;
+  return `[Attachment observed: ${mime}${approxBytes ? `, ~${approxBytes} bytes` : ""}]`;
+}
+
 function formatContent(content: any, cap: number): string {
   if (typeof content === "string") return truncateText(redactSecrets(content), cap);
   if (!Array.isArray(content)) return truncateText(redactSecrets(JSON.stringify(redactDeep(content))), cap);
   return content.map(part => {
     if (part.type === "text") return redactSecrets(part.text);
     if (part.type === "thinking") return `[Thinking]: ${redactSecrets(part.thinking ?? "")}`;
-    if (part.type === "image") return runtime.state?.settings.observeAttachments === false ? "[Image omitted: attachment observation disabled]" : `[Image: ${part.mimeType ?? "unknown"}]`;
+    if (part.type === "image") return shouldObserveAttachment(part, runtime.state) ? formatAttachmentPart(part) : "[Image omitted: attachment observation disabled or unsupported]";
     if (part.type === "toolCall") return `[Tool Call ${part.name}: ${truncateText(redactSecrets(JSON.stringify(redactDeep(part.arguments ?? {}))), 2_000)}]`;
     return `[${part.type}: ${truncateText(redactSecrets(JSON.stringify(redactDeep(part))), 2_000)}]`;
   }).join("\n").slice(0, cap);
@@ -1443,6 +2053,20 @@ function parseBooleanSetting(key: string, value: string): boolean {
   throw new Error(`OM setting ${key} must be on/off or true/false`);
 }
 
+function parseAttachmentSetting(key: string, value: string): ObserveAttachmentsMode {
+  const normalized = value.trim().toLowerCase();
+  if (["auto", "default"].includes(normalized)) return "auto";
+  if (["1", "true", "yes", "on", "enabled"].includes(normalized)) return "on";
+  if (["0", "false", "no", "off", "disabled"].includes(normalized)) return "off";
+  throw new Error(`OM setting ${key} must be auto/on/off`);
+}
+
+function parseRetrievalProvider(key: string, value: string): RetrievalProvider {
+  const normalized = value.trim().toLowerCase();
+  if (["off", "local", "gemini"].includes(normalized)) return normalized as RetrievalProvider;
+  throw new Error(`OM setting ${key} must be off, local, or gemini`);
+}
+
 function applySetting(state: PiOMRecord, key: string | undefined, value: string): void {
   if (!key) throw new Error("Missing OM setting key");
   if (!value) throw new Error(`Missing OM setting value for ${key}`);
@@ -1455,10 +2079,20 @@ function applySetting(state: PiOMRecord, key: string | undefined, value: string)
   else if (["observation-model", "observer-model", "observe-model"].includes(normalized)) state.settings.observationModel = value.trim();
   else if (["reflection-model", "reflector-model", "reflect-model"].includes(normalized)) state.settings.reflectionModel = value.trim();
   else if (normalized === "caveman") state.settings.caveman = parseBooleanSetting(key, value);
-  else if (["attachments", "observe-attachments", "attachment-observation"].includes(normalized)) state.settings.observeAttachments = parseBooleanSetting(key, value);
+  else if (["attachments", "observe-attachments", "attachment-observation"].includes(normalized)) state.settings.observeAttachments = parseAttachmentSetting(key, value);
+  else if (["retrieval", "retrieval-provider", "vector-provider"].includes(normalized)) {
+    state.settings.retrieval.provider = parseRetrievalProvider(key, value);
+    if (state.settings.retrieval.provider === "gemini" && state.settings.retrieval.model === "local/hash-bow-v1") state.settings.retrieval.model = "gemini-embedding-001";
+    if (state.settings.retrieval.provider === "local" && state.settings.retrieval.model === "gemini-embedding-001") state.settings.retrieval.model = "local/hash-bow-v1";
+    state.vectorIndex = undefined;
+  }
+  else if (["retrieval-model", "embedding-model", "vector-model"].includes(normalized)) { state.settings.retrieval.model = value.trim(); state.vectorIndex = undefined; }
+  else if (["retrieval-top-k", "top-k", "vector-top-k"].includes(normalized)) state.settings.retrieval.topK = clampInt(parsePositiveIntSetting(key, value), 1, 30);
+  else if (["retrieval-threshold", "vector-threshold"].includes(normalized)) state.settings.retrieval.threshold = parseRatioSetting(key, value);
   else if (normalized === "scope") {
     const scope = value.trim().toLowerCase();
     if (scope !== "session" && scope !== "project") throw new Error("OM scope must be session or project");
+    if (state.scope !== scope) state.vectorIndex = undefined;
     state.scope = scope;
   } else {
     throw new Error(`Unknown OM setting: ${key}`);
@@ -1475,14 +2109,21 @@ function formatSettingsText(state: PiOMRecord): string {
     `  observation-model: ${state.settings.observationModel}`,
     `  reflection-model: ${state.settings.reflectionModel}`,
     `  caveman: ${state.settings.caveman ? "on" : "off"}`,
-    `  attachments: ${state.settings.observeAttachments ? "on" : "off"}`,
+    `  attachments: ${state.settings.observeAttachments}`,
     `  scope: ${state.scope}`,
+    `  retrieval: ${state.settings.retrieval.provider}`,
+    `  retrieval-model: ${state.settings.retrieval.model}`,
+    `  retrieval-top-k: ${state.settings.retrieval.topK}`,
+    `  retrieval-threshold: ${state.settings.retrieval.threshold}`,
+    `  vector-index: ${state.vectorIndex ? `${state.vectorIndex.entries.length} entries (${state.vectorIndex.provider}/${state.vectorIndex.model})` : "none"}`,
     "",
     "usage:",
     "  /om set observation-threshold 30000",
     "  /om set reflection-threshold 40000",
     "  /om set observation-model google/gemini-2.5-flash",
     "  /om set caveman on",
+    "  /om set retrieval local",
+    "  /om vector status|rebuild|search <query>",
     "  /om reset",
   ].join("\n");
 }
@@ -1670,7 +2311,7 @@ async function writeDebug(ctx: any, name: string, payload: unknown): Promise<voi
     const dir = runtime.debugDir || join(ctx.cwd, CONFIG_DIR_NAME, "om", "debug");
     await mkdir(dir, { recursive: true });
     const file = join(dir, `${new Date().toISOString().replace(/[:.]/g, "-")}-${sanitizeFileName(name)}.json`);
-    await writeFile(file, redactSecrets(JSON.stringify(redactDeep({ extension: EXTENSION_ID, sessionId: state.sessionId, ...payload as any }), null, 2)) + "\n", "utf8");
+    await writeFile(file, JSON.stringify(redactDeep({ extension: EXTENSION_ID, sessionId: state.sessionId, ...payload as any }), null, 2) + "\n", "utf8");
   } catch (error) {
     // Ignore debug write errors if context is stale
   }
@@ -1709,6 +2350,14 @@ export const __test = {
   atomicWriteJson,
   applySetting,
   formatSettingsText,
+  suppressDuplicateObservations,
+  buildLocalVectorIndex,
+  retrieveRelevantObservations,
+  buildObserverInput,
+  modelSupportsAttachments,
+  formatVectorStatus,
+  shouldObserveAttachment,
+  geminiEmbedding,
 };
 
 class OMOverlay {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-observational-memory-extension",
-  "version": "0.1.3",
+  "version": "0.2.0",
   "description": "Mastra-style Observational Memory extension for Pi compaction and runtime context.",
   "license": "MIT",
   "author": "Nikita Nosov <20nik.nosov21@gmail.com>",

package/scripts/test.mjs

@@ -40,25 +40,86 @@ __test.applySetting(state, "observation-model", "google/gemini-2.5-flash");
 __test.applySetting(state, "reflection-threshold", "12345");
 __test.applySetting(state, "caveman", "on");
 __test.applySetting(state, "attachments", "off");
+__test.applySetting(state, "retrieval", "local");
+__test.applySetting(state, "retrieval-top-k", "4");
+__test.applySetting(state, "retrieval-threshold", "25%");
+__test.applySetting(state, "scope", "project");
 assert.equal(state.settings.observationModel, "google/gemini-2.5-flash");
 assert.equal(state.thresholds.reflection, 12345);
 assert.equal(state.settings.caveman, true);
-assert.equal(state.settings.observeAttachments, false);
+assert.equal(state.settings.observeAttachments, "off");
+assert.equal(state.settings.retrieval.provider, "local");
+assert.equal(state.settings.retrieval.topK, 4);
+assert.equal(state.settings.retrieval.threshold, 0.25);
+assert.equal(state.scope, "project");
+state.vectorIndex = { provider: "local", model: "local/hash-bow-v1", scopeKey: "session:old", updatedAt: new Date().toISOString(), entries: [] };
+__test.applySetting(state, "scope", "session");
+assert.equal(state.scope, "session");
+assert.equal(state.vectorIndex, undefined);
 assert.throws(() => __test.applySetting(state, "buffer-activation", "2"), /ratio/);
 
 // Migration/normalization: v1 state keeps observations and receives v2 settings.
 const migrated = __test.normalizeState({ version: 1, observations: "token=do-not-keep-this-secret", thresholds: { observation: 10 } }, { sessionId: "m", cwd: process.cwd() });
-assert.equal(migrated.version, 2);
+assert.equal(migrated.version, 3);
 assert.equal(migrated.thresholds.observation, 10);
 assert.equal(migrated.settings.caveman, false);
+assert.equal(migrated.settings.observeAttachments, "auto");
+assert.equal(migrated.settings.retrieval.provider, "off");
 assert(!migrated.observations.includes("do-not-keep-this-secret"));
 
+// Duplicate suppression and local retrieval: exact/similar repeats are skipped, relevant chunks rank first.
+const deduped = __test.suppressDuplicateObservations("* 🔴 User likes Pi loopflows", "* 🔴 User likes Pi loopflows\n* 🟡 Vector retrieval implemented locally");
+assert(!deduped.includes("User likes Pi loopflows"));
+assert(deduped.includes("Vector retrieval"));
+state.observations = "* 🔴 User builds Pi loopflows\n* 🟡 unrelated cooking note\n* ✅ Vector retrieval implemented locally";
+state.vectorIndex = __test.buildLocalVectorIndex(state);
+const retrieved = await __test.retrieveRelevantObservations({ }, state, "local vector retrieval for loopflows");
+assert(retrieved.includes("Vector retrieval"));
+assert(__test.formatVectorStatus(state).includes("provider: local"));
+
+// Attachment observation: auto mode accepts small images, rejects non-images and oversized payloads.
+state.settings.observeAttachments = "auto";
+assert.equal(__test.shouldObserveAttachment({ type: "image", mimeType: "image/png", data: "abc" }, state), true);
+assert.equal(__test.shouldObserveAttachment({ type: "image", mimeType: "application/pdf", data: "abc" }, state), false);
+assert.equal(__test.shouldObserveAttachment({ type: "image", mimeType: "image/png", data: "x".repeat(3_000_000) }, state), false);
+state.settings.observeAttachments = "off";
+assert.equal(__test.shouldObserveAttachment({ type: "image", mimeType: "image/png", data: "abc" }, state), false);
+
+// Attachment input builder: off/auto never leaks base64 into text; on + unsupported model hard-fails; Gemini model includes image part.
+const attachmentEntry = {
+  id: "att-1",
+  type: "message",
+  timestamp: new Date().toISOString(),
+  message: { role: "user", content: [{ type: "text", text: "see image" }, { type: "image", mimeType: "image/png", data: "aGVsbG8=" }] },
+};
+state.settings.observationModel = "local/text-only";
+state.settings.observeAttachments = "off";
+let built = __test.buildObserverInput({ model: { provider: "local", id: "text-only" } }, [attachmentEntry], state);
+assert(built.text.includes("Attachment omitted"));
+assert(!built.text.includes("aGVsbG8="));
+assert.equal(built.attachmentParts.length, 0);
+state.settings.observeAttachments = "auto";
+built = __test.buildObserverInput({ model: { provider: "local", id: "text-only" } }, [attachmentEntry], state);
+assert(built.text.includes("unsupported observer model"));
+assert.equal(built.attachmentParts.length, 0);
+state.settings.observeAttachments = "on";
+assert.throws(() => __test.buildObserverInput({ model: { provider: "local", id: "text-only" } }, [attachmentEntry], state), /unsupported observer model/);
+state.settings.observationModel = "google/gemini-2.5-flash";
+built = __test.buildObserverInput({ model: { provider: "google", id: "gemini-2.5-flash" } }, [attachmentEntry], state);
+assert.equal(built.attachmentParts.length, 1);
+assert.equal(built.attachmentParts[0].type, "image");
+
 // Atomic write: writes valid JSON, creates backup on subsequent write, removes temp files.
 const dir = await mkdtemp(join(tmpdir(), "pi-om-test-"));
 try {
   const file = join(dir, "state.json");
-  await __test.atomicWriteJson(file, { a: 1, token: "supersecrettoken" });
-  assert.equal(JSON.parse(await readFile(file, "utf8")).token, "[REDACTED_SECRET]");
+  await __test.atomicWriteJson(file, { a: 1, accessToken: "supersecrettoken", observationTokens: 238, pendingMessageTokens: 12, observations: 'Secret check: api_key="sk-live-test-should-redact-1234567890" stays JSON-safe' });
+  const firstWrite = JSON.parse(await readFile(file, "utf8"));
+  assert.equal(firstWrite.accessToken, "[REDACTED_SECRET]");
+  assert.equal(firstWrite.observationTokens, 238);
+  assert.equal(firstWrite.pendingMessageTokens, 12);
+  assert(firstWrite.observations.includes("[REDACTED_SECRET]"));
+  assert(!firstWrite.observations.includes("sk-live-test-should-redact"));
   await __test.atomicWriteJson(file, { a: 2 });
   assert.equal(JSON.parse(await readFile(file, "utf8")).a, 2);
   assert.equal(existsSync(`${file}.bak`), true);