clawmem 0.6.0 → 0.7.0

package/AGENTS.md

@@ -252,7 +252,7 @@ ClawMem hooks handle ~90% of retrieval automatically. Agent-initiated MCP calls
 | `precompact-extract` | PreCompact | — | extracts decisions, file paths, open questions → writes `precompact-state.md` to auto-memory. Query-aware decision ranking. Reindexes auto-memory collection. |
 | `decision-extractor` | Stop | — | LLM extracts observations → `_clawmem/agent/observations/`, infers causal links, detects contradictions, extracts SPO triples from decision/preference/milestone/problem facts. Background consolidation worker synthesizes deductive observations from related facts (Phase 3, every ~15 min). |
 | `handoff-generator` | Stop | — | LLM summarizes session → `_clawmem/agent/handoffs/` |
-| `feedback-loop` | Stop | — | tracks referenced notes → boosts confidence, records usage relations + co-activations between co-referenced docs, tracks utility signals (surfaced vs referenced ratio for lifecycle automation) |
+| `feedback-loop` | Stop | — | tracks referenced notes → boosts confidence, records usage relations + co-activations between co-referenced docs, tracks utility signals (surfaced vs referenced ratio for lifecycle automation), per-turn recall attribution (marks which surfaced docs were cited in which turn) |
 
 **Default behavior:** Read injected `<vault-context>` first. If sufficient, answer immediately.
 

package/CLAUDE.md

@@ -252,7 +252,7 @@ ClawMem hooks handle ~90% of retrieval automatically. Agent-initiated MCP calls
 | `precompact-extract` | PreCompact | — | extracts decisions, file paths, open questions → writes `precompact-state.md` to auto-memory. Query-aware decision ranking. Reindexes auto-memory collection. |
 | `decision-extractor` | Stop | — | LLM extracts observations → `_clawmem/agent/observations/`, infers causal links, detects contradictions, extracts SPO triples from decision/preference/milestone/problem facts. Background consolidation worker synthesizes deductive observations from related facts (Phase 3, every ~15 min). |
 | `handoff-generator` | Stop | — | LLM summarizes session → `_clawmem/agent/handoffs/` |
-| `feedback-loop` | Stop | — | tracks referenced notes → boosts confidence, records usage relations + co-activations between co-referenced docs, tracks utility signals (surfaced vs referenced ratio for lifecycle automation) |
+| `feedback-loop` | Stop | — | tracks referenced notes → boosts confidence, records usage relations + co-activations between co-referenced docs, tracks utility signals (surfaced vs referenced ratio for lifecycle automation), per-turn recall attribution (marks which surfaced docs were cited in which turn) |
 
 **Default behavior:** Read injected `<vault-context>` first. If sufficient, answer immediately.
 

package/README.md

@@ -85,7 +85,7 @@ Runs fully local with no API keys and no cloud services. Integrates via Claude C
 **Optional integrations:**
 
 - [Claude Code](https://docs.anthropic.com/en/docs/claude-code) — for hooks + MCP integration
-- [OpenClaw](https://github.com/openclawai/openclaw) — for ContextEngine plugin integration
+- [OpenClaw](https://github.com/openclaw/openclaw) — for ContextEngine plugin integration
 - [Hermes Agent](https://github.com/NousResearch/hermes-agent) — for MemoryProvider plugin integration
 - [bd CLI](https://github.com/dolthub/dolt) v0.58.0+ — for Beads issue tracker sync (only if using Beads)
 
@@ -885,6 +885,17 @@ Documents are split into semantic fragments (sections, lists, code blocks, front
 
 Uses the LLM server (shared with query expansion and intent classification) to extract structured observations from session transcripts. Observation types: `decision`, `bugfix`, `feature`, `refactor`, `discovery`, `change`, `preference`, `milestone`, `problem`. Each observation includes title, facts, narrative, concepts, and files read/modified. Preferences, milestones, and problems get first-class content_type treatment with dedicated confidence baselines and half-lives instead of being flattened to generic "observation". Falls back to regex patterns if the model is unavailable.
 
+### Recall Tracking
+
+Empirical tracking of which documents are surfaced by retrieval, which queries surfaced them, and whether the assistant actually cited them. Provides signals beyond raw search relevance for lifecycle decisions:
+
+- **Per-query diversity**: docs surfaced by multiple distinct queries have proven cross-domain relevance
+- **Multi-day spacing**: docs surfaced across separate calendar days (spaced frequency) are more valuable than binge recalls in one session
+- **Negative signals**: docs surfaced frequently but rarely referenced are noise candidates for snooze
+- **Per-turn attribution**: feedback-loop segments the transcript into turns and attributes references to specific context-surfacing invocations, not the session globally
+
+Data feeds `lifecycle_status` (pin/snooze candidate reports) and `lifecycle_sweep` (recall-based recommendations). Adapted from [OpenClaw](https://github.com/openclaw/openclaw) dreaming promotion patterns.
+
 ### User Profile
 
 Two-tier auto-curated profile extracted from your decisions and hub documents:
@@ -1124,6 +1135,7 @@ Built on the shoulders of:
 - [MAGMA](https://arxiv.org/abs/2501.13956) — multi-graph memory agent
 - [MemPalace](https://github.com/milla-jovovich/mempalace) — conversation import patterns, broadened observation taxonomy (preference/milestone/problem), session-bootstrap synthesis
 - [memory-lancedb-pro](https://github.com/CortexReach/memory-lancedb-pro) — retrieval gate, length normalization, MMR diversity, access reinforcement algorithms
+- [OpenClaw](https://github.com/openclaw/openclaw) — recall tracking patterns (per-query diversity, multi-day spacing, negative signal tracking, promotion scoring) extracted from the dreaming memory consolidation system
 - [OpenViking](https://github.com/volcengine/OpenViking) — query decomposition patterns, collection-scoped retrieval, transaction-safe indexing
 - [QMD](https://github.com/tobi/qmd) — search backend (BM25 + vectors + RRF + reranking)
 - [SAME](https://github.com/sgx-labs/statelessagent) — agent memory concepts (recency decay, confidence scoring, session tracking)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.6.0",
+  "version": "0.7.0",
   "description": "On-device context engine and memory for AI agents. Claude Code and OpenClaw. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {

package/src/consolidation.ts

@@ -124,6 +124,17 @@ async function tick(store: Store, llm: LlamaCpp): Promise<void> {
     if (tickCount % 3 === 0) {
       await generateDeductiveObservations(store, llm);
     }
+
+    // Phase 4: Recall stats recomputation (every tick — lightweight SQL aggregation)
+    try {
+      const updated = store.recomputeRecallStats();
+      if (updated > 0) {
+        console.log(`[consolidation] Phase 4: recomputed recall_stats for ${updated} docs`);
+      }
+    } catch (err) {
+      // Non-critical — recall stats are informational, not retrieval-blocking
+      console.error("[consolidation] Phase 4 recall stats failed:", err);
+    }
   } catch (err) {
     console.error("[consolidation] Tick failed:", err);
   } finally {

package/src/hooks/context-surfacing.ts

@@ -30,6 +30,7 @@ import { enrichResults } from "../search-utils.ts";
 import { sanitizeSnippet } from "../promptguard.ts";
 import { shouldSkipRetrieval, isRetrievedNoise } from "../retrieval-gate.ts";
 import { MAX_QUERY_LENGTH } from "../limits.ts";
+import { writeRecallEvents, hashQuery } from "../recall-buffer.ts";
 
 // =============================================================================
 // Config
@@ -69,18 +70,44 @@ export async function contextSurfacing(
   input: HookInput
 ): Promise<HookOutput> {
   let prompt = input.prompt?.trim();
-  if (!prompt || prompt.length < MIN_PROMPT_LENGTH) return makeEmptyOutput("context-surfacing");
+
+  // Compute turn_index FIRST, before any early returns.
+  // Every transcript-visible early return must log an empty context_usage row
+  // to keep turn_index aligned with transcript turns for per-turn attribution.
+  if (input.sessionId) {
+    try {
+      let turnIndex = 0;
+      try {
+        const existing = store.db.prepare(
+          `SELECT COUNT(*) as cnt FROM context_usage WHERE session_id = ? AND hook_name = 'context-surfacing'`
+        ).get(input.sessionId) as { cnt: number };
+        turnIndex = existing.cnt;
+      } catch { /* fallback to 0 */ }
+      (input as any)._turnIndex = turnIndex;
+    } catch { /* non-fatal */ }
+  }
+
+  if (!prompt || prompt.length < MIN_PROMPT_LENGTH) {
+    logEmptyTurn(store, input);
+    return makeEmptyOutput("context-surfacing");
+  }
 
   // Bound query length to prevent DoS on search indices
   if (prompt.length > MAX_QUERY_LENGTH) prompt = prompt.slice(0, MAX_QUERY_LENGTH);
 
-  // Skip slash commands
-  if (prompt.startsWith("/")) return makeEmptyOutput("context-surfacing");
+  // Skip slash commands — log empty turn for alignment
+  if (prompt.startsWith("/")) {
+    logEmptyTurn(store, input);
+    return makeEmptyOutput("context-surfacing");
+  }
 
   // Adaptive retrieval gate: skip greetings, shell commands, affirmations, etc.
-  if (shouldSkipRetrieval(prompt)) return makeEmptyOutput("context-surfacing");
+  if (shouldSkipRetrieval(prompt)) {
+    logEmptyTurn(store, input);
+    return makeEmptyOutput("context-surfacing");
+  }
 
-  // Heartbeat / duplicate suppression (IO4)
+  // Heartbeat / duplicate suppression (IO4) — NOT transcript-visible user turns
   if (isHeartbeatPrompt(prompt)) return makeEmptyOutput("context-surfacing");
   if (wasPromptSeenRecently(store, "context-surfacing", prompt)) {
     return makeEmptyOutput("context-surfacing");
@@ -157,7 +184,7 @@ export async function contextSurfacing(
     }
   }
 
-  if (results.length === 0) return makeEmptyOutput("context-surfacing");
+  if (results.length === 0) { logEmptyTurn(store, input); return makeEmptyOutput("context-surfacing"); }
 
   // Budget-aware deep escalation (deep profile only):
   // If the fast path finished quickly and found results, spend remaining time budget
@@ -215,7 +242,7 @@ export async function contextSurfacing(
     !FILTERED_PATHS.some(p => r.displayPath.includes(p))
   );
 
-  if (results.length === 0) return makeEmptyOutput("context-surfacing");
+  if (results.length === 0) { logEmptyTurn(store, input); return makeEmptyOutput("context-surfacing"); }
 
   // Filter out snoozed documents
   const now = new Date();
@@ -231,7 +258,7 @@ export async function contextSurfacing(
     return true;
   });
 
-  if (results.length === 0) return makeEmptyOutput("context-surfacing");
+  if (results.length === 0) { logEmptyTurn(store, input); return makeEmptyOutput("context-surfacing"); }
 
   // Deduplicate by filepath (keep best score per path)
   const deduped = new Map<string, SearchResult>();
@@ -273,7 +300,7 @@ export async function contextSurfacing(
       : 0;
 
     // Activation floor: if even the best result is too weak, bail entirely
-    if (bestScore < profile.activationFloor) return makeEmptyOutput("context-surfacing");
+    if (bestScore < profile.activationFloor) { logEmptyTurn(store, input); return makeEmptyOutput("context-surfacing"); }
 
     const adaptiveMin = Math.max(bestScore * profile.minScoreRatio, profile.absoluteFloor);
     scored = allScored.filter(r => r.compositeScore >= adaptiveMin);
@@ -282,7 +309,7 @@ export async function contextSurfacing(
     scored = allScored.filter(r => r.compositeScore >= minScore);
   }
 
-  if (scored.length === 0) return makeEmptyOutput("context-surfacing");
+  if (scored.length === 0) { logEmptyTurn(store, input); return makeEmptyOutput("context-surfacing"); }
 
   // Spreading activation (E11): boost results co-activated with top HOT results
   if (scored.length > 3) {
@@ -325,11 +352,62 @@ export async function contextSurfacing(
   // Build context within token budget (profile-driven)
   const { context, paths, tokens } = buildContext(scored, prompt, tokenBudget);
 
-  if (!context) return makeEmptyOutput("context-surfacing");
+  if (!context) {
+    logEmptyTurn(store, input);
+    return makeEmptyOutput("context-surfacing");
+  }
 
-  // Log the injection
+  // Use pre-computed turn_index from top of function
   if (input.sessionId) {
-    logInjection(store, input.sessionId, "context-surfacing", paths, tokens);
+    const turnIndex = (input as any)._turnIndex ?? 0;
+
+    // Log the injection — returns usage_id for recall event linkage
+    const usageId = logInjection(store, input.sessionId, "context-surfacing", paths, tokens, turnIndex);
+
+    // Record recall events ONLY for docs that made it into the injected context
+    // (post-budget). Docs trimmed by token budget were never seen by the model.
+    // Each event links to its context_usage row via usage_id + turn_index.
+    // Multi-vault: route docs to origin vault's store. Mirror context_usage there too.
+    try {
+      const qHash = hashQuery(prompt);
+      const injectedSet = new Set(paths);
+      const injectedScored = scored.filter(r => injectedSet.has(r.displayPath));
+
+      // Group by vault origin (undefined = general vault)
+      const byVault = new Map<string | undefined, typeof injectedScored>();
+      for (const r of injectedScored) {
+        const vault = (r as any)._fromVault as string | undefined;
+        let group = byVault.get(vault);
+        if (!group) { group = []; byVault.set(vault, group); }
+        group.push(r);
+      }
+
+      const validUsageId = usageId > 0 ? usageId : undefined;
+      for (const [vault, docs] of byVault) {
+        const mappedDocs = docs.map(r => ({ displayPath: r.displayPath, searchScore: r.compositeScore }));
+        if (!vault) {
+          writeRecallEvents(store, input.sessionId, qHash, mappedDocs, validUsageId, turnIndex);
+        } else {
+          try {
+            const vaultStore = resolveStore(vault);
+            // Mirror context_usage row into named vault for correct FK + attribution
+            const vaultPaths = docs.map(r => r.displayPath);
+            const vaultUsageId = vaultStore.insertUsage({
+              sessionId: input.sessionId,
+              timestamp: new Date().toISOString(),
+              hookName: "context-surfacing",
+              injectedPaths: vaultPaths,
+              estimatedTokens: 0,
+              wasReferenced: 0,
+              turnIndex,
+            });
+            writeRecallEvents(vaultStore, input.sessionId, qHash, mappedDocs, vaultUsageId > 0 ? vaultUsageId : undefined, turnIndex);
+          } catch { /* vault unavailable — skip */ }
+        }
+      }
+    } catch {
+      // Non-critical — don't block context surfacing on recall tracking errors
+    }
   }
 
   // Routing hint: detect query intent signals and prepend a tool routing directive
@@ -351,6 +429,19 @@ export async function contextSurfacing(
 // Helpers
 // =============================================================================
 
+/**
+ * Log an empty context_usage row for a skipped turn.
+ * Keeps turn_index aligned with transcript turns so per-turn recall
+ * attribution doesn't drift when some prompts are gated.
+ */
+function logEmptyTurn(store: Store, input: HookInput): void {
+  if (!input.sessionId) return;
+  try {
+    const turnIndex = (input as any)._turnIndex ?? 0;
+    logInjection(store, input.sessionId, "context-surfacing", [], 0, turnIndex);
+  } catch { /* non-fatal */ }
+}
+
 /**
  * Detect causal/temporal/discovery signals in the prompt and return a
  * routing hint that makes the correct tool choice salient at the moment

package/src/hooks/feedback-loop.ts

@@ -10,12 +10,18 @@
  */
 
 import type { Store } from "../store.ts";
+import { resolveStore } from "../store.ts";
+import { listVaults } from "../config.ts";
 import type { HookInput, HookOutput } from "../hooks.ts";
 import {
   makeEmptyOutput,
   readTranscript,
   validateTranscriptPath,
 } from "../hooks.ts";
+import {
+  segmentTranscriptIntoTurns,
+  attributeRecallReferences,
+} from "../recall-attribution.ts";
 
 // =============================================================================
 // Handler
@@ -129,6 +135,33 @@ export async function feedbackLoop(
     // Non-critical — don't block feedback loop on utility tracking errors
   }
 
+  // Recall tracking: per-turn attribution using transcript segmentation.
+  // Reads full transcript, segments into turns, zips with context_usage rows,
+  // checks references per-turn rather than session-globally.
+  try {
+    const allMessages = readTranscript(transcriptPath, 500);
+    const turns = segmentTranscriptIntoTurns(allMessages);
+    const usages = store.getUsageForSession(sessionId);
+
+    // General vault attribution
+    attributeRecallReferences(store, sessionId, usages, turns);
+
+    // Cross-vault: attribute recall events in any configured named vaults.
+    // Each vault has its own context_usage rows (mirrored during context-surfacing).
+    const vaultNames = listVaults();
+    for (const vaultName of vaultNames) {
+      try {
+        const vaultStore = resolveStore(vaultName);
+        const vaultUsages = vaultStore.getUsageForSession(sessionId);
+        if (vaultUsages.length > 0) {
+          attributeRecallReferences(vaultStore, sessionId, vaultUsages, turns);
+        }
+      } catch { /* vault unavailable — skip */ }
+    }
+  } catch {
+    // Non-critical — don't block feedback loop on recall tracking errors
+  }
+
   // Silent return — feedback loop doesn't inject context
   return makeEmptyOutput("feedback-loop");
 }
@@ -195,6 +228,13 @@ function trackUtilitySignals(
 // Reference Detection
 // =============================================================================
 
+// Recall attribution logic is in src/recall-attribution.ts
+// (attributeRecallReferences, segmentTranscriptIntoTurns)
+
+// =============================================================================
+// Reference Detection
+// =============================================================================
+
 function checkTitleReference(store: Store, path: string, text: string): boolean {
   try {
     const parts = path.split("/");

package/src/hooks.ts

@@ -385,23 +385,28 @@ export function logInjection(
   sessionId: string,
   hookName: string,
   injectedPaths: string[],
-  estimatedTokens: number
-): void {
+  estimatedTokens: number,
+  turnIndex?: number
+): number {
   try {
-    store.insertUsage({
+    const usageId = store.insertUsage({
       sessionId,
       timestamp: new Date().toISOString(),
       hookName,
       injectedPaths,
       estimatedTokens,
       wasReferenced: 0,
+      turnIndex,
     });
 
     // Record co-activation for all injected paths (E3)
     if (injectedPaths.length >= 2) {
       store.recordCoActivation(injectedPaths);
     }
+
+    return usageId;
   } catch {
     // Non-fatal: don't crash hook if usage logging fails
+    return -1;
   }
 }

package/src/mcp.ts

@@ -2277,6 +2277,11 @@ This is the recommended entry point for ALL memory queries.`,
       const config = loadConfig();
       const policy = config.lifecycle;
 
+      // Recall tracking summary
+      const recallStats = store.getRecallStatsAll(1);
+      const highDiversity = recallStats.filter(r => r.diversityScore >= 0.4 && r.spacingScore >= 0.5 && r.recallCount >= 3);
+      const highNoise = recallStats.filter(r => r.recallCount >= 5 && r.negativeCount > r.recallCount * 0.8);
+
       const lines = [
         `Active: ${stats.active}`,
         `Archived (auto): ${stats.archived}`,
@@ -2286,6 +2291,10 @@ This is the recommended entry point for ALL memory queries.`,
         `Never accessed: ${stats.neverAccessed}`,
         `Oldest access: ${stats.oldestAccess?.slice(0, 10) || "n/a"}`,
         "",
+        `Recall tracking: ${recallStats.length} docs tracked`,
+        `  Pin candidates (high diversity+spacing): ${highDiversity.length}`,
+        `  Snooze candidates (surfaced often, rarely referenced): ${highNoise.length}`,
+        "",
         `Policy: ${policy ? `archive after ${policy.archive_after_days}d, purge after ${policy.purge_after_days ?? "never"}, dry_run=${policy.dry_run}` : "none configured"}`,
       ];
 
@@ -2322,7 +2331,29 @@ This is the recommended entry point for ALL memory queries.`,
         const lines = candidates.map(c =>
           `- ${c.collection}/${c.path} (${c.content_type}, modified ${c.modified_at.slice(0, 10)}, accessed ${c.last_accessed_at?.slice(0, 10) || "never"})`
         );
-        return { content: [{ type: "text", text: `Would archive ${candidates.length} document(s):\n${lines.join("\n") || "(none)"}` }] };
+
+        // Recall-based recommendations
+        const recallStats = store.getRecallStatsAll(3);
+        const pinCandidates = recallStats.filter(r => r.diversityScore >= 0.4 && r.spacingScore >= 0.5 && r.recallCount >= 3);
+        const snoozeCandidates = recallStats.filter(r => r.recallCount >= 5 && r.negativeCount > r.recallCount * 0.8);
+
+        const recallLines: string[] = [];
+        if (pinCandidates.length > 0) {
+          recallLines.push("", "Pin candidates (high diversity, multi-day spread, recall≥3):");
+          for (const r of pinCandidates.slice(0, 5)) {
+            const label = r.collection && r.path ? `${r.collection}/${r.path}` : `doc#${r.docId}`;
+            recallLines.push(`  - ${label} (recalls=${r.recallCount}, queries=${r.uniqueQueries}, days=${r.recallDays}, diversity=${r.diversityScore.toFixed(2)}, spacing=${r.spacingScore.toFixed(2)})`);
+          }
+        }
+        if (snoozeCandidates.length > 0) {
+          recallLines.push("", "Snooze candidates (surfaced often, rarely referenced):");
+          for (const r of snoozeCandidates.slice(0, 5)) {
+            const label = r.collection && r.path ? `${r.collection}/${r.path}` : `doc#${r.docId}`;
+            recallLines.push(`  - ${label} (recalls=${r.recallCount}, referenced=${r.recallCount - r.negativeCount}, noise_ratio=${(r.negativeCount / r.recallCount * 100).toFixed(0)}%)`);
+          }
+        }
+
+        return { content: [{ type: "text", text: `Would archive ${candidates.length} document(s):\n${lines.join("\n") || "(none)"}${recallLines.join("\n")}` }] };
       }
 
       const archived = store.archiveDocuments(candidates.map(c => c.id));

package/src/recall-attribution.ts

@@ -0,0 +1,182 @@
+/**
+ * Recall Attribution — per-turn reference detection for recall tracking.
+ *
+ * Extracted into a standalone module for testability (per GPT 5.4 High review turn 4).
+ *
+ * Architecture:
+ * 1. Segment the transcript into ordered turns (user → assistant pairs)
+ * 2. Zip context_usage rows (by turn_index) with transcript turns (by position)
+ * 3. For each pair, detect references in that turn's assistant text only
+ * 4. Mark recall_events linked to the usage rows whose turn actually cited the doc
+ */
+
+import type { Store, UsageRow } from "./store.ts";
+
+// =============================================================================
+// Types
+// =============================================================================
+
+export type TranscriptTurn = {
+  userText: string;
+  assistantText: string;
+};
+
+// =============================================================================
+// Transcript Segmentation
+// =============================================================================
+
+/**
+ * Segment a flat message array into ordered turns.
+ * A turn starts on each "user" message and includes all following "assistant"
+ * messages until the next "user" message.
+ *
+ * @param messages - Ordered array of {role, content} from transcript JSONL
+ * @returns Ordered array of turns
+ */
+export function segmentTranscriptIntoTurns(
+  messages: { role: string; content: string }[]
+): TranscriptTurn[] {
+  const turns: TranscriptTurn[] = [];
+  let currentUser = "";
+  let currentAssistant = "";
+
+  for (const msg of messages) {
+    if (msg.role === "user") {
+      // New turn: flush previous if it has assistant content
+      if (currentUser || currentAssistant) {
+        turns.push({ userText: currentUser, assistantText: currentAssistant });
+      }
+      currentUser = msg.content;
+      currentAssistant = "";
+    } else if (msg.role === "assistant") {
+      currentAssistant += (currentAssistant ? "\n" : "") + msg.content;
+    }
+    // Ignore system/tool messages for attribution purposes
+  }
+
+  // Flush final turn
+  if (currentUser || currentAssistant) {
+    turns.push({ userText: currentUser, assistantText: currentAssistant });
+  }
+
+  return turns;
+}
+
+// =============================================================================
+// Per-Turn Reference Detection
+// =============================================================================
+
+/**
+ * Check if a displayPath (collection/path) is referenced in text.
+ * Matches by: full path, filename (without extension), or doc title.
+ */
+function isPathReferenced(
+  store: Store,
+  displayPath: string,
+  text: string
+): boolean {
+  if (!text || !displayPath) return false;
+
+  // Full path match
+  if (text.includes(displayPath)) return true;
+
+  // Filename match (without extension, min 4 chars)
+  const filename = displayPath.split("/").pop()?.replace(/\.(md|txt)$/i, "");
+  if (filename && filename.length > 3 && text.toLowerCase().includes(filename.toLowerCase())) {
+    return true;
+  }
+
+  // Title match from DB
+  const parts = displayPath.split("/");
+  if (parts.length >= 2) {
+    const collection = parts[0]!;
+    const docPath = parts.slice(1).join("/");
+    const doc = store.findActiveDocument(collection, docPath);
+    if (doc?.title && doc.title.length >= 5 && text.toLowerCase().includes(doc.title.toLowerCase())) {
+      return true;
+    }
+  }
+
+  return false;
+}
+
+// =============================================================================
+// Attribution Core
+// =============================================================================
+
+/**
+ * Attribute recall events to specific turns using per-turn reference detection.
+ *
+ * For each context_usage row (ordered by turn_index), finds the corresponding
+ * transcript turn and checks which of that turn's injected docs were cited in
+ * that turn's assistant text. Only marks recall_events linked to turns where
+ * the doc was actually referenced.
+ *
+ * @param store - Store instance for doc resolution and event marking
+ * @param sessionId - Session identifier
+ * @param usages - context_usage rows for this session, ordered by turn_index
+ * @param turns - Transcript turns, ordered by position
+ */
+export function attributeRecallReferences(
+  store: Store,
+  sessionId: string,
+  usages: UsageRow[],
+  turns: TranscriptTurn[]
+): void {
+  // Filter to context-surfacing usages only
+  const surfacingUsages = usages.filter(u => u.hookName === "context-surfacing");
+
+  for (const usage of surfacingUsages) {
+    // Match usage to transcript turn by turn_index
+    const turn = turns[usage.turnIndex];
+    if (!turn || !turn.assistantText) continue;
+
+    // Parse injected paths for this turn
+    let injectedPaths: string[];
+    try { injectedPaths = JSON.parse(usage.injectedPaths) as string[]; }
+    catch { continue; }
+    if (injectedPaths.length === 0) continue;
+
+    // Check which docs from THIS turn were referenced in THIS turn's assistant text
+    const referencedDocIds: number[] = [];
+    for (const path of injectedPaths) {
+      if (!isPathReferenced(store, path, turn.assistantText)) continue;
+
+      const parts = path.split("/");
+      if (parts.length < 2) continue;
+      const collection = parts[0]!;
+      const docPath = parts.slice(1).join("/");
+      const doc = store.findActiveDocument(collection, docPath);
+      if (doc) referencedDocIds.push(doc.id);
+    }
+
+    if (referencedDocIds.length === 0) continue;
+
+    // Mark only recall events linked to THIS usage row
+    for (const docId of referencedDocIds) {
+      // Primary: usage_id-linked events (current schema)
+      const linked = store.db.prepare(`
+        SELECT id FROM recall_events
+        WHERE usage_id = ? AND doc_id = ? AND was_referenced = 0
+      `).all(usage.id, docId) as { id: number }[];
+
+      if (linked.length > 0) {
+        const ids = linked.map(r => r.id);
+        const placeholders = ids.map(() => "?").join(",");
+        store.db.prepare(`
+          UPDATE recall_events SET was_referenced = 1
+          WHERE id IN (${placeholders})
+        `).run(...ids);
+      } else {
+        // Fallback: pre-migration events without usage_id — match by turn_index
+        store.db.prepare(`
+          UPDATE recall_events SET was_referenced = 1
+          WHERE id IN (
+            SELECT id FROM recall_events
+            WHERE session_id = ? AND doc_id = ? AND turn_index = ? AND was_referenced = 0
+          )
+        `).run(sessionId, docId, usage.turnIndex);
+      }
+    }
+  }
+}

package/src/recall-buffer.ts

@@ -0,0 +1,85 @@
+/**
+ * Recall Tracking — direct-write recall event recording.
+ *
+ * Context-surfacing writes recall events directly to SQLite (single transaction,
+ * <0.4ms for ~12 rows). This replaces the original in-memory buffer design which
+ * failed in Claude Code mode where each hook is a separate process invocation.
+ *
+ * Per GPT 5.4 High review (Codex turn 1):
+ * - Direct INSERT is preferred over buffer for cross-process correctness
+ * - WAL mode handles concurrent writes safely (busy_timeout=5000ms)
+ * - Negative signals (surfaced but not referenced) marked retroactively by feedback-loop
+ */
+
+import { createHash } from "crypto";
+import type { Store } from "./store.ts";
+
+// =============================================================================
+// Query Hashing
+// =============================================================================
+
+/**
+ * Hash a query string for recall tracking.
+ * SHA1 truncated to 12 hex chars (same as OpenClaw's approach).
+ */
+export function hashQuery(query: string): string {
+  return createHash("sha1")
+    .update(query.toLowerCase().trim())
+    .digest("hex")
+    .slice(0, 12);
+}
+
+// =============================================================================
+// Direct Write (replaces in-memory buffer)
+// =============================================================================
+
+/**
+ * Record surfaced documents as recall events directly to SQLite.
+ * Called from context-surfacing hook — single transaction, ~0.4ms.
+ *
+ * Resolves displayPath → doc_id inline. Docs that can't be resolved
+ * (deleted between search and write) are silently skipped.
+ *
+ * @param store - Store instance with DB access
+ * @param sessionId - Current session identifier
+ * @param queryHash - SHA1 hash of the search query
+ * @param docs - Array of {displayPath, searchScore} for each surfaced result
+ * @returns Number of events recorded
+ */
+export function writeRecallEvents(
+  store: Store,
+  sessionId: string,
+  queryHash: string,
+  docs: { displayPath: string; searchScore: number }[],
+  usageId?: number,
+  turnIndex?: number
+): number {
+  if (!sessionId || docs.length === 0) return 0;
+
+  const resolved: { docId: number; queryHash: string; searchScore: number; sessionId: string }[] = [];
+
+  for (const doc of docs) {
+    const parts = doc.displayPath.split("/");
+    if (parts.length < 2) continue;
+    const collection = parts[0]!;
+    const docPath = parts.slice(1).join("/");
+    const found = store.findActiveDocument(collection, docPath);
+    if (!found) {
+      console.debug?.(`[recall] skipping unresolvable displayPath: ${doc.displayPath}`);
+      continue;
+    }
+
+    resolved.push({
+      docId: found.id,
+      queryHash,
+      searchScore: doc.searchScore,
+      sessionId,
+      usageId,
+      turnIndex,
+    });
+  }
+
+  if (resolved.length === 0) return 0;
+  return store.insertRecallEvents(resolved);
+}
+

package/src/store.ts

@@ -301,6 +301,10 @@ function initializeDatabase(db: Database): void {
   sqliteVec.load(db);
   db.exec("PRAGMA journal_mode = WAL");
   db.exec("PRAGMA foreign_keys = ON");
+  // Set generous busy_timeout during DDL — concurrent Stop hooks (decision-extractor,
+  // handoff-generator, feedback-loop) all run initializeDatabase simultaneously.
+  // 15s is well within the 30s Stop hook timeout. Reset to normal after DDL completes.
+  db.exec("PRAGMA busy_timeout = 15000");
 
   // Drop legacy tables that are now managed in YAML
   db.exec(`DROP TABLE IF EXISTS path_contexts`);
@@ -491,11 +495,18 @@ function initializeDatabase(db: Database): void {
       hook_name TEXT NOT NULL,
       injected_paths TEXT NOT NULL DEFAULT '[]',
       estimated_tokens INTEGER NOT NULL DEFAULT 0,
-      was_referenced INTEGER NOT NULL DEFAULT 0
+      was_referenced INTEGER NOT NULL DEFAULT 0,
+      turn_index INTEGER NOT NULL DEFAULT 0
     )
   `);
   db.exec(`CREATE INDEX IF NOT EXISTS idx_context_usage_session ON context_usage(session_id)`);
 
+  // Migration: add turn_index to existing context_usage
+  const cuCols = db.prepare("PRAGMA table_info(context_usage)").all() as { name: string }[];
+  if (!cuCols.some(c => c.name === "turn_index")) {
+    try { db.exec(`ALTER TABLE context_usage ADD COLUMN turn_index INTEGER NOT NULL DEFAULT 0`); } catch { /* exists */ }
+  }
+
   // Hook prompt dedupe: suppress duplicate/heartbeat prompts to reduce GPU churn.
   db.exec(`
     CREATE TABLE IF NOT EXISTS hook_dedupe (
@@ -785,6 +796,64 @@ function initializeDatabase(db: Database): void {
   `);
 
   db.exec(`CREATE INDEX IF NOT EXISTS idx_intent_cache_time ON intent_classifications(cached_at)`);
+
+  // Recall tracking: append-only event log for every doc surfaced by retrieval
+  // usage_id is informational (no FK) — links to context_usage.id in the same vault
+  // but may reference a different vault's row in cross-vault scenarios.
+  // Cross-vault linkage uses session_id + turn_index instead.
+  db.exec(`
+    CREATE TABLE IF NOT EXISTS recall_events (
+      id INTEGER PRIMARY KEY AUTOINCREMENT,
+      doc_id INTEGER NOT NULL,
+      query_hash TEXT NOT NULL,
+      search_score REAL NOT NULL,
+      session_id TEXT NOT NULL,
+      usage_id INTEGER,
+      turn_index INTEGER NOT NULL DEFAULT 0,
+      surfaced_at TEXT NOT NULL DEFAULT (datetime('now')),
+      was_referenced INTEGER NOT NULL DEFAULT 0,
+      FOREIGN KEY (doc_id) REFERENCES documents(id) ON DELETE CASCADE
+    )
+  `);
+  // Migration: add usage_id + turn_index columns to existing recall_events tables
+  const reCols = db.prepare("PRAGMA table_info(recall_events)").all() as { name: string }[];
+  const reColNames = new Set(reCols.map(c => c.name));
+  if (!reColNames.has("usage_id")) {
+    try { db.exec(`ALTER TABLE recall_events ADD COLUMN usage_id INTEGER`); } catch { /* exists */ }
+  }
+  if (!reColNames.has("turn_index")) {
+    try { db.exec(`ALTER TABLE recall_events ADD COLUMN turn_index INTEGER NOT NULL DEFAULT 0`); } catch { /* exists */ }
+  }
+  db.exec(`CREATE INDEX IF NOT EXISTS idx_recall_events_usage ON recall_events(usage_id)`);
+  db.exec(`CREATE INDEX IF NOT EXISTS idx_recall_events_doc ON recall_events(doc_id)`);
+  db.exec(`CREATE INDEX IF NOT EXISTS idx_recall_events_session ON recall_events(session_id)`);
+  db.exec(`CREATE INDEX IF NOT EXISTS idx_recall_events_surfaced ON recall_events(surfaced_at)`);
+
+  // Recall stats: derived summary recomputed by background worker
+  db.exec(`
+    CREATE TABLE IF NOT EXISTS recall_stats (
+      doc_id INTEGER PRIMARY KEY,
+      recall_count INTEGER NOT NULL DEFAULT 0,
+      unique_queries INTEGER NOT NULL DEFAULT 0,
+      recall_days INTEGER NOT NULL DEFAULT 0,
+      total_score REAL NOT NULL DEFAULT 0,
+      max_score REAL NOT NULL DEFAULT 0,
+      first_recalled_at TEXT,
+      last_recalled_at TEXT,
+      diversity_score REAL NOT NULL DEFAULT 0,
+      spacing_score REAL NOT NULL DEFAULT 0,
+      negative_count INTEGER NOT NULL DEFAULT 0,
+      updated_at TEXT NOT NULL DEFAULT (datetime('now')),
+      FOREIGN KEY (doc_id) REFERENCES documents(id) ON DELETE CASCADE
+    )
+  `);
+
+  // Migration: add contradict_confidence to memory_relations
+  const mrCols = db.prepare("PRAGMA table_info(memory_relations)").all() as { name: string }[];
+  const mrColNames = new Set(mrCols.map(c => c.name));
+  if (!mrColNames.has("contradict_confidence")) {
+    try { db.exec(`ALTER TABLE memory_relations ADD COLUMN contradict_confidence REAL`); } catch { /* column exists */ }
+  }
 }
 
 
@@ -898,7 +967,7 @@ export type Store = {
   getRecentSessions: (limit: number) => SessionRecord[];
 
   // SAME: Context usage tracking
-  insertUsage: (usage: UsageRecord) => void;
+  insertUsage: (usage: UsageRecord) => number;
   getUsageForSession: (sessionId: string) => UsageRow[];
   markUsageReferenced: (id: number) => void;
 
@@ -944,6 +1013,13 @@ export type Store = {
   queryEntityTriples: (entityId: string, options?: { asOf?: string; direction?: "outgoing" | "incoming" | "both" }) => { id: number; direction: string; subject: string; predicate: string; object: string; validFrom: string | null; validTo: string | null; confidence: number; current: boolean }[];
   getTripleStats: () => { totalTriples: number; currentFacts: number; expiredFacts: number; predicateTypes: string[] };
 
+  // Recall tracking
+  insertRecallEvents: (events: { docId: number; queryHash: string; searchScore: number; sessionId: string; usageId?: number; turnIndex?: number; wasReferenced?: boolean }[]) => number;
+  recomputeRecallStats: () => number;
+  getRecallStats: (docId: number) => RecallStatsRow | null;
+  getRecallStatsAll: (minRecallCount?: number) => RecallStatsRow[];
+  markRecallEventsReferenced: (sessionId: string, docIds: number[]) => void;
+
   // Co-activation tracking
   recordCoActivation: (paths: string[]) => void;
   getCoActivated: (path: string, limit?: number) => { path: string; count: number }[];
@@ -987,9 +1063,9 @@ export function createStore(dbPath?: string, opts?: { readonly?: boolean; busyTi
     db.exec("PRAGMA journal_mode = WAL");
     db.exec("PRAGMA query_only = ON");
   }
-  if (opts?.busyTimeout !== undefined) {
-    db.exec(`PRAGMA busy_timeout = ${opts.busyTimeout}`);
-  }
+  // Reset busy_timeout to operational value after DDL init (which uses 15s).
+  // Default 5000ms for normal operations — callers can override via opts.
+  db.exec(`PRAGMA busy_timeout = ${opts?.busyTimeout ?? 5000}`);
 
   return {
     db,
@@ -1075,7 +1151,7 @@ export function createStore(dbPath?: string, opts?: { readonly?: boolean; busyTi
     getRecentSessions: (limit: number) => getRecentSessionsFn(db, limit),
 
     // SAME: Context usage tracking
-    insertUsage: (usage: UsageRecord) => insertUsageFn(db, usage),
+    insertUsage: (usage: UsageRecord) => insertUsageFn(db, usage) as number,
     getUsageForSession: (sessionId: string) => getUsageForSessionFn(db, sessionId),
     markUsageReferenced: (id: number) => markUsageReferencedFn(db, id),
 
@@ -1216,6 +1292,165 @@ export function createStore(dbPath?: string, opts?: { readonly?: boolean; busyTi
     },
 
     // Co-activation tracking
+    // Recall tracking: batch insert surfacing events
+    insertRecallEvents: (events: { docId: number; queryHash: string; searchScore: number; sessionId: string; usageId?: number; turnIndex?: number; wasReferenced?: boolean }[]) => {
+      if (events.length === 0) return 0;
+      const stmt = db.prepare(`
+        INSERT INTO recall_events (doc_id, query_hash, search_score, session_id, usage_id, turn_index, surfaced_at, was_referenced)
+        VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+      `);
+      const now = new Date().toISOString();
+      const tx = db.transaction(() => {
+        for (const e of events) {
+          stmt.run(e.docId, e.queryHash, e.searchScore, e.sessionId, e.usageId ?? null, e.turnIndex ?? 0, now, e.wasReferenced ? 1 : 0);
+        }
+      });
+      tx();
+      return events.length;
+    },
+
+    // Recall tracking: recompute derived stats from events
+    // Uses SQL GROUP BY for aggregation (O(1) queries), then JS for diversity/spacing formulas
+    recomputeRecallStats: () => {
+      const aggregated = db.prepare(`
+        SELECT
+          doc_id,
+          COUNT(*) AS recall_count,
+          COUNT(DISTINCT query_hash) AS unique_queries,
+          COUNT(DISTINCT date(surfaced_at, 'utc')) AS recall_days,
+          SUM(search_score) AS total_score,
+          MAX(search_score) AS max_score,
+          SUM(CASE WHEN was_referenced = 0 THEN 1 ELSE 0 END) AS negative_count,
+          MIN(surfaced_at) AS first_recalled_at,
+          MAX(surfaced_at) AS last_recalled_at,
+          GROUP_CONCAT(DISTINCT date(surfaced_at, 'utc')) AS day_list
+        FROM recall_events
+        GROUP BY doc_id
+      `).all() as {
+        doc_id: number; recall_count: number; unique_queries: number; recall_days: number;
+        total_score: number; max_score: number; negative_count: number;
+        first_recalled_at: string; last_recalled_at: string; day_list: string;
+      }[];
+
+      if (aggregated.length === 0) return 0;
+
+      const upsert = db.prepare(`
+        INSERT INTO recall_stats (doc_id, recall_count, unique_queries, recall_days, total_score, max_score,
+          first_recalled_at, last_recalled_at, diversity_score, spacing_score, negative_count, updated_at)
+        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+        ON CONFLICT(doc_id) DO UPDATE SET
+          recall_count = excluded.recall_count,
+          unique_queries = excluded.unique_queries,
+          recall_days = excluded.recall_days,
+          total_score = excluded.total_score,
+          max_score = excluded.max_score,
+          first_recalled_at = excluded.first_recalled_at,
+          last_recalled_at = excluded.last_recalled_at,
+          diversity_score = excluded.diversity_score,
+          spacing_score = excluded.spacing_score,
+          negative_count = excluded.negative_count,
+          updated_at = excluded.updated_at
+      `);
+
+      const now = new Date().toISOString();
+      const tx = db.transaction(() => {
+        for (const row of aggregated) {
+          // Diversity: clamped max(uniqueQueries, recallDays) / 5
+          const diversityScore = Math.min(1, Math.max(row.unique_queries, row.recall_days) / 5);
+
+          // Spacing: multi-day spread
+          let spacingScore = 0;
+          if (row.recall_days > 1 && row.day_list) {
+            const days = row.day_list.split(",").sort();
+            const spacing = Math.min(1, Math.log1p(days.length - 1) / Math.log1p(4));
+            const firstDay = new Date(days[0]! + "T00:00:00Z").getTime();
+            const lastDay = new Date(days[days.length - 1]! + "T00:00:00Z").getTime();
+            const spanDays = Math.max(0, (lastDay - firstDay) / (24 * 60 * 60 * 1000));
+            const span = Math.min(1, spanDays / 7);
+            spacingScore = Math.min(1, 0.55 * spacing + 0.45 * span);
+          } else if (row.recall_days === 1) {
+            spacingScore = 0.2;
+          }
+
+          upsert.run(
+            row.doc_id, row.recall_count, row.unique_queries, row.recall_days,
+            row.total_score, row.max_score,
+            row.first_recalled_at, row.last_recalled_at,
+            diversityScore, spacingScore, row.negative_count, now
+          );
+        }
+      });
+      tx();
+      return aggregated.length;
+    },
+
+    getRecallStats: (docId: number) => {
+      const row = db.prepare(`SELECT * FROM recall_stats WHERE doc_id = ?`).get(docId) as any;
+      if (!row) return null;
+      return {
+        docId: row.doc_id,
+        recallCount: row.recall_count,
+        uniqueQueries: row.unique_queries,
+        recallDays: row.recall_days,
+        totalScore: row.total_score,
+        maxScore: row.max_score,
+        firstRecalledAt: row.first_recalled_at,
+        lastRecalledAt: row.last_recalled_at,
+        diversityScore: row.diversity_score,
+        spacingScore: row.spacing_score,
+        negativeCount: row.negative_count,
+        updatedAt: row.updated_at,
+      } as RecallStatsRow;
+    },
+
+    getRecallStatsAll: (minRecallCount: number = 1) => {
+      return (db.prepare(`
+        SELECT rs.*, d.collection, d.path, d.title
+        FROM recall_stats rs
+        JOIN documents d ON rs.doc_id = d.id
+        WHERE rs.recall_count >= ? AND d.active = 1
+        ORDER BY rs.recall_count DESC
+      `).all(minRecallCount) as any[]).map(row => ({
+        docId: row.doc_id,
+        recallCount: row.recall_count,
+        uniqueQueries: row.unique_queries,
+        recallDays: row.recall_days,
+        totalScore: row.total_score,
+        maxScore: row.max_score,
+        firstRecalledAt: row.first_recalled_at,
+        lastRecalledAt: row.last_recalled_at,
+        diversityScore: row.diversity_score,
+        spacingScore: row.spacing_score,
+        negativeCount: row.negative_count,
+        updatedAt: row.updated_at,
+        collection: row.collection,
+        path: row.path,
+        title: row.title,
+      } as RecallStatsRow));
+    },
+
+    markRecallEventsReferenced: (sessionId: string, docIds: number[]) => {
+      if (docIds.length === 0) return;
+      // Mark only the LATEST event per doc in this session, not all events.
+      // This preserves negative signals: if a doc was surfaced across 5 prompts
+      // but only cited once, 4 events stay was_referenced=0 (genuine negatives).
+      const stmt = db.prepare(`
+        UPDATE recall_events SET was_referenced = 1
+        WHERE id = (
+          SELECT id FROM recall_events
+          WHERE session_id = ? AND doc_id = ?
+          ORDER BY surfaced_at DESC
+          LIMIT 1
+        )
+      `);
+      const tx = db.transaction(() => {
+        for (const docId of docIds) {
+          stmt.run(sessionId, docId);
+        }
+      });
+      tx();
+    },
+
     recordCoActivation: (paths: string[]) => {
       if (paths.length < 2) return;
       const now = new Date().toISOString();
@@ -1451,6 +1686,7 @@ export type UsageRecord = {
   injectedPaths: string[];
   estimatedTokens: number;
   wasReferenced: number;
+  turnIndex?: number;
 };
 
 export type UsageRow = {
@@ -1461,6 +1697,26 @@ export type UsageRow = {
   injectedPaths: string;
   estimatedTokens: number;
   wasReferenced: number;
+  turnIndex: number;
+};
+
+export type RecallStatsRow = {
+  docId: number;
+  recallCount: number;
+  uniqueQueries: number;
+  recallDays: number;
+  totalScore: number;
+  maxScore: number;
+  firstRecalledAt: string | null;
+  lastRecalledAt: string | null;
+  diversityScore: number;
+  spacingScore: number;
+  negativeCount: number;
+  updatedAt: string;
+  // Joined from documents (only populated by getRecallStatsAll)
+  collection?: string;
+  path?: string;
+  title?: string;
 };
 
 export type DocumentRow = {
@@ -3647,19 +3903,22 @@ function getRecentSessionsFn(db: Database, limit: number): SessionRecord[] {
 // SAME: Context Usage Tracking
 // =============================================================================
 
-function insertUsageFn(db: Database, usage: UsageRecord): void {
+function insertUsageFn(db: Database, usage: UsageRecord): number {
   db.prepare(`
-    INSERT INTO context_usage (session_id, timestamp, hook_name, injected_paths, estimated_tokens, was_referenced)
-    VALUES (?, ?, ?, ?, ?, ?)
-  `).run(usage.sessionId, usage.timestamp, usage.hookName, JSON.stringify(usage.injectedPaths), usage.estimatedTokens, usage.wasReferenced);
+    INSERT INTO context_usage (session_id, timestamp, hook_name, injected_paths, estimated_tokens, was_referenced, turn_index)
+    VALUES (?, ?, ?, ?, ?, ?, ?)
+  `).run(usage.sessionId, usage.timestamp, usage.hookName, JSON.stringify(usage.injectedPaths), usage.estimatedTokens, usage.wasReferenced, usage.turnIndex ?? 0);
+  // Return the rowid of the just-inserted row for recall event linkage
+  const row = db.prepare("SELECT last_insert_rowid() as id").get() as { id: number };
+  return row.id;
 }
 
 function getUsageForSessionFn(db: Database, sessionId: string): UsageRow[] {
   return db.prepare(`
     SELECT id, session_id AS sessionId, timestamp, hook_name AS hookName,
            injected_paths AS injectedPaths, estimated_tokens AS estimatedTokens,
-           was_referenced AS wasReferenced
-    FROM context_usage WHERE session_id = ? ORDER BY timestamp
+           was_referenced AS wasReferenced, turn_index AS turnIndex
+    FROM context_usage WHERE session_id = ? ORDER BY turn_index, timestamp
   `).all(sessionId) as UsageRow[];
 }