clawmem 0.1.7 → 0.2.0

package/AGENTS.md

@@ -96,6 +96,7 @@ curl http://host:8090/v1/models
 | `CLAWMEM_ENABLE_AMEM` | enabled | A-MEM note construction + link generation during indexing. |
 | `CLAWMEM_ENABLE_CONSOLIDATION` | disabled | Background worker backfills unenriched docs. Needs long-lived MCP process. |
 | `CLAWMEM_CONSOLIDATION_INTERVAL` | 300000 | Worker interval in ms (min 15000). |
+| `CLAWMEM_NUDGE_INTERVAL` | `15` | Prompts between lifecycle tool use before `<vault-nudge>` injection. 0 to disable. |
 
 **Note:** The `bin/clawmem` wrapper sets all endpoint defaults. Always use the wrapper — never `bun run src/clawmem.ts` directly. For remote GPU setups, add the same env vars to the watcher service via a systemd drop-in.
 
@@ -415,6 +416,8 @@ Note: intent disables BM25 strong-signal bypass, forcing full expansion+rerankin
 
 ## Composite Scoring (automatic, applied to all search tools)
 
+**Observation invalidation (v0.2.0):** Documents with `invalidated_at` set are excluded from search results. Soft invalidation uses `invalidated_at`, `invalidated_by`, and `superseded_by` columns. When `decision-extractor` detects a contradiction that drops confidence ≤ 0.2, the observation is marked invalidated. Invalidated docs can be restored via lifecycle tools.
+
 ```
 compositeScore = (0.50 × searchScore + 0.25 × recencyScore + 0.25 × confidenceScore) × qualityMultiplier × coActivationBoost
 ```
@@ -482,11 +485,15 @@ The `memory_relations` table is populated by multiple independent sources:
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP tool or watcher (.beads/ change) | Queries `bd` CLI (Dolt backend). Maps beads deps: blocks→causal, discovered-from→supporting, relates-to→semantic, plus conditional-blocks→causal, caused-by→causal, supersedes→supporting. Metadata: `{origin: "beads"}`. |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
+| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → canonical normalization (FTS5 + Levenshtein) → `entity_mentions` + `entity_cooccurrences` tables. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
+| `consolidated_observations` | supporting | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.
 
 **Graph traversal asymmetry:** `adaptiveTraversal()` traverses all edge types outbound (source→target) but only `semantic` and `entity` edges inbound (target→source). Temporal and causal edges are directional only.
 
+**MPFP graph retrieval (v0.2.0):** Multi-Path Fact Propagation runs predefined meta-path patterns in parallel (`[semantic, semantic]`, `[entity, temporal]`, `[semantic, causal]`, etc.), fuses via RRF. Hop-synchronized edge cache batches DB queries per hop instead of per pattern. Forward Push with α=0.15 teleport probability bounds active nodes sublinearly. Tier 3 only (`query`/`intent_search`), not hooks. Patterns selected per MAGMA intent: WHY → `[semantic, causal]`, ENTITY → `[entity, semantic]`, WHEN → `[temporal, semantic]`.
+
 ### When to Run `build_graphs`
 
 - After **bulk ingestion** (many new docs at once) — adds temporal backbone and fills semantic gaps where A-MEM links are sparse.
@@ -505,11 +512,14 @@ The `memory_relations` table is populated by multiple independent sources:
 
 ```
 User Query + optional intent hint
+  → Temporal Extraction (regex date range from query: "last week", "March 2026" → WHERE modified_at BETWEEN filters)
   → BM25 Probe → Strong Signal Check (skip expansion if top hit ≥ 0.85 with gap ≥ 0.15; disabled when intent provided)
   → Query Expansion (LLM generates text variants; intent steers expansion prompt)
   → Parallel: BM25(original) + Vector(original) + BM25(each expanded) + Vector(each expanded)
+       + Temporal Proximity (date-range filtered, if temporal constraint extracted)
+       + Entity Graph (conditional 1-hop entity walk from top seeds, if entity signals present)
   → Original query lists get positional 2× weight in RRF; expanded get 1×
-  → Reciprocal Rank Fusion (k=60, top candidateLimit)
+  → Reciprocal Rank Fusion (k=60, top candidateLimit, up to 4-way parallel legs)
   → Intent-Aware Chunk Selection (intent terms at 0.5× weight alongside query terms at 1.0×)
   → Cross-Encoder Reranking (4000 char context; intent prepended to rerank query; chunk dedup; batch cap=4)
   → Position-Aware Blending (α=0.75 top3, 0.60 mid, 0.40 tail)
@@ -662,6 +672,8 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
 
 ## Integration Notes
 
+- **Memory nudge (v0.2.0):** Every N prompts (default 15) without a lifecycle MCP tool call (`memory_pin`/`memory_forget`/`memory_snooze`), context-surfacing appends `<vault-nudge>` prompting proactive memory management. Counter resets on lifecycle tool use. Configure via `CLAWMEM_NUDGE_INTERVAL` (0 to disable).
+- **Entity resolution (v0.2.0):** A-MEM enrichment now extracts named entities via LLM, resolves to canonical forms using `entities_fts` + Levenshtein fuzzy matching, and tracks co-occurrence. Entity graph traversal available for ENTITY intent queries via `intent_search`.
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
 - Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.

package/CLAUDE.md

@@ -96,6 +96,7 @@ curl http://host:8090/v1/models
 | `CLAWMEM_ENABLE_AMEM` | enabled | A-MEM note construction + link generation during indexing. |
 | `CLAWMEM_ENABLE_CONSOLIDATION` | disabled | Background worker backfills unenriched docs. Needs long-lived MCP process. |
 | `CLAWMEM_CONSOLIDATION_INTERVAL` | 300000 | Worker interval in ms (min 15000). |
+| `CLAWMEM_NUDGE_INTERVAL` | `15` | Prompts between lifecycle tool use before `<vault-nudge>` injection. 0 to disable. |
 
 **Note:** The `bin/clawmem` wrapper sets all endpoint defaults. Always use the wrapper — never `bun run src/clawmem.ts` directly. For remote GPU setups, add the same env vars to the watcher service via a systemd drop-in.
 
@@ -415,6 +416,8 @@ Note: intent disables BM25 strong-signal bypass, forcing full expansion+rerankin
 
 ## Composite Scoring (automatic, applied to all search tools)
 
+**Observation invalidation (v0.2.0):** Documents with `invalidated_at` set are excluded from search results. Soft invalidation uses `invalidated_at`, `invalidated_by`, and `superseded_by` columns. When `decision-extractor` detects a contradiction that drops confidence ≤ 0.2, the observation is marked invalidated. Invalidated docs can be restored via lifecycle tools.
+
 ```
 compositeScore = (0.50 × searchScore + 0.25 × recencyScore + 0.25 × confidenceScore) × qualityMultiplier × coActivationBoost
 ```
@@ -482,11 +485,15 @@ The `memory_relations` table is populated by multiple independent sources:
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP tool or watcher (.beads/ change) | Queries `bd` CLI (Dolt backend). Maps beads deps: blocks→causal, discovered-from→supporting, relates-to→semantic, plus conditional-blocks→causal, caused-by→causal, supersedes→supporting. Metadata: `{origin: "beads"}`. |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
+| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → canonical normalization (FTS5 + Levenshtein) → `entity_mentions` + `entity_cooccurrences` tables. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
+| `consolidated_observations` | supporting | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.
 
 **Graph traversal asymmetry:** `adaptiveTraversal()` traverses all edge types outbound (source→target) but only `semantic` and `entity` edges inbound (target→source). Temporal and causal edges are directional only.
 
+**MPFP graph retrieval (v0.2.0):** Multi-Path Fact Propagation runs predefined meta-path patterns in parallel (`[semantic, semantic]`, `[entity, temporal]`, `[semantic, causal]`, etc.), fuses via RRF. Hop-synchronized edge cache batches DB queries per hop instead of per pattern. Forward Push with α=0.15 teleport probability bounds active nodes sublinearly. Tier 3 only (`query`/`intent_search`), not hooks. Patterns selected per MAGMA intent: WHY → `[semantic, causal]`, ENTITY → `[entity, semantic]`, WHEN → `[temporal, semantic]`.
+
 ### When to Run `build_graphs`
 
 - After **bulk ingestion** (many new docs at once) — adds temporal backbone and fills semantic gaps where A-MEM links are sparse.
@@ -505,11 +512,14 @@ The `memory_relations` table is populated by multiple independent sources:
 
 ```
 User Query + optional intent hint
+  → Temporal Extraction (regex date range from query: "last week", "March 2026" → WHERE modified_at BETWEEN filters)
   → BM25 Probe → Strong Signal Check (skip expansion if top hit ≥ 0.85 with gap ≥ 0.15; disabled when intent provided)
   → Query Expansion (LLM generates text variants; intent steers expansion prompt)
   → Parallel: BM25(original) + Vector(original) + BM25(each expanded) + Vector(each expanded)
+       + Temporal Proximity (date-range filtered, if temporal constraint extracted)
+       + Entity Graph (conditional 1-hop entity walk from top seeds, if entity signals present)
   → Original query lists get positional 2× weight in RRF; expanded get 1×
-  → Reciprocal Rank Fusion (k=60, top candidateLimit)
+  → Reciprocal Rank Fusion (k=60, top candidateLimit, up to 4-way parallel legs)
   → Intent-Aware Chunk Selection (intent terms at 0.5× weight alongside query terms at 1.0×)
   → Cross-Encoder Reranking (4000 char context; intent prepended to rerank query; chunk dedup; batch cap=4)
   → Position-Aware Blending (α=0.75 top3, 0.60 mid, 0.40 tail)
@@ -662,6 +672,8 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
 
 ## Integration Notes
 
+- **Memory nudge (v0.2.0):** Every N prompts (default 15) without a lifecycle MCP tool call (`memory_pin`/`memory_forget`/`memory_snooze`), context-surfacing appends `<vault-nudge>` prompting proactive memory management. Counter resets on lifecycle tool use. Configure via `CLAWMEM_NUDGE_INTERVAL` (0 to disable).
+- **Entity resolution (v0.2.0):** A-MEM enrichment now extracts named entities via LLM, resolves to canonical forms using `entities_fts` + Levenshtein fuzzy matching, and tracks co-occurrence. Entity graph traversal available for ENTITY intent queries via `intent_search`.
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
 - Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.

package/README.md

@@ -42,6 +42,18 @@ ClawMem turns your markdown notes, project docs, and research dumps into persist
 
 Runs fully local with no API keys and no cloud services. Integrates via Claude Code hooks and MCP tools, or as an OpenClaw ContextEngine plugin. Both modes share the same vault for cross-runtime memory. Works with any MCP-compatible client.
 
+### v0.2.0 Enhancements
+
+Seven patterns extracted from competitor analysis ([Hindsight](https://github.com/vectorize-io/hindsight), [Hermes Agent](https://github.com/NousResearch/hermes-agent), [claude-mem](https://github.com/thedotmack/claude-mem)):
+
+- **Entity resolution + co-occurrence graph** — LLM entity extraction during A-MEM enrichment, canonical normalization via FTS5 + Levenshtein fuzzy matching, co-occurrence tracking, entity graph traversal for ENTITY intent queries
+- **MPFP graph retrieval** — Multi-Path Fact Propagation with meta-path patterns per intent, hop-synchronized edge cache, Forward Push with α=0.15 teleport probability. Replaces single-beam traversal for causal/entity/temporal queries.
+- **Temporal query extraction** — regex-based date range extraction from natural language queries ("last week", "March 2026"), wired as WHERE filters into BM25 and vector search
+- **4-way parallel retrieval** — temporal proximity and entity graph channels added as parallel RRF legs in `query` tool (Tier 3 only), alongside existing BM25 + vector channels
+- **3-tier consolidation** — facts to observations (auto-generated, with proof_count and trend enum) to mental models. Background worker synthesizes clusters of related observations into consolidated patterns.
+- **Observation invalidation** — soft invalidation (invalidated_at/invalidated_by/superseded_by columns). Observations with confidence ≤ 0.2 after contradiction are filtered from search results.
+- **Memory nudge** — periodic ephemeral `<vault-nudge>` injection prompting lifecycle tool use after N turns of inactivity. Configurable via `CLAWMEM_NUDGE_INTERVAL`.
+
 ## Architecture
 
 <p align="center">

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.1.7",
+  "version": "0.2.0",
   "description": "On-device context engine and memory for AI agents. Claude Code and OpenClaw. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {

package/src/amem.ts

@@ -8,6 +8,7 @@
 import type { Database } from "bun:sqlite";
 import type { LlamaCpp } from "./llm.ts";
 import type { Store } from "./store.ts";
+import { enrichDocumentEntities } from "./entity.ts";
 
 export interface MemoryNote {
   keywords: string[];
@@ -612,11 +613,21 @@ export async function postIndexEnrich(
       return;
     }
 
-    // Step 2: Generate memory links (new documents only)
+    // Step 2: Entity extraction + resolution + co-occurrence (new documents only)
+    try {
+      const entityCount = await enrichDocumentEntities(store.db, llm, docId);
+      if (entityCount > 0) {
+        console.log(`[amem] Resolved ${entityCount} entities for docId ${docId}`);
+      }
+    } catch (err) {
+      console.log(`[amem] Entity enrichment failed for docId ${docId}:`, err);
+    }
+
+    // Step 3: Generate memory links (new documents only)
     const linksCreated = await generateMemoryLinks(store, llm, docId);
     console.log(`[amem] Created ${linksCreated} links for docId ${docId}`);
 
-    // Step 3: Evolve memories based on new evidence (new documents only)
+    // Step 4: Evolve memories based on new evidence (new documents only)
     // The new document triggers evolution of its linked neighbors
     if (linksCreated > 0) {
       // Get neighbors this new document links to (outbound links from generateMemoryLinks)

package/src/clawmem.ts

@@ -1237,8 +1237,22 @@ async function cmdWatch() {
     // Skill vault not configured — skip
   }
 
+  // Periodic WAL checkpoint: the watcher holds a long-lived DB connection which
+  // prevents SQLite auto-checkpoint from shrinking the WAL file. Without this,
+  // the WAL grows unbounded (observed 77MB+), slowing every concurrent DB access
+  // (hooks, MCP) and eventually causing UserPromptSubmit hook timeouts.
+  const WAL_CHECKPOINT_INTERVAL = 5 * 60 * 1000; // 5 minutes
+  const checkpointTimer = setInterval(() => {
+    try {
+      s.db.exec("PRAGMA wal_checkpoint(PASSIVE)");
+    } catch {
+      // Checkpoint failed (busy) — will retry next interval
+    }
+  }, WAL_CHECKPOINT_INTERVAL);
+
   // Keep running until Ctrl+C
   process.on("SIGINT", () => {
+    clearInterval(checkpointTimer);
     watcher.close();
     skillWatcher?.close();
     closeStore();

package/src/consolidation.ts

@@ -1,12 +1,17 @@
 /**
  * ClawMem Consolidation Worker
  *
- * Background worker that enriches documents missing A-MEM metadata.
- * Runs periodically to backfill memory notes for documents indexed before A-MEM.
+ * Two-phase background worker:
+ * 1. A-MEM backfill: enriches documents missing memory notes
+ * 2. 3-tier consolidation: synthesizes clusters of related observations
+ *    into higher-order consolidated observations with proof counts and trends
+ *
+ * Pattern H from ENHANCEMENT-PLAN.md (source: Hindsight consolidator.py)
  */
 
 import type { Store } from "./store.ts";
 import type { LlamaCpp } from "./llm.ts";
+import { extractJsonFromLLM } from "./amem.ts";
 
 // =============================================================================
 // Types
@@ -18,19 +23,40 @@ interface DocumentToEnrich {
   title: string;
 }
 
+export type TrendEnum = 'NEW' | 'STABLE' | 'STRENGTHENING' | 'WEAKENING' | 'STALE';
+
+export interface ConsolidatedObservation {
+  id: number;
+  observation: string;
+  proof_count: number;
+  source_doc_ids: number[];
+  trend: TrendEnum;
+  status: string;
+  created_at: string;
+  updated_at: string;
+  collection: string | null;
+}
+
+interface ObservationCluster {
+  docs: { id: number; title: string; facts: string; context: string; modified_at: string }[];
+  collection: string;
+}
+
 // =============================================================================
 // Worker State
 // =============================================================================
 
 let consolidationTimer: Timer | null = null;
 let isRunning = false;
+let tickCount = 0;
 
 // =============================================================================
 // Worker Functions
 // =============================================================================
 
 /**
- * Starts the consolidation worker that enriches documents missing A-MEM metadata.
+ * Starts the consolidation worker that enriches documents missing A-MEM metadata
+ * and periodically consolidates observations.
  *
  * @param store - Store instance with A-MEM methods
  * @param llm - LLM instance for memory note construction
@@ -69,7 +95,7 @@ export function stopConsolidationWorker(): void {
 }
 
 /**
- * Single worker tick: find and enrich up to 3 documents missing A-MEM metadata.
+ * Single worker tick: A-MEM backfill + periodic observation consolidation.
  */
 async function tick(store: Store, llm: LlamaCpp): Promise<void> {
   // Reentrancy guard
@@ -79,45 +105,337 @@ async function tick(store: Store, llm: LlamaCpp): Promise<void> {
   }
 
   isRunning = true;
+  tickCount++;
 
   try {
-    // Find documents missing A-MEM keywords (primary indicator of unenriched docs)
-    const docs = store.db
-      .prepare<DocumentToEnrich, []>(
-        `SELECT id, hash, title
-         FROM documents
-         WHERE amem_keywords IS NULL AND active = 1
-         ORDER BY created_at ASC
-         LIMIT 3`
+    // Phase 1: A-MEM backfill (every tick)
+    await backfillAmem(store, llm);
+
+    // Phase 2: Observation consolidation (every 6th tick, ~30 min at default interval)
+    if (tickCount % 6 === 0) {
+      await consolidateObservations(store, llm);
+    }
+  } catch (err) {
+    console.error("[consolidation] Tick failed:", err);
+  } finally {
+    isRunning = false;
+  }
+}
+
+/**
+ * Phase 1: Find and enrich up to 3 documents missing A-MEM metadata.
+ */
+async function backfillAmem(store: Store, llm: LlamaCpp): Promise<void> {
+  const docs = store.db
+    .prepare<DocumentToEnrich, []>(
+      `SELECT id, hash, title
+       FROM documents
+       WHERE amem_keywords IS NULL AND active = 1
+       ORDER BY created_at ASC
+       LIMIT 3`
+    )
+    .all();
+
+  if (docs.length === 0) return;
+
+  console.log(`[consolidation] Enriching ${docs.length} documents`);
+
+  for (const doc of docs) {
+    try {
+      const note = await store.constructMemoryNote(llm, doc.id);
+      await store.storeMemoryNote(doc.id, note);
+      await store.generateMemoryLinks(llm, doc.id);
+      console.log(`[consolidation] Enriched doc ${doc.id} (${doc.title})`);
+    } catch (err) {
+      console.error(`[consolidation] Failed to enrich doc ${doc.id}:`, err);
+    }
+  }
+}
+
+// =============================================================================
+// Phase 2: 3-Tier Observation Consolidation
+// =============================================================================
+
+/**
+ * Find clusters of related observations and synthesize into consolidated observations.
+ * Runs per-collection to prevent cross-vault false merges.
+ */
+async function consolidateObservations(store: Store, llm: LlamaCpp): Promise<void> {
+  console.log("[consolidation] Starting observation consolidation");
+
+  // Find observation-type documents not yet consolidated
+  const observations = store.db.prepare(`
+    SELECT d.id, d.title, d.facts, d.amem_context as context, d.modified_at, d.collection
+    FROM documents d
+    WHERE d.active = 1
+      AND d.content_type = 'observation'
+      AND d.facts IS NOT NULL
+      AND d.id NOT IN (
+        SELECT value FROM (
+          SELECT json_each.value as value
+          FROM consolidated_observations co, json_each(co.source_doc_ids)
+          WHERE co.status = 'active'
+        )
       )
-      .all();
+    ORDER BY d.collection, d.modified_at DESC
+    LIMIT 50
+  `).all() as { id: number; title: string; facts: string; context: string; modified_at: string; collection: string }[];
+
+  if (observations.length === 0) {
+    console.log("[consolidation] No unconsolidated observations found");
+    return;
+  }
+
+  // Group by collection
+  const clusters = new Map<string, ObservationCluster>();
+  for (const obs of observations) {
+    if (!clusters.has(obs.collection)) {
+      clusters.set(obs.collection, { docs: [], collection: obs.collection });
+    }
+    clusters.get(obs.collection)!.docs.push(obs);
+  }
+
+  // Process each collection cluster
+  for (const [collection, cluster] of clusters) {
+    if (cluster.docs.length < 2) continue; // Need at least 2 observations to consolidate
+
+    try {
+      await synthesizeCluster(store, llm, cluster);
+    } catch (err) {
+      console.error(`[consolidation] Failed to consolidate cluster for ${collection}:`, err);
+    }
+  }
+
+  // Update trends on existing consolidated observations
+  updateTrends(store);
+
+  console.log("[consolidation] Observation consolidation complete");
+}
+
+/**
+ * Synthesize a cluster of observations into consolidated observations using LLM.
+ */
+async function synthesizeCluster(
+  store: Store,
+  llm: LlamaCpp,
+  cluster: ObservationCluster
+): Promise<void> {
+  const docsText = cluster.docs.map((d, i) =>
+    `${i + 1}. [${d.modified_at}] "${d.title}"\n   Facts: ${d.facts?.slice(0, 300) || 'none'}\n   Context: ${d.context?.slice(0, 200) || 'none'}`
+  ).join('\n\n');
+
+  const prompt = `Analyze these ${cluster.docs.length} session observations and identify recurring patterns or cross-session themes.
+
+Observations:
+${docsText}
+
+For each pattern you identify:
+1. Write a clear, actionable observation (1-2 sentences)
+2. Count how many source observations support it (proof_count)
+3. List which source numbers (1-indexed) contribute
+
+Return ONLY valid JSON array:
+[
+  {
+    "observation": "Clear statement of the pattern",
+    "proof_count": 3,
+    "source_indices": [1, 3, 5]
+  }
+]
+
+Rules:
+- Only include patterns supported by 2+ observations
+- Be specific — "user frequently modifies X" > "user works on code"
+- 1-5 patterns maximum
+Return ONLY the JSON array. /no_think`;
 
-    if (docs.length === 0) {
-      // No work to do
-      return;
+  const result = await llm.generate(prompt, {
+    temperature: 0.3,
+    maxTokens: 500,
+  });
+
+  if (!result) return;
+
+  const parsed = extractJsonFromLLM(result.text) as Array<{
+    observation: string;
+    proof_count: number;
+    source_indices: number[];
+  }> | null;
+
+  if (!Array.isArray(parsed)) return;
+
+  for (const pattern of parsed) {
+    if (!pattern.observation || !Array.isArray(pattern.source_indices) || pattern.source_indices.length < 2) continue;
+
+    // Map source indices to doc IDs
+    const sourceDocIds = pattern.source_indices
+      .filter(i => i >= 1 && i <= cluster.docs.length)
+      .map(i => cluster.docs[i - 1]!.id);
+
+    if (sourceDocIds.length < 2) continue;
+
+    // Check for existing similar consolidated observation (avoid duplicates)
+    const existing = findSimilarConsolidation(store, pattern.observation, cluster.collection);
+    if (existing) {
+      // Update existing: merge source docs, increment proof count
+      const existingSourceIds: number[] = JSON.parse(existing.source_doc_ids as unknown as string || '[]');
+      const mergedIds = [...new Set([...existingSourceIds, ...sourceDocIds])];
+
+      store.db.prepare(`
+        UPDATE consolidated_observations
+        SET proof_count = ?,
+            source_doc_ids = ?,
+            updated_at = datetime('now'),
+            observation = ?
+        WHERE id = ?
+      `).run(mergedIds.length, JSON.stringify(mergedIds), pattern.observation, existing.id);
+
+      console.log(`[consolidation] Updated observation #${existing.id}: proof_count=${mergedIds.length}`);
+    } else {
+      // Insert new consolidated observation
+      store.db.prepare(`
+        INSERT INTO consolidated_observations (observation, proof_count, source_doc_ids, trend, status, collection)
+        VALUES (?, ?, ?, 'NEW', 'active', ?)
+      `).run(pattern.observation, sourceDocIds.length, JSON.stringify(sourceDocIds), cluster.collection);
+
+      console.log(`[consolidation] Created new observation: "${pattern.observation.slice(0, 60)}..." (proof=${sourceDocIds.length})`);
     }
+  }
+}
 
-    console.log(`[consolidation] Enriching ${docs.length} documents`);
+/**
+ * Find an existing consolidated observation similar to the given text.
+ * Uses simple word overlap (Jaccard) to detect near-duplicates.
+ */
+function findSimilarConsolidation(
+  store: Store,
+  observation: string,
+  collection: string
+): { id: number; source_doc_ids: string } | null {
+  const existing = store.db.prepare(`
+    SELECT id, observation, source_doc_ids
+    FROM consolidated_observations
+    WHERE status = 'active' AND collection = ?
+  `).all(collection) as { id: number; observation: string; source_doc_ids: string }[];
 
-    // Enrich each document (note + links, skip evolution to avoid cascades)
-    for (const doc of docs) {
-      try {
-        // Construct and store memory note
-        const note = await store.constructMemoryNote(llm, doc.id);
-        await store.storeMemoryNote(doc.id, note);
+  const queryWords = new Set(observation.toLowerCase().split(/\s+/).filter(w => w.length > 3));
 
-        // Generate memory links (skip evolution for backlog)
-        await store.generateMemoryLinks(llm, doc.id);
+  for (const obs of existing) {
+    const obsWords = new Set(obs.observation.toLowerCase().split(/\s+/).filter(w => w.length > 3));
+    const intersection = [...queryWords].filter(w => obsWords.has(w)).length;
+    const union = new Set([...queryWords, ...obsWords]).size;
+    const jaccard = union > 0 ? intersection / union : 0;
 
-        console.log(`[consolidation] Enriched doc ${doc.id} (${doc.title})`);
-      } catch (err) {
-        console.error(`[consolidation] Failed to enrich doc ${doc.id}:`, err);
-        // Continue with remaining docs (don't let one failure block the queue)
-      }
+    if (jaccard > 0.5) {
+      return { id: obs.id, source_doc_ids: obs.source_doc_ids };
     }
-  } catch (err) {
-    console.error("[consolidation] Tick failed:", err);
-  } finally {
-    isRunning = false;
   }
+
+  return null;
+}
+
+/**
+ * Update trend labels on consolidated observations based on evidence timestamps.
+ * Trends: NEW (< 7 days), STRENGTHENING (proof growing), WEAKENING (no new evidence 30+ days),
+ * STABLE (steady), STALE (60+ days without new evidence).
+ */
+function updateTrends(store: Store): void {
+  const observations = store.db.prepare(`
+    SELECT id, proof_count, source_doc_ids, trend, created_at, updated_at
+    FROM consolidated_observations
+    WHERE status = 'active'
+  `).all() as {
+    id: number; proof_count: number; source_doc_ids: string;
+    trend: string; created_at: string; updated_at: string;
+  }[];
+
+  const now = Date.now();
+  const DAY_MS = 86400000;
+
+  for (const obs of observations) {
+    const createdAge = (now - new Date(obs.created_at).getTime()) / DAY_MS;
+    const updatedAge = (now - new Date(obs.updated_at).getTime()) / DAY_MS;
+
+    let newTrend: TrendEnum;
+    if (createdAge < 7) {
+      newTrend = 'NEW';
+    } else if (updatedAge > 60) {
+      newTrend = 'STALE';
+    } else if (updatedAge > 30) {
+      newTrend = 'WEAKENING';
+    } else if (obs.proof_count >= 4 && updatedAge < 14) {
+      newTrend = 'STRENGTHENING';
+    } else {
+      newTrend = 'STABLE';
+    }
+
+    if (newTrend !== obs.trend) {
+      store.db.prepare(`UPDATE consolidated_observations SET trend = ? WHERE id = ?`).run(newTrend, obs.id);
+    }
+  }
+}
+
+// =============================================================================
+// Public API for MCP / CLI
+// =============================================================================
+
+/**
+ * Get consolidated observations, optionally filtered.
+ */
+export function getConsolidatedObservations(
+  store: Store,
+  options?: { collection?: string; trend?: TrendEnum; minProof?: number; limit?: number }
+): ConsolidatedObservation[] {
+  let sql = `SELECT * FROM consolidated_observations WHERE status = 'active'`;
+  const params: any[] = [];
+
+  if (options?.collection) {
+    sql += ` AND collection = ?`;
+    params.push(options.collection);
+  }
+  if (options?.trend) {
+    sql += ` AND trend = ?`;
+    params.push(options.trend);
+  }
+  if (options?.minProof) {
+    sql += ` AND proof_count >= ?`;
+    params.push(options.minProof);
+  }
+
+  sql += ` ORDER BY proof_count DESC, updated_at DESC LIMIT ?`;
+  params.push(options?.limit || 20);
+
+  return store.db.prepare(sql).all(...params) as ConsolidatedObservation[];
+}
+
+/**
+ * Manually trigger consolidation (for CLI or MCP tool).
+ */
+export async function runConsolidation(
+  store: Store,
+  llm: LlamaCpp,
+  dryRun: boolean = false
+): Promise<{ clustersFound: number; observationsCreated: number }> {
+  if (dryRun) {
+    // Count unconsolidated observations
+    const count = store.db.prepare(`
+      SELECT COUNT(*) as cnt FROM documents
+      WHERE active = 1 AND content_type = 'observation' AND facts IS NOT NULL
+        AND id NOT IN (
+          SELECT value FROM (
+            SELECT json_each.value as value
+            FROM consolidated_observations co, json_each(co.source_doc_ids)
+            WHERE co.status = 'active'
+          )
+        )
+    `).get() as { cnt: number };
+
+    return { clustersFound: count.cnt, observationsCreated: 0 };
+  }
+
+  const before = store.db.prepare(`SELECT COUNT(*) as cnt FROM consolidated_observations WHERE status = 'active'`).get() as { cnt: number };
+  await consolidateObservations(store, llm);
+  const after = store.db.prepare(`SELECT COUNT(*) as cnt FROM consolidated_observations WHERE status = 'active'`).get() as { cnt: number };
+
+  return { clustersFound: 0, observationsCreated: after.cnt - before.cnt };
 }