clawmem 0.1.8 → 0.2.1

package/AGENTS.md

@@ -96,6 +96,7 @@ curl http://host:8090/v1/models
 | `CLAWMEM_ENABLE_AMEM` | enabled | A-MEM note construction + link generation during indexing. |
 | `CLAWMEM_ENABLE_CONSOLIDATION` | disabled | Background worker backfills unenriched docs. Needs long-lived MCP process. |
 | `CLAWMEM_CONSOLIDATION_INTERVAL` | 300000 | Worker interval in ms (min 15000). |
+| `CLAWMEM_NUDGE_INTERVAL` | `15` | Prompts between lifecycle tool use before `<vault-nudge>` injection. 0 to disable. |
 
 **Note:** The `bin/clawmem` wrapper sets all endpoint defaults. Always use the wrapper — never `bun run src/clawmem.ts` directly. For remote GPU setups, add the same env vars to the watcher service via a systemd drop-in.
 
@@ -415,6 +416,8 @@ Note: intent disables BM25 strong-signal bypass, forcing full expansion+rerankin
 
 ## Composite Scoring (automatic, applied to all search tools)
 
+**Observation invalidation (v0.2.0):** Documents with `invalidated_at` set are excluded from search results. Soft invalidation uses `invalidated_at`, `invalidated_by`, and `superseded_by` columns. When `decision-extractor` detects a contradiction that drops confidence ≤ 0.2, the observation is marked invalidated. Invalidated docs can be restored via lifecycle tools.
+
 ```
 compositeScore = (0.50 × searchScore + 0.25 × recencyScore + 0.25 × confidenceScore) × qualityMultiplier × coActivationBoost
 ```
@@ -482,11 +485,15 @@ The `memory_relations` table is populated by multiple independent sources:
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP tool or watcher (.beads/ change) | Queries `bd` CLI (Dolt backend). Maps beads deps: blocks→causal, discovered-from→supporting, relates-to→semantic, plus conditional-blocks→causal, caused-by→causal, supersedes→supporting. Metadata: `{origin: "beads"}`. |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
+| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → canonical normalization (FTS5 + Levenshtein) → `entity_mentions` + `entity_cooccurrences` tables. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
+| `consolidated_observations` | supporting | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.
 
 **Graph traversal asymmetry:** `adaptiveTraversal()` traverses all edge types outbound (source→target) but only `semantic` and `entity` edges inbound (target→source). Temporal and causal edges are directional only.
 
+**MPFP graph retrieval (v0.2.0):** Multi-Path Fact Propagation runs predefined meta-path patterns in parallel (`[semantic, semantic]`, `[entity, temporal]`, `[semantic, causal]`, etc.), fuses via RRF. Hop-synchronized edge cache batches DB queries per hop instead of per pattern. Forward Push with α=0.15 teleport probability bounds active nodes sublinearly. Tier 3 only (`query`/`intent_search`), not hooks. Patterns selected per MAGMA intent: WHY → `[semantic, causal]`, ENTITY → `[entity, semantic]`, WHEN → `[temporal, semantic]`.
+
 ### When to Run `build_graphs`
 
 - After **bulk ingestion** (many new docs at once) — adds temporal backbone and fills semantic gaps where A-MEM links are sparse.
@@ -505,11 +512,14 @@ The `memory_relations` table is populated by multiple independent sources:
 
 ```
 User Query + optional intent hint
+  → Temporal Extraction (regex date range from query: "last week", "March 2026" → WHERE modified_at BETWEEN filters)
   → BM25 Probe → Strong Signal Check (skip expansion if top hit ≥ 0.85 with gap ≥ 0.15; disabled when intent provided)
   → Query Expansion (LLM generates text variants; intent steers expansion prompt)
   → Parallel: BM25(original) + Vector(original) + BM25(each expanded) + Vector(each expanded)
+       + Temporal Proximity (date-range filtered, if temporal constraint extracted)
+       + Entity Graph (conditional 1-hop entity walk from top seeds, if entity signals present)
   → Original query lists get positional 2× weight in RRF; expanded get 1×
-  → Reciprocal Rank Fusion (k=60, top candidateLimit)
+  → Reciprocal Rank Fusion (k=60, top candidateLimit, up to 4-way parallel legs)
   → Intent-Aware Chunk Selection (intent terms at 0.5× weight alongside query terms at 1.0×)
   → Cross-Encoder Reranking (4000 char context; intent prepended to rerank query; chunk dedup; batch cap=4)
   → Position-Aware Blending (α=0.75 top3, 0.60 mid, 0.40 tail)
@@ -624,9 +634,20 @@ Symptom: embed --force with new model produces 3 docs stuck as "Unembedded" but
   → Fix: Delete partial content_vectors + vectors_vec for the stuck hashes, then re-run embed (no --force).
     The vec0 DELETE try-catch prevents cascading failures during the re-embed.
 
+Symptom: reindex --force after v0.2.0 upgrade shows no entity extraction
+  → `reindex --force` treats existing docs as updates (isNew=false). The A-MEM pipeline
+    skips entity extraction, link generation, and evolution for updates to avoid churn.
+  → Fix: Use `clawmem reindex --enrich` instead. The `--enrich` flag forces the full
+    enrichment pipeline (entity extraction + links + evolution) on all documents.
+  → `--force` alone only refreshes A-MEM notes (keywords, tags, context). `--enrich`
+    is needed after major upgrades that add new enrichment stages.
+
 Symptom: CLI reindex/update falls back to node-llama-cpp Vulkan (not GPU server)
   → GPU env vars only in systemd drop-in, not in wrapper script. CLI invocations missed them.
   → Fixed 2026-02-12: bin/clawmem wrapper exports CLAWMEM_EMBED_URL/LLM_URL/RERANK_URL defaults.
+  → Always run ClawMem via the `bin/clawmem` wrapper, not `bun run src/clawmem.ts` directly.
+    The wrapper sets CLAWMEM_EMBED_URL/LLM_URL/RERANK_URL defaults. Scripts or inline bun
+    commands that bypass the wrapper will fall back to in-process node-llama-cpp (slow, CPU).
 
 Symptom: "UserPromptSubmit hook error" on context-surfacing hook (intermittent)
   → SQLite contention between the watcher and the hook. The watcher processes filesystem events
@@ -662,6 +683,8 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
 
 ## Integration Notes
 
+- **Memory nudge (v0.2.0):** Every N prompts (default 15) without a lifecycle MCP tool call (`memory_pin`/`memory_forget`/`memory_snooze`), context-surfacing appends `<vault-nudge>` prompting proactive memory management. Counter resets on lifecycle tool use. Configure via `CLAWMEM_NUDGE_INTERVAL` (0 to disable).
+- **Entity resolution (v0.2.0):** A-MEM enrichment now extracts named entities via LLM, resolves to canonical forms using `entities_fts` + Levenshtein fuzzy matching, and tracks co-occurrence. Entity graph traversal available for ENTITY intent queries via `intent_search`.
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
 - Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.

package/CLAUDE.md

@@ -96,6 +96,7 @@ curl http://host:8090/v1/models
 | `CLAWMEM_ENABLE_AMEM` | enabled | A-MEM note construction + link generation during indexing. |
 | `CLAWMEM_ENABLE_CONSOLIDATION` | disabled | Background worker backfills unenriched docs. Needs long-lived MCP process. |
 | `CLAWMEM_CONSOLIDATION_INTERVAL` | 300000 | Worker interval in ms (min 15000). |
+| `CLAWMEM_NUDGE_INTERVAL` | `15` | Prompts between lifecycle tool use before `<vault-nudge>` injection. 0 to disable. |
 
 **Note:** The `bin/clawmem` wrapper sets all endpoint defaults. Always use the wrapper — never `bun run src/clawmem.ts` directly. For remote GPU setups, add the same env vars to the watcher service via a systemd drop-in.
 
@@ -415,6 +416,8 @@ Note: intent disables BM25 strong-signal bypass, forcing full expansion+rerankin
 
 ## Composite Scoring (automatic, applied to all search tools)
 
+**Observation invalidation (v0.2.0):** Documents with `invalidated_at` set are excluded from search results. Soft invalidation uses `invalidated_at`, `invalidated_by`, and `superseded_by` columns. When `decision-extractor` detects a contradiction that drops confidence ≤ 0.2, the observation is marked invalidated. Invalidated docs can be restored via lifecycle tools.
+
 ```
 compositeScore = (0.50 × searchScore + 0.25 × recencyScore + 0.25 × confidenceScore) × qualityMultiplier × coActivationBoost
 ```
@@ -482,11 +485,15 @@ The `memory_relations` table is populated by multiple independent sources:
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP tool or watcher (.beads/ change) | Queries `bd` CLI (Dolt backend). Maps beads deps: blocks→causal, discovered-from→supporting, relates-to→semantic, plus conditional-blocks→causal, caused-by→causal, supersedes→supporting. Metadata: `{origin: "beads"}`. |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
+| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → canonical normalization (FTS5 + Levenshtein) → `entity_mentions` + `entity_cooccurrences` tables. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
+| `consolidated_observations` | supporting | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.
 
 **Graph traversal asymmetry:** `adaptiveTraversal()` traverses all edge types outbound (source→target) but only `semantic` and `entity` edges inbound (target→source). Temporal and causal edges are directional only.
 
+**MPFP graph retrieval (v0.2.0):** Multi-Path Fact Propagation runs predefined meta-path patterns in parallel (`[semantic, semantic]`, `[entity, temporal]`, `[semantic, causal]`, etc.), fuses via RRF. Hop-synchronized edge cache batches DB queries per hop instead of per pattern. Forward Push with α=0.15 teleport probability bounds active nodes sublinearly. Tier 3 only (`query`/`intent_search`), not hooks. Patterns selected per MAGMA intent: WHY → `[semantic, causal]`, ENTITY → `[entity, semantic]`, WHEN → `[temporal, semantic]`.
+
 ### When to Run `build_graphs`
 
 - After **bulk ingestion** (many new docs at once) — adds temporal backbone and fills semantic gaps where A-MEM links are sparse.
@@ -505,11 +512,14 @@ The `memory_relations` table is populated by multiple independent sources:
 
 ```
 User Query + optional intent hint
+  → Temporal Extraction (regex date range from query: "last week", "March 2026" → WHERE modified_at BETWEEN filters)
   → BM25 Probe → Strong Signal Check (skip expansion if top hit ≥ 0.85 with gap ≥ 0.15; disabled when intent provided)
   → Query Expansion (LLM generates text variants; intent steers expansion prompt)
   → Parallel: BM25(original) + Vector(original) + BM25(each expanded) + Vector(each expanded)
+       + Temporal Proximity (date-range filtered, if temporal constraint extracted)
+       + Entity Graph (conditional 1-hop entity walk from top seeds, if entity signals present)
   → Original query lists get positional 2× weight in RRF; expanded get 1×
-  → Reciprocal Rank Fusion (k=60, top candidateLimit)
+  → Reciprocal Rank Fusion (k=60, top candidateLimit, up to 4-way parallel legs)
   → Intent-Aware Chunk Selection (intent terms at 0.5× weight alongside query terms at 1.0×)
   → Cross-Encoder Reranking (4000 char context; intent prepended to rerank query; chunk dedup; batch cap=4)
   → Position-Aware Blending (α=0.75 top3, 0.60 mid, 0.40 tail)
@@ -624,9 +634,20 @@ Symptom: embed --force with new model produces 3 docs stuck as "Unembedded" but
   → Fix: Delete partial content_vectors + vectors_vec for the stuck hashes, then re-run embed (no --force).
     The vec0 DELETE try-catch prevents cascading failures during the re-embed.
 
+Symptom: reindex --force after v0.2.0 upgrade shows no entity extraction
+  → `reindex --force` treats existing docs as updates (isNew=false). The A-MEM pipeline
+    skips entity extraction, link generation, and evolution for updates to avoid churn.
+  → Fix: Use `clawmem reindex --enrich` instead. The `--enrich` flag forces the full
+    enrichment pipeline (entity extraction + links + evolution) on all documents.
+  → `--force` alone only refreshes A-MEM notes (keywords, tags, context). `--enrich`
+    is needed after major upgrades that add new enrichment stages.
+
 Symptom: CLI reindex/update falls back to node-llama-cpp Vulkan (not GPU server)
   → GPU env vars only in systemd drop-in, not in wrapper script. CLI invocations missed them.
   → Fixed 2026-02-12: bin/clawmem wrapper exports CLAWMEM_EMBED_URL/LLM_URL/RERANK_URL defaults.
+  → Always run ClawMem via the `bin/clawmem` wrapper, not `bun run src/clawmem.ts` directly.
+    The wrapper sets CLAWMEM_EMBED_URL/LLM_URL/RERANK_URL defaults. Scripts or inline bun
+    commands that bypass the wrapper will fall back to in-process node-llama-cpp (slow, CPU).
 
 Symptom: "UserPromptSubmit hook error" on context-surfacing hook (intermittent)
   → SQLite contention between the watcher and the hook. The watcher processes filesystem events
@@ -662,6 +683,8 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
 
 ## Integration Notes
 
+- **Memory nudge (v0.2.0):** Every N prompts (default 15) without a lifecycle MCP tool call (`memory_pin`/`memory_forget`/`memory_snooze`), context-surfacing appends `<vault-nudge>` prompting proactive memory management. Counter resets on lifecycle tool use. Configure via `CLAWMEM_NUDGE_INTERVAL` (0 to disable).
+- **Entity resolution (v0.2.0):** A-MEM enrichment now extracts named entities via LLM, resolves to canonical forms using `entities_fts` + Levenshtein fuzzy matching, and tracks co-occurrence. Entity graph traversal available for ENTITY intent queries via `intent_search`.
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
 - Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.

package/README.md

@@ -42,6 +42,18 @@ ClawMem turns your markdown notes, project docs, and research dumps into persist
 
 Runs fully local with no API keys and no cloud services. Integrates via Claude Code hooks and MCP tools, or as an OpenClaw ContextEngine plugin. Both modes share the same vault for cross-runtime memory. Works with any MCP-compatible client.
 
+### v0.2.0 Enhancements
+
+Seven patterns extracted from competitor analysis ([Hindsight](https://github.com/vectorize-io/hindsight), [Hermes Agent](https://github.com/NousResearch/hermes-agent), [claude-mem](https://github.com/thedotmack/claude-mem)):
+
+- **Entity resolution + co-occurrence graph** — LLM entity extraction during A-MEM enrichment, canonical normalization via FTS5 + Levenshtein fuzzy matching, co-occurrence tracking, entity graph traversal for ENTITY intent queries
+- **MPFP graph retrieval** — Multi-Path Fact Propagation with meta-path patterns per intent, hop-synchronized edge cache, Forward Push with α=0.15 teleport probability. Replaces single-beam traversal for causal/entity/temporal queries.
+- **Temporal query extraction** — regex-based date range extraction from natural language queries ("last week", "March 2026"), wired as WHERE filters into BM25 and vector search
+- **4-way parallel retrieval** — temporal proximity and entity graph channels added as parallel RRF legs in `query` tool (Tier 3 only), alongside existing BM25 + vector channels
+- **3-tier consolidation** — facts to observations (auto-generated, with proof_count and trend enum) to mental models. Background worker synthesizes clusters of related observations into consolidated patterns.
+- **Observation invalidation** — soft invalidation (invalidated_at/invalidated_by/superseded_by columns). Observations with confidence ≤ 0.2 after contradiction are filtered from search results.
+- **Memory nudge** — periodic ephemeral `<vault-nudge>` injection prompting lifecycle tool use after N turns of inactivity. Configurable via `CLAWMEM_NUDGE_INTERVAL`.
+
 ## Architecture
 
 <p align="center">
@@ -137,6 +149,25 @@ clawmem update --embed
 clawmem doctor
 ```
 
+### Upgrading
+
+```bash
+bun update -g clawmem   # or: npm update -g clawmem
+```
+
+Database schema migrates automatically on next startup (new tables and columns are added via `CREATE IF NOT EXISTS` / `ALTER TABLE ADD COLUMN`).
+
+After **major version updates** (e.g. 0.1.x → 0.2.0) that add new enrichment pipelines, run a full enrichment pass to backfill existing documents:
+
+```bash
+clawmem reindex --enrich  # Full enrichment: entity extraction + links + evolution for all docs
+clawmem embed             # Re-embed if upgrading embedding models (not needed for most updates)
+```
+
+`--enrich` forces the complete A-MEM pipeline (entity extraction, link generation, memory evolution) on all documents, not just new ones. Without it, reindex only refreshes metadata for existing docs.
+
+Routine patch updates (e.g. 0.2.0 → 0.2.1) do not require reindexing.
+
 ### Integration
 
 #### Claude Code

package/SKILL.md

@@ -272,7 +272,7 @@ Once escalated, route by query type:
 | `beads_sync` | Sync Beads issues from Dolt backend into memory. |
 | `index_stats` | Doc counts, embedding coverage, content type distribution. |
 | `status` | Quick index health. |
-| `reindex` | Force re-index (BM25 only, does NOT embed). |
+| `reindex` | Force re-index (BM25 only, does NOT embed). Use `--enrich` after major upgrades to backfill entity extraction + links on existing docs. |
 | `memory_evolution_status` | Track how a doc's A-MEM metadata evolved over time. |
 | `timeline` | Temporal neighborhood around a document — what was modified before/after. Progressive disclosure: search → timeline → get. Supports same-collection scoping and session correlation. |
 | `list_vaults` | Show configured vault names and paths. Empty in single-vault mode. |
@@ -660,6 +660,14 @@ echo "user query" | clawmem surface --context --stdin
 echo "session-id" | clawmem surface --bootstrap --stdin
 ```
 
+### Enrichment Commands
+
+```bash
+clawmem reindex --enrich        # Full A-MEM pipeline on ALL docs (entity extraction,
+                                # link generation, memory evolution). Use after major upgrades.
+                                # Without --enrich, reindex only refreshes metadata for changed docs.
+```
+
 ### Analysis Commands
 
 ```bash

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.1.8",
+  "version": "0.2.1",
   "description": "On-device context engine and memory for AI agents. Claude Code and OpenClaw. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {

package/src/amem.ts

@@ -8,6 +8,7 @@
 import type { Database } from "bun:sqlite";
 import type { LlamaCpp } from "./llm.ts";
 import type { Store } from "./store.ts";
+import { enrichDocumentEntities } from "./entity.ts";
 
 export interface MemoryNote {
   keywords: string[];
@@ -612,11 +613,21 @@ export async function postIndexEnrich(
       return;
     }
 
-    // Step 2: Generate memory links (new documents only)
+    // Step 2: Entity extraction + resolution + co-occurrence (new documents only)
+    try {
+      const entityCount = await enrichDocumentEntities(store.db, llm, docId);
+      if (entityCount > 0) {
+        console.log(`[amem] Resolved ${entityCount} entities for docId ${docId}`);
+      }
+    } catch (err) {
+      console.log(`[amem] Entity enrichment failed for docId ${docId}:`, err);
+    }
+
+    // Step 3: Generate memory links (new documents only)
     const linksCreated = await generateMemoryLinks(store, llm, docId);
     console.log(`[amem] Created ${linksCreated} links for docId ${docId}`);
 
-    // Step 3: Evolve memories based on new evidence (new documents only)
+    // Step 4: Evolve memories based on new evidence (new documents only)
     // The new document triggers evolution of its linked neighbors
     if (linksCreated > 0) {
       // Get neighbors this new document links to (outbound links from generateMemoryLinks)

package/src/clawmem.ts

@@ -1269,6 +1269,7 @@ async function cmdWatch() {
 
 async function cmdReindex(args: string[]) {
   const force = args.includes("--force") || args.includes("-f");
+  const enrich = args.includes("--enrich");
   const collections = collectionsList();
 
   if (collections.length === 0) {
@@ -1283,9 +1284,13 @@ async function cmdReindex(args: string[]) {
     console.log(`${c.yellow}Force reindex: deactivated all documents${c.reset}`);
   }
 
+  if (enrich) {
+    console.log(`${c.cyan}Full enrichment: entity extraction + links + evolution for all documents${c.reset}`);
+  }
+
   for (const col of collections) {
     console.log(`Indexing ${c.bold}${col.name}${c.reset} (${col.path})...`);
-    const stats = await indexCollection(s, col.name, col.path, col.pattern);
+    const stats = await indexCollection(s, col.name, col.path, col.pattern, { forceEnrich: enrich });
     console.log(`  +${stats.added} added, ~${stats.updated} updated, =${stats.unchanged} unchanged, -${stats.removed} removed`);
   }
 }
@@ -2202,7 +2207,7 @@ ${c.bold}Setup:${c.reset}
 ${c.bold}Indexing:${c.reset}
   clawmem update [--pull] [--embed]    Re-scan collections (--embed auto-embeds)
   clawmem embed [-f]                   Generate fragment embeddings
-  clawmem reindex [--force]            Full re-index
+  clawmem reindex [--force] [--enrich]  Full re-index (--enrich: run entity extraction + links on all docs)
   clawmem watch                        File watcher daemon
   clawmem status                       Show index status