clawmem 0.2.2 → 0.2.4

package/AGENTS.md

@@ -89,7 +89,7 @@ curl http://host:8090/v1/models
 | `CLAWMEM_EMBED_MAX_CHARS` | `6000` | Max chars per embedding input. Default fits EmbeddingGemma (2048 tokens). Set to `1100` for granite-278m (512 tokens). Cloud providers skip truncation. |
 | `CLAWMEM_EMBED_TPM_LIMIT` | `100000` | Tokens-per-minute limit for cloud embedding pacing. Match to your provider tier: Free 100000, Paid 2000000, Premium 50000000. |
 | `CLAWMEM_EMBED_DIMENSIONS` | (none) | Output dimensions for OpenAI `text-embedding-3-*` Matryoshka models (e.g. `512`, `1024`). Only sent when URL contains `openai.com`. |
-| `CLAWMEM_LLM_URL` | `http://localhost:8089` | LLM server for intent, expansion, A-MEM. Falls to `node-llama-cpp` if unset + `NO_LOCAL_MODELS=false`. |
+| `CLAWMEM_LLM_URL` | `http://localhost:8089` | LLM server for intent, expansion, A-MEM, and entity extraction. Falls to `node-llama-cpp` if unset + `NO_LOCAL_MODELS=false`. For better entity extraction quality, point at a 7B+ model or cloud API during `reindex --enrich` (see `docs/internals/entity-resolution.md`). |
 | `CLAWMEM_RERANK_URL` | `http://localhost:8090` | Reranker server. Falls to `node-llama-cpp` if unset + `NO_LOCAL_MODELS=false`. |
 | `CLAWMEM_NO_LOCAL_MODELS` | `false` | Blocks `node-llama-cpp` from auto-downloading GGUF models. Set `true` for remote-only setups. |
 | `CLAWMEM_VAULTS` | (none) | JSON map of vault name → SQLite path for multi-vault mode. E.g. `{"work":"~/.cache/clawmem/work.sqlite"}` |
@@ -485,7 +485,7 @@ The `memory_relations` table is populated by multiple independent sources:
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP tool or watcher (.beads/ change) | Queries `bd` CLI (Dolt backend). Maps beads deps: blocks→causal, discovered-from→supporting, relates-to→semantic, plus conditional-blocks→causal, caused-by→causal, supersedes→supporting. Metadata: `{origin: "beads"}`. |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
-| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → canonical normalization (FTS5 + Levenshtein) → `entity_mentions` + `entity_cooccurrences` tables. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
+| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → quality filters (title/length/blocklist/location validation) → type-agnostic canonical resolution within compatibility buckets (person, org, location, tech=project/service/tool/concept) → `entity_mentions` + `entity_cooccurrences` tables. Entity edges use IDF-based specificity scoring. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
 | `consolidated_observations` | supporting | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.
@@ -555,28 +555,6 @@ User Query → Intent Classification (WHY/WHEN/ENTITY/WHAT)
 | `candidateLimit` | Yes (default 30) | No |
 | Best for | Most queries, progressive disclosure | Causal chains spanning multiple docs |
 
-## Operational Issue Tracking
-
-When encountering tool failures, instruction contradictions, retrieval gaps, or workflow friction that would benefit from a fix:
-
-Write to `docs/issues/YYYY-MM-DD-<slug>.md` with: category, severity, what happened, what was expected, context, suggested fix.
-
-**File structure:**
-```
-# <title>
-- Category: tool-failure | instruction-gap | workflow-friction | retrieval-gap | inconsistency
-- Severity: critical | high | medium
-- Status: open | resolved
-
-## Observed
-## Expected
-## Context
-## Suggested Fix
-```
-
-**Triggers:** repeated tool error, instruction that contradicts observed behavior, retrieval consistently missing known content, workflow requiring unnecessary steps.
-
-**Do NOT log:** one-off transient errors, user-caused issues, issues already recorded.
 
 ## Troubleshooting
 
@@ -658,6 +636,15 @@ Symptom: "UserPromptSubmit hook error" on context-surfacing hook (intermittent)
   → Default hook timeout is 8s (since v0.1.1). If you have an older install, re-run
     `clawmem setup hooks`. If persistent, restart the watcher: `systemctl --user restart
     clawmem-watcher.service`. Healthy memory is under 100MB — if 400MB+, restart clears it.
+
+Symptom: WSL hangs or becomes unresponsive during long sessions / watcher has 100K+ FDs
+  → Pre-v0.2.3: fs.watch(recursive: true) registered inotify watches on EVERY subdirectory,
+    including excluded dirs (gits/, node_modules/, .git/). Broad collection paths like
+    ~/Projects with 67K subdirs exhausted inotify limits.
+  → v0.2.3 fix: watcher walks dir trees at startup, skips excluded subtrees, watches
+    non-excluded dirs individually. 500-dir cap per collection path.
+  → Diagnosis: `ls /proc/$(pgrep -f "clawmem.*watch")/fd | wc -l` — healthy < 15K.
+  → If still high: narrow broad collection paths. See docs/troubleshooting.md for details.
 ```
 
 ## CLI Reference
@@ -684,7 +671,7 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
 ## Integration Notes
 
 - **Memory nudge (v0.2.0):** Every N prompts (default 15) without a lifecycle MCP tool call (`memory_pin`/`memory_forget`/`memory_snooze`), context-surfacing appends `<vault-nudge>` prompting proactive memory management. Counter resets on lifecycle tool use. Configure via `CLAWMEM_NUDGE_INTERVAL` (0 to disable).
-- **Entity resolution (v0.2.0):** A-MEM enrichment now extracts named entities via LLM, resolves to canonical forms using `entities_fts` + Levenshtein fuzzy matching, and tracks co-occurrence. Entity graph traversal available for ENTITY intent queries via `intent_search`.
+- **Entity resolution (v0.2.0+):** A-MEM enrichment extracts named entities via LLM, resolves to canonical forms using FTS5 + Levenshtein fuzzy matching with **type-agnostic compatibility buckets** (person, org, location stay separate; project/service/tool/concept merge freely as "tech" bucket). Quality filters reject title-as-entity, long names, template placeholders, and invalid locations. Entity edges use IDF-based specificity scoring (rare entities create edges; ubiquitous entities alone cannot). See `docs/internals/entity-resolution.md` for customization (extending type vocabulary and buckets).
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
 - Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.

package/CLAUDE.md

@@ -89,7 +89,7 @@ curl http://host:8090/v1/models
 | `CLAWMEM_EMBED_MAX_CHARS` | `6000` | Max chars per embedding input. Default fits EmbeddingGemma (2048 tokens). Set to `1100` for granite-278m (512 tokens). Cloud providers skip truncation. |
 | `CLAWMEM_EMBED_TPM_LIMIT` | `100000` | Tokens-per-minute limit for cloud embedding pacing. Match to your provider tier: Free 100000, Paid 2000000, Premium 50000000. |
 | `CLAWMEM_EMBED_DIMENSIONS` | (none) | Output dimensions for OpenAI `text-embedding-3-*` Matryoshka models (e.g. `512`, `1024`). Only sent when URL contains `openai.com`. |
-| `CLAWMEM_LLM_URL` | `http://localhost:8089` | LLM server for intent, expansion, A-MEM. Falls to `node-llama-cpp` if unset + `NO_LOCAL_MODELS=false`. |
+| `CLAWMEM_LLM_URL` | `http://localhost:8089` | LLM server for intent, expansion, A-MEM, and entity extraction. Falls to `node-llama-cpp` if unset + `NO_LOCAL_MODELS=false`. For better entity extraction quality, point at a 7B+ model or cloud API during `reindex --enrich` (see `docs/internals/entity-resolution.md`). |
 | `CLAWMEM_RERANK_URL` | `http://localhost:8090` | Reranker server. Falls to `node-llama-cpp` if unset + `NO_LOCAL_MODELS=false`. |
 | `CLAWMEM_NO_LOCAL_MODELS` | `false` | Blocks `node-llama-cpp` from auto-downloading GGUF models. Set `true` for remote-only setups. |
 | `CLAWMEM_VAULTS` | (none) | JSON map of vault name → SQLite path for multi-vault mode. E.g. `{"work":"~/.cache/clawmem/work.sqlite"}` |
@@ -485,7 +485,7 @@ The `memory_relations` table is populated by multiple independent sources:
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP tool or watcher (.beads/ change) | Queries `bd` CLI (Dolt backend). Maps beads deps: blocks→causal, discovered-from→supporting, relates-to→semantic, plus conditional-blocks→causal, caused-by→causal, supersedes→supporting. Metadata: `{origin: "beads"}`. |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
-| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → canonical normalization (FTS5 + Levenshtein) → `entity_mentions` + `entity_cooccurrences` tables. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
+| Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → quality filters (title/length/blocklist/location validation) → type-agnostic canonical resolution within compatibility buckets (person, org, location, tech=project/service/tool/concept) → `entity_mentions` + `entity_cooccurrences` tables. Entity edges use IDF-based specificity scoring. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
 | `consolidated_observations` | supporting | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.
@@ -555,28 +555,6 @@ User Query → Intent Classification (WHY/WHEN/ENTITY/WHAT)
 | `candidateLimit` | Yes (default 30) | No |
 | Best for | Most queries, progressive disclosure | Causal chains spanning multiple docs |
 
-## Operational Issue Tracking
-
-When encountering tool failures, instruction contradictions, retrieval gaps, or workflow friction that would benefit from a fix:
-
-Write to `docs/issues/YYYY-MM-DD-<slug>.md` with: category, severity, what happened, what was expected, context, suggested fix.
-
-**File structure:**
-```
-# <title>
-- Category: tool-failure | instruction-gap | workflow-friction | retrieval-gap | inconsistency
-- Severity: critical | high | medium
-- Status: open | resolved
-
-## Observed
-## Expected
-## Context
-## Suggested Fix
-```
-
-**Triggers:** repeated tool error, instruction that contradicts observed behavior, retrieval consistently missing known content, workflow requiring unnecessary steps.
-
-**Do NOT log:** one-off transient errors, user-caused issues, issues already recorded.
 
 ## Troubleshooting
 
@@ -658,6 +636,15 @@ Symptom: "UserPromptSubmit hook error" on context-surfacing hook (intermittent)
   → Default hook timeout is 8s (since v0.1.1). If you have an older install, re-run
     `clawmem setup hooks`. If persistent, restart the watcher: `systemctl --user restart
     clawmem-watcher.service`. Healthy memory is under 100MB — if 400MB+, restart clears it.
+
+Symptom: WSL hangs or becomes unresponsive during long sessions / watcher has 100K+ FDs
+  → Pre-v0.2.3: fs.watch(recursive: true) registered inotify watches on EVERY subdirectory,
+    including excluded dirs (gits/, node_modules/, .git/). Broad collection paths like
+    ~/Projects with 67K subdirs exhausted inotify limits.
+  → v0.2.3 fix: watcher walks dir trees at startup, skips excluded subtrees, watches
+    non-excluded dirs individually. 500-dir cap per collection path.
+  → Diagnosis: `ls /proc/$(pgrep -f "clawmem.*watch")/fd | wc -l` — healthy < 15K.
+  → If still high: narrow broad collection paths. See docs/troubleshooting.md for details.
 ```
 
 ## CLI Reference
@@ -684,7 +671,7 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
 ## Integration Notes
 
 - **Memory nudge (v0.2.0):** Every N prompts (default 15) without a lifecycle MCP tool call (`memory_pin`/`memory_forget`/`memory_snooze`), context-surfacing appends `<vault-nudge>` prompting proactive memory management. Counter resets on lifecycle tool use. Configure via `CLAWMEM_NUDGE_INTERVAL` (0 to disable).
-- **Entity resolution (v0.2.0):** A-MEM enrichment now extracts named entities via LLM, resolves to canonical forms using `entities_fts` + Levenshtein fuzzy matching, and tracks co-occurrence. Entity graph traversal available for ENTITY intent queries via `intent_search`.
+- **Entity resolution (v0.2.0+):** A-MEM enrichment extracts named entities via LLM, resolves to canonical forms using FTS5 + Levenshtein fuzzy matching with **type-agnostic compatibility buckets** (person, org, location stay separate; project/service/tool/concept merge freely as "tech" bucket). Quality filters reject title-as-entity, long names, template placeholders, and invalid locations. Entity edges use IDF-based specificity scoring (rare entities create edges; ubiquitous entities alone cannot). See `docs/internals/entity-resolution.md` for customization (extending type vocabulary and buckets).
 - QMD retrieval (BM25, vector, RRF, rerank, query expansion) is forked into ClawMem. Do not call standalone QMD tools.
 - SAME (composite scoring), MAGMA (intent + graph), A-MEM (self-evolving notes) layer on top of QMD substrate.
 - Three `llama-server` instances (embedding, LLM, reranker) on local or remote GPU. Wrapper defaults to `localhost:8088/8089/8090`.

package/README.md

@@ -44,7 +44,7 @@ Runs fully local with no API keys and no cloud services. Integrates via Claude C
 
 ### v0.2.0 Enhancements
 
-- **Entity resolution + co-occurrence graph** — LLM entity extraction during A-MEM enrichment, canonical normalization via FTS5 + Levenshtein fuzzy matching, co-occurrence tracking, entity graph traversal for ENTITY intent queries
+- **Entity resolution + co-occurrence graph** — LLM entity extraction with quality filters, type-agnostic canonical resolution within [compatibility buckets](docs/internals/entity-resolution.md) (extensible type vocabulary), IDF-based entity edge scoring, co-occurrence tracking, entity graph traversal for ENTITY intent queries
 - **MPFP graph retrieval** — Multi-Path Fact Propagation with meta-path patterns per intent, hop-synchronized edge cache, Forward Push with α=0.15 teleport probability. Replaces single-beam traversal for causal/entity/temporal queries.
 - **Temporal query extraction** — regex-based date range extraction from natural language queries ("last week", "March 2026"), wired as WHERE filters into BM25 and vector search
 - **4-way parallel retrieval** — temporal proximity and entity graph channels added as parallel RRF legs in `query` tool (Tier 3 only), alongside existing BM25 + vector channels

package/SKILL.md

@@ -642,6 +642,15 @@ Symptom: "UserPromptSubmit hook error" on context-surfacing hook (intermittent)
   -> Default hook timeout is 8s (since v0.1.1). If you have an older install, re-run
      `clawmem setup hooks`. If persistent, restart the watcher: `systemctl --user restart
      clawmem-watcher.service`. Healthy memory is under 100MB — if 400MB+, restart clears it.
+
+Symptom: WSL hangs or becomes unresponsive during long sessions / watcher has 100K+ FDs
+  -> Pre-v0.2.3: fs.watch(recursive: true) registered inotify watches on EVERY subdirectory,
+     including excluded dirs (gits/, node_modules/, .git/). Broad collection paths like
+     ~/Projects with 67K subdirs exhausted inotify limits.
+  -> v0.2.3 fix: watcher walks dir trees at startup, skips excluded subtrees, watches
+     non-excluded dirs individually. 500-dir cap per collection path.
+  -> Diagnosis: `ls /proc/$(pgrep -f "clawmem.*watch")/fd | wc -l` — healthy < 15K.
+  -> If still high: narrow broad collection paths. See docs/troubleshooting.md.
 ```
 
 ---
@@ -677,29 +686,6 @@ clawmem consolidate [--dry-run] # Find and archive duplicate low-confidence docu
                                 # Jaccard similarity within same collection
 ```
 
----
-
-## Operational Issue Tracking
-
-When encountering tool failures, instruction contradictions, retrieval gaps, or workflow friction:
-
-Write to `docs/issues/YYYY-MM-DD-<slug>.md`:
-
-```
-# <title>
-- Category: tool-failure | instruction-gap | workflow-friction | retrieval-gap | inconsistency
-- Severity: critical | high | medium
-- Status: open | resolved
-
-## Observed
-## Expected
-## Context
-## Suggested Fix
-```
-
-**Triggers:** repeated tool error, instruction contradicting observed behavior, retrieval consistently missing known content.
-
-**Do NOT log:** one-off transient errors, user-caused issues, already recorded issues.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.2.2",
+  "version": "0.2.4",
   "description": "On-device context engine and memory for AI agents. Claude Code and OpenClaw. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {

package/src/entity.ts

@@ -9,7 +9,7 @@
 
 import type { Database } from "bun:sqlite";
 import { createHash } from "crypto";
-import type { LlamaCpp } from "./llm.ts";
+import type { LLM } from "./llm.ts";
 import { extractJsonFromLLM } from "./amem.ts";
 
 // =============================================================================
@@ -77,6 +77,77 @@ function similarityRatio(a: string, b: string): number {
   return 1 - levenshtein(a, b) / maxLen;
 }
 
+// =============================================================================
+// Quality Filters
+// =============================================================================
+
+const ENTITY_BLOCKLIST = new Set([
+  'entity name', 'entity type', 'description', 'example',
+  'name', 'type', 'value', 'item',
+  'exampletool', 'jane smith', // prompt examples the LLM echoes
+]);
+
+/**
+ * Check if an extracted entity is low quality and should be rejected.
+ * Catches: title-as-entity, long names, template placeholders, heading labels.
+ */
+function isLowQualityEntity(name: string, type: string, docTitle: string): boolean {
+  const normalized = name.toLowerCase().trim();
+  const normalizedTitle = docTitle.toLowerCase().trim();
+
+  // Exact or near-exact title match (Levenshtein > 0.85)
+  if (normalizedTitle.length > 0 && similarityRatio(normalized, normalizedTitle) > 0.85) return true;
+
+  // Too long — likely a title or sentence fragment
+  if (name.length > 60) return true;
+
+  // Template placeholders / generic words
+  if (ENTITY_BLOCKLIST.has(normalized)) return true;
+
+  // Heading labels (trailing colon)
+  if (name.endsWith(':')) return true;
+
+  // Location low-trust: if type is location, validate it
+  if (type === 'location' && !isValidLocation(name)) return true;
+
+  return false;
+}
+
+/**
+ * Validate that a location entity is actually geographic / infrastructure.
+ * Rejects long non-geographic names that the LLM mistyped as location.
+ */
+function isValidLocation(name: string): boolean {
+  // IP addresses
+  if (/\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}/.test(name)) return true;
+  // VM identifiers (e.g., "VM 202", "VM 200")
+  if (/^VM\s+\d+/i.test(name)) return true;
+  // Positive-signal only — no length-based or FQDN fallback
+  return false;
+}
+
+// =============================================================================
+// Compatibility Buckets (for type-agnostic canonical resolution)
+// =============================================================================
+
+// Each bucket contains types that are semantically interchangeable for merging.
+// Types in the same bucket can merge; cross-bucket merges are rejected.
+// The 'tech' bucket captures the common LLM confusion between project/service/tool/concept.
+// Unknown types default to their own isolated bucket (no false merges).
+const ENTITY_BUCKETS: Record<string, string> = {
+  person: 'person',
+  org: 'org',
+  location: 'location',
+  project: 'tech',
+  service: 'tech',
+  tool: 'tech',
+  concept: 'tech',
+};
+
+function getEntityBucket(type: string): string {
+  return ENTITY_BUCKETS[type] ?? type; // unknown types form their own bucket
+}
+
 // =============================================================================
 // Entity ID Generation
 // =============================================================================
@@ -99,7 +170,7 @@ function makeEntityId(name: string, type: string, vault: string = 'default'): st
  * Returns a list of (name, type) pairs.
  */
 export async function extractEntities(
-  llm: LlamaCpp,
+  llm: LLM,
   title: string,
   content: string
 ): Promise<ExtractedEntity[]> {
@@ -113,15 +184,16 @@ Content:
 ${truncated}
 
 Return ONLY valid JSON array:
-[
-  {"name": "Entity Name", "type": "person|project|service|tool|concept|org|location"}
-]
+[{"name": "...", "type": "person|project|service|tool|concept|org|location"}]
 
 Rules:
 - Only include specific, named entities (not generic concepts like "database" or "testing")
 - Normalize names: "VM 202" not "vm202", "ClawMem" not "clawmem"
-- 3-15 entities maximum
+- 0-10 entities. Return empty array [] if no specific entities found
 - Include the most specific type for each entity
+- Do NOT extract the document's title as an entity
+- Do NOT extract heading labels, section names, or sentence fragments
+- Only extract entities that could meaningfully appear in OTHER documents
 Return ONLY the JSON array. /no_think`;
 
   try {
@@ -135,7 +207,7 @@ Return ONLY the JSON array. /no_think`;
     const parsed = extractJsonFromLLM(result.text) as ExtractedEntity[] | null;
     if (!Array.isArray(parsed)) return [];
 
-    // Validate and filter
+    // Validate, filter, and quality-check
     return parsed
       .filter(e =>
         typeof e.name === 'string' &&
@@ -144,7 +216,8 @@ Return ONLY the JSON array. /no_think`;
         e.name.length <= 100 &&
         ['person', 'project', 'service', 'tool', 'concept', 'org', 'location'].includes(e.type)
       )
-      .slice(0, 15);
+      .filter(e => !isLowQualityEntity(e.name, e.type, title))
+      .slice(0, 10);
   } catch (err) {
     console.log(`[entity] LLM extraction failed:`, err);
     return [];
@@ -159,7 +232,13 @@ Return ONLY the JSON array. /no_think`;
  * Resolve an entity name to its canonical form.
  * Uses FTS5 candidate lookup + Levenshtein fuzzy matching.
  *
- * Scoped per vault (via vault parameter) to prevent cross-vault false merges.
+ * Type-agnostic within compatibility buckets:
+ * - person: only merges with person
+ * - org: only merges with org
+ * - location: only merges with location
+ * - tech (project/service/tool/concept): merges freely within bucket
+ *
+ * Scoped per vault to prevent cross-vault false merges.
  *
  * @returns entity_id of canonical match, or null if no match (new entity)
  */
@@ -171,34 +250,41 @@ export function resolveEntityCanonical(
   threshold: number = 0.75
 ): string | null {
   const normalizedName = name.toLowerCase().trim();
+  const inputBucket = getEntityBucket(type);
 
-  // Step 1: FTS5 candidate lookup — join to entity_nodes for vault scoping
+  // Use lower threshold for person names (enables "Andre (Dre) Konrad" ↔ "Dre Konrad")
+  const effectiveThreshold = inputBucket === 'person' ? 0.65 : threshold;
+
+  // Step 1: FTS5 candidate lookup — type-agnostic, vault-scoped
   let candidates: { entity_id: string; name: string; entity_type: string }[] = [];
   try {
     candidates = db.prepare(`
       SELECT f.entity_id, f.name, f.entity_type
       FROM entities_fts f
       JOIN entity_nodes e ON e.entity_id = f.entity_id
-      WHERE entities_fts MATCH ? AND f.entity_type = ? AND e.vault = ?
+      WHERE entities_fts MATCH ? AND e.vault = ?
       LIMIT 20
-    `).all(normalizedName.split(/\s+/).map(w => `"${w}"`).join(' OR '), type, vault) as typeof candidates;
+    `).all(normalizedName.split(/\s+/).map(w => `"${w}"`).join(' OR '), vault) as typeof candidates;
   } catch {
     // FTS5 match may fail on special chars — fall back to LIKE on entity_nodes directly
     candidates = db.prepare(`
       SELECT entity_id, name, entity_type
       FROM entity_nodes
-      WHERE LOWER(name) LIKE ? AND entity_type = ? AND vault = ?
+      WHERE LOWER(name) LIKE ? AND vault = ?
       LIMIT 20
-    `).all(`%${normalizedName}%`, type, vault) as typeof candidates;
+    `).all(`%${normalizedName}%`, vault) as typeof candidates;
   }
 
   if (candidates.length === 0) return null;
 
-  // Step 2: Fuzzy rank candidates by Levenshtein similarity
+  // Step 2: Fuzzy rank candidates, filtering by bucket compatibility
   let bestMatch: { entity_id: string; score: number } | null = null;
   for (const candidate of candidates) {
+    // Reject cross-bucket matches (e.g., don't merge "Andrea" person with "Andrea" project)
+    if (getEntityBucket(candidate.entity_type) !== inputBucket) continue;
+
     const score = similarityRatio(normalizedName, candidate.name.toLowerCase());
-    if (score >= threshold && (!bestMatch || score > bestMatch.score)) {
+    if (score >= effectiveThreshold && (!bestMatch || score > bestMatch.score)) {
       bestMatch = { entity_id: candidate.entity_id, score };
     }
   }
@@ -362,7 +448,7 @@ function clearDocEntityState(db: Database, docId: number): void {
  */
 export async function enrichDocumentEntities(
   db: Database,
-  llm: LlamaCpp,
+  llm: LLM,
   docId: number,
   vault: string = 'default'
 ): Promise<number> {
@@ -444,8 +530,11 @@ export async function enrichDocumentEntities(
         return 0; // Another worker already committed this exact enrichment
       }
 
-      // Clear old derived state if re-enriching (content changed)
-      if (txState || existingState) {
+      // Clear old derived state if re-enriching (content changed or state was externally wiped)
+      const hasOldMentions = db.prepare(
+        `SELECT 1 FROM entity_mentions WHERE doc_id = ? LIMIT 1`
+      ).get(docId);
+      if (txState || existingState || hasOldMentions) {
         clearDocEntityState(db, docId);
       }
 
@@ -459,33 +548,92 @@ export async function enrichDocumentEntities(
         return 0;
       }
 
-      // Now mutate counters — one upsert per unique canonical ID (no inflation)
+      // Mutate counters using precomputed canonical IDs (no redundant re-resolution)
       const resolvedIds: string[] = [];
       for (const { entity, canonicalId } of resolvedPairs) {
-        const entityId = upsertEntity(db, entity.name, entity.type, vault);
-        resolvedIds.push(entityId);
-        recordEntityMention(db, entityId, docId, entity.name);
+        // Check if canonical entity already exists
+        const existing = db.prepare(
+          `SELECT entity_id FROM entity_nodes WHERE entity_id = ?`
+        ).get(canonicalId) as { entity_id: string } | undefined;
+
+        if (existing) {
+          // Existing canonical — increment count
+          db.prepare(`
+            UPDATE entity_nodes SET mention_count = mention_count + 1, last_seen = datetime('now')
+            WHERE entity_id = ?
+          `).run(canonicalId);
+        } else {
+          // New entity — insert
+          db.prepare(`
+            INSERT OR IGNORE INTO entity_nodes (entity_id, entity_type, name, description, created_at, mention_count, last_seen, vault)
+            VALUES (?, ?, ?, NULL, datetime('now'), 1, datetime('now'), ?)
+          `).run(canonicalId, entity.type, entity.name, vault);
+          try {
+            db.prepare(`
+              INSERT OR IGNORE INTO entities_fts (entity_id, name, entity_type)
+              VALUES (?, ?, ?)
+            `).run(canonicalId, entity.name.toLowerCase(), entity.type);
+          } catch { /* FTS insert non-fatal */ }
+        }
+
+        resolvedIds.push(canonicalId);
+        recordEntityMention(db, canonicalId, docId, entity.name);
       }
 
-      // Step 4: Track co-occurrences (deduplicated by canonical ID)
-      trackCoOccurrences(db, resolvedIds);
+      // Step 4: Track co-occurrences (deduplicate resolvedIds to prevent self-pairs)
+      const uniqueResolvedIds = [...new Set(resolvedIds)];
+      trackCoOccurrences(db, uniqueResolvedIds);
+
+      // Step 5: Create entity edges with IDF-based specificity scoring
+      // Rare entities justify edges; ubiquitous entities alone cannot
+      const totalDocs = (db.prepare(`SELECT COUNT(*) as cnt FROM documents WHERE active = 1`).get() as { cnt: number }).cnt;
 
-      // Step 5: Create entity edges in memory_relations
+      // Collect candidate target docs and their shared entities
+      const targetEntityMap = new Map<number, string[]>(); // docId → [entityIds]
       for (const entityId of resolvedIds) {
         const otherDocs = db.prepare(`
           SELECT doc_id FROM entity_mentions
           WHERE entity_id = ? AND doc_id != ?
-          LIMIT 10
+          LIMIT 20
         `).all(entityId, docId) as { doc_id: number }[];
 
         for (const other of otherDocs) {
-          db.prepare(`
-            INSERT OR IGNORE INTO memory_relations (source_id, target_id, relation_type, weight, metadata, created_at)
-            VALUES (?, ?, 'entity', 0.7, ?, datetime('now'))
-          `).run(docId, other.doc_id, JSON.stringify({ entity: entityId }));
+          const existing = targetEntityMap.get(other.doc_id) || [];
+          existing.push(entityId);
+          targetEntityMap.set(other.doc_id, existing);
+        }
+      }
+
+      // Compute IDF per entity (cache for this enrichment)
+      const entityIdf = new Map<string, number>();
+      for (const entityId of resolvedIds) {
+        if (!entityIdf.has(entityId)) {
+          const docFreq = (db.prepare(
+            `SELECT COUNT(DISTINCT doc_id) as cnt FROM entity_mentions WHERE entity_id = ?`
+          ).get(entityId) as { cnt: number }).cnt;
+          entityIdf.set(entityId, Math.log((totalDocs + 1) / (docFreq + 1)));
         }
       }
 
+      // Create edges only when max entity IDF exceeds threshold
+      const idfThreshold = 3.0; // ln-based: filters entities in >5% of docs (e.g., 13+ docs in 262-doc corpus)
+      for (const [targetDocId, sharedEntities] of targetEntityMap) {
+        const maxIdf = Math.max(...sharedEntities.map(eid => entityIdf.get(eid) || 0));
+        if (maxIdf < idfThreshold) continue; // Skip — only ubiquitous entities shared
+
+        // Weight: IDF specificity + shared-count bonus (multi-entity overlap outranks single)
+        const sharedBonus = Math.min(0.15, 0.05 * (sharedEntities.length - 1));
+        const weight = Math.min(1.0, 0.3 + 0.12 * maxIdf + sharedBonus);
+        const bestEntity = sharedEntities.reduce((best, eid) =>
+          (entityIdf.get(eid) || 0) > (entityIdf.get(best) || 0) ? eid : best
+        );
+
+        db.prepare(`
+          INSERT OR IGNORE INTO memory_relations (source_id, target_id, relation_type, weight, metadata, created_at)
+          VALUES (?, ?, 'entity', ?, ?, datetime('now'))
+        `).run(docId, targetDocId, weight, JSON.stringify({ entity: bestEntity, shared: sharedEntities.length }));
+      }
+
       // Persist enrichment state LAST — only after all derived data written
       db.prepare(`
         INSERT OR REPLACE INTO entity_enrichment_state (doc_id, input_hash, enriched_at)

package/src/indexer.ts

@@ -37,7 +37,7 @@ export interface IndexStats {
 // Exclusion Rules
 // =============================================================================
 
-const EXCLUDED_DIRS = new Set([
+export const EXCLUDED_DIRS = new Set([
   "_PRIVATE",
   ".clawmem",
   ".git",

package/src/intent.ts

@@ -10,7 +10,7 @@
 
 import type { Database } from "bun:sqlite";
 import { createHash } from "crypto";
-import type { LlamaCpp } from "./llm.ts";
+import type { LLM } from "./llm.ts";
 
 export type IntentType = 'WHY' | 'WHEN' | 'ENTITY' | 'WHAT';
 
@@ -179,7 +179,7 @@ function classifyIntentHeuristic(query: string): IntentResult {
  */
 export async function classifyIntent(
   query: string,
-  llm: LlamaCpp,
+  llm: LLM,
   db: Database
 ): Promise<IntentResult> {
   // Check cache first (1 hour TTL)
@@ -268,7 +268,7 @@ export type QueryClause = {
  */
 export async function decomposeQuery(
   query: string,
-  llm: LlamaCpp,
+  llm: LLM,
   db: Database,
   sessionContext?: string
 ): Promise<QueryClause[]> {

package/src/watcher.ts

@@ -1,9 +1,14 @@
 /**
  * ClawMem File Watcher - fs.watch with debounce for incremental reindex
+ *
+ * Walks each directory tree at startup, skipping excluded dirs (gits/,
+ * node_modules/, .git/, etc.), and watches only non-excluded directories.
+ * This prevents inotify FD exhaustion on trees with large cloned repos.
  */
 
-import { watch, type WatchEventType } from "fs";
-import { shouldExclude } from "./indexer.ts";
+import { watch, readdirSync, statSync, type WatchEventType } from "fs";
+import { join, relative } from "path";
+import { shouldExclude, EXCLUDED_DIRS } from "./indexer.ts";
 
 export type WatcherOptions = {
   debounceMs?: number;
@@ -11,6 +16,42 @@ export type WatcherOptions = {
   onError?: (error: Error) => void;
 };
 
+/**
+ * Walk a directory tree, returning only directories that are NOT excluded.
+ * Stops recursion into excluded subtrees (gits/, node_modules/, .git/, etc.).
+ */
+function walkNonExcludedDirs(root: string): string[] {
+  const dirs: string[] = [root];
+  const queue: string[] = [root];
+
+  while (queue.length > 0) {
+    const current = queue.pop()!;
+    let entries: string[];
+    try {
+      entries = readdirSync(current);
+    } catch {
+      continue; // Permission denied or deleted
+    }
+
+    for (const entry of entries) {
+      // Skip excluded directory names before stat
+      if (EXCLUDED_DIRS.has(entry) || (entry.startsWith(".") && entry !== ".")) continue;
+
+      const fullPath = join(current, entry);
+      try {
+        if (statSync(fullPath).isDirectory()) {
+          dirs.push(fullPath);
+          queue.push(fullPath);
+        }
+      } catch {
+        // stat failed — skip
+      }
+    }
+  }
+
+  return dirs;
+}
+
 export function startWatcher(
   directories: string[],
   options: WatcherOptions
@@ -20,34 +61,54 @@ export function startWatcher(
   const watchers: ReturnType<typeof watch>[] = [];
 
   for (const dir of directories) {
-    try {
-      const watcher = watch(dir, { recursive: true }, (event, filename) => {
-        if (!filename) return;
-        // Accept .md files (indexing) and .jsonl only within .beads/ (Dolt backend)
-        const isMd = filename.endsWith(".md");
-        const isBeadsJsonl = filename.endsWith(".jsonl") && filename.includes(".beads/");
-        if (!isMd && !isBeadsJsonl) return;
-        if (shouldExclude(filename)) return;
-
-        const fullPath = `${dir}/${filename}`;
-        const existing = pending.get(fullPath);
-        if (existing) clearTimeout(existing);
-
-        pending.set(fullPath, setTimeout(async () => {
-          pending.delete(fullPath);
-          try {
-            await onChanged(fullPath, event);
-          } catch (err) {
-            onError?.(err instanceof Error ? err : new Error(String(err)));
-          }
-        }, debounceMs));
-      });
-      watcher.on("error", (err) => {
-        onError?.(err instanceof Error ? err : new Error(String(err)));
-      });
-      watchers.push(watcher);
-    } catch (err) {
-      onError?.(err instanceof Error ? err : new Error(`Failed to watch ${dir}: ${err}`));
+    // Walk the tree, skipping excluded dirs — watch each non-excluded dir individually
+    const watchableDirs = walkNonExcludedDirs(dir);
+
+    // Safety: warn and cap if a single collection path produces too many dirs
+    const MAX_WATCH_DIRS = 500;
+    if (watchableDirs.length > MAX_WATCH_DIRS) {
+      console.log(`[watcher] WARNING: ${dir} has ${watchableDirs.length} dirs — capping at ${MAX_WATCH_DIRS} to prevent FD exhaustion. Consider narrowing the collection path.`);
+      watchableDirs.length = MAX_WATCH_DIRS;
+    } else {
+      console.log(`[watcher] ${dir}: watching ${watchableDirs.length} dirs`);
+    }
+
+    for (const watchDir of watchableDirs) {
+      try {
+        // Non-recursive watch — each dir watched individually
+        const watcher = watch(watchDir, (event, filename) => {
+          if (!filename) return;
+          // Accept .md files (indexing) and .jsonl only within .beads/ (Dolt backend)
+          const isMd = filename.endsWith(".md");
+          const isBeadsJsonl = filename.endsWith(".jsonl") && filename.includes(".beads/");
+          if (!isMd && !isBeadsJsonl) return;
+
+          const relativeToDirRoot = relative(dir, join(watchDir, filename));
+          if (shouldExclude(relativeToDirRoot)) return;
+
+          const fullPath = join(watchDir, filename);
+          const existing = pending.get(fullPath);
+          if (existing) clearTimeout(existing);
+
+          pending.set(fullPath, setTimeout(async () => {
+            pending.delete(fullPath);
+            try {
+              await onChanged(fullPath, event);
+            } catch (err) {
+              onError?.(err instanceof Error ? err : new Error(String(err)));
+            }
+          }, debounceMs));
+        });
+        watcher.on("error", (err) => {
+          onError?.(err instanceof Error ? err : new Error(String(err)));
+        });
+        watchers.push(watcher);
+      } catch (err) {
+        // Individual dir watch failure is non-fatal — skip it
+        if (onError) {
+          onError(err instanceof Error ? err : new Error(`Failed to watch ${watchDir}: ${err}`));
+        }
+      }
     }
   }