@velvetmonkey/flywheel-memory 2.0.76 → 2.0.78

package/README.md

@@ -12,13 +12,13 @@
 [![Scale](https://img.shields.io/badge/scale-100k--line%20files%20%7C%202.5k%20entities-brightgreen.svg)](https://github.com/velvetmonkey/flywheel-memory/blob/main/docs/TESTING.md#performance-benchmarks)
 [![Tests](https://img.shields.io/badge/tests-2,456%20passed-brightgreen.svg)](https://github.com/velvetmonkey/flywheel-memory/blob/main/docs/TESTING.md)
 
-| | Without Flywheel | With Flywheel |
+| | Grep approach | Flywheel |
 |---|---|---|
-| "What's overdue?" | Read every file | Indexed query, <10ms |
-| "What links here?" | grep every file | Pre-indexed backlink graph |
+| "What's overdue?" | Grep + read matches (~500-2,000 tokens) | Indexed metadata query (~50-200 tokens) |
+| "What links here?" | Grep for note name (flat list, no graph) | Pre-indexed backlink graph (<10ms) |
 | "Add a meeting note" | Raw write, no linking | Structured write + auto-wikilink |
-| "What should I link?" | Manual or grep | Smart scoring + semantic understanding |
-| Token cost | 2,000-250,000 | 50-200 |
+| "What should I link?" | Not possible | 10-dimension scoring + semantic search |
+| Hubs, orphans, paths? | Not possible | Pre-indexed graph analysis |
 
 51 tools across 17 categories. 6-line config. Zero cloud dependencies.
 
@@ -35,8 +35,6 @@ Then ask: *"How much have I billed Acme Corp?"*
 
 ## See It Work
 
-![Flywheel demo](https://raw.githubusercontent.com/velvetmonkey/flywheel-memory/main/demos/flywheel-demo.gif)
-
 ### Read: "How much have I billed Acme Corp?"
 
 From the [carter-strategy](https://github.com/velvetmonkey/flywheel-memory/tree/main/demos/carter-strategy/) demo -- a solo consultant with 3 clients, 5 projects, and $27K in invoices.
@@ -44,49 +42,47 @@ From the [carter-strategy](https://github.com/velvetmonkey/flywheel-memory/tree/
 ```
 ❯ How much have I billed Acme Corp?
 
-● flywheel › get_backlinks
-  path: "clients/Acme Corp.md"
-  ← invoices/INV-2025-047.md
-    invoices/INV-2025-048.md
-    projects/Acme Data Migration.md
-    proposals/Acme Analytics Add-on.md
+● flywheel › search
+  query: "Acme Corp"
+  → clients/Acme Corp.md, invoices/INV-2025-047.md,
+    invoices/INV-2025-048.md, projects/Acme Data Migration.md
+
+● read_file("clients/Acme Corp.md")
+  ← total_billed: 156000, rate: $300/hr, status: active
 
-● flywheel › get_note_metadata
-  path: "invoices/INV-2025-047.md"
+● read_file("invoices/INV-2025-047.md")
   ← amount: 15000, status: paid, period: November 2025
 
-● flywheel › get_note_metadata
-  path: "invoices/INV-2025-048.md"
+● read_file("invoices/INV-2025-048.md")
   ← amount: 12000, status: pending, period: December 2025
 
 ┌─ RESULT ──────────────────────────────────────────────┐
-│ Acme Corp Billing: $27,000                            │
+│ Acme Corp: $156K total billed                         │
 │                                                       │
 │   Paid:    $15,000 — Acme Data Migration (Nov 2025)   │
 │   Pending: $12,000 — Acme Data Migration (Dec 2025)   │
 │                                                       │
-│ 4 tool calls · ~160 tokens · 0 files read             │
+│ Also: $35K pending proposal (Analytics Add-on)        │
 └───────────────────────────────────────────────────────┘
 ```
 
-Claude didn't read any files. It navigated the graph: backlinks to find related notes, metadata to extract the numbers.
+Flywheel's indexed search found all Acme-related notes in one call. Claude read the files it needed for billing details. No grepping, no guessing paths.
 
-```
-Same 3 queries without Flywheel: 11,150 tokens (reading files repeatedly)
-Same 3 queries with Flywheel:       300 tokens (querying the index)
-                                     37x savings
-```
+Flywheel's search found all related notes in one call. Without it, Claude would grep for "Acme" and scan every matching file.
+
+The bigger difference isn't just tokens — it's that Flywheel answers structural questions (backlinks, hubs, shortest paths, schema analysis) that file-level access can't answer at all.
 
 ### Write: Auto-wikilinks on every mutation
 
 ```
-❯ Log that I finished the Acme strategy deck
+❯ Log that Stacy Thompson reviewed the API Security Checklist for Acme before the Beta Corp Dashboard kickoff
 
 ● flywheel › vault_add_to_section
   path: "daily-notes/2026-01-04.md"
   section: "Log"
-  content: "finished the [[Acme Corp]] strategy deck"
-                       ↑ auto-linked because Acme Corp.md exists
+  content: "[[Stacy Thompson]] reviewed the [[API Security Checklist]] for [[Acme Corp|Acme]] before the [[Beta Corp Dashboard]] kickoff → [[GlobalBank API Audit]], [[Acme Analytics Add-on]], [[Acme Data Migration]]"
+            ↑ 4 entities auto-linked — "Acme" resolved to Acme Corp via alias
+            → 3 contextual suggestions appended (scored ≥12 via co-occurrence with linked entities)
 ```
 
 Try it yourself: `cd demos/carter-strategy && claude`
@@ -194,8 +190,8 @@ graph LR
 ```
 
 ```
-Input:  "Met with Sarah about the data migration"
-Output: "Met with [[Sarah Mitchell]] about the [[Acme Data Migration]]"
+Input:  "Stacy Thompson finished reviewing the API Security Checklist for the Beta Corp Dashboard"
+Output: "[[Stacy Thompson]] finished reviewing the [[API Security Checklist]] for the [[Beta Corp Dashboard]]"
 ```
 
 No manual linking. No broken references. Use compounds into structure, structure compounds into intelligence.

package/dist/index.js

@@ -1886,24 +1886,42 @@ async function initEmbeddings() {
   if (pipeline) return;
   if (initPromise) return initPromise;
   initPromise = (async () => {
-    try {
-      const transformers = await Function("specifier", "return import(specifier)")("@huggingface/transformers");
-      pipeline = await transformers.pipeline("feature-extraction", activeModelConfig.id, {
-        dtype: "fp32"
-      });
-      if (activeModelConfig.dims === 0) {
-        const probe = await pipeline("test", { pooling: "mean", normalize: true });
-        activeModelConfig.dims = probe.data.length;
-        console.error(`[Semantic] Probed model ${activeModelConfig.id}: ${activeModelConfig.dims} dims`);
-      }
-    } catch (err) {
-      initPromise = null;
-      if (err instanceof Error && (err.message.includes("Cannot find package") || err.message.includes("MODULE_NOT_FOUND") || err.message.includes("Cannot find module") || err.message.includes("ERR_MODULE_NOT_FOUND"))) {
-        throw new Error(
-          "Semantic search requires @huggingface/transformers. Install it with: npm install @huggingface/transformers"
-        );
+    const MAX_RETRIES = 3;
+    const RETRY_DELAYS = [2e3, 5e3, 1e4];
+    for (let attempt = 1; attempt <= MAX_RETRIES; attempt++) {
+      try {
+        const transformers = await Function("specifier", "return import(specifier)")("@huggingface/transformers");
+        console.error(`[Semantic] Loading model ${activeModelConfig.id} (~23MB, cached after first download)...`);
+        pipeline = await transformers.pipeline("feature-extraction", activeModelConfig.id, {
+          dtype: "fp32"
+        });
+        console.error(`[Semantic] Model loaded successfully`);
+        if (activeModelConfig.dims === 0) {
+          const probe = await pipeline("test", { pooling: "mean", normalize: true });
+          activeModelConfig.dims = probe.data.length;
+          console.error(`[Semantic] Probed model ${activeModelConfig.id}: ${activeModelConfig.dims} dims`);
+        }
+        return;
+      } catch (err) {
+        if (err instanceof Error && (err.message.includes("Cannot find package") || err.message.includes("MODULE_NOT_FOUND") || err.message.includes("Cannot find module") || err.message.includes("ERR_MODULE_NOT_FOUND"))) {
+          initPromise = null;
+          throw new Error(
+            "Semantic search requires @huggingface/transformers. Install it with: npm install @huggingface/transformers"
+          );
+        }
+        if (attempt < MAX_RETRIES) {
+          const delay = RETRY_DELAYS[attempt - 1];
+          console.error(`[Semantic] Model load failed (attempt ${attempt}/${MAX_RETRIES}): ${err instanceof Error ? err.message : err}`);
+          console.error(`[Semantic] Retrying in ${delay / 1e3}s...`);
+          await new Promise((resolve2) => setTimeout(resolve2, delay));
+          pipeline = null;
+        } else {
+          console.error(`[Semantic] Model load failed after ${MAX_RETRIES} attempts: ${err instanceof Error ? err.message : err}`);
+          console.error(`[Semantic] Semantic search disabled. Keyword search (BM25) remains available.`);
+          initPromise = null;
+          throw err;
+        }
       }
-      throw err;
     }
   })();
   return initPromise;
@@ -2552,7 +2570,7 @@ function deserializeVaultIndex(data) {
     builtAt: new Date(data.builtAt)
   };
 }
-function loadVaultIndexFromCache(stateDb2, scannedFileCount, maxAgeMs = 24 * 60 * 60 * 1e3, tolerancePercent = 5) {
+function loadVaultIndexFromCache(stateDb2, scannedFileCount, maxAgeMs = 24 * 60 * 60 * 1e3, tolerancePercent = 5, newestFileMtime) {
   const info = getVaultIndexCacheInfo(stateDb2);
   if (!info) {
     console.error("[Flywheel] No cached index found");
@@ -2569,6 +2587,10 @@ function loadVaultIndexFromCache(stateDb2, scannedFileCount, maxAgeMs = 24 * 60
     console.error(`[Flywheel] Cache invalid: too old (${Math.round(age / 1e3 / 60)} minutes)`);
     return null;
   }
+  if (newestFileMtime && newestFileMtime.getTime() > info.builtAt.getTime()) {
+    console.error(`[Flywheel] Cache invalid: files modified since cache was built (newest file: ${newestFileMtime.toISOString()}, cache built: ${info.builtAt.toISOString()})`);
+    return null;
+  }
   const data = loadVaultIndexCache(stateDb2);
   if (!data) {
     console.error("[Flywheel] Failed to load cached index data");
@@ -9570,7 +9592,8 @@ function registerHealthTools(server2, getIndex, getVaultPath, getConfig = () =>
 import { z as z4 } from "zod";
 import {
   searchEntities,
-  searchEntitiesPrefix
+  searchEntitiesPrefix,
+  getEntityByName as getEntityByName2
 } from "@velvetmonkey/vault-core";
 function matchesFrontmatter(note, where) {
   for (const [key, value] of Object.entries(where)) {
@@ -9618,6 +9641,37 @@ function inFolder(note, folder) {
   const normalizedFolder = folder.endsWith("/") ? folder : folder + "/";
   return note.path.startsWith(normalizedFolder) || note.path.split("/")[0] === folder.replace("/", "");
 }
+function enrichResult(result, index, stateDb2) {
+  const note = index.notes.get(result.path);
+  const normalizedPath = result.path.toLowerCase().replace(/\.md$/, "");
+  const backlinks = index.backlinks.get(normalizedPath) || [];
+  const enriched = {
+    path: result.path,
+    title: result.title
+  };
+  if (result.snippet) enriched.snippet = result.snippet;
+  if (note) {
+    enriched.frontmatter = note.frontmatter;
+    enriched.tags = note.tags;
+    enriched.aliases = note.aliases;
+    enriched.backlink_count = backlinks.length;
+    enriched.outlink_count = note.outlinks.length;
+    enriched.modified = note.modified.toISOString();
+    if (note.created) enriched.created = note.created.toISOString();
+  }
+  if (stateDb2) {
+    try {
+      const entity = getEntityByName2(stateDb2, result.title);
+      if (entity) {
+        enriched.category = entity.category;
+        enriched.hub_score = entity.hubScore;
+        if (entity.description) enriched.description = entity.description;
+      }
+    } catch {
+    }
+  }
+  return enriched;
+}
 function sortNotes(notes, sortBy, order) {
   const sorted = [...notes];
   sorted.sort((a, b) => {
@@ -9825,22 +9879,20 @@ function registerQueryTools(server2, getIndex, getVaultPath, getStateDb) {
             const fts5Map = new Map(fts5Results.map((r) => [normalizePath2(r.path), r]));
             const semanticMap = new Map(semanticResults.map((r) => [normalizePath2(r.path), r]));
             const entityMap = new Map(entityResults.map((r) => [normalizePath2(r.path), r]));
-            const merged = Array.from(allPaths).map((p) => ({
-              path: p,
-              title: fts5Map.get(p)?.title || semanticMap.get(p)?.title || entityMap.get(p)?.name || p.replace(/\.md$/, "").split("/").pop() || p,
-              snippet: fts5Map.get(p)?.snippet,
-              rrf_score: Math.round((rrfScores.get(p) || 0) * 1e4) / 1e4,
-              in_fts5: fts5Map.has(p),
-              in_semantic: semanticMap.has(p),
-              in_entity: entityMap.has(p)
-            }));
-            merged.sort((a, b) => b.rrf_score - a.rrf_score);
+            const merged = Array.from(allPaths).map((p) => {
+              const title = fts5Map.get(p)?.title || semanticMap.get(p)?.title || entityMap.get(p)?.name || p.replace(/\.md$/, "").split("/").pop() || p;
+              const snippet = fts5Map.get(p)?.snippet;
+              const rrfScore = rrfScores.get(p) || 0;
+              return { rrfScore, ...enrichResult({ path: p, title, snippet }, index, getStateDb()) };
+            });
+            merged.sort((a, b) => b.rrfScore - a.rrfScore);
+            const results = merged.slice(0, limit).map(({ rrfScore, ...rest }) => rest);
             return { content: [{ type: "text", text: JSON.stringify({
               scope,
               method: "hybrid",
               query,
               total_results: Math.min(merged.length, limit),
-              results: merged.slice(0, limit)
+              results
             }, null, 2) }] };
           } catch (err) {
             console.error("[Semantic] Hybrid search failed, falling back to FTS5:", err instanceof Error ? err.message : err);
@@ -9849,11 +9901,10 @@ function registerQueryTools(server2, getIndex, getVaultPath, getStateDb) {
         if (entityResults.length > 0) {
           const normalizePath2 = (p) => p.replace(/\\/g, "/").replace(/\/+/g, "/");
           const fts5Map = new Map(fts5Results.map((r) => [normalizePath2(r.path), r]));
-          const entityNormMap = new Map(entityResults.map((r) => [normalizePath2(r.path), r]));
           const entityRanked = entityResults.filter((r) => !fts5Map.has(normalizePath2(r.path)));
           const merged = [
-            ...fts5Results.map((r) => ({ path: r.path, title: r.title, snippet: r.snippet, in_entity: entityNormMap.has(normalizePath2(r.path)) })),
-            ...entityRanked.map((r) => ({ path: r.path, title: r.name, snippet: void 0, in_entity: true }))
+            ...fts5Results.map((r) => enrichResult({ path: r.path, title: r.title, snippet: r.snippet }, index, getStateDb())),
+            ...entityRanked.map((r) => enrichResult({ path: r.path, title: r.name }, index, getStateDb()))
           ];
           return { content: [{ type: "text", text: JSON.stringify({
             scope: "content",
@@ -9863,12 +9914,13 @@ function registerQueryTools(server2, getIndex, getVaultPath, getStateDb) {
             results: merged.slice(0, limit)
           }, null, 2) }] };
         }
+        const enrichedFts5 = fts5Results.map((r) => enrichResult({ path: r.path, title: r.title, snippet: r.snippet }, index, getStateDb()));
         return { content: [{ type: "text", text: JSON.stringify({
           scope: "content",
           method: "fts5",
           query,
-          total_results: fts5Results.length,
-          results: fts5Results
+          total_results: enrichedFts5.length,
+          results: enrichedFts5
         }, null, 2) }] };
       }
       return { content: [{ type: "text", text: JSON.stringify({ error: "Invalid scope" }, null, 2) }] };
@@ -10679,7 +10731,8 @@ async function findSections(index, headingPattern, vaultPath2, folder) {
 }
 
 // src/tools/read/primitives.ts
-function registerPrimitiveTools(server2, getIndex, getVaultPath, getConfig = () => ({})) {
+import { getEntityByName as getEntityByName3 } from "@velvetmonkey/vault-core";
+function registerPrimitiveTools(server2, getIndex, getVaultPath, getConfig = () => ({}), getStateDb = () => null) {
   server2.registerTool(
     "get_note_structure",
     {
@@ -10707,8 +10760,31 @@ function registerPrimitiveTools(server2, getIndex, getVaultPath, getConfig = ()
           }
         }
       }
+      const note = index.notes.get(path33);
+      const enriched = { ...result };
+      if (note) {
+        enriched.frontmatter = note.frontmatter;
+        enriched.tags = note.tags;
+        enriched.aliases = note.aliases;
+        const normalizedPath = path33.toLowerCase().replace(/\.md$/, "");
+        const backlinks = index.backlinks.get(normalizedPath) || [];
+        enriched.backlink_count = backlinks.length;
+        enriched.outlink_count = note.outlinks.length;
+      }
+      const stateDb2 = getStateDb();
+      if (stateDb2 && note) {
+        try {
+          const entity = getEntityByName3(stateDb2, note.title);
+          if (entity) {
+            enriched.category = entity.category;
+            enriched.hub_score = entity.hubScore;
+            if (entity.description) enriched.description = entity.description;
+          }
+        } catch {
+        }
+      }
       return {
-        content: [{ type: "text", text: JSON.stringify(result, null, 2) }]
+        content: [{ type: "text", text: JSON.stringify(enriched, null, 2) }]
       };
     }
   );
@@ -19363,7 +19439,7 @@ var ALL_CATEGORIES = [
   "policy",
   "memory"
 ];
-var DEFAULT_PRESET = "full";
+var DEFAULT_PRESET = "minimal";
 function parseEnabledCategories() {
   const envValue = (process.env.FLYWHEEL_TOOLS ?? process.env.FLYWHEEL_PRESET)?.trim();
   if (!envValue) {
@@ -19565,7 +19641,7 @@ registerSystemTools(
 registerGraphTools(server, () => vaultIndex, () => vaultPath, () => stateDb);
 registerWikilinkTools(server, () => vaultIndex, () => vaultPath);
 registerQueryTools(server, () => vaultIndex, () => vaultPath, () => stateDb);
-registerPrimitiveTools(server, () => vaultIndex, () => vaultPath, () => flywheelConfig);
+registerPrimitiveTools(server, () => vaultIndex, () => vaultPath, () => flywheelConfig, () => stateDb);
 registerGraphAnalysisTools(server, () => vaultIndex, () => vaultPath, () => stateDb, () => flywheelConfig);
 registerVaultSchemaTools(server, () => vaultIndex, () => vaultPath);
 registerNoteIntelligenceTools(server, () => vaultIndex, () => vaultPath, () => flywheelConfig);
@@ -19669,7 +19745,8 @@ async function main() {
       const files = await scanVault(vaultPath);
       const noteCount = files.length;
       serverLog("index", `Found ${noteCount} markdown files`);
-      cachedIndex = loadVaultIndexFromCache(stateDb, noteCount);
+      const newestMtime = files.reduce((max, f) => f.modified > max ? f.modified : max, /* @__PURE__ */ new Date(0));
+      cachedIndex = loadVaultIndexFromCache(stateDb, noteCount, void 0, void 0, newestMtime);
     } catch (err) {
       serverLog("index", `Cache check failed: ${err instanceof Error ? err.message : err}`, "warn");
     }
@@ -19870,30 +19947,45 @@ async function runPostIndexWork(index) {
     if (hasEmbeddingsIndex() && !modelChanged) {
       serverLog("semantic", "Embeddings already built, skipping full scan");
     } else {
-      setEmbeddingsBuilding(true);
-      buildEmbeddingsIndex(vaultPath, (p) => {
-        if (p.current % 100 === 0 || p.current === p.total) {
-          serverLog("semantic", `Embedding ${p.current}/${p.total} notes...`);
-        }
-      }).then(async () => {
-        if (stateDb) {
-          const entities = getAllEntitiesFromDb3(stateDb);
-          if (entities.length > 0) {
-            const entityMap = new Map(entities.map((e) => [e.name, {
-              name: e.name,
-              path: e.path,
-              category: e.category,
-              aliases: e.aliases
-            }]));
-            await buildEntityEmbeddingsIndex(vaultPath, entityMap);
+      const MAX_BUILD_RETRIES = 2;
+      const attemptBuild = async (attempt) => {
+        setEmbeddingsBuilding(true);
+        try {
+          await buildEmbeddingsIndex(vaultPath, (p) => {
+            if (p.current % 100 === 0 || p.current === p.total) {
+              serverLog("semantic", `Embedding ${p.current}/${p.total} notes...`);
+            }
+          });
+          if (stateDb) {
+            const entities = getAllEntitiesFromDb3(stateDb);
+            if (entities.length > 0) {
+              const entityMap = new Map(entities.map((e) => [e.name, {
+                name: e.name,
+                path: e.path,
+                category: e.category,
+                aliases: e.aliases
+              }]));
+              await buildEntityEmbeddingsIndex(vaultPath, entityMap);
+            }
           }
+          loadEntityEmbeddingsToMemory();
+          setEmbeddingsBuildState("complete");
+          serverLog("semantic", "Embeddings ready \u2014 searches now use hybrid ranking");
+        } catch (err) {
+          const msg = err instanceof Error ? err.message : String(err);
+          if (attempt < MAX_BUILD_RETRIES) {
+            const delay = 1e4;
+            serverLog("semantic", `Build failed (attempt ${attempt}/${MAX_BUILD_RETRIES}): ${msg}. Retrying in ${delay / 1e3}s...`, "error");
+            await new Promise((resolve2) => setTimeout(resolve2, delay));
+            return attemptBuild(attempt + 1);
+          }
+          serverLog("semantic", `Embeddings build failed after ${MAX_BUILD_RETRIES} attempts: ${msg}`, "error");
+          serverLog("semantic", "Keyword search (BM25) remains fully available", "error");
+        } finally {
+          setEmbeddingsBuilding(false);
         }
-        loadEntityEmbeddingsToMemory();
-        setEmbeddingsBuildState("complete");
-        serverLog("semantic", "Embeddings ready");
-      }).catch((err) => {
-        serverLog("semantic", `Embeddings build failed: ${err instanceof Error ? err.message : err}`, "error");
-      });
+      };
+      attemptBuild(1);
     }
   } else {
     serverLog("semantic", "Skipping \u2014 FLYWHEEL_SKIP_EMBEDDINGS");

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@velvetmonkey/flywheel-memory",
-  "version": "2.0.76",
+  "version": "2.0.78",
   "description": "MCP server that gives Claude full read/write access to your Obsidian vault. Select from 51 tools for search, backlinks, graph queries, mutations, agent memory, and hybrid semantic search.",
   "type": "module",
   "main": "dist/index.js",
@@ -52,7 +52,7 @@
   },
   "dependencies": {
     "@modelcontextprotocol/sdk": "^1.25.1",
-    "@velvetmonkey/vault-core": "^2.0.75",
+    "@velvetmonkey/vault-core": "^2.0.78",
     "better-sqlite3": "^11.0.0",
     "chokidar": "^4.0.0",
     "gray-matter": "^4.0.3",