@theglitchking/semantic-pages 0.4.3 → 0.4.5

package/README.md

@@ -6,7 +6,7 @@
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
 > [!IMPORTANT]
-> Semantic Pages runs a local embedding model (~80MB) on first launch. This download happens once and is cached at `~/.semantic-pages/models/`. No API key required. No data leaves your machine.
+> Semantic Pages runs a local embedding model (~22MB) on first launch. This download happens once and is cached at `~/.semantic-pages/models/`. No API key required. No data leaves your machine.
 
 ---
 
@@ -18,7 +18,7 @@ When you have markdown notes scattered across a project — a `vault/`, `docs/`,
 
 ## Operational Summary
 
-The server indexes all `.md` files in a directory you point it at. Each file is parsed for YAML frontmatter, `[[wikilinks]]`, `#tags`, and headings. The text content is split into ~512-token chunks and embedded locally using the `nomic-embed-text-v1.5` model running via WebAssembly in Node.js. These embeddings are stored in an HNSW index for fast approximate nearest neighbor search. Simultaneously, a directed graph is built from wikilinks and shared tags using graphology.
+The server indexes all `.md` files in a directory you point it at. Each file is parsed for YAML frontmatter, `[[wikilinks]]`, `#tags`, and headings. The text content is split into chunks and embedded locally using `all-MiniLM-L6-v2` — a 22MB model that runs natively in Node.js via ONNX. These embeddings are stored in an HNSW index for fast approximate nearest neighbor search. Simultaneously, a directed graph is built from wikilinks and shared tags using graphology.
 
 When Claude calls `search_semantic`, the query is embedded and compared against all chunks via cosine similarity. When Claude calls `search_graph`, it does a breadth-first traversal from matching nodes. `search_hybrid` combines both — semantic results re-ranked by graph proximity. Beyond search, Claude can create, read, update, delete, and move notes, manage YAML frontmatter fields, add/remove/rename tags vault-wide, and query the knowledge graph for backlinks, forwardlinks, shortest paths, and connectivity statistics.
 
@@ -223,7 +223,7 @@ semantic-pages --notes ./vault --reindex
 - If the index seems stale or corrupted
 - After changing the embedding model
 
-**What to expect**: Full re-parse, re-embed, and re-index of all markdown files. Takes 10-60 seconds depending on vault size and whether the model is cached.
+**What to expect**: Full re-parse, re-embed, and re-index of all markdown files. Takes 30 seconds to ~20 minutes depending on vault size and hardware. See [Performance Tuning](./.documentation/performance-tuning.md) for details.
 
 ---
 
@@ -480,8 +480,8 @@ src/
 | Markdown parsing | `unified` + `remark-parse` | AST-based, handles wikilinks |
 | Frontmatter | `gray-matter` | YAML/TOML frontmatter extraction |
 | Wikilinks | `remark-wiki-link` | `[[note-name]]` extraction from AST |
-| Embeddings | `@huggingface/transformers` | WASM runtime, no Python, no API key |
-| Embedding model | `nomic-embed-text-v1.5` | High quality, ~80MB, runs locally |
+| Embeddings | `@huggingface/transformers` + `onnxruntime-node` | Native ONNX runtime, no Python, no API key |
+| Embedding model | `all-MiniLM-L6-v2` (default) | ~22MB, fast (~3 min / 3K chunks), excellent retrieval quality |
 | Vector index | `hnswlib-node` | HNSW algorithm, same as production vector DBs |
 | Knowledge graph | `graphology` | Directed graph, serializable, rich algorithms |
 | Graph algorithms | `graphology-traversal` + `graphology-shortest-path` | BFS, shortest path |
@@ -519,7 +519,7 @@ Plain text → split at sentence boundaries → ~512 token chunks
 
 #### Step 3: Embed
 ```
-Each chunk → nomic-embed-text-v1.5 (WASM) → normalized Float32Array
+Each chunk → all-MiniLM-L6-v2 (native ONNX) → normalized Float32Array
 ```
 
 #### Step 4: Index
@@ -573,14 +573,16 @@ const path = graph.findPath("overview.md", "auth.md");
 
 | Metric | Value |
 |--------|-------|
-| Index 100 notes | ~5 seconds |
-| Index 1,000 notes | ~30 seconds |
+| Index 100 notes (~600 chunks) | ~30 seconds |
+| Index 500 notes (~3,000 chunks) | ~3–5 minutes |
+| Index 2,000 notes (~12,000 chunks) | ~15–20 minutes |
 | Semantic search latency | <100ms |
 | Text search latency | <10ms |
 | Graph traversal latency | <5ms |
-| Model download (first run) | ~80MB, cached at `~/.semantic-pages/models/` |
-| Index size (100 notes) | ~10MB |
-| npm package size | 85.7 kB |
+| Subsequent server starts (warm cache) | <1 second |
+| Model download (first run) | ~22MB, cached at `~/.semantic-pages/models/` |
+| Index size (500 notes) | ~30–50MB |
+| npm package size | ~112 kB |
 
 ---
 
@@ -592,6 +594,18 @@ const path = graph.findPath("overview.md", "auth.md");
 
 ---
 
+## Documentation
+
+Deep-dive guides are in [`.documentation/`](./.documentation/):
+
+- [**How It Works**](./.documentation/how-it-works.md) — architecture, processing pipeline, index format, search mechanics
+- [**Performance Tuning**](./.documentation/performance-tuning.md) — model selection, batch size, workers, benchmarks
+- [**Embedder Guide**](./.documentation/embedder-guide.md) — when/how to tune the embedder, model switching, cache management
+- [**Troubleshooting**](./.documentation/troubleshooting.md) — common problems and fixes
+- [**Changelog**](./.documentation/changelog.md) — version history with rationale
+
+---
+
 ## Troubleshooting
 
 ### Installation Issues

package/dist/{chunk-TDC45FQJ.js → chunk-VAPQ4NA3.js}

@@ -10,7 +10,7 @@ import remarkParse from "remark-parse";
 import remarkWikiLink from "remark-wiki-link";
 import matter from "gray-matter";
 import { glob } from "glob";
-import { readFile } from "fs/promises";
+import { readFile, stat } from "fs/promises";
 import { basename, join } from "path";
 var CHUNK_TARGET_CHARS = 2e3;
 var Indexer = class {
@@ -28,7 +28,10 @@ var Indexer = class {
     return docs;
   }
   async indexFile(absolutePath, relativePath) {
-    const raw = await readFile(absolutePath, "utf-8");
+    const [raw, fileStat] = await Promise.all([
+      readFile(absolutePath, "utf-8"),
+      stat(absolutePath)
+    ]);
     const { data: frontmatter, content } = matter(raw);
     const tree = this.processor.parse(content);
     const wikilinks = this.extractWikilinks(tree);
@@ -37,6 +40,12 @@ var Indexer = class {
     const plainText = this.stripMarkdown(content);
     const chunks = this.chunkText(plainText);
     const title = frontmatter.title || headers[0] || basename(relativePath, ".md");
+    const mtime = this.resolveMtime(frontmatter, fileStat.mtime);
+    const loadPriority = typeof frontmatter.load_priority === "number" ? Math.min(10, Math.max(1, frontmatter.load_priority)) : void 0;
+    const status = typeof frontmatter.status === "string" ? frontmatter.status : void 0;
+    const tier = typeof frontmatter.tier === "string" ? frontmatter.tier : void 0;
+    const domains = Array.isArray(frontmatter.domains) ? frontmatter.domains : void 0;
+    const purpose = typeof frontmatter.purpose === "string" ? frontmatter.purpose : void 0;
     return {
       path: relativePath,
       title,
@@ -45,9 +54,35 @@ var Indexer = class {
       wikilinks,
       tags,
       headers,
-      chunks
+      chunks,
+      mtime,
+      ...loadPriority !== void 0 && { loadPriority },
+      ...status !== void 0 && { status },
+      ...tier !== void 0 && { tier },
+      ...domains !== void 0 && { domains },
+      ...purpose !== void 0 && { purpose }
     };
   }
+  /**
+   * Resolve the best available modification date for a document.
+   * Priority: last_updated → updated → date → lastmod → fs.stat mtime
+   * Accepts YYYY-MM-DD strings or full ISO timestamps.
+   */
+  resolveMtime(frontmatter, statMtime) {
+    const candidates = [
+      frontmatter.last_updated,
+      frontmatter.updated,
+      frontmatter.date,
+      frontmatter.lastmod
+    ];
+    for (const val of candidates) {
+      if (!val) continue;
+      const str = val instanceof Date ? val.toISOString() : String(val);
+      const parsed = new Date(str);
+      if (!isNaN(parsed.getTime())) return parsed.toISOString();
+    }
+    return statMtime.toISOString();
+  }
   extractWikilinks(tree) {
     const links = [];
     const walk = (node) => {
@@ -111,4 +146,4 @@ export {
   __export,
   Indexer
 };
-//# sourceMappingURL=chunk-TDC45FQJ.js.map
+//# sourceMappingURL=chunk-VAPQ4NA3.js.map

package/dist/chunk-VAPQ4NA3.js.map

@@ -0,0 +1 @@
+{"version":3,"sources":["../src/core/indexer.ts"],"sourcesContent":["import { unified } from \"unified\";\nimport remarkParse from \"remark-parse\";\nimport remarkWikiLink from \"remark-wiki-link\";\nimport matter from \"gray-matter\";\nimport { glob } from \"glob\";\nimport { readFile, stat } from \"node:fs/promises\";\nimport { basename, join, relative } from \"node:path\";\nimport type { IndexedDocument } from \"./types.js\";\n\nconst CHUNK_TARGET_CHARS = 2000; // ~512 tokens\n\nexport class Indexer {\n  private notesPath: string;\n  private processor: any;\n\n  constructor(notesPath: string) {\n    this.notesPath = notesPath;\n    this.processor = unified().use(remarkParse).use(remarkWikiLink);\n  }\n\n  async indexAll(): Promise<IndexedDocument[]> {\n    const files = await glob(\"**/*.md\", { cwd: this.notesPath });\n    const docs = await Promise.all(\n      files.map((file) => this.indexFile(join(this.notesPath, file), file))\n    );\n    return docs;\n  }\n\n  async indexFile(\n    absolutePath: string,\n    relativePath: string\n  ): Promise<IndexedDocument> {\n    const [raw, fileStat] = await Promise.all([\n      readFile(absolutePath, \"utf-8\"),\n      stat(absolutePath),\n    ]);\n    const { data: frontmatter, content } = matter(raw);\n    const tree = this.processor.parse(content);\n\n    const wikilinks = this.extractWikilinks(tree);\n    const tags = this.extractTags(content, frontmatter);\n    const headers = this.extractHeaders(tree);\n    const plainText = this.stripMarkdown(content);\n    const chunks = this.chunkText(plainText);\n\n    const title =\n      (frontmatter.title as string) ||\n      headers[0] ||\n      basename(relativePath, \".md\");\n\n    // Resolve modification time: prefer frontmatter date fields over fs.stat\n    // Supports hit-em-with-the-docs (last_updated) and common alternatives\n    const mtime = this.resolveMtime(frontmatter, fileStat.mtime);\n\n    // Optional hit-em-with-the-docs fields (only populated when present)\n    const loadPriority =\n      typeof frontmatter.load_priority === \"number\"\n        ? Math.min(10, Math.max(1, frontmatter.load_priority))\n        : undefined;\n    const status =\n      typeof frontmatter.status === \"string\" ? frontmatter.status : undefined;\n    const tier =\n      typeof frontmatter.tier === \"string\" ? frontmatter.tier : undefined;\n    const domains = Array.isArray(frontmatter.domains)\n      ? (frontmatter.domains as string[])\n      : undefined;\n    const purpose =\n      typeof frontmatter.purpose === \"string\" ? frontmatter.purpose : undefined;\n\n    return {\n      path: relativePath,\n      title,\n      content: plainText,\n      frontmatter,\n      wikilinks,\n      tags,\n      headers,\n      chunks,\n      mtime,\n      ...(loadPriority !== undefined && { loadPriority }),\n      ...(status !== undefined && { status }),\n      ...(tier !== undefined && { tier }),\n      ...(domains !== undefined && { domains }),\n      ...(purpose !== undefined && { purpose }),\n    };\n  }\n\n  /**\n   * Resolve the best available modification date for a document.\n   * Priority: last_updated → updated → date → lastmod → fs.stat mtime\n   * Accepts YYYY-MM-DD strings or full ISO timestamps.\n   */\n  private resolveMtime(\n    frontmatter: Record<string, unknown>,\n    statMtime: Date\n  ): string {\n    const candidates = [\n      frontmatter.last_updated,\n      frontmatter.updated,\n      frontmatter.date,\n      frontmatter.lastmod,\n    ];\n    for (const val of candidates) {\n      if (!val) continue;\n      const str = val instanceof Date ? val.toISOString() : String(val);\n      const parsed = new Date(str);\n      if (!isNaN(parsed.getTime())) return parsed.toISOString();\n    }\n    return statMtime.toISOString();\n  }\n\n  private extractWikilinks(tree: any): string[] {\n    const links: string[] = [];\n    const walk = (node: any) => {\n      if (node.type === \"wikiLink\") {\n        links.push(node.value || node.data?.alias || \"\");\n      }\n      if (node.children) {\n        for (const child of node.children) walk(child);\n      }\n    };\n    walk(tree);\n    return [...new Set(links.filter(Boolean))];\n  }\n\n  private extractTags(content: string, frontmatter: Record<string, unknown>): string[] {\n    const inlineTags = [...content.matchAll(/(?:^|\\s)#([a-zA-Z][\\w-/]*)/g)].map(\n      (m) => m[1]\n    );\n\n    const fmTags = Array.isArray(frontmatter.tags)\n      ? (frontmatter.tags as string[])\n      : [];\n\n    return [...new Set([...fmTags, ...inlineTags])];\n  }\n\n  private extractHeaders(tree: any): string[] {\n    const headers: string[] = [];\n    const walk = (node: any) => {\n      if (node.type === \"heading\") {\n        const text = this.nodeToText(node);\n        if (text) headers.push(text);\n      }\n      if (node.children) {\n        for (const child of node.children) walk(child);\n      }\n    };\n    walk(tree);\n    return headers;\n  }\n\n  private nodeToText(node: any): string {\n    if (node.type === \"text\") return node.value;\n    if (node.children) return node.children.map((c: any) => this.nodeToText(c)).join(\"\");\n    return \"\";\n  }\n\n  private stripMarkdown(content: string): string {\n    return content\n      .replace(/```[\\s\\S]*?```/g, \"\")\n      .replace(/`[^`]+`/g, \"\")\n      .replace(/!\\[.*?\\]\\(.*?\\)/g, \"\")\n      .replace(/\\[([^\\]]+)\\]\\(.*?\\)/g, \"$1\")\n      .replace(/#{1,6}\\s+/g, \"\")\n      .replace(/[*_~]{1,3}/g, \"\")\n      .replace(/>\\s+/g, \"\")\n      .replace(/\\|.*\\|/g, \"\")\n      .replace(/-{3,}/g, \"\")\n      .replace(/\\n{3,}/g, \"\\n\\n\")\n      .trim();\n  }\n\n  chunkText(text: string): string[] {\n    if (text.length <= CHUNK_TARGET_CHARS) return [text];\n\n    const sentences = text.match(/[^.!?\\n]+[.!?\\n]+|[^.!?\\n]+$/g) || [text];\n    const chunks: string[] = [];\n    let current = \"\";\n\n    for (const sentence of sentences) {\n      if (current.length + sentence.length > CHUNK_TARGET_CHARS && current) {\n        chunks.push(current.trim());\n        current = \"\";\n      }\n      current += sentence;\n    }\n    if (current.trim()) chunks.push(current.trim());\n\n    return chunks;\n  }\n}\n"],"mappings":";;;;;;;AAAA,SAAS,eAAe;AACxB,OAAO,iBAAiB;AACxB,OAAO,oBAAoB;AAC3B,OAAO,YAAY;AACnB,SAAS,YAAY;AACrB,SAAS,UAAU,YAAY;AAC/B,SAAS,UAAU,YAAsB;AAGzC,IAAM,qBAAqB;AAEpB,IAAM,UAAN,MAAc;AAAA,EACX;AAAA,EACA;AAAA,EAER,YAAY,WAAmB;AAC7B,SAAK,YAAY;AACjB,SAAK,YAAY,QAAQ,EAAE,IAAI,WAAW,EAAE,IAAI,cAAc;AAAA,EAChE;AAAA,EAEA,MAAM,WAAuC;AAC3C,UAAM,QAAQ,MAAM,KAAK,WAAW,EAAE,KAAK,KAAK,UAAU,CAAC;AAC3D,UAAM,OAAO,MAAM,QAAQ;AAAA,MACzB,MAAM,IAAI,CAAC,SAAS,KAAK,UAAU,KAAK,KAAK,WAAW,IAAI,GAAG,IAAI,CAAC;AAAA,IACtE;AACA,WAAO;AAAA,EACT;AAAA,EAEA,MAAM,UACJ,cACA,cAC0B;AAC1B,UAAM,CAAC,KAAK,QAAQ,IAAI,MAAM,QAAQ,IAAI;AAAA,MACxC,SAAS,cAAc,OAAO;AAAA,MAC9B,KAAK,YAAY;AAAA,IACnB,CAAC;AACD,UAAM,EAAE,MAAM,aAAa,QAAQ,IAAI,OAAO,GAAG;AACjD,UAAM,OAAO,KAAK,UAAU,MAAM,OAAO;AAEzC,UAAM,YAAY,KAAK,iBAAiB,IAAI;AAC5C,UAAM,OAAO,KAAK,YAAY,SAAS,WAAW;AAClD,UAAM,UAAU,KAAK,eAAe,IAAI;AACxC,UAAM,YAAY,KAAK,cAAc,OAAO;AAC5C,UAAM,SAAS,KAAK,UAAU,SAAS;AAEvC,UAAM,QACH,YAAY,SACb,QAAQ,CAAC,KACT,SAAS,cAAc,KAAK;AAI9B,UAAM,QAAQ,KAAK,aAAa,aAAa,SAAS,KAAK;AAG3D,UAAM,eACJ,OAAO,YAAY,kBAAkB,WACjC,KAAK,IAAI,IAAI,KAAK,IAAI,GAAG,YAAY,aAAa,CAAC,IACnD;AACN,UAAM,SACJ,OAAO,YAAY,WAAW,WAAW,YAAY,SAAS;AAChE,UAAM,OACJ,OAAO,YAAY,SAAS,WAAW,YAAY,OAAO;AAC5D,UAAM,UAAU,MAAM,QAAQ,YAAY,OAAO,IAC5C,YAAY,UACb;AACJ,UAAM,UACJ,OAAO,YAAY,YAAY,WAAW,YAAY,UAAU;AAElE,WAAO;AAAA,MACL,MAAM;AAAA,MACN;AAAA,MACA,SAAS;AAAA,MACT;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA;AAAA,MACA,GAAI,iBAAiB,UAAa,EAAE,aAAa;AAAA,MACjD,GAAI,WAAW,UAAa,EAAE,OAAO;AAAA,MACrC,GAAI,SAAS,UAAa,EAAE,KAAK;AAAA,MACjC,GAAI,YAAY,UAAa,EAAE,QAAQ;AAAA,MACvC,GAAI,YAAY,UAAa,EAAE,QAAQ;AAAA,IACzC;AAAA,EACF;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA,EAOQ,aACN,aACA,WACQ;AACR,UAAM,aAAa;AAAA,MACjB,YAAY;AAAA,MACZ,YAAY;AAAA,MACZ,YAAY;AAAA,MACZ,YAAY;AAAA,IACd;AACA,eAAW,OAAO,YAAY;AAC5B,UAAI,CAAC,IAAK;AACV,YAAM,MAAM,eAAe,OAAO,IAAI,YAAY,IAAI,OAAO,GAAG;AAChE,YAAM,SAAS,IAAI,KAAK,GAAG;AAC3B,UAAI,CAAC,MAAM,OAAO,QAAQ,CAAC,EAAG,QAAO,OAAO,YAAY;AAAA,IAC1D;AACA,WAAO,UAAU,YAAY;AAAA,EAC/B;AAAA,EAEQ,iBAAiB,MAAqB;AAC5C,UAAM,QAAkB,CAAC;AACzB,UAAM,OAAO,CAAC,SAAc;AAC1B,UAAI,KAAK,SAAS,YAAY;AAC5B,cAAM,KAAK,KAAK,SAAS,KAAK,MAAM,SAAS,EAAE;AAAA,MACjD;AACA,UAAI,KAAK,UAAU;AACjB,mBAAW,SAAS,KAAK,SAAU,MAAK,KAAK;AAAA,MAC/C;AAAA,IACF;AACA,SAAK,IAAI;AACT,WAAO,CAAC,GAAG,IAAI,IAAI,MAAM,OAAO,OAAO,CAAC,CAAC;AAAA,EAC3C;AAAA,EAEQ,YAAY,SAAiB,aAAgD;AACnF,UAAM,aAAa,CAAC,GAAG,QAAQ,SAAS,6BAA6B,CAAC,EAAE;AAAA,MACtE,CAAC,MAAM,EAAE,CAAC;AAAA,IACZ;AAEA,UAAM,SAAS,MAAM,QAAQ,YAAY,IAAI,IACxC,YAAY,OACb,CAAC;AAEL,WAAO,CAAC,GAAG,oBAAI,IAAI,CAAC,GAAG,QAAQ,GAAG,UAAU,CAAC,CAAC;AAAA,EAChD;AAAA,EAEQ,eAAe,MAAqB;AAC1C,UAAM,UAAoB,CAAC;AAC3B,UAAM,OAAO,CAAC,SAAc;AAC1B,UAAI,KAAK,SAAS,WAAW;AAC3B,cAAM,OAAO,KAAK,WAAW,IAAI;AACjC,YAAI,KAAM,SAAQ,KAAK,IAAI;AAAA,MAC7B;AACA,UAAI,KAAK,UAAU;AACjB,mBAAW,SAAS,KAAK,SAAU,MAAK,KAAK;AAAA,MAC/C;AAAA,IACF;AACA,SAAK,IAAI;AACT,WAAO;AAAA,EACT;AAAA,EAEQ,WAAW,MAAmB;AACpC,QAAI,KAAK,SAAS,OAAQ,QAAO,KAAK;AACtC,QAAI,KAAK,SAAU,QAAO,KAAK,SAAS,IAAI,CAAC,MAAW,KAAK,WAAW,CAAC,CAAC,EAAE,KAAK,EAAE;AACnF,WAAO;AAAA,EACT;AAAA,EAEQ,cAAc,SAAyB;AAC7C,WAAO,QACJ,QAAQ,mBAAmB,EAAE,EAC7B,QAAQ,YAAY,EAAE,EACtB,QAAQ,oBAAoB,EAAE,EAC9B,QAAQ,wBAAwB,IAAI,EACpC,QAAQ,cAAc,EAAE,EACxB,QAAQ,eAAe,EAAE,EACzB,QAAQ,SAAS,EAAE,EACnB,QAAQ,WAAW,EAAE,EACrB,QAAQ,UAAU,EAAE,EACpB,QAAQ,WAAW,MAAM,EACzB,KAAK;AAAA,EACV;AAAA,EAEA,UAAU,MAAwB;AAChC,QAAI,KAAK,UAAU,mBAAoB,QAAO,CAAC,IAAI;AAEnD,UAAM,YAAY,KAAK,MAAM,+BAA+B,KAAK,CAAC,IAAI;AACtE,UAAM,SAAmB,CAAC;AAC1B,QAAI,UAAU;AAEd,eAAW,YAAY,WAAW;AAChC,UAAI,QAAQ,SAAS,SAAS,SAAS,sBAAsB,SAAS;AACpE,eAAO,KAAK,QAAQ,KAAK,CAAC;AAC1B,kBAAU;AAAA,MACZ;AACA,iBAAW;AAAA,IACb;AACA,QAAI,QAAQ,KAAK,EAAG,QAAO,KAAK,QAAQ,KAAK,CAAC;AAE9C,WAAO;AAAA,EACT;AACF;","names":[]}

package/dist/cli/index.js

@@ -215,7 +215,7 @@ program.command("serve", { isDefault: true }).description("Start the MCP server
     process.exit(1);
   }
   if (opts.stats) {
-    const { Indexer } = await import("../indexer-HSCSXWIO.js");
+    const { Indexer } = await import("../indexer-55PTBSTU.js");
     const indexer = new Indexer(notesPath);
     const docs = await indexer.indexAll();
     console.log(`Notes: ${docs.length}`);

package/dist/core/index.d.ts

@@ -10,6 +10,13 @@ interface IndexedDocument {
     tags: string[];
     headers: string[];
     chunks: string[];
+    /** ISO timestamp — prefers frontmatter last_updated/updated/date/lastmod, falls back to fs.stat mtime */
+    mtime: string;
+    loadPriority?: number;
+    status?: string;
+    tier?: string;
+    domains?: string[];
+    purpose?: string;
 }
 interface SearchResult {
     path: string;
@@ -17,6 +24,11 @@ interface SearchResult {
     score: number;
     snippet: string;
     matchedChunk?: string;
+    mtime?: string;
+    loadPriority?: number;
+    status?: string;
+    tier?: string;
+    domains?: string[];
 }
 interface GraphNode {
     path: string;
@@ -78,6 +90,12 @@ declare class Indexer {
     constructor(notesPath: string);
     indexAll(): Promise<IndexedDocument[]>;
     indexFile(absolutePath: string, relativePath: string): Promise<IndexedDocument>;
+    /**
+     * Resolve the best available modification date for a document.
+     * Priority: last_updated → updated → date → lastmod → fs.stat mtime
+     * Accepts YYYY-MM-DD strings or full ISO timestamps.
+     */
+    private resolveMtime;
     private extractWikilinks;
     private extractTags;
     private extractHeaders;

package/dist/core/index.js

@@ -10,7 +10,7 @@ import {
 } from "../chunk-6ZT5TGKT.js";
 import {
   Indexer
-} from "../chunk-TDC45FQJ.js";
+} from "../chunk-VAPQ4NA3.js";
 export {
   Embedder,
   FrontmatterManager,

package/dist/indexer-55PTBSTU.js

@@ -0,0 +1,7 @@
+import {
+  Indexer
+} from "./chunk-VAPQ4NA3.js";
+export {
+  Indexer
+};
+//# sourceMappingURL=indexer-55PTBSTU.js.map

package/dist/mcp/server.js

@@ -11,7 +11,7 @@ import {
 import {
   Indexer,
   __export
-} from "../chunk-TDC45FQJ.js";
+} from "../chunk-VAPQ4NA3.js";
 
 // src/mcp/server.ts
 import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
@@ -4073,6 +4073,7 @@ async function createServer(notesPath, options = {}) {
   const frontmatterManager = new FrontmatterManager(notesPath);
   const tagManager = new TagManager(notesPath);
   let documents = [];
+  let docByPath = /* @__PURE__ */ new Map();
   let vectorIndex = null;
   let indexState = "empty";
   let indexProgress = { embedded: 0, total: 0 };
@@ -4098,6 +4099,7 @@ async function createServer(notesPath, options = {}) {
       const graphLoaded = await graph.load(indexPath);
       if (!graphLoaded) return false;
       documents = await indexer.indexAll();
+      docByPath = new Map(documents.map((d) => [d.path, d]));
       textSearch.setDocuments(documents);
       vectorIndex = tempVector;
       indexState = "stale";
@@ -4165,6 +4167,7 @@ async function createServer(notesPath, options = {}) {
       allChunks.map((c) => ({ docPath: c.docPath, chunkIndex: c.chunkIndex, text: c.text }))
     );
     documents = newDocs;
+    docByPath = new Map(documents.map((d) => [d.path, d]));
     textSearch.setDocuments(newDocs);
     graph.buildFromDocuments(newDocs);
     vectorIndex = newVector;
@@ -4200,19 +4203,62 @@ async function createServer(notesPath, options = {}) {
     }
     return "Indexing in progress... Try again shortly.";
   }
+  function enrichResult(result) {
+    const doc = docByPath.get(result.path);
+    if (!doc) return result;
+    return {
+      ...result,
+      mtime: doc.mtime,
+      ...doc.loadPriority !== void 0 && { loadPriority: doc.loadPriority },
+      ...doc.status !== void 0 && { status: doc.status },
+      ...doc.tier !== void 0 && { tier: doc.tier },
+      ...doc.domains !== void 0 && { domains: doc.domains }
+    };
+  }
+  function applyPriorityBoost(score, path) {
+    const doc = docByPath.get(path);
+    if (doc?.loadPriority === void 0) return score;
+    return score * (1 + (doc.loadPriority - 5) * 0.04);
+  }
+  function applyDateFilter(results, modifiedAfter, modifiedBefore) {
+    if (!modifiedAfter && !modifiedBefore) return results;
+    const after = modifiedAfter ? new Date(modifiedAfter).getTime() : -Infinity;
+    const before = modifiedBefore ? new Date(modifiedBefore).getTime() : Infinity;
+    return results.filter((r) => {
+      const doc = docByPath.get(r.path);
+      if (!doc) return true;
+      const t = new Date(doc.mtime).getTime();
+      return t >= after && t <= before;
+    });
+  }
   const server = new McpServer({
     name: "semantic-pages",
     version: "0.2.0"
   });
   server.tool(
     "search_semantic",
-    "Vector similarity search \u2014 find notes similar to a query by meaning",
-    { query: external_exports.string(), limit: external_exports.number().optional().default(10) },
-    async ({ query, limit }) => {
+    "Vector similarity search \u2014 find notes similar to a query by meaning. Scores are boosted by load_priority when present.",
+    {
+      query: external_exports.string(),
+      limit: external_exports.number().optional().default(10),
+      modifiedAfter: external_exports.string().optional().describe("ISO date \u2014 only return notes modified after this date (e.g. '2026-01-01')"),
+      modifiedBefore: external_exports.string().optional().describe("ISO date \u2014 only return notes modified before this date"),
+      status: external_exports.string().optional().describe("Filter by frontmatter status (e.g. 'active', 'draft')"),
+      tier: external_exports.string().optional().describe("Filter by frontmatter tier (e.g. 'guide', 'reference')"),
+      domain: external_exports.string().optional().describe("Filter by frontmatter domain (e.g. 'api', 'security')")
+    },
+    async ({ query, limit, modifiedAfter, modifiedBefore, status, tier, domain }) => {
       if (!vectorIndex) return textResponse(indexState === "empty" ? indexingMessage() : "Index not built. Run reindex first.");
       const queryEmbed = await embedder.embed(query);
-      const results = vectorIndex.search(queryEmbed, limit);
-      return textResponse(JSON.stringify(results, null, 2));
+      let results = vectorIndex.search(queryEmbed, limit * 3);
+      results = results.map((r) => ({ ...r, score: applyPriorityBoost(r.score, r.path) }));
+      results.sort((a, b) => b.score - a.score);
+      results = applyDateFilter(results, modifiedAfter, modifiedBefore);
+      if (status) results = results.filter((r) => docByPath.get(r.path)?.status === status);
+      if (tier) results = results.filter((r) => docByPath.get(r.path)?.tier === tier);
+      if (domain) results = results.filter((r) => docByPath.get(r.path)?.domains?.includes(domain));
+      const enriched = results.slice(0, limit).map(enrichResult);
+      return textResponse(JSON.stringify(enriched, null, 2));
     }
   );
   server.tool(
@@ -4224,12 +4270,22 @@ async function createServer(notesPath, options = {}) {
       caseSensitive: external_exports.boolean().optional().default(false),
       pathGlob: external_exports.string().optional(),
       tagFilter: external_exports.array(external_exports.string()).optional(),
-      limit: external_exports.number().optional().default(20)
+      limit: external_exports.number().optional().default(20),
+      modifiedAfter: external_exports.string().optional().describe("ISO date \u2014 only return notes modified after this date"),
+      modifiedBefore: external_exports.string().optional().describe("ISO date \u2014 only return notes modified before this date"),
+      status: external_exports.string().optional().describe("Filter by frontmatter status"),
+      tier: external_exports.string().optional().describe("Filter by frontmatter tier"),
+      domain: external_exports.string().optional().describe("Filter by frontmatter domain")
     },
-    async (opts) => {
+    async ({ modifiedAfter, modifiedBefore, status, tier, domain, ...opts }) => {
       if (documents.length === 0 && indexState !== "ready") return textResponse(indexingMessage());
-      const results = textSearch.search(opts);
-      return textResponse(JSON.stringify(results, null, 2));
+      let results = textSearch.search(opts);
+      results = applyDateFilter(results, modifiedAfter, modifiedBefore);
+      if (status) results = results.filter((r) => docByPath.get(r.path)?.status === status);
+      if (tier) results = results.filter((r) => docByPath.get(r.path)?.tier === tier);
+      if (domain) results = results.filter((r) => docByPath.get(r.path)?.domains?.includes(domain));
+      const enriched = results.map(enrichResult);
+      return textResponse(JSON.stringify(enriched, null, 2));
     }
   );
   server.tool(
@@ -4244,20 +4300,36 @@ async function createServer(notesPath, options = {}) {
   );
   server.tool(
     "search_hybrid",
-    "Combined semantic + graph search \u2014 vector results re-ranked by graph proximity",
-    { query: external_exports.string(), limit: external_exports.number().optional().default(10) },
-    async ({ query, limit }) => {
+    "Combined semantic + graph search \u2014 vector results re-ranked by graph proximity and load_priority",
+    {
+      query: external_exports.string(),
+      limit: external_exports.number().optional().default(10),
+      modifiedAfter: external_exports.string().optional().describe("ISO date \u2014 only return notes modified after this date"),
+      modifiedBefore: external_exports.string().optional().describe("ISO date \u2014 only return notes modified before this date"),
+      status: external_exports.string().optional().describe("Filter by frontmatter status"),
+      tier: external_exports.string().optional().describe("Filter by frontmatter tier"),
+      domain: external_exports.string().optional().describe("Filter by frontmatter domain")
+    },
+    async ({ query, limit, modifiedAfter, modifiedBefore, status, tier, domain }) => {
       if (!vectorIndex) return textResponse(indexState === "empty" ? indexingMessage() : "Index not built. Run reindex first.");
       const queryEmbed = await embedder.embed(query);
-      const semanticResults = vectorIndex.search(queryEmbed, limit * 2);
+      const semanticResults = vectorIndex.search(queryEmbed, limit * 3);
       const graphResults = graph.searchGraph(query, 2);
       const graphPaths = new Set(graphResults.map((r) => r.path));
-      const hybrid = semanticResults.map((r) => ({
+      let hybrid = semanticResults.map((r) => ({
         ...r,
-        score: graphPaths.has(r.path) ? r.score * 1.3 : r.score
+        score: applyPriorityBoost(
+          graphPaths.has(r.path) ? r.score * 1.3 : r.score,
+          r.path
+        )
       }));
       hybrid.sort((a, b) => b.score - a.score);
-      return textResponse(JSON.stringify(hybrid.slice(0, limit), null, 2));
+      hybrid = applyDateFilter(hybrid, modifiedAfter, modifiedBefore);
+      if (status) hybrid = hybrid.filter((r) => docByPath.get(r.path)?.status === status);
+      if (tier) hybrid = hybrid.filter((r) => docByPath.get(r.path)?.tier === tier);
+      if (domain) hybrid = hybrid.filter((r) => docByPath.get(r.path)?.domains?.includes(domain));
+      const enriched = hybrid.slice(0, limit).map(enrichResult);
+      return textResponse(JSON.stringify(enriched, null, 2));
     }
   );
   server.tool(
@@ -4282,16 +4354,37 @@ async function createServer(notesPath, options = {}) {
   );
   server.tool(
     "list_notes",
-    "List all indexed notes with metadata (title, tags, link count)",
-    {},
-    async () => {
+    "List all indexed notes with metadata (title, tags, timestamps, link count). Supports filtering by date, status, tier, and domain.",
+    {
+      modifiedAfter: external_exports.string().optional().describe("ISO date \u2014 only return notes modified after this date (e.g. '2026-01-01')"),
+      modifiedBefore: external_exports.string().optional().describe("ISO date \u2014 only return notes modified before this date"),
+      status: external_exports.string().optional().describe("Filter by frontmatter status (e.g. 'active', 'deprecated')"),
+      tier: external_exports.string().optional().describe("Filter by frontmatter tier (e.g. 'guide', 'reference')"),
+      domain: external_exports.string().optional().describe("Filter by frontmatter domain (e.g. 'api', 'security')")
+    },
+    async ({ modifiedAfter, modifiedBefore, status, tier, domain }) => {
       if (documents.length === 0 && indexState !== "ready") return textResponse(indexingMessage());
-      const list = documents.map((d) => ({
+      const after = modifiedAfter ? new Date(modifiedAfter).getTime() : -Infinity;
+      const before = modifiedBefore ? new Date(modifiedBefore).getTime() : Infinity;
+      let list = documents.filter((d) => {
+        const t = new Date(d.mtime).getTime();
+        if (t < after || t > before) return false;
+        if (status && d.status !== status) return false;
+        if (tier && d.tier !== tier) return false;
+        if (domain && !d.domains?.includes(domain)) return false;
+        return true;
+      }).map((d) => ({
         path: d.path,
         title: d.title,
+        mtime: d.mtime,
         tags: d.tags,
         wikilinks: d.wikilinks.length,
-        chunks: d.chunks.length
+        chunks: d.chunks.length,
+        ...d.loadPriority !== void 0 && { loadPriority: d.loadPriority },
+        ...d.status !== void 0 && { status: d.status },
+        ...d.tier !== void 0 && { tier: d.tier },
+        ...d.domains !== void 0 && { domains: d.domains },
+        ...d.purpose !== void 0 && { purpose: d.purpose }
       }));
       return textResponse(JSON.stringify(list, null, 2));
     }
@@ -4472,11 +4565,17 @@ async function createServer(notesPath, options = {}) {
       );
     }
   );
-  const cached = await tryLoadCachedIndex();
   if (options.waitForReady) {
+    await tryLoadCachedIndex();
     await fullIndex();
-  } else if (!cached) {
-    backgroundIndex();
+  } else {
+    tryLoadCachedIndex().then((cached) => {
+      if (!cached) backgroundIndex();
+    }).catch((err) => {
+      process.stderr.write(`Startup error: ${err?.message ?? err}
+`);
+      backgroundIndex();
+    });
   }
   if (options.watch !== false) {
     const watcher = new Watcher(notesPath);