@disco_trooper/apple-notes-mcp 1.2.0 → 1.3.0

package/README.md

@@ -1,21 +1,30 @@
 # apple-notes-mcp
 
+[![npm version](https://img.shields.io/npm/v/@disco_trooper/apple-notes-mcp)](https://www.npmjs.com/package/@disco_trooper/apple-notes-mcp)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![macOS](https://img.shields.io/badge/macOS-000000?logo=apple&logoColor=white)](https://www.apple.com/macos/)
+[![Bun](https://img.shields.io/badge/Bun-000000?logo=bun&logoColor=white)](https://bun.sh)
+[![Claude](https://img.shields.io/badge/Claude-MCP-blueviolet)](https://modelcontextprotocol.io)
+
 MCP server for Apple Notes with semantic search and CRUD operations. Claude searches, reads, creates, updates, and manages your Apple Notes through natural language.
 
 ## Features
 
+- **Chunk-Based Search** - Long notes split into chunks for accurate matching (NEW!)
+- **Query Caching** - 60x faster repeated searches (NEW!)
+- **Knowledge Graph** - Tags, links, and related notes discovery (NEW!)
+- **Hybrid Search** - Vector + keyword search with Reciprocal Rank Fusion
 - **Semantic Search** - Find notes by meaning, not keywords
-- **Hybrid Search** - Combine vector and keyword search for better results
 - **Full CRUD** - Create, read, update, delete, and move notes
 - **Incremental Indexing** - Re-embed only changed notes
-- **Dual Embedding Support** - Local HuggingFace or OpenRouter API
-- **Claude Code Integration** - Works with Claude Code CLI
+- **Dual Embedding** - Local HuggingFace or OpenRouter API
 
-## Requirements
+## What's New in 1.3
 
-- macOS (uses Apple Notes via JXA)
-- [Bun](https://bun.sh) runtime
-- Apple Notes app with notes
+- **Parent Document Retriever** - Splits long notes into 500-char chunks with 100-char overlap. Searches match specific sections, returns full notes.
+- **60x faster cached queries** - Query embedding cache eliminates redundant API calls.
+- **Auto-filters Base64/encoded content** - Skips images and attachments during indexing.
+- **4-6x faster indexing** - Parallel processing and optimized chunk generation.
 
 ## Installation
 
@@ -23,34 +32,43 @@ MCP server for Apple Notes with semantic search and CRUD operations. Claude sear
 
 ```bash
 npm install -g @disco_trooper/apple-notes-mcp
+apple-notes-mcp
 ```
 
+The setup wizard guides you through:
+1. Choosing your embedding provider (local or OpenRouter)
+2. Configuring API keys if needed
+3. Setting up Claude Code integration
+4. Indexing your notes
+
 ### From source
 
 ```bash
 git clone https://github.com/disco-trooper/apple-notes-mcp.git
 cd apple-notes-mcp
 bun install
+bun run start
 ```
 
+## Requirements
+
+- macOS (uses Apple Notes via JXA)
+- [Bun](https://bun.sh) runtime
+- Apple Notes app with notes
+
 ## Quick Start
 
-Run the setup wizard:
+Run the command after installation:
 
 ```bash
-bun run setup
+apple-notes-mcp
 ```
 
-The wizard:
-1. Chooses your embedding provider (local or OpenRouter)
-2. Configures API keys if needed
-3. Sets auto-indexing preferences
-4. Adds to Claude Code configuration
-5. Indexes your notes
+The setup wizard starts automatically on first run. Restart Claude Code after setup to use the MCP tools.
 
 ## Configuration
 
-Store configuration in `.env`:
+Configuration stored in `~/.apple-notes-mcp/.env`:
 
 | Variable | Description | Default |
 |----------|-------------|---------|
@@ -61,6 +79,14 @@ Store configuration in `.env`:
 | `INDEX_TTL` | Auto-reindex interval in seconds | - |
 | `DEBUG` | Enable debug logging | `false` |
 
+To reconfigure:
+
+```bash
+apple-notes-mcp setup
+# or from source:
+bun run setup
+```
+
 ### Embedding Providers
 
 **Local (default)**: Uses HuggingFace Transformers with `Xenova/multilingual-e5-small`. Free, runs locally, ~200MB download.
@@ -74,14 +100,14 @@ See [docs/models.md](docs/models.md) for model comparison.
 ### Search & Discovery
 
 #### `search-notes`
-Search notes with hybrid vector + fulltext search.
+Hybrid vector + fulltext search.
 
 ```
 query: "meeting notes from last week"
 folder: "Work"           # optional, filter by folder
-limit: 10                 # default: 20
-mode: "hybrid"            # hybrid, keyword, or semantic
-include_content: false    # include full content vs preview
+limit: 10                # default: 20
+mode: "hybrid"           # hybrid, keyword, or semantic
+include_content: false   # include full content vs preview
 ```
 
 #### `list-notes`
@@ -91,7 +117,7 @@ Count indexed notes.
 List all Apple Notes folders.
 
 #### `get-note`
-Get a note's full content by title.
+Get note content by title.
 
 ```
 title: "My Note"          # or "Work/My Note" for disambiguation
@@ -100,13 +126,15 @@ title: "My Note"          # or "Work/My Note" for disambiguation
 ### Indexing
 
 #### `index-notes`
-Index all notes for semantic search.
+Index notes for semantic search.
 
 ```
 mode: "incremental"       # incremental (default) or full
 force: false              # force reindex even if TTL hasn't expired
 ```
 
+Use `mode: "full"` to create the chunk index for better long-note search. First full index takes longer as it generates chunks, but subsequent searches run fast.
+
 #### `reindex-note`
 Re-index a single note after manual edits.
 
@@ -150,16 +178,65 @@ title: "My Note"
 folder: "Archive"
 ```
 
+### Knowledge Graph
+
+#### `list-tags`
+List all tags with occurrence counts.
+
+#### `search-by-tag`
+Find notes with a specific tag.
+
+```
+tag: "project"
+folder: "Work"    # optional
+limit: 20         # default: 20
+```
+
+#### `related-notes`
+Find notes related to a source note.
+
+```
+title: "My Note"
+types: ["tag", "link", "similar"]  # default: all
+limit: 10                          # default: 10
+```
+
+#### `export-graph`
+Export knowledge graph for visualization.
+
+```
+format: "json"     # json or graphml
+folder: "Work"     # optional filter
+```
+
+**Supported Formats:**
+- `json` - For custom visualization (D3.js, web apps)
+- `graphml` - For professional tools (Gephi, yEd, Cytoscape)
+
 ## Claude Code Setup
 
-### Automatic (via setup wizard)
+### Automatic (recommended)
 
-Run `bun run setup` and select "Add to Claude Code configuration".
+The setup wizard automatically adds apple-notes-mcp to Claude Code. Run `apple-notes-mcp` after installation.
 
 ### Manual
 
 Add to `~/.claude.json`:
 
+For npm installation:
+```json
+{
+  "mcpServers": {
+    "apple-notes": {
+      "command": "apple-notes-mcp",
+      "args": [],
+      "env": {}
+    }
+  }
+}
+```
+
+For source installation:
 ```json
 {
   "mcpServers": {
@@ -205,6 +282,9 @@ Ensure Apple Notes runs and contains notes. Grant automation permissions when pr
 # Type check
 bun run check
 
+# Run tests
+bun run test
+
 # Run with debug logging
 DEBUG=true bun run start
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@disco_trooper/apple-notes-mcp",
-  "version": "1.2.0",
+  "version": "1.3.0",
   "description": "MCP server for Apple Notes with semantic search and CRUD operations",
   "type": "module",
   "main": "src/index.ts",
@@ -13,15 +13,16 @@
     "test:watch": "vitest"
   },
   "dependencies": {
-    "@modelcontextprotocol/sdk": "^1.0.0",
-    "@lancedb/lancedb": "^0.13.0",
+    "@clack/prompts": "^0.8.0",
     "@huggingface/transformers": "^3.0.0",
-    "turndown": "^7.2.0",
+    "@lancedb/lancedb": "^0.13.0",
+    "@modelcontextprotocol/sdk": "^1.0.0",
+    "apache-arrow": "^21.1.0",
+    "dotenv": "^16.4.0",
     "marked": "^15.0.0",
     "run-jxa": "^3.0.0",
-    "zod": "^3.24.0",
-    "@clack/prompts": "^0.8.0",
-    "dotenv": "^16.4.0"
+    "turndown": "^7.2.0",
+    "zod": "^3.24.0"
   },
   "devDependencies": {
     "@types/bun": "^1.1.0",
@@ -39,7 +40,8 @@
   "author": "disco-trooper",
   "license": "MIT",
   "bin": {
-    "apple-notes-mcp": "./src/index.ts"
+    "apple-notes-mcp": "./src/index.ts",
+    "apple-notes-mcp-setup": "./src/setup.ts"
   },
   "files": [
     "src",

package/src/config/claude.test.ts

@@ -0,0 +1,47 @@
+// src/config/claude.test.ts
+import { describe, it, expect, vi, beforeEach } from "vitest";
+
+// Mock paths.js before importing claude.ts
+vi.mock("./paths.js", () => ({
+  isNpmInstall: vi.fn(),
+  getProjectRoot: vi.fn(() => "/mock/project"),
+}));
+
+describe("claude config", () => {
+  beforeEach(() => {
+    vi.clearAllMocks();
+    vi.resetModules();
+  });
+
+  describe("getClaudeConfigEntry", () => {
+    it("should return npm command when npm install", async () => {
+      const { isNpmInstall } = await import("./paths.js");
+      vi.mocked(isNpmInstall).mockReturnValue(true);
+
+      const { getClaudeConfigEntry } = await import("./claude.js");
+      const entry = getClaudeConfigEntry();
+
+      expect(entry.command).toBe("apple-notes-mcp");
+      expect(entry.args).toEqual([]);
+    });
+
+    it("should return bun command when source install", async () => {
+      const { isNpmInstall } = await import("./paths.js");
+      vi.mocked(isNpmInstall).mockReturnValue(false);
+
+      const { getClaudeConfigEntry } = await import("./claude.js");
+      const entry = getClaudeConfigEntry();
+
+      expect(entry.command).toBe("bun");
+      expect(entry.args[0]).toBe("run");
+      expect(entry.args[1]).toContain("/mock/project/src/index.ts");
+    });
+  });
+
+  describe("getClaudeConfigPath", () => {
+    it("should return path to ~/.claude.json", async () => {
+      const { getClaudeConfigPath } = await import("./claude.js");
+      expect(getClaudeConfigPath()).toMatch(/\.claude\.json$/);
+    });
+  });
+});

package/src/config/claude.ts

@@ -0,0 +1,106 @@
+// src/config/claude.ts
+/**
+ * Claude Code configuration management.
+ */
+
+import * as fs from "node:fs";
+import * as path from "node:path";
+import * as os from "node:os";
+import { isNpmInstall, getProjectRoot } from "./paths.js";
+
+const CLAUDE_CONFIG_PATH = path.join(os.homedir(), ".claude.json");
+
+interface ClaudeServerEntry {
+  command: string;
+  args: string[];
+  env: Record<string, string>;
+}
+
+interface ClaudeConfig {
+  mcpServers?: Record<string, ClaudeServerEntry>;
+  [key: string]: unknown;
+}
+
+/**
+ * Generate the appropriate MCP server entry based on install method.
+ */
+export function getClaudeConfigEntry(): ClaudeServerEntry {
+  if (isNpmInstall()) {
+    return {
+      command: "apple-notes-mcp",
+      args: [],
+      env: {},
+    };
+  } else {
+    const projectRoot = getProjectRoot();
+    return {
+      command: "bun",
+      args: ["run", path.join(projectRoot, "src", "index.ts")],
+      env: {},
+    };
+  }
+}
+
+/**
+ * Read existing Claude config.
+ */
+export function readClaudeConfig(): ClaudeConfig | null {
+  if (!fs.existsSync(CLAUDE_CONFIG_PATH)) {
+    return null;
+  }
+
+  try {
+    const content = fs.readFileSync(CLAUDE_CONFIG_PATH, "utf-8");
+    return JSON.parse(content);
+  } catch {
+    return null;
+  }
+}
+
+/**
+ * Write Claude config with our MCP server entry.
+ */
+export function writeClaudeConfig(entry: ClaudeServerEntry): boolean {
+  let config = readClaudeConfig();
+
+  if (!config) {
+    config = {
+      mcpServers: {
+        "apple-notes": entry,
+      },
+    };
+  } else {
+    const mcpServers = (config.mcpServers || {}) as Record<string, ClaudeServerEntry>;
+    mcpServers["apple-notes"] = entry;
+    config.mcpServers = mcpServers;
+  }
+
+  try {
+    fs.writeFileSync(CLAUDE_CONFIG_PATH, JSON.stringify(config, null, 2) + "\n");
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+/**
+ * Check if existing Claude config uses different install method.
+ * Returns null if no existing config, otherwise returns the method.
+ */
+export function getExistingInstallMethod(): "npm" | "source" | null {
+  const config = readClaudeConfig();
+  const entry = config?.mcpServers?.["apple-notes"];
+
+  if (!entry) {
+    return null;
+  }
+
+  return entry.command === "apple-notes-mcp" ? "npm" : "source";
+}
+
+/**
+ * Get Claude config path for display.
+ */
+export function getClaudeConfigPath(): string {
+  return CLAUDE_CONFIG_PATH;
+}

package/src/config/constants.ts

@@ -32,10 +32,19 @@ export const ERROR_MESSAGE_MAX_LENGTH = 200;
 export const MAX_RETRIES = 3;
 export const RATE_LIMIT_BACKOFF_BASE_MS = 2000;
 
-// Indexing
-export const EMBEDDING_DELAY_MS = 300;
+// Indexing (EMBEDDING_DELAY_MS removed - not needed with batch processing)
 
 // Search tuning
 export const HYBRID_SEARCH_MIN_FETCH = 40;
 export const FOLDER_FILTER_MULTIPLIER = 3;
 export const PREVIEW_TRUNCATE_RATIO = 0.7;
+
+// Knowledge Graph
+export const DEFAULT_RELATED_NOTES_LIMIT = 10;
+export const GRAPH_TAG_WEIGHT = 0.8;
+export const GRAPH_LINK_WEIGHT = 1.0;
+export const GRAPH_SIMILAR_WEIGHT = 0.5;
+
+// Chunking settings
+export const DEFAULT_CHUNK_SIZE = 500;
+export const DEFAULT_CHUNK_OVERLAP = 100;

package/src/config/paths.test.ts

@@ -0,0 +1,40 @@
+// src/config/paths.test.ts
+import { describe, it, expect } from "vitest";
+
+describe("config paths", () => {
+  describe("getConfigDir", () => {
+    it("should return ~/.apple-notes-mcp", async () => {
+      const { getConfigDir } = await import("./paths.js");
+      expect(getConfigDir()).toMatch(/\.apple-notes-mcp$/);
+    });
+  });
+
+  describe("getEnvPath", () => {
+    it("should return config dir + .env", async () => {
+      const { getEnvPath, getConfigDir } = await import("./paths.js");
+      expect(getEnvPath()).toBe(`${getConfigDir()}/.env`);
+    });
+  });
+
+  describe("getDataDir", () => {
+    it("should return config dir + data", async () => {
+      const { getDataDir, getConfigDir } = await import("./paths.js");
+      expect(getDataDir()).toBe(`${getConfigDir()}/data`);
+    });
+  });
+
+  describe("isNpmInstall", () => {
+    it("should detect npm install from path", async () => {
+      const { isNpmInstall } = await import("./paths.js");
+      // Running from source, should be false
+      expect(isNpmInstall()).toBe(false);
+    });
+  });
+
+  describe("getProjectRoot", () => {
+    it("should return project root directory", async () => {
+      const { getProjectRoot } = await import("./paths.js");
+      expect(getProjectRoot()).toMatch(/apple-notes-mcp$/);
+    });
+  });
+});

package/src/config/paths.ts

@@ -0,0 +1,86 @@
+// src/config/paths.ts
+/**
+ * Unified configuration paths for both source and npm installations.
+ * All user config lives in ~/.apple-notes-mcp/ regardless of install method.
+ */
+
+import * as fs from "node:fs";
+import * as path from "node:path";
+import * as os from "node:os";
+
+/**
+ * Get the config directory path.
+ * Always ~/.apple-notes-mcp/ for consistency across install methods.
+ */
+export function getConfigDir(): string {
+  return path.join(os.homedir(), ".apple-notes-mcp");
+}
+
+/**
+ * Get the .env file path.
+ */
+export function getEnvPath(): string {
+  return path.join(getConfigDir(), ".env");
+}
+
+/**
+ * Get the data directory path (for LanceDB).
+ */
+export function getDataDir(): string {
+  return path.join(getConfigDir(), "data");
+}
+
+/**
+ * Check if configuration exists.
+ */
+export function hasConfig(): boolean {
+  return fs.existsSync(getEnvPath());
+}
+
+/**
+ * Ensure config directory exists.
+ */
+export function ensureConfigDir(): void {
+  const configDir = getConfigDir();
+  if (!fs.existsSync(configDir)) {
+    fs.mkdirSync(configDir, { recursive: true });
+  }
+}
+
+/**
+ * Detect if running from npm global install vs source.
+ * Used to generate appropriate Claude Code config.
+ */
+export function isNpmInstall(): boolean {
+  const scriptPath = new URL(import.meta.url).pathname;
+  return (
+    scriptPath.includes("/node_modules/") ||
+    scriptPath.includes("/.npm/") ||
+    // Global npm install on macOS
+    scriptPath.includes("/lib/node_modules/")
+  );
+}
+
+/**
+ * Get the project root directory (for source installs).
+ */
+export function getProjectRoot(): string {
+  // Navigate up from src/config/paths.ts to project root
+  return path.resolve(path.dirname(new URL(import.meta.url).pathname), "../..");
+}
+
+/**
+ * Check if legacy config exists in project root.
+ * Used for migration from old setup.
+ */
+export function hasLegacyConfig(): boolean {
+  const legacyPath = path.join(getProjectRoot(), ".env");
+  return fs.existsSync(legacyPath) && !isNpmInstall();
+}
+
+/**
+ * Get legacy config path for migration.
+ */
+export function getLegacyEnvPath(): string {
+  return path.join(getProjectRoot(), ".env");
+}

package/src/db/arrow-fix.test.ts

@@ -0,0 +1,101 @@
+import { describe, it, expect, beforeEach, afterEach } from "vitest";
+import * as fs from "fs";
+import * as os from "os";
+import * as path from "path";
+import { LanceDBStore } from "./lancedb.js";
+
+describe("Arrow type inference fix", () => {
+  let tempDir: string;
+  let store: LanceDBStore;
+
+  beforeEach(() => {
+    tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "arrow-test-"));
+    store = new LanceDBStore(tempDir);
+  });
+
+  afterEach(() => {
+    fs.rmSync(tempDir, { recursive: true, force: true });
+  });
+
+  it("handles records where ALL tags and outlinks are empty", async () => {
+    // This is the exact scenario that triggers the Arrow bug
+    const records = [
+      {
+        id: "1",
+        title: "Note 1",
+        content: "Hello world",
+        vector: new Array(384).fill(0.1),
+        folder: "Notes",
+        created: new Date().toISOString(),
+        modified: new Date().toISOString(),
+        indexed_at: new Date().toISOString(),
+        tags: [],  // Empty!
+        outlinks: [],  // Empty!
+      },
+      {
+        id: "2",
+        title: "Note 2",
+        content: "Test content",
+        vector: new Array(384).fill(0.2),
+        folder: "Notes",
+        created: new Date().toISOString(),
+        modified: new Date().toISOString(),
+        indexed_at: new Date().toISOString(),
+        tags: [],  // Empty!
+        outlinks: [],  // Empty!
+      },
+    ];
+
+    // This should NOT throw "Cannot infer list vector from empty array"
+    await store.index(records);
+
+    // Verify data was stored
+    const count = await store.count();
+    expect(count).toBe(2);
+
+    // Verify tags are empty (placeholders removed)
+    const all = await store.getAll();
+    expect(all[0].tags).toEqual([]);
+    expect(all[0].outlinks).toEqual([]);
+  });
+
+  it("handles mixed empty and non-empty arrays", async () => {
+    const records = [
+      {
+        id: "1",
+        title: "Note with tags",
+        content: "Hello",
+        vector: new Array(384).fill(0.1),
+        folder: "Notes",
+        created: new Date().toISOString(),
+        modified: new Date().toISOString(),
+        indexed_at: new Date().toISOString(),
+        tags: ["tag1", "tag2"],
+        outlinks: [],
+      },
+      {
+        id: "2",
+        title: "Note without tags",
+        content: "World",
+        vector: new Array(384).fill(0.2),
+        folder: "Notes",
+        created: new Date().toISOString(),
+        modified: new Date().toISOString(),
+        indexed_at: new Date().toISOString(),
+        tags: [],
+        outlinks: ["Note 1"],
+      },
+    ];
+
+    await store.index(records);
+
+    const all = await store.getAll();
+    const note1 = all.find(n => n.id === "1");
+    const note2 = all.find(n => n.id === "2");
+
+    expect(note1?.tags).toEqual(["tag1", "tag2"]);
+    expect(note1?.outlinks).toEqual([]);
+    expect(note2?.tags).toEqual([]);
+    expect(note2?.outlinks).toEqual(["Note 1"]);
+  });
+});