@softerist/heuristic-mcp 2.0.0

package/README.md

@@ -0,0 +1,249 @@
+# Smart Coding MCP
+
+[![npm version](https://img.shields.io/npm/v/smart-coding-mcp.svg)](https://www.npmjs.com/package/smart-coding-mcp)
+[![npm downloads](https://img.shields.io/npm/dm/smart-coding-mcp.svg)](https://www.npmjs.com/package/smart-coding-mcp)
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
+[![Node.js](https://img.shields.io/badge/Node.js-%3E%3D18-green.svg)](https://nodejs.org/)
+
+An extensible Model Context Protocol (MCP) server that provides intelligent semantic code search for AI assistants. Built with local AI models (RAG), inspired by Cursor's semantic search research.
+
+## What This Does
+
+AI coding assistants work better when they can find relevant code quickly. Traditional keyword search falls short - if you ask "where do we handle authentication?" but your code uses "login" and "session", keyword search misses it.
+
+This MCP server solves that by indexing your codebase with AI embeddings. Your AI assistant can search by meaning instead of exact keywords, finding relevant code even when the terminology differs.
+
+![Example](example.png)
+
+## Why Use This
+
+**Better Code Understanding**
+
+- Search finds code by concept, not just matching words
+- Works with typos and variations in terminology
+- Natural language queries like "where do we validate user input?"
+
+**Performance**
+
+- Pre-indexed embeddings are faster than scanning files at runtime
+- Smart project detection skips dependencies automatically (node_modules, vendor, etc.)
+- Incremental updates - only re-processes changed files
+
+**Privacy**
+
+- Everything runs locally on your machine
+- Your code never leaves your system
+- No API calls to external services
+
+## Installation
+
+Install globally via npm:
+
+```bash
+npm install -g smart-coding-mcp
+```
+
+To update to the latest version:
+
+```bash
+npm update -g smart-coding-mcp
+```
+
+## Configuration
+
+Add to your MCP configuration file. The location depends on your IDE and OS:
+
+| IDE                  | OS      | Config Path                                                       |
+| -------------------- | ------- | ----------------------------------------------------------------- |
+| **Claude Desktop**   | macOS   | `~/Library/Application Support/Claude/claude_desktop_config.json` |
+| **Claude Desktop**   | Windows | `%APPDATA%\Claude\claude_desktop_config.json`                     |
+| **Cascade (Cursor)** | All     | Configured via UI Settings > Features > MCP                       |
+| **Antigravity**      | macOS   | `~/.gemini/antigravity/mcp_config.json`                           |
+| **Antigravity**      | Windows | `%USERPROFILE%\.gemini\antigravity\mcp_config.json`               |
+
+Add the server configuration to the `mcpServers` object in your config file:
+
+### Option 1: Specific Project (Recommended)
+
+```json
+{
+  "mcpServers": {
+    "smart-coding-mcp": {
+      "command": "smart-coding-mcp",
+      "args": ["--workspace", "/absolute/path/to/your/project"]
+    }
+  }
+}
+```
+
+### Option 2: Multi-Project Support
+
+```json
+{
+  "mcpServers": {
+    "smart-coding-mcp-project-a": {
+      "command": "smart-coding-mcp",
+      "args": ["--workspace", "/path/to/project-a"]
+    },
+    "smart-coding-mcp-project-b": {
+      "command": "smart-coding-mcp",
+      "args": ["--workspace", "/path/to/project-b"]
+    }
+  }
+}
+```
+
+## Environment Variables
+
+Override configuration settings via environment variables in your MCP config:
+
+| Variable                         | Type    | Default                   | Description                           |
+| -------------------------------- | ------- | ------------------------- | ------------------------------------- |
+| `SMART_CODING_VERBOSE`           | boolean | `false`                   | Enable detailed logging               |
+| `SMART_CODING_BATCH_SIZE`        | number  | `100`                     | Files to process in parallel          |
+| `SMART_CODING_MAX_FILE_SIZE`     | number  | `1048576`                 | Max file size in bytes (1MB)          |
+| `SMART_CODING_CHUNK_SIZE`        | number  | `25`                      | Lines of code per chunk               |
+| `SMART_CODING_MAX_RESULTS`       | number  | `5`                       | Max search results                    |
+| `SMART_CODING_SMART_INDEXING`    | boolean | `true`                    | Enable smart project detection        |
+| `SMART_CODING_WATCH_FILES`       | boolean | `false`                   | Enable file watching for auto-reindex |
+| `SMART_CODING_SEMANTIC_WEIGHT`   | number  | `0.7`                     | Weight for semantic similarity (0-1)  |
+| `SMART_CODING_EXACT_MATCH_BOOST` | number  | `1.5`                     | Boost for exact text matches          |
+| `SMART_CODING_EMBEDDING_MODEL`   | string  | `Xenova/all-MiniLM-L6-v2` | AI embedding model to use             |
+| `SMART_CODING_WORKER_THREADS`    | string  | `auto`                    | Worker threads (`auto` or 1-32)       |
+
+**Example with environment variables:**
+
+```json
+{
+  "mcpServers": {
+    "smart-coding-mcp": {
+      "command": "smart-coding-mcp",
+      "args": ["--workspace", "/path/to/project"],
+      "env": {
+        "SMART_CODING_VERBOSE": "true",
+        "SMART_CODING_BATCH_SIZE": "200",
+        "SMART_CODING_MAX_FILE_SIZE": "2097152"
+      }
+    }
+  }
+}
+```
+
+**Note**: The server starts instantly and indexes in the background, so your IDE won't be blocked waiting for indexing to complete.
+
+## Available Tools
+
+**semantic_search** - Find code by meaning
+
+```
+Query: "Where do we validate user input?"
+Returns: Relevant validation code with file paths and line numbers
+```
+
+**index_codebase** - Manually trigger reindexing
+
+```
+Use after major refactoring or branch switches
+```
+
+**clear_cache** - Reset the embeddings cache
+
+```
+Useful when cache becomes corrupted or outdated
+```
+
+## How It Works
+
+The server indexes your code in four steps:
+
+1. **Discovery**: Scans your project for source files
+2. **Chunking**: Breaks code into meaningful pieces (respecting function boundaries)
+3. **Embedding**: Converts each chunk to a vector using a local AI model
+4. **Storage**: Saves embeddings to `.smart-coding-cache/` for fast startup
+
+When you search, your query is converted to the same vector format and compared against all code chunks using cosine similarity. The most relevant matches are returned.
+
+![How It Works](how-its-works.png)
+
+## Examples
+
+**Natural language search:**
+
+Query: "How do we handle cache persistence?"
+
+Result:
+
+```javascript
+// lib/cache.js (Relevance: 38.2%)
+async save() {
+  await fs.writeFile(cacheFile, JSON.stringify(this.vectorStore));
+  await fs.writeFile(hashFile, JSON.stringify(this.fileHashes));
+}
+```
+
+**Typo tolerance:**
+
+Query: "embeding modle initializashun"
+
+Still finds embedding model initialization code despite multiple typos.
+
+**Conceptual search:**
+
+Query: "error handling and exceptions"
+
+Finds all try/catch blocks and error handling patterns.
+
+## Privacy
+
+- AI model runs entirely on your machine
+- No network requests to external services
+- No telemetry or analytics
+- Cache stored locally in `.smart-coding-cache/`
+
+## Technical Details
+
+**Embedding Model**: all-MiniLM-L6-v2 via transformers.js
+
+- Fast inference (CPU-friendly)
+- Small model size (~100MB)
+- Good accuracy for code search
+
+**Vector Similarity**: Cosine similarity
+
+- Efficient comparison of embeddings
+- Normalized vectors for consistent scoring
+
+**Hybrid Scoring**: Combines semantic similarity with exact text matching
+
+- Semantic weight: 0.7 (configurable)
+- Exact match boost: 1.5x (configurable)
+
+## Research Background
+
+This project builds on research from Cursor showing that semantic search improves AI coding agent performance by 12.5% on average across question-answering tasks. The key insight is that AI assistants benefit more from relevant context than from large amounts of context.
+
+See: https://cursor.com/blog/semsearch
+
+## License
+
+MIT License
+
+Copyright (c) 2025 Omar Haris
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/config.json

@@ -0,0 +1,66 @@
+{
+  "searchDirectory": ".",
+  "fileExtensions": [
+    "js",
+    "ts",
+    "jsx",
+    "tsx",
+    "mjs",
+    "cjs",
+    "css",
+    "scss",
+    "sass",
+    "less",
+    "html",
+    "htm",
+    "xml",
+    "svg",
+    "py",
+    "pyw",
+    "java",
+    "kt",
+    "scala",
+    "c",
+    "cpp",
+    "h",
+    "hpp",
+    "cs",
+    "go",
+    "rs",
+    "rb",
+    "php",
+    "swift",
+    "sh",
+    "bash",
+    "json",
+    "yaml",
+    "yml",
+    "toml",
+    "sql"
+  ],
+  "excludePatterns": [
+    "**/node_modules/**",
+    "**/dist/**",
+    "**/build/**",
+    "**/.git/**",
+    "**/coverage/**",
+    "**/.next/**",
+    "**/target/**",
+    "**/vendor/**",
+    "**/.smart-coding-cache/**"
+  ],
+  "smartIndexing": true,
+  "chunkSize": 25,
+  "chunkOverlap": 5,
+  "batchSize": 100,
+  "maxFileSize": 1048576,
+  "maxResults": 5,
+  "enableCache": true,
+  "cacheDirectory": "./.smart-coding-cache",
+  "watchFiles": false,
+  "verbose": false,
+  "embeddingModel": "Xenova/all-MiniLM-L6-v2",
+  "semanticWeight": 0.7,
+  "exactMatchBoost": 1.5,
+  "workerThreads": "auto"
+}

package/features/clear-cache.js

@@ -0,0 +1,75 @@
+export class CacheClearer {
+  constructor(embedder, cache, config, indexer) {
+    this.cache = cache;
+    this.config = config;
+    this.indexer = indexer;
+    this.isClearing = false;
+  }
+
+  async execute() {
+    // Check if indexing is in progress
+    if (this.indexer && this.indexer.isIndexing) {
+      throw new Error("Cannot clear cache while indexing is in progress. Please wait for indexing to complete.");
+    }
+
+    // Check if cache is currently being saved (race condition prevention)
+    if (this.cache.isSaving) {
+      throw new Error("Cannot clear cache while cache is being saved. Please try again in a moment.");
+    }
+
+    // Check if a clear operation is already in progress (prevent concurrent clears)
+    if (this.isClearing) {
+      throw new Error("Cache clear operation already in progress. Please wait for it to complete.");
+    }
+
+    this.isClearing = true;
+
+    try {
+      await this.cache.clear();
+      return {
+        success: true,
+        message: `Cache cleared successfully. Next indexing will be a full rebuild.`,
+        cacheDirectory: this.config.cacheDirectory
+      };
+    } finally {
+      this.isClearing = false;
+    }
+  }
+}
+
+export function getToolDefinition() {
+  return {
+    name: "c_clear_cache",
+    description: "Clears the embeddings cache, forcing a complete reindex on next search or manual index operation. Useful when encountering cache corruption or after major codebase changes.",
+    inputSchema: {
+      type: "object",
+      properties: {}
+    },
+    annotations: {
+      title: "Clear Embeddings Cache",
+      readOnlyHint: false,
+      destructiveHint: true,
+      idempotentHint: true,
+      openWorldHint: false
+    }
+  };
+}
+
+export async function handleToolCall(request, cacheClearer) {
+  try {
+    const result = await cacheClearer.execute();
+    return {
+      content: [{
+        type: "text",
+        text: `${result.message}\n\nCache directory: ${result.cacheDirectory}`
+      }]
+    };
+  } catch (error) {
+    return {
+      content: [{
+        type: "text",
+        text: `Failed to clear cache: ${error.message}`
+      }]
+    };
+  }
+}

package/features/find-similar-code.js

@@ -0,0 +1,127 @@
+import path from "path";
+import { dotSimilarity } from "../lib/utils.js";
+
+/**
+ * FindSimilarCode feature
+ * Given a code snippet, finds similar patterns elsewhere in the codebase
+ */
+export class FindSimilarCode {
+  constructor(embedder, cache, config) {
+    this.embedder = embedder;
+    this.cache = cache;
+    this.config = config;
+  }
+
+  async execute({ code, maxResults = 5, minSimilarity = 0.3 }) {
+    const vectorStore = this.cache.getVectorStore();
+
+    if (vectorStore.length === 0) {
+      return {
+        results: [],
+        message: "No code has been indexed yet. Please wait for initial indexing to complete."
+      };
+    }
+
+    // Generate embedding for the input code
+    const codeEmbed = await this.embedder(code, { pooling: "mean", normalize: true });
+    const codeVector = Array.from(codeEmbed.data);
+
+    // Score all chunks by similarity
+    const scoredChunks = vectorStore.map(chunk => {
+      const similarity = dotSimilarity(codeVector, chunk.vector);
+      return { ...chunk, similarity };
+    });
+
+    // Filter by minimum similarity and sort
+    const filteredResults = scoredChunks
+      .filter(chunk => chunk.similarity >= minSimilarity)
+      .sort((a, b) => b.similarity - a.similarity);
+
+    // Deduplicate: if input code is from indexed file, skip exact matches
+    const normalizedInput = code.trim().replace(/\s+/g, ' ');
+    const results = filteredResults
+      .filter(chunk => {
+        const normalizedChunk = chunk.content.trim().replace(/\s+/g, ' ');
+        // Skip if it's essentially the same code (>95% similar text)
+        return normalizedChunk !== normalizedInput;
+      })
+      .slice(0, maxResults);
+
+    return {
+      results,
+      message: results.length === 0 ? "No similar code found above the similarity threshold." : null
+    };
+  }
+
+  formatResults(results) {
+    if (results.length === 0) {
+      return "No similar code patterns found in the codebase.";
+    }
+
+    return results.map((r, idx) => {
+      const relPath = path.relative(this.config.searchDirectory, r.file);
+      return `## Similar Code ${idx + 1} (Similarity: ${(r.similarity * 100).toFixed(1)}%)\n` +
+             `**File:** \`${relPath}\`\n` +
+             `**Lines:** ${r.startLine}-${r.endLine}\n\n` +
+             "```" + path.extname(r.file).slice(1) + "\n" +
+             r.content + "\n" +
+             "```\n";
+    }).join("\n");
+  }
+}
+
+// MCP Tool definition
+export function getToolDefinition(config) {
+  return {
+    name: "d_find_similar_code",
+    description: "Find similar code patterns in the codebase. Given a code snippet, returns other code chunks that are semantically similar. Useful for finding duplicate code, understanding patterns, and refactoring opportunities.",
+    inputSchema: {
+      type: "object",
+      properties: {
+        code: {
+          type: "string",
+          description: "The code snippet to find similar patterns for"
+        },
+        maxResults: {
+          type: "number",
+          description: "Maximum number of similar code chunks to return (default: 5)",
+          default: 5
+        },
+        minSimilarity: {
+          type: "number",
+          description: "Minimum similarity threshold 0-1 (default: 0.3 = 30%)",
+          default: 0.3
+        }
+      },
+      required: ["code"]
+    },
+    annotations: {
+      title: "Find Similar Code",
+      readOnlyHint: true,
+      destructiveHint: false,
+      idempotentHint: true,
+      openWorldHint: false
+    }
+  };
+}
+
+// Tool handler
+export async function handleToolCall(request, findSimilarCode) {
+  const code = request.params.arguments.code;
+  const maxResults = request.params.arguments.maxResults || 5;
+  const minSimilarity = request.params.arguments.minSimilarity || 0.3;
+
+  const { results, message } = await findSimilarCode.execute({ code, maxResults, minSimilarity });
+
+  if (message) {
+    return {
+      content: [{ type: "text", text: message }]
+    };
+  }
+
+  const formattedText = findSimilarCode.formatResults(results);
+
+  return {
+    content: [{ type: "text", text: formattedText }]
+  };
+}

package/features/hybrid-search.js

@@ -0,0 +1,173 @@
+import path from "path";
+import fs from "fs/promises";
+import { dotSimilarity } from "../lib/utils.js";
+
+export class HybridSearch {
+  constructor(embedder, cache, config) {
+    this.embedder = embedder;
+    this.cache = cache;
+    this.config = config;
+    this.fileModTimes = new Map(); // Cache for file modification times
+  }
+
+  async populateFileModTimes(files) {
+    const uniqueFiles = new Set(files);
+    const missing = [];
+
+    for (const file of uniqueFiles) {
+      if (!this.fileModTimes.has(file)) {
+        missing.push(file);
+      }
+    }
+
+    if (missing.length === 0) {
+      return;
+    }
+
+    const BATCH_SIZE = 200;
+    for (let i = 0; i < missing.length; i += BATCH_SIZE) {
+      const batch = missing.slice(i, i + BATCH_SIZE);
+      await Promise.all(batch.map(async file => {
+        try {
+          const stats = await fs.stat(file);
+          this.fileModTimes.set(file, stats.mtimeMs);
+        } catch {
+          this.fileModTimes.set(file, null);
+        }
+      }));
+    }
+  }
+
+  // Cache invalidation helper
+  clearFileModTime(file) {
+    this.fileModTimes.delete(file);
+  }
+
+  async search(query, maxResults) {
+    const vectorStore = this.cache.getVectorStore();
+
+    if (vectorStore.length === 0) {
+      return {
+        results: [],
+        message: "No code has been indexed yet. Please wait for initial indexing to complete."
+      };
+    }
+
+    // Generate query embedding
+    const queryEmbed = await this.embedder(query, { pooling: "mean", normalize: true });
+    const queryVector = Array.from(queryEmbed.data);
+
+    if (this.config.recencyBoost > 0) {
+      await this.populateFileModTimes(vectorStore.map(chunk => chunk.file));
+    }
+
+    // Score all chunks (synchronous map now, much faster)
+    const scoredChunks = vectorStore.map(chunk => {
+      // Semantic similarity (vectors are normalized)
+      let score = dotSimilarity(queryVector, chunk.vector) * this.config.semanticWeight;
+
+      // Exact match boost
+      const lowerQuery = query.toLowerCase();
+      const lowerContent = chunk.content.toLowerCase();
+
+      if (lowerContent.includes(lowerQuery)) {
+        score += this.config.exactMatchBoost;
+      } else {
+        // Partial word matching
+        const queryWords = lowerQuery.split(/\s+/);
+        const matchedWords = queryWords.filter(word =>
+          word.length > 2 && lowerContent.includes(word)
+        ).length;
+        score += (matchedWords / queryWords.length) * 0.3;
+      }
+
+      // Recency boost - recently modified files rank higher
+      if (this.config.recencyBoost > 0) {
+        const mtime = this.fileModTimes.get(chunk.file);
+        if (typeof mtime === "number") {
+          const daysSinceModified = (Date.now() - mtime) / (1000 * 60 * 60 * 24);
+          const decayDays = this.config.recencyDecayDays || 30;
+
+          // Linear decay: full boost at 0 days, no boost after decayDays
+          const recencyScore = Math.max(0, 1 - (daysSinceModified / decayDays));
+          score += recencyScore * this.config.recencyBoost;
+        }
+      }
+
+      return { ...chunk, score };
+    });
+
+    // Get top results
+    const results = scoredChunks
+      .sort((a, b) => b.score - a.score)
+      .slice(0, maxResults);
+
+    return { results, message: null };
+  }
+
+  formatResults(results) {
+    if (results.length === 0) {
+      return "No matching code found for your query.";
+    }
+
+    return results.map((r, idx) => {
+      const relPath = path.relative(this.config.searchDirectory, r.file);
+      return `## Result ${idx + 1} (Relevance: ${(r.score * 100).toFixed(1)}%)\n` +
+             `**File:** \`${relPath}\`\n` +
+             `**Lines:** ${r.startLine}-${r.endLine}\n\n` +
+             "```" + path.extname(r.file).slice(1) + "\n" +
+             r.content + "\n" +
+             "```\n";
+    }).join("\n");
+  }
+}
+
+// MCP Tool definition for this feature
+export function getToolDefinition(config) {
+  return {
+    name: "a_semantic_search",
+    description: "Performs intelligent hybrid code search combining semantic understanding with exact text matching. Ideal for finding code by meaning (e.g., 'authentication logic', 'database queries') even with typos or variations. Returns the most relevant code snippets with file locations and line numbers.",
+    inputSchema: {
+      type: "object",
+      properties: {
+        query: {
+          type: "string",
+          description: "Search query - can be natural language (e.g., 'where do we handle user login') or specific terms"
+        },
+        maxResults: {
+          type: "number",
+          description: "Maximum number of results to return (default: from config)",
+          default: config.maxResults
+        }
+      },
+      required: ["query"]
+    },
+    annotations: {
+      title: "Semantic Code Search",
+      readOnlyHint: true,
+      destructiveHint: false,
+      idempotentHint: true,
+      openWorldHint: false
+    }
+  };
+}
+
+// Tool handler
+export async function handleToolCall(request, hybridSearch) {
+  const query = request.params.arguments.query;
+  const maxResults = request.params.arguments.maxResults || hybridSearch.config.maxResults;
+
+  const { results, message } = await hybridSearch.search(query, maxResults);
+
+  if (message) {
+    return {
+      content: [{ type: "text", text: message }]
+    };
+  }
+
+  const formattedText = hybridSearch.formatResults(results);
+
+  return {
+    content: [{ type: "text", text: formattedText }]
+  };
+}