mcp-code-indexer 2.2.0__tar.gz → 2.3.0__tar.gz

{mcp_code_indexer-2.2.0/src/mcp_code_indexer.egg-info → mcp_code_indexer-2.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mcp-code-indexer
-Version: 2.2.0
+Version: 2.3.0
 Summary: MCP server that tracks file descriptions across codebases, enabling AI agents to efficiently navigate and understand code through searchable summaries and token-aware overviews.
 Author: MCP Code Indexer Contributors
 Maintainer: MCP Code Indexer Contributors
@@ -59,8 +59,8 @@ Dynamic: requires-python
 
 # MCP Code Indexer 🚀
 
-[![PyPI version](https://badge.fury.io/py/mcp-code-indexer.svg?15)](https://badge.fury.io/py/mcp-code-indexer)
-[![Python](https://img.shields.io/pypi/pyversions/mcp-code-indexer.svg?15)](https://pypi.org/project/mcp-code-indexer/)
+[![PyPI version](https://badge.fury.io/py/mcp-code-indexer.svg?17)](https://badge.fury.io/py/mcp-code-indexer)
+[![Python](https://img.shields.io/pypi/pyversions/mcp-code-indexer.svg?17)](https://pypi.org/project/mcp-code-indexer/)
 [![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 
 A production-ready **Model Context Protocol (MCP) server** that revolutionizes how AI agents navigate and understand codebases. Built for high-concurrency environments with advanced database resilience, the server provides instant access to intelligent descriptions, semantic search, and context-aware recommendations while maintaining 800+ writes/sec throughput.

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0}/README.md

@@ -1,7 +1,7 @@
 # MCP Code Indexer 🚀
 
-[![PyPI version](https://badge.fury.io/py/mcp-code-indexer.svg?15)](https://badge.fury.io/py/mcp-code-indexer)
-[![Python](https://img.shields.io/pypi/pyversions/mcp-code-indexer.svg?15)](https://pypi.org/project/mcp-code-indexer/)
+[![PyPI version](https://badge.fury.io/py/mcp-code-indexer.svg?17)](https://badge.fury.io/py/mcp-code-indexer)
+[![Python](https://img.shields.io/pypi/pyversions/mcp-code-indexer.svg?17)](https://pypi.org/project/mcp-code-indexer/)
 [![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 
 A production-ready **Model Context Protocol (MCP) server** that revolutionizes how AI agents navigate and understand codebases. Built for high-concurrency environments with advanced database resilience, the server provides instant access to intelligent descriptions, semantic search, and context-aware recommendations while maintaining 800+ writes/sec throughput.

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0}/docs/api-reference.md

@@ -327,7 +327,14 @@ const result = await mcp.callTool("update_missing_descriptions", {
 
 ### search_descriptions
 
-Searches through all file descriptions in a project to find files related to specific functionality. Use this for large codebases instead of loading the entire structure. Returns files ranked by relevance.
+Searches through all file descriptions in a project to find files related to specific functionality using intelligent query preprocessing. Features include:
+
+- **Multi-word search**: `"grpc proto"` finds files containing both terms regardless of order
+- **Operator escaping**: FTS5 operators (`AND`, `OR`, `NOT`, `NEAR`) are treated as literal search terms
+- **Whole word matching**: Prevents partial matches for more precise results
+- **Case insensitive**: Works regardless of case in query or descriptions
+
+Use this for large codebases instead of loading the entire structure. Returns files ranked by relevance using BM25 scoring.
 
 #### Parameters
 
@@ -394,10 +401,50 @@ const result = await mcp.callTool("search_descriptions", {
 }
 ```
 
+#### Enhanced Search Examples
+
+**Multi-word search (order-agnostic):**
+```javascript
+// Both queries find the same results
+await mcp.callTool("search_descriptions", {
+  projectName: "api-service",
+  folderPath: "/projects/api-service", 
+  branch: "main",
+  query: "grpc proto"        // Finds files with both "grpc" AND "proto"
+});
+
+await mcp.callTool("search_descriptions", {
+  projectName: "api-service",
+  folderPath: "/projects/api-service",
+  branch: "main", 
+  query: "proto grpc"        // Same results as above
+});
+```
+
+**FTS5 operator escaping:**
+```javascript
+// Search for files containing literal "AND" as a term
+await mcp.callTool("search_descriptions", {
+  projectName: "error-handling",
+  folderPath: "/projects/error-handling",
+  branch: "main",
+  query: "logging AND error"  // Finds files with all three: "logging", "AND", "error"
+});
+```
+
+**Case insensitive matching:**
+```javascript
+// All variations return same results
+const queries = ["HTTP client", "http CLIENT", "Http Client"];
+// Each finds files containing both "http" and "client" regardless of case
+```
+
 🔍 **Search Tips**:
+- **Use multiple words**: "grpc proto" finds files with both terms
+- **Try different orders**: "api client" vs "client api" yield same results  
 - **Be descriptive**: "authentication logic" vs "auth"
-- **Combine concepts**: "database connection pooling"
-- **Try variations**: If no results, try different terms
+- **Don't worry about operators**: "AND", "OR" are treated as literal search terms
+- **Case doesn't matter**: "HTTP", "http", "Http" all work the same
 - **Use technical terms**: "middleware", "controller", "utils"
 - **Search by purpose**: "error handling", "data validation"
 

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "mcp-code-indexer"
-version = "2.2.0"
+version = "2.3.0"
 description = "MCP server that tracks file descriptions across codebases, enabling AI agents to efficiently navigate and understand code through searchable summaries and token-aware overviews."
 readme = "README.md"
 license = {text = "MIT"}

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0}/src/mcp_code_indexer/database/database.py

@@ -30,6 +30,7 @@ from mcp_code_indexer.database.exceptions import (
 from mcp_code_indexer.database.connection_health import (
     ConnectionHealthMonitor, DatabaseMetricsCollector
 )
+from mcp_code_indexer.query_preprocessor import preprocess_search_query
 
 logger = logging.getLogger(__name__)
 
@@ -848,7 +849,16 @@ class DatabaseManager:
         query: str,
         max_results: int = 20
     ) -> List[SearchResult]:
-        """Search file descriptions using FTS5."""
+        """Search file descriptions using FTS5 with intelligent query preprocessing."""
+        # Preprocess query for optimal FTS5 search
+        preprocessed_query = preprocess_search_query(query)
+        
+        if not preprocessed_query:
+            logger.debug(f"Empty query after preprocessing: '{query}'")
+            return []
+        
+        logger.debug(f"Search query preprocessing: '{query}' -> '{preprocessed_query}'")
+        
         async with self.get_connection() as db:
             cursor = await db.execute(
                 """
@@ -866,7 +876,7 @@ class DatabaseManager:
                 ORDER BY bm25(file_descriptions_fts)
                 LIMIT ?
                 """,
-                (query, project_id, branch, max_results)
+                (preprocessed_query, project_id, branch, max_results)
             )
             rows = await cursor.fetchall()
             

mcp_code_indexer-2.3.0/src/mcp_code_indexer/query_preprocessor.py

@@ -0,0 +1,181 @@
+"""
+Query preprocessing module for intelligent FTS5 search.
+
+This module provides intelligent query preprocessing for SQLite FTS5 full-text search
+to enable multi-word search with case insensitive matching, whole word enforcement,
+and proper handling of FTS5 operators as literal search terms.
+
+Key features:
+- Multi-word queries: "grpc proto" becomes "grpc" AND "proto" for order-agnostic matching
+- FTS5 operator escaping: "AND OR" becomes '"AND" AND "OR"' to treat operators as literals
+- Whole word matching: prevents partial matches by relying on proper tokenization
+- Case insensitive: leverages FTS5 default behavior
+- Special character handling: preserves special characters in quoted terms
+"""
+
+import re
+import logging
+from typing import List, Set
+
+logger = logging.getLogger(__name__)
+
+
+class QueryPreprocessor:
+    """
+    Preprocesses user queries for optimal FTS5 search performance.
+    
+    Handles multi-word queries, operator escaping, and special character preservation
+    while maintaining BM25 ranking performance.
+    """
+    
+    # FTS5 operators that need to be escaped when used as literal search terms
+    FTS5_OPERATORS: Set[str] = {
+        'AND', 'OR', 'NOT', 'NEAR'
+    }
+    
+    def __init__(self):
+        """Initialize the query preprocessor."""
+        pass
+    
+    def preprocess_query(self, query: str) -> str:
+        """
+        Preprocess a user query for FTS5 search.
+        
+        Args:
+            query: Raw user query string
+            
+        Returns:
+            Preprocessed query string optimized for FTS5
+            
+        Examples:
+            >>> preprocessor = QueryPreprocessor()
+            >>> preprocessor.preprocess_query("grpc proto")
+            '"grpc" AND "proto"'
+            >>> preprocessor.preprocess_query("error AND handling")
+            '"error" AND "AND" AND "handling"'
+            >>> preprocessor.preprocess_query('config "file system"')
+            '"config" AND "file system"'
+        """
+        if not query or not query.strip():
+            return ""
+        
+        # Normalize whitespace
+        query = query.strip()
+        
+        # Split into terms while preserving quoted phrases
+        terms = self._split_terms(query)
+        
+        if not terms:
+            return ""
+        
+        # Process each term: escape operators and add quotes
+        processed_terms = []
+        for term in terms:
+            processed_term = self._process_term(term)
+            if processed_term:  # Skip empty terms
+                processed_terms.append(processed_term)
+        
+        if not processed_terms:
+            return ""
+        
+        # Join with AND for multi-word matching
+        result = " AND ".join(processed_terms)
+        
+        logger.debug(f"Preprocessed query: '{query}' -> '{result}'")
+        return result
+    
+    def _split_terms(self, query: str) -> List[str]:
+        """
+        Split query into terms while preserving quoted phrases.
+        
+        Args:
+            query: Input query string
+            
+        Returns:
+            List of terms and quoted phrases
+            
+        Examples:
+            'grpc proto' -> ['grpc', 'proto']
+            'config "file system"' -> ['config', '"file system"']
+            'error AND handling' -> ['error', 'AND', 'handling']
+        """
+        terms = []
+        
+        # Regex to match quoted phrases or individual words
+        # This pattern captures:
+        # 1. Double-quoted strings (including the quotes)
+        # 2. Single words (sequences of non-whitespace characters)
+        pattern = r'"[^"]*"|\S+'
+        
+        matches = re.findall(pattern, query)
+        
+        for match in matches:
+            # Skip empty matches
+            if match.strip():
+                terms.append(match)
+        
+        return terms
+    
+    def _process_term(self, term: str) -> str:
+        """
+        Process a single term: escape operators and ensure proper quoting.
+        
+        Args:
+            term: Single term or quoted phrase
+            
+        Returns:
+            Processed term ready for FTS5
+            
+        Examples:
+            'grpc' -> '"grpc"'
+            'AND' -> '"AND"'
+            '"file system"' -> '"file system"'
+            'c++' -> '"c++"'
+        """
+        if not term:
+            return ""
+        
+        # If already quoted, return as-is (user intentional phrase)
+        if term.startswith('"') and term.endswith('"') and len(term) >= 2:
+            return term
+        
+        # Check if term is an FTS5 operator (case-insensitive)
+        if term.upper() in self.FTS5_OPERATORS:
+            # Escape operator by quoting
+            escaped_term = f'"{term}"'
+            logger.debug(f"Escaped FTS5 operator: '{term}' -> '{escaped_term}'")
+            return escaped_term
+        
+        # Quote all terms to ensure whole-word matching and handle special characters
+        return f'"{term}"'
+    
+    def _escape_quotes_in_term(self, term: str) -> str:
+        """
+        Escape internal quotes in a term for FTS5 compatibility.
+        
+        Args:
+            term: Term that may contain quotes
+            
+        Returns:
+            Term with escaped quotes
+            
+        Examples:
+            'say "hello"' -> 'say ""hello""'
+            "test's file" -> "test's file"
+        """
+        # In FTS5, quotes inside quoted strings are escaped by doubling them
+        return term.replace('"', '""')
+
+
+def preprocess_search_query(query: str) -> str:
+    """
+    Convenience function for preprocessing search queries.
+    
+    Args:
+        query: Raw user query
+        
+    Returns:
+        Preprocessed query ready for FTS5
+    """
+    preprocessor = QueryPreprocessor()
+    return preprocessor.preprocess_query(query)

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0}/src/mcp_code_indexer/server/mcp_server.py

@@ -324,7 +324,7 @@ class MCPCodeIndexServer:
                 ),
                 types.Tool(
                     name="search_descriptions",
-                    description="Searches through all file descriptions in a project to find files related to specific functionality. Use this for large codebases instead of loading the entire structure. Always start with the fewest terms possible; if the tool returns a lot of results (more than 20) or the results are not relevant, then narrow it down by increasing the number of search terms. Start broad, then narrow the focus only if needed!",
+                    description="Searches through all file descriptions in a project to find files related to specific functionality. Use this for large codebases instead of loading the entire structure. Always start with the fewest terms possible (1 to 3 words AT MOST); if the tool returns a lot of results (more than 20) or the results are not relevant, then narrow it down by increasing the number of search words one at a time and calling the tool again. Start VERY broad, then narrow the focus only if needed!",
                     inputSchema={
                         "type": "object",
                         "properties": {

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0/src/mcp_code_indexer.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mcp-code-indexer
-Version: 2.2.0
+Version: 2.3.0
 Summary: MCP server that tracks file descriptions across codebases, enabling AI agents to efficiently navigate and understand code through searchable summaries and token-aware overviews.
 Author: MCP Code Indexer Contributors
 Maintainer: MCP Code Indexer Contributors
@@ -59,8 +59,8 @@ Dynamic: requires-python
 
 # MCP Code Indexer 🚀
 
-[![PyPI version](https://badge.fury.io/py/mcp-code-indexer.svg?15)](https://badge.fury.io/py/mcp-code-indexer)
-[![Python](https://img.shields.io/pypi/pyversions/mcp-code-indexer.svg?15)](https://pypi.org/project/mcp-code-indexer/)
+[![PyPI version](https://badge.fury.io/py/mcp-code-indexer.svg?17)](https://badge.fury.io/py/mcp-code-indexer)
+[![Python](https://img.shields.io/pypi/pyversions/mcp-code-indexer.svg?17)](https://pypi.org/project/mcp-code-indexer/)
 [![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
 
 A production-ready **Model Context Protocol (MCP) server** that revolutionizes how AI agents navigate and understand codebases. Built for high-concurrency environments with advanced database resilience, the server provides instant access to intelligent descriptions, semantic search, and context-aware recommendations while maintaining 800+ writes/sec throughput.

{mcp_code_indexer-2.2.0 → mcp_code_indexer-2.3.0}/src/mcp_code_indexer.egg-info/SOURCES.txt

@@ -26,6 +26,7 @@ src/mcp_code_indexer/git_hook_handler.py
 src/mcp_code_indexer/logging_config.py
 src/mcp_code_indexer/main.py
 src/mcp_code_indexer/merge_handler.py
+src/mcp_code_indexer/query_preprocessor.py
 src/mcp_code_indexer/token_counter.py
 src/mcp_code_indexer.egg-info/PKG-INFO
 src/mcp_code_indexer.egg-info/SOURCES.txt