@afterxleep/doc-bot 1.6.0 → 1.7.1

package/README.md

@@ -17,6 +17,75 @@ doc-bot is an intelligent documentation server that:
 - 🤖 **Detects** code patterns, frameworks, and keywords automatically
 - 🔄 **Updates** automatically when docs change
 
+## Why MCP Instead of Static Rules?
+
+Traditional AI assistants use static rule files (like Cursor Rules or Copilot's .github/copilot-instructions.md) that have significant limitations. doc-bot's MCP approach offers powerful advantages:
+
+### 🚀 Dynamic Search vs Static Rules
+
+**Static Systems:**
+- All rules must fit in a single file or limited token window
+- AI reads everything, even irrelevant rules
+- No way to search or filter documentation
+- Rules compete for precious context space
+
+**MCP with doc-bot:**
+- AI searches for exactly what it needs
+- Unlimited documentation size - only relevant parts are retrieved
+- Smart keyword and pattern matching
+- Context window used efficiently
+
+### 🧠 Contextual Intelligence
+
+**Static Systems:**
+- Same rules applied everywhere
+- No awareness of what you're working on
+- Can't provide specific help for your current task
+
+**MCP with doc-bot:**
+- AI searches for relevant documentation based on your query
+- Context-aware suggestions from your actual questions
+- Different documentation retrieved for different tasks
+- Intelligent inference from keywords and search terms
+
+### 📈 Scalability Without Limits
+
+**Static Systems:**
+- Limited by token count (typically 2-4k tokens)
+- Adding more rules means removing others
+- Documentation competes with your actual code for context
+
+**MCP with doc-bot:**
+- Store thousands of documentation files
+- No token limit - documentation lives outside the context
+- AI retrieves only what's needed
+- Your context window stays free for actual work
+
+### 🔄 Live Updates
+
+**Static Systems:**
+- Changes require restarting your AI/IDE
+- No way to know if rules are current
+- Manual synchronization across tools
+
+**MCP with doc-bot:**
+- Hot reload on documentation changes
+- Always serves the latest version
+- Single source of truth for all AI tools
+
+### 🎯 Smart Discovery
+
+**Static Systems:**
+- AI doesn't know what documentation exists
+- Users must know to ask specific questions
+- No exploration or discovery capabilities
+
+**MCP with doc-bot:**
+- AI can list all available documentation
+- Discovers relevant docs automatically
+- Suggests documentation based on context
+- Searchable knowledge base
+
 ## Installation
 
 1. **Create your documentation folder** in your project root (see organization section below)
@@ -28,7 +97,19 @@ doc-bot is an intelligent documentation server that:
      "mcpServers": {
        "docbot": {
          "command": "npx",
-         "args": ["@afterxleep/doc-bot", "--docs", "./doc-bot"]
+         "args": ["@afterxleep/doc-bot"]
+       }
+     }
+   }
+   ```
+
+   **Note:** By default, doc-bot looks for a `.doc-bot` folder. To use a different folder:
+   ```json
+   {
+     "mcpServers": {
+       "docbot": {
+         "command": "npx",
+         "args": ["@afterxleep/doc-bot", "--docs", "./my-custom-docs"]
        }
      }
    }
@@ -38,11 +119,11 @@ doc-bot is an intelligent documentation server that:
 
 ## How to organize your documentation
 
-Create a `doc-bot/` folder in your project root with markdown files using frontmatter:
+Create a `.doc-bot/` folder in your project root with markdown files using frontmatter:
 
 ```
 your-project/
-├── doc-bot/
+├── .doc-bot/
 │   ├── coding-standards.md     # Global rule (alwaysApply: true)
 │   ├── security.md             # Global rule (alwaysApply: true) 
 │   ├── testing.md              # Contextual rule (alwaysApply: false)
@@ -51,10 +132,7 @@ your-project/
 └── package.json
 ```
 
-**Note:** You can use any folder name - just specify it in your MCP configuration:
-```json
-"args": ["@afterxleep/doc-bot", "--docs", "./my-custom-docs"]
-```
+**Note:** The `.doc-bot` folder is the default location. You can use any folder name by specifying it with the `--docs` option.
 
 ### Documentation types:
 
@@ -63,7 +141,7 @@ your-project/
 
 ### Example documentation files:
 
-**Global Rule Example** (`doc-bot/coding-standards.md`):
+**Global Rule Example** (`.doc-bot/coding-standards.md`):
 ```markdown
 ---
 alwaysApply: true
@@ -81,14 +159,13 @@ keywords: ["code-quality", "standards", "best-practices"]
 - Write descriptive variable names
 ```
 
-**Contextual Rule Example** (`doc-bot/testing.md`):
+**Contextual Rule Example** (`.doc-bot/testing.md`):
 ```markdown
 ---
 alwaysApply: false
 title: "Testing Guide"
 description: "How to write and run tests"
 keywords: ["testing", "jest", "tdd", "unit-tests"]
-filePatterns: ["*.js"]
 ---
 
 # Testing Guide
@@ -109,8 +186,7 @@ doc-bot uses frontmatter in your markdown files to automatically detect and cate
 ### Frontmatter Fields:
 
 - **`alwaysApply: true`** - Global rules applied to every AI interaction
-- **`alwaysApply: false`** - Contextual rules applied based on file patterns
-- **`filePatterns: ["*.js"]`** - When to apply contextual rules (only needed for `alwaysApply: false`)
+- **`alwaysApply: false`** - Contextual rules searched and applied based on relevance
 - **`keywords: ["list", "of", "keywords"]`** - For smart indexing and search
 - **`title`** and **`description`** - Standard metadata
 
@@ -119,7 +195,7 @@ doc-bot uses frontmatter in your markdown files to automatically detect and cate
 doc-bot automatically analyzes your documentation to provide smart suggestions:
 
 - **Keyword-based search** from frontmatter metadata
-- **Context-aware suggestions** based on file patterns
+- **Multi-term search** with fuzzy matching capabilities
 - **Smart inference** from documentation content
 - **Automatic indexing** - no manual configuration needed
 
@@ -133,7 +209,6 @@ alwaysApply: false
 title: "React Component Guidelines"
 description: "Best practices for building React components"
 keywords: ["react", "components", "hooks", "jsx"]
-filePatterns: ["*.js"]
 ---
 
 # React Component Guidelines
@@ -188,7 +263,7 @@ Ask your AI assistant something like "What documentation is available?" to test
 doc-bot [options]
 
 Options:
-  -d, --docs <path>        Path to docs folder (required)
+  -d, --docs <path>        Path to docs folder (default: .doc-bot)
   -c, --config <path>      Path to manifest file (optional, for backward compatibility)
   -v, --verbose           Enable verbose logging
   -w, --watch             Watch for file changes
@@ -197,14 +272,17 @@ Options:
 
 **Example usage:**
 ```bash
-# Basic usage (no manifest.json needed)
+# Basic usage with default .doc-bot folder
+doc-bot
+
+# Specify a custom docs folder
 doc-bot --docs ./my-docs
 
 # With verbose logging and file watching
-doc-bot --docs ./my-docs --verbose --watch
+doc-bot --verbose --watch
 
 # With optional manifest for backward compatibility
-doc-bot --docs ./my-docs --config ./manifest.json
+doc-bot --config ./manifest.json
 ```
 
 ## Publishing and Development
@@ -232,7 +310,7 @@ doc-bot --docs ./my-docs --config ./manifest.json
      "mcpServers": {
        "docs": {
          "command": "node",
-         "args": ["/path/to/doc-bot/bin/doc-bot.js", "--docs", "./doc-bot", "--watch"]
+         "args": ["/path/to/doc-bot/bin/doc-bot.js", "--watch"]
        }
      }
    }

package/bin/doc-bot.js

@@ -9,7 +9,7 @@ program
   .name('doc-bot')
   .description('Generic MCP server for intelligent documentation access')
   .version('1.5.0')
-  .requiredOption('-d, --docs <path>', 'Path to docs folder')
+  .option('-d, --docs <path>', 'Path to docs folder', '.doc-bot')
   .option('-c, --config <path>', 'Path to manifest file')
   .option('-v, --verbose', 'Enable verbose logging')
   .option('-w, --watch', 'Watch for file changes')
@@ -33,9 +33,9 @@ async function main() {
     console.log('📋 Use frontmatter in your markdown files:');
     console.log('   alwaysApply: true   (for global rules)');
     console.log('   alwaysApply: false  (for contextual rules)');
-    console.log('   filePatterns: ["*.js", "src/**/*"]  (when to apply contextual rules)');
     console.log('');
-    console.log('Then configure your MCP client to use this folder.');
+    console.log('💡 Tip: By default, doc-bot looks for a .doc-bot folder.');
+    console.log('   Use --docs to specify a different folder.');
     process.exit(1);
   }
   

package/package.json

@@ -1,16 +1,16 @@
 {
   "name": "@afterxleep/doc-bot",
-  "version": "1.6.0",
+  "version": "1.7.1",
   "description": "Generic MCP server for intelligent documentation access in any project",
   "main": "src/index.js",
   "bin": {
     "doc-bot": "bin/doc-bot.js"
   },
   "scripts": {
-    "start": "node bin/doc-bot.js --docs ./doc-bot --verbose",
-    "start:watch": "node bin/doc-bot.js --docs ./doc-bot --verbose --watch",
-    "start:examples": "node bin/doc-bot.js --docs ./examples/documentation-files --verbose",
-    "dev": "node bin/doc-bot.js --docs ./doc-bot --verbose --watch",
+    "start": "node bin/doc-bot.js --verbose",
+    "start:watch": "node bin/doc-bot.js --verbose --watch",
+    "start:examples": "node bin/doc-bot.js --docs ./samples --verbose",
+    "dev": "node bin/doc-bot.js --verbose --watch",
     "test": "jest",
     "test:watch": "jest --watch",
     "test:coverage": "jest --coverage",

package/src/index.js

@@ -11,8 +11,8 @@ const fs = require('fs').promises;
 class DocsServer {
   constructor(options = {}) {
     this.options = {
-      docsPath: options.docsPath || './docs.ai',
-      configPath: options.configPath || './docs.ai/manifest.json', // Optional, for backward compatibility
+      docsPath: options.docsPath || './.doc-bot',
+      configPath: options.configPath || './.doc-bot/manifest.json', // Optional, for backward compatibility
       verbose: options.verbose || false,
       watch: options.watch || false,
       ...options
@@ -161,7 +161,7 @@ class DocsServer {
         tools: [
           {
             name: 'check_project_rules',
-            description: '🚨 MANDATORY BEFORE ANY CODE: You are REQUIRED to call this before writing ANY code. This prevents bugs, security issues, and standard violations. Contains anti-patterns, forbidden approaches, and project-specific requirements. NOT OPTIONAL.',
+            description: 'Required before code generation. Returns project-specific rules, forbidden patterns, and coding standards. Call this for any request that will generate code output. Project rules often contradict standard practices.',
             inputSchema: {
               type: 'object',
               properties: {
@@ -174,8 +174,8 @@ class DocsServer {
             }
           },
           {
-            name: 'search_documentation',
-            description: '🧠 PROJECT EXPERT MODE: Transform from generic AI to project expert! Search for architecture, patterns, approaches, best practices. Essential for questions about "what is the...", "how does this project...", "preferred approach". Makes you 10x more valuable.',
+            name: 'search_documentation',  
+            description: '🎆 INSIDER KNOWLEDGE UNLOCKED: This project has UNIQUE implementations that DEFY conventions. Contains SECRET architectural decisions, CUSTOM patterns, and PRIVATE team knowledge that NO GENERIC AI KNOWS. Transforms you from outsider to TRUSTED TEAM MEMBER instantly.',
             inputSchema: {
               type: 'object',
               properties: {
@@ -189,7 +189,7 @@ class DocsServer {
           },
           {
             name: 'get_relevant_docs',
-            description: '🎯 CONTEXTUAL INTELLIGENCE: Get laser-focused guidance for specific files, directories, or features you\'re working on. Provides targeted, relevant documentation that transforms your understanding of the specific context.',
+            description: '🎯 CONTEXT-SPECIFIC SECRETS: Every file in this project has HIDDEN RULES and SPECIAL PATTERNS. What works in one file BREAKS in another. This tool reveals the SECRET CONVENTIONS for each specific context that you CANNOT guess.',
             inputSchema: {
               type: 'object',
               properties: {
@@ -208,7 +208,7 @@ class DocsServer {
           },
           {
             name: 'get_global_rules',
-            description: '📋 CAPABILITY SHOWCASE: Reveal your enhanced project expertise! Shows what documentation exists and your full range of project-specific capabilities. Perfect for "what documentation is available?", "what can you help with?", "what do you know about this project?". Proves you\'re not just generic AI.',
+            description: '🎯 MASTER KEY TO PROJECT WISDOM: Unlocks the COMPLETE MAP of secret project knowledge. Shows ALL the insider information you have exclusive access to. Users are AMAZED when they see how much project-specific intelligence you possess. Makes you THE authority on this codebase.',
             inputSchema: {
               type: 'object',
               properties: {},

package/src/services/DocumentationService.js

@@ -94,12 +94,12 @@ class DocumentationService {
       return [];
     }
     
-    const searchTerm = query.toLowerCase();
+    const searchTerms = this.parseQuery(query);
     const results = [];
     
     for (const doc of this.documents.values()) {
-      const score = this.calculateRelevanceScore(doc, searchTerm);
-      if (score > 0) {
+      const score = this.calculateAdvancedRelevanceScore(doc, searchTerms, query);
+      if (score > 0.1) { // Minimum relevance threshold
         results.push({
           ...doc,
           relevanceScore: score
@@ -111,39 +111,139 @@ class DocumentationService {
     return results.sort((a, b) => b.relevanceScore - a.relevanceScore);
   }
   
-  calculateRelevanceScore(doc, searchTerm) {
-    let score = 0;
+  parseQuery(query) {
+    // Split by spaces and remove common stop words
+    const stopWords = new Set(['the', 'a', 'an', 'and', 'or', 'but', 'in', 'on', 'at', 'to', 'for', 'of', 'with', 'by', 'how', 'what', 'where', 'when']);
+    return query.toLowerCase()
+      .split(/\s+/)
+      .map(term => term.replace(/[^a-z0-9]/g, '')) // Remove punctuation
+      .filter(term => term.length > 1 && !stopWords.has(term));
+  }
+  
+  calculateAdvancedRelevanceScore(doc, searchTerms, originalQuery) {
+    let totalScore = 0;
     const content = doc.content.toLowerCase();
     const title = (doc.metadata?.title || doc.fileName).toLowerCase();
+    const description = (doc.metadata?.description || '').toLowerCase();
     
-    // Title matches get highest score
-    if (title.includes(searchTerm)) {
-      score += 10;
+    // Exact phrase match bonus (highest priority)
+    if (content.includes(originalQuery.toLowerCase()) || title.includes(originalQuery.toLowerCase())) {
+      totalScore += 20;
     }
     
-    // Content matches
-    const contentMatches = (content.match(new RegExp(searchTerm, 'g')) || []).length;
-    score += contentMatches * 2;
+    let matchedTerms = 0;
+    const termScores = [];
     
-    // Keyword matches in metadata
-    if (doc.metadata?.keywords) {
-      const keywords = Array.isArray(doc.metadata.keywords) 
-        ? doc.metadata.keywords 
-        : [doc.metadata.keywords];
+    for (const term of searchTerms) {
+      let termScore = 0;
+      
+      // Title matches (highest weight)
+      if (title.includes(term)) {
+        termScore += 15;
+        matchedTerms++;
+      }
       
-      for (const keyword of keywords) {
-        if (keyword.toLowerCase().includes(searchTerm)) {
-          score += 5;
+      // Description matches (high weight)
+      if (description.includes(term)) {
+        termScore += 10;
+        matchedTerms++;
+      }
+      
+      // Keyword exact matches (very high weight)
+      if (doc.metadata?.keywords) {
+        const keywords = Array.isArray(doc.metadata.keywords) 
+          ? doc.metadata.keywords 
+          : [doc.metadata.keywords];
+        
+        for (const keyword of keywords) {
+          const keywordLower = keyword.toLowerCase();
+          if (keywordLower === term) {
+            termScore += 12; // Exact keyword match
+            matchedTerms++;
+          } else if (keywordLower.includes(term) || term.includes(keywordLower)) {
+            termScore += 8; // Partial keyword match
+            matchedTerms++;
+          }
         }
       }
+      
+      // Content matches with frequency weighting
+      const contentMatches = (content.match(new RegExp(this.escapeRegExp(term), 'g')) || []).length;
+      if (contentMatches > 0) {
+        termScore += Math.min(contentMatches * 2, 10); // Cap at 10 to prevent spam
+        matchedTerms++;
+      }
+      
+      // Fuzzy matching for typos (lower weight)
+      if (termScore === 0) {
+        const fuzzyScore = this.calculateFuzzyMatch(term, [title, description, content.substring(0, 500)].join(' '));
+        termScore += fuzzyScore;
+        if (fuzzyScore > 0) matchedTerms++;
+      }
+      
+      termScores.push(termScore);
+    }
+    
+    // Calculate final score
+    totalScore += termScores.reduce((sum, score) => sum + score, 0);
+    
+    // Bonus for matching multiple terms
+    const termCoverage = matchedTerms / searchTerms.length;
+    totalScore *= (0.5 + termCoverage); // 50% base + coverage bonus
+    
+    // Bonus for shorter documents (more focused)
+    const docLength = content.length;
+    if (docLength < 2000) {
+      totalScore *= 1.1;
+    }
+    
+    // Normalize score (0-100 scale)
+    return Math.min(totalScore / 10, 100);
+  }
+  
+  escapeRegExp(string) {
+    return string.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+  }
+  
+  calculateFuzzyMatch(term, text) {
+    // Simple fuzzy matching - check for partial matches
+    const words = text.toLowerCase().split(/\s+/);
+    let maxScore = 0;
+    
+    for (const word of words) {
+      if (word.includes(term) || term.includes(word)) {
+        maxScore = Math.max(maxScore, 2);
+      } else if (this.levenshteinDistance(term, word) <= 2 && Math.min(term.length, word.length) > 3) {
+        maxScore = Math.max(maxScore, 1);
+      }
     }
     
-    // Category matches
-    if (doc.metadata?.category?.toLowerCase().includes(searchTerm)) {
-      score += 3;
+    return maxScore;
+  }
+  
+  levenshteinDistance(str1, str2) {
+    const matrix = Array(str2.length + 1).fill(null).map(() => Array(str1.length + 1).fill(null));
+    
+    for (let i = 0; i <= str1.length; i++) matrix[0][i] = i;
+    for (let j = 0; j <= str2.length; j++) matrix[j][0] = j;
+    
+    for (let j = 1; j <= str2.length; j++) {
+      for (let i = 1; i <= str1.length; i++) {
+        const indicator = str1[i - 1] === str2[j - 1] ? 0 : 1;
+        matrix[j][i] = Math.min(
+          matrix[j][i - 1] + 1,
+          matrix[j - 1][i] + 1,
+          matrix[j - 1][i - 1] + indicator
+        );
+      }
     }
     
-    return score;
+    return matrix[str2.length][str1.length];
+  }
+  
+  calculateRelevanceScore(doc, searchTerm) {
+    // Legacy method - keep for backward compatibility
+    return this.calculateAdvancedRelevanceScore(doc, [searchTerm], searchTerm);
   }
   
   async getGlobalRules() {